• Kalpesh Agrawal

What is Data Mining & The 9 Laws of Data Mining

Data mining is an interdisciplinary field of Insurance, Business, Computer Science, economic, and others to discover new patterns from large datasets. Data mining refers to extracting or "mining" knowledge from the large dataset. For Data mining, another synonym term used, knowledge discovery from data or "KDD".

Data mining is the compiled process of statistics, machine learning, and database systems. The term "Data mining" is incorrectly applied, because the goal of data mining is the extraction (mining) of patterns and knowledge from a large amount of data.

History of Data Mining

The manual extraction of patterns and knowledge from data has occurred for centuries. As the process shows below, early methods of identifying patterns in data include Baye's theorem (1763) and Regression analysis (1805). As time passes, with the increasing power of computer technology have dramatically increased data collection, storage, and manipulation ability. Also as data sets grown in terms of size and complexity, the tools of "Data Analysis" has increased by other discoveries in computer science, mainly in the field of Machine Learning, such as a neural network (1943), cluster analysis, genetic algorithms (1950), decision trees and decision rules (1960), and support vector machines-SVM(1992).

Data Mining is the process which applies these methods to uncover the hidden pattern and knowledge from the data.

The 9 Laws of Data Mining by TOM KHABAZA

Tom Khabaza is a great personality who expressed the data mining philosophy, in his 9 laws of Data Mining.

Tom Khabaza is a London based data mining expert, who offers some help in the form of a simple way to explain important analytics concepts. The "9 laws of Data Mining" is widely accepted in the analytics community.

So, let's start to briefly introduce "9 Laws of Data mining".

1. "Business Goals Law"

Business objectives are the origin of every data mining solution.

If you don't understand the business problem, you can't solve it. This means if you don't know what is the business goals, what are the business objectives you will unable to solve the problems facing by the business.

2. "Business Knowledge law"

Business knowledge is central to every step of the data mining.

3. "Data Preparation Law"

This law is the most important law, as data preparation is more than half of every data mining process.

There are two aspects to this " Problem space solving". The first is putting the data into a form in which it can be easily analyzed, like in a single table, with one record example.

The second aspect is making the data more informative with respect to the business problem.

This law also explains the otherwise paradoxical observation that even after all the data acquisition, cleaning, and organization that goes into creating a data warehouse, data preparation is still crucial to, and more than half of the data mining process.

4. "No free lunch for Data Miner (NFL-DM)"

The right model for a given application can only be discovered by experiment.

Data Miner has to implement the trial and error to find predictive methods that work for the company.

5. "Watkins Law"

This law is given by " David Watkins"...which states that there are always patterns. In practice, the data always holds useful information to support decision making.

6. "Insight Law"

Data mining amplifies perception in the business domain. Data mining provides a kind of intelligent amplifier, helping business experts to solve the business problem, because the more analysis, helps to understand more than before.

7. "Prediction Law"

Prediction increases information locally by generalization. By good analysis, we can better predict and understand business situations.

8. "Value Law"

The values of data mining results are not determined by the accuracy or stability of the predictive models or technical measure. Data miners have to judge the results by the value they yield to the business, not by the technical or mathematical details.

9. "Law of Change"

All patterns are subject to change because they reflect not only a changing world but also our changing understanding.

Special thanks to TOM KHABAZA, Sir...

I hope you learn a lot from this article.

Thanks for reading.

If you want to get a regular update from us, please subscribe to the form below.