What is Data Mining: Part 1

Data mining tells us about very large and complex data sets, the kinds of information that would be readily apparent about small and simple things.

The Big Data bubble is everywhere. Businesses need to be on top of their game to understand and execute effectively. In the recent years, there has been an extensive surge in the amount of data that is being accumulated. Every transaction over the internet is getting captured and stored. At these scales patterns are often too subtle and relationships too complex or multi-dimensional to observe by simply looking at the data.

It is here that Data Mining comes in to the picture. It essentially enables corporations to use the information that is available to decode more about the users than what they wish to declare. Data mining is a means of automating this part of the process to detect interpretable patterns; it helps one see the forest without getting lost in the trees.

The application of Data Mining process depends highly on finding the pattern. These generic processes illustrate what data mining actually do.

• Anomaly detection
• Association learning
• Cluster detection
• Classification
• Regression

Data mining can grant immense inferential power. If an algorithm can correctly classify a case into known category based on limited data, it is possible to estimate a wide-range of other information about that case based on the properties of all the other cases in that category. This may sound dry, but it is how most successful Internet companies make their money and from where they draw their power.

The next part of the Data Mining series will look at What Data Analysts expect from IT?