If you have managed to construct a data warehouse, you now have corporate and operational data organized to support informed decision making. The challenge is to find effective ways to analyze those data so as to extract valuable information.

Many people new to the field find it difficult to determine exactly what constitutes data mining. Unfortunately, many software vendors are also unclear, so many identify any summarizing and reporting activity as "data mining." If you are truly to benefit from the revolutionary technology known as data mining, you need to distinguish the descriptive reporting provided by OLAP from the predictive, artificial intelligence models generated by the latest in machine learning technology.

Data Mining versus OLAP

In a nutshell: OLAP tells you what, but only data mining can also tell you why. Furthermore, OLAP can answer generic, unstructured questions.

We can illustrate the difference between OLAP and data mining with an example from our consulting experience. A major residential mortgage lender wanted to be able to predict which of their customers would refinance their loans at any point in time. Because refinancing normally means going to another lender, refinancers are often customers to competitors. With advance information, however, the bank could give each borrower a refinance probability score, classify each borrower as desirable or undesirable on other criteria, and then attempt to keep desirable customers by offering a reduced interest rate. The data warehouse we constructed contained monthly information on each borrower; data on current employment, real estate, and financial markets; the difference between employment, real estate, and financial markets; the difference between the current interest rate and the rate of the original mortgage; and detailed credit bureau information. Using OLAP reporting alone, we could have examined refinance behavior from many perspectives, including by region, borrower credit score, trend in borrower credit score, local real estate conditions, trend in market interest rates, and so forth. To generate reports the analyst then would have had to decide what was relevant and request reports based on those variables. Generally, such reports are useful and comprehensible only if they have a small number of dimensions.