Data mining, or knowledge discovery, is the computer-assisted process of digging

through and analyzing enormous sets of data and then extracting the meaning of the data.
Data mining tools predict behaviors and future trends, allowing businesses to make
proactive, knowledge-driven decisions. Data mining tools can answer business questions
that traditionally were too time consuming to resolve. They scour databases for hidden
patterns, finding predictive information that experts may miss because it lies outside their
expectations.
Data mining derives its name from the similarities between searching for valuable
information in a large database and mining a mountain for a vein of valuable ore. Both
processes require either sifting through an immense amount of material, or intelligently
probing it to find where the value resides.
Notable uses

Games
Since the early 1960s, with the availability of oracles for certain combinatorial games,
also called tablebases (e.g. for 3x3-chess) with any beginning configuration, small-board
dots-and-boxes, small-board-hex, and certain endgames in chess, dots-and-boxes, and
hex; a new area for data mining has been opened. This is the extraction of human-usable
strategies from these oracles. Current pattern recognition approaches do not seem to fully
acquire the high level of abstraction required to be applied successfully. Instead,
extensive experimentation with the tablebases combined with an intensive study of
tablebase-answers to well designed problems, and with knowledge of prior art (i.e. pretablebase knowledge) is used to yield insightful patterns. Berlekamp (in dots-andboxes, etc.) and John Nunn (in chess endgames) are notable examples of researchers
doing this work, though they were not and are not involved in tablebase generation.

Business
Data mining in customer relationship management applications can contribute
significantly to the bottom line.[citation needed] Rather than randomly contacting a prospect or
customer through a call center or sending mail, a company can concentrate its efforts on
prospects that are predicted to have a high likelihood of responding to an offer. More
sophisticated methods may be used to optimize resources across campaigns so that one
may predict to which channel and to which offer an individual is most likely to respond
(across all potential offers). Additionally, sophisticated applications could be used to
automate mailing. Once the results from data mining (potential prospect/customer and
channel/offer) are determined, this "sophisticated application" can either automatically
send an e-mail or a regular mail. Finally, in cases where many people will take an action
without an offer, "uplift modeling" can be used to determine which people have the
greatest increase in response if given an offer. Uplift modeling thereby enables marketers
to focus mailings and offers on persuadable people, and not to send offers to people who
will buy the product without an offer. Data clustering can also be used to automatically
discover the segments or groups within a customer data set.

Businesses employing data mining may see a return on investment, but also they
recognize that the number of predictive models can quickly become very large. Rather
than using one model to predict how many customers will churn, a business could build a
separate model for each region and customer type. Then, instead of sending an offer to all
people that are likely to churn, it may only want to send offers to loyal customers. Finally,
the business may want to determine which customers are going to be profitable over a
certain window in time, and only send the offers to those that are likely to be profitable.
In order to maintain this quantity of models, they need to manage model versions and
move on to automated data mining.
Data mining can also be helpful to human resources (HR) departments in identifying the
characteristics of their most successful employees. Information obtained such as
universities attended by highly successful employees can help HR focus recruiting
efforts accordingly. Additionally, Strategic Enterprise Management applications help a
company translate corporate-level goals, such as profit and margin share targets, into
operational decisions, such as production plans and workforce levels.[17]
Another example of data mining, often called the market basket analysis, relates to its use
in retail sales. If a clothing store records the purchases of customers, a data mining
system could identify those customers who favor silk shirts over cotton ones. Although
some explanations of relationships may be difficult, taking advantage of it is easier. The
example deals with association rules within transaction-based data. Not all data are
transaction based and logical, or inexact rules may also be present within a database.
Market basket analysis has also been used to identify the purchase patterns of the Alpha
Consumer. Alpha Consumers are people that play a key role in connecting with the
concept behind a product, then adopting that product, and finally validating it for the rest
of society. Analyzing the data collected on this type of user has allowed companies to
predict future buying trends and forecast supply demands.[citation needed]
Data mining is a highly effective tool in the catalog marketing industry.[citation needed]
Catalogers have a rich database of history of their customer transactions for millions of
customers dating back a number of years. Data mining tools can identify patterns among
customers and help identify the most likely customers to respond to upcoming mailing
campaigns.
Data mining for business applications is a component which needs to be integrated into a
complex modeling and decision making process. Reactive business intelligence (RBI)
advocates a "holistic" approach that integrates data mining, modeling, and interactive
visualization into an end-to-end discovery and continuous innovation process powered by
human and automated learning.[18]
In the area of decision making, the RBI approach has been used to mine knowledge that
is progressively acquired from the decision maker, and then self-tune the decision method
accordingly.[19]
An example of data mining related to an integrated-circuit production line is described in
the paper "Mining IC Test Data to Optimize VLSI Testing."[20] In this paper, the

application of data mining and decision analysis to the problem of die-level functional
testing is described. Experiments mentioned demonstrate the ability to apply a system of
mining historical die-test data to create a probabilistic model of patterns of die failure.
These patterns are then utilized to decide, in real time, which die to test next and when to
stop testing. This system has been shown, based on experiments with historical test data,
to have the potential to improve profits on mature IC products.

Science and engineering

In recent years, data mining has been used widely in the areas of science and engineering,
such as bioinformatics, genetics, medicine, education and electrical power engineering.
In the study of human genetics, sequence mining helps address the important goal of
understanding the mapping relationship between the inter-individual variations in human
DNA sequence and the variability in disease susceptibility. In simple terms, it aims to
find out how the changes in an individual's DNA sequence affects the risks of developing
common diseases such as cancer, which is of great importance to improving methods of
diagnosing, preventing, and treating these diseases. The data mining method that is used
to perform this task is known as multifactor dimensionality reduction.[21]
In the area of electrical power engineering, data mining methods have been widely used
for condition monitoring of high voltage electrical equipment. The purpose of condition
monitoring is to obtain valuable information on, for example, the status of the insulation
(or other important safety-related parameters). Data clustering techniques such as the
self-organizing map (SOM), have been applied to vibration monitoring and analysis of
transformer on-load tap-changers (OLTCS). Using vibration monitoring, it can be
observed that each tap change operation generates a signal that contains information
about the condition of the tap changer contacts and the drive mechanisms. Obviously,
different tap positions will generate different signals. However, there was considerable
variability amongst normal condition signals for exactly the same tap position. SOM has
been applied to detect abnormal conditions and to hypothesize about the nature of the
abnormalities.[22]
Data mining methods have also been applied to dissolved gas analysis (DGA) in power
transformers. DGA, as a diagnostics for power transformers, has been available for many
years. Methods such as SOM has been applied to analyze generated data and to determine
trends which are not obvious to the standard DGA ratio methods (such as Duval
Triangle).[22]
Another example of data mining in science and engineering is found in educational
research, where data mining has been used to study the factors leading students to choose
to engage in behaviors which reduce their learning,[23] and to understand factors
influencing university student retention.[24] A similar example of social application of data
mining is its use in expertise finding systems, whereby descriptors of human expertise are
extracted, normalized, and classified so as to facilitate the finding of experts, particularly
in scientific and technical fields. In this way, data mining can facilitate institutional
memory.

Other examples of application of data mining methods are biomedical data facilitated by
domain ontologies,[25] mining clinical trial data,[26] and traffic analysis using SOM.[27]
In adverse drug reaction surveillance, the Uppsala Monitoring Centre has, since 1998,
used data mining methods to routinely screen for reporting patterns indicative of
emerging drug safety issues in the WHO global database of 4.6 million suspected adverse
drug reaction incidents.[28] Recently, similar methodology has been developed to mine
large collections of electronic health records for temporal patterns associating drug
prescriptions to medical diagnoses.[29]
Data mining has been applied software artifacts within the realm of software engineering:
Mining Software Repositories

Visual data mining

In the process of turning from analogical into digital, large data sets have been generated,
collected, and stored discovering statistical patterns, trends and information which is
hidden in data, in order to build predictive patterns. Studies suggest visual data mining is
faster and much more intuitive than is traditional data mining.[38][39]

[edit] Music data mining

Data mining techniques, and in particular co-occurrence analysis, has been used to
discover relevant similarities among music corpora (radio lists, CD databases) for the
purpose of classifying music into genres in a more objective manner.[40]

[edit] Surveillance
Data mining has been used to stop terrorist programs under the U.S. government,
including the Total Information Awareness (TIA) program, Secure Flight (formerly
known as Computer-Assisted Passenger Prescreening System (CAPPS II)), Analysis,
Dissemination, Visualization, Insight, Semantic Enhancement (ADVISE),[41] and the
Multi-state Anti-Terrorism Information Exchange (MATRIX).[42] These programs have
been discontinued due to controversy over whether they violate the 4th Amendment to
the United States Constitution, although many programs that were formed under them
continue to be funded by different organizations or under different names.[43]
In the context of combating terrorism, two particularly plausible methods of data mining
are "pattern mining" and "subject-based data mining".

[edit] Pattern mining

"Pattern mining" is a data mining method that involves finding existing patterns in data.
In this context patterns often means association rules. The original motivation for
searching association rules came from the desire to analyze supermarket transaction data,
that is, to examine customer behavior in terms of the purchased products. For example,

an association rule "beer potato chips (80%)" states that four out of five customers that
bought beer also bought potato chips.
In the context of pattern mining as a tool to identify terrorist activity, the National
Research Council provides the following definition: "Pattern-based data mining looks for
patterns (including anomalous data patterns) that might be associated with terrorist
activity these patterns might be regarded as small signals in a large ocean of noise."[44]
[45][46]
Pattern Mining includes new areas such a Music Information Retrieval (MIR) where
patterns seen both in the temporal and non temporal domains are imported to classical
knowledge discovery search methods.

[edit] Subject-based data mining

"Subject-based data mining" is a data mining method involving the search for
associations between individuals in data. In the context of combating terrorism, the
National Research Council provides the following definition: "Subject-based data mining
uses an initiating individual or other datum that is considered, based on other
information, to be of high interest, and the goal is to determine what other persons or
financial transactions or movements, etc., are related to that initiating datum