Data Mining:

Concepts and
Techniques
— Chapter 1 —
— Introduction —

Jiawei Han and Micheline Kamber

Department of Computer Science
University of Illinois at Urbana-Champaign
www.cs.uiuc.edu/~hanj
©2006 Jiawei Han and Micheline Kamber. All rights reserved.
Data Mining: Concepts and
November 23, 2024 Techniques 1
 Data Mining: Concepts and
November 23, 2024 Techniques 6
 Chapter 1. Introduction

 Motivation: Why data mining?

 What is data mining?
 Data Mining: On what kind of data?
 Data mining functionality
 Classification of data mining systems
 Top-10 most popular data mining algorithms
 Major issues in data mining
 Overview of the course
Data Mining: Concepts and
November 23, 2024 Techniques 7
 Why Data Mining?
 The Explosive Growth of Data: from terabytes to petabytes

Data collection and data availability

Automated data collection tools, database systems, Web,
computerized society

Major sources of abundant data

Business: Web, e-commerce, transactions, stocks, …

Science: Remote sensing, bioinformatics, scientific
simulation, …

Society and everyone: news, digital cameras, YouTube
 We are drowning in data, but starving for knowledge!
 “Necessity is the mother of invention”—Data mining—
Automated analysis of massive data sets
Data Mining: Concepts and
November 23, 2024 Techniques 8
 Evolution of Sciences
 Before 1600, empirical science
 1600-1950s, theoretical science
 Each discipline has grown a theoretical component. Theoretical models often
motivate experiments and generalize our understanding.
 1950s-1990s, computational science
 Over the last 50 years, most disciplines have grown a third, computational
branch (e.g. empirical, theoretical, and computational ecology, or physics, or
linguistics.)
 Computational Science traditionally meant simulation. It grew out of our
inability to find closed-form solutions for complex mathematical models.
 1990-now, data science
 The flood of data from new scientific instruments and simulations
 The ability to economically store and manage petabytes of data online
 The Internet and computing Grid that makes all these archives universally
accessible
 Scientific info. management, acquisition, organization, query, and visualization
tasks scale almost linearly with data volumes. Data mining is a major new
challenge!
 Jim Gray and Alex Szalay, The World Wide Telescope: An Archetype for Online
Data Mining: Concepts and
Science,
November 23, 2024Comm. ACM, 45(11): 50-54, Nov. 2002
Techniques 9
 What Is Data Mining?

 Data mining (knowledge discovery from data)

 Extraction of interesting (non-trivial, implicit, previously
unknown and potentially useful) patterns or knowledge
from huge amount of data
 Data mining: a misnomer?
 Alternative names
 Knowledge discovery (mining) in databases (KDD),
knowledge extraction, data/pattern analysis, data
archeology, data dredging, information harvesting,
business intelligence, etc.
 Watch out: Is everything “data mining”?
 Simple search and query processing
 (Deductive) expert Data
systems
Mining: Concepts and
November 23, 2024 Techniques 11
 Knowledge Discovery (KDD) Process

 Data mining—core of Pattern Evaluation

knowledge discovery
process
Data Mining

Task-relevant Data

Data Selection
Warehouse
Data Cleaning

Data Integration

Databases Data Mining: Concepts and

November 23, 2024 Techniques 12
 Data Mining and Business
Intelligence
Increasing potential
to support
business decisions End User
Decisio
n
Making
Data Presentation Business
Analyst
Visualization Techniques
Data Mining Data
Information Discovery Analyst

Data Exploration
Statistical Summary, Querying, and Reporting

Data Preprocessing/Integration, Data Warehouses

DBA
Data Sources
Paper, Files, Web documents, Scientific experiments, Database Systems
Data Mining: Concepts and
November 23, 2024 Techniques 13
 Data Mining: Confluence of Multiple
Disciplines

Database
Technology Statistics

Machine Visualization
Learning Data Mining

Pattern
Recognition Other
Algorithm Disciplines

Data Mining: Concepts and

November 23, 2024 Techniques 14
 Why Not Traditional Data
Analysis?
 Tremendous amount of data
 Algorithms must be highly scalable to handle such as tera-
bytes of data
 High-dimensionality of data
 Micro-array may have tens of thousands of dimensions
 High complexity of data
 Data streams and sensor data
 Time-series data, temporal data, sequence data
 Structure data, graphs, social networks and multi-linked data
 Heterogeneous databases and legacy databases
 Spatial, spatiotemporal, multimedia, text and Web data
 Software programs, scientific simulations
 New and sophisticated applications
Data Mining: Concepts and
November 23, 2024 Techniques 15
 Multi-Dimensional View of Data
Mining
 Data to be mined
 Relational, data warehouse, transactional, stream, object-
oriented/relational, active, spatial, time-series, text, multi-
media, heterogeneous, legacy, WWW
 Knowledge to be mined
 Characterization, discrimination, association, classification,
clustering, trend/deviation, outlier analysis, etc.
 Multiple/integrated functions and mining at multiple levels
 Techniques utilized
 Database-oriented, data warehouse (OLAP), machine learning,
statistics, visualization, etc.
 Applications adapted
 Retail, telecommunication, banking, fraud analysis, bio-data
mining, stock market analysis, text mining, Web mining, etc.
Data Mining: Concepts and
November 23, 2024 Techniques 16
 Data Mining: Concepts and
November 23, 2024 Techniques 17
 Data Mining: Concepts and
November 23, 2024 Techniques 18
 Data Mining: Concepts and
November 23, 2024 Techniques 19
 Data Mining: Concepts and
November 23, 2024 Techniques 20
 Data Mining: Classification Schemes

 General functionality
 Descriptive data mining
 Predictive data mining
 Different views lead to different classifications
 Data view: Kinds of data to be mined
 Knowledge view: Kinds of knowledge to be
discovered
 Method view: Kinds of techniques utilized
 Application view: Kinds of applications adapted
Data Mining: Concepts and
November 23, 2024 Techniques 21
 Data Mining: On What Kinds of
Data?
 Database-oriented data sets and applications
 Relational database, data warehouse, transactional database
 Advanced data sets and advanced applications
 Data streams and sensor data
 Time-series data, temporal data, sequence data (incl. bio-
sequences)
 Structure data, graphs, social networks and multi-linked data
 Object-relational databases
 Heterogeneous databases and legacy databases
 Spatial data and spatiotemporal data
 Multimedia database
 Text databases
 The World-Wide Web Data Mining: Concepts and
November 23, 2024 Techniques 22
 Data Mining Functionalities
 Multidimensional concept description: Characterization and
discrimination
 Generalize, summarize, and contrast data
characteristics, e.g., dry vs. wet regions
 Frequent patterns, association, correlation vs. causality
 Diaper  Beer [0.5%, 75%] (Correlation or causality?)
 Classification and prediction
 Construct models (functions) that describe and
distinguish classes or concepts for future prediction

E.g., classify countries based on (climate), or classify
cars based on (gas mileage)
 Predict some unknown or missing numerical values
Data Mining: Concepts and
November 23, 2024 Techniques 23
 Data Mining Functionalities (2)
 Cluster analysis

Class label is unknown: Group data to form new classes, e.g.,
cluster houses to find distribution patterns

Maximizing intra-class similarity & minimizing interclass similarity
 Outlier analysis

Outlier: Data object that does not comply with the general behavior
of the data

Noise or exception? Useful in fraud detection, rare events analysis
 Trend and evolution analysis

Trend and deviation: e.g., regression analysis

Sequential pattern mining: e.g., digital camera  large SD memory

Periodicity analysis

Similarity-based analysis
 Other pattern-directed or statistical analyses

Data Mining: Concepts and

November 23, 2024 Techniques 24
 Major Issues in Data Mining
 Mining methodology
 Mining different kinds of knowledge from diverse data types, e.g., bio,
stream, Web
 Performance: efficiency, effectiveness, and scalability
 Pattern evaluation: the interestingness problem
 Incorporation of background knowledge
 Handling noise and incomplete data
 Parallel, distributed and incremental mining methods
 Integration of the discovered knowledge with existing one: knowledge
fusion
 User interaction
 Data mining query languages and ad-hoc mining
 Expression and visualization of data mining results
 Interactive mining of knowledge at multiple levels of abstraction
 Applications and social impacts
 Domain-specific data mining & invisible data mining
 Protection of data security,
Dataintegrity, and privacy
Mining: Concepts and
November 23, 2024 Techniques 29
 Summary

 Data mining: Discovering interesting patterns from large

amounts of data
 A natural evolution of database technology, in great demand,
with wide applications
 A KDD process includes data cleaning, data integration, data
selection, transformation, data mining, pattern evaluation, and
knowledge presentation
 Mining can be performed in a variety of information repositories
 Data mining functionalities: characterization, discrimination,
association, classification, clustering, outlier and trend analysis,
etc.
 Data mining systems and architectures
 Major issues in data mining
Data Mining: Concepts and
November 23, 2024 Techniques 34
 Why Data Mining?—Potential
Applications
 Data analysis and decision support
 Market analysis and management

Target marketing, customer relationship management
(CRM), market basket analysis, cross selling, market
segmentation
 Risk analysis and management

Forecasting, customer retention, improved underwriting,
quality control, competitive analysis
 Fraud detection and detection of unusual patterns (outliers)
 Other Applications
 Text mining (news group, email, documents) and Web mining
 Stream data mining
 Bioinformatics and bio-data analysis
Data Mining: Concepts and
November 23, 2024 Techniques 37
 Ex. 1: Market Analysis and
Management
 Where does the data come from?—Credit card transactions, loyalty
cards, discount coupons, customer complaint calls, plus (public) lifestyle
studies
 Target marketing
 Find clusters of “model” customers who share the same characteristics:
interest, income level, spending habits, etc.
 Determine customer purchasing patterns over time
 Cross-market analysis—Find associations/co-relations between product
sales, & predict based on such association
 Customer profiling—What types of customers buy what products
(clustering or classification)
 Customer requirement analysis
 Identify the best products for different groups of customers
 Predict what factors will attract new customers
 Provision of summary information
 Multidimensional summary reports
 Statistical summary information (data central tendency and variation)
Data Mining: Concepts and
November 23, 2024 Techniques 38
 Ex. 2: Corporate Analysis & Risk
Management
 Finance planning and asset evaluation

cash flow analysis and prediction

contingent claim analysis to evaluate assets

cross-sectional and time series analysis (financial-ratio,
trend analysis, etc.)
 Resource planning

summarize and compare the resources and spending
 Competition

monitor competitors and market directions

group customers into classes and a class-based pricing
procedure

set pricing strategy in a highly competitive market
Data Mining: Concepts and
November 23, 2024 Techniques 39
 Ex. 3: Fraud Detection & Mining Unusual
Patterns
 Approaches: Clustering & model construction for frauds, outlier
analysis
 Applications: Health care, retail, credit card service, telecomm.

Auto insurance: ring of collisions
 Money laundering: suspicious monetary transactions

Medical insurance

Professional patients, ring of doctors, and ring of references

Unnecessary or correlated screening tests

Telecommunications: phone-call fraud

Phone call model: destination of the call, duration, time of day
or week. Analyze patterns that deviate from an expected norm
 Retail industry

Analysts estimate that 38% of retail shrink is due to dishonest
employees
 Anti-terrorism
Data Mining: Concepts and
November 23, 2024 Techniques 40
 KDD Process: Several Key
Steps
 Learning the application domain

relevant prior knowledge and goals of application
 Creating a target data set: data selection
 Data cleaning and preprocessing: (may take 60% of effort!)
 Data reduction and transformation

Find useful features, dimensionality/variable reduction, invariant
representation
 Choosing functions of data mining

summarization, classification, regression, association, clustering
 Choosing the mining algorithm(s)
 Data mining: search for patterns of interest
 Pattern evaluation and knowledge presentation

visualization, transformation, removing redundant patterns, etc.
 Use of discovered knowledge
Data Mining: Concepts and
November 23, 2024 Techniques 41
 Are All the “Discovered” Patterns
Interesting?
 Data mining may generate thousands of patterns: Not all of
them are interesting
 Suggested approach: Human-centered, query-based, focused
mining
 Interestingness measures
 A pattern is interesting if it is easily understood by humans, valid
on new or test data with some degree of certainty, potentially
useful, novel, or validates some hypothesis that a user seeks to
confirm
 Objective vs. subjective interestingness measures
 Objective: based on statistics and structures of patterns, e.g.,
support, confidence, etc.
 Subjective: based on user’s belief in the data, e.g.,
Data actionability,
unexpectedness, novelty, Mining: Concepts and
etc.
November 23, 2024 Techniques 42
 Find All and Only Interesting
Patterns?
 Find all the interesting patterns: Completeness
 Can a data mining system find all the interesting patterns?
Do we need to find all of the interesting patterns?
 Heuristic vs. exhaustive search
 Association vs. classification vs. clustering
 Search for only interesting patterns: An optimization problem
 Can a data mining system find only the interesting
patterns?
 Approaches

First general all the patterns and then filter out the
uninteresting ones

Generate only the interesting patterns—mining query
Data Mining: Concepts and
optimization
November 23, 2024 Techniques 43
 Why Data Mining Query Language?

 Automated vs. query-driven?


Finding all the patterns autonomously in a database?—
unrealistic because the patterns could be too many but
uninteresting
 Data mining should be an interactive process

User directs what to be mined
 Users must be provided with a set of primitives to be used to
communicate with the data mining system
 Incorporating these primitives in a data mining query
language

More flexible user interaction

Foundation for design of graphical user interface

Standardization of data mining industry and practice
Data Mining: Concepts and
November 23, 2024 Techniques 46
 Primitives that Define a Data Mining
Task
 Task-relevant data
 Database or data warehouse name
 Database tables or data warehouse cubes
 Condition for data selection
 Relevant attributes or dimensions
 Data grouping criteria
 Type of knowledge to be mined
 Characterization, discrimination, association, classification,
prediction, clustering, outlier analysis, other data mining
tasks
 Background knowledge
 Pattern interestingness measurements
 Visualization/presentation of discovered
Data Mining: Concepts and patterns
November 23, 2024 Techniques 47
 Primitive 3: Background Knowledge

 A typical kind of background knowledge: Concept hierarchies

 Schema hierarchy
 E.g., street < city < province_or_state < country
 Set-grouping hierarchy
 E.g., {20-39} = young, {40-59} = middle_aged
 Operation-derived hierarchy
 email address: [email protected]
login-name < department < university < country
 Rule-based hierarchy
 low_profit_margin (X) <= price(X, P1) and cost (X, P2) and
(P1 - P2) < $50
Data Mining: Concepts and
November 23, 2024 Techniques 48
 Primitive 4: Pattern Interestingness
Measure

 Simplicity
e.g., (association) rule length, (decision) tree size
 Certainty
e.g., confidence, P(A|B) = #(A and B)/ #(B), classification
reliability or accuracy, certainty factor, rule strength, rule
quality, discriminating weight, etc.
 Utility
potential usefulness, e.g., support (association), noise
threshold (description)
 Novelty
not previously known, surprising (used to remove
redundant rules, e.g., Illinois vs. Champaign rule
implication support Data
ratio)
Mining: Concepts and
November 23, 2024 Techniques 49
 Primitive 5: Presentation of Discovered
Patterns

 Different backgrounds/usages may require different forms of

representation
 E.g., rules, tables, crosstabs, pie/bar chart, etc.
 Concept hierarchy is also important
 Discovered knowledge might be more understandable
when represented at high level of abstraction
 Interactive drill up/down, pivoting, slicing and dicing
provide different perspectives to data
 Different kinds of knowledge require different
representation: association, classification, clustering, etc.

Data Mining: Concepts and

November 23, 2024 Techniques 50
 DMQL—A Data Mining Query
Language

 Motivation
 A DMQL can provide the ability to support ad-hoc and
interactive data mining
 By providing a standardized language like SQL

Hope to achieve a similar effect like that SQL has
on relational database

Foundation for system development and evolution

Facilitate information exchange, technology
transfer, commercialization and wide acceptance
 Design
 DMQL is designed with the primitives described earlier

Data Mining: Concepts and

November 23, 2024 Techniques 51
 An Example Query in DMQL

Data Mining: Concepts and

November 23, 2024 Techniques 52
 Architecture: Typical Data Mining
System

Graphical User Interface

Pattern Evaluation
Know
Data Mining Engine ledge
-Base
Database or Data
Warehouse Server

data cleaning, integration, and selection

Data World-Wide Other Info

Database Repositories
Warehouse Web
Data Mining: Concepts and
November 23, 2024 Techniques 56