Introduction to Data Mining

Mauro Sozio
HKU:[email protected]

Telecom Paristech (Paris): [email protected]

 Big Data: a Paradigm Shift

! Study conducted by Stanley Milgram in 1969.

! Questions:
! What is the probability that two random people in
the world know each other?
! How many hops between them? (e.g. friend of
friend of friend = 3 hops.)

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Small-world experiment and
six degree of separation

! Experiment:
! Random people from Nebraska, Kansas,..., were
sent a letter with the goal of forwarding it to a
random person in Boston.
! If the person knew that person then he/she could
send him/her the letter directly.
! Otherwise she could forward the letter to a
relative or a friend who might know the person.
! Some basic information about the target person
were included.
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Small-world experiment and
six degree of separation

! Results:
! only 64 out of 296
letters reach the
destination (some people
refused to participate)
! among those reaching
the destination, the
average number of hops
was ~5-6.

-> six degree of separation

COMP7103: Introduction to Data Mining HKU, Hong Kong
 Six degree of separation
in the BigData era

! Similar study on Facebook with more than 1

billion users!
! Sophisticated algorithms estimated the
average path length between users: 4!

References:
Travers, Jeffrey & Stanley Milgram. 1969. "An Experimental Study of the Small World Problem." Sociometry, Vol.
32, No. 4, pp. 425-443.
Lars Backstrom, Paolo Boldi, Marco Rosa, Johan Ugander, Sebastiano Vigna: Four degrees of separation. WebSci 2012:33-42

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Google Trends

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Google Books Ngram

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Ranking Web Pages
! Googling “Stanford University”
! Stanford University Official web page
! Stanford University Press...
! Stanford health care...
! ...

! Pages are ranked according to their

importance... How? PageRank Algorithm...

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Finding best football teams with PageRank

COMP7103: Introduction to Data Mining HKU, Hong Kong

 The BigData Revolution
! BigData is revolutionizing:
! Crime prevention. We can
predict crimes by mining
past data.
! Healthcare. Mining query
logs and Twitter for finding
flu trends.
! Detecting earthquakes with
Twitter. Flu Trends around the world

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Why Mine Data? Commercial Viewpoint

! Lots of data is generated:

! Web data, e-commerce
! purchases at department/grocery store
! Bank/Credit card transactions

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Why Mine Data? Scientific Viewpoint

! Data collected from:

! remote sensors on a satellite
! telescopes scanning the skies
! microarrays generating expression
data
! scientific simulations
! people using Facebook, Twitter,
Google
! Data mining may help scientists
! in classifying and segmenting data
! in hypothesis formations
! modeling real-world phenomenon,
human behaviour
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Data Mining: Definition

! Many definitions:
! Non-trivial extraction of implicit, previously unknown

and potentially useful information from data

! Exploration & analysis, by automatic or semi-

automatic means, of large quantities of data

in order to discover meaningful patterns.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Origins of Data Mining
! Uses ideas and techniques from: machine learning/AI, pattern recognition,
statistics, theory of algorithms, database systems,...

Machine Learning/
Statistics/ Pattern
AI Recognition

Data Mining

Database
systems
! Issues:
! Massive amount of data
! High dimensionality
! Heterogenous, distributed nature of data
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Data Mining Tasks
! Prediction Methods
! Use some variables to predict unknown or future
values of other variables.

! Description Methods
! Find human-interpretable patterns that describe
the data.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Data Mining Tasks
! Classification [Predictive]
! Clustering [Descriptive]
! Association Rule Discovery [Descriptive]
! Sequential Pattern Discovery [Descriptive]
! Regression [Predictive]
! Deviation Detection [Predictive]
! Ranking [Descriptive]
! Recommendation Systems [Predictive]
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Classification: Definition

! Given a collection of records (training set)

! Each record contains a set of attributes, one of the

attributes is the class.

! Find a model for class attribute as a function of the
values of other attributes.
! Goal: previously unseen records should be assigned a
class as accurately as possible.
! A test set is used to determine the accuracy of the

model. Usually, the given data set is divided into

training and test sets, with training set used to build
the model and test set used to validate it.
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Classification Example
c al c al us
i i o
gor gor inu
te te nt ss
a a o a
c c c cl

Test
Set

Learn
Training Model
Set Classifier

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Classification: Application 1
l Direct Marketing
– Goal: Target a set of consumers likely to buy a new
cell-phone product.
– Approach:
" Use the data for a similar product introduced before.
" We know which customers decided to buy and which
decided otherwise. This {buy, don’t buy} decision forms the
class attribute.
" Collect various demographic, lifestyle, and company-
interaction related information about all such customers.
! Type of business, where they stay, how much they earn, etc.
" Use this information as input attributes to learn a classifier
model.
From [Berry & Linoff] Data Mining Techniques, 1997

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Classification: Application 2
l Fraud Detection
– Goal: Predict fraudulent cases in credit card
transactions.
– Approach:
" Use credit card transactions and the information on its
account-holder as attributes.
! When does a customer buy, what does he buy, how often he
pays on time, etc
" Label past transactions as fraud or fair transactions. This
forms the class attribute.
" Learn a model for the class of the transactions.
" Use this model to detect fraud by observing credit card
transactions on an account.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Classification: Application 3
l Customer Attrition/Churn:
– Goal: To predict whether a customer is likely to
be lost to a competitor.
– Approach:
" Use detailed record of transactions with each of the
past and present customers, to find attributes.
! How often the customer calls, where he calls, what time-of-
the day he calls most, his financial status, marital status, etc.
" Label the customers as loyal or disloyal.
" Find a model for loyalty.

From [Berry & Linoff] Data Mining Techniques, 1997

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Classification: Application 4
l Sky Survey Cataloging
– Goal: To predict class (star or galaxy) of sky objects,
especially visually faint ones, based on the telescopic
survey images (from Palomar Observatory).
! 3000 images with 23,040 x 23,040 pixels per image.
– Approach:
" Segment the image.
" Measure image attributes (features) - 40 of them per object.

" Model the class based on these features.

" Success Story: Could find 16 new high red-shift quasars, some of
the farthest objects that are difficult to find!

From [Fayyad, et.al.] Advances in Knowledge Discovery and Data Mining,

1996
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Classifying Galaxies
Courtesy: http://aps.umn.edu

Early Class: Attributes:

• Stages of Formation • Image features,
• Characteristics of light
waves received, etc.
Intermediate

Late

Data Size:
• 72 million stars, 20 million galaxies
• Object Catalog: 9 GB
• Image Database: 150 GB

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Clustering Definition
l Given a set of data points, each having a set
of attributes, and a similarity measure among
them, find clusters such that
– Data points in one cluster are more similar to one
another.
– Data points in separate clusters are less similar to
one another.
l Similarity Measures:
– Euclidean Distance if attributes are continuous.
– Other Problem-specific Measures.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Illustrating Clustering
x Euclidean Distance Based Clustering in 3-D space.

Intracluster distances Intercluster distances

are minimized are maximized

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Clustering: Application 1
l Market Segmentation:
– Goal: subdivide a market into distinct subsets of
customers where any subset may conceivably be
selected as a market target to be reached with a
distinct marketing mix.
– Approach:
" Collect different attributes of customers based on their
geographical and lifestyle related information.
" Find clusters of similar customers.
" Measure the clustering quality by observing buying patterns
of customers in same cluster vs. those from different
clusters.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Clustering: Application 2
l Document Clustering:
– Goal: To find groups of documents that are
similar to each other based on the important
terms appearing in them.
– Approach: To identify frequently occurring terms
in each document. Form a similarity measure
based on the frequencies of different terms. Use
it to cluster.
– Gain: Information Retrieval can utilize the
clusters to relate a new document or search term
to clustered documents.
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Document Clustering
l Clustering Points: 3204 Articles of Los Angeles Times.
l Similarity Measure: How many words are common in
these documents (after some word filtering).

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Clustering of S&P 500 Stock
Observe Stock Movements every day.
Clustering points: Stock-{UP/DOWN}
Similarity Measure: Two points are more similar if the events
described by them frequently happen together on the same day.
We used association rules to quantify a similarity measure.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Association Rule Discovery:
! Given a set of records each of which contain some number of
items from a given collection;
! Produce dependency rules which will predict occurrence of

an item based on occurrences of other items.

Rules Discovered:
{Milk} --> {Coke}
{Diaper, Milk} --> {Beer}

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Association Rule Discovery: Application 1

l Marketing and Sales Promotion:

– Let the rule discovered be
{Bagels, … } --> {Potato Chips}
– Potato Chips as consequent => Can be used to
determine what should be done to boost its sales.
– Bagels in the antecedent => Can be used to see
which products would be affected if the store
discontinues selling bagels.
– Bagels in antecedent and Potato chips in consequent
=> Can be used to see what products should be sold
with Bagels to promote sale of Potato chips!

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Association Rule Discovery: Application 2

l Supermarket shelf management.

– Goal: To identify items that are bought together
by sufficiently many customers.
– Approach: Process the point-of-sale data
collected with barcode scanners to find
dependencies among items.
– A classic rule --
" If a customer buys diaper and milk, then he is very
likely to buy beer.
" So, don’t be surprised if you find six-packs stacked
next to diapers!
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Association Rule Discovery: Application 3

! Inventory Management:
! Goal: A consumer appliance repair company wants to

anticipate the nature of repairs on its consumer products

and keep the service vehicles equipped with right parts to
reduce on number of visits to consumer households.
! Approach: Process the data on tools and parts required in

previous repairs at different consumer locations and

discover the co-occurrence patterns.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Sequential Pattern Discovery:
l Given is a set of objects, with each object associated with its own timeline
of events, find rules that predict strong sequential dependencies among
different events.

(A B) (C) (D E)

l Rules are formed by first disovering patterns. Event occurrences in the

patterns are governed by timing constraints.

(A B) (C) (D E)
<= xg >ng <= ws

<= ms

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Sequential Pattern Discovery:
! In telecommunications alarm logs,
! (Inverter_Problem Excessive_Line_Current)
(Rectifier_Alarm) --> (Fire_Alarm)
! In point-of-sale transaction sequences,
! Computer Bookstore:
(Intro_To_Visual_C) (C++_Primer) -->
(Python_for_dummies,Tcl_Tk)
! Athletic Apparel Store:
(Shoes) (Racket, Racketball) --> (Sports_Jacket)

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Regression
l Predict a value of a given continuous valued variable
based on the values of other variables, assuming a linear
or nonlinear model of dependency.
l Greatly studied in statistics, neural network fields.
l Examples:
– Predicting sales amounts of new product based on
advertising costs.
– Predicting wind velocities as a function of
temperature, humidity, air pressure, etc.
– Time series prediction of stock market indices.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Deviation/Anomaly Detection
! Detect significant deviations from normal
behavior
! Applications:
! Credit Card Fraud Detection

! Network Intrusion
Detection

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Challenges of Data Mining
l Scalability
l Dimensionality
l Complex and Heterogeneous Data
l Data Quality
l Data Ownership and Distribution
l Privacy Preservation
l Streaming Data

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Real-World Graphs
! Graphs represent pairwise relations between entities in the
real world (web pages, proteins, social network users,..)

! Real-world graphs are big! Facebook, 1.2 billion users,

Twitter 600M active users, 5k tweets per sec.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 MapReduce

Initially developed by Google, it is used by several

companies (Yahoo!, IBM) and universities (Cornell,
CMU... Telecom ParisTech).
COMP7103: Introduction to Data Mining HKU, Hong Kong
 Our class
! We will cover:
! PageRank, Clustering (K-Means), Frequent
Itemsets, Recommender Systems, Short
introduction to MapReduce, Classification,...
! Books:
! Mining of Massive Datasets. J. Leskovec, A. Rajaraman, J. D.
Ullman (available online).
! Introduction to Data Mining P. Tan, M. Steinbach, V. Kumar.
! Evaluation: 50% final exam, 20% midterm evaluation, 15%
for each of the two assignments.

COMP7103: Introduction to Data Mining HKU, Hong Kong

 Schedule
! To be finalised in a couple of days… stay
tuned!

COMP7103: Introduction to Data Mining HKU, Hong Kong