Data Mining And Data Warehousing

(CSA4003)

Dr. Rudra Kalyan Nayak

Assistant Professor
School of Computing Science & Engineering

2/28/2022
 Module-2 Data Mining

 Motivation: Why data mining?

 What is data mining?
 Data Mining: On what kind of data?
 Data mining functionality
 Classification of data mining systems
 Major issues in data mining
 Data Mining Task Primitives
 Integration of a data mining system with a database or data warehouse
system

2/28/2022
2/28/2022
Motivation: Why data mining?

2/28/2022
 Evolution of Database Technology

 1960s and earlier:

 Data Collection and Database Creation
 Primitive file processing

2/28/2022
 Evolution of Database Technology
 1970s - early 1980s:
 Data Base Management Systems
 Hierarchical and network database systems
 Relational database Systems
 Query languages: SQL
 Transactions, concurrency control and recovery.
 On-line transaction processing (OLTP)

2/28/2022
 Evolution of Database Technology

 Mid -1980s - present:

 Advanced data models
 Extended relational, object-relational
 Advanced application-oriented DBMS
 spatial, scientific, engineering, temporal, multimedia,
active, stream and sensor, knowledge-based

2/28/2022
 Evolution of Database Technology

 Late 1980s-present
 Advanced Data Analysis
 Data warehouse and OLAP
 Data mining and knowledge discovery
 Advanced data mining applications
 Data mining and society
 1990s-present:
 XML-based database systems
 Integration with information retrieval
 Data and information integration

2/28/2022
 Evolution of Database Technology

 Present – future:
 New generation of integrated data and information
system.

2/28/2022
What Is Data Mining?

2/28/2022
 What Is Data Mining?

 Data mining refers to extracting or mining knowledge

from large amounts of data.
 Mining of gold from rocks or sand
 Knowledge mining from data, knowledge extraction,
data/pattern analysis, data archeology, and data
dredging.
 Knowledge Discovery from data, or KDD

2/28/2022
 Data Mining: A KDD Process

Pattern Evaluation
– Data mining: the core of
knowledge discovery process.
Data Mining

Task-relevant Data

Data Warehouse Selection

Transformation

Data Cleaning

Data Integration

Databases 2/28/2022
 Steps of a KDD Process

1. Data cleaning
2. Data integration
3. Data selection
4. Data transformation
5. Data mining
6. Pattern evaluation
7. Knowledge presentaion

2/28/2022
 Steps of a KDD Process
 Learning the application domain:
 relevant prior knowledge and goals of application
 Creating a target data set: data selection
 Data cleaning and preprocessing
 Data reduction and transformation:
 Find useful features, dimensionality/variable reduction,
invariant representation.

2/28/2022
 Steps of a KDD Process

 Choosing functions of data mining

 summarization, classification, regression, association,
clustering.
 Choosing the mining algorithms
 Data mining: search for patterns of interest
 Pattern evaluation and knowledge presentation
 visualization, transformation, removing redundant
patterns, etc.
 Use of discovered knowledge

2/28/2022
Architecture of a Typical Data
Mining System
Graphical user interface

Pattern evaluation

Data mining engine

Knowledge-base
Database or data
warehouse server
Data cleaning & data integration Filtering

Data
Databases Warehouse
2/28/2022
Data Mining and Business Intelligence
Increasing potential
to support
business decisions End User
Making
Decisions

Data Presentation Business

Analyst
Visualization Techniques
Data Mining Data
Information Discovery Analyst

Data Exploration
Statistical Analysis, Querying and Reporting

Data Warehouses / Data Marts

OLAP, MDA DBA
Data Sources
Paper, Files, Information Providers, Database Systems, OLTP
2/28/2022
Data Mining: On What Kind of Data?

2/28/2022
 Answer : On any kind of data.

2/28/2022
2/28/2022
 Data Mining: On What Kind of Data?

 Relational databases
 Data warehouses
 Transactional databases

2/28/2022
 Data Mining: On What Kind of Data?

 Advanced DB and information repositories

 Object-oriented and object-relational databases
 Spatial databases
 Time-series data and temporal data
 Text databases and multimedia databases
 Heterogeneous and legacy databases
 WWW

2/28/2022
Data Mining Functionalities -What
kind of patterns can be mined?

2/28/2022
Data mining tasks generally classified into two categories.

1. Descriptive :It characterize the general properties of data

in the database.
2. Predictive :It performs inference(interpretation) on the
current data in order to make predictions.

2/28/2022
 Data Mining Functionalities
 Concept description: Characterization and
discrimination
 Data can be associated with classes or concepts
 Ex. All Electronics store classes of items for sale include
computer and printers.
 Description of class or concept is called class/concept
description.
 Data characterization : summarization of general features
of target class of data.
 Data discrimination : comparison of target class with one
or more contrasting classes.
2/28/2022
 Data Mining Functionalities

 Association Analysis
 Multi-dimensional vs. single-dimensional association
 age(X, “20..29”) ^ income(X, “20..29K”) => buys(X,
“PC”) [support = 2%, confidence = 60%]
 contains(T, “computer”) => contains(x, “software”)
[support=1%, confidence=75%]

2/28/2022
 Data Mining Functionalities
 Classification and Prediction
 Finding models (functions) that describe and distinguish data
classes or concepts for predict the class whose label is
unknown
 E.g., classify countries based on climate, or classify cars based
on gas mileage
 Models: decision-tree, classification rules (if-then), neural
network

 Prediction: Predict some unknown or missing

numerical values

2/28/2022
 Data Mining Functionalities

 Cluster analysis
 Analyze class-labeled data objects, clustering analyze
data objects without consulting a known class label.
 Clustering based on the principle: maximizing the intra-
class similarity and minimizing the interclass similarity

2/28/2022
 Data Mining Functionalities
 Outlier analysis
 Outlier: a data object that does not comply(fulfill) with the general
behavior of the model of the data
 It can be considered as noise or exception but is quite useful in fraud
detection, rare events analysis

 Trend and evolution analysis

 Trend and deviation: regression analysis
 Sequential pattern mining, periodicity analysis
 Similarity-based analysis

2/28/2022
Data Mining: Classification Schemes

2/28/2022
 Data Mining: Confluence of Multiple
Disciplines
Database
Statistics
Technology

Information
Science Data Mining MachineLearning

Visualization Other
Disciplines
2/28/2022
Data Mining systems: Classification Schemes

 General functionality
 Descriptive data mining
 Predictive data mining

 Data mining various criteria's:

 Kinds of databases to be mined
 Kinds of knowledge to be discovered
 Kinds of techniques utilized
 Kinds of applications adopted

2/28/2022
 Data Mining: Classification Schemes
 Databases to be mined
 Relational, transactional, object-oriented, object-
relational, active, spatial, time-series, text, multi-media,
heterogeneous, legacy, WWW, etc.
 Knowledge to be mined
 Characterization, discrimination, association,
classification, clustering, trend, deviation and outlier
analysis, etc.
 Multiple/integrated functions and mining at multiple
levels

2/28/2022
 Data Mining: Classification Schemes

 Techniques utilized
 Database-oriented, data warehouse
(OLAP), machine learning, statistics,
visualization, neural network, etc.
 Applications adopted
 Retail, telecommunication, banking,
fraud analysis, DNA mining, stock market
2/28/2022
Major Issues in Data Mining

2/28/2022
 Major Issues in Data Mining

1. Mining methodology and user interaction issues

 Mining different kinds of knowledge in databases
 Interactive mining of knowledge at multiple levels of
abstraction
 Incorporation of background knowledge
 Data mining query languages and ad-hoc data mining
 Expression and visualization of data mining results
 Handling noise and incomplete data
 Pattern evaluation: the interestingness problem

2/28/2022
 Major Issues in Data Mining

2. Performance issues
 Efficiency and scalability of data mining algorithms
 Parallel, distributed and incremental mining methods

2/28/2022
 Major Issues in Data Mining

3. Issues relating to the diversity of data types

 Handling relational(simple) and complex types of data.

 Mining information from heterogeneous databases and global

information systems (WWW)

2/28/2022
 Data Mining Task Primitives

Datamining query is defined in terms of data mining task primitives.

2/28/2022
2/28/2022
2/28/2022
2/28/2022
ARCHITECTURE OF A TYPICAL DATA MINING
SYSTEM

2/28/2022
2/28/2022
2/28/2022
2/28/2022
2/28/2022
2/28/2022
Q::::: Describe the differences between the following approaches for
the integration of a data mining system with a database or data
warehouse system: no coupling, loose coupling, semitight coupling, and
tight coupling.
State which approach you think is the most popular and why?

2/28/2022