Scribd
Upload
23 views14 pages

Data Mining CSE4052 Lecture-13

Uploaded by

1941012358.j.hiaakundu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
23 views14 pages

Data Mining CSE4052 Lecture-13

Uploaded by

1941012358.j.hiaakundu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 14

Lecture-13

Data Mining

Data Warehouse and OLAP Technology for


Data Mining
(Basic Concepts)

Dr. Kaberi Das


Associate Professor
Department of Computer Science and Engineering
ITER, Siksha ‘O’ Anusandhan University.
Content

 What is Data Warehouse?


 Data Warehouse vs. Heterogeneous
 Data Warehouse vs. Operational DBMS
 OLTP vs. OLAP
 Why Separate Data Warehouse?
 Summary

04/14/2025 Data Preprocessing (Basic Concepts) 2


What is Data Warehouse?

• Defined in many different ways, but not rigorously.


• A decision support database that is maintained separately from the
organization’s operational database
• Support information processing by providing a solid platform of consolidated,
historical data for analysis.
• “A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile
collection of data in support of management’s decision-making process.”—W. H.
Inmon
• Data warehousing:
• The process of constructing and using data warehouses

04/14/2025 Data Preprocessing (Basic Concepts) 3


Data Warehouse—Subject-Oriented

• Organized around major subjects, such as customer, product, sales.


• Focusing on the modeling and analysis of data for decision makers, not on daily
operations or transaction processing.
• Provide a simple and concise view around particular subject issues by excluding
data that are not useful in the decision support process.

04/14/2025 Data Preprocessing (Basic Concepts) 4


Data Warehouse—Integrated

• Constructed by integrating multiple, heterogeneous data sources


• relational databases, flat files, on-line transaction records
• Data cleaning and data integration techniques are applied.
• Ensure consistency in naming conventions, encoding structures, attribute
measures, etc. among different data sources
• E.g., Hotel price: currency, tax, breakfast covered, etc.
• When data is moved to the warehouse, it is converted.

04/14/2025 Data Preprocessing (Basic Concepts) 5


Data Warehouse—Time Variant

• The time horizon for the data warehouse is significantly longer than that of
operational systems.
• Operational database: current value data.
• Data warehouse data: provide information from a historical perspective (e.g.,
past 5-10 years)
• Every key structure in the data warehouse
• Contains an element of time, explicitly or implicitly
• But the key of operational data may or may not contain “time element”.

04/14/2025 Data Preprocessing (Basic Concepts) 6


Data Warehouse—Non-Volatile

• A physically separate store of data transformed from the operational environment.


• Operational update of data does not occur in the data warehouse environment.
• Does not require transaction processing, recovery, and concurrency control
mechanisms
• Requires only two operations in data accessing:
• initial loading of data and access of data.

04/14/2025 Data Preprocessing (Basic Concepts) 7


Data Warehouse vs. Heterogeneous DBMS

• Traditional heterogeneous DB integration:


• Query driven approach
• When a query is posed to a client site, a meta-dictionary is used to translate the
query into queries appropriate for individual heterogeneous sites involved, and
the results are integrated into a global answer set
• Complex information filtering, compete for resources
• Data warehouse: update-driven, high performance
• Information from heterogeneous sources is integrated in advance and stored in
warehouses for direct query and analysis

04/14/2025 Data Preprocessing (Basic Concepts) 8


Data Warehouse vs. Operational DBMS

 OLTP (on-line transaction processing)


◦ Major task of traditional relational DBMS
◦ Day-to-day operations: purchasing, inventory, banking, manufacturing, payroll,
registration, accounting, etc.
 OLAP (on-line analytical processing)
◦ Major task of data warehouse system
◦ Data analysis and decision making

04/14/2025 Data Preprocessing (Basic Concepts) 9


Data Warehouse vs. Operational DBMS

 Distinct features (OLTP vs. OLAP):


◦ User and system orientation: customer vs. market
◦ Data contents: current, detailed vs. historical, consolidated
◦ Database design: ER + application vs. star + subject
◦ View: current, local vs. evolutionary, integrated
◦ Access patterns: update vs. read-only but complex queries

04/14/2025 Data Preprocessing (Basic Concepts) 10


OLTP vs. OLAP

OLTP OLAP
users clerk, IT professional knowledge worker
function day to day operations decision support
DB design application-oriented subject-oriented
data current, up-to-date historical,
detailed, flat relational summarized, multidimensional
isolated integrated, consolidated
usage repetitive ad-hoc
access read/write lots of scans
index/hash on prim. key
unit of work short, simple transaction complex query
# records accessed tens millions
#users thousands hundreds
DB size 100MB-GB 100GB-TB
metric transaction throughput query throughput, response

04/14/2025 Data Preprocessing (Basic Concepts) 11


Why Separate Data Warehouse?

 High performance for both systems


◦ DBMS— tuned for OLTP: access methods, indexing, concurrency control, recovery
◦ Warehouse—tuned for OLAP: complex OLAP queries, multidimensional view,
consolidation.
 Different functions and different data:
◦ Missing data: Decision support requires historical data which operational DBs do not
typically maintain.
◦ Data consolidation: DS requires consolidation (aggregation, summarization) of data
from heterogeneous sources.
◦ Data quality: different sources typically use inconsistent data representations, codes
and formats which have to be reconciled.

04/14/2025 Data Preprocessing (Basic Concepts) 12


Summary

• A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile


collection of data organized in support of management decision making.

• Several factors distinguish data warehouses from operational databases.

• Because the two systems provide quite different functionalities and require
different kinds of data, it is necessary to maintain data warehouses separately
from operational databases.

04/14/2025 Data Preprocessing (Basic Concepts) 13


Thank You

04/14/2025 Data Preprocessing (Basic Concepts) 14

You might also like