UNIT- I

INTRODUCTION AND DATA

WAREHOUSING

Contents

Introduction to Data Warehouse

Multidimensional Data Model
Data Warehouse Architecture
Implementation
Further Development
Data Warehousing to Data Mining

Introduction to Data Warehouse

Data warehousing provides architectures and tools for business executives to

systematically organize, understand, and use their data to make strategic decisions.
Data warehouse systems are valuable tools in todays competitive, fast-evolving
world.
In the last several years, many firms have spent millions of dollars in building
enterprise-wide data warehouses.
Many people feel that with competition mounting in every industry, data
warehousing is the latest must-have marketing weapona way to retain customers
by learning more about their needs.
Lets take a closer look at each of these key features.

Cont..

Subject-oriented: A data warehouse is organized around major subjects, such as

customer, supplier, product, and sales.
Rather than concentrating on the day-to-day operations and transaction processing
of an organization, a data warehouse focuses on the modeling and analysis of data
for decision makers.
Hence, data warehouses typically provide a simple and concise view around
particular subject issues by excluding data that are not useful in the decision
support process.
Integrated: A data warehouse is usually constructed by integrating multiple
heterogeneous sources, such as relational databases, flat files, and on-line
transaction records.
Data cleaning and data integration techniques are applied to ensure consistency in
naming conventions, encoding structures, attribute measures, and so on.

Cont..

Time-variant: Data are stored to provide information from a historical perspective

(e.g., the past 510 years). Every key structure in the data warehouse contains,
either implicitly or explicitly, an element of time.

Nonvolatile: A data warehouse is always a physically separate store of data

transformed from the application data found in the operational environment. Due to
this separation, a data warehouse does not require transaction processing, recovery,
and concurrency control mechanisms. It usually requires only two operations in
data accessing: initial loading of data and access of data.

A data warehouse is also often viewed as an architecture, constructed by integrating

data from multiple heterogeneous sources to support structured and/or ad hoc
queries, analytical reporting, and decision making.

Multidimensional Data Model

Data warehouses and OLAP tools are based on a multidimensional data model.
This model views data in the form of a data cube
What is a data cube? A data cube allows data to be modeled and viewed in
multiple dimensions. It is defined by dimensions and facts.
In general terms, dimensions are the perspectives or entities with respect to which
an organization wants to keep records. For example, All Electronics may create a
sales data warehouse in order to keep records of the stores sales with respect to the
dimensions time, item, branch, and location. These dimensions allow the store to
keep track of things like monthly sales of items and the branches and locations at
which the items were sold.
Each dimension may have a table associated with it, called a dimension table,
which further describes the dimension. For example, a dimension table for item
may contain the attributes item name, brand, and type. Dimension tables can be
specified by users or experts, or automatically generated and adjusted based on data
distributions.

Cont..

Cont..

A multidimensional data model is typically organized around a central theme, like

sales, for instance. This theme is represented by a fact table. Facts are numerical
measures.
Think of them as the quantities by which we want to analyze relationships between
dimensions. Examples of facts for a sales data warehouse include dollars sold
(sales amount in dollars), units sold (number of units sold), and amount budgeted.
The fact table contains the names of the facts, or measures, as well as keys to each
of the related dimension tables.
To gain a better understanding of data cubes and the multidimensional data model,
lets start by looking at a simple 2-D data cube that is, in fact, a table or spreadsheet
for sales data from All Electronics.

Data Warehouse Architecture

Steps for the Design and Construction of Data Warehouses

This subsection presents a business analysis framework for data warehouse design.
The basic steps involved in the design process are also described.
The Design of a Data Warehouse: A Business Analysis Framework
What can business analysts gain from having a data warehouse?
First, having a data warehouse may provide a competitive advantage by presenting
relevant information from which to measure performance and make critical
adjustments in order to help win over competitors.
Second, a data warehouse can enhance business productivity because it is able to
quickly and efficiently gather information that accurately describes the organization.
Third, a data warehouse facilitates customer relationship management because it
provides a consistent view of customers and items across all lines of business, all
departments, and all markets.
Finally, a data warehouse may bring about cost reduction by tracking trends,
patterns, and exceptions over long periods in a consistent and reliable manner.

Implementation
Four different views regarding the design of a data warehouse must be considered: the
top-down view, the data source view, the data warehouse view, and the business query
view.
The top-down view allows the selection of the relevant information necessary for the
data warehouse. This information matches the current and future business needs.
The data source view exposes the information being captured, stored, and managed by
operational systems. This information may be documented at various levels of detail and
accuracy, from individual data source tables to integrated data source tables. Data
sources are often modeled by traditional data modeling techniques, such as the entityrelationship model or CASE (computer-aided software engineering) tools.
The data warehouse view includes fact tables and dimension tables. It represents the
information that is stored inside the data warehouse, including precalculated totals and
counts, as well as information regarding the source, date, and time of origin, added to
provide historical context.
Finally, the business query view is the perspective of data in the data warehouse from
the viewpoint of the end user.

A Three-Tier Data Warehouse

Architecture

Further Development

Further Development of Data Cube and OLAP Technology

It describes data mining by discovery-driven exploration of data cubes, where
anomalies in the data are automatically detected and marked for the user with
visual cues.
It describes multi feature cubes for complex data mining queries involving multiple
dependent aggregates at multiple granularity.
It presents methods for constrained gradient analysis in data cubes, which
identifies cube cells that have dramatic changes in value in comparison with their
siblings, ancestors, or descendants.

Discovery-Driven Exploration of Data

Cubes

Discovery-driven exploration is such a cube exploration approach. In discovery

driven exploration, precomputed measures indicating data exceptions are used to
guide the user in the data analysis process, at all levels of aggregation.

We hereafter refer to these measures as exception indicators. Intuitively, an

exception is a data cube cell value that is significantly different from the value
anticipated, based on a statistical model.

The model considers variations and patterns in the measure value across all of the
dimensions to which a cell belongs. For example, if the analysis of item-sales data
reveals an increase in sales in December in comparison to all other months, this
may seem like an exception in the time dimension.

Complex Aggregation at Multiple

Granularity:
Multifeature Cubes

Data cubes facilitate the answering of data mining queries as they allow the
computation of aggregate data at multiple levels of granularity.

Many complex data mining queries can be answered by Multifeature cubes without
any significant increase in computational cost, in comparison to cube computation
for simple queries with standard data cubes.

Constrained Gradient Analysis in Data

Cubes

Many data cube applications need to analyze the changes of complex measures in
multidimensional space.

For example, in real estate, we may want to ask what are the changes of the
average house price in the Vancouver area in the year 2004 compared against
2003, and the answer could be the average price for those sold to professionals in
the West End went down by 20%, while those sold to business people in Metro
town went up by 10%, etc.

Expressions such as professionals in the West End correspond to cuboids cells

and describe sectors of the business modeled by the data cube.

Data Warehousing to Data Mining

Cont..

Data mining is not confined to the analysis of data stored in data warehouses.
It may analyze data existing at more detailed granularities than the summarized data
provided in a data warehouse.
It may also analyze transactional, spatial, textual, and multimedia data that are difficult
to model with current multidimensional database technology.
In this context, data mining covers a broader spectrum than OLAP with respect to data
mining functionality and the complexity of the data handled.
Because data mining involves more automated and deeper analysis than OLAP, data
mining is expected to have broader applications.
Data mining can help business managers find and reach more suitable customers, as well
as gain critical business insights that may help drive market share and raise profits.
In addition, data mining can help managers understand customer group characteristics
and develop optimal pricing strategies accordingly, correct item bundling based not on
intuition but on actual item groups derived from customer purchase patterns, reduce
promotional spending, and at the same time increase the overall net effectiveness of
promotions