Data Warehousing & DATA

MINING (SE-409)
Lecture-4
Dimensional Modeling

Dr. Huma
Software Engineering department

University of Engineering and Technology, Taxila

1
Dimensional Modeling (DM)

2
 The need for ER modeling?
• Problems with early COBOLian data processing
systems.
• Collection of data

• Data redundancies

• From flat file to Table, each entity ultimately

becomes a Table in the physical schema.

• Simple O(n2) Join to work with Tables

3
Why ER Modeling has been so successful?
– Coupled with normalization drives out all the
redundancy out of the database.

– Change (or add or delete) the data at just one

point.

– Can be used with indexing for very fast access.

– Resulted in success of OLTP systems.

4
 ER Modeling
CITY DISTRICT M DIVISION PROVINCE
1 district BACK
1 1
zone M division
M DISTRICT DIVISION
ZONE CITY
1
store M week
1
STORE # STREET ZONE ... DATE WEEK
1 M
sale_header quarter
M M
RECEIPT # STORE # DATE ... MONTH QTR
1 1
M M
1
WEEK MONTH
M sale_detail month 1
RECEIPT # ITEM # ... $
YEAR QTR
1 M M
1 year
ITEM # CATEGORY
ITEM # SUPPLIER
item_x_cat M
1 item_x_splir
CATEGORY DEPT
cat_x_dept 5
 Need for DM: Un-answered Qs
• Lets have a look at a typical ER data model first.

• Some Observations:
– All tables look-alike, as a consequence it is difficult to identify:

• Which table is more important ?

• Which is the largest?

• Which tables contain numerical measurements of the

business?

• Which table contain nearly static descriptive attributes?

[dimension info]
6
Need for DM: Complexity of Representation
– Many topologies for the same ER diagram, all
appearing different.
• Very hard to visualize and remember.
12
7 6
3 12 7
11 4 8
8
9
1 10
10 9 11
6 1

3 2 5
2 5 4

• A large number of possible connections to any

two (or more) tables
7
 Need for DM: The Paradox
• The Paradox: Trying to make information accessible using tables
resulted in an inability to query them!

• ER and Normalization result in large number of tables which are:

– Hard to understand by the users (DB programmers): EPR
system span on multiple tables

– Hard to navigate optimally by DBMS software

• Real value of ER is in using tables individually or in pairs[ good

performance in one or less table in join operation ]

• Too complex for queries that span multiple tables with a large
number of records
8
 ER vs. DM
ER DM
Constituted to optimize OLTP Constituted to optimize DSS
performance. query performance.

Models the
Models the micro/detail macro[aggregate]
relationships among data relationships among data
elements. elements with an overall
deterministic strategy.
All dimensions serve as
A wild variability of the
equal entry points to the
structure of ER models.
fact table.
Very vulnerable to changes in Changes in users' querying
the user's querying habits, habits can be
because such schemas are accommodated by
asymmetrical. automatic SQL generators.
9
 How to simplify a ER data model?

• Bring it to DSS
• Two general methods:

– De-Normalization

– Dimensional Modeling (DM)

10
 What is DM?…
• A simpler logical model optimized for decision support.
• Inherently dimensional in nature[fact + dimension] , with a
single central fact table and a set of smaller dimensional
tables.
• Multi-part key for the fact table (long in terms of data, contain
numerical data, how many item sale, what revenue we get
from sale+ how much sale we need + single column primary
key).

• Dimensional tables with a single-part PK.(one and more but

small + single column key+ info regarding time, geography,
product dimension).

11
 What is DM?...

• Results in a star like structure, called star

schema or star join. Fact in center and
dimension around it.

– All relationships mandatory M-1.

– Single path between any two levels.[fact vs

dimensional table]

12
 Dimensions have Hierarchies

Items

Books Cloths

Fiction Text Men Women

Engg Medical

Analysts tend to look at the data through dimension at a

particular “level” in the hierarchy

13
 The two Schemas

Star
Snow-flake

14
 “Simplified” 3NF (Retail)
CITY DISTRICT M DIVISION PROVINCE
1 district BACK
1 1
zone M division
M DISTRICT DIVISION
ZONE CITY
1
store M week
1
STORE # STREET ZONE ... DATE WEEK
1 M
sale_header quarter
M M
RECEIPT # STORE # DATE ... MONTH QTR
1 1
M M
1
WEEK MONTH
M sale_detail month 1
RECEIPT # ITEM # ... $
YEAR QTR
1 M M
1 year
ITEM # CATEGORY
ITEM # SUPPLIER
item_x_cat M
1 item_x_splir
CATEGORY DEPT
cat_x_dept 15
 Vastly Simplified Star Schema
Product Dim
Geography Dim
1 ITEM#
STORE# 1
Fact Table CATEGORY
ZONE
RECEIPT#
DEPT
CITY
STORE#
M SUPPLIER
DISTRICT
ITEM# M
DIVISION
DATE Time Dim
M
PROVINCE . DATE
. 1
facts . WEEK

Sale Rs. MONTH

QUARTER

YEAR

16
 The Benefit of Simplicity

Beauty lies in close correspondence

with the business, obvious even to
business users.[means simplicity]

17
 Features of Star Schema
Dimensional hierarchies are collapsed into a single table for
each dimension. Loss of Information? Relationship lost

A single fact table created with a single header from the

detail records, resulting in:

– A vastly simplified physical data model!

– Fewer tables (thousands of tables in some ERP systems).

– Fewer joins resulting in high performance.

–
18
Process of Dimensional Modeling
 The Process of Dimensional Modeling
Four Step Method from ER to DM: ER covers all business.[ Visualization high,
complexity is high, whether requirement or not].

1. Choose the Business Process like [accounting, inventory, invoices

and even processes not running simultaneously and focus on it].
2. Choose the Grain [high grain high detail, each row showing grain
level]
3. Choose the Facts
4. Choose the Dimensions [small in size more in numbers]
 Step-1: Choose the Business Process
• A business process is a major operational
process in an organization.
• Typically supported by a legacy system
(database) or an OLTP.
– Examples: Orders, Invoices, Inventory etc.

• Business Processes are often termed as Data

Marts and that is why many people criticize
DM as being data mart oriented. [Two school
of thoughts]
 Step-1: Separating the Process

Star-1

Snow-flake
 Step-2: Choosing the Grain
• Grain is the fundamental, atomic [not further break down] level
of data to be represented.

• Grain is also termed as the unit of analyses.

• Example grain statements

• Typical grains
– Individual Transactions [single + multiple
– Daily aggregates (snapshots)
– Monthly aggregates

• Relationship between grain and expressiveness.[ more detail

more know about business]

• Grain vs. hardware trade-off.[ keep focus on grian not on

hardware]
 Step-2: Relationship b/w Grain
LOW Granularity HIGH Granularity

Four aggregates per week

4 x 4 = 16 values

Two aggregates per week Daily aggregates

2 x 4 = 8 values 6 x 4 = 24 values
 The case FOR data aggregation
• Works well for repetitive queries.

• Justifiable if used for max number of queries.

• Provides a “big picture” or macroscopic view.

• Application dependent, usually fixed to business

changes. [ construct according to bussiness]
 The case AGAINST data aggregation
• Aggregation is irreversible.
– Can create monthly sales data from weekly sales data,
but the reverse is not possible.
– [do not through detail data]

• Aggregation limits the questions that can be

answered.
– What, when, why, where, what-else, what-next
– When [time]
– Where[zone 1 or zone 2]
 The case AGAINST data aggregation
• Aggregation can hide crucial facts.
–The average of 100 & 100 is same as 150 & 50
Aggregation hides crucial facts Example

Week-1 Week-2 Week-3 Week-4 Average

Zone-1 100 100 100 100 100
Zone-2 50 100 150 100 100
Just looking at the averages i.e. aggregate
Zone-3 50 100 100 150 100
Zone-4 200 100 50 50 100
Average 100 100 100 100

SALE DATA: SAME IN EVERY ZONE AND EVERY WEEK

SAY SOME ONE RUN SOME PROMOTION SCHEME AND SEEING HOW
PEOPLE RESPOND ON IT
 Aggregation hides crucial facts chart
250
Z1 Z2 Z3 Z4
200
Sale
wise 150
Wrong
grain
100
setting

50

0
Week-1 Week-2 Week-3 Week-4
Week wise
Z1: Sale is constant (need to work on it)
Z2: Sale went up, then fell (need of concern)
Z3: Sale is on the rise, why?
Z4: Sale dropped sharply, need to look deeply.
W2: Static sale
Step 3: Choose Facts statement

Facts
“We need monthly sales: data
volume and Rs. by
week, product and Zone : reference”

Dimensions

Decision maker ask and you being

WH architect
 Step 3: Choose Facts
• Choose the facts that will populate each
fact table record.

– Remember that best Facts are Numeric,

Continuously Valued and Additive.

– Example: Quantity Sold, Amount etc.

 Step 4: Choose Dimensions
• Choose the dimensions that apply to each
fact in the fact table.

– Typical dimensions: time, product, geography

etc. WHERE CLAUSE

– Identify the descriptive attributes that explain

each dimension.

– Determine hierarchies within each dimension.

 Step-4: How to Identify a Dimension?

• The single valued attributes during recording of a transaction

are dimensions. : MEANS VALUES DOES NOT CHANGE
Fact Table
Calendar_Date
Time_of_Day
Dim Account _No
ATM_Location
Transaction_Type
Transaction_Rs

Time_of_day: Morning, Mid Morning, Lunch Break etc.

Transaction_Type: Withdrawal, Deposit, Check balance etc.