HISTORY MANAGEMENT OF DATA – SLOWLY CHANGING

DIMENSIONS

Marek Wancerz, Paweł Wancerz

Politechnika Lubelska, Wydział Elektrotechniki i Informatyki, Politechnika Lubelska, Wydział Elektrotechniki i Informatyki

Abstract: The article describes few methods of managing data history in databases and data marts. There are many types of dealing with the
history of the data. This article will show us some examples, point advantages and disadvantages of each of the method and show us possible
scenarios of use.

Keywords: managing data history, databases, data marts

ZARZĄDZANIE HISTORIĄ DANYCH – SLOWLY CHANGING DIMENSIONS

Streszczenie: Artykuł opisuje sposób zarządzania historią tabel wymiarowych w bazach danych i hurtowniach danych. Istnieje kilka spo-
sobów na archiwizowanie historii. Artykuł ma na celu przybliżenie ich funkcjonalności popartej przykładami, wskazanie zalet i wad oraz
możliwych scenariuszy użycia.

Słowa kluczowe: zarządzanie historią, bazy danych, hurtownie danych

Introduction
Nowadays, almost everyone use data in they lives. But how to To understand the way of data history management we should take
understand word „data”? In IT, we can name it as a set of values a look on the main methods.
or variables belonging to a set of items. It is very often represented
in a tabular form (columns + rows), data tree (parent-child rela-
tionship) or in a graphical structure (tables models with graphical
2. Slowly Changing Dimensions – main types
representation). The data we keep doesn't have to be in a text
The easiest way to discuss about the Slowly Changing Dimensions
form. We can keep it as a number or even an image.
types is to go through all of them with some examples, pointing
The data kept in our databases (or data marts) and its quality gives
advantages and disadvantages and possible usage scenarios. We
us a huge advantage for the data visualization and management.
have 3 basic types of Slowly Changing Dimensions.
But we have to remember that the dimensional data is not a stable
entity and it might change over time. To manage the data history I) SCD Type 1 – overwriting the old values
Slowly Changing Dimensions was invented.
Figure (Fig. 2) shows the change of data for an item. The Cat-
egory for an Item RoboticBook was Science Fiction in 2012. In
1. Slowly Changing Dimensions overview 2013 the Item changed its Category to Reality.

Slowly Changing Dimensions was invented by Ralph Kimball,

who is regarded as one of the original architects of data ware-
housing. His methodology became a standard. Slowly Changing
Dimensions is a set of methods to manage the data history in the
Dimension tables.
Data might change over time and we should take it into account
Fig. 2 SCD Type 1 overview.
while developing our system (Fig. 1):
 Source data–source data should be delivered in a unified form,
The change of Category is a result of an error or a change of
 ETL (Extract Transform Load) – the ETL system should have
structure. Nevertheless it sometimes doesn't meet business re-
the mechanism to operate with data incoming to the target system.
quirements. The big advantage for such a method of development
All bigger ETL tools have the option SCD already implemented
is the simplicity of database structure and ETL system. But main
(as an option to use) – Informatica PowerCenter Tool, SSIS,
disadvantage of this kind of method is the fact that we lose the
Oracle Warehous Builder,
data history! We can only see descriptive attributes as they exist
 Database structure - based on the method of history today. To give a better understanding of business requirements
management, the database structure has to be adapted. inaccuracy after such a change, please have a look the picture
below (Fig. 3).

Fig. 1 Data mart load process

Fig. 3 SCD Type 1 example.
There are 2 People in dimension Person with their payments in a 3) Conclusion
separate fact table – Payments. Then on 30/03/2013 there is a new
payment from the same person – Smith but under a new address (it We have analyzed 3 types of managing historical data in
was changed in the Dimensional table). When we analyze the Dimensional tables. Each of them has advantages and disad-
Payments by City, we will see that Texas has 500 but in fact it has vantages and can be used in totally different business needs. It is
only 200 (300 was NewYork but it is no longer available!). up the data modeler to set up such an environment to make it easy
to implement, maintain and develop. The most popular method
II) SCD Type 2– new record in the dimension (from those 3) is definitely the SCD Type 2 which gives us a full
history of a Dimension value and helps us to build reports not only
Let’s take the same example as in Type 1. The same sets of values
on current but also on the historical data. The types we described
were assigned to the Item, but with additional fields – Effective
together with the whole concept were invented in 90'. But after
Date, Expiration Date, FlagCurrent (Fig. 4).
publish of SCD's 1,2,3 Kimball Group started working on modifi-
cations of methods, their fusions and new ones. The result is a new
book The Data Warehouse Toolkit (Wiley, Jun/Jul 2013) where
we can find 7 Types of SCD's! You can check the overview here
on the Figure (Fig. 7). I will describe them in the next publication.

Fig. 4 SCD Type 2 overview.

As we can see, for Type 2 Dimension, we have 3 additional indi-

cators which help us control the data. The dates show us the pe-
riod of time when the Item is valid. The Current flag is giving us a
fast information if the row is Valid (Y) or not (N). The huge ad-
vantage for this approach is that we keep all the history rows in
the dimension and we track all the historical entries. On the other
hand, this approach is more complicated for the end user (report
developer). The dimension table growth has also be taken into
account while development of the project schema.
Let’s have a look again at the example from SCD Type 1. But here
we will use SCD Type 2 for history data management. (Fig. 5)

Fig. 5 SCD Type 2 example. Fig. 7 New types of SCD [5].

This example shows us correct values grouped by Cities. This is

because we created a new row for the changed Smith person with Bibliography:
updated City. [1] DataModelling: http://www.learndatamodeling.com.
[2] Dimensional Modeling in Depth (Kimball, Ross) – coursebook.
III) SCD Type 3– new dimension column [3] The Data Warehouse Toolkit (Wiley, 2013).
[4] Informatica PowerCenter official: http://www.informatica.com.
Let’s have a look at the last primary SCD – Type 3. The same [5] Kimball Group webpage: http://www.kimballgroup.com.
example will be taken into account while trying to visualize the
method. (Fig. 6)
dr inż. Marek Wancerz
E-mail: [email protected]

Marek Wancerz, ukończył Wydział Elektryczny Politechniki Lubelskiej. Obecnie

pracuje w Katedrze Sieci i Zabezpieczeń swojej macierzystej uczelni. Jego
zainteresowania naukowe koncentrują się wokół zagadnień automatyki
zabezpieczeniowej, bezpieczeństwa pracy systemu elektroenergetycznego oraz
Fig. 6 SCD Type 2 overview. wykorzystania informatyki i baz danych w energetyce. Współautor wielu krajowych i
zagranicznych publikacji o tej tematyce, a także wielu prac naukowo-badawczych dla

In this method, a new dimension column is created to keep the

historical value of the item. This kind of method is used relatively
infrequently. Type 3 SCD is good for tracking soft changes, like
item or business reorganization. It gives a good view of the situa-
tion today and prior the change. But in case of more frequent and
important changes this method will lose the historical data, as only
current and original values are retained. The history of changes instytucji związanych z energetyką i informatyką.
can't be reproduced as it is done in SCD Type 2.
mgr inż. Paweł Wancerz
E-mail: [email protected]

Paweł Wancerz is an employee of Atos IT Solutions & Services Company located in

Wroclaw. He is currently attending PhD Studies at Politechnika Lubelska. His major
interests in IT are Business Intelligence, Data Warehousing and ETL. He participated
in professional training which enabled him to acquire in-depth knowledge in this field.