OLTP and OLAP

DATA B A S E DE S IG
N

Lis Sulmont
Curriculum
Manager
Our motivating
question:
How should we organize and manage data?
Schemas: How should my data be logically organized?

Normalization: Should my data have minimal dependency and redundancy?

Views: What joins will be done most often?

Access control: Should all users of the data have the same level of access

DBMS: How do I pick between all the SQL and noSQL options?

and more!

DATABASE DESIGN
Our motivating
question:
How should we organize and manage data?
Schemas: How should my data be logically organized?

Normalization: Should my data have minimal dependency and redundancy?

Views: What joins will be done most often?

Access control: Should all users of the data have the same level of access

DBMS: How do I pick between all the SQL and noSQL options?

and more!

It depends on the intended use of the data.

DATABASE DESIGN
Approaches to processing data
OLTP OLAP
Online Transaction Online Analytical
Processing Processing

DATABASE DESIGN
Some concrete examples
OLTP tasks OLAP tasks
Find the price of a book Calculate books with best proht
margin
Update latest customer
Find most loyal customers
transaction Keep track of
Decide employee of the
employee hours
month

DATABASE DESIGN
OLAP vs. OLTP
OLTP OLAP

Purpose support daily transactions report and analyze data

Design application-oriented subject-oriented
Data up-to-date, operational consolidated, historical
Size snapshot, gigabytes archive, terabytes
Queries simple transactions & frequent complex, aggregate queries & limited
updates updates
Users thousands hundreds

DATABASE DESIGN
Working together

DATABASE DESIGN
Takeaways
Step back and hgure out business
requirements

Difference between OLAP and

OLTP OLAP? OLTP? Or something

else?

DATABASE DESIGN
Let's practice!
DATA B A S E DE S IG
N
 Storing data
DATA B A S E DE S IG
N

Lis Sulmont
Curriculum
Manager
Structuring data
1. Structured data 2. Unstructured data

Follows a schema Schemaless

Dehned data types & Makes up most of data in the world

relationships
e.g., photos, chat logs, MP3
e.g., SQL, tables in a relational database

# Example of a JSON file

3. Semi-structured data "user": {
"profile_use_background_image": true,
Does not follow larger
"statuses_count": 31,
schema Self-describing "profile_background_color": "C0DEED",
"followers_count": 3066,
structure ...
e.g., NoSQL, XML, JSON
DATABASE DESIGN
 Structuring data

1https:/ commons.wikimedia.org/wiki/File:Flower_jtca001.jpg 2
https:/ commons.wikimedia.org/wiki/File:Languages_xml.png 3

DATABASE DESIGN
Storing data beyond traditional databases
Traditional databases
For storing real-time relational structured data ? OLTP

Data warehouses
For analyzing archived structured data ? OLAP

Data lakes
For storing data of all structures = hexibility and
scalability

For analyzing big data

DATABASE DESIGN
Data warehouses
Optimized for analytics - OLAP
Organized for reading & aggregating
data

Usually read-only

Contains data from multiple sources

Massively Parallel Processing (MPP)

Typically uses a denormalized schema

and dimensional modeling

Data marts

Subset of data

warehouses Dedicated to

a specihc topic DATABASE DESIGN

Data lakes
Store all types of data at a lower cost:
e.g., raw, operational databases, IoT device logs, real-time, relational and non-
relational

Retains all data and can take up petabytes

Schema-on-read as opposed to schema-on-

write

Need to catalog data otherwise becomes a

data swamp

Run big data analytics using services such as

Apache Spark and Hadoop
Useful for deep learning and data discovery
because activities require so much data

DATABASE DESIGN
ET
L

EL
T

DATABASE DESIGN
Let's practice!
DATA B A S E DE S IG
N
 Database design
DATA B A S E DE S IG
N

Lis Sulmont
Curriculum
Manager
What is database design?
Determines how data is logically stored
How is data going to be read and updated?

Uses database models: high-level specihcations for database

structure Most popular: relational model

Some other options: NoSQL models, object-oriented model, network

model Uses schemas: blueprint of the database

Dehnes tables, helds, relationships, indexes, and views

When inserting data in relational databases, schemas must be

respected

DATABASE DESIGN
 Data modeling
Process of creating a data model for the data to be stored

1. Conceptual data model: describes entities, relationships, and attributes

Tools: data structure diagrams, e.g., entity-relational diagrams and U ML

diagrams

2. Logical data model: dehnes tables, columns, relationships

Tools: database models and schemas, e.g., relational model and star schema

3. Physical data model: describes physical storage

Tools: partitions, CPUs, indexes, backup systems and tablespaces

1 https:/ en.wikipedia.org/wiki/Data_model

DATABASE DESIGN
Conceptual - ER diagram Logical - schema

Entities, relationships, and Fastest conversion: entities become the tables

attributes

DATABASE DESIGN
Other database design options

Determining tables

DATABASE DESIGN
Beyond the relational model
Dimensional modeling
Adaptation of the relational model for data warehouse design

Optimized for OLAP queries: aggregate data, not updating

(OLTP) Built using the star schema

Easy to interpret and extend schema

DATABASE DESIGN
Elements of dimensional modeling
Fact tables

Decided by business use-

case Holds records of a

metric Changes regularly

Connects to dimensions via

foreign keys
Organize by: Dimension tables
Holds descriptions of
What is being analyzed? attributes
How often do entities Does not change as often
change?

DATABASE DESIGN
Let's practice!
DATA B A S E DE S IG
N