Outline

n  Introduction & architectural issues

n  Data distribution
q  Distributed query processing
q Query Processing Methodology
q Localization
q Distributed query optimization
q Distributed transactions & concurrency control
q Distributed reliability
q Data replication
q Parallel database systems
q Database integration & querying
q Advanced topics

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 1

Query Processing

high level user query

query
processor

low level data manipulation

commands

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 2

Page 1
 Query Processing Components
n  Query language that is used

l  SQL: “intergalactic dataspeak”

n  Query execution methodology

l  The steps that one goes through in executing high-level

(declarative) user queries.

n  Query optimization

l  How do we determine the “best” execution plan?

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 3

Selecting Alternatives
SELECT ENAME
FROM EMP,ASG
WHERE EMP.ENO = ASG.ENO
AND RESP = “Manager”

Strategy 1
ΠENAME(σRESP=“Manager” ∧ EMP.ENO=ASG.ENO (EMP ⋅ ASG))
Strategy 2
ΠENAME(EMP ⋈ENO (σRESP=“Manager” (ASG)))
Strategy 2 avoids Cartesian product, so is “better”

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 4

Page 2
 What is the Problem?
Site 1 Site 2 Site 3 Site 4 Site 5
ASG1=σENO≤“E3”(ASG) ASG2=σENO>“E3”(ASG) EMP1=σENO≤“E3”(EMP) EMP2=σENO>“E3”(EMP) Result

Site 5 Site 5
' '
result = EMP ∪ EMP
1 2 result= (EMP1 ∪ EMP2) ENOσRESP=“Manager”(ASG1 ∪ ASG2)
' '
EMP 1 EMP 2
Site 3 Site 4 ASG1 ASG2 EMP1 EMP2
' ' ' '
EMP = EMP1
1 ENO ASG 1 EMP = EMP2
2 ENO ASG 2 Site 1 Site 2 Site 3 Site 4

ASG1' ASG'2
Site 1 Site 2
ASG1' = σRESP="Manager" ASG1 ASG'2 = σRESP="Manager" ASG2

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 5

Cost of Alternatives
n  Assume:
l  size(EMP) = 400, size(ASG) = 1000
l  tuple access cost = 1 unit; tuple transfer cost = 10 units
n  Strategy 1
  produce ASG': (10+10)∗tuple access cost 20
  transfer ASG' to the sites of EMP: (10+10)∗tuple transfer cost 200
  produce EMP': (10+10) ∗tuple access cost∗2 40
  transfer EMP' to result site: (10+10) ∗tuple transfer cost 200
Total cost 460
n  Strategy 2
  transfer EMP to site 5: 400∗tuple transfer cost 4,000
  transfer ASG to site 5 :1000∗tuple transfer cost 10,000
  produce ASG':1000∗tuple access cost 1,000
  join EMP and ASG':400∗20∗tuple access cost 8,000
Total cost 23,000
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 6

Page 3
Query Optimization Objectives
Minimize a cost function
I/O cost + CPU cost + communication cost
These might have different weights in different
distributed environments
Wide area networks
l  communication cost will dominate
u  low bandwidth
u  low speed
u  high protocol overhead
l  most algorithms ignore all other cost components

Local area networks

l  communication cost not that dominant
l  total cost function should be considered

Can also maximize throughput

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 7

Query Optimization Issues –

Types Of Optimizers
n  Exhaustive search
l  Cost-based
l  Optimal
l  Combinatorial complexity in the number of relations
n  Heuristics

l  Not optimal

l  Regroup common sub-expressions
l  Perform selection, projection first
l  Replace a join by a series of semijoins
l  Reorder operations to reduce intermediate relation size
l  Optimize individual operations
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 8

Page 4
 Query Optimization Issues –
Optimization Granularity

n  Single query at a time

l  Cannot use common intermediate results

n  Multiple queries at a time

l  Efficient if many similar queries

l  Decision space is much larger

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 9

Query Optimization Issues –

Optimization Timing
n  Static
l  Compilation ⇒ optimize prior to the execution
l  Difficult to estimate the size of the intermediate
results ⇒ error propagation
l  Can amortize over many executions
l  R*
n  Dynamic
l  Run time optimization
l  Exact information on the intermediate relation sizes
l  Have to reoptimize for multiple executions
l  Distributed INGRES
n  Hybrid
l  Compile using a static algorithm
l  If the error in estimate sizes > threshold, reoptimize
at run time
l  Mermaid
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 10

Page 5
 Query Optimization Issues –
Statistics

n  Relation
l  Cardinality
l  Size of a tuple
l  Fraction of tuples participating in a join with another
relation
n  Attribute
l  Cardinality of domain
l  Actual number of distinct values
n  Common assumptions
l  Independence between different attribute values
l  Uniform distribution of attribute values within their
domain

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 11

Query Optimization Issues –

Decision Sites

n  Centralized
l  Single site determines the “best” schedule
l  Simple
l  Need knowledge about the entire distributed database
n  Distributed
l  Cooperation among sites to determine the schedule
l  Need only local information
l  Cost of cooperation
n  Hybrid
l  One site determines the global schedule
l  Each site optimizes the local subqueries

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 12

Page 6
 Query Optimization Issues –
Network Topology
n  Wide area networks (WAN) – point-to-point
l  Characteristics
u Low bandwidth
u Low speed
u  High protocol overhead
l  Communication cost will dominate; ignore all other cost
factors
l  Global schedule to minimize communication cost
l  Local schedules according to centralized query
optimization
n  Local area networks (LAN)
l  Communication cost not that dominant
l  Total cost function should be considered
l  Broadcasting can be exploited (joins)
l  Special algorithms exist for star networks

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 13

Distributed Query
Processing Methodology
Calculus Query on Distributed
Relations

Query GLOBAL
Decomposition SCHEMA

Algebraic Query on Distributed

Relations
CONTROL FRAGMENT
SITE Data
Localization SCHEMA

Fragment Query

Global STATS ON
Optimization FRAGMENTS

Optimized Fragment Query

with Communication Operations

Local LOCAL
LOCAL
Optimization SCHEMAS
SITES

Optimized Local
Queries

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 14

Page 7
 Step 1 – Query Decomposition
n  Input : calculus query on global relations
n  Normalization
l  Manipulate query quantifiers and qualification
n  Analysis
l  Detect and reject “incorrect” queries
l  Possible for only a subset of relational calculus
n  Simplification
l  Eliminate redundant predicates
n  Restructuring
l  Calculus query ⇒ algebraic query
l  More than one translation is possible
l  Use transformation rules

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 15

Restructuring
n  Convert relational calculus to
relational algebra ΠENAME Project
n  Make use of query trees
n  Example σDUR=12 ∧ DUR=24
Find the names of employees other than
J. Doe who worked on the CAD/CAM
project for either 1 or 2 years.
σPNAME=“CAD/CAM” Select
SELECT ENAME
FROM EMP, ASG, PROJ
WHERE EMP.ENO = ASG.ENO σENAME≠“J. DOE”
AND ASG.PNO = PROJ.PNO
AND ENAME ≠ “J. Doe” ⋈PNO
AND PNAME = “CAD/CAM”
AND DUR = 12 or DUR = 24)
⋈ENO Join

PROJ ASG EMP

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 16

Page 8
 Restructuring –
Transformation Rules
n  Commutativity of binary operations
l  R × S ⇔ S × R
l  R ⋈S ⇔ S ⋈R
l  R ∪ S ⇔ S ∪ R
n  Associativity of binary operations
l  ( R × S) × T ⇔ R × (S × T)
l  (R ⋈S) ⋈T ⇔ R ⋈ (S ⋈T)
n  Idempotence of unary operations
l  ΠA’(Π A’(R)) ⇔ Π A’(R)
l  σp (σp2(A2)(R)) ⇔ σp1(A1)∧p2(A2)(R)
1(A1)
where R[A] and A' ⊆ A, A" ⊆ A and A' ⊆ A"
n  Commuting selection with projection

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 17

Previous Example – Equivalent

Query
ΠENAME ΠENAME

σDUR=12 ∧ DUR=24 σPNAME=“CAD/CAM” ∧

(DUR=12 ∧ DUR=24) ∧

σPNAME=“CAD/CAM” ENAME≠“J. Doe”

σENAME≠“J. DOE” ⋈PNO,ENO

⋈PNO
×

⋈ENO
PROJ ASG EMP EMP PROJ ASG

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 18

Page 9
 Restructuring
ΠENAME

⋈PNO
ΠPNO,ENAME

⋈ENO

ΠPNO ΠPNO,ENO ΠPNO,ENAME

σPNAME = "CAD/CAM" σDUR=24∧DUR=24 σENAME ≠ "J. Doe"

PROJ ASG EMP

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 19

Step 2 – Data Localization

Input: Algebraic query on distributed relations
n  Determine which fragments are involved
n  Localization program
l  substitute for each global query its materialization
program
l  optimize

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 20

Page 10
 Example
Assume ΠENAME
l  EMP is fragmented into EMP1, EMP2,
EMP3 as follows: σDUR=12 ∧DUR=24
u  EMP1= σENO≤“E3”(EMP)
σPNAME=“CAD/CAM”
u  EMP2= σ“E3”<ENO≤“E6”(EMP)
u  EMP3= σENO≥“E6”(EMP)
σENAME≠“J. DOE”
l  ASG fragmented into ASG1 and ASG2 as
follows:
⋈PNO
u  ASG1= σENO≤“E3”(ASG)
u  ASG2= σENO>“E3”(ASG) ⋈ENO
Replace EMP by (EMP1 ∪ EMP2 ∪
EMP3) PROJ ∪
∪

and ASG by (ASG1 ∪ ASG2) in any
query EMP1EMP2 EMP3 ASG1 ASG2
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 21

Provides Parallellism

⋈ENO ⋈ENO ⋈ENO ⋈ENO

EMP1 ASG1 EMP2 ASG2 EMP3 ASG1 EMP3 ASG2

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 22

Page 11
 Eliminates Unnecessary
Work

⋈ENO ⋈ENO ⋈ENO

EMP1 ASG1 EMP2 ASG2 EMP3 ASG2

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 23

Reduction for PHF

n  Reduction with selection
l  Relation R and FR={R1, R2, …, Rw} where Rj=σp (R)
j

σpi(Rj)=∅ if ∀x in R: ¬(pi(x)∧ pj(x))

l  Example
SELECT *
FROM EMP
WHERE ENO="E5"
σENO=“E5” σENO=“E5”

EMP1 EMP2 EMP3 EMP2

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 24

Page 12
 Reduction for PHF
n  Reduction with join
l  Possible if fragmentation is done on join attribute

l  Distribute join over union

(R1∪ R2)⋈S ⇔ (R1⋈S) ∪ (R2⋈S)

l  Given Ri =σp (R) and Rj = σp (R)

i j

Ri ⋈Rj =∅ if ∀x in Ri, ∀y in Rj: ¬(pi(x)∧ pj(y))

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 25

Reduction for PHF

n  Assume EMP is fragmented as ⋈ENO

before and
l  ASG1: σENO ≤ "E3"(ASG) ∪
∪

l  ASG2: σENO > "E3"(ASG)
n  Consider the query
SELECT * EMP1 EMP2 EMP3 ASG1 ASG2
FROM EMP,ASG
WHERE EMP.ENO=ASG.ENO
n  Distribute join over unions
∪
n  Apply the reduction rule
⋈ENO ⋈ENO ⋈ENO

EMP1 ASG1 EMP2 ASG2 EMP3 ASG2

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 26

Page 13
 Reduction for VF
n  Find useless (not empty) intermediate relations
Relation R defined over attributes A = {A1, ..., An} vertically
fragmented as Ri =ΠA'(R) where A'⊆ A:
ΠD,K(Ri) is useless if the set of projection attributes D is not in A'
Example: EMP1=ΠENO,ENAME (EMP); EMP2=ΠENO,TITLE (EMP)
SELECT ENAME
FROM EMP

ΠENAME ΠENAME

⋈ENO

EMP1 EMP2 EMP1

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 27

Reduction for DHF

n  Rule :
l  Distribute joins over unions
l  Apply the join reduction for horizontal fragmentation
n  Example
ASG1: ASG ⋉ENO EMP1
ASG2: ASG ⋉ENO EMP2
EMP1: σTITLE=“Programmer” (EMP)
EMP2: σTITLE=“Programmer” (EMP)
n  Query
SELECT *
FROM EMP, ASG
WHERE ASG.ENO = EMP.ENO
AND EMP.TITLE = "Mech. Eng."

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 28

Page 14
 Reduction for DHF
Generic query ⋈ENO
σTITLE=“Mech. Eng.”

∪
∪

ASG1 ASG2 EMP1 EMP2

Selections first ⋈ENO

∪
σTITLE=“Mech. Eng.”

ASG1 ASG2 EMP2

CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 29

Reduction for DHF

∪

Joins over unions

⋈ENO ⋈ENO

σTITLE=“Mech. Eng.” σTITLE=“Mech. Eng.”

ASG1 EMP2 ASG2 EMP2

Elimination of the empty intermediate relations
(left sub-tree) ⋈ENO

σTITLE=“Mech. Eng.”

ASG2 EMP2
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 3. 30

Page 15