Outline

• Introduction
• Background
• Distributed Database Design
• Fragmentation
• Data distribution
• Database Integration
• Semantic Data Control
• Distributed Query Processing
• Multidatabase Query Processing
• Distributed Transaction Management
• Data Replication
• Parallel Database Systems
• Distributed Object DBMS
• Peer-to-Peer Data Management
• Web Data Management
• Current Issues

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 1 of 65

9-Sep-19
Design Problem

• In the general setting :

Making decisions about the placement of data and programs
across the sites of a computer network as well as possibly
designing the network itself.

• In Distributed DBMS, the placement of applications

entails
• placement of the distributed DBMS software; and

• placement of the applications that run on the database

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 2 of 65

9-Sep-19
Dimensions of the Problem
Access pattern behavior
dynamic
static
partial
information

data
Level of knowledge
data +
complete
program
information

Level of sharing

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 3 of 65

9-Sep-19
Distribution Design

•Top-down
• mostly in designing systems from scratch

• mostly in homogeneous systems

•Bottom-up
• when the databases already exist at a number of
sites

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 4 of 65

9-Sep-19
Top-Down Design

Requirements
Analysis

Objectives
User Input
Conceptual View Integration View Design
Design

Access
GCS Information ES’s

Distribution
Design User Input

LCS’s

Physical
Design

LIS’s
Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 5 of 65
9-Sep-19
Distribution Design Issues

Why fragment at all?

How to fragment?

How much to fragment?

How to test correctness?

How to allocate?

Information requirements?

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 6 of 65

9-Sep-19
Fragmentation

• Can't we just distribute relations?

• What is a reasonable unit of distribution?
• relation
• views are subsets of relations locality
• extra communication
• fragments of relations (sub-relations)
• concurrent execution of a number of transactions
that access different portions of a relation
• views that cannot be defined on a single fragment
will require extra processing
• semantic data control (especially integrity
enforcement) more difficult

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 7 of 65

9-Sep-19
 Fragmentation Alternatives – Horizontal

PROJ1 : projects with budgets less than PROJ

$200,000 PNO PNAME BUDGET LOC

PROJ2 : projects with budgets greater P1 Instrumentation 150000 Montreal

P2 Database Develop. 135000 New York
than or equal to $200,000 P3 CAD/CAM 250000 New York
P4 Maintenance 310000 Paris
P5 CAD/CAM 500000 Boston

PROJ1 PROJ2

PNO PNAME BUDGET LOC PNO PNAME BUDGET LOC

P1 Instrumentation 150000 Montreal P3 CAD/CAM 250000 New York
P2 Database Develop. 135000 New York P4 Maintenance 310000 Paris
P5 CAD/CAM 500000 Boston

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 8 of 65

9-Sep-19
 Fragmentation Alternatives – Vertical
PROJ
PROJ1: information about project PNO PNAME BUDGET LOC
budgets P1 Instrumentation 150000 Montreal
PROJ2: information about project P2 Database Develop. 135000 New York
P3 CAD/CAM 250000 New York
names and locations P4 Maintenance 310000 Paris
P5 CAD/CAM 500000 Boston

PROJ1 PROJ2
PNO BUDGET PNO PNAME LOC

P1 150000 P1 Instrumentation Montreal

P2 135000 P2 Database Develop. New York
P3 250000 P3 CAD/CAM New York
P4 310000 P4 Maintenance Paris
P5 500000 P5 CAD/CAM Boston

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 9 of 65

9-Sep-19
Degree of Fragmentation
finite number of alternatives

tuples relations
or
attributes

Finding the suitable level of partitioning within this

range

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 10 of 65

9-Sep-19
Correctness of Fragmentation

• Completeness
• Decomposition of relation R into fragments R1, R2, ..., Rn is
complete if and only if each data item in R can also be found
in some Ri
• Reconstruction
• If relation R is decomposed into fragments R1, R2, ..., Rn, then
there should exist some relational operator ∇ such that
R = ∇1≤i≤nRi

• Disjointness
• If relation R is decomposed into fragments R1, R2, ..., Rn, and
data item di is in Rj, then di should not be in any other
fragment Rk (k ≠ j ).

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 11 of 65

9-Sep-19
Allocation Alternatives

•Non-replicated
• partitioned : each fragment resides at only one
site
•Replicated
• fully replicated : each fragment at each site
• partially replicated : each fragment at some of
the sites
•Rule of thumb:
If read-only queries << 1, replication is advantageous,
update queries
otherwise replication may cause problems

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 12 of 65

9-Sep-19
Comparison of Replication Alternatives

Full-replication Partial-replication Partitioning

QUERY Same Difficulty

Easy
PROCESSING

DIRECTORY Easy or Same Difficulty

MANAGEMENT Non-existant

CONCURRENCY
Moderate Difficult Easy
CONTROL

RELIABILITY Very high High Low

Possible Possible
REALITY Realistic
application application
Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 13 of 65
9-Sep-19
Information Requirements

•Four categories:
• Database information
• Application information
• Communication network information
• Computer system information

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 14 of 65

9-Sep-19
Fragmentation

•Horizontal Fragmentation (HF)

• Primary Horizontal Fragmentation (PHF)
• Derived Horizontal Fragmentation (DHF)

•Vertical Fragmentation (VF)

•Hybrid Fragmentation (HF)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 15 of 65

9-Sep-19
 PHF – Information Requirements
• Database Information
• relationship

SKILL
TITLE, SAL

L1
EMP PROJ
ENO, ENAME, TITLE PNO, PNAME, BUDGET,
LOC

ASG
ENO, PNO, RESP, DUR

• cardinality of each relation: card(R)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 16 of 65

9-Sep-19
PHF - Information Requirements
• Application Information
• simple predicates : Given R[A1, A2, …, An], a simple predicate
pj is
pj : Ai θValue
where θ  {=,<,≤,>,≥,≠}, Value  Di and Di is the domain of Ai.
For relation R we define Pr = {p1, p2, …,pm}
Example :
PNAME = "Maintenance"
BUDGET ≤ 200000
• minterm predicates : Given R and Pr = {p1, p2, …,pm}
define M = {m1,m2,…,mr} as
M = { mi | mi = pjPr pj* }, 1≤j≤m, 1≤i≤z
where pj* = pj or pj* = ¬(pj).

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 17 of 65

9-Sep-19
PHF – Information Requirements

Example
m1: PNAME="Maintenance"  BUDGET≤200000

m2: NOT(PNAME="Maintenance")  BUDGET≤200000

m3: PNAME= "Maintenance"  NOT(BUDGET≤200000)

m4: NOT(PNAME="Maintenance")  NOT(BUDGET≤200000)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 18 of 65

9-Sep-19
PHF – Information Requirements
• Application Information
• minterm selectivities: sel(mi)
• The number of tuples of the relation that would be
accessed by a user query which is specified according to a
given minterm predicate mi.
• access frequencies: acc(qi)
• The frequency with which a user application qi accesses
data.
• Access frequency for a minterm predicate can also be
defined.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 19 of 65

9-Sep-19
Primary Horizontal Fragmentation

Definition :
Rj = Fj(R), 1 ≤ j ≤ w
where Fj is a selection formula, which is (preferably) a minterm
predicate.
Therefore,
A horizontal fragment Ri of relation R consists of all the tuples of R
which satisfy a minterm predicate mi.

Given a set of minterm predicates M, there are as many horizontal
fragments of relation R as there are minterm predicates.
Set of horizontal fragments also referred to as minterm fragments.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 20 of 65

9-Sep-19
PHF – Algorithm

Given: A relation R, the set of simple predicates Pr

Output: The set of fragments of R = {R1, R2,…,Rw}
which obey the fragmentation rules.

Preliminaries :
• Pr should be complete
• Pr should be minimal

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 21 of 65

9-Sep-19
Completeness of Simple Predicates
• A set of simple predicates Pr is said to be complete if and
only if the accesses to the tuples of the minterm fragments
defined on Pr requires that two tuples of the same minterm
fragment have the same probability of being accessed by any
application.

• Example :
• Assume PROJ[PNO,PNAME,BUDGET,LOC] has two
applications defined on it.
• Find the budgets of projects at each location. (1)
• Find projects with budgets less than $200000. (2)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 22 of 65

9-Sep-19
Completeness of Simple Predicates

According to (1),
Pr={LOC=“Montreal”,LOC=“New York”,LOC=“Paris”}
which is not complete with respect to (2).
Modify
Pr ={LOC=“Montreal”,LOC=“New York”,LOC=“Paris”,
BUDGET≤200000,BUDGET>200000}
which is complete.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 23 of 65

9-Sep-19
Minimality of Simple Predicates
• If a predicate influences how fragmentation is performed,
(i.e., causes a fragment f to be further fragmented into, say, fi
and fj) then there should be at least one application that
accesses fi and fj differently.
• In other words, the simple predicate should be relevant in
determining a fragmentation.
• If all the predicates of a set Pr are relevant, then Pr is
minimal.

acc(mi ) acc(m j )
=
card( fi ) card( f j )

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 24 of 65

9-Sep-19
Minimality of Simple Predicates

Example :
Pr ={LOC=“Montreal”,LOC=“New York”, LOC=“Paris”,
BUDGET≤200000,BUDGET>200000}

is minimal (in addition to being complete). However, if we

add
PNAME = “Instrumentation”
then Pr is not minimal.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 25 of 65

9-Sep-19
COM_MIN Algorithm

Given: a relation R and a set of simple

predicates Pr
Output: a complete and minimal set of simple
predicates Pr' for Pr

Rule 1: a relation or fragment is partitioned into

at least two parts which are accessed
differently by at least one application.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 26 of 65

9-Sep-19
COM_MIN Algorithm

Initialization :
• find a pi  Pr such that pi partitions R according to Rule
1
• set Pr' = pi ; Pr Pr – {pi} ; F  {fi}
Iteratively add predicates to Pr' until it is complete
• find a pj  Pr such that pj partitions some fk defined
according to minterm predicate over Pr' according to
Rule 1
• set Pr' = Pr'  {pi}; Pr Pr – {pi}; F  F  {fi}
• if pk  Pr' which is nonrelevant then
Pr'  Pr – {pi}
F  F – {fi}

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 27 of 65

9-Sep-19
PHORIZONTAL Algorithm
Makes use of COM_MIN to perform fragmentation.
Input: a relation R and a set of simple predicates Pr
Output: a set of minterm predicates M according to which
relation R is to be fragmented

 Pr'  COM_MIN (R,Pr)

 determine the set M of minterm predicates
 determine the set I of implications among pi  Pr
 eliminate the contradictory minterms from M

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 28 of 65

9-Sep-19
PHF – Example
• Two candidate relations : PAY and PROJ.
• Fragmentation of relation PAY
• Application: Check the salary info and determine raise.
• Employee records kept at two sites  application run at two
sites
• Simple predicates
p1 : SAL ≤ 30000
p2 : SAL > 30000
Pr = {p1,p2} which is complete and minimal Pr'=Pr
• Minterm predicates
m1 : (SAL ≤ 30000)
m2 : NOT(SAL ≤ 30000) = (SAL > 30000)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 29 of 65

9-Sep-19
PHF – Example
PAY1 PAY2

TITLE SAL TITLE SAL

Mech. Eng. 27000 Elect. Eng. 40000

Programmer 24000 Syst. Anal. 34000

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 30 of 65

9-Sep-19
PHF – Example
• Fragmentation of relation PROJ
• Applications:
• Find the name and budget of projects given their no.
• Issued at three sites
• Access project information according to budget
• one site accesses ≤200000 other accesses >200000
• Simple predicates
• For application (1)
p1 : LOC = “Montreal”
p2 : LOC = “New York”
p3 : LOC = “Paris”
• For application (2)
p4 : BUDGET ≤ 200000
p5 : BUDGET > 200000
• Pr = Pr' = {p1,p2,p3,p4,p5}

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 31 of 65

9-Sep-19
PHF – Example

• Fragmentation of relation PROJ continued

• Minterm fragments left after elimination
m1 : (LOC = “Montreal”)  (BUDGET ≤ 200000)
m2 : (LOC = “Montreal”)  (BUDGET > 200000)
m3 : (LOC = “New York”)  (BUDGET ≤ 200000)
m4 : (LOC = “New York”)  (BUDGET > 200000)
m5 : (LOC = “Paris”)  (BUDGET ≤ 200000)
m6 : (LOC = “Paris”)  (BUDGET > 200000)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 32 of 65

9-Sep-19
PHF – Example

PROJ1 PROJ2

PNO PNAME BUDGET LOC PNO PNAME BUDGET LOC

Database
P1 Instrumentation 150000 Montreal P2 135000 New York
Develop.

PROJ4 PROJ6

PNO PNAME BUDGET LOC PNO PNAME BUDGET LOC

P3 CAD/CAM 250000 New P4 Maintenance 310000 Paris
York

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 33 of 65

9-Sep-19
PHF – Correctness

• Completeness
• Since Pr' is complete and minimal, the selection predicates are
complete

• Reconstruction
• If relation R is fragmented into FR = {R1,R2,…,Rr}

R = Ri FR Ri
• Disjointness
• Minterm predicates that form the basis of fragmentation
should be mutually exclusive.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 34 of 65

9-Sep-19
Derived Horizontal Fragmentation

• Defined on a member relation of a link according to a selection

operation specified on its owner.
• Each link is an equijoin.
• Equijoin can be implemented by means of semijoins.

SKILL
TITLE, SAL

L1
EMP PROJ

ENO, ENAME, TITLE PNO, PNAME, BUDGET, LOC

L2 L3

ASG

ENO, PNO, RESP, DUR

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 35 of 65

9-Sep-19
DHF – Definition

Given a link L where owner(L)=S and member(L)=R,

the derived horizontal fragments of R are defined as
Ri = R ⋉F Si, 1≤i≤w
where w is the maximum number of fragments that
will be defined on R and
Si = Fi (S)

where Fi is the formula according to which the

primary horizontal fragment Si is defined.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 36 of 65

9-Sep-19
DHF – Example

Given link L1 where owner(L1)=SKILL and member(L1)=EMP

EMP1 = EMP ⋉ SKILL1
EMP2 = EMP ⋉ SKILL2
where
SKILL1 = SAL≤30000(SKILL)
SKILL2 = SAL>30000(SKILL)

EMP1 EMP2
ENO ENAME TITLE ENO ENAME TITLE

E3 A. Lee Mech. Eng. E1 J. Doe Elect. Eng.

E4 J. Miller Programmer E2 M. Smith Syst. Anal.
E7 R. Davis Mech. Eng. E5 B. Casey Syst. Anal.
E6 L. Chu Elect. Eng.
E8 J. Jones Syst. Anal.
Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 37 of 65
9-Sep-19
DHF – Correctness
• Completeness
• Referential integrity
• Let R be the member relation of a link whose owner is relation
S which is fragmented as FS = {S1, S2, ..., Sn}. Furthermore, let A
be the join attribute between R and S. Then, for each tuple t
of R, there should be a tuple t' of S such that
t[A] = t' [A]
• Reconstruction
• Same as primary horizontal fragmentation.
• Disjointness
• Simple join graphs between the owner and the member
fragments.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 38 of 65

9-Sep-19
Vertical Fragmentation

• Has been studied within the centralized context

• design methodology
• physical clustering
• More difficult than horizontal, because more
alternatives exist.
Two approaches :
• grouping
• attributes to fragments
• splitting
• relation to fragments

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 39 of 65

9-Sep-19
Vertical Fragmentation

• Overlapping fragments
• grouping
• Non-overlapping fragments
• splitting
We do not consider the replicated key attributes to be
overlapping.
Advantage:
Easier to enforce functional dependencies
(for integrity checking etc.)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 40 of 65

9-Sep-19
VF – Information Requirements
• Application Information
• Attribute affinities
• a measure that indicates how closely related the attributes are
• This is obtained from more primitive usage data
• Attribute usage values
• Given a set of queries Q = {q1, q2,…, qq} that will run on the
relation R[A1, A2,…, An],

1 if attribute Aj is referenced by query qi


use(qi,Aj) = 
 0 otherwise

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 41 of 65

9-Sep-19
VF – Definition of use(qi,Aj)
Consider the following 4 queries for relation PROJ
q1: SELECT BUDGET q2: SELECT PNAME,BUDGET
FROM PROJ FROM PROJ
WHERE PNO=Value
q3: SELECT PNAME q4: SELECT SUM(BUDGET)
FROM PROJ FROM PROJ
WHERE LOC=Value WHERE LOC=Value
Let A1= PNO, A2= PNAME, A3= BUDGET, A4= LOC

A1 A2 A3 A4
q1 1 0 1 0

q2 0 1 1 0

q3 0 1 0 1

q4 0 0 1 1

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 42 of 65

9-Sep-19
VF – Affinity Measure aff(Ai,Aj)

The attribute affinity measure between two attributes Ai and Aj of a

relation R[A1, A2, …, An] with respect to the set of applications Q =
(q1, q2, …, qq) is defined as follows :

aff (Ai, Aj) =  (query access)

all queries that access A and A i j


access
query access = access frequency of a query 
execution
all sites

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 43 of 65

9-Sep-19
VF – Calculation of aff(Ai, Aj)

Assume each query in the previous example accesses the

attributes once during each execution.
S S S
1 2 3
Also assume the access frequencies
q1 15 20 10
q2 5 0 0
q3 25 25 25

q
4 3 0 0
Then
aff(A1, A3) = 15*1 + 20*1+10*1 A1 A2 A3 A4
= 45 A1 45 0 45 0
and the attribute affinity matrix AA is A2 0 80 5 75
A3 45 5 53 3
A4 0 75 3 78

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 44 of 65

9-Sep-19
VF – Clustering Algorithm

• Take the attribute affinity matrix AA and reorganize the

attribute orders to form clusters where the attributes in each
cluster demonstrate high affinity to one another.
• Bond Energy Algorithm (BEA) has been used for clustering of
entities. BEA finds an ordering of entities (in our case
attributes) such that the global affinity measure is
maximized.

AM =  (affinity of Ai and Aj with their neighbors)

i j

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 45 of 65

9-Sep-19
Bond Energy Algorithm

Input: The AA matrix

Output: The clustered affinity matrix CA which is a
perturbation of AA
Initialization: Place and fix one of the columns of AA in
CA.
Iteration: Place the remaining n-i columns in the
remaining i+1 positions in the CA matrix. For each
column, choose the placement that makes the most
contribution to the global affinity measure.
Row order: Order the rows according to the column
ordering.

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 46 of 65

9-Sep-19
Bond Energy Algorithm
“Best” placement? Define contribution of a placement:

cont(Ai, Ak, Aj) = 2bond(Ai, Ak)+2bond(Ak, Al) –2bond(Ai, Aj)

where

n
bond(Ax,Ay) =  aff(Az,Ax)aff(Az,Ay)

z =1

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 47 of 65

9-Sep-19
BEA – Example
Consider the following AA matrix and the corresponding CA matrix
where A1 and A2 have been placed. Place A3:

Ordering (0-3-1) :
cont(A0,A3,A1) = 2bond(A0 , A3)+2bond(A3 , A1)–2bond(A0 , A1)
= 2* 0 + 2* 4410 – 2*0 = 8820
Ordering (1-3-2) :
cont(A1,A3,A2) = 2bond(A1 , A3)+2bond(A3 , A2)–2bond(A1,A2)
= 2* 4410 + 2* 890 – 2*225 = 10150
Ordering (2-3-4) :
cont (A2,A3,A4) = 1780
Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 48 of 65
9-Sep-19
BEA – Example
• Therefore, the CA matrix has the form A1 A3 A2

45 45 0
0 5 80
45 53 5
0 3 75

• When A4 is placed, the final form of the CA A1 A3 A2 A4

matrix (after row organization) is A 1 45 45 0 0
A3 45 53 5 3
A2 0 5 80 75
A4 0 3 75 78
Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 49 of 65
9-Sep-19
VF – Algorithm
How can you divide a set of clustered attributes {A1, A2, …,
An} into two (or more) sets {A1, A2, …, Ai} and {Ai, …, An}
such that there are no (or minimal) applications that
access both (or more than one) of the sets.

A1
A2
A3 … Ai Ai+1 . . .Am
A1 A2 TA

Ai

Ai+1
BA

Am

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 50 of 65

9-Sep-19
VF – ALgorithm

Define
TQ = set of applications that access only TA
BQ = set of applications that access only BA
OQ = set of applications that access both TA and BA
and
CTQ = total number of accesses to attributes by applications
that access only TA
CBQ = total number of accesses to attributes by applications
that access only BA
COQ = total number of accesses to attributes by applications
that access both TA and BA
Then find the point along the diagonal that maximizes

CTQCBQCOQ2

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 51 of 65

9-Sep-19
VF – Algorithm
Two problems :
Cluster forming in the middle of the CA matrix
• Shift a row up and a column left and apply the algorithm to
find the “best” partitioning point
• Do this for all possible shifts
• Cost O(m2)
More than two clusters
• m-way partitioning
• try 1, 2, …, m–1 split points along diagonal and try to find the
best point for each of these
• Cost O(2m)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 52 of 65

9-Sep-19
VF – Correctness

A relation R, defined over attribute set A and key K, generates

the vertical partitioning FR = {R1, R2, …, Rr}.
• Completeness
• The following should be true for A:
A =  ARi
• Reconstruction
• Reconstruction can be achieved by
R = ⋈•K Ri, Ri  FR

• Disjointness
• TID's are not considered to be overlapping since they are
maintained by the system
• Duplicated keys are not considered to be overlapping

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 53 of 65

9-Sep-19
Hybrid Fragmentation

HF HF

R1 R2

VF VF VF VF VF

R11 R12 R21 R22 R23

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 54 of 65

9-Sep-19
Fragment Allocation
• Problem Statement
Given
F = {F1, F2, …, Fn} fragments
S ={S1, S2, …, Sm} network sites
Q = {q1, q2,…, qq} applications
Find the "optimal" distribution of F to S.
• Optimality
• Minimal cost
• Communication + storage + processing (read & update)
• Cost in terms of time (usually)
• Performance
Response time and/or throughput
• Constraints
• Per site constraints (storage & processing)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 55 of 65

9-Sep-19
Information Requirements
• Database information
• selectivity of fragments
• size of a fragment
• Application information
• access types and numbers
• access localities
• Communication network information
• unit cost of storing data at a site
• unit cost of processing at a site
• Computer system information
• bandwidth
• latency
• communication overhead

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 56 of 65

9-Sep-19
Allocation

File Allocation (FAP) vs Database Allocation (DAP):

• Fragments are not individual files
• relationships have to be maintained

• Access to databases is more complicated

• remote file access model not applicable
• relationship between allocation and query processing

• Cost of integrity enforcement should be considered

• Cost of concurrency control should be considered

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 57 of 65

9-Sep-19
Allocation – Information Requirements
• Database Information
• selectivity of fragments
• size of a fragment
• Application Information
• number of read accesses of a query to a fragment
• number of update accesses of query to a fragment
• A matrix indicating which queries updates which fragments
• A similar matrix for retrievals
• originating site of each query
• Site Information
• unit cost of storing data at a site
• unit cost of processing at a site
• Network Information
• communication cost/frame between two sites
• frame size
Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 58 of 65
9-Sep-19
Allocation Model
General Form
min(Total Cost)
subject to
response time constraint
storage constraint
processing constraint

Decision Variable

1 if fragment Fi is stored at site Sj

xij =
0 otherwise

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 59 of 65

9-Sep-19
Allocation Model

• Total Cost

 query processing cost 

all queries

  cost of storing a fragment at a site

all sites all fragments

• Storage Cost (of fragment Fj at Sk)

(unit storage cost at Sk)  (size of Fj)  xjk

• Query Processing Cost (for one query)

processing component + transmission component

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 60 of 65

9-Sep-19
Allocation Model
• Query Processing Cost
Processing component
access cost + integrity enforcement cost + concurrency control
cost

• Access cost

  (no. of update accesses+ no. of read accesses) 

all sites all fragments
xij  local processing cost at a site

• Integrity enforcement and concurrency control costs

• Can be similarly calculated

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 61 of 65

9-Sep-19
Allocation Model
• Query Processing Cost
Transmission component
cost of processing updates + cost of processing retrievals
• Cost of updates

  update message cost 

all sites all fragments
  acknowledgment cost
all sites all fragments
• Retrieval Cost

 min
all sites
(cost of retrieval command 

all fragments cost of sending back the result)

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 62 of 65

9-Sep-19
Allocation Model

• Constraints
• Response Time
execution time of query ≤ max. allowable response time for that
query
• Storage Constraint (for a site)

 storage requirement of a fragment at that site 

storage capacity at that site
all fragments

• Processing constraint (for a site)

 processing load of a query at that site 

all queries processing capacity of that site

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 63 of 65

9-Sep-19
Allocation Model

• Solution Methods
• FAP is NP-complete
• DAP also NP-complete

• Heuristics based on
• single commodity warehouse location (for FAP)
• knapsack problem
• branch and bound techniques
• network flow

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 64 of 65

9-Sep-19
Allocation Model

•Attempts to reduce the solution space

•assume all candidate partitionings known;

select the “best” partitioning

•ignore replication at first

•sliding window on fragments

Department of Computer Science &Engineering, National Institute of Technology Karnataka, Surathkal 65 of 65

9-Sep-19