Unit 3

Divide and Conquer Strategy: Introduction, The

2-dimensional maxima finding problem, The
closest pair problem, The convex hull problem.
The Greedy method: The 2-way merge problem,
The minimum cycle basis problem, The 2-terminal
one to any problem. Dynamic Programming:
The resource allocation problem, The longest
common subsequence problem
 A simple example
 finding the maximum of a set S of n numbers

5 -3
 Time complexity
 Time complexity:
2
T(
n
/2
)
+1
,n
>
2
T
(
n
)
=
1 , 
n
2
 Calculation of T(n):
Assume n = 2k,
T(n) = 2T(n/2)+1
= 2(2T(n/4)+1)+1
= 4T(n/4)+2+1
:
=2k-1T(2)+2k-2+…+4+2+1
=2k-1+2k-2+…+4+2+1
=2k-1 = n-1

5 -4
A general divide-and-conquer
algorithm
Step 1: If the problem size is small, solve this
problem directly; otherwise, split the
original problem into 2 sub-problems
with equal sizes.
Step 2: Recursively solve these 2 sub-problems
by applying this algorithm.
Step 3: Merge the solutions of the 2 sub-
problems into a solution of the original
problem.

5 -5
Time complexity of the general
algorithm
 Time complexity:


2
T(
n
/2
)
+S
(
n)
+
M(
n),
nc
T
(
n
)=
b ,
n<c

where S(n) : time for splitting

M(n) : time for merging
b : a constant
c : a constant
 e.g. Binary search
 e.g. quick sort
 e.g. merge sort
5 -6
 Master Theorem
The master theorem is a recipe that gives asymptotic estimates for a
class of recurrence relations that often show up when analyzing
recursive algorithms.
Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and let T(n) be
a function over the positive numbers defined by the recurrence
T(n) = a T(n/b) + f(n).
If f(n) = Θ(nd), where d ≥ 0,
then
T(n) = Θ(nd) if a < bd,
T(n) = Θ(ndlog n) if a = bd,
T(n) = Θ(nlogba) if a > bd.
Example 1
T(n) = 4T(n/2) + n

Here, a = 4, b = 2, and d = 1.
Since a > bd, ( 4 >2)

T(n) ∈ Θ(nlog24) = Θ(n2).

Example 2
T(n) = 4T(n/2) + n2

Here, a = 4, b = 2, and d = 2

Since a = bd ( 4 = 4)

T(n) ∈ Θ(n2 log n).

Example 3
T(n) = 4T(n/2) + n3

Here, a = 4, b = 2, and d = 3

Since a < bd, ( 4 < 8)

T(n) ∈ Θ(n3)
 Binary search

 sorted sequence : (search 9)

1 4 5 7 9 10 12 15
step 1 
step 2 
step 3 
 best case: 1 step = (1)

 worst case: (log2 n+1) steps = (log n)

 average case: (log n) steps

2 -11
 n cases for successful search
n+1 cases for unsuccessful search

Average # of comparisons done in the binary tree:

A(n) = , where k = log n+1

1  k i 1 
  i 2  k ( n  1)
2 n  1  i 1 
2 -12
 k
A
s
k
s
um
en
=
2 p
r
ov
ed
by
i
nd
uc
t
i
on


i
2
i

1 k
2(
k
1
)1 on
k

i1

A(n) = 1
(( k  1) 2 k  1  k (2 k  1))
2n  1
k as n is very large
= log n
= (log n)

2 -13
 2-D ranking finding
 Def: Let A = (a1,a2), B = (b1,b2). A dominates B iff a1> b1 and a2 > b2
 Def: Given a set S of n points, the rank of a point x is the number of
points dominated by x.

D
B

A C
E

rank(A)= 0 rank(B) = 1 rank(C) = 1

rank(D) = 3 rank(E) = 0

2 -14
 Straightforward algorithm:
compare all pairs of points : O(n2)

2 -15
 Divide-and-conquer 2-D ranking finding
Input: A set S of planar points P1,P2,…,Pn
Output: The rank of every point in S
Step 1: (Split the points along the median line L into A and B.)
a.If S contains only one point, return its rank its rank as 0.
b.Otherwise,choose a cut line L perpendicular to the x-axis such that n/2
points of S have X-values L (call this set of points A) and the remainder
points have X-values L(call this set B).Note that L is a median X-value of
this set.
Step 2:
Find ranks of points in A and ranks of points in B, recursively.
Step 3:
Sort points in A and B according to their y-values.
Scan these points sequentially and determine, for each point in B, the
number of points in A whose y-values are less than its y-value. The rank of
this point is equal to the rank of this point among points in B, plus the
number of points in A whose y-values are less than its y-value.

2 -16
2 -17
 2-D maxima finding problem

 Def : A point (x1, y1) dominates (x2, y2) if x1 > x2

and y1 > y2. A point is called a maxima if no
other point dominates it
 Straightforward method : Compare every pair
of points.

Time complexity:
O(n2)

5 -18
Divide-and-conquer for
maxima finding

The maximal points of SL and SR

5 -19
The algorithm:
 Input: A set of n planar points.
 Output: The maximal points of S.

Step 1: If S contains only one point, return it as

the maxima. Otherwise, find a line L
perpendicular to the X-axis which separates the
set of points into two subsets SLand SR , each of
which consisting of n/2 points.
Step 2: Recursively find the maximal points of SL
and SR .
Step 3: Find the largest y-value of SR. Project the
maximal points of SL onto L. Discard each of
the maximal points of SL if its y-value is less
than the largest y-value of SR .
5 -20
 Time complexity: T(n)
Step 1: O(n)
Step 2: 2T(n/2)
Step 3: O(n)
2
T
(n
/
2
)+
O(
n
)+
O(
n
) ,
n>1
T
(
n
)
=
1 ,
n=1

Assume n = 2k
T(n) = O(n log n)

5 -21
 The closest pair problem
 Given a set S of n points, find a pair of points
which are closest together.
 1-D version :  2-D version

Solved by sorting
Time complexity :
O(n log n)

5 -22
 at most 6 points in area A:

5 -23
The algorithm:
 Input: A set of n planar points.
 Output: The distance between two closest
points.
Step 1: Sort points in S according to their y-values
and x-values.
Step 2: If S contains only two points, return
infinity as their distance.
Step 3: Find a median line L perpendicular to the
X-axis to divide S into two subsets, with equal
sizes, SL and SR.
Step 4: Recursively apply Step 2 and Step 3 to
solve the closest pair problems of SL and SR.
Let dL(dR) denote the distance between the
closest pair in SL (SR). Let d = min(dL, dR). 5 -24
Step 5: For a point P in the half-slab bounded by
L-d and L, let its y-value by denoted as yP . For
each such P, find all points in the half-slab
bounded by L and L+d whose y-value fall
within yP +d and yP -d. If the distance d
between P and a point in the other half-slab is
less than d, let d=d . The final value of d is the
answer.
 Time complexity: O(n log n)

Step 1: O(n log n)

Steps 2~5:
2
T
(n
/
2
)+
O(
n
)+
O(
n
),n
>1
T
(
n
)
=
1 ,n
=1
T(n) = O(n log n)
5 -25
 The convex hull problem
concave polygon: convex polygon:

 The convex hull of a set of planar points is the

smallest convex polygon containing all of the
points.

5 -26
 The divide-and-conquer strategy to solve
the problem:

5 -27
 The merging procedure:
1. Select an interior point p.
2. There are 3 sequences of points which have
increasing polar angles with respect to p.
(1) g, h, i, j, k
(2) a, b, c, d
(3) f, e
1. Merge these 3 sequences into 1 sequence:
g, h, a, b, f, c, e, d, i, j, k.
1. Apply Graham scan to examine the points
one by one and eliminate the points which
cause reflexive angles.
(See the example on the next page.)
5 -28
 e.g. points b and f need to be deleted.

Final result:

5 -29
 Divide-and-conquer for convex hull
 Input : A set S of planar points
 Output : A convex hull for S

Step 1: If S contains no more than five points, use

exhaustive searching to find the convex hull
and return.
Step 2: Find a median line perpendicular to the X-
axis which divides S into SL and SR ; SL lies to
the left of SR .
Step 3: Recursively construct convex hulls for SL
and SR. Denote these convex hulls by Hull(SL)
and Hull(SR) respectively.
5 -30
 Step 4: Apply the merging procedure to merge
Hull(SL) and Hull(SR) together to form a convex
hull.

 Time complexity:
T(n) = 2T(n/2) + O(n)
= O(n log n)

5 -31
 The 2-way merging problem
 # of comparisons required for the linear 2-way
merge algorithm is m1+ m2 -1 where m1 and m2
are the lengths of the two sorted lists
respectively.
 2-way merging example
2 3 5 6
1 4 7 8
 The problem: There are n sorted lists, each of
length mi. What is the optimal sequence of
merging process to merge these n lists into one
sorted list ?

3 -32
 Extended binary trees
 An extended binary tree representing a 2-way
merge

3 -33
An example of 2-way merging
 Example: 6 sorted lists with lengths 2,
3, 5, 7, 11 and 13.

3 -34
 Time complexity for
generating an optimal
extended binary
tree:O(n log n)

3 -35
 Huffman codes
 In telecommunication, how do we represent a
set of messages, each with an access frequency,
by a sequence of 0’s and 1’s?
 To minimize the transmission and decoding
costs, we may use short strings to represent
more frequently used messages.
 This problem can by solved by using an
extended binary tree which is used in the 2-way
merging problem.

3 -36
 An example of Huffman algorithm
 Symbols: A, B, C, D, E, F, G
freq. : 2, 3, 5, 8, 13, 15, 18

 Huffman codes:
A: 10100 B: 10101 C: 1011
D: 100 E: 00 F: 01
G: 11

A Huffman code Tree

3 -37
 The minimal cycle basis
problem
 3 cycles:
A1 = {ab, bc, ca}
A2 = {ac, cd, da}
A3 = {ab, bc, cd, da}
where A3 = A1  A2
(A  B = (AB)-(AB))
A2 = A1  A3
A1 = A2  A3
Cycle basis : {A1, A2} or {A1, A3} or {A2, A3}
3 -38
 Def : A cycle basis of a graph is a set of
cycles such that every cycle in the graph
can be generated by applying  on some
cycles of this basis.
 Minimal cycle basis : smallest total
weight of all edges in this cycle.
 e.g. {A1, A2}

3 -39
 Algorithm for finding a minimal cycle basis:
Step 1: Determine the size of the minimal cycle basis, demoted as k.
Step 2: Find all of the cycles. Sort all cycles( by weight).
Step 3: Add cycles to the cycle basis one by one. Check if the added cycle is
a linear combination of some cycles already existing in the basis. If it is,
delete this cycle.
Step 4: Stop if the cycle basis has k cycles.

3 -40
 Detailed steps for the minimal
cycle basis problem
 Step 1 :
A cycle basis corresponds to the fundamental set of cycles with respect to a
spanning tree.

a graph a spanning tree a fundamental set of

cycles

# of cycles in a cycle
basis :

=k

= |E| - (|V|- 1)

= |E| - |V| + 1

3 -41
 Step 2:
How to find all cycles in a graph?
How many cycles in a graph in the worst case?

In a complete digraph of n vertices and n(n-1) edges:

 Step 3:
How to check if a cycle is a linear combination of some cycles?
Using Gaussian elimination.

3 -42
 Gaussian elimination
2 cycles C1 and C2 are represented by a 0/1
matrix
 E.g. e
1 e
2 e
3 e
4 e
5
C
1 111
C
2 111

  on rows 1 and 3
e
1 e
2 e
3 e
4 e
5
 Add C3 C
1 1 1 1

e C
2 111
1 e
2 e
3 e
4 e
5
C
3 111
C
1 1 1 1
C
2 111
 on rows 2 and 3 : empty
C
3 1 1 11
∵C3 = C1  C2
3 -43
| E| = 8 | V | = 6 K = 8 – 6 - 1 = 3

C1 is added
C2 is added
C3 is added and deleted because C3 is a linear combination
of C1 and C2
C4 is added and the minimum cycle basis is { C1, C2, C4}
 e2 e8

e3
e1 e5 e7

e4 e6
 The 2-terminal one to any special
channel routing problem
 Def: Given two sets of terminals on the upper and lower rows,
respectively, we have to connect each upper terminal to the lower row in a
one to one fashion. This connection requires that # of tracks used is
minimized.

3 -47
2 feasible solutions
via

3 -48
 Redrawing solutions
(a) Optimal solution

(b) Another solution

3 -49
 At each point, the local density of the solution
is # of lines the vertical line intersects.
 The problem: to minimize the density. The
density is a lower bound of # of tracks.
 Upper row terminals: P1 ,P2 ,…, Pn from left to
right
 Lower row terminals: Q1 ,Q2 ,…, Qm from left
to right m > n.
 It would never have a crossing connection:

3 -50
 Suppose that we have a method to determine
the minimum density, d, of a problem
instance.
 The greedy algorithm:

Step 1 : P1 is connected Q1.

Step 2 : After Pi is connected to Qj, we check
whether Pi+1 can be connected to Qj+1. If the
density is increased to d+1, try to connect Pi+1
to Qj+2.
Step 3 : Repeat Step2 until all Pi’s are connected.

3 -51
 Fibonacci sequence
 Fibonacci sequence: 0 , 1 , 1 , 2 , 3 , 5 , 8 , 13 , 21 , …
Fi = i if i  1
Fi = Fi-1 + Fi-2 if i  2
 Solved by a recursive program:
f5

f4 f3

f3 f2 f2 f1

f2 f1 f1 f0 f1 f0

f1 f0

 Much replicated computation is done.

 It should be solved by a simple loop.

8 -52
 Chapter 7

Dynamic Programming

8 -53
 Dynamic Programming
 Dynamic Programming is an algorithm
design method that can be used when
the solution to a problem may be viewed
as the result of a sequence of decisions

8 -54
 The shortest path
 To find a shortest path in a multi-stage graph
3 2 7

1 4
S A B 5
T

5 6

 Apply the greedy method :

the shortest path from S to T :
1+2+5=8

8 -55
The shortest path in multistage
graphs
 e.g. A
4
D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F

 The greedy method can not be applied to this

case: (S, A, D, T) 1+4+18 = 23.
 The real shortest path is:
(S, C, F, T) 5+2+2 = 9.
8 -56
 Dynamic programming approach
 Dynamic programming approach (forward approach):

1 A
d(A, T)

2 d(B, T)
S B T

d(C, T)
5
C
 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}

4
 d(A,T) = min{4+d(D,T), 11+d(E,T)} A D
d(D, T)
= min{4+18, 11+13} = 22.
11
E T
d(E, T)

8 -57
 Dynamic programming

 d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}

= min{9+18, 5+13, 16+2} = 18.
 d(C, T) = min{ 2+d(F, T) } = 2+2 = 4
 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
 The above way of reasoning is called
backward reasoning.

8 -58
 Backward approach
(forward reasoning)
 d(S, A) = 1
d(S, B) = 2
d(S, C) = 5
 d(S,D)=min{d(S, A)+d(A, D),d(S, B)+d(B, D)}

= min{ 1+4, 2+9 } = 5

d(S,E)=min{d(S, A)+d(A, E),d(S, B)+d(B, E)}
= min{ 1+11, 2+5 } = 7
d(S,F)=min{d(S, A)+d(A, F),d(S, B)+d(B, F)}
= min{ 2+16, 5+2 } = 7
8 -59
 d(S,T) = min{d(S, D)+d(D, T),d(S,E)+
d(E,T), d(S, F)+d(F, T)}
= min{ 5+18, 7+13, 7+2 }
=9

8 -60
 Principle of optimality
 Principle of optimality: Suppose that in solving a
problem, we have to make a sequence of
decisions D1, D2, …, Dn. If this sequence is
optimal, then the last k decisions, 1  k  n must
be optimal.
 e.g. the shortest path problem
If i, i1, i2, …, j is a shortest path from i to j, then i1,
i2, …, j must be a shortest path from i1 to j
 In summary, if a problem can be described by a
multistage graph, then it can be solved by
dynamic programming.

8 -61
 Dynamic programming
 Forward approach and backward approach:
 Note that if the recurrence relations are formulated using the forward
approach then the relations are solved backwards . i.e., beginning
with the last decision
 On the other hand if the relations are formulated using the backward
approach, they are solved forwards.
 To solve a problem by using dynamic
programming:
 Find out the recurrence relations.
 Represent the problem by a multistage graph.

8 -62
 The resource allocation
problem
 m resources, n projects
profit p(i, j) : j resources are allocated to project
i.
maximize the total profit.
Resou
rc
e
P
ro
je
ct 1 2 3
1 2 8 9
2 5 6 7
3 4 4 4
4 2 4 5

8 -63
 The multistage graph solution
A E I
0,1 0 0,2 0 0,3
5 4
0 5
B 7 6 F 4 4 J
1,1 0 1,2 0 1,3
2 4
5 4
S 8 C
6
G
4
K 2
T
2,1 0 2,2 0 2,3
9 0
5 4
D H L
3,1 0 3,2 0 3,3

 The resource allocation problem can be described as a multistage graph.

 (i, j) : i resources allocated to projects 1, 2, …, j
e.g. node H=(3, 2) : 3 resources allocated to projects 1, 2.

8 -64
 Find the longest path from S to T :
(S, C, H, L, T), 8+5+0+0=13
2 resources allocated to project 1.
1 resource allocated to project 2.
0 resource allocated to projects 3, 4.

8 -65