Algorithm Design
Techniques
Dr M Muzammal
Divide and Conquer
The most-well known algorithm design strategy:
Divide instance of problem into two or more smaller
instances
Solve smaller instances recursively
Obtain solution to original (larger) instance by combining
these solutions
Divide and Conquer
Divide the problem into sub-problems that are similar to
the original but smaller in size
Conquer the sub-problems by solving them recursively. If
they are small enough, just solve them in a straightforward
manner.
Combine the solutions to create a solution to the original
problem
Analysis:
Divide and Conquer Algorithm
D&C algorithms result into following general recurrence relation:
T(n) = aT(n/b) + f (n) where f(n) (nd), d 0
Master Theorem: If a < bd, T(n) (nd)
If a = bd, T(n) (nd log n)
If a > bd, T(n) (nlog b a )
Examples:
T(n) = 4T(n/2) + n T(n) ?
Let’s Solve it
Analysis:
Divide and Conquer Algorithm
Examples
T(n) = 9T(n/3) + n
T(n) = 8T(n/2) + n2
Divide and Conquer:
Merge Sort
Divide: Divide the n-element sequence to be sorted into two
subsequences of n/2 elements each
Conquer: Sort the two subsequences recursively using merge
sort.
Combine: Merge the two sorted subsequences to produce
the sorted answer
Divide and Conquer:
Merge Sort
8 3 2 9 7 1 5 4
8 3 2 9 7 1 5 4
8 3 2 9 71 5 4
8 3 2 9 7 1 5 4
3 8 2 9 1 7 4 5
2 3 8 9 1 4 5 7
1 2 3 4 5 7 8 9
Divide and Conquer:
Analysis of Merge Sort
Divide: Computing the middle takes (1)
– copying two n/2 elements arrays take 2*n/2 = (n) time
Conquer: Solving 2 sub-problems takes 2T(n/2)
Combine: Merging n elements takes (n)
Total Execution Time:
T(n) = (1) if n = 1
T(n) = 2T(n/2) + (n) if n > 1
Divide and Conquer:
Quick Sort
Select a pivot (partitioning element) – here, the first element
Rearrange the list so that all the elements in the first s
position are smaller than or equal to the pivot and all the
elements in the remaining n-s position are larger than or
equal to the pivot
p
A[i]p A[i]p
Divide and Conquer:
Quick Sort
Exchange the pivot with the last element in the first (i.e., )
subarray — the pivot is now in its final position
Sort the two subarrays recursively
Efficiency:
Best case: split in the middle — Θ(n log n)
Worst case: sorted array! — Θ(n2)
Average case: random arrays — Θ(n log n)
Divide and Conquer:
Binary Tree Traversal
Graph algorithms: binary tree traversals
Inorder traversal:
traverse left subtree of current vertex
visit current vertex
traverse right subtree of current vertex
Preorder traversal similar, but visit current vertex first
Postorder traversal similar, but visit current vertex last
All three take O(n) time, where n is number of nodes in tree
Note difference from searching in a binary tree
Divide and Conquer:
Closest Pair Problem
Recall the problem: Given n points in the plane, find two that
are the minimum distance apart.
Brute force algorithm took Θ(n2) time.
Try to do better with divide and conquer:
― divide points into two disjoint subsets
― recursively find closest pairs in the two subsets
― somehow combine results to get final answer
Divide and Conquer:
Closest Pair Problem
Separate points into two equal-sized groups on either side of a
vertical line
Recursively compute closest pair for left group and for right
group
– what should base of the recursion be?
Check if there is a smaller distance between two points on
opposite sides of the vertical line
– This is the tricky part
Divide and Conquer:
Closest Pair Problem
d is min. of min. distance on right and min.
distance on left
any pair with distance < d must be in this
strip of width 2d centered around dividing
line
consider points in strip from bottom to top
for each such point, compare it against
other points in the strip that could possibly
be closer
there are only a constant number of these
other points!
Divide and Conquer:
Closest Pair Problem
Each box is d/2 by d/2
No point in comparing p
against points in red area –
more than d away
d Just need to worry about
the six blue boxes
p
Each box contains at most one
d d point, since maximum distance
in a box is d/√2, which is < d 15
Divide and Conquer:
Closest Pair Problem
ClosestPairDist(P):
– if n is small then return result of brute force algorithm
– Pl := left half of P w.r.t. x-coordinate
– Pr := right half of P w.r.t. x-coordinate
– dl := ClosestPairDist(Pl)
– dr := ClosestPairDist(Pr)
– d := min(dl,dr)
– for each point p in S (2d-wide center strip) do
o for each point q in one of the six boxes do
• d := min(dist(p,q),d)
– return d
Divide and Conquer:
Closest Pair Problem
Before calling recursive code, preprocess:
– sort P into array PX by increasing x-coordinate
– sort P into array PY by increasing y-coordinate
Use PX to efficiently divide P into half w.r.t. x-coordinates
Use PY to efficiently scan up the 2d-wide center strip
Divide and Conquer:
Closest Pair Problem Analysis
Preprocessing takes O(n log n) time
Recursive code, if implemented carefully, has running time
described by this recurrence:
T(n) = 2T(n/2) + O(n)
– I.e., two recursive calls (left half and right half)
– rest of the work takes time linear in the number of points
being handled
– Solution is T(n) = O(n log n)
Total time is O(n log n); beats brute force
Divide and Conquer:
Multiplication of Large Integers
Consider the problem of multiplying two (large) n-digit
integers represented by arrays of their digits such as:
A = 12345678901357986429 B = 87654321284820912836
The grade-school algorithm:
a1 a2 … an
b1 b2 … bn
(d10) d11d12 … d1n
(d20) d21d22 … d2n
…………………
(dn0) dn1dn2 … dnn
Efficiency: Θ(n2) single-digit multiplications
Divide and Conquer:
Multiplication of Large Integers
Divide And Conquer Solution
Let X and Y be two n-digit numbers
X= a b
Y= c d
X a10n / 2 b, Y c10n / 2 d
X *Y a * c10n (a * d b * c)10n / 2 b * d
Divide and Conquer:
Multiplication of Large Integers
1 if n = 1
T ( n)
4T (n / 2) (n) if n > 1
Solution by Master Theorem:
T (n) n 2
Divide and Conquer:
Large Matrix Multiplication
Input:
Output:
Divide and Conquer:
Large Matrix Multiplication
Idea:
n×n matrix = 2×2 matrix of (n/2)×(n/2) submatrices
8 mults of (n/2)×(n/2) submatrices
4 adds of (n/2)×(n/2) submatrices s submatrices
Divide and Conquer:
Large Matrix Multiplication
8 multiplications of (n/2)×(n/2) sub-matrices
4 additions of (n/2)×(n/2) sub-matrices
Recurrence:
T(n) = 8 T(n/2) + 4n2
O(n3) solution
Divide and Conquer:
Strassen’s Matrix Multiplication
Reduce Multiplications
Divide and Conquer:
Strassen’s Matrix Multiplication
Recurrence:
T(n) = 7 T(n/2) + 18n2
T(n) = Θ(n 2.81)
Best to date: Θ(n2.376)
Algorithm Design Technique
Transform and Conquer
Transform and Conquer
A group of design methods based on the idea of transformation
Two stage procedure:
― Transformation: problem instance is modified (transformed)
― Conquer: modified problem instance is solved
Variations:
– Instance Simplification
– Representation Change
– Problem Reduction
Transform and Conquer:
Instance Simplification
Solve a problem’s instance by transforming it into another
simpler/easier instance of the same problem
Presorting
Many problems involving lists are easier when list is sorted.
― Searching
― Computing the median (selection problem)
― Checking if all elements are distinct (element uniqueness)
Instance Simplification:
Searching With Presorting
Problem: Search for a given K in A[0..n-1]
Presorting-based algorithm:
― Stage 1 Sort the array by an efficient sorting algorithm
― Stage 2 Apply binary search
Efficiency: Θ(nlog n) + O(log n) = Θ(nlog n)
Transform and Conquer:
Representation Change
Solve the problem by transforming into a different
representation of the same problem
Example:
Polynomial Evaluation
Transform and Conquer:
Representation Change
Polynomial Evaluation:
Given a polynomial of degree n
p(x) = anxn + an-1xn-1 + … + a1x + a0
and a specific value of x, find the value of p at that point.
Two brute-force algorithms:
p0 p a0; power 1
for i n downto 0 do for i 1 to n do
power 1 power power * x
for j 1 to i do p p + ai * power
power power * x return p
p p + ai * power
return p
Representation Change:
Polynomial Evaluation
Horner’s rule:
Example: p(x) = 2x4 - x3 + 3x2 + x - 5
= x(2x3 - x2 + 3x + 1) - 5
= x(x(2x2 - x + 3) + 1) - 5
= x(x(x(2x - 1) + 3) + 1) - 5
Substitution into the last formula leads to a faster
algorithm
Representation Change:
Polynomial Evaluation
Same sequence of computations are obtained by simply
arranging the coefficient in a table and proceeding as follows:
coefficients 2 -1 3 1 -5
x=3 2 3*2+(-1)=5 3*5+3=18 3*18+1=55 3*55+(-5)=160
Transform and Conquer:
Problem Reduction
Solve a problem by reducing it to another problem whose
solution is known to you.
To be of practical value, the combined time of the
transformation and solving the other problem should be
smaller than solving the problem as given by another method.
Problem Reduction:
Find LCM
Problem Reduction:
Circuit Reduction to Graph
Problem Reduction:
Maps Reduction Graph
END