Advanced algorithms

University of Technology and Engineering

Vietnam National University Hanoi

1
 Outlines

1.Divide and conquer

2.Dynamic programming
3.Greedy algorithms

2
 Divide-and-Conquer

➢ Divide-and conquer is a general ➢ The base case for the

algorithm design paradigm: recursion are subproblems of
❖ Divide: divide the input data S size 0 or 1
in two disjoint subsets S1 and
S2 ➢ Merge-sort is a sorting
❖ Recur: solve the subproblems algorithm based on the divide-
associated with S1 and S2 and-conquer paradigm
❖ Conquer: combine the
solutions for S1 and S2 into a
solution for S

3
 Binary Search Algorithm
Problem: Given a list A sorted increasingly. Your task is to write a
function to check if an item X is in A?
Algorithm:
➢ Step 1:
❖ If A is empty, return False
❖ Else: compare X with Y which is the item at the middle of A.
➢ Step 2:
❖ If X = Y, return True.
❖ If X < Y, search on the left half of A
❖ If X > Y, search on the right half of A
Complexity: O(log n)

4
 Example

1 1 1 1 1
0 1 3 4 5 7 8 9
1 4 6 8 9
l m r
1 1 1 1 1
0 1 3 4 5 7 8 9
1 4 6 8 9
l m r
1 1 1 1 1
0 1 3 4 5 7 8 9
1 4 6 8 9
l m r
1 1 1 1 1
0 1 3 4 5 7 8 9
1 4 6 8 9
l=m =r

5
Merge Sort

7 2|9 4 → 2 4 7 9

7|2 → 2 7 9|4 → 4 9

7→7 2→2 9→9 4→4

6
 Merge-Sort

➢ Merge-sort on an input Algorithm mergeSort(S, C)

sequence S with n Input sequence S with n
elements consists of three
elements, comparator C
steps:
Output sequence S sorted
❖ Divide: partition S into two according to C
sequences S1 and S2 of
about n/2 elements each if S.size() > 1
(S1, S2) ← partition(S,
❖ Recur: recursively sort S1
n/2);
and S2
mergeSort(S1, C);
❖ Conquer: merge S1 and S2
into a unique sorted mergeSort(S2, C);
sequence S ← merge(S1, S2);

7
 Merging Two Sorted Sequences
➢ The conquer step of Algorithm merge(A, B)
merge-sort consists of Input sequences A and B with
merging two sorted n/2 elements each
sequences A and B Output sorted sequence of A ∪ B
into a sorted sequence
S containing the union S ← empty sequence
of the elements of A while ¬A.isEmpty() ∧ ¬B.isEmpty()
and B if A.first().element() <
B.first().element()
➢ Merging two sorted
sequences, each with
S.insertLast(A.remove(A.first()));
n/2 elements and
implemented by means else
of a doubly linked list,
S.insertLast(B.remove(B.first()));
takes O(n) time
while ¬A.isEmpty()
S.insertLast(A.remove(A.first()));
while ¬B.isEmpty() 8
S.insertLast(B.remove(B.first()));
 Execution Example
➢Partition
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6

7 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 2→ 9→ 4→ 3→ 8→ 6→ 1→
7 2 9 4 3 8 6 1
9
 Execution Example (cont.)
➢Recursive call, partition
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 8 6

7 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 2→ 9→ 4→ 3→ 8→ 6→ 1→
7 2 9 4 3 8 6 1
10
 Execution Example (cont.)
➢Recursive call, partition
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 8 6

7⏐2→2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 2→ 9→ 4→ 3→ 8→ 6→ 1→
7 2 9 4 3 8 6 1
11
 Execution Example (cont.)
➢Recursive call, base case
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 8 6

7⏐2→2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 2→ 9→ 4→ 3→ 8→ 6→ 1→
7 2 9 4 3 8 6 1
12
 Execution Example (cont.)
➢Recursive call, base case
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 8 6

7⏐2→2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 9→ 4→ 3→ 8→ 6→ 1→
2→2
7 9 4 3 8 6 1
13
 Execution Example (cont.)
➢Merge
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 8 6

7⏐2→2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 9→ 4→ 3→ 8→ 6→ 1→
2→2
7 9 4 3 8 6 1
14
 Execution Example (cont.)
➢Recursive call, …, base case, merge
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 8 6

7⏐2→2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 9→ 4→ 3→ 8→ 6→ 1→
2→2
7 9 4 3 8 6 1
15
 Execution Example (cont.)
➢Merge
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 8 6

7⏐2→2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 9→ 4→ 3→ 8→ 6→ 1→
2→2
7 9 4 3 8 6 1
16
 Execution Example (cont.)
➢Recursive call, …, merge, merge
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 6 8

7⏐2→2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 9→ 4→ 3→ 8→ 6→ 1→
2→2
7 9 4 3 8 6 1
17
 Execution Example (cont.)
➢Merge
7 2 9 4⏐3 8 6 1 → 1 2 3 4 6 7 8 9

7 2⏐9 4→ 2 4 7 9 3 8 6 1 → 1 3 6 8

7⏐2→2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6

7→ 9→ 4→ 3→ 8→ 6→ 1→
2→2
7 9 4 3 8 6 1
18
 Analysis of Merge-Sort
➢ The height h of the merge-sort tree is O(log n)
❖ at each recursive call we divide in half the sequence,
➢ The overall amount or work done at the nodes of depth i is O(n)
❖ we partition and merge 2i sequences of size n/2i
❖ we make 2i+1 recursive calls
➢ Thus, the total running time of merge-sort is O(n log n)

depth #seqs size

0 1 n

1 2 n/2

i 2i n/2i

… … … 19
Summary of Sorting Algorithms
Expected
Algorithm Notes
time
• slow
selection-sort O(n2)
• for small data sets (< 1K)

• slow
insertion-sort O(n2)
• for small data sets (< 1K)

• fast
heap-sort O(n log n)
• for large data sets (1K — 1M)

• fast
merge-sort O(n log n)
• for huge data sets (> 1M)
• fast and most common in
Quick-sort O(n log n) practice
• for huge data sets (> 1M)
20
 Exercise 1

Your task is to illustrate the execution of the merge-sort algorithm

on following numbers:
3, 13, 89, 34, 21, 44, 99, 56, 9

21