ΑΡΙΣΤΟΤΕΛΕΙΟ ΠΑΝΕΠΙΣΤΗΜΙΟ ΘΕΣΣΑΛΟΝΙΚΗΣ ΠΟΛΥΤΕΧΝΙΚΗ ΣΧΟΛΗ

Ανάλυση και Σχεδιασμός Αλγορίθμων

Divide and Conquer

Παναγιώτης Χ. Πετραντωνάκης
Επίκουρος Καθηγητής
Τμήμα Ηλεκτρολόγων Μηχ. & Μηχ. Υπολογιστών
[email protected]

Τετάρτη, 15 Μαρτίου 2023

Divide-and-conquer strategy

• It is a strategy of algorithm design that:

• Divide into subproblems

• smaller instances of the same problem

• Recursively solving subproblems

• Base case: if the subproblems are small enough you can solve them by brute force

• Combine answers

Divide and Conquer 1

Divide-and-conquer strategy

• Where is the work done:

• Dividing the problems into subproblems

• At the end of the recursion where the subproblems are small and are solved
outright

• Gluing together the intermediate answers

Divide and Conquer 2

Multiplication
• We have seen algorithms achieving 𝑂 𝑛2

• Can we do better with D&C?

Divide and Conquer 3

Multiplication
• We have seen algorithms achieving 𝑂 𝑛2

• Can we do better with D&C?

• What if divide the numbers in to parts?

𝑥 = 𝑥𝐿 + 𝑥𝑅

Divide and Conquer 4

Multiplication
• In particular, assume 𝑥 and 𝑦 two 𝑛bit integers (for the sake of
simplicity 𝑛 is power of 2). Thus,

𝑛
𝑥= 𝑥𝐿 22 + 𝑥𝑅
and
𝑛
𝑦= 𝑦𝐿 22 + 𝑦𝑅

We want to compute the multiplication 𝑥𝑦

Divide and Conquer 5

Multiplication
• We can rewrite the multiplication 𝑥𝑦 as:

𝑛 𝑛
𝑥𝑦 = 𝑥𝐿 22 + 𝑥𝑅 𝑦𝐿 22 + 𝑦𝑅 =

𝑛
= 2𝑛 𝑥𝐿 𝑦𝐿 + 22 𝑥𝐿 𝑦𝑅 + 𝑥𝑅 𝑦𝐿 + 𝑥𝑅 𝑦𝑅

What is the number of computer steps needed 𝑇(𝑛)?

Divide and Conquer 6

Multiplication
• We can rewrite the multiplication 𝑥𝑦 as:

𝑛 𝑛
𝑥𝑦 = 𝑥𝐿 22 + 𝑥𝑅 𝑦𝐿 22 + 𝑦𝑅 =
𝑛
= 2𝑛 𝑥𝐿 𝑦𝐿 + 22 𝑥𝐿 𝑦𝑅 + 𝑥𝑅 𝑦𝐿 + 𝑥𝑅 𝑦𝑅

What is the number of computer steps needed 𝑇(𝑛)?

𝑇 𝑛 = 4𝑇 𝑛ൗ2 + 𝑂(𝑛)
Divide and Conquer 7
 # of steps

Recursive Calls 𝑐𝑛 = 20 𝑐𝑛

𝑛 4
4𝑐 = 𝑐𝑛 = 2𝑐𝑛
2 2

. . . 𝑛 42
16𝑐 = 𝑐𝑛 2 = 4𝑐𝑛
4 2

. . . 𝑛 43
64𝑐 = 𝑐𝑛 3 = 8𝑐𝑛
8 2
. . .

. . .
4𝑘
𝑐𝑛 𝑘 = 2𝑘 𝑐𝑛
2

Divide and Conquer 8

Total steps of D&C multiplication and Big 𝑂
• 𝑇 𝑛 = 𝑐𝑛 1 + 2 + 4 + 8 + ⋯ + 2𝑘 = 𝑐𝑛 1 + 21 + 22 + ⋯ + 2𝜅

Divide and Conquer 9

Total steps of D&C multiplication and Big 𝑂
• 𝑇 𝑛 = 𝑐𝑛 1 + 2 + 4 + 8 + ⋯ + 2𝑘 = 𝑐𝑛 1 + 21 + 22 + ⋯ + 2𝜅

2𝑘+1 −1
• 1 + 21 + 22 + ⋯ + 2𝜅 = = 2log2 𝑛+1 − 1 = 2 ∙ 2log2 𝑛 − 1 =
2−1
2𝑛 − 1

Divide and Conquer 10

Total steps of D&C multiplication and Big 𝑂
• 𝑇 𝑛 = 𝑐𝑛 1 + 2 + 4 + 8 + ⋯ + 2𝑘 = 𝑐𝑛 1 + 21 + 22 + ⋯ + 2𝜅

2𝑘+1 −1
• 1 + 21 + 22 + ⋯ + 2𝜅 = = 2log2 𝑛+1 − 1 = 2 ∙ 2log2 𝑛 − 1 =
2−1
2𝑛 − 1

• 𝑇 𝑛 = 4𝑇 𝑛Τ2 + 𝑂 𝑛 = 𝑂(𝑛2 ) (what??)

Divide and Conquer 11

Multiplication
𝑇 𝑛 = 4𝑇 𝑛ൗ2 + 𝑂(𝑛)

• This particular algorithm works for 𝑂 𝑛2

• Thus, what did we achieve with regard to efficiency?

• At this point nothing, but…

Divide and Conquer 12

Carl Friedrich Gauss (1777-1855)
• Gauss realized that for the multiplication of two complex
numbers: Britannica

𝑎 + 𝑏𝑖 𝑐 + 𝑑𝑖 = 𝑎𝑐 − 𝑏𝑑 + 𝑏𝑐 + 𝑎𝑑 𝑖

Divide and Conquer 13

Carl Friedrich Gauss (1777-1855)
• Gauss realized that for the multiplication of two complex
numbers: Britannica

𝑎 + 𝑏𝑖 𝑐 + 𝑑𝑖 = 𝑎𝑐 − 𝑏𝑑 + 𝑏𝑐 + 𝑎𝑑 𝑖

We do not need 4 multiplications but three as:

𝑏𝑐 + 𝑎𝑑 = 𝑎 + 𝑏 𝑐 + 𝑑 − 𝑎𝑐 − 𝑏𝑑

Divide and Conquer 14

Multiplication (revisited)
• Using the Gauss trick we need three multiplications instead of four

𝑛
𝑥𝑦 = 2𝑛 𝑥𝐿 𝑦𝐿 + 22 𝑥𝐿 𝑦𝑅 + 𝑥𝑅 𝑦𝐿 + 𝑥𝑅 𝑦𝑅 =

𝑛
= 2𝑛 𝑥𝐿 𝑦𝐿 + 22 𝑥𝐿 + 𝑥𝑅 𝑦𝐿 + 𝑦𝑅 − 𝑥𝐿 𝑦𝐿 − 𝑥𝑅 𝑦𝑅 + 𝑥𝑅 𝑦𝑅

Thus:
𝑛
𝑇 𝑛 = 3𝑇 + 𝑂(𝑛)
2
Divide and Conquer 15
 # of steps

Recursive Calls 𝑐𝑛 = 20 𝑐𝑛

𝑛 3
3𝑐 = 𝑐𝑛
2 2

. . . 𝑛 32
9𝑐 = 𝑐𝑛 2
4 2

. . . 𝑛 33
27𝑐 = 𝑐𝑛 3
8 2
. . .

. . .
3𝑘
𝑐𝑛 𝑘
2

Divide and Conquer 16

Total steps of D&C multiplication and Big 𝑂
3 𝑘+1
3 3 2 3 𝑘 3 𝑘 −1
• 𝑇 𝑛 = 𝑐𝑛 1 + + + + ⋯+ = 𝑐𝑛 2
3 =
2 2 2 2 −1
2

𝑘+1 log2 𝑛+1 log2 𝑛

3 3 3 3 3log2 𝑛
= 2𝑐𝑛 − 1 ≤ 2𝑐𝑛 = 2𝑐𝑛 = 3𝑐𝑛
2 2 2 2 𝑛

log2 𝑛 log2 3
= 3𝑐3log2 𝑛 = 3𝑐 2log2 3 = 3𝑐 2log2 𝑛 = 3𝑐𝑛log2 3

Divide and Conquer 17

Total steps of D&C multiplication and Big 𝑂
3 𝑘+1
3 3 2 3 𝑘 3 𝑘 −1
• 𝑇 𝑛 = 𝑐𝑛 1 + + + +⋯+ = 𝑐𝑛 2
3 =
2 2 2 2 −1
2

𝑘+1 log2 𝑛+1 log2 𝑛

3 3 3 3 3log2 𝑛
= 2𝑐𝑛 − 1 ≤ 2𝑐𝑛 = 2𝑐𝑛 = 3𝑐𝑛
2 2 2 2 𝑛

log2 𝑛 log2 3
= 3𝑐3log2 𝑛 = 3𝑐 2log2 3
= 3𝑐 2log2 𝑛
= 3𝑐𝑛log2 3

• 𝑇 𝑛 = 3𝑇 𝑛Τ2 + 𝑂 𝑛 = 𝑂 𝑛log2 3 = 𝑂(𝑛1.59 ) (wow!)

Divide and Conquer 18

In general…
𝑛
• For every problem of size 𝑛 that is divided into 𝑎 sub problems of size ,
𝑏
then:
• depth of recursion tree is log 𝑏 𝑛
log𝑏 𝑛 log𝑏 𝑎
• width (i.e., # leaves) is 𝑎log𝑏 𝑛 = 𝑏 log𝑏 𝑎 = 𝑏 log𝑏 𝑛 = 𝑛log𝑏 𝑎

𝑛
• Theorem: if 𝑇 𝑛 = 𝑎𝑇 + 𝑂 𝑛𝑑 , for 𝑎 > 0, 𝑏 > 1, 𝑑 ≥ 0 then:
𝑏

𝑂 𝑛𝑑 𝑖𝑓 𝑑 > log 𝑏 𝑎
𝑇 𝑛 = 𝑂 𝑛𝑑 log 𝑏 𝑛 𝑖𝑓𝑑 = log 𝑏 𝑎
𝑂 𝑛log𝑏 𝑎 𝑖𝑓 𝑑 < log 𝑏 𝑎

Divide and Conquer 19

Binary search
• Definition of the problem: Find a key 𝑘 in a large file containing keys
𝑧[0, 1, … , 𝑛 − 1] in sorted order.

𝑛
• Algorithm: Comprare 𝑘 with Depending on the
𝑧[ ].
2
result recurse either on the first or the second
half of 𝑧.

• 𝑎 =? , 𝑏 =? , 𝑑 =?

• Running time=?

Divide and Conquer 20

Sorting
• Problem: Sort a list o numbers in ascending order.

• How can we implement an algorithm using D&C concept?

• What would be the 𝑎, 𝑏, 𝑎𝑛𝑑 𝑑 parameters

9 1 5 4 7 11 2 6

Divide and Conquer 21

Insertion sort

CLRS

Divide and Conquer 22

Insertion sort

CLRS

Divide and Conquer 23

Insertion sort

• Thus the running time of Insertion-Sort:

𝑇 𝑛 = 𝑛 𝑛 𝑛

= 𝑐1 𝑛 + 𝑐2 𝑛 − 1 + 𝑐4 𝑛 − 1 + 𝑐5 ෍ 𝑡𝑗 + 𝑐6 ෍(𝑡𝑗 − 1) + 𝑐7 ෍(𝑡𝑗 − 1)
𝑗=2 𝑗=2 𝑗=2
+ 𝑐8 (𝑛 − 1)

Divide and Conquer 24

Insertion sort

• Best case: the array is already sorted!

• All 𝑡𝑗 = 1 (while condition is checked only once)

• Then:

𝑇 𝑛 = 𝑐1 𝑛 + 𝑐2 𝑛 − 1 + 𝑐4 𝑛 − 1 + 𝑐5 𝑛 − 1 + 𝑐8 𝑛 − 1 = 𝑂(𝑛)

Divide and Conquer 25

Insertion sort

• Worst case: the array is reverse sorted order!

• All 𝑡𝑗 = 𝑗 (have to compare key with all elements on its left, 𝑗 − 1 elements)

𝑛 𝑛+1
• We know: σ𝑛𝑗=2 𝑡𝑗 = σ𝑛𝑗=2 𝑗 = σ𝑛𝑗=1 𝑗 − 1 = −1
2

• Thus:

𝑇 𝑛 = 𝑂(𝑛2 )

Divide and Conquer 26

Merge sort

• Divide: Divide the 𝑛-element sequence to be sorted into two

subsequences of 𝑛/2 elements each.

• Conquer: Sort the two subsequences recursively using merge sort.

• Combine: Merge the two sorted subsequences to produce the sorted

answer.

Divide and Conquer 27

Merge sort

𝑛
• 𝑇 𝑛 = 𝑎𝑇 + 𝑂(𝑛𝑑 )
𝑏

Divide and Conquer 28

Merge sort

𝑛
• 𝑇 𝑛 = 𝑎𝑇 + 𝑂(𝑛𝑑 )
𝑏

• 𝑎 = 2, 𝑏 = 2

• What is 𝑑? We need to define merging procedure.

Divide and Conquer 29

Merge sort algorithm

Dasgupta et al. (Algorithms)

Divide and Conquer 30

Merge

Dasgupta et al. (Algorithms)

Divide and Conquer 31

Merge sort

𝑛
• 𝑇 𝑛 = 𝑎𝑇 + 𝑂(𝑛𝑑 )
𝑏

• 𝑎 = 2, 𝑏 = 2, 𝑑 = 1.

𝑂 𝑛𝑑 𝑖𝑓 𝑑 > log 𝑏 𝑎
• 𝑇 𝑛 = 𝑂 𝑛𝑑 log 𝑏 𝑛 𝑖𝑓𝑑 = log 𝑏 𝑎
𝑂 𝑛log𝑏 𝑎 𝑖𝑓 𝑑 < log 𝑏 𝑎

• 𝑇 𝑛 = 𝑛 log 𝑛

Divide and Conquer 32

 # of steps

Recursive Calls 𝑐𝑛

𝑛
2𝑐 = 𝑐𝑛
2

𝑛
4𝑐 = 𝑐𝑛
4

. . . 8𝑐
𝑛
8
= 𝑐𝑛
. . .

. . .
𝑛
. . . log 𝑛 𝑐
log 𝑛
= 𝑐𝑛

Divide and Conquer 33

Examples

CLRS

Divide and Conquer 34

Can we do better?

Dasgupta et al. (Algorithms)

Divide and Conquer 35

Can we do better?

Dasgupta et al. (Algorithms)

• The depth of the tree is the worst case complexity of the algorithm

• A binary tree with depth 𝑑 has at most 2𝑑 leaves.

• What is the maximum number of leaves of the above defined tree?

Divide and Conquer 36

Can we do better?

Dasgupta et al. (Algorithms)

• The depth of the tree is the worst case complexity of the algorithm

• A binary tree with depth 𝑑 has at most 2𝑑 leaves.

• What is the maximum number of leaves of the above defined tree?

• 𝑛!

Divide and Conquer 37

Can we do better?

Dasgupta et al. (Algorithms)

• Thus

𝑛
𝑑
𝑛 2 𝑛 𝑛
2 ≥ 𝑛! → 𝑑 ≥ log 𝑛! ≥ log = log
2 2 2

Running time: 𝑇 𝑛 = 𝑑 = Ω(𝑛𝑙𝑜𝑔𝑛)

Divide and Conquer 38

ΑΡΙΣΤΟΤΕΛΕΙΟ ΠΑΝΕΠΙΣΤΗΜΙΟ ΘΕΣΣΑΛΟΝΙΚΗΣ ΠΟΛΥΤΕΧΝΙΚΗ ΣΧΟΛΗ

Ερωτήσεις;
Ανάλυση και Σχεδιασμός Αλγορίθμων
Divide and Conquer

Τετάρτη, 15 Μαρτίου 2023