CECS 419/619 COMPUTER ALGORITHMS Fall 2016

Homework Set 2
Student Name: Khaled Gamal Abdel Maksoud
Student ID: 1280703

Problem 1 (30 points)

Give asymptotic upper and lower bounds for T(n) in each of the following recurrences. Assume that T(n) is constant
for sufficiently small n. Make your bounds as tight as possible, and justify your answers.
a) 𝑇(𝑛) = 4𝑇(𝑛/3) + 𝑛 lg 𝑛
Solution
Using the master method and comparing the given recurrence with the general form T(n) = a T(n/b) + f(n), we
get:
a = 4, b = 3 and f(n) = 𝑛 lg𝑛 (f(n) > 0 for all n ≥ 2)
𝑛log𝑏 𝑎 = 𝑛log3 4 = 𝑛d (where 2 > d > 1)
𝑛 lg 𝑛 = O(𝑛 ) for some > 0 such that d - > 1 (lg 𝑛 is asymptotically less than 𝑛k for any k > 0 )
It is clear now that we can apply Case 1 of the master method.
Therefore, T(n) = Θ(𝑛log3 4)

b) 𝑇(𝑛) = 4𝑇(𝑛/2) + √𝑛
Solution
Comparing the given recurrence with the general form T(n) = a T(n/b) + f(n), we get:
a = 4, b = 2 and f(n) = √𝑛 (f(n) > 0 for all n ≥ 1)
𝑛log𝑏 𝑎 = 𝑛log2 4 =
√𝑛 = =Ω(𝑛 ) for some >0
√𝑛 = Ω(𝑛 ) (we selected = 0.5)
Now, we need to prove that a f(n/b) ≤ C f(n) for some positive constants C and n0 where C < 1 and n ≥ n0
4 𝑛 √𝑛 ≤C √𝑛
 1/√ ≤ C

Page 1 of 18
 C = 0.71 and n ≥ 1
It is clear now that we can apply Case 3 of the master method.
Therefore, T(n) = Θ( √𝑛 )

c) 𝑇(𝑛) = 𝑇(𝑛/2) + 𝑇(𝑛/4) + 𝑇(𝑛/8) + 𝑛

Solution
The construction of a recursion tree corresponding to the given recurrence along with its progressive expansions
are shown in figures a, b, c, and d.

T(n) n

T(n/2) T(n/4) T(n/8)

(a) (b)

n/2 n/4 n/8

T(n/4) T(n/8) T(n/16) T(n/8) T(n/16) T(n/32) T(n/16) T(n/32) T(n/64)

(c)

Page 2 of 18
 n n

n/2 n/4 n/8 𝑛

n/4 n/8 n/16 n/8 n/16 n/32 n/16 n/32 n/64 𝑛

lg n

T(1)

T(1)

T(1)

T(1)

T(1)

T(1)

T(1)

T(1)

T(1) C (constant > 0)

(d)

On the ith level of the recursion tree, the maximum problem size is n/2i .
When we reach the leaves level, we have n/2i = 1
Accordingly, i = lg n (i.e. the depth of the recursion tree is lg n)
T(n) = + (7/8)n (7/8)2n … C [where C is a constant > 0]

T(n) ≤ [∑ ]+C [since not all branches of the recursion tree will reach level lg n]

≤ [∑ ]+C

=[ ]+C [since ∑ = 1/1-x]

C (constant > 0)
=8 C
≤8 Cn [for all n ≥ 1]
Page 3 of 18
 = (8 + C)
=D (where D = 8 + C)
Accordingly, we have T(n) ≤ D where D is a constant > 0 and n ≥ 1.
Therefore, T(n) = O( ).
To obtain an asymptotic lower bound for T(n), we will consider only the running time represented by the
leftmost branch at each level of the recursion tree shown in figure d.
T(n) ≥ + (1/2)n (1/2)2 … C [where C is a constant > 0]

= [∑ ]+C

= ] + C [since ∑ 𝑟 where r is the common ratio of the series]

=2 C
= C
= C
≥ [since C > 0]
=
≥n or n ≥ 2
Accordingly, we have T(n) ≥ K where K = 1 > 0 and n ≥ 2.
Therefore, T(n) = Ω( ).
We can use the substitution method to verify our upper and lower bounds obtained above.
Assume T(k) ≤ C k for k < n and C is a constant > 0.
T(n) = T(n/2) + T(n/4) + T(n/8) + n
≤ C (n/2) + C (n/4) + C (n/8) + n
= C 7/8 n + n
= C n – (1/8 C n – n) (desired - residual)
≤ C n whenever (1/8 C n – n)
Hence, T(n) ≤ C n for C and n ≥ 1
Therefore, T(n) = O( ).
Similarly, assume T(m) ≥ D m for m < n and D is a constant > 0.
T(n) = T(n/2) + T(n/4) + T(n/8) + n
≥ D (n/2) + D (n/4) + D (n/8) + n
= D 7/8 n + n

Page 4 of 18
 = D n + (n - 1/8 D n) (desired + residual)
≥ D n whenever (n - 1/8 D n)
Hence, T(n) ≥ D n for and n ≥ 1
Therefore, T(n) = Ω( ).

Page 5 of 18
Problem 2 (20 points)
Using the partitioning Example in Slides 4-7 of Lecture 6 as a model, illustrate the operation of PARTITION on
the array A=<13, 19, 9, 5, 12, 8, 7, 4, 21, 2, 6, 11>. Pick first element as the pivot.
Solution
The partition function can be written as follows:

PARTITION (𝐴, 𝑝, 𝑞)
𝑥 = A[𝑝] //pivot = 1st element of array
𝑖=𝑝
for 𝑗 = 𝑝 + 1 to 𝑞
if 𝐴[𝑗] ≤ 𝑥
𝑖=𝑖+1
exchange 𝐴[𝑖] with 𝐴[𝑗]
exchange 𝐴[𝑝] with 𝐴[𝑖]
return 𝑖 //return position of pivot element

In our case, we have p = 1 and q = 12.

We will start with:
Pivot x = A[1] = 13
i=1
j=2
Our input array is:
1 2 3 4 5 6 7 8 9 10 11 12

13 19 9 5 12 8 7 4 21 2 6 11

i j

Keep moving j until element is ≤ pivot

1 2 3 4 5 6 7 8 9 10 11 12

13 19 9 5 12 8 7 4 21 2 6 11

i j

Swap 19 & 9

Page 6 of 18
1 2 3 4 5 6 7 8 9 10 11 12

13 9 19 5 12 8 7 4 21 2 6 11

i j

Keep moving j until element is ≤ pivot

1 2 3 4 5 6 7 8 9 10 11 12

13 9 19 5 12 8 7 4 21 2 6 11

i j

Swap 19 & 5

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 19 12 8 7 4 21 2 6 11

i j

Keep moving j until element is ≤ pivot

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 19 12 8 7 4 21 2 6 11

i j

Swap 19 & 12

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 19 8 7 4 21 2 6 11

i j

Keep moving j until element is ≤ pivot

Page 7 of 18
1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 19 8 7 4 21 2 6 11

i j

Swap 19 & 8

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 19 7 4 21 2 6 11

i j

Keep moving j until element is ≤ pivot

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 19 7 4 21 2 6 11

i j

Swap 19 & 7

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 19 4 21 2 6 11

i j

Keep moving j until element is ≤ pivot

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 19 4 21 2 6 11

i j

Swap 19 & 4

Page 8 of 18
1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 4 19 21 2 6 11

i j

Keep moving j until element is ≤ pivot

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 4 19 21 2 6 11

i j

Swap 19 & 2

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 4 2 21 19 6 11

i j

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 4 2 21 19 6 11

i j

Swap 21 & 6

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 4 2 6 19 21 11

i j

Page 9 of 18
1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 4 2 6 19 21 11

i j

Swap 19 & 11

1 2 3 4 5 6 7 8 9 10 11 12

13 9 5 12 8 7 4 2 6 11 21 19

i j
Loop terminates
Swap 13 & 11

1 2 3 4 5 6 7 8 9 10 11 12

11 9 5 12 8 7 4 2 6 13 21 19

≤ 13 > 13

Page 10 of 18
Problem 3 (30 points)
In the algorithm SELECT (Slide 29 of lecture 9), the input elements are divided into groups of 5. Will the
algorithm work in linear time if they are divided into groups of 7? Argue that SELECT does not run in linear
time if groups of 3 are used.
Solution
Scenario No 1 (groups of 7)
The SELECT (i , n) function will execute the following steps:
1. Divide the n elements into groups of 7. Find the median of each 7-element group.
2. Recursively SELECT the median x of the ⌊n/ ⌋ group medians to be the pivot.
3. Partition around the pivot x. Let k = rank(x).
4. If i = k then return x
elseif i < k then
Recursively SELECT the ith smallest element in the lower part
else
Recursively SELECT the (i-k)th smallest element in the upper part
At least half group medians are ≤ x
 At least ⌊⌊n/7⌋/ ⌋ elements ≤ x
 At least ⌊n/ 4⌋ elements ≤ x
 At least 4⌊n/ 4⌋ elements ≤ x (since we have 4 elements in each group ≤ the group median)
 At least ⌊ ⌋ elements ≤ x
The recursive call to SELECT in Step 4 written above is executed recursively on ≤ 5n/7 elements.
Thus, the recurrence for running time can assume that Step 4 takes time T(5n/7) in the worst case.
Referring to the four steps written above, we have:
Running time of Step 1 = Θ
Running time of Step 2 = T(n/7)
Running time of Step 3 = Θ
Running time of Step 4 = T(5n/7)
Accordingly, the total running time T(n) can be expressed by the following recurrence:
T(n) = T(n/7) + T(5n/7) + Θ
We will try to prove that T(n) Cn for some constant C > 0 and n ≥ n0
Using the substitution method, we have:
T(n) Cn/7 + 5Cn/7 + Θ
Page 11 of 18
 T(n) = 6Cn/7 + Θ
= Cn - (Cn/7- Θ ) (desired - residual)
Cn when (Cn/7- Θ ) 0
Let Θ = Dn for some D > 0
 T(n) Cn when (Cn/7- Dn) 0
 Cn/7 Dn
C 7D > 0 for all n 1
Therefore, T(n) = O(n) (A)
Now, we will try to prove that T(n) Kn for some constant K > 0 and n ≥ n0
Using the substitution method, we have:
T(n) Kn/7 + 5Kn/7 + Θ
= 6Kn/7 + Θ
= Kn + (Θ n) - Kn/7)
Kn when (Θ n) - Kn/7) 0
Let Θ = Dn for some D > 0
 T(n) Kn when (Dn - Kn/7) 0
 Kn/7 Dn
0<K 7D for all n 1
Therefore, T(n) = Ω (B)
Using relations A and B, we get T(n) = Θ
Accordingly, the algorithm will work in linear time if the input elements are divided into groups of 7.
Scenario No 2 (groups of 3)
The SELECT (i , n) function will execute the following steps:
1. Divide the n elements into groups of 3. Find the median of each 3-element group.
2. Recursively SELECT the median x of the ⌊n/3⌋ group medians to be the pivot.
3. Partition around the pivot x. Let k = rank(x).
4. If i = k then return x
elseif i < k then
Recursively SELECT the ith smallest element in the lower part
else
Recursively SELECT the (i-k)th smallest element in the upper part
At least half group medians are ≤ x
Page 12 of 18
 At least ⌊⌊n/3⌋ ⌋ elements ≤ x
 At least ⌊n/6⌋ elements ≤ x
 At least 2⌊n/6⌋ elements ≤ x (since we have 2 elements in each group ≤ the group median)
 At least ⌊n/3⌋ elements ≤ x
The recursive call to SELECT in Step 4 written above is executed recursively on ≤ 2n/3 elements.
Thus, the recurrence for running time can assume that Step 4 takes time T(2n/3) in the worst case.
Referring to the four steps written above, we have:
Running time of Step 1 = Θ
Running time of Step 2 = T(n/3)
Running time of Step 3 = Θ
Running time of Step 4 = T(2n/3)
Accordingly, the total running time T(n) can be expressed by the following recurrence:
T(n) = T(n/3) + T(2n/3) + Θ
We will try to prove that T(n) Cn for some constant C > 0 and n ≥ n0
Using the substitution method, we have:
T(n) Cn/3 + 2Cn/3 + Θ
 T(n) Cn + Θ
Cn when Θ 0 which is impossible
 T(n) ≠ O(n)
Therefore, T(n) ≠ Θ
Accordingly, the algorithm does not run in linear time if groups of 3 are used.

Page 13 of 18
Problem 4 (20 points)
Using Figure 8.2 as a model, illustrate the operation of COUNTING-SORT on the array A=<6, 0, 2, 0, 1, 3, 4,
6, 1, 3, 2>
Solution
We are given an input array A[1 . . 11] of length 11. We need two more arrays, the array B[1 . . 11] holds the
sorted output and the array C[0 . . 6] provides temporary working storage.
The Counting Sort algorithm can be written as follows:
for i = 0 to 6
do c[i] = 0
for j = 1 to 11
do C[A[j]] = C[A[j]] + 1
//C[i] now contains the number of elements equal to i
for i = 1 to 6
do C[i] = C[i] + C[i-1]
//C[i] now contains the number of elements less than or equal to i
for j = 11 downto 1
do B[C[A[j]]] = A[j]
C[A[j]] = C[A[j]] - 1

Our input array is:

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

We will start with the execution of the following part:

for i = 0 to 6
do c[i] = 0
The result is:

0 1 2 3 4 5 6

C: 0 0 0 0 0 0 0

Then, we will execute the following part:

for j = 1 to 11
do C[A[j]] = C[A[j]] + 1
The result is:

0 1 2 3 4 5 6

C: 2 2 2 2 1 0 2

Page 14 of 18
Then, we will execute the following part:
for i = 1 to 6
do C[i] = C[i] + C[i-1]
The result is:

0 1 2 3 4 5 6

C: 2 4 6 8 9 9 11

Finally, we will execute the following part:

for j = 11 downto 1
do B[C[A[j]]] = A[j]
C[A[j]] = C[A[j]] - 1
The result is:

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 2

0 1 2 3 4 5 6

C: 2 4 5 8 9 9 11

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 2 3

0 1 2 3 4 5 6

C: 2 4 5 7 9 9 11

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 1 2 3
Page 15 of 18
 0 1 2 3 4 5 6

C: 2 3 5 7 9 9 11

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 1 2 3 6

0 1 2 3 4 5 6

C: 2 3 5 7 9 9 10

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 1 2 3 4 6

0 1 2 3 4 5 6

C: 2 3 5 7 8 9 10

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 1 2 3 3 4 6

0 1 2 3 4 5 6

C: 2 3 5 6 8 9 10

Page 16 of 18
 1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 1 1 2 3 3 4 6

0 1 2 3 4 5 6

C: 2 2 5 6 8 9 10

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 0 1 1 2 3 3 4 6

0 1 2 3 4 5 6

C: 1 2 5 6 8 9 10

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 0 1 1 2 2 3 3 4 6

0 1 2 3 4 5 6

C: 1 2 4 6 8 9 10

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

Page 17 of 18
 1 2 3 4 5 6 7 8 9 10 11

B: 0 0 1 1 2 2 3 3 4 6

0 1 2 3 4 5 6

C: 0 2 4 6 8 9 10

1 2 3 4 5 6 7 8 9 10 11

A: 6 0 2 0 1 3 4 6 1 3 2

1 2 3 4 5 6 7 8 9 10 11

B: 0 0 1 1 2 2 3 3 4 6 6

0 1 2 3 4 5 6

C: 0 2 4 6 8 9 9

Page 18 of 18