Handout 36: Final Exam Solutions 2

Problem 1. Recurrences [15 points] (3 parts)

Give a tight asymptotic upper bound (O notation) on the solution to each of the following recur-
rences. You need not justify your answers.

√
(a) T (n) = 2T (n/8) + 3
n.

Solution: Θ(n1/3 lg n) by Case 2 of the Master Method.

(b) T (n) = T (n/3) + T (n/4) + 5n

Solution: Θ(n).

8T (n/2) + Θ(1) if n2 > M ,

%
(c) T (n) =
M if n2 ≤ M ;
where M is a variable independent from n.

√ " √
Solution: Θ n3 / M . The recursion tree has approximately lg n − lg M =
!
! √ "
lg n/ M levels. In the tree, every internal node has 8 children, each with cost
√ ! √ "3
O(1). At the bottom of the tree, there are approximately 8lg(n/ M ) = n/ M
leaves. Since#each leaf costs$ M , and the total cost is dominated by the leaves, the
! √ "3 √ "
solution is Θ M n/ M = Θ n3 / M .
!
Handout 36: Final Exam Solutions 3

Problem 2. Algorithms and running times [9 points]

Match each algorithm below with the tightest asymptotic upper bound for its worst-case running
time by inserting one of the letters A, B, . . ., I into the corresponding box. For sorting algorithms,
n is the number of input elements. For matrix algorithms, the input matrix has size n × n. For
graph algorithms, the number of vertices is n, and the number of edges is Θ(n).
You need not justify your answers. Some running times may be used multiple times or not at all.
Because points will be deducted for wrong answers, do not guess unless you are reasonably sure.

Insertion sort A: O(lg n)

Heapsort B: O(n)

B UILD -H EAP C: O(n lg n)

Strassen’s D: O(n2 )

Bellman-Ford E: O(n2 lg n)

Depth-first search F: O(n2.5)

Floyd-Warshall G: O(nlg 7 )

Johnson’s H: O(n3 )

Prim’s I: O(n3 lg n)

Solution: From top to bottom: D, C, B, G, D, B, H, E, C.

Handout 36: Final Exam Solutions 4

Problem 3. Substitution method [10 points]

Use the substitution method to prove a tight asymptotic lower bound (Ω-notation) on the solution
to the recurrence

T (n) = 4T (n/2) + n2 .

Solution: By the master method, we know T (n) = Θ(n2 lg n). Therefore, our induction hypoth-
esis is T (m) ≥ cm2 lg m for all m < n.
For m = 1, we have the base case that T (1) = 1 > c12 lg 1 for all c > 0.
For the inductive step, assume for all m < n, T (m) ≥ cm2 lg m. The induction hypothesis yields

T (n) = 4T (n/2) + n2
# $2 # $
n n
≥ 4c lg + n2
2 2
= cn2 lg n − cn2 lg 2 + n2
= cn2 lg n + (1 − c)n2 .

For c < 1, this quantity is always greater than cn2 lg n. Therefore, T (n) = Ω(cn2 lg n).
Handout 36: Final Exam Solutions 5

Problem 4. True or False, and Justify [35 points] (7 parts)

Circle T or F for each of the following statements to indicate whether the statement is true or
false, respectively. If the statement is correct, briefly state why. If the statement is wrong, explain
why. The more content you provide in your justification, the higher your grade, but be brief. Your
justification is worth more points than your true-or-false designation.

T F Let A1 , A2 , and A3 be three sorted arrays of n real numbers (all distinct). In the comparison
model, constructing a balanced binary search tree of the set A1 ∪A2 ∪A3 requires Ω(n lg n)
time.

Solution: False. First, merge the three arrays, A1 , A2 , and A3 in O(n) time. Second,
construct a balanced binary search tree from the merged array: the median of the array
is the root; recursively build the left subtree from the first half of the array and the right
subtree from the second half of the array. The resulting running time is T (n) = 2T (n/2)+
O(1) = O(n).

T F Let T be a complete binary tree with n nodes. Finding a path from the root of T to a given
vertex v ∈ T using breadth-first search takes O(lg n) time.

Solution: False. Breadth-first search requires Ω(n) time. Breadth-first search examines
each node in the tree in breadth-first order. The vertex v could well be the last vertex
explored. (Also, notice that T is not necessarily sorted.)
Handout 36: Final Exam Solutions 6

T F Given an unsorted array A[1 . . n] of n integers, building a max-heap out of the elements of
A can be performed asymptotically faster than building a red-black tree out of the elements
of A.

Solution: True. Building a heap takes O(n) time, as described in CLRS. On the other
hand, building a red-black tree takes Ω(n lg n) time, since it is possible to produce a sorted
list of elements from a red-black tree in O(n) time by doing an in-order tree walk (and
sorting requires Ω(n lg n) time in a comparison model).

T F Suppose we use a hash function h to hash n distinct keys into an array T of length m.
Assuming simple uniform hashing, the expected number of colliding pairs of elements is
Θ(n2 /m).

Solution: True. Let Xi,j be an indicator random variable equal to 1 if elements i and j
collide, and equal to 0 otherwise. Simple uniform hashing means that the probability of
element i hashing to slot k is 1/m. Therefore, the probability that i and j both hash to the
same slot Pr(Xi,j ) = 1/m. Hence, E [Xi,j ] = 1/m. We now use linearity of expectation
to sum over all possible pairs i and j:
⎡ ⎤
n )
n
E [number of colliding pairs] =
)
E⎣ Xi,j ⎦
i=1 j=i+1
n
) )n
= E [Xi,j ]
i=1 j=i+1
)n ) n
= 1/m
i=1 j=i+1
n(n + 1)
=
2m
= Θ(n2 /m)
Handout 36: Final Exam Solutions 7

T F Every sorting network with n inputs has depth Ω(lg n).

Solution: True. Let d be the depth of the network. Since there are n inputs to the
network, there can be at most nd comparators. (One way of seeing this is by the pigeon-
hole principle: if there are n wires and more than nd comparators, then some wire must
traverse more than d comparators, resulting in a depth > d.)
In the comparison-based model, it is possible to simulate the sorting network one compara-
tor at a time. The running time is equal to the number of comparators (times a constant
factor overhead). Therefore, every sorting network must have at least Ω(n lg n) compara-
tors, by the lower-bound for sorting in the comparison-based model.
Therefore, nd > Ω(n lg n), implying that the depth d is Ω(lg n).

T F If a dynamic-programming problem satisfies the optimal-substructure property, then a lo-

cally optimal solution is globally optimal.

Solution: False. The property which implies that locally optimal solutions are globally
optimal is the greedy-choice property. Consider as a counterexample the edit distance
problem. This problem has optimal substructure, as was shown on the problem set, how-
ever a locally optimal solution—making the best edit next—does not result in a globally
optimal solution.
Handout 36: Final Exam Solutions 8

T F Let G = (V, E) be a directed graph with negative-weight edges, but no negative-weight

cycles. Then, one can compute all shortest paths from a source s ∈ V to all v ∈ V faster
than Bellman-Ford using the technique of reweighting.

Solution: False. The technique of reweighting preserves the shortest path by assigning
a value h(v) to each vertex v ∈ V , and using this to calculate new weights for the edges:
w̃(u, v) = w(u, v) + h(u) − h(v). However, to determine values for h(v) such that the
edge weights are all non-negative, we use Bellman-Ford to solve the resulting system
of difference constraints. Since the technique of reweighting relies on Bellman-Ford, it
cannot run faster than Bellman-Ford.
Handout 36: Final Exam Solutions 9

Problem 5. Red-black trees [15 points] (3 parts)

(a) Assign the keys 2, 3, 5, 7, 11, 13, 17, 19 to the nodes of the binary search tree below so
that they satisfy the binary-search-tree property.

nil nil

nil nil

nil nil nil

nil nil

Solution: 5 points for the correct answer:

Handout 36: Final Exam Solutions 10

(b) Explain why this binary search tree cannot be colored to form a legal red-black tree.

Solution: We prove this by contradiction. Suppose that a valid coloring exists. In a

red-black tree, all paths from a node to descendant leaves contain the same number of
black nodes. The path (17, 19, NIL) can contain at most three black nodes. Therefore
the path (17, 11, 3, 5, 7, NIL) must also contain at most three black nodes and at least
three red nodes. By the red-black tree properties, the root 17 must be black and the
NIL node must also be black. This means that there must be three red nodes in the
path (11, 3, 5, 7), but this would mean there are two consecutive red nodes, which
violates the red-black tree properties. This is a contradiction, therefore the tree cannot
be colored to form a legal red-black tree.
5 points for correct answer. 2-4 points for stating the red-black tree rules, but not
proving why the tree does not satisfy the rules. 1 point for stating that the tree is
unbalanced.
Handout 36: Final Exam Solutions 11

(c) The binary search tree can be transformed into a red-black tree by performing a single
rotation. Draw the red-black tree that results, labeling each node with “red” or “black.”
Include the keys from part (a).

Solution: Rotate right around the root. There are several valid ways to color the
resulting tree. Here is one possible answer (all nodes are colored black except for 7).

5 points for the correct answer. 2 points for the correct rotation but incorrect coloring.
Handout 36: Final Exam Solutions 12

Problem 6. Wiggly arrays [10 points]

An array A[1 . . 2n + 1] is wiggly if A[1] ≤ A[2] ≥ A[3] ≤ A[4] ≥ . . . ≤ A[2n] ≥ A[2n + 1].
Given an unsorted array B[1 . . 2n + 1] of real numbers, describe an efficient algorithm that outputs
a permutation A[1 . . 2n + 1] of B such that A is a wiggly array.

Solution: There are several ways to solve this problem in Θ(n) time. You can find the median k
of B using the deterministic Θ(n) select algorithm and partition B around the median k into two
equal sized sets Blow and Bhigh . Assign A[1] ← k. Then for each i > 1, if i is even, assign an
element from Bhigh to A[i], otherwise assign an element from Blow to A[i]. Since all elements in
Bhigh are greater than or equal to all elements in Blow and the median is less than or equal to all
elements in Bhigh , the array is wiggly. The overall running time is Θ(n).
10 points for correct Θ(n) solution and correct analysis. 8-9 points for correct solution but missing
a minor step in the analysis.
5 points for correct Θ(n lg n) solution and correct analysis. 2-4 points for correct solution but
incomplete analysis.
Handout 36: Final Exam Solutions 13

Problem 7. Difference constraints [12 points] (2 parts)

Consider the following linear-programming system of difference constraints (note that one con-
straint is an equality):

x1 − x4 ≤ −1
x1 − x5 ≤ −4
x2 − x1 ≤ −4
x2 − x3 = −9
x3 − x1 ≤ 5
x3 − x5 ≤ 2
x4 − x3 ≤ −3
x5 − x1 ≤ 5
x5 − x4 ≤ 1

(a) Draw the constraint graph for these constraints.

Solution: The equality constraint can be written as two inequality constraints, x2 −

x3 ≤ −9, and x3 − x2 ≤ 9. A completely correct constraint graph received 6 points.
Solutions that had edges in the wrong direction received only 4 points, and solutions
that did not handle the equality constraint received 3 points.
(b) Solve for the unknowns x1 , x2 , x3 , x4 , and x5 , or explain why no solution exists.

Solution: No solution to this system exists because the constraint graph has a negative-
weight cycle. For example, x1 ❀ x3 ❀ x4 ❀ x5 ❀ x1 has weight 5−3+1−4 = −1.
A full-credit solution (6 points) must exhibit the negative weight cycle.
Handout 36: Final Exam Solutions 14

Problem 8. Amortized increment [12 points]

k−1
An array A[0 . . k − 1] of bits (each array element is 0 or 1) stores a binary number x = A[i] · 2i .
)

i=0
To add 1 (modulo 2k ) to x, we use the following procedure:

I NCREMENT (A, k)
1 i←0
2 while i < k and A[i] = 1
3 do A[i] ← 0
4 i← i+1
5 if i < k
6 then A[i] ← 1

Given a number x, define the potential Φ(x) of x to be the number of 1’s in the binary representation
of x. For example, Φ(19) = 3, because 19 = 100112. Use a potential-function argument to prove
that the amortized cost of an increment is O(1), where the initial value in the counter is x = 0.

Solution: Φ(x) is a valid potential function the number of 1’s in the binary representation of x is
always nonnegative, i.e., Φ(x) ≥ 0 for all x. Since the initial value of the counter is 0, Φ 0 = 0.
Let ck be the real cost of the operation I NCREMENT (A, k), and let dk be the number of times the
while loop body in Lines 3 and 4 execute (i.e., dk is the number of consecutive 1’s counting from
the least-significant bit of the binary representation of x). If we assume that executing all of Lines
1, 2, 5, and 6 require unit cost, and executing the body of the while loop requires unit cost, then
the real cost is ck = 1 + dk ,
The potential decreases by one every time the while loop is executed. Therefore, the change in
potential, ∆Φ = Φk − Φk−1 is at most 1 − dk . More specifically, ∆Φ = 1 − dk if we execute Line
6 , and ∆Φ = −dk if we do not.
Thus, using the formula for amortized cost, we get

cˆk = ck + ∆Φ
= 1 + dk + ∆Φ
≤ 1 + dk + 1 − dk
= 2.

Therefore, the amortized cost of an increment is O(1).

Only solutions that explicitly calculate/explain the real cost ck of a single increment received full
credit. Solutions that give an aggregate analysis or any other method that did not use the potential
function received at most 6 points. One or two points were deducted from correct solutions that
did not explain why the potential function is valid or calculate Φ0 .
Handout 36: Final Exam Solutions 15

Problem 9. Minimum spanning trees [12 points]

Let G = (V, E) be a connected, undirected graph with edge-weight function w : E → , and
assume all edge weights are distinct. Consider a cycle ⟨v1 , v2 , . . . , vk , vk+1⟩ in G, where vk+1 = v1 ,
and let (vi , vi+1 ) be the edge in the cycle with the largest edge weight. Prove that (vi , vi+1 ) does
not belong to the minimum spanning tree T of G.

Solution: Proof by contradiction. Assume for the sake of contradiction that (v i , vi+1 ) does be-
long to the minimum spanning tree T . Removing (vi , vi+1 ) from T divides T into two connected
components P and Q, where some nodes of the given cycle are in P and some are in Q. For any
cycle, at least two edges must cross this cut, and therefore there is some other edge (v j , vj+1) on
the cycle, such that adding this edge connects P and Q again and creates another spanning tree T ′ .
Since the weight of (vj , vj+1 ) is less than (vi , vi+1 ), the weight of T ′ is less than T and T cannot
be a minimum spanning tree. Contradiction.
Handout 36: Final Exam Solutions 16

Problem 10. Multithreaded scheduling [10 points]

A greedy scheduler runs a multithreaded computation in 260 seconds on 4 processors and in 90
seconds on 32 processors. Is it possible that the computation has work T 1 = 1024 and critical-path
length T∞ = 64? Justify your answer.

Solution: For greedy schedulers, we know that min{T1 /P, T∞ } ≤ TP ≤ T1 /P + T∞ . The

numbers given above satisfy all of the above inequalities for both values of P . Therefore, it is
possible that the work and critical path are correct.
Handout 36: Final Exam Solutions 17

Problem 11. Transposing a matrix [40 points] (4 parts)

Let X be an N × N matrix, where N is an exact power of 2. The following code computes
Y = X T:

T RANS(X, Y, N)
1 for i ← 1 to N
2 do for j ← 1 to N
3 do Y [j, i] ← X[i, j]

Consider the cache-oblivious two-level memory model with a cache of M elements and blocks of
B elements. Assume that both matrices X and Y are stored in row-major order, that is, the linear
order of X in memory is X[1, 1], X[1, 2], . . . , X[1, N], X[2, 1], X[2, 2], . . . , X[2, N], . . . , X[N, 1],
X[N, 2], . . . , X[N, N], and similarly for Y .

(a) Analyze the number MT(N) of memory transfers incurred by T RANS when N ≫ M.

Solution: Since the loop scans through matrix X one row at a time, the number of
memory transfers required for accessing X is O(N 2 /B). We incur O(N 2 ) memory
transfers to access Y , however, because Y is accessed one column at a time. When we
access the block containing Y [j, i], since N ≫ M and we scan through all of column
i before accessing Y [j + 1, i], each access to Y may incur a memory transfer.
Therefore, MT (N) = O(N 2 ).
Handout 36: Final Exam Solutions 18

Now, consider the following divide-and-conquer algorithm for computing the transpose:

R-T RANS(X, Y, N)
1 if N = 1
2 then Y [1, 1] ← X[1, 1]
3 else Partition X into four (N/2) × (N/2) submatrices X11 , X12 , X21 , and X22 .
4 Partition Y into four (N/2) × (N/2) submatrices Y11 , Y12 , Y21 , and Y22 .
5 R-T RANS (X11 , Y11 , N/2)
6 R-T RANS (X12 , Y21 , N/2)
7 R-T RANS (X21 , Y12 , N/2)
8 R-T RANS (X22 , Y22 , N/2)

Assume that the cost of partitioning is O(1).

(b) Give and solve a recurrence for the number MT(N) of memory transfers incurred by
R-T RANS when N ≫ M.

Solution: If we make either the tall-cache assumption (i.e., M = Ω(B 2 )), or we

assume that the matrix is stored in the recursive block layout, then

4T (n/2) + Θ(1) if cn2 > M ,

%
MT (n) = .
M/B if cn2 ≤ M

The recursion tree has approximately lg N − lg( M/c) levels. Since every level
+

has 4 times as many nodes as the previous level, the solution

√ √
to the recurrence is
dominated by the cost of the leaves. The tree has 4 lg(N c/ M )
leaves, each of cost
M/B. Therefore, MT (N) is

, √ -2
M N c
MT (N) = √
B M
2
cN
=
B, -
N2
= O .
B
Handout 36: Final Exam Solutions 19

The following multithreaded algorithm computes the transpose in parallel:

P-T RANS(X, Y, N)
1 if N = 1
2 then Y [1, 1] ← X[1, 1]
3 else Partition X into four (N/2) × (N/2) submatrices X11 , X12 , X21 , and X22 .
4 Partition Y into four (N/2) × (N/2) submatrices Y11 , Y12 , Y21 , and Y22 .
5 spawn P-T RANS (X11 , Y11 , N/2)
6 spawn P-T RANS (X12 , Y21 , N/2)
7 spawn P-T RANS (X21 , Y12 , N/2)
8 spawn P-T RANS (X22 , Y22 , N/2)
9 sync

(c) Give and solve recurrences describing the work T1 (N) and critical-path length T∞ (N)
of P-T RANS . What is the asymptotic parallelism of the algorithm?

Solution:
N
# $
T1 (N) = 4T1 + O(1)
2
= Θ(N 2 ).
N
# $
T∞ (N) = T∞ + O(1)
2
= Θ(lg N).

The parallelism of the algoritm is T1 /T∞ , or Θ(N 2 / lg N).

Handout 36: Final Exam Solutions 20

Professor Kellogg inadvertantly places two additional sync statements into his code as follows:

K-T RANS(X, Y, N)
1 if N = 1
2 then Y [1, 1] ← X[1, 1]
3 else Partition X into four (N/2) × (N/2) submatrices X11 , X12 , X21 , and X22 .
4 Partition Y into four (N/2) × (N/2) submatrices Y11 , Y12 , Y21 , and Y22 .
5 spawn K-T RANS (X11 , Y11 , N/2)
6 sync
7 spawn K-T RANS (X12 , Y21 , N/2)
8 sync
9 spawn K-T RANS (X21 , Y12 , N/2)
10 spawn K-T RANS (X22 , Y22 , N/2)
11 sync

(d) Give and solve recurrences describing the work T1 (N) and critical-path length T∞ (N)
of K-T RANS . What is the asymptotic parallelism of this algorithm?

Solution: The work for the algorithm remains the same.

N
# $
T1 (N) = 4T1 + O(1)
2
= Θ(N 2 ).

The critical path increases, however, because we only solve one of the subproblems in
parallel.

N
# $
T∞ (N) = 3T∞ + O(1)
2
= Θ(N lg 3 ).

The parallelism of the algorithm is T1 /T∞ , or Θ(N 2−lg 3 ).

SCRATCH PAPER — Please detach this page before handing in your exam.
SCRATCH PAPER — Please detach this page before handing in your exam.