2.

ALGORITHM ANALYSIS
Running Time Analysis
• Mathematical analysis of worst-case running time of an algorithm as
function of input size. Why these choices?
• Mathematical: describes the algorithm. Avoids hard-to-control
experimental factors (CPU, programming language, quality of
implementation), while still being predictive.
• Worst-case: just works. (“average case” appealing, but hard to
analyze)
• Function of input size: allows predictions. What will happen on a new
input?
Efficiency
• When is an algorithm efficient?
• Stable Matching Brute force: Ω(n!)
• Propose-and-Reject?: O(n2)
• We must have done something clever
• Question: Is it Ω(n2) ?
Efficient implementation

Goal. Implement Gale–Shapley to run in O(n2) time.

GALE–SHAPLEY (preference lists for n hospitals and n students)

INITIALIZE M to empty matching.

WHILE (some hospital h is unmatched)
s  first student on h’s list to whom h has not yet
proposed. IF (s is unmatched)
Add h–s to matching M.
ELSE IF (s prefers h to current
partner hʹ) Replace hʹ–s with h–s in
matching M.
ELSE
s rejects h.

RETURN stable matching M.

25
Efficient implementation

Goal. Implement Gale–Shapley to run in O(n 2) time.

Representing hospitals and students. Index hospitals and students 1, …,

n. Representing the matching.

・ Maintain two arrays student[h] and hospital[s].
- if h matched to s, then student[h] = s and hospital[s] =
- huse value 0 to designate that hospital or student is
unmatched
・ Can add/remove a pair from matching in O(1) time.

・ Maintain set of unmatched hospitals in a queue (or

stack).
・ Can find an unmatched hospital in O(1) time.

26
Data representation: making a proposal

Hospital makes a proposal.

・ Key operation: find hospital’s next favorite student.
・ For each hospital: maintain a list of students, ordered by
preference.
・ For each hospital: maintain a pointer to student for next proposal.

next proposal
to

hospital h 3 4 1 5 2 null

favorit least
e favorite

Bottom line. Making a proposal takes O(1)

time.
27
Data representation: accepting/rejecting a proposal

Student accepts/rejects a proposal.

・ Does student s prefer hospital h to hospital hʹ ?
・ For each student, create inverse of preference list of

hospitals.

pref[] 1st 2nd 3rd 4th 5th 6th 7th 8th

8 3 7 1 4 5 6 2 student prefers hospital 4 to

6 since rank[4] < rank[6]

1 2 3 4 5 6 7 8
rank[]

4th 8th 2nd 5th 6th 7th 3rd 1st

for i = 1 to n
rank[pref[i]] = i

Bottom line. After Θ(n2 ) preprocessing time (to create the n ranking
arrays), it takes O(1) time to accept/reject a proposal. 28
Stable matching:
summary

Theorem. Can implement Gale–Shapley to run in O(n2)

time. Pf.
・ Θ(n2) preprocessing time to create the n ranking
arrays.
・ There are O(n2) proposals; processing each proposal
takes O(1) time.

Theorem. In the worst case, any algorithm to find a stable matching

must query the hospital’s preference list Ω(n2) times.

29
Survey of common running times
Constant time

Constant time. Running time is

O(1).
bounded by a constant,
Examples. which does not depend on input size
・ Conditional branch. n

・ Arithmetic/logic operation.
・ Declare/initialize a
variable.
・ Access
Follow aelement in an array.
link in ai linked list.
・ Compare/exchange two elements in an
array.
・…

31
Linear time

Linear time. Running time is O(n).

Definition: an algorithm runs in polynomial time if its running time is O(nd ) for some constant d

Merge two sorted lists. Combine two sorted linked lists A = a1, a2, …, an and
B = b1, b2, …, bn into a sorted whole.

O(n) algorithm. Merge in mergesort.

i  1; j  1.
WHILE (both lists are nonempty)
IF (ai ≤ bj) append ai to output list and
increment i. ELSE append bj
to output list and increment j.
32
Append remaining elements from nonempty list to
TARGET

S UM
TARGET-SUM. Given a sorted array of n distinct integers and an integer
T, find two that sum to exactly T ?

input
−20 10 20 30 35 40 60 70 T = 60
( s orted)

i j

33
Logarithmic time

Logarithmic time. Running time is O(log n).

Search in a sorted array. Given a sorted array A of n distinct integers and

an integer x, find index of x in array.

remaining
O(log n) algorithm. Binary elements

・ Invariant: If x is in the array, then x is in A[lo ..

search.

hi]. ‹ k ≤ 1 + log2 n.
・ After k iterations of WHILE loop, (hi − lo + 1) ≤ n / 2k
lo  1; hi  n.
WHILE (lo ≤ hi)
mid  ⎣(lo +
hi) / 2⎦.
IF (x < A[mid]) hi  mid − 1.
ELSE IF (x > A[mid]) lo  mid + 1.
ELSE RETURN mid.
RETURN −1. 35
 Logarithmic time

O(log n)

h t t p s : / / w w w. f a c e b o o k . c o m / p g / n p c o m p l e t e t e e n s
S EARCH IN A
SORTED ROTATED ARRAY

SEARCH-IN-SORTED-ROTATED-ARRAY. Given a rotated sorted array of n

distinct integers and an element x, determine if x is in the array.

s o r t e d circular array
20 30
95

35
90

50
85

60
80
65
75 67

s o r t e d rotated array

80 85 90 95 20 30 35 50 60 65 67 75

1 2 3 4 5 6 7 8 9 10 11 12

37
Linearithmic time

Linearithmic time. Running time is O(n log n).

Sorting. Given an array of n elements, rearrange them in ascending

order.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
O(n log n) algorithm. Mergesort. M E R G E S O R T E X A M P L E
E M R G E S O R T E X A M P L E
E MG R E S O R T E X A M P L E
E GM R E S O R T E X A M P L E
E GM R E S O R T E X A M P L E
E GM R E S O R T E X A M P L E
E GM R E O R S T E X A M P L E
E E G M O R R S T E X A M P L E
E E G M O R R S E T X A M P L E
E E G M O R R S E T A X M P L E
E E G M O R R S A E T X M P L E
E E G M O R R S A E T X M P L E
E E G M O R R S A E T X M P E L
E E G M O R R S A E T X E L M P
E E G M O R R S A E E L M P T X39
LARGEST EMPTY INTERVAL

LARGEST-EMPTY-INTERVAL. Given n timestamps x1, …, xn on which copies of a

file arrive at a server, what is largest interval when no copies of file
arrive?

40
Quadratic time

Quadratic time. Running time is O(n2).

Closest pair of points. Given a list of n points in the plane (x1, y1), …, (xn,
yn), find the pair that is closest to each other.

O(n2) algorithm. Enumerate all pairs of points (with i < j).

min  ∞.
FOR i=1
TO n
FOR j
= i + 1 TO n
d

 (xi −
xj)2 + (yi −
Remark. Ω(n ) seems inevitable,
2
yj)2. but this is just an illusion. [see
§5.4]
IF 42
Cubic time

Cubic time. Running time is O(n3).

3-SUM. Given an array of n distinct integers, find three that sum to

0.

O(n3) algorithm. Enumerate all triples (with i < j < k).

FOR i =1 TO n
FOR j = i+1 TO n
FOR k= j+1 TO n
IF (ai + aj
+ ak = 0)
RETURN (ai, aj,
ak).

Remark. Ω(n3) seems inevitable, but O(n2) is not hard. [see next
slide]
43
3-
S UM

3-SUM. Given an array of n distinct integers, find three that sum to

0.

O(n3) algorithm. Try all triples.

O(n2) algorithm.

44
Polynomial time

Polynomial time. Running time is O(nk) for some constant k > 0.

Independent set of size k. Given a graph, find k nodes such that no

two are joined by an edge.
k is a constant

O(nk) algorithm. Enumerate all subsets of k nodes.

FOREACH subset S of k nodes:

Check whether S is an independent set.
IF (S is an independent set)
RETURN S.
i nde pe nde nt s e t o f s i z e 3

・ Check whether S is an independent set of size k takes O(k2)

Σ
time. n n(n — 1)(n — 2) z ·· · z (n — k k
= ≤ n
+ 1) k
・ Number of k-element subsets =
・ O(k2 nk / k!) = O(nk). k k(k — 1)(k — 2) z ·· · z 1 !

poly-time for k = 17, but not 46

Exponential time

Exponential time. Running time is O(2nk ) for some constant k > 0.

Independent set. Given a graph, find independent set of max

cardinality.

O(n2 2n) algorithm. Enumerate all subsets of n elements.

S*  .
FOREACH subset S of n nodes:
Check whether S is an independent set.
IF (S is an independent set and ⎢S⎟ >
⎢S*⎟)
i nde pe nde nt s e t o f ma x cardinality
S* 
S. RETURN

S*.

47
Exponential time

Exponential time. Running time is O(2nk ) for some constant k > 0.

Euclidean TSP. Given n points in the plane, find a tour of minimum

length.

O(n 𐄂 n!) algorithm. Enumerate all permutations of length n.

π*  .
FOREACH permutation π of n points:
Compute length of tour corresponding to π.
IF (length(π) < length(π*))
π*  π.
for simplicity, we’ll assume
RETURN π*. Euclidean
distances are rounded to nearest
integer (to avoid issues with infinite
precision)

48
Analysis of algorithms: quiz 4

Which i s an equivalent definition o f exponential time?

A. O( 2n) doesn’t include

3n

B. O(2cn) for some constant c > 0. includes 3n but doesn’t include n! = 2Θ(n log n)

C. Both A and B.

D. Neither A nor B.

49