2.

A LGORITHM A NALYSIS

‣ computational tractability
‣ asymptotic order of growth
‣ implementing Gale–Shapley
‣ survey of common running times

Lecture slides by Kevin Wayne


Copyright © 2005 Pearson-Addison Wesley

http://www.cs.princeton.edu/~wayne/kleinberg-tardos

Last updated on 12/16/21 6:55 PM

 2. A LGORITHM A NALYSIS

‣ computational tractability
‣ asymptotic order of growth
‣ implementing Gale–Shapley
‣ survey of common running times

SECTION 2.1
A strikingly modern thought

“ As soon as an Analytic Engine exists, it will necessarily guide the future

course of the science. Whenever any result is sought by its aid, the question
will arise—By what course of calculation can these results be arrived at by
the machine in the shortest time? ” — Charles Babbage (1864)

how many times do you

have to turn the crank?

Analytic Engine
3
Models of computation: Turing machines

Deterministic Turing machine. Simple and idealistic model.

Running time. Number of steps.

Memory. Number of tape cells utilized.

Caveat. No random access of memory.

・Single-tape TM requires ≥ n2 steps to detect n-bit palindromes.
・Easy to detect palindromes in ≤ cn steps on a real computer.

4
Models of computation: word RAM

assume w ≥ log2 n
Word RAM.
・Each memory location and input/output cell stores a w-bit integer.
・Primitive operations: arithmetic/logic operations, read/write memory,
array indexing, following a pointer, conditional branch, …

constant-time C-style operations

(w = 64)
input … memory

program a[i] i

.
.
.
output …

Running time. Number of primitive operations.

Memory. Number of memory cells utilized.

Caveat. At times, need more refined model (e.g., multiplying n-bit integers). 5
Brute force

Brute force. For many nontrivial problems, there is a natural brute-force

search algorithm that checks every possible solution.
・Typically takes 2 n steps (or worse) for inputs of size n.
・Unacceptable in practice.

Ex. Stable matching problem: test all n! perfect matchings for stability. 6
Polynomial running time

Desirable scaling property. When the input size doubles, the algorithm
should slow down by at most some multiplicative constant factor C.

Def. An algorithm is poly-time if the above scaling property holds.

There exist constants a > 0 and b > 0 such that,

for every input of size n, the algorithm performs
b corresponds
≤ a n primitive computational steps.
to C = 2b

von Neumann Nash Gödel Cobham Edmonds Rabin

(1953) (1955) (1956) (1964) (1965) (1966)

7
Polynomial running time

We say that an algorithm is efficient if it has a polynomial running time.

Theory. Definition is (relatively) insensitive to model of computation.

Practice. It really works!

・The poly-time algorithms that people develop have both
small constants and small exponents.
・Breaking through the exponential barrier of brute force typically
exposes some crucial structure of the problem. n120

Map graphs in polynomial time

Map graphs in polynomial time
Exceptions. Some poly-time algorithms in the wild Department of
Mikkel Thorup
Computer Science, University of Copenhag
Mikkel Thorup
Universitetsparken
Department of Computer Science, University of1,Copenhagen
DK-2100 Copenhagen East, Denma
have galactic constants and/or huge exponents. Universitetsparken 1, DK-2100 Copenhagen East,
[email protected]
[email protected]
Denmark

Abstract
AbstractChen, Grigni, and Papadimitriou (WADS’97 and STOC’98)
have introduced a modified notion of planarity, where two
Chen, Grigni, and Papadimitriou (WADS’97 and STOC’98)
faces are considered adjacent if they share at least one point.
have introduced a modified notion of planarity, where two
The corresponding abstract graphs are called map graphs.
faces are considered adjacent if they share at least one point.
Chen et.al. raised the question of whether map graphs can be
The corresponding abstract graphs are called map graphs.
recognized in polynomial time. They showed that the decision Figure 1. Large
Chen et.al. raised the question of whether map graphs can be

Q. Which would you prefer: 20 n120 or n1 + 0.02 ln n ?

problem is in NP and presented a polynomial time algorithm
recognized in polynomial time. They showed that the decision Figure 1. Large cliques in maps
for the special case where we allow at most 4 faces to intersect
problem is in NP and presented a polynomial time algorithm addition, the hamantach may h
in any point — if only 3 are allowed to intersect in a point, we
for the special case where we allow at most 4 faces to intersect touching all three corners. In
get the usual planar graphs. addition, the hamantach may have at most two triangle faces
in any point — if only 3 are allowed to intersect in a point, we all the different types of large
Chen et.al. conjectured that maptouching
graphs can be recognized
all three corners. In [5] there is a classification of
get the usual planar graphs. showed that recognizing map
in polynomial time, and in this paper,
alltheir conjecture
the different is settled
types of large cliques in maps. Chen et.al. [5]
Chen et.al. conjectured that map graphs can be recognized recognition can be done in sing
affirmatively. showed that recognizing map graphs is in NP, hence that the
in polynomial time, and in this paper, their conjecture is settled they conjectured that, in fact, m
affirmatively. recognition can be done in singly exponential time. However,
polynomial time. They suppor
they conjectured that, in fact, map graphs can be recognized in
that if we allow at most 4 faces
1. Introduction polynomial time. They supported their conjecture by showing
resulting map graphs can be rec
that if we allow at most 4 faces to meet in any single point, the
1. Introduction
resulting map
Recently Chen, Grigni, and Papadimitriou [4, graphs
8
this paper, we settle the genera
can be recognized in polynomial time. In
5] suggested
a graph, we can decide in polyn
this paper,The
the study of a modified notion of planarity. we settle
basic the general conjecture, showing that given
frame-
Recently Chen, Grigni, and Papadimitriou [4, 5] suggested The algorithm can easily be m
Worst-case analysis

Worst case. Running time guarantee for any input of size n.

・Generally captures efficiency in practice.
・Draconian view, but hard to find effective alternative.

Exceptions. Some exponential-time algorithms are used widely in practice

because the worst-case instances don’t arise.

simplex algorithm Linux grep k-means algorithm

9
Other types of analyses

Probabilistic. Expected running time of a randomized algorithm.

Ex. The expected number of compares to quicksort n elements is ~ 2n ln n.

Amortized. Worst-case running time for any sequence of n operations.

Ex. Starting from an empty stack, any sequence of n push and pop
operations takes O(n) primitive computational steps using a resizing array.

Also. Average-case analysis, smoothed analysis, competitive analysis, ...

10
 2. A LGORITHM A NALYSIS

‣ computational tractability
‣ asymptotic order of growth
‣ implementing Gale–Shapley
‣ survey of common running times

SECTION 2.2
Big O notation

Upper bounds. f(n) is O(g(n)) if there exist constants c > 0 and n0 ≥ 0

such that 0 ≤ f(n) ≤ c · g (n) for all n ≥ n0.
c · g(n)

Ex. f(n) = 32n2 + 17n + 1. f(n)

・f(n) is O(n2). choose c = 50, n = 1

0

・f(n) is neither O(n) nor O(n log n).

n0 n

Typical usage. Insertion sort makes O(n2) compares to sort n elements.

12
Analysis of algorithms: quiz 1

Let f(n) = 3n2 + 17 n log2 n + 1000. Which of the following are true?

A. f(n) is O(n2).
 choose c = 1020, n0 = 1

B. f(n) is O(n3).
 choose c = 1020, n0 = 1

C. Both A and B.


D. Neither A nor B.

13
Big O notational abuses

One-way “equality.” O(g(n)) is a set of functions, but computer scientists

often write f(n) = O(g(n)) instead of f(n) ∈ O(g(n)).

Ex. Consider g1(n) = 5n3 and g2(n) = 3n2.

・We have g1(n) = O(n3) and g2(n) = O(n3).
・But, do not conclude g1(n) = g2(n).

Domain and codomain. f and g are real-valued functions.

・The domain is typically the natural numbers: ℕ → ℝ.
・Sometimes we extend to the reals: ℝ ≥ 0 → ℝ. input size, recurrence relations

・Or restrict to a subset. plotting, limits, calculus

Bottom line. OK to abuse notation in this way; not OK to misuse it.

14
Big O notation: properties

Reflexivity. f is O( f).

Constants. If f is O(g) and c > 0, then c f is O(g).

Products. If f 1 is O(g1) and f 2 is O(g2), then f 1 f 2 is O(g1 g2).

Pf.
・∃ c1 > 0 and n1 ≥ 0 such that 0 ≤ f 1(n) ≤ c1 · g1(n) for all n ≥ n1.
・∃ c2 > 0 and n2 ≥ 0 such that 0 ≤ f 2(n) ≤ c2 · g2(n) for all n ≥ n2.
・Then, 0 ≤ f 1(n) · f 2(n) ≤ c1 · c2 · g1(n) · g2(n) for all n ≥ max { n1, n2 }. ▪
c n0

Sums. If f 1 is O(g1) and f2 is O(g2), then f 1 + f 2 is O(max {g1, g2}).

ignore lower-order terms

Transitivity. If f is O(g) and g is O(h), then f is O(h).

Ex. f(n) = 5n3 + 3n2 + n + 1234 is O(n3).

15
Big Omega notation

Lower bounds. f(n) is Ω(g(n)) if there exist constants c > 0 and n0 ≥ 0

such that f(n) ≥ c · g(n) ≥ 0 for all n ≥ n0.
f(n)

Ex. f(n) = 32n2 + 17n + 1. c · g(n)

・f(n) is both Ω(n2) and Ω(n). choose c = 32, n0 = 1

・f(n) is not Ω(n3).

n0 n

Typical usage. Any compare-based sorting algorithm requires Ω(n log n)

compares in the worst case.

Vacuous statement. Any compare-based sorting algorithm requires

at least O(n log n) compares in the worst case.

16
Analysis of algorithms: quiz 2

Which is an equivalent definition of big Omega notation?

A. f(n) is Ω(g(n)) iff g (n) is O( f(n)).


B. f(n) is Ω(g(n)) iff there exists a constant c > 0 such that f(n) ≥ c · g(n) ≥ 0
for infinitely many n.


C. Both A and B.


D. Neither A nor B.

17
Analysis of algorithms: quiz 2

Which is an equivalent definition of big Omega notation?

A. f(n) is Ω(g(n)) iff g (n) is O( f(n)).


B. f(n) is Ω(g(n)) iff there exist constants c > 0 such that f(n) ≥ c · g(n) ≥ 0
for infinitely many n.
 why not use this definition instead?

C. Both A and B.


f(n) is Ω(g(n)) if there exist constants c1 > 0 and n0 ≥ 0

D. Neither A nor B.
such that f(n) ≥ c1 · g(n) ≥ 0 for all n ≥ n0

g(n) is O( f(n)) if there exist constants c2 > 0 and n0 ≥ 0

such that 0 ≤ g(n) ≤ c2 · f(n) for all n ≥ n0

c1 = 1 / c2
18
Big Theta notation

Tight bounds. f(n) is Θ(g(n)) if there exist constants c1 > 0, c2 > 0, and n0 ≥ 0
such that 0 ≤ c1 · g(n) ≤ f(n) ≤ c2 · g(n) for all n ≥ n0. c2 · g(n)

f(n)
Ex. f(n) = 32n2 + 17n + 1. c1 · g(n)
・f(n) is Θ(n2). choose c = 32, c = 50, n = 1
1 2 0

・f(n) is neither Θ(n) nor Θ(n3).

n0 n

Typical usage. Mergesort makes Θ(n log n) compares to sort n elements.

between ½ n log2 n
and n log2 n

19
Analysis of algorithms: quiz 3

Which is an equivalent definition of big Theta notation?

A. f(n) is Θ(g(n)) iff f(n) is both O( g(n)) and Ω( g(n)).


f (n)
B. f(n) is Θ(g(n)) iff nlim for0 some constant 0 < c < ∞.

= c >
g(n)

C. Both A and B.


D. Neither A nor B. counterexample

2n n
f (n) =
3n n

g(n) = n

f(n) is O(n) but limit does not exist

20
Asymptotic bounds and limits

f (n)
Proposition. If nlim for0 some constant 0 < c < ∞ then f(n) is Θ(g(n)).
= c >
g(n)

Pf.
・By definition of the limit, for any ε > 0, there exists n0 such that
f (n)
c c+
<latexit sha1_base64="4471u2rVjPFvRKTehhYE1ZF2UoY=">AAACbnicbVDRSiMxFE1HV7vdVas+7IOIYYtQEctUBJV9EZaFfXRhq0KnlMztnRrMJENyR1qGfpFf4+PqV/gHm6lFtnVvSHI4597c3BNnSjoKwz+VYGn5w8pq9WPt0+e19Y365taVM7kF7IBRxt7EwqGSGjskSeFNZlGkscLr+O57qV/fo3XS6N80zrCXiqGWiQRBnurXf3DgRzzCzEllNI+++aWwvHiUWAFF0tQHk2JYnnMq8MO3slq/3ghb4TT4e9CegQabxWV/s7IVDQzkKWoCJZzrtsOMeoWwJEHhpBblDjMBd2KIXQ+1SNH1ium8E77vmQFPjPVbE5+y/1YUInVunMY+MxV06xa1kvyf1s0pOesVUmc5oYbXRkmuOBlemscH0iKQGnsgwEr/Vw63wttE3uK5LtO3M4S5SYpRriWYAS6wikZkxcS72F707D3oHLfOW+Gvk8bF2czOKtthX1mTtdkpu2A/2SXrMGAP7JE9sefKS/Al2A32XlODyqxmm81F0PwL9Tm8EA==</latexit>
g(n)

for all n ≥ n0.

・Choose ε = ½ c > 0.
・Multiplying by g(n) yields 1/2 c · g(n) ≤ f(n) ≤ 3/2 c · g(n) for all n ≥ n0.
・Thus, f(n) is Θ(g(n)) by definition, with c1 = 1/2 c and c2 = 3/2 c. ▪

f (n)
Proposition. If lim = 0 , then f(n) is O(g(n)) but not Ω(g(n)).
n g(n)
<latexit sha1_base64="a6Ox4O/LCYsW5EpcWyBdjU0iDhE=">AAACZHicbVDLSgMxFE3HV62vqrgSJFgE3ZSpCFZEKLhxqWBV6JSSSe+0wUwyJHfUMszH+DVudekX+Btmahe2eiCXw7mv3BMmUlj0/c+SNze/sLhUXq6srK6tb1Q3t+6sTg2HNtdSm4eQWZBCQRsFSnhIDLA4lHAfPl4W+fsnMFZodYujBLoxGygRCc7QSb3qeSBF3MsUDYwYDJEZo59pIFSEo5wG5zSIDONZdKiO8mxQxEK8KIJf6VVrft0fg/4ljQmpkQmue5ulraCveRqDQi6ZtZ2Gn2A3YwYFl5BXgtRCwvgjG0DHUcVisN1sfGVOD5zSp5E27imkY/V3R8Zia0dx6CpjhkM7myvE/3KdFKNmNxMqSREU/1kUpZKipoVltC8McJQjRxg3wv2V8iFzvqAzdmrLeHYCfOqS7CVVgus+zKgSX9Cw3LnYmPXsL2kf18/q/s1JrdWc2Fkmu2SfHJIGOSUtckWuSZtw8kreyDv5KH15a962t/NT6pUmPdtkCt7eNxcjuXA=</latexit>

f (n)
Proposition. If lim = , then f(n) is Ω(g(n)) but not O(g(n)).
n g(n) <latexit sha1_base64="7KIVqEi//Gxt4Gr531t4InKKrao=">AAACaXicbVDRatswFFW8dk2zdkuzl7K+iIZB+xKcMVhHGQT20scU5iUQhyAr14moLBnpumsw/p1+TV9b6D/0Iyq7fmjSHZA4nHOvru6JUiks+v5jw3u3tf1+p7nb+rC3//FT+6Dz1+rMcAi4ltqMI2ZBCgUBCpQwTg2wJJIwiq5+l/7oGowVWv3BVQrThC2UiAVn6KRZexBKkcxyRUMjFktkxuh/NBQqxlVBw3MaxobxPD5Rp0W+KO9S/FU5VVFr1u76Pb8CfUv6NemSGsPZQaMTzjXPElDIJbN20vdTnObMoOASilaYWUgZv2ILmDiqWAJ2mlerFvSrU+Y01sYdhbRSX3fkLLF2lUSuMmG4tJteKf7Pm2QYn01zodIMQfGXQXEmKWpa5kbnwgBHuXKEcSPcXylfMhcOunTXplRvp8DXNslvMiW4nsOGKvEGDStciv3NzN6S4FvvZ8+//N4dnNVxNskROSYnpE9+kAG5IEMSEE5uyR25Jw+NJ6/jHXpfXkq9Rt3zmazB6z4DXwy7+A==</latexit>

21
Asymptotic bounds for some common functions

Polynomials. Let f(n) = a0 + a1 n + … + ad n d with ad > 0. Then, f(n) is Θ(n d ).

Pf.
a0 + a1 n + . . . + ad nd
lim d
= ad > 0
n n

Logarithms. loga n is Θ(logb n) for every a > 1 and every b > 1.

Pf. loga n 1 no need to specify base
= (assuming it is a constant)
logb n logb a
<latexit sha1_base64="DQO+CMgFJE1qjJ2qgd/1EHNxfVE=">AAACWHicbZDbSiNBEIZ7Zj3E4ybx0pvGIHgVZ0TQvVgQvNlLBaNCEkJNpyY29nQP3TWLYZjH8Gm8dR9i8WXsHBATLWj4+P/qrq4/yZV0FEX/g/DHyuraem1jc2t7Z/dnvdG8daawAjvCKGPvE3CopMYOSVJ4n1uELFF4lzxeTvy7v2idNPqGxjn2MxhpmUoB5KVB/biXWhBlT5nRALiuZpR44r/5zIs/RKj4oN6K2tG0+FeI59Bi87oaNIJmb2hEkaEmocC5bhzl1C/BkhQKq81e4TAH8Qgj7HrUkKHrl9PNKn7olSFPjfVHE5+qn2+UkDk3zhLfmQE9uGVvIn7ndQtKz/ul1HlBqMVsUFooToZPYuJDaVGQGnsAYaX/KxcP4OMgH+bClOnbOYqFTcqnQkthhrikKnoiC5VPMV7O7Ct0Ttq/2tH1aevifB5nje2zA3bEYnbGLtgfdsU6TLBn9sJe2b/gLQzC9XBj1hoG8zt7bKHC5ju/4bZS</latexit>

Logarithms and polynomials. loga n is O(n d ) for every a > 1 and every d > 0.
Pf. loga n
lim d
= 0
n n

Exponentials and polynomials. n d is O(r n ) for every r > 1 and every d > 0.
Pf. nd
lim n = 0
n r

Factorials. n! is 2Θ(n log n).

n n
Pf. Stirling’s formula: n! 2 n
e 22
Big O notation with multiple variables

Upper bounds. f(m, n) is O(g(m, n)) if there exist constants c > 0, m0 ≥ 0,

and n0 ≥ 0 such that 0 ≤ f(m, n) ≤ c · g (m, n) for all n ≥ n0 or m ≥ m0.

Ex. f(m, n) = 32mn2 + 17mn + 32n3.

・f(m, n) is both O(mn2 + n3) and O(mn3).
・f(m, n) is O(n3) if a precondition to the problem implies m ≤ n.
・f(m, n) is neither O(n3) nor O(mn2).
Typical usage. In the worst case, breadth-first search takes O(m + n) time
to find a shortest path from s to t in a digraph with n nodes and m edges.

23
 2. A LGORITHM A NALYSIS

‣ computational tractability
‣ asymptotic order of growth
‣ implementing Gale–Shapley
‣ survey of common running times

SECTION 2.3
Efficient implementation

Goal. Implement Gale–Shapley to run in O(n 2) time.

GALE–SHAPLEY (preference lists for n hospitals and n students)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
____________________________________________________________________________________________________________________

INITIALIZE M to empty matching.

WHILE (some hospital h is unmatched)
s ← first student on h’s list to whom h has not yet proposed.
IF (s is unmatched)
Add h–s to matching M.
ELSE IF (s prefers h to current partner hʹ)
Replace hʹ–s with h–s in matching M.
ELSE
s rejects h.

RETURN stable matching M.

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
____________________________________________________________________________________________________________________

25
Efficient implementation

Goal. Implement Gale–Shapley to run in O(n 2) time.

Representing hospitals and students. Index hospitals and students 1, …, n.

Representing the matching.

・Maintain two arrays student[h] and hospital[s].
- if h matched to s, then student[h] = s and hospital[s] = h
- use value 0 to designate that hospital or student is unmatched
・Can add/remove a pair from matching in O(1) time.

・Maintain set of unmatched hospitals in a queue (or stack).

・Can find an unmatched hospital in O(1) time.

26
Data representation: making a proposal

Hospital makes a proposal.

・Key operation: find hospital’s next favorite student.
・For each hospital: maintain a list of students, ordered by preference.
・For each hospital: maintain a pointer to student for next proposal.

next proposal to

hospital h 3 4 1 5 2 null

favorite least favorite

Bottom line. Making a proposal takes O(1) time.

27
Data representation: accepting/rejecting a proposal

Student accepts/rejects a proposal.

・Does student s prefer hospital h to hospital hʹ ?
・For each student, create inverse of preference list of hospitals.

pref[]
1st 2nd 3rd 4th 5th 6th 7th 8th

8 3 7 1 4 5 6 2 student prefers hospital 4 to 6

since rank[4] < rank[6]

1 2 3 4 5 6 7 8
rank[]
4th 8th 2nd 5th 6th 7th 3rd 1st

for i = 1 to n
rank[pref[i]] = i

Bottom line. After Θ(n2) preprocessing time (to create the n ranking arrays),
it takes O(1) time to accept/reject a proposal. 28
Stable matching: summary

Theorem. Can implement Gale–Shapley to run in O(n2) time.

Pf.
・Θ(n2) preprocessing time to create the n ranking arrays.
・There are O(n2) proposals; processing each proposal takes O(1) time. ▪

Theorem. In the worst case, any algorithm to find a stable matching must
query the hospital’s preference list Ω(n2) times.

29
 2. A LGORITHM A NALYSIS

‣ computational tractability
‣ asymptotic order of growth
‣ implementing Gale–Shapley
‣ survey of common running times

SECTION 2.4
Constant time

Constant time. Running time is O(1).

bounded by a constant,
Examples. which does not depend on input size n

・Conditional branch.
・Arithmetic/logic operation.
・Declare/initialize a variable.
・Follow a link in a linked list.
・Access element i in an array.
・Compare/exchange two elements in an array.
・…

31
Linear time

Linear time. Running time is O(n).

Merge two sorted lists. Combine two sorted linked lists A = a1, a2, …, an and
B = b1, b2, …, bn into a sorted whole.

O(n) algorithm. Merge in mergesort.

i ← 1; j ← 1.
WHILE (both lists are nonempty)
IF (ai ≤ bj) append ai to output list and increment i.
ELSE append bj to output list and increment j.
Append remaining elements from nonempty list to output list.
32
TARGET SUM

TARGET-SUM. Given a sorted array of n distinct integers and an integer T,

find two that sum to exactly T ?

input
−20 10 20 30 35 40 60 70 T = 60
(sorted)

i j

33
TARGET SUM

TARGET-SUM. Given a sorted array of n distinct integers and an integer T,

find two that sum to exactly T ?

O(n2) algorithm. Try all pairs.

O(n) algorithm. Exploit sorted order.

input
−20 10 20 30 35 40 60 70 T = 60
(sorted)

i j

Invariant. No element to the left of i or right of j in pair that sums to T.

34
Logarithmic time

Logarithmic time. Running time is O(log n).

Search in a sorted array. Given a sorted array A of n distinct integers and an

integer x, find index of x in array.

remaining elements
O(log n) algorithm. Binary search.
・Invariant: If x is in the array, then x is in A[lo .. hi].
・After k iterations of WHILE loop, (hi − lo + 1) ≤ n / 2k k ≤ 1 + log2 n.

lo ← 1; hi ← n.
WHILE (lo ≤ hi)
mid ← ⎣(lo + hi) / 2⎦.
IF (x < A[mid]) hi ← mid − 1.
ELSE IF (x > A[mid]) lo ← mid + 1.
ELSE RETURN mid.
RETURN −1.
35
 Logarithmic time

O(log n)

https://www.facebook.com/pg/npcompleteteens
36
SEARCH IN A SORTED ROTATED ARRAY

SEARCH-IN-SORTED-ROTATED-ARRAY. Given a rotated sorted array of n distinct

integers and an element x, determine if x is in the array.

sorted circular array

20 30
95

90

35
50
85
80

60
65
75 67

sorted rotated array

80 85 90 95 20 30 35 50 60 65 67 75

1 2 3 4 5 6 7 8 9 10 11 12

37
SEARCH IN A SORTED ROTATED ARRAY

SEARCH-IN-SORTED-ROTATED-ARRAY. Given a rotated sorted array of n distinct

integers and an element x, determine if x is in the array.

O(log n) algorithm.
・Find index k of smallest element.
・Binary search for x in either A[1 .. k−1] or A[k .. n].
find index of smallest element

lo ← 1; hi ← n.
IF (A[lo] ≤ A[hi]) RETURN 0 sorted

WHILE (lo + 2 ≤ hi) at least 3 elements

mid ← ⎣(lo + hi) / 2⎦.

loop invariant
IF (A[mid] < A[hi]) hi ← mid. A[lo] > A[hi]

ELSE IF (A[mid] > A[hi]) lo ← mid.

RETURN hi
38
 Linearithmic time

Linearithmic time. Running time is O(n log n).

Sorting. Given an array of n elements, rearrange them in ascending order.

O(n log n) algorithm. Mergesort.

a[]
lo hi 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
M E R G E S O R T E X A M P L E
a, aux, 0, 0, 1) E M R G E S O R T E X A M P L E
a, aux, 2, 2, 3) E M G R E S O R T E X A M P L E
aux, 0, 1, 3) E G M R E S O R T E X A M P L E
a, aux, 4, 4, 5) E G M R E S O R T E X A M P L E
a, aux, 6, 6, 7) E G M R E S O R T E X A M P L E
aux, 4, 5, 7) E G M R E O R S T E X A M P L E
ux, 0, 3, 7) E E G M O R R S T E X A M P L E
a, aux, 8, 8, 9) E E G M O R R S E T X A M P L E
a, aux, 10, 10, 11) E E G M O R R S E T A X M P L E
aux, 8, 9, 11) E E G M O R R S A E T X M P L E
a, aux, 12, 12, 13) E E G M O R R S A E T X M P L E
a, aux, 14, 14, 15) E E G M O R R S A E T X M P E L
aux, 12, 13, 15) E E G M O R R S A E T X E L M P
ux, 8, 11, 15) E E G M O R R S A E E L M P T X
, 0, 7, 15) A E E E E G L M M O P R R S T X
Trace of merge results for top-down mergesort 39
LARGEST EMPTY INTERVAL

LARGEST-EMPTY-INTERVAL. Given n timestamps x1, …, xn on which copies of a

file arrive at a server, what is largest interval when no copies of file arrive?

40
LARGEST EMPTY INTERVAL

LARGEST-EMPTY-INTERVAL. Given n timestamps x1, …, xn on which copies of a

file arrive at a server, what is largest interval when no copies of file arrive?

O(n log n) algorithm.

・Sort the array a.
・Scan the sorted list in order, identifying the maximum gap between
successive timestamps.

41
Quadratic time

Quadratic time. Running time is O(n2).

Closest pair of points. Given a list of n points in the plane (x1, y1), …, (xn, yn),
find the pair that is closest to each other.

O(n2) algorithm. Enumerate all pairs of points (with i < j).

min ← ∞.
FOR i = 1 TO n
FOR j = i + 1 TO n
d ← (xi − xj)2 + (yi − yj)2.
IF (d < min)
min ← d.

Remark. Ω(n2) seems inevitable, but this is just an illusion. [see §5.4]
42
Cubic time

Cubic time. Running time is O(n3).

3-SUM. Given an array of n distinct integers, find three that sum to 0.

O(n3) algorithm. Enumerate all triples (with i < j < k).

FOR i = 1 TO n
FOR j = i + 1 TO n
FOR k = j + 1 TO n
IF (ai + aj + ak = 0)
RETURN (ai, aj, ak).

Remark. Ω(n3) seems inevitable, but O(n2) is not hard. [see next slide]
43
3-SUM

3-SUM. Given an array of n distinct integers, find three that sum to 0.

O(n3) algorithm. Try all triples.

O(n2) algorithm.

44
3-SUM

3-SUM. Given an array of n distinct integers, find three that sum to 0.

O(n3) algorithm. Try all triples.

O(n2) algorithm.
・Sort the array a.
・For each integer ai : solve TARGET-SUM on the array containing all
elements except ai with the target sum T = −ai.

Best-known algorithm. O(n2 / (log n / log log n)).

Conjecture. No O(n2−ε) algorithm for any ε > 0.

45
Polynomial time

Polynomial time. Running time is O(nk) for some constant k > 0.

Independent set of size k. Given a graph, find k nodes such that no two
are joined by an edge.
k is a constant

O(nk) algorithm. Enumerate all subsets of k nodes.

FOREACH subset S of k nodes:

Check whether S is an independent set.
IF (S is an independent set)
RETURN S.
independent set of size 3

・Check whether S is an independent set of size k takes O(k2) time.

・Number of k-element subsets = n = n(n 1)(n 2) · · · (n k + 1) nk

・O(k n / k!) = O(n ).

2 k k k k(k 1)(k 2) · · · 1 k!

poly-time for k = 17, but not practical 46

Exponential time

k
Exponential time. Running time is O(2n ) for some constant k > 0.

Independent set. Given a graph, find independent set of max size.

O(n2 2n) algorithm. Enumerate all subsets of n elements.

S* ← ∅.
FOREACH subset S of n nodes:
Check whether S is an independent set.
IF (S is an independent set and ⎢S⎟ > ⎢S*⎟)
S* ← S. independent set of max size
RETURN S*.

47
Exponential time

k
Exponential time. Running time is O(2n ) for some constant k > 0.

Euclidean TSP. Given n points in the plane, find a tour of minimum length.

O(n 𐄂 n!) algorithm. Enumerate all permutations of length n.

π* ← ∅.
FOREACH permutation π of n points:
Compute length of tour corresponding to π.
IF (length(π) < length(π*))
π* ← π.
for simplicity, we’ll assume Euclidean
RETURN π*. distances are rounded to nearest integer
(to avoid issues with infinite precision)

48
Analysis of algorithms: quiz 4

Which is an equivalent definition of exponential time?

A. O( 2n)
 doesn’t include 3n

B. O(2cn ) for some constant c > 0.
 includes 3n but doesn’t include n! = 2Θ(n log n)

C. Both A and B.


D. Neither A nor B.

49