Lec1

Syllabus

Graph Algorithm DFS/BFS

Greedy Algorithms

Dynamic Programming

Divide & Conquer

NP-Completeness Theory

Approximation Algorithms

Randomized Algorithms (Probably)

Computational Geometry

Streaming Algorithms

References

Algorithm Design, Kleinberg & Tardos

[CLRS] Intro to Algorithm

Scores

hw: 30%

mid: 30%, in-class

final: 40%, 3 hrs

bonus: 3~5%

research project: publication level - A+; else - bonus

Stable Matching

(Gale & Sharpley)

statement: given two equally sized sets, every element having a preference list of the
elements in the other set.

a pair (A,B) is unstable if both of them prefer each other to their current partner.

G-S algo (propose-reject algo)

while(there is somebody single)

A <- an arbitraty single boy
X <- first girl A hasn't proposed
if(X is single)
A and X engage with each other
else // if X is engaged
if(A is better than her current bf)
A and X engage with each other
else
X rejects A

Lemma: G-S terminates with a matching.

 Proof: Once a girl is engaged, she will stay engaged forever. Consider a boy, if he already
exhausted his preference list, then every girl must be engaged, so a matching exists.
Lemma: G-S finds a stable matching.

Proof: By contradiction, suppose (A, X) is an unstable pair. Assume currently A-Y, B-X,
then X > Y in A's list and A > B in X's. Since A's matching is Y, all previous girls must have
rejected A, and so does X. But when A proposes to X, X's current boyfriend must be not
better than B, so X cannot reject A, which leads to a contradiction.

(worst) running time

running time expectation analysis

random instance: the preference list of each boy is a random permutation =

trick: how to generate a random permutation. from to , every time randomly pick
one element in the remaining.

Theorem: .

Proof: key observation: consider some algorithm G-S'.

G-S: each time a boy proposes to a random girl he has not proposed yet.

G-S': each time a boy proposes to a random girl. (possibly he has proposed yet)

Then intuitively . And G-S' is basically a coupon collector problem

with expectation time.

BFS/DFS

BFS, DFS, SCC, DAG (omitted)

Lec2
Parenthesis theorem: [discovery time, finishing time] in DFS do not intersect.

white-path THM: Consider the time DFS discovery M, V is a descendant of M (in DFS tree) iff

white path from

Proof: trivial.

Kosaraju's Algorithm: see Introduction to Algorithms, P617.

DFS to compute finishing time

DFS but consider tree roots in order of decreasing .

induction: for some scc define , every time the algorithm

extracts the scc with largest .

Greedy Algorithm (ItA P414 & AD P115)

Introduction: activity-selection problem (maximum-size subset problem of segments)

Algorithm: Repeat selecting the available activity with earliest finishing time.

Optimality: Compare with optimal answer.

Example: scheduling to "minimize" lateness

 Description: Given n jobs, each job i has deadline and length , if a job is finished
after we define lateness , otherwise . Goal: find a permutation of
[n] and minimize the max lateness.

Algorithm: Schedule jobs in increasing order of the deadlines.

Optimality: Assume initially jobs are in arbitrary order, swap the neighboring two jobs if
their lateness forms an inversion, and the max lateness does not increase.
Example: Dijkstra Algorithm for shortest path

Algorithm: Maintaining a set whose shortest path has been determined, and every
time from all edge , choose one with minimal , and
append to .

Optimality: By contradiction, if violates, consider the time is about to be

appended into .

Lec3
Minimal spanning tree for undirected graph

Kruskal's algorithm: Adding edges (if possible) in value-ascending order

implementation using union-find set

Prim's algorithm: Every time adding a nearest vertex, like Dijkstra

Reverse deletion: remove edge one by one.

Union-find set and its time complexity

Heap sorting

Fibonacci Heap (?)

The Huffman code

Every time combining two least-used nodes

Laplacian matrix and incidence matrix (in-homework)

https://en.m.wikipedia.org/wiki/Laplacian_matrix

Lec4
Greedy Algorithm (Cont'd)

Optimal Caching (AD 4.3)

: pieces of elements stored in main memory; Cache: hold pieces.

Initially, cache holds elements. We need to process a sequence of elements .

Design an evicting algorithm that minimize cache misses.

Algorithm: evict the item that is needed the farthest into the future
 Proof: First prove reduced unreduced. Next transform any non-ff schedule into the ff-
schedule, by showing if agrees with in first steps then that agrees with
in first steps and have same time of misses as .
Online version: LRU used. Bound of LRU: .

Minimum-Cost Arborescences (AD 4.9)

Arborenscence: Another word of 'tree'

Goal: Find a min-cost directed spanning tree rooted at in a directed graph .

Algorithm:

for each $v \in V \backslash\{r\}$

let $y_v$ be the minimum cost of an edge entering $v$
**Subtract all cost of edges entering $v$ by $y_v$**
Choose one 0 edge entering $v$
Let $F^*$ be the set of chosen edges, if $F^*$ is a tree, done.
Else there is a directed cycle, contract it, recursively find a minimum-
cost arborescence in the contracted graph

Proof is by showing for every zero cycle , an optimal arborescence will have only one edge
entering , so contracting is WLOG

Tree-Matrix Theorem

#(spanning tree) in a graph =

Proof (?)

Dynamic Programming (AD 6)

Weighted Interval Scheduling (AD 6.1)

Goal: Given intervals with values, pick a subset of intervals that are mutually compatible and
maximize sum of values.

Algorithm: Sort intervals in ascending finishing time, let be the last interval finishing
before starts, be the answer considering only the first intervals, then
.

Subset Sums & Knapsacks (AD 6.4)

Subset sums: Find maximum subset sum that . Is a special case of knapsacks ( 背包).
The knapsacks algorithm is a pseudo-poly algorithm; It relies on the scale of volume, but poly
algo should only polynomially rely on the input size.

RNA Secondary Structure (AD 6.5)

Similar to parenthesis matching; f =

Tree Decomposition (AD 10.4)

Seeking to decompose according to a tree-like pattern

A tree decomposition of consists of a tree and a subset associated with each

node of .

The collection of must satisfy:

(Node Coverage) Every node of belongs to at least one piece .

 (Edge Coverage) For every edge of , there is some piece containing both ends of
.

(Coherence) Let , , and be three nodes of such that lies on the path from
to . Then, if a node of belongs to both and , it also belongs to .
Coherence ensures that when we delete a node or edge in , correspondingly breaks
apart like a tree does. See AD Thm 10.13 (P575) & 10.14 (P577).

Def. The tree-width of a tree decomposition

Thm10.15. A connected graph has tree-width iff it's a tree.

Intuitively, applying dynamic programming on tree decomposition is like on a tree.

Lec5
Bellman-ford Algorithm (ItA 24.1)

Assume no negative cycle

An intuitive algorithm:

// OPT(i, v): shortest v-t path that uses at most i edges

OPT(i, v) = min(OPT(i - 1, v), min_w(OPT(i - 1,w) + c(w, v)))

An elegant implementation:

for (int i = 1; i <= n - 1; i++)

for (int e = 1; e <= m; e++)
dis[e.v] = min(dis[e.v], dis[e.u] + e.w)

Shortest path tree (ItA P647)

P = union of (v, last[v])

If P contains a cycle C, then cost of C < 0

Proof = Telescoping

Robertson-Seymour THM (Graph Minor THM)

Minor: A minor of G can be obtained by contracting edges and deletion of nodes or edges
from G.

Def. a graph family of a graph is minor-closed if minor of

Ex.

Wagner/Kuratowski Thm: Planar graphs doesn't have or as minors

Robertson-Seymour Thm: every family of graphs that is closed under minors can be defined
by a finite set of forbidden minors.

NP-completeness (ItA 34)

P: problems that are solvable in polynomial time.

NP: problems that are verifiable in polynomial time. (Informal)

NPC (NP-completeness): Formal definition TBA

Polynomial-time reduction
 Def. Polynomial-time reduction: If a problem can be solved in poly-time plus an oracle
that solves problem (in O(1) time), we call is poly-time reducible to , denoted
.

Cook & Karp. If edges are defined by poly-time reduction, all problems in NP has path into
NPC, and within NPC, there is a complete graph.

Examples of NPC

Independent Set Problem: Given a graph , ask whether there is an independent set of size
at least .

Vertex Cover ( 最小边覆盖): Given , a vertex cover is s.t. ,

either or . Ask whether where is a vertex cover of size at most .

Proof of : is an iff is a .

Definition of NP

We focus on decision problems (output is either Y or N)

For a decision problem , we can think as a collection of Yes instances, if is

Yes.

Def. (Efficient Verifier) B is an efficient verifier for problem if:

1. B is a poly-time (in ) algorithm with input and , where is an instance of , is a

certificate

2. iff s.t. and

is problem, is example, is candidate answer, is jury.

Def. is the class of all problems for which there is an efficient verifier for .

NP-Completeness definition

Def: A problem is NPC if:

1.

2. For all , ( is harder than all other problems in )

Observation: Suppose , , then . (Obviously )

Cook-Levin Thm (1971): SAT is NPC. (SAT = Boolean satisfiability problem = Determining whether
there exists an assignment of truth values (true or false) to a set of Boolean variables that makes a
given Boolean formula true)

High-level proof: Suppose is some problem, is an efficient verifier for . We

? B to a circuit, (See ItA)

Lec6
Overview:
CIRCUIT-SAT: Given a boolean combinational circuit composed of AND, OR, and NOT gates, is
it satifiable?

It's NPC due to the workings of computer hardware (composed of circuits) (ItA Lemma
34.6)

SAT: Determing whether a boolean formula is satisfiable.

CIRCUIT-SAT SAT, because we can assume every node in the circuit correspond to a
boolean; Then we take the AND of clauses of every nodes in the circuit together, with
the final node. So this boolean formula is satisfiable iff the original circuit is satisfiable.
(ItA Thm 34.9)

3-CNF-SAT: 3-conjunctive normal form SAT

A literal is an occurence of a variable or its negation.

A conjective normal form (CNF) is a boolean formula that is expressed as an AND of

clauses, each are the OR of one or more literals.

A 3-CNF is a CNF with each clause having exactly three distinct literals.

SAT 3-CNF-SAT. Proof: First we can use the method as above to construct the clause
tree of an SAT, every node of tree having 1 or 2 children. Then we obtain a formula
composed of conjuctions of clauses each having at most 3 literals, but not necessarily an
OR of them. Then for every clause, we bruteforcely enumerate over all possible
inputs, and use the 1-output tuples to rebuild an OR of AND clauses. Finally we apply
DeMorgan's laws to change it into an AND of OR clauses, i.e. a CNF. (ItA Thm 34.10)

CLIQUE: A clique is a subset of vertices that are pairwisely connected in . The CLIQUE
problem asks whether a graph contains a clique of size .
 3-CNF-SAT CLIQUE. Proof: for a 3-CNF-SAT problem with clauses, we constuct a
graph with vertices, corresponding to every literal. Two literals are connected with
edges iff (1) They belong to different clauses (2) They are not complements, e.g. and
. Then the constructed having clique of size The SAT is satisfiable. (ItA Thm 34.11)
VERTEX-COVER: A subset of vertices that covers all edges. The problem asks whether a graph
contains a vertex cover of size or not.

CLIQUE VERTEX-COVER. Proof: Construct the complement graph of the original

graph , then having clique of size having vertex cover of size . (ItA
Thm 34.12)

HAM-CYCLE ?

TSP ?

SUBSET-SUM ?

3-matching: ?

3-color: ?

Lec7
NP-completeness: decision problem

NP-hardess: optimization problem (NPO)

We say an optimization problem is NP-hard, iff its corresponding decision problem is

NP-complete.

Extending the limits of tractability (AD 10)

We want to further define the set of problems that we can polynomially solve, this can be
done by parametrization.

Def. FPT (fixed parameter tractable) Let the input size of a problem be and we have a
parameter , we say a problem is FPT if there is an algo that solves the problem in
time. e.g. is feasible but is not.

Finding Small Vertex Covers (AD 10.1)

We revisit the vertex cover problem, the parameter being the size of vc.

Then a algorithm is more appealing than an one.

Key observation: let some then iff or

.

ALGO check(G, k)
if G has no edge, the empty set is a VC, return T
if |E(G)| > k|V| (the maximum number of edges coverable by k vertices),
return F
randomly pick an edge e = (u, v),
return check(G - u, k - 1) OR check(G - v, k - 1)

The running time of this algo is : levels of recursion.

Best FPT:

Kernel method:

Approximation Algorithms (poly-time) (AD 11)

 Def. Approximation ratio ( )

For minimization problems,

For maximization problems,

: exact solution : const approximation

PTAS (Poly-Time Approximation Scheme) algorithms such that , is a

constant, and running time is poly if is constant. e.g.

FPTAS (Fully Poly-Time Approximation Scheme) running time is

Load Balancing Problem (AD 11.1)

Problem statement: Given a set of machines and a set of jobs, each job
has processing time . We want to assign the jobs to the machines such that the max load of
machines is minimized, formally, if denotes the set of jobs assigned to machine
then machine 's load is and we want to minimize .

This problem is NP-hard!

3-partition problem: Given numbers , partition then into groups (each

group 3 numbers) s.t. the sum of each group is the same (you can assume each is
poly-sized)

3-partition is . And if load balancing is solved then setting jobs = and

max load = average will resolve the 3-partition problem.

Another approach is using subset sum. Subset sum is , and if load balancing is
solved then giving two machines and asking the minimal of maximal
load to be will resolve the subset sum for with target .

A greedy algorithm: Process the jobs in any order, one by one. Assign the job to the machine
with then minimum load.

THM: The greedy algorithm achieves an approximation rate .

Proof: Assume the optimal load is . Then note that , and

. We claim that the output of the greedy algorithm
.

Assume the machine with largest load is . Consider the last time we assign to
then and .

Worst case of the greedy algorithm: we have jobs of time and one job of time
.
 An improvement of the greedy algorithm: we do not process in arbitrary order, but in
decreasing order.

THM: The sorted greedy algorithm achieves approximation rate .

Proof: Similarly assume the machine with maxload is . If only has one job then
our answer is optimal. Else let be the last job assigned to , then , and
thus .

K-Center Problem (AD 11.2)

Problem statement: Given a metric graph (i.e. complete graph,

), cluster the graph into
clusters and minimize the maximum of cluster radius.

An greedy algorithm: Guess the optimal radius (at most possibilities)

S = G, C = \emptyset
while (S <> \emptyset)
Choose an arbitrary vertex v \in S, C <- C + {v}
Remove all node u s.t. d(u, v) <= 2r
If |C| > k, our guess is too small.
If |C| <= k, succeed.

Proof of : It suffices to show if then our guess succeeds. Assume the

optimal clusters are then every time we will remove at least all nodes in one
cluster, so within times we will clear .

Next we will prove inapproximability for .

Define the Promise Problem: for a given input, we guarantee the optimal value is not in
, ask whether the optimal value is or .

Then an -approximate algo can be used to solve the above promise problem.

Next we show when the promise problem for k-center is NPC.

Proof: Dominating Set promise k-center. Because for some graph , we

construct a complete graph such that an edge in has weight if it's in and
if it's not. Then promising 's k-center with is equivalent to promising
's k-size dominating set.