Lecture 28.

Approximation Algorithms
Introduction to Algorithms
Sungkyunkwan University

Hyunseung Choo
[email protected]

Superintelligence Laboratory Copyright 2000-2022 Superintelligence

H.CHOO
Laboratory
1/21
 Lecture Contents
 Approximation Algorithms

 Vertex Cover Problem

 Traveling Salesman Problem

 Job Scheduling Problem

Superintelligence Laboratory H.CHOO 2/21

 Approximation Algorithms
 Possible ways to deal with NP-completeness
 If the input is small, an algorithm with exponential running
time may be satisfactory
 Isolate special cases that can be solved in polynomial time
 Find near-optimal solutions in polynomial time
 An approximation algorithm is an algorithm which can
provide a near-optimal solution for a NP-complete
problem in polynomial time

Superintelligence Laboratory H.CHOO 3/21

 Approximation Ratio
 An approximation algorithm for a problem has an
approximation ratio of 𝜌(𝑛) if for any input of size 𝑛:
𝐶 𝐶∗
𝑚𝑎𝑥 ∗ , ≤ 𝜌(𝑛)
𝐶 𝐶
 𝐶 is the cost of the approximation algorithm
 𝐶 ∗ is the optimal cost of an optimal solution
𝐶∗ 𝐶
 𝜌(𝑛) ≥ 1 ( ≥ 1 for maximization problems, ∗ ≥ 1 for
𝐶 𝐶
minimization problems)
 The algorithm is called an 𝜌(𝑛)-𝑎𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑖𝑜𝑛 𝑎𝑙𝑔𝑜𝑟𝑖𝑡ℎ𝑚
 An optimal algorithm has 𝜌 𝑛 = 1

Superintelligence Laboratory H.CHOO 4/21

 Vertex Cover Problem
 A vertex cover of an undirected graph 𝐺 = (𝑉, 𝐸) is a
subset 𝑉 ′ ⊆ 𝑉 such that if (𝑢, 𝑣) is an edge of 𝐺, then
either 𝑢 ∈ 𝑉′ or 𝑣 ∈ 𝑉′ (or both); the size of a vertex
cover is the number of vertices in it
 The vertex cover problem is to find a vertex cover of
minimum size in a given undirected graph

Input graph Optimal vertex cover

includes vertices b,d,e

Superintelligence Laboratory H.CHOO 5/21

 Vertex Cover Problem: Algorithm

Superintelligence Laboratory H.CHOO 6/21

 Vertex Cover Problem: Example (1/3)

(a) Input graph 𝐺. (b) The edge (𝑏, 𝑐), shown heavy, is the first edge chosen by
APPROX-VERTEX-COVER. Vertices 𝑏 and 𝑐, shown lightly shaded, are added
to the set 𝐶 containing the vertex cover being created. Edges (𝑎, 𝑏), (𝑐, 𝑒), and
(𝑐, 𝑑), shown dashed, are removed since they are now covered by some vertex
in 𝐶.

Superintelligence Laboratory H.CHOO 7/21

 Vertex Cover Problem: Example (2/3)

(c) Edge (𝑒, 𝑓) is chosen; vertices 𝑒 and 𝑓 are added to 𝐶; edges (𝑑, 𝑒) and
(𝑑, 𝑓) are removed. (d) Edge (𝑑, 𝑔) is chosen; vertices 𝑑 and 𝑔 are added to 𝐶

Superintelligence Laboratory H.CHOO 8/21

 Vertex Cover Problem: Example (3/3)

(e) The set 𝐶 which is the vertex cover produced by APPROX-VERTEX-

COVER, contains six vertices: 𝑏, 𝑐, 𝑑, 𝑒, 𝑓, 𝑔. (f) The optimal vertex cover for this
problem contains only three vertices: 𝑏, 𝑑, and 𝑒.

Superintelligence Laboratory H.CHOO 9/21

 Cover Vertex Problem: Analysis
 APPROX-VERTEX-COVER is a 2-approximation algorithm
Proof:
- Let 𝐴 denote the set of edges that line 4 of APPROX-
VERTEX-COVER picked
- An optimal solution must include at least one vertex of
each edge in A, so |𝐶 ∗ | ≥ 𝐴
- In line 5 of APPROX-VERTEX-COVER, for each edge of 𝐴,
two vertices are added to 𝐶, so, number of vertices in C is
𝐶 = 2 𝐴 ≤ 2 𝐶 ∗ that means
𝐶
∗
≤2
𝐶

Superintelligence Laboratory H.CHOO 10/21

 Traveling Salesman Problem
 In traveling salesman problem, we are given a complete
undirected graph 𝐺 = (𝑉, 𝐸) that has a non-negative
integer cost 𝑐(𝑢, 𝑣) associated with each edge
(𝑢, 𝑣) ∈ 𝐸, and we must find a Hamiltonian cycle of 𝐺
with minimum cost

Input graph A Hamiltonian cycle with minimum cost

Superintelligence Laboratory H.CHOO 11/21

 Traveling Salesman Problem: Algorithm
 The following algorithm uses minimum spanning tree to
create a tour whose cost is no more than twice that of
the minimum spanning tree’s weight. MST-PRIM
algorithm is used to compute the minimum spanning
tree

Superintelligence Laboratory H.CHOO 12/21

 Traveling Salesman Problem: Example

(a) A complete undirected graph. (b) A minimum spanning tree 𝑇 of the graph
computed by MST-PRIM, a is the root vertex. (c) A walk of 𝑇, starting at a. A full
walk of the tree visits the vertices in the order
𝑎; 𝑏; 𝑐; 𝑏; ℎ; 𝑏; 𝑎; 𝑑; 𝑒; 𝑓; 𝑒; 𝑔; 𝑒; 𝑑; 𝑎. A preorder walk of 𝑇 lists a vertex just
when it is first encountered, as indicated by the dot next to each vertex, yielding
the ordering 𝑎; 𝑏; 𝑐; ℎ; 𝑑; 𝑒; 𝑓; 𝑔.

Superintelligence Laboratory H.CHOO 13/21

 Traveling Salesman Problem: Example

(d) A tour obtained by visiting the vertices in the order given by the preorder
walk, which is the tour 𝐻 returned by APPROX-TSP-TOUR. (e) An optimal tour
𝐻 ∗ for the original complete graph

Superintelligence Laboratory H.CHOO 14/21

 Traveling Salesman Problem: Analysis
 APPROX-TSP-TOUR is a 2-approximation algorithm
Proof:
- Let 𝐻 ∗ denote an optimal tour. A spanning tree is
obtained if one edge is removed from the tour. Therefore,
the weight of the minimum spanning tree in line 2 of the
algorithm is a lower bound for the cost of 𝐻 ∗
𝑐(𝑇) ≤ 𝑐(𝐻 ∗ )
- Let 𝑊 denote a full walk of the minimum spanning tree
𝑇. Since the full walk traverses every edge of T exactly
twice
𝑐 𝑊 = 2𝑐(𝑇) ≤ 2𝑐(𝐻 ∗ )

Superintelligence Laboratory H.CHOO 15/21

 Traveling Salesman Problem: Analysis
 APPROX-TSP-TOUR is a 2-approximation algorithm
Proof (continue):
- The tour 𝐻 is obtained by removing vertices from the full
walk 𝑊, so
𝑐(𝐻) ≤ 𝑐(𝑊) ≤ 2𝑐(𝐻 ∗ )
𝑐(𝐻)
∗
≤2
𝑐(𝐻 )

Superintelligence Laboratory H.CHOO 16/21

 Job Scheduling Problem
 Suppose we have 𝑛 jobs, each of which take time 𝑡𝑖 to
process, and 𝑚 identical machines on which we
schedule the jobs. Jobs cannot be split between
machines. For a given scheduling, let 𝐴𝑗 be the set of
jobs assigned to machine 𝑗. Let 𝑇𝑗 = σ𝑖∈𝐴𝑗 𝑡𝑖 be the load
of machine 𝑗. The job scheduling problem asks for an
assignment of jobs to machines that minimizes the
makespan, where the makespan is defined as the
maximum load over all machines.
 Example: 5 jobs with processing time {3,3,2,2,2}, and 2
machines
 The optimal schedule: {3,3} and {2,2,2}, makespan = 6

Superintelligence Laboratory H.CHOO 17/21

 Job Scheduling Problem: Algorithm
 Consider the following greedy algorithm which
iteratively allocates the next job to the machine with
the least load.

𝐴: sets of jobs assigned to machines

𝑇: loads of machines
𝑡: processing times of jobs
𝑚: number of machines
𝑛: number of jobs

Superintelligence Laboratory H.CHOO 18/21

 Job Scheduling Problem: Example
 We are given 5 jobs with processing time {3,3,2,2,2},
and 2 machines
 The optimal schedule: {3,3} and {2,2,2}, makespan = 6
 The schedule given by GREEDY-JOB-SCHEDULING: {3,2,2} and
{3,2}, makespan = 7

Superintelligence Laboratory H.CHOO 19/21

 Job Scheduling Problem: Analysis
 GREEDY-JOB-SCHEDULING is a 2-approximation
algorithm
Proof:
- Let 𝑇 ∗ denote the optimal makespan, then:

𝑇 ∗ ≥ max 𝑡𝑖
𝑖
𝑛
1
𝑇∗ ≥ ෍ 𝑡𝑖
𝑚
𝑖

Superintelligence Laboratory H.CHOO 20/21

 Job Scheduling Problem: Analysis
 GREEDY-JOB-SCHEDULING is a 2-approximation
algorithm
Proof (continue):
- Consider machine 𝑗 with maximum load 𝑇𝑗 . Let 𝑖 be the
last job scheduled on machine 𝑗. Before 𝑖 was scheduled, 𝑗
had the smallest load, so 𝑗 must have had load smaller
than the average load. Then,
𝑛
1
𝑇𝑗 = 𝑇𝑗 − 𝑡𝑖 + 𝑡𝑖 ≤ ෍ 𝑡𝑖 + max 𝑡𝑖 ≤ 𝑇 ∗ + 𝑇 ∗ = 2𝑇 ∗
𝑚 𝑖
𝑖

Superintelligence Laboratory H.CHOO 21/21

 Thanks to contributors
Mr. Pham Van Nguyen (2022)
Prof. Hyunseung Choo (2001 – 2022)

Superintelligence Laboratory Copyright 2000-2022 Superintelligence

H.CHOOLaboratory
22/21