lOMoARcPSD|37374107

AA Unit 5 Notes

Advanced Algorithms (Jawaharlal Nehru Technological University, Hyderabad)

Scan to open on Studocu

Studocu is not sponsored or endorsed by any college or university

Downloaded by kisara rishitha ([email protected])
 lOMoARcPSD|37374107

ADVANCED ALGORITHMS
UNIT V
5.1. NP Completeness
Almost all the algorithms we have studied thus far have been polynomial -time
algorithms: on inputs of size n, their worst-case running time is O(nk) for some constant k.
You might wonder whether all problems can be solved in polynomial time. The answer is
no.
Generally, we think of problems that are solvable by polynomial-time algorithms as
being tractable, or easy, and problems that require superpolynomial time as being
intractable, or hard.
The subject of this chapter, however, is an interesting class of problems, called the
“NP-complete” problems, whose status is unknown. No polynomial-time algorithm has yet
been discovered for an NP-complete problem, nor has anyone yet been able to prove that
no polynomial-time algorithm can exist for any one of them.
Several NP-complete problems are particularly tantalizing because they seem on the
surface to be similar to problems that we know how to solve in polynomial time. In each of
the following pairs of problems, one is solvable in polynomial time and the other is NP-
complete, but the difference between problems appears to be slight:
 Shortest vs. longest simple paths
 Euler tour vs. hamiltonian cycle
 2-CNF satisfiability vs. 3-CNF satisfiability
The class P consists of those problems that are solvable in polynomial time. More
specifically, they are problems that can be solved in time O(nk) for some constant k, where
n is the size of the input to the problem. Most of the problems examined so far are in P.
The class NP consists of those problems that are “verifiable” in polynomial time.
What do we mean by a problem being verifiable? If we were somehow given a “certificate”
of a solution, then we could verify that the certificate is correct in time polynomial in the
size of the input to the problem.
Any problem in P is also in NP, since if a problem is in P then we can solve it in
polynomial time without even being supplied a certificate.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

A problem is in the class NP Complete if it is in NP and is as “hard” as any problem

in NP.
Many problems of interest are optimization problems, in which each feasible
solution has an associated value, and we wish to find a feasible solution with the best value.
NP Completeness applies directly not to optimization problems, however, but to
decision problems, in which the answer is simply “yes” or “no” (or, more formally, “1” or
“0”).
We call such a procedure a polynomial-time reduction algorithm and, as the
following figure shows, it provides us a way to solve problem A in polynomial time:
1. Given an instance α of problem A, use a polynomial-time reduction algorithm
to transform it to an instance β of problem B.
2. Run the polynomial-time decision algorithm for B on the instance β.
3. Use the answer for β as the answer for α.

In other words, by “reducing” solving problem A to solving problem B, we use the

“easiness” of B to prove the “easiness” of A.

5.1.1. Polynomial Time

To understand the class of polynomial-time solvable problems, we must first have a
formal notion of what a “problem” is. We define an abstract problem Q to be a binary
relation on a set I of problem instances and a set S of problem solutions.
This formulation of an abstract problem is more general than we need for our
purposes. As we saw above, the theory of NP-completeness restricts attention to decision
problems: those having a yes/ no solution. In this case, we can view an abstract decision
problem as a function that maps the instance set I to the solution set {0, 1}.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

In order for a computer program to solve an abstract problem, we must represent

problem instances in a way that the program understands. An encoding of a set S of
abstract objects is a mapping e from S to the set of binary strings.
For example, we are all familiar with encoding the natural numbers N = {0, 1, 2, 3, 4,
. . . } as the strings {0, 1, 10, 11, 100, . . . }. Using this encoding, e(17) = 10001.
Thus, a computer algorithm that “solves” some abstract decision problem actually
takes an encoding of a problem instance as input. We call a problem whose instance set is
the set of binary strings a concrete problem.
A concrete problem is polynomial-time solvable, therefore, if there exists an
algorithm to solve it in time O(nk) for some constant k.
We can now formally define the complexity class P as the set of concrete decision
problems that are polynomial-time solvable.

5.1.2. Polynomial Time Verification

Hamiltonian cycles
The problem of finding a hamiltonian cycle in an undirected graph has been studied
for over a hundred years. Formally, a hamiltonian cycle of an undirected graph G= (V, E) is
a simple cycle that contains each vertex in V. A graph that contains a hamiltonian cycle is
said to be hamiltonian; otherwise, it is nonhamiltonian.
Following figure (a) shows one hamiltonian cycle. Not all graphs are hamiltonian,
however. For example, figure (b) shows a bipartite graph with an odd number of vertices.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

We can define the hamiltonian-cycle problem, “Does a graph G have a hamiltonian

cycle?” as a formal language:
HAM-CYCLE = {<G> : G is a hamiltonian graph}.
How might an algorithm decide the language HAM-CYCLE? Given a problem
instance <G>, one possible decision algorithm lists all permutations of the vertices of G and
then checks each permutation to see if it is a hamiltonian path.

Verification algorithms
We define a verification algorithm as being a two-argument algorithm A, where
one argument is an ordinary input string x and the other is a binary string y called a
certificate. A two-argument algorithm A verifies an input string x if there exists a
certificate y such that A(x, y) = 1.

The complexity class NP

The complexity class NP is the class of languages that can be verified by a
polynomial-time algorithm.

5.1.3. NP Completeness and Reducibility

Reducibility
A problem Q can be reduced to another problem Q’ if any instance of Q can be
“easily rephrased” as an instance of Q’, the solution to which provides a solution to the
instance of Q.
For example, the problem of solving linear equations in an indeterminate x reduces
to the problem of solving quadratic equations. Given an instance ax + b = 0, we transform it
to 0 x2 + ax + b = 0, whose solution provides a solution to ax + b = 0. Thus, if a problem Q
reduces to another problem Q’, then Q is, in a sense, “no harder to solve” than Q’.
Returning to our formal-language framework for decision problems, we say that a
language L1 is polynomial-time reducible to a language L2, written L1 ≤P L2, if there
exists a polynomial-time computable function f : {0, 1}*  {0, 1}* such that for all x {0,
1}*,
x L1 if and only if f(x) L2.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

We call the function f the reduction function, and a polynomial-time algorithm F

that computes f is a reduction algorithm.

NP-completeness
Polynomial-time reductions provide a formal means for showing that one problem
is at least as hard as another, to within a polynomial-time factor.
A language L ⊆{0, 1}* is NP-complete if
1. L NP, and
2. L’ ≤P L for every L1 NP.
If a language L satisfies property 2, but not necessarily property 1, we say that L is
NP-hard. We also define NPC to be the class of NP-complete languages.

Circuit satisfiability
Boolean combinational circuits are built from boolean combinational el ements that
are interconnected by wires. A boolean combinational element is any circuit element
that has a constant number of boolean inputs and outputs and that performs a well-defined
function. Boolean values are drawn from the set {0, 1}, where 0 represents FALSE and 1
represents TRUE.
The boolean combinational elements that we use in the circuit -satisfiability problem
compute simple boolean functions, and they are known as logic gates. Following figure
shows the three basic logic gates that we use in the circuit-satisfiability problem: the NOT
gate (or inverter), the AND gate, and the OR gate.
We can describe the operation of each gate, and of any boolean combinational
element, by a truth table, shown under each gate in Figure.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

A boolean combinational circuit consists of one or more boolean combinational

elements interconnected by wires. A wire can connect the output of one element to the
input of another, thereby providing the output value of the first element as an input value
of the second. Following figure shows two similar boolean combinational circuits, differing
in only one gate.

A truth assignment for a boolean combinational circuit is a set of boolean input

values. We say that a one-output boolean combinational circuit is satisfiable if it has a
satisfying assignment: a truth assignment that causes the output of the circuit to be 1.
The circuit-satisfiability problem is, “Given a boolean combinational circuit
composed of AND, OR, and NOT gates, is it satisfiable?”
As a formal language, we can therefore define
CIRCUIT-SAT = {<C> : C is a satisfiable boolean combinational circuit}.
Given a circuit C, we might attempt to determine whether it is satisfiable by simply
checking all possible assignments to the inputs.
5.1.4. NP Complete Problems
The structure of NP-completeness proofs is given below in the figure. All proofs
ultimately follow by reduction from the NP-completeness of CIRCUIT-SAT.
The clique problem
A clique in an undirected graph G = (V, E) is a subset V’ ⊆V of vertices, each pair of
which is connected by an edge in E. In other words, a clique is a complete subgraph of G.
The size of a clique is the number of vertices it contains.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

The clique problem is the optimization problem of finding a clique of maximum

size in a graph. As a decision problem, we ask simply whether a clique of a given size k
exists in the graph. The formal definition is
CLIQUE = {<G, k> : G is a graph containing a clique of size k}.
The clique problem is NP-complete.

The vertex-cover problem

A vertex cover of an undirected graph G=(V,E) is a subset V’ V such that if (u, v)
E, then u V’ or v V’ (or both). That is, each vertex “covers” its incident edges, and a vertex
cover for G is a set of vertices that covers all the edges in E. The size of a vertex cover is the
number of vertices in it.

Figure (a) is an undirected graph G= (V, E) with clique V’ = {u, v, x, y}. (b) The graph
G produced by the reduction algorithm that has vertex cover V – V’ = {w, z}.
The vertex-cover problem is to find a vertex cover of minimum size in a given
graph.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

For example, the graph in following figure (b) has a vertex cover {w, z} of size 2.
As a language, we define
VERTEX-COVER = {<G, k> : graph G has a vertex cover of size k}.
The vertex-cover problem is NP-complete.

The hamiltonian-cycle problem

The hamiltonian cycle problem is NP-complete. And this problem is discussed in
5.1.2.

The traveling-salesman problem

In the traveling-salesman problem, which is closely related to the
hamiltoniancycle problem, a salesman must visit n cities. Modeling the problem as a
complete graph with n vertices, we can say that the salesman wishes to make a tour, or
hamiltonian cycle, visiting each city exactly once and finishing at the city he starts from.
The salesman incurs a nonnegative integer cost c(i, j) to travel from city I to city j,
and the salesman wishes to make the tour whose total cost is minimum, where the total
cost is the sum of the individual costs along the edges of the tour. For example, in the
following figure, a minimum-cost tour is {u, w, v, x, u}, with cost 7.
The traveling-salesman problem is NP-complete.

The subset-sum problem

In the subset-sum problem, we are given a finite set S of positive integers and an
integer target t > 0. We ask whether there exists a subset S’ ⊆S whose elements sum to t.
For example, if S = {1, 2, 7, 14, 49, 98, 343, 686, 2409, 2793, 16808, 17206, 117705,
117993} and t = 138457, then the subset S’ = {1, 2, 7, 98, 343, 686, 2409, 17206, 117705} is
a solution.
The subset-sum problem is NP-complete.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

5.2. Approximation Algorithms

Even if a problem is NP-complete, there may be hope. We have at least three ways to
get around NP-completeness.
First, if the actual inputs are small, an algorithm with exponential running time may
be perfectly satisfactory.
Second, we may be able to isolate important special cases that we can solve in
polynomial time.
Third, we might come up with approaches to find near -optimal solutions in
polynomial time (either in the worst case or the expected case).
In practice, nearoptimality is often good enough. We call an algorithm that returns
near-optimal solutions an approximation algorithm.
We say that an algorithm for a problem has an approximation ratio of (n) if, for
any input of size n, the cost C of the solution produced by the algorithm is within a factor of
(n) of the cost C* of an optimal solution:
max(C/ C*, C*/ C) ≤ n).
If an algorithm achieves an approximation ratio of (n), we call it a (n)-
approximation algorithm. The definitions of the approximation ratio and of a (n)-
approximation algorithm apply to both minimization and maximization problems.
An approximation scheme for an optimization problem is an approximation
algorithm that takes as input not only an instance of the problem, but also a value >0
such that for any fixed , the scheme is a (1 + ϵ)-approximation algorithm.
We say that an approximation scheme is a polynomial-time approximation
scheme if for any fixed > 0, the scheme runs in time polynomial in the size n of its input
instance.
The running time of a polynomial-time approximation scheme can increase very
rapidly as decreases.
We say that an approximation scheme is a fully polynomial-time approximation
scheme if it is an approximation scheme and its running time is polynomial in both 1/ and
the size n of the input instance.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

5.2.1. Vertex Cover Problem

A vertex cover of an undirected graph G = (V, E) is a subset V’ ⊆V such that if (u, v)
is an edge of G, then either u V’ or v V’ (or both). The size of a vertex cover is the number
of vertices in it.
The vertex-cover problem is to find a vertex cover of minimum size in a given
undirected graph. We call such a vertex cover an optimal vertex cover. This problem is the
optimization version of an NP-complete decision problem.
Even though we don’t know how to find an optimal vertex cover in a graph G in
polynomial time, we can efficiently find a vertex cover that is near -optimal. The following
approximation algorithm takes as input an undirected graph G and returns a vertex cover
whose size is guaranteed to be no more than twice the size of an optimal vertex cover.
APPROX-VERTEX-COVER(G)
1 C= ∅
2 E’ = G, E
3 while E’ ≠ ∅
4 let (u, v) be an arbitrary edge of E’
5 C= C∪ {u, v}
6 remove from E’ every edge incident on either u or v
7 return C

The following figure illustrates how APPROX-VERTEX-COVER operates on an

example graph. The variable C contains the vertex cover being constructed. Line 1
initializes C to the empty set. Line 2 sets E’ to be a copy of the edge set G.E of the graph. The
loop of lines 3–6 repeatedly picks an edge (u, v) from E’, adds its endpoints u and v to C,
and deletes all edges in E’ that are covered by either u or v. Finally, line 7 returns the vertex
cover C. The running time of this algorithm is O(V + E), using adjacency lists to represent E’.

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

The operation of APPROX-VERTEX-COVER illustrated in the above figure. (a) The

input graph G, which has 7 vertices and 8 edges. (b) The edge (b, c), shown heavy, is the
first edge chosen by APPROX-VERTEXCOVER. Vertices b and c, shown lightly shaded, are
added to the set C containing the vertex cover being created. Edges (a, b), (c, e), and (c, d),
shown dashed, are removed since they are now covered by some vertex in C. (c) Edge (e, f)
is chosen; vertices e and f are added to C. (d) Edge (d, g) is chosen; vertices d and g are
added to C. (e) The set C, which is the vertex cover produced by APPROX-VERTEX-COVER,
contains the six vertices b, c, d, e, f, g. (f) The optimal vertex cover for this problem contains
only three vertices: b, d, and e.
APPROX-VERTEX-COVER is a polynomial-time 2-approximation algorithm.

5.2.2. Travelling Sales Person Problem

In the traveling-salesman problem, we are given a complete undirected graph G =
(V, E) that has a nonnegative integer cost c(u, v) associated with each edge (u, v) E, and
we must find a hamiltonian cycle (a tour) of G with minimum cost. As an extension of our
notation, let c(A) denote the total cost of the edges in the subset A ⊆E:

In many practical situations, the least costly way to go from a place u to a place w is
to go directly, with no intermediate steps. Put another way, cutting out an intermediate
stop never increases the cost. We formalize this notion by saying that the cost function c
satisfies the triangle inequality if, for all vertices u, v, w V, c(u, w) ≤ c(u, v) + c(v, w).

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

The triangle inequality seems as though it should naturally hold, and it is

automatically satisfied in several applications. For example, if the vertices of the graph are
points in the plane and the cost of traveling between two vertices is the ordinary euclidean
distance between them, then the triangle inequality is satisfi ed.
Applying the methodology of the above, we shall first compute a structure - a
minimum spanning tree - whose weight gives a lower bound on the length of an optimal
traveling-salesman tour. We shall then use the minimum spanning tree to create a tour
whose cost is no more than twice that of the minimum spanning tree’s weight, as long as
the cost function satisfies the triangle inequality.
The following algorithm implements this approach, calling the minimum-spanning-
tree algorithm MST-PRIM from Chapter 2 as a subroutine. The parameter G is a complete
undirected graph, and the cost function c satisfies the triangle inequality.

APPROX-TSP-TOUR(G, c)
1 select a vertex r G.V to be a “root” vertex
2 compute a minimum spanning tree T for G from root r using MST-PRIM(G,c,r)
3 let H be a list of vertices, ordered according to when they are first visited in a
preorder tree walk of T
4 return the hamiltonian cycle H

Downloaded by kisara rishitha ([email protected])

 lOMoARcPSD|37374107

The following figure illustrates the operation of APPROX-TSP-TOUR. Part (a) of the
figure shows a complete undirected graph, and part (b) shows the minimum spanning tree
T grown from root vertex a by MST-PRIM. Part (c) shows how a preorder walk of T visits
the vertices, and part (d) displays the corresponding tour, which is the tour returned by
APPROX-TSP-TOUR. Part (e) displays an optimal tour, which is about 23% shorter.
APPROX-TSP-TOUR is a polynomial-time 2-approximation algorithm for the
traveling-salesman problem with the triangle inequality.

Downloaded by kisara rishitha ([email protected])