CSCE 310J CSCE 310J

Data Structures & Algorithms Data Structures & Algorithms

‹ Giving credit where credit is due:
» Most of the lecture notes are based on slides
P, NP, and NP-Complete created by Dr. Cusack and Dr. Leubke.
» I have modified them and added new slides
Dr. Steve Goddard
[email protected]

http://www.cse.unl.edu/~goddard/Courses/CSCE310J

1 2

Tractability Polynomial-Time Algorithms

‹ Some problems are intractable: ‹ Are some problems solvable in polynomial time?
as they grow large, we are unable to solve them in » Of course: every algorithm we’ve studied provides
reasonable time polynomial-time solution to some problem
‹ What constitutes reasonable time? Standard ‹ Are all problems solvable in polynomial time?
working definition: polynomial time » No: Turing’s “Halting Problem” is not solvable by any
» On an input of size n the worst-case running time is computer, no matter how much time is given
O(nk) for some constant k ‹ Most problems that do not yield polynomial-time
» Polynomial time: O(n2), O(n3), O(1), O(n lg n) algorithms are either optimization or decision
» Not in polynomial time: O(2n), O(nn), O(n!) problems.

3 4

Optimization/Decision Optimization/Decision
Problems Problems
‹ Optimization Problems: ‹ An optimization problem tries to find an optimal solution
» An optimization problem is one which asks, “What is the optimal ‹ A decision problem tries to answer a yes/no question
solution to problem X?” ‹ Many problems will have decision and optimization
» Examples: versions.
™ 0-1 Knapsack » Eg: Traveling salesman problem
™ Fractional Knapsack ™ optimization: find hamiltonian cycle of minimum weight
™ Minimum Spanning Tree ™ decision: find hamiltonian cycle of weight < k
‹ Decision Problems ‹ Some problems are decidable, but intractable:
» An decision problem is one which asks, “Is there a solution to as they grow large, we are unable to solve them in
problem X with property Y?” reasonable time
» Examples: » What constitutes “reasonable time”?
™ Does a graph G have a MST of weight ≤ W? » Is there a polynomial-time algorithm that solves the problem?

5 6

Page 1
The Class P Sample Problems in P
P: the class of decision problems that have ‹ Fractional Knapsack
polynomial-time deterministic algorithms. ‹ MST
» That is, they are solvable in O(p(n)), where p(n) is a
polynomial on n ‹ Single-source shortest path
» A deterministic algorithm is (essentially) one that ‹ Sorting
always computes the correct answer
‹ Others?

Why polynomial?
» if not, very inefficient
» nice closure properties
» machine independent in a strong sense

7 8

The class NP Nondeterminism

NP: the class of decision problems that are solvable in ‹ Think of a non-deterministic computer as a
polynomial time on a nondeterministic machine (or with a
non-deterministic algorithm) computer that magically “guesses” a solution, then
‹ (A determinstic computer is what we know)
has to verify that it is correct
» If a solution exists, computer always guesses it
‹ A nondeterministic computer is one that can “guess” the
right answer or solution » One way to imagine it: a parallel computer that can
» Think of a nondeterministic computer as a parallel machine that freely spawn an infinite number of processes
can freely spawn an infinite number of processes ™ Have one processor work on each possible solution
‹ Thus NP can also be thought of as the class of problems ™ All processors attempt to verify that their solution works
™ If a processor finds it has a working solution
» whose solutions can be verified in polynomial time; or
» that can be solved in polynomial time on a machine that can pursue » So: NP = problems verifiable in polynomial time
infinitely many paths of the computation in parallel
‹ Note that NP stands for “Nondeterministic Polynomial-
time”
9 10

Sample Problems in NP Hamiltonian Cycle

‹ Fractional Knapsack ‹ A hamiltonian cycle of an undirected graph is a
‹ MST simple cycle that contains every vertex
‹ Single-source shortest path ‹ The hamiltonian-cycle problem: given a graph G,
‹ Sorting does it have a hamiltonian cycle?
‹ Others? ‹ Describe a naïve algorithm for solving the
» Hamiltonian Cycle (Traveling Sales Person) hamiltonian-cycle problem. Running time?
» Satisfiability (SAT)
‹ The hamiltonian-cycle problem is in NP:
» Conjunctive Normal Form (CNF) SAT
» No known deterministic polynomial time algorithm
» 3-CNF SAT
» Easy to verify solution in polynomial time (How?)

11 12

Page 2
The Satisfiability (SAT) Conjunctive Normal Form
Problem (CNF) and 3-CNF
‹ Satisfiability (SAT): ‹ Even if the form of the Boolean expression is
» Given a Boolean expression on n variables, can we
simplified, no known polynomial time algorithm exists
assign values such that the expression is TRUE? » Literal: an occurrence of a Boolean or its negation
» Ex: ((x1 →x2) ∨ ¬((¬x1 ↔ x3) ∨ x4)) ∧¬x2 » A Boolean formula is in conjunctive normal form, or CNF, if
it is an AND of clauses, each of which is an OR of literals
» Seems simple enough, but no known deterministic
™ Ex: (x1 ∨ ¬x2) ∧ (¬x1 ∨ x3 ∨ x4) ∧ (¬x5)
polynomial time algorithm exists
» Easy to verify in polynomial time!
» 3-CNF: each clause has exactly 3 distinct literals
™ Ex: (x1 ∨ ¬x2 ∨ ¬x3) ∧ (¬x1 ∨ x3 ∨ x4) ∧ (¬x5 ∨ x3 ∨ x4)
™ Notice: true if at least one literal in each clause is true

13 14

Example: CNF satisfiability Review: P And NP Summary

‹ This problem is in NP. Nondeterministic algorithm: ‹ P = set of problems that can be solved in
» Guess truth assignment
» Check assignment to see if it satisfies CNF formula
polynomial time
» Examples: Fractional Knapsack, …
‹ Example: ‹ NP = set of problems for which a solution can be
(A∨¬B ∨ ¬C ) ∧ (¬A ∨ B) ∧ (¬ B ∨ D ∨ F ) ∧ (F ∨ ¬ D) verified in polynomial time
‹ Truth assignments:
ABCDEF
» Examples: Fractional Knapsack,…, Hamiltonian Cycle,
1. 0 1 1 0 1 0 CNF SAT, 3-CNF SAT
2. 1 0 0 0 0 1 ‹ Clearly P ⊆ NP
3. 1 1 0 0 0 1
4. ... (how many more?) ‹ Open question: Does P = NP?
» Most suspect not
Checking phase: Θ(n)
15 16

NP-complete problems Review: Reduction

‹ A decision problem D is NP-complete iff ‹ A problem P can be reduced to another problem Q if
1. D ∈ NP any instance of P can be rephrased to an instance of Q,
2. every problem in NP is polynomial-time reducible to D the solution to which provides a solution to the instance
of P
‹ Cook’s theorem (1971): CNF-sat is NP-complete » This rephrasing is called a transformation
‹ Intuitively: If P reduces in polynomial time to Q, P is
‹ Other NP-complete problems obtained through “no harder to solve” than Q
polynomial-time reductions of known NP-
complete problems

17 18

Page 3
NP-Hard and NP-Complete Proving NP-Completeness
‹ If P is polynomial-time reducible to Q, we denote ‹ What steps do we have to take to prove a problem
this P ≤p Q Q is NP-Complete?
‹ Definition of NP-Hard and NP-Complete: » Pick a known NP-Complete problem P
» If all problems R ∈ NP are reducible to P, then P is NP- » Reduce P to Q
Hard ™ Describe a transformation that maps instances of P to instances
of Q, s.t. “yes” for Q = “yes” for P
» We say P is NP-Complete if P is NP-Hard
™ Prove the transformation works
and P ∈ NP
™ Prove it runs in polynomial time
‹ If P ≤p Q and P is NP-Complete, Q is also » Oh yeah, prove Q ∈ NP (What if you can’t?)
NP-Complete

19 20

Directed Hamiltonian Cycle ⇒ Directed Hamiltonian Cycle ⇒

Undirected Hamiltonian Cycle Undirected Hamiltonian Cycle
‹ What was the hamiltonian cycle problem again?
y1 y2 y3
‹ For my next trick, I will reduce the directed
hamiltonian cycle problem to the undirected y x1 x2 x3
hamiltonian cycle problem before your eyes x

» Which variant am I proving NP-Complete?

u v
‹ Given a directed graph G,
» What transformation do I need to effect?
v1 v2 v3
u1 u2 u3

21 22

Transformation: Undirected Hamiltonian

Directed ⇒ Undirected Ham. Cycle Cycle
‹ Transform graph G = (V, E) into G’ = (V’, E’): ‹ Thus we can reduce the directed problem to the
» Every vertex v in V transforms into 3 vertices undirected problem
v1, v2, v3 in V’ with edges (v1,v2) and (v2,v3) in E’
‹ What’s left to prove the undirected hamiltonian
» Every directed edge (v, w) in E transforms into the
undirected edge (v3, w1) in E’ (draw it) cycle problem NP-Complete?
» Can this be implemented in polynomial time? ‹ Argue that the problem is in NP
» Argue that a directed hamiltonian cycle in G implies an
undirected hamiltonian cycle in G’
» Argue that an undirected hamiltonian cycle in G’
implies a directed hamiltonian cycle in G

23 24

Page 4
Hamiltonian Cycle ⇒ TSP Hamiltonian Cycle ⇒ TSP
‹ The well-known traveling salesman problem: ‹ The steps to prove TSP is NP-Complete:
» Optimization variant: a salesman must travel to n cities,
» Prove that TSP ∈ NP (Argue this)
visiting each city exactly once and finishing where he
begins. How to minimize travel time? » Reduce the undirected hamiltonian cycle problem to the
TSP
» Model as complete graph with cost c(i,j) to go from city i to
™ So if we had a TSP-solver, we could use it to solve the
city j hamilitonian cycle problem in polynomial time
‹ How would we turn this into a decision problem? ™ How can we transform an instance of the hamiltonian cycle
problem to an instance of the TSP?
» A: ask if ∃ a TSP with cost < k
™ Can we do this in polynomial time?

25 26

The TSP Review: P and NP

‹ Random asides: ‹ What do we mean when we say a problem
» TSPs (and variants) have enormous practical is in P?
importance
» A: A solution can be found in polynomial time
™ E.g., for shipping and freighting companies
™ Lots of research into good approximation algorithms ‹ What do we mean when we say a problem
» Recently made famous as a DNA computing problem is in NP?
» A: A solution can be verified in polynomial time
‹ What is the relation between P and NP?
» A: P ⊆ NP, but no one knows whether P = NP

27 28

Review:
Review: NP-Complete Proving Problems NP-Complete
‹ What, intuitively, does it mean if we can reduce
‹ How do we usually prove that a problem R
problem P to problem Q?
is NP-Complete?
» P is “no harder than” Q
» A: Show R ∈NP, and reduce a known
‹ How do we reduce P to Q? NP-Complete problem Q to R
» Transform instances of P to instances of Q in polynomial
time s.t. Q: “yes” iff P: “yes”
‹ What does it mean if Q is NP-Hard?
» Every problem P∈NP ≤p Q
‹ What does it mean if Q is NP-Complete?
» Q is NP-Hard and Q ∈ NP

29 30

Page 5
Other NP-Complete
Problems General Comments
‹ K-clique
» A clique is a subset of vertices fully connected to each other, i.e. a ‹ Literally hundreds of problems have been shown
complete subgraph of G to be NP-Complete
» The clique problem: how large is the maximum-size clique in a
graph? ‹ Some reductions are profound, some are
» No turn this into a decision problem? comparatively easy, many are easy once the key
» Is there a clique of size k?
insight is given
‹ Subset-sum: Given a set of integers, does there exist a
subset that adds up to some target T?
‹ 0-1 knapsack: when weights not just integers
‹ Hamiltonian path: Obvious
‹ Graph coloring: can a given graph be colored with k colors
such that no adjacent vertices are the same color?
‹ Etc…
31 32

Page 6