‫ر‬

‫الجامعة السعودية االلكتونية‬

‫ر‬
‫االلكتونية‬ ‫الجامعة السعودية‬

‫‪26/12/2021‬‬
Theory of Computing

Headline separator Module 12

Additional Classes of Problems
Contents
1. A Restricted Satisfiability Problem
2. Additional NP-Complete Problems
3. Problems Solvable in Polynomial Space
4. A Problem That Is Complete for PS
Weekly Learning Outcomes
1. Explain various complexities of problems.
Required Reading
1. A Restricted Satisfiability Problem
2. Additional NP-Complete Problems
3. Problems Solvable in Polynomial Space
4. A Problem That Is Complete for PS
(Introduction to Automata Theory, Languages, and Computation
(2013) Global Edition 3rd Edition)
Recommended Reading

https://www.diva-portal.org/smash/get/diva2:1087096/FULLTEXT03.pdf
http://infolab.stanford.edu/~ullman/focs/ch03.pdf

This Presentation is mainly dependent on the textbook: Introduction to Automata Theory, Languages, and Computation: Global Edition, 3rd edition (2013) PHI
by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman
• A Restricted Satisfiability Problem
Restricted Satisfiability Problem
• Our plan is to demonstrate a wide variety of problem, such as the TSP
problem to be NP-complete.
• Cook’s theorem showed the first NP complete problem - whether a
boolean expression is satisfiable, by reducing all problems in NP to
the SAT problem in polynomial time.
• In addition, the problem remains NP-complete even if the expression
is restricted to consist of a product of clauses, each of which consists
of only three literals (the problem 3-SAT).
• 3-SAT is an important yet much easier than SAT to reduce to typical
problems.
• 3-SAT is a problem about satisfiability of boolean expression, but
these expressions have a very regular form: they are the AND of
clauses each of which is the OR of exactly three variables or negated
variables.

7
Normal Forms for Boolean Expressions
Three essential definitions:
1. A literal is either a variable, or a negated variable. Examples
are x and ¬y.
2. A clause (or sum of literals) is the logical OR of one or more
literals. Examples are x, x ˅ y, and x ˅ ¬y ˅ z.
3. A boolean expression is said to be in conjunctive normal
form or CNF (or product of clauses), if it is the AND of
clauses.
Note: ‘+’ is used for ˅ and ‘.’ or nothing is written for the AND.
So, the above clauses are x, x + y, etc.

8
What is Restricted Satisfiability?
▪ An expression is said to be in k-conjunctive normal form
(k-CNF) if it is the product of clauses, each of which is the
sum of exactly k distinct literals.
For instance, (x + y) (y + z) (z + x) is in 2-CNF, because each of its
clauses has exactly two literals.
▪ CSAT is the problem: given a boolean expression in CNF, is
it satisfiable?
▪ kSAT is the problem: given a boolean expression in k-CNF,
is it satisfiable?
▪ We shall see that CSAT, 3SAT, and kSAT for all k higher than
3 are NP-complete.
▪ There are linear-time algorithms for 1SAT and 2SAT.

9
NP-Completeness of CSAT
▪ The proof of Cook’s theorem can be modified to produce a
formula in CNF.
▪ Unique is already the AND of clauses.
▪ Starts Right is the AND of clauses, each with one variable.
▪ Finishes Right is the OR of variables, i.e., a single clause.
• Only Moves Right is a problem, and not much of a problem.
• It is the product of formulas for each i and j.
• Those formulas are fixed, independent of n.
• You can convert any formula to CNF.
• It may exponentiate the size of the formula and therefore
take time to write down that is exponential in the size of the
original formula, but these numbers are all fixed for a given
NTM M and independent of n.

10
k-SAT

• If a boolean formula is in CNF and every clause consists of exactly k

literals, we say the boolean formula is an instance of k-SAT.
• Say the formula is in k-CNF.
• Example: 3-SAT formula
(x + y + z)(x + -y + z)(x + y + -z)(x + -y + -z)

11
k-SAT Facts

• Every boolean formula has an equivalent CNF formula.

• But the size of the CNF formula may be exponential in the size of the original.
• Not every boolean formula has a k-SAT equivalent.
• 2SAT is in P; 3SAT is NP-complete.

12
Proof: 2SAT is in P (Sketch)

• Pick an assignment for some variable, say x = true.

• Any clause with –x forces the other literal to be true.
• Example: (-x + -y) forces y to be false.
• Keep seeing what other truth values are forced by variables with
known truth values.

13
Proof (2)
• One of three things can happen:
1. You reach a contradiction (e.g., z is forced to be both true and false).
2. You reach a point where no more variables have their truth value
forced, but some clauses are not yet made true.
3. You reach a satisfying truth assignment.
• Case 1: (Contradiction) There can only be a satisfying assignment if
you use the other truth value for x.
• Simplify the formula by replacing x by this truth value and repeat the
process.
• Case 3: You found a satisfying assignment, so answer “yes.”
• Case 2: (You force values for some variables, but other variables and
clauses are not affected).
• Adopt these truth values, eliminate the clauses that they satisfy, and repeat.
• In Cases 1 and 2 you have spent O(n2) time and have reduced the
length of the formula by > 1, so O(n3) total.

14
3-SAT
▪ 3-SAT: A formula is 3-SAT if, each clause has exactly 3 terms
• Variables x1, . . ., xn
• Clauses C1, . . ., Ck
• Cj = (tj1 or tj2 or tj3)
• This problem is NP-complete.
• Clearly it is in NP, since SAT is.
• It is not true that every Boolean formula can be converted to an
equivalent 3-CNF formula, even if we exponentiate the size of the
formula.
▪ Fact: Every instance of SAT can be converted in polynomial time to an
equivalent instance of 3-SAT.
• But we don’t need equivalence.
• We need to reduce every CNF formula F to some 3-CNF formula that is
satisfiable if and only if F is.
• Reduction involves introducing new variables into long clauses, so we
can split them apart.

15
Reduction of CSAT to 3SAT
• Let (x1+…+xn) be a clause in some CSAT instance, with n > 4.
• Note: the x’s are literals, not variables; any of them could be negated
variables.
• Introduce new variables y1,…,yn-3 that appear in no other clause.
• Replace (x1+…+xn) by (x1+x2+y1)(x3+y2+ -y1) … (xi+yi-1+ -yi-2) … (xn-
2+yn-3+ -yn-4)(xn-1+xn+ -yn-3)
• If there is a satisfying assignment of the x’s for the CSAT instance,
then one of the literals xi must be made true.
• Assign yj = true if j < i-1 and yj = false for larger j.
• We are not done.
• We also need to show that if the resulting 3SAT instance is satisfiable,
then the original CSAT instance was satisfiable.

16
CSAT to 3SAT (2)
• Suppose (x1+x2+y1)(x3+y2+ -y1) … (xn-2+yn-3+ -yn-4)(xn-1+xn+ -yn-3)
is satisfiable, but none of the x’s is true.
• The first clause forces y1 = true.
• Then the second clause forces y2 = true.
• And so on … all the y’s must be true.
• But then the last clause is false.
• There is a little more to the reduction, for handling clauses of 1
or 2 literals.
• Replace (x) by (x+y1+y2) (x+y1+ -y2) (x+ -y1+y2) (x+ -y1+ -y2).
• Replace (w+x) by (w+x+y)(w+x+ -y).
• Remember: the y’s are different variables for each CNF clause.

17
CSAT to 3SAT Running Time

• This reduction is surely polynomial.

• In fact it is linear in the length of the CSAT instance.
• Thus, we have polytime-reduced CSAT to 3SAT.
• Since CSAT is NP-complete, so is 3SAT.

18
NP-Completeness of CSAT

• The proof of Cook’s theorem can be modified to produce a formula in

CNF.
• Unique is already the AND of clauses.
• Starts Right is the AND of clauses, each with one variable.
• Finishes Right is the OR of variables, i.e., a single clause.

19
NP-Completeness of CSAT (1)

• Only Moves Right is a problem, and not much of a problem.

• It is the product of formulas for each i and j.
• Those formulas are fixed, independent of n.

20
NP-Completeness of CSAT (2)

• You can convert any formula to CNF.

• It may exponentiate the size of the formula and therefore take time
to write down that is exponential in the size of the original formula,
but these numbers are all fixed for a given NTM M and independent
of n.

21
k-SAT

• If a boolean formula is in CNF and every clause consists of exactly k

literals, we say the boolean formula is an instance of k-SAT.
• Say the formula is in k-CNF.
• Example: 3-SAT formula
(x + y + z)(x + -y + z)(x + y + -z)(x + -y + -z)

22
k-SAT Facts

• Every boolean formula has an equivalent CNF formula.

• But the size of the CNF formula may be exponential in the size of the original.
• Not every boolean formula has a k-SAT equivalent.
• 2SAT is in P; 3SAT is NP-complete.

23
Proof: 2SAT is in P (Sketch)

• Pick an assignment for some variable, say x = true.

• Any clause with –x forces the other literal to be true.
• Example: (-x + -y) forces y to be false.
• Keep seeing what other truth values are forced by variables with
known truth values.

24
Proof (2)
• One of three things can happen:
1. You reach a contradiction (e.g., z is forced to be both true and false).
2. You reach a point where no more variables have their truth value
forced, but some clauses are not yet made true.
3. You reach a satisfying truth assignment.
• Case 1: (Contradiction) There can only be a satisfying assignment if
you use the other truth value for x.
• Simplify the formula by replacing x by this truth value and repeat the
process.
• Case 3: You found a satisfying assignment, so answer “yes.”
• Case 2: (You force values for some variables, but other variables and
clauses are not affected).
• Adopt these truth values, eliminate the clauses that they satisfy, and repeat.
• In Cases 1 and 2 you have spent O(n2) time and have reduced the
length of the formula by > 1, so O(n3) total.

25
3SAT

• This problem is NP-complete.

• Clearly it is in NP, since SAT is.
• It is not true that every Boolean formula can be converted to an
equivalent 3-CNF formula, even if we exponentiate the size of the
formula.
• But we don’t need equivalence.
• We need to reduce every CNF formula F to some 3-CNF formula that
is satisfiable if and only if F is.
• Reduction involves introducing new variables into long clauses, so we
can split them apart.

26
Reduction of CSAT to 3SAT
• Let (x1+…+xn) be a clause in some CSAT instance, with n > 4.
• Note: the x’s are literals, not variables; any of them could be negated
variables.
• Introduce new variables y1,…,yn-3 that appear in no other clause.
• Replace (x1+…+xn) by (x1+x2+y1)(x3+y2+ -y1) … (xi+yi-1+ -yi-2) … (xn-2+yn-
3+ -yn-4)(xn-1+xn+ -yn-3)
• If there is a satisfying assignment of the x’s for the CSAT instance,
then one of the literals xi must be made true.
• Assign yj = true if j < i-1 and yj = false for larger j.
• We are not done.
• We also need to show that if the resulting 3SAT instance is satisfiable,
then the original CSAT instance was satisfiable.

27
CSAT to 3SAT (2)
• Suppose (x1+x2+y1)(x3+y2+ -y1) … (xn-2+yn-3+ -yn-4)(xn-1+xn+ -yn-3)
is satisfiable, but none of the x’s is true.
• The first clause forces y1 = true.
• Then the second clause forces y2 = true.
• And so on … all the y’s must be true.
• But then the last clause is false.
• There is a little more to the reduction, for handling clauses of 1 or 2
literals.
• Replace (x) by (x+y1+y2) (x+y1+ -y2) (x+ -y1+y2) (x+ -y1+ -y2).
• Replace (w+x) by (w+x+y)(w+x+ -y).
• Remember: the y’s are different variables for each CNF clause.

28
CSAT to 3SAT Running Time
• This reduction is surely polynomial.
• In fact it is linear in the length of the CSAT instance.
• Thus, we have polytime-reduced CSAT to 3SAT.
• Since CSAT is NP-complete, so is 3SAT.

29
• Additional NP-Complete Problems
Additional NP-Complete Problems
▪ There is a vast collection of known NP-complete problems each is
proved NP-complete by a polynomial-time reduction from some
previously known NP-complete problem.
▪ We have given reductions that show the following problems NP-
complete:
▪ independent set
▪ node cover
▪ directed and undirected versions of the Hamilton circuit problem
▪ the traveling-salesman problem.
▪ This process
▪ The process of discovering new NP-complete problems has two
important aspects:
▪ When we discover a problem to be NP-complete, it tells us that there is
little chance an efficient algorithm can be developed to solve it.
▪ Each time we add a new NP-complete problem P to the list, we
reenforce the idea that all NP-complete problems require exponential
time.

31
Describing NP-complete Problems
▪ As we introduce new NP-complete problems, we shall use a
stylized form of definition:
1. The name of the problem, and usually an abbreviation like 3SAT or
TSP.
2. The input to the problem: what is represented, and how.
3. The output desired: under what circumstances should the output
be “yes”?
4. The problem from which a reduction is made to prove the problem
NP-Complete.

32
The Problem of Independent Sets
• Let G be an undirected graph. We say a subset I of the
nodes of G is an independent set if no two nodes of I are
connected by an edge of G.
• An independent set is maximal if it is as large (has as many
nodes) as any independent set for the same graph.
• Example: In the adjacent, figure {1, 4} is the maximal
independent set. It is the only solution.
• {1} is an independent set but not maximal.

33
The Node Cover Problem

• Given a graph G, we say N is a node cover for G if every edge of G

has at least one end in N.
• The problem Node Cover is: given a graph G and a “budget” k, does G
have a node cover of k or fewer nodes?

34
The formal definition of the MIS problem

• PROBLEM: Independent Set (IS)

• INPUT: A graph G and a lower bound k, which must be between 1 and
the number of nodes of G.
• OUTPUT: “Yes” if and only if G has an independent set of k nodes.
• REDUCTION FROM: 3SAT.

35
Example: Node Cover

A B C D

E F

One possible node cover

of size 3: {B, C, E}

36
NP-Completeness of Node Cover

• Reduction from 3SAT.

• For each clause (X+Y+Z) construct a “column” of three nodes, all
connected by vertical edges.
• Add a horizontal edge between nodes that represent any variable
and its negation.
• Budget = twice the number of clauses.

37
 Example: Reduction – (3)

(x + y + z)(-x + -y + -z)(x + -y +z)(-x + y + -z)

Truth assignment: x = y = T; z = F
Pick a true node in each column

x -x x -x

y -y -y y

z -z z -z

38
 Example: Reduction – (4)

(x + y + z)(-x + -y + -z)(x + -y +z)(-x + y + -z)

Truth assignment: x = y = T; z = F
The other nodes form a node cover

x -x x -x

y -y -y y

z -z z -z

39
Hamiltonian-Circuit Problem
▪ A Hamiltonian-circuit in a graph is a cycle that visits each vertex
exactly once
Problem Statement
Given: A directed graph G = (V,E)
To Find: If the graph contains a Hamiltonian cycle
G: Hamiltonian-circuit of G:

▪ Hamiltonian Circuit is NP-Complete

▪ Reduction from 3-SAT
Traveling Salesman Problem
• Given a complete graph with edge weights, TSP
determines the shortest tour (starts and gets back
to the starting point) that includes all of the
vertices exactly once.

5
6
8
10
10
8
10 9
7

Find the minimum cost tour

• Problems Solvable in Polynomial Space
The Class of Languages Co-NP
▪ We believe that none of the NP-complete problems have
their complements in NP, and therefore no NP-complete
problem is in co-NP.
▪ We believe that the complements of NP-complete
problems, which are by definition in co-NP, are not in NP.
▪ A problem/language whose complement is in NP is said to
be in Co-NP.
▪ P is closed under complementation.
▪ Thus, P  Co-NP.
▪ Also, if P = NP, then P = NP = Co-NP.
Theorem: NP = co-NP if and only if there is some NP-
complete problem whose complement is in NP.

43
co-NP

• co-NP = {A | Σ* - A ε NP}
NP ∩ co-NP

So far, no natural problem has been found in NP ∩ co-NP, but not in P.

NP

co-NP P
The Class of Languages PS
▪ There are complete problems P for polynomial space, in the
sense that all problems in this class are reducible in polynomial
time to P. Thus if P is in P or in NP, then all languages with
polynomial-space-bounded TM’s are in P or NP respectively.
▪ Polynomial-Space Turing Machines: There is some polynomial
p(n) such that when given input w of length n, the TM never
visits more than p(n) cells of its tape.
▪ The Class of Languages PS (Polynomial space): Define the class
of languages PS to include all and only the languages that are
L(M) for some polynomial-space-bounded deterministic Turing
machine M.

46
Polynomial Space (PS)
• The emphasis on the computation resources is usually on the
time used to perform the computation process (especially
when measuring a performance issue or complexity).
• However, the amount of space required is often just as
important ( called the space requirement ).
Polynomial Space (PS)
In any Turing machine:
• the time used is the number of steps taken before halting or entering
a final state.
• The space required is defined as the number of distinct tape squares
(cells) “visited” by the read/write head.
Proof
Since the time taken to visit all cells < = the time used, then
the number of cells visited < = the time used.
→ the number of cells visited cannot be more than the steps of
computation

This implies:
Any Polynomial Solvable in polynomial time, It’s solvable in polynomial
space.
Polynomial Space (PS)

Although all problems solvable in polynomial time can be solved in

polynomial space, it is still a controversial and unresolved question
whether there exist problems solvable in polynomial space which
cannot be solved in polynomial time.

- This is so difficult to conclude since all problems in NP,

including P and NP-C problems are solved in both polynomial
space and polynomial time.
Deterministic Polynomial Space (PS)

• PS
is the class of all languages recognizable by
polynomial space bounded DTM that halts
on all inputs.

i.e. All and Only the languages that are LM

for some polynomial space bounded DTM M.
Nondeterministic Polynomial Space (NPS)

▪ NPS is the class of all languages recognizable by

polynomial space bounded NDTM that halts on at least
one of the possible input structures.
i.e. All and Only the languages that are LM for some
polynomial space bounded NDTM M.

Evidently, PS ⊆ NPS, since every DTM is technically

NDTM.
However, The surprising result is that:

PS = NPS
Relationships among Classes of Languages

52
• A Problem That Is Complete for PS
Polynomial-Space-Bounded TM’s

• A TM M is said to be polyspace-bounded if there is a polynomial p(n)

such that, given input of length n, M never uses more than p(n) cells
of its tape.
• L(M) is in the class polynomial space, or PS.

54
Nondeterministic Polyspace

• If we allow a TM M to be nondeterministic but to use only p(n) tape

cells in any sequence of ID’s when given input of length n, we say M
is a nondeterministic polyspace-bounded TM.
• And L(M) is in the class nondeterministic polyspace, or NPS.

55
Relationship to Other Classes

• Obviously, P  PS and NP  NPS.

• If you use polynomial time, you cannot reach more than a polynomial
number of tape cells.
• Alas, it is not even known whether P = PS or NP = PS.
• On the other hand, we shall show PS = NPS.

56
Exponential Polytime Classes

• A DTM M runs in exponential polytime if it makes at most cp(n)

steps on input of length n, for some constant c and polynomial
p.
• Say L(M) is in the class EP.
• If M is an NTM instead, say L(M) is in the class NEP
(nondeterministic exponential polytime ).

57
More Class Relationships

• P  NP  PS  EP, and at least one of these is proper.

• A diagonalization proof shows that P  EP.
• PS  EP requires proof.
• Key Point: A polyspace-bounded TM has only cp(n) different ID’s.
• We can count to cp(n) in polyspace and stop it after it surely repeated an ID.

58
Savitch’s Theorem: PS = NPS
• Key Idea: a polyspace NTM has “only” cp(n) different ID’s it can enter.
• Implement a deterministic, recursive function that decides, about
the NTM, whether I⊦*J in at most m moves.
• Assume m < cp(n), since if the NTM accepts, it does so without
repeating an ID.
• Recursive doubling trick: to tell if I⊦*J in < m moves, search for an
ID K such that I⊦*K and K⊦*J, both in < m/2 moves.
• Complete algorithm: ask if I0⊦*J in at most cp(n) moves, where I0 is
the initial ID with given input w of length n, and J is any of the ID’s
with an accepting state and length < p(n).

59
Recursive Doubling

boolean function f(I, J, m) {

for (all ID’s K using p(n) tape)
if (f(I, K, m/2) && f(K, J, m/2))
return true;
return false;
}

60
Stack Implementation of f

I, J, m I, K, m/2 L, K, m/4 ... M, N, 1

O(p(n)) O(p(n)) O(p(n)) O(p(n))

space space space space

O(p2(n)) space

61
Space for Recursive Doubling

• f(I, J, m) requires space O(p(n)) to store I, J, m, and the current K.

• m need not be more than cp(n), so it can be stored in O(p(n)) space.
• How many calls to f can be active at once?
• Largest m is cp(n).

62
Space for Recursive Doubling (2)

• Each call with third argument m results in only one call with
argument m/2 at any one time.
• Thus, at most log2cp(n) = O(p(n)) calls can be active at any one time.
• Total space needed by the DTM is therefore O(p2(n)) – a polynomial.

63
PS-Complete Problems

• A problem P in PS is said to be PS-complete if there is a polytime

reduction from every problem in PS to P.
• Note: it has to be polytime, not polyspace, because:
1. Polyspace can exponentiate the output size.
2. Without polytime, we could not deal with the question P = PS?

64
What PS-Completeness Buys

• If some PS-complete problem is:

1. In P, then P = PS.
2. In NP, then NP = PS.

65
Quantified Boolean Formulas

• We shall meet a PS-complete problem, called QBF : is a given

quantified boolean formula true?
• But first we meet the QBF’s themselves.
• We shall give a recursive (inductive) definition of QBF’s along with the
definition of free/bound variable occurrences.

66
QBF’s (2)

• First-order predicate logic, with variables restricted to true/false.

• Basis:
1. Constants 0 (false) and 1 (true) are QBF’s.
2. A variable is a QBF, and that variable occurrence is free in this QBF.

• Induction: If E and F are QBF’s, so are:

1. E AND F, E OR F, and NOT F.
Variables are bound or free as in E or F.
2. (x)E and (x)E for any variable x.
All free occurrences x are bound to this quantifier, and other occurrences of
variables are free/bound as in E.
• Use parentheses to group as needed.
Precedence: quantifiers, NOT, AND, OR.

67
 Example: QBF

bound

(x)(y)(((x)(x OR y)) AND NOT (x AND y))

bound

bound

68
The QBF Problem

• The problem QBF is:

• Given a QBF with no free variables, is its value 1 (true)?
• Theorem: QBF is PS-complete.
• Comment: What makes QBF extra hard? Alternation of quantifiers.
• Example: if only  used, then the problem is really SAT.

69
Part I: QBF is in PS
• Suppose we are given QBF F of length n.
• F has at most n operators.
• We can evaluate F using a stack of subexpressions that never has more
than n subexpressions, each of length < n.
• Thus, space used is O(n2).
• Suppose we have subexpression E on top of the stack, and E = G
OR H.
1. Push G onto the stack.
2. Evaluate it recursively.
3. If true, return true.
4. If false, replace G by H, and return what H returns.

70
QBF is in PS (2)

• Cases E = G AND H and E = NOT G are handled similarly.

• If E = (x)G, then treat E as if it were E = E0 OR E1.
• Observe: difference between  and OR is succinctness; you don’t write both E0
and E1.
• But E0 and E1 must be almost the same.
• If E = (x)G, then treat E as if it were E = E0 AND E1.

71
Part II: All of PS Polytime Reduces to QBF
• Recall that if a polyspace-bounded TM M accepts its input w of length
n, then it does so in cp(n) moves, where c is a constant and p is a
polynomial.
• Use recursive doubling to construct a QBF saying “there is a sequence
of cp(n) moves of M leading to acceptance of w.”

72
Main Reference
1. A Restricted Satisfiability Problem
2. Additional NP-Complete Problems
3. Problems Solvable in Polynomial Space
4. A Problem That Is Complete for PS
(Introduction to Automata Theory, Languages, and Computation
(2013) Global Edition 3rd Edition)
Additional References

https://www.diva-portal.org/smash/get/diva2:1087096/FULLTEXT03.pdf
http://infolab.stanford.edu/~ullman/focs/ch03.pdf

This Presentation is mainly dependent on the textbook: Introduction to Automata Theory, Languages, and Computation: Global Edition, 3rd edition (2013) PHI
by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman
Thank You