Theoretical Computer Science 2

University of Technology Graz

Lecture notes of a student

Lukas Prokop
February 14, 2024

Contents

1
This lecture took place on 2nd of Oct 2013.

0.1 Structure of this course

Evaluation:
Practicals 2-3 board presentations
Lecture oral exam (ask for date via email)

Contents:
• Extended view on complexity theory and modern applications
• Focus on algorithms, optimization problems and cryptography
2 sections:

1. Complexity theory
• randomized complexity classes
• polynomial hierarchy
• interactive protocols (zero knowledge systems)
• probabilistically checkable proof (PCP theorem)
2. Approximation of complexity
• complexity theory of heuristic algorithms

0.2 Literature
• C. Papadimitriou, “Computational complexity”, 1994 [old, classic].
• I. Wegener, „Komplexitätstheorie (Grenzen in der Effizienz von Algorith-
men)”, 2003 [modern, non-comprehensive].
• S. Arora, B. Barak, Computational complexity (A modern approach),
2009 [modern, exhaustive].

2
1 Randomized complexity classes

1.1 Problem: Does a perfect matching exist in a given

bipartite graph?

1.1.1 Problem and approach

Given. G = (U, V, E) is an undirected, bipartite graph with U =

{u1 , u2 , . . . , un } and V = {v1 , v2 , . . . , vn }.

Find. Does a perfect matching exist in the bipartite graph G?

Reference. “Computational Complexity” (Christos Papadimitriou),

“Symbolic determinants” in section 11.1

Definition. A matching M in G is a set of pairwise non-adjacent edges. So

no two edges of the matching share a common vertex. A perfect matching is a
matching covering all vertices of G.

Note. This question can be answered in polynomial time with a deterministic

algorithm (“Edmonds’s algorithm”, O(|V |4 )). We take a different approach.

tcs2_bipartite_graph_perfect_matching.pdf

Figure 1: Perfect matchings in a bipartite graph. Red edges show a perfect

matching in the left graph. In the right graph no perfect matching exists.

Consider a matrix A(G) = (ai,j ) for graph G. We distinguish 2 cases:

(ui , vj ) ∈ E

 xi,j
|{z}
ai,j = symbolic variable
(ui , vj ) ∈

0 /E

Sn is the set of permutations over {1, . . . , n}.

X n
Y
det A(G) = σ(π) · aj,πj
π∈Sn j=1
(
1 π is an even permutation
σ(π) =
−1 π is an odd permutation

3
Observation:

1. Non-zero terms of the product in det A correspond to a perfect matching.

2. G has a perfect matching if the det A(G) ̸= 0 (hence is a zero function).

Remarks:

• This alternative method given above is based on the evaluation of det A(G).
• We are interested in the evaluation of a symbolic determinant.

For given values of xi,j ∈ Q we can evaluate det A(G) in polynomial time (eg. via
gaussian elimination). For the symbolic determinant evaluation no algorithm is
known (actually, the question whether a specific coefficient is the coefficient of
the symbolic non-zero determinant, is an NP-complete problem).

In our case we want to know whether or not det A(G) = 0.

1.1.2 Idea and Lemma

Select a random configuration for xi,j and evaluate the resulting determinant.
If the determinant is 0, then we have found the zero function or a root in the
function. In the other case G has a perfect matching (result: YES). In the first
case, we have to proceed with further evaluations.

Lemma 1.1: Assume p(z1 , z2 , . . . , zm ) to be a polynomial with m variables

(must not be the zero polynomial). d is the maximum degree of those variables.
Furthermore M > 0 and M ∈ Z. It is fact that the number of m-tuples
(z1 , . . . , zm ) ∈ {0, 1, . . . , M − 1} with p(z1 , . . . , zm ) = 0 is smaller or equal to
mdM m−1 (so this is the upper bound).

Proof of the Lemma: Complete induction over m.

m = 1: Number of zeros ≤ d (with respect to the fundamental theorem of alge-

bra) for a polynomial of 1 variable of degree ≤ d.
m − 1 → m: Polynomial p in z1 , . . . , zm . Consider p to be a polynomial in zm
with coefficients which are in z1 , . . . , zm−1 .

Remark. p results in evaluation of “gzz” test tuple 0.

Case distinction:

The coefficient with highest degree of zm in p is 0. Apply induction to the

coefficient of zm with highest degree. The coefficient is a polynomial
in z1 , . . . , zm−1 . Can therefore ≤ (m − 1) of M m−2 permutations of
(z1 , . . . , zm−1 ). In combination with the selection of zm (M possibilities).
≤ (m − 1) of M m−1 .

4
Is not 0. We have a polynomial of degree ≤ d in zm such that ≤ d null points
can exist for each combination of z1 , . . . , zm−1 . Therefore ≤ d M m−1
further null points of p. Addition of (m − 1) in M m−1 with d with M m−1
results in ≤ m · d with M m−1 . So this lemma is proven.

Consider the following algorithm:

Algorithm 1 Mulmuley, Vazirani and Vazirani (1987)

1: Choose m random integers (i1 , . . . , im ) for the assignment of xi,j with
{i, j} ∈ E. (here m = |E|) with xi,j ∈ {0, . . . , M − 1} whereas in the
case of a matching M = 2m = 2 · |E|.
2: Evaluate det A(G) for xi,j assignment (i1 , . . . , im ).
3: if det A(G)(i1 , . . . , im ) ̸= 0 then
4: return “Yes, G has a perfect matching”.
5: else
6: return “I don’t know” (we need further repetitions).
7: end if

This algorithm returns with “no error” or “I don’t know”. This algorithm is
part of the complexity class RP.

1.1.3 Analysis of the algorithm

Properties of the algorithm (false-biased Monte-Carlo algorithm):

• If the answer is Yes (there exists an perfect matching), the answer is always
correct (no false positives).
• If the graph does not have a perfect matching, the answer of the algorithm
is not complete. Consider the answer “No” if the number of repetitions
with the answer “I don’t know” exceeds some threshold (therefore, false
negatives are possible).

What’s the probability that we get a false negative?

In the case of a Yes-answer we can see that the error probability (applying the
algorithm once) is ≤ 12 (recognize that d = 1 here). With k applications, we get
a probability ≤ 21k .

1.2 Problem: SAT problem

Given. A boolean equation ϕ in conjunctive normal form.

Find. Is ϕ satisfiable?

5
 2-SAT. Restrict boolean equation to CNF with 2 literals per clause.

3-SAT. Restrict boolean equation to CNF with 3 literals per clause.

2-SAT has been show to be polynomially decidable (eg. with flow theory). Here
the following algorithm shall be given:

Algorithm 2 Algorithm to decide 2-SAT in polynomial time (Even, Itai,

Shamir, 1976)
1: All variables are unassigned. Recent branch point is unknown.
2: while unassigned variables exist do
3: if both literals of clause falsify clause then
4: if Backtracking not possible then
5: return Unsatisfiable
6: else
7: Go back to most recent branch point.
8: end if
9: else if one of both variables of clause are set then
10: Assign the remaining variable in a way to make clause satisfied
11: else if clause is satisfied or both variables are free then
12: Create branch point
13: Set one variable to an arbitrary value
14: end if
15: end while

3-SAT is known to be NP-complete and cannot be solved efficiently.

Consider the following algorithm for SAT:

1: Start with a random assignment T for the variables x1 , . . . , xn .

2: for r times do
3: if T is a satisfiable assignment then
4: return yes.
5: end if
6: Let c be one of the clauses which are not satisfied.
7: Take one literal of T and negate its variable in the assignment.
8: end for
9: return “no” or “probably no”.

Is also a Monte-Carlo-Algorithm. Open issues: Probability for false negatives?

How to choose r?

Is a polynomial upper bound of r (in regards of the number of clauses) enough

for a practical error probability? (exponentially, it is easily feasible)

• No, even for 3-SAT there are known instances of the problem where a

6
 polynomial r fails.
• For 2-SAT for r = 2n2 we have an error probability of ≤ 12 . Therefore we
have a Monte-Carlo algorithm for 2-SAT.

This lecture took place on 7th of Oct 2013.

1.3 Problem: Polynomial Identity Testing

Given. A polynomial

Find. Is the given polynomial a zero-polynomial (hence equals to 0)?

Equivalence. Be aware that polynomial identity testing equals polynomial

equivalence checking. p1 (x) = p2 (x) ⇔ [p1 (x) − p2 (x)] = 0

Example. −x · x − y · y + (x − y)(x + y) is a zero-polynomial whereas x · x

is not.

The instance can be encoded in various forms. Possible forms include:

• multivariate arithmetic expression over integers (arithmetic formula)

• matrix with polynomial entries

Polynomials are represented here as equations of an algebraic circuit (instead

of the operators ∧, ∨ and ¬ in classical propositional logic, we have +, − and ·
available) with n variables x1 , x2 , . . . , xn represented as special directed graph.
All vertices (which are not sources and are not inner vertices) do have in-degree
2 und are annotated with operators {+, ·, −}.

tcs2_algebraic_circuit.pdf

Figure 2: Visual representation of algebraic circuit x1 x2 + x3

Remarks:

• Extensions to this problem are possible.

• Such a circuit creates a polynomials in x1 , x2 , . . . , xn .

7
1.3.1 The class ZeroP

Let ZeroP be the complexity class of algebraic circuits, which evaluate whether
a polynomial (which is identical) is a zero-polynomial.

c = c′ ⇔ c − c′ = 0
This is why the name ZeroP was chosen. A special case of this problem is the
previous problem to determine whether a perfect matching exists.

Problem: Detection of ZeroP is non-trivial (ie. is a polynomial P ∈ ZeroP).

Consider for example

n
Y
(1 + xi )
i=1
This is a very compact description, but the circuit has size 2n = O(n) but
polynomial has 2n terms.

For this problem no deterministic, efficient algorithms is known, but a nice

Monte-Carlo algorithm.

We can use the following Lemma 1.2 (analogous to Lemma 1.1):

˜ For example
Is p(x1 , . . . , xn ) a polynomial of total degree max d.
x21 x32 x3 x74 has max-degree 13 for all terms.
S is a finite set of integers. If aj , . . . , an is selected randomly out of
S.
In this case:
d˜
P (p(a1 , . . . , an ) ̸= 0) ≥ 1 −
|S|

A circuit C of size m contains ≤ m multiplications and therefore degree ≤ 2m .

Idea for probabilistic algorithm:

Algorithm 3 Probabilistic algorithm to solve “is polynomial zero?”

1: Select random integers I for x1 , . . . , xn ∈ {1, . . . , 10, . . . , 2m } (requires
O(nm) random bits).
2: Let Y be the result of the evaluation of C for I.
3: If Y = 0, answer “Yes”. Else “I don’t know”.

If C ∈ ZEROP, then we accept C always. If C ∈ / ZEROP and we do not

1
answer with “NO” for y ̸= 0, we have a error probability. We reject ≤ 10 ·
9
(Mil probability ≥ 10 ).

Problem: Degree can be at maximum 2m . y and the intermediate results

m
can grow up to (10 · 2m )2 . An exponential number of bits is necessary for
representation. Therefore we would like to switch to modulo calculations.

8
 
We evaluate C mod k where k is randomly selected from 1, . . . , 22m (further
constraints are mentioned later). Instead of y, we get y mod k. If y = 0, then
also y mod k = 0.
1
If y ̸= 0, then we can show that k (with probability ≥ δ = 4m ) does not split.
1
This probability is small enough that O( δ ) are enough repetitions to gain our
result. We do only accept ak , if for all runs the result was 0.

Is y ̸= 0 and B = {p1 , . . . , pn } the set of distinct prime factors of y. This satisfies

that K is not a prime number of B is given with probability ≥ δ. With the
prime number theorem,  we can conclude2m that for sufficient large m the number
of prime numbers in 1, . . . , 22m is ≥ 22·m .
2m
y can have max log y ≤ 5, m · 2m = o( 22m ) prime factors.

For sufficiently large m it is given that the number of k ∈ 1, . . . , 22m such
2m
that k is prime and k ∈ 1
/ B: ≥ 24m with probability δ = 4m a random k has the
desired properties.

1.4 Problem: Primality testing

In the meantime a deterministic polynomial primality test is known (Agrawal,

Kayal, Saxena). For a few years only randomized primality tests (which repre-
sent Monte Carlo algorithms) were known.

Given. n is odd

Find. Is n not a prime number?

1.5 Probabilistic complexity classes overview

In this section we want to given a quick overview for the definitions required to
discuss this topic. See table ?? for the class overview.

Now we would like to formalize Monte-Carlo algorithms. Monte-Carlo turing

machines are special cases of probabilistic turing machines. Probabilistic turing
machines are a generalization.

1.6 Monte-Carlo turing machines and RP

Definition. N is a polynomial, non-deterministic turingmachine. We assume

for N that all computations for an input x take the same number of steps.
This number has to be polynomial in |x|, because N is a polynomial turingma-
chine. Furthermore in every step there are exactly 2 non-deterministic decision
possibilities.

9
 failure/error probability failure/error probability
(bounded) (not bounded)
Two-sided error BPP PP
One-sided error RP, co-RP NP, co-NP
RP = Monte-Carlo algorithms
No error, ZPP = RP ∩ co − RP NP ∩ co − NP
but failure is possible (Las Vegas Algorithms)
No error, no failure P

Table 1: Probabilistic complexity classes overview

L is a language. N is called Monte-Carlo turing machine for L iff N satisfies

the definition above and the computations for an input of size N are finished in
p(n) runtime (with p as a polynomial) . Furthermore it satisfies:

• If x ∈
/ L then all computations of N for x end with the result “No” (no
false positives).
• If x ∈ L then more than half of the evaluations for x result in “Yes”. Hence
the probability of false negatives is ≤ 21 where 12 is a configurable constant.

RP (randomized polynomial) is a historical name and does not really distinguish

it from other classes.

The set of all languages for which there exists a Monte-Carlo turing machine is
called RP.

Remark:

• This concept corresponds to the informal concept of Monte-Carlo algo-

rithms (of our exercises).
• If we replace the allowed error probability for false negatives to a value
̸= 12 , but < 1, no new complexity class is created.

Repetitions:
1
1 − ϵ with ϵ < is the error probability of false positives
2
For k repetitions: Error probability for false negatives is (1 − ϵ)k . Choose k:
 
1 1
k= − probability for false negatives is ≤
log2 (1 − ϵ) 2

Runtime k · p(n) is polynomial for polynomial k.

RP lies between P and NP. co-RP is the complement of RP (ie. the result
“Yes”/“No”/“I don’t know” is exchanged).

10
1.7 Complexity class ZPP

ZPP (polynomial randomized with zero probability of error)

ZPP = RP ∩ co-RP

In other words, there exists a Monte-Carlo algorithm which does not return any
false positives and an analogon which does not return false negatives (no error,
but failure is possible). This is the definition of Las-Vegas algorithms.
1
If probability of false positives and false negatives ≤ 2 (after k repetitions).

1.8 Complexity class PP

PP (probabilistic polynomial) is not defined algorithmically. We consider MajSat

(majority SAT).

Given. A SAT equation with n variables.

Find. Determine whether the majority of 2n ≥ 2n−1 +1 assignments satisfy

the equation?

It is unknown whether or not MajSat ∈ NP (how should the polynomial cer-

tificate look like?). It is even more unlikely that MajSat ∈ RP.

Definition. (PP) A language L is in PP, iff there is a non-deterministic poly-

nomial TM N (N satisfies the basic constraints mentioned above) such that for
all inputs x it states that
1
x∈L ⇔ #(computations of N for x result in “YES”) >
2
Therefore this turing machine has an inherent majority evaluation capability.

Remark: Definition of PP is syntactically, but not semantically. We can easily

show MajSat ∈ PP and MajSat is PP-complete.

This lecture took place on 9. Okt 2013.

1.8.1 NP ⊆ PP

Proof. L ∈ NP. N is a non-deterministic turing machine for L. We construct

another turing machine N ′ , which decides L with a majority vote. N ′ almost
looks alike N . We introduce a new Start state and another branching (non-
deterministic decision), which is executed in the Start state.

Start state - non-deterministic choice which branch to take:

1. Evaluation with N

11
 2. Randomized selection of one evaluation path (for all inputs x of same size
answer always with “YES”).

Consider input x. x takes p(|x|) steps in N (fundamental assumption as always).

2|x| evaluation paths of N for x

2|x|+1 evaluation paths of N’ for x

A majority of evaluations for N ’ for x ends with “Yes”. This corresponds to:
more than 1 evaluation of N for x ends with “YES”. This corresponds to x ∈ L.

We can conclude L ∈ PP.

Remarks:

• PP is closed under complement.

• PP is (in difference to our previous mentioned complexity classes) not
algorithmically defined.

1.9 The class BPP

Definition. BPP (bounded error probabilistic polynomial) contains all lan-

guages L for which there is a non-deterministic polynomial turing machine N
(without further constraints of the assumptions with our fundamental assump-
tions) such that:

3
• For all inputs x with x ∈ L ends with probability ≥ 4 of the evaluations
of N of x with acceptance.
3
• For all inputs x with x ∈
/ L ends with probability ≥ 4 of the evaluations
of N of x with rejection.

Remarks:

1
• Any value > 2 and < 1 can be used instead of 34 .
• For 34 : 1
4 error probability for false positives and false negatives.
1
• RP ⊆ BPP (Monte-Carlo algorithm with error probability 2 repeated
twice in different direction)
• co-RP ⊆ BPP
• BPP ⊆ PP (can easily be derived from the definition; has fewer con-
straints)
• Open issues:

12
 – BPP ⊆ NP?
– ∃ BPP-complete problems?
• BPP is closed under complement.
• BPP is basically a model for “practical polynomial randomized algorithm”.

All these complexity classes require some concept of probability; for example
random bits set on the turing machine. Question in practice:

Assume we don’t have any fair dice / coins (probability (heads) ̸= 12 ). Does this
define a new complexity class?

You can create a fair die with polynomial effort using an unfair die. So RP and
BPP (etc) are not influenced.

Take a coin with probability (heads) = ρ with ρ ̸= 21 . The following two lemmas
apply:

1. (Lemma 1.3) A coin with probability (heads) = ρ can be simulated by

a probabilistic, polynomial turing machine in expected time O(1), if the
i-th bit of ρ is determinable in polynomial time in i. Notice that the last
assumption is only relevant for irrational numbers.
1
2. (Lemma 1.4) A coin with probability (heads) = (fair coin) canbe simu-
2 
1
lated with a coin of probability(heads) = ρ in expected time O ρ(1−ρ) .

1.9.1 Proof of Lemma 1.3

Is 0.p1 p2 p3 . . . a binary fractional representation of ρ. The probabilistic turing-

machine creates a sequence of random bits b1 b2 . . . step by step, where bit bi is
created in step i. If bi < pi then the turing machine returns “heads” as result
and stops. If bi > pi then the turing machine returns “tails” as result and stops.
In any other case, go on (increment i).

If turing machine reaches bit i + i, then

bj = pj ∀j ∈ {1, . . . , i}

The probability for this is defined by ( 21 )l .

1
P
The probability that the result is heads is i pi · 2i = ρ. Expected runtime is
c 1
P
i i · 2i for constant c.

1.9.2 Proof of Lemma 1.4

We construct a turing machine which simulates (using the possibility to use a

coin with probability(heads) = ρ multiple times) a fair coin.

13
M throws pair of coins (2 coins) with bias ρ. Repeat these paired throws until
2 different results are given. We return “heads” for the result heads-tails and
“tails” for tails-heads.

Probability for heads-tails is ρ · (1 − ρ). Probability for tails-heads is (1 − ρ) · ρ =

ρ · (1 − ρ). Probability that turing machine stops in every step is 2ρ · (1 − ρ).
The conditional probability for stopping in step k is equal for heads and tails.
So we can create a fair coin with probability(heads) = 21 . This proof does work
for any value of ρ.

1.9.3 Remark for Lemma 1.2

The “probabilistic turing machine” must have two transition function δ0 ad δ1 .

We decide in every step whether we want to use (with probability 12 ) δ0 or (with
probability 12 ) δ1 . In the general model for a probabilistic turing machine which
is considered for efficient algorithms we allow expected polynomial time.

From this lemma it follows that (if we allow expected polynomial time1 ) we
don’t lose any properties for a biased coin. We can also show this for polynomial
runtime rather than expected runtime (by splitting the bitstring into blocks).

1.10 Relations of probabilistic complexity classes

randomized_complexity_classes.pdf

Figure 3: Overview of randomized complexity classes

Figure ?? gives an overview of randomized complexity classes.

The position of BPP is partially unknown; partially we will discuss them later
on.

The aspect of expected polynomial runtime is considered at a different place.

2 The class FP

Definition. (Complexity class of function problems for polynomial hierarchy).

Many problems in practice are no decision problems (they result in complexer
1 which is realistic for practical applications

14
answers than “YES” or “NO”). Two examples are FSAT and TSP.

2.1 Problem: FSAT

Given. A SAT equation ϕ.

Find. Is there any satisfying assignment for ϕ? If so, return it. Otherwise
report “No”.

Assume that a polynomial SAT algorithm exists for the decision problem SAT,
then we can solve FSAT in polynomial time: ϕ is a SAT equation with variables
x1 , x2 , . . . , xn .

1. Apply SAT algorithm to ϕ. If answer is “no”, then ϕ is not satisifiable

and we return “there is no satisfying assignment”.
2. (Successive fixation of variables.) We know that in one of the cases
(true/false) the answer is “Yes” and an assignment can be found. Fix
variable v in such a way (i.e. ϕ1 or ϕ2 ) that ϕ satys satisfiable. Stop if all
variables got fixated.

fsat.pdf

This approach takes O(n) SAT calls.

2.2 Problem: Travelling Salesman Problem (TSP)

D = (dij ) dij ∈ Z+
0

where dij specifies the distance between the vertices i and j. D is called distance
matrix.

Classical definition of TSP

Given. Distance matrix D

Find. A hamiltonian path with minimum length/weight

15
Specifically, TSP is given as decision problem TSPDEC :

Given. n × n matrix D = (dij ) with dij ∈ Z0+ with upper bound d∗ of

costs

Find. Is there any hamiltonian path of length ≤ d∗ ?

This lecture took place on 14th of Oct 2013.

We show that when a polynomial algorithm for TSPDEC exists, then there is a
polynomial algorithm for TSP:

Step 1 Determine the requested target function value (length of optimal tour).

Value of an optimal tour is in {0, 1, . . . , 2q } with q as the coding length of the

instance.

Binary search for the optimal value for the optimal value. Always ask: is there
any tour with length (≤ 2q−1 )? We get a binary tree always with two branches
“Yes” and “No”. On the second level we ask is there any tour with length
(≤ 2q−2 ).

After a polynomial number O(log2 (2q ) = O(q) of steps, c∗ —the length of the
requested tour—is known.

Step 2 Determine a required tour (a tour with length c∗ ). Consider the values
of D in arbitrary order one after another. dij is the current item we look at.
Then set dij temporarily to c∗ +1 (Idea: restrict usage of (i, j)) and ask whether
there is a tour of length ≤ c∗ for the current matrix.

If answer is no, then we have to use the current edge (i, j) and we set dij back
to its old value. In the other case we keep dij as c∗ + 1.

In the end (after O(n2 ) steps) we know the optimal tour.

Formally L is a language in NP. We want to define the associated function

problem to L: FL. For L ∈ NP there is a binary relation RL , which is deter-
minable in polynomial time such that ∀x strings ∃ string y with RL (x, y) ⇔ x ∈
| {z }
(x,y)∈RL
L. RL (x, y) is the certificate of an accepted instance.

Definition. (FL) Is L ∈ NP the associated problem FL for L. Given a string x

we have to find a string y such that RL (x, y) or answer that there is no such y.

Next task: Generalize the class / reduction term of decision problems for
function problems.

16
Definition: A and B are two function problems. A is reducible to B, iff ∃ string
functions R and S (both computable in polynomial space) such that the relation
illustrated in Figure ?? is given.

function_problem_reduction.pdf

Figure 4: Reduction term for function problems

x is an input string of an instance of A such that R(x) is an input string of an

instance of B. Is z a correct output for the instance of B with input R(x) then
S(z) is a correct output for the instance with input x.

Remark: Any logarithmic boundary for space implies polynomial time.

Definition. A function problem A is says to be complete for a class FC of

function problems iff A ∈ FC and all problems in FC can be reduced to A.

We can show that FSAT ∈ FNP is FNP-complete.

We want to consider a special class of function problems. They are problems

for which there are always valid solution and as such the answer is never “No”.
Example for such a problem: ∃ always y with RL (x, y)?

2.3 Problem: Factorization of integers

Given. A number n ∈ N

Find. prime number decomposition

The corresponding decision problem is not interesting (answer is always yes).

Function problems are not trivial. It’s an open question whether factorization
is possible in polynomial time. Factorization is fundamentally different than
FSAT where a decision problem cannot be solved efficiently.

Definition. (total functions) For such function problems there is always a valid
solution.

Definition. We consider a problem L in FNP total, when for all strings x∃

string y ∃ with RL (x, y).

Definition. (TFNP) partial class of all function problems with total functions.

17
2.4 Exercise 2: Integer sets of same sum

Pn
Given. n numbers a1 , . . . , an ∈ Z+ with i=1 ai < 2n − 1.

Find. I, J ⊂ P P I ̸= J such that I ∩ J =  (not always

{1, . . . , n} with
required) and i∈I ai = i∈J ai (or answer “No” if there are no such
sets).

Pn
For the special case i=1 ai < 2n − 1 there is always a solution due to the pid-
geonhole principle. Without this constraint, the decision problem is complete.

2.5 Exercise 3: Happy net problem

Given. Undirected graph G, edge weights we ∈ Z for all e ∈ E. State

function S: V → {+1, −1}.

Find. Find a state S in which all vertices are happy.

A vertex i ∈ V is called “happy for S” iff

X
S(i) · S(j) · wij ≥ 0
{i,j}∈E

happy_vertices.pdf

Claim. There always is such a state.

P
Proof. Potential function ρ[s] := {i,j}∈E S(i) · S(j) · wij .

Remark. i is unhappy for S.

X
S(i) · S(j) wij < 0
|{z}
{i,j}∈E
∈Z
| {z }
=−δ with δ>0,δ∈Z and therefore δ≥1

Transition S → S ′ with S ′ (j) = S(j)∀j ̸= i and S ′ (i) = −S(i) (flip) otherwise.

We can show that:
ρ[S ′ ] = ρ[S] + 2δ

We come up with an algorithm from this:

18
 Start with a random start state. As long as there are unhappy
P flip them. Because there are ρ values in [−W, W ] with
vertices,
W = {i,j}∈E |wij | and ρ increases with ≥ 2{i,j}∈E , we have shown
the finiteness of the algorithm.

This is also a total function problem. The algorithm has pseudo-polynomial

runtime (because it depends on pseudo-polynomial property of W ). It is an
open question whether there exists a polynomial algorithm.

Given. An undirected graph (G = (V, E)) and a hamiltonian cycle Q in

G.

Find. A second hamiltonian cycle (but does not have to exist) (different
from Q).

For cubic graphs (3-regular2 ) there is always a second hamiltonian cycle (if there
is at least one).

Is a total function problem and it is open whether there is a polynomial algo-

rithm.

fnp.pdf

2.6 Four variants of TSP

2.6.1 TSPDEC

Given. n × n matrix D and a boundary c∗ .

Find. Is there a tour with length ≤ c∗ ?

2.6.2 Exact-TSP

Given. n × n matrix D, value c∗

Find. Does the optimal tour has length c∗ ?

2 d(v) = 3∀v ∈ V

19
2.6.3 TSP-Cost

Given. n × n matrix D.

Find. Length of the optimal tour

2.6.4 TSP

Given. n × n matrix D.

Find. optimal tour.

3 Complexity class DP

Definition. (difference polynomial) L ∈ DP ⇔ ∃ languages L1 ∈ NP and

L2 ∈ co-NP with L = L1 ∩ L2 .

Remark. DP ̸= P ∩ co-NP.

3.1 Examples for problems in DP

Examples:

• EXACT-TSP ∈ DP.
Split problem in ≤ c∗ (in NP) and ≥ c∗ (in co-NP) questions.
• SAT-UNSAT
Given a SAT formular ϕ and ϕ′ .
Question: Is ϕ satisfiable or ϕ′ not satisfiable?

Statement 2.1 SAT-UNSAT ∈ DP-complete.

Proof:

1. SAT-UNSAT ∈ DP.
L1 = {(ϕ, ϕ′ ) : ϕ is satisfiable}.
L1 ∈ P due to SAT ∈ NP.
L2 = {(ϕ, ϕ′ ) : ϕ is not satisfiable}.
L2 ∈ co-NP due to co-SAT ∈ co-NP (L = L1 ∩ L2 ).
2. SAT-UNSAT is DP-complete.
Is L ∈ DP and L is reducible to SAT-UNSAT.
L ∈ DP → ∃l1 ∈ NP, l2 ∈ co-NP
∃ reduction R1 of L1 to SAT. ∃ reduction R2 of L2 (complement) to SAT.

20
 Construct reduction R with R(x) = (R1 (x), R2 (x)) of L to SAT-UNSAT.
R(x) is YES-instance of SAT-UNSAT ⇔ R1 (x) is satisfiable, R2 (x) is not
satisfiable ⇔ x ∈ L1 , x ∈ L2 ⇔ x ∈ L.

Statement 2.2 EXACT-TSP ∈ DP-complete.

(repetition)

L ∈ DP : L = L1 ∩ L2 with L1 ∈ NP, L2 ∈ co-NP.

SAT-UNSAT ∈ DP-complete.

Satz 2.2. EXACT-TSP is DP-complete.

Proof:

1. EXACT-TSP ∈ DP has been proven above.

2. DP-Completeness (reduction of SAT-UNSAT)
We start from the classical reduction of 3SAT to the hamiltonian cycle
problem. Using the equations ϕ, ϕ′ of the SAT-UNSAT instance we get 2
graph G and G′ such that G and G′ posess a hamiltonian path ⇔ ϕ and ϕ′
is satisfiable

Remark: Idea of reduction 3SAT to hamiltonian path problem: Given. 3SAT

equation ϕ = (x1 , x2 , . . . , xn ) and clauses c1 , c2 , . . . , cn with ≤ 3 literals per
clause.

Construction of a graph G = R(ϕ) which has a hamiltonian path ⇒ ϕ is satis-

fiable.

Construction consists of the components, which are connected at the ends.

3sat_hamiltonian_path.pdf

The image shows the selection gadget for each variable.

• Inner nodes only occur it the corresponding component.

• They relate to the outside.

There are always 2 possibilities to follow a path. The horizontal lines are fol-
lowed by 2 paths where the paths can be described by a XOR (upper horizontal
line is assigned to a different path than the corresponding lower horizontal line).

21
 consistency_graph.pdf

clause_gadgets.pdf

This ensures that the variable are kept consistent.

Clause gadget. without constraining the assumptions, 3 literals per clause.

The selection gadgets for x1 to xn are connected in series.

gadgets.pdf

(x1 ∨ x2 ∨ x3 ) for hamiltonian path 1

(x1 ∨ x2 ∨ x3 ) for hamiltonian path 2

All 3m triangular nodes (of the m clause gadgets) and the last node of the
“Kelle” of the variable gadgets and a new node 3 are connected by all possible
nodes (complete subgraph). We finally add node 2 and edge {2, 3}.

G(ϕ) is now finalized. We can show that the hamiltonian path ∃ ⇔ ϕ is sat-
isfiable. Such a hamiltonian path has to exist between 1 and 2 and without
constrainting of our previous assumption, we start in 1.

(Further details shall be studies as home exercise)

3.2 Back to our EXACT-TSP construction

Independent of the satisfiability of ϕ and ϕ′ , G and G′ contain a so-called broken

hamiltonian path. This corresponds to an “almost satisfiable truth assignment”

22
(all but one variable are satisfied). Introduce a new variable z and insert ∨z to
all clauses and another clause z (can be reduced to a 3SAT normal form). The
new SAT equation requies an almost satisfying assignment (set z = ⊤).

If we apply the reduction of the variable-introduction, so we get an almost

satisfying truth assignment T and its corresponding graph, which has an broken
hamiltonian path: We start in 1, traverse all variable gadgets according to T
and traverse all clause gadgets. We interrupt this hamiltonian path only for the
second-last clause.

So the path is interrupted at exactly for one edge.

Apply this construction to ϕ and ϕ′ ; G and G′ are created where each of them
contains a broken hamiltonian path.

vertex_combination.pdf

In the left graph we define the edge weights to get an EXACT-TSP instance.

Set 
1
 if {i, j} is an edge in G or edge in G’
dij := 2 if {i, j} is no edge in G but i,j are vertices of G

3 else


Question: What is the length of an optimal tour?

Case 1: ϕ and ϕ′ are satisfiable. G and G′ have a hamiltonian path (together:

hamiltonian cycle). So we only have 1-edges (optimal tour has length n).
Case 2: ϕ and ϕ′ are not satisfiable. We get broken satisfiable paths which
have to be fixed. This has additional cost of +1 + 2. Optimal tour has
length n + 3.
Case 3: ϕ is satisfiable, ϕ is not satifiable
Optimal tour has length n + 2
Case 4: ϕ is not satisfiable, ϕ is satisfiable
Optimal tour has length n − 1

(ϕ, ϕ′ ) is Yes-instance of SAT-UNSAT ⇔ in our EXACT-TSP instance (c∗ =

n + 2) exists an optimal tour with length n + 2.

23
3.3 Other problems

Many other NP-hard combinatorial optimization problems lead in the EXACT-

version to DP-complete problems. DP is the natural class of EXACT-COST.

Other problems of DP:

Critical SAT Given a SAT equation ϕ.

Is it true that ϕ is not satisfiable, but by deleting one clause we get a
satisfying one?
Critical Hamiltonian Path Given an undirected graph G = (V, E)
Is it true that G does not have a hamiltonian path, but adding one arbi-
trary edge leads to the existence of such.
Unique SAT Given a SAT equation ϕ
Has ϕ got exactly one satisfying assignment?

All problems above (except Unique-SAT) are DP-complete. For Unique-SAT it

is unknown.

Theorem 2.3. (A first step towards polynomial hierarchy.) A different ap-

proach to DP: Use a SAT oracle and one co-SAT oracle. Answer “Yes” whenever
first answer (L1 ) is yes and second answer (L2 ) is “No”.

Definition. A turing machine M ? with an oracle is a turing machine with a

specfici string (query string of oracle call) and 3 specified, additional states: q?
is the state of the oracle call.

qYes State for Yes-answer of the oracle

qNo State for No-answer of the oracle

A ⊆ σ ∗ is a language.

M ? with A as oracle runs like a traditional turing machine except for the tran-
sition of the query state.

M A (x) . . . applies oracle TM M ? with oracle A to input x

Instead of the single language, we also consider language classes. P SAT is a

generalization of DP and has an polynomial number of oracle call available.
P SAT corresponds to P NP , which can be achieved by oracle turing machine
with oracle of NP. Polynomial number of steps (including number of oracle
calls, excluding time for running the oracle). Analogous: FPNP .

Question: Are there any FPNP -complete problems?

24
3.4 MAX-OUTPUT

Given a non-deterministic turing machine N with input string 1n . N behaves

in such a way that for this input and for an arbitrary sequence of the non-
deterministic O(n) steps stops. It returns a binary string of length n as output.

Given: Greatest output of N (interpreted as binary number)

Theorem 2.3 MAX-OUTPUT is FPNP complete.

Will be a helping subproblem to show FPnp completeness of natural problems.

Proof: First we show that MAX OUTPUT ∈ FPNP . Idea: Binary search.
Iterate question: Is there an output greater ≥ x? (or > x) (for integers x ∈

0, . . . , 2n−1 ).

We determine the optimal value in O(log n) (polynomial number of) steps.

Each question is a question in NP. Therefore: FPNP .

Secondly, we get a little bit more technical: We show FPNP completeness. F

is a function of strings to string with F ∈ FPNP . There exists an oracle tur-
ingmachine M ? which evaluates F with a polynomial time boundary. Thus,
M NP (x) = F (x) or correspondingly M SAT (x) = F (x).

We need a reduction from F to MAX-OUTPUT: We need a function R con-

verting F to MAX-OUTPUT (R(x)) and a function S to convert the output of
R to F (x) (solution, output).

We need two functions R and S such that

• R and S are determinable with logarithmic space requirements..

• For all strings x, R(x) is an instance of MAX-OUTPUT.
• S applied to the output of MAX-OUTPUT for an instance R(x) which
evaluates to F (x).

Regarding the construction of R We have to build a non-deterministic

TM N and input 1n . We set n = p2 (|x|) with p(.) as the polynomial time
boundary of TM M SAT (informally this definition requires that sufficient time
is available to simulate M SAT ).

Regarding N : B starts with an input 1n and creates x as string and simulates

M SAT to x afterwards.

This simulation M SAT is deterministic and correct for all operations, that are no
calls to the oracle. Assume that M SAT stops at the first oracle call. An oracle
call corresponds to the question whether this SAT equation ϕ is satisfiable.

N guesses the answer to this question and sets z1 = 1, if the guesses answer is
“Yes” and z1 = 0 otherwise. In the case of z1 = 0, N continues the simulation

25
with state qNo . If z1 = 1, then N guesses an assignment T1 for the variables in
ϕ1 and checks whether T1 satisifies ϕ1 . If this test succeedes, the simulation is
continued in state qYes . Otherwise to stop and resign. N returns the string 0n
as output and halts (non-successful evaluation).

Application to all oracle calls:

ϕ . . . equation in oracle call i

zi ∈ {0, 1} analoguous to z1

In the case that the turingmachine does not halt yet ({z1 , z2 , . . . , zr } as the set
of oracle calls), N writes string z1 , z2 , . . . , zr and possibly padded with zeros at
the end for length n and output of M SAT (x) = F (x).

• A partial set of the successful evaluations are faulty simulations of M SAT .

ϕi can be satisfiable, but N probably uses zi = 0. In case zi = 1 we don’t
have any faulty case. At this point stop for this problem.

Main Observation: Each successful evaluation which returns the greatest

output corresponds to a correct simulation.

Proof of the Main Observation: Assume in a successful evaluation which

returns the maximal output, zj = 0 but ϕj is satisfiable.

Furthermore assume j is chosen minimally (i.e. no faulty evaluations before-

hand). Therefore there must be another successful simulation of N , for which
the current and all until the j-th oracle call do correspond, but now zj = 1 and a
satisfying assignment Tj for each ϕj is guessed. Afterwards all remaining oracle
calls create correct simulations.

Output of the first successful evaluations: z1 , . . . , zj−1 , 0, . . ..

Output of the current evaluations: z1 , . . . , zj−1 , 1, . . ..

Thus the Main Observation is valid.

N therefore solves the MAX-OUTPUT problem and N can be constructed with

a logarithmic space boundary. S is trivial, because F (x) is written by N to the
tape.

3.5 MAX-WEIGHT

Given. a SAT equation ϕ and a weight for each clause.

Find. Truth assignment in such a way that the sum of weights of satisfying
clauses is maximal.

Theorem 2.4. MAX-WEIGHT-SAT ∈ FPNP -complete.

26
Proof:

1. Using binary search and a SAT oracle we can evaluate the greatest total
weight of satisfiable clauses. With variable fixitation like in FSAT we get
an assignment which has the target function value. Thus MAX-WEIGHT
∈ FPNP .
2. Reduce MAX-OUTPUT to MAX-WEIGHT-SAT. The proof of the the-
orem by Cook uses a non-deterministic N and an input 1n . A boolean
equation ϕ(N, n) is constructed such that each satisfying assignment of
ϕ(N, n) corresponds to a correct evaluation of N to 1n .

We use this construction. All clauses of ϕ(N, n) are assigned a high weight
value; for example 2n .

We introduce new clauses. The weight are chosen in such a way that in every
optimal solution of MAX-WEIGHT-SAT all clauses of ϕ(N, n) are satisfied.

ϕ(N, n) contains a variable for the symbol at each position for each string of
N for every step. There are n variables (y1 , y2 , . . . , yn ). Those represent the
positions in the output string if N halts. We add n clauses to ϕ(N, n). Each of
them contains only one literal:

(yi ) i = 1, . . . , n

The weights of this clause 2n−i . Remark: Those clauses together are weighted
less than each ϕ(N, n).

Hypothesis: Every optimal solution of MAX-WEIGHT-SAT corresponds to a

correct evaluation of N for 1n such results in the greatest-possible output of N
to 1n .

This defines the R-function reduction of the proof. Coming to the S-function. . .

From an optimal solution of MAX-WEIGHT-SAT we can construct an optimal

output for MAX-OUTPUT easily (even from the optimal target function value).

Also we can show that

TSP ∈ FPNP complete

Reduction (for example) via MAX-WEIGHT-SAT. A lot of other combinatorial

optimization problems are FPNP -complete. For example the Knapsack-problem
is a weighted version of MAX-CUT and BISECTION-WIDTH. The following
algorithms are not in FPNP (they distinguish from the previous ones because
those algorithms have a polynomial upper bound for the maximized target func-
tion value (for example number of edges of a clique ≤ n):

• MAX-CLIQUE (find complete subgraph of undirected graph G with max-

imum cardinality).
• Unweighted MAX-CLIQUE (maximize number of edges)

27
 • Unweighted MAX-WEIGHT-SAT (maximize number of satisfiable clauses)

For such problems (specifically MAX-CLIQUE) O(log n) oracle calls are suffi-
cient. We get new complexity classes:

PNP[log n] FPNP[log n]

A class of languages which can be decided by a polynomial oracle M to the

input x with O(log |X|) SAT-oracle calls.

Theorem 2.5 MAX-CLIQUE is FPNP[log n] -complete.

Other possible constraints in these classes: The polynomial number of oracle

calls do not occur adaptively (therefore without knowledge of the previous oracle
call results) The calls will get parallelizable (para-classes) [correctly denoted
with a vertical =-sign at the bottom].
NP[log n]
NP
Theorem 2.6 PPARA = P NP[log n] . FPNP
PARA = FP .
NP
Proof of PPARA = P NP[log n]

P NP[log n] ⊆ PPARA
NP

Consider an oracle-TM which makes O(log n) (adaptive) NP-oracle calls. For

the first call: 2 possible outcomes (depending on yes/no answer). For second call
the same, etc. This gives us a complete decision tree describing 2k·logn ∈ O(nk )
oracle calls and answers.

For the simulation with a non-adaptive Oracle-TM determine all O(nk ) oracle
calls and answers beforehand. The correct decision path can be derived.

NP
PPARA ⊆ P NP[log n]

NP
L is a language ∈ PPARA . L can be evaluated with a polynomial number of
non-adaptive oracle calls. Firstly we determine with O(log n) NP-oracle calls
(binary search) the number K of Yes-answers for the given non-adaptive oracle
calls.

Remark. The question whether the given set of SAT equations ≥ K is satisfi-
able is a NP-question.

Finally, are there K satisfying truth assignments for K of the n expressions such
that if all others are not satisfiable (for us this is the case because of K max.)
the oracle-TM halts with acceptance? L ∈ P NP[log n] .

4 The polynomial hierarchy

δ0 P = Σ0 P = Π0 P := P

28
 δi+1 P := P Σi P
Σi+1 P := NPΣi P
Πi+1 P := co-NPΣi P
[
PH := Σi P
i≥0

Remark: This definition is based on the oracle turingmachine concept. We

will consider alternative definitions later on. Instead of Σi P also ΣP
i is denoted
in literature.

4.1 Layer 1

δ1 P = P P = P
Σ1 P = NPP = NP
Π1 P = co-NPP = co-NP

4.2 Layer 2

∆2 P = P NP
Σ2 P = NPNP
Π2 P = co-NPNP

Goal: Characterization of Σi and Πi (recursive and non-recursive).

Theorem 2.7. L is a language and i ≥ 1. We can state:

L ∈ Σi P ⇔ ∃ polynomial balanced relation R

R is a polynomial balanced relation such that the language {x; y : (x, y) ∈ R}

originates from Πi−1 P and L = {x : ∃y with (x, y) ∈ R}.

Term “polynomial balanced”. A binary relation R is called polynomial

balanced iff ∃k ≥ 1 such that (x, y) ∈ R ⇒ |y| ≤ |x|k .

Proof by induction (by i):

1. With i = 1, Σ1 P = NP The stated theorem results from the known

characteristics of NP with a certificate. We know:

L ∈ NP ⇔ ∃ in polynomial time decidabe, polynomial balanced relation R

with L = {x : (x, y) ∈ R for some y}

Induction introduction accomplished.

29
 2. i ≥ 2 and step i − 1 → i
• “⇐”: There exists a relation R according to the theorem (right part
of the equation) additionally L ∈ Σi O = NPΣi−1 P .
Thus, we have to define a non-deterministic oracle turingmachine
which uses an oracle from Σi−1 P and decides L.
For the input x this TM guesses the associated y and asks Σi−1 P
oracle, whether (x, y) ∈ R (or actually— because R is a Σi−1 P
relation—whether (x, y) ∈/ R).
• “⇒”: L ∈ Σi P (additionally with a corresponding relation R).
Because L ∈ Σi P there exists a non-deterministic oracle turingma-
chine M ? with oracle K ∈ Σi−1 P such that M ? decides L. Because
K ∈ Σi−1 P there exists a polynomial balanced relation S with S
decidable in Πi−2 such that z ∈ K ⇔ ∃w : (z, w) ∈ S.
3. We construct R with the help of S. We know x ∈ L ⇔ there exists a valid,
accepted computation of the oracle turingmachine M k for x (several steps
of M k are oracle calls [return Yes/No answers]). The associated certificate
is y.
4. For each oracle call zi with answer “Yes” a certificate wi with (zi , wi ) ∈ S
is returned.
5. Define R has followed: (x, y) ∈ R ⇔ y describes an accepted evaluation of
M k to x together with the certificates wi for all oracle calls zi with answer
“Yes”.
6. Additionally R has the desired properties.
Hypothesis. (x, y) ∈ R can be decided in Πi−1 P .
Proof of hypothesis. First of all we have to test whether all steps M ?
(with k computed by the oracle) are valid. This is possible in determin-
istic polynomial time. Then we have to test for a polynomial number of
pairs (zi , wi ) whether (zi , wi ) ∈ S. This can be computed in Πi−2 P and
therefore also in Πi−1 P . Because k ∈ Σi−1 P , this question is a Πi−1 P
question.
7. So (x, y) ∈ R ⇔ A sequence of questions of Πi−1 P have the answer “Yes”.
Show in Πi−1 P .

Corollary 2.8. (recursive characteristic of Πi P ) L is a language and i ≥ 1.

Then: L ∈ Πi P ⇔ there exists a polynomial balanced
 relation R such that the
language {x; y : (x, y) ∈ R} is in Σi−1 P and L = x : ∀y with |y| ≤ |x|k : (x, y) ∈ R .

Proof of Corollary 2.8. Is a direct result from Theorem 2.7.

We are looking into non-recursive characteristics now:

Corollary 2.9. (non-recursive characteristic for Σi P ) L is a language and i ≥ 1.

Then L ∈ Σi P ⇔ there exists a poly., balanced, in polynomial time decidable
(i+1)-ary relation R with L = {x : ∃y1 ∀y2 ∃y3 . . . Qyi : (x, y1 , y2 , . . . , yi ) ∈ R}
with Q as quantifier (∀ in even case, ∃ in odd case). This . . . describes an
alternating quantifier sequence.

30
Proof results from the repeated application of Theorem 2.7 and Corollary 2.8
and adhesion of certificates.

Remarks:

• Analogously the non-recursive characteristic of Πi P . But this sequence

starts with an all-quantifier.
• An alternating introduction of the clases Σi P and Πi P would be by an
alternating quantifier usage in the relational representation.
• Another alternative for this introduction would be by using a so-called
“alternating oracle turing machine” where every oracle call can be labelled
with quantifier ∃ or ∀ (for layer i with i − 1 quantifier changes allowed).

Theorem 2.10. If there is any i ≥ 1, Σi P = Πi P , then ∀j > 1, Σj P = Πj P =

δj P .

Remark. For i = 1 we can conclude that (under the assumption that P = NP)
the layer 1 already does not tell us anything and the whole hierarchy definition
is useless (“collapse at layer 1”). If the assumption does not hold we still do not
know whether or not the hierarchy collapses at any finite layer.

Conjecture. P ̸= NP and polynomial hierarchy does not collapse for any finite
layer.

Proof. It suffices
Σi P = Πi P
⇒ Σi+1 P = Πi+1 P

Theorem 2.7
Consider L ∈ Σi+1 P ⇒ ∃ relation R in Πi P (= Σi P according to
assumption) with L = {x : ∃y : (x, y) ∈ R}. R is also in Σi P ⇒ (x, y) ∈ R ⇔ ∃z
with (x, y, z) ∈ S for any relation S of Πi−1 P .

Thus x ∈ L ⇔ ∃ a string y; z such that (x, y, z) ∈ S with S ∈ Πi−1 P ⇒ L ∈ Σi P

(analogously for Πi P ).

4.3 QSATi

Consider a quantified SAT problem QSATi with i quantifiers.

Given. A SAT equation ϕ. The variables in ϕ are partitioned in i classes

X1 , X2 , . . . , Xi .

Find. Is it true, that a partial truth assignment for X1 exists such that
for all partial truth assignment of variables in X2 there exists partial truth

31
 assignments for variables in X3 such that . . . X1 .
Problem can analogously defined to start with ∀ quantifier.
∃X1 ∀X2 ∃X3 . . . ϕ

Theorem 2.11. QSATi is Σi P -complete.

Remark: The variant with an ∀ quantifier at the start is Πi P complete.

This is an example at the i-th layer. We now know that at the i-th layer complete
problems do exist.

This lecture took place on 28th of Oct 2013.

Proof.

1. QSAT ∈ Σi P follows directly from the non-recursive representation of

Σi P .

2. We show Σi P -hardness. Consider L ∈ Σi P . Find a reduction of L to

QSAT: Transform L to the non-recursive characteristic of Σi P (corollary
2.9). Relation R can be decided in polynomial time. ⇒ ∃ polynomial
turingmachine M which accepts all input strings (xi ; y1 ; . . . ; yi ) such that
(x; y1 ; . . . ; yi ) ∈ R.

Remark. (for Σi P -hardness) If i is odd (i even analogously) we use the method

introduced in the proof of the Cook-Levin Theorem. ∃ SAT equation ϕ which
reflects the computation of M . We arrange the variables in i + 2 variables. The
set X corresponds to the variables in the input string of the form (xi ; y1 ; . . . ; yi )
as mentioned previously (each set is separated by a semicolon). Show: All
variables that describe the resulting components of M .

For a fixed assignment of the variables X, Y1 , . . . , Yi we state: The resulting

boolean expression is satisfiable ⇔ the variables of the input string describe a
string accepted by M .

x is a random string of appropriate length and substitutes in ϕ the corresponding

assignment for the variable X.

We know:

x ∈ L ⇔ ∃y1 ∀y2 , . . . , ∃yi such that R(x, y1 , . . . , yn )

| {z }
i is odd

Thus for the partial assignment of the variable X with x there are values in Y1
such that for all values of variables in Y2 . . . there are values for variables in Yi
and there are values for the variable in Z such that ϕ is satisfied.

Another example. Multilevel-Optimization is Σi P -complete.

32
Theorem 2.12. If there is a PH-complete problem, the polynomial hierarchy
collapses at finite level.

Proof. Assume L is PH-complete. Thus there exists a i ≥ 0 : L ∈ Σi P . Each

language L′ ∈ Σi+1 P can be reduced to L. Because all levels of PH are closed
under reductions, we conclude L′ ∈ Σi P ⇒ Σi+1 P = Σi P . Thus the polynomial
hierarchy collapses.

Remark: It’s an open question whether PH-complete problems exist (conjec-

ture: no).

Theorem 2.13: PH ⊆ PSPACE (conjecture: PH ⊂ PSPACE).

Remarks:

1. Quantified boolean equation problem.

Q1 x1 Q2 x2 . . . Qk xk ϕ(x1 , . . . , xk ) Qi ∈ {∀, ∃} .
| {z }
quantifier-free

is PSPACE-complete.
2. If we define Σi P , Πi P with an alternating oracle-TM, we have to assume
that the number i of quantifiers is not part of the input. This is a con-
straint for the number of allowed quantifiers. There is no constraint for
PSPACE.
3. Because there are PSPACE-complete problems, we can conclude from
PSPACE = PH that the polynomial hierarchy collapses at finite level.

4.4 BPP ⊆ Σ2 P

Theorem 2.14. BPP ⊆ Σ2 P

Proof. L ∈ BPP. So there exists a turingmachine M with computations of

length p(n) (p polynomial) for inputs of length n such that M decides L using
majority vote (probability for false negatives is for example ≤ 14 ). For each
p(n)
input x of length n denote A(x) ∈ {0, 1} the set of accepted computations.

We can assume that

1 1
∀x ∈ L : |A(x)| ≥ 2p(n) (1 − ) error probability ≤ n
2n 2
1 1
∀x ∈
/ L : |A(x)| ≤ · 2p(n) error probability3 ≤
2n 2n

Denote U the set of bit strings of size p(n).

Definition. a, b ∈ U with a ⊕ b as component-wise XOR operation.

3 can easily be achieved by repetition

33
Observation.

1. a ⊕ b = c ⇔ c ⊕ b = a
2. (a ⊕ b) ⊕ b = a. The function ⊕b is reversible with the same argument.
3. a ∈ U is constant, r ∈ U is random value. So a ⊕ r is random value.

Definition. (Translation) t ∈ U . A(x) ⊕ t := {a ⊕ t| a ∈ A(x)} (translation of

A with t).

From the second property we can conclude that |A(x) ⊕ t| = |A(x)|∀t ∈ U .

We will show that we can return for x ∈ L a relatively small number of transla-
tions which cover U (cardinality p(n)) whereas for every x ∈
/ L there is no such
representation.

x ∈ L. t1 , t2 , . . . , tp(n) ∈ U is a random sequence of p(n) translations (p(n)2

coin throws).

b ∈ U . b is “cover by” t1 , . . . , tp(n) if b ∈ A(x) ⊕ tj for any j ∈ {1, . . . , p(n)}.

Question: What’s the probability that b is covered?

observation
b ∈ A(x) ⊕ ty ⇔ b ⊕ tj ∈ A(x)
| {z }
randomized because of third property

It has the same distribution like t1 , . . . , tn .

Because x ∈ L ⇒ probability(b ∈
/ A(x) ⊕ ti ) ≤ 1
2n = 2−n .

• Probability that b is not covered by any tj is ≤ 2−n·p(n) .

• Every element of U is not covered with probability ≤ 2−n·p(n) .
– Probability that some element of U is not covered:

≤ 2−np(n) · |U | = 2−(n−1)·p(n) ≪ 1
|{z}
2p(n)

A sequence of p(n) random translations covers the complete U with correspond-

ingly high probability.

In the case x ∈ / L A(x) is an exponentially small portion of U . Thus for any

sufficiently large n there is no sequence of p(n) translations that cover U com-
pletely.

Conclusion. x ∈ L ⇔ there exists a sequence of polynomial (p(n)) number of

translations that cover U .


L = x : ∃T = (t1 , t2 , . . . , tp(n) ) [tj translations]

34
such that ∀b ∈ U ∃j ∈ {1, . . . , p(n)} : b ⊕ tj ∈ A(x).

The last exists operator can be rewritten as: (b ⊕ t1 ∈ A(x)) ∨ (b ⊕ t2 ∈ A(x)) ∨

. . . ∨ (b ⊕ tp(n) ∈ A(x)) (can be tested in polynomial time). Therefore we get a
Σ2 p-characterisation of L; thus L ∈ Σ2 p.

We know that BPPi−1 is closed under complement ⇒ BPP ⊆ Σ2 P ∩ Π2 P .

5 Counting Problems

5.1 Three counting problems

Not combinatorics, but relations (specially in graph theoretical) problems. From

a complexity theoretical point of view: How many solutions are there for a given
problem?

5.1.1 #SAT

Given a SAT equation, find the number of satisfying truth assignments. Analo-
gously the number of hamiltonian paths - number of hamiltonian paths. Anal-
ogously the number of cliques - number of cliques with ≥ k vertices.

Those examples are counting variants for problems where the basic decision
problem is NP-complete.

5.1.2 #Matching / Permanent

Given. A bipartite graph G = (V ∪ U, E) with U = {u1 , . . . , un } and

V = {v1 , . . . , vn }.

Find. Number of perfect matchings in G (this corresponds to a 1-by-1

mapping of vertices in U to vertices in V ).

We consider the adjacency matrix AG which is associated to G. A(G) = (aij )

with (
1 if {ui , vi } ∈ E
aij =
0 else

When does the association π (permutation π ∈ Sn ) describe a perfect matching?

 Y
aiπ(i) = 1∀i ∈ {1, . . . , n} ⇔ ui , vπ(i) ∈ E∀i ∈ {1, . . . , n} ⇔ aiπ(i) = 1
i=1

35
The number of perfect matchings is given by the permanent
n
X Y
perm(A) = aiπ(i)
π∈Sn i=1

π ∈ Sn is the set of permutation of {1, . . . , n}. Compare

X n
Y
det A(G) = (−1)sign(π) · aiπ(i)
π∈Sn i=1

Question. There exists an efficient algorithm for computation of perm(A) or

at least for 01-matrices (restriction of values)?

5.1.3 Subgraph paths

Given a graph with m edges and n vertices. Find the number of subgraphs of
G (2m at maximum) which contain a path from 1 to n.

5.2 #P

Pronounced as “number P” or “sharp P”. Is the class of counting problems that

are associated using relation Q. All previously mentioned problems are part of
#P.

Q is a polynomial balanced relation decidable in polynomial time. The associ-

ated counting problem for Q: Given an x, find the number of y with (x, y) ∈ Q?

Question. Are there any of these problems #P-complete? (be aware an ap-
propriate reduction term is necessary) #SAT ∈ #P-complete?

This lecture took place on 4th of Nov 2013.

For the definition of #P-completeness we have two approaches:

• by using the reduction term, we defined for function problems (number of

solutions corresponds to solution). We need an S which gives us the num-
ber of solutions for B with the number of solutions for A. This subclass
of reductions is popular for counting problems: “paisimonious reductions”
(dt. „sparsam”) (number of solutions is preserved or with a factor k,
k ∈ N).
• Informally:

A function problem f is #P-complete iff problem is in #P

and the existence of a polynomial algorithm for evaluation of f
implies #P = FP.

36
 We extend the concept of oracle TMs to simple function languages (for
∗
oracle TMs: access to an oracle L ⊆ {0, 1} and we can answer questions
∗
like q ∈ L? in one oracle call). Now we provide a function f : {0, 1} →
∗
{0, 1} to the turingmachine M (including an oracle to decide f ). (f :
∗
{0, 1} → N can be applied to binary representation). Thus m has no
access to the language L = {(x, i) with fi (x) = 1} with i as bit index. For
∗ ∗
function f : {0, 1} → {0, 1} FPf denotes the set of functions which can
be decided with a polynomial turingmachine with oracle calls to f .
Formally: Function f is #P-complete iff f ∈ #P and ∀g ∈ #P we conclude
g ∈ FPf . Obviously f ∈ FP ⇒ FPf = FP.

Theorem 3.1. f is #P-complete and f ∈ FP. Thus FP = #P.

Theorem 3.2. #SAT is #P-complete.

Sketch of proof. Consider the proof of the Cook-Levin theorem. Reduction

of an arbitrary language to SAT. In polynomial time evaluable function f :
∗ ∗ ∗
{0, 1} → {0, 1} such that ∀x ∈ {0, 1} : f (x) ∈ SAT ⇔ x ∈ L.

This proof however provides more than that: It provides a method to convert a
certificate x ∈ L to a certificate f (x) ∈ SAT [ie. a satisfying truth assignment]
and vice versa. This is a bijective relation.

Thus the number of satisfying assignments is the number of certificates for x

(the reduction is a paisimonious reduction).

Theorem 3.3. #Hamiltonian-Path Cycle ∈ #P-complete.

Reduction via #SAT (can be dervied from the proof that TSP is FPNP com-
plete).

Question. Are there any #P-complete counting problems whose base problem
(as decision problem) lie in P?

Answer. Yes. Two examples:

• Permanent is #P-complete (see theorem 3.4).

• #CYCLE (given a graph, count number of cycles). There exists a poly-
nomial algorithm for #CYCLE ⇒ P = NP.

Theorem 3.4. (Valiant, 1979) Permanent ∈ #P-complete.

Proof. Goal: perm(A) evaluation for 01 matrices. We use a proof using general
matrices (integers as entries of the matrix).

1. For 01-matrix A we already know perm(A) = the number of perfect match-

ings in a bipartite graph G(A) which are associated to A.

 matrix A
2. For with ai,j ∈ {0, ±1}.
 It is easy to state that perm(A) =
n n
| π ∈ Sn : πj=1 aiπ(i) = 1 | − | π ∈ Sn : πi=1 aiπ(i) = −1 |. Two #SAT
call are sufficient.

37
 3. A is a n × n matrix with integers as entries. We can consider A as
weighted adjacency matrix of the directed, complete4 graph (rows) with
V = {v1 , . . . , vn } (columns) and the following vertices set: The edge
(vi , vj ) has weight ai,j . Loops are allowed (and are contained).

graphrepresentation.pdf

Observation. Every permutation π ∈ S corresponds to in G(A).

So-called cycle cover. Each component is a directed cycle and each vertex occurs
in exactly one component.

Define the weight of one cycle cover as the product of the weights of contained
edges: perm(A) is the sume of weights of all cycle covers.

Remark. This way we can show that Permanent ∈ FP#SAT (for arbitrary
integer matrices).

In the following we will omit edges with weight 0 in our drawings (cycle cov-
ers containing such edges have weight 0; therefore we can ignore it). In the
construction we will also allow multiple edges.

cycle_cover.pdf

Observation. (cycle cover observation) Furthermore we observe: G consists of

G′ and G′′ (G′ can be arbitrary). G′′ has 2 cycle covers with weight ̸= 0 (be
aware that self-loops are not drawn).

Each cycle cover of G consists of one cycle cover of G′ and one cycle cover of
G′′ . Therefore we have two possibilities: one with 1 and one with −1 (with
weight ̸= p). For cycle cover with weight ω for G′ we get the terms ω and −ω.
The overall weight sum of all cycle covers is 0.

Now we reduce #3SAT (which is #P-complete) to Permanent.

Now a 3SAT equation ϕ with n variables and m clauses is given. We will con-
struct an integer matrix A or an equivalent directed, weighted graph G(A) (let’s
4 complete, because we assign weight 0 to missing edges

38
call him G̃) (negative entries will be used) such that perm(A) = perm(G̃) = 43m
(#ϕ; which is the number of satisfying assignments of ϕ). G̃ ⇒ P(A).

Later on G̃ → Ĝ with 0-1 weights such that perm(G̃) evaluates perm(Ĝ) (then
we have shown that 01-problems are #P-complete).

The central idea is that there are two types of cycle covers in G̃. The ones that
are assigned to a satisfying assignment of ϕ and the other ones. Similar to the
cycle cover observation we will use negative edge weights to achieve that the
values of cycle covers (which do not contribute to the satisfying assignment)
drop out each other.

Furthermore every cycle cover contributes 43m to each satisfying assignment for
perm(G).

perm(Ĝ) = perm(A) = 43m

(#ϕ as considered).

Question: ϕ reduces to G̃? There are 3 kinds of gadgets:

• clause-gadgets
• variable-gadgets
• consistency-gadgets (XOR-gadgets)

Consider the following 4 × 4 matrix or the corresponding graph

 
0 1 −1 −1
1 −1 1 1
A= 0

1 1 2
0 1 3 0

example_graph.pdf

Figure 5: Example graph

We can now explicitly evaluate that

• perm(A) = 0
• For perm(B): We can dervice B from A by deleting the first row and
first column ⇒ perm(B) = 0. Or we could also delete the fourth row

39
 and fourth column. Or we could also delete the first and fourth row and
column. Also for matrix it results if first and fourth row and first and
fourth column is deleted.
• perm(C) = 4 where B is created by deleting of the first row and fourth
column or the fourth row and first column of matrix A.

schematic.pdf

Figure 6: Schematic representation

The upper graph in the illustration can be derived from the upper graph and
the lower graph represents the remaining graph.

example_graph_with_g.pdf

Figure 7: Example graph with g added

For the modified example graph with g we state: The overall weight of all cycle
covers is 8 (contribution value 4 results from cycle cover with (g, 4) and (1, g) -
this corresponds to deletion of the first row and fourth column and the second
value 4 results from (g, 1) and (4, g) correspondingly.)

The contribution of the cycle cover which contains g is 0 (corresponds to matrix

A without deleting rows and columns). Also: For all cycle covers that traverse
(g, 1) and (1, g) (deleting first row and first column) we get contribution value
0. Correspondingly for (g, 4) and (4, g) for the fourth row and column.

We can use this to rewrite XOR: Assume that we have a graph H which contains
the two edges (1, 1′ ) and (2, 2′ ).

Consider the lower graph in the schematic drawing. In combination with the
definitions from above: The sum of all weights, the cycle cover of H which
traverses (1, 1′ ), but does not traverse (2, 2′ ). All other cycle covers result in
value 0.

Regarding clause gadgets: The traversal of an external edge in one cycle cover
corresponds to one not-satisfied corresponding variable. There are three external
edges, one per variable.

40
 xor_gadget.pdf

Figure 8: XOR gadget (upper drawing) and its schema (lower drawing)

clause_gadget2.pdf

Figure 9: Clause gadget (upper drawing) and its schema (lower drawing). Red
edges are external ones.

In each clause we have 3 variables which are represented by the two external
edges of this clause. These external edges are connected to the edges of the
corresponding variable gadgets via XOR gadgets (which is equivalent to our
construction of for the hamiltonian path problem).

Observation. There does not exist a cycle cover which goes through all 3
external edges. Furthermore for all proper subsets of the set of 3 external edges
(including the empty set): There exists 1 cycle cover which contains exactly all
those edges and no other ones.

With this specific setup we achieve that the remaining graph G′ satisfies:

perm(G′ ) = 43m (#ϕ)

with #ϕ has the number of satisfying assignments for ϕ.

Remark. The corresponding external edges are omitted for clauses which con-
tain xi positively and added for clauses containing xi negatively.

5.2.1 Reduction to 0-1 matrices

Computed in 2 steps:

1. Construct a graph G′′ with edge weights ∈ {0, ±1} and perm(G′′ ) =
perm(G′ ). Edge weights which are powers of 2 (2k ) can be represented
by a path of k edges of weight 2 (all new inner edges of this path do only
′
occur in this path). For an edge of weight 2k + 2k we construct to two
′
parallel paths with weights 2k and 2k . Each edge whose weight is not a

41
 variable_gadget.pdf

Figure 10: A variable gadget (2 possible cycle covers)

power of 2 can be represented as a set of parallel paths which use binary

representation. If L is the number of bits to describe the weights in G′ ,
the graph is blown back with factor O(n · L2 log n). 2 can be replaced by
parallel 1-edges. Therefore we get only {0, ±1} edges.
2. Removal of negative weights by transition to modulo arithmetics. Transi-
tion to mod 2q + 1 with q = n2 . Now we can state −1 ≡ 2q mod 2q + 1.
The permanent mod 2q + 1 keeps the same, if edges with weight −1 get
replaced by edges with weight 2q . Those get replaced by a subgraph with
all weights 1 edges (size O(q) = O(n log n)). Graph G′′′ with weights 0, 1
such that the permanent of G′′′ is used to evaluate the original permanent
(rest mod 2q + 1). Blow back with factor O(< n log n).

Remark. The exact counting of solutions is sometimes computationally expen-

sive. Let’s do approximations!

α-Approximation
0<α<1
∗ f (x)
A provides an α-approximation for f : {0, 1} → N if f (x) ≤ A(x) ≤ α ∀x.

There are counting problems for which even an approximative algorithm for
constant α > 0 is difficult. But there are also problems which are easy to
address approximatively.

For the (01) PERMANENT problem: There exists a FPRAS (fully polynomial
randomized approximation scheme) algorithm which (for a given ε and δ) eval-
∗
uates a (1 − ε) approximation for requested function f : {0, 1} → N (for 01
perm = number of perfect matchings) with probability 1 − δ (with probability
1 1
δ the algorithm is allowed to be faulty) in time p(n, log , log ).
| δ
{z ε}
polynomial

Remark. In case N = NP, then for all problems in #P a FPRAS5 can be

found. Even a FPTAS6 (see last chapter of this lecture).
5 randomized
6 deterministic

42
5.2.2 Classification of #P: How powerful is counting?

Problems in #P are solvable with polynomial limited space (eg. lexiographical

enumeration). Therefore #P is not more powerful than PSPACE. PH is not
more powerful than PSPACE.

What’s the relation of #P and PH

|{z} ?
|{z}
class of functions class of languages

Theorem 3.5. (Theorem of Toda) PH ⊆ P#P and PH ⊆ P#SAT .

Therefore we can solve every problem in the polynomial hierarchy if an oracle

for a #P-complete problem is provided.

Remark. #P = FP ⇒ NP = P ∧ PH = P.

Conclusion. Informally: Counting is more powerful than the polynomial hier-

archy.

Relation between PP and #P? So the question of PP is whether or not

half of the evaluations result in acceptance. Thus we are only interested in the
MSB of the number of accepted evaluations. #P is interested in the values of
all bits.

Idea. We are only interested in the LSB (parity). This defines the complexity
class ⊕P (“parity P”, “odd P”).

A language L is in ⊕P if there is a polynomial turingmachine M such that for

all strings x we can state that:

x ∈ L ⇔ number of accepting evaluations of x at M is odd

(equivalently there exists a polynomially balanced relation R decidable in poly-

nomial time such that x ∈ L ⇔ the number of y with (x, y) ∈ R is odd).

5.2.3 Problem: ⊕ SAT

Given. A SAT equation.

Find. Is the number of satisfying assignments odd?

Theorem 3.6. ⊕ SAT is ⊕P-complete.

Proof. ⊕ SAT ∈ ⊕ P according to definition. ⊕ P completeness follows from

parsimonious reducibility of a problem in #P to #SAT.

Analogously, ⊕ Hamiltonian Path/Cycle ∈ ⊕ P-complete.

43
5.2.4 ⊕ Perm in ⊕ P-complete

Can be solved in polynomial time because the parity of the determinant corre-
sponds to the parity of the permanent (polynomial time!).

Theorem 3.7. ⊕ P is closed with respect to complement. Proof will be done

in practicals.

Theorem 3.8 NP ⊆ RP⊕ P . We will define a polynomial Monte Carlo algorithm

for SAT when using a ⊕ SAT oracle.

5.3 NP ⊆ RP⊕ P

Proof. We will define a polynomial Monte-Carlo algorithm for SAT which has
access to a ⊕ SAT oracle. Given a SAT equation ϕ with variabes x1 , . . . , xn .

Is S ⊆ {1, . . . , n}.

Definition. (hyperplane ηs ) A hyperplane ηs is a boolean expression requiring

that an odd number of variables with indices of S have value true.

y0 , . . . , yn are new variables. ηs can be constructed with the following set of

rules:

• Add (y0 ) as clause.

• Add (yn ) as clause.
• For all i ∈ {1, . . . , n} add clause C̃i
(
(yi−1 ) ⊕ xi i∈S
C̃i =
(yi−1 ) i∈
/S

(Transform this into a DNF.)

Is ϕ0 = ϕ the whole SAT equation. For i = 1 to n repeat the following steps:

1. Create a random set Si ⊆ {1, . . . , n} and set

ϕi = ϕi−1 ∧ ηsi

Apply ⊕-SAT oracle to ϕi . If ϕi ∈ ⊕ SAT, then ϕ satisfiable. Return “YES”. If

ϕi ∈
/ ⊕ SAT ∀j = 1, . . . , n, return “No or probably no”.

Claim. The algorithm above is a Monte-Carlo algorithm.

Proof.

44
• No false positive answers.

ϕi ∈ ⊕SAT ⇒ ϕi has ≥ 1 satisfying assignment

⇒ ϕ has more than 1 satisfying assignment

7
• Probability for false negatives? Claim: ≤ 8 (6 repetitions required for
probability ≤ 12 ).
Observation. If the number of satisfying assignments for ϕ is between 2k
and 2k+1 for 0 ≤ k < n, then ϕk+2 has exactly one satisfying assignment
with probability ≥ 81 .
Proof. T is the set of satisfying assignments for ϕ. 2k ≤ |T | ≤ 2k+1 .
We define that two truth assignments in hyper planes ηs do correspond if
both satisfy ηs or both do not satisfy ηs .
Let’s fixate t ∈ T . Consider t̂ ∈ T . The probability that t corresponds
with t′ at the first k + 2 hyper planes (ηs1 , ηs2 , . . . , ηsk+2 ) is 2k+2
1
(which
results from the fact that all (k + 2) sets s1 , . . . , sk+2 contain an even
number of variables where t and t̂ do not correspond and these events are
indepedent and occur with probability 21 ).
Build sum over t̂ ∈ T \ {t}. The probability that t corresponds to some
t̂ ∈ T \ {t} at the first k + 2 hyper planes is

|T | − 1 1
≤ <
2k+2 2
Opposite probability (they do not correspond) is
1
≥
2

Obviously we conclude that t satisfies the first k + 2 hyperplanes with

1
probability 2k+2 .
We now state: If t satisfies the first k + 2 hyper planes then t is the only
assignment of T with this property with probability ≥ 21 (the probability
that t satisfies the first k + 2 hyper planes does not interfere with the
probability that t̂ ̸= t do not correspond).
1
With probability ≥ 2k+3
(= 21 2k+2
1
) t is the only satisfying assignment for
ϕk+2 .
This is valid ∀t ∈ T . Because of |T | ≥ 2k the probability that such t ∈ T
exists is
1 1
≥ 2k · k+3 =
|{z} 2 8
lower bound for |T |

Our observation is proven.

If ϕ is satisfiable, then ∃k ∈ {0, . . . , n − 1} with number of satisfying
assignments for ϕ between 2k and 2k+1 . Therefore at least one of the
equations ϕi has exactly one satisfying assignment with probability ≥ 81
and thus an odd number and thus ϕi ∈ ⊕ SAT.
Our claim is proven.

45
6 Interactive protocols

6.1 Introduction and Terminology

In math the classical concept of proofs is strongly related to the idea of NP

certificates. But the verification is done without any interaction. We want to
introduce 2 roles/actors:

• Prover
• Verifier

A prover creates proofs and the verifier checks those proofs. During this process
they can interact by exchanging messages. At the end the verifier has to decide
whether or not the proof gets accepted.

Open question. Which power do prover and verifier have?

6.1.1 Variant 1: Prover and verifier act deterministically

∗ ∗
Definition. (Message exchange) f, g : {0, 1} → {0, 1} , k ∈ N0 (can depend on
∗
input length). k rounds of interaction is a result of strings a1 , . . . , ak ∈ {0, 1} .

a1 = f (x)

a2 = g(x, a1 )
..
.
a2i+1 = f (x, a1 , . . . , a2i ) 2i < k
a2i+2 = g(x, a1 , . . . , a2i+1 ) 2i + 1 < k

In the end the verifier decides whether or not to accept the proof. This decision
only depends on x and ai ⇒ {0, 1} for acceptance or rejection.

Requirement for a deterministic proof system. The language L has a

deterministic proof system with k rounds if there is a deterministic turing ma-
chine M which satisfies the following constraint for an input x, a1 , a2 , . . . in
polynomial time in |x| and with k message exchanges:
∗ ∗
x ∈ L ⇒ ∃ |{z}
P : {0, 1} → {0, 1}
proof

with probability(verifier accepts P ) = 1

∗ ∗
x∈
/ L ⇒ ∀ |{z}
P : {0, 1} → {0, 1}
proof

with probability(verifier does not accept P ) = 1

46
Complexity class d|P (“d” for deterministic) contains all languages for which a
deterministic proof system with poly(n) rounds can be found.

Lemma 4.1. d|P = NP. (Proof is given in practicals.)

Consequence: With a purely deterministic approach we don’t gain any new

results. Interaction is not relevant.

6.1.2 Variant 2

More interestingly, we consider a randomized verifier. Verifier can return false

answer, but we add constraints for the probability.

Power of verifier. Polynomial randomized algorithm.

Power of prover. Exponential runtime. Deterministic is enough (or alterna-

tively PSPACE).

6.2 Example Scenario

The input x is known to both actors. Bob (prover) and Veronica (verifier)
act alternatingly. Bob runs algorithm B̃ and Alice runs algorithm Ṽ . They
exchange messages m1 , m2 , . . . of polynomial size in |x|.

Assume we start with Bob (could also be Veronica).

m1 = B̃(x) first message of Bob, depending on x

..
.
m2i = Ṽ (x, m1 , . . . , m2i−1 , Vi ) message by Veronica with Vi as private random bits
m2i−1 = B̃(x, m1 , . . . , m2i−2 )

The protocol ends with Veronica’s message “YES” (accept) or “No” (reject).
The tuple (B, V ) decides a language L for all inputs iff

• if x ∈ L, then the probability that x gets accepted by (B, V ) is greater

equal some constant between 0 and 1 (eg. ≥ 43 ).
/ L, then the probability that x gets accepted by (B ′ , V ) (with B ′ as
• if x ∈
any algorithm with exponential runtime (or PSPACE algorithm)) is ≤ 14
(or any other constant).

Remark. We can again use repeated runs to reduce the error probability.

Definition. (complexity class IP) The class of languages which can be de-
cided with an interactive protocol (of the previously described manner) with
polynomial rounds.

47
Definition. (complexity class IP(l)) The number of rounds is restricted to l.

Obviously
NP ⊆ IP (no randomization necessary)
BPP ⊆ IP (verifier does not require prover)

Result of Shamir.
IP = PSPACE

Remark. In our definition of IP we had

3
x ∈ L ⇒ ∃ proof P : probability(V accepts P) ≥
4
1
x∈
/ L ⇒ ∀ proof P : probability(V rejects P) ≤
4

3
• We could also replace ≥ 4 with = 1 without modifying complexity class
IP (non-trivial result).
1
• But ≤ 4 cannot be replaced by = 0 without falling back to NP.

• In definition of IP the verifier uses private random bits. The constraint to

public random bits leads to Arthur-Merlin proofs (AM).

We take a look at interactive protocols and their complements.

6.3 Graph isomorphism (GI)

Given. 2 graphs G0 = (V0 , E0 ), G1 = (V1 , E1 ) with |V0 | = |V1 | and

|E0 | = |E1 |.

Find. Are G0 and G1 isomorphic (G0 ∼

= G1 )?

6.4 Complement of graph isomorphism (GI)

Given. Same configuration as previously.

Find. Are G0 and G1 not isomorphic?

GI ∈ NP
⇒ GI ∈ IP(1) ⊆ IP

48
 graph_isomorphism.pdf

What about GI?

Complexity state of GI is an open question. We don’t know know whether GI

is NP-complete. Experts assume No.

Theorem 4.2. (without proof) GI ∈ IP(2) also GI ∈ IP.

Proof. Construct with an appropriate protocol:

Verifier: Select random bit b ∈ {0, 1}. Create random permutation π ∈ Sn

and determine graph H = π(Gb ). Verifier sends H to Prover.
Prover: Prover’s goal is to determine whether or not which graph (G0 or G1 )
was permuted by verifier. Determine bit b̃ ∈ {0, 1} and send it to Verifier.
(If G0 ∼
= G1 , Prover chooses bit b̃ randomly and otherwise select b̃ in such
a way that H ∼ = π(Gb̃ )).
Verifier: Accepts, if b = b̃ and rejects for b ̸= b̃.

Communication protocol with 2 rounds: Essential part of protocol: Random

bits of Verifier are private.

To prove that this is an IP protocol, consider the following 2 cases:

1. G0 ∼
= G1 : Prover always gets a graph H, which is isomorphic to G0 and
G1 independent of b. Prover cannot derive anything. Can only estimate b
with probability 12 and b̃ is the guessed value. Probability with error = 12 .
2. G0 ≇ G1 : Prover is capable of determining bit b of the Verifier and sets
b̃ = b. Therefore Veronica accepts. Probability for an error = 0.

For GI ∈ IP(2) private random bits of V are essentially important.

Definition. (notation) AM[K] or AM(k).

Definition. AM(k) results from IP(k) if the verifier has no access to private
random bits. Obvisouly AM(k) ⊆ IP(k)

Theorem 4.3. (Goldwasser, Sipser)

IP(k) ⊆ AM(k + 2)

In the practicals we will consider an AM(2) protocol for GI.

49
6.5 Regarding complexity of GI

The following theorem holds:

Theorem 4.4. (without proof) If GI is NP-complete, then Σ2 P = Π2 P . The

proof uses an AM(2) protocol.

IP protocols can be traced back to research by Goldwasser, Micali and Radkoff.

AM protocols were considered by Babai.

6.6 Interactive protocols with zero knowledge property

(ZKP)

We distinguish between perfect ZKP and its attenuations. In perfect ZKP the
prover doesn’t want to share “any” information with the verifier (to avoid abuse).
For example if the question is: Does any hamiltonian cycle exist? But prover
does not want to share the hamiltonian path itself. So generally, we want to
show that some X satisfies some property, but we don’t want to share X itself.
X might be a certificate of a NP problem.

6.7 ZKP Problem: Magic door

Given. Given is a room A and room B. They are separated by a magic door
which can only be opened by a secret. An anteroom is externally accessible
and allows to enter room A and room B through separate (simple) doors.

Find. How can Bob verify that he knows the secret without telling Alice
the secret?

Alice stays in the anteroom. Bob goes into room A and comes back from room
B. He must have changed the room through the magic door.

1. Prover and Verifier is in the external area. Prover enters the anteroom
and closes door behind himself. Prover choses bit i randomly and enters
room i. He closes the door behind himself.
2. Verifier enters the anteroom and choses bit j randomly and tells Prover
her choice.
3. Bob appears from one of the doors.
4. Verifier accepts, if Bob comes from room j.

Claim. This protocol has perfect zero knowledge property.

50
Definition. (perfect zero knowledge property) An interactive protocol has the
perfect ZKP (=PZKP) iff the following algorithm does not reveal any informa-
tion:

1. Consider a malicious Verifier, which executes (instead of V) any other

arbitrary efficient algorithm (with the intention to determine information
about Bob).
2. There is an efficient simulation algorithm which produces exactly the same
messages like the protocol (B, V ′ ).

An interactive protocol for a decision problem L has PZKP iff there is a random-
ized simulation algorithm S for every randomized algorithm V ′ with polynomial
runtime, whose expected maximum runtime is polynomial and which for x ∈ L
can evaluate all information shared via messages between Prover and Verifier
with the same probability.

Defintion. (PZK) Class of all problems with an interactive protocol for which
the verification is possible in polynomial time and the protocol is a PZKP.

For our example: This protocol is a correct interactive protocol and is a

perfect zero knowledge protocol.

Regarding interactive protocols:

Case 1 Prover does not posses secret and detects it with probability 21 .
Case 2 Prover knows secret. He can also come from the correct room and the
Verifier will accept it.

Regarding PZK: We need a Prover simulation (“Prover Double”) which looks

like Prover but does not know secret.

Bob enters anteroom. Alice accepts if Bob’s Double comes from door j. If Alice
rejects, we start a new trial. We have a finite number of trials.

Theorem 4.5. GI ∈ PZK.

Proof. Constructive proof by providing an appropriate protocol.

Given. graphs G0 , G1 (n vertices).

Prover selects one permutation π ∈ Sn randomly and determines ϕ1 = ϕ(G1 ).

He sends ϕ1 to verifier. Verifier randomly choses a bit b ∈ {0, 1} and sends b to
the prover. If b = 1, prover sends ϕ1 to verifier otherwise it sends π2 = π1 ◦ π.
π was selected (by prover) in such a way that G1 = π(G0 ). In case that this
proves isomorphism, the prover is in possession of an appropriate π.
verifier accepts ⇔ H = π2 (Gb )

Remark. We could also let the prover choose one random bit and let him
permute the graph. However, this does not change the theoretical points of it.

51
If G0 is isomorphic to G1 the prover can make verifier to accept every input.
Otherwise the probability to detect the false claim is 21 (as always: repetition is
possible).

Regarding PZK: Ṽ is an arbitrary V algorithm. We have to define a simula-

tion algorithm S for prover (ie. the corresponding protocol).

Simulation algorithm. Choose bit b̃ ∈ {0, 1} and random permutation π̃ ∈

Sn . Determine H = π̃(Gb̃ ). ϕ1 is input for V ∗ . V ∗ provides bit b∗ ∈ {0, 1}. If
b̃ = b∗ , S ∗ sends π̃ to V ∗ and provides as a decision whatever V ∗ has provided.
If b̃ ̸= b∗ , discards trial. Restart and try again.

We obtain b̃ = b∗ with probability 1 −k

P∞ −k 2 . Probability for k iterations is 2 . Ex-
pected runtime is T (n) = k=1 2 = 2.

Interesting is only the case if G0 is isomorphic to G1 because otherwise there is

no secret in runtime of V ∗ .

Regarding distribution of messages. The first message of S ∗ has the same

distribution like in case of the first message of the prover. The message cor-
responds to a random graph, which is isomorphic to G0 and G1 . With ϕ1 we
don’t share b̃.
1
b̃ = b∗ with probability =
2

In this case (b̃ = b∗ ) the messages ϕ1 and π̃ which gets received by V ∗ is

identical distributed to messages, which result from the real communication
between prover and verifier.

Remark. Further examples for PZK protocols: see practicals.

Is there any PZK protocol for Hamiltonian Path? We guess not (as for any
other NP-complete problem).

GI ∈ IP(1) ∩ co − IP(2)
| {z } | {z }
because in NP we have shown GI ∈ IP(2)

GI NP-complete ⇒ Σ2 P = Π2 (P )

If there is a ZPK protocol for NP-complete problems, polynomial hierarchy

would collapse to layer 2. Is some less strict definition satisfied for NP-complete
problems?

First idea for attenuation. Leads to SZK(P) (statistical zero knowledge

property). Simulator’s generated distribution of messages has insignificant dis-
tance to distribution of the real protocol.

ε(n) is insignificant if ε(n) is superpolynomially smaller. Therefore for all poly-

nomials p and a sufficiently large n:
1
ε(n) <
p(n)

52
The complexity SZK has some theoretically interesting properties. For example
we assume that this class lies exactly between P and NP. But it is (like for PZK)
unlikely that there exists protocols in SZK for NP-complete problems.

SZK ⊆ IP(2) ∩ co − IP(2)

Second idea of attenuation. Leads to CZK(P) (computational zero knowl-

edge property) Randomized algorithm with polynomial runtime can only distin-
guish the distribution generated by the simulator from the distribution of the
real protocol with insignificant probability.

It turns out that (under the assumption mentioned below) CZK protocols exist
for NP-complete problems.

Definition. (Standard assumption in formal cryptography) There exists a one-

way function7 .

There are several approaches to define one-way functions formally.

∗ ∗
• A function f : {0, 1} → {0, 1} (computable in polynomial time) is a
one-way function, if for all randomized polynomial algorithms A we can
state that

probability(A(y) = x′ with f (x′ ) = y) < ε(n)

∗
x ∈ {0, 1} y = f (x) ε is very small

Typical candidates for one-way functions:

• Integer factorization
• Robin functions

• Discrete logarithm

6.8 Complexity class UP

Definition. (complexity class UP) A non-deterministic turingmachine is called

unambiguous, iff for every input x there is exactly one accepting computational
path. UP is the class of languages which can be decided by unambiguous NTMs
in polynomial time.

P ⊆ UP ⊆ NP
7 f (x) is computationally “easy” to determine, but f −1 (x) is computationally infeasible.

Easy means for example a deterministic polynomial algorithm. Infeasible means for example
a randomized algorithm with polynomial runtime cannot compute x.

53
In the following consider the following variants for the concept of a one-way
function f .

∗ 1
1. f is injective and for x ∈ {0, 1} : |x| k ≤ |f (x)| ≤ |x|k for some k > 0.
2. f ∈ FP.
3. f −1 ∈
/ FP.

Theorem 4.6. UP = P ⇔ there does not exist a one-way function (as defined
above).

Proof. There exists a one-way function f . Consider a language Lf with

Lf = {(x, y) : ∃z with f (z) = y ∧ z ≼ x}

≼ is defined here by length and (secondary) lexiographically (0 < 1 < 00 <

01 < 10 < 11 < 000 < . . .).

Claim. Lf ∈ UP \ P.

Proof. We can easily show that Lf ∈ UP (ie. there exists an unambiguous TM

U , which accepts Lf ). U guesses z for input (x, y) such that |z| ≤ |y|k and tests
whether y = f (z). If result is yes, we test whether z ≼ x and accept.

Assumption. There exists a polynomial algorithm for Lf .

Claim. We can invert f with binary-search-like approach. For given y we ask

k
whether (1|y| ) ∈ Lf (a string with |y|k ones). If answer is no, then we conclude
k
that no x exists with f (x) = y (if one would exist, it would satisfy x ≤ 1|y| )
1
because |y| ≥ |x| k . If the answer is No, we ask
k−1
(1|y| ) ∈ Lf , . . .
k
until one request (1l−1 , y) ∈ Lf results in answer No. Length l of x with 1|y| .
After 2nk calls of a polynomial algorithm for Lf x is determined (an x with
f (x) = y if one exists).

So we have inverted f to y in polynomial time. f −1 ∈ FP is a contradiction to

f as a one-way function.

We now have to show that ∃L ∈ UP \ P. Is U an unambiguous turing machine

which accepts L and x is an accepting computation of U to the input y. We
define a function fu with
fu (x) = |{z}
1 y
prefix

• Therefore we have a polynomial dependency between parameter and func-

tion value because the computations of U run in polynomial time.
• fu is injective, because U is unambiguous and we use 0 as 1 at the begin-
ning (prefix) as a flag. We use 0y for an accepting computation.

54
 • If we can invert fu in polynomial time, we could decide L in deterministic
polynomial time, because inverting of fu to 1y tells us whether U accepts
y or not.

This deterministic polynomial time is a contradiction to UP \ P.

Remark. The existence of a one-way function is also relevant for other areas.
For example the existence of so-called pseudo random numbers generators and
the Goldreich-Levin theorem.

6.9 Returning to CZK problems

Important aspect: bit commitment.

2 operations:

fixitation (Festlegung) Prover chooses 1 bit b ∈ {0, 1}. Instead of sending

the bit, he sends a bitstring c(b, K) to the verifier with k as secret key of
prover.
revelation (Aufdeckung) At any later point in time verifier can request (in
case of doubt), that the prover publishes his secret b.

Desired properties.

• Fixitation should be “hidden”: Without knowledge of the key, verifier

should not be able to retrieve any information from c(b, K) about b. More
precisely: For a uniformly distributed K a randomized algorithm with
polynomial runtime has an insignificant probability to guess b from c(b, K).

• Fixitation should be bound. For no K it results that c(b, K) = c(1−b, K ′ ).

Under the assumption that an one-way function exists, there exists such a bit
fixitation method (eg. using RSA).

Theorem 4.7. (Relevant for RSA-like approach) For the Hamiltonian cycle
problem there exists a CZK protocol under the cryptographical standard as-
sumption (of one-way functions).

Proof. Consider the following interactive protocol. Is G = (V, E) an undirected

graph with n vertices. We assume that a functional bit commitment method is
provided. Now:

1. Bob chooses random permutations π ∈ Sn and permutes G (→ π(G)). He

sends for π and the vertices list of π(G) the bit commitment to Verifier.

2. Veronica selects randomly a bit i ∈ {0, 1} and sends it to Bob.

55
 3. If i = 0, Bob publishes the bits for π and π(G). If i = 1, Bob publishes
the bits of n edges of π(G) (this corresponds to the number of n edges
of G). If G has an hamiltonian cycle he also publishes the edges of the
hamiltonian cycle.
4. If i = 0, verifier accepts, if permutation π ′ ∈ Sn and the corresponding
edge list for π ′ (G) was published. If i = 1, verifier accepts if the prover
provided the edges of a hamiltonian cycle. In other cases the verifier
rejects.

Correctness as interactive protocol:

1. If G has a hamiltonian cycle, the prover and verifier can always follow the
protocol.
2. If there is no hamiltonian cycle, prover cannot prepare himself for i = 0
and i = 1. If prover permutes the correct graph correctly, he can pass the
test for i = 0, but will fail for i = 1. A malicious prover will be detected
in case i = 0 and will pass i = 1. We get an one-sided error probability 21
(can be reduced by repetition). Therefore our protocol is correct.

Assumption: G has hamiltonian cycle. The prover uses a random secret of fixed
length. Consider the protocol (B, V ′ ). Consider the prover’s revelation as the
secret publication of bit fixitation for π and π(G). Simulation algorithm:

1. Simulation of prover: Select random bit i ∈ {0, 1} and work with hy-
pothesis, such that V ′ algorithm will select i = i′ . If i′ = 0 for random
π ∈ Sn , send bit fixitation for (π, π(G)). If i′ = 1 for random π ∈ Sn ,
send bit fixitation for (π, π(H ′ )) with H ′ as the graph only consists of the
hamiltonian cycle 1, 2, . . . , n.
2. Simulate V ′ for the simulated transmitted data by prover.
3. If i ̸= i′ , we make a new start with step 1.
4. Simulate algorithm of prover. Uncover the information (as given by i).

Following from the properties of bit commitment, probability(i = i′ ) = 21 is the

expected number of iterations constant. Distribution of messages of simulation
algorithm S vs protocol (B, V ′ ). The difference is only with insignificant prob-
ability distinguishable for V . Analogously we can define CZK protocols for all
other NP-complete problems.

As a second example: Colorization of a graph with 3 colors.

Given. Undirected graph G = (V, E)

Find. 3-colorization of G: ϕ (with V ′ → {1, 2, 3}) such that

{i, j} ∈ E ⇒ ϕ(i) ̸= ϕ(j)

56
 Consider the following interactive protocol

1. The prover selects a random permutation π of {1, 2, 3}. For i = 1 to

n the prover sends bit fixitation for π(ϕ(i)) to the verifier. [If Bob has
3-colorization, he takes those as ϕ].
2. Verifier select random edge e ∈ E and sends it to the prover.
3. e = {i, j}. Prover has to publish the values transmitted in step 1 for i or
j.
4. Verifier check whether the uncovered colors are different and equate with
the values of step 1.
5. If yes, verifier accepts, else rejects them.

Probability for malicious prover detection is very small. If we repeat t = |E|

times, then error probability ∼ e−t .

Implementation of this stuff follows in RSA-like approach.

Possible approaches:

Factorization Prover chooses randomly a large prime number p. p is in binary represen-

tation
p = (pl−1 , . . . , p0 )
| {z }
digits
p is large enough such that b = p0 ⊕ p1 ⊕ . . . ⊕ pl−1 . Furthermore prover
selects prime number q with q < p. Determine n = pq. Fixitation: Prover
sends n. Revelation: Bob sends p, q. (Is hidden under the assumption that
factorization of n is difficult and bound because factorization is distinct)
RSA-like 3 colors = {00, 11, 01}. Prover generates random permutation π of the
color set and n pairs of edges of RSA key pairs (pi , qi , di , ei ) for i ∈ V .
For each vertex i ∈ V , prover computes encryption (yi , yi′ ) of the color of i
under π (when using the corresponding RSA-system). Consider bi b′i as the
two bits of π(ϕ(i)), then yi = (2xi + bi )ei mod pi qi and yi′ = (2x′i + b′i )ei
mod pi qi with xi , x′i random integers ≤ pi2qi (private computation of the
prover). Prover publishes (ei , pi , qi , yi , yi′ ) for i ∈ V (ie. public RSA key
and encrypted colors). The verifier randomly selects edge {i, j} ∈ E.
Prover now provides verifier di and dj . Verifier can now compute:
bi = (yidi mod pi qi ) mod 2
d
b′i = (yi′ i mod pi qi ) mod 2
Analogously bj , b′j
She tests whether bi b′i ̸= bj b′j . Verifier does not learn anything about the
colorization of the prover.

Conclusio. If we allow 2 or more independent proofs, we come up with multi-

prover interactive systems (complexity class mIP). This provides us NEXP
(very powerful!).

57
7 PCPs and the PCP theorem

Definition. (PCP) probabilistically checkable proofs. Or: randomized verifi-

able proofs.

Definition. Let r, q : N → N. A (r(n), q(n))-bounded PCP is a randomized

verifiable algorithm V with polynomial runtime and the following properties:

k
• For the input x of length n and a proof B = {0, 1} , the algorithm V has
r(n)
access to x and a random vector r ∈ {0, 1} (ie. V has r(n) random
bits available).
• Based on this information V computes up to O(q(n)) positions and gets
the corresponding bits of the proof as additional information (in general
V does not know the whole proof).

• Finally V computes the decision whether x gets accepted.

 
 
V (x, r, B) ∈ |{z}
0 , |{z}
1
 
reject accept

Because V must have polynomial runtime only polynomial functions are con-
sidered for r(n) and q(n) (this is some kind of resource constraint).

Definition. A decision problem L is part of the class PCP(r(n), q(n)) if there

exists a (r(n), q(n))-bounded PCP verifier V such that

∀x ∈ L ∃ proof B with probabilityz (V (x, z, B) = 1) = 1

1
/ L ∧ ∀ proofs B’ : probabilityz (V (x, z, B ′ )) ≤
∀x ∈
2
The second definition defines a constrained one-sided error.

Relation of PCP, NP and P.

NP = PCP(0, poly(n))

P = PCP(0, log (n))

PCP theorem. NP = PCP(log n, 1).

We are going to prove NP = PCP(n3 , 1).

Theorem 5.1. L ∈ PCP(r(n), q(n)) ⇒ there exists a non-deterministic algo-

rithm which decides L in 2O(r(n)+log n) runtime.

Proof. Is p(n) the runtime of V → p(n) as polynomial in n. In total ≤

g(n) · 2O(r(n)) are read.

58
Guess those bits non-deterministically (random bitstring). Simulate for ever
random bitstring the control flow of V and accept if all simulations return
acceptance. Computational runtime boundary: 2O(r(n)+log n) .

Consequence.
[
NP = PCP(log n, poly(n)) = PCP(log n, nk )
k≥0

Theorem 5.2. (PCP theorem by Arora, Lund, Motwani, 1992) NP = PCP(log n, 1).

We construct a PCP(n3 , 1) verifier for 3 SAT. A 3 SAT instance is given.

C1 , C2 , . . . , Cm clauses
x1 , x2 , . . . , xn variables

Idea. Arithmetization of 3 SAT equation.

boolean arithmetic
xi 1 − xi
xi xi
∧ +
∨ ·

Arithmetic returns polynomial of maximum degree 3.

n
Observation. Is a ∈ {0, 1} . P (a) = 0 ⇔ a is a satisfying assignment for ϕ.
P (a) > 0. P (a) returns the number of satisfying clauses.

Transition to arithmetic modulo 2.

P (a) ≡ 1 mod 2 a is not satisfying (no error possible)

P (a) ≡ 0 mod 2 not all clauses must be satisfied

Idea. We introduce randomization. We hide randomly a set of clauses.

Let Pi be the polynomial corresponding to the ith clause (after arithmetization).

m
ρ is a random vector ∈ {0, 1} (for m clauses). P (ρ) is the sum of all polynomials
Pi with ρi = 1 (terms of all visible clauses).
m
X
P (ρ) = ρ i · Pi
i=0

For satisfying assignments a, P (ρ) (a) = 0. probability(P (ρ) (a) ≡ 0 mod 2) =

probability(P (ρ) (a) ≡ 1 mod 2) = 12 .

We still have the problem that we depend on a and a requires n bits and is
therefore inappropriate for our purposes.

59
We need method to compute p(a) (=p(ρ) (a)) without knowledge of a.

V must be capable to derive from the available information a constant (here

3) number of proof positions and from the values of those proof positions the
decision about acceptance.

In general: Consider a polynomial Ψ = (x1 , x2 , . . . , xn ) of degree ≤ 3 over Z2

(p and p(ρ) mod 2 of this structure). Ψ consists (probably multiplication is
necessary) of the following terms:

xi ∈ {0, 1} constant term

1
xi i ∈ IΨ
2
xi · xj (i, j) ∈ IΨ
3
xi · xj · xk (i, j, k) ∈ IΨ

Define now linear functions La1 , La2 , La3 .

n
X
La1 : Zn2 → Z2 La1 (y1 , . . . , yn ) = ai · yi
i=1

n X
n
2 X
La2 : Zn2 → Z2 La2 (y11 , . . . , ynn ) = ai · aj · yij
i=1 j=1
n X
n X
n
3 X
La3 : Zn2 → Z2 La3 (y111 , . . . , ynnn ) = ai · aj · ak · yijk
i=1 j=1 k=1

We can store those function as function tables. For La1 length 2n , for La2 length
2 3 2 3
2n and for La3 length 2n . So the total length is 2n + 2n + 2n . γΨ 1 2
, γΨ and
3 1 2 3 2
γΨ are the characteristical vectors of IΨ , IΨ and IΨ (for example γΨ has a One
at position (i, j) ⇔ (i, j) ∈ IΨ 2
). Now we can state Ψ(a) = γΨ + La1 (γΨ 1
)+
a 2 a 3
L2 (γp si ) + L3 (γΨ ). Ψ(a) can be evaluated without access to a. V can evaluate
La1 (γΨ
1
), La2 (γΨ
2
) and La3 (γΨ
3
) (which is p(a) or pγ (a)) without using a.

We have a function table for La1 , La2 and La3 . We check: p(ρ) (a) for constant,
randomized ρ and accept only if all values are zero.

The number of required random bits is O(n3 ), because number of clauses in

non-trivial 3 SAT instance is
   
n n
≤ 2n + 4 +8 = O(n3 )
2 3

We need an approach for proofs of random kind. We can assume that proofs
have the “correct” length.

A proof verifier for the general case consists of the following components:

60
 • linearity testing
• robust function evaluator
• consistency test
• proof verifier for well-formed proofs

7.1 Linearity testing

Definition. f : Zm m
2 → Z2 is linear if f (x) + f (y) = f (x + y)∀x, y ∈ Z2 . f is
m
δ-close to a function g (f, g : Z2 → Z2 ) if (for a uniform distribution of x to
Zm2 ) we can state:
probabilityx (f (x ̸= g(x))) ≤ δ

Followingly we call a function almost-linear, if for a usefully selected δ (we will

discuss this later on) the function is δ-close to a linear function.

Idea of linearity tests. Select randomly and independent of each other x, y ∈

Zm
2 and name f non-linear if f (x) + f (y) ̸= f (x + y) (otherwise linear). In this
case, the linearity test fails.

Properties:

• If f is linear, dann f will pass the test.

1
• If f is not δ-close for any δ < 3 to a linear function, then f will pass the
test with probability 1 − 2δ .

To show the first property is immediate. For the second property we have to
show probability(f (x + y) ̸= f (x) + f (y)) ≤ 2δ (f is δ-close to linear function g).
This is a constructive proof. We define a linear function g, which is δ-close to f .
a ∈ Zm2 . We define g(a) as follows: Evaluate all function values f (a + b) − f (b)
with b ∈ Zm 2 (fyi, f (a + b) ∈ [0, 1] and f (b) ∈ Z2 ). We set g(a) = 0 if 0 occurs
more often (or equal) as result for f (a + b) − f (b) then 1. Otherwise g(a) = 1.

Remainingly we have to show that

• g is linear to u.
• g is δ-close to f .

Regarding point 1: Assume f and g are not δ-close to each other,

⇒ probabilityx (f (x) ̸= g(x)) > δ

Because of the construction g, we can state that

1
probabilityy (g(a) = f (a + y) − f (y)) ≥
2

61
Therefore
probabilityx,y (f (x + y) − f (y) ̸= f (x))
≥ probabilityx,y (f (x + y) − f (y) = g(x), g(x) ̸= f (x))
X
= probabilityy (f (a + y) − f (y) = g(a), g(a) ̸= f (a))
a∈Zm
2

For all a ∈ Zm
2 we can state:

f (a) = g(a) ∨ f (a) ̸= g(a)

In the first case, the probability (from above) is 0 and in the second case we can
omit the precondition f (a) ̸= g(a).

probabilityx,y (f (x + y) − f (y) ̸= f (x))

1 X
≥ · probabilityy (f (a + y) − f (y) = g(a))
2m
a∈Zm
2 ,f (a)̸=g(a)

1 X 1 δ
≥ · >
2m 2 2
a∈Zm
2 ,g(a)̸=f (a)

The last inequation follows, because from

probabilityx (f (x) ̸= g(x)) > δ

it follows that more than 2m · δ of all a ∈ Zm 2 it requires f (a) ̸= g(a). We get a

contradiction to probabilityx,y (f (x + y) ̸= f (x) + f (y)) ≤ 2δ .

For the second bullet point: To determine the linearity of g we consider

p(a) = probabilityx (g(a) = f (a + x) − f (x))

1
Obviously, p(a) ≥ 2 (choice of g). We want to show a stronger statement:
p(a) ≥ 1 − δ.

random x ∈ Zm m
2 → x + a ∈ Z2 is random
| {z } | {z }
uniformly distri. uniformly distri.

δ
probabilityx,y (f (x + a) + f (y) + f (x + a + y)) ≤
2
We call it event 1. This is a requirement in the inequation of bullet point 2.

Analogously, event 2,
δ
probabilityx,y (f (x) + f (y + a) ̸= f (x + a + y)) ≤
2

The probability for the union of both events is bounded by above δ and the
probability of the complement is bounded below by 1 − δ.

62
According to the DeMorgan rules, the intersection of f (x)+f (y+a) is f (x+a+y)
and f (x + a) + f (y) is f (x + a + y). This corresponds to the subset of the event
f (x + a) + f (y) = f (y + a) + f (x).

probabilityx,y (f (x + a) + f (y) = f (y + a) + f (x)) ≥ 1 − δ

⇔ f (x + a) − f (x) = f (y + a) − f (y)
⇒ P = probabilityx,y (f (x + a) − f (x) = f (y + a) − f (y)) ≥ 1 − δ
X
P = probabilityx,y (f (x + a) − f (x) = z, f (y + a) − f (y) = z)
z∈{0,1}

iid
X
= probabilityx (f (x + a) − f (x) = z). probabilityy (f (y + a) − f (y) = z)
z∈{0,1}
X 2
= [probabilityb (f (x + a) − f (x) = z)]
z∈{0,1}

For z = g(a) is probabilityx (f (x + a) − f (x) = z) = p(a). For z ̸= g(a) we get

probabilityx (f (x + a) − f (x) = z) = 1 − p(a).

Therefore it follows from the derivation so far:

1 − δ ≤ p(a)2 + (1 − p(a))2

(we know that p(a) ≥ 21 .) Thus

1
1 − p(a) ≤ ≤ p(a)
2
⇒ p(a)2 + (1 − p(a))2 ≤ p(a)2 + p(a)(1 − p(a))
p(a)2 + (1 − p(a))2 = p(a)
So we have proven p(a) ≥ 1 − δ. We will use this result now at three different
occations:

p(a) = probabilityx (g(a) = f (a + x) − f (x)) ≥ 1 − δ

p(b) = probabilityx (g(b) = f (b + a + x) − f (a + x)) ≥ 1 − δ
p(a + b) = probabilityx (g(a + x) = f (a + b + x) − f (x)) ≥ 1 − δ

The intersection of all these three events has probability ≥ 1 − 3δ. Add the first
two events and substract the third event:

probabilityx (g(a) + g(b) = g(a + b)) ≥ 1 − 3δ

1
Resulting from the requirement δ < 3 we get:

probability(g(a) + g(b) = g(a + b)) > 0

63
Because the inner condition is independent of x, we get

probabilityx (g(a) + g(b) = g(a + b)) = 1

and therefore g is linear

g(a + b) = g(a) + g(b)∀a, b

The linearity test is proven.

7.2 Robust function evaluator

1
Goal. For δ < 3 the following properties have to be satisfied:

• If f is linear, then function evaluator has to provide f (a)∀a ∈ Zm

2 .

• If f is δ-close to linear function g, then function evaluator (randomized

algorithm) has to provide value g(a) with a error probability bounded by
2δ.

We take the following approach to determine f (a): Select x ∈ Zm 2 randomly

and evaluate f (x + a) − f (x). It remains to show the two cases we just stated
in the bullet points.

The first case is trivial, because f is linear: f (x + a) = f (x) + f (a)∀a.

For the second case f and g are δ-close:

probabilityx (f (x) = g(x)) ≥ 1 − δ

probabilityx (f (x + a) = g(x + a)) ≥ 1 − δ

The probability that both events will occur is ≥ 1 − 2δ. From those two events
it follows
f (x + a) − f (x) = g(x + a) − g(x)
| {z }
=g(a)

Because g is linear: f (x+a)−f (x) = g(a). Function evaluator has been proven.

7.3 Consistency test

We have 3 function stabilizors f1 , f2 and f3 (in our proof this is La1 , La2 , La3 ).
f1 , f2 and f3 are each linear or δ-close for a linear function (otherwise they will
fail the linearity test).
1
For the consistency test we assume δ < 24 .

64
 1. If function stabilizors of f1 , f2 and f3 are linear functions of type La1 , La2
n
and La3 (for any a ∈ {0, 1} ) the consistency test has to succeed.
n
2. If there is no a ∈ {0, 1} such that the functions (represented by the
function tables of f1 , f2 , f3 ) are δ-close to La1 , La2 and La3 , the consistency
test has to succeed with error probability bounded by a constant c.

Construction. We choose randomly and independent x, x′ , x′′ ∈ Zm

2 and y ∈
2
Zn2 .

Define x = x′ by
(x = x′ )ij = xi · x′j → dim n2
x′′ ◦ y by (x′′ ◦ y)ijk = x′′ijk

Use function evaluator to get estimator for

b ⇒ f1 (x) b′ ⇒ f1 (x′ ) b′′ ⇒ f1 (x′′ ) c ⇒ f2 (x ◦ x′ )

c′ ⇒ f2 (y) d ⇒ f3 (x′′ ◦ y)

The consistency test will succeed if bb′ = c and b′′ c′ = d.

Remark. The linear functions La1 , La2 and La3 will succeed the test of course.
For them the function evaluation value is error free and we can state:

n
!  n

n
n X
X X X
La1 (x) · La2 (x′ ) = ai xi · aj x′j  = ai aj vi xj
i=1 j=1 i=1 j=1

Analogously
La1 (x′′ ) · La2 (y) = La3 (x′′ ◦ y)

1 1
For our two desired properties for the consistency test: Because δ < 24 , 2δ < 12 .
Because we have 6 calls of the function evaluator and each of them has error
1
probability < 2δ < 12 , the total error probability for function evaluations is 12 .

Consider now the case that all function evaluations are passed successfully. A
linear δ-close function f1 has coefficient ai . A δ-close function f2 has coefficient
bij (Matrix B = (bij )).

Define a matrix A = (aij ) with aij = ai · aj .

Remark. Function evaluations is correct, but function stabilizor is inconsistent.

We now consider the case bb′ ̸= c. Then follows A ̸= B. Consider x and x′ as
column vectors, then the consistency test compares xt Ax′ and xt Bx′ . This test
is based on the fact that A ̸= B and random x and x′ with probability ≥ 41 the
values are difference and therefore inconsistency is shown.

If A and B differe in the j-th columns, the probability that xt A and xt B differ
at the j-th position is 21 .

65
Combining all these thoughts, we get a PCP(n3 , 1) verifier for 3 SAT.

Remark. PCP(poly(n), 1) = NEXP = c≥1 PCP(nc , 1).

S

In the approximation chapter we will have a fresh look at the PCP theorem from
a different perspective (reformulation with focus on non-approximative results).

8 Approximation from complexity theory per-

spective

Motivation. Many optimization problems are difficult to solve exactly.

Similar Question. Can we efficiently compute approximative solutions for

such problems?

Basic question. How do we measure the quality of an approximation? Most

basically this is the relative error.

Instance 1 Opt. value 17 Approx. solution: 27

Instance 2 Opt. value 1700 Approx. solution: 1710

Table 2: Example for quality of an approximation. Instance 2 has the smaller

relative error.

8.1 A notion of optimization

In the following we will discuss optimization problems. x is the input. S(x)

denotes the set of valid solutions for the instance with input x.

S(x) ̸= 0 Not optimizable in the other case

Denote the target function value of the solution with s ∈ S(x). Vopt (x) denotes
the optimal target function value for the input x. Furthermore we assume
V (x, s) > 0∀x, s (not required, but makes quality function less complex).

∀x, s ∈ S(x) the encoding length of s and v(x, s) is polynomial in |x| (Back-
ground: our approximation algorithm shall have polynomial runtime).

8.2 A definition of approximation quality

Definition. Approximation quality (here: one of multiple variations)

v(x, s)
r(x, s) = with s ∈ S(x) for min opt problems
vopt (x)

66
 vopt (x)
r(x, s) = with s ∈ S(x) for max opt problems
v(x, s)

Remark.

• This definition is possible due to v(x, s) > 0 (no absolute value or division
required).
• We always have r(x, s) ≥ 1. The closer r(x, s) is to 1, the better is s.
• r(x, s) ≤ c with c ≥ 1 is “c-Approximation”.

Algorithmical view. Consider optimization problem Q. x is the input of Q

and A ist an algorithm for Q: A computes valid solutions SA (x) ∈ S(x). The
approximation quality of A for the input x is given by r̂A (x) := r(x, sA (x)).

Worst case consideration. For the algorithm A rA (n) is called maximum

approximation quality of A.

rA (n) = max {rˆA (x) ||x| ≤ n}

Naming. If rA (n) ≤ 1 + ε is given for some ε ≥ 0 and all n ∈ N, A computes

an ε-optimal solution.

If rA (n) ≤ c for c ≥ 1 and all n ∈ N, A computes a c-approximation (approxi-

mates Q to factor c).

Remark. Also the quality of a maximum optimization problem is defined in a

way such that the quality is ≤ 1 (inverse of our definition).

8.3 Bin-Pack with an approximation algorithm

Given. n numbers ai ∈ [0, 1] (item size). Bins have capacity 1.

Find. Pack items in minimum number of bins.

Algorithm 4 Best-First Decreasing algorithm for Bin-Pack (BFD)

1: Sort elements in descending order.
2: Pick largest number and place it in fitting bin such that the smallest amount
of capacity is left.

For the BFD algorithm the following statements are known:

11
v(x, sBFD (x)) ≤ · vopt (x) + 4
9
11 4
rBFD (x) ≤ +
9 vopt (x)

67
Without loss of generality, we assume that vopt (x) ≥ 2 because vopt (x) = 1
instances are trivially recognizable. Then
29
rBFD (x) ≤
9

29 11
For large values of vopt (x) 9 is far away from 9 and serious quality.

8.4 A notion of asymptotic approximation quality

We therefore introduce the asymptotic approximation quality

∞
rA = inf {b|∀ε > 0∃v(ε) > 0∀xvopt ≥ v(ε), rA (x) ≤ b + ε}

∞ 11 y
For example: rBFD = 9 . When n goes to infinity, vopt (x) = ∞ and vopt (x) to
0.

We will almost only consider classical quality. But there are some problems (bin
packing is one of them) for which asymptotical quality is more powerful.

8.5 Complexity classes APX and APX*

Let r(n) : N 7→ [0, ∞] with r(n + 1) ≥ r(n)∀n.

The complexity class APX(r(n)) contains all optimization problems for which
an algorithm A with maximum approximation quality rA (n) ≤ r(n) can be
found with polynomial runtime.
[
APX := APX(c)
c>1,c const

Intuitively, “in constant quality approximable in polynomial time”.

\
APX∗ := APX(c)
c>1,c const

Problem for every c > 1 with quality c is approximable. Algorithm can depend
on c.

Definition (PTAS). polynomial time approximation scheme.

A PTAS for an optimization problem Q is an algorithm (a class of algorithms)

with inputs of structure (x, ε) (x is input for Q, ε > 0, ε ∈ Q) which computes a
solution for Q with quality (1 + ε) in polynomial time in |x| (“ε optimization”).

A PTAs as complexity class: contains all optimization problems for which there
exists a PTAs.

68
Definition (FPTAS). fully polynomial time approximation scheme (runtime is
polynomial in |x| and in 1ε ).
1
For PTAS runtime O(n ε ) with n = O(|x|) is allowed. For FPTAS it is not.

Obviously P ⊆ FPTAS ⊆ PTAS ⊆ APX.

8.6 Approximation of Max-Clique

Given. An undirected graph G = (V, E) with |V | = n

Find. A complete subgraph of G with maximum vertex number

Trivial algorithm 1. Return random vertices. Quality ≤ n.

Trivial algorithm 2. Fixate k ∈ N, k < n. Consider all subsets of V with size

k and return the greatest complete subgraph ⇒ quality nk .

Runtime is fixed for k.

Later on we will see: Max-Clique ∈

/ AFX.

8.7 Approximation of Vertex-Cover

Given. An undirected graph G = (V, E) with |V | = n

Find. V ′ ⊆ V such that every edge e in E of V ′ is covered (each edge in

E has at least one end vertex in V ′ and V ′ has minimal cardinality).

For general graph this problem is NP-hard. The approximation algorithm is

interesting.

Algorithm 5 Approximation algorithm for Vertex-Cover

1: E ′ = {}.
2: while ∃ an edge {u, v} such that no u and v is covered by an edge in E ′ do
3: select this edge and add it to E ′
4: end while
5: V ′ is set of all end vertices of edges in E ′ .

Claim. The algorithm above returns vertex cover V ′ and has approximation
quality 2.

Central point is the property of E ′ on termination.

• E ′ contains all edges which that the degree of each vertex is ≤ 1. Therefore
E ′ is maximal matching. Therefore V ′ is vertex cover.

69
 • Furthermore |E ′ | = k ⇒ |V ′ | ≤ 2k. Quality 2 follows, because for a cover
of E ′ at least k vertices are necessary.

Example 3. (Max-3-SAT) Given a 3 SAT equation l with variables x1 , . . . , xm

and clauses c1 , . . . , cn . Find an assignment such that the maximum number of
clauses is satisfied.

Consider the following randomized algorithm: Assign x1 , . . . , xn independently

uniformly distributed with 0, 1. Random variable:
(
1 clause cj is satisfied
xj =
0 else

Denote
7
E(xj ) = 7 of 8 clauses satisfied
8

Random variable (we want X at maximum)

m
X
X := xj
j=1

m
X 7
E(X) = E(Xj ) = m
j=1
8

Randomized approximation algorithm with quality 78 . We desire a deterministic

algorithm.

Idea. Fixate the values xi one after another. We have to ensure that for random
selection of an assignment for the remaining variables the number of satisfied
clauses has to be keep ≥ 78 m.

Situation during the running algorithm: 0–3 literals per clause are fixed.

Consider xn = b ∈ {0, 1}.

aj = E(xj |xn = b) j = 1, . . . , m

If with assignment xn = b the clause j is satisfied, aj = 1. Otherwise there are

k = 1, 2, 3 not-fixed literals in cj .
2k −1 1
Then this implies: aj = 2k
=1− 2k
. If the literal is fixed and false, aj = 0.

m
X
E(X|xn = b) = E(Xj |xn = b) = a1 + . . . + am
j=1

Decidable in O(m)
1 1
E(X) = · E(X|Xn = 1) + · E(X|xn = 0)
2 2

70
 7
We know: E(X) ≥ m
8
7 7
⇒ E(X|xn = 1) ≥ m E(X|xn = 0) ≥ m
8 8

We are allowed to fixate xn = bn such that E(x|xn = b) ≥ 78 m. So xn is fixed.

Now iterate. We gain assignment b1 , . . . , bn .
7
E(x|x1 , . . . , xn = b) ≥ m
8
Deterministic approximation algorithm with runtime O(n · m) with quality 87 .

8.8 Example 4: Scheduling

A simple 2-machine scheduling problem as example for a problem with a PTAS.

Given. A set J = {1, . . . , n} of tasks and we have to identical machines.

A task j takes Pj time.

Find. Find assignment of tasks such total runtime is minimal. A task

cannot be split.

Problem is NP-hard (proof via partition problem).

Pn
Set T = j=1 pj . Obviously vopt (x) ≥ T2 .
 
X X 
min max pj , pj
 
j∈M⊮ j∈{1,...,n}\M1

Idea. Split tasks in 2 sets. We denote task j to be “large”, if pj ≥ ε · T (goal:

quality 1 + ε). Other jobs are called “small”.

Observation. There are ≤ ⌊ 1ε ⌋ large tasks.

1
For each of the maximal 2⌊ ε ⌋ possible assignments of the large tasks to the two
machines, apply the following algorithm:

• Extend the currently existing machine assignment with assignment of

small tasks with “least loaded” heuristics (traverse from the smallest to
the greatest task and assign it to the machine with the smaller total time).
• In the end select the best solution.

1
Runtime is O(n2 ε ) polynomial in n. So it satisfies the condition to be in PTAS.

71
We have to show that this is a (1 + ε) approximation. Show that:

V (x, s
|{z} ) ≤ (1 + ε)vopt (x)
schedule, solution

Proof of quality. Consider optimal solution:

• Corresponding assignment of large tasks to M1 and M2 .

• Also this partition has been considered in the approximation algorithm
(because all of them have been considered)

Case distinction:

1. All small tasks are assigned to one machine.

2. Not all small tasks are assigned to one machine.
• The maximum difference of both assignment total time of the ma-
chines ≤ εT .
T
• The maximum difference of completeness time of 2 is limited by ε T2 .
T ε
v(x, s) ≤ + T
2 2
T
(1 + ε) ≤ (1 + ε) · vopt (x)
2

1
Disadvantage: Runtime (PTAS which is no FPTAS, because of 2 ε ).

8.9 Example 5: Knapsack problem

The knapsack problem is an example for a FPTAS. NP-complete problem.

Given are n items with weights. g1 , . . . , gn ∈ N and values v1 ,P . . . , vn ∈ N. Also

given
P is a capacity b ∈ N. Find I ⊆ {1, . . . , n} such that i∈I gi ≤ b and
i∈I v i is maximal.

Classical dynamic programming algorithm:

 
Xk k
X 
F (k, y) := max vj xj subject to gi xi ≤ y, xi ∈ {0, 1} with i = 1, . . . , k
 
j=1 i=1

Maximum value when considering the first k items. Knapsack capacity

(
F (k, y − gk+1 ) + vk + 1
F (k + 1, y) = max
F (k, y)

72
F (n, b) is decidable in O(nb) (pseudopolynomial) time.

Dual point of view. (to two variables k and v) M [k, v] is minimal knapsack
weight to reach total value.

 
Xk k
X 
M [k, v] = min gj xj | vj x j = v
 
j=1 j=1

Initialization:
M [0, v] = ∞ ∀v > 0
M [i, v] = ∞ ∀v < 0, iin {1, . . . , n}

Recursion:

M [k + 1, v] = min {M [k, v], M [K, v − vk+1 ] + gk+1 }

Runtime:
O(n · Vmax ) = O(n2 · vmax )
| {z }
O(n · Vmax )
| {z }
max vj with j∈{1,...,n}

Runtime is practical iff vmax is not too large!

Idea. Scaling of values using t as factor for values.

t will be selected as
ε · maxi=1,...,n Vi
t=
(1 + ε)n

New resulting knapsack instance: Values ṽ1 , ṽ2 , . . . , ṽn , weights g1 , . . . , gn and
capacity b.

Observation. I ⊆ {1, . . . , n} is valid solution for original instance. I is the set

of selected items ⇔ I is a valid solution for the new problem instance (gi , b did
not get modified)
n o
Runtime. O(n2 ·maxi=1,...,n ṽi ) = O(n2 ·max ⌊ (1+ε)·n·v
ε·max vi
i
⌋ ) = O(n2 ·⌊ (1+ε)n
ε ⌋) =
3
O( nε + n3 ) so polynomial in n and 1
ε !. This is fine for FPTAS.

Still to show: (1 + ε) approximation.

Regarding quality. Is I ∗ ⊆ {1, . . . , n} and optimal solution for the original

instance. Is It the solution determined by the approximative algorithm. I ∗ and
It are both valid solution for original and new instance. We can state that
X X
v˜i ≥ v˜i because It is optimal solution for new instance
i∈It i∈I ∗

73
Because ṽi = ⌊ vti ⌋, we conclude vi vi P
t − 1 < ṽi ≤ t . Is OPT := i∈I ∗ vi value of
an optimal solution.

X X vi X
ṽi ≥ t · ( − 1) = vi − t · |I ∗ | = OPT − t · |I ∗ |
∗
t ∗
|{z}
i∈I i∈I
≤n

ε 1
≥ OPT − · OPT = OPT ·
1+ε 1+ε

So we get
OPT
P ≤1+ε
i∈It vi
X OPT
vi ≥
1+ε
i∈It

9 Positive and negative approximation results

This lecture took place on After christmas 2013..

In the field of approximation algorithms we distinguish “positive results” (by

providing an appropriate approximation algorithm) and “negative results” (re-
sults of the structure “Problem Q cannot be approximated with quality guar-
antee < c if . . . (eg. P ̸= NP)”).

Several technique to derive negative results (non-approximations) do exist:

9.1 Gap technique

This technique is derived from the NP-hardness results. We use a gap-problem

as basis.
More precisely: Given an optimization problem Q where all inputs x satisfy
vopt (x) ≤ a or vopt (x) ≥ b > a and which is NP-complete. The problem of
determining which case (a or b) is given, is called (a, b)-gap-problem.

Assuming there exists an approximation algorithm A (therefore A has poly-

nomial runtime) with performance guarantee < ab for the (a, b)-gap-problem,
P = NP would follow. Such an approximation algorithm allows us to distin-
guish between both cases.

Theorem. Is A an (a, b)-gap-problem. There is no approximation algorithm

for Q with performance guarantee ab if P ̸= NP.

74
9.1.1 Example: TSP

The Hamiltonian cycle problem is NP-complete. Given an undirected graph

G = (V, E) and |V | = n. Construct a distance matrix n × n for TSP D = (dij ).
(
1 {i, j} ∈ E
dij = n
n·2 {i, j} ∈
/E

Observation. If G is hamiltonian, the shortest tour in regards of D has length

n = 1+1+. . .+1. If G is not hamiltonian, every tour has length ≥ (n−1)·1+n·2n .

This gives us a (n, (n − 1) + n · 2n )-gap-problem. If P ̸= NP there does not exist

an approximation algorithm for TSP with performance guarantee 2n .

Thus if P ̸= NP, T SP ∈
/ AP X.

Remark. For particular special cases of TSP better approximation results are
possible. If D is symmetrical and triangle inequation is satisfied (metrical case),
a 32 -approximation algorithm can be provided (Christofides heuristic).

Theorem. (by Arora) If dij are euclidean distances of n points, a PTAS can
be provided.

9.1.2 Special case b = a + 1: (k, k + 1)-gap-problem

b = a + 1, a = k, k ∈ N

Theorem. If

• P ̸= NP and
• for a minimization problem only values with integers as target function
values are possible and
• decision whether vopt (x) ≤ k is NP-complete

then there is no approxmation algorithm with performance guarantee < 1 + k1 .

This is a result for (k, k + 1)-gap-problems.

We provide 2 examples for this theorem.

9.1.3 Example: BIN-Packing

Given. a1 , . . . , an ∈ (0, 1].

Find. Minimum number of bins, each bin has capacity 1.

75
No approximation algorithm with performance guarantee < 32 = 1.5 for Bin-
packing exists if P ̸= NP. But there exists an asymptotical PTAS for BIN-
Packing.

9.1.4 Example: Vertex coloring

Given. Undirected graph G = (V, E).

Find. Coloring of V with minimum number of colors (= χ(G)). Decide

whether χ(G) ≤ k.

We can decide this in polynomial time for k = 1 and k = 2. For k ≥ 3 this is

an NP-complete problem ((3, 4)-gap-problem). Thus there is no approximation
algorithm with performance guarantee < 43 if P ̸= NP.

In comparison to BIN-Packing we cannot provide any better asymptotical re-

sults.

Theorem. If P ̸= NP there is no approximation algorithm for vertex coloring

with asymptotical performance guarantee c < 34 .

Proof of the theorem. (expansion argument) We construct a graph GK =

(VK , EK ) and χ(GK ) = K · χ(G) with K ∈ K.

Idea. (3, 4)-gap-problem for G becomes a (3K, 4K)-gap.

Consider K disjoint copies of G. We connect every vertex with all vertices in

different copies. We have to show that χ(GK ) = K · χ(G).

• χ(GK ) ≤ K · χ(G) can be ensured: Use in every copy a different set of

colors.
• χ(GK ) ≥ K · χ(G): We use χ(G) colors in G and because of the construc-
tion of GK no color can occur in more than 1 copy.

Theorem. If P ̸= NP, NP-hard optimization problems with “small solutions”

(ie. ∃ polynomial p: ∀x ∈ S(x) : v(x, s) ∈ Zt0 and v(x, s) ≤ p(|x|) with S(x) as
the set of valid solutions and v(x, s) as target function value) have no FPTAS.

Corollary. If P ̸= NP there does not exist a FPTAS for strong NP-hard

problems.

Typical problems with “small solution values”:

• Clique
• MaxSAT
• Vertex-Cover

76
And unlike Knapsack, Partition.

9.2 Construction at basis of PCP results

PCP theorem in our previous definition: NP = PCP(log n, 1). From this point
of view we cannot find any relation to approximation algorithms.

Now consider Max-3-SAT.

Given. 3SAT equation ϕ

Find. Maximum number of satisfiable clauses.

Until 1992 it was an open question whether an ρ approximation algorithm exists

for MAX-3-SAT for arbitrary ρ > 1. With the PCP results we gain: No, if
P ̸= NP.

The relation is that the PCP theorem has the following equivalent definition
(PCP theorem in regards of hardness of approximation):

There exists a constant ρ̂ > 1 such that all languages L ∈ NP there

exists a polynomial time computable function f which maps strings
to 3SAT equations such that

x ∈ L ⇒ val(f (x)) = 1

x∈
/ L ⇒ val(f (x)) < ρ̂
with val(x) as the quotient of maximum number of satisfied clauses
and number of clauses. Distinguishing those cases is a gap-problem.

It immediately follows the following corollary.

Corollary. There exists a constant ρ > 1 such that if an ρ approxima-

tion algorithm for Max-3-SAT exists, P = NP. Therefore approximation for
Max-3-SAT is not arbitrary. It can be close to 1.

With an ρ approximation algorithm A for Max-3-SAT we get a polynomial

algorithm for deciding whether x ∈ L.

Remark. The classical Cook-Levin idea is not applicable close to 1. We need

the gap!

Now we want to discuss why both definitions of the PCP theorem are equivalent.
Consider this generalization of 3SAT (CSP, constraint satisfaction problem):

Is q ∈ N. An instance of q-CSP is a collection of functions ϕ1 , . . . , ϕm (so-called

n
“constraints”) with ϕi : {0, 1} → {0, 1} such that every ϕi depends on at most
q input values.

77
Therefore for all i ∈ {1, . . . , m} there exists j1 , . . . , jq ∈ {1, . . . , n} such that
n
ϕi (u) = f (uj1 , . . . , ujq )∀u ∈ {0, 1} .
n
If ϕi (u) = 1 we state that u ∈ {0, 1} satisfies the constraint ϕi (otherwise
unsatisfiable).

TheP quotient of the number of satisfied constraints and all constraints is given
m
ϕi (u)
by i=1
m . Denote val(ϕ) as the maximum of all quotients. ϕ is called
satisfiable if val(ϕ) = 1.

Remark. 3SAT is given with q = 3 and all constraints are disjunctions of

literals.

Definition. (GAP-CSP) Is q ∈ N and is ρ < 1. Then we define ρ-GAP q-CSP

as the problem to determine whether a given instance ϕ of q-CSP

• val(ϕ) = 1 (denote ϕ as Yes instance of ρ-GAP q-CSP)

• val(ϕ) < ρ (denote ϕ as No instance of ρ-GAP q-CSP)

We state that ρ-GAP q-CSP is NP-hard for every language L ∈ NP, if there is a
polynomial time computable function f which maps strings to q-CSP instances
such that
x ∈ L ⇒ val(f (x)) = 1
x∈
/ L ⇒ val(f (x)) < ρ

9.3 Three definitions of the PCP theorem

Now we have 3 equivalent definitions of the PCP theorem:

1. NP = PCP(log n, 1)
2. The previous theorem in the statement above. A constant g < I such
that ∀L ∈ NP there exists a polynomial computable function f : Strings
→ (Representation) ∨ 3SAT equations with

x ∈ L ⇒ val(f (x)) = 1

x∈
/ L ⇒ val(f (x)) < g

3. There exists a q ∈ N and ρ ∈ (0, 1) such that ρ-GAP q-CSP is NP-hard.

9.3.1 The first definition implies the third

Is NP ⊆ PCP(log n, 1). We show that 21 -GAP q-CSP is NP-hard for appropriate

constant.

78
It’s enough that show only a single NP-complete language L (eg. reduce 3SAT
to 12 -GAP q-CSP for appropriate constant q). From NP ⊆ PCP(log n, 1) it
follows that there is a PCP verifier such that the verifier V has a constant
number of proof bits q requested/verified and used c · log n random bits (with c
c·log n
as constant). For given input x and random vector r ∈ {0, 1} define Vx,r
as the function returning 1 as output iff the proof B gets accepted by V .

Because q proof bits are relevant, Vx,r only depends on q positions. Therefore
n
∀x ∈ {0, 1} returns the collection

ϕ = {Vx,r }r∈{0,1}c·log n

a q-CSP instance of polynomial size.

Because V has polynomial runtime the transformation of x to ϕ has also poly-

nomial runtime. Because V is our PCP verifier, for x ∈ 3SAT (x must be
/ 3SAT, val(ϕ) ≤ 12 . Therefore
satisfiable) its given that ϕ = 1 and for x ∈
1
2 -GAP q-CSP NP-hard.

9.4 The third definition implies the first

Assume ρ-GAP q-CSP is NP-complete for constant q ∈ N, ρ ∈ (0, 1). This can
be translated to PCP-system with q proof bits, logarithmical number of random
bits and error probability ≤ ρ for every language L:

Given input x. The verifier calls the reduction f (x) to gain the q-CSP instance
ϕ = {ϕi }i=1,...,m .

It is expected that the proof B is an assignment of values to the variables

in ϕ. Verification: Select randomly i ∈ {1, . . . , m} and check whether ϕi is
satisfied (this involves q position requests). Obviously the verifier accepts with
probability 1 if x ∈ L. If x ∈
/ L, acceptance with probability ≤ ρ.

This error probability can be improved to 12 with costs of a constant factor in

regards to the number of random bits and the number of proof positions.

9.5 The second definition equates the third definition

Because 3SAT is special case of ρ-CSP, it immediately follows that the second
definition implies the third. We have to show that the third definition implies
the second.

Is ε > 0 and q ∈ N such that (1 − ε)-GAP q-CSP is NP-hard (q, ε∃ because of

the third definition).

Is {ϕi } a q-CSP instance with n variables and m constraints.

Each constaint ϕi can be represented as conjunction of ≤ 2q SAT clauses (thus

each clause is an disjunction of ≤ q literals).

79
Is ϕ′ the collection of ≤ m · 2ϕ clauses resulting from all ϕi , i ∈ {1, . . . , m}. If ϕ
is a Yes instance of (1 − ε)-GAP q-CSP (ie. ϕ is satisfiable), then there exists a
truth assignment such that all clauses of ϕ′ are satisfied. If ϕ is a No instance
of (1 − ε)-GAP q-CSP then at least the portion ε of constraints of ϕ is violated
in each truth assignment.

At least a portion of 2εq of constraints of ϕ′ is violated. Use proof technique

for theorem of Cook to transform each clause C to q variables u1 , . . . , uq in a
set of clauses C1 , . . . , Cq to the variables u1 , . . . , uq and additional temporary
variables y1 , . . . , yq such that

• every clause Cj is a 3SAT clause

• if u1 , . . . , uq is selected such that C is satisfied, then there exists an as-
signment of y1 , . . . , yq such that (u1 , . . . , uq , y1 , . . . , yq ) satisfies the clauses
c1 , . . . , cq .
• if u1 , . . . , uq such that C is not satisfied, then for all assignments of
y1 , . . . , yq there is one clause Ci such that Ci is not satisfied by u1 , . . . , uq , yi , . . . , yq .

From ϕ′ generate ϕ′′ as follows: Apply the technique from above to all clauses
of ϕ′ ; we get a collection of ≤ q · m · 2q clauses.

ϕ′′ is a 3SAT instance. In total ϕ → ϕ′′ .

• If ϕ is satisfiable, then ϕ and ϕ′′ are satisfiable.

• If ϕ′ is unsatisfiable, then every assignment of ϕ violates at least a portion
ε of constraints. This corresponds to a 2εq portion of ϕ′ and q·2 ε
q portion

of constraints of ϕ′′ .

Let A be the version showing PCP theorem’s hardness from approximation’s

point of view (definition 1). Let B be the second definition and C the third.

PCP verifier For A, it is PCP-verifier V. For B it is an CSP instance ϕ.

PCP proof For A, it is proof B. For B is it assignment of variables in ϕ.

Length of proof Not given for A. For B it is the number of variables.

Number of position lookups (q) For B and C, it is the maximum number
of variables per constraint.
Number of random bits (r) For B and C, it is the logarithm of the con-
straint number m.
1
Error probability (typically 2) For B and C, it is the maximum for val(ϕ)
for a No instance.

80
We now know that (if P ̸= NP) Max-3-SAT cannot be approximated as close
to q as we wish (NP-hard problem).
∼ ∼
But no particular value g follows from the ideas above such that q -approximation
of Max-3-SAT is NP-hard.

Hastad was able (with an improved version of the PCP theorem) to show for
3SAT that for all ε > 0 if there exists a ( 78 − ε)-approximation of Max-3-SAT,
P ̸= NP follows.

9.6 Further negative approximation results

Other approximation results that follow from the second PCP-theorem defini-
tion. So far we have considered Max-3-SAT and CSP.

Consider Vertex-Cover (minimum cardinality) and Independent-Set (max-

imum cardinality, corresponds to Stable-Set).

Theorem. There exists a constant γ > 1 such that γ-approximation for

Vertex-Cover is NP-hard. For all β > 1, a β-approximation of Independent-Set
is NP-hard.

The second theorem is stronger than the first theorem (“for all”). Vertex-Cover ∈
APX (compare with 2-approximation algorithm from start of this chapter).

9.6.1 Independent-Set

Clique ∈
/ APX if P ̸= NP

Independent-Set and Vertex-Cover are equivalent in regards of the corre-

sponding optimization problem, but no statement (or equivalence) for approxi-
mation follows from this.

In a Vertex-Cover all vertices are covered (complement of Vertex-Cover

is an Independent-Set).

Is NV C the minimal cardinality of a Vertex-Cover and is NIS the maximal

cardinality of an Independent-Set. Is n the number of vertices in G. Then:

NV C = n − NIS

A β-approximation for Independent-Set returns Independent-Set of size

1
β NIS . If you want to use this for Vertex-Cover, we get Vertex-Cover
with cardinality n − β1 NIS . This provide us approximation guarantee:

n − β1 NIS
n − NIS

81
This can be arbitrarily bad, especially if IS is close to n. This is no surprise if
we consider the last theorem. We want to provide a proof for this theorem.

9.6.2 Proof of the last theorem

First we show (using the PCP theorem) for Independent-Set and Vertex-Cover
that there is some constant ρ > 1 such that ρ-approximation is NP-hard.

Lemma. There exists a polynomially computable transformation f of 3SAT

equations to graphs such that for every 3SAT equation ϕ, f (ϕ) is an undirected
graph with n vertices which largest independent set has cardinality val(ϕ) · n7 .

Proof. The proof follows from the classical NP-completeness reduction of

Independent-Set.

Corollary. If P ̸= NP there exists constants ρ, ρ′ > 1 such Independent-Set

is not ρ-approximable and Vertex-Cover is not ρ′ -approximable.

Proof. Is L a language in NP. From the second PCP-theorem definition we

know that decision version of L is reducible to approximation of Max-3-SAT
Reduction provides a 3SAT equation ϕ which is either satisfiable val(ϕ) = 1 or
val(ϕ) < ρ with ρ < 1. Now we use the reduction from the Lemma:
1 1
approximation of Independent-Set → approximation of Max-3-SAT
ρ ρ

This is a statement thatn 1

ρ -approximation (ρ′ > 1) for Independent-Set is
NP-hard.

Regarding Vertex-Cover: Minimal vertex cover which can be derived from

graph with the reduction has cardinality n − val(ϕ) n7 . So if Vertex-Cover
7− 1
has a ρ′′ approximation with ρ′′ = 6 ρ then we could find a vertex cover with
size ρ′ · (n − n7 ) in case val(ϕ) = 1 and this is ≤ n − ρ · h7 . We could distinguish
between val(ϕ) = 1 and val(ϕ) < ρ which is NP-hard.

proof the theorem. Given is a graph G with n vertices. Consider graph

Now we
Gk nk vertices correspond to k-sized subsets of V . Two subsets S1 and S2 are
connected by an edge if S1 ∪ S2 is an independent set in G.

The largest independent set in Gk corresponds to all k-sized
subsets of the

largest independent set in G and has therefore cardinality NkIS . We now take
the graph from the reduction in the corollary above and construct the product
with k. The quotient from the size of the largest independent set in the two
 k
(NIS )
cases is ρ·NkIS . This is an approximation of about ρ1 . If k is large enough,
( k )
then the approximation guarantee is arbitrarily bad. The runtime is therefore
nk ; polynomial.

In the following lectures we want to develop a reduction term for approximation.

82
9.7 Reduction of approximation problems

We need an appropriate reduction term for approximation.

There are a number of terms available. We discuss PTAS-reductions.

approximation_reduction.pdf

We need an appropriate “translation” of approximation guarantee s′ in regards

of B to s in regards of A.

Especially for PTAS reduction we want to insert a PTAS for B and we require

rB (x′ , s′ ) ≤ 1 + α(ε) ⇒ rA (x, s) ≤ 1 + ε

Definition. A PTAS reduction of problem A to problem B (A ≤PTAS B) is

denominated by a triple (f, g, α) with

• For input x of A, input f (x) for B is computable in polynomial time.

• For input x of A and solution s′ ∈ SB (f (x)) and ε ⊆ Qt , g(x, s′ , e) can be
computed in polynomial time.
• α : Q+ → Q+ is surjective, polynomially computable and rB (f (x), s′ ) ≤
1 + α(ε) ⇒ rA (x, g(x, s′ , ε)) ≤ 1 + ε.

Lemma. A ≤PTAS B, B ∈ PTAS ⇒ A ∈ PTAS.

Proof. We need a PTAS for A (input x, ε). Apply PTAS to B with input f (x)
and guarantee 1 + α(ε). We get solution s′ for B with rB (f (x), s′ ) ≤ 1 + α(ε).
Return s = g(x, s′ , ε). From the third property of the definition we know that

rA (x, s) ≤ 1 + ε

Further properties of ≤PTAS :

• ≤PTAS is reflective.
(f, g, α = id).
• ≤PTAS is transitive.

≤PTAS is of partial order. We can define =PTAS by A =PTAS B ⇔ A ≤PTAS B

and B ≤PTAS A.

83
Example. P1 : Determine clique with maximum number of vertices. P2 : De-
termine an independent set of maximum number of vertices. P1 =PTAS P2 .

Theorem. MAX-3-SAT ≤PTAS Clique.

Proof. Consider the classical reduction. 3SAT equation ϕ with clauses C1 , . . . , Cm .

3SAT ≤p Clique.

f (ϕ) = G with G = (V, E). In the classical reduction we use, that ϕ is satisfiable
⇔ f (ϕ) has clique of cardinality m.

We now use an observation: At least k clauses are satisfiable ⇔ ∃ clique with

cardinality ≥ k.

literal_construction.pdf

Edge are introduced between vertices in different rows, which do not correspond
to contradictory clauses → Clique with k vertices.

Let clauses Ci1 , . . . , Cik be (simultaneously) satisfiable.

For rows i1 , . . . , ik there exists a satisfiable literal. The corresponding vertices

are pairwise connected by edges (in different rows and are not contradictory) →
clique with k vertices.

Assume Vi1,j1 , . . . , Vik,jk create a clique with k vertices. Literals at position

(i1 , j1 ) (j1 -th literal in clause Ci1 ) . . . (ik , jk ) is satisfiable. Choose a truth as-
signment which is arbitrarily consistent with the literal assignment → k clauses
are satisfied.

From input ϕ and clique V ′ with |V ′ | = k in G = f (ϕ) the variable assignment

g(ϕ, V ′ )

Remark. Not every classical reduction is appropriate for approximation reduc-

tion.

Example. MAX-4-SAT → MAX-3-SAT.

4-SAT equation ϕ with m clauses. Clause Ci = (li1 ∨ li2 ∨ li3 ∨ li4 ). This is
represented as 2 3SAT clauses

(li1 ∨ li2 ∨ zi ) ∧ (li3 ∨ li4 ∨ ¬zi )

zi is a new variable. We get a 3SAT equation ϕ′ with 2m clauses.

Problem. Here it is trivialy ≥ half of clauses in ϕ′ is satisfiable. So this is

inappropriate for PTAS reduction.

84
Next goal. (completeness term) Introduce class NPO as abstract class for
optimization problem analogously to NP for decision problems.

Definition. (NPO) An optimization problem A corresponds to class NPO, if

• for all x and s(x) the target function value v(x, s) is an integer (could be
relaxed to Q).
• for all x and s, the test whether s ∈ S(x) (s is valid) must be computable
in polynomial time.
• v(x, s)∀x, s is computable in polynomial time.

MIN-NPO Minimization problem in NPO

MAX-NPO Maximization problem in NPO

Definition. An optimization problem A is NPO-complete iff

• A ∈ NPO
• ∀B ∈ NPO : B ≤PTAS A

In the following we consider an optimization problem of NPO. Therefore we are

interested for APX, PTAS, . . . now the intersection of NPO with the classes
defined previously.

The completeness definition for PTAS and APX is analogous.

There are MAX-NPO and MIN-NPO complete problems. It can be proven that
MAX-W-SAT is MAX-NPO complete and MIN-W-SAT is MIN-NPO complete.

Given. SAT equation ϕ with variables x1 , . . . , xn with clauses C1 , . . . , Cn .

Weights w1 , . . . , wn ∈ N0 .

Target function value.

( Pn
max 1, i=1 wi · ai a is satisfiable
v(a) =
1 else
n
a ∈ {0, 1} truth assignments.

Furthermore we can show that MAX-W-SAT =PTAS MIN-W-SAT. MAX-W-

SAT and MIN-W-SAT are NPO-complete.

Another complexity class in relation with approximation is MaxSNP (concept

of Paradimitriou and Yannakakis).

Definition. (Strict NP) All properties of the structure

∃s∀x1 ∀x2 , . . . , ∀xn : Φ(S, G, x1 , . . . , xn )

85
where Φ is a quantifier-free expression of first order, which contains the variables
x1 , . . . , xn and contains graph structure G and S.

Step towards optimization problem MaxSNP:

max |{(x1 , . . . , xn ) ∈ V n |Φ(G1 , G2 , . . . , Gm , S, x1 , . . . , xn )}|

S

where G1 , . . . Gm is input relation over a finite set V .

Find. Relation S such that the number of n-tuples with Φ is satisfied.

9.7.1 Example 1

Given. Graph G = (V, E).

Find. Cut which cuts the most number of edges.

Relation G (which also corresponds to the graph) describes set of edges

max |{(x, y) : G(x, y) ∨ G(y, x) ∧ S(x) ∧ ¬S(y)}|

S⊆V

(meaning there exists an edge between x and y and and an intersecting edge
exists).

9.7.2 Example 2: MAX-2-SAT

3 input relations: G0 , G1 , G2 . Gi describes all clauses with i intersected literals.

G0 (x, y) : x ∨ y is clause
G1 (x, y) : ¬x ∨ y is clause
G2 (x, y) : ¬x ∨ ¬y is clause

S represents a variable with truth value True.

max |{(x, y) : Φ(G0 , G1 , G2 , S, x, y)}|

S⊆V

Φ : [G0 (x, y)∧(S(x)∨S(y))]∨[G1 (x, y)∧(¬s(x)∨s(y))]∨[G2 (x, y)∧(¬s(x)∨¬s(y))]

9.7.3 Example 3: MAX

Independent-Set in graph G with restricted degree k.

Representation of G as (k + 1)-ary relation H. H contains |V | (k + 1)-tuples.

86
(y1 , . . . , yk ) where yi neighbors of x are repeated vertices, if they are of degree
x < k.

max |{(x, y1 , . . . , yk ) : (x, y1 , . . . , yk ) ∈ H ∧ x ∈ S ∧ yi ∈

/ S, i ∈ {1, . . . , K}}|
S⊆V

S is an independent set.

Remark. We can show the following.

• Closure of MaxSNP under PTAS reduction is APX.

• Every problem in MaxSNP has an approximation algorithm with con-
stant guarantee.
• Independent-Set ∈
/ MaxSNP, if P ̸= NP.

9.8 Linear-integer optimization models (“rounding”)

The following section discusses positive results.

9.8.1 MaxSAT

Given. SAT equation ϕ with variables x1 , . . . , xn and clauses C1 , . . . , Cm .

Find. Maximum number of satisfiable clauses

For clause Ci denominate Vi+ the set of non-negative variables in Ci . Vi− is the
set of negative variables in Ci .

We have 2 groups of variables:

1. x is variable. xi ∈ {0, 1}. xi = 1 if xi is true in ϕ. Else xi = 0.

2. z is variable. zj ∈ {0, 1}. zj = 1 if clause Cj is satisfied. Else zj = 0.

The number of satisfied clauses is

m
X
max zj
j=1

We require (IPMS ):
X X
xi + (1 − xi ) ≥ zj j = 1, . . . , m
xj ∈Vj+ xi ∈Vj−

87
with
xi ∈ {0, 1} , i = 1, . . . , n zj ∈ {0, 1} , j = 1, . . . , m

This requires integers.

Linear programming relaxation (LPMS ) Instead we define “with”:

0 ≤ xi ≤ 1

0 ≤ zj ≤ 1

This can be solved in polynomial time. Denote (x∗ , z ∗ ) as the optimal solution
of LPMS .

Problem. In general xk will not be an integer. Thus ∃i with xi ∈ (0, 1).

Idea. Round x∗ to an integer value x̃ which is used as approximation solution.

Idea for rounding. Randomized rounding. Set

(
1 ∈ ϕ with probability x∗i
xi =
0 ∈ ϕ with probability 1 − x∗i

We can show the following: The randomized algorithm above has expected
e
guarantee of 1−1 1 = e−1 .
e

Define random variable X as the number of satisfied clauses. Define Xi ∈ {0, 1}

whether clause Ci is satisfied.

m
X
X= Xi
i=1
m
X
E(X) = E(Xi )
i=1

E(Xi ) = prob(Xi = 1) = 1 − prob(Xi = 0)

Y Y
prob(Xj = 0) = (1 − x∗j ) · x∗j
xi ∈Vj+ xi ∈Vj−

Pk
Helping result. ∀t1 , . . . , tk ∈ R+ +
0 and t ∈ R0 with i=1 ti ≥ t it follows
Qk t k
i=1 (1 − ti ) ≤ (1 − k ) .

(x∗ , z ∗ ) is solution of LPMS therefore it follows that xi ∈V + x∗i + xj ∈V − (1 −

P P
j j
x∗j ) ≥ zj∗ ∀j.

88
With helping result
zj∗ kj
probability(Xj = 0) ≤ (1 − )
kj

z∗
where kj is the number of literals in Cj . Thus probability Xj = 1 ≥ 1−(1− kjj )kj .

Another helping result:

t k 1?
1 − (1 − ) ≥ (1 − (1 − )k ) · t ∀t ∈ [0, 1], k ∈ N
k k
1 kj
⇒ P (Xj = 1) ≥ (1 − (1 − ) ) · zj∗
kj
m
X 1 kj
⇒ E(X) ≥ (1 − (1 − ) ) · zj∗
j=1
kj

Optimal value of MaxSAT ≤ OPT value of LPMS .

m
X
zj∗ ≥ OP T ( ϕ )
|{z}
j=1
MAX SAT equation

1 kj 1
1 − (1 − ) ≥1−
kj e

QED.

Remark. By combination of the method above with RANDOM SAT (set

xi = 0 or xi = 1 with probability 12 statistically independent) we get a 43
approximation.

Further improvements can be achieved with semidefinite optimization.

9.8.2 Min-Weight Vertex Cover

Given. Undirected graph G = (V, E). Edge weights wi ≥ 0∀i ∈ V .

Find. Vertex-Cover with minimal weight sum.

In IPVC (different formulation, but again no optimization follows immediately)

(
1 i ∈ VC
xi ∈
0 i∈ / VC
n
X
min wi · xi
i=1

89
 xi + xj ≥ 1∀ {i, j} ∈ E
xj ∈ {0, 1} i = 1, . . . , n

Now consider again LP relaxation. Replace the last line with xi ≥ 0 (0 ≤ xi ≤

1).

In LPVC : Is x∗ optimal solution of LPVC . Apply rounding as follows:

(
0 if x∗i < 21
x̃i =
1 x∗i ≥ 12

x̃ is valid solution for IPVC . This gives us guarantee 2 (compare with Vertex
Cover problem at beginning of lecture notes).

Website: “A compendium of NP optimization problems”.

Appendices
A List of algorithms or interactive protocols
• Design and analyze an algorithm to determine whether a perfect matching
exists in a bipartite graph (page ??).
• Design and analyze an algorithm to decide 2-SAT and 3-SAT instances
probabilistically (page ??).

• Design and analyze an algorithm to test whether two given polynomials

are identical (page ??).
• Design and analyze an algorithm for primality testing (page ??).
• Design and analyze an algorithm for the FSAT problem using a SAT oracle
(page ??).

• Design and analyze an algorithm for the Happy Net Problem (page ??).
• Design and analyze an interactive protocol to show that GI ∈ IP (page ??).
• Design and analyze an approximation algorithm for Bin Packing (page ??).

• Design and analyze an approximation algorithm for Scheduling and Knap-

sack (page ?? and ??)

90
B List of mentioned complexity classes

More than 496 complexity classes are currently maintained in the Complexity
Zoo. We mentioned 23 complexity classes.
• mIP • PSPACE
• APX
• NEXP • PTAS
• BPP
• NP • PZK
• coIP
• NPO • RP
• DP
• P • UP
• FP
• PCP • ZPP
• FPTAS
• PH • #P
• IP
• PP • ⊕P
The following definitions for these 23 complexity classes got extracted from
Complexity Zoo.

APX The subclass of NPO problems that admit constant-factor approximation

algorithms. (I.e., there is a polynomial-time algorithm that is guaranteed
to find a solution within a constant factor of the optimum cost.)
Contains PTAS. Equals the closure of MaxSNP and of MaxNP under
PTAS reduction.
BPP The class of decision problems solvable by an NP machine such that
2
1. If the answer is “yes” then at least 3 of the computation paths accept.
1
2. If the answer is “no” then at most 3 of the computation paths accept.
(Here all computation paths have the same length.)
Often identified as the class of feasible problems for a computer
Q with access
to a genuine random-number source. Contained in σ2P ∩ 2P , and indeed
in ZPPNP.
If BPP contains NP, then RP = NP and PH is contained in BPP.
DP DP = BH2, the second level of the Boolean hierarchy.
FP Sometimes defined as the class of functions computable in polynomial time
by a Turing machine. (Generalizes P, which is defined in terms of decision
problems only.)
However, if we want to compare FP to FNP, we should instead define it as
the class of FNP problems (that is, polynomial-time predicates P (x, y))
for which there exists a polynomial-time algorithm that, given x, outputs
any y such that P (x, y). That is, there could be more than one valid
output, even though any given algorithm only returns one of them.
FP = FNP if and only if P = NP.

91
FPTAS The subclass of NPO problems that admit an approximation scheme in
the following sense. For any ε > 0, there is an algorithm that is guaranteed
to find a solution whose cost is within a 1 + ε factor of the optimum cost.
Furthermore, the running time of the algorithm is polynomial in n (the
size of the problem) and in 1ε .
Contained in PTAS and FPT.
IP The class of decision problems for which a “yes” answer can be verified by
an interactive proof. Here a probabilistic polynomial-time verifier sends
messages back and forth with an all-powerful prover. They can have poly-
nomially many rounds of interaction. Given the verifier’s algorithm, at
the end:
1. If the answer is “yes”, the prover must be able to behave in such a
way that the verifier accepts with probability at least 23 (over the
choice of the verifier’s random bits).
2. If the answer is “no”, then however the prover behaves the verifier
must reject with probability at least 23 .
With the motivation of providing a framework for the introduction of zero-
knowledge proofs. Interestingly, the power of general interactive proof
systems is not decreased if the verifier is only allowed random queries (i.e.,
it merely tosses coins and sends any outcome to the prover). The latter
model, known as the Arthur-Merlin (or public-coin) model was introduced
independently (but later), and a strong equivalent (which preserves the
number of rounds) is proved.
Often, it is required that the prover can convince the verifier to accept
correct assertions with probability 1; this is called perfect completeness.
However, the definitions with one-sided and two-sided error can be shown
to be equivalent.
First demonstration to the power of interactive proofs was given by show-
ing that for graph nonisomorphism (a problem not known in NP) has
such proofs. Five years later is was shown that IP contains PH, and in-
deed (this was discovered only a few weeks later) equals PSPACE. On the
other hand, coNP is not contained in IP relative to a random oracle.
mIP Same as IP, except that now the verifier can exchange messages with many
provers, not just one. The provers cannot communicate with each other
during the execution of the protocol, so the verifier can “cross-check” their
assertions (as with suspects in separate interrogation rooms).
Let MIP[k] be the class of decision problems for which a “yes” answer can
be verified with k provers. Then for all k > 2, MIP[k] = MIP[2] = MIP.
MIP equals NEXP; this is a famous non-relativizing result.

NEXP Nondeterministic exponential time (i.e. NTIME(2p(n)) for p a polyno-

mial). Equals MIP (but not relative to all oracles). Does not equal NP.
Does not equal EXP if and only if there is a sparse set in NP that is not
in P.

92
NP An “NP machine” is a nondeterministic polynomial-time Turing machine.
Then NP is the class of decision problems solvable by an NP machine such
that
1. If the answer is “yes”, at least one computation path accepts.
2. If the answer is “no”, all computation paths reject.
Equivalently, NP is the class of decision problems such that, if the answer
is “yes”, then there is a proof of this fact, of length polynomial in the size of
the input, that can be verified in P (i.e. by a deterministic polynomial-time
algorithm). On the other hand, if the answer is “no”, then the algorithm
must declare invalid any purported proof that the answer is “yes”.
For example, the SAT problem is to decide whether a given Boolean
formula has any satisfying truth assignments. SAT is in NP, since a “yes”
answer can be proved by just exhibiting a satisfying assignment.
A decision problem is NP-complete if (1) it is in NP, and (2) any problem
in NP can be reduced to it (under some notion of reduction). The class
of NP-complete problems is sometimes called NPC.
That NP-complete problems exist is immediate from the definition. The
seminal result of Cook, Karp, and Levin is that many natural problems
(that have nothing to do with Turing machines) are NP-complete.
The first such problem to be shown NP-complete was SAT. Other classic
NP-complete problems include:
NP contains P.
If P equals NP, then NP equals its complement coNP. Whether NP equals
coNP is also open. NP and coNP can be extended to the polynomial
hierarchy PH.
If NP is in BPP then PH collapses to σ2P .
An alternate characterization is NP = PCP(log n, O(1)).
Also, showed that NP is precisely the class of decision problems reducible
to a graph-theoretic property expressible in second-order existential logic.
This leads to the subclass SNP.
NPO The class of function problems of the form, “Find any n-bit string x
that maximizes a cost function C(x), where C is computable in FP (i.e.
polynomial-time).” Contains APX.
P The class of decision problems solvable in polynomial time by a Turing ma-
chine. Contains some highly nontrivial problems, including linear pro-
gramming and finding a maximum matching in a general graph.
Contains the problem of testing whether an integer is prime, an important
result that improved on a proof requiring an assumption of the generalized
Riemann hypothesis.
A decision problem is P-complete if it is in P, and if every problem in P
can be reduced to it in L (logarithmic space). The canonical P-complete
problem is Circuit Evaluation: given a Boolean circuit and an input,
decide what the circuit outputs when given the input.
Important subclasses of P include L, NL, NC, and SC.

93
PCP The class of decision problems such that a “yes” answer can be verified
by a probabilistically checkable proof, as follows.
The verifier is a polynomial-time Turing machine with access to O(r(n))
uniformly random bits. It has random access to a proof (which might be
exponentially long), but can query only O(q(n)) bits of the proof.
Then we require the following:
• If the answer is “yes”, there exists a proof such that the verifier
accepts with certainty.
• If the answer is “no”, then for all proofs the verifier rejects with
probability at least 21 (over the choice of the O(r(n)) random bits).
By definition NP = PCP(0, poly(n)). MIP = PCP(poly(n), poly(n)).
NP = PCP(log n, log n). In fact, NP = PCP(log n, 1)!
On the other hand, if NP is contained in PCP(o(log n), o(log n)), then P
= NP.
Q
PH Let △ 0P = σ0P = 0P = P. Then for i > 0, let
• △i P = P with σi−1 P oracle.
• σi P = NP with σi−1 P oracle.
Q
• i P = coNP with σi−1 P oracle.
Then PH is the union of these classes for all nonnegative constants i.
PH can also be defined using alternating quantifiers: it’s the class of prob-
lems of the form, “given an input x, does there exist a y such that for
all z, there exists a w . . . such that ϕ(x, y, z, w, . . .)”, where y, z, w, . . .
are polynomial-size strings and ϕ is a polynomial-time computable predi-
cate. It’s not totally obvious that this is equivalent to the first definition,
since the first one involves adaptive NP oracle queries and the second one
doesn’t, but it is.
Contained in P with a PP oracle. Contains BPP.
Relative to a random oracle, PH is strictly contained in PSPACE with
probability 1.
PP The class of decision problems solvable by an NP machine such that
1
1. If the answer is “yes” then at least 2 of computation paths accept.
2. If the answer is “no” then less than 12 of computation paths accept.
PP is closed under union and intersection (this was an open problem for
14 years). Contains PNP[log].
PSPACE The class of decision problems solvable by a Turing machine in poly-
nomial space.
Equals NPSPACE, AP, IP, and, assuming the existence of one-way func-
tions, CZK. Contains P with #P oracle.
Relative to a random oracle, PSPACE strictly contains PH with probabil-
ity 1.

94
 PSPACE has a complete problem that is both downward self-reducible
and random self-reducible. It is the largest class with such a complete
problem.

PTAS The subclass of NPO problems that admit an approximation scheme in

the following sense. For any ε > 0, there is a polynomial-time algorithm
that is guaranteed to find a solution whose cost is within a 1 + ε factor
of the optimum cost. (However, the exponent of the polynomial might
depend strongly on ε.)
Contains FPTAS, and is contained in APX. As an example, the Traveling
Salesman Problem in the Euclidean plane is in PTAS.
PZK Same as SZK, but now the two distributions must be identical, not merely
statistically close. (The “two distributions” are (1) the distribution over
the verifier’s view of his interaction with the prover, conditioned on the
verifier’s random coins, and (2) the distribution over views that the verifier
can simulate without the prover’s help.) Contained in SZK.
RP The class of decision problems solvable by an NP machine such that
1
1. If the answer is “yes”, at least 2 of computation paths accept.
2. If the answer is “no”, all computation paths reject.
Contains the problem of testing whether an integer is prime, although this
problem was subsequently shown to be in P.
UP The class of decision problems solvable by an NP machine such that

1. If the answer is “yes”, exactly one computation path accepts.

2. If the answer is “no”, all computation paths reject.
“Worst-case” one-way functions exist if and only if P does not equal UP.
“Worst-case” one-way permutations exist if and only if P does not equal
UP ∩ coUP. Note that these are weaker than the one-way functions and
permutations that are needed for cryptographic applications.
There exists an oracle relative to which P is strictly contained in UP is
strictly contained in NP; indeed, these classes are distinct with probability
1 relative to a random oracle.
ZPP Defined as RP ∩ coRP. Defined as the class of problems solvable by ran-
domized algorithms that always return the correct answer, and whose
expected running time (on any input) is polynomial.
Contains the problem of testing whether an integer is prime.
coIP Complementary class of IP.

#P The class of function problems of the form “compute f (x),” where f is the
number of accepting paths of an NP machine. The canonical #P-complete
problem is #SAT. Any function in #P can be approximated to within a
polynomial factor in BPP with NP oracle. Likewise, any problem in #P
can be approximated to within a constant factor ε by a machine in FP||NP
running in poly(n, ε−1 ) time.

95
⊕P The class of decision problems solvable by an NP machine such that
• If the answer is “yes”, then the number of accepting paths is odd.
• If the answer is “no”, then the number of accepting paths is even.

C Exam questions
1. What does the Cook-Levin theorem state?

2. What are Monte-Carlo algorithms? What are Las-Vegas algorithms? De-

fine the probabilistic complexity classes RP, co-RP, BPP, PP and ZPP.
Define their relation towards each other and P and NP.
3. Formally define a Monte-Carlo turing machine.

4. Define the MajSat problem. Classify MajSat in terms of complexity

classes.
5. Can you create a fair die with polynomial effort using an unfair die? How?
What does it tell about probabilistic complexity classes?
6. What are function problems? Define the complexity class FP. Give 2
example problems. What are total functions?
7. Define the TSP variants TSPDEC , Exact-TSP, TSP-Cost and TSP.
8. Define the complexity class DP and prove that Sat-UnSAT and Exact-TSP
are DP-complete.

9. What is the polynomial hierarchy? What does it define? Which conclu-

sions can we draw? Consider the QSATi problem. In which level of the
hierarchy is it contained?
10. What does polynomially balanced mean?

11. What is the relation between BPP and the polynomial hierarchy?
12. Define what a counting problem is. How is the class #P defined? Show
at least two problems that are #P-complete.
13. Prove that Permanent ∈ #P-complete.

14. What does the Theorem of Toda say?

15. What are parsimonious reductions?
16. What is the relation between PP and #P?
17. Define the complexity class ⊕P. Discuss ⊕SAT, ⊕PERM. Show that NP
⊆ RP⊕P .
18. Discuss which interactive protocol variants exist and what their properties
are. Discuss their relation to P, NP and PSPACE. Define the class IP.

96
19. Define the perfect zero knowledge property. What does bit commitment
mean?

20. What are the statistical zero knowledge property and the computational
zero knowledge property?
21. Define the complexity class UP. What is the relation between UP and P?
22. What does the PCP-Theorem say? When is a decision problem part of
PCP(r(n), q(n))?

23. Discuss the linearity and consistency tests for functions.

24. Define the notion of approximation quality and what does c-approximation
mean? Define the complexity classes APX and APX*.
25. What does PTAS and FPTAS stand for? Discuss the Max-Clique and
Vertex-Cover problems in terms of approximability.
26. What is the gap technique? Illustrate it with TSP as an example. Derive
the third definition of the PCP-Theorem.
27. How does reduction work for approximation problems?

28. Define the complexity class NPO.

29. Consider MaxSAT and Min-Weight Vertex Cover as linear program-
ming problems.

97