CS 341: Chapter 5 5-2

Chapter 5
Reducibility
CS 341: Foundations of CS II
Contents
• Reducing One Problem to Another
• Examples of Undecidable Problems
Marvin K. Nakayama • Mapping Reducibility
Computer Science Department
New Jersey Institute of Technology
Newark, NJ 07102

CS 341: Chapter 5 5-3 CS 341: Chapter 5 5-4

Introduction Reducibility
• Previously, we saw
• Reduction always involves two problems, A and B .
Church-Turing Thesis
Many problems are solvable using TMs • Definition: If A reduces to B , then any solution of B solves A.
One problem, ATM, is unsolvable by TMs, where • Remarks:
ATM = { M, w | M is a TM that accepts string w } We showed that ANFA is decidable by reducing ANFA to ADFA.
Definition of reduction says nothing about solving A or B alone.
• We now will see many other computationally unsolvable problems.
If A is reducible to B , then A cannot be harder than B .
• We will do this by using reductions. The statement “p ⇒ q ” is equivalent to “¬q ⇒ ¬p”.
Suppose A reduces to B . Then
• Example:  If I can solve B , then I can solve A.
Finding your way around a city  Equivalently, if I can’t solve A, then I can’t solve B .
reduces to  Equivalently, if A is undecidable, then B is undecidable.
obtaining a city map.
CS 341: Chapter 5 5-5 CS 341: Chapter 5 5-6
Reducibility Halting Problem for TMs is Undecidable
• It required some effort to prove that ATM is not decidable. • Recall that ATM (acceptance problem for TMs) is undecidable, where
• But now we can build on this result as follows: ATM = { M, w | M is a TM and M accepts string w }.
To show another language L is undecidable, • Define related problem:
we typically show ATM reduces to L.
HALT TM = { M, w | M is a TM and M halts on string w }.
If “language L is decidable” implies “ATM is decidable,”
then L is not decidable. • In this case (but not others), ATM and HALT TM have same universe
• Typical approach to show L is undecidable via reduction from ATM to L Ω = { M, w | M is TM, w is string }.

Suppose L is decidable. • Given a specific M, w ∈ Ω of a specific pair of TM M and string w,

Let R be a TM that decides L. if M halts on input w, then M, w ∈ HALT TM,
Using R as subroutine, if M doesn’t halt on input w, then M, w ∈ HALT TM.
 can construct another TM S that decides ATM .
• How does HALT TM differ from ATM?
But ATM is not decidable.
Conclusion: L is not decidable. Theorem 5.1
HALT TM is undecidable.

CS 341: Chapter 5 5-7 CS 341: Chapter 5 5-8

Basic Idea of Proof that HALT TM is Undecidable Proof that HALT TM is Undecidable

ATM = { M, w | M is a TM and M accepts string w }, ATM = { M, w | M is a TM and M accepts string w },
HALT TM = { M, w | M is a TM and M halts on string w }. HALT TM = { M, w | M is a TM and M halts on string w }.

Basic idea of proof by contradiction is to reduce ATM to HALT TM: • Assume ∃ TM R that decides HALT TM.
• Define TM S to decide ATM using TM R as follows:
• Suppose ∃ TM R that decides HALT TM.
S = “On input M, w ∈ Ω, where M is a TM and w a string:
• How could we use R to construct TM to decide ATM? 1. Run R on input M, w.
• Recall universal TM U recognizes ATM: 2. If R rejects, reject.
3. If R accepts, simulate M on input w until it halts.
U = “On input M, w, where M is a TM and w is a string:
4. If M accepts, accept; otherwise, reject.”
1. Simulate M on input w.
2. If M ever enters its accept state, accept; • TM S always halts and decides ATM
if M ever enters its reject state, reject.” S accepts M, w ∈ ATM, and S rejects M, w ∈ ATM.
• U doesn’t decide ATM since M may loop on w in stage 1. • Thus, deciding ATM is reduced to deciding HALT TM.
• Solution: first run R on M, w to see if it’s safe to run M on w. • But ATM is undecidable, so HALT TM must also be undecidable.
CS 341: Chapter 5 5-9 CS 341: Chapter 5 5-10
Emptiness Problem for TMs is Undecidable Constructing Decider S for ATM From Decider R for ETM
• Does a TM M recognize the empty language? ETM = { M  | M is a TM and L(M ) = ∅ }
ETM = { M  | M is a TM and L(M ) = ∅ }.
• Bad Idea: When S receives input M, w, it calls R with input M .
• Universe Ω = { M  | M is a TM } of all Turing machines. If R accepts, then L(M ) = ∅.
• For a specific encoded TM M  ∈ Ω,  In particular, M does not accept w , so S rejects input M, w.
if M accepts at least one string, then M  ∈ ETM, If R rejects, then L(M ) = ∅, so M accepts at least one string.
if M accepts no strings, then M  ∈ ETM.  But don’t know if M accepts w , so TM S can’t decide ATM.

Theorem 5.2 • Fix: Create another TM M1 from TM M and w as follows:

ETM is undecidable. M1 = “On input x:
Proof Idea: Reduce ATM to ETM. 1. If x = w, reject.
2. If x = w, run M on input w, and accept if M accepts.”
• Suppose ETM is decidable.
w is only string M1 could accept, so one of 2 cases occurs:
• Let R be a TM that decides ETM.
• Use TM R to construct another TM S that decides ATM.  If M, w ∈ ATM, then L(M1) = {w}, so M1 ∈ ETM.
 If M, w ∈ ATM, then L(M1) = ∅, so M1 ∈ ETM.
• But since ATM is undecidable, ETM must also be.

CS 341: Chapter 5 5-11 CS 341: Chapter 5 5-12

Proof: ETM = { M  | M is TM and L(M ) = ∅ } is Undecidable TM Recognizing Regular Language is Undecidable
• Suppose ∃ TM R that decides ETM. • Does a TM M recognize a regular language?
• Define TM S to decide ATM using decider R for ETM as follows: REG TM = { M  | M is a TM and L(M ) is a regular language }
S = “On input M, w, where M is a TM and w is a string:
• Universe Ω = { M  | M is a TM } of all Turing machines.
1. Construct TM M1 from M and w as follows:
M1 = “On input x: • For a specific encoded TM M  ∈ Ω,
(1) If x = w, reject. if L(M ) is regular, then M  ∈ REG TM,
(2) If x = w, run M on input w, if L(M ) is nonregular, then M  ∈ REG TM.
and accept if M accepts.”
2. Run R on input M1. Theorem 5.3
3. If R accepts, reject; if R rejects, accept.” REG TM is undecidable.
• Note that Proof Idea: Reduce ATM to REG TM.
M1 ∈ ETM ⇐⇒ L(M1) = ∅ ⇐⇒ M accepts w • Assume REG TM is decidable.
⇐⇒ M, w ∈ ATM. • Let R be a TM that decides REG TM.
• But then TM S decides ATM, which is undecidable. • Use TM R to construct TM S that decides ATM.
• Therefore, TM R cannot exist, so ETM is undecidable. • But how do we do this?
CS 341: Chapter 5 5-13 CS 341: Chapter 5 5-14
Constructing Decider S for ATM from Decider R for REG TM Proof that REG TM is Undecidable

REG TM = { M  | M is a TM and L(M ) is a regular language }. • Suppose that REG TM is decidable.

• Let R be a TM that decides REG TM.
• TM S is given input M, w.
• Use R to construct TM S to decide ATM:
• TM S first constructs a TM M
using M, w so that TM M

recognizes a regular language if and only if M accepts w. S = “On input M, w, where M is a TM and w is a string:

If M does not accept w, then M

recognizes language 1. Construct following TM M
from M and w:

{ 0n 1n | n ≥ 0 }, M
= “On input x:
1. If x ∈ { 0n1n | n ≥ 0 }, accept.
which is nonregular.
2. If x ∈ { 0n1n | n ≥ 0 }, run M on input w
If M accepts w, then M
recognizes language Σ∗, which is regular.
and accept if M accepts w.”
• We construct M
as follows: 2. Run R on input M
.
M
automatically accepts all strings in { 0n 1n | n ≥ 0 }. 3. If R accepts, accept; if R rejects, reject.”
In addition, if M accepts w, then M
accepts all other strings. • M
 ∈ REG TM ⇐⇒ M, w ∈ ATM,
so S decides ATM, which is impossible.

CS 341: Chapter 5 5-15 CS 341: Chapter 5 5-16

Equivalence of 2 TMs is Undecidable Proof that EQTM is Undecidable
• Do 2 TMs recognize the same language? • Recall
EQTM = { M1, M2 | M1, M2 are TMs and L(M1) = L(M2) }. EQTM = { M1, M2 | M1, M2 are TMs and L(M1) = L(M2) }.

• Universe • Reduce ETM to EQTM as follows:

Ω = { M1, M2 | M1, M2 are TMs } Let M2 = M∅ be a TM with L(M∅ ) = ∅.
of all possible pairs of Turing machines. A TM that decides EQTM can also decide ETM by deciding if
M1, M∅ ∈ EQTM.
• For any specific encoded pair M1, M2 ∈ Ω,
 M1  ∈ ETM ⇐⇒ M1, M∅ ∈ EQTM
if L(M1) = L(M2), then M1, M2 ∈ EQTM,
if L(M1) = L(M2), then M1, M2 ∈ EQTM. • Since ETM is undecidable (Theorem 5.2), EQTM must be undecidable.
• We’ll see later that EQTM is
Theorem 5.4 not Turing-recognizable
EQTM is undecidable. not co-Turing-recognizable
CS 341: Chapter 5 5-17 CS 341: Chapter 5 5-18
Other Undecidable Problems Proof of Rice’s Theorem: Reduce ATM to P

• Does a TM recognize a finite language? • Suppose P is decided by TM RP .

• Does a TM recognize a context-free language? • Let T∅ be a TM that always rejects, so L(T∅) = ∅.
• Does a TM recognize a decidable language? • Without loss of generality, assume T∅ ∈ P . (Otherwise, consider P .)
• Does a TM halt on all inputs? • Because we assumed P is nontrivial, ∃ TM T with T  ∈ P .
• Does a TM have a state that is never entered on any input string? • Now design TM S to decide ATM using RP ’s ability to distinguish
between T∅ and T .
Rice’s Theorem. S = “On input M, w, where M is a TM and w a string:
• Informally: Every non-trivial property P of languages of Turing 1. Use M and w to construct the following TM Mw :
machines is undecidable. Mw = “On input x:
• Formally: Let P be a language consisting of TM descriptions such that 1. Simulate M on input w. If it halts and rejects, reject.
2. Simulate T on input x. If it accepts, accept.”
1. P contains some, but not all, TM descriptions, and
2. Use TM RP to determine whether Mw  ∈ P .
2. whenever L(M1) = L(M2), we have M1 ∈ P iff M2 ∈ P .
If YES, accept. If NO, reject.”
Then P is undecidable.

CS 341: Chapter 5 5-19 CS 341: Chapter 5 5-20

Proof of Rice’s Theorem: Reduce ATM to P (cont.) Limited Success Thus Far
• Note that TM Mw simulates T if M accepts w. • Our reductions have been straightforward:
• Hence, Transform TM for some language into a similar TM that decides
another language
L(Mw ) = L(T ) if M accepts w,
L(Mw ) = ∅ if M does not accept w. • As a result, the languages we proved are undecidable are similar:

• Therefore, Mw  ∈ P iff M accepts w. ATM, EQTM, HALT TM, etc.

• Hence, S decides ATM, which is impossible since ATM is undecidable. • For languages concerning questions not about TMs,
we have to use a different approach.
• Thus, P is undecidable.
e.g., Hilbert’s 10th problem
• Recall interpretation of TM configuration:
1011q701
current state is q7
LHS of tape is 1011, and RHS of tape is 01
tape head is on RHS 0
CS 341: Chapter 5 5-21 CS 341: Chapter 5 5-22
Computation Histories Remarks About Computation Histories
• Accepting and rejecting computation histories are finite.
Definition: An accepting computation history for a TM M on a
string w is a sequence of configurations • If M does not halt on w,
C1, C2, . . . , Ck then no accepting or rejecting computation history exists.
for some k ≥ 1 such that the following properties hold:
• Useful for both
1. C1 is the start configuration of M on w.
deterministic TMs (one history)
2. Each Cj yields Cj+1.
nondeterministic TMs (many histories).
3. Ck is an accepting configuration.
• “M, w ∈ ATM” is equivalent to
Definition: A rejecting computation history for M on w is the “ ∃ accepting computation history C1, . . . , Ck for M on w”
same except last configuration Ck is a rejecting configuration of M .
“All histories C1, . . . , Ck are non-accepting ones for M on w”.

CS 341: Chapter 5 5-23 CS 341: Chapter 5 5-24

Context-Free Languages Mapping Reducibility
Does a CFG generate all strings over Σ?
ALLCFG = { G | G is a CFG with L(G) = Σ∗ }. • Thus far, we have seen several ways to reduce one problem to another.

• Reductions appear in
Theorem 5.13
ALLCFG is undecidable. decidability theory
complexity theory (as we’ll see later in Chapter 7).
Proof Idea: (see Sipser for full proof)
• Approach: Reduce ATM to ALLCFG. • Now we want to formalize the notion of reducibility.
• Construct a CFG G from TM M and input w.
If M does not accept w, then G generates all strings.
If M accepts w, then G generates all strings except the accepting
computation histories for M on w.
• CFG G generates all strings iff TM M does not accept w.
CS 341: Chapter 5 5-25 CS 341: Chapter 5 5-26
Computable Functions Computable Functions
• Suppose we have 2 languages A and B , where One useful class of computable functions transforms one TM into another.
A is defined over alphabet Σ1, so A ⊆ Σ∗1, i.e., universe Ω1 = Σ∗1
B is defined over alphabet Σ2, so B ⊆ Σ∗2, i.e., universe Ω2 = Σ∗2 Example:

• Informally speaking, A is reducible to B if we can use a “black box” for T = “On input w:
B to build an algorithm for A. 1. If w = M , where M is some TM,
• Construct M
, where M
is a TM such that
• Definition: A function L(M
) = L(M ), but
M
never tries to move tape head off LHS of tape.”
f : Σ∗1 → Σ∗2
is a computable function if some TM M , on every input w ∈ Σ∗1, The function T accomplishes this by adding several states to the
halts with just f (w) ∈ Σ∗2 on its tape. description of M .
• All the usual integer computations are computable:
Addition, multiplication, sorting, etc.

CS 341: Chapter 5 5-27 CS 341: Chapter 5 5-28

Mapping Reducibility Language A is Mapping Reducible to B

Definition: Suppose f
• A is defined over alphabet Σ1, so A ⊆ Σ∗1, i.e., universe Ω1 = Σ∗1 Σ∗1 Σ∗2
• B is defined over alphabet Σ2, so B ⊆ Σ∗2, i.e., universe Ω2 = Σ∗2
f B
Then A is mapping reducible to B , written A
A ≤m B
if there is a computable function
f : Σ∗1 → Σ∗2
such that, for every w ∈ Σ∗1,
w∈A ⇐⇒ f (w) ∈ B. w∈A ⇐⇒ f (w) ∈ B
YES instance for problem A ⇐⇒ YES instance for problem B
The function f is called a reduction of A to B .
(f is also called a many-one reduction.)
CS 341: Chapter 5 5-29 CS 341: Chapter 5 5-30
Example: Mapping Reduction ATM ≤m HALT TM Example: Mapping Reduction ATM ≤m HALT TM
• Recall that
The following TM F computes this function f .
ATM = { M, w | M is a TM that accepts string w },
HALT TM = { M, w | M is a TM that halts on string w }. F = “On input M, w, where M is a TM and w is a string:
• Instance for acceptance problem is M, w.
1. Construct the following TM M
:
• Instance for halting problem is M, w.
• We previously proved that HALT TM is undecidable by showing M
= “On input x:
ATM reduces to HALT TM. 1. Run M on input x.
2. If M accepts, accept.
• To show ATM ≤m HALT TM, need computable function f with
3. If M rejects, enter a loop.”
input M, w, which is an instance for acceptance problem for TMs
output f (M, w) = M
, w
, which is an instance for halting 2. Output M
, w.”
problem for TMs
M, w ∈ ATM ⇐⇒ f (M, w) = M
, w
 ∈ HALT TM. Note that M, w ∈ ATM ⇐⇒ M
, w ∈ HALT TM.

CS 341: Chapter 5 5-31 CS 341: Chapter 5 5-32

Decidability obeys ≤m Ordering Undecidability obeys ≤m Ordering
Theorem 5.22 Corollary 5.23
If A ≤m B and B is decidable, then A is decidable. If A ≤m B and A is undecidable, then B is undecidable also.
Proof. f Proof. Language A undecidable and B decidable contradicts the
Σ∗1 Σ∗2
• Let MB be TM that decides B . previous theorem.
f B
A
• Let f be reducing fcn from A to B . Recall: Complements A = Σ∗1 − A and B = Σ∗2 − B .
• Consider the following TM:
MA = “On input w: Fact: If A ≤m B , then A ≤m B .
1. Compute f (w). Proof.
f
2. Run MB on input f (w) and give the same result.” • Let f be reducing fcn of A to B : Σ∗1 Σ∗2

• Since f is a reducing function, w ∈ A ⇐⇒ f (w) ∈ B . w ∈ A ⇐⇒ f (w) ∈ B. A

f B

If w ∈ A, then f (w) ∈ B , so MB and MA accept. • Same fcn f shows A ≤m B since

If w ∈
 A, then f (w) ∈ B , so MB and MA reject. w ∈ A ⇐⇒ f (w) ∈ B.
• Thus, MA decides A.
CS 341: Chapter 5 5-33 CS 341: Chapter 5 5-34
Recognizability and ≤m Unrecognizability and ≤m
Theorem 5.28 Corollary 5.29
If A ≤m B and B is Turing-recognizable, then A is Turing-recognizable. If A ≤m B and A is not Turing-recognizable,
Proof. then B is not Turing-recognizable.
Proof. Language A not Turing-recognizable and B Turing-recognizable
• Let MB be a TM that recognizes B
contradicts the previous theorem.
• Let f be a reducing function from A to B .
• Define a new TM as follows: Fact: If A ≤m B and A is not co-Turing-recognizable,
MA = “On input w: then B is not co-Turing-recognizable.
1. Compute f (w). Proof.
2. Run MB on input f (w) and give the same result.”
• If A is not co-Turing-recognizable,
• Since f is a reducing function, w ∈ A ⇐⇒ f (w) ∈ B . then its complement A is not Turing-recognizable.
If w ∈ A, then f (w) ∈ B , so MB and MA accept. • A ≤m B implies A ≤m B (see slide 5-32).
If w ∈
 A, then f (w) ∈ B , so MB and MA reject or loop. • Corollary 5.29 implies B is not Turing-recognizable.
• Thus, MA recognizes A. • Hence, B is not co-Turing-recognizable.

CS 341: Chapter 5 5-35 CS 341: Chapter 5 5-36

ETM is not Turing-recognizable Theorem 5.30: EQTM is not Turing-recognizable
Recall: the emptiness problem for TMs: EQTM = { M1, M2 | M1, M2 are TMs with L(M1) = L(M2) }.
ETM = { M  | M is a TM with L(M ) = ∅ }.
Proof. Reduce ATM ≤m EQTM, and apply Corollary 5.29.
Proof. Reduce ATM ≤m ETM, and apply Corollary 5.29.
• Define reducing function f (M, w) = M1, M2, where
• Define reducing function f (M, w) = M
,
M1 = “reject on all inputs.”
where M
is following TM:
M2 = “On input x:
M
= “On input x:
1. Ignore input x, and run M on w.
1. Ignore input x, and run M on input w.
2. If M accepts w, accept.”
2. If M accepts w, then accept.”
• L(M1) = ∅.
• If M accepts w (i.e., M, w ∈ ATM), then L(M
) = Σ∗;
if M doesn’t accept w (i.e., M, w ∈ ATM), then L(M
) = ∅. • If M accepts w (i.e., M, w ∈ ATM), then L(M2) = Σ∗.
If M doesn’t accept w (i.e., M, w ∈ ATM), then L(M2) = ∅.
• Thus, M, w ∈ ATM ⇐⇒ f (M, w) = M
 ∈ ETM.
• Thus, M, w ∈ ATM ⇐⇒ f (M, w) = M1, M2 ∈ EQTM.
• Cor. 5.29 implies ETM not TM-recog. since ATM also isn’t (Cor. 4.23).
• ATM not TM-recognizable (Cor. 4.23),
so EQTM not TM-recognizable by Corollary 5.29.
CS 341: Chapter 5 5-37 CS 341: Chapter 5 5-38
Theorem 5.30: EQTM is not co-Turing-recognizable Summary of Chapter 5
EQTM = { M1, M2 | M1, M2 are TMs with L(M1) = L(M2) }. • Computable function f : Σ∗1 → Σ∗2 has TM that maps
Proof. Reduce ATM ≤m EQTM, and apply Fact on slide 5-34. strings in Σ∗1 (i.e., instances of one problem)
to strings in Σ∗2 (i.e., instances of another problem)
• Define reducing function f (M, w) = M1, M2, where
• Mapping reduction A ≤m B :
M1 = “accept on all inputs.” w ∈ A ⇐⇒ f (w) ∈ B , for some computable function f .
M2 = “On input x:
If I can solve B , then I can solve A.
1. Ignore input, and run M on w.
If I can’t solve A, then I can’t solve B .
2. If M accepts w, accept.”
• Undecidable problems: ATM, HALT TM, ETM, REG TM, EQTM,
• L(M1) = Σ∗.
ALLCFG
• If M accepts w (i.e., M, w ∈ ATM), then L(M2) = Σ∗.
• Rice’s Theorem: any nontrivial property of the language of a TM is
If M doesn’t accept w (i.e., M, w ∈ ATM), then L(M2) = ∅.
undecidable.
• M, w ∈ ATM ⇐⇒ f (M, w) = M1, M2 ∈ EQTM.
• ETM is not Turing-recognizable.
• Because ATM is not co-Turing-recognizable,
• EQTM is neither Turing-recognizable nor co-Turing-recognizable.
EQTM is not co-Turing-recognizable by Fact on slide 5-34.