CS53 THEORY OF COMPUTATION

Einstein College of Engineering

UNIT-I
AUTOMATA
Introduction
Why do we study Theory of Computation ?

Importance of Theory of Computation

Languages
Languages and Problems

What is Computation ?
Sequence of mathematical operations ?

What are, and are not, mathematical operations?

Sequence of well-defined operations

How many operations ?

The fewer, the better.

Which operations ?

The simpler, the better.

What do we study in Theory of Computation ?

What is computable, and what is not ?

Basis of
Algorithm analysis

Complexity theory

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

What a computer can and cannot do

Are you trying to write a non-existing program?

Can you make your program more efficient?

What do we study in Complexity Theory ?

What is easy, and what is difficult, to compute ?

What is easy, and what is hard for computers to do?

Is your cryptograpic scheme safe?

Applications in Computer Science

Analysis of algorithms
Complexity Theory
Cryptography

Compilers
Circuit design

History of Theory of Computation

1936 Alan Turing invented the Turing machine, and proved that there exists an
unsolvable problem.
1940s Stored-program computers were built.
1943 McCulloch and Pitts invented finite automata.
1956 Kleene invented regular expressions and proved the equivalence of regular
expression and finite automata.
1956 Chomsky defined Chomsky hierarchy, which organized languages
recognized by different automata into hierarchical classes.
1959 Rabin and Scott introduced nondeterministic finite automata and proved its
equivalence to (deterministic) finite automata.
1950s-1960s More works on languages, grammars, and compilers

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

1965 Hartmantis and Stearns defined time complexity, and Lewis, Hartmantis and
Stearns defined space complexity.
1971 Cook showed the first NP-complete problem, the satisfiability prooblem.
1972 Karp Showed many other NP-complete problems.

Alphabet and Strings

An alphabet is a finite, non-empty set of symbols.

{0,1 } is a binary alphabet.
{ A, B, , Z, a, b, , z } is an English alphabet.

A string over an alphabet is a sequence of any number of symbols from .

0, 1, 11, 00, and 01101 are strings over {0, 1 }.
Cat, CAT, and compute are strings over the English alphabet.

An empty string, denoted by , is a string containing no symbol.

is a string over any alphabet.

The length of a string x, denoted by length(x), is the number of positions of

symbols in the string.

Let = {a, b, , z}
length(automata) = 8
length(computation) = 11
length() = 0
x(i), denotes the symbol in the ith position of a string x, for 1 i length(x).

String Operations

Concatenation
Substring
Reversal

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

The concatenation of strings x and y, denoted by xy or x y, is a string z such that:

z(i) = x(i) for 1 i length(x)
z(i) = y(i) for length(x)<ilength(x)+length(y)

Example
automatacomputation = automatacomputation
The concatenation of string x for n times, where n0, is denoted by xn
x0 =
x1 = x
x2 = x x
x3 = x x x

Substring
Let x and y be strings over an alphabet
The string x is a substring of y if there exist strings w and z over such that y = w
x z.
is a substring of every string.
For every string x, x is a substring of x itself.
Example
, comput and computation are substrings of computation.
Reversal
Let x be a string over an alphabet
The reversal of the string x, denoted by x r, is a string such that
if x is , then xr is .
If a is in , y is in * and x = a y, then xr = yr a.
(automata)r
= (utomata)r a
= (tomata)r ua
= (omata)r tua
= (mata)r otua
= (ata)r motua
= (ta)r amotua
= (a)r tamotua
= ()r atamotua

= atamotua The set of strings created from any number (0 or 1 or ) of symbols

in an alphabet is denoted by *.
That is, * = i=0 i
Let = {0, 1}.
* = {, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, }.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

The set of strings created from at least one symbol (1 or 2 or ) in an alphabet

is denoted by +.

That is, + = i=1 i

= i=0.. i - 0
= i=0.. i - {}
Let = {0, 1}. + = {0, 1, 00, 01, 10, 11, 000, 001, 010, 011, }.
* and + are infinite sets.

A language over an alphabet is a set of strings over .

Let = {0, 1} be the alphabet.
Le = {* | the number of 1s in is even}.
, 0, 00, 11, 000, 110, 101, 011, 0000, 1100, 1010, 1001, 0110, 0101,
0011, are in Le

Operations on LanguagesComplementation

Union
Intersection
Concatenation
Reversal
Closure

Complementation
Let L be a language over an alphabet .
The complementation of L, denoted byL, is *L.
Example:
Let = {0, 1} be the alphabet.
Le = {* | the number of 1s in is even}.
Le= {* | the number of 1s in is not even}.
Le= {* | the number of 1s in is odd}.

Union
Let L1 and L2 be languages over an alphabet .
The union of L1 and L2, denoted by L1L2, is {x | x is in L1 or L2}.
Example:
{x{0,1}*|x begins with 0} {x{0,1}*|x ends with 0}

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

= {x {0,1}*| x begins or ends with 0}

Intersection
Let L1 and L2 be languages over an alphabet .
The intersection of L1 and L2, denoted by L1L2, is { x | x is in L1 and L2}.
Example:
{ x{0,1}*| x begins with 0} { x{0,1}*| x ends with 0}
= { x{0,1}*| x begins and ends with 0}

Concatenation
Let L1 and L2 be languages over an alphabet .
The concatenation of L1 and L2, denoted by L1L2, is {w1w2| w1 is in L1 and w2
is in L2}.
Example
{ x {0,1}*| x begins with 0}{x {0,1}*| x ends with 0}
= { x {0,1}*| x begins and ends with 0 and length(x) 2}
{ x {0,1}*| x ends with 0}{x {0,1}*| x begins with 0}
= { x {0,1}*| x has 00 as a substring}
Reversal
Let L be a language over an alphabet .
The reversal of L, denoted by Lr, is {wr| w is in L}.
Example
{x {0,1}*| x begins with 0} r
= {x {0,1}*| x ends with 0}
{x {0,1}*| x has 00 as a substring} r
= {x {0,1}*| x has 00 as a substring}

Closure
Let L be a language over an alphabet .
The closure of L, denoted by L+, is { x |for an integer n 1, x = x1x2xn and x1,
x2 , , xn are in L}

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

That is, L+ = i= 1 Li

Example:
Let = {0, 1} be the alphabet.
Le = {* | the number of 1s in is even}
Le+ = {* | the number of 1s in is even} = Le*
Observation about Closure
L+ = L* {} ?
Example:
L = {* | the number of 1s in is even}
L+ = {* | the number of 1s in is even} = Le*
Why?
L* = L+ {} ?

Languages and ProblemsProblem

Example: What are prime numbers > 20?

Decision problem
Problem with a YES/NO answer
Example: Given a positive integer n, is n a prime number > 20?

Language
Example: {n | n is a prime number > 20}

Finite Automata

A simple model of computation

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Deterministic finite automata (DFA)

How a DFA works
How to construct a DFA
Non-deterministic finite automata (NFA)
How an NFA works
How to construct an NFA
Equivalence of DFA and NFA
Closure properties of the class of languages accepted by FA

Finite Automata (FA)

Read an input string from tape

Determine if the input string is in a language
Determine if the answer for the problem is YES or NO for the given input on
the tape

How does an FA work?

At the beginning,
an FA is in the start state (initial state)
its tape head points at the first cell
For each move, FA
reads the symbol under its tape head
changes its state (according to the transition function) to the next state
determined by the symbol read from the tape and its current state
move its tape head to the right one cell
When does an FA stop working?
When it reads all symbols on the tape
Then, it gives an answer if the input is in the specific language:
Answer YES if its last state is a final state
Answer NO if its last state is not a final state

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

f
1

(q,a)

How to define a DFA

a 5-tuple (Q, , , s, F), where
a set of states Q is a finite set
an alphabet is a finite, non-empty set
a start state s in Q
a set of final states F contained in Q
a transition function is a function Q Q

See formal definition

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

How an FA works

Definition

Let M = (Q, , , s, F) be a DFA, and *. We say M accepts if (s, )

*M (f, ), when f F. Otherwise, we say M rejects .
(s, 001101) *M (f, ) M accepts 001101
(s, 01001) *M (s, ) M rejects 01001

Language accepted by a DFA

Let M = (Q, , , s, F ) be a DFA. The language accepted by M, denoted by
L(M ) is the set of strings accepted by M. That is, L(M) = {*|(s, ) *M (f, ) for
some f F }
Example:
L(M) = {x {0,1}* | the number of 1s in x is odd}.

f
1

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

How to construct a DFA

Determine what a DFA need to memorize in order to recognize strings in the

language.
Hint: the property of the strings in the language
Determine how many states are required to memorize what we want.
final state(s) memorizes the property of the strings in the language.
Find out how the thing we memorize is changed once the next input symbol is
read.
From this change, we get the transition function.

Constructing a DFA: Example

Consider L= {{0,1}*| has both 00 and 11 as substrings}.

Step 1: decide what a DFA need to memorize
Step 2: how many states do we need?
Step 3: construct the transition diagram

Constructing a DFA: Example

Consider L= {{0,1}*| represents a binary number divisible by 3}.

L = {0, 00, 11, 000, 011, 110, 0000, 0011, 0110, 1001, 00000, ...}.

Step 1: decide what a DFA need to memorize

remembering that the portion of the string that has been read so far is
divisible by 3

Step 2: how many states do we need?

2 states remembering that

the string that has been read is divisible by 3
the string that has been read is indivisible by 3.

3 states remembering that

the string that has been read is divisible by 3

the string that has been read - 1 is divisible by 3.
the string that has been read - 2 is divisible by 3.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Using 2 states

Reading a string w representing a number divisible by 3.

Next symbol is 0. w 0, which is 2*w, is also divisible by 3.
If w=9 is divisible by 3, so is 2*w=18.
Next symbol is 1. w 1, which is 2*w +1, may or may not be divisible by 3.
If 8 is indivisible by 3, so is 17.
If 4 is indivisible by 3, but 9 is divisible.
Using these two states is not sufficient.
Using 3 states
Each state remembers the remainder of the number divided by 3.
If the portion of the string that has been read so far, say w, represents the
number whose remainder is 0 (or, 1, or 2),
If the next symbol is 0, what is the remainder of w 0?
If the next symbol is 1, what is the remainder of w 1?

Current
number

Current
remainder

Next symbol New number New remainder

3n

6n

3n

6n+1

3n+1

6n+2

3n+1

6n+3

3n+2

6n+4

3n+2

6n+5

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

How to define an NFA

a 5-tuple (Q, , , s, F), where

a set of states Q is a finite set
an alphabet is a finite, non-empty set
a start state s in Q
a set of final states F contained in Q
a transition function is a function Q({})2Q
See formal definition

Definition

Let M = (Q, , , s, F) be a non-deterministic finite automaton, and (q0, 0) and

(q1, 1) be two configurations of M.
We say (q0, 0) yields (q1, 1) in one step, denoted by (q0, 0) M (q1, 1), if
q1 (q0, a,), and 0=a 1, for some a {}.

Definition

Let M = (Q, , , s, F) be an NFA, and (q0, 0) and (q1, 1) be two

configurations of M. (q0, 0) yields (q1, 1) in zero step or more, denoted by
(q0, 0) *M (q1, 1), if
q0= q1 and 0 = 1, or
(q0, 0) M (q2, 2) and (q2, 2) *M (q1, 1) for some q2 and 2.

Definition
Let M = (Q, , , s, F) be an NFA, and *. We say M accepts if (s, )
*M (f, ), when f F. Otherwise, we say M rejects .

Language accepted by an NFA

Let M = (Q, , , s, F) be an NFA.

The language accepted by M, denoted by L(M) is the set of strings accepted by M.
That is, L(M) = {*| (s,) *M (f, ) for some fF}

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

DFA and NFA are equivalent

Md and Mn are equivalent L(Md) = L(Mn).
DFA and NFA are equivalent
For any DFA Md, there exists an NFA Mn such that Md and Mn are equivalent.
(part 1)
For any NFA Mn, there exists a DFA Md such that Md and Mn are equivalent.
(part 2)

Part 1 of the equivalence proof

For any DFA Md, there exists an NFA Mn such that Md and Mn are equivalent

Proof: Let Md be any DFA. We want to construct an NFA Mn such that L(Mn) = L(Md).
From the definitions of DFA and NFA, if M is a DFA then it is also an NFA.
Then, we let Mn = Md.
Thus, L(Md) = L(Mn).

For any NFA Mn, there exists a DFA Md such that Md and Mn are equivalent.

Proof: Let Mn = (Q, , , s, F) be any NFA. We want to construct a DFA Md such that
L(Md) = L(Mn).
First define the closure of q, denoted by E(q).
Second, construct a DFA Md=(2Q, , ', E(s), F')
Finally, prove
f F (s, ) |-*Mn (f, ) f 'F ' (E(s), ) |*Md (f ' , ).

Closure of state q
Let M = (Q, , , s, F) be an NFA, and qQ.
The closure of q, denoted by E(q), is
the set of states which can be reached from q without reading any symbol.
{pQ| (q, ) |-M* (p, )}
If an NFA is in a state q, it can also be in any state in the closure of q without
reading any input symbol.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Example of closure
Constructing the equivalent DFA
Let Mn = (Q, , , s, F) be any NFA. We construct a DFA Md =(2Q, , ', E(s), F'),
where :
'(q',a) = {rE(p)| p (q,a) } and
F' = {f Q | f F })

E(q0)

E(q1)

E(q2)

E(q3)

E(q4)

q0, q1, q2, q3

q1, q2, q3

q2

q3

q3,q4

Prove property of and '

Let Mn = (Q, , , s, F) be any NFA, and Md = (2Q, , ', E(s), F') be a DFA, where
'(q', a) = {rE(p)| p(q,a)} and
F' = {f Q | f F }
Prove , fF (s, ) |-*Mn (f, ) f 'F ' (E(s), ) |-*Md (f', ) and ff' by
induction.
Prove a more general statement , p, qQ (p, ) |-*Mn (q, ) (E(p), ) |*Md (q', ) and qq'.

Proof
Part I:
For any string in *, and states q and r in Q, there exists R Q such that
(q, ) *Mn (r, ) (E(q), ) *Md (R, ) and rR.
Basis:
Let be a string in *, q and r be states in Q, and (q, ) *Mn (r, ) in 0 step.
Because (q, ) *Mn (r, ) in 0 step, we know (1) q=r , and (2) = .
Then, (E(q), ) = (E(r), ).
Thus, (E(q), ) *Md (E(r), ) .
That is, there exists R=E(r) such that r R and (E(q), ) *Md (R, ).
Induction hypothesis:
For any non-negative integer k, string in *, and states q and r in Q, there exists R
Q:
(q, ) *Mn (r, ) in k steps -> (E(q), ) *Md (R, ) and rR.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Induction step:
Prove, for any non-negative integer k, string in *, and states q and r in Q, there
exists R Q:
(q, ) *Mn (r, ) in k+1 steps -> (E(q), ) *Md (R, ) and rR.
Let be a string in *, q and r be states in Q, and (q, ) *Mn (r, ) in k+1 steps.
Because (q, ) *Mn (r, ) in k+1 steps and k0, there exists a state p in Q and a string
* such that (q, ) *Mn (p, a) in k steps and (p, a) Mn (r, ) for some a {}.
From the induction hypothesis and (q, ) *Mn (p, a) in k steps, we know that there
exists PQ such that (E(q), ) *Md (P, a) and pP.
Since (p, a) Mn (r, ), r(p, a).
From the definition of of Md, E((p, a)) (P, a) because pP.
Because r(p, a) and E((p, a)) (P, a), r(P, a).
Then, for R=(P, a), (P, a) *Md (R, ) and rR.
Thus, (E(q), ) *Md (P, a) *Md (R, ) and rR.

Part II:
For any string in *, and states q and r in Q, there exists R Q such that rR and
(E(q), ) *Md (R, ) -> (q, ) *Mn (r, ).
Proof
Basis:
Let be a string in *, q and r be states in Q, R be a subset of Q such that r R and
(E(q), ) *Md (R, ) in 0 step.
Because (E(q), ) *Md (R, ) in 0 step, E(q)=R and =.
From the definition of E, (q, )=R because E(q)=R.
Then, for any rR, (q, ) *Mn (r, ).
That is, there exists R=E(q) such that r R and (q, ) *Mn (r, ).
Induction hypothesis:
For any non-negative integer k, string in *, and states q and r in Q, there exists R
Q such that rR and:
(E(q), ) *Md(R, ) in k steps ->(q, ) *Mn(r, ).
Induction step:
Prove, for any non-negative integer k, string in *, and states q and r in Q, there
exists R Q such that rR:
(E(q), )*Md(R, ) in k+1 steps ->(q, ) *Mn(r, ).
Let be a string in *, q and r be states in Q, and (E(q), ) *Md (R, ) in k+1 steps.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Because (E(q), ) *Md (R, ) in k+1 steps and k0, there exists P2Q (i.e. PQ) and
a string * such that =a, (E(q), ) *Md (P,) in k steps and (P, a) Md (R, ) for
some a.
From the induction hypothesis and (E(q), ) *Md (P, ) in k steps, we know that there
exists pP such that (q, )*Mn(p,) (i.e. (q, a) *Mn (p, a) ).
Since (P, a) Md (R, ), there exists rR such that r= (p, a).
Then, for some rR, (p, a) *Mn (r, ).
Thus, (q, ) *Mn (p, a) *Mn (r, ) for some rR.
Closure Properties

The class of languages accepted by FAs is closed under the operations

Union
Concatenation
Complementation
Kleenes star
Intersection
The class of languages accepted by FA is closed under union.
Proof:
Let MA = (QA, , A, sA, FA) and
MB = (QB, , B, sB, FB) be any FA.
We construct an NFA M =
(Q, , , s, F) such that
Q = QA QB {s}
= A A {(s, , {sA, sB})}
F = FA FB
To prove L(M) = L(MA) L(MB), we prove:
I. For any string * L(MA) or L(MB) L(M) &
II. For any string * L(MA) and L(MB). L(M)
For I, consider (a) L(MA) or (b) L(MB).
For (a), let L(MA).
From the definition of strings accepted by an FA, there is a state fA in FA such
that (sA, ) |-*MA (fA, ).
Because A,
(sA, ) |-*M (fA, ) also.
Because sA(s,), (s, ) |-M (sA, ).
Thus, (s, ) |-M (sA, ) |-*M (fA, ).
Because fA F,
L(M).
Similarly for (b).
For (II), let L(MA)L(MB).
Because (s, , {sA, sB}), either (s, ) |-M (sA, ) or (s, ) |-M (sB, ) only.
Because L(MA), there exists no fA in FA such that (sA, ) |-*MA (fA,).
Because L(MB), there exists no fB in FB such that (sB, ) |-*MB (fB, ).
Since there is no transition between states in QA and QB in M, there exists no state f in
F=FAFB such that (s, ) |-M (sA, ) |-*M (fA, ) or (s, ) |-M (sB, ) |-*M (fB, ).

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

That is, L(M).

Thus, L(M) = L(MA)L(MB).

Closure under concatenation

The class of languages accepted by FA is closed under intersection.
Proof: Let L1 and L2 be languages accepted by FA.
L1 L2 = (L1 L2)
By the closure property under complementation, there are FA acceptingL1 andL2.
By the closure property under union, there is an FA acceptingL1 L2.
By the closure property under complementation, there is an FA accepting(L1 L2).
Thus, the class of languages accepted by FA is closed under intersection.
Let MA = (QA, , A, sA, FA) and
MB = (QB, , B, sB, FB) be any FA.
We construct an NFA M = (Q, , , s, F) such that
Q = QA QB
= A A (i.e. ((qA,qB),a) = A(qA,a)B(qB,a))
s = (sA, sB)
F = FA FB

Basic

Check list

Explain how DFA/NFA work (configuration, yield next configuration)

Find the language accepted by DFA/NFA
Construct DFA/NFA accepting a given language
Find closure of a state
Convert an NFA into a DFA
Prove a language accepted by FA
Construct FA from other FAs

Advanced
Prove DFA/NFA accepting a language
Prove properties of DFA/NFA
Configuration change
Under some modification
etc.
Prove some properties of languages accepted by DFA/NFA
Under some modification
Surprise!

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

UNIT: II : REGULAR EXPRESSIONS &LANGUAGES

Regular Languages

Regular expressions
Regular languages
Equivalence between languages accepted by FA and regular languages
Closure Properties

Regular Expressions

Regular expression over alphabet

is a regular expression.
is a regular expression.
For any a, a is a regular expression.
If r1 and r2 are regular expressions, then
(r1 + r2) is a regular expression.
(r1 r2) is a regular expression.
(r1* ) is a regular expression.
Nothing else is a regular expression.

is a regular language corresponding to the regular expression .

{} is a regular language corresponding to the regular expression .
For any symbol a, {a} is a regular language corresponding to the regular

expression a.
If L1 and L2 are regular languages corresponding to the regular expression r1
and r2, then
L1L2, L1L2, and L1* are regular languages corresponding to (r1 + r2)
, (r1 r2), and (r1*).

Simple examples

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Let = {0,1}.
{*| does not contain 1s}
(0*)
{*| contains 1s only}
(1(1*)) (which can can be denoted by (1+))
*
((0+1)*)
{*| contains only 0s or only 1s}
((00*)+(11*))

Some more notations

Let = {0,1}.
Parentheses in regular expressions can be omitted when the order of evaluation is
clear.
((0+1)*) 0+1*
((0*)+(1*)) = 0* + 1*
For concatenation, can be omitted.
r r r r is denoted by rn.
Let = {0,1}.

{*| contains odd number of 1s}

0*(10*10*)*10*
{*| any two 0s in are separated by three 1s}
1*(0111)*01* + 1*
{*| is a binary number divisible by 4}
(0+1)*00
{*| does not contain 11}
0*(10+)* (1+) or (0+10)* (1+)

Notation

Let r be a regular expression.

The regular language corresponding to the regular expression r is denoted by L(r).

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Some rules for language operations

Let r, s and t be languages over {0,1}

r+=+r=r
r+s=s+r
r = r = r
r = r =
r(s + t) = rs + rt
r+ = r r*

Rewrite rules for regular expressions

Let r, s and t be regular expressions over {0,1}.

* =
* =
(r + )+ = r*

r* = r*(r + ) = r* r* = (r*)*
(r*s*)* = (r + s)*

Closure properties of the class of regular languages (Part 1 )

Theorem: The class of regular languages is closed under union, concatenation, and
Kleenes star.

Proof: Let L1 and L2 be regular languages over .

Then, there are regular expressions r1 and r2 corresponding to L1 and L2.
By the definition of regular expression and regular languages, r1+r2 ,r1 r2, and r1* are
regular expressions corresponding to L1L2, L1L2, and L1*.
Thus, the class of regular languages is closed under union, concatenation, and Kleenes
star.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Equivalence of language accepted by FA and regular languages

To show that the languages accepted by FA and regular languages are equivalent, we
need to prove:
For any regular language L, there exists an FA M such that L = L(M).
For any FA M, L(M) is a regular language.

For any regular language L, there exists an FA M such that L = L(M)

Proof:

Let L be a regular language.

Then, a regular expression r corresponding to L.
We construct an NFA M, from the regular expression r, such that L=L(M).
Basis:
If r = , M is
If r = , M is
If r = {a} for some a , M is

Proof (contd)
Induction hypotheses: Let r1 and r2 be regular expressions with
less than n operations. And, there are NFAs M1 and M2 accepting
regular languages corresponding to L(r1) and L(r2).
Induction step: Let r be a regular expression with n operations.
We construct an NFA accepting L(r).

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

r can be in the form of either r1+r2, r1r2, or r1*, for regular

expressions r1 and r2 with less than n operations.
If r = r1+r2, then M is
If r = r1r2, then M is
If r = r1*, then M is
Therefore, there is an NFA accepting L(r) for any regular expression r.

s
1

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Constructing NFA for regular expressions

+ *

0 (10 )
(1+)

Can these two states be merged?

NO
Be careful when you decide to
merge some

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

For any FA M, L(M) is a regular language

Proof: Let M = (Q, , , q1, F) be an FA, where Q={qi| 1 i n} for some

positive integer n.
Let R(i, j, k) be the set of all strings in that drive M from state qi to
state qj while passing through any state ql , for l k. (i and j can be any
states)

ql
ql

qj

'
Proof (contd)
We prove that L(M) is a regular language by showing that there is a regular
expression corresponding to L(M), by induction.
Basis: R(i, j, 0) corresponds to a regular expression a if i j and a + if i= j
for some a.
Induction hypotheses: Let R(i, j,k-1) correspond to a regular expression, for
any i, j, k n.
Induction step: R(i, j, k) = R(i, j, k-1) R(i, k, k-1) R(k, k, k-1)* R(k, j, k-1)
also corresponds to a regular expression because R(i, j, k-1), R(i, k, k-1), R(k,
k, k-1) and R(k, j, k-1) correspond to some regular expressions and union,
concatenation, and Kleenes star are allowed in regular expressions.
Therefore, L(M) is also a regular language because L(M) = + R(1, f, n) for all
qf in F.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Pumping Lemma
Let L be a regular language.
Then, there exists an integer n0 such that for every string x in L that |x|n, there are
strings u, v, and w such that
x = u v w,
v ,
|u v| n, and
for all k 0, u vk w is also in L
Any language L is not a regular language if for any integer n0 , there is a string x in L
such that |x|n, for any strings u, v, and w,
x u v w, or
v = , or
Not (|u v| n), or
there is k 0, u vk w is not in L

Any language L is not a regular language if

for any integer n0 ,
there is a string x in L such that |x|n,
for any strings u, v and w, such that x = u v w, v , and |u v| n,
there is k 0, u vk w is not in L

Given a language L.
Let n be any integer 0 .
Choose a string x in L that |x|n.
Consider all possible ways to chop x into u, v and w such that v , and |uv| n.
For all possible u, v, and w, show that there is k 0 such that u vk w is not in L.
Then, we can conclude that L is not regular.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Prove {0i 1i| i 0} is not regular

Let L = {0i1i| i 0}.
Let n be any integer 0.
Let x = 0n 1n.
Make sure that x is in L and |x|n.
The only possible way to chop x into u, v, and w such that v, and |u v| n is:
u = 0p, v = 0q, w = 0n-p-q 1n, where 0p<n and 0<qn
Show that there is k 0, u vk w is not in L.
u vk w = 0p 0qk 0n-p-q 1n = 0p+qk+(n-p-q) 1n = 0n+q(k-1) 1n
If k 1, then n+q(k-1) n and u vk w is not in L.
Then, L is not regular.
Let L = {0i1i| i 0}.
Let n be any integer 0, and m= n/2.
Let x = 0m 1m.
Make sure that x is in L and |x|n.
Possible ways to chop x into u, v, and w such that v , and |u v| n are:
u = 0p, v = 0q, w = 0m-p-q 1m, where 0p<m and 0<qm
u = 0p, v = 0 m-p 1q, w = 1m-q, where 0p<m and 0<qm
u = 0 m 1p, v = 1q, w = 1m-p-q, where 0p<m and 0<qm

Show that there is k 0, u vk w is not in L.

u=0p, v=0q, w= 0m-p-q 1m, where where 0p<m and 0<qm

u vk w = 0p 0qk 0m-p-q 1m = 0m+q(k-1)1m is not in L if

k1.

u=0p, v=0m-p 1q, w=1m-q, where where 0p<m and 0<qm

u vk w = 0p (0m-p 1q)k 1m-q is not in L if k 1.
u=0m 1p, v=1q, w=1m-p-q, where where 0p<m and 0<qm
u vk w = 0m 1p 1qk 1m-p-q = 0m 1m+q(k-1) is not in L if

k1.
Then, L is not regular.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Prove {1i|i is prime} is not regular

Let L = {1i| i is prime}.

Let n be any integer 0.
Let p be a prime n, and w = 1p.
Only one possible way to chop w into x, y, and z such that y , and |x y| n is:
x = 1q, y = 1r, z = 1p-q-r, where 0q<n and 0<r<n
Show that there is k 0, x yk z is not in L.
x yk z = 1q 1rk 1p-q-r = 1q+rk+(p-q-r) = 1p+r(k-1)
If k=p+1, then p+r(k-1) = p(r+1), which is not a prime.
Then, x yk z is not in L.
Then, L is not regular.

Using closure property

Let be a binary operation on languages and the class of regular languages is closed
under . ( can be , , or -)
If L1 and L2 are regular, then L1L2 is regular.
If L1L2 is not regular, then L1 or L2 are not regular.
If L1L2 is not regular but L2 is regular, then L1 is not regular.
Let L={w{0,1}*| the number of 0s and 1s in w are equal}.
Let R= {0i1i| i 0}.
R = 0*1* L
We already prove that R is not regular.
But 0*1* is regular.
Then, L is not regular.
Let be a unary operation on a language and the class of regular languages is closed
under .
( can be complement or *)
If L is regular, then L is regular.
If L is not regular, then L is not regular.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Prove that {w{0,1}*| the number of 0s and 1s in w are not

equal} is not regular
Let L = {w{0,1}*| the number of 0s and 1s in w are not equal}.
Let R =L = {w{0,1}*| the number of 0s and 1s in w are equal}.
We already prove that R is not regular.
Then, L is not regular.

Check list
Find the language described by a regular exp.
Construct regular exp. describing a given language
Convert a regular exp. into an FA
Convert an FA into a regular exp.
Prove a language is regular
By constructing a regular exp.
By constructing an FA
By using closure properties
Construct an FA or a regular exp. for the intersection,
union, concatenation, complementation, and Kleenes star
of regular languages
Prove other closure properties of the class of regular lang

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

UNIT-III
CONTEXT FREE GRAMMAR AND LANGUAGES
Pushdown automata

Pushdown automata differ from finite state machines in two ways:

1. They can use the top of the stack to decide which transition to take.
2. They can manipulate the stack as part of performing a transition.
Pushdown automata choose a transition by indexing a table by input signal, current state, and the symbol at
the top of the stack. This means that those three parameters completely determine the transition path that is
chosen. Finite state machines just look at the input signal and the current state: they have no stack to work
with. Pushdown automata add the stack as a parameter for choice.
Pushdown automata can also manipulate the stack, as part of performing a transition. Finite state machines
choose a new state, the result of following the transition. The manipulation can be to push a particular
symbol to the top of the stack, or to pop off the top of the stack. The automaton can alternatively ignore the
stack, and leave it as it is. The choice of manipulation (or no manipulation) is determined by the tr ansition

table.
Put together: Given an input signal, current state, and stack symbol, the automaton can
follow a transition to another state, and optionally manipulate (push or pop) the stack.
In general pushdown automata may have several computations on a given input string,
some of which may be halting in accepting configurations while others are not. Thus we
have a model which is technically known as a "nondeterministic pushdown automaton"
(NPDA). Nondeterminism means that there may be more than just one transition
available to follow, given an input signal, state, and stack symbol. If in every situation
only one transition is available as continuation of the computation, then the result is a
deterministic pushdown automaton (DPDA), a strictly weaker device.
If we allow a finite automaton access to two stacks instead of just one, we obtain a more
powerful device, equivalent in power to a Turing machine. A linear bounded automaton
is a device which is more powerful than a pushdown automaton but less so than a Turing
machine.
Pushdown automata are equivalent to context-free grammars: for every context-free
grammar, there exists a pushdown automaton such that the language generated by the
grammar is identical with the language generated by the automaton, which is easy to
prove. The reverse is true, though harder to prove: for every pushdown automaton there
exists a context-free grammar such that the language generated by the automaton is
identical with the language generated by the grammar.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Formal Definition
A PDA is formally defined as a 7-tuple:
where

is a finite set of states

is a finite set which is called the input alphabet
is a finite set which is called the stack alphabet
*
is a mapping of into , the transition relation, where means "a finite (maybe
empty) list of element of " and denotes the empty string.
is the start state
is the initial stack symbol
is the set of accepting states

An element is a transition of M. It has the intended meaning that M, in state , with on the
input and with as topmost stack symbol, may read a, change the state to q, pop A,
replacing it by pushing . The letter (epsilon) denotes the empty string and the
component of the transition relation is used to formalize that the PDA can either read a
letter from the input, or proceed leaving the input untouched.
In many texts the transition relation is replaced by an (equivalent) formalization, where

is the transition function, mapping into finite subsets of .

Here (p,a,A) contains all possible actions in state p with A on the stack, while reading
a on the input. One writes for the function precisely when for the relation. Note that finite
in this definition is essential.
Computations

a step of the pushdown automaton

In order to formalize the semantics of the pushdown automaton a description of the
current situation is introduced. Any 3-tuple is called an instantaneous description (ID) of
M, which includes the current state, the part of the input tape that has not been read, and
the contents of the stack (topmost symbol written first). The transition relation defines
the step-relation of M on instantaneous descriptions. For instruction there exists a step ,
for every and every .
In general pushdown automata are nondeterministic meaning that in a given
instantaneous description (p,w,) there may be several possible steps. Any of these steps

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

can be chosen in a computation. With the above definition in each step always a single
symbol (top of the stack) is popped, replacing it with as many symbols as necessary. As a
consequence no step is defined when the stack is empty.
Computations of the pushdown automaton are sequences of steps. The computation starts
in the initial state q0 with the initial stack symbol Z on the stack, and a string w on the
input tape, thus with initial description (q0,w,Z). There are two modes of accepting. The
pushdown automaton either accepts by final state, which means after reading its input the
automaton reaches an accepting state (in F), or it accepts by empty stack (), which means
after reading its input the automaton empties its stack. The first acceptance mode uses the
internal memory (state), the second the external memory (stack).
Formally one defines
1. with and (final state)
2. with (empty stack)
Here represents the reflexive and transitive closure of the step relation meaning any
number of consecutive steps (zero, one or more).
For each single pushdown automaton these two languages need to have no relation: they
may be equal but usually this is not the case. A specification of the automaton should also
include the intended mode of acceptance. Taken over all pushdown automata both
acceptance conditions define the same family of languages.
Theorem. For each pushdown automaton M one may construct a pushdown automaton
M' such that L(M) = N(M'), and vice versa, for each pushdown automaton M one may
construct a pushdown automaton M' such that N(M) = L(M')
The following is the formal description of the PDA which recognizes the language by
final state:

PDA for (by final state)

, where

Q = {p,q,r}
= {0,1}
= {A,Z}

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

F = {r}
consists of the following six instructions:
(p,0,Z,p,AZ), (p,0,A,p,AA), (p,,Z,q,Z), (p,,A,q,A), (q,1,A,q,), and
(q,,Z,r,Z).
In words, in state p for each symbol 0 read, one A is pushed onto the stack. Pushing
symbol A on top of another A is formalized as replacing top A by AA. In state q for each
symbol 1 read one A is popped. At any moment the automaton may move from state p to
state q, while it may move from state q to accepting state r only when the stack consists
of a single Z.
There seems to be no generally used representation for PDA. Here we have depicted the
instruction (p,a,A,q,) by an edge from state p to state q labelled by a;A / (read a;
replace A by ).

Understanding the computation process

accepting computation for 0011

The following illustrates how the above PDA computes on different input strings. The
subscript M from the step symbol is here omitted.
(a) Input string = 0011. There are various computations, depending on the moment the
move from state p to state q is made. Only one of these is accepting.
(i) . The final state is accepting, but the input is not accepted this way as it has not
been read.
(ii) . No further steps possible.
(iii) . Accepting computation: ends in accepting state, while complete input has
been read.
(b) Input string = 00111. Again there are various computations. None of these is
accepting.
(i) . The final state is accepting, but the input is not accepted this way as it has not
been read.
(ii) . No further steps possible.
(iii) . The final state is accepting, but the input is not accepted this way as it has
not been (completely) read.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Pushdown Automata
As Fig. 5.1 indicates, a pushdown automaton consists of three components: 1) an input
tape, 2) a control unit and 3) a stack structure. The input tape consists of a linear
configuration of cells each of which contains a character from an alphabet. This tape can
be moved one cell at a time to the left. The stack is also a sequential structure that has a
first element and grows in either direction from the other end. Contrary to the tape head
associated with the input tape, the head positioned over the current stack element can
read and write special stack characters from that position. The current stack element is
always the top element of the stack, hence the name ``stack''. The control unit contains
both tape heads and finds itself at any moment in a particular state.

Figure 5.1: Conceptual Model of a Pushdown Automaton

Definition

A (non-deterministic) finite state pushdown automaton (abbreviated PDA or, when the
context is clear, an automaton) is a 7-tuple = (X, Z, , R, zA, SA, ZF), where

X = {x1, ... , xm} is a finite set of input symbols. As above, it is also called an
alphabet. The empty symbol is not a member of this set. It does, however, carry
its usual meaning when encountered in the input.
Z = {z1, ... zn} is a finite set of states.
= {s1, ... , sp} is a finite set of stack symbols. In this case

R ((X { })Z )(Z

zA is the initial state.
SA is the initial stack symbol.

ZF

)) is the transition relation.

K is a distinguished set of final states.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Figure 5.3: Derivation of the String a3bc3

Context-Free Languages
As will be recalled from the last chapter there were two basic ways to determine whether
a given string belongs to the language generated by some finite state automaton: One
could verify that the string brings the automaton to a final state or one could derive, or,
better, produce, the string in the regular grammar corresponding to the automaton. The
same option holds for PDAs.
Definition

A context-free grammar is a grammar

productions, in R have the special form A

B.VIJAYAKUMAR B.E. M.Tech (PhD)

= (X, T, S, R) for which all rules, or

, for A

X - T and

X*.

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Additionally, for any two strings u, v

only if (1) u = u1Au2 for u1, u2

X* write u

X* and A

...

v (u directly produces v ) if and

X - T and (2) v = v1 v2 and A

X*, is a production from R. The reduction u

Finally, write u
u1
u2

Einstein College of Engineering

v is also called a direct production.

v for two strings u, v X* (u derives v) if there is a sequence u = u0

un = v of direct productions ui
ui+1 from R. The length of the

derivation is n. The language generated by

is {x

T*| S

x}.

Thus, the definition just articulates the reduction of A to in any context in which A
occurs. It is trivial that every regular language is context-free. The obverse, as will be
seen presently, is not true. Before proving the central theorem for this section two typical
examples are given.
Example 1

Consider

= (X, T, R, S) with T = {a, b} and X = {S, a, b, }. The productions, or

grammar rules, are: S

aSb | . Then it is clear that L( ) = {anbn| n
the previous chapter it is known that this language is not regular.

0}. From

Example 2: A Grammar for Arithmetic Expressions

Let

X = {E, T, F, id, + , - ,*,/,(,), a, b, c}

and T = {a, b, c, + , - ,*,/,(,)}. The start symbol S is E and the productions are as
follows:
E

E+T | E-T | T

T*F | T/F | F

(E) | id

id

a | b | c

Then the string (a + b)*c belongs to L( ). Indeed, it is easy to write down a derivation
of this string:
E

T*F

F*F

(E)*F

(E + T)*F

(T + T)*F

(F + T)*F

(id + T)*F

(a + T)*F

(a + F)*F

(a + id )*F

(a + b)*F

(a + b)*id

B.VIJAYAKUMAR B.E. M.Tech (PhD)

(a + b)*c

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

The derivation just adduced is leftmost in the sense that the leftmost nonterminal was
always substituted. Although derivations are in general by no means unique, the leftmost
one is. The entire derivation can also be nicely represented in a tree form, as Fig. 5.4
suggests.

Figure 5.4: Derivation Tree for the Expression (a + b)*c

The internal nodes of the derivation, or syntax, tree are nonterminal symbols and the
frontier of the tree consists of terminal symbols. The start symbol is the root and the
derived symbols are nodes. The order of the tree is the maximal number of successor
nodes for any given node. In this case, the tree has order 3. Finally, the height of the tree
is the length of the longest path from the root to a leaf node, i.e. a node that has no
successor. The string (a + b)*c obtained from the concatenation of the leaf nodes
together from left to right is called the yield of the tree.
The expected relation between pushdown automata and context-free languages is
enunciated in the following theorem.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

B.VIJAYAKUMAR B.E. M.Tech (PhD)

Einstein College of Engineering

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Figure 5.5: Derivation of the String a2bcba2

Conversely, assume is a PDA. To clarify the subsequent definitions the following
discussion on the internal operation of is offered. The goal is, of course, to concoct a
context-free grammar that executes a leftmost derivation of every string that accepts. If
were as simple as the example in the first part of this proof, namely, that after pushing
the very first nontrivial symbol (not SA) onto the stack remains in a single state z1,
then it would be very straightforward to reverse the above process and construct from
. Basically, if x is the input string write x = x ax , where x is that part of x that has
already been processed (a so-called prefix of x) and ax is the rest of x whose first input
symbol is a. Then the direct production of configurations of of the form (ax , z1,
AA )
(x , z1, A ) corresponds to the grammar rule A
a , resulting in the
reduction x AA
xa A . Thus the sequence of stack moves from the abovementioned example commences with SA and, after popping that symbol, derives the
string a2bcba2, as can be seen by inspecting the stack column in Fig. 5.5.
Unfortunately, the general case is considerably more complicated, because 's state
transitions also enter into the picture. Proceeding naively, one could reduce to a 2 state
PDA
of the aforementioned type by pushing pairs (z, A) of states and stack symbols
from onto
's stack, thus imitating 's calculation of input strings. Thus, when
is in state z and pushes A onto the stack,
pushes (z, A) onto its stack. The reader is
invited to pause to discover the fatal shortcoming of this method before reading further.
The problem becomes immediately transparent when one considers what happens when
pops a stack element (z, A). State z is no longer relevant for
's further operationwas in state z when A got pushed, but what state was in when the pop occurred?
Therefore, it is necessary to push triples (z, A, z ), where z is 's state when the pop
takes place. Since it is not known what 's state z is going to be when it pops A, has
to guess what it is going to be, .i.e. it nondeterministically pushes (z, A, z ), where z
Z is arbitrary. The only restriction is that when executing two (or more) push operations
the unknown state z must be manipulated consistently. This means if A1A2 is pushed,
then after pops A1, or, equivalently,
pops (z1, A1, z1 ), then finds itself in state
z1 . Since
does not use its own state information in imitating 's state transitions,
's current state must be available in describing the next element of
's stack, or, in
other words, better be in state z1 when popping A2 from its stack, and so must be of
the form (z1 , A2, z2) for some (predicted) z2
formalized.

Z. This train of thought will now be

For simplicity, assume that pushes at most two symbols and that it has a single
acceptance statei zF. A moment's reflection shows that these assumptions are not
restrictive; but they do eliminate some extra preprocessing. The nonterminals of G are
triples (z, A, z ) of states z, z and a stack symbol A. The basic idea is to imitate what

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

the machine undergoes in state z finally to pop symbol A and to wind up thereby in state
z , having processed some string of input characters. Thus the rules for the sought-after
context-free grammar are posited as follows:
1. For the (extra) start symbol put S

(zA, SA, zF).

2. For each transition ((a, z, B),(z , C))

R put for each z1

(z, B, z1)

a(z , C, z1)

3. In case two symbols are pushed, i.e. ((a, z, B),(z , C1C2))

pair z1, z2

R, then put for each

Z
(z, B, z1)

4. For each z

Z put (z, , z)

a(z , C1, z2)(z2, C2, z1).

.

It is important to notice the free choice of z1 and z1, z2 in rules 2. and 3. Consider, for
example, processing the string a2bc2 from the PDA from Section 5.1. Then posit the start
rule
S
(z1, SA, z3),
since there is only one final state. Now mechanically translate each of the transitions
from this PDA into their grammatical equivalents as shown in Table 5.1.
Table 5.1: Translation of the PDA Transition Rules into Grammatical Productions
Nr. Transition Function

Nr Production

((a, zA, SA),(zA, SSA)) 1'

(zA, SA, z')

a(zA, S, z'')(z'', SA, z')

((a, zA, S),(zA, SS))

2'

(zA, S, w')

a(zA, s, w'')(w'', s, w')

((b, zA, S),(z2, ))

3'

(zA, S, v')

b(z2, , v')

((c, z2, S),(z2, ))

4'

(z2, S, u')

c(z2, , u')

((c, z2, SA),(z3, ))

5'

z2, SA, t')

c(z3, , t')

It is important to note that states z', z'', w', w'', v', u', t' can be chosen at will.
Hopefully, a proper choice will lead to success in accordance with the philosophy of
nondeterminism.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Properties of Context-Free Langauges

Syntax Trees

Tree representations of derivations, also known as syntax trees, were briefly introduced in
the preceding section to promote intuition of derivations. Since these are such important
tools for the investigation of context-free languages, they will be dealt with a little more
systematically here.
Definition

Let = (X, T, R, S) be a context-free grammar. A syntax tree for this grammar consists
of one of the following
1. A single node x for an x
2. An edge

T. This x is both root and leaf node.

corresponding to a production A

R.

3. A tree

where the A1, A2, ... , An are the root nodes of syntax trees. Their yields are read
from left to right.
Ambiguity

Until now the syntax trees were uniquely determined-even if the sequence of direct
derivations were not. Separating the productions corresponding to the operator hierarchy,
from weakest to strongest, in the expression grammar + , - ,*,/,() preserves this natural
hierarchy. If this is not done, then syntax trees with a false evualation sequence are often
the result. Suppose, for instance, that the rules of the expression grammar were written E
E + E | E*E | id, then two different syntax trees are the result. If the first
production E
E + E were chosen then the result would be the tree

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

On the other hand, choosing the production E

entirely different ilk.

Einstein College of Engineering

E*E first results in a syntax tree of an

Thus this grammar is ambiguous, because it is possible to generate two different syntax
trees for the expression a + b*c.
Chomsky Normal Form

Work with a given context-free grammar is greatly facilitated by putting it into a socalled normal form. This provides some kind of regularity in the appearance of the righthand sides of grammar rules. One of the most important normal forms is the Chomsky
normal form.
Definition

The context-free Grammar = (X, T, R, S) is said to be in Chomsky normal form if

all grammar rules have the form

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

for a
rule

T and B, C

Einstein College of Engineering

(5.1)

a | BC,

X - T. There is one exception. If

L(

), then the single extra

(5.2)

is permitted. If
1. vy

L(

) then production rule 5.2 is not allowed.

(that is, v

or y

).

2. The length of vwy satisfies | vwy|

3. For each integer k

n.

0, it follows that uvkwykz

L(

).

Proof

Assume that is in Chomsky normal form. For x L( ) consider the (binary) syntax
tree for the derivation of x. Assume the height of this tree is h as illustrated in Fig. 5.6.

Figure 5.6: Derivation Tree for the string x

Then it follows that | x|
most 2h-1. If

L(

2h-2 + 2h-2 = 2h-1, i.e. the yield of the tree with height h is at

has k nonterminal symbols, let n = 2k. Then let x

L(

) be a string

with | x| n. Thus the syntax tree for x has height at least k + 1, thus on the path from
the root downwards that defines the height of the tree there are at least k + 2 nodes, i.e. at

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

least k + 1 nonterminal symbols. It then follows that there is some nonterminal symbol A
that appears at least twice. Consulting Fig. 5.7, it is seen that the partial derivation S
uAz

uvAyz obtains.

Figure 5.7: Nonterminal A appears twice in the derivation of x

If, now, both u and z were empty, then derivations of the form S
uAz
A would be
possible, contrary to the assumption of Chomsky normal form. For the same reason either
v or y are nonempty. If | vwy| > n then apply the procedure anew until the condition |
vwy|

n holds. Finally, since the derivation A

pleases, it follows that S

uAz
generated. This completes the proof.

uvAyz

vAy can be repeated as often as one

uv2Ay2z

uv2wy2zi, etc. can be

Example 1

The language L = {aibici | i

1} is not context free.

Proof

Assume L were context-free. Then let n be the n from the preceding theorem and put x =
anbncn. Ogden's lemma then provides the decomposition x = uvwyz with the stated
properties. There are several cases to consider.
Case 1 The string vy contains only a's. But then the string uwz L, which is
impossible, because it contains fewer a's than b's and c's. .
Case 2,3 vy contains only b's or c's. This case is similar to case 1.
Case 4,5 vy contains only a's and b's or only b's or c's. Then it follows that uwz contains

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

more c's than a's and b's or more a's than b's and c's. This is again a contradiction.
Since | vwy|

n it is not possible that vy contain a's and c's.

it is seen that the complements

and

are not in general context-free.

Push Down Automata and Context-Free Grammars

Definition

An algorithm is called polynomial in case there is an integer k 2 such that the number
of steps after which the algorithm halts is (nk). The argument n depends only on the
input.
Theorem 5..7 There is a polynomial algorithm that constructs to any given push down
automaton

with L(

a context-free grammar

) = L(

). Conversely, there is a

polynomial algorithm that constructs to any given context-free grammar

a push down

with L( ) = L( ).
Theorem 5..8 There is a polynomial algorithm that decides, given any context-free
automaton

grammar G = (X, T, R, S) and x

T* whether x

L(

).

Proof

The proof of this theorem sometimes goes under the name CYK algorithm after their
discoverers Cocke, Younger and Kasami. It proceeds as follows:
1. Rewrite in Chomsky normal form. It is easily seen that this can be done in
polynomial time.
2. If x = x1x2 ... xn, then for 0

i, j

n put xij = xixi+1 ... xi+j-1. It is noteworthy that

| xij| = j. The idea is to determine all A

Vij = {A

X - T for which A

X-T | A

1. For j = 1 it is readily seen that Vi1 = {A

2. For general j it is also seen that A
A
BC is a rule from R and B
some k = 1, 2, ... j - 1.

Vij

xij. Thus set

xij}.
X-T | A
A

xi}.

xixi+1 ... xi+j-1

xi ... xi+k-1 and C

xi+k ... xi+j-1 for

Thus the algorithm can be formulated as follows:

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

for i : = 1 to n do

Vi1 : = {A

X-T | A

xi

R};

for j : = 2 to n do
for i : = 1 to n - j + 1 do begin

Vij : =

for k : = 1 to j - 1 do

Vij : = Vij

{A

X-T | A

BC, B

Vik, C

Vi+k, j-k};

end

Figure 5.8: Diagonal Procedure for CYK Algorithm

There is a nice interpretation of the innermost for loop. Formally one processes the pairs
Vi1Vi+1, j-1, Vi2Vi+2, j-2, ... , Vi, j-1Vi+j-1, 1. As evidenced in Fig. 5.8 go down the ith column
and simultaneously traverse the diagonal from Vi+1, j-1 up and to the right. The
corresponding elements are compared with each other.
Finally, it is seen that x
length(x).

L(

V1, n, because then S

x1 ... xn, where n =

This technique of producing increasingly larger solutions from smaller ones is called
dynamic programming.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Example

Consider the Grammar

S

AB | BC

BA | a

CC | b

AB | a

and the string x = baaba with n = 5. Then proceeding as above, the following triangular
matrix results:
b
a
a
b a
B
A, C A, C B A, C
S, A B
S, C S, A
B

S, A, C
S, A, C
Since S V15 it follows that x L( ). It is quite remarkable that the algorithm time is
(n3). It is also remarkable that the CYK algorithm actually shows how to construct the
derivation, which has great practical importance.
Then it is easy to derive the string abc:

aBC

abC

abc

Similarly, one derives the string a2b2c2:

aSBC

a2BCBC

a2B2C2

a2bBC2

a2b2C2

a2b2cC

a2b2c2

a2b2C2

a2b2cC

a2b2c2

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

It is then a routine application of mathematical induction to prove the general formula S

anbncn.
E + b*c

T + b*c

F + b*c

id + b*c

a + b*c.

At each stage of the derivation the sentential form of the stage is of the form uv, where u
X* and v T*. Tracing this derivation backwards, now proceed as follows: Starting
from the leftmost input symbol reduce that symbol to a rule for which it is the right-hand
side, in this case id
a. Then reduce id to F, etc. until an E has been produced. All of
the previous symbols are handles or right-hand sides of rules that allow successful (in the
sense that the start symbol will eventually be produced). After E has been obtained, the
next input symbol `+' is kept, or better, appended to E. Thus the sentential form `E +' is
produced. This sentential form is called a viable prefix because there is a rule of the form
E
E + T (a trivial one). If it recognized that E + is a viable prefix, then, starting with
the next input symbol, continue this process from that point onwards until the rest of the
right-hand side has been produced, i.e. a handle has been found. Then reduce this handle
to the left-hand side of the ``correct'' rule until the start symbol alone has been produced.
This process can be nicely realized using a push-down automaton. Thus, proceeding from
left to right on the input string, shift or push one or more input symbols onto the stack
until a handle is found. The reduce or pop that handle from the stack and push the lefthand side of the associated rule onto the stack. On a successful parse, if no reduction is
presently forthcoming then the contents of the stack constitute a viable prefix for some
rule yet to be determined. Another way of saying the same thing is that the contents of the
stack, read from bottom up, are the prefix of a sentential form produced on the way back
to the start symbol during a rightmost derivation.
A correct parse of the string a + b*c as a sequence of shift/reduce actions is given in
Table 5.3. Notice the decision to handle multiplication before addition is governed by
``looking ahead'' one symbol.
Table 5.3: Predictive Parse of the expression a + b*c

B.VIJAYAKUMAR B.E. M.Tech (PhD)

Stack

Input

Action

a + b*c$ Shift

id$

+ b*c$ Reduce

F$

+ b*c$ Reduce

T$

+ b*c$ Reduce

E$

+ b*c$ Reduce

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

+ E$

b*c$ Shift

b + E$

*c$ Shift

id + E$

*c$ Reduce

F + E$

*c$ Reduce

T + E$

*c$ Reduce

*T + E$

c$ Shift

c*T + E$

$ Reduce

id*T + E$

$ Reduce

F*T + E$

$ Reduce

T + E$

$ Reduce

E$

$ Accept

Stack

Input

a + b*c$ Shift

Action

id$

+ b*c$ Reduce

F$

+ b*c$ Reduce

T$

+ b*c$ Reduce

E$

+ b*c$ Reduce

+ E$

B.VIJAYAKUMAR B.E. M.Tech (PhD)

Einstein College of Engineering

b*c$ Shift

b + E$

*c$ Shift

id + E$

*c$ Reduce

F + E$

*c$ Reduce

T + E$

*c$ Reduce

*T + E$

c$ Shift

c*T + E$

$ Reduce

id*T + E$

$ Reduce

F*T + E$

$ Reduce

T + E$

$ Reduce

E$

$ Accept

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

UNIT-IV
PROPERTIES OF CONTEXT FREE LANGUAGES

Turing Machines (TM)

Structure of Turing machines

Deterministic Turing machines (DTM)
Accepting a language
Computing a function
Composite Turing machines
Multitape Turing machines
Nondeterministic Turing machines (NTM)
Universal Turing machines (UTM)

Determine if an input x is in a Determine if an input x is in a language.

That is, answer if the answer of a problem P for the instance x is yes.
Compute a function
Given an input x, what is f(x)?
language.
That is, answer if the answer of a problem P for the instance x is yes.
Compute a function
Given an input x, what is f(x)?

How does a TM work?

At the beginning,
A TM is in the start state (initial state)
its tape head points at the first cell
The tape contains , following by input string, and the rest of the tape
contains .
For each move, a TM
reads the symbol under its tape head

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

According to the transition function on the symbol read from the tape and
its current state, the TM:
write a symbol on the tape
move its tape head to the left or right one cell or not
changes its state to the next state

When does a TM stop working?

A TM stops working,
when it gets into the special state called halt state. (halts)
The output of the TM is on the tape.
when the tape head is on the leftmost cell and is moved to the left. (hangs)
when there is no next state. (hangs)

How to define deterministic TM (DTM)

a quintuple (Q, , , , s), where

the set of states Q is finite, not containing halt state h,
the input alphabet is a finite set of symbols not including the blank
symbol ,
the tape alphabet is a finite set of symbols containing , but not
including the blank symbol ,
the start state s is in Q, and
the transition function is a partial function from Q ({}) Q{h}
({}) {L, R, S}.

Example of a DTM
Definition
Let T = (Q, , , , s) be a DTM, and (q1, 1a11) and (q2, 2a22) be two
configurations of T.
We say
(q1, 1a11) yields (q2, 2a22) in one step,
denoted by (q1, 1a11) T (q2, 2a22), if
(q1, a1) = (q2,a2,s), 1=2 and 1=2,
(q1, a1) = (q2,b,R), 2=1b and 1=a22,

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

(q1, a1) = (q2,b,L), 1=2a2 and 2=b

Definition
Let T=(Q, , , , s) be a DTM, and (q1, 1a11) and (q2, 2a22) be two
configurations of T.
We say
(q1, 1a11) yields (q2, 1a22) in zero step or
more, denoted by (q1, 1a11) -*T (q2, 1a22), if
q1=q2, 1 =2, a1= a2, and 1= 2, or
(q1,1a11)-T (q, a) and (q, a)-*T (q2,1a22) for some q in Q,
and in *, and a in .

Yield in zero step or more: Example

s,0001000)
(p1,@0001000)
(p2,@001000)
(p2,@001000)
(p3,@001000)
(p4,@00100)
(p4,@00100)
(p1,@00100)
(p2,@0100) (p4,@010)
(p4,@010)
(p1,@010)
(p2,@10)
(p2,@10)
(p2,@10)
(p3,@10)
B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

(p4,@1)
(p4,@1)
(p1,@1)
(q1,@)
(q1,@)
(q2,)
(h ,1)
(p2,@0100)
(p3,@0100)

TM accepting a language
Definition
Let T=(Q, , , , s) be a TM, and w*.
T accepts w if (s, , , w) |-T* (h, , , 1).
The language accepted by a TM T, denoted by L(T), is the
set of strings accepted by T.

L(T)={0n10n | n0}
T halts on 0n10n
T hangs on 0n+110n at p3
T hangs on 0n10n+1 at q1
T hangs on 0n 12 0n at q1

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

TM computing a function

Definition

Let T=(Q, , , , s) be a TM, and f be a function from * to *.T computes f if, for any
string w in *,

Example of TM Computing Function

0/0,L
1/1,L
0/0,L
1/1,L

q2

/0,L

@/0,R

/,R

r2

Jaruloj Chongstitvatana

q1

0/0,R

p1

h
0/0,R
1/1,R

0/@,R

@/1,R

0/0,L
1/1,L
0/0,L
1/1,L

0/0,R
1/1,R

1/1,R

p2

1/@,R

/1,L

2301379

r1

/,S
/,L

p3
1/1,L
0/0,L

0/0,R
1/1,R

Turing Machines

19

Let T1 and T2 be TMs.

T1 T2 means executing T1 until T1 halts and then executing T2.
T1 -a T2 means executing T1 until T1 halts and if the symbol under the tape
head when T1 halts is a then executing T2.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Nondeterministic TM

An NTM starts working and stops working in the same way as a DTM.
Each move of an NTM can be nondeterministic.
Each Move in an NTM

reads the symbol under its tape head

According to the transition relation on the symbol read from the tape and its
current state, the TM choose one move nondeterministically to:
write a symbol on the tape
move its tape head to the left or right one cell or not
changes its state to the next state

How to define nondeterministic TM (NTM)

a quintuple (Q, , , , s), where

the set of states Q is finite, and does not contain halt state h,
the input alphabet is a finite set of symbols, not including the blank
symbol ,
the tape alphabet is a finite set of symbols containing , but not
including the blank symbol ,
the start state s is in Q, and

the transition fn :Q({})2Q{h}({}){L,R,S}.

Configuration of an NTM
Definition
Let T = (Q, , , , s) be an TM.
A configuration of
Can be written as

B.VIJAYAKUMAR B.E. M.Tech (PhD)

T is an element of Q

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

(q,l,a,r) or
(q,lar) Definition
Let T = (Q, , , , s) be an NTM, and (q1, 1a11) and (q2, 2a22) be
two configurations of T.
We say
(q1, 1a11) yields (q2, 2a22) in one step,
denoted by (q1, 1a11) T (q2, 2a22), if
(q2,a2,S) (q1, a1), 1=2 and 1=2,
(q2,b,R) (q1, a1), 2=1b and 1=a22,
(q2,b,L) (q1, a1), 1=2a2 and 2=b1.

NTM accepting a language/computing a function

Definition
Let T = (Q, , , , s) be an NTM.
Let w* and f be a function from * to *.
T accepts w if (s, , , w) |-T* (h, , , 1).
The language accepted by a TM T, denoted by L(T), is the
set of strings accepted by T.
T computes f if, for any string w in *, (s, ,

, w) |-T*

(h, , , f(w)).

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Example of NTM
Let L={ww| w{0,1}*}
0/0,L
1/1,L
/,L

s
/@,R

0/0,L
1/1,L

/@,L

,
0/

0/0,R
1/1,R

q0

1/
,

2301379

0/
,

,L
1/

q1

0/0,R
R 1/1,R

@/,L

t0
h

,R

@/
,R

r1
Turing Machines

0/@
,

,R
/

/,R

0/0,L
1/1,L
/,L
Jaruloj Chongstitvatana

r0

,R
@/

1
,R
1/@
0/0,R
1/1,R
28

Multitape TM
TM with more than one tape.
Each tape has its own tape head.
Each tape is independent.
CONTROL
UNIT

TAPE
TAPE
Jaruloj Chongstitvatana

2301379

B.VIJAYAKUMAR B.E. M.Tech (PhD)

Turing Machines

29

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

2-Tape Turing Machine

a quintuple (Q, , , , s), where

the set of states Q is finite, and does not contain the halt state h,
the input alphabet is a finite set of symbols, not including the blank
symbol ,
the tape alphabet is a finite set of symbols containing , but not
including the blank symbol ,
the start state s is in Q, and
the transition function is a partial function from
Q ({})2
Q{h} ({})2 {L, R, S}2

Example of 2-Tape Turing Machine

s

,/(,),(R,S)

p1

p2

0,/(0,0),(L,R)
1,/(1,1),(L,R)

,/(1,),(L,L)

,/(,),(R,S)

p3

0,/(0,),(R,S)
1,/(1,),(R,S)

q2

,/(,),(R,R)

,/(,),(L,L)

0,/(0,),(R,S)
1,/(1,),(R,S)

,/(,),(L,R)

p4

0,0/(,),(L, L)
1,1/(,),(L, L)

Jaruloj Chongstitvatana

2301379

Turing Machines

31

Equivalence of 2-tape TM and single-tape TM

Theorem:
For any 2-tape TM T, there exists a single-tape TM M
such that for any string in *:
if T halts on with on its tape, then M halts on with on its tape, and
if T does not halt on , then M does not halt

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

How 1-tape TM simulates 2-tape TM

Marking the position of each tape head in the content of the tape
Encode content of 2 tapes on 1 tape
When to convert 1-tape symbol into 2-tape symbol

Construct 1-tape TM simulating a transition in 2-tape TM

Convert the encoding of 2-tape symbols back to 1-tape symbols

Encoding 2 tapes in 1 tape

0 1 1 1 0
0 1 0 1

0 1 1 1 0
0 1 0 1

New alphabet contains:

old alphabet
encoding of a symbol on tape 1
encoding of a symbol on tape 1
symbol on tape 2
encoding of a symbol on tape 1
by its tape head
encoding of a symbol on tape 1
symbol on tape 2 po inted by its
Jaruloj Chongstitvatana

2301379

B.VIJAYAKUMAR B.E. M.Tech (PhD)

and a symbol on tape 2

pointed by its tape head and a
and a symbol on tape 2 po inted
pointed by its tape head and a
tape head
Turing Machines

34

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Tape format

c(b,)

c(a,) c(b,) c(c,) c(d,)

Encoded tape content

seperator
Whats read on tape 1 and 2

Jaruloj Chongstitvatana

2301379

Turing Machines

36

Simulating transitions in 2-tape TM in 1-tape TM

p

a1,a2/(b1,b2),(d1,d2 )

c(a1,a2)

T_tape1 (a1 ,b1 ,d1 )

T_tape2 (a2 ,b2 ,d2)

Jaruloj Chongstitvatana

2301379

B.VIJAYAKUMAR B.E. M.Tech (PhD)

Turing Machines

37

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Update
the first
cell symbol
T_tape1(0,1,d)
Convert
1-tape

,L
,x)
(
)/ c ),L
,x
c( (,
/c

not #/not #,L

c(?,x)/c(?,x),R
c(?,x)/c(?,x),R

It is
Remember
not possible
symbol
that
c(1,?)
under
is found
tape head
because
in
c(0,?) is wriiten
tape 1in cell 1.
Jaruloj Chongstitvatana

c(?,x)/c(1,x),R

2301379

#/#,L

,R

c(1,x)/c(1,x),L
#/#,L
1/c(1,),L

)/
c(
,
x)

c(0,?)/c(1,?),d
c(0,?)/c(1,?),d

not #/not #,L

c(
? ,x

),R
,x
(0
/c

#/#,R

c(?,?)/c(?,?),R

#/#,L

x)
?,
c(

? and x are 0, 1, or

c(
0/ 0,x
c( ) /c
0, (0
) ,x
,L
),L

into 2-tape symbol

not #/not #,L

Turing Machines

38

Equivalence of 2-tape TM and single-tape TM

Proof:
Let T = (Q, , , , s) be a 2-tape TM.
We construct a 1-tape TM M=(K, , , , s) such that
= {c(a,b)| a,b are in {}} {c(a,b)| a,b are in {}}
{c(a,b)|a,b are in {}} {c(a,b)|a,b are in {}} {#}
We need to prove that:
if T halts on with output , then M halts on with output , and
if T does not halt on

If T loops, then M loops.

If T hangs in a state p, M hangs somewhere from p to the next state.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Equivalence of NTM and DTM

Theorem:
For any NTM Mn, there exists a DTM Md such that:
if Mn halts on input with output , then Md halts on input with output
, and
if Mn does not halt on input , then Md does not halt on input .
Proof:
Let Mn = (Q, , , , s) be an NTM.

Construct a DTM equivalent to an NTM

WriteInitialConfiguration

Tape 1: simulate Mns tape

Tape 2: store configuration tree

Set WorkingTape

FindNewConfiguration

FindStateinCurrentConfiguration

EraseCurrentConfiguration

a,q

a,h

WriteAllPossibleNextConfiguration

a is any symbol, q is any state in Q

Jaruloj Chongstitvatana

2301379

B.VIJAYAKUMAR B.E. M.Tech (PhD)

Depend on Mn

Turing Machines

45

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

How Md works
WriteInitialConfiguration

Tape 1

Set Working Tape

FindNewConfiguration

FindStateinCurrentC onfiguration
a, q
a, h

EraseCurrentC onfiguration

@
0 1 1 0
Current
Current state:
state: ssq

Tape 2

WriteAllPossibleNex tC onfiguration

* s 0 1 1 0
- # *
0 q 1 1 0 #
*
# @ s 0 1 1 0 # 0 1 q 1 0 #
*
/@,S

Jaruloj Chongstitvatana

/,R

2301379

0/0,R
Turing Machines

46

Then, there is a positive integer n such that the initial configuration (s, ) of Mn
yeilds a halting configuration (h, ) in n steps.
From the construction of Md, the configuration (h, ) must appear on tape 2 at
some time.
Then, Md must halt with on tape 1.

if Mn does not halt on input

Then, Mn cannot reach the halting configuration. That is, (s, ) never yields a
halting configuration (h, ).
From the construction of Md, the configuration (h,) never appears on tape 2.
Then, Md never halt.

Universal Turing Machine

Given the description of a DTM T and an input string z, a universal TM simulates

how T works on input z.
Whats need to be done?

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

How to describe T and z on tape

Use an encoding function
How to simulate T

Encoding function

Let T=(Q, , , s) be a TM. The encoding function e(T) is defined as follows:

e(T)=e(s)#e(),
e()=e(m1)#e(m2)#...#e(mn)#, where = {m1, m2,..., mn}
e(m)=e(p),e(a),e(q),e(b),e(d), where m = (p, a, q, b, d)
e(z)=1e(z1)1e(z2)11e(zm)1, where z=z1z2zm is a string
e()=0, e(ai)=0i+1, where ai is in
e(h)=0, e(qi)=0i+1, where qi is in Q
e(S)=0, e(L)=00, e(R)=000

Example of Encoded TM

e()=0 ,
e(a1)=00 , e(a2)=000
e(h)=0,
e(q1)=00,
e(q2)=000
e(S)=0,
e(L)=00,
e(R)=000
e(a1a1a2) = 1e()1e(a1)1e(a1)1e(a2)1e()1
= 101001001000101
e(m1) = (q1),e(a1),e(q2),e(a2),e(R)
= 00,00,000,000,000
e(m2) = e(q2),e(),e(h),e(),e(S)
= 000,0,0,0,0
e() = e(m1)#e(m2)#...#
= 00,00,000,000,000#000,0,0,0,0#...#
e(T) = e(s)#e()
= 00#00,00,000,000,000#000,0,0,0,0#...#
Input = e(Z)|e(T)|

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

=
101001001000101|00#00,00,000,000,000#000,0,0,0,0#...#|

Universal Turing Machine

Tape 1: I/O tape, store the transition function of T and
input of T
Tape 2: simulate Ts tape
Tape 3: store Ts state
CopyInputToTape2

CopyStartStateToTape3

UpdateTape2

FindRightMove
0

UpdateStateOnTape3
Jaruloj Chongstitvatana

(halt)

2301379

CopyTape2ToTape1

Turing Machines

54

How UTM Works

12 0 0 1 0 1 | 0 0
a
Tape 1 # 0 0 , 0 0 , 0 0 0 , 0 0 0 , 0 0 0
# 0 0 0 , 0 , 0 , 0 , 0 # ... # |
Tape 2
Copy InputToTape2

1 0 0 1
0 0
1 1
0 1

Tape 3

UpdateTape2

FindRightMov e

alt
N ot h
UpdateStateOnTape3

0 0 0
Jaruloj Chongstitvatana

CopyStartStateToTape3

2301379

B.VIJAYAKUMAR B.E. M.Tech (PhD)

halt

Turing Machines

CopyTape2ToTape1

55

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Church-Turing Thesis
Turing machines are formal versions of algorithms.
No computational procedure will be considered an algorithm unless it
can be presented as a Turing machine.

Checklist
Construct a DTM, multitape TM, NTM accepting languages
or computing function
Construct composite TM
Prove properties of languages accepted by specific TM
Prove the relationship between different types
Describe the relationship between TM and FA
Prove the relationship between TM and FA

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

UNIT -V
Decidability
Decidable/Undecidable problems
Accepting:
Definition

Let T = (Q, , , , s) be a TM.

T accepts a string w in * if
(s,w) |-T* (h, 1) .

T accepts a language L* if, for any string w in L, T accepts w.

Characteristic function
For any language L*, the characteristic function of L is the function L(x)
such that
L(x) = 1
if x L
L(x) = 0
otherwise
Example
Let L = { {0,1}* | n1() <n0() <2n1() }, where nx() is the number of xs in
}.
L() = 1
if n1() <n0() <2n1()
L() = 0
otherwise

Deciding: Definition

Let T = (Q, , , , s) be a TM.

T decides a language L* if T computes the characteristic function of L.
T decides a language L* if
for any string w in L, T halts on w with output 1,
for any string w inL, T halts on w with output

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Accepting/Deciding: Example
S

/@,R

/,L

R
,
/

0/0,L
1/1,L

0/0,R
p2 1/1,R

0/

,L

p4

1/,L
/,L

Jaruloj Chongstitvatana

p3

Hang when
input = 02n

q1

q2

/1,L

/0,L

1/,R

/,L

p1

@/,R

r2

/,L

/,L

0/,R

@/,R

r1

n10
n10
n |n0}
n |n0}
TM decidinging
accepting L={0
L={0

0/,L
1/,L

2301379

Hang when input

= 0n 1 0n+m

If the input x is in L,
T halts with output 1.
If the input x is not in L,
T hangs.
Hang when input
= 0n+m 0n
Decidability

Recursively enumerable languages

A language L is recursively enumerable if there is a Turing machine T

accepting L.
A language L is Turing-acceptable if there is a Turing machine T
accepting L.
Example:
{0n10n|n0} is a recursively-enumerable
language.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Recursive languages

A language L is recursive if there is a Turing machine T deciding L.

A language L is Turing-decidable if there is a Turing machine T
deciding L.

Example:
{0n10n|n0} is a recursive language.

Closure Properties of the Class of Recursive Languages

Theorem:

Let L be a recursive language over . Then,L is recursive.

Proof:
Let L be a recursive language over .
Then, there exists a TM T computing L.
Construct a tape TM M computing L. as follows:
T TmoveRight 0 Twrite1
1
Twrite0
Then,L is recursive.
Closure Property Under Union

Theorem: Let L1 and L2 be recursive languages over . Then, L1L2 is recursive.

Proof:
Let L1 and L2 be recursive languages over .
Then, there exist TMs T1 and T2 computing L1 and L2, respectively.
Construct a 2-tape TM M as follows:
TcopyTape1ToTape2 T1 TmoveRight 0 TcopyTape2ToTape1 T2

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Closure Property Under Union

TcopyTape1ToTape2 T1 TmoveRight 0 TcopyTape2ToTape1 T2

If the input w is not in L1 and L2, L1(w) and L2(w)=0. Thus, both T1 and T2
must run, and M halts with output 0.
If the input w is in L1, L1(w)=1. Thus, M halts with output 1.
If the input w is not in L1 but is in L2, L1(w)=0 and L2(w)=1. Thus, M halts with
output 1.
That is, M computes characteristic function of L.
Then, L1L2 is recursive.

Closure Property Under Intersection

Theorem: Let L1 and L2 be recursive languages over . Then, L1L2 is recursive.

Proof:
Let L1 and L2 be recursive languages over .
Then, there exist TMs T1 and T2 computing L1 and L2, respectively.

Construct a 2-tape TM M as follows:

TcopyTape1ToTape2 T1 TmoveRight 1 TcopyTape2ToTape1 T2

TcopyTape1ToTape2 T1 TmoveRight 1 TcopyTape2ToTape1 T2

If the input w is in L1L2, L1(w) and L2(w)=1. Thus, M halts with output 1.
If the input w is not in L1, L1(w)=0. Thus, M halts with output 0.
If the input w is in L1 but is not in L2, L1(w)=1 and L2(w)=0. Thus, M halts with
output 0.
That is, M computes characteristic function of L1L2.
Then, L1L2 is recursive.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Closure Properties of the Class of Recursively Enumerable

Languages
Theorem: Let L1 and L2 be recursively enumerable languages over . Then, L1L2
is also recursively enumerable.
Proof:
Let L1 and L2 be recursively enumerable languages over .
Then, there exist TMs T1 and T2 accepting L1 and L2, respectively.
Construct an NTM M as follows.

Closure Property Under Union

If w is in L1, but not in L2, then T1 in M runs and halts.

If w is in not L1, but in L2, then T2 in M runs and halts.
If w is in both L1 and L2, then either T1 or T2 runs and halts.
For these 3 cases, M halts.
If w is neither in L1 nor in L2, then either T1 or T2 runs but both never halt. Then, M does not
halt.
Thus, M accepts L1L2. That is, L1L2 is recursively enumerable.

Closure Property Under Intersection

Theorem: Let L1 and L2 be recursively enumerable languages over . Then, L1L2 is also
recursively enumerable.
Proof:
Let L1 and L2 be recursively enumerable languages over .
Then, there exist TMs T1 and T2 accepting L1 and L2, respectively.
Construct an NTM M as follows.
TcopyTape1ToTape2 T1 TmoveRight 1 TcopyTape2ToTape1 T2

Closure Property Under Intersection

If w is in not L1, then T1 in M does not halt. Then, M does not halt.
If w is in L1, but not in L2, then T1 in M halts and T2 can finally start, but does not halt. Then,
M does not halt.
If w is in both L1 and L2, then T1 in M halts and T2 can finally start, and finally halt. Then, M
halts.
Thus, M accepts L1L2. That is, L1L2 is recursively enumerable.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Closure Property Under Intersection (II)

Theorem:
Let L1 and L2 be recursively enumerable languages over . Then, L1L2 is also recursively
enumerable.
Proof:
Let L1 and L2 be recursively enumerable languages over .
Then, there exist DTMs T1 =(Q1, , , 1, s1) and T2 =(Q2, , , 2, s2) accepting L1 and L2,
respectively.
Construct a 2-tape TM M which simulates T1 and T2 simultaneously. Tape 1 represents T1s
tape and Tape 2 represents T2s tape.

Closure Property Under Intersection (II)

Let M = ((Q1{h})(Q2{h}), , , , (s1,s2)) where
((p1,p2),a1,a2) = ((q1,q2),b1,b2,d1,d2) for 1(p1,a1)=(q1,b1,d1) and 2(p2,a2
)=(q2,b2,d2)
((h,p2),a1,a2) = ((h,q2),a1,b2,S,d2) for all p2,a1,a2 and 2(p2,a2)=(q2,b2,d2)
((p1,h),a1,a2) = ((q1,h),b1,a2,d1,S) for all p1,a1,a2 and 1(p1,a1)=(q1,b1,d1)
((h,h),a1,a2) = (h,a1,a2,S,S) for all a1,a2
If neither T1 nor T2 halt, M never gets to the state h.
If T1 halts and T2 does not halt, M gets to the state (h,p).
If T2 halts and T1 does not halt, M gets to the state (p,h).
If both T1 and T2 halt, M finally gets to the state h.
Relationship Between the Classes of Recursively Enumerable and Recursive Languages

Theorem: If L is a recursive language, then L is recursively enumerable.

Proof:
Let L be a recursive language over .
Then, there is a TM T deciding L.
Then, T also accepts L.
Thus, L is recursively enumerable.

Relationship between RE and Recursive Languages

Theorem: Let L be a language. If L andL are recursively enumerable, then L is recursive.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Proof:
Let L andL be recursively-enumerable languages over .
Then, there are a TM T accepting L, and a TMT acceptingL.
For any string w in *, w is either in L or inL.
That is, either T orT must halt on w, for any w in *.
We construct an NTM M as follows:
If w is in L, T halts on w and thus, M accepts w.
If w is not in L,T halts on w and thus, M rejects w.
Then, M computes the characteristic function of L. Then, L is recursive.

Decision Problems

A decision problem is a prob. whose ans. is either yes or no

A yes-instance (or no-instance) of a problem P is the instance of P whose answer is
yes (or no, respectively)
A decision problem P can be encoded by fe over as a language {fe(X)| X is a yesinstance of P}.

Encoding of decision problems

Is X a prime ?
{1X | X is a prime}
Does TM T accept string e(T)?
{e(T) | T is a TM accepting string e(T)}
Does TM T accept string w?
{e(T)e(w) | T is a TM accepting string w} or
{<T,w> | T is a TM accepting string w}

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Decidable (or solvable) problems

Definition:
If fe is a reasonable encoding of a decision problem P
over , we say P is decidable (or solvable) if the associated language {fe(X)| X is a
yes-instance of P} is recursive.
A problem P is undecidable (or unsolvable) if P is not

decidable.

Self-Accepting
SA (Self-accepting) = {w{0,1,#, ,}*| w=e(T) for some TM T and wL(T)}
NSA (Non-self-accepting) = {w {0,1,#, ,}*| w=e(T) for some TM T and
wL(T)}
E (Encoded-TM) = {w{0,1,#, ,}*| w=e(T) for some TM T}

NSA is not recursively enumerable

We prove by contradiction.
Assume NSA is recursively enumerable .
Then, there is TM T0 such that L(T0)=NSA.
Is e(T0) in NSA?
If e(T0)NSA, then e(T0)L(T0) by the definition of NSA But
L(T0)=NSA. Thus, contradiction.
If e(T0) NSA, then e(T0) SA and e(T0)L(T0) by the definition of
SA. But L(T0)=NSA. Thus, contradiction.
Then, the assumption is false.
That is, NSA is not recursively enumerable.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

E is recursive

Theorem: E is recursive.
Proof:
We can construct a regular expression for E according to the
definition of the encoding function as follows:
R = S 1 (M #)+
S=0
M = Q , A, Q , A, D
Q = 0+
A = 0+
D = 0 + 00 + 000
Then, E is regular, and thus recursive.

SA is recursively enumerable

Construct a TM S accepting SA
If w is not e(T) for some TM T, S rejects w.
If w is e(T) for some TM T, S accepts e(T) iff T
accepts e(T).
L(S) = {w| w=e(T) for some TM T accepting e(T) =
SA.
Then, SA is recursively enumerable.

Encode

UTM

Reject
Jaruloj Chongstitvatana

2301379

B.VIJAYAKUMAR B.E. M.Tech (PhD)

Decidability

32

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

SA is not recursive

NSA = E SA
NSA is not recursively enumerable (from previous theorem), and thus not recursive.
But E is recursive.
From the closure property, if L1 and L2 are recursive, then L1 - L2 is recursive.
Using its contrapositive, if L1 - L2 is not recursive, then L1 or L2 are not recursive.
Since NSA is not recursive and E is recursive, SA is not recursive.

Co-R.E.

Definition
A language L is co-R.E. if its complement L is R.E.
It does not mean L is not R.E.
Examples:
SA is R.E. SA=ENSA is not R.E.
SA is co-R.E., but not R.E.
NSA is not R.E. NSA=ESA is R.E.
NSA is co-R.E., but not R.E.
E is recursive, R.E., and co-R.E.
Relationship between R.E., co-R.E. and Recursive Languages

Theorem: Let L be any language. L is R.E. and co-R.E. iff L is recursive.

Proof:
() Let L be R.E. and co-R.E. Then, L is R.E. Thus, L is recursive.
() Let L be recursive. Then, L is R.E. From the closure under complementation
of the class of recursive languages,L is also recursive. Then, L is also R.E.
Thus, L is co-R.E.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Observation
A language L is either
recursive
R.E., bot not recursive
co-R.E., but not recursive
Neither R.E. nor co-R.E.

R.E.

recursive co-R.E.

Neither R.E. nor co-R.E.

Jaruloj Chongstitvatana

2301379

Decidability

36

Reduction Definition:
Let L1 and L2 be languages over 1 and 2, respectively.
L1 is (many-one) reducible to L2, denoted by L1L2, if there is a TM M computing a
function f: 1*2* such that wL1 f(w)L2.
Definition:
Let P1 and P2 be problems. P1 is (many-one) reducible to
P2 if there is a TM M computing a function f: 1*2* such that w is a yes-instance of P1
f(w) is a yes-instance of P2.

Reduction

Definition:
A function f: 1*2* is a Turing-computable function if there is a Turing
machine computing f.
Definition:
Let L1 and L2 be languages over 1 and 2,
respectively. L1 is (many-one) reducible to L2, denoted by L1L2, if there
is a Turing-computable function f: 1*2* such that wL1 f(w)L2.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Meaning of Reduction
P1 is reducible to P2 if TM M computing a function f: 1*2* such
that w is a yes-instance of P1 f(w) is a yes-instance of P2.
If you can map yes-instances of problem A to yes-instances of
problem B, then
we can solve A if we can solve B
it doesnt mean we can solve B if we can solve A
the decidability of B implies the decidability of A

Properties of reduction
Theorem:
Proof:

Let L be a language over . LL.

Let L be a language over .
Let f be an identity function from **.
Then, there is a TM computing f.
Because f is an identity function, wL f(w)=wL.
By the definition, LL.

Properties of reduction

Theorem:

Let L1 and L2 be languages over .

If L1L2, thenL1L2.

Proof:

Let L1 and L2 be languages over .

Because L1L2, there is a function f such that wL1
f(w)L2, and a TM T computing f.
wL1 f(w)L2.
By the definition,L1L2.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Properties of reduction
Theorem:

Let L1, L2 and L3 be languages over .

If L1L2 and L2L3, then L1L3.

Proof:

Let L1, L2 and L3 be languages over .

There is a function f such that wL1 f(w)L2, and
a TM T1 computing f because L1L2.
There is a function g such that wL2 g(w)L3, and
a TM T2 computing g because L2L3.
wL1f(w)L2g(f(w))L3, and T1T2 computes
g(f(w)).
By the definition, L1L3.

Using reduction to prove decidability

Theorem: If L2 is recursive, and L1L2, then L1 is also recursive.
Proof:
Let L1 and L2 be languages over , L1L2, and L2 be recursive.
Because L2 is recursive, there is a TM T2 computing L2.
Because L1L2, there is a TM T1 computing a function f such that wL1
f(w)L2.

Using reduction to prove decidability

Construct a TM T=T1T2. We show that T computes L1.
If wL1, T1 in T computes f(w)L2 and T2 in T computes L2(f(w)),
which is 1.
If wL1, T1 in T computes f(w) L2 and T2 in T computes L2(f(w)),
which is 0.
Thus, L1 is also recursive.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Using reduction to prove R.E

Theorem: If L2 is R.E., and L1L2, then L1 is also R.E.
Proof:
Let L1 and L2 be languages over , L1L2, and L2 be R.E.
Because L2 is R.E, there is a TM T2 accepting L2.
Because L1L2, there is a TM T1 computing a function f such that wL1
f(w)L2.

Using reduction to prove R.E.

Construct a TM T=T1T2. We show that T accepts L1.
If wL1, T1 in T computes f(w)L2 and T2 in T accepts f(w). Thus, T
accepts w.
If wL1, T1 in T computes f(w)L2 and T2 in T does not accept (f(w)).
Thus, T does not accept w.
Thus, L1 is also R.E.

Using reduction to prove co-R.E.

Theorem:
If L2 is co-R.E., and L1L2, then L1 is also co-R.E.
Proof:
Let L1 and L2 be languages over , L1L2, and L2 be co-R.E.
Because L2 is co-R.E,L2 is R.E.
Because L1L2,L1L2. Then,L1 is R.E.
Thus, L1 is co-R.E.

Theorem:
If L2 is co-R.E., and L1L2, then L1 is also co-R.E.
Proof:
Let L1 and L2 be languages over , L1L2, and L2 be co-R.E.
Because L2 is co-R.E,L2 is R.E.
Because L1L2,L1L2. Then,L1 is R.E.
Thus, L1 is co-R.E.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Another way to prove undecidability

R.E.

recursive co-R.E.

Neither R.E. nor co-R.E.

Let L1L2.
If L1 is not recursive /
R.E. /
co-R.E.,
then L2 is not recursive /
R.E. /
co-R.E.

To prove a language L is not recursive:

1. Guess where L is (not R.E. or not co-R.E.)
2. Choose another non-recursive language R which is of the
same type
3. Show R L.
Jaruloj Chongstitvatana

2301379

Decidability

51

Guess if its rec., R.E., co-R.E., or neither

Given a TM T,
does T get to state q on blank tape?
does T accept ?
does T output 1?
does T accept everything?
is L(T) finite?
Problem of accepting an empty string
We will prove that the problem if a TM accepts an empty string is
undecidable.
This problem is corresponding to the following language.
Accept = {e(M)| M is a TM accepting }
Thus, we will prove that Accept is not recursive.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Accept is not recursive.

Proof:
(Guess Accept is in R.E., but not co-R.E.)
Show SA Accept
(We want a Turing-computable f n f(<T>)=<M> such that
T accepts e(T) M accepts
T does not accept e(T) M does not accept
Let f(T)=M is a TM that first writes e(T) after its input and then runs
T.
M writes e(T) after its input. If its input is , T has e(T) as input.

Accept is not coR.E.

Verify that T accepts e(T) M accepts
M writes e(T) and lets T run. If the input of M is :
when T accepts e(T), M accepts .
when T doesnt accept e(T), then M doesnt accept .
Accept is not coR.E.

Next, we show that there is a TM TF computing f.

TF works as follows:
changes the start state of T in e(T) to a new state
add e(Write<T>), make its start state the start state of TF, and make
the transition from its halt state to Ts start state.
Then, SA Accept.
Then,Accept is not co-R.E, and is not recursive.

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Halting problem
Problem
Given a Turing machine T and string z, does T halt on z?
Given a program P and input z, does P halt on z?
Language
Halt = {w*| w=e(T)e(z) for a Turing machine T halting on
z}.
Halt = {<T,z>| T is a Turing machine halting on z}.

Halting problem is undecidable

Proof:
Let Halt = {<T,z>| T is a Turing machine halting on z}.
(Guess Halt is in R.E., but not co-R.E.)
Show SA Halt
(We want a Turing-computable f n f(<T1>)=<T2 ,z> such that
T1 accepts e(T1) T2 halts on z
T1 does not accept e(T1) T2 does not halt on z
Then, a possible function is f(<T>) = <T, e(T)> because T accepts e(T) T
halts on e(T).)

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING

CS53 THEORY OF COMPUTATION

Einstein College of Engineering

Some other undecidable problems

FINITE
Given a TM T, is L(T) finite?
Guess FINITE is neither R.E. nor co-R.E.
To assure L(T) is finite, we need to run T on all possible input and
count if T accepts a finite number of strings.
To assure L(T) is infinite, we need to run T on all possible input and
count if T accepts an infinite number of strings.

FINITE is not recursive FINITE is not recursive

Let FINITE={<T>| T is a TM such that L(T) is finite.}
Guess FINITE is neither R.E. nor co-R.E.
Choose NSA which is not co-R.E. to show that NSAFINITE.
We want to find a Turing-computable function f such that <T>NSA f(<T>)=MFINITE
<T>NSA M accepts , and thus L(M) is finite.
<T>NSAM accepts *, and thus L(M) is infinite.
Then, let M=f(<T>) be a TM that runs T on its input, and accepts everything if T halts.

FINITE is not recursive

Now, we will show that <T>NSA <M>FINITE
If <T>NSA, then T does not accept <T>. Then, M does not get to start AccAll. Thus, M accepts
nothing and L(M) is finite.
If <T>NSA, then T accepts <T>. Then, M gets pass T, and accept everything. Thus, M accepts

Checklist

Prove a language is recursive, R.E., or co-R.E.

Prove closure properties of these classes of languages
Prove properties of reduction
Prove a language is not recursive, not R.E., or not co-R.E.

prove a problem is decidable

Prove a problem is undecidable

B.VIJAYAKUMAR B.E. M.Tech (PhD)

EINSTEIN COLLEGE OF ENGINEERING