THEORY OF COMPUTATION

Dr. K. Umamaheswari
Professor,
Dept. of Information Technology,
PSG College of Technology,
Coimbatore

Department of Information Technology

PSG College of Technology 1
 Outline
• Introduction to Theory of Computation
• Pushdown Automata
• Equivalence of CFG and PDA
• Turing Machine

Department of Information Technology

PSG College of Technology 2
 INTRODUCTION

Department of Information Technology

PSG College of Technology 3
 Theory of Computation
began early in the 20th century (before modern electronic
computers had been invented)

mathematicians were trying to find which (math) problems

could be solved by simple methods and which could not

define what is meant by a “simple method for solving a

problem”

need for a formal model of computation

Department of Information Technology

PSG College of Technology 4
 What is a problem?
• Decision problem:
f:Σ* → {accept, reject}
• Equivalent notion: language
L  Σ*
the set of strings that map to “accept”
• Example: L = set of pairs (m,k) for which m
has a prime factor p < k

Department of Information Technology

PSG College of Technology 5
 Computation
• accept
input machine • reject
• loop forever

• the set of strings that lead to “accept” is

the language recognized by this
machine
• if every other string leads to “reject”,
then this language is decided by the
machine

Department of Information Technology

PSG College of Technology 6
 What goes inside the box?
• accept
input machine • reject
• loop forever

• We want the simplest mathematical

formalization of computation possible.
• Strategy:
– endow box with a feature of computation
– try to characterize the languages decided
– identify language we “know” real computers can
decide that machine cannot
– add new feature to overcome limits
Department of Information Technology
PSG College of Technology 7
 CHOMSKY
HIERARCHY
a containment hierarchy classes
of formal languages.

- Noam Chomsky

Department of Information Technology

PSG College of Technology 8
 Context Free Grammar
and Normal Forms

Department of Information Technology

PSG College of Technology 9
• Given a context-free grammar G = (VN,VT,P,S), the language
generated by G is the set L(G) = { w ∈ Σ∗ | S ⇒ w }.
• A language L ⊆ Σ∗ is a context-free language (CFL) iff
L = L(G) for some context-free grammar G.
• Given Grammar:
E → E+E | E*E | (E) | a | b
• Extended Grammar:
E → E+E | E*E | (E) | E-E | E^E | a | b
G = ( {E}, {+, *, (, ), -, ^, a, b}, P, E)
L = {Generates a set of arithmetic expressions with
operands a and b }
Department of Information Technology
PSG College of Technology 10
 Equivalence of CFG and PDA

Department of Information Technology

PSG College of Technology 11
• Context-Free Languages (CFLs) can be
described by a Context Free Grammar
(CFG) and can be processed by a
pushdown automaton.

Department of Information Technology

PSG College of Technology 12
 Procedure 1 for CFG to PDA

• Given a CFG G = (V, ∑, S, P), we define a PDA

P(G) = ({q0}, ∑, V ∪ ∑, δ, q0, S)
where δ is defined as follows:
• δ(q0, ∈, A) = {(q0, α) | A → α ∈ P} for all A ∈ V
• δ(q0, a, a) = {(q0, ∈)} for all a ∈ ∑.

Department of Information Technology

PSG College of Technology 13
 Example:
• G = ({E, T, F}, {(, ), a, +, *}, E, P)
where the set of productions P are given
by
• E→T|E+T
• T→F|T*F
• F → a | (E )

Department of Information Technology

PSG College of Technology 14
 • we define P(G) = ({q0}, {(, ), a, +, *}, {E, T, F, (, ), a,
+, *}, δ, q0, E) where

• δ(q0, ∈,E) = {(q0, T), (q0,E+T)}

• δ(q0, ∈, T) = {(q0, F), (q0, T*F)}
• δ(q0, ∈, F) = {(q0, a), (q0, (E))}
• δ(q0, (, () = {(q0, ∈)}
• δ(q0, ), )) = {(q0, ∈)}
• δ(q0, a, a) = {(q0, ∈)}
• δ(q0, +, +) = {(q0, ∈)}
• δ(q0, *, *) = {(q0, ∈)}
• δ(q, x, z) = ∅

Department of Information Technology

PSG College of Technology 15
 Trace for the String w = a+(a*a)
(q0, a+(a*a), E) ⊢ (q0, a+(a*a), ⊢ (q0, a*a), T))
E+T)
⊢ (q0, a*a), T*F))
⊢ (q0, a+(a*a), T+T)
⊢ (q0, a*a), F*F))
⊢ (q0, a+(a*a), F+T)
⊢ (q0, a*a), a*F))
⊢ (q0, a+(a*a), a+T)
⊢ (q0, *a), *F))
⊢ (q0, +(a*a), +T)
⊢ (q0, a), F))
⊢ (q0, (a*a), T)
⊢ (q0, a), a))
⊢ (q0, (a*a), F)
⊢ (q0, ), ))
⊢ (q0 (a*a), (E))
⊢ (q0, ∈, ∈ )
⊢ (q0, a*a), E))
Hence a+(a*a) ∈ L(P(G)).

Department of Information Technology

PSG College of Technology 16
 • w ∈ L(G) ⇐⇒ S ∗⇒ w ⇐⇒ (q0,w, S)∗⊢ (q0, ∈, ∈)
⇐⇒ w ∈ L(P(G))
• Shows Non – Deterministic Pushdown Automata

Department of Information Technology

PSG College of Technology 17
 Example 2:
L = {anbmcpdq : m + n = p + q} 0 (p, ε, ε), (q, S)
1 (q, ε, S), (q, aSd)
(1) S → aSd
2 (q, ε, S), (q,T)
(2) S → T
3 (q, ε, S), (q,U)
(3) S → U 4 (q, ε, T), (q, aTc)
(4) T → aTc 5 (q, ε, T), (q, V)
(5) T → V 6 (q, ε, U), (q, bUd)
(6) U → bUd 7 (q, ε, U), (q, V)
(7) U → V 8 (q, ε, V), (q, bVc)
(8) V → bVc 9 (q, ε, V), (q, ε)
(9) V → ε 10 (q, a, a), (q, ε)
11 (q, b, b), (q, ε)
input = a a b c d d 12 (q, c, c), (q, ε)
13 (q, d, d), (q, ε)

Department of Information Technology

PSG College of Technology 18
 GNF to PDA
• Definition: Let G = (V, T, P, S) be a CFL. If every production in P is of
the form
A –> aα
Where A is in V, a is in T, and α is in V*, then G is said to be in Greibach
Normal Form (GNF).
• Example:
S –> aAB | bB
A –> aA | a
B –> bB | c
• Theorem: Let L be a CFL. Then L – {ε} is a CFL.

• Theorem: Let L be a CFL not containing {ε}. Then there exists a GNF
grammar G such that L = L(G).

Department of Information Technology

PSG College of Technology 19
• Lemma 1: Let L be a CFL. Then there exists a PDA M such that L = LE(M).
• Proof: Assume without loss of generality that ε is not in L. The construction can
be modified to include ε later.
Let G = (V, T, P, S) be a CFG, and assume without loss of generality that G is in
GNF. Construct M = (Q, Σ, Г, δ, q, z, Ø) where:
Q = {q}
Σ=T
Г=V
z=S
δ: for all a in Σ and A in Г, δ(q, a, A) contains (q, γ) if A –> aγ is in P or rather:
δ(q, a, A) = {(q, γ) | A –> aγ is in P and γ is in Г*}, for all a in Σ and A in Г
• For a given string x in Σ* , M will attempt to simulate a leftmost derivation of x
with G.

Department of Information Technology

PSG College of Technology 20
 • Example 1:
• Consider the following CFG in GNF.
S –> aS G is in GNF
S –> a L(G) = a+
Construct M as:
Q = {q}
Σ = T = {a}
Г = V = {S}
z=S
δ(q, a, S) = {(q, S), (q, ε)}
δ(q, ε, S) = Ø
• Question: Is that all? Is δ complete? Recall that δ: Q x (Σ U {ε}) x Г –>
finite subsets of Q x Г*

Department of Information Technology

PSG College of Technology 21
 • Example 2:.
Consider the following CFG in GNF
(1) S –> aA
(2) S –> aB
(3) A –> aA
(4) A –> aB
(5) B –> bB
(6) B –> b
Construct M as: Q = {q} Σ = T = {a, b} Г = V = {S, A, B} z=S
(1) δ(q, a, S) = {(q, A), (q, B)} From productions #1 and 2, S->aA, S->aB
(2) δ(q, a, A) = {(q, A), (q, B)} From productions #3 and 4, A->aA, A->aB
(3) δ(q, a, B) = Ø
(4) δ(q, b, S) = Ø
(5) δ(q, b, A) = Ø
(6) δ(q, b, B) = {(q, B), (q, ε)} From productions #5 and 6, B->bB, B->b
(7) δ(q, ε, S) = Ø
(8) δ(q, ε, A) = Ø
(9) δ(q, ε, B) = Ø Recall δ: Q x (Σ U {ε}) x Г –> finite subsets of Q x Г*

Department of Information Technology

PSG College of Technology 22
 • Tracing the PDA for the string w = aaaabb
S => aA (1)
=> aaA (3)
=> aaaA (3)
=> aaaaB (4)
=> aaaabB (5)
=> aaaabb (6)
• The corresponding computation of the PDA:
• q, aaaabb, S) |— (q, aaabb, A) (1)/1
|— (q, aabb, A) (2)/1
|— (q, abb, A) (2)/1
|— (q, bb, B) (2)/2
|— (q, b, B) (6)/1
|— (q, ε, ε) (6)/2
– String is read
– Stack is emptied
– Therefore the string is accepted by the PDA

Department of Information Technology

PSG College of Technology 23
 • Example 2:
• Consider the following CFG in GNF.
(1) S –> aABC
(2) A –> a
(3) B –> b
(4) C –> cAB
(5) C –> cC
Construct M as: Q = {q} Σ = T = {a, b, c} Г = V = {S, A, B, C} z = S
(1) δ(q, a, S) = {(q, ABC)} S->aABC (9) δ(q, c, S) = Ø
(2) δ(q, a, A) = {(q, ε)} A->a (10) δ(q, c, A) = Ø
(3) δ(q, a, B) = Ø (11) δ(q, c, B) = Ø
(4) δ(q, a, C) = Ø (12) δ(q, c, C) = {(q, AB), (q, C))} // C->cAB|cC
(5) δ(q, b, S) = Ø (13) δ(q, ε, S) = Ø
(6) δ(q, b, A) = Ø (14) δ(q, ε, A) = Ø
(7) δ(q, b, B) = {(q, ε)} B->b (15) δ(q, ε, B) = Ø
(8) δ(q, b, C) = Ø (16) δ(q, ε, C) = Ø

Department of Information Technology

PSG College of Technology 24
 Exercises:

• Generate PDA for the the following grammar and

Trace for the string w = 00111
– S  AS | 
– A  0A1 | A1 | 01
• Consider the following grammar G with
productions
S → 0B / 1A A → 0 / 0S / 1AA B → 1 / 1S / 0BB
Trace for the sentence w = 00110101

Department of Information Technology

PSG College of Technology 25
 Normal Forms
• A NF for a grammar has additional
conditions imposed upon its productions
and is equivalent to the given grammar.
• Two types
– Chomsky NF (CNF)
– Greibach NF (GNF)
• Simple form of productions.
• Rules in CNF has both theoretical and
practical implications.
Department of Information Technology
PSG College of Technology 26
 Significance
• Language preserving transformations
• Improve parsing efficiency
• Prove properties about languages and derivations
CNF:
CYK membership algorithm to find if a string is in
the language represented by the grammar.
GNF:
is used for conversion from CFG to NDPA and vice
versa.

Department of Information Technology 27

PSG College of Technology 27
 T0

T1
T2 GNF
T3
CNF

Department of Information Technology

PSG College of Technology 28
 Preprocess Steps
1. Remove λ productions
2. Remove Unit Productions.
3. Remove useless productions and non-
terminals.
4. Eliminate Left Recursion
5. Eliminate Left Factoring

Department of Information Technology

PSG College of Technology 29
 Elimination of l-rules
Reduces the length of the derivation

S  aBS | l S  aS | aBS | l
B  bB | l B  bB | b

L( B)  b * L( B)  b 
L( S )  (ab*) * L( S )  (ab*) *
S 7 aaa S 4 aaa
Department of Information Technology
PSG College of Technology 30
 Example
S  AB S'  S | l
B  bB | l S  AB | A | B
A  aA | l B  bB | b
S'  S A  aA | a
S  AB | A | B
a *b* 
B  bB | l | b
l  a   b  a  b
A  aA | l | a
Department of Information Technology
PSG College of Technology 31
 Elimination of useless symbols

• A variable is useful if it occurs in a

derivation that begins with the start
symbol and generates a terminal string.
• Reachable from S S * uXv where X V
G

u,v  (V  ) *

• Derives terminal string X *G 

where   *

Department of Information Technology

PSG College of Technology 32
Determination of nullable non-terminals

S  AC A  aA | C
C  cC | l B  AbC

{C}  {A, C}  {S, A, C}

Bottom-up flow of information

Department of Information Technology

PSG College of Technology 33
 Elimination of Unit Productions
Removing renaming rules: redundant procedure calls.

A  aA | a | B
B  bB | b | C
Cc
A  aA | a | bB | b | c
B  bB | b | c
Cc
Top-down flow of information

Department of Information Technology

PSG College of Technology 34
 Examples

S  AB | A | B S  AB | aA | a | b
A  aA | a | B A  aA | a | b
Bb Bb

S  aA | b | A S  aA | b | Sa | bB
A  Sa | B A  Sa | bB | aA | b
B  bB | S B  bB | aA | b | Sa

Department of Information Technology

PSG College of Technology 35
 Chomsky Normal Form (CNF)
• A CFG is in Chomsky Normal Form if
each rule is of the form:
A  BC
Aa
S l
where B,C V - {S}
• Theorem: There is an algorithm to
construct a grammar G’ in CNF that is
equivalent to a CFG G.

Department of Information Technology

PSG College of Technology 36
 CNF Conversion
CNF -- Rules are of the form A  BC or A  a
Eg: S  AS | a; A  SA | b
Steps
1. Eliminate λ, unit and useless productions
2. Add production of form A a or A  BC to P1
3. Consider A  x1x2x3……x n
1. If n = 1 then x1 should be a Terminal (T)
2. If n >=2, introduce Ba for a ε T and
1. Convert A  x1x2x3……x n to A  B1 B2 … B3 and add B1
 x1 … to P1
3. For n>2 introduce new Var D1,D2…
1. Eg. A BCDE is written as
2. A BD1 D1  C D2 D2  D D3 D3 E D4
Department of Information Technology
PSG College of Technology 37
 Significance of CNF
• Length of derivation of a string of length
n in CNF = (2n-1)
(Cf. Number of nodes of a strictly binary tree with n-leaves)
• Maximum depth of a parse tree = n
• Minimum depth of a parse tree = log 2 n  1

Department of Information Technology

PSG College of Technology 38
 Example
• Obtain an equivalent grammar that does
not contain l-rules, chain rules, and
useless variables.
• Apply following conversion on rules of
the form: A  bBcC
A  PQ Pb
Q  BR R  WC
W c
Department of Information Technology
PSG College of Technology 39
• Every CFG G can also be converted to an equivalent
grammar in Greibach Normal Form(GNF).
• A context-free grammar G = (VN,VT,P,S) is in Greibach
Normal Form iff its productions are of the form
– A→a
– A → aA1A2…An
– S→l
Where A1A2…An ∈ VN, a ∈ Σ ∗ .
• S → l is in P iff l ∈ L(G), and S does not occur on the
right-hand side of any production.
• An important consequence of the Greibach Normal Form
is that every non-terminal is not left recursive.
Department of Information Technology
PSG College of Technology 40
 In the context-free grammar, a non-terminal  left-
recursive
if the left symbol in one of its productions is itself (in case of
direct left recursion) A Aα/β

Can be made itself by some sequence of substitutions (in the

case of indirect left recursion)
Direct Recursion : occurs when the definition can be satisfied
with only 1 substitutions
 It requires rules in the form of
α  sequence of non-terminals & terminals
β  does not start with A

Department of Information Technology

PSG College of Technology 41
 To eliminate left recursion

A  β A’
A’  λ / α A’

INDIRECT LEFT RECURSION

Occurs when the definition is satisfied via several
substitutions
Example :
S  AA / 0 ASA/1

A  A α1 / A α2 /….. / A αn / β1 / β2 A  β1 A’ / β2 A’/ …… / βm A’
/ …… / βm A’  λ / α1 A’ / α2 A’ /….. / αn A’

Department of Information Technology
PSG College of Technology 42
• E  E + E / E * E / (E) / a / b

Department of Information Technology

PSG College of Technology 43
No useless symbol
A  β1 A’ / β2 A’/ …… / βm A’
No nullable (λ) production A’  λ / α1 A’ / α2 A’ /….. / αn A’
No unit production
LEFT RECURSION :
A α1 A α2 β1 β2 β3
E  E + E / E * E / (E) / a / b

E  (E) E’ / a E’ / b E’
E’  + E E’ / * E E’ / λ

Department of Information Technology

PSG College of Technology 44
E  (E) E’ / a E’ / b E’
E’  + E E’ / * E E’ / λ
E  (E) E’ / a E’ / b E’ / (E) / a / b
E’  + E E’ / * E E’ / + E / * E
1 2 3 4 5 6
E  (E) E’ / a E’ / b E’ / (E) / a / b
7 8 9 10

E’  + E E’ / * E E’ / + E / * E
Department of Information Technology
PSG College of Technology 45
 1 2 3 4 5 6
E  (E) E’ / a E’ / b E’ / (E) / a / b
7 8 9 10
E’  + E E’ / * E E’ / + E / * E

VT  + , * , ( , ) , a , b VN  E , E’

E’  + E E’ / * E E’ / + E / * E are already in GNF

E  a E’ / b E’ / a / b are already in GNF

4 1
E(E) E  (E) E’

A) E  ( E A E’
E(EA

Department of Information Technology

PSG College of Technology 46
 E  ( E A E’ / a E’ / b E’ / ( E A / a / b
A)

Department of Information Technology

PSG College of Technology 47
Question
Consider the following CFG G = (VN,VT,P,S),
where VN = { S, T, X }, VT = { a, b }, the start
variable is S, and the rules P are
S→aT Xb, T→XT S | ε, X→ a | b.
Construct PDA after converting to GNF.

Department of Information Technology

PSG College of Technology 48
 Prerequisites
• CFG: A context-free grammar (CFG)
consisting of a finite set of grammar rules is a
quadruple (VN,VT,P,S) where
VN is a set of non-terminal symbols.
VT is a set of terminals where VN ∩ VT = NULL.
P is a set of rules, P: VN → (VN ∪ VT)*, i.e., the
left-hand side of the production rule P does
have any right context or left context.
S is the start symbol.

Department of Information Technology

PSG College of Technology 49
 Prerequisites
• GNF: A CFG is in Greibach Normal Form
if the Productions are in the following
forms −
A→b
A → bD1…Dn
S→ε
where A, D1,....,Dn are non-terminals
and b is a terminal.
Department of Information Technology
PSG College of Technology 50
 GNF
G = ({ S, T, X }, { a, b }, R, S)
S→aTXb | aXb, T→XTS | XS, X→ a | b

S → aTXP | aXP
T → aTS| bTS | aS | bS
X→a|b
P→b

Department of Information Technology

PSG College of Technology 51
 Conversion of CFG to GNF
G = ({ S,T, X }, { a, b }, R, S)
S→aTXb, T→XTS | ε, X→ a | b
• Eliminating λ-production:
S → aTXb | aXb
T → XTS | XS
X→a|b

Department of Information Technology

PSG College of Technology 52
 Conversion of CFG to GNF
G = ({ S, T, X }, { a, b }, R, S)
S→aTXb | aXb, T→XTS | XS, X→ a | b

X→a|b
T → aTS| bTS | aS | bS
S → aTXb | aXb
P→b
S → aTXP | aXP
Department of Information Technology
PSG College of Technology 53
 GNF
G = ({ S, T, X }, { a, b }, R, S)
S→aTXb | aXb, T→XTS | XS, X→ a | b

S → aTXP | aXP
T → aTS| bTS | aS | bS
X→a|b
P→b

Department of Information Technology

PSG College of Technology 54
 Prerequisites
• PDA: A PDA can be formally described as a 7-
tuple (Q, ∑, S, δ, q0, z, F) −
Q is the finite number of states
∑ is input alphabet
S is stack symbols
δ is the transition function: Q ×(∑ ∪ {ε}) S × Q × S*
q0 is the initial state (q0 ∈ Q)
z is the initial stack top symbol (I ∈ S)
F is a set of accepting states (F ∈ Q)

Department of Information Technology

PSG College of Technology 55
 GNF to PDA
• Definition: Let G = (VN, VT, P, S) be a CFL. If every production
in P is of the form A –> aα
Where A is in VN, a is in VT, and α is in VN*, then G is said to be
in Greibach Normal Form (GNF).
• Example:
S –> aAB | bB
A –> aA | a
B –> bB | c
• Theorem: Let L be a CFL. Then L – {λ} is a CFL.
• Theorem: Let L be a CFL not containing {λ}. Then there exists
a GNF grammar G such that L = L(G).

Department of Information Technology

PSG College of Technology 56
• Lemma 1: Let L be a CFL. Then there exists a PDA M such that L = LE(M).
• Proof: Assume without loss of generality that ε is not in L. The construction
can be modified to include ε later.
Let G = (V, T, P, S) be a CFG, and assume without loss of generality that G
is in GNF. Construct M = (Q, Σ, Г, δ, q, z, Ø) where:
Q = {q}
Σ=T
Г=V
z=S
δ: for all a in Σ and A in Г, δ(q, a, A) contains (q, γ) if A –> aγ is in P or rather:
δ(q, a, A) = {(q, γ) | A –> aγ is in P and γ is in Г*}, for all a in Σ and A in Г
• For a given string x in Σ* , M will attempt to simulate a leftmost derivation of
x with G.

Department of Information Technology

PSG College of Technology 57
 PDA
S → aTXP | aXP,
T→aTS|bTS|aS|bS, X→a|b, P→b

δ(𝑞0 , λ, 𝑧0 )=(𝑞0 ,S𝑧0 )

δ(𝑞1 ,a, S)=(𝑞1 ,TXP) (𝑞1 ,XP)
δ(𝑞1 ,a, T) =(𝑞1 ,TS)(𝑞1 ,S)
δ(𝑞1 ,b, T) =(𝑞1 ,TS)(𝑞1 ,S)
δ(𝑞1 ,a, X) =(𝑞1 , λ)
δ(𝑞1 ,b, X) =(𝑞1 , λ)

Department of Information Technology

PSG College of Technology 58
• Exercise 1:
• Consider the following CFG in GNF.
S –> aS G is in GNF
S –> a L(G) = a+
Construct M as:
Q = {q}
Σ = T = {a}
Г = V = {S}
z=S
δ(q, a, S) = {(q, S), (q, ε)}
δ(q, ε, S) = Ø
• Question: Is that all? Is δ complete? Recall that δ: Q x (Σ U {ε}) x Г –> finite subsets of Q x Г*

Department of Information Technology

PSG College of Technology 59
• Exercise 2:.
Consider the following CFG in GNF
(1) S  aA
(2) S  aB
(3) A  aA
(4) A  aB
(5) B  bB
(6) B  b
Construct M as: Q = {q} Σ = T = {a, b} Г = V = {S, A, B} z = S
(1) δ(q, a, S) = {(q, A), (q, B) }From productions #1 and 2, SaA, S  aB
(2) δ(q, a, A) = {(q, A), (q, B)}From productions #3 and 4, AaA, AaB
(3) δ(q, a, B) = Ø
(4) δ(q, b, S) = Ø
(5) δ(q, b, A) = Ø
(6) δ(q, b, B) = {(q, B), (q, ε)} From productions #5 and 6, BbB, Bb
(7) δ(q, ε, S) = Ø
(8) δ(q, ε, A) = Ø
(9) δ(q, ε, B) = Ø Recall δ: Q x (Σ U {ε}) x Г –> finite subsets of Q x Г*
Department of Information Technology
PSG College of Technology 60
 • Tracing the PDA for the string w = aaaabb
S => aA (1)
=> aaA (3)
=> aaaA (3)
=> aaaaB (4)
=> aaaabB (5)
=> aaaabb (6)
• The corresponding computation of the PDA:
• q, aaaabb, S) |— (q, aaabb, A) (1)/1
|— (q, aabb, A) (2)/1
|— (q, abb, A) (2)/1
|— (q, bb, B) (2)/2
|— (q, b, B) (6)/1
|— (q, ε, ε) (6)/2
– String is read
– Stack is emptied
– Therefore the string is accepted by the PDA

Department of Information Technology

PSG College of Technology 61
• Example 2:
• Consider the following CFG in GNF.
(1) S  aABC (2) A  a (3) B  b (4) C  cAB (5) C  cC
Construct M as: Q = {q} Σ = T = {a, b, c} Г = V = {S, A, B, C} z = S
(1) δ(q, a, S) = {(q, ABC)} S  aABC (9) δ(q, c, S) = Ø
(2) δ(q, a, A) = {(q, ε)} Aa (10) δ(q, c, A) = Ø
(3) δ(q, a, B) = Ø (11) δ(q, c, B) = Ø
(4) δ(q, a, C) = Ø (12)δ(q, c, C) = {(q, AB), (q, C))} // C->cAB|cC
(5) δ(q, b, S) = Ø (13) δ(q, ε, S) = Ø
(6) δ(q, b, A) = Ø (14) δ(q, ε, A) = Ø
(7) δ(q, b, B) = {(q, ε)} Bb (15) δ(q, ε, B) = Ø
(8) δ(q, b, C) = Ø (16) δ(q, ε, C) = Ø

Department of Information Technology

PSG College of Technology 62
 CYK algorithm
• Cocke-Younger-Kasami algorithm - To
recognise CF language
• To prove membership of strings to CFL
• To construct a possible parse tree.
• Can also process Stochastic CFG, where
probabilities a stored in a table.
• Asymptotic time complexity is θ(n3),
where, n -- strlen
Department of Information Technology
PSG College of Technology 63
I/P string a 1 ... a n of length ‘n’. G has ‘r’ terminals.
Grammar has nonterminals R 1 ... R r & R 1 is start symbol.
P[n n r] is an array of booleans initialized to false.
For each i = 1 to n
For each unit production R j → a i set P[i 1 j] = true.
For each i = 2 to n -- Length of span
For each j = 1 to n-i+1 -- Start of span
For each k = 1 to i-1 -- Partition of span
For each production R A -> R B R C
if P[j k B] and P[j+k i-k C]
then set P[j i A] = true
if P[1 n 1] is true
then string is member of language
else string is not member of language
Department of Information Technology
PSG College of Technology 64
 CYK Parsers
i/p str: w = w1w2 w3……wn;
wij = wi wi+1……wj and Vij = { A ε V | A * wij }
W belongs to L iff S ε V1n.
Vii. Is found by examining RHS of rules

O(n3) – n is string length

Department of Information Technology

PSG College of Technology 65
S  AB
A  BB | a
B  AB | b
To generate string aabbb

Department of Information Technology

PSG College of Technology 66
 String: aaabbb

Department of Information Technology

PSG College of Technology 67
 String : baaa

Department of Information Technology

PSG College of Technology 68
 Summary

Basic definition on Normal forms

Solved examples in CNF and GNF

Department of Information Technology

PSG College of Technology 69
PUSHDOWN AUTOMATA

Department of Information Technology

3/14/2020
PSG College of Technology 70
 Equivalence of CFG and PDA
• Theorem: For a language L ⊆ ∑∗ the
following two statements are equivalent:
• L is given by a CFG G, L = L(G).
• L is the language of a PDA P, L = L(P).

Department of Information Technology

PSG College of Technology 71
• Context-Free Languages (CFLs) can be
described by a Context Free Grammar
(CFG) and can be processed by a
pushdown automaton.

Department of Information Technology

PSG College of Technology 72
 Procedure 1 for CFG to PDA

• Given a CFG G = (V, ∑, S, P), we define a PDA

P(G) = ({q0}, ∑, V ∪ ∑, δ, q0, S)
where δ is defined as follows:
• δ(q0, ∈, A) = {(q0, α) | A → α ∈ P} for all A ∈ V
• δ(q0, a, a) = {(q0, ∈)} for all a ∈ ∑.

Department of Information Technology

PSG College of Technology 73
 Example:
• G = ({E, T, F}, {(, ), a, +, *}, E, P)
where the set of productions P are given
by
• E→T|E+T
• T→F|T*F
• F → a | (E )

Department of Information Technology

PSG College of Technology 74
• we define P(G) = ({q0}, {(, ), a, +, *}, {E, T, F, (, ),
a, +, *}, δ, q0, E) where
• δ(q0, ∈,E) = {(q0, T), (q0,E+T)}
• δ(q0, ∈, T) = {(q0, F), (q0, T*F)}
• δ(q0, ∈, F) = {(q0, a), (q0, (E))}
• δ(q0, (, () = {(q0, ∈)}
• δ(q0, ), )) = {(q0, ∈)}
• δ(q0, a, a) = {(q0, ∈)}
• δ(q0, +, +) = {(q0, ∈)}
• δ(q0, *, *) = {(q0, ∈)}
• δ(q, x, z) = ∅

Department of Information Technology

PSG College of Technology 75
 Trace for the String w = a+(a*a)
(q0, a+(a*a), E) ⊢ (q0, ⊢ (q0, a*a), T))
a+(a*a), E+T) ⊢ (q0, a*a), T*F))
⊢ (q0, a+(a*a), T+T) ⊢ (q0, a*a), F*F))
⊢ (q0, a+(a*a), F+T) ⊢ (q0, a*a), a*F))
⊢ (q0, *a), *F))
⊢ (q0, a+(a*a), a+T)
⊢ (q0, a), F))
⊢ (q0, +(a*a), +T)
⊢ (q0, a), a))
⊢ (q0, (a*a), T) ⊢ (q0, ), ))
⊢ (q0, (a*a), F) ⊢ (q0, ∈, ∈ )
⊢ (q0 (a*a), (E))
⊢ (q0, a*a), E)) Hence a+(a*a) ∈ L(P(G)).

Department of Information Technology

PSG College of Technology 76
 • w ∈ L(G) ⇐⇒ S ∗⇒ w ⇐⇒ (q0,w, S)∗⊢ (q0, ∈, ∈)
⇐⇒ w ∈ L(P(G))
• Shows Non – Deterministic Pushdown Automata

Department of Information Technology

PSG College of Technology 77
 Example 2:
0 (p, ε, ε), (q, S)
L = {anbmcpdq : m + n = p + q}
1 (q, ε, S), (q, aSd)
(1) S → aSd
2 (q, ε, S), (q,T)
(2) S → T
3 (q, ε, S), (q,U)
(3) S → U
4 (q, ε, T), (q, aTc)
(4) T → aTc
5 (q, ε, T), (q, V)
(5) T → V
6 (q, ε, U), (q, bUd)
(6) U → bUd
7 (q, ε, U), (q, V)
(7) U → V
8 (q, ε, V), (q, bVc)
(8) V → bVc
9 (q, ε, V), (q, ε)
(9) V → ε
10 (q, a, a), (q, ε)
input = a a b c d d 11 (q, b, b), (q, ε)
12 (q, c, c), (q, ε)
13 (q, d, d), (q, ε)
Department of Information Technology
PSG College of Technology 78
 GNF to PDA
• Definition: Let G = (V, T, P, S) be a CFL. If every production in P is of
the form
A –> aα
Where A is in V, a is in T, and α is in V*, then G is said to be in Greibach
Normal Form (GNF).
• Example:
S –> aAB | bB
A –> aA | a
B –> bB | c
• Theorem: Let L be a CFL. Then L – {ε} is a CFL.

• Theorem: Let L be a CFL not containing {ε}. Then there exists a GNF
grammar G such that L = L(G).

Department of Information Technology

PSG College of Technology 79
• Lemma 1: Let L be a CFL. Then there exists a PDA M such that L = LE(M).
• Proof: Assume without loss of generality that ε is not in L. The construction can
be modified to include ε later.
Let G = (V, T, P, S) be a CFG, and assume without loss of generality that G is in
GNF. Construct M = (Q, Σ, Г, δ, q, z, Ø) where:
Q = {q}
Σ=T
Г=V
z=S
δ: for all a in Σ and A in Г, δ(q, a, A) contains (q, γ) if A –> aγ is in P or rather:
δ(q, a, A) = {(q, γ) | A –> aγ is in P and γ is in Г*}, for all a in Σ and A in Г
• For a given string x in Σ* , M will attempt to simulate a leftmost derivation of x
with G.

Department of Information Technology

PSG College of Technology 80
 • Example 1:
• Consider the following CFG in GNF.
S –> aS G is in GNF
S –> a L(G) = a+
Construct M as:
Q = {q}
Σ = T = {a}
Г = V = {S}
z=S
δ(q, a, S) = {(q, S), (q, ε)}
δ(q, ε, S) = Ø
• Question: Is that all? Is δ complete? Recall that δ: Q x (Σ U {ε}) x Г –>
finite subsets of Q x Г*

Department of Information Technology

PSG College of Technology 81
• Example 2:.
Consider the following CFG in GNF
(1) S  aA
(2) S  aB
(3) A  aA
(4) A  aB
(5) B  bB
(6) B  b
Construct M as: Q = {q} Σ = T = {a, b} Г = V = {S, A, B} z = S
(1) δ(q, a, S) = {(q, A), (q, B) }From productions #1 and 2, SaA, S  aB
(2) δ(q, a, A) = {(q, A), (q, B)}From productions #3 and 4, AaA, AaB
(3) δ(q, a, B) = Ø
(4) δ(q, b, S) = Ø
(5) δ(q, b, A) = Ø
(6) δ(q, b, B) = {(q, B), (q, ε)} From productions #5 and 6, BbB, Bb
(7) δ(q, ε, S) = Ø
(8) δ(q, ε, A) = Ø
(9) δ(q, ε, B) = Ø Recall δ: Q x (Σ U {ε}) x Г –> finite subsets of Q x Г*
Department of Information Technology
PSG College of Technology 82
 • Tracing the PDA for the string w = aaaabb
S => aA (1)
=> aaA (3)
=> aaaA (3)
=> aaaaB (4)
=> aaaabB (5)
=> aaaabb (6)
• The corresponding computation of the PDA:
• q, aaaabb, S) |— (q, aaabb, A) (1)/1
|— (q, aabb, A) (2)/1
|— (q, abb, A) (2)/1
|— (q, bb, B) (2)/2
|— (q, b, B) (6)/1
|— (q, ε, ε) (6)/2
– String is read
– Stack is emptied
– Therefore the string is accepted by the PDA

Department of Information Technology

PSG College of Technology 83
 • Example 2:
• Consider the following CFG in GNF.
(1) S  aABC
(2) A  a
(3) B  b
(4) C  cAB
(5) C  cC
Construct M as: Q = {q} Σ = T = {a, b, c} Г = V = {S, A, B, C} z
=S
(1) δ(q, a, S) = {(q, ABC)} S  aABC (9) δ(q, c, S) = Ø
(2) δ(q, a, A) = {(q, ε)} Aa (10) δ(q, c, A) = Ø
(3) δ(q, a, B) = Ø (11) δ(q, c, B) = Ø
(4) δ(q, a, C) = Ø (12)δ(q, c, C) = {(q, AB), (q, C))} // C->cAB|cC
(5) δ(q, b, S) = Ø (13) δ(q, ε, S) = Ø
(6) δ(q, b, A) = Ø (14) δ(q, ε, A) = Ø
(7) δ(q, b, B) = {(q, ε)} Bb (15) δ(q, ε, B) = Ø
(8) δ(q, b, C) = Ø (16) δ(q, ε, C) = Ø

Department of Information Technology

PSG College of Technology 84
 Exercises

• Generate PDA for the the following grammar and

Trace for the string w = 00111
– S  AS | 
– A  0A1 | A1 | 01
• Consider the following grammar G with
productions
S → 0B / 1A A → 0 / 0S / 1AA B → 1 / 1S / 0BB
Trace for the sentence w = 00110101

Department of Information Technology

PSG College of Technology 85
 TURING MACHINE

Department of Information Technology

3/14/2020
PSG College of Technology 86
 Structure of TM
Finite set of states

CONTROL Start state

UNIT
A single halt state

TAPE Move left or right one cell

HEAD at a time

TAPE

Store input for the TM

Can be of any length
Can read from and write on tape
87
Department of Information Technology
PSG College of Technology 87
 What does a TM do?

Determine if an input x is in a
language.
• That is, answer if the answer of a problem P
for the instance x is “yes”. ACCEPTOR

Compute a function
• Given an input x, what is f(x)? TRANSDUCER

Department of Information Technology

PSG College of Technology 88
 How does a TM work?

At the beginning,
• A TM is in the start state (initial
state)
• its tape head points at the first cell
• The tape contains , following by
input string, and the rest of the
tape contains .
Department of Information Technology
JarulojPSG
Chongstitvatana 2301379
College of Technology 89
 How does a TM work?

For each move, a TM

• reads the symbol under its tape head
• According to the transition function on the
symbol read from the tape and its current
state, the TM:
• write a symbol on the tape
• move its tape head to the left or right one
cell or not
• changes its state to the next state
Department of Information Technology
PSG College of Technology 90
When does a TM stop working?

A TM stops working,
• when it gets into the special state
called halt state. (halts)
• The output of the TM is on the tape.
• when the tape head is on the leftmost
cell and is moved to the left. (hangs)
• when there is no next state. (hangs)
Department of Information Technology
PSG College of Technology 91
THE ORIGIN OF TURING MACHINE
But Turing’s a-machine became famous for
something else other than its original intentions.

The a-machine was found to be more expressive

than CFGs. It can recognize strings that
cannot be modeled by CFLs like anbncn.

The a-machines came to be more popularly known

as Turing Machines.

Department of Information Technology

PSG College of Technology 92
 TURING MACHINE
 In 1936 Alan Turing formulated an abstract machine that is a model of
computation.

 He attempted to isolate the primitive steps into which computations could

be decomposed.

 By defining a machine capable of the same basic steps, he was able to

argue that any conceivable algorithm could be carried out by his
machine.

 Any algorithmic procedure that can be carried out by a human, a team of

humans or a computer can be carried out by some TM. This is known as
Church-Turing thesis.
 Department of Information Technology
93
PSG College of Technology
A Turing machine consists of two parts

• 1. a head which can read or write a symbol and can move left or right or
stay in position, corresponding to square cells marked-off on a tape
• 2.an infinite tape, extending on either side of the head, marked-off into
square cells, on which symbols from an alphabet can be written.
• Simple operations must include
• i. changes of the symbol on one of the observed squares
• ii. Changes of one of the squares observed to another square
• a. A possible change of symbol together with a possible change of state of
mind
• b. A possible change of observed squares together with a possible change
of state of mind.
Department of Information Technology
PSG College of Technology 94
 Tape String

b a c n r
Blank tape Blank
Tape
Read/Write Head

States
(α, β, γ,…)

Turing Machine
Department of Information Technology
PSG College of Technology 95
 Definition of Turing Machine
A Turing Machine is a 5-tuple T = (Q, ∑, Γ, q0, δ)
Q A finite set of states excluding halt state {h}
∑ A finite set of input alphabet
Γ A finite set of tape alphabet
∑ Ϲ Γ Γ is assumed not to contain blank {Δ}
q0 Initial state is an element of Q
δ δ : Q X (Γ U {Δ}) → (Q U {h}) X (Γ U {Δ}) X (R, L, S) is a partial
function (that is possibly undefined at certain points)
The tape may also be viewed as having a left end.
Each square of the tape contains a single symbol of Γ U {Δ}
If q є Q, r є Q U {h} X, Y є Γ U {Δ} and D є {R, L, S} the computation
step is
δ(q, X) = (r, Y, D)
If the tape is at left end and D = L then the TM crashes.
If r = h then TM halts because δ (h, X) is not defined.

Department of Information Technology

PSG College of Technology 96
To describe the status of a TM at some point during
computation, we must specify the state in which the
machine is in, the contents of the tape, and the
current tape head position.
Configuration of the TM is (q, xay)
where q is a state, x and y are strings over Γ U {Δ},
a Γ U {Δ}, and the underscore symbol represents
the current position of the tape head.
A move is given by (q, xay)| (r, zbw)
(q, xay)|---*T (r, zbw) zero or more steps.

Department of Information Technology

PSG College of Technology 97
 Deciders
• A Turing machine is called a decider if
every string in * is either accepted or
rejected
• A language is called Turing-decidable if
some Turing machine decides it
– These languages are often just called
decidable

Department
3/14/2020 of Information Technology
PSG College of Technology 98
 The Church-Turing Hypothesis
Everything computable is computable by a Turing machine
– Physical interpretation
– Conceptual interpretation
– Definition of Computation
– Everything computable is computable by a Java program
– All “good enough” computers are equivalent

ANY "COMPUTABLE" FUNCTION IS A PARTIAL RECURSIVE

FUNCTION.
══
ANY "COMPUTABLE" FUNCTION HAS A TM TO COMPUTE IT

Partial recursive functions are the only computable functions, and these
are the functions computable by TMs.

Department
3/14/2020 of Information Technology
PSG College of Technology 99
The Turing machine: Computability
What cannot be computed by TMs?
• There are only countably many TM
machines -> there exist uncountably many
real numbers that are not computable
• Uncomputable functions, e.g. Halting
Problem

Department
3/14/2020 of Information Technology
PSG College of Technology 100
 Equivalent TM Models
Many variations on TMs are equivalent.
1. Stay option. Read-write can stay at current
square.
2. Semi-infinite tape. Tape is infinite to the right,
but not the left.
3. Off-line TM. Two tapes, one for input, and the
other for working storage.
4. Multitape TMs. TM has a finite number of tapes.
5. Multidimensional TMs. Tape is two-dimensional
like a matrix.
6. Non-deterministic TM
7. Universal TM
Department
3/14/2020 of Information Technology
PSG College of Technology 101
 Robustness
Other models of TM like, k-tape TMs,
nondeterministic Turing machines are not more
powerful than simple TMs:
Every nondeterministic TM has an equivalent
3-tape Turing machine, which –in turn– has an
equivalent 1-tape Turing machine.

The Turing machine model is extremely robust.

Department of Information Technology

3/14/2020
PSG College of Technology 102
 1. TM with stay option
• r/w head remains in the same position after
rewriting

The transitions defined by δ N can be partitioned into

two subsets:
(1) those which move the r/w head to left or to the right
and
(2) those which keep the r/w head at the same cell.

Transitions in the first subset is automatically included in

M of Turing Machines
Department of Information Technology
3/14/2020
PSG College of Technology 103
Transitions in the second subset, such as
ε(q; a) = (p; b; S), is replaced with two transitions:
1. Transition 1 rewrites the cell and moves the
r/w head to the right;
2. Transition 2 simply moves the r/w head to the
left.

Department of Information Technology

3/14/2020
PSG College of Technology 104
 2. TM with multiple tracks
• Each storage cell is composite and has sub-
cells.
• Makes designing TM easier.
• Does not increase the computational power.

Department of Information Technology

3/14/2020
PSG College of Technology 105
 3. TM with semi-infinite
• Bound on left but not on right

The state of TM contains two components:

1)The first indicates the upper or lower track,
2)The second indicates the current state.
When N is in state p and the r/w head is on the right of the
breaking point,
TM would be in state (up; p). Similarly, when N is in state q and
the r/w head is on the left of the breaking point, M would be in
state (down; q).
Department of Information Technology
3/14/2020
PSG College of Technology 106
 4. Offline TM
• In a standard Turing machine, the input is
the initial tape contents.
• When we put the input file back, a move
is governed by the current internal state,
the current symbol read from the input
file and the current symbol read from the
tape.

Department of Information Technology

3/14/2020
PSG College of Technology 107
• It simulates TM by copying file contents to the
Tape and proceeding as TM.
• A 4-track TM can also simulate OL TM.
– The 1st track is for the input file.
– The 2nd track indicates the current position of the input
file.
– The 3rd track is for the tape.
– The 4th track indicates the current position of the tape.

Department of Information Technology

3/14/2020
PSG College of Technology 108
 5. TM with complex storage
– Multi-tape TM
• The TM has more than one tape, each with an
independent r/w head.

Department of Information Technology

3/14/2020
PSG College of Technology 109
 Department of Information Technology
3/14/2020
PSG College of Technology 110
 6. Multidimensional TM
• A standard TM’s tape is 1-dimensional in that
it can only extend to the left and to the right.
• A 2-dimensional tape is viewed as a grid plain.
• A TM equipped with a 2-dimensional tape still
has 1 r/w head. This r/w head can move left,
right, up, or down.

Department of Information Technology

3/14/2020
PSG College of Technology 111
 Department of Information Technology
3/14/2020
PSG College of Technology 112
 7. Non deterministic TM
• A nondeterministic Turing machine (NTM) is identical
to a standard Turing machine except that the transition
function is

• A standard TM can be used to simulate a NTM.

Replication is used - When more than one move is
possible, the machine simply makes as many
replicas as needed and each replica tries out a
possible move.
• A mechanical machine cannot replicate itself.
Department of Information Technology
3/14/2020
PSG College of Technology 113
 8. Universal TM
• A Turing machine can carry out one
specific task.
• A digital computer can be programmed to
perform many different tasks at different
times.
• UTM is programmable
• A UTM’s input is the description of a TM M
and an input string w.
• It can determine if M accepts w.
Department
3/14/2020 of Information Technology
PSG College of Technology 114
• A universal TM is a 3-tape TM:
– The 1st tape is the encoding of a TM M.
– The 2nd is the tape used by M.
– The 3rd is the encoding of the internal state of
M.

Department of Information Technology

3/14/2020
PSG College of Technology 115
• The UTM executes in cycles, similar to the
instruction cycles of a digital computer.
– It first reads the contents of the 2nd and 3rd
tapes to determine the current configuration of
M.
– Then it consults the 1st tape to find an
applicable transition.
– The UTM performs the transition by changing
the 2nd and 3rd tapes.

Department
3/14/2020 of Information Technology
PSG College of Technology 116
 References
John E.Hopcroft, Rajeev Motwani, Jeffery D
Ullman, “Introduction to automata theory,
languages and computation”, Pearson, 2012

John C. Martin,”Introduction to languages

and the theory of computation”, McGraw
Hill, 2012

Peter Linz,”An Introdution to formal

languages and automata”,
Jones&Barlett,2011

Department of Information Technology

PSG College of Technology 117
Department of Information Technology
PSG College of Technology 118