Theory of Automata

&
Formal Languages

1
 BOOKS
Theory of computer Science: K.L.P.Mishra &
N.Chandrasekharan

Intro to Automata theory, Formal languages and

computation: Ullman,Hopcroft
Motwani

Elements of theory of computation Lewis &

papadimitrou

2
 Syllabus
Introduction
Deterministic and non deterministic Finite
Automata, Regular Expression,Two way
finite automata,Finite automata with
output,properties of regular sets,pumping
lemma, closure properties,Myhill nerode
theorem

3
Context free Grammar: Derivation trees,
Simplification forms
Pushdown automata: Def, Relationship
between PDA and context free
language,Properties, decision algorithms
Turing Machines: Turing machine
model,Modification of turing
machines,Church’s
thesis,Undecidability,Recursive and
recursively enumerable languages Post
correspondence problems recursive
functions
4
Chomsky Hierarchy: Regular grammars,
unrestricted grammar, context sensitive
language, relationship among languages

5
6
7
 Finite Automaton
Input
• String

Output
Finite String
Automaton

8
 Finite Accepter
Input
• String
Output
“Accept”
Finite
or
Automaton
“Reject”

9
 Transition Graph
a, b
Abba -Finite Accepter
•
q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

initial final
state state
transition
state “accept”
10
 Initial Configuration
Input String
a b b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

11
 Reading the Input
a b b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

12
a b b a

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

13
a b b a

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

14
a b b a

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

15
 Input finished

a b b a

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

Output: “accept”16
 Rejection
a b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

17
a b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

18
a b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

19
a b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

20
 Input finished

a b a

a, b
Output:
q5 “reject”
a a, b
b a b
q0 a q1 b q2 b q3 a q4

21
 Another Example
a a b

a a,b

q0 b q1 a, b q2

22
a a b

a a,b

q0 b q1 a, b q2

23
a a b

a a,b

q0 b q1 a, b q2

24
a a b

a a,b

q0 b q1 a, b q2

25
 Input finished

a a b

a a,b
Output: “accept”

q0 b q1 a, b q2

26
 Rejection
b a b

a a,b

q0 b q1 a, b q2

27
b a b

a a,b

q0 b q1 a, b q2

28
b a b

a a,b

q0 b q1 a, b q2

29
b a b

a a,b

q0 b q1 a, b q2

30
 Input finished

b a b

a a,b

q0 b q1 a, b q2

Output: “reject”
31
• Deterministic Finite Accepter (DFA)

M   Q, ,  , q0 , F 

Q : Finite set of states

 : input alphabet
 : transition function  : Q X Q
q0 : initial state is a member of Q
F : set of final states 32
 Input Alphabet 
   a, b

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
33
 Set of States Q
Q   q0 , q1, q2 , q3 , q4 , q5 
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

34
 Initial State q0

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

35
 Set of Final States F
F   q4 

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

36
 Transition Function 
 :Q  Q

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
37
   q0 , a   q1

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
38
   q0 , b   q5

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
39
   q2 , b   q3

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
40
 a b Transition Function 
q0 q1 q5
q1• q5 q2
q2 q2 q3
q3 q4 q5 a, b
q4 q5 q5
q5 q5 q5 q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
41
Extended Transition Function  *

 * : Q  *  Q

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
42
  *  q0 , ab   q2

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
43
  *  q0 , abba   q4

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
44
  *  q0 , abbbaa   q5

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
45
Observation: There is a walk from q0 to q5
with label abbbaa

 *  q0 , abbbaa   q5
a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
46
 Recursive Definition
 *  q,    q
•  *  q, wa    ( * (q, w), a )
a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
47
 *  q0 , ab  
   * (q0 , a ), b  
     *  q0 ,  , a , b  
    q0 , a , b  
  q1 , b  
q2 a , b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
48
 Languages Accepted by DFAs
• Take DFA M

• Definition:
– The language L M  contains
– all input strings accepted by M

L M–  = { strings that drive M to a final state}

49
 Example
L M    abba M
•
a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
accept
50
 Another Example
L M     , ab, abba M
•
a, b

q5
b a a a, b
b
q0 a q1 b q2 b q3 a q4
accept accept accept
51
 Formally

• For a DFA Language accepted by M :

L M    w  * :  *  q0 , w  F 
M   Q, ,  , q0 , F 
alphabet transition initial final
function state states
52
 Observation
• Language accepted by M:
L M    w  * :  *  q0 , w  F 

• Language rejected by M :
L M    w  * :  *  q0 , w  F 
53
• More Examples
L M   {a b : n  0}
n

a a,b

q0 b q1 a, b q2

accept trap state

or dead state
54
L M  = { all strings with prefix ab }
• a, b

q0 a q1 b q2

b a accept

q3 a, b
55
 L  M  = { all strings without
substring 001 }
•

1 0 0,1
1

 0 1
0 00 001

0
56
 Regular Languages

• A language L is regular if there is

• a DFA M such that L  L M 

• All regular languages form a language

family
–
57
 Example
L   awa : w   a, b *
a
• The language b
• is regular:
b
q0 a q2 q3
•
b a
q4

a, b 58
Non Deterministic
Automata

59
 Non Deterministic Finite
Accepter

M   Q, ,  , q0 , F 

 :Q  2 Q

60
Nondeterministic Finite Accepter (NFA)

Alphabet = {a}

q1 a q2
a
q0
a
q3

61
Nondeterministic Finite Accepter (NFA)

Alphabet = {a}

Two choices q1 a q2
a
q0
a
q3

62
Nondeterministic Finite Accepter (NFA)

Alphabet = {a}

Two choices q1 a q2 No transition

a
q0
a
q3 No transition

63
 First Choice

a a

q1 a q2
a
q0
a
q3

64
 First Choice

a a

q1 a q2
a
q0
a
q3

65
 First Choice

a a

q1 a q2
a
q0
a
q3

66
 First Choice

a a
All input is consumed
q1 a q2 “accept”
a
q0
a
q3

67
 Second Choice

a a

q1 a q2
a
q0
a
q3

68
 Second Choice

a a

q1 a q2
a
q0
a
q3

69
 Second Choice

a a

q1 a q2
a
q0
a
No transition:
q3
the automaton hangs
70
 Second Choice

a a
Input cannot be consumed
q1 a q2
a
q0
a
q3 “reject”

71
An NFA accepts a string:

when there is a computation of the NFA

that accepts the string

•All the input is consumed and the automaton

is in a final state

72
An NFA rejects a string:

when there is no computation of the NFA

that accepts the string

• All the input is consumed and the

automaton is in a non final state

• The input cannot be consumed

73
 Example

aa is accepted by the NFA:

“accept”
q1 a q2 q1 a q2
a a
q0
a
q0
a
q3 q3 “reject”
because this computation
accepts aa 74
 Rejection example

q1 a q2
a
q0
a
q3

75
 First Choice

q1 a q2
a
q0
a
q3

76
 First Choice

a
“reject”
q1 a q2
a
q0
a
q3

77
 Second Choice

q1 a q2
a
q0
a
q3

78
 Second Choice

q1 a q2
a
q0
a
q3

79
 Second Choice

q1 a q2
a
q0
a
q3 “reject”

80
 Example
a is rejected by the NFA:
“reject”
q1 a q2 q1 a q2
a a
q0 q0
a a
q3
“reject” q3

All possible computations lead to rejection

81
 Rejection example

a a a

q1 a q2
a
q0
a
q3

82
 First Choice

a a a

q1 a q2
a
q0
a
q3

83
 First Choice

a a a

q1 a q2
a
q0 No transition:
a
the automaton hangs
q3

84
 First Choice

a a a
Input cannot be consumed
q1 a q2 “reject”
a
q0
a
q3

85
 Second Choice

a a a

q1 a q2
a
q0
a
q3

86
 Second Choice

a a a

q1 a q2
a
q0
a
q3

87
 Second Choice

a a a

q1 a q2
a
q0
a
No transition:
q3
the automaton hangs
88
 Second Choice

a a a
Input cannot be consumed
q1 a q2
a
q0
a
q3 “reject”

89
aaa is rejected by the NFA:

“reject”
q1 a q2 q1 a q2
a a
q0
a
q0
a
q3 q3 “reject”

All possible computations lead to rejection

90
Language accepted: L  {aa}

q1 a q2
a
q0
a
q3

91
 Lambda →Transitions

q0 a q1  q2 a q3

92
a a

q0 a q1  q2 a q3

93
a a

q0 a q1  q2 a q3

94
 (read head doesn’t move)

a a

q0 a q1  q2 a q3

95
a a

q0 a q1  q2 a q3

96
 all input is consumed

a a

“accept”

q0 a q1  q2 a q3

String aa is accepted 97
 Rejection Example

a a a

q0 a q1  q2 a q3

98
a a a

q0 a q1  q2 a q3

99
 (read head doesn’t move)

a a a

q0 a q1  q2 a q3

100
a a a

q0 a q1  q2 a q3

No transition:
the automaton hangs
101
Input cannot be consumed

a a a

“reject”

q0 a q1  q2 a q3

String aaa is rejected 102

Language accepted: L  {aa}

q0 a q1  q2 a q3

103
 Another NFA Example

q0 a q1 b q2  q3


104
a b

q0 a q1 b q2  q3


105
a b

q0 a q1 b q2  q3


106
a b

q0 a q1 b q2  q3


107
a b

“accept”
q0 a q1 b q2  q3


108
 Another String

a b a b

q0 a q1 b q2  q3


109
a b a b

q0 a q1 b q2  q3


110
a b a b

q0 a q1 b q2  q3


111
a b a b

q0 a q1 b q2  q3


112
a b a b

q0 a q1 b q2  q3


113
a b a b

q0 a q1 b q2  q3


114
a b a b

q0 a q1 b q2  q3


115
a b a b

“accept”
q0 a q1 b q2  q3


116
 Language accepted

L   ab, abab, ababab, ...


  ab

q0 a q1 b q2  q3


117
Another NFA Example

0
q0 q1 0, 1 q2
1

118
 Language accepted

L(M ) = { λ, 10, 1010, 101010, ...}

= { 10} *
0
q0 q1 0, 1 q2
1

119
Remarks:
•The  symbol never appears on the
input tape

•Extreme automata:

M1 M2
q0 q0

L(M1 ) = {} L(M 2 ) = {λ}

120
•NFAs are interesting because we can
express languages easier than DFAs

a,b
NFA M1 DFA M2
q2
q0 a q1 b
a,b

q0 a q1

L( M1 ) = {a} L( M 2 ) = {a} 121

 Formal Definition of NFAs
M   Q, ,  , q0 , F 
Q : Set of states, i.e.  q0 , q1, q2 
 : Input alphabet, i.e.  a, b
: Transition function

q0 : Initial state

F : Final states 122

Transition Function 

  q0 , 1   q1

0
q0 q1 0, 1 q
2
1

123
  (q1,0)  {q0 , q2}

0
q0 q1 0, 1 q
2
1

124
  (q0 ,  )  {q0 , q2}

0
q0 q1 0, 1 q
2
1

125
  (q2 ,1)  

0
q0 q1 0, 1 q
2
1

126
 Extended Transition Function  *
 *  q0 , a    q1
•

q4 q5
a a
q0 a q1 b q2  q3
 127
  *  q0 , aa    q4 , q5 

q4 q5
a a
q0 a q1 b q2  q3
 128
  *  q0 , ab    q2 , q3 , q0 

q4 q5
a a
q0 a q1 b q2  q3
 129
 Formally
q j   *  qi , w

It holds
if and only if

there is a walk from qi to q j

with label w
130
 The Language of an NFA M
F   q0 ,q5 
q4 q5
• a a
q0 a q1 b q2  q3


 *  q0 , aa    q4 , q5 aa  L(M )
131
 F   q0 ,q5 
q4 q5
a a
q0 a q1 b q2  q3


 *  q0 , ab    q2 , q3 , q0  ab  L M 
132
F   q0 ,q5 
q4 q5
• a a
q0 a q1 b q2  q3


 *  q0 , abaa    q4 , q5 aaba  L(M )

133
F   q0 ,q5 
q4 q5
a a
q0 a q1 b q2  q3


 *  q0 , aba    q1 aba  L M 

134
 q4 q5
a a
q0 a q1 b q2  q3



L M    aa   ab *   ab  aa
135
 Formally

• The language accepted by NFA is:

M
L M    w1, w2 , w3 ,...
 * (q0 , wm )  {qi , q j ,...}
• where

• and there is some qk  F (final state)

136
w L M   * (q0 , w)
• qi
w

q0
qk qk  F
w
w qj

137
Equivalence of NFAs and DFAs

138
 Equivalence of Machines

• For DFAs or NFAs:

• Machine is equivalent to machine

M1 M2
• if
L M1   L M 2 
•
139
 NFA M1
L M1   {10} * 0
q0 q1 0, 1 q2
•
1


DFA M2 0,1
0
L M 2   {10} *
q0 q1 1 q2
1
0 140
• Since L M1   L M 2   10 *

• machines M1 and M 2are equivalent

0
NFA M1 q0 q1 0, 1 q2
1


0,1
0
DFA M2 q0 q1 1 q2
1
0 141
 Equivalence of NFAs and DFAs
Question: NFAs = DFAs ?

Same power?
Accept the same languages?

142
 Equivalence of NFAs and DFAs

Question: NFAs = DFAs ? YES!

Same power?
Accept the same languages?

143
 We will prove:


Languages Languages
accepted accepted
by NFAs by DFAs

NFAs and DFAs have the same

computation power

144
 Step 1

Languages Languages
accepted  accepted
by NFAs by DFAs

Proof: Every DFA is trivially an NFA

A language accepted by a DFA

is also accepted by an NFA 145
 Step 2

Languages Languages
accepted  accepted
by NFAs by DFAs

Proof: Any NFA can be converted to an

equivalent DFA

A language accepted by an NFA

is also accepted by a DFA 146
 NFA to DFA
M a
NFA
q0 a q1  q2
• b

DFA M
 q0 

147
 M a
NFA
q0 a q1  q2
• b

DFA M
 q0  a
 q1, q2 

148
 M a
NFA
q0 a q1  q2
• b

DFA M
 q0  a
 q1, q2 
b

 149
 M a
NFA
q0 a q1  q2
• b

a
DFA M
 q0  a
 q1, q2 
b

 150
 M a
NFA
q0 a q1  q2
• b

b a
DFA M
 q0  a
 q1, q2 
b

 151
 M a
NFA
q0 a q1  q2
• b

b a
DFA M
 q0  a
 q1, q2 
b

 a, b
152
NFA M a L M   L(M )
q0 a q1  q2
• b
a
DFA M b

 q0  a
 q1, q2 
b

 a, b
153
 NFA to DFA: Remarks

• We are given an NFA M

• We want to convert it to an equivalent DFA M 

L M   L(M )
154
• If the NFA has states

q0 , q1, q2 ,...

• the DFA has states in the power set

•

,  q0  ,  q1 ,  q1, q2  ,  q3 , q4 , q7  ,....

155
 Procedure NFA to DFA
•

• 1. Initial state of NFA: q0

• Initial state of DFA:  q0 

156
 M a
NFA
q0 a q1  q2
• b

DFA M
 q0 

157
 Procedure NFA to DFA

• 2. For every DFA’s state {qi , q j ,..., qm }

• Compute in the NFA

 *  qi , a  ,
•  *  q j , a ,  {qi , q j ,..., qm }

... to DFA
 {qi , q j ,..., qm }, a   {qi , qj ,..., qm
• Add transition
}
158
 M a
NFA
q0 a q1  q2
• b
 * (q0 , a )  {q1, q2 }

DFA M 

 q0  a
 q1, q2 

   q0  , a    q1, q2  159
 Procedure NFA to DFA

• Repeat Step 2 for all letters in alphabet,

• until
• no more transitions can be added.

160
 M a
NFA
q0 a q1  q2
• b

b a
DFA M
 q0  a
 q1, q2 
b

 a, b
161
 Procedure NFA to DFA
• 3. For any DFA state {qi , q j ,..., qm }

• If some is a final state in the NFA

qj
• Then,
• is a final state in the DFA
• {qi , q j ,..., qm }
162
 M a
NFA
q0 a q1  q2 q1  F
• b
a
DFA M b

 q0  a
 q1, q2 
b  q1, q2   F 
 a, b
163
 Theorem
Take NFA M
•
Apply procedure to obtain DFA M

Then M and M are equivalent :

L M   L M  
164
 Finally
We have proven


Languages Languages
accepted accepted
by NFAs by DFAs

165
 We have proven


Languages Languages
accepted accepted
by NFAs by DFAs

Regular Languages

166
 We have proven


Languages Languages
accepted accepted
by NFAs by DFAs

Regular Languages Regular Languages

167
 We have proven


Languages Languages
accepted accepted
by NFAs by DFAs

Regular Languages Regular Languages

Thus, NFAs accept the regular languages

168
 Single Final State
for NFAs and DFAs

169
 Observation

• Any Finite Automaton (NFA or DFA)

• can be converted to an equivalent NFA

• with a single final state

170
 Example
a
NFA
a b

a Equivalent NFA

a b

b 

171
 In General
NFA

Equivalent NFA

 Single
 final state
172
 Extreme Case

NFA without final state

Add a final state

Without transitions

173
Some Properties of
Regular Languages

174
 properties

For regular languages L1 and L

2
we will prove that:
Union: L1  L2

Concatenation: L1L2 Are regular

Languages
Star: L1 *

175
 We Say:
Regular languages are closed under

Union: L1  L2

Concatenation: L1L2

Star: L1 *
176
 Properties
For regular languages L1 and L2

we will prove that:

Union: L1  L2

Are regular
Concatenation: L1L2
Languages

Star: L1 *
177
Regular language L1 Regular language L2

L M1   L1 L M 2   L2

NFA M1 NFA M2

Single final state Single final state

178
 Example
a
n
L1  {a b} b M1

M2
a
L2   ba b

179
 L1  L2 Union
M1
• NFA for

 

 M2 

180
 n
L1  L2  {a b}  {ba}
NFA for
n
• L1  {a b}
a
b
 

 L2  {ba} 
b a
181
 Concatenation
L1L2

• NFA for
M1 M2
 

182
 Example
n n
L1L2  {a b}{ba}  {a bba}
•
• NFA for
n
L1  {a b}
a L2  {ba}
b  b a 

183
 Star Operation
• NFA for L *
1 
  L1 *
M1
 


184
 Example
n
L1*  {a b} *

• NFA for
n
L1  {a N>=
b}0
a
 b 

 185
 Procedure: NFA to DFA
1 Create a graph with vertex {q0}.Identify this vertex as
initial vertex of DFA.
2 Repeat the following steps until no more edges are
missing.Take any vertex {qi,qj,….qk} of G that has no
outgoing edge for some symbol a of the alphabet.
Compute  *(qi, a),  * (qj, a)….  *(qk, a)
Then form the union of all these yielding the set  *{ql,
qm, …qn}.
Create a vertex for G labeled {ql, qm,…qn} if it does not
already exist.
Add to G an edge from {qi, qj,…qk} to {ql,qm…qn} and
label it with a.
3 Every state of G whose label contains any qf of F is
identified as a final vertex of DFA.
4 If NFA accepts  then vertex {q0} in G is also made
186
a
final vertex.
 1
0

q0 q1 q2
0,1 0,1

187
 Start
Equivalent q0 1
0
DFA q
{q0, 1
q1} 1
0,1
1 q2
0
0,1 0
q0,q1,q2 1 q1,q2

0
0,1

188