MITWPU - Unit 1-Theory of Computation
MITWPU - Unit 1-Theory of Computation
MITWPU - Unit 1-Theory of Computation
T.Y. B.Tech
Theory of Computation
(CS314)
7/28/2021 1
Course Objective & Course Outcomes
• Course Objectives:
By participating in and understanding all facets of this Course a student will be able:
1. To understand the basics of automata theory and its operations.
2. To understand problem classification and problem solving by machines.
3. To study computing machines by describing, classifying and comparing
different types of computational models.
4. To understand the fundamentals of decidability and computational complexity.
• Course Outcomes:
After successful completion of this course students will be able:
1. To construct Finite State Machines to solve problems in computing.
2. To build mathematical expressions and syntax verification for the formal
languages.
3. To construct and analyze Push Down Automata and Turing Machine for formal
languages.
4. To classify the problems of decidability and complexity. 2
7/28/2021
Text Books & Reference Books
• Text Books
• John C. Martin, Introduction to Language and Theory of Computation, TMH,
3rd Edition, ISBN: 978-0-07-066048-9.
• Vivek Kulkarni, Theory of computation, Oxford University Press, ISBN-13: 978-0-
19-808458-7.
• Reference Books
7/28/2021 3
Important Instructions
• Be ready Be Attentive
• Tutorials : 4
• Add your profile picture and use only MITWPU email-id for TOC communications and
submissions.
• Tutorial Instructions:
⮚Tutorial on Assignment/Ruled pages
⮚Before uploading label your pages with your Rollno., Name, Tutorialno.
⮚You maybe asked to keep your video ON anytime during the tutorial
7/28/2021 4
Unit I
• Introduction to Formal language, Basic concepts: Symbol, Alphabet, String
• Introduction to Finite Automata, State transition graph, Transition table,
Acceptance of a string, Acceptance of a Language
• Deterministic finite Automata (DFA)-Formal Definition
• NonDeterministic finite Automata (NFA)-Formal Definition
• NonDeterministic finite Automata (NFA) with epsilon transition
• Equivalence of NFA and DFA, Conversion from NFA to DFA, Conversion from
NFA with epsilon transition to DFA
• Finite Automata with output: Moore and Mealy Machine, Moore to Mealy
conversion, Mealy to Moore conversion
7/28/2021 5
Planner - Lectures
Lecture No Topics Covered
1 Introduction to TOC, Introduction to Formal language, Basic
concepts: Symbol, Alphabet, String
2 Introduction to FA, State transition graph, Transition table,
Acceptance of a string, Acceptance of a Language,
3 DFA -Formal Definition, NFA-Formal Definition, NFA with epsilon
transition, Equivalence of NFA and DFA
4 Conversion from NFA to DFA
7 Minimization of FA
Tutorial topics:
FA, DFA, NFA, FSM, Conversion from NFA to DFA
7/28/2021 6
About TOC
• Course is about models of computation, their power, and relationships.
• Hierarchy of models of increasing power:
-DFA < PDA < TM.
For each model, we study:
-What can be computed.
-What cannot be computed.
7/28/2021 7
History of TOC
• 1930-45: Alan Turing studied Turing machine
Describe boundary between what a computing machine could do and
what it could not do
• 1940-50: Researchers studies Finite Automata
• Late 1950s: N. Chomsky studied formal
“Grammars”
• 1969-70: S. Cook extended Turing’s study
Separated problems that can be solved efficiently by computers and the
problems that can be solved theoretically, but takes so much time by
computers to solve
7/28/2021 8
Automata theory
• The study of abstract 'mathematical' machines or systems and
the computational problems that can be solved using these
machines.
7/28/2021 9
Application of TOC
• Finite Automata:
• Software for designing and checking behavior of
digital circuits
• The “Lexical Analyzer” of a compiler
• Software for scanning large bodies of text
E.g. to find occurrences of words in collections of
web pages
• Software for verifying systems of all types that have a
finite number of distinct states
E.g. communication Protocols
7/28/2021 10
Application of TOC (cont…)
• Turing Machine:
• Prototype/Abstract of modern computer
• To find out what a computer can do and what it can
not do
• Help us understand what we can expect from our
software
• To decide, whether we can solve the problem, which
requires more time to solve, in less amount of time.
7/28/2021 11
Basic Definitions
7/28/2021 12
Basic Definitions
• Alphabet - a finite set of symbols.
Notation: Σ .
E.g.: Binary alphabet {0,1},
English alphabet {a,...,z,!,?}
7/28/2021 14
Basic Machine and Finite State Machine
• Basic Machine:
• A machine that recognizes input set I and
produces output set O,
where I & O are finite.
• Machine Function(MAF): I-> O
• Deals with what output is generated, not how
output is generated
E.g.: Switch
7/28/2021 15
Basic machine
It is the most primitive machine with a set of inputs I and set of output O.
E.g. AND,OR,NOR… gates , Vending machine, Decimal to binary
converter, Electric fan…
Weighing machine : I/p- Coin
O/p- Printed weight ticket
It just maps input set to output set.
Mapping function is called Machine function MAF I -> O
I O
(0,0) 0
(0,1) 0
(1,0) 0
(1,1) 1
Limitations:
Cannot remember the intermediate states.
7/28/2021 16
Finite State machine(FSM)
In FSM the internal state of machine changes when it
receives I/P.
It has a pair of functions:
Machine Fn MAF : I x S -> 0
For a given i/p there will be always only one final state.
7/28/2021 17
Finite Automata
• 5-tuple:
• M= (Q,Σ, 𝛿, qo, F)
where,
• Q: Finite set of states
• Σ: Finite input alphabet
• 𝛿: STF that maps Q X Σ to Q, i.e. Q X Σ Q
• qo: Initial state of FA, qo ∈ Q
• F: Set of final states, F ⊆ Q
7/28/2021 18
Finite Automata (cont…)
Input
String
Output
Finite
Automaton Accept/Reject
7/28/2021 19
Block diagram of FA
0 0 1 Input Tape
Tape Head
Finite Control
q0
7/28/2021 20
Acceptance of a String
• A string ‘x’ is said to be accepted by an FA
(given by):
M= (Q,Σ, 𝛿, qo, F)
if 𝛿 (qo,x) = p,
7/28/2021 21
Acceptance of a Language
• If there is a language L such that:
7/28/2021 22
Finite Automata (cont…)
• Finite number of states
• No memory to store intermediate results
• Limited power due to lack of memory
• Input string is considered to be accepted, if the FA resides in any
of the final states
• Input string is considered to be rejected, if the FA resides in any
of the non-final states
7/28/2021 23
Preferred Notation for FA
❑Transition Diagram
❑Transition Table
7/28/2021 24
Transition Diagram
Transition diagram for FA(Q,Ʃ,q0,F,δ) is a graph
defined as:
⮚For each state there is a node.
⮚For each state q in Q and for input symbol in Ʃ .
let δ(q,a)=p then
a
q p
7/28/2021 25
Transition Table
7/28/2021 26
Deterministic Finite Automata
• An FA is said to be deterministic, if for every state there is a
unique input symbol, which takes the state to the required next
unique state
• Given a state Sj, the same input symbol does not cause the FA to
move into more than one state-there is always a unique next state
for an input symbol
0
0
S0 S1
1 1
0 S2
7/28/2021 27
Designing FA
• Accept strings consisting of a & b, that has at
least one ‘a’
i)Min string :a
ii)∑ ={a,b}
b
a
q0 q1 a,b
7/28/2021 28
Finite Automata Example
❑FA for language over {1,0} in which every
string ends with 10.
0
1
1 0
Q0 Q1 Qf
0 1
7/28/2021 29
FA(DFA & NFA) Examples
1. Construct a FA to recognize language containing strings starting
with ‘10’
2. Construct a FA to recognize language containing strings ending
with ‘101’
3. Construct a FA to recognize language of strings with exactly two
0s anywhere
4. Construct a FA to recognize language containing strings
starting with ‘a’
5. Construct a FA to identify string containing even number of
‘a’s over ∑ ={a}
7/28/2021 30
Non-Deterministic Finite Automata
• NFA has the power to be in several states at once
• We can always convert an NFA to a DFA, but the latter may have
exponentially more states than the NFA (a rare case)
• The difference between the DFA and the NFA is the type of transition
function δ
• 5-tuple:
• M= (Q,Σ, 𝛿, qo, F)
where,
• Q: Finite set of states
• Σ: Finite input alphabet
• 𝜹: STF that maps Q X 𝜮 to Q, i.e. Q X 𝜮 2Q
• qo: Initial state of FA, qo ∈ Q
• F: Set of final states, F ⊆ Q
7/28/2021 32
Example : pair of 0’s or pair of 1’s
q2 {} {q2}
q3
{q2} {q2}
q4
{q4} {}
{q4} {q4}
7/28/2021 33
NFA
7/28/2021 34
NFA Examples
1. Construct a FA to recognize language containing strings starting
with ‘10’
2. Construct a FA to recognize language containing strings ending
with ‘101’
3. Construct a FA to recognize language of strings with exactly two
0s anywhere
4. Construct a FA that recognizes the language of strings that end in
01
7/28/2021 35
NFA
0,1
1
q0 0 q1 q2
7/28/2021 36
Equivalence of NFA and DFA
For every NFA, there exists an equivalent DFA.
NFA and DFA have equal powers
• Example: NFA vs DFA
7/28/2021 37
Conversion of NFA to DFA
(Method I)
7/28/2021 38
Conversion of NFA to DFA
Method I (cont…)
7/28/2021 39
Conversion of NFA to DFA
Method I (cont…)
Example:
Convert the NFA : M = ({q0, q1}, {0, 1}, 𝛿, q0,
{q1}) to its equivalent DFA
State Transition Table 𝜹 for example NFA
0 1
0,1
q0 q1
7/28/2021 40
Conversion of NFA to DFA
Method I (cont…)
State Transition table for resultant NFA State Transition Graph for resultant NFA
7/28/2021
41
Conversion of NFA to DFA
Method II
7/28/2021 42
Conversion of NFA to DFA
Method II (cont…)
7/28/2021 43
Conversion of NFA to DFA
Method II (cont…)
• Example:
– Convert the NFA: M = ({q0, q1}, {0, 1}, 𝛿, q0,
{q1}) to its equivalent DFA
State Transition Table 𝜹 for example NFA Transition Graph for example NFA
7/28/2021 44
Conversion of NFA to DFA
Method II (cont…)
DFA construction from the given NFA:
(a) Step 1 (b) Step 2 (c) Step 3 (d) Step 4 (e)Final DFA
7/28/2021 45
DFA Minimization
• Identify equivalent states and keep any one of them.
7/28/2021
46
DFA Minimization (cont..)
• Equivalent states:
– go to same next state on reading the same input
symbol
– same type (final or non-final)
• Unreachable states:
– Can not be reached from initial state on reading any
input symbol
• Dead (or Trap) states:
– Have no outgoing transitions, except to themselves
7/28/2021 47
DFA Minimization Rules
• One non-final state can be replaced by its equivalent
non-final state only
7/28/2021 48
DFA Minimization Rules (cont…)
• Replacing state A by state B means deleting all entries related to state A
i.e. deleting all the transitions for state A from the state transition table.
• If more equivalent states are found after applying all the rules,
repeat the same five steps to further reduce the DFA.
Example
𝜮 0 1
Q
p p,q p
q r r
r s -
s s s
7/28/2021 49
DFA Minimization (cont…)
State Transition Table of the DFA to be minimized
7/28/2021 50
DFA Minimization (cont…)
Minimized DFA
7/28/2021 51
DFA Minimization (cont…)
Further minimized DFA
7/28/2021
52
NFA with 𝜀 − Transitions
• 5-tuple:
• M= (Q,Σ, 𝛿, qo, F) where,
• Q: Finite set of states
• Σ: Finite input alphabet
• 𝜹: STF that maps Q x (𝜮∪ { 𝜺} ) to 2Q
i.e. Q X (𝜮 ∪{𝜺 }) 🡪 2Q
• qo: Initial state of FA, qo ∈ Q
• F: Set of final states, F ⊆ Q
7/25/21 53
NFA with 𝜀 − Transitions
(cont..)
• Example:
Transition Graph for example NFA with 𝛆 − 𝐭𝐫𝐚𝐧𝐬𝐢𝐭𝐢𝐨𝐧𝐬
7/25/2021 54
NFA with ℇ moves Example
7/28/2021 55
NFA with 𝜀 − Transitions
(Examples)
• Example:
Transition Graph for example NFA with 𝛆 − 𝐭𝐫𝐚𝐧𝐬𝐢𝐭𝐢𝐨𝐧𝐬
7/25/2021 56
NFA with 𝜀 − Transitions
(Examples)
• Example:
Convert the NFA with 𝜀 −transitions given in the figure to its equivalent DFA
State Transition table for example NFA with State Transition Graph for example NFA
𝛆 −transitions with 𝛆 −transitions
57
NFA with 𝜀 − Transitions
(Examples)
• Example:
Transition Graph for example NFA with 𝛆 − 𝐭𝐫𝐚𝐧𝐬𝐢𝐭𝐢𝐨𝐧𝐬
7/25/2021 58
Significance of NFA with 𝜀 − Transitions
7/25/2021 59
C Conversion of NFA with 𝜀 − Transitions to
DFA
Indirect Method
Direct Method
NFA without 𝜀
moves
Equivalent DFA
7/29/201 60
9
Indirect Conversion Method
7/25/2021 61
Indirect Conversion Method
(cont…)
• 𝜀-closure of a state:
• Set of all states p, such that there is a path from state q to
state p labeled ‘𝜀’, is known as 𝜀-closure (q)
• Set of all the states having distance zero from state q
• Every state is at distance zero from itself
• Denoted by 𝛿^
7/28/2021
7/29/201 62
9
Indirect Conversion Method
(cont…)
• Example:
Convert the NFA with 𝜀 −transitions given in the figure to its equivalent DFA
State Transition table for example NFA with State Transition Graph for example NFA
𝛆 −transitions with 𝛆 −transitions
63
Indirect Conversion Method
(cont…)
64
Indirect Conversion Method
(cont…)
2. Conversion of NFA without 𝛆 −transitions to an equivalent DFA:
(a) Step 1 (b) Step 2 (c) Final DFA
65
Indirect Conversion Method
(cont…)
2. Conversion of NFA without 𝛆 −transitions to an equivalent DFA:
State Transition table for the equivalent DFA Reduced State Transition table for the
equivalent DFA
66
Direct Conversion Method
• Begin with the initial state
• Go on adding the states as & when required to the diagram
• Follow the same process until there exists no state without
having the transitions specified
• Each state label consists of:
• Name of state label
• Combination of all the state symbols, which are reachable
from the given state
67
Direct Conversion Method (cont…)
• Start building an equivalent DFA with the transition diagram of
given NFA
• Find the transition, one state at a time
• If the next state of a given transition is a new combination, add
that to the set of states for the resultant DFA
68
Direct Conversion Method
(cont…)
• Example (Refer slide no. 53 for example NFA with 𝜺 −transitions
Resultant DFA (a) Step 1 (b) Step 2 (c) Step 3 (d) Final DFA
69
Convert Є-NFA to DFA example
a b c Є
Solution Є-closure(p) = { p}
NFA Є-closure(q) = { p, q}
a b c Є-closure(r) ={ p, q,r}
76
Convert Є-NFA to DFA example
0 1 Є
A {A} {B}
B {C} {D}
C {B} {D}
*D {D} {D}
Solution
NFA
0 1 DFA using direct method
A {A,B,C,D} {D} 0 1
B {C,D} {D} [A] [ABCD] [D]
[ABCD] [ABCD] [BD]
C --- {B,D}
*[D] -[D] [D]
*D {D} {D} [BD} [CD] [D]
[CD} [D] [BD]
77
FA with output
Machine generates an output on every input. The value
of output is a function of current state and the current
input.
78
Mealy Machine (cont…)
• Six-tuple
• M= (Q,Σ, Δ, 𝛿, 𝜆, qo)
where,
• Q: Finite set of states
• Σ: Finite input alphabet
• Δ: An output alphabet
• 𝛿: STF that maps Q X 𝛴 to Q,
• i.e. Q X 𝛴 Q
• 𝝀: Machine function (MAF); 𝝀: Q X 𝜮 𝜟
• qo: Initial state of the machine, qo ∈ Q
79
Mealy Machine Example
Mealy machine for exclusive-or of the two most-recent
input values
80
Mealy Machine
• The output symbol at any given time depends on the current
input symbol and the current state (i.e.transition)
• An output symbol is associated with the transition
81
Mealy Machine
• A machine with finite number of states, and for which the output
symbol at any given time is a function of (i.e. it depends on) the
current input symbol as well as the current state of the machine
• Example:
• Construct a Mealy machine to find 2’s complement of a given binary
number
Example Mealy Machine
82
Moore Machine (cont…)
• Six-tuple
• M= (Q,Σ, Δ, 𝛿, 𝜆, qo)
where,
• Q: Finite set of states
• Σ: Finite input alphabet
• Δ: An output alphabet
• 𝛿: STF that maps Q X 𝛴 to Q,
• i.e. Q X 𝛴 Q
• 𝝀: Machine function (MAF); 𝝀: Q 𝜟
• qo: Initial state of the machine, qo ∈ Q
83
Moore Machine (cont…)
• Example:
• Construct a Moore machine that takes set of all strings over {a,b} as an
input and prints ‘1’ as an output for every occurrence of ‘ab’ as a
substring
84
Moore Machine
• The output symbol at any given time depends only upon the
current state of the machine (and not on the input symbol read)
• An output symbol is associated with each state
• When the machine is in particular state, it generates the output,
irrespective of the input that caused the transition
85
Moore Machine Example
7/28/2021 86
Moore Machine Example
Machine that counts occurrences of aab
7/28/2021 87
Conversion of Moore Machine to
Mealy Machine
• Moore Machine:
• M1= (Q,Σ, Δ, 𝛿, 𝜆, qo)
• Equivalent Mealy Machine:
• M2= (Q,Σ, Δ, 𝛿, 𝜆′,
qo) where,
• 𝛌′ 𝐪, 𝐚 = 𝛌 (𝛅(q, a))
88
Moore to Mealy Machine
❑Let us take moore machine and it’s transition
Table.
7/28/2021 89
Moore to Mealy Machine
❑It’s transition Table.
7/28/2021 90
Algorithm
Step1:Construct an empty mealy machine using
all states of moore machine as shown in Table
7/28/2021 91
Algorithm
Step 2: Next state for each state can also be directly found fro
m moore machine transition Table as:
7/28/2021 92
Algorithm
Step 3: As we can see output corresponding to each
input in moore machine transition table. Use this to fill the Output
entries.
7/28/2021 93
Algorithm
Step4:As we can see from above
table, q10 and q11 are similar to each other (same value of next st
ate and Output for different Input). Similarly, q20 and q21 are also
similar. So, q11 and q21 can be
eliminated.
7/28/2021 94
Final Mealy Machine
Number of states in Mealy machine can’t be greater than
number of states in Moore machine
7/28/2021 95
Conversion of Moore Machine to
Mealy Machine (cont…)
• Example:
• Convert the given Moore machine to an equivalent
Mealy Machine
Example Moore Machine Equivalent Mealy Machine
96
Conversion of Mealy Machine to
Moore Machine
• Mealy Machine:
• M1= (Q,Σ, Δ, 𝛿, 𝜆, qo)
• Equivalent Moore Machine:
• M2= ([QX Δ]),Σ, Δ, 𝛿′, 𝜆′, [qo, bo]
where,
• bo is an arbitrarily selected member of Δ
• 𝜹′ 𝒒, 𝒃 , 𝒂= [𝜹 (𝒒, 𝒂), 𝝀 (q, 𝒂)]
• 𝝀′ ([q, b]) = b
b: output symbol a: input
symbol
97
Mealy to Moore Machine
❑Let us take mealy
machine and it’s transition
Table.
7/28/2021 98
Algorithm
Step 1.Find states having more than 1 outputs
associated with them.
Here we have q1 and q2.
7/28/2021 99
Algorithm
Step 3. Create an empty moore machine with new
generated state. For moore machine, Output will be
associated to each state irrespective of inputs.
7/28/2021 100
Algorithm
Step 4. Fill the entries of next state using mealy machine tr
ansition table shown in Table. For q0 on
input 0, next state is q10 (q1 with output 0).
Similarly, for q0 on input 1, next state is q20 (q2 with output 0).
7/28/2021 101
Final Moore Machine
7/28/2021 102
Conversion of Mealy Machine to
Moore Machine (cont…)
• Example:
• Convert the given Mealy Machine to an equivalent
Moore Machine
Example Mealy Machine Equivalent Moore Machine
103
Comparison of Moore and Mealy
Machine
Moore Machine Mealy Machine
104