Dr.

Arshi Husain,
Asst. Professor
USICT, GGSIPU
TOC / TAFL Playlist:
https://shorturl.at/xzuIM
[email protected]
https://shorturl.at/4sLZo
 Theory Of Computation
What is Automata Theory?

Automata theory is the study of abstract machines and automata,& the computational problems that can be

solved using them. It is the combinational study of theoretical computer science and discrete mathematics.

TOC is a branch of computer science that explores the fundamental capabilities and limitations of

computers. It describes the various abstract models of computation, represented mathematically & focuses

on understanding what problems can be solved by a computer, how efficiently they can be solved, and how to

design systems to perform those computations. It focuses on:

1. What can we compute? (Decidability)

2. How can we compute it? (Models of Computation)
3. How efficiently can we compute it? (Complexity)
 Theory Of Computation
The Theory of Computation is divided into three main areas:

1. Automata Theory: Focuses on abstract machines (like finite automata and

Turing machines) and how they process inputs.
2. Formal Languages: Studies the structure of

languages that computers can understand,

like programming languages.

1. Computational Complexity: Examines how

hard or easy a problem is to solve, based

on resources needed.
 Basic Terminology
Automata: It is a self-operating machine or abstract computational systems that
follows predefined rules to process inputs and produce outputs. Eg, Elevator,
Automatic doors etc

Symbol: Basic building blocks. Example: a,b,c,… , 0,1,2,3,….9 and special characters.

Alphabet -: is a finite , nonempty set of symbols denoted by ∑ in theory of

computation. Example1-:∑= {0,1} Example2-:∑={a,b,c....z}

String or Word-: Finite sequence of symbols (Symbols could be chosen from

alphabet) and denoted by symbol w (depends on writer) Example-: ∑={0,1} Then
0101001 , 010, 00, 11, 111 ,0, 0011,..... etc are the string we can from by sequence
of symbols. Then a,b,ab,aa,bb,aab,abb,....etc are the string we can write by
sequence of symbol.

Empty String : Symbol ‘ε’ (epsilon) is used to denote empty string and that is the
string consisting of zero symbol. It is also called as null string
 Basic Terminology
Length of String : The length of a string is number of symbols in the string. It
denoted by |w|.
Example 1: If ∑={0,1} & w=01010 Then, |w|=5 Example2: If w =ε then |w|=0

Prefix: It is any number of leading symbol of string. Example-: let w=abc Prefix of
w are ε,a,ab,abc.

Suffix: Any number of trailing symbols of string. If w=abc, suffix are ε,c,bc,abc

Substring:Any sequence of symbol over the given string. Example-: w=abc

The substring of w are ε,a,b,c,bc,ab,abc.

Concatenation of String-: if w and x are two string then the concatenation of two
string is denoted by wx. Example-: w=1110 and x=0101 Then xy=11100101
 Basic Terminology
Language-: It is the set of string over the alphabet ∑ which follows a set of
rules called the grammar L Or otherwise we can say set of string generated by
grammar taking symbol from ∑ .

Example 1-:All string start with 11 from ∑={0,1}

L={110,111,1100,1111,1101,1110,........}
Example 2-:set of binary number whose value is prime. L={10,11,101,111,1011,......}
Note-: L={ϕ} is called empty language. And L={ε} Language consisting of only the
empty string.

Grammar-: A set of rules used to generate the string for a particular language.
 Power of an alphabet
If Σ is an alphabet, the set of all strings can be expressed as a certain length from
that alphabet by using exponential notation. The power of an alphabet is denoted by
and is the set of strings of length k.

Example: If ∑={0,1}

Σn= { w , |w| = n} using the symbols from the alphabet

Set of all strings from alphabet
Σ of length exactly n
Kleene Closure / Star Σ*
Positive Closure Σ+
 Finite
Automata
 Finite Automata
❏ Finite Automata(FA) is the simplest machine to recognize patterns. It is a
mathematical model of a system, with discrete inputs, outputs, finite states
and set of transitions from state to state that occurs on input symbols from
alphabet Σ.
❏ It takes the string of symbol as input and changes its state accordingly. When
the desired symbol is found, then the transition occurs. At the time of
transition, the automata can either move to the next state or stay in the same
state. They are widely used for Pattern matching, Lexical Analysis, Parsing and
other applications.

It representations:

○ Graphical (Transition/State Diagram or Transition Table)

○ Tabular (Transition Table)
○ Mathematical (Transition Function/ID or Mapping Function)
 Finite Automata
Applications:

❏ In switching theory and design and analysis of digital circuits automata theory
is applied.
❏ Design and analysis of complex software and hardware systems.
❏ It plays an important role in complier design
❏ To prove the correctness of the program automata theory is used.
❏ It is base for the formal languages and these formal languages are useful of
the programming languages.
 Finite Automata Model

The various components of the block diagram are as follows:

1) Input tape: The input tape is divided into cells, each cell
containing a single symbol from the input alphabet ∑. The
end cells of the tape contain the end marker ¢ at the left
end and the end marker $ at the right end. The absence of
end markers indicates that the tape is of infinite length.
The left-to-right sequence of symbols between the two
end markers is the input string to be processed

1) Finite control: It represents the "brain" or logic of the

finite automaton. It decides the next state on receiving
particular input from input tape.

1) Reading head : The head is unidirectional, it examines only

one cell at a time and reads the cells one by one from left
to right,
 Computation of Finite Automata
❏ The automata M receives the input string ‘w’ designed from the alphabet S.

❏ After reading each symbol, control moves from one state to another along
the transition that has that symbol as its label.

❏ When M reads the last symbol of w, then the string ‘w’ is said to be
accepted by the finite automata, if there exist a transition for which it
starts at the initial state and ends in any one of the final states,. δ* (q0, w)
= qf for some qf ∈ F.

Mathematically, it can be represented as:

L(M) =
Types of Finite Automata
 Deterministic Finite Automata (DFA)
Deterministic Finite Automaton is a FA in which there is exactly one path for a

specific input from current state to next state. i.e., there is a unique

transition on each input symbol.

Formal Definition of a DFA
 Representation
1. Transition State Diagram (Transition graph): It is a finite 5-tuple directed
graph in which each circle represents a state and the edges represent the
transition of one state to another state. The initial state is represented by a
circle with an arrow pointing towards it, the final state by two concentric
circles.
 Representation (cont.)
2. Transition Table: It is a two-dimensional table where number of columns is
equal to number of input alphabets and number of rows is equal to number of
states.
 Representation (cont.)
3. Transition ID/Function: The mapping/transition function is denoted by δ.

Two parameters are passed to this transition function: (i) current state and
(ii) input symbol. The transition function returns a state which can be called as next
state.
δ ( current_state, current_input_symbol ) = next_state

Example 1: δ ( q0, a ) = q1, means on a state q0 take an input symbol a machine will
make a transition q1

Example 2: δ {qi , a) = qj , this means that on a state qi take an input symbol a

machine will make a transition qj
 Practice Questions
Design a minimal DFA that accepts
1) L={ε, a}, Σ={a}
2) L={a2}, Σ={a}
3) L={a,a2}, Σ={a}
4) L={ε, a,a2}, Σ={a}
5) L={an | n >=0 }, Σ={a}
6) L={an | n >=1 }, Σ={a}
7) L={a,b}* , Σ={a,b}
8) L={a,b}+, Σ={a,b}
Q9 Design a minimal DFA that accepts all strings over
the alphabet ∑ = {a, b} such that every accepted
string w, starts with ‘a’
 Q10 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, starts and ends with ‘a’
 Q11 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, starts and ends with the same symbol

b
 Q12 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, starts and ends with the different symbol
 Q13 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, starts with the substring s=’abb’
 Q14 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, ends with the substring s=’bab’
 Q14 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, ends with the substring s=’bab’
 Q15 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, contains the substring s=’aba’
 Q16 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, is like w=AX. A=aaa/bbb?
 Q17. Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, is like w=XS. S=aaa/bbb?
 Q18. Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string w, is like w=XSX. S=aaa/bbb?
 Q19 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string ‘w’ should be:
a. |w|=4
b. |w|<=4
c. |w|>=4
 Q20 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string ‘w’ must consist of exactly two a’s.,i.e.|w|a=2
 Q21 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string ‘w’ must consist of at least two a’s.,i.e.|w|a>=2
 Q22 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string ‘w’ must consist of at most two a’s.,i.e.|w|a<=2
 Q23 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that such that
every accepted string ‘w’ should be like |w|=0 (mod 3)
 Q24 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that such that
every accepted string ‘w’ should be like |w|=1 (mod 4)
 Q25 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string ‘w’, must contain 0(mod 2) number of a’s, i.e.
|w|a = 0(mod 2)
 Q26 Design a minimal DFA that accepts all strings
over the alphabet ∑ = {a, b} such that every accepted
string ‘w’, must contain 2(mod 3) number of a’s, i.e.
|w|a = 2(mod 3)
Q27 Design a minimal DFA to check whether given
unary number is divisible by three.
Q28 Design a minimal DFA to check whether given
binary number is divisible by 2.
Q29 Design a minimal DFA to check whether given
binary number is divisible by 3.
Q30 Design a minimal DFA to check whether given
binary number is divisible by 4.
 NFA (Non-Deterministic finite automata)

1. Q: finite set of states

2. ∑: finite set of the input symbol
3. S: initial state
4. F: final state
5. δ: Transition function

δ: Q x ∑ →2Q
 POINTS TO REMEMBER

Accepting power of NDFA= Accepting power of DFA

No concept of dead state

Acceptance by NFA
Q1. Design a NFA that accepts all strings over the alphabet
∑ = {a, b} such that every accepted string start and end
with same symbol
Q1. Design a NFA that accepts all strings over the alphabet
∑ = {a, b} such that every accepted string start and end
with same symbol
Q2. Design a NFA that accepts all strings over the
alphabet ∑ = {a, b} such that every accepted string start
and end with different symbol
Q2. Design a NFA that accepts all strings over the
alphabet ∑ = {a, b} such that every accepted string
start and end with different symbol
Q3. Design a minimal NFA that accepts all strings over
the alphabet ∑ = {a, b}. where every accepted string ‘w’
contains substring s, Where s = ’aba
Q4. Design a NFA that accepts all strings over the
alphabet ∑ = {a, b}. where every accepted string ‘w’
ends with substring ‘s’, Where s = ‘bab’?
Q5. Design a NFA that accepts all strings over the
alphabet ∑ = {a, b}, where every accepted string ‘w’
starts with substring s, Where s = ‘aba
Q6. Design a NDFA that accepts all strings over the
alphabet ∑ = {a, b} such that for every accepted string
3rd from right end is always a ?
DFA Vs NFA
Conversion from NFA to DFA(NFA and DFA Equivalence)
● In NFA, when a specific input is given to the current state, the
machine goes to multiple states. It can have zero, one or more than
one move on a given input symbol. On the other hand, in DFA, when a
specific input is given to the current state, the machine goes to only
one state. DFA has only one move on a given input symbol.
● Since every NFA and DFA has equal power that means, for every
language if a NFA is possible, then DFA is also possible. So, every
NFA can be converted to DFA
● Let, M = (Q, ∑, δ, q0, F) is an NFA which accepts the language L(M).
There should be equivalent DFA denoted by M' = (Q', ∑', q0', δ', F')
such that L(M) = L(M').
Steps for converting NFA to DFA:
Step 1: Convert the given NFA to its equivalent transition table
Step 2: Create the DFA’s start state. Initial state will always
remain same.
Step 3: Create the DFA’s transition table
Step 4: Step 4: Create the DFA’s final states
The DFA’s final states are the sets of states that contain at least
one final state from the NFA.

Step 5: Simplify the DFA

Step1:
Step 2:
Step 1
Points to Remember
● After conversion, the number of states in the resulting DFA may or may not be
same as NFA.
● The maximum no. of states that may be present in the DFA are 2Number of states in the
NFA
.
In general, the following relationship exists between the number of states in the NFA and
DFA-

where m = Number of states in the NFA

n = Number of states in the DFA
 NULL/ε Closure
The ε closure(P) is a set of states which are reachable from state P
along € labelled transition path.

The Null closure Q is always a non-empty and finite state, because

every state’s null closure is that state only.

€-closure(Φ) = Φ
EQUIVALENCE BETWEEN NULL NFA TO NFA

● There will be no change in the initial state.

● No change in the total no. of states
● May be change in the number of final states.
● All the states whose €-closure consists of at least one final
state in the initial €-NFA will get the status of the final state
in the resulting NFA
How to convert Epsilon NFA to NFA
Step 1: Find null closure of all states

Step 2: Follow the process of null closure, input, null closure

ẟ(q0, a) = ε-Closure [ẟ[ε-Closure(q0), a]]

Step 3: Resulting NFA

Chomsky hierarchy
Type 0: Unrestricted Grammars
Language recognized by Turing Machine is known as Type 0
Grammar. They are also known as Recursively Enumerable
Languages.
Grammar Production for Type 0 is given by
α —> β
where α and β are strings of non-terminal and terminal
symbols, with α containing at least one non-terminal.
Type 1: Context-Sensitive Grammars

Languages recognized by Linear Bounded Automata are known as Type 1

Grammar. Context-sensitive grammar represents context-sensitive languages.

Grammar Production for Type 1 is given by

α —> β (ensuring count symbol in LHS must be less than or equal to RHS)

Special Case for Start Symbol

The only exception to this rule is when the start symbol S produces the empty string
directly:This special case is allowed to ensure that the grammar can generate the
empty string as part of the language. However, this exception applies strictly to the
start symbol and not to other non-terminals.
Type 2: Context-Free Grammar
Languages recognized by Pushdown Automata are known as Type 2 Grammar.
Context-free grammar represents context-free languages.
For grammar to be context-free, it must be context-sensitive. Grammar Production
for Type 2 is given by
A→γ, where A is a single non-terminal and γ is a string of terminals and
non-terminals. The left-hand side of the rule is always a single non-terminal.

Type 1:Regular Grammars

Languages recognized by Finite Automata are known as Type 3 Grammar.

Regular grammar represents regular languages. Rules are of the form A→aB
or A→a , where A and B are non-terminals and a is a terminal. Regular
grammars can be further divided into right-linear and left-linear grammars
based on whether the non-terminal appears on the right or left of the terminal
in the production rules.
Summary
 Minimization of DFA
Minimization of DFA means reducing the number of states (whose
presence or absence doesn’t affect the language accepting capability of
the given FA.)
NOTE: A minimal FA is always unique for a language

ELIMINATION OF NON- PRODUCTIVE STATES-

These states don’t add anything to the language accepting power to the
machine. They can further be divided into
• Unreachable Sate • Dead State • Equal State
NON- PRODUCTIVE STATES

Dead State- It is basically created to make the system complete, can be defined
as a state from which there is no transition possible to the final state
Unreachable State- It is that state which cannot be reached starting from initial
state by parsing any input string

Equivalent states- Two states q1 and q2 are equivalent (denoted by q1≅ q2) if
both δ (q1, x) and δ (q2, x) are final states or both of them are non-final states for
all x ∈ ∑*. If q1 and q2 are k-equivalent for all k ≥ 0, then they are k-equivalent.

These states behave in same manner on each and every input string. That is for
any string w where w Î S* either both of the states will go to final state or both will
go to non-final state
 Regular Expression
● The language accepted by finite automata can be easily described by simple expressions
called Regular Expressions. It is the most effective way to represent any language.
● The languages accepted by some regular expression are referred to as Regular languages.
● A regular expression can also be described as a sequence of pattern that defines a string.
● Regular expressions are used to match character combinations in strings.
● Two regular expressions P and Q are equivalent (we write P = Q),if P and Q represent the same
set of strings.

For instance:

In a regular expression, x* means zero or more occurrence of x. It can generate {e, x, xx, xxx, xxxx,
.....}

In a regular expression, x+ means one or more occurrence of x. It can generate {x, xx, xxx, xxxx, .....}
 Regular Language

Regular languages are formal languages that regular expressions can

describe and can also be recognized by finite automata.
If for instance, a, b ∈ ∑, then
• R = a denotes the L = {a}
• R = a.b denotes L = {ab} concatenation
• R = a + b denotes L = {a, b} Union
• R = a* denotes the set {∈, a, aa, aaa, ...} known as Kleene
closure.
• R = a+ Positive closure {a, aa, aaa…}
• R =(a + b)* denotes {a, b}*
 Operators
When we view a in ∑ as a regular expression, we denote it by a.
• If R is a regular expression, then (R) is also a regular expression.
• The iteration (or closure) of a regular expression R written as R*, is also a regular
expression.
• The iteration (or closure) of a regular expression R written as R+ , is also a
regular expression.
• The concatenation of two regular expressions R1 and R2, written as R1 R2, is
also a regular expression.
• The union of two regular expressions R1 and R2, written as R1 + R2, is also a
regular expression
 Operator precedence
The precedence order to solve is
• ()Bracket
• * (Kleene Closure)
• + Positive Closure
• Concatenation
• Union
Finite Automata to regular expression Conversion
Finite Automata to regular expression Conversion
Q design a regular expression that represent all strings over the alphabet
∑ = {a, b}, where every accepted string ‘w’ starts with substring s = ‘abb’
Q design a regular expression that represent all strings over the alphabet ∑ =
{a, b}. where every accepted string ‘w’ ends with substring s = ‘bab’?
Q Design a regular expression that represent all strings over the alphabet
∑ = {a, b}. where every accepted string ‘w’ contains substring s = ‘aba’ ?
Q Design a regular expression that represent all strings over the alphabet ∑
= {a, b} such that every accepted string start and end with same symbol?
Q Design a regular expression that represent all strings over the alphabet ∑ = {a,
b} such that every accepted string start and end with different symbol ?
Q Design a regular expression that represent all strings over the alphabet ∑
= {a, b} such that every accepted string w, is like w=AX where A = ‘aaa/bbb’
?
Q Design a regular expression that represent all strings over the alphabet ∑
= {a, b} such that every accepted string w, is like w=XS. S = ‘aaa/bbb’ ?
Q Design a regular expression that represent all strings over the alphabet ∑ =
{a, b} such that every accepted string w, is like w=XSX. S = aaa/bbb ?
Q Design a regular expression that represent all strings over the alphabet ∑ = {a,
b}, such that every string ‘w’ accepted must be like i) |w| = 3 ii) |w|<=3 iii) |w|>=3
Q Design a regular expression that represent all strings over the alphabet ∑ = {a,
b} such that for every accepted string 2nd symbol from left end is always b
Q Design a regular expression that represent all strings over the alphabet ∑ = {a,
b} such that for every accepted string 4th symbol from right end is always a.
Q Design a regular expression that represent all strings over the alphabet ∑=
{a, b}, such that every string ‘w’ where |W| = 0(mod 3)?
Q Design a regular expression that represent all strings over the alphabet ∑=
{a, b}, such that every string ‘w’ where |W| = 3(mod 4)?
Q Design a regular expression that represent all strings over the alphabet ∑=
{a, b}, such that every string ‘w’ where |W|a = 0(mod 3)
Q Design a regular expression that represent all strings over the alphabet ∑=
{a, b}, such that every string ‘w’ where |W|b = 2(mod 3)
Closure properties of Regular languages
Consider that L and M are regular languages
Myhill-Nerode Theorem
Myhill-Nerode Theorem
 Moore Machine
● In a Moore machine, each state is associated with a specific output
● They are FS M/c’s that help in producing outputs
● There is no concept of final states and dead states and no concepts of final
states
● Input: 11
● Output: 000
● If the input string has a length of n, then the length of the output string will be
n+1
● A Moore machine's response to an empty input string is the output associated
with the initial state.
A Moore machine is a six-tuple (Q, ∑, O, δ, λ, q0), where
• Q is a finite set of states
• ∑ is the input alphabet:
• O is the output alphabet.
• δ is the transition function Q x ∑ into Q
• λ is the output function mapping Q into O
• q0 is the initial state
Design a Moore m/c from the given transition table
Construct a Moore machine that prints ‘a’ whenever the sequence ‘01’ occurs,
∑ = {0, 1}, Δ = {a, b}?
Construct a Moore machine that counts the occurrences of sequence ‘ab’ in
terms of 1, ∑ = {a, b}, Δ = {0, 1}?
 Mealy machines
● In a Mealy machine, the output symbol depends on the current state and the
current input.
● They are FS M/c’s that help in producing outputs
● There is no concept of final states and dead states and no concepts of final
states
● Mealy machine do not response for empty string
● If the input string has a length of n, then the length of the output string will be n
● Input: 11
● Output: 00
 Mealy machines
Mealy machines are also finite state machines with output value and
their output depends on the present state and current input symbol. It
can be defined as (Q, q0, ∑, O, δ, λ’) where:
● Q is a finite set of states.
● q0 is the initial state.
● ∑ is the input alphabet.
● O is the output alphabet.
● δ is the transition function which maps Q×∑ → Q.
● ‘λ’ is the output function that maps Q×∑→ O.
Design a Mealy m/c from the given transition table
Construct a Mealy machine that prints ‘a’ whenever the sequence ‘01’ occurs,
∑ = {0, 1}, Δ = {a, b}?
Construct a Mealy machine that counts the occurrences of sequence ‘ab’ in
terms of 1, ∑ = {a, b}, Δ = {0, 1}?
 Moore Mealy

Output depends on the present state as well as

Output depends only upon the present state.
present input.

Produces an output corresponding to the initial

Does not produce any output.
state.

Moore machine also places its output in the Mealy Machine places its output on the
state. transition.

Generates an output string of length n+1. Generates an output string of length n

If input changes, output does not change, as

If input changes, output also changes, because
moore machine is a type of finite state machine
in mealy machine output depends on the present
where the output is determined solely by the
state as well as present input.
current state, not by the current input.
Conversion from Moore machine to Mealy Machine
Conversion from Moore machine to Mealy Machine
Conversion from Moore machine to Mealy Machine
Conversion from Moore machine to Mealy Machine
If n pigeonholes are occupied by n+1 or more pigeons, then at
least one pigeonhole is occupied by greater than one pigeon.
Eg: At least how many people must have their birthday on the
same day if 8 people are assembled in a room?
 Decidability and Undecidability

Decidable Decidable problem is one for which there exists an algorithm that can
determine the answer (yes or no) for every input in a finite amount of time. In the
context of regular languages, decidability means that there exists a finite
procedure (algorithm) to determine certain properties of these languages.

Undecidable Undecidable problems are those for which no algorithm can

determine the answer (yes or no) for all possible inputs.
Decision properties of regular languages are specific questions about these languages,
such as emptiness, finiteness, membership, equivalence.
Membership

Problem: Determine whether a string www belongs to a regular language L.

Algorithm:

● Construct a finite automaton M for L.

● Simulate M on input w.
● If M ends in an accepting state, w is in L; otherwise, it is not.
Example:
Conversion from Mealy machine to Moore Machine
Conversion from Mealy machine to Moore Machine
Conversion from Mealy machine to Moore Machine
For Practice Questions, refer to unit 2 playlist
Transition Graphs
final
Example
Kleene’s Theorem
Regular Grammar to Finite Automata
 Non-regular languages
Non-regular languages are languages that cannot be recognized by a finite automaton
(DFA or NFA) and cannot be expressed using regular expressions. These languages
require more computational power, such as a pushdown automaton (for context-free
languages) or a Turing machine. They fail the pumping lemma for regular languages,
which is often used to prove their non-regularity.

Example:
The language L={ a^n b^n | n ≥ 0 } is non-regular because a finite automaton cannot
remember how many aa's have been seen to match them with bb's.

Proving non-regularity using Pumping Lemma.:

1⃣ Assume language is regular.

2⃣ Given ppp, choose a string that is at least length ppp.

3⃣ Show how pumping & splitting the string does NOT satisfy the pumping lemma.
"Pumping" refers to the idea that a certain
segment of a string within a language can be
repeated ("pumped") multiple times without
causing the string to fall out of the language.
=
 Arden's Theorem
If P & Q are two regular expressions over Σ & if P does not contain ε, then the following equation in R given by

R = Q + RP

has a unique solution

R = QP*

Conditions

● FA should not contain ε-transition

● FA should have only one initial state

Why?

Arden's Theorem

● To check the equivalence between two REs

● Conversion of DFA to RE
STEPS
1. Write state equations by looking at the incoming transitions
2. Add epsilon to the start state equation
3. Perform any combination of substitution rearrangement and apply ARDEN‘S
THEOREM until we are able to get the final states equation in terms of
alphabets.
1. Write state equations by looking at the
incoming transitions
2. Add epsilon to the start state equation
3. Perform any combination of substitution
rearrangement and apply ARDEN‘S
THEOREM until we are able to get the final
states equation in terms of alphabets.
 Context Free Grammar
Context Free Grammar is formal grammar, the syntax or structure of a formal
language can be described using context-free grammar (CFG), a type of formal
grammar. The grammar has four tuples: (Vn,∑,P,S).
NOTE: {(Vn,∑,P,S) also denoted as (Vn,T,P,S). Where T (∑) is a set of terminals}

Where, Vn: is a non empty set of Variables or Non Terminals, V is finite.

Points to remember
Consider the following grammar & and identify its language (CFG TO CFL)

Consider the following grammar and identify its language?

Consider the following grammar & and identify its language (CFG TO CFL)
Consider the following grammar & and identify its language (CFG TO CFL)

S→aSb/ε
Consider the following grammar & and identify its language (CFG TO CFL)
Ques. Consider the following grammar & and identify its language (CFG TO CFL)
Ques. Consider the following grammar & and identify its language (CFG TO CFL)
L={a^nb^m |n≥0,m>0}
Consider the following grammar & and identify its language (CFG TO CFL)
Convert CFL to CFG
1. S → aSb
S → ab
1. S → aSb | bb
Convert CFL to CFG
3. S→aaSb
S→/ϵ
4. S→aaSb
S→aaa
Convert CFL to CFG
Convert CFL to CFG
Convert CFL to CFG
Write CFG that generates a language which accepts all
palindromic strings of even length over the alphabet {a,b}
Write CFG that generates a language which accepts all
palindromic strings of odd length over the alphabet {a,b}
Write CFG that generates a language which accepts all
palindromic strings over the alphabet {a,b}
 REGULAR GRAMMARS
Regular grammar generates regular language.
Right and Left linear Regular Grammars
 Conversion of RLG to LLG for language L
Step 1: Reverse the FA for language L
Step 2: Write the RLG for it.
Step 3: Reverse the right linear grammar
(RHS).
Ex: FA for accepting strings that
start with b
FA TO CFG CONVERSION
EXAMPLES
EXAMPLES
PDA to CFG Conversion
CFG to PDA Conversion
 Conversion of RLG to LLG for language L
Step 1: Reverse the FA for language L
Step 2: Write the RLG for it.
Step 3: Reverse the right linear grammar.
Example: FA for accepting strings that
start with b
 Simplification of CFG
The process of Removal of Unit productions, Null productions and Useless
Symbols is known as Simplification of CFG.

S→AbB/bB/Ab/b
S →a
S →b
S→aSb/ε S' → S / ε
S→ a S b/ ab

S' → S / ε
S→BA/A/AB/B/AA
A→0A/0
B→1B/1

S' → S / ε
S→AB
S→AB/B/A
A→a/ε
A→a
B→b/ε
B→b
S → 0X | 1Y | 01
X → 0S | 00
Y → 0S | 00 | 1

Step 1: To remove A → B, add production A → x to the grammar rule whenever B → x occurs in

the grammar.

Step 2: Delete A → B from the grammar.

Step 3: Repeat the first and the second steps until all the unit productions are removed.
 Remove Unit Productions from:
S → 0X | 1Y | Z
X → 0S | 00
Y → 1 | X
Z → 01

So,the grammar
becomes:
S → 0X | 1Y | 01
X → 0S | 00
Y → 0S | 00 | 1
 Remove Unit Productions from:
S → aA b
A → B / a
B → C / b
C → D / c
D → d

So,the grammar
becomes:
S → a A b
A → a | d | c | b
 Remove Useless Productions from:
S → aAB | bA | aC Grammar after
A → aB | b
B → aC | d removing useless C
S → aAB | bA
A → aB | b
B → d
Step 1.2: Check Reachability

Every symbol can be reached from

S, and every non-terminal produces
terminal strings
Remove Useless Productions from:
A context free grammar (CFG) is in Chomsky Normal Form (CNF) if all
production rules satisfy one of the following conditions:

● A non-terminal generating a terminal (e.g.; X->x)

● A non-terminal generating two non-terminals (e.g.; X->YZ)
● Exception:Start symbol generating ε is allowed (e.g.; S-> ε)
 Conversion of CFG to CNF
Step 1. Eliminate null, unit and useless productions.

Step 2. Eliminate terminals from RHS if they exist with other terminals or
non-terminals. e.g,; production rule X->xY can be decomposed as:
X->ZY
Z->x

Step 3. Eliminate RHS with more than two non-terminals.

e.g,; production rule X->XYZ can be decomposed as:
X->PZ
P->XY
 Convert the following CFG to CNF
1. S → aSb | ab
Step 1: Introduce New Variables for Terminals

Since CNF does not allow mixed terminals and non-terminals, we introduce new variables

A→a

B→b

Now, rewrite the productions using these:

S → aSb | ab becomes: S → ASB | AB

Step 2: Convert to CNF Format

The rule S → ASB is not in CNF because the right-hand side has three symbols.

We introduce a new variable: C → SB

Now, rewrite S → ASB as:

S → AC

C → SB
 EXAMPLES

1. 2.
 Greibach Normal Form (GNF)

● A non-terminal that generates a terminal. For instance, X →

x.
● A start symbol that generates ε. For instance, S → ε.
● A non-terminal that generates a terminal followed by any
number of non-terminals. For instance, S → xXSY.
 EXAMPLES

S → AY

X → x | SX

Y→y

A→x
EXAMPLES
 Pushdown Automata
A Pushdown Automata (PDA) is a way to implement context-free
Grammar
1. It is more powerful than FSM.
2. FSM has very limited memory but PDA has more memory.
3. PDA= Finite State Machine + Stack
Block diagram of pushdown automata

A Pushdown Automata has three Components:

1. An input tape
2. A finite Control Unit.
3. A stack with infinite size.
 Formal Definition
A Pushdown Automata (PDA) can be defined as as a 7-tuple (Q, ∑, Γ, δ, q0, I, F):
● Q is the set of states
● ∑is the set of input symbols
● Γ is the set of pushdown symbols (which can be pushed and popped from stack)
● q0 is the initial state
● Z is the initial pushdown symbol (which is initially present in stack)
● F is the set of final states
● δ is a transition function which maps Q x {Σ ∪ ∈} x Γ into Q x Γ*. In a given state, PDA
will read input symbol and stack symbol (top of the stack) and move to a new state
and change the symbol of stack.
Push,Pop and NOOP/Skip Operations on PDA
 Construct a PDA for the language { 0^n 1^n | n >=1 }
To construct a PDA accepting by empty stack for the language L = {0^n1^n |
n>=1}, we can follow these steps:
1. Push a special symbol Z0 (not in the input alphabet) to the stack to mark the
bottom of the stack.
2. Read the input symbol 0 and push it to the stack.
3. Repeat step 2 until all 0s are read.
4. Read the input symbol 1 and pop the topmost symbol (which should be 0).
5. Repeat step 4 until all 1s are read.
6. Check if the stack is empty/contains only Z0 at this point. If yes, accept the
input.
Construct a PDA for the language { 0^n 1^n | n >=1 }
Transition Diagram
 Construct a PDA for the language { 0^n 1^n | n >=1 }
Check the IP string: Transition
w = 0011 Diagram

Instantaneous Description (ID)

Transition function δ of a (PDA)

Construct a PDA for the language { a^n b^n | n >=1 }
Construct a PDA for the language { a^n b^n | n >=1 }

Check the
IP string:
w=aabb
Construct a PDA for the language { a^n b^2n | n >=1 }
Construct a PDA for the language { a^n b^2n | n >=1 }
Construct a PDA for the language {a^n b^n U a^n b^2n | n >=1 }
Construct a PDA for the language {a^n b^n U a^n b^2n | n >=1 }
 Two stack PDA for L= aⁿbⁿcⁿ |n>=1
Push 'a' into Stack 1: For every 'a' in the input, push it onto the first stack.

Pop 'a' and Push 'b' into Stack 2: When 'b' appears, pop 'a' from Stack 1 and push 'b' onto Stack 2.

Pop 'b' for each 'c': When 'c' appears, pop 'b' from Stack 2.

Accept if Both Stacks are Empty: If the input is fully read and both stacks are empty, the string is
accepted.
 Two stack PDA
L=aⁿbⁿcⁿdⁿ |n>=1
Push 'a' into Stack 1: For each 'a' in the input, push it onto Stack 1.

Pop 'a' and Push 'b' into Stack 2: When 'b' appears, pop 'a' from Stack
1 and push 'b' onto Stack 2.

Pop 'b' and Push 'c' into Stack 1: When 'c' appears, pop 'b' from Stack
2 and push 'c' onto Stack 1.

Pop 'c' for each 'd': When 'd' appears, pop 'c' from Stack 1.

Accept if Both Stacks are Empty: If the input is fully read and both
stacks are empty, the string is accepted.
 PDA for wcw^R w ε (a,b)+
STACK Transition Function

δ(q0, a, Z) = (q0, aZ)

δ(q0, a, a) = (q0, aa)
δ(q0, b, Z) = (q0, bZ)
δ(q0, b, b) = (q0, bb)
δ(q0, a, b) = (q0, ab)
δ(q0, b, a) = (q0, ba)

// Decision step
δ(q0, c, a) = (q1, a)
δ(q0, c, b) = (q1, b)

δ(q1, b, b) = (q1, ε)
δ(q1, a, a) = (q1, ε)

δ(q1, ε, Z) = (qf, Z)
Example : Process the input string: "abbcbba" for the above DPDA

Scan string from left to right

First input is 'a' and follow the rule:
on input 'a' and STACK alphabet Z, push the two 'a's into STACK as : (a,Z/aZ) and state will be q0
Second input is 'b' and so follow the rule:
on input 'b' and STACK alphabet 'a', push the 'b' into STACK as : (b,a/ba) and state will be q0
Third input is 'b' and so follow the rule:
on input 'b' and STACK alphabet 'b', push the 'b' into STACK as : (b,b/bb) and state will be q0
Fourth input is 'c' and so follow the rule:
on input 'c' and STACK alphabet 'a' or 'b' and state q0, do nothing as : (c,b/b) and state will be q1
Fifth input is 'b' and so follow the rule:
on input 'b' and STACK alphabet 'b' (state is q1), pop one 'b' from STACK as : (b,b/ε) and state will be q1
Sixth input is 'b' and so follow the rule:
on input 'b' and STACK alphabet 'b' (state is q1), pop one 'b' from STACK as : (b,b/ε) and state will be q1
Seventh input is 'a' and so follow the rule:
on input 'a' and STACK alphabet 'a' and state q1, pop one 'a' from STACK as : (a,a/ε) and state will remain q1
We reached end of the string, so follow the rule:
on input ε and STACK alphabet Z, go to final state(qf) as : (ε, Z/Z)
NPDA for wwR w ε (a,b)+ (EVEN PALINDROME)

Eg: w=ab
DPDA Vs NPDA
 Pumping Lemma for CFLs
1. Purpose: Used to prove that a language is not context-free by showing
that no matter how a string is divided, it cannot be pumped while staying in
the language.
2. Existence of a Constant p: There exists a pumping length p (depends on
the language) such that any string w with ∣w∣≥p can be split as: w =
uvwxy
3. Contradiction Approach: To prove L is not context-free:
● Assume L is context-free.
● Pick a string w with ∣w∣≥p
● Show that for every possible split, at least one pumped version
● This contradiction proves L is not context-free.
Pumping Lemma for CFLs
Example1
Example2
Example2

Let z = aabbbb
 Closure properties of Context-Free Language
Context-Free languages are closed under:
a. Union
b. Concatenation
c. Kleene Closure
Not closed under:
a. Intersection
b. Complementation
Union
Union Example 2:
Concatenation
Concatenation Example 2
 Kleene Closure

L1 = L* is also a CFL
Intersection
Complement

Which is invalid,
therefore , not closed
under complement
 LL(k) Grammar
If k=1, then we have, LL(1) Grammar
● 1st L indicates → Reading input string from left to right.
● 2nd L indicates → Leftmost derivation
● 1 indicates →Looking ahead terminal symbol in the input string.
● Used for Top Down Parsing Explicit Parse Tree
Example: Consider the Grammar:
Implicit Parse Tree

& the string,

w=aaabd
Determine
Whether the
given grammar
is LL(1) or not.
Example2: Consider the following grammar and the string w=abd. Determine
whether the given grammar is LL(1) or not
LL(k) Grammar
Introduction to Turing Machine
FA

PDA

TM
TMs are able to do any of the following Operations on Tape
1) Read a symbol from the tape.
2) Write a symbol to the tape.
3) Move the tape head, one step LEFT.
4) Move the tape head, one step RIGHT.
 Components of Turing Machine
TAPE:
A 2-way infinite tape, divided into cells.Each cell is capable of holding only one tape symbol.
Purpose: Acts as the machine’s memory, allowing it to read from and write to the tape.

TAPE HEAD:

A read/write head that moves along the tape. It can read the symbol in the current cell, write a new
symbol, and move left or right.

Purpose: Facilitates interaction with the tape by reading symbols and updating them.

FINITE CONTROL UNIT:

The finite control unit is the part of the Turing Machine that contains the transition function and
manages the state transitions.
Purpose: It acts as the decision-making component that directs the machine's actions based on its
current state and the input symbol it encounters
Formal Definition
M = (Q, Σ, Γ, δ, q0, B, F)

● Q: The finite set of states

● Σ: The finite set of input symbols.
● Γ: The complete set of tape symbols, Σ is always a subset of Γ.
● δ: The transition function. Defines the next state assumed by the machine, the symbol

to be written in the tape, and the direction which the head will moves. δ:Q×Γ→Q×Γ×
The transition δ(q0, a) = (q1, X, R), means that at q0 state if ‘a’ occurs
{L,R} then replace it with ‘X’ ,change the state to q1 and move to right.
● q0: The start state, a member of Q, where the finite control is found initially.
● B: the blank symbol. This symbol is in Γ but not in Σ.
● F: the set of final or accepting states, a subset of Q.
 Modifications of Turing Machine
1. Multi-tape Turing Machine
2. Turing Machine with stay option d: Q ×ℾ -> Q× ℾ × (L/R/S)
3. Multi-dimensional Turing Machine d: Q ×ℾ -> Q× ℾ × (L/R/U/D)
4. Turing Machine with one way infinite tape (Semi infinite tape)
5. Multi Read/Write head points
6. Multi-head Turing Machine
7. Offline TM: have a restriction that input can’t be changed (2 tapes)
8. Jumping TM: where the tape head can jump to non-adjacent positions on the tape
9. Non erasing TM: where input can not be converted to blank
10. Always writing TM
11. UTM
12. Non-Deterministic Turing Machine
13. TM without Writing Capacity (FA)
14. TM with Tape used as Stack
15. TM with finite tape
16. TM with Input Size Tape (LBA)
Church’s Thesis
 Church’s Thesis
According to the Church-Turing thesis:

-Anything that can be done on any existing digital computer can also be done by a Turing
machine/ Everything that can be computed, can be computed by a Turing machine / What
could naturally be called as an effective procedure, can be realized by a turing machine.

- This thesis cannot be proven but is believed on the basis of the amount of evidence that
supports it. No one has yet been able to suggest a problem, solvable by what we intuitively
consider an algorithm, for which a Turing machine program cannot be written.

- Computer scientists have devised several alternative models of computation, but none of
them is more powerful than the Turing machine model.

"Any real-world computation can be translated into an equivalent Turing machine

computation."
Design a Turing machine for a^n b^n | n ≥ 1
Design a Turing machine for a^n b^n c^n| n ≥ 1
Turing machine for 2’s Complement

Let M be the Turing Machine for above AIM, hence it can be define as M(Q, Σ, Г, δ, q0, B, F) where
Q: set of states: {q0, q1, q2}
Σ: set of input symbols: {0, 1}
Г: Set of Tape symbols: {0, 1, B}
q0: initial state (q0)
B: Blank Symbol (B)
F: set of Final states: { } [Note: Here, set of final states is null as here we have to design turing
machine as enumerator.]
δ: Transition Function:
Turing machine for 2’s Complement

1. Move the input head at right direction, till it reaches to B. If B encounter then
change state from q0 to q1.
2. Now, start reading input symbol right to left one by one.
3. If the input symbol is 0, move the input head left and do the same for all 0 till 1
encounter.
4. If the input symbol is 1, move the input head left and change its state from q1 to
q2.
5. Now, if the input symbol is 0, make it 1 and if input symbol is 1 make it 0 till
reach to the starting point.
 Transition Table

Transition Diagram
Turing machine for L = {w c w | w ∊ {0, 1}* }?
Turing machine for addition of two number in unary
Turing machine for converting unary number into binary number
 Universal Turing Machine (UTM)
Motivation: We need a single m/c which can solve all type of problems
❖ A Universal Turing Machine is a reprogrammable Turing machine capable of simulating the
behavior of any other Turing machine, effectively serving as a programmable computer that can
execute any computable function.

❖ A standard Turing machine can simulate only a single computation problem, so if we want to
solve another computation problem, then another Turing machine is required to be constructed,
but in case of a UTM, the same UTM can be used to solve any computation problem.

❖ Power of a UTM is the same as the power of a standard TM as both solve the same set of
computational problems. (also a digital computer)

❖ Turing complete means a system can perform any computation that a Turing machine can,
given enough time and memory.
 Universal Turing Machine (UTM)
Input to UTM is description of TM () and string w.
It consists of 3 tapes:
1. 1st tape stores the binary encoding of TM.
2. 2nd tape stores string w.
3. 3rd tape stores the state of TM.
Action: Accept, Reject, Loop
 Linear Bounded Automaton (LBA)

An LBA is a restricted type of Turing Machine where the tape is bounded by the length of the input.
LBAs are precisely the machines that recognize context-sensitive languages.
LBA is more powerful than PDA but less powerful than unrestricted Turing Machines.

A linear bounded automaton can be defined as an 8-tuple (Q, X, ∑, q0, ML, MR,
δ, F) where −

Q is a finite set of states

X is the tape alphabet
∑ is the input alphabet
q0 is the initial state
ML is the left end marker
MR is the right end marker where MR ≠ ML
δ is a transition function
F is the set of final states
 Recursive & Recursive Enumerable Language
Recursive Language (REC) {Turing Dec.}
A recursive language (subset of RE) can be decided by Turing machine which means it will
enter into final state for the strings of language and rejecting state for the strings which are
not part of the language. The TM will always halt in this case. REC languages are also
called as Turing decidable languages. Recursive languages are not closed under following
operations • Kleene closure • Homomorphism • Substitution
Recursive Enumerable (RE) or Type -0 Language {Turing Rec.}
RE languages or type-0 languages are generated by type-0 grammars. An RE language can
be accepted or recognized by Turing machine which means it will enter into final state for
the strings of language and may or may not enter into rejecting state for the strings which
are not part of the language. It means TM can loop forever for the strings which are not a
part of the language. RE languages are also called as Turing recognizable languages. RE are
not closed under following operations • Complement • Set Difference
 Halting Problem
Halt : The state of Turing machine, where no transition is defined or required is
known as Halt.
After taking an input string, there are three possibilities for Turing Machine, i.e. it
may go to:
● Final halt
● Non- Final halt
● Infinite loop
In case of Final Halt, Machine halts on final state, and hence it shows that string
is accepted. In Non-Final Halt, Machine halts on non- final state, and hence the
string is rejected.If the machine goes to infinite loop after taking an input string,
then we can’t say whether the string is Accepted/Rejected.
 Halting Problem
The Halting Problem in Automata theory is a problem that asks whether a
given computer program (Turing machine) will eventually stop running (halt)
or continue to run forever when provided with a specific input.
Problem: Determine if a program halts on a given input.
The answer is no we cannot design a generalized algorithm which can
appropriately say that given a program will ever halt or not?
The only way is to run the program and check whether it halts or not.
 What is the Halting Problem of a TM?

This is the classic undecidable problem:

Given a Turing Machine M and an input string w,

Does M eventually halt on w?

Halting means:

● TM stops (accepts or rejects).

● Not looping forever.

Alan Turing proved: No algorithm can decide whether any TM halts on all
inputs.
That’s what we call undecidable.
 Turing Machine as Computer of Integer Functions

● An integer function is a function that accepts integers as input and produces an

integer as output.

● Let an integer function be f (x,y) = x+y. We have to solve this function using Turing
Machine. so , input tape must represent the input arguments.

● Let the number of '0' be used to represent the argument. But there must be a separator
to differentiate the number of '0' for each argument. In Turing machine, blank space is
used to separate the arguments on the tape.
Undecidability
 Undecidability Problems
Undecidable problems are decision problems where no algorithm can be constructed to always provide a correct "yes" or "no"

Eg. Fermat's Last theorem is undecidable, which states that no three positive integers x, y, and z can satisfy the following equation
for any integer value of n >2.

xⁿ + yⁿ = zⁿ

1. The Halting Problem:

Question:
Given a program and an input, will the program eventually halt (stop running) or run forever?
Why it's undecidable:
It's impossible to create an algorithm that can always correctly determine if any arbitrary program will halt. Alan Turing proved this
in 1936, demonstrating the existence of undecidable problems.

2. The Post Correspondence Problem (PCP):

Question:
Given a list of pairs of strings (dominoes), can you arrange them in a way such that concatenating the top strings equals the
concatenation of the bottom strings?
Why it's undecidable:
There's no algorithm that can always determine if such an arrangement exists.
 Post Correspondence Problem (PCP)

It is a popular undecidable problem that was introduced by Emil Leon Post in

1946.

The problem involves two lists of words over a common alphabet.

The challenge is to find a sequence of indices where the concatenation of the
words in the first list, in that sequence, is equal to the concatenation of the words
in the second list in the same sequence

Def 2: we have N number of Dominos (tiles). The aim is to arrange tiles in such
order that string made by Numerators is same as string made by Denominators.
 Post Correspondence Problem
Q. Find whether the lists M = (ab, bab, bbaaa) and N = (a, ba, bab) have a post
correspondence solution? For x2 x1 x3 ∆ y2 y1 y3

Q. A = {1, 110, 0111} • B = {111, 001, 11}

 Post Correspondence Problem

Find whether the lists M = (abb, aa, aaa) and N = (bba,

aaaa, aa) have a post correspondence solution.
 PCP Examples

2113
PCP Examples
Check does PCP with two lists X= (0101, 000111, 001, 10, 01,
00) and Y= (0101000, 11, 1001,
100, 10, 0) have a solution?
 Reducibility
Reducibility means to transfer knowledge about one problem to another
Formal definition: If there exists a computable function f such that: x ∈ P₁ ⇔ f(x) ∈
P₂

Then, P₁ ≤ P₂ or P₁ ≤ₘ P₂ , (P₁ is reducible to P₂)

If we know P₂ is decidable, and P₁ reduces to P₂, then P₁ is also decidable.
If we know P₁ is undecidable, and P₁ reduces to P₂, then P₂ is also undecidable

Let +ve → Decidable

-ve → Undecidable, them
 Reducibility
If P₁ is decidable, then P₂ ________?

If P₁ is undecidable, then P₂________?

If P₂ is decidable, then P₁________?

If P₂ is undecidable, then P₁________?

Q1. Given P₁ ≤ P₂ and P₂ ≤ P₃

a) P₁ is decidable
P₃ is undecidable, then,

P₂ is ________ ?

Q2. Given L₁ ≤ L₂ and L₂ ≤ L₃

1) If L₂ is decidable then L₁ is ______ and L₃ is ______

2) If L₂ is undecidable then L₁ is ______and L₃ is ______

 Undecidability of the Post Correspondence Problem
PROOF:
Approach: Take a problem that is already proven to be undecidable and try to convert it
to PCP. If we can successfully convert it to an equivalent PCP then we prove that PCP is
also undecidable.

ACCEPTANCE (Halting) PROBLEM OF A TURING MACHINE (This is an undecidable

problem)
↓
Convert this to PCP (called Modified PCP - MPCP)

So, to prove that PCP is undecidable, we want to reduce the TM halting problem to
PCP.

So we take:

● A Turing Machine M and an input string w = aba

 Undecidability of the Post Correspondence Problem

Given:
● TM States: q0, q1, q2 (final)

● Input: w = aba

● Alphabet:

○ Input symbols: Σ = {a, b}

○ Tape symbols: Γ = {a, b, x, B} (includes blank B and symbol x used for marking)
 Undecidability of the Post Correspondence Problem
We have a Domino [#/#] , and the given TM Transitions are :

1. From q0, if a → write x, move Right, go to q1

Notation: q0 -- a → x, R --> q1

2. From q1, if b → write x, move Left, go to q2

Notation: q1 -- b → x, L --> q2

Now we need to follow the following steps to convert the given Turing m/c into a format that can be simulated
using Post Correspondence Problem (PCP).
Step 1:

This becomes the starting

tile in the PCP setup
 Undecidability of the Post Correspondence Problem
Step 2:
 Undecidability of the Post Correspondence Problem
Step 3:
Undecidability of the Post Correspondence Problem

Step 4:

q2
 Undecidability of the Post Correspondence Problem
Step 6:
 Undecidability of the Post Correspondence Problem

Solution: From Step 1, 2 ,4 and 6

From Step 3, 4 and 6

From Step 5,4 and 6

From Step 5,4 and 6

From Step 5 and 6

From Step 7

So,we could reduce are acceptance

problem of TM TO our PCP problem,
hence PCP is also undecidable
 Recursive Function Theory
Not all functions, even those with precise definitions, can be computed by Turing Machines.
Example: Busy Beaver Functions
If you can define a given function using initial primitive recursive functions, then it is a Turing computable function/primitive
recursive function.
So, some initial functions are taken as primitive recursive functions.
Recursive Function Theory
 Recursive Function Theory
Recursive Function
A recursive function is a function that calls itself during its execution. Eg:factorial

Partial Recursive Function

A function is called a partial recursive function if it is defined for some of its arguments. They may not produce
a result for every possible input. In other words, they can be undefined for some inputs.

Example: Subtraction of two positive numbers m and n, f(m, n) = m-n; if m>=n, otherwise it's not defined.

Primitive Recursive Functions: These are a subset of recursive functions that are defined using only basic
initial functions (such as zero, successor, and projection functions) and can be formed through a finite
combination of composition and recursion. They are always total functions, meaning they produce a result for
every possible input. (We can create recursive func using the basic initial func)
Complexity Theory
 Complexity Theory

Solvable - A problem is said to be solvable if either we can solve it or if we can prove that the
problem cannot be solved

Unsolvable - A problem is said to be unsolvable if neither we can solve it, nor we can proof
that the problem can not be solved
Decidable- Decidable problem is one for which there exists an algorithm that can determine
the answer (yes or no) for every input in a finite amount of time. In terms of formal languages,
a language is decidable if there exists a halting Turing machine that always decides
membership in the language. Such a language is also called a recursive language
Undecidable-Undecidable problems are those for which no algorithm can determine the
correct answer (yes or no) for all possible inputs. In other words, there is no Turing machine
that always halts with a correct decision for every input. In terms of formal languages, a
language is undecidable if there does not exist a halting Turing machine that decides
membership in the language. Such languages are called non-recursive languages.
 Complexity Classes
Polynomial Time Algorithms

● Linear Search (O(n))

● Binary Search (O(log n))
● Insertion Sort (O(n²))
● Merge Sort (O(n log n))

Non-Polynomial Time (Exponential Time)

Algorithms
Ex:If n = 10,
● 0/1 Knapsack (2ⁿ)
● Traveling Salesman (2ⁿ) then (10)² = 100 and 2¹⁰ = 1024
● Graph Coloring (2ⁿ)
● Sudoku (2ⁿ)
● Scheduling (2ⁿ)
 P-Class Problems
■ P (polynomial time) problems are a set of problems that can be solved in polynomial time by
deterministic algorithms. A deterministic algorithm is an algorithm that always follows a fixed
sequence of steps and produces the same output for the same input every time.
■ P is also known as PTIME or DTIME complexity class.
■ P problems are a set of all decision problems which can be solved in polynomial time using the
deterministic Turing machine.
■ They are simple to solve, easy to verify and take computationally acceptable time for solving any
instance of the problem. Such problems are also known as “tractable”.

Examples of P Problems:

■ Insertion sort
■ Merge sort
■ Linear search
■ Matrix multiplication
■ Finding minimum and maximum elements from the array
 NP-Class Problems
■ NP (Non-deterministic polynomial time) problems are a set of problems that can not be solved in
polynomial time, but given the solution, it can be verified in polynomial time.
■ NP includes all problems of P, i.e. P ⊆ NP.

Examples of NP problems: Knapsack problem (O(2n)), Travelling salesman

problem (O(n!)), Tower of Hanoi (O(2n – 1)), Hamiltonian cycle (O(n!))

Polynomial time Reduction
 NP-Hard & NPC (NP-Complete) Problems
NP-Hard: A decision problem p is called NP-hard, if
every problem in NP can be reduced to p in
polynomial time.
● If we can solve any NP-hard problem in polynomial time,
we would be able to solve all the problems in NP in
polynomial time.
● A well-known example of the NP-hard problem is the
Halting problem. The halting problem is stated as, “Given
an algorithm and set of inputs, will it run forever ?” The
answer to this question is Yes or No, so this is a decision
problem.
● There does not exist any known algorithm which can
decide the answer for any given input in polynomial time.
So halting problem is an NP-hard problem.
 NP-Hard & NPC (NP-Complete) Problems
NPC (NP-Complete):A decision problem A is called
NP-complete if it has the following two properties :
■ It belongs to class NP.
■ It is NP-Hard
Examples of the well-known NP-complete problems are :

■ Boolean satisfiability problem.

■ Knapsack problem.
■ Hamiltonian path problem.
■ Travelling salesman problem.
■ Subset sum problem.
■ Vertex covers the problem.
■ Graph colouring problem.
■ Clique problem.
 Space Complexity
The Power of Space: Space seems to be more powerful than time because space can be
reused.

What is Space Complexity of a Turing Machine?

Measuring Space: A Turing Machine M runs in space S(n) if, for all inputs of length n, M
uses at most S(n) cells in total on its work tapes.

Space Complexity of a Turing machine = The maximum number of tape cells it scans
or uses during computation on any input of size n.

It is a function f(n), where n is the length of the input string.

Formal Definition: Let M be a Turing machine, and let f(n) be a function, We say that
M runs in space f(n) if : For every input of length n, the Turing machine never scans
more than f(n) tape cells on any computation branch (for non-deterministic machines,
on any branch)
 Space Complexity
The Turing machine never scans more than f(n) tape cells on any computation
branch (for non-deterministic machines, on any branch), i.e

● No matter which path (or set of choices) the NDTM follows,

● It never uses more than f(n) space (tape cells) along that path.

So even if there are many possible branches, each one individually must obey the
space limit f(n).
 Space Complexity Classes
1. DSPACE(f(n))

● Problems solvable by a Deterministic Turing Machine (DTM) using at most f(n)

space.
● Example: DSPACE(n) means space grows linearly with input, for instance, SAT
problem can be decided in linear space

2. NSPACE(f(n))

● Problems solvable by a Non-deterministic Turing Machine (NTM) using at most

f(n) space on any branch of computation.
● Example: NSPACE(n^2) = all problems solvable using quadratic space,
non-deterministically
 Space Complexity Classes
3. PSPACE: Class of all problems decidable by a deterministic TM using polynomial
space.
PSPACE = ⋃ DSPACE(n^k), for all k ∈ ℕ

● Includes problems solvable in space: O(n), O(n²), O(n³), ...

● Contains P, NP, and many more

● Example problems: Quantified Boolean Formula (QBF)

4. NPSPACE: Class of problems decidable by a non-deterministic TM using polynomial

space.

NPSPACE = ⋃ NSPACE(n^k), for all k ∈ ℕ

Note: By Savitch’s Theorem, PSPACE = NPSPACE, As it proves that non-deterministic

space doesn’t give more power than deterministic space for polynomial bounds.
 Boolean Satisfiability Problem (SAT Problem)
This problem determines whether there exists a set of Boolean variable assignments
that satisfy a given Boolean formula.

Objective: We aim to find values for Boolean

variables (TRUE, FALSE, or 1, 0)

such that the Boolean formula

evaluates to TRUE (or 1)

Input: A Boolean Formula

Output: Find for which combination

of the variables (in the form of TRUE and FALSE),

the Boolean formula becomes TRUE.

Steps for proving NP-Complete:

Step 1: Prove that B is in NP

Step 2: Select an NP-Complete Language A.

Step 3: Construct a function f that maps members of A to members of B.

Step 4: Show that x is in A iff f(x) is in B.

Step 5: Show that f can be computed in polynomial time.

 Complexity Theory

Problem

Solvable Unsolvable

Decidable Undecidable

P-Type NP-Type
 Ackermann Function
The Ackermann function is a two-parameter function that takes non-negative integers as
inputs and produces a non-negative integer as its result.The Ackermann function is total. The
function is defined as follows:

A(m,n)= {n+1, if m=0

A(m−1,1) if m>0 and n=0
A(m−1,A(m,n−1)) if m>0 and n>0
 UNIT 1: DFA, NFA, EQ., MINMZTN
IMPORTANT TOPICS UNIT 2: RE, ARDEN’S, PL

UNIT 3: CFG, CFG SMPLFCTN,

AMBIGUOUS, RLG, RLG,
CHOMSKY,CNF,GNF

UNIT 4: NPDA
DPDA
Pumping Lemma for CFL
Closure Properties of CFL
Decision Problems of CFL

UNIT 5: TM, UTM, LBA, PCP