Automata Theory Handout
Automata Theory Handout
The term "Automata" is derived from the Greek word "αὐτόματα" which means "self-
acting". An automaton (Automata in plural) is an abstract self-propelled computing
device which follows a predetermined sequence of operations automatically.
An automaton with a finite number of states is called a Finite Automaton (FA)
or Finite State Machine (FSM).
Related Terminologies
Alphabet
String
Length of a String
Kleene Star
Definition − The Kleene star, ∑*, is a unary operator on a set of symbols or strings, ∑, that
gives the infinite set of all possible strings of all possible lengths over ∑ including λ.
Representation − ∑* = ∑0 ∪ ∑1 ∪ ∑2 ∪……. where ∑p is the set of all possible strings of
length p.
Example − If ∑ = {a, b}, ∑* = {λ, a, b, aa, ab, ba, bb,………..}
Definition − The set ∑+ is the infinite set of all possible strings of all possible lengths over ∑
excluding λ.
Representation − ∑+ = ∑1 ∪ ∑2 ∪ ∑3 ∪…….
∑+ = ∑* − { λ }
Example − If ∑ = { a, b } , ∑+ = { a, b, aa, ab, ba, bb,………..}
Language
Example
Q = {a, b, c},
∑ = {0, 1},
q0 = {a},
F = {c}, and
Transition function δ as shown by the following table −
Present State Next State for Input 0 Next State for Input 1
a a b
b c a
c b c
In NDFA, for a particular input symbol, the machine can move to any combination of
the states in the machine. In other words, the exact state to which the machine moves
cannot be determined. Hence, it is called Non-deterministic Automaton. As it has
finite number of states, the machine is called Non-deterministic Finite
Machine or Non-deterministic Finite Automaton.
Q = {a, b, c}
∑ = {0, 1}
q0 = {a}
F = {c}
The transition function δ as shown below −
Present State Next State for Input 0 Next State for Input 1
a a, b b
b c a, c
c b, c c
DFA vs NDFA
The following table lists the differences between DFA and NDFA.
DFA NDFA
The transition from a state is to a single particular next The transition from a state can be to multiple next
state for each input symbol. Hence it is states for each input symbol. Hence it is called non-
called deterministic. deterministic.
Empty string transitions are not seen in DFA. NDFA permits empty string transitions.
A string is accepted by a DFA, if it transits to a final state. A string is accepted by a NDFA, if at least one of all
possible transitions ends in a final state.
An automaton that computes a Boolean function is called an acceptor. All the states of
an acceptor is either accepting or rejecting the inputs given to it.
Classifier
A classifier has more than two final states and it gives a single output when it
terminates.
Transducer
An automaton that produces outputs based on current input and/or previous state is
called a transducer. Transducers can be of two types −
Mealy Machine − The output depends both on the current state and the current input.
Moore Machine − The output depends only on the current state.
Problem Statement
Let X = (Qx, ∑, δx, q0, Fx) be an NDFA which accepts the language L(X). We have to
design an equivalent DFA Y = (Qy, ∑, δy, q0, Fy) such that L(Y) = L(X). The following
procedure converts the NDFA to its equivalent DFA −
Algorithm
Input − An NDFA
Output − An equivalent DFA
Step 1 − Create state table from the given NDFA.
Step 2 − Create a blank state table under possible input alphabets for the equivalent
DFA.
Step 3 − Mark the start state of the DFA by q0 (Same as the NDFA).
Step 4 − Find out the combination of States {Q 0, Q1,... , Qn} for each possible input
alphabet.
Step 5 − Each time we generate a new DFA state under the input alphabet columns,
we have to apply step 4 again, otherwise go to step 6.
Step 6 − The states which contain any of the final states of the NDFA are the final
states of the equivalent DFA.
Example
Let us consider the NDFA shown in the figure below.
q δ(q,0) δ(q,1)
a {a,b,c,d,e} {d,e}
b {c} {e}
c ∅ {b}
d {e} ∅
e ∅ ∅
Using the above algorithm, we find its equivalent DFA. The state table of the DFA is
shown in below.
q δ(q,0) δ(q,1)
[d,e] [e] ∅
[b,d,e] [c,e] [e]
[e] ∅ ∅
[c, e] ∅ [b]
[c] ∅ [b]
Input − DFA
Output − Minimized DFA
Step 1 − Draw a table for all pairs of states (Q i, Qj) not necessarily connected directly
[All are unmarked initially]
Step 2 − Consider every state pair (Q i, Qj) in the DFA where Qi ∈ F and Qj ∉ F or vice
versa and mark them. [Here F is the set of final states]
Step 3 − Repeat this step until we cannot mark anymore states −
If there is an unmarked pair (Qi, Qj), mark it if the pair {δ (Q i, A), δ (Qi, A)} is marked for
some input alphabet.
Step 4 − Combine all the unmarked pair (Qi, Qj) and make them a single state in the
reduced DFA.
Example
a b c d e f
a
b
a b c d e f
c ✔ ✔
d ✔ ✔
e ✔ ✔
f ✔ ✔ ✔
Step 3 − We will try to mark the state pairs, with green colored check mark, transitively.
If we input 1 to state ‘a’ and ‘f’, it will go to state ‘c’ and ‘f’ respectively. (c, f) is already
marked, hence we will mark pair (a, f). Now, we input 1 to state ‘b’ and ‘f’; it will go to
state ‘d’ and ‘f’ respectively. (d, f) is already marked, hence we will mark pair (b, f).
a b c d e f
a
c ✔ ✔
d ✔ ✔
e ✔ ✔
f ✔ ✔ ✔ ✔ ✔
After step 3, we have got state combinations {a, b} {c, d} {c, e} {d, e} that are
unmarked.
We can recombine {c, d} {c, e} {d, e} into {c, d, e}
Hence we got two combined states as − {a, b} and {c, d, e}
So the final minimized DFA will contain three states {f}, {a, b} and {c, d, e}
Step 1 − All the states Q are divided in two partitions − final states and non-final
states and are denoted by P0. All the states in a partition are 0th equivalent. Take a
counter k and initialize it with 0.
Step 2 − Increment k by 1. For each partition in P k, divide the states in Pk into two
partitions if they are k-distinguishable. Two states within this partition X and Y are k-
distinguishable if there is an input S such that δ(X, S) and δ(Y, S) are (k-1)-
distinguishable.
Step 3 − If Pk ≠ Pk-1, repeat Step 2, otherwise go to Step 4.
Step 4 − Combine kth equivalent sets and make them the new states of the reduced
DFA.
Example
q δ(q,0) δ(q,1)
a b c
b a d
c e f
d e f
e e f
f f f
Q δ(q,0) δ(q,1)
Finite automata may have outputs corresponding to each transition. There are two
types of finite state machines that generate output −
Mealy Machine
Moore machine
Mealy Machine
A Mealy Machine is an FSM whose output depends on the present state as well as the
present input.
It can be described by a 6 tuple (Q, ∑, O, δ, X, q 0) where −
Q is a finite set of states.
∑ is a finite set of symbols called the input alphabet.
O is a finite set of symbols called the output alphabet.
δ is the input transition function where δ: Q × ∑ → Q
X is the output transition function where X: Q × ∑ → O
q0 is the initial state from where any input is processed (q0 ∈ Q).
The state table of a Mealy Machine is shown below −
Next state
→a b x1 c x1
b b x2 d x3
c d x3 c x1
d d x3 d x2
Moore machine is an FSM whose outputs depend on only the present state.
A Moore machine can be described by a 6 tuple (Q, ∑, O, δ, X, q 0) where −
Q is a finite set of states.
∑ is a finite set of symbols called the input alphabet.
O is a finite set of symbols called the output alphabet.
δ is the input transition function where δ: Q × ∑ → Q
X is the output transition function where X: Q → O
q0 is the initial state from where any input is processed (q0 ∈ Q).
The state table of a Moore Machine is shown below −
Next State
Present state Output
Input = 0 Input = 1
→a b c x2
b b d x1
c c d x2
d d d x3
The state diagram of the above Moore Machine is −
The following table highlights the points that differentiate a Mealy Machine from a
Moore Machine.
Output depends both upon the present state and the Output depends only upon the present state.
present input
Generally, it has fewer states than Moore Machine. Generally, it has more states than Mealy Machine.
The value of the output function is a function of the The value of the output function is a function of the
transitions and the changes, when the input logic on current state and the changes at the clock edges,
the present state is done. whenever state changes occur.
Mealy machines react faster to inputs. They generally In Moore machines, more logic is required to decode the
react in the same clock cycle. outputs resulting in more circuit delays. They generally
react one clock cycle later.
Example
Next State
Present State Output
a=0 a=1
→a d b 1
b a d 0
c c c 0
d b a 1
Next State
b a d
c c c
d b a
Step 3 −
Next State
=> a d 1 b 0
b a 1 d 1
c c 0 c 0
d b 0 a 1
Example
Next State
→a d 0 b 1
b a 1 d 0
c c 1 c 0
d b 0 a 1
Here, states ‘a’ and ‘d’ give only 1 and 0 outputs respectively, so we retain states ‘a’
and ‘d’. But states ‘b’ and ‘c’ produce different outputs (1 and 0). So, we
divide b into b0, b1 and c into c0, c1.
Next State
Present State Output
a=0 a=1
→a d b1 1
b0 a d 0
b1 a d 1
c0 c1 C0 0
c1 c1 C0 1
d b0 a 0
n the literary sense of the term, grammars denote syntactical rules for conversation in
natural languages. Linguistics have attempted to define grammars since the inception
of natural languages like English, Sanskrit, Mandarin, etc.
The theory of formal languages finds its applicability extensively in the fields of
Computer Science. Noam Chomsky gave a mathematical model of grammar in 1956
which is effective for writing computer languages.
Grammar
A grammar G can be formally written as a 4-tuple (N, T, S, P) where −
N or V is a set of variables or non-terminal symbols.
N
Example
Grammar G1 −
({S, A, B}, {a, b}, S, {S → AB, A → a, B → b})
Here,
S, A, and B are Non-terminal symbols;
a and b are Terminal symbols
S is the Start symbol, S ∈ N
Productions, P : S → AB, A → a, B → b
Example
Grammar G2 −
(({S, A}, {a, b}, S,{S → aAb, aA → aaAb, A → ε } )
Here,
S and A are Non-terminal symbols.
a and b are Terminal symbols.
ε is an empty string.
S is the Start symbol, S ∈ N
Production P : S → aAb, aA → aaAb, A → ε
Example
Example
If there is a grammar
G: N = {S, A, B} T = {a, b} P = {S → AB, A → a, B → b}
Here S produces AB, and we can replace A by a, and B by b. Here, the only accepted
string is ab, i.e.,
L(G) = {ab}
Example
Example
Problem − Suppose, L (G) = {am bn | m ≥ 0 and n > 0}. We have to find out the
grammar G which produces L(G).
Solution
Since L(G) = {am bn | m ≥ 0 and n > 0}
the set of strings accepted can be rewritten as −
L(G) = {b, ab,bb, aab, abb, …….}
Here, the start symbol has to take at least one ‘b’ preceded by any number of ‘a’
including null.
To accept the string set {b, ab, bb, aab, abb, …….}, we have taken the productions −
S → aS , S → B, B → b and B → bB
S → B → b (Accepted)
S → B → bB → bb (Accepted)
S → aS → aB → ab (Accepted)
S → aS → aaS → aaB → aab(Accepted)
S → aS → aB → abB → abb (Accepted)
Thus, we can prove every single string in L(G) is accepted by the language generated
by the production set.
Hence the grammar −
G: ({S, A, B}, {a, b}, S, { S → aS | B , B → b | bB })
Example
Problem − Suppose, L (G) = {am bn | m > 0 and n ≥ 0}. We have to find out the
grammar G which produces L(G).
Solution −
Since L(G) = {am bn | m > 0 and n ≥ 0}, the set of strings accepted can be rewritten as −
L(G) = {a, aa, ab, aaa, aab ,abb, …….}
Here, the start symbol has to take at least one ‘a’ followed by any number of ‘b’
including null.
To accept the string set {a, aa, ab, aaa, aab, abb, …….}, we have taken the
productions −
S → aA, A → aA , A → B, B → bB ,B → λ
S → aA → aB → aλ → a (Accepted)
S → aA → aaA → aaB → aaλ → aa (Accepted)
S → aA → aB → abB → abλ → ab (Accepted)
S → aA → aaA → aaaA → aaaB → aaaλ → aaa (Accepted)
S → aA → aaA → aaB → aabB → aabλ → aab (Accepted)
S → aA → aB → abB → abbB → abbλ → abb (Accepted)
Thus, we can prove every single string in L(G) is accepted by the language generated
by the production set.
Hence the grammar −
G: ({S, A, B}, {a, b}, S, {S → aA, A → aA | B, B → λ | bB })
Take a look at the following illustration. It shows the scope of each type of grammar −
Type - 3 Grammar
Type-3 grammars generate regular languages. Type-3 grammars must have a single
non-terminal on the left-hand side and a right-hand side consisting of a single terminal
or single terminal followed by a single non-terminal.
The productions must be in the form X → a or X → aY
where X, Y ∈ N (Non terminal)
and a ∈ T (Terminal)
The rule S → ε is allowed if S does not appear on the right side of any rule.
Example
X → ε
X → a | aY
Y → b
Type - 2 Grammar
Type-2 grammars generate context-free languages.
The productions must be in the form A → γ
where A ∈ N (Non terminal)
and γ ∈ (T ∪ N)* (String of terminals and non-terminals).
These languages generated by these grammars are be recognized by a non-
deterministic pushdown automaton.
Example
S → X a
X → a
X → aX
X → abc
X → ε
Type - 1 Grammar
Type-1 grammars generate context-sensitive languages. The productions must be in
the form
αAβ→αγβ
where A ∈ N (Non-terminal)
and α, β, γ ∈ (T ∪ N)* (Strings of terminals and non-terminals)
The strings α and β may be empty, but γ must be non-empty.
The rule S → ε is allowed if S does not appear on the right side of any rule. The
languages generated by these grammars are recognized by a linear bounded
automaton.
Example
AB → AbBc
A → bcA
B → b
Type - 0 Grammar
Type-0 grammars generate recursively enumerable languages. The productions have
no restrictions. They are any phase structure grammar including all formal grammars.
They generate the languages that are recognized by a Turing machine.
The productions can be in the form of α → β where α is a string of terminals and
nonterminals with at least one non-terminal and α cannot be null. β is a string of
terminals and non-terminals.
Example
S → ACaB
Bc → acB
CB → DB
aD → Db
Regular Expressions
A Regular Expression can be recursively defined as follows −
ε is a Regular Expression indicates the language containing an empty string. (L (ε) = {ε})
φ is a Regular Expression denoting an empty language. (L (φ) = { })
x is a Regular Expression where L = {x}
If X is a Regular Expression denoting the language L(X) and Y is a Regular Expression
denoting the language L(Y), then
o X + Y is a Regular Expression corresponding to the language L(X) ∪
L(Y) where L(X+Y) = L(X) ∪ L(Y).
o X . Y is a Regular Expression corresponding to the language L(X) .
L(Y) where L(X.Y) = L(X) . L(Y)
o R* is a Regular Expression corresponding to the language L(R*)where L(R*) =
(L(R))*
If we apply any of the rules several times from 1 to 5, they are Regular Expressions.
Some RE Examples
Regular Expressions Regular Set
(a+b)* Set of strings of a’s and b’s of any length including the null string. So L = { ε, a, b, aa ,
ab , bb , ba, aaa…….}
(a+b)*abb Set of strings of a’s and b’s ending with the string abb. So L = {abb, aabb, babb, aaabb,
ababb, …………..}
(11)* Set consisting of even number of 1’s including empty string, So L= {ε, 11, 1111,
111111, ……….}
(aa)*(bb)*b Set of strings consisting of even number of a’s followed by odd number of b’s , so L =
{b, aab, aabbb, aabbbbb, aaaab, aaaabbb, …………..}
(aa + ab + ba + bb)* String of a’s and b’s of even length can be obtained by concatenating any combination
of the strings aa, ab, ba and bb including null, so L = {aa, ab, ba, bb, aaab, aaba,
…………..}
Regular Sets
ny set that represents the value of the Regular Expression is called a Regular Set.
∅* = ε
ε* = ε
RR* = R*R
R*R* = R*
(R*)* = R*
RR* = R*R
(PQ)*P =P(QP)*
(a+b)* = (a*b*)* = (a*+b*)* = (a+b*)* = a*(ba*)*
R + ∅ = ∅ + R = R (The identity for union)
R ε = ε R = R (The identity for concatenation)
∅ L = L ∅ = ∅ (The annihilator for concatenation)
R + R = R (Idempotent law)
L (M + N) = LM + LN (Left distributive law)
(M + N) L = ML + NL (Right distributive law)
ε + RR* = ε + R*R = R*
Arden's Theorem
n order to find out a regular expression of a Finite Automaton, we use Arden’s
Theorem along with the properties of regular expressions.
Statement −
Let P and Q be two regular expressions.
If P does not contain null string, then R = Q + RP has a unique solution that is R = QP*
Proof −
R = Q + (Q + RP)P [After putting the value R = Q + RP]
= Q + QP + RPP
When we put the value of R recursively again and again, we get the following equation
−
R = Q + QP + QP2 + QP3…..
R = Q (ε + P + P2 + P3 + …. )
R = QP* [As P* represents (ε + P + P2 + P3 + ….) ]
Hence, proved.
Method
Step 1 − Create equations as the following form for all the states of the DFA having n
states with initial state q1.
q1 = q1R11 + q2R21 + … + qnRn1 + ε
q2 = q1R12 + q2R22 + … + qnRn2
…………………………
…………………………
…………………………
…………………………
qn = q1R1n + q2R2n + … + qnRnn
Rij represents the set of labels of edges from qi to qj, if no such edge exists, then Rij = ∅
Step 2 − Solve these equations to get the equation for the final state in terms of Rij
Problem
Construct a regular expression corresponding to the automata given below −
Solution −
Here the initial state and final state is q1.
The equations for the three states q1, q2, and q3 are as follows −
q1 = q1a + q3a + ε (ε move is because q1 is the initial state0
q2 = q1b + q2b + q3b
q3 = q2a
Now, we will solve these three equations −
q2 = q1b + q2b + q3b
= q1b + q2b + (q2a)b (Substituting value of q3)
= q1b + q2(b + ab)
= q1b (b + ab)* (Applying Arden’s Theorem)
q1 = q1a + q3a + ε
= q1a + q2aa + ε (Substituting value of q3)
= q1a + q1b(b + ab*)aa + ε (Substituting value of q2)
= q1(a + b(b + ab)*aa) + ε
= ε (a+ b(b + ab)*aa)*
= (a + b(b + ab)*aa)*
Hence, the regular expression is (a + b(b + ab)*aa)*.
Problem
Construct a regular expression corresponding to the automata given below −
Solution −
Here the initial state is q1 and the final state is q2
Now we write down the equations −
q1 = q10 + ε
q2 = q11 + q20
q3 = q21 + q30 + q31
Now, we will solve these three equations −
q1 = ε0* [As, εR = R]
So, q1 = 0*
q2 = 0*1 + q20
So, q2 = 0*1(0)* [By Arden’s theorem]
Hence, the regular expression is 0*10*.
Construction of an FA from an RE
We can use Thompson's Construction to find out a Finite Automaton from a Regular
Expression. We will reduce the regular expression into smallest regular expressions
and converting these to NFA and finally to DFA.
Some basic RA expressions are the following −
Case 1 − For a regular expression ‘a’, we can construct the following FA −
Case 2 − For a regular expression ‘ab’, we can construct the following FA −
Case 3 − For a regular expression (a+b), we can construct the following FA −
Case 4 − For a regular expression (a+b)*, we can construct the following FA −
Method
Step 1 Construct an NFA with Null moves from the given regular expression.
Step 2 Remove Null transition from the NFA and convert it into its equivalent DFA.
Problem
Convert the following RA into its equivalent DFA − 1 (0 + 1)* 0
Solution
We will concatenate three expressions "1", "(0 + 1)*" and "0"
Now we will remove the ε transitions. After we remove the ε transitions from the NDFA,
we get the following −
It is an NDFA corresponding to the RE − 1 (0 + 1)* 0. If you want to convert it into a
DFA, simply apply the method of converting NDFA to DFA discussed in Chapter 1.
Step 3 −
Here q1 is an initial state, so we make qf also an initial state.
So the FA becomes −
Step 4 −
Here qf is a final state, so we make q1 also a final state.
So the FA becomes −
Pumping Lemma For Regular Grammars
Theorem
Let L be a regular language. Then there exists a constant ‘c’ such that for every
string w in L −
|w| ≥ c
We can break w into three strings, w = xyz, such that −
|y| > 0
|xy| ≤ c
For all k ≥ 0, the string xykz is also in L.
DFA Complement
f (Q, ∑, δ, q0, F) be a DFA that accepts a language L, then the complement of the DFA
can be obtained by swapping its accepting states with its non-accepting states and vice
versa.
We will take an example and elaborate this below −
The derivation or the yield of a parse tree is the final string obtained by concatenating
the labels of the leaves of the tree from left to right, ignoring the Nulls. However, if all
the leaves are Null, derivation is Null.
Example
Let a CFG {N,T,P,S} be
N = {S}, T = {a, b}, Starting symbol = S, P = S → SS | aSb | ε
One derivation from the above CFG is “abaabb”
S → SS → aSbS → abS → abaSb → abaaSbb → abaabb
Sentential Form and Partial Derivation Tree
A partial derivation tree is a sub-tree of a derivation tree/parse tree such that either all
of its children are in the sub-tree or none of them are in the sub-tree.
Example
If in any CFG the productions are −
S → AB, A → aaA | ε, B → Bb| ε
the partial derivation tree can be the following −
If a partial derivation tree contains the root S, it is called a sentential form. The above
sub-tree is also in sentential form.
Solution
Let’s find out the derivation tree for the string "a+a*a". It has two leftmost derivations.
Derivation 1 − X → X+X → a +X → a+ X*X → a+a*X → a+a*a
Parse tree 1 −
Union
Concatenation
Kleene Star operation
Union
Let L1 and L2 be two context free languages. Then L1 ∪ L2 is also context free.
Example
Let L1 = { anbn , n > 0}. Corresponding grammar G1 will have P: S1 → aAb|ab
Let L2 = { cmdm , m ≥ 0}. Corresponding grammar G2 will have P: S2 → cBb| ε
Union of L1 and L2, L = L1 ∪ L2 = { anbn } ∪ { cmdm }
The corresponding grammar G will have the additional production S → S1 | S2
Concatenation
If L1 and L2 are context free languages, then L1L2 is also context free.
Example
Union of the languages L1 and L2, L = L1L2 = { anbncmdm }
The corresponding grammar G will have the additional production S → S1 S2
Kleene Star
If L is a context free language, then L* is also context free.
Example
CFG Simplification
n a CFG, it may happen that all the production rules and symbols are not needed for
the derivation of strings. Besides, there may be some null productions and unit
productions. Elimination of these productions and symbols is called simplification of
CFGs. Simplification essentially comprises of the following steps −
Reduction of CFG
Removal of Unit Productions
Removal of Null Productions
Reduction of CFG
CFGs are reduced in two phases −
Phase 1 − Derivation of an equivalent grammar, G’, from the CFG, G, such that each
variable derives some terminal string.
Derivation Procedure −
Step 1 − Include all symbols, W1, that derive some terminal and initialize i=1.
Step 2 − Include all symbols, Wi+1, that derive Wi.
Step 3 − Increment i and repeat Step 2, until Wi+1 = Wi.
Step 4 − Include all production rules that have Wi in it.
Phase 2 − Derivation of an equivalent grammar, G”, from the CFG, G’, such that each
symbol appears in a sentential form.
Derivation Procedure −
Step 1 − Include the start symbol in Y1 and initialize i = 1.
Step 2 − Include all symbols, Yi+1, that can be derived from Yi and include all production
rules that have been applied.
Step 3 − Increment i and repeat Step 2, until Yi+1 = Yi.
Problem
Solution
Phase 1 −
T = { a, c, e }
W1 = { A, C, E } from rules A → a, C → c and E → aA
W2 = { A, C, E } U { S } from rule S → AC
W3 = { A, C, E, S } U ∅
Since W2 = W3, we can derive G’ as −
G’ = { { A, C, E, S }, { a, c, e }, P, {S}}
where P: S → AC, A → a, C → c , E → aA | e
Phase 2 −
Y1 = { S }
Y2 = { S, A, C } from rule S → AC
Y3 = { S, A, C, a, c } from rules A → a and C → c
Y4 = { S, A, C, a, c }
Since Y3 = Y4, we can derive G” as −
G” = { { A, C, S }, { a, c }, P, {S}}
where P: S → AC, A → a, C → c
Removal of Unit Productions
Any production rule in the form A → B where A, B ∈ Non-terminal is called unit
production..
Removal Procedure −
Step 1 − To remove A → B, add production A → x to the grammar rule whenever B →
x occurs in the grammar. [x ∈ Terminal, x can be Null]
Step 2 − Delete A → B from the grammar.
Step 3 − Repeat from step 1 until all unit productions are removed.
Problem
Remove unit production from the following −
S → XY, X → a, Y → Z | b, Z → M, M → N, N → a
Solution −
There are 3 unit productions in the grammar −
Y → Z, Z → M, and M → N
At first, we will remove M → N.
As N → a, we add M → a, and M → N is removed.
The production set becomes
S → XY, X → a, Y → Z | b, Z → M, M → a, N → a
Now we will remove Z → M.
As M → a, we add Z→ a, and Z → M is removed.
The production set becomes
S → XY, X → a, Y → Z | b, Z → a, M → a, N → a
Now we will remove Y → Z.
As Z → a, we add Y→ a, and Y → Z is removed.
The production set becomes
S → XY, X → a, Y → a | b, Z → a, M → a, N → a
Now Z, M, and N are unreachable, hence we can remove those.
The final CFG is unit production free −
S → XY, X → a, Y → a | b
Removal of Null Productions
In a CFG, a non-terminal symbol ‘A’ is a nullable variable if there is a production A →
ε or there is a derivation that starts at A and finally ends up with
ε: A → .......… → ε
Removal Procedure
A CFG is in Chomsky Normal Form if the Productions are in the following forms −
A→a
A → BC
S→ε
where A, B, and C are non-terminals and a is terminal.
Algorithm to Convert into Chomsky Normal Form −
Step 1 − If the start symbol S occurs on some right side, create a new start
symbol S’ and a new production S’→ S.
Step 2 − Remove Null productions. (Using the Null production removal algorithm
discussed earlier)
Step 3 − Remove unit productions. (Using the Unit production removal algorithm
discussed earlier)
Step 4 − Replace each production A → B1…Bn where n > 2 with A → B1C where C →
B2 …Bn. Repeat this step for all productions having two or more symbols in the right
side.
Step 5 − If the right side of any production is in the form A → aB where a is a terminal
and A, B are non-terminal, then the production is replaced by A → XB and X → a.
Repeat this step for every production which is in the form A → aB.
Problem
Solution
Problem
Solution
Here, S does not appear on the right side of any production and there are no unit or
null productions in the production rule set. So, we can skip Step 1 to Step 3.
Step 4
Now after replacing
X in S → XY | Xo | p
with
mX | m
we obtain
S → mXY | mY | mXo | mo | p.
And after replacing
X in Y → Xn | o
with the right side of
X → mX | m
we obtain
Y → mXn | mn | o.
Two new productions O → o and P → p are added to the production set and then we
came to the final GNF as the following −
S → mXY | mY | mXC | mC | p
X → mX | m
Y → mXD | mD | o
O→o
P→p
Lemma
If L is a context-free language, there is a pumping length p such that any string w ∈
L of length ≥ p can be written as w = uvxyz, where vy ≠ ε, |vxy| ≤ p, and for all i ≥ 0,
uvixyiz ∈ L.
Problem
Find out whether the language L = {xnynzn | n ≥ 1} is context free or not.
Solution
an input tape,
a control unit, and
a stack with infinite size.
The stack head scans the top symbol of the stack.
A stack does two operations −
Push − a new symbol is added at the top.
Pop − the top symbol is read and removed.
A PDA may or may not read an input symbol, but it has to read the top of the stack in
every transition.
A PDA can be formally described as a 7-tuple (Q, ∑, S, δ, q 0, I, F) −
Q is the finite number of states
∑ is input alphabet
S is stack symbols
δ is the transition function: Q × (∑ ∪ {ε}) × S × Q × S*
q0 is the initial state (q0 ∈ Q)
I is the initial stack top symbol (I ∈ S)
F is a set of accepting states (F ∈ Q)
The following diagram shows a transition in a PDA from a state q 1 to state q2, labeled
as a,b → c −
This means at state q1, if we encounter an input string ‘a’ and top symbol of the stack
is ‘b’, then we pop ‘b’, push ‘c’ on top of the stack and move to state q2.
Turnstile Notation
The "turnstile" notation is used for connecting pairs of ID's that represent one or many
moves of a PDA. The process of transition is denoted by the turnstile symbol "⊢".
Consider a PDA (Q, ∑, S, δ, q 0, I, F). A transition can be mathematically represented by
the following turnstile notation −
(p, aw, Tβ) ⊢ (q, w, αb)
This implies that while taking a transition from state p to state q, the input symbol ‘a’ is
consumed, and the top of the stack ‘T’ is replaced by a new string ‘α’.
Note − If we want zero or more moves of a PDA, we have to use the symbol (⊢*) for it.
Solution
Example
Initially we put a special symbol ‘$’ into the empty stack. At state q2, the w is being
read. In state q3, each 0 or 1 is popped when it matches the input. If any other input is
given, the PDA will go to a dead state. When we reach that special symbol ‘$’, we go to
the accepting state q4.
Problem
Solution
Example
Design a top-down parser for the expression "x+y*z" for the grammar G with the
following production rules −
P: S → S+X | X, X → X*Y | Y, Y → (S) | id
Solution
If the PDA is (Q, ∑, S, δ, q0, I, F), then the top-down parsing is −
(x+y*z, I) ⊢(x +y*z, SI) ⊢ (x+y*z, S+XI) ⊢(x+y*z, X+XI)
⊢(x+y*z, Y+X I) ⊢(x+y*z, x+XI) ⊢(+y*z, +XI) ⊢ (y*z, XI)
⊢(y*z, X*YI) ⊢(y*z, y*YI) ⊢(*z,*YI) ⊢(z, YI) ⊢(z, zI) ⊢(ε, I)
Example
Design a top-down parser for the expression "x+y*z" for the grammar G with the
following production rules −
P: S → S+X | X, X → X*Y | Y, Y → (S) | id
Solution
If the PDA is (Q, ∑, S, δ, q0, I, F), then the bottom-up parsing is −
(x+y*z, I) ⊢ (+y*z, xI) ⊢ (+y*z, YI) ⊢ (+y*z, XI) ⊢ (+y*z, SI)
⊢(y*z, +SI) ⊢ (*z, y+SI) ⊢ (*z, Y+SI) ⊢ (*z, X+SI) ⊢ (z, *X+SI)
⊢ (ε, z*X+SI) ⊢ (ε, Y*X+SI) ⊢ (ε, X+SI) ⊢ (ε, SI)
Definition
A Turing Machine (TM) is a mathematical model which consists of an infinite length
tape divided into cells on which input is given. It consists of a head which reads the
input tape. A state register stores the state of the Turing machine. After reading an
input symbol, it is replaced with another symbol, its internal state is changed, and it
moves from one cell to the right or left. If the TM reaches the final state, the input string
is accepted, otherwise rejected.
A TM can be formally described as a 7-tuple (Q, X, ∑, δ, q 0, B, F) where −
Q is a finite set of states
X is the tape alphabet
∑ is the input alphabet
δ is a transition function; δ : Q × X → Q × X × {Left_shift, Right_shift}.
q0 is the initial state
B is the blank symbol
F is the set of final states
The following table shows a comparison of how a Turing machine differs from Finite
Automaton and Pushdown Automaton.
Tape alphabet symbol Present State ‘q0’ Present State ‘q1’ Present State ‘q2’
a 1Rq1 1Lq0 1Lqf
Here the transition 1Rq1 implies that the write symbol is 1, the tape moves right, and
the next state is q1. Similarly, the transition 1Lq2 implies that the write symbol is 1, the
tape moves left, and the next state is q2.
Example 1
α BRq2 BRq1
Example 2
Design a Turing Machine that reads a string representing a binary number and erases
all leading 0’s in the string. However, if the string comprises of only 0’s, it keeps one 0.
Solution
Let us assume that the input string is terminated by a blank symbol, B, at each end of
the string.
The Turing Machine, M, can be constructed by the following moves −
Let q0 be the initial state.
If M is in q0, on reading 0, it moves right, enters the state q1 and erases 0. On reading 1, it
enters the state q2 and moves right.
If M is in q1, on reading 0, it moves right and erases 0, i.e., it replaces 0’s by B’s. On
reaching the leftmost 1, it enters q2 and moves right. If it reaches B, i.e., the string
comprises of only 0’s, it moves left and enters the state q3.
If M is in q2, on reading either 0 or 1, it moves right. On reaching B, it moves left and enters
the state q4. This validates that the string comprises only of 0’s and 1’s.
If M is in q3, it replaces B by 0, moves left and reaches the final state qf.
If M is in q4, on reading either 0 or 1, it moves left. On reaching the beginning of the string,
i.e., when it reads B, it reaches the final state qf.
Hence,
M = {{q0, q1, q2, q3, q4, qf}, {0,1, B}, {1, B}, δ, q0, B, {qf}}
where δ is given by −
Tape alphabet Present State Present State Present State Present State Present State
symbol ‘q0’ ‘q1’ ‘q2’ ‘q3’ ‘q4’
It is a two-track tape −
Upper track − It represents the cells to the right of the initial head position.
Lower track − It represents the cells to the left of the initial head position in reverse order.
The infinite length input string is initially written on the tape in contiguous tape cells.
The machine starts from the initial state q0 and the head scans from the left end marker
‘End’. In each step, it reads the symbol on the tape under its head. It writes a new
symbol on that tape cell and then it moves the head either into left or right one tape
cell. A transition function determines the actions to be taken.
It has two special states called accept state and reject state. If at any point of time it
enters into the accepted state, the input is accepted and if it enters into the reject state,
the input is rejected by the TM. In some cases, it continues to run infinitely without
being accepted or rejected for some certain input symbols.
Note − Turing machines with semi-infinite tape are equivalent to standard Turing
machines.
Language Decidability
A language is called Decidable or Recursive if there is a Turing machine which
accepts and halts on every input string w. Every decidable language is Turing-
Acceptable.
A decision problem P is decidable if the language L of all yes instances to P is
decidable.
For a decidable language, for each input string, the TM halts either at the accept or the
reject state as depicted in the following diagram −
Example 1
Find out whether the following problem is decidable or not −
Is a number ‘m’ prime?
Solution
Example 2
Given a regular language L and string w, how can we check if w ∈ L?
Solution
Undecidable Languages
F or an undecidable language, there is no Turing Machine which accepts the language
and makes a decision for every input string w (TM can make decision for some input
string though). A decision problem Pis called “undecidable” if the language L of all yes
instances to P is not decidable. Undecidable languages are not recursive languages,
but sometimes, they may be recursively enumerable languages.
Example
The halting problem of Turing machine
The mortality problem
The mortal matrix problem
The Post correspondence problem, etc.
Rice Theorem
Rice theorem states that any non-trivial semantic property of a language which is
recognized by a Turing machine is undecidable. A property, P, is the language of all
Turing machines that satisfy that property.
Formal Definition
If P is a non-trivial property, and the language holding the property, L p , is recognized
by Turing machine M, then Lp = {<M> | L(M) ∈ P} is undecidable.
Proof
Suppose, a property P is non-trivial and φ ∈ P.
Since, P is non-trivial, at least one language satisfies P, i.e., L(M 0) ∈ P , ∋ Turing
Machine M0.
Let, w be an input in a particular instant and N is a Turing Machine which follows −
On input x
Run M on w
If M does not accept (or doesn't halt), then do not accept x (or do not halt)
If M accepts w then run M0 on x. If M0 accepts x, then accept x.
A function that maps an instance ATM = {<M,w>| M accepts input w} to a N such that
If M accepts w and N accepts the same language as M0, Then L(M) = L(M0) ∈ p
If M does not accept w and N accepts φ, Then L(N) = φ ∉ p
Since ATM is undecidable and it can be reduced to Lp, Lp is also undecidable.
Example 1
Find whether the lists
M = (abb, aa, aaa) and N = (bba, aaa, aa)
have a Post Correspondence Solution?
Solution
x1 x2 x3
M Abb aa aaa
N Bba aaa aa
Here,
x2x1x3 = ‘aaabbaaa’
and y2y1y3 = ‘aaabbaaa’
We can see that
x2x1x3 = y2y1y3
Hence, the solution is i = 2, j = 1, and k = 3.
Example 2
Find whether the lists M = (ab, bab, bbaaa) and N = (a, ba, bab) have a Post
Correspondence Solution?
Solution
x1 x2 x3
M ab bab bbaaa
N a ba bab
Related Terminologies
Alphabet
String
Length of a String
Kleene Star
Definition − The Kleene star, ∑*, is a unary operator on a set of symbols or strings, ∑, that
gives the infinite set of all possible strings of all possible lengths over ∑ including λ.
Representation − ∑* = ∑0 ∪ ∑1 ∪ ∑2 ∪……. where ∑p is the set of all possible strings of
length p.
Example − If ∑ = {a, b}, ∑* = {λ, a, b, aa, ab, ba, bb,………..}
Kleene Closure / Plus
Definition − The set ∑+ is the infinite set of all possible strings of all possible lengths over ∑
excluding λ.
Representation − ∑+ = ∑1 ∪ ∑2 ∪ ∑3 ∪…….
∑+ = ∑* − { λ }
Example − If ∑ = { a, b } , ∑+ = { a, b, aa, ab, ba, bb,………..}
Language
Example
Q = {a, b, c},
∑ = {0, 1},
q0 = {a},
F = {c}, and
Transition function δ as shown by the following table −
Present State Next State for Input 0 Next State for Input 1
a a b
b c a
c b c
Q = {a, b, c}
∑ = {0, 1}
q0 = {a}
F = {c}
The transition function δ as shown below −
Present State Next State for Input 0 Next State for Input 1
a a, b b
b c a, c
c b, c c
DFA NDFA
The transition from a state is to a single particular next The transition from a state can be to multiple next
state for each input symbol. Hence it is states for each input symbol. Hence it is called non-
called deterministic. deterministic.
Empty string transitions are not seen in DFA. NDFA permits empty string transitions.
A string is accepted by a DFA, if it transits to a final state. A string is accepted by a NDFA, if at least one of all
possible transitions ends in a final state.
An automaton that computes a Boolean function is called an acceptor. All the states of
an acceptor is either accepting or rejecting the inputs given to it.
Classifier
A classifier has more than two final states and it gives a single output when it
terminates.
Transducer
An automaton that produces outputs based on current input and/or previous state is
called a transducer. Transducers can be of two types −
Mealy Machine − The output depends both on the current state and the current input.
Moore Machine − The output depends only on the current state.
Strings accepted by the above DFA: {0, 00, 11, 010, 101, ...........}
Strings not accepted by the above DFA: {1, 011, 111, ........}
NDFA to DFA Conversion
Problem Statement
Let X = (Qx, ∑, δx, q0, Fx) be an NDFA which accepts the language L(X). We have to
design an equivalent DFA Y = (Qy, ∑, δy, q0, Fy) such that L(Y) = L(X). The following
procedure converts the NDFA to its equivalent DFA −
Algorithm
Input − An NDFA
Output − An equivalent DFA
Step 1 − Create state table from the given NDFA.
Step 2 − Create a blank state table under possible input alphabets for the equivalent
DFA.
Step 3 − Mark the start state of the DFA by q0 (Same as the NDFA).
Step 4 − Find out the combination of States {Q 0, Q1,... , Qn} for each possible input
alphabet.
Step 5 − Each time we generate a new DFA state under the input alphabet columns,
we have to apply step 4 again, otherwise go to step 6.
Step 6 − The states which contain any of the final states of the NDFA are the final
states of the equivalent DFA.
Example
Let us consider the NDFA shown in the figure below.
q δ(q,0) δ(q,1)
a {a,b,c,d,e} {d,e}
b {c} {e}
c ∅ {b}
d {e} ∅
e ∅ ∅
Using the above algorithm, we find its equivalent DFA. The state table of the DFA is
shown in below.
q δ(q,0) δ(q,1)
[d,e] [e] ∅
[e] ∅ ∅
[c, e] ∅ [b]
[c] ∅ [b]
The state diagram of the DFA is as follows −
DFA Minimization
DFA Minimization using Myphill-Nerode Theorem
Algorithm
Input − DFA
Output − Minimized DFA
Step 1 − Draw a table for all pairs of states (Q i, Qj) not necessarily connected directly
[All are unmarked initially]
Step 2 − Consider every state pair (Q i, Qj) in the DFA where Qi ∈ F and Qj ∉ F or vice
versa and mark them. [Here F is the set of final states]
Step 3 − Repeat this step until we cannot mark anymore states −
If there is an unmarked pair (Qi, Qj), mark it if the pair {δ (Q i, A), δ (Qi, A)} is marked for
some input alphabet.
Step 4 − Combine all the unmarked pair (Qi, Qj) and make them a single state in the
reduced DFA.
Example
a b c d e f
a b c d e f
a
c ✔ ✔
d ✔ ✔
e ✔ ✔
f ✔ ✔ ✔
Step 3 − We will try to mark the state pairs, with green colored check mark, transitively.
If we input 1 to state ‘a’ and ‘f’, it will go to state ‘c’ and ‘f’ respectively. (c, f) is already
marked, hence we will mark pair (a, f). Now, we input 1 to state ‘b’ and ‘f’; it will go to
state ‘d’ and ‘f’ respectively. (d, f) is already marked, hence we will mark pair (b, f).
a b c d e f
c ✔ ✔
d ✔ ✔
e ✔ ✔
f ✔ ✔ ✔ ✔ ✔
After step 3, we have got state combinations {a, b} {c, d} {c, e} {d, e} that are
unmarked.
We can recombine {c, d} {c, e} {d, e} into {c, d, e}
Hence we got two combined states as − {a, b} and {c, d, e}
So the final minimized DFA will contain three states {f}, {a, b} and {c, d, e}
Algorithm 3
Step 1 − All the states Q are divided in two partitions − final states and non-final
states and are denoted by P0. All the states in a partition are 0th equivalent. Take a
counter k and initialize it with 0.
Step 2 − Increment k by 1. For each partition in P k, divide the states in Pk into two
partitions if they are k-distinguishable. Two states within this partition X and Y are k-
distinguishable if there is an input S such that δ(X, S) and δ(Y, S) are (k-1)-
distinguishable.
Step 3 − If Pk ≠ Pk-1, repeat Step 2, otherwise go to Step 4.
Step 4 − Combine kth equivalent sets and make them the new states of the reduced
DFA.
Example
a b c
b a d
c e f
d e f
e e f
f f f
Mealy Machine
Moore machine
Mealy Machine
A Mealy Machine is an FSM whose output depends on the present state as well as the
present input.
It can be described by a 6 tuple (Q, ∑, O, δ, X, q 0) where −
Q is a finite set of states.
∑ is a finite set of symbols called the input alphabet.
O is a finite set of symbols called the output alphabet.
δ is the input transition function where δ: Q × ∑ → Q
X is the output transition function where X: Q × ∑ → O
q0 is the initial state from where any input is processed (q0 ∈ Q).
The state table of a Mealy Machine is shown below −
Next state
→a b x1 c x1
b b x2 d x3
c d x3 c x1
d d x3 d x2
Moore Machine
Moore machine is an FSM whose outputs depend on only the present state.
A Moore machine can be described by a 6 tuple (Q, ∑, O, δ, X, q 0) where −
Q is a finite set of states.
∑ is a finite set of symbols called the input alphabet.
O is a finite set of symbols called the output alphabet.
δ is the input transition function where δ: Q × ∑ → Q
X is the output transition function where X: Q → O
q0 is the initial state from where any input is processed (q0 ∈ Q).
The state table of a Moore Machine is shown below −
Next State
Present state Output
Input = 0 Input = 1
→a b c x2
b b d x1
c c d x2
d d d x3
The following table highlights the points that differentiate a Mealy Machine from a
Moore Machine.
Generally, it has fewer states than Moore Machine. Generally, it has more states than Mealy Machine.
The value of the output function is a function of the The value of the output function is a function of the
transitions and the changes, when the input logic on current state and the changes at the clock edges,
the present state is done. whenever state changes occur.
Mealy machines react faster to inputs. They generally In Moore machines, more logic is required to decode the
react in the same clock cycle. outputs resulting in more circuit delays. They generally
react one clock cycle later.
Example
Next State
Present State Output
a=0 a=1
→a d b 1
b a d 0
c c c 0
d b a 1
Next State
→a d b
b a d
c c c
d b a
Step 3 −
Next State
b a 1 d 1
c c 0 c 0
d b 0 a 1
Example
Next State
→a d 0 b 1
b a 1 d 0
c c 1 c 0
d b 0 a 1
Here, states ‘a’ and ‘d’ give only 1 and 0 outputs respectively, so we retain states ‘a’
and ‘d’. But states ‘b’ and ‘c’ produce different outputs (1 and 0). So, we
divide b into b0, b1 and c into c0, c1.
Next State
Present State Output
a=0 a=1
→a d b1 1
b0 a d 0
b1 a d 1
c0 c1 C0 0
c1 c1 C0 1
d b0 a 0
Introduction to Grammars
n the literary sense of the term, grammars denote syntactical rules for conversation in
natural languages. Linguistics have attempted to define grammars since the inception
of natural languages like English, Sanskrit, Mandarin, etc.
The theory of formal languages finds its applicability extensively in the fields of
Computer Science. Noam Chomsky gave a mathematical model of grammar in 1956
which is effective for writing computer languages.
Grammar
A grammar G can be formally written as a 4-tuple (N, T, S, P) where −
N or V is a set of variables or non-terminal symbols.
N
Example
Grammar G1 −
({S, A, B}, {a, b}, S, {S → AB, A → a, B → b})
Here,
S, A, and B are Non-terminal symbols;
a and b are Terminal symbols
S is the Start symbol, S ∈ N
Productions, P : S → AB, A → a, B → b
Example
Grammar G2 −
(({S, A}, {a, b}, S,{S → aAb, aA → aaAb, A → ε } )
Here,
S and A are Non-terminal symbols.
a and b are Terminal symbols.
ε is an empty string.
S is the Start symbol, S ∈ N
Production P : S → aAb, aA → aaAb, A → ε
Example
Example
If there is a grammar
G: N = {S, A, B} T = {a, b} P = {S → AB, A → a, B → b}
Here S produces AB, and we can replace A by a, and B by b. Here, the only accepted
string is ab, i.e.,
L(G) = {ab}
Example
Example
Problem − Suppose, L (G) = {am bn | m ≥ 0 and n > 0}. We have to find out the
grammar G which produces L(G).
Solution
Since L(G) = {am bn | m ≥ 0 and n > 0}
the set of strings accepted can be rewritten as −
L(G) = {b, ab,bb, aab, abb, …….}
Here, the start symbol has to take at least one ‘b’ preceded by any number of ‘a’
including null.
To accept the string set {b, ab, bb, aab, abb, …….}, we have taken the productions −
S → aS , S → B, B → b and B → bB
S → B → b (Accepted)
S → B → bB → bb (Accepted)
S → aS → aB → ab (Accepted)
S → aS → aaS → aaB → aab(Accepted)
S → aS → aB → abB → abb (Accepted)
Thus, we can prove every single string in L(G) is accepted by the language generated
by the production set.
Hence the grammar −
G: ({S, A, B}, {a, b}, S, { S → aS | B , B → b | bB })
Example
Problem − Suppose, L (G) = {am bn | m > 0 and n ≥ 0}. We have to find out the
grammar G which produces L(G).
Solution −
Since L(G) = {am bn | m > 0 and n ≥ 0}, the set of strings accepted can be rewritten as −
L(G) = {a, aa, ab, aaa, aab ,abb, …….}
Here, the start symbol has to take at least one ‘a’ followed by any number of ‘b’
including null.
To accept the string set {a, aa, ab, aaa, aab, abb, …….}, we have taken the
productions −
S → aA, A → aA , A → B, B → bB ,B → λ
S → aA → aB → aλ → a (Accepted)
S → aA → aaA → aaB → aaλ → aa (Accepted)
S → aA → aB → abB → abλ → ab (Accepted)
S → aA → aaA → aaaA → aaaB → aaaλ → aaa (Accepted)
S → aA → aaA → aaB → aabB → aabλ → aab (Accepted)
S → aA → aB → abB → abbB → abbλ → abb (Accepted)
Thus, we can prove every single string in L(G) is accepted by the language generated
by the production set.
Hence the grammar −
G: ({S, A, B}, {a, b}, S, {S → aA, A → aA | B, B → λ | bB })
Take a look at the following illustration. It shows the scope of each type of grammar −
Type - 3 Grammar
Type-3 grammars generate regular languages. Type-3 grammars must have a single
non-terminal on the left-hand side and a right-hand side consisting of a single terminal
or single terminal followed by a single non-terminal.
The productions must be in the form X → a or X → aY
where X, Y ∈ N (Non terminal)
and a ∈ T (Terminal)
The rule S → ε is allowed if S does not appear on the right side of any rule.
Example
X → ε
X → a | aY
Y → b
Type - 2 Grammar
Type-2 grammars generate context-free languages.
The productions must be in the form A → γ
where A ∈ N (Non terminal)
and γ ∈ (T ∪ N)* (String of terminals and non-terminals).
These languages generated by these grammars are be recognized by a non-
deterministic pushdown automaton.
Example
S → X a
X → a
X → aX
X → abc
X → ε
Type - 1 Grammar
Type-1 grammars generate context-sensitive languages. The productions must be in
the form
αAβ→αγβ
where A ∈ N (Non-terminal)
and α, β, γ ∈ (T ∪ N)* (Strings of terminals and non-terminals)
The strings α and β may be empty, but γ must be non-empty.
The rule S → ε is allowed if S does not appear on the right side of any rule. The
languages generated by these grammars are recognized by a linear bounded
automaton.
Example
AB → AbBc
A → bcA
B → b
Type - 0 Grammar
Type-0 grammars generate recursively enumerable languages. The productions have
no restrictions. They are any phase structure grammar including all formal grammars.
They generate the languages that are recognized by a Turing machine.
The productions can be in the form of α → β where α is a string of terminals and
nonterminals with at least one non-terminal and α cannot be null. β is a string of
terminals and non-terminals.
Example
S → ACaB
Bc → acB
CB → DB
aD → Db
Regular Expressions
A Regular Expression can be recursively defined as follows −
ε is a Regular Expression indicates the language containing an empty string. (L (ε) = {ε})
φ is a Regular Expression denoting an empty language. (L (φ) = { })
x is a Regular Expression where L = {x}
If X is a Regular Expression denoting the language L(X) and Y is a Regular Expression
denoting the language L(Y), then
o X + Y is a Regular Expression corresponding to the language L(X) ∪
L(Y) where L(X+Y) = L(X) ∪ L(Y).
o X . Y is a Regular Expression corresponding to the language L(X) .
L(Y) where L(X.Y) = L(X) . L(Y)
o R* is a Regular Expression corresponding to the language L(R*)where L(R*) =
(L(R))*
If we apply any of the rules several times from 1 to 5, they are Regular Expressions.
Some RE Examples
Regular Expressions Regular Set
(a+b)* Set of strings of a’s and b’s of any length including the null string. So L = { ε, a, b, aa ,
ab , bb , ba, aaa…….}
(a+b)*abb Set of strings of a’s and b’s ending with the string abb. So L = {abb, aabb, babb, aaabb,
ababb, …………..}
(11)* Set consisting of even number of 1’s including empty string, So L= {ε, 11, 1111,
111111, ……….}
(aa)*(bb)*b Set of strings consisting of even number of a’s followed by odd number of b’s , so L =
{b, aab, aabbb, aabbbbb, aaaab, aaaabbb, …………..}
(aa + ab + ba + bb)* String of a’s and b’s of even length can be obtained by concatenating any combination
of the strings aa, ab, ba and bb including null, so L = {aa, ab, ba, bb, aaab, aaba,
…………..}
Regular Sets
Any set that represents the value of the Regular Expression is called a Regular Set.
∅* = ε
ε* = ε
RR* = R*R
R*R* = R*
(R*)* = R*
RR* = R*R
(PQ)*P =P(QP)*
(a+b)* = (a*b*)* = (a*+b*)* = (a+b*)* = a*(ba*)*
R + ∅ = ∅ + R = R (The identity for union)
R ε = ε R = R (The identity for concatenation)
∅ L = L ∅ = ∅ (The annihilator for concatenation)
R + R = R (Idempotent law)
L (M + N) = LM + LN (Left distributive law)
(M + N) L = ML + NL (Right distributive law)
ε + RR* = ε + R*R = R*
Arden's Theorem
In order to find out a regular expression of a Finite Automaton, we use Arden’s
Theorem along with the properties of regular expressions.
Statement −
Let P and Q be two regular expressions.
If P does not contain null string, then R = Q + RP has a unique solution that is R = QP*
Proof −
R = Q + (Q + RP)P [After putting the value R = Q + RP]
= Q + QP + RPP
When we put the value of R recursively again and again, we get the following equation
−
R = Q + QP + QP2 + QP3…..
R = Q (ε + P + P2 + P3 + …. )
R = QP* [As P* represents (ε + P + P2 + P3 + ….) ]
Hence, proved.
Method
Step 1 − Create equations as the following form for all the states of the DFA having n
states with initial state q1.
q1 = q1R11 + q2R21 + … + qnRn1 + ε
q2 = q1R12 + q2R22 + … + qnRn2
…………………………
…………………………
…………………………
…………………………
qn = q1R1n + q2R2n + … + qnRnn
Rij represents the set of labels of edges from qi to qj, if no such edge exists, then Rij = ∅
Step 2 − Solve these equations to get the equation for the final state in terms of Rij
Problem
Construct a regular expression corresponding to the automata given below −
Solution −
Here the initial state and final state is q1.
The equations for the three states q1, q2, and q3 are as follows −
q1 = q1a + q3a + ε (ε move is because q1 is the initial state0
q2 = q1b + q2b + q3b
q3 = q2a
Now, we will solve these three equations −
q2 = q1b + q2b + q3b
= q1b + q2b + (q2a)b (Substituting value of q3)
= q1b + q2(b + ab)
= q1b (b + ab)* (Applying Arden’s Theorem)
q1 = q1a + q3a + ε
= q1a + q2aa + ε (Substituting value of q3)
= q1a + q1b(b + ab*)aa + ε (Substituting value of q2)
= q1(a + b(b + ab)*aa) + ε
= ε (a+ b(b + ab)*aa)*
= (a + b(b + ab)*aa)*
Hence, the regular expression is (a + b(b + ab)*aa)*.
Problem
Construct a regular expression corresponding to the automata given below −
Solution −
Here the initial state is q1 and the final state is q2
Now we write down the equations −
q1 = q10 + ε
q2 = q11 + q20
q3 = q21 + q30 + q31
Now, we will solve these three equations −
q1 = ε0* [As, εR = R]
So, q1 = 0*
q2 = 0*1 + q20
So, q2 = 0*1(0)* [By Arden’s theorem]
Hence, the regular expression is 0*10*.
Construction of an FA from an RE
We can use Thompson's Construction to find out a Finite Automaton from a Regular
Expression. We will reduce the regular expression into smallest regular expressions
and converting these to NFA and finally to DFA.
Some basic RA expressions are the following −
Case 1 − For a regular expression ‘a’, we can construct the following FA −
Case 2 − For a regular expression ‘ab’, we can construct the following FA −
Case 3 − For a regular expression (a+b), we can construct the following FA −
Case 4 − For a regular expression (a+b)*, we can construct the following FA −
Method
Step 1 Construct an NFA with Null moves from the given regular expression.
Step 2 Remove Null transition from the NFA and convert it into its equivalent DFA.
Problem
Convert the following RA into its equivalent DFA − 1 (0 + 1)* 0
Solution
We will concatenate three expressions "1", "(0 + 1)*" and "0"
Now we will remove the ε transitions. After we remove the ε transitions from the NDFA,
we get the following −
Step 3 −
Here q1 is an initial state, so we make qf also an initial state.
So the FA becomes −
Step 4 −
Here qf is a final state, so we make q1 also a final state.
So the FA becomes −
|y| > 0
|xy| ≤ c
For all k ≥ 0, the string xykz is also in L.
DFA Complement
If (Q, ∑, δ, q0, F) be a DFA that accepts a language L, then the complement of the DFA
can be obtained by swapping its accepting states with its non-accepting states and vice
versa.
We will take an example and elaborate this below −
Representation Technique
The derivation or the yield of a parse tree is the final string obtained by concatenating
the labels of the leaves of the tree from left to right, ignoring the Nulls. However, if all
the leaves are Null, derivation is Null.
Example
Let a CFG {N,T,P,S} be
N = {S}, T = {a, b}, Starting symbol = S, P = S → SS | aSb | ε
One derivation from the above CFG is “abaabb”
S → SS → aSbS → abS → abaSb → abaaSbb → abaabb
A partial derivation tree is a sub-tree of a derivation tree/parse tree such that either all
of its children are in the sub-tree or none of them are in the sub-tree.
Example
If in any CFG the productions are −
S → AB, A → aaA | ε, B → Bb| ε
the partial derivation tree can be the following −
If a partial derivation tree contains the root S, it is called a sentential form. The above
sub-tree is also in sentential form.
Solution
Let’s find out the derivation tree for the string "a+a*a". It has two leftmost derivations.
Derivation 1 − X → X+X → a +X → a+ X*X → a+a*X → a+a*a
Parse tree 1 −
Union
Concatenation
Kleene Star operation
Union
Let L1 and L2 be two context free languages. Then L1 ∪ L2 is also context free.
Example
Let L1 = { anbn , n > 0}. Corresponding grammar G1 will have P: S1 → aAb|ab
Let L2 = { cmdm , m ≥ 0}. Corresponding grammar G2 will have P: S2 → cBb| ε
Union of L1 and L2, L = L1 ∪ L2 = { anbn } ∪ { cmdm }
The corresponding grammar G will have the additional production S → S1 | S2
Concatenation
If L1 and L2 are context free languages, then L1L2 is also context free.
Example
Kleene Star
If L is a context free language, then L* is also context free.
Example
CFG Simplification
In a CFG, it may happen that all the production rules and symbols are not needed for
the derivation of strings. Besides, there may be some null productions and unit
productions. Elimination of these productions and symbols is called simplification of
CFGs. Simplification essentially comprises of the following steps −
Reduction of CFG
Removal of Unit Productions
Removal of Null Productions
Reduction of CFG
CFGs are reduced in two phases −
Phase 1 − Derivation of an equivalent grammar, G’, from the CFG, G, such that each
variable derives some terminal string.
Derivation Procedure −
Step 1 − Include all symbols, W1, that derive some terminal and initialize i=1.
Step 2 − Include all symbols, Wi+1, that derive Wi.
Step 3 − Increment i and repeat Step 2, until Wi+1 = Wi.
Step 4 − Include all production rules that have Wi in it.
Phase 2 − Derivation of an equivalent grammar, G”, from the CFG, G’, such that each
symbol appears in a sentential form.
Derivation Procedure −
Step 1 − Include the start symbol in Y1 and initialize i = 1.
Step 2 − Include all symbols, Yi+1, that can be derived from Yi and include all production
rules that have been applied.
Step 3 − Increment i and repeat Step 2, until Yi+1 = Yi.
Problem
Solution
Phase 1 −
T = { a, c, e }
W1 = { A, C, E } from rules A → a, C → c and E → aA
W2 = { A, C, E } U { S } from rule S → AC
W3 = { A, C, E, S } U ∅
Since W2 = W3, we can derive G’ as −
G’ = { { A, C, E, S }, { a, c, e }, P, {S}}
where P: S → AC, A → a, C → c , E → aA | e
Phase 2 −
Y1 = { S }
Y2 = { S, A, C } from rule S → AC
Y3 = { S, A, C, a, c } from rules A → a and C → c
Y4 = { S, A, C, a, c }
Since Y3 = Y4, we can derive G” as −
G” = { { A, C, S }, { a, c }, P, {S}}
where P: S → AC, A → a, C → c
Step 1 − To remove A → B, add production A → x to the grammar rule whenever B →
x occurs in the grammar. [x ∈ Terminal, x can be Null]
Step 2 − Delete A → B from the grammar.
Step 3 − Repeat from step 1 until all unit productions are removed.
Problem
Remove unit production from the following −
S → XY, X → a, Y → Z | b, Z → M, M → N, N → a
Solution −
There are 3 unit productions in the grammar −
Y → Z, Z → M, and M → N
At first, we will remove M → N.
As N → a, we add M → a, and M → N is removed.
The production set becomes
S → XY, X → a, Y → Z | b, Z → M, M → a, N → a
Now we will remove Z → M.
As M → a, we add Z→ a, and Z → M is removed.
The production set becomes
S → XY, X → a, Y → Z | b, Z → a, M → a, N → a
Now we will remove Y → Z.
As Z → a, we add Y→ a, and Y → Z is removed.
The production set becomes
S → XY, X → a, Y → a | b, Z → a, M → a, N → a
Now Z, M, and N are unreachable, hence we can remove those.
The final CFG is unit production free −
S → XY, X → a, Y → a | b
A→a
A → BC
S→ε
where A, B, and C are non-terminals and a is terminal.
Problem
Solution
Problem
Solution
Here, S does not appear on the right side of any production and there are no unit or
null productions in the production rule set. So, we can skip Step 1 to Step 3.
Step 4
Now after replacing
X in S → XY | Xo | p
with
mX | m
we obtain
S → mXY | mY | mXo | mo | p.
And after replacing
X in Y → Xn | o
with the right side of
X → mX | m
we obtain
Y → mXn | mn | o.
Two new productions O → o and P → p are added to the production set and then we
came to the final GNF as the following −
S → mXY | mY | mXC | mC | p
X → mX | m
Y → mXD | mD | o
O→o
P→p
Problem
Find out whether the language L = {xnynzn | n ≥ 1} is context free or not.
Solution
an input tape,
a control unit, and
a stack with infinite size.
The stack head scans the top symbol of the stack.
A stack does two operations −
Push − a new symbol is added at the top.
Pop − the top symbol is read and removed.
A PDA may or may not read an input symbol, but it has to read the top of the stack in
every transition.
Turnstile Notation
The "turnstile" notation is used for connecting pairs of ID's that represent one or many
moves of a PDA. The process of transition is denoted by the turnstile symbol "⊢".
Consider a PDA (Q, ∑, S, δ, q 0, I, F). A transition can be mathematically represented by
the following turnstile notation −
(p, aw, Tβ) ⊢ (q, w, αb)
This implies that while taking a transition from state p to state q, the input symbol ‘a’ is
consumed, and the top of the stack ‘T’ is replaced by a new string ‘α’.
Note − If we want zero or more moves of a PDA, we have to use the symbol (⊢*) for it.
Example
Solution
Example
Initially we put a special symbol ‘$’ into the empty stack. At state q2, the w is being
read. In state q3, each 0 or 1 is popped when it matches the input. If any other input is
given, the PDA will go to a dead state. When we reach that special symbol ‘$’, we go to
the accepting state q4.
Problem
Construct a PDA from the following CFG.
G = ({S, X}, {a, b}, P, S)
where the productions are −
S → XS | ε , A → aXb | Ab | ab
Solution
Example
Design a top-down parser for the expression "x+y*z" for the grammar G with the
following production rules −
P: S → S+X | X, X → X*Y | Y, Y → (S) | id
Solution
If the PDA is (Q, ∑, S, δ, q0, I, F), then the top-down parsing is −
(x+y*z, I) ⊢(x +y*z, SI) ⊢ (x+y*z, S+XI) ⊢(x+y*z, X+XI)
⊢(x+y*z, Y+X I) ⊢(x+y*z, x+XI) ⊢(+y*z, +XI) ⊢ (y*z, XI)
⊢(y*z, X*YI) ⊢(y*z, y*YI) ⊢(*z,*YI) ⊢(z, YI) ⊢(z, zI) ⊢(ε, I)
Example
Design a top-down parser for the expression "x+y*z" for the grammar G with the
following production rules −
P: S → S+X | X, X → X*Y | Y, Y → (S) | id
Solution
If the PDA is (Q, ∑, S, δ, q0, I, F), then the bottom-up parsing is −
(x+y*z, I) ⊢ (+y*z, xI) ⊢ (+y*z, YI) ⊢ (+y*z, XI) ⊢ (+y*z, SI)
⊢(y*z, +SI) ⊢ (*z, y+SI) ⊢ (*z, Y+SI) ⊢ (*z, X+SI) ⊢ (z, *X+SI)
⊢ (ε, z*X+SI) ⊢ (ε, Y*X+SI) ⊢ (ε, X+SI) ⊢ (ε, SI)
Definition
A Turing Machine (TM) is a mathematical model which consists of an infinite length
tape divided into cells on which input is given. It consists of a head which reads the
input tape. A state register stores the state of the Turing machine. After reading an
input symbol, it is replaced with another symbol, its internal state is changed, and it
moves from one cell to the right or left. If the TM reaches the final state, the input string
is accepted, otherwise rejected.
A TM can be formally described as a 7-tuple (Q, X, ∑, δ, q 0, B, F) where −
Q is a finite set of states
X is the tape alphabet
∑ is the input alphabet
δ is a transition function; δ : Q × X → Q × X × {Left_shift, Right_shift}.
q0 is the initial state
B is the blank symbol
F is the set of final states
The following table shows a comparison of how a Turing machine differs from Finite
Automaton and Pushdown Automaton.
Tape alphabet symbol Present State ‘q0’ Present State ‘q1’ Present State ‘q2’
Here the transition 1Rq1 implies that the write symbol is 1, the tape moves right, and
the next state is q1. Similarly, the transition 1Lq2 implies that the write symbol is 1, the
tape moves left, and the next state is q2.
Example 1
α BRq2 BRq1
Example 2
Design a Turing Machine that reads a string representing a binary number and erases
all leading 0’s in the string. However, if the string comprises of only 0’s, it keeps one 0.
Solution
Let us assume that the input string is terminated by a blank symbol, B, at each end of
the string.
The Turing Machine, M, can be constructed by the following moves −
Let q0 be the initial state.
If M is in q0, on reading 0, it moves right, enters the state q1 and erases 0. On reading 1, it
enters the state q2 and moves right.
If M is in q1, on reading 0, it moves right and erases 0, i.e., it replaces 0’s by B’s. On
reaching the leftmost 1, it enters q2 and moves right. If it reaches B, i.e., the string
comprises of only 0’s, it moves left and enters the state q3.
If M is in q2, on reading either 0 or 1, it moves right. On reaching B, it moves left and enters
the state q4. This validates that the string comprises only of 0’s and 1’s.
If M is in q3, it replaces B by 0, moves left and reaches the final state qf.
If M is in q4, on reading either 0 or 1, it moves left. On reaching the beginning of the string,
i.e., when it reads B, it reaches the final state qf.
Hence,
M = {{q0, q1, q2, q3, q4, qf}, {0,1, B}, {1, B}, δ, q0, B, {qf}}
where δ is given by −
Tape alphabet Present State Present State Present State Present State Present State
symbol ‘q0’ ‘q1’ ‘q2’ ‘q3’ ‘q4’
It is a two-track tape −
Upper track − It represents the cells to the right of the initial head position.
Lower track − It represents the cells to the left of the initial head position in reverse order.
The infinite length input string is initially written on the tape in contiguous tape cells.
The machine starts from the initial state q0 and the head scans from the left end marker
‘End’. In each step, it reads the symbol on the tape under its head. It writes a new
symbol on that tape cell and then it moves the head either into left or right one tape
cell. A transition function determines the actions to be taken.
It has two special states called accept state and reject state. If at any point of time it
enters into the accepted state, the input is accepted and if it enters into the reject state,
the input is rejected by the TM. In some cases, it continues to run infinitely without
being accepted or rejected for some certain input symbols.
Note − Turing machines with semi-infinite tape are equivalent to standard Turing
machines.
Language Decidability
A language is called Decidable or Recursive if there is a Turing machine which
accepts and halts on every input string w. Every decidable language is Turing-
Acceptable.
Solution
Example 2
Given a regular language L and string w, how can we check if w ∈ L?
Solution
Undecidable Languages
For an undecidable language, there is no Turing Machine which accepts the language
and makes a decision for every input string w (TM can make decision for some input
string though). A decision problem Pis called “undecidable” if the language L of all yes
instances to P is not decidable. Undecidable languages are not recursive languages,
but sometimes, they may be recursively enumerable languages.
Example
The halting problem of Turing machine
The mortality problem
The mortal matrix problem
The Post correspondence problem, etc.
Rice Theorem
Rice theorem states that any non-trivial semantic property of a language which is
recognized by a Turing machine is undecidable. A property, P, is the language of all
Turing machines that satisfy that property.
Formal Definition
If P is a non-trivial property, and the language holding the property, L p , is recognized
by Turing machine M, then Lp = {<M> | L(M) ∈ P} is undecidable.
Proof
Suppose, a property P is non-trivial and φ ∈ P.
Since, P is non-trivial, at least one language satisfies P, i.e., L(M 0) ∈ P , ∋ Turing
Machine M0.
Let, w be an input in a particular instant and N is a Turing Machine which follows −
On input x
Run M on w
If M does not accept (or doesn't halt), then do not accept x (or do not halt)
If M accepts w then run M0 on x. If M0 accepts x, then accept x.
A function that maps an instance ATM = {<M,w>| M accepts input w} to a N such that
If M accepts w and N accepts the same language as M0, Then L(M) = L(M0) ∈ p
If M does not accept w and N accepts φ, Then L(N) = φ ∉ p
Since ATM is undecidable and it can be reduced to Lp, Lp is also undecidable.
Example 1
Find whether the lists
M = (abb, aa, aaa) and N = (bba, aaa, aa)
have a Post Correspondence Solution?
Solution
x1 x2 x3
M Abb aa aaa
N Bba aaa aa
Here,
x2x1x3 = ‘aaabbaaa’
and y2y1y3 = ‘aaabbaaa’
We can see that
x2x1x3 = y2y1y3
Hence, the solution is i = 2, j = 1, and k = 3.
Example 2
Find whether the lists M = (ab, bab, bbaaa) and N = (a, ba, bab) have a Post
Correspondence Solution?
Solution
x1 x2 x3
M ab bab bbaaa
N a ba bab