Formal Language and Automata Theory

Prof. Parth Singh,Prof.Ashish kumar,Prof. Amit Solanki

Assistant Professor
Computer Science & Engineering
CHAPTER-3
Grammar
Context Free Language

• CFLs are produced by context free language

structure. The arrangement of all setting free
dialects is indistinguishable from the set of
dialects acknowledged by push down
automata, and the set of regular language is a
subset of CFG
Context Free Language

• Grammar:
-Sentence structure is a lot of rules which check whether a string have a
place with a specific language or not. This defined Sentence structure is
called Grammar

-Syntax of language is defined by this notation

- Designing of Parser is done by the help of CFG

Derivation Tree/ Parse Tree
• Generation of Derivation Tree
- A deduction tree or parse tree is an arranged established tree that
graphically speaks to the semantic data a string got from a CFG.
• Representation Technique
- Root vertex: Must be labeled by the start symbol.
- Vertex: Labeled by a non-terminal symbol.
- Leaves: Labeled by a terminal symbol or ε.

Image source : Google

Approaches to draw a derivation tree

• Top-down Approach
- Starts with the starting symbol S.
- Goes down to tree leaves using productions.
• Bottom-up Approach
- Starts from tree leaves.
- Proceeds upward to the root which is the starting symbol S.
.
Derivation of a Tree

• The induction or the yield of a parse tree is the last string acquired by
connecting the marks of the leaves of the tree from left to right, overlooking
the Nulls. In any case, if all the leaves are Null, deduction is Null
Derivation of a Tree : Example
• Let a CFG {N,T,P,S} be N = {S}, T = {a, b}, Starting symbol = S, P = S → SS | aSb
| ε One derivation from the above CFG is “abaabb”
Answer:-
S → SS
S → aSbS (replace S →aSb)
S →abS (replace S →ε)
S → abaSb (replace S →aSb)
S → abaaSbb (replace S →aSb)

S→ abaabb (replace S →ε)

Sentential Form and Partial Derivation Tree
• A fractional deduction tree is a sub-tree of an induction tree/parse tree with
the end goal that either the entirety of its kids are in the sub-tree or none of
them are in the sub-tree

•Example
In the event that in any CFG the creations/production are:
S → AB, A → aaA | ε, B →Bb| ε
•The fractional deduction tree can be the accompanying:
-In the event that an incomplete determination tree contains the root S, it is
known as a sentential form.
-The above example is likewise in sentential form.
Types of Derivation Tree
• Leftmost and Rightmost Derivation of a String
- Leftmost derivation: A furthest left induction is acquired by applying
creation to the furthest left factor in each progression

- Rightmost derivation: A furthest right deduction is acquired by applying

creation to the furthest right factor in each progression.
Left and Right Recursive Grammars
• In a CFG "G", if there is a creation in the structure X → Xa where X is a non-
terminal and 'a' will be a series of terminals, it is known as a left recursive
creation. The syntax having a left recursive creation is known as a left recursive
production

• Also, if in a CFG "G", if there is a creation is in the structure X → aX where X is

a non-terminal and 'a' will be a series of terminals, it is known as a right
recursive production. The language structure having a privilege recursive
creation is known as a right recursive production.
Ambiguity in Context-Free Grammars

• On the off chance that a setting free sentence structure G has more than
one induction tree for some string w ∈ L(G), it is called an ambiguous
grammar. There exist various right-most or left-most determinations for some
string produced from that language structure.
CFG Simplification
• In a CFG, it might happen that all the production rules and symbol are not
required for the determination of strings. Furthermore, there might be some
invalid production and unit creations. End of these production and symbols is
called disentanglement of CFGs.

Steps to simply
•Reduction of CFG
•Removal of Unit Productions
•Removal of Null Productions
Reduction of CFG
Derivation of an equivalent grammar, G’, from the CFG, G, such that each
variable derives some terminal string.

Derivation Procedure Part 1:

Step 1: Include all symbols, W1, that derive some terminal and initialize i=1.
Step 2: Include all symbols, Wi+1, that derive Wi.
Step 3: Increment i and repeat Step 2, until Wi+1 = Wi.
Step 4: Include all production rules that have Wi in it.
Reduction of CFG
Derivation of an equivalent grammar, G”, from the CFG, G’ such that each
symbol appears in a sentential form.

Derivation Procedure part 2:

Step 1: Include the start symbol in Y1 and initialize i = 1.
Step 2: Include all symbols, Yi+1, that can be derived from Yi and include all
production rules that have been applied.
Step 3: Increment i and repeat Step 2, until Yi+1 = Yi.
Removal of Unit Productions
Any production rule in the form A → B where A, B ∈ Non-terminal is called
unit production

Removal Procedure:-
Step 1: To remove A→B, add production A → x to the grammar rule whenever
B → x occurs in the grammar. [x ∈ Terminal, x can be Null]
Step 2: Delete A→B from the grammar.
Step 3: Repeat from step 1 until all unit productions are removed.
Removal of Null Productions
In a CFG, a non-terminal symbol ‘A’ is a nullable variable if there is a production
A → ϵ or there is a derivation that starts at A and finally ends up with
ϵ: A → .......… → ϵ
Removal Procedure:
Step1 Find out nullable non-terminal variables which derive ϵ.
Step2 For each production A → a, construct all productions A → x where x is
obtained from ‘a’ by removing one or multiple non-terminals from Step 1.
Step3 Combine the original productions with the result of step 2 and remove ϵ-
productions.
Chomsky Normal Form
In formal language theory, a CFG ”G ” is said to be in Chomsky normal form if all
of its production rules are of the form:

Q → RE,
or Q → f,
or S → ε,

Here Q,R,E are Non terminals and S is starting symbol. f and ε(epsilon) are
terminals where ε represent NULL
Greibach Normal Form
In formal language theory, a CFG is in Greibach normal form (GNF) if the right-
hand sides of all production rules start with a terminal symbol, optionally
followed by some variables.

Either that in the form:

P → aQWERTY
P→a
S→ε

Here P,Q,W,E,R,T,Y are non terminals and S is start symbol. ε and a are
terminals
Non Deterministic Push down Automata
•A nondeterministic pushdown automaton (npda) is basically an nfa with a stack
added to it.
•A nondeterministic pushdown automaton or npda is a 7-tuple
M = (Q,∑, Γ, δ,q0, z, F)
•Q is a finite set of states,
∑ is a the input alphabet,
Γ is the stack alphabet,
δ is a transition function,
q0 ∈ Q is the initial state,
z ∈ Γ is the stack start symbol, and
F ⊆ Q is a set of final states.
Block diagram of PDA
Example of NPDA

[1]https://www.google.co.in/imgres?imgurl=https%3A%2F%2Ffanyv88.com%3A443%2Fhttps%2Fmedia.geeksforgeeks.org%2Fwp-
content%2Fuploads%2F33333.jpg&imgrefurl=https%3A%2F%2Ffanyv88.com%3A443%2Fhttps%2Fwww.geeksforgeeks.org%2Fnpda-for-l-0i1j2k-ij-or-jk-i-j-k-
1%2F&tbnid=7m8Az1XEry4VSM&vet=12ahUKEwjS3IKF-tbpAhVQRSsKHa-1DH
PDA equivalence to CFG
•Algorithm to find PDA corresponding to a given CFG:

Input − A CFG, G = (V, T, P, S)

Output − Equivalent PDA, P = (Q, ∑, S, δ, q0, I, F)

Step 1 − Convert the productions of the CFG into GNF.

Step 2 − The PDA will have only one state {q}.
Step 3 − The start symbol of CFG will be the start symbol in the PDA.
Step 4 − All non-terminals of the CFG will be the stack symbols of the PDA and
all the terminals of the CFG will be the input symbols of the PDA.
Step 5 − For each production in the form A → aX where a is terminal and A, X are
combination of terminal and non-terminals, make a transition δ (q, a, A)
What is Parse Tree??
SxCKJbdjkSBAKJDbsjXBZCVZXvcVZXBNc
● Parse trees are a representation of derivations that is much more compact.
Several derivations may correspond to the same parse tree. For example, in
the balanced parenthesis grammar, the following parse tree:HC ADSA

Image source : Google

What is Parse Tree??

Corresponds to the derivation S ⇒ SS ⇒ S(S) ⇒ (S)(S) ⇒ (S)() ⇒ ()()

as well as this one:
S ⇒ SS ⇒ (S)S ⇒ (S)(S) ⇒ ()(S) ⇒ ()()
Parse Tree

• In a parse tree, the points are called nodes. Each node has a label on it.
• The topmost node is called the root. The bottom nodes are called leaves.
• In a parse tree for a grammar G, the leaves must be labelled with terminal
symbols from G, or with ǫ. The root is often labeled with the start symbol of G,
but not always.
• If a node N labeled with A has children N1, N2, . . . , Nk from left to right,
labeled with A1, A2, . . . , Ak, respectively, then A → A1A2, . . . Ak must be a
production in the grammar G.

Image source : Google

 Parse Tree
• The yield of a parse tree is the concatenation of the labels of the leaves, from
left to right. The yield of the tree above is ()().

● Leftmost and Rightmost Derivations

• In a leftmost derivation, at each step the leftmost nonterminal is
replaced. In a rightmost derivation, at each step the rightmost
nonterminal is replaced.
• Such replacements are indicated by ͢R and ͢L respectively.
• Their transitive closures are ͢R* and ͢L* respectively.
Parse Tree
•In the balanced parenthesis grammar, this is a leftmost derivation:
S ⇒ SS ⇒ (S)S ⇒ ()S ⇒ ()(S) ⇒ ()().
This is a rightmost derivation:
S ⇒ SS ⇒ S(S) ⇒ S() ⇒ (S)() ⇒ ()()
Ambiguity In CFG
● A context-free grammar G = (V, Σ, R, S) is ambiguous if there is
some string w ∈ Σ ∗ such that there are two distinct parse trees T 1
and T 2 having S at the root and having yield w.
● Equivalently, w has two or more leftmost derivations, or two or
more rightmost derivations.
● Note that languages are not ambiguous; grammars are. Also, it has
to be the same string w with two different (leftmost or rightmost)
derivations for a grammar to be ambiguous.
Ambiguity In CFG
● Here is an example of an ambiguous grammar:
● E→E+E
● E→E∗E
● E → (E)
● E→a
● E→b
● E→c
Ambiguity In CFG
● In this grammar, the string a + b ∗ c can be parsed in two different
ways corresponding to doing the addition before or after the
multiplication. This is very bad for a compiler, because the compiler
uses the parse tree to generate code, meaning that this string could
have two very different semantics. Here are two parse trees for the
string a + b ∗ c in this grammar:
Here are two parse trees for the string a + b ∗ c in this grammar:
Ambiguity In CFG
Ambiguity In CFG
● There is a notion of inherent ambiguity for context-free languages;
a context-free language L is inherently ambiguous if every context-
free grammar G for L is ambiguous. As an example, the
language{anbncmdm : n ≥ 1, m ≥ 1} ∪ {anbmcmdn : n ≥ 1, m ≥ 1} is
inherently ambiguous. In any context-free grammar for L, some
strings of the form anbncndn will have two distinct parse trees.
Unfortunately, the problem of whether a context-free grammar is
ambiguous, is undecidable. However, there are some patterns in a
context-free grammar that frequently indicate ambiguity:
Ambiguity In CFG
● S → SS
● S→a
● S→A
● A → AA
● A→a
● S → AA
● A→S
● A→a
● S → SbS
Ambiguity in CFG
● S→a
● S → AbA
● A→S
● A→a
Ambiguity In CFG
The following is not ambiguous:
S → aS
S → bS
S→ε
In general, a production A → AA causes ambiguity if it is reachable
from the start symbol and some terminal string is derivable from A.
Pumping Lemma of CFG

● For every CFL L there is a constant k ≥ 0 such that for any word z in
L of length at least k, there are strings u, v,w, x, y such that
● z = uvwxy,
● Vx ≠ ϵ
● |vwx| ≤ k,
● and for each i ≥ 0, the string uviwxiy belongs to L.
Parse trees for CFG’s
Parse trees for CFG’s

● Cutting and pasting in parse trees

Proof idea
● A long string must have a deep parse tree, which in turn means a
path with a repeated non-terminal.
Proof
Proof II
Proof II
 CHAPTER-3
Push Down Automata
Deterministic Push Down Automata
Deterministic Push Down Automata

● A pushdown automaton is deterministic if for every pair of

compatible transitions, these transitions are identical.

● Let L be a language defined over the alphabet Σ, the language L is

deterministic context-free if and only if it is accepted by a
deterministic pushdown automaton.
Deterministic Push Down Automata
Let L be a language defined over the alphabet Σ, the language L is
deterministic context-free if and only if it is accepted by a
deterministic pushdown automaton.
 CHAPTER-3
Closure Properties of CFL
Closure properties of CFLs
• CFLs are closed under
- Union
- Concatenation
- Kleene Star
- Reversal
- Homomorphism
• Regular Languages closed under
- Intersection
- Difference
- Complementation
• CFLs are not closed under intersection, difference, or complementation, why?
Union
• If L1 and If L2 are two context free languages, their union L1 ∪ L2 will also be
context free.
• Let L1 and L2 be CFLs with grammars G1 and G2, respectively.
• Assume G1 and G2 have no variables in common.
• Let S1 and S2 be the start symbols of G1 and G2.
• Form a grammar for (L1 ∪ L2) by combining all the symbols and productions
of G1 and G2.
• Then, add a new start symbol S. Add productions S -> S1 | S2.
Union
• Example:
L1 = { anbncm | m >= 0 and n >= 0 }
L2 = { anbmcm | n >= 0 and m >= 0 }
L3 = L1 ∪ L2 = { anbncm ∪ anbmcm | n >= 0, m >= 0 } is also context free.

• Here Language L1 generates all strings that contains occurrence of a’s equals
to occurrence of b’s and L2 generates all strings that contains occurrence of b’s
equals to the occurrence of c’s.

• Union require either of two condition require to be true. It can be accepted by

pushdown automata.

• So, Language L3 is also CFL.

Concatenation
• L1 and L2 are CFLs, then their concatenation L1.L2 will also be context free
• Let L1 and L2 be CFL’s with grammars G1 and G2, respectively.
• Assume G1 and G2 have no variables in common.
• Let S1 and S2 be the start symbols of G1 and G2.
• Form a new grammar for L1.L2 by starting with all symbols and productions of
G1 and G2.
• Add a new start symbol S and production S -> S1S2.
• Every derivation from S results in a string in L1 followed by one in L2.
Concatenation
• Example :
L1 = { anbn | n >= 0 }
L2 = { cmdm | m >= 0 }
L3 = L1.L2 = {anbn cmdm | m >= 0 and n >= 0} is also context free.
• Here L1 grammar generate all strings which contains equal occurrence of a’s
and b’s and L2 generate equal occurrence of c’s and d’s. Language L3 can be
accepted by Pushdown automata.
• Hence, Concatenation is closed under CFLs.
Kleene Star
• L1 is context free, then its Kleene closure L1* will also be context free.
• Let L have grammar G, with start symbol S1.
• Grammar for L* by introducing to G a new start symbol S and the productions
S -> S1S | ε.
• A rightmost derivation from S generates a sequence of zero or more S1’s,
each of which generates some string in L.
• Example :
L1 = { anbn | n >= 0 }
L1* = { anbn | n >= 0 }* is also context free.
Reversal
• L is a CFL with grammar G
• Grammar for LR by reversing the right side of every production.
• Example:
Let G have S -> 0S1 | 01.
S -> 1S0 | 10.
• LR is also Context free grammar.
Homomorphism
• Let L be a CFL with grammar G.
• Let h be a homomorphism on the terminal symbols of G.
• Construct a grammar for h(L) by replacing each terminal symbol a by h(a).
• Example:
G has productions S -> 0S1 | 01.
h is defined by h(0) = ab, h(1) = ε.
h(L(G)) has the grammar with productions S -> abS | ab.
Non-Closure Properties of CFL’s - Intersection
•Let L1 and L2 are two context free languages
L1 = { anbncm | n >= 0 and m >= 0 }
L2 = (ambncn | n >= 0 and m >= 0 }
L3 = L1 ∩ L2 = { anbncn | n >= 0 }
• L1 generate all strings of number of a’s should be equal to number of b’s and
L2 generate all strings of number of b’s should be equal to number of c’s.
• Intersection require both conditions need to be true
• It cannot be accepted by pushdown automata, so it is not context free.
Complementation
• Let L1 and L2 are two context free languages
L1 = { anbncm | n >= 0 and m >= 0 }
L2 = (ambncn | n >= 0 and m >= 0 }

• CFLs are not closed under intersection.

• Language is not context free and it can not accepted by Pushdown automata.
• Thus, CFLs are not closed under Complementation.
Difference
• Let L1 and L2 are two context free languages
• Proof: L1 ∩ L2 = L1 – (L1 – L2).
• CFLs are not closed under Intersection.
• If CFLs were closed under difference, they require to be closed under
intersection, but they are not.
• Thus, CFLs are not closed under difference
 CHAPTER-3
Context Sensitive Language
Context Sensitive Grammar and Languages
• Hierarchy of languages.
- Type-0 : Recursively enumerable language
- Type-1 : Context Sensitive language
- Type-2 : Context Free language
- Type-3 : Regular language
• We discuss Context Sensitive Language and corresponding state machine,
(Linear Bounded Automaton(LBA)), equivalence and properties of Context
Sensitive Languages.
Definition : Context Sensitive Grammar(CSG)
• Context Sensitive Grammar (CSG) is a quadruple G=(N,∑,P,S) where,
- N is set of non-terminal symbols
- ∑ is set of terminal symbols
- S is set of start symbol
- P is set of production in form of αAβ → αɤβ where ɤ ≠ ϵ
• Derivation non-terminal A will be changed to ɤ only
• CSG is Non-contracting grammar as ɤ ≠ ϵ then α → β => |α| ≤ |β|
Context Sensitive Language(CSL)
• The language that can be defined by • Derivation of CSL
context-sensitive grammar is called S → aAbc
Context sensitive language(CSL). → abAc
→ abBbcc
• Example: → aBbbcc
Consider the following CSG. → aaAbbcc
→ aabAbcc
S → abc/aAbc → aabbAcc
Ab → bA → aabbBbccc
→ aabBbbccc
Ac → Bbcc → aaBbbbccc
bB → Bb → aaabbbccc
Context sensitive language
aB → aa/aaA L= {anbncn | n≥1}.
Closure properties of CSLs
• Union
• Intersection
• Complement
• Concatenation
• Kleene closure
• Reversal
Equivalence of CSL
• The following grammar(G) is context-sensitive.
S -> aTb | ab
aT -> aaTb | ac

• The language generated by grammar G

L(G) = {ab} ∪ {ancbn | n>0}
• Language is also a context-free.

• For example, Context free grammar(G1) for this.

S -> aTb | ab
T -> aTb | c
• Any context-free language is context sensitive.
• Not all context sensitive but it need not be context free.
Equivalence of CSL
Theorem: Every context-sensitive language L is recursive.

• Let L be CSL, G be CSG

• Derivation of string w, S ->S1 ,S1->S2, S2->S3........ = w
• No of steps are Bound on possible derivations. We know that |xi|< |xi +1| (G is
non-contracting).
• Check whether w is in L(G) as follows
-Construct a transition graph whose vertices are the strings of length|w|.
-To find Paths correspond to derivation in G. Add edge from x to y if x -> y
-w ∈ L(G) if there is a path from S to w.
Equivalence of CSL
Theorem: There exists some recursive language that is not context sensitive.
• Language L is recursive
-Create possible CSG Gi = (Ni , {0; 1; 2; 3; 4; 5; 6; 7; 8; 9}, Si , Pi ) which generates
numbers.
-We can define language L, which contains the numbers of the grammars which
does not generate the number of its position in the list.
L = {i | i ∉ L(Gi )}.
-We can create a list of all context-sensitive generative grammars which
generates numbers, and we can decide whether or not a context-sensitive
grammar generates its position in the list.
-Hence language L is recursive.
Equivalence of CSL
Theorem: There exists a recursive language that is not context sensitive.
• Language L is not context sensitive
-Assume that L is a CSL
-So there is a CSG Gc , s.t L(Gc ) = L for some c.
-If c ∈ L(Gc), by the denition of L, we have c ∉ L, but L = L(Gc ). So a contradiction.
-If c ∉ L(Gc ), then c ∈ L is also a contradiction since L = L(Gc ).
-Hence language L is not context sensitive.
Chomsky Hierarchy
 Chomsky Hierarchy
Type Language Automaton Production rules

Type 0 Recursively Turing Machine α→β

Unrestricted enumerable

Type 1 Context-sensitive Linear-bounded αAβ → αɤβ

Context- automaton
sensitive
Type 2 Context-free Pushdown automaton A→ɤ
Context-free

Type 3 Regular Finite state automaton A → a and A → aB

Regular
Linear Bounded Automata
• Linear Bounded Automata is a single tape non-deterministic Turing Machine
with two special tape symbols call them left marker < and right marker >.
• The transitions should satisfy these conditions:
- It should not replace the marker symbols by any other symbol.
- It should not write on cells beyond the marker symbols.
• Configuration will be: < q0a1a2a3a4a5.......an > = <q0w>
Linear Bounded Automata
• Linear Bounded Automata is a non-deterministic Turing Machine,
M=(Q, Σ ,T, 𝛿, B, ,F, q0, <, >, t, r) Where,

- Q is set of all states

- Σ is set of all terminals
- T is set of all tape alphabets
- 𝛿 is set of transitions
- B is blank symbol
- q0 is the initial state
- < is left marker and > is right marker
- t is accept state
- r is reject state
Linear Bounded Automata
• Turing Machine for Context sensitive language L= {anbncn | n≥1}.
www.paruluniversity.ac.in