THEORY OF COMPUTATION

DR O. J Falana

1
CSC 206: THEORY OF COMPUTATION (2 Units)

Formal grammars and automata; meaning of alphabet, string, concatenation, language

and level of language; regular expression, regular grammar and context-free languages,
deterministic and non-deterministic parsing of context free languages; recursive
language, finite state automata, turing machine, pumping lemma, chomsk normal form
and CYK algorithm. Pre-requisites MTS 101
 Outlines
1. Introduction to Computation Theory
Definition and importance of Computation Theory
Relationship between Computation Theory and Computer Science
Overview of key topics in the course

2. Formal Grammars and Automata

Definition of Grammar and Automata
Types of Formal Grammars (Regular, Context-Free, Context-Sensitive, Unrestricted)
Automata Theory: Finite Automata, Pushdown Automata, Turing Machines

3. Alphabet, Strings, Concatenation, Language, and Levels of Language

Definition of Alphabet (Σ), Strings and their properties
String operations (Concatenation, Substring, Reverse)
Definition of Language and Examples, Chomsky Hierarchy of Languages
 Outlines contd..
4. Regular Expressions and Regular Grammar
Definition of Regular Expressions, Operations on Regular Expressions (Union, Concatenation,
Kleene Star), Regular Grammars and their Types (Left-Linear, Right-Linear), Equivalence
between Regular Expressions and Finite Automata

5. Context-Free Languages and Parsing

Definition of Context-Free Languages, Context-Free Grammars (CFGs) and their
Representation, Derivations and Parse Trees, Deterministic vs. Non-Deterministic Parsing
Applications of CFGs in Programming Languages

6. Recursive Languages and Decidability

Definition of Recursive and Recursively Enumerable Languages, Decidability and
Undecidability in Computation, Examples of Recursive and Non-Recursive Languages
7. Finite State Automata (FSA), Definition and Types (Deterministic and Non-Deterministic)
Transition Functions and State Diagrams, Conversion between NFA and DFA
Outlines contd..
8. Turing Machines and Computability
Introduction to Turing Machines, Components and Functioning of a Turing Machine
Variants of Turing Machines (Multi-Tape, Non-Deterministic), Church-Turing Thesis
and Computability

9. Pumping Lemma
Pumping Lemma for Regular Languages, Pumping Lemma for Context-Free Languages
Applications in Proving Language Non-Regularity

10. Chomsky Normal Form (CNF) and CYK Algorithm

Definition of Chomsky Normal Form, Conversion of CFG to CNF
CYK Algorithm for Parsing Strings in CNF, Application in Compiler Design
LECTURE 1
Introduction to Computation Theory
Definition of Computation Theory
The Theory of Computation is a branch of computer science that focuses on the study of
problems that can be solved using algorithms and the efficiency of these solutions.

Automata theory, also known as the Theory of Computation, is a field within computer
science and mathematics that focuses on studying abstract machines to understand the
capabilities and limitations of computation by analyzing mathematical models of how
machines can perform calculations.

It answers fundamental questions like:

• What problems can be solved using a computer?

• How efficient can these solutions be?
• Are there problems that are unsolvable by any computer?
1.2 Branches of Computation Theory
The theory of computation is divided into three main areas:

A. Automata Theory
• Studies abstract computing devices (automata) and their capabilities.
• Explains how computers process languages and recognize patterns.

B. Computability Theory
• Focuses on what problems are solvable using algorithms.
• Deals with decidable vs. undecidable problems.

C. Complexity Theory
• Examines how efficiently problems can be solved.
• Categorizes problems into complexity classes like P, NP, NP-complete.
1.3 Why Study Computation Theory?
1. Understanding Computation Limits – Helps in identifying which problems are
solvable and which are impossible to compute.
2. Algorithm Design – Provides techniques for designing better and more efficient
algorithms.
3. Artificial Intelligence & Machine Learning – Automata concepts are used in pattern
recognition, natural language processing, and AI.
4. Compiler Design – Used to design programming language parsers and interpreters.
5. Cybersecurity & Cryptography – Complexity theory is used to design secure
cryptographic algorithms.
6. Regular Expressions (RE) in Systems -Regular expressions are powerful tools for
pattern matching and text processing used extensively in many systems.
7. Finite Automata in Modeling Systems -Modeling Protocols and Circuits: Finite
automata (FA) are used to model protocols, like those in network communication,
and to design electronic circuits that operate based on a set of predefined rules or
states.
1.4 Fundamental Concepts in Computation Theory
The study of computation is based on fundamental mathematical models
and formal systems, including:

• Alphabet (Σ) – A set of symbols used in computation. Example: Σ = {0,

1}.
• String – A sequence of symbols from an alphabet. Example: "1011".
• Language (L) – A set of valid strings formed using an alphabet.
• Formal Grammar – A set of rules that define how strings are generated in
a language.
• Automata (Computing Models) – Machines that process languages (e.g.,
Finite Automata, Pushdown Automata, Turing Machines).
1.5 Basic Terminologies of Theory of Computation
Now, let’s understand the basic terminologies, which are important and
frequently used in the Theory of Computation.
1. Symbol
A symbol (often also called a character) is the smallest building block, which
can be any alphabet, letter, or picture.

2. Alphabets (Σ)
A finite, non-empty set of symbols used to construct strings and languages.
For example, Σ = {a, b}.
1.5 Basic Terminologies of Theory of Computation
3. String
A string is a finite sequence of symbols from some alphabet. A string is
generally denoted as w and the length of a string is denoted as |w|. Empty
string is the string with zero occurrence of symbols, represented as ε.
2. GRAMMARS: REGULAR EXPRESSIONS AND
LANGUAGES, CONTEXT-FREE LANGUAGES
Having learnt about Strings and Alphabets in the previous lecture, another important
concept in formal language and automata theory, is Grammar.
Learning Outcomes
At the end of this lecture, you should be able to:
• define Grammar
• state the types of grammars available in the field of Computer Science
• describe the class of automata that can recognize strings generated by each grammar
• identify strings that are generated by a particular grammar
• describe the Chomsky hierarchy
• explain the relevance of grammar and formal languages to computer programming.

1
3
GRAMMAR
Grammar is a set of rules for forming strings in a formal language.
The rules describe how to form strings from the language's alphabet that are valid
according to the language's syntax. A grammar does not describe the meaning of the
strings or what can be done with them in whatever context - only their form.
Formal language theory, which is the discipline that studies formal grammars and
languages, is a branch of Applied Mathematics. Its applications are found in theoretical
computer science, theoretical linguistics, formal semantics, mathematical logic, and other
areas.
Grammar is a set of rules for rewriting strings, along with a "start symbol" from which
rewriting must start.
Therefore, a grammar is usually thought of as a language generator. However, it can also
sometimes be used as the basis for a "recognizer", a function in computing that
determines whether a given string belongs to the language or is grammatically incorrect.
To describe such recognisers, formal language theory uses separate formalisms, known as
automata theory.
1
4
Lecture 2: Grammars and Automata
A grammar mainly consists of a set of rules for transforming strings.
To generate a string in the language, one begins with a string consisting of only a single
start symbol.The productionrulesare then applied in any order, until a string that contains
neither the start symbol nor designated nonterminal symbols is produced.
The language formed by the grammar consists of all distinct strings that can be generated
in this manner.
Any particular sequence of production rules on the start symbol yields a distinct string in
the language.
If there are multiple ways of generating the same single string, the grammar is said to be
ambiguous.
For example, assume the alphabet consists of a and b, the start symbol is S, and we have
the following production rules:
S -► aSb
S -► ba

1
6
Definitions:
1. The Syntax of Grammars
The syntax of a Grammar refers to the structure of the grammar.
In the classic formalisation of generative grammar first proposed by Noam Chomsky in
the 1950s, a grammar G consists of the following components:
• A finite set N of nonterminal symbols, none of which appear in strings formed from G.
• A finite set T of terminal symbols that is disjoint from N.
• A finite set P of production rules
• A distinguished symbol S is the start symbol.
Thus, Grammar G can be formally defined as the tuple (N,T,P,S).
The language of G, denoted as L(G), is defined as all those sentences that can be derived
in a finite number of steps from the start symbol S.

1
7
2. The Semantics of Grammars
Semantics is the linguistic and philosophical study of meaning, in language, programming
languages and formal logics.
It is concerned with the relationship between signifiers, like words, phrases, signs and
symbols, and their denotations.

For example, assume the alphabet consists of a and b, the start symbol is S, and we have
the following production rules:
S => aSb
S => ba
Then we start with S, and can choose a rule to apply to it. If we choose rule 1, we obtain
the string aSb. If we choose rule 1 again, we replace S with aSb and obtain the string
aaSbb. If we now choose rule 2, we replace S with ba and obtain the string aababb, and are
done.
We can write this series of choices more briefly, using symbols:
S => aSb => aaSbb => aababb. 6
Example 1: Consider the Grammar G where S = {a, b, c} is the start symbol, and P
consists of the following production rules:
1. S => aBSc
2. S => abc
3. Ba => aB
4. Bb => bb
Construct the grammar of language L(G) = {anbncn n > 1}
Solution: the language is the set of strings that consist of 1 or more a's, followed by the
same number of b's, followed by the same number of c's.
Some examples of the derivation of strings in L(G) are:
S => aBSc => aBabcc => aaBbcc => aabbcc
OR
S => aBSc => aBaBScc => aBaBabccc => aaBBabccc => aaaBBbccc => aaaBbbccc =>
aaabbbccc
1
9
Language theory is a branch of Mathematics concerned with describing languages as a set
of operations over an alphabet. It is closely linked with automata theory, as automata are
used to generate and recognize formal languages.
There are several classes of formal languages, each allowing more complex language
specification than the one before it, i.e. Chomsky hierarchy, and each corresponding to a
class of automata which recognizes it.
Because automata are used as models for computation, formal languages are the preferred
mode of specification for any problem that must be computed.

2
0
 TYPES OF GRAMMARS

1. REGULAR EXPRESSIONS: are means to describe regular languages.

Before formally defining the notion of a regular expression, we give some examples.
Consider the expression:
(0 𝖴 1)01*.
The language described by this expression is the set of all binary strings:
1. that start with either 0 or 1 (this is indicated by (0 𝖴 1)),
2. for which the second symbol is 0 (this is indicated by 0), and
3. that end with zero or more 1s (this is indicated by 1*). 9
That is, the language described by this expression is:
{00, 001, 0011, 00111, . . . , 10, 101, 1011, 10111, . . .}

Here are some more examples (in all cases, the alphabet is {0, 1}):
• The language {w : w contains exactly two 0s} can be described by the expression
1*01*01*
• The language {w : w contains at least two 0s}can be described by the expression
(0 𝖴 1)*0(0 𝖴 1)*0(0 𝖴 1)*.
• The language {w : 1011 is a substring of w} can be described by the expression
(0 𝖴 1)*1011(0 𝖴 1)*.

22
• The language {w : the length of w is odd} can be described by the expression:
(0 𝖴 1) ((0 𝖴 1)(0 𝖴 1))* .

• The language {1011, 0} can be described by the expression:

1011 𝖴 0.
• The language {w : the first and last symbols of w are equal} can be described by the
expression: 0(0 𝖴 1)*0 𝖴 1(0 𝖴 1)*1 𝖴 0 𝖴 1.

After these examples, we give a formal definition of regular expressions:

23
Definition:
Let Σ be a non-empty alphabet.
1. є is a regular expression.
2. ϕ is a regular expression.
3. For each a∈Σ, a is a regular expression.
4. If R1 and R2 are regular expressions, then R1 𝖴 R2 is a regular expression.
5. If R1 and R2 are regular expressions, then R1 R2 is a regular expression.
6. If R is a regular expression, then R* is a regular expression.

You can regard 1, 2, and 3 as being the “building blocks” of regular expressions. Items 4, 5
and 6 give rules that can be used to combine regular expressions into new (and larger)
regular expressions.

24
To give an example, we claim that:
(0 𝖴 1)*101(0 𝖴 1)*
is a regular expression (where the alphabet Σ is equal to {0, 1}). In order to prove this, we
have to show that this expression can be built using the “rules” given in Definition above.
Here we go:
• By point 3, 0 is a regular expression.
• By point 3, 1 is a regular expression.
• Since 0 and 1 are regular expressions, by point 4, 0𝖴1 is a regular expression.
• Since 0𝖴1 is a regular expression, by point 6, (0𝖴1)* is a regular expression.
• Since 1 and 0 are regular expressions, by point 5, 10 is a regular expression.
• Since 10 and 1 are regular expressions, by point 5, 101 is a regular expression.
•Since (0 𝖴 1)* and 101 are regular expressions, by point 5, (0 𝖴 1)*101 is a
regular expression.
• Since (0 𝖴 1)*101 and (0 𝖴 1)* are regular expressions, by point 5, (0 𝖴1)*101(0 𝖴 1)* is a
regular expression. 13
Next we define the language that is described by a regular expression.

Let Σ be a non-empty alphabet.

1. The regular expression є describes the language {є}
2. The regular expression ϕ describes the language ϕ
3. For each a∈Σ, the regular expression a describes the language {a}.
4.Let R1 and R2 be regular expressions and let L1 and L2 be the languages described by
them, respectively. The regular expression R1 𝖴 R2 describes the language L1 𝖴 L2.
5.Let R1 and R2 be regular expressions and let L1 and L2 be the languages described by
them, respectively. The regular expression R1 R2 describes the language L1 L2.
6.Let R be a regular expression and let L be the language described by it. The regular
expression R* describes the language L*.

26
For example:
• The regular expression (0𝖴є)(1𝖴є) describes the language {01, 0, 1, є}.
• The regular expression 0𝖴є describes the language {0, є}, whereas the regular
expression 1* describes the language {є, 1, 11, 111, . . .}.
Therefore, the regular expression (0𝖴є)1* describes the language {0, 01, 011, 0111, . . . , є,
1, 11, 111, . . .}.

Observe that this language is also described by the regular expression 01* 𝖴 1*
• The regular expression 1*∅ describes the empty language, i.e., the language ∅.
• The regular expression ∅* describes the language {є}.

27
2. CONTEXT-FREE GRAMMARS
A context-free grammar is a set of recursive rules used to generate patterns of strings.
Context-free grammars are used for defining the syntax of programming languages and
their compilation.
Context-free grammars (CFGs) are used to describe context-free languages. A context-free
grammar can describe all regular languages and more, but they cannot
describe all possible languages.
Context-free grammars are studied in fields of theoretical computer science, compiler
design (in particular parsing), and linguistics. CFG’s are used to describe programming
languages and parser programs in compilers.

28
Definition:
A Context-Free Grammar (CFG) is a 4-tuple (V, T, S, P) where:
( i ) V is a finite set called the variables
(ii)T is a finite set, disjoint from V, called the terminals
(iii) P is a finite set of rules, with each rule being a variable and a string of variables and
terminals
(iv) S ∈ V is the start variable.

Examples of Context free langauages:

(a) The grammar G = ({S}, {a, b}, S, P) with productions
S => aSa,
S => bSb,
S => γ is context free.
29
S => aSa
=> aaSaa
=> aabSbaa
=> aabbaa

(b) The grammar G, with production rules given by

S => abB,
A => aaBb,
B => bbAa,
A => γ is context free.

30
31
Relationship of CFG with other computation models
Context-free languages are described by context-free grammars, which can be generated
by pushdown automata, just as regular languages can be generated by finite state
machines. Regular languages and finite state machines can describe some context-free
languages, but not all. Turing machines can generate all regular languages, all context-
free languages, and more.
Any language that can be generated using regular expressions can be generated by a
context-free grammar. All regular languages are context-free languages, but not all
context-free languages are regular.

There are grammars called context-sensitive grammars which are more powerful
(meaning they can generate more complex languages that might require more memory)
than both regular languages and context-free languages.

32
3. CONTEXT SENSITIVE GRAMMARS AND LANGUAGES
A context-sensitive grammar is a formal grammar in which the left-hand sides and right-
hand sides of any production rules may be surrounded by a context of terminal and non-
terminal symbols.
Context-sensitive grammar are more general than context-free grammars, in the sense that
there are languages that can be described by CSG but not by context-free grammar
A context-sensitive Language is a language generated by a context sensitive grammar.
Definition:
A context-sensitive grammar is one whose productions are all of the form
xAy => xvy
where A ∈v and x, v, y ∈ (V 𝖴T )*
“Context-sensitive” implies the fact that the actual string modification is given by A=> v,
while the x and y provide the context in which the rule may be applied.

33
For example: S => abc│aAbc
Ab => bA
Ac => Bbcc
bB => Bb
aB => aa │aaA

RELATIONSHIPS BETWEEN GRAMMARS

THEOREM 1: Every context-free language is context-sensitive.
THEOREM 2: There exists a context-sensitive language that is not context-free.
THEOREM 3: Every context-sensitive language is recursive.

34
Other Forms of Generative Grammars
Many extensions and variations on Chomsky's original hierarchy of formal grammars have
been developed, both by linguists and by computer scientists, usually either in order to
increase their expressive power or in order to make them easier to analyse or parse.
Some forms of grammars developed include:
• Tree-adjoining grammars increase the expressiveness of conventional generative
grammars by allowing rewrite rules to operate on parse trees instead of just strings.
• Affix grammars and attribute grammars allow rewrite rules to be augmented with
semantic attributes and operations, useful both for increasing grammar expressiveness
and for constructing practical language translation tools.
• Analytic Grammars

35
THE CHOMSKY HIERARCY
The Chomsky hierarchy is an hierarchy of the classes of formal grammars.
The Chomsky Hierarchy, as originally defined by Noam Chomsky in 1956, comprises
four types of languages and their associated grammars and the type of machines
that recognizes it.

The Chomsky Hierarchy is shown in Table 1:

36
Table 1 : Chomsky Hierarchy

37
• The Unrestricted grammars are classified as Type 0.
• Type 1 grammars generate context-sensitive languages.
• Type 2 grammars generate context-free languages and
• Type 3 grammars generate regular languages.

Each type of grammars is recognised by a particular type of automata.

For example:
Type-2 grammars are recognised by pushdown automata
Type-3 grammars are recognised by finite state automata.
The Unrestricted grammars can express any language that can be accepted by a
Turing machine.

38
39
PARSING
A grammar can be used in two ways:
(a) Using the grammar to generate strings of the language.
(b) Using the grammar to recognize the strings.

“Parsing” a string is finding a derivation (or a derivation tree) for that string.
Parsing a string is like recognizing a string. The only realistic way to recognize a string
of a context-free grammar is to parse it.

40
41
42
43
CONCLUSION
In this lecture, you have been introduced to the concept of formal grammars. Grammars
are very important in the field of automata theory since they are the building blocks of
languages.

SELF EXERCISE
1. What you understand by Grammars?
2. Give examples of Context-Free Grammar
3. Distinguish among the following grammar types:
a. Regular Grammars
b. Context-Free Grammars
c. Analytical Grammars.
4. Discuss the Chomsky hierarchy.
What is the relationship amongst the various types of grammars described in the Chomsky
32
hierarchy?
 COURSE OUTLINE

FINITE AUTOMATA
• NFA
• Regular Expressions
• Regular Languages
• Two-way finite automata
• Finite automata with output

4
5
Definition: A Nondeterministic Finite Automata (NFA) is also
defined by a 5- tuple

4
6
NFA differs from DFA in that, the range of δ in NFA is in the powerset 2Q . A string is accepted by
an NFA if there is some sequence of possible moves that will put the machine in the final state at the
end of the string.

Example 1:Obtain an NFA for a language consisting of all strings over {0,1} containing a 1in
the third position from the end.

Solution:

q1 , q2 , q3 are initial states, q4 is the final state

Note that this is an NFA as δ(q2 , 0) = q3 and δ(q2, 1) = q3.

4
7
Example 2: Determine an NFA accepting the language

Solution:

4
8
We shall come back to NFA later
REGULAR EXPRESSION
Regular Languages.
The regular languages are those languages that can be constructed from the “big three” set
operations viz., (a) Union (b) Concatenation (c) Kleene star. A regular language is defined as
follows.

Definition: Let Σ be an alphabet. The class of “regular languages” over Σ is defined inductively
as follows:

6
50
Regular Expressions:
Regular expressions are designed to represent regular languages with a mathematical
tool, a tool built from a set of primitives and operations. This representation involves
a combination of strings of symbols from some alphabet Σ, parentheses and the
operators +, ⋅ and *. A regular expression is obtained from the symbol {a, b, c},
empty string ∈, and empty-set ∅ with the operations +, ⋅ and * (i.e union,
concatenation and Kleene star).

Examples:
0 + 1 represents the set {0, 1}
1 represents the set {1}
0 represents the set {0}
(0 + 1) 1 represents the set {01, 11}
(a + b ).(b + c) represents the set {ab, bb, ac, bc}
(0 + 1)* = ∈ + (0 + 1) + (0 + 1) (0 + 1) + … = Σ*
∈(0+1)+ = (0+1)(0+1)(0+1)* = Σ+ = Σ*-{∈} 51
Building Regular Expressions
Assume that Σ = {a b, c}
a* means “zero or more instances of a concatenated together”, So a* ={λ,a, aa, aaa,
…}
To say “zero or more ab’s,” = {λ, ab abab, …} = (ab)*.

Also a+ means one or more instances of a concatenated together

E.g. a+ = {a,, aa, aaa, …}

52
Languages defined by Regular Expressions
There is a very simple correspondence between regular expressions and the languages they denote:

Regular Expression L(Regular Expression)

x, for each x ∈Σ {x}

λ {λ}
∅ {}
(r1) L(r1)
r 1∗ L(r1) ∗
r1r2 L(r1)L(r2)
r1 + r 2 L(r1)UL(r2)

53
 TWO-WAY FINITE AUTOMATA
Two-way finite automata are machines that can read input string in either direction.
This type of machines have a “read head”, which can move left or right over the input
string. Like the finite automata, the two-way finite automata also have a finite set Q of
states and they can be either deterministic (2DFA) or nondeterministic (2NFA). They
accept only regular sets like the ordinary finite automata. Let us assume that the
symbols of the input string are occupying cells of a finite tape, one symbol per cell as
shown below. The left and right end markers |— and —| enclose the input string. The
end markers are not part of the input alphabet Σ.

54
Definition:
A 2DFA is an octuple M = (Q, Σ |—, —|, δ, s, t, r)
where, Q is a finite set of states
Σ is a finite set of input alphabet.
|— is the left end marker, |— ∉Σ,
—| is the right end marker, —|∉ Σ,
δ: Q × (Σ 𝖴{|—, —|}) ( → Q × {L, R}) is the transition function.
s∈Q is the start state,
t∈Q is the accept state, and
r∈Q is the reject state, r ≠ t

such that for all the states q,

55
δ takes a state and a symbol as arguments and returns a new state
and a direction to move the head i.e., if δ(p, b) = (q, d), then
whenever the machine is in state p and scanning a tape cell
containing symbol b, it moves its head one cell in the direction d and
enters the state q.

56
FINITE AUTOMATA WITH OUTPUT
Definition: A finite-state machine M = (Q, Σ, O, δ, λ, q0) consists of a finite set Q of states, a finite
input alphabet Σ, a finite output alphabet O, a transition function δ that assigns to each state and input
pair a new state, an output function λ that assigns to each state and input pair an output, and an initial
state q0 . Let M = M = (Q, Σ, O, δ, λ, q0) be a finite state machine. A state table is used to denote the

values of the transition function δ and the output function λ for all pairs of states and input.

Mealey Machine: Usually the finite automata have binary output, i.e., they accept the string or do not
accept the string. This is basically decided on the basis of whether the final state is reached by the initial
state. Removing this restriction, we are trying to consider a model where the outputs can be chosen from
some other alphabet.

57
The values of the output function F(t) in the most general case is a function of the present state q(t) and
present input x(t).

F(t) = λ(q(t), x(t))

where λ is called the output function. This model is called the “Mealey machine”.
A Mealey machine is a six-tuple (Q, Σ, O, δ, λ q0) where all the symbols except λ have the same meaning
as discussed in the section above. λ is the output function mapping Σ × Q into O.

Moore Machine: (To be continued)

58
DETERMINISTIC FINITE AUTOMATA – DFA
What is an Automaton? An automaton is an abstract model of a digital
computer. It has a mechanism to read input, which is a string over a given
alphabet. This input is place on an “input file”, which can be read by the
automaton but cannot change it.
The input file is divided into cells, each of which can
hold one symbol. The automaton has a temporary
“storage” device, which has unlimited number of cells,
the contents of which can be altered by the
automaton. Automaton has a control unit, which is
said to be in one of a finite number of “internal states”.
The automaton can change state in a defined way.
A model of Automaton
Types of Automaton- We have two types of
Automaton
(a) Deterministic Automata
(b) Non-deterministic Automata
A deterministic automata is one in which each move (i.e.
transition from one state to another) is determined by the
current configuration. If the internal state, input and contents
of the storage are known, it is possible to predict the future
behaviour of the automaton. This type of automaton is said to
be deterministic automata, otherwise it is non-determinist
automata.
Definition:

A Deterministic Finite Automaton (DFA) is a 5-tuple

The input mechanism can move only from left to right and
reads exactly one symbol on each step. The transition from
one internal state to another are governed by the transition
function δ.
If δ(q0, a) = q1, then if the DFA is in state q0 and the current

input symbol is a, the DFA will go into state q1 .

Discussion:
The machine, M, accepts the specified language, as long as three consecutive b’s
have not been read. It should be noted that:

(i) M is in state qi (where i = 0,1, or 2) immediately after reading a run of i

consecutive b’s that either began the input string or was preceded by an ‘a’.
(ii) If an ‘a’ is read and M is in state, q0 , q1 , or M returns to its initial state q0.

q0 , q1 and q1 are “final states” (as given in the problem). Therefore any input string
not containing three consecutive b’s will be accepted. In case we get three
consecutive b’s then the q3 state is reached (which is not a final state), hence M will

remain in this state, irrespective of any other symbol in the rest of the string. This
state q3 is said to be a “dead state” or M is said to be “trapped” at q3 . The DFA

schematic is shown below based on the discussion above

Finite Automaton with four states
Example 2: Determine the DFA schematic for M = (Q, Σ, δ, q, F)
where Q = {q1 , q2 , q3 }, Σ = {0,1}, q1 is the start state, F = {q2 }

and δ is given by the table below.

Also determine a Language L recognized by the DFA.
Solution:

From the given table for δ, the DFA is drawn, where q2 is the only final

state.
(It is to be noted that a DFA “accept” a string it can “recognize” a
language. The catch here is that “accept” is used for strings and
“recognize” for of a language).
It could be seen that the DFA accepts strings that has at least one 1 and
an even number of 0s following the last 1. Hence the language L is given
by
Example 3: Sketch the DFA given

M = ({q1 q2 }, {0, 1}, δ, q1, {q2})

and δ is given by

δ (q1, 0) = q1

δ (q2, 0) = q1

δ (q1, 1) = q2

δ (q2, 1) = q2

Determine a Language L(M), that the DFA recognizes.

Solution:
From the given data, it is easy to predict the schematic of DFA as
follows.
Internal states = q1 , q2 .
Symbols = 0, 1.
Transition function = δ (as defined above in the given problem)
q1 = Initial state
q2 = Final state

The state diagram of the DFA

 If a string ends in a 0, it is “rejected” and “accepted” only if
Therefore the language L(M) = {w | w ends in a 1}.
the string ends in a 1.

Example 4: Design a DFA, the language recognized by the

Automaton being
L = { anb : n  0}
Solution:
Therefore the DFA accepts all strings consisting of an
arbitrary number of a’s, followed by a single b. All other
input strings are rejected.
Example 5: Obtain the state table and state transition
diagram (DFA Schematic) of the finite state automaton:
M = (Q, Σ, δ, q0, F),
where Q = {q0 q1 q2 q3}, Σ = {a b}, q0 is the initial state, F is
the final state with the transition defined by
δ(q0,a) = q2 δ(q3,a) = q1 δ(q2,b) = q3 δ(q2,a) = q0

δ(q1,a) = q3 δ(q0,b) = q1 δ(q3,b) = q2 δ(q1,b) = q0

Solution:
The State Table diagram is as shown below

With the given definitions, the State

Transition diagram/DFA Schematic is shown on next slide.
20
Example 6: Obtain the DFA that accepts/recognizes the
language L(M) = {w | w ∈ {a, b, c}* and w contains the pattern
abac}
(Note: This is an application of DFA’s involving searching a text
for a specified pattern
Solution: Let us begin by “hard coding” the pattern into the
machines states as shown in fig. (a) below.
As the pattern ‘abac’ has length four,
there are four states required in
addition to one initial state q0, to
remember the
q4 can be reached only after reading ‘abac’. The complete DFA
pattern. q4 is the only accepting state
is as shown
required andbelow in in this slide.
this state
Exercise: Determine the languages produced by the following FAs

(1)

(i) (ii)

(2) *Construct a finite state machine that accepts only positive integers that are
evenly divisible by 4. Hint: Use Σ = {0, 1}.

* Intermediate
Pushdown
Automata
(PDA)
8
3
8
4
8
5
8
6
Basically a pushdown automaton is
"Finite state machine" + "a stack“

The stack head scans the top symbol of the stack.

A stack does two operations −
• Push − a new symbol is added at the top.
• Pop − the top symbol is read and removed.
A PDA may or may not read an input symbol, but it has to read the top of the stack in
every transition.

8
7
A PDA can be formally described as a 7-tuple (Q, ∑, , δ, q0, Z0, F)
• Q is the finite number of states
• ∑ is input alphabet
•  the set of pushdown symbols on the stack (which can be pushed and popped
from stack)
• δ is a transition function which maps Q x {Σ 𝖴 ∈} x  into Q x *. In a given
state, PDA will read input symbol and stack symbol (top of the stack) and
move to a new state and change the symbol of stack.
• q0 is the initial state (q0 ∈ Q)
• Z0, (Z0 ∈ ), is the initial pushdown symbol (which is initially present in stack)
• F is a set of accepting states (F ∈ Q)

8
8
The arguments of δ are the current state of the control unit, the current input symbol, and the
current symbol on the top of the stack.
The result is a set of pairs (q, x) where q = next state of the control unit, and x = string that is
put on top of the stack in place of the single symbol there before.
The ‘stack’ is an additional component available as part of PDA. The ‘stack’ increases its
memory. With respect to {anbn : n≥1}, we can store a’s in the stack. When the symbol ‘b’ is
encountered, an ‘a’ from the stack can be removed. If the stack becomes empty on the
completion of processing a given string, then the PDA accepts the string.

8
9
Remember that δ is a function of 3 arguments:
δ (current state, current input symbol, current symbol of stack top)
The result is a set of pairs (q, x)
where q = next state of the control unit
x = string that is put on top of the stack in place of the single symbol there before.

Note that while the second argument may be ∈, rather than a member of the input alphabet (so that
no input symbol is consumed), there is no such option for the third argument. δ always consumes a
symbol from the stack, no move is possible if the stack is empty.
There may also be an ∈ -transition, where the second argument may be ∈, which means that a
move that does not consume an input symbol is possible.

9
0
The following diagram shows a transition in a PDA from a state q1 to state q2, labelled as a, b → c
This means at state q1, if we encounter an input string ‘a’ and top symbol of the stack is
‘b’, then we pop ‘b’, push ‘c’ on top of the stack and move to state q2.
Terminologies Related to PDA
Instantaneous Description
Instantaneous Description (ID) is an informal notation of how a PDA “computes” an
input string and makes a decision that string is accepted or rejected.
The instantaneous description (ID) of a PDA is represented by a triplet (q, w, α) where
- q is the current state.
- w is the remaining (unconsumed) input.
- α is the stack contents, top at the left.

92
Execution of NPDA
Assume that someone is in the middle of stepping through a string with a DFA, and we need to take
over and finish the job. There are two things that are required to be known:
(a) the state of the DFA is in, and
(b) what the remaining input is.
But if the automaton is an NPDA we need to know one more viz, contents of the stack.

Instantaneous Description of a PDA Revisited

The Instantaneous description of a PDA is a triplet (q, w, α),
where q = current state of the automaton
w = unread part of the input string
α = stack contents (written as a string, with the leftmost symbol at the top of the stack).
Let the symbol “|–” denote a move of the NPDA, and suppose that δ(q1 , a, x) = {(q2 , y), …}, then the
following is possible:
(q1 , aw, xz) |– (q2 ,w, yz)
where w indicates the rest of the string following ‘a’ and z indicates the rest of the stack contents
underneath the x.
This notation tells that in moving from state q1 to state q2, an ‘a’ is
consumed from the input string aw, and the x at the top (left) of the stack xz is replaced with y, leaving
yz on the stack.

93
Turnstile Notation
⊢ sign is called a “turnstile notation” and represents one move.
⊢* sign represents a sequence of moves (‘zero or more moves’).
E.g. (p, bw, T) ⊢ (q, w, α)

This implies that while taking a transition from state p to state q, the input symbol ‘b’ is
consumed, and the top of the stack ‘T’ is replaced by a new string ‘α’

94
Why PDA?
Let us consider a finite automata which accepts the language
. L1(M) = {ambn: m, n ≥ 1}
We see that machine M will move from state q0 to q1, on the occurrence of a’s. On seeing ‘b’,
M moves from q1 to q2 and continues to be in the state q2 on getting more b’s.
Assume that the input string is given by ambn , then the resulting state is final state and so M
accepts ambn.
Now consider the language L2 (M) = {anbn: n ≥ 1}where the number of b’s and a’s are equal.
The FA constructed for L1 will differ from that of L2.

For the language L1 (M) there is not necessity to remember the number of a’s. The following
have to be remembered.
(a) Whether the first symbol is ‘b’ (to reject the string)
(b) Whether ‘a’ follows ‘b’ (to reject the string)
(c) Whether ‘a’ follows ‘a’ and ‘b’ follows ‘b’ (to accept the string).

95
We know that FA has only a finite number of states, M cannot remember the number
of a’s in anbn where ‘n’ is larger than the number of states of M. Thus the FA does not
accept the sets of the form {anbn: n ≥ 1}

This is taken care by a “PUSHDOWN AUTOMATA”.

96
Example: Consider the set of transition rules of an NPDA given by δ(q1 , a, b) = {(q2 , cd), (q3 , ∈)}

If at any time the control unit is in state q1, the input symbol read is ‘a’,
and the symbol on the top of stack is ‘b’, then one of the following two cases can occur:
(a) The control unit tends to go into the state q2 and the string ‘cd’ replaces ‘b’ on top of the stack.
(b) The control unit goes into state q3 with the symbol b removed from the top of the stack.

In the deterministic case, when the function δ is applied, the automaton moves to a new state q∈Q
and pushes a new string of symbols x ∈* onto the stack. As we are dealing with nondeterministic
pushdown automaton, the result of applying δ is a finite set of (q, x) pairs.

97
NPDAs are not usually drawn. However, with a few minor extensions, we can draw an NPDA similar to the
way we draw an NFA. Instead of labelling an are with an element of , we can label arcs with a | x, y where
a , x  and y  .

Let us consider an example of NPDA given by

98
An Elaborate example:

Construct a PDA that accepts Even Palindromes of the form

L = {wwR | w = (a + b)+}

Before we attempt a solution, let understand the problem. What are Palindromes?
Palindromes are words that spells same forwards and backwards. Even Palindromes
have even length.
e.g.: noon, 123321, abba, racecar, no lemon no melon

99
Also let us take a look once again at the moves of
PDA

10
0
Pushdown
Automata
(PDA)
1
0
1
0
1
0
1
0
Basically a pushdown automaton is
"Finite state machine" + "a stack“

The stack head scans the top symbol of the stack.

A stack does two operations −
• Push − a new symbol is added at the top.
• Pop − the top symbol is read and removed.
A PDA may or may not read an input symbol, but it has to read the top of the stack in
every transition.

1
0
A PDA can be formally described as a 7-tuple (Q, ∑, , δ, q0, Z0, F)
• Q is the finite number of states
• ∑ is input alphabet
•  the set of pushdown symbols on the stack (which can be pushed and popped
from stack)
• δ is a transition function which maps Q x {Σ 𝖴 ∈} x  into Q x *. In a given
state, PDA will read input symbol and stack symbol (top of the stack) and
move to a new state and change the symbol of stack.
• q0 is the initial state (q0 ∈ Q)
• Z0, (Z0 ∈ ), is the initial pushdown symbol (which is initially present in stack)
• F is a set of accepting states (F ∈ Q)

1
0
The arguments of δ are the current state of the control unit, the current input symbol, and the
current symbol on the top of the stack.
The result is a set of pairs (q, x) where q = next state of the control unit, and x = string that is
put on top of the stack in place of the single symbol there before.
The ‘stack’ is an additional component available as part of PDA. The ‘stack’ increases its
memory. With respect to {anbn : n≥1}, we can store a’s in the stack. When the symbol ‘b’ is
encountered, an ‘a’ from the stack can be removed. If the stack becomes empty on the
completion of processing a given string, then the PDA accepts the string.

1
0
Remember that δ is a function of 3 arguments:
δ (current state, current input symbol, current symbol of stack top)
The result is a set of pairs (q, x)
where q = next state of the control unit
x = string that is put on top of the stack in place of the single symbol there before.

Note that while the second argument may be ∈, rather than a member of the input alphabet (so that
no input symbol is consumed), there is no such option for the third argument. δ always consumes a
symbol from the stack, no move is possible if the stack is empty.
There may also be an ∈ -transition, where the second argument may be ∈, which means that a
move that does not consume an input symbol is possible.

1
0
The following diagram shows a transition in a PDA from a state q1 to state q2, labelled as a, b → c
This means at state q1, if we encounter an input string ‘a’ and top symbol of the stack is
‘b’, then we pop ‘b’, push ‘c’ on top of the stack and move to state q2.
Terminologies Related to PDA
Instantaneous Description
Instantaneous Description (ID) is an informal notation of how a PDA “computes” an
input string and makes a decision that string is accepted or rejected.
The instantaneous description (ID) of a PDA is represented by a triplet (q, w, α) where
- q is the current state.
- w is the remaining (unconsumed) input.
- α is the stack contents, top at the left.

11
1
Execution of NPDA
Assume that someone is in the middle of stepping through a string with a DFA, and we need to take
over and finish the job. There are two things that are required to be known:
(a) the state of the DFA is in, and
(b) what the remaining input is.
But if the automaton is an NPDA we need to know one more viz, contents of the stack.

Instantaneous Description of a PDA Revisited

The Instantaneous description of a PDA is a triplet (q, w, α),
where q = current state of the automaton
w = unread part of the input string
α = stack contents (written as a string, with the leftmost symbol at the top of the stack).
Let the symbol “|–” denote a move of the NPDA, and suppose that δ(q1 , a, x) = {(q2 , y), …}, then the
following is possible:
(q1 , aw, xz) |– (q2 ,w, yz)
where w indicates the rest of the string following ‘a’ and z indicates the rest of the stack contents
underneath the x.
This notation tells that in moving from state q1 to state q2, an ‘a’ is
consumed from the input string aw, and the x at the top (left) of the stack xz is replaced with y, leaving
yz on the stack.

11
2
Turnstile Notation
⊢ sign is called a “turnstile notation” and represents one move.
⊢* sign represents a sequence of moves (‘zero or more moves’).
E.g. (p, bw, T) ⊢ (q, w, α)

This implies that while taking a transition from state p to state q, the input symbol ‘b’ is
consumed, and the top of the stack ‘T’ is replaced by a new string ‘α’

11
3
Why PDA?
Let us consider a finite automata which accepts the language
. L1(M) = {ambn: m, n ≥ 1}
We see that machine M will move from state q0 to q1, on the occurrence of a’s. On seeing ‘b’,
M moves from q1 to q2 and continues to be in the state q2 on getting more b’s.
Assume that the input string is given by ambn , then the resulting state is final state and so M
accepts ambn.
Now consider the language L2 (M) = {anbn: n ≥ 1}where the number of b’s and a’s are equal.
The FA constructed for L1 will differ from that of L2.

For the language L1 (M) there is not necessity to remember the number of a’s. The following
have to be remembered.
(a) Whether the first symbol is ‘b’ (to reject the string)
(b) Whether ‘a’ follows ‘b’ (to reject the string)
(c) Whether ‘a’ follows ‘a’ and ‘b’ follows ‘b’ (to accept the string).

11
4
We know that FA has only a finite number of states, M cannot remember the number
of a’s in anbn where ‘n’ is larger than the number of states of M. Thus the FA does not
accept the sets of the form {anbn: n ≥ 1}

This is taken care by a “PUSHDOWN AUTOMATA”.

11
5
Example: Consider the set of transition rules of an NPDA given by δ(q1 , a, b) = {(q2 , cd), (q3 , ∈)}

If at any time the control unit is in state q1, the input symbol read is ‘a’,
and the symbol on the top of stack is ‘b’, then one of the following two cases can occur:
(a) The control unit tends to go into the state q2 and the string ‘cd’ replaces ‘b’ on top of the stack.
(b) The control unit goes into state q3 with the symbol b removed from the top of the stack.

In the deterministic case, when the function δ is applied, the automaton moves to a new state q∈Q
and pushes a new string of symbols x ∈* onto the stack. As we are dealing with nondeterministic
pushdown automaton, the result of applying δ is a finite set of (q, x) pairs.

11
6
NPDAs are not usually drawn. However, with a few minor extensions, we can draw an NPDA similar to the
way we draw an NFA. Instead of labelling an are with an element of , we can label arcs with a | x, y where
a , x  and y  .

Let us consider an example of NPDA given by

11
7
An Elaborate example:

Construct a PDA that accepts Even Palindromes of the form

L = {wwR | w = (a + b)+}

Before we attempt a solution, let understand the problem. What are Palindromes?
Palindromes are words that spells same forwards and backwards. Even Palindromes
have even length.
e.g.: noon, 123321, abba, racecar, no lemon no melon

11
8
Also let us take a look once again at the moves of
PDA

11
9
To generate a string of terminal symbols from a CFG, we:
• Begin with a string consisting of the start symbol;
•Apply one of the productions with the start symbol on the left hand side, replacing the start
symbol with the right hand side of the production;
•Repeat the process of selecting non-terminal symbols in the string, and replacing them with the
right hand side of some corresponding production, until all non-terminals have been replaced by
terminal symbols.

Example 1: Given Σ = {a, b} obtain Σ*.

(a) Give an example of a finite language in Σ.
(b) Given L = {a nbn : n ≥ 0}, check if the strings aabb, aaaabbbb, abb are in the language L.

Solution
Σ = {a, b}
Therefore we have Σ* = {ε, a, b, aa, ab, ba, bb, aaa,}
(a) {a, aa, aab} is an example of a finite language in S.
(b) (i) aa bb → a string in L. (n = 2)
(ii) aaaa bbbb → a string in L. (n = 4)
(iii) abb → not a string in L (since there is no n satisfying this)

Example 2: Given L = {a nbn : n ≥ 0}

Obtain:
(a) L2
(b) LR
Solution
Given L = {a nbn : n ≥ 0}
(a)L2 = {a nbna mbm : n ≥ 0, m ≥ 0}
where n and m are unrelated.
Hence, the string aabbaaabbb is in L2.

(b) Reverse of L is given by :

LR = {bnan : n ≥ 0}

Example 3: Let L = {ab, aa, baa}. Which of the following strings are in L*?
(a) abaabaaabaa
(b) aaaabaaaa
(c) baaaaabaaaab
(d) baaaaabaa
Solution
Please note that L* is the “star-closure” of a language L, given by
L* = L0 ᴗ L1 ᴗ L2 …….
and L+ is the “positive closure” defined by
L+ = L1 ᴗ L2 ……
(a) ab aa baa ab aa → This string is in L*
(b) aa aa baa aa → This string is in L*
(c) baa aa ab aa aa b or baa aa ab aa a ab

(undefined)
(undefined)
This string is not in L*.
(d) baa aa ab aa → This string is in L*.

DERIVATION TREES
A „derivation tree‟ is an ordered tree which the nodes are labeled with the left sides of
productions and in which the children of a node represent its corresponding right sides.

For a given CFG with productions:

S → aA
A→ aB
B → bB
B→a
The derivation tree is as shown below:

The resultant of the derivation tree is the word w = aaba.

Example 1: A grammar G which is context-free has the productions

S → aAB
A → Bba
B → bB
B →c
(The word w = acbabc is derived as follows):
Example 2: A CFG given by productions:
S →a
S→aAS
A→ bS
Obtain the derivation tree of the word w = abaabaa.
Solution
Example 3: Given a CFG given by G = (N, T, P, S)
with N = {S}, T = {a, b},
P : S →aSb
S →ab
Obtain the derivation tree and the language generated L(G).
Solution

Example 4: Given G = (N, T, P, S) with

N = {E}, S = E, T = {id, +, *, c}
and
P: 1. E →E +E
2. E →E *E
3. E → (E)
4. E → id
Obtain the derivation tree.
Solution
Example 5: Obtain the language generated L(G) for a CFG given G = (N, T, P, S) with N = {S},
T{a},
P: 1. S →SS
2. S →a
Solution

Example 6: Obtain the language generated by each of the following production rules.

Solution
Example 7: Given a CFG G = (N,T,P,S)
AMBIGUITY

Example 1: Show that the grammar S → S | S, S → a is ambiguous.

Solution
In order to show that G is ambiguous, we need to find a w  L(G), which is ambiguous.