BCS503

Theory of Computation
Some Notes
 DISCLAIMER

• Students are urged to read the textbook. Please read

these notes only if you need a gist or summary.
1. These notes might contain mistakes. Please read and
refer to the textbook when in doubt. Send me any
mistakes you find and I will correct them.
• These notes are subject to modifications.
• They focus on theory and not on examples or
problems. You are requested to read the examples
from the textbook and solve exercises given there. Try
all the assignment questions. Try all the questions
from the question bank.
• In case of any difficulty, contact me.
Version History
Version 1.0 08 November 2024
Version 2.0 07 December 2024
Version 2.1 08 December 2024
Table of Contents
Version History .................................................................................................................................. 3
Central Concepts ............................................................................................................................... 6
Alphabet ........................................................................................................................................... 6
String................................................................................................................................................. 6
Power of an Alphabet ................................................................................................................... 7
Language .......................................................................................................................................... 7
Languages and Problems ............................................................................................................ 7
Simple and Powerful Automata ................................................................................................ 8
Finite Automata ................................................................................................................................. 8
Deterministic Finite Automata ................................................................................................. 8
Non-deterministic Finite Automata ......................................................................................... 9
Non-deterministic Automata with ε-transitions................................................................ 10
Conversions between DFA, NFA and ε-NFA ....................................................................... 10
Definition of ECLOSE ................................................................................................................ 11
Regular Expressions ...................................................................................................................... 11
Proving Languages not to be regular ........................................................................................ 12
Example 1 ...................................................................................................................................... 12
Examples 2 (Example 4.3 from textbook) ........................................................................... 12
Context-Free Languages and Pushdown Automata............................................................. 13
Context-Free Grammars and Context-Free Languages .................................................. 13
Pushdown Automata .................................................................................................................. 13
Instantaneous Description or ID for a PDA ........................................................................ 15
Transition Diagram for a PDA ................................................................................................. 15
PDA Example for L = {0n1n | n≥1} ...................................................................................................... 15
PDA Example (Equal Number of 0’s and 1’s) .................................................................................... 16
Deterministic Context-Free Languages (DCFLs) ............................................................... 17
DCFLs and Ambiguity............................................................................................................ 18
DCFLs and Programming Languages ............................................................................... 18
Normal Forms ....................................................................................................................................... 18
Chomsky Normal Form (CNF)............................................................................................................ 19
Greibach Normal Form (GNF) ........................................................................................................... 20
Pumping Lemma for Context-Free Languages ...................................................................................... 20
Turing Machines.............................................................................................................................. 20
 Instantaneous Description or ID of a TM ............................................................................ 21
Transition Diagram for a TM ................................................................................................... 22
Language of a TM ........................................................................................................................ 22
Example of TM .................................................................................................................................. 22
Example of TM which recognizes L = {0n1n | n ≥ 0} .......................................................................... 23
Decidability and Undecidability ................................................................................................. 23
Recursively Enumerable (RE) .................................................................................................. 24
Recursive........................................................................................................................................ 24
The Diagonal Language Ld........................................................................................................ 24
The Universal Turing Machine................................................................................................ 25
The Universal Language Lu ...................................................................................................... 25
Central Concepts
Automata theory is the study of different types of formal
automata or machines which have applications in
computer science. For example, finite state machines or
finite automata are used in the design of hardware such
as sequential logic, registers and processors. Finite
automata are used in compilers for recognizing tokens
such as identifiers, keywords, literal and operators.
Pushdown automata or automata with a single stack are
used for checking the syntax of a written program and
understanding it. That is, the program is checked for
errors and target code is generated. Automata theory is
thus a core subject in computer science.

An automaton is used in general to accept a language. A

language is made up of symbols.

Alphabet
An alphabet is a finite non-empty set of symbols. It is
usually denoted by the Greek capital letter ∑. Typical
examples of alphabets are the binary alphabet {0, 1} and
the set of lower case English letters {a, b, c, ..., z}.

String
A string is a finite sequence of symbols from an alphabet.
It is usually denoted by w. The length of a string w,
denoted |w|, is the number of symbols in the sequence
w. For example |abc| = 3 and |01001| = 5. We also have
the empty string containing no symbol (or 0 symbols),
which is denoted by the Greek letter ͼ (epsilon). |ͼ| = 0.
Power of an Alphabet
If n ≥ 0 is an integer, ∑n represents the set of all strings
of length n, using ∑ as the alphabet. Note that ∑0 = {ͼ}.
Now, the set of all strings over ∑, represented as ∑* is
defined as ∑0 U ∑1 U ∑2 U ∑3 U ∑4 ... where U represents
set union operator.

Language
A language over an alphabet is a set of strings using that
alphabet. That is, a language L over ∑ is given by L
subset of ∑*. Note that L can be either finite or infinite.
An example of a finite language over {0, 1} is {01, 11,
0011, 000011} and an example of an infinite language
over {0, 1} is L = {w | w has the same number of 0’s as
1’s}

Finite Automata are associated with regular expressions.

The language accepted by a finite automaton can be
represented as a regular expression. Similarly, a regular
expression that represents a language can be converted
to a finite automaton which accepts that language.

Languages and Problems

Associated with every language is a decision problem. If
w is a string, and L is a language, there is a decision
problem whether x in L. This problem has an yes or no
answer.
Similarly, every decision problem P, defines a language L
= {x | P(x) is yes}.
Decision problems are special cases of general
computational problems. A general computational
problem can be approximately worded using a decision
problem. For example, instead of computing the radius of
a sphere, we can ask questions like is the radius greater
than 10 cm?

Simple and Powerful Automata

Automata try to decide some decision problems. That is
each automaton can accept a language. The simplest
kinds of automata we study are finite automata. The next
type of automata which are more powerful and can
accept more types of languages are called pushdown
automata. The most powerful automata are Turing
machines. Turing machines can do whatever any general
computer can do. However there are some problems
which even Turing machines cannot solve. These are
called undecidable problems.

Finite Automata
A finite automaton is an automaton or machine which
can exist in only one of a finite number of states at any
given time.
Deterministic Finite Automata
A deterministic finite automaton or DFA has a finite set
of states Q, an alphabet ∑, a state transition function δ,
an initial or start state q0, and a set of final states F
subset of Q. In other words, a DFA A = (Q, ∑, δ, q0, F) is a
5-tuple. The transition function δ, can be represented by
a transition diagram or a transition table. For ever state
and a symbol, there is a single next state.
δ : Qx∑→Q
δ can be extended to a string w in ∑*. Let δ^ be the
extended transition function given by:
1. δ^(q, ε) = q
2. δ^(q, xa) = δ(δ^ (q, x), a) for x in Σ* and a in Σ.
For all q in Q.
The language of A is L(A) = {w| δ^(q0,w) is in F}.

Non-deterministic Finite Automata

A non-deterministic finite automaton or NFA has a finite
set of states Q, an alphabet ∑, a state transition function
δ, an initial or start state q0, and a set of final states F
subset of Q. In other words, an NFA N = (Q, ∑, δ, q 0, F) is
a 5-tuple. The transition function δ, can be represented
by a transition diagram or a transition table. For every -
state and a symbol, there are zero or more next states,
giving a subset of Q.
δ : Qx∑→2Q
δ can be extended to a string w in ∑*. Let δ^ be the
extended transition function given by:
1. δ^(q, ε) = q
2. δ^(q, xa) = Union of δ(δ^(q, x), a) for x in Σ* and a in
Σ.
For all q in Q.
The language of N is L(N) = {w| δ^(q0,w) contains a state
in F}.

Non-deterministic Automata with ε-transitions

We might add the possibility of moving from one state to
another in an NFA without consuming any input. Such
transitions are called ε-transitions. An NFA with one or
more ε-transitions is definitely an ε-NFA.
Formally, it is of the form N = (Q, ∑, δ, q0, F) where:
δ : Qx(∑ union {ε})→2Q.

Conversions between DFA, NFA and ε-NFA

The three kinds of finite automata are equivalent.
A DFA is (like) an NFA and an NFA is like a special case
of an ε-NFA. We can show that given an NFA N, we can
convert it into an equivalent DFA.
Let N = (Q, Σ, δ, q0, F) be an NFA. We construct DFA as D
= (2Q, Σ, δD, {q0}, FD). Everything is known here except for
δD and FD. We define them as:
1. δD(S, a) = union over p in S of δ(p, a)
2. FD is {S | S intersection F is not empty}
It can then be shown by induction on the length of a
string that both these machines accept the same
language.
For proving that an ε-NFA can be converted to an
equivalent DFA, we follow a similar approach, except
that we also use the ECLOSE function.

Definition of ECLOSE
Basis: For any state q, ECLOSE(q) contains q.
Induction: If p is in ECLOSE(q) and there is an ε-
transition from state p to state r, then r is also in
ECLOSE(q).

Regular Expressions
Basis:
1. The empty set ϕ is a regular expression representing
the language ϕ = {}.
2. The empty string ε is a regular expression
representing the language {ε}.
3. If a is in the alphabet, a is a regular expression
representing {a}.
Induction:
Let E and F be regular expressions standing for L(E) and
L(F) respectively. Then:
1. E+F represents L(E) union L(F)
2. E.F or simply EF represents L(E).L(F)
3. E* represents (L(E))*
4. (E) represents the same language as E with
parentheses giving importance to the operators
contained within.
Proving Languages not to be regular
We use the pumping lemma for regular languages
(Theorem 4.1 from the textbook) to show that some
languages are not regular.

Example 1
L = {0n1n | n ≥ 0}. Let, if possible, L be regular. Let n be
the constant as given by the pumping lemma. (n is the
pumping length.) Consider w = 0n1n which is in L. |w| >
n. Hence, as per the pumping lemma, w=xyz where |xy|
≤ n. Let |y|=m>0. Consider the string xyiz. The number
of 0’s in it is n-m+im and the number of 1’s is n. These
two numbers are not equal unless i=1. For example, if
i=2, the number of 0’s is n+m, which is different from n.
This goes against the lemma. Therefor L is not regular.

Examples 2 (Example 4.3 from textbook)

L = {0p | p is prime}. Let, if possible, L be regular. Let n
be the constant as given by the pumping lemma. (n is the
pumping length.) Let p be a prime number greater than
n+1. We can definitely find such a p, because we know
that there are infinitely many primes. Consider w = 0p
which is is L. |w| > n. Hence, as per the pumping
lemma, w=xyz where |xy| ≤ n. Let |y|=m>0. Consider
the string xyiz. The number of 0’s in it is p-m+im. Choose
i=p-m. The number of 0’s is p-m+(p-m)m=(p-m)(m+1)
which is a composite number. This goes against the
lemma. Therefor L is not regular.
Context-Free Languages and Pushdown Automata
Context-Free Grammars and Context-Free Languages
A context-free grammar (CFG) is G = (V, T, P, S), where T
is a set of terminals, V is a set of variables, P is a set of
productions and S is a special variable from V called the
start symbol. Note that T and V are disjoint. So, variables
are also called non-terminals. Every production in P has
the form A → β, where A is a variable from V and β in (V
union T)* is a string of variables and terminals.
A single step derivation from a variable A is obtained by
replacing A with the right-hand side of a production
starting from variable A. We write this single step as A =>
β. If β contains a variable B (which may or may not be
the same as A), we can replace B with the right-hand
side of a production involving B. Then we get a two-step
derivation as A => αBγ => αδγ where B → δ is a
production. We can thus have a derivation from a
variable A in 0 or more steps of a string ω of variables
and terminals. This is represented A =>* ω. If S =>* w
and w is a string consisting only of terminals, then w is
in the language given by G. That is, L(G) = {w|w in T* and
S=>*w}. The languages given by context-free grammars
are called context-free languages.

Pushdown Automata
A pushdown automaton P is a 7-tuple (Q, Σ, Γ, δ, q0, Z0,
F). Q is a finite set of states the machine can be in. Σ is
the input alphabet. The input is scanned from left to
right and is either accepted or rejected. P has a stack
which can grow without any limit. The symbols which
can be pushed on the stack are given by the stack
alphabet Γ. The state q0 in Q is the starting state of P. F
is a subset of Q (not necessarily proper). If P reaches a
state in F, it accepts the input seen so far. The string of
symbols on the stack is written from left to right, with
the left most symbol being on the top of the stack. When
P starts, it has only one symbol Z0 on the stack. Moves of
P are given by the transition function δ. When the
current state of P is q, the input scanned is symbol a
from Σ, and the stack contains a symbol X from Γ on the
top, the next state can be given by p and symbol X can
be replaced by string γ from Γ*. So, δ(q, a, X) contains (p,
γ). Note that δ is, in general multi-valued. δ(q, a, X) is in
general a set of pairs of the form (q, γ). This means PDA P
is in general non-deterministic. Not only that, P can
have ε-moves, that is, moves without consuming any
input. We write an ε-move as δ(q, ε, X) which is a set of
values of the form (p, γ).

Given a PDA PF = (Q, Σ, Γ, δ, q0, Z0, F), it can accept

when the state becomes one in F. This is called
acceptance by final state. Given a PDA PN = (Q, Σ, Γ, δ,
q0, Z0, F), it can accept when the stack becomes empty.
This is called acceptance by empty stack. F is irrelevant
for PN; we can write PN = (Q, Σ, Γ, δ, q0, Z0). Both the ways
of acceptance are equivalent. It can be shown that given
a PN, we can construct an equivalent PF. Likewise, given a
PF, we can construct an equivalent PN. The proofs are
detailed in the textbook along with examples.
It can be shown that a CFG and a PN are equivalent.
Thus context-free grammars and pushdown automata
define the same set of languages called the context-free
languages.

Instantaneous Description or ID for a PDA

An ID for a PDA is (q, w, γ), where q is the current state,
w is the remainder of the input and γ is the string of
symbols on the stack. The moves of a PDA can be
thought of as changes in ID following the δ function.

Transition Diagram for a PDA

The transition diagram for a PDA contains all states in Q
as nodes in a graph. If δ(q, a, X) contains (p, γ), where a
is either in Σ or a is ε, we draw an arc from q to p
labelled a,X/γ. Note that q and p might even be the same
node. The start state q0 is specially marked with an
incoming (horizontal) arrow. All nodes in F are
represented using double concentric circles.

PDA Example for L = {0n1n | n≥1}

This PDA has three states, q0, q1 and q2. q0 is the start
state and q2 is the only accepting state. The stack
alphabet is {Z0, 0, 1}. If Z0 is on the stack and the input is
0, push 0 on the stack. If 0 is on the stack and 0 is in the
input, push 0. If 0 is on the stack and 1 is the input,
change to state q1 and pop. When in state q1, if 0 is on
the stack and 1 is in the input, pop. If the state is q1 and
Z0 is exposed on the top of the stack, without consuming
any input, move to state q2.
The transition diagram is given below.

PDA Example (Equal Number of 0’s and 1’s)

The following PDA accepts the language of all strings over
{0, 1} which contain the same number of 0’s as 1’s by
empty stack: The stack symbols are {Z0, 0, 1}. If the top
of the stack contains Z0 and the input symbol is 0, push
0 on the stack. If the top of the stack contains Z0 and the
input symbol is 1, push 1 on the stack. If the top of the
stack contains 0 and the input symbol is 0, push 0 on
the stack. If the top of the stack contains 1 and the input
symbol is 0, push 0 on the stack. If the top contains 1
and the input is 0, or, if the top contains 0 and the input
is 1, pop the symbol from the stack. If the top symbol is
Z0, pop it without consuming any input. This PDA
accepts all strings with the same number of 0’s and 1.s
by empty stack.
The transition diagram for the PDA is given below.

Note that there is only one state q0 in this PDA. Also,

here there is no need for any accepting state.

Deterministic Context-Free Languages (DCFLs)

A PDA is deterministic if:
1. |δ(q, a, X)| ≤ 1 for a in Σ or a = ε, and
2. If |δ(q, a, X)| = 1 for any a in Σ, δ(q, ε, X) is empty.

This means, given any ID for the PDA, there is at most

one next ID. The moves of the PDA are deterministic.
There is never a choice of moves.
The languages corresponding to DPDAs are called
deterministic context-free languages (DCFLs). It can be
shown that DCFLs are a proper subset of CFLs.
Specifically, the language L = {wwR| w in Σ* and wR is the
reverse of w} is a CFL but not a DCFL. We do not have aa
proof for this but can show by intuition that it can be
recognized by a PDA but not by a DPDA.

It can be easily shown that any regular language is a

DCFL. Using the pumping lemma for regular languages,
we can show how some DCFLs are not regular. Thus,
there is a hierarchy of three distinct language classes:
regular languages, DCFLs. and CFLs.

DCFLs and Ambiguity

It can be shown that any DCFL can have an
unambiguous grammar.

DCFLs and Programming Languages

Most Programming Languages can be modelled using
DCFLs. So, there is a DPDA for recognizing a
Programming Language. This is useful for constructing
Compilers.

Normal Forms
A context-free grammar can be put in a normal form. A
normal for restricts the types of productions. We discuss
two normal forms: Chomsky Normal Form and Greibach
Normal Form.
Chomsky Normal Form (CNF)
A context-free grammar is in CNF if every production is
of the form A → BC or A → a where A, B and C are
variables and a is a terminal.
For any context-free language which is not empty and
which does not contain the null string ε, we can find a
context-free grammar which is in CNF.
To do this, we start with a context-free grammar for the
language and then follow the three steps:
1. Eliminate ε-Productions (Productions of the form A →
ε).
2. Eliminate Unit productions (of the form A → B).
3. Eliminate Useless Symbols.
A grammar symbol X (variable or terminal) is useful if S
=>* αXβ =>* γ for some γ in Σ*. That is X occurs in a
derivation from the start symbol which yields a string of
terminals. X is useless if it is not useful. To eliminate all
useless symbols, the following steps need to be followed
in proper order:
1. Eliminate non-generating variables. That is,
eliminate any variable A from which we cannot
derive a string of terminals.
2. Eliminate unreachable symbols, That is, eliminate
all grammar symbols which cannot be reached from
S.
After these steps, it is easy to convert the grammar into
CNF by the following procedure:
1. Let all productions of the form A → a or A → BC to
remain.
 2. Introduce new variables so that the remaining
productions are of the form A → α where α is a string
of variables.
3. Break the productions A → α where |α| > 2: Let α =
Bβ. Replace it with productions A → BC and C → β.
Repeat 3 till CNF is achieved.

Greibach Normal Form (GNF)

In Greibach Normal Form, each production is of the form
A → aα, where A is a variable, a is a terminal and α is a
(possibly empty) string of variables. Every context-free
language which is not empty and does not contain ε can
be put in GNF.

Pumping Lemma for Context-Free Languages

The existence of CNF can be used to prove the following
pumping lemma:
If L is a context-free language, there is a constant n
(depending on L) such that if z is a string in L and
|z| ≥ n, z can be written as z = uvwxy, where |vwx|
≤ , vx is not ε and for all i ≥ 0, uviwxiy must be in L.

The pumping lemma can be used to show that some

languages are not context-free.

Turing Machines
A Turing Machine (TM) is one of the most general models
of computation. TMs are the most powerful automata in
their ability to recognize languages. A TM M = (Q, Σ, Γ, δ,
q0, B, F). Q is a finite set of states. The special state q0 in
Q is the start state. M is in the start state q0 when the
machine starts. M has infinitely many cells or squares on
which tape symbols can be written stretching from left to
right. The symbols are from the tape alphabet Γ. Σ is the
input alphabet. In the beginning, the input is provided
on the tape between two infinite sequences of blanks.
The blank B is a special tape symbol which is not in Σ.
All symbols from Σ as well as B are part of Γ. F is the set
of final accepting states. M has a tape head which is
positioned over one of the cells. Initially, the tape head is
scanning the first input symbol which is on the right of
the a blank.
The transition function δ is given by δ(q, X) = (p, Y, D)
where p and q are states, X and Y are symbols and D is a
direction, either left (L) or right (R). What this means, is
that when M is in state q, and the tape head has finished
scanning symbol X, the state changes to p, X is replaced
by Y and the head moves in the direction given by D.
Note that p can be the same as q and Y can be the same
as X. In some cases, δ(q, X) might not even be defined.
Then, M has no further moves; it has halted in state q.

Instantaneous Description or ID of a TM
The ID is given by X1X2…Xi-1qXiXi+1…Xn. The n tape
symbols X1 to Xn are present between the leftmost non-
blank symbol and the rightmost non-blank symbol on
the tape. The tape head is positioned over symbol Xi. The
current state is q. The moves of a TM can be visualized
as a change of IDs. For example, if δ(q, Xi) = (p, Y, R), the
ID changes to X1X2…Xi-1YpXi+1…Xn Similarly, if δ(q, Xi) =
(p, Y, L), the next ID is X1X2…pXi-1YXi+1…Xn.

Transition Diagram for a TM

The transition diagram is a graph with a node for each
state in Q. All states in F are represented with double
concentric circles. The start state q0 is marked with an
incoming (horizontal) arrow. If δ(q, X) = (p, Y, R), the edge
from q to p has a label X/Y→. If δ(q, X) = (p, Y, L), the
edge from q and p has the mark X/Y.

Language of a TM
The language of a TM is the set of all strings w such that
starting from the ID q0w, M reaches a final state in F.
Once a final state is reached, we can assume, there is no
further move. The Turing Machine is said to halt if there
is no further move.

Example of TM
This ia a TM which replaces a sequence of 0’s with 1’s
and accepts
Example of TM which recognizes L = {0n1n | n ≥ 0}

Please verify these examples.

Decidability and Undecidability

Every TM can be coded in binary. So, every TM
corresponds to a distinct natural number. The set of TMs
is infinite, but it is countably infinite. The cardinality of
the set of TMs is the same as N, the set of natural
numbers.
Now, consider decision problems. Every decision problem
has a binary answer (yes or no) for every input instance
which can be coded in binary. Thus, the cardinality of
the set of functions (decision problems) is the same as
the cardinality of 2N (the power set of natural numbers).
By Cantor’ss Theorm, the cardinality of the set of
functions is strictly much more than the cardinality of
the set of Turing Machnes (programs). This counting
argument show that there exist problems which cannot
be solved using computers.

Recursively Enumerable (RE)

A language L is said to be recursively enumerable is
there is a Turing Machine M such that if w is in L, M on
input w halts and accepts. (Note that if w is not in L, M
will either halt and reject or go on forever.)
Recursive
A language L is recursive if there is a Turing Machine M
which halts on all inputs and if w is in L, M on input w
accepts.

The Diagonal Language Ld

Consider the diagonal language Ld = {M | M on input M
does not accept}.
Let, if possible, Md be a Turing Machine accepting Ld.
Does Md accept Md? If it does, then Md does not belong to
Ld, so Md does not accept Md. On the other hand, if Md
does not accept Md, by definition, Md belongs to Ld and so
Md accepts Md. This contradiction shows that there can
be no TM accepting Ld. Thus Ld is not RE.

The Universal Turing Machine

It has been shown that there exists a Universal Turing
Machine U which when given an input containing the
encoding of a TM M and its input w, simulates M on w.
That is U on input <M, w> halts and accepts if M
accepts, and halts and rejects if M rejects w and goes on
forever if M does not halt on w. An example of U with 3
tapes is given in chapter 9 of the textbook.

The Universal Language Lu

Consider the language Lu = {<M, w> | M on input w halts
and accepts). We find that Lu is RE because we can
simply feed <M, w> to U and it will halt and accept if
<M. w> is in Lu.
We now show that Lu is not recursive. Suppose Lu were
recursive. Let Mu be the TM which halts on all inputs and
accepts Lu. We can construct an algorithm for Ld as
follows: On input M, feed <M, M> to Mu. If Mu says yes
then say no. If Mu says no then say yes. Thus, we have a
TM which halts on all inputs and decides Ld. This is not
possible because Ld is not RE. Thus, Mu cannot exist. Lu
is not recursive.