THEORY

OF
COMPUTATION
U19CS503 THEORY OF COMPUTATION 3003

COURSE OUTCOMES

At the end of the course, students will be able to

Compare and analyse various Finite Automata

Construct finite automata from regular expression and identify the properties of
regular language

 Design recognizers for different formal languages and Push Down Automata

Convert CFG to normal forms and design turing machines for various problems

 Determine the decidability and intractability of computational problems

UNIT I AUTOMATA
9

Introduction to formal proof − Additional forms of proof −

Inductive proofs – Finite Automata (FA) − Deterministic Finite
Automata (DFA) − Non- deterministic Finite Automata (NFA) −
Finite automata with epsilon transitions. Case Study: Cruise
Control

UNIT II REGULAR EXPRESSIONS AND LANGUAGES 9

Regular expression − FA and Regular expressions − Proving

languages not to be regular − Closure properties of regular
languages − Equivalence and minimization of automata.
UNIT III CONTEXT-FREE GRAMMAR AND LANGUAGES 9

Context Free Grammar (CFG) − Parse trees − Ambiguity in grammars and

languages − Definition of the pushdown automata − Languages of a
pushdown automata − Equivalence of pushdown automata and CFG −
Deterministic pushdown automata- Pumping lemma for CFL − Closure
properties of CFL

UNIT IV TURING MACHINE 9

Normal forms for CFG − −Turing machines −Language of TM -

Programming Techniques for TM, Variants in TM Case study: Church’s
Thesis-Godelization.

UNIT V UNDECIDABILITY 9

Recursively Enumerable (RE) - An Undecidable problem that is RE –

Halting Problem− Post’s correspondence problem – Rice
Theorem−Classes P and NP –NP Completeness-Relationship between
Time Complexity and Space Complexity.
General Info for the Course

Introduction to
Automata Theory,
Languages and
Computation
J.E. Hopcroft, R.
Motwani, J.D.
Ullman
2nd Edition
 INTRODUCTION

• What is Theory of Computation?

• Why do we study theory of computation?
• What is the use of theory of computation?
• History of Theory of Computation
• Alphabet, strings and length of string
What is TOC?
In the theory of computation area, we use
mathematical techniques.

It deals with whether and how efficiently problems

can be solved on a model of computation, using
an algorithm.

The field is divided into three major branches:

automata theory, computability theory and
computational complexity theory
 Automata theory
Automata theory is the study of abstract machines (or more
appropriately, abstract 'mathematical' machines or systems) and the
computational problems that can be solved using these machines.

These abstract machines are called automata.

Automata comes from the Greek word (Αυτόματα) which means

that something is doing something by itself.

Automata theory is also closely related to formal language theory

8
 Computability theory
Computability theory deals primarily with the question of the extent
to which a problem is solvable on a computer.

Computability theory is closely related to the branch of

mathematical logic called recursion theory, which removes the
restriction of studying only models of computation which are
reducible to the Turing model.

Eg:
Halting problem cannot be solved by a Turing machine

Rice's theorem, which states that for all non-trivial properties of

partial functions, it is undecidable whether a Turing machine
computes a partial function with that property.
9
 Computational complexity
theory
Complexity theory considers not only whether a problem can be
solved at all on a computer, but also how efficiently the problem
can be solved.

Two major aspects are considered: time complexity and space

complexity
-How many steps does it take to perform a computation, and
-How much memory is required to perform that computation.

In order to analyze how much time and space a given algorithm

requires, computer scientists express the time or space required to
solve the problem as a function of the size of the input problem.

10
Why do we study theory of computation?
a computer is what??
it is simply a machine made of hardware??

but why we need computer....

we need computer to perform computation on a specific task....

how this computation will occur in the computer

is defined by a set of rules ...

to understand these rules ,we need to study theory

of computation....
theory drives practice and practice drives theory....to
make efficient machines for computation tasks we
need to study theory of computation.
 Why we still need TC?
• Technologies become obsolete but basic
theories remain forever.
• TC provides tools for solving
computational problems like regular
expressions for string parsing and pattern
matching.
• Studying different types of grammars like
CFG would help in many other areas like
compilers design and natural language
processing.
What is the use of theory of computation?
In simple words to learn any natural language like
ENGLISH, HINDI,FRENCH....
firstly we need to learn the vocabulary and grammar of
that language. That means we have to learn how the
language is actually specified.

In the same way programming languages (formal

languages) like C,C++, JAVA....
has their own vocabulary and grammar and such
grammar is specified with the help of mathematical
model that is called as Theory of Computation.
History of Theory of Computation
1936 Alan Turing invented the Turing machine, and proved that
there exists an unsolvable problem.

1940’s Stored-program computers were built.

1943 McCulloch and Pitts invented finite automata.

1956 Kleene invented regular expressions and proved the

equivalence of regular expression and finite automata.

1956 Chomsky defined Chomsky hierarchy, which organized

languages recognized by different automata into hierarchical
classes.

1959 Rabin and Scott introduced nondeterministic finite automata

and proved its equivalence to (deterministic) finite automata.
1950’s-1960’s More works on languages, grammars,
and compilers
1965 Hartmantis and Stearns defined time complexity,
and Lewis, Hartmantis and Stearns defined space
complexity.
1971 Cook showed the first NP-complete problem, the
satisfiability problem.
1972 Karp Showed many other NP-complete problems.
1976 Diffie and Helllman defined Modern Cryptography
based on NP-complete problems.
1978 Rivest, Shamir and Adelman proposed a public-
key encryption scheme, RSA.
 Chomsky’s Hierarchy of formal
languages

All the languages

Recursively Enumerable
Recursive

Context Sensitive
Context Free
Regular
16
 Language recognizers
Language Type Recognizer

Regular 3 Finite Automata

Context Free 2 Push Down Automata

Linear Bounded
Context Sensitive 1
Automata
Recursive 0 Turing Machine
Unrestric
ted or
Recursively Turing
recogniza Turing Machine
Enumerable ble

Non Recursively
NO DEVICE
Enumerable
17
Mathematical Preliminaries
Set
•Set is defined as collection of objects.
•These objects are called elements of the
set.
A= {a,b,c}, B={0,1,2}
Subset
The subset A is called subset of set B, if
every element of set A is present in the set
B but reverse is not true.
It is denoted by A ⊆ B
Eg: A={1,2,3} and B={1,2,3,4,5}, then A ⊆ B
Empty Set
The set having no element in it is called
empty set. It is denoted by A={ } and it can
be written as Φ.

Power Set
The power set is a set of all the subsets of
its element.
Eg: A={1,2,3}
Find power set of Q is,
Power set of Q={{Φ},{1},{2},{3},{1,2},{1,3},{2,3},{1,2,3}}
The number of elements are always equal to 2 n where
n is number of elements in original set. As in set A
there are 3 elements.
Therefore in power set Q there are 2 3=8 elements

Equal Set
The two sets are said to be equal (A=B) if A ⊆ B and
B ⊆ A, i.e, every element of set A is an element of B
and every element of B is an element of A.
A={1,2,3} and B={1,2,3} then A=B
 Alphabet and Strings
•An alphabet is a finite, non-empty set of
symbols.
-{0,1 } is a binary alphabet.
-{ A, B, …, Z, a, b, …, z } is an English alphabet.

Example:
1. Σ = {a,b}, an alphabet of 2 symbols a
and b.
2. Σ = {0,1,2} an alphabet of 3 symbols
0,1,2
 A string or word over an alphabet set Σ is a
finite sequence of symbols from Σ
-0, 1, 11, 00, and 01101 are strings over {0, 1 }.
-Cat, CAT, and compute are strings over the
English alphabet.

Examples:
1. Σ = {a,b} then abab, aabba, aaabba are all strings over the
alphabet Σ ={a,b}

2. Σ ={a} then a,aa,aaa are all string over the alphabet Σ ={a}

We use lower case letters a,b,c for elements of Σ and

u,v,w for string names
 Empty String or Null string

An empty string, denoted by , is a string

containing no symbol.
The string consisting of zero symbols or the
length of a string is zero.
It is denoted by Є or λ
-  is a string over any alphabet.
Length
The length of a string x, denoted by length(x), is
the number of positions of symbols in the string.

Let Σ = {a, b, …, z}
length(automata) = 8
length(computation) = 11
length(ε) = 0

x(i), denotes the symbol in the ith position of a

string x, for 1 i  length(x).
Examples:
1. If W = abcd then | w| =4
2. If x=01010110 then | x | = 8
3. If y = 0110 then | y | = 4
4. | Є | = 0 =| λ|
Concatenation of string:
The concatenation of two strings w and v is
the string obtained by appending the
symbols of v to the right end of w.

i.e , w = a1a2a3 ………am

v= b1b2b3………….bn
then wv= a1a2a3 ………amb1b2b3………….bn
ex:

i) x = 010 y = 1
xy = 0101 yx= 1010

ii) x = third y = year

xy = thirdyear

iii) Empty string is the identity element for

concatenation operator ie, w = w = w
Reverse of a string:
The reverse of a string is obtained by writing the
symbols in reverse order.
Let w be a string. Then its reverse is W R

ie W= a1a2……am
WR = am…….a2a1

Ex:
Let u= 0101011
uR = 1101010
Power of an alphabet:
Let Σ be an alphabet
Σ * denotes the set of all strings over the alphabet Σ .
Σ m denotes the set of all strings over the alphabet Σ of length m.

Ex:
If Σ = {0,1} then
Σ 0 ={ ε } empty string
Σ 1 ={0,1} set of all strings of length one over Σ = {0,1}
Σ 2 = {00,01,10,11} set of all strings of length two over Σ = {0,1}
Σ 3 ={000,001,010,011,100,101,110,111} set of all strings of length
three over Σ = {0,1}
Kleene closure:
Let Σ be an alphabet. Then “Kleene closure Σ *” denotes the set
of all strings (including ε, empty string ) over the alphabet Σ.

Ex:
i) If Σ = {a} then Σ* = { ε,a,aa,aaa,…..} that is
Σ 0 ={ ε }
Σ1={a}
Σ 2 = {aa}

ii) If Σ = {0,1} the Σ * = { ε, 0,1,00,01,10,……}

iii) If Σ = {0} the Σ * = (ε, 0,00,000,…….}

Therefore Σ * = Σ 0 U Σ 1 U Σ 2 U………….
Substring:
A string v appears within another string w(w=uv) is
called “ substring of w”. If w=uv, the substrings u & v
are said to be prefix and suffix of w respectively.
Ex:
i) w = abbab be a string over an alphabet Σ = {a,b}
then a,ab,abb,ba,bab,….. are all substrings of w.

ii) w = 123
prefixes = { ε , 1,12,123}
suffixes = { ε , 3, 23, 123}

iii) w= abbab
prefixes = { ε ,a,ab,abb,abba,abbab}
suffixes = { ε ,b,ab,bab,bbab,abbab}
Palindrome:
A palindrome is a string which is same whether
written forward or backward.

Ex: madam, Malayalam, non,nun,121

If the length of a palindrome is even, then it can be

obtained by concatenation of a string and its reverse.
Ex:
If u=01 uR = 10
Then even palindrome = 0110
Properties of string operation:
i) concatenation is associative ; that is for all strings u,v
and w.
(uv)w = u(vw)

ii) If u and v are strings, then the length of their

concatenation is the sum of the individual lengths, i.e,
| uv| = |u| + |v|

Ex:
X = abc Y = 123
XY = abc123
|XY| = 6 i.e |X| = 3 |Y| = 3
Hence |XY| = |X| + |Y|.
 Σ*
• The set of strings created from any number
(0 or 1 or …) of symbols in an alphabet  is
denoted by *.
• That is, * = i=0 i
– Let  = {0, 1}.
 * = {, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, … }.
 Σ +

• The set of strings created from at least one

symbol (1 or 2 or …) in an alphabet  is
denoted by +.
• That is, + = i=1  i
= i=0.. i - 0
=  i=0.. i - {}
• Let  = {0, 1}. + = {0, 1, 00, 01, 10, 11, 000, 001, 010,
011, … }.
* and + are infinite sets.
 Languages
• A language over an alphabet Σ is a set of strings over
Σ.
– Let Σ = {0, 1} be the alphabet.

– Le = {Σ* | the number of 1’s in  is even}.

 , 0, 00, 11, 000, 110, 101, 011, 0000, 1100, 1010, 1001, 0110, 0101,
0011, … are in Le
 Operations on Languages
• Complementation
• Union
• Intersection
• Concatenation
• Reversal
• Closure
 Complementation
Let L be a language over an alphabet Σ.
The complementation of L, denoted byL, is Σ*–L.

Example:
Let Σ = {0, 1} be the alphabet.
Le = {Σ* | the number of 1’s in  is even}.
Le= {Σ* | the number of 1’s in  is not even}.
Le= {Σ* | the number of 1’s in  is odd}.
 Union
Let L1 and L2 be languages over an alphabet Σ.
The union of L1 and L2, denoted by L1L2, is {x | x is in
L1 or L2}.
Example:
{x{0,1}*|x begins with 0}  {x{0,1}*|x ends with 0}
= {x  {0,1}*| x begins or ends with 0}
 Intersection
Let L1 and L2 be languages over an alphabet Σ.
The intersection of L1 and L2, denoted by L1L2, is { x |
x is in L1 and L2}.

Example:
{ x{0,1}*| x begins with 0}  { x{0,1}*| x ends with 0}
= { x{0,1}*| x begins and ends with 0}
 Concatenation
Let L1 and L2 be languages over an alphabet Σ.
The concatenation of L1 and L2, denoted by L1L2, is {w1w2| w1 is in
L1 and w2 is in L2}.
Example
{ x  {0,1}*| x begins with 0}{x  {0,1}*| x ends with 0}
= { x  {0,1}*| x begins and ends with 0 and length(x)  2}
{ x  {0,1}*| x ends with 0}{x  {0,1}*| x begins with 0}
= { x  {0,1}*| x has 00 as a substring}
 Reversal
Let L be a language over an alphabet Σ.
The reversal of L, denoted by Lr, is {wr| w is in L}.
Example
{x  {0,1}*| x begins with 0} r
= {x  {0,1}*| x ends with 0}
{x  {0,1}*| x has 00 as a substring} r
= {x  {0,1}*| x has 00 as a substring}
Consider the string X=110 and Y=0110 then
find
i) xy
ii) yx
iii) x2
iv) ϵ y
Answer
xy = 1100110
yx= 0110110
x2 =110110
ϵ y = 0110