20CS2054

THEORY OF COMPUTATION

Dr. T. Mathu,
Assistant Professor/ DSCS,
School of CST.
 INTRODUCTION
• WHAT IS COMPUTATION?
– is any type of arithmetic or non-arithmetic
calculation that follows a well-defined model
(algorithm – step by step solution to a
problem)
Computations may be simple or complex
Examples of Computational Problems:
Multiplication of two numbers
Finding a word in the dictionary
Finding distance between two cities
Predicting weather, etc
 COMPUTATIONAL DEVICES / COMPUTATION MODEL

• The kind of model / device that is used for solving

the computational problems or to run the
algorithm is the computation model /
computational device.
• Examples:
– Calculator
– Smartphone
– Computer
– Pen and Paper
• In Theory of Computation, we study various
abstract models of computation devices based on
the resources that they use.
(abstract -- existing in thought or as an idea (theory) but not having
a physical or concrete existence.)
Alan Turing (1912-1954)
(A pioneer of automata theory)

►Father of Modern Computer Science

►English mathematician
►Studied abstract machines called
Turing machines even before
computers existed
►Heard of the Turing test?

5
• Turing’s goal is to describe precisely the
boundary between what a computing could
do and what it could not do.
ie. What kind of problems can be solved
and what cannot be solved
 Computational
‘Models’
• To model the hardware of
a computer, we introduce
the notion of an
automaton (plural,
automata).
• An automaton is a
construct that possesses
all the indispensable
features of a digital
computer.
• It accepts input, produces
output, may have some
temporary storage, and
can make decisions in
transforming the input
into the output.
 What is Automata Theory?

►Study of abstract computing devices, or “machines”

►Automaton = an abstract computing device
►Note: A “device” need not even be a physical
hardware!
►A fundamental question in computer science:
►Find out what different models of machines can do and
cannot do
►The theory of computation
►Computability vs. Complexity
8
 Why Study Automata Theory?
1. To give introduction to Finite Automata which is a
useful model for many important kinds of hardware
and software. Some of them are..
► Software for designing and checking the behavior of digital circuits.

► Lexical Analyzer of the compiler.

► Software

► For text processing,

► To find occurrences of words, phrases or patterns

► Software for verifying systems of all types that have a finite number of
distinct states, such as communication protocols or protocols for secure
exchange of information.
• There are many systems or components such
as those enumerated above that may be
viewed as always being in one of a finite
number of states.
• The purpose of a state is to remember the
relevant portion of the system’s history.
 Structural Representations
2. To study two important notations that play
an important role in the study of automata
and their applications but they are not
automaton-like.
• Grammars-- Grammars are useful models when
designing software that processes data with a
recursive structure
• Regular Expressions--denote the structure of
data especially text strings. (e.g. email address)
3. It is essential for the study of the limits of
computation ►What a computer can and
cannot do
►What is computable, ►Are you trying to write a non-
and what is not ? existing program?
►Can you make your program
more efficient?

►What a computer can do?

►Basis of
►decidable
►Algorithm analysis ►What a computer can do
►Complexity theory efficiently?
►tractable and intractable
Applications in Computer Science

►Analysis of algorithms
►Compilers
►Complexity Theory
►Circuit design
►Cryptography

13
 Chomsky
Hierarchy
• Computation models
– Finite State
Automata
– Push Down
Automata
– Linear Bounded
Automata
– Turing Machines
COMPUTATION

CPU memory
temporary memory

input memory
CPU
output memory

Program memory
 temporary memory

Example: input memory

CPU
output memory
Program memory
compute

compute
 temporary memory
input memory

CPU
output memory
Program memory
compute

compute
 temporary memory

input memory

CPU
output memory
Program memory
compute

compute
 input memory
temporary memory

CPU

Program memory output memory

compute

compute
 Automaton

temporary memory

Automaton
input memory
CPU
output memory

Program memory
 Different Kinds of Automata

Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

 Finite Automaton

temporary memory

input memory
Finite
Automaton
output memory

Example: Vending Machines

(small computing power)
 Stack Push, Pop

input memory
Pushdown
Automaton
output memory
Example: Compilers for Programming Languages
(medium computing power)
 Turing Machine

Random Access Memory

input memory
Turing Machine

output memory

Examples: Any Algorithm

(highest computing power)
 Power of Automata

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
 Formal Language
• A formal language is an abstraction of the
general characteristics of programming
languages.
• The formal language consists of a set of
symbols and some rules of formation by which
these symbols can be combined into entities
called sentences.
• A formal language is the set of all sentences
permitted by the rules of formation.
 Formal Language
• A formal language consists of words whose
letters are taken from an alphabet and are
well-formed according to a specific set of
rules.
• The alphabet of a formal language consist of
symbols, letters, or tokens that concatenate
into strings of the language.
 Course Objectives

Enable the student to

1. describe the logical and mathematical foundations of
computer science.
2. design and construct various abstract models of
computation.
3. apply the mathematical concepts to solve
computational problems
 Course Outcomes

The student will be able to

1. understand the concept of languages, basics of grammars and
the hierarchy of languages.
2. construct and compare deterministic and non-deterministic
finite automata
3. develop push down automata from a given context free
language or context free grammar
4. recognize context sensitive grammars
5. design turing machine for recursively enumerable language
6. compare computability & non-computability and Decidability
& Un-decidability.
Module 1: Introduction
Alphabet - strings, languages and grammars, productions and
derivation, Chomsky hierarchy of languages.

Module 2: Regular Languages and Finite Automata

Regex Grep Tool for text searching, AI Chatbot using regular
expression engine. Regular expressions and languages, deterministic
finite automata (DFA) & equivalence with regular expressions,
nondeterministic finite automata (NFA) & equivalence with DFA,
regular grammars & equivalence with finite automata, properties of
regular languages, pumping lemma for regular languages, minimization
of finite automata
Module 3: Context-Free Languages and Pushdown
Named Entity Recognition by Parsing XML, Tower of Hanoi using
PushDown Automata, Palindrome checking. Context-free grammars
(CFG) and languages (CFL), Chomsky and Greibach normal forms,
nondeterministic pushdown automata (PDA) and equivalence with CFG,
parse trees, ambiguity in CFG, pumping lemma for context-free
languages, deterministic pushdown automata, closure properties of
CFLs,

Module 4: Context-Sensitive Languages

Context-sensitive grammars (CSG) and languages, linear bounded
automata and equivalence with CSG.
Module 5: Turing Machines
Turing Test in Artificial Intelligence, Design of a Calculator using Turing
Machine, List Appending. The basic model for Turing machines (TM),
Turing recognizable (recursively enumerable) and Turing-decidable
(recursive) languages and their closure properties, variants of Turing
machines, nondeterministic TMs and equivalence with deterministic TMs,
unrestricted grammars and equivalence with Turing machines, TMs as
enumerators

Module 6: Undecidability
Halting Problem, Wicked Problem. Church-Turing thesis, Universal Turing
machine, the universal and diagonalization languages, reduction between
languages and Rice’s theorem, undecidable problems about languages.
 Textbooks
1. John E. Hopcroft, Rajeev
Motwani and Jeffrey D.
Ullman, “Introduction to
Automata Theory, Languages,
and Computation”, 3rd
edition, Pearson Education
Asia. 2014, ISBN: 978-93-
325-3586-2.

2. Peter Linz, “An Introduction to

Formal Languages and
Automata”, 6th Edition, Jone
and Bartlett Student Edition,
2016. ISBN 9789384323219.
 Reference Books

1. Harry R. Lewis and Christos H. Papadimitriou, “Elements of the

Theory of Computation”, Pearson Education Asia. 2nd edition 2015.
ISBN 9789332549890.
2. Dexter C. Kozen, “Automata and Computability,” Undergraduate
Texts in Computer Science, Springer.2012, ISBN 9781461273097.
3. John Martin, “Introduction to Languages and The Theory of
Computation”, Tata McGraw Hill. 3rd edition 2009, ISBN
9780070660489.
4. Kamala Krithivasan, Rama R, “Introduction to Formal Languages,
Automata Theory and Computation”, Pearson Edition, 2009. ISBN:
9788131723562.
5. https://thenewstack.io/tutorial-find-strings-in-text-files-using-grep-
with-regular-expressions/
6. http://botartisanz.com/blog/create-your-eliza-chatbot-in-20-
minutes-with-regular-expressions-day-6
7. https://towardsdatascience.com/parsing-xml-named-entity-
recognition-in-one-shot-629a8b9846ee
8. https://link.springer.com/chapter/10.1007%2F978-3-540-
39886-8_6
9. https://www.geeksforgeeks.org/turing-test-artificial-
intelligence/
10. https://www.geeksforgeeks.org/turing-machine-addition/
11. https://www.geeksforgeeks.org/halting-problem-in-theory-
of-computation/