Theory of Computation: C S I S
Theory of Computation: C S I S
CS 2204
Theory of Computation
Introduction
Prafulla Kalapatapu
Dept of CSE, Program Coordinator for AI
Center for Sustainable Infrastructure and Systems
(CSIS) CS2204-TOC
CSIS - MU
École Centrale School of Engineering, Mahindra University
Centre for Sustainable Infrastructure
and Systems (CSIS)
Lecture -1
Introduction
Course Details, Components, Motivation
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Course Content :
- Slides will be shared to your outlook mails on every Friday.
Text
- “Introduction to the Theory of Computation” Sipser, 3rd Edition.
(Other editions ok but are missing some Exercises and Problems).
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Course Coverage
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Course Expectation
Prerequisites
• Prior substantial experience and comfort with mathematical concepts, theorems,
and proofs.
• Creativity & practice will be needed for attempting exams well.
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Course Motivation
1. Applications
2. Basic Research
3. Connections to other fields
4. What is the nature of computation?
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
What is computable?
Examples:
– check if a number n is prime
– compute the product of two numbers
– sort a list of numbers
– find the maximum number from a list
Hard but computable:
– Given a set of linear inequalities, maximize a linear function
Eg. maximize 5x+2y
3x+2y < 53
x < 32
5x –9y > 22
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Theory of Computation?
Primary aim of the course: What is “computation”?
• Can a computer solve any problem, given enough time and disk-space? Or
are they fundamental limits to computation?
Logic Building
Algorithms, flow charts, logics
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Problem Statement 1
• Write an algorithm to find sum of numbers from 1 to 10
Pre-Analysis :
1 2 3 4 5 6 7 8 9 10
3
Logic = prev sum + nxt numb
6
10
15
21
CSIS - MU CS2204-TOC
Centre for Sustainable Infrastructure
and Systems (CSIS)
Algorithm
step 1 : Begin
step 2 : input n
step 3 : initialize sum=0, i=1
step 4 : sum= sum+i
step 5 : i=i+1
step 6 : if (i<=n) then goto step4
step 7 : display sum
step 8 : end
CSIS - MU CS2204-TOC
Centre for Sustainable Infrastructure
and Systems (CSIS)
Problem Statement 2
• Write an algorithm to find sum of individual digits of a given number
Pre-Analysis :
Ex: given number n = 376
3 7 6
13 Logic = individual digit(unit’s place)
+ prev sum
16
First get individual digit of Unit’s
place
CSIS - MU CS2204-TOC
Centre for Sustainable Infrastructure
and Systems (CSIS)
Algorithm
step 1 : Begin
step 2 : initialize n=376 or input n
step 3 : initialize sum=0, r
step 4 : r = n % 10 Observations :
• Why r=n%10 to be written
step 5 : sum = sum + r • Why n=n/10
step 6 : n = n /10 • n>0
step 7 : if (n>0) then goto step4
step 8 : display sum
step 9 : end
CSIS - MU CS2204-TOC
Centre for Sustainable Infrastructure
and Systems (CSIS)
Problem Statement 3
Write an algorithm to know given number is Armstrong number or not
Pre-Analysis :
Ex: given number n = 153
1 5 3
3*3*3 = 27 Logic = cube of individual digit(unit’s
5*5*5 =125 place) + prev sum
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Algorithm
step 1 : Begin
step 2 : initialize n=153 or input n
step 3 : initialize sum=0, d=n , r
step 4 : r = n % 10
Observations :
step 5 : sum = sum + r*r*r • Why d=n to be written
• What if instead of d=n, n=d written?
step 6: n = n /10 • r*r*r
step 7 : if (n>0) then goto step4 • sum == d , why not sum == n
step 8 : if(sum==d) then
step 8.1 : display “Armstrong”
else
step 8.2 : display “not Armstrong”
step 9 : end
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Problem Statement 4
Write an algorithm to find the given number is even or odd
Analysis : 0 even
Logic = n % 2 =
1 odd
Algorithm
step 1 : begin
step 2 : Read n
step 3 : if(n%2 == 0) then
step 3.1 : print “even”
else
step 3.2 : print “odd”
step 4 : end
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Problem Statement 5
Write an algorithm to swap two numbers
(i) using temp variable
(ii) without using temp variable
Logic :
How many variables required ?
3
c=a
- First value a=b
- Second value b=c
- Temp variable
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Lecture -2
Automata, computation, complexity
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
• Theory of computation
- We develop models of computation, and ask:
-> what can and cannot be computed in these models, and how
quickly? with how much memory?
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Theory of computation:
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
• Automata Theory
• Computability Theory
• Complexity Theory
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Automata Theory
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Computability Theory
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Complexity Theory
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Lecture -3
Automata Theory
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
28
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
(A pioneer of automata theory)
Alan Turing (1912-1954)
• Father of Modern Computer Science
• English mathematician
• Studied abstract machines called Turing
machines even before computers existed
• Heard of the Turing test?
CSIS - MU CS2204-TOC
29
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
30
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU Image source: Nowak et al. Nature, vol 417, 2002 CS2204-TOC
31
Centre for Sustainable Infrastructure
and Systems (CSIS)
Regular Context-
(DFA) free Context- Recursively-
(PDA) sensitive enumerable
(LBA) (TM)
CSIS - MU CS2204-TOC
32
Prafulla Kalapatapu
33 Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
Centre for Sustainable Infrastructure
and Systems (CSIS)
Alphabet
An alphabet is a finite, non-empty set of symbols
• We use the symbol ∑ (sigma) to denote an alphabet
• Examples:
• Binary: ∑ = {0,1}
• All lower case letters: ∑ = {a,b,c,..z}
• Alphanumeric: ∑ = {a-z, A-Z, 0-9}
• DNA molecule letters: ∑ = {a,c,g,t}
• …
CSIS - MU CS2204-TOC
34
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Strings
A string or word is a finite sequence of symbols chosen from ∑
• Empty string is (or “epsilon”)
CSIS - MU CS2204-TOC
35
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Powers of an alphabet
Let ∑ be an alphabet.
• ∑* = ∑0 U ∑1 U ∑2 U …
• ∑+ = ∑1 U ∑2 U ∑3 U …
CSIS - MU CS2204-TOC
36
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Languages
L is a said to be a language over alphabet ∑, only if L ∑*
➔ this is because ∑* is the set of all strings (of all possible length including 0) over the given
alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed by n 1’s:
L = {, 01, 0011, 000111,…}
2. Let L be the language of all strings of with equal number of 0’s and 1’s:
L = {, 01, 10, 0011, 1100, 0101, 1010, 1001,…}
CSIS - MU CS2204-TOC
37
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Example:
Let w = 100011
Q) Is w the language of strings with equal number of 0s and 1s?
CSIS - MU CS2204-TOC
38
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Finite Automata
• Some Applications
• Software for designing and checking the behavior of digital circuits
• Lexical analyzer of a typical compiler
• Software for scanning large bodies of text (e.g., web pages) for pattern finding
• Software for verifying systems of all types that have a finite number of states
(e.g., stock market transaction, communication/network protocol)
CSIS - MU CS2204-TOC
39
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
40
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Structural expressions
• Grammars
• Regular expressions
• E.g., unix style to capture city names such as “Palo Alto CA”:
CSIS - MU CS2204-TOC
41
Prafulla Kalapatapu
42 Centre for Sustainable Infrastructure
and Systems (CSIS)
Formal Proofs
CSIS - MU CS2204-TOC
Centre for Sustainable Infrastructure
and Systems (CSIS)
Deductive Proofs
From the given statement(s) to a conclusion statement (what we want to prove)
• Logical progression by direct implications
CSIS - MU CS2204-TOC
43
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Claim 2:
Given x and assuming that Claim 1 is true, prove that 2x≥x2
Proof:
1) Given: x = a2 + b2 + c2 + d2
2) Given: a≥1, b≥1, c≥1, d≥1
3) ➔ a2≥1, b2≥1, c2≥1, d2≥1 (by 2)
4) ➔x≥4 (by 1 & 3)
5) ➔ 2x ≥ x2 (by 4 and Claim 1)
“implies” or “follows”
CSIS - MU CS2204-TOC
44
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
An example:
Theorem: The height of an n-node binary tree is at least floor(lg n)
CSIS - MU CS2204-TOC
45
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Quantifiers
“For all” or “For every”
• Universal proofs
• Notation=
“There exists”
• Used in existential proofs
• Notation=
CSIS - MU CS2204-TOC
46
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Proving techniques
• By contradiction
• Start with the statement contradictory to the given statement
• E.g., To prove (A => B), we start with:
• (A and ~B)
• … and then show that could never happen
CSIS - MU CS2204-TOC
47
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Proving techniques…
• By counter-example
• Show an example that disproves the claim
CSIS - MU CS2204-TOC
48
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
49
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
“If-and-Only-If” statements
• “A if and only if B” (A <==> B)
• (if part) if B then A ( <= )
• (only if part) A only if B ( => )
(same as “if A then B”)
• “If and only if” is abbreviated as “iff”
• i.e., “A iff B”
• Example:
• Theorem: Let x be a real number. Then floor of x = ceiling of x if and only if x is
an integer.
• Proofs for iff have two parts
• One for the “if part” & another for the “only if part”
CSIS - MU CS2204-TOC
50
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
• Read chapter 1 for more examples and exercises from prescribed TextBook
CSIS - MU CS2204-TOC
51
Prafulla Kalapatapu
52 Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
53
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
CSIS - MU CS2204-TOC
55
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
• Input: a word w in ∑*
• Question: Is w acceptable by the DFA?
• Steps:
• Start at the “start state” q0
• For every input symbol in the sequence w do
• Compute the next state from the current state, given the current input symbol in w and
the transition function
• If after all symbols in w are consumed, the current state is one of the
accepting states (F) then accept w;
• Otherwise, reject w.
CSIS - MU CS2204-TOC
56
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Regular Languages
• Let L(A) be a language recognized by a DFA ‘A’.
• Then L(A) is called a “Regular Language”.
CSIS - MU CS2204-TOC
57
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Regular Context-
(DFA) free Context-
Recursively-
(PDA) sensitive
enumerable
(LBA)
(TM)
CSIS - MU CS2204-TOC
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
3. L= DFA which accepts all the strings containing atleast 3 a’s with {a,b}
into three broad categories atleast, exactly, atmost.
CSIS - MU CS2204-TOC
60
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Example #2
DFA with ∑ = {0, 1} accepts all starting with 0.
CSIS - MU CS2204-TOC
61
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
Example #3
• Build a DFA for the following language:
• L = {w | w is a binary string that contains 01 as a substring}
• Steps for building a DFA to recognize L:
• ∑ = {0,1}
• Decide on the states: Q
• Designate start state and final state(s)
• δ: Decide on the transitions:
• “Final” states == same as “accepting states”
• Other states == same as “non-accepting states”
CSIS - MU CS2204-TOC
62
Prafulla Kalapatapu
Centre for Sustainable Infrastructure
and Systems (CSIS)
1 0,1 • ∑ = {0,1}
0
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Accepting • Transition table
state symbols
0 1
q0 q1 q0
• What if the language allows
states
q1 q1 q2
empty strings? *q2 q2 q2
CSIS - MU CS2204-TOC