Theory of Automata and

Formal Languages

1
What is Automata Theory?
! Study of abstract (existing in thoughts or as an
idea) computing devices, or “machines”
! Automaton = an abstract computing device
! Note: A “device” need not even be a physical
hardware!
! A fundamental question in computer science:
! Find out what different models of machines can do and
cannot do
! The theory of computation
! Computability vs. Complexity
2
(A pioneer of automata theory)

Alan Turing (1912-1954)

! Father of Modern Computer Science
! English mathematician
! Studied abstract machines called
Turing machines even before
computers existed
! Heard of the Turing test?

3
Theory of Computation: A
Historical Perspective
1930s • Alan Turing studied Turing machines
• Decidability
• Halting problem
1940-1950s • “Finite automata” machines studied
• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages
1969 Cook introduces “intractable” problems
or “NP-Hard” problems
1970- Modern computer science: compilers,
computational & complexity theory evolve
4
 Languages & Grammars
! Languages: “A language is a
collection of sentences of finite
length all constructed from a
finite alphabet of symbols”
! Grammars: “A grammar can be
regarded as a device that
enumerates the sentences of a
language” - nothing more,
nothing less

! N. Chomsky, Information and

Control, Vol 2, 1959

Image source: Nowak et al. Nature, vol 417, 2002

5
The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) Context-
free Recursively-
sensitive
(PDA) enumerable
(LBA) (TM)

6
The Central Concepts of
Automata Theory

7
Alphabet
An alphabet is a finite, non-empty set of symbols
! We use the symbol ∑ (sigma) to denote an
alphabet
! Examples:
! Binary: ∑ = {0,1}
! All lower case letters: ∑ = {a,b,c,..z}
! Alphanumeric: ∑ = {a-z, A-Z, 0-9}
! DNA molecule letters: ∑ = {a,c,g,t}
! …

8
Strings
A string or word is a finite sequence of symbols
chosen from ∑
! Empty string is ε (or “epsilon”)

! Length of a string w, denoted by “|w|”, is equal to

the number of (non- ε) characters in the string
! E.g., x = 010100 |x| = 6
! y = 1010101 |x| = ?

! xy = concatentation of two strings x and y

9
Powers of an alphabet
Let ∑ be an alphabet.

! ∑k = the set of all strings of length k

! ∑* = ∑0 U ∑1 U ∑2 U …

! ∑+ = ∑1 U ∑2 U ∑3 U …

10
Languages
L is said to be a language over alphabet ∑, only if L ⊆ ∑*
" this is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed
by n 1’s:
L = {ε, 01, 0011, 000111,…}
2. Let L be the language of all strings of with equal number of 0’s
and 1’s:
L = {ε, 01, 10, 0011, 1100, 0101, 1010, 1001,…}
Canonical ordering of strings in the language

Definition: Ø denotes the Empty language

! Let L = {ε}; Is L=Ø? NO
11
The Membership Problem
Given a string w ∈∑*and a language L over
∑, decide whether or not w ∈L.

Example:
Let w = 100011
Q) Is w ∈ the language of strings with equal
number of 0s and 1s?

12
Finite Automata
! Some Applications
! Software for designing and checking the behavior of
digital circuits
! Lexical analyzer of a typical compiler
! Software for scanning large bodies of text (e.g., web
pages) for pattern finding
! Software for verifying systems of all types that have a
finite number of states (e.g., stock market transaction,
communication/network protocol)

13
Finite Automata : Examples
action
! On/Off switch state

! Modeling recognition of the word “then”

Start state Transition Intermediate Final state

state
14
Structural expressions
! Grammars
! Regular expressions
! E.g., unix style to capture city names such as
“Palo Alto CA”:
! [A-Z][a-z]*([ ][A-Z][a-z]*)*[ ][A-Z][A-Z]

Start with a letter

A string of other
letters (possibly Should end w/ 2-letter state code
empty)

Other space delimited words

(part of city name) 15
Formal Proofs

16
 Deductive Proofs
• A deductive proof consists of a sequence of statement whose truth leads us from some
initial statement (hypothesis or given statements) to a conclusion statement.

• Each step of a deductive proof MUST follow from a given fact or previous statements
(or their combinations) by an accepted logical principle.

• The theorem that is proved when we go from a hypothesis H to a conclusion C is the

statement ’’if H then C’’. We say that C is deduced from H.

BBM401 Automata Theory and Formal Languages 22

 Deductive Proofs
Example: Proof of a Theorem
• Assume that the following theorem (initial statement) is given:
– Given Thm. (initial statement): If x ≥ 4, then 2x ≥ x2
– We are not going to prove this theorem, we assume that it is true.
• If we want we can prove this theorem using proof by induction.

• Theorem to be proved:
If x is the sum of the squares of four positive integers, then 2x ≥ x2

Hypothesis Conclusion

BBM401 Automata Theory and Formal Languages 23

 Deductive Proofs
Example: Proof of a Theorem
Proof of
If x is the sum of the squares of four positive integers, then 2x ≥ x2

Statement Justification
1. If x ≥ 4, then 2x ≥ x2 Given theorem
2. x = a2 + b2 + c2 + d2 Given
3. a ≥ 1 b≥1 c≥1 d≥1 Given
4. a2 ≥ 1 b2 ≥ 1 c2 ≥ 1 d2 ≥ 1 From (3) and principle of arithmetic
5. x ≥ 4 From (2), (4) and principle of arithmetic
6. 2x ≥ x2 From (1) and (5)

BBM401 Automata Theory and Formal Languages 24

 Read it by yourself

Second Induction Example

• If x ≥4 then 2x ≥ x2
• Basis: If x=4, then 2x is 16 and x2 is 16.
Thus, the theorem holds.
• Induction: Suppose for some x ≥4 that 2x ≥
x2. With this statement as the hypothesis,
we need to prove the same statement, with
x+1 in place of x: 2(x+1) ≥ (x+1)2

Second Induction Example

• 2(x+1) ≥ (x+1)2 ? (i)
• Rewrite in terms of S(x)
– 2(x+1) = 2*2x
– We are assuming 2x ≥ x2
– So therefore 2(x+1) = 2*2x ≥ 2x2 (ii)
• Substitute (ii) into (i)
– 2x2 ≥ (x+1)2
– 2x2 ≥ (x2+2x+1)
– x2 ≥ 2x+1
– x ≥ 2 + 1/x
– Since x >=4, we get some value >=4 on the left side. The right
side will equal at most 2.25 and in fact gets smaller and
approaches 2 as x increases. Consequently, we have proven the
theorem to be true by induction.

28
 On Theorems, Lemmas and Corollaries
We typically refer to:
! A major result as a “theorem”
! An intermediate result that we show to prove a larger result as a
“lemma”
! A result that follows from an already proven result as a “corollary”

An example:
Theorem: The height of an n-node binary
tree is at least floor(lg n)
Lemma: Level i of a perfect binary tree has
2i nodes.
Corollary: A perfect binary tree of height h
has 2h+1-1 nodes.
19
Quantifiers
“For all” or “For every”
! Universal proofs
! Notation=
“There exists”
! Used in existential proofs
! Notation=
Implication is denoted by =>
! E.g., “IF A THEN B” can also be written as “A=>B”

20
Proving techniques
! By contradiction
! Start with the statement contradictory to the given statement
! E.g., To prove (A => B), we start with:
! (A and ~B)
! … and then show that could never happen

! By induction
! (3 steps) Basis, inductive hypothesis, inductive step
! By contrapositive statement
! If A then B ≡ If ~B then ~A

21
Proving techniques…
! By counter-example
! Show an example that disproves the claim

! Note: There is no such thing called a

“proof by example”!
! So when asked to prove a claim, an example that
satisfied that claim is not a proof

22
Different ways of saying the same
thing
! “If H then C”:
i. H implies C
ii. H => C
iii. C if H
iv. H only if C
v. Whenever H holds, C follows

23
 Proof of an iff Theorem
Let x be a real number. Then x = x if and only if x is an integer.
If-Part:
• Given that x is an integer.
• By definitions of ceiling and floor operations. x = x and x = x
• Thus, x = x .

Only-If-Part:
• Given that x = x
• By definitions of ceiling and floor operations. x ≤ x and x ≥ x
• Since given that x = x , x ≤ x and x ≥ x
• By the properties of arithmetic inequalities, x = x
• Since x is always an integer, x MUST be integer too.

BBM401 Automata Theory and Formal Languages 26

Summary
! Automata theory & a historical perspective
! Chomsky hierarchy
! Finite automata
! Alphabets, strings/words/sentences, languages
! Membership problem
! Proofs:
! Deductive, induction, contrapositive, contradiction,

counterexample
! If and only if

! Read chapter 1 for more examples and exercises

25