Lecture 1: Introduction to

Automata Theory

Theory of Automata
 Marks Distribution
• Quizzes = 4 (each of 2.5 marks)
• Assignments
– Programming =3 (each of 2 marks)
– Reading/ writing=3 (each of 2 marks)
– Complex Computing Problem=1 (of 8 marks)
• Midterm (20 marks)
• Final Term (50 marks)
Text Book:Introduction to automata theory, languages and
computation" by JE Hopcroft, R Motwani and JD Ullman.
 Objectives
At the end of the course the students will be able to:
1.Explain and manipulate the different concepts in automata theory
and formal languages such as formal proofs, automata, regular
expressions, Turing machines etc;
2.Prove properties of languages, grammars and automata with
rigorously formal mathematical methods
3.Design automata, RE and CFG
4.Transform between equivalent NFAs, DFAs and REs
5.Define Turing machines performing simple tasks.
6.Differentiate and manipulate formal descriptions of languages,
automata and grammars with focus on regular and context-free
languages, finite automata and regular expressions.
 Course Learning
Outcomes(CLOs)
• Explain the different concepts in automata
theory and formal languages.(C2)
• Analyze properties of languages,
grammars, and automata with formal
methods.(C4)
• Design grammars and models for different
languages. (C5)
 What is Automata Theory?
• Study of abstract computing devices, or
“machines”
• Automaton = an abstract computing device
– Note: A “device” need not even be a physical
hardware!
• A fundamental question in computer science:
– Find out what different models of machines can do
and cannot do
– The theory of computation?
• Computability vs. Complexity

5
 Alan Turing (1912-1954)
(A pioneer of automata theory)

• Father of Modern Computer

Science
• English mathematician
• Studied abstract machines called
Turing machines even before
computers existed
• Heard of the Turing test?

6
Theory of Computation: A
Historical Perspective

1930s • Alan Turing studies Turing machines

• Decidability
• Halting problem

1940-1950s • “Finite automata” machines studied

• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages

1969 Cook introduces “intractable” problems

or “NP-Hard” problems

1970- Modern computer science: compilers,

computational & complexity theory evolve 7
 The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) free Context-
Recursively-
(PDA) sensitive
enumerable
(LBA)
(TM)

8
 Mathematical Concepts
• Sets
• What is a SET? A set is the mathematical
• Elements of a set? model for a collection of
different things
• Set membership
• Set cardinality
• Empty, Singleton
• Set representation :S = {x : x is prime }
• Operations: Union, Intersection and difference
• Power set :2|S| = {x : x ⊆ S}
• Set partitioning
• Cartesian Product
Set partitioning
 Functions
• A function f that is a subset of A × B is
written:
–f:A→B
– (a, b) ∈ f ,is written f(a) = b
• A is the domain of the function
 Pigeonhole Principle
• A, B are finite sets, with |A| > |B|, then
there is no one-to-one function from A to B
• If there are n pigeonholes, and > n
pigeons, and every pigeon is in a
pigeonhole, there must be at least one
hole with > 1 pigeon.
The Central Concepts of
Automata Theory

13
 Alphabet
An alphabet is a finite, non-empty set of symbols
• We use the symbol ∑ (sigma) to denote an
alphabet
• Examples:
– Binary: ∑ = {0,1}
– All lower case letters: ∑ = {a,b,c,..z}
– Alphanumeric: ∑ = {a-z, A-Z, 0-9}
– DNA molecule letters: ∑ = {a,c,g,t}
– …

14
 What will be the alphabets?
• Language of circles
• A language contains basic shapes (circle,
square, rectangle, triangle
• High level language
• Language that may contain binary, octal,
and hexadecimal numbers.
• Language of m As and n Bs.
 Strings
A string or word is a finite sequence of symbols
chosen from ∑
• Empty string is  (or “epsilon”)
• The empty string  is a string of 0 characters
• Length of a string w, denoted by “|w|”, is equal to
the number of (non- ) characters in the string
– E.g., x = 010100 |x| = 6
– x = 01  0  1  00  |x| = ?
– If x is any string then, x  =?

– xy = concatenation of two strings x and y

16
 Powers of an alphabet
Let ∑ be an alphabet.

– ∑k = the set of all strings of length k

– ∑* = ∑0 U ∑1 U ∑2 U …

– ∑+ = ∑ 1 U ∑ 2 U ∑ 3 U …

17
 Languages
L is a said to be a language over alphabet ∑, only if L  ∑*
 this is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed
by n 1’s:
L = {, 01, 0011, 000111,…}
2. Let L be the language of all strings of with equal number of 0’s
and 1’s:
L = {, 01, 10, 0011, 1100, 0101, 1010, 1001,…}

Canonical ordering of strings in the language

Definition: Ø denotes the Empty language
• Let L = {}; Is L=Ø?
NO 18
 The Membership Problem
Given a string w ∑*and a language L over
∑, decide whether or not w L.

Example:
Let w = 100011
Q) Is w  the language of strings with
equal number of 0s and 1s?

19
 Strings (operations)
• We can concatenate a string with itself:
– w1 = w
– w2 = ww
– w3 = www
• By definition, w0 = 
• Reverse a string: wR
– KarachiR =icharaK
 Formal Language
• A formal language (or just language) is a
set of strings
– L1 ={ab, aabb, aaabbb ....}
– L2 ={apple, banana, mango…}
– L3 ={1, 11, 111, 1111, 11111, . . .}
• A language can be either finite or infinite
 Languages (Operations)
• Concatenation
– L1L2 = {wv : w ∈ L1 ∧ v ∈ L2}
– {a, ab}{bb, b} = {abb, ab, abbb}
– {a, ab}{a, ab} = {aa, aab, aba, abab}
– {a, aa}{a, aa} = {aa, aaa, aaaa}
• Union
• Intersection
 Finite Automata
• Some Applications
– Software for designing and checking the behavior of
digital circuits
– Lexical analyzer of a typical compiler
– Software for scanning large bodies of text (e.g., web
pages) for pattern finding
– Software for verifying systems of all types that have a
finite number of states (e.g., stock market transaction,
communication/network protocol)

23
 Finite Automata : Examples
• On/Off switch action

state

• Modeling recognition of the word “then”

Start state Transition Intermediate Final state

state
24
Formal Proofs

25
 Deductive Proofs
From the given statement(s) to a conclusion
statement (what we want to prove)
• Logical progression by direct implications

Example for parsing a statement:

• “If y≥4, then 2y≥y2.”

given conclusion

(there are other ways of writing this).

26
 Example: Deductive proof
Let Claim 1: If y≥4, then 2y≥y2.

Let x be any number which is obtained by adding the squares of 4

positive integers.
Claim 2:
Given x and assuming that Claim 1 is true, prove that 2 x≥x2
• Proof:
– Given: x = a2 + b2 + c2 + d2
– Given: a≥1, b≥1, c≥1, d≥1
  a2≥1, b2≥1, c2≥1, d2≥1 (by 2)
 x≥4 (by 1 & 3)
  2x ≥ x 2 (by 4 and Claim 1)

“implies” or “follows” 27
 On Theorems, Lemmas and Corollaries

We typically refer to:

• A major result as a “theorem”
• An intermediate result that we show to prove a larger result as a
“lemma”
• A result that follows from an already proven result as a
“corollary”

An example:
Theorem: The height of an n-node binary
tree is at least floor(lg n)
Lemma: Level i of a perfect binary tree has
2i nodes.
Corollary: A perfect binary tree of height h
has 2h+1-1 nodes. 28
 Quantifiers
“For all” or “For every”
– Universal proofs
– Notation=
“There exists”
– Used in existential proofs
– Notation=
Implication is denoted by =>
• E.g., “IF A THEN B” can also be written as “A=>B”

29
 Proving techniques
• By contradiction
– Start with the statement contradictory to the given
statement
– E.g., To prove (A => B), we start with:
• (A and ~B)
• … and then show that could never happen

What if you want to prove that “(A and B => C or D)”?

• By induction
– (3 steps) Basis, inductive hypothesis, inductive step
• By contrapositive statement
– If A then B ≡ If ~B then ~A
30
 Proving techniques…
• By counter-example
– Show an example that disproves the claim

• Note: There is no such thing called a

“proof by example”!
– So when asked to prove a claim, an example that
satisfied that claim is not a proof

31
Different ways of saying the same thing

• “If H then C”:

i. H implies C
ii. H => C
iii. C if H
iv. H only if C
v. Whenever H holds, C follows

32
 “If-and-Only-If” statements
• “A if and only if B” (A <==> B)
– (if part) if B then A ( <= )
– (only if part) A only if B ( => )
(same as “if A then B”)
• “If and only if” is abbreviated as “iff”
– i.e., “A iff B”
• Example:
– Theorem: Let x be a real number. Then floor of x =
ceiling of x if and only if x is an integer.
• Proofs for iff have two parts
– One for the “if part” & another for the “only if part”
33
 Conditions
• Which one is true?
• ∀(A,B) A × B = B × A
– If and only if A = B
• ∀(A,B) A × B ≠ B × A
– If and only if A ≠ B
 Summary
• Automata theory & a historical perspective
• Chomsky hierarchy
• Finite automata
• Alphabets, strings/words/sentences, languages
• Membership problem
• Proofs:
– Deductive, induction, contrapositive, contradiction,
counterexample
– If and only if

35
References