Week 1-Introduction Advanced Theory of Computation

Theory of Automata & Formal

Languages

Thursday, January 18, 2024

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Acknowledgement

 While most of the slides are from the Automata course at

coursera.org by Jeff Ullman, and
 from the Theory of Automata course given in 2013 @ FAST-
ISL,
 some slides are also taken from the lecture notes @
http://www.eecs.wsu.edu/~ananth/CptS317/Lectures/index.htm
 The slides used in this lecture are taken from Automata course by Dr.
Azam at FAST-CFD campus.

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Books
• Introduction to Theory of Computation,
Michael Sipser, 3rd Edition
• Introduction to Computer Theory, Danial I. A.
Cohen
• Introduction to Automata Theory,
Languages, and Computations, John Hopcroft
and Jeffery Ullman, 3rd Edition
• An Introduction to Formal Languages and
Automata, Peter Linz
• Introduction to Languages and the Theory of
Computation, John C. Martin. Fourth Edition

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

What is the study of Automata Theory?

• The study of abstract computing devices, or “machines.”

• Days before digital computers
– What is possible to compute with an abstract machine
– Seminal work by Alan Turing
• Why is this useful?
– Direct application to creating compilers, programming languages,
designing applications.
– Formal framework to analyze new types of computing devices, e.g.
biocomputers or quantum computers.
– Develop mathematically mature computer scientists capable of
precise and formal reasoning!
• Some major topics in Automata Theory

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Finite State Automata

• Deterministic and non-deterministic finite state
machines.
• Regular expressions and languages.
• Techniques for identifying and describing regular
languages; techniques for showing that a
language is not regular. Properties of such
languages.

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Context-Free Languages
• Context-free grammars, parse trees.
• Derivations and ambiguity.
• Relation to pushdown automata.
• Properties of such languages and techniques for
showing that a language is not context-free.

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Turing Machines
• Basic definitions and relation to the notion of an
algorithm or program.
• Power of Turing Machines.

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Computation

CPU memory

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

temporary memory

input
CPU
output

Program memory

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Example: f ( x)  x 3

temporary memory

input
CPU
output
Program memory
compute xx
compute x x
2
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

f ( x)  x 3

temporary memory
input
x2
CPU
output
Program memory
compute xx
compute x x
2
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation
temporary memory
z  2*2  4 f ( x)  x 3

f ( x)  z * 2  8
input
x2
CPU
output
Program memory
compute xx
compute x x
2
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation
temporary memory
z  2*2  4 f ( x)  x 3

f ( x)  z * 2  8
input
x2
CPU
f ( x)  8
Program memory output
compute xx
compute x x
2
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Automaton
temporary memory

Automaton
input
CPU
output

Program memory

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Automaton
temporary memory

Automaton
input

output
transition

state

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Different Kinds of Automata

Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Finite Automaton

temporary memory

input
Finite
Automaton
output

Example: Elevators, Vending Machines

(small computing power)
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Temp.
Pushdown Automaton
memory Stack Push, Pop

Pushdown input

Automaton
output

Example: Compilers for Programming Languages

(medium computing power)
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Turing Machine
Temp.
memory Random Access Memory

input
Turing
Machine
output

Examples: Any Algorithm

(highest computing power)
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Power of Automata
Simple More complex Hardest
problems problems problems

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Turing Machine is the most powerful

computational model known

Question: Are there computational

problems that a Turing Machine
cannot solve?

Answer: Yes (unsolvable problems)

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Mathematical notions and

terminology

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Set
• A set is a group of objects represented as a unit.
• Sets may contain any type of object, including
numbers, symbols, and even other sets.
• The objects in a set are called its elements or
members.

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Languages

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

• Language: a set of strings

• String: a sequence of symbols from some alphabet

• Example:
• Strings: cat, dog, house
• Language: {cat, dog, house}

Alphabet:   a, b, c,, z

Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Alphabets and Strings

• An alphabet is a set of symbols

• Example Alphabet:   a, b  
• A string is a sequence of symbols from the alphabet
• Example Strings
a
u  ab
ab
v  bbbaaa
abba
w  abba
aaabbbaaba b
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Decimal numbers alphabet   { 0,1,2, ,9}

102345 567463386

Binary numbers alphabet   { 0,1}

100010001 101101111

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

String Operations
w  a1a2  an abba
v  b1b2 bm bbbaaa

Concatenation

wv  a1a2 anb1b2 bm abbabbbaaa

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

w  a1a2 an ababaaabbb

Reverse

w  an a2a1
R
bbbaaababa

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

String Length
w  a1a2 an

• Length: w n

• Examples: abba  4
aa  2
a 1
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

String Length

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Length of Concatenation
uv  u  v

• Example:u  aab, u  3
v  abaab, v  5

uv  aababaab  8
uv  u  v  3  5  8
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Empty String
• A string with no letters is denoted:
 or 
• Observations:   0

w  w   w

abba  abba  abba  abba

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Substring
• Substring of string:
– a subsequence of consecutive
characters
• String Substring
abbab ab
abbab abba
abbab b
abbab bbab
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Prefix and Suffix

abbab
• Prefixes Suffixes
 abbab w  uv
a bbab
prefix
ab bab
suffix
abb ab
abba b
abbab 
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Another Operation
n
w  ww


w
n

• Example: abba   abbaabba

2

• Definition: w 
0
–

abba    0
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

The * Operation

 * : the set of all possible strings

from alphabet 

  a, b
*   , a, b, aa, ab, ba, bb, aaa, aab,

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Example

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Example 2

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Closure of Null

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation
The + Operation
 : the set of all possible strings from alphabet except 

  a, b
*   , a, b, aa, ab, ba, bb, aaa, aab,

   * 

  a, b, aa, ab, ba, bb, aaa, aab,

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation
Languages
• A language over alphabet 
is any subset of  *
• Examples:

  a, b
*   , a, b, aa, ab, ba, bb, aaa,
Language: 
Language: a, aa, aab
Language: { , abba, baba, aa, ab, aaaaaa}
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

More Language Examples

Alphabet   {a , b }
An infinite language
L  {a b : n  0}n n


ab
L abb L
aabb
aaaaabbbbb
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Prime numbers

Alphabet   { 0,1,2, ,9}

Language:

PRIMES  {x : x   and x is prime} *

PRIMES  {2,3,5,7,11,13,17,}

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Even and odd numbers

Alphabet   { 0,1,2, ,9}

EVEN  {x : x   and x is even} *

EVEN  { 0,2,4,6, }

ODD  {x : x   and x is odd} *

ODD  {1,3,5,7, }
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Unary Addition

Alphabet:   {1,,  }
Language:

ADDITION  {x  y  z : x  1n , y  1m , z  1k ,
nm k}

11  111  11111  ADDITION

111  111  111  ADDITION
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Squares
Alphabet:   {1, # }
Language:

SQUARES  {x # y : x  1 , y  1 , m  n } n m 2

11#1111  SQUARES
111#1111  SQUARES
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Note that:

Sets   { }  {}

Set size {}    0

Set size {}  1

String length  0
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Operations on Languages
• The usual set operations

a, ab, aaaa  bb, ab  {a, ab, bb, aaaa}

a, ab, aaaa  bb, ab  {ab}
a, ab, aaaa  bb, ab  a, aaaa
• Complement: L   * L

a, ba   , b, aa, ab, bb, aaa,

Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Reverse

• Definition: L  {w : w  L}
R R

• Examples: ab, aab, babaR  ba, baa, abab

L  {a b : n  0}
n n

L  {b a : n  0}
R n n
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Concatenation

• Definition: L1L2  xy : x  L1, y  L2 

• Example:a, ab, bab, aa

 ab, aaa, abb, abaa, bab, baaa

Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Concatenation (cont..)
• Concatenation does not always produce word in
same language.

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Another Operation
• Definition: L 
n

LL L
n

a, b  a, ba, ba, b 

3

aaa, aab, aba, abb, baa, bab, bba, bbb

0
• Special case: L  

a , bba , aaa 0  

Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

L  {a b : n  0}
n n

L  {a b a b : n, m  0}
2 n n m m

2
aabbaaabbb L

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Star-Closure (Kleene *)
• All strings that can be constructed from L

• Definition: L*  L  L  L 
0 1 2

• Example:
 , 
a, bb, 
 
a, bb*   
 aa , abb, bba , bbbb, 
• aaa, aabb, abba, abbbb,
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Positive Closure

• Definition: L  L  L 
 1 2

Same with L* but without the 

a, bb, 
  
a, bb  aa, abb, bba, bbbb, 
aaa, aabb, abba, abbbb,
 
Government College University, Faisalabad
Week 1-Introduction Advanced Theory of Computation

Palindrome

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Theorem
• S* = S**

• S* != S+

• When S* == S+ ?

Government College University, Faisalabad

Week 1-Introduction Advanced Theory of Computation

Exercise

Government College University, Faisalabad