CSC312

Automata Theory

Lecture # 1

Introduction
 Administrative Stuff
• Instructor: Dr. Ashfaq Ahmad
[email protected]

•Office Hrs: Mon-Thu 1:00 – 16:00 hrs

(or by appointment)
•Prerequisite: CSC102 - Discrete Structures
 Course Organization
Text Book:
i) Denial I. A. Cohen Introduction to Computer Theory,
Second Edition, John Wiley & Sons.
Reference Books:
i) J. E. Hopcroft, R. Motwani, & J. D. Ullman
Introduction to Automata Theory,
Languages, and Computation, Third Edition,
Pearson, 2008.
Instruments:
There will be 3~4 assignments, 3~4 quizzes,
Weights: Assignments 10%
Quizzes 15%
S-I 10%
S-II 15%
Final Exam 50%
 Course Objectives:
This course is designed to enable the students to
study and mathematically model various abstract
computing machines that serve as models for
computations and examine the relationship between
these automata and formal languages.
Our models will not be used to
discuss the practical engineering details of the
hardware of computers, but the
more abstract questions of the frontiers of
capability of these mechanical devices.
There are separate courses that deal with
circuits and switching theory (computer
logic) and with instruction sets Such as:
Data structures, algorithms, operating
systems, compiler design and artificial
intelligence and so forth.
All of these courses
have a theoretical component, but they
differ from our study in two basic
ways.

Courtesy Costas Busch - RPI 5

First, they deal only with computers that
already exist; our models, on the other hand
will encompass all computers that do exist,
will exist, and that can ever be dreamed of.

Second, they are interested in how best to

do things; we shall not be interested in
optimality at all, but rather we shall be
concerned with the question of (from the
perspective of what language structures the
machines we describe can and cannot accept
as input, and what possible meaning their
output may have. Courtesy Costas Busch - RPI 6
 Course Outline
Regular expressions, TG, DFA and NFAs. Core
concepts of Regular Languages and Finite
Automata; Kleene’s Theorem, Moore and Mealy
Machines, Decidability for Regular Languages;
Non-regular Languages; Transducers (automata
with output). Context-free Grammar and
Languages; Decidability for Context-free
Languages; Non-context-free Languages; Pushdown
Automata, Turing machines.
 Schedule of Lectures
Lect.# Topics/Contents
1 Introduction to Automata theory, Its background, Mathematical Preliminaries,
Sets, Functions, Graphs, Proof Techniques
2 Formal Languages, Introduction to defining languages, alphabet, language,
word, null string, length of a string, reverse of a string, Palindrome, Kleene
closure.
3 Formal definition of Regular Expressions, Defining languages with regular
expressions, Languages associated with regular expressions.
4 Equality of Regular Expressions, Introducing the language EVEN-EVEN.
5 More examples related to regular expressions.
6 Introducing Finite Automata., Defining languages using Finite Automata.
Constructing Finite Automata for different languages.
7 Recognizing the language defined by the given Finite Automata.
8 More examples related to Finite Automata.
9 Transition Graphs with examples, Generalized Transition Graphs, Non-
determinism in case of Transition Graphs.
10 Non-deterministic FA’s. Differences between FA, TG and NFA.
11 Sessional I
 Schedule of Lectures
Lect.# Topics/Contents
12 Discussion on the solution of S-I
13 Kleene’s Theorem, Algorithm for turning TGs into REs
14 Kleene’s Theorem, Algorithm for turning REs into FAs
15 Different Rules for turning Res into FAs
16 Nondeterminism, NFA, converting NFA into FA. Union of two FAs using
NFA.
17 Finite Automata with output, Moore’s machines and Mealy machines with
examples. 1’s Complement machine, Increment machine.
18 Theorems for Converting Moore machines into Mealy machines and vice
versa.
19 Transducers as models of sequential circuits.
20 Regular Languages, Closure properties (i.e. , Concatenation and Kleene
closure) of Regular Languages with examples.
21 Complements and Intersections of Regular Languages, Theorems relating to
regular languages and the related examples.
22 Non-Regular Languages, The pumping Lemma, Examples relating to
Pumping Lemma.
 Schedule of Lectures
23 Sessional-II
24 Decidability, decision procedure, Blue-paint method, Effective decision
procedure to prove whether two given RE’s or FA’s are equivalent. Myhill-
Nerode theorem, Related Examples.
25 Context-Free Grammars, CFG’s for Regular Languages with examples.
CFG’s for non-regular languages.
26 CFG’s of PALINDROME, EQUAL and EVEN-EVEN languages, Backus-
Naur Form.
27 Parse Trees, Examples relating to Parse Trees, Lukasiewicz notation, Prefix
and Postfix notations and their evaluation.
28 Ambiguous and Unambiguous CFG’s, Total language tree. Regular
Grammars, Semi-word, Word, Working String, Converting FA’s into
CFG’s.
29 Regular Grammars, Constructing Transition Graphs from Regular
Grammars. Killing null productions. Killing unit productions,
30 Chomsky Normal form with examples, Left most derivations.
Pushdown Automata, Constructing PDA’s for FA’s, Pushdown stack.
 Some basics
• Automaton = A self-operating machine or
mechanism (Dictionary definition), plural is
Automata.
• Automata = abstract computing devices
• Automata theory = the study of abstract
machines (or more appropriately, abstract
'mathematical' machines or systems), and the
computational problems that can be solved
using these machines.
• Mathematical models of computation
• Finite automata
• Push-down automata
• Turing machines
 History
• 1930s : Alan Turing defined machines more powerful
than any in existence, or even any that we could
imagine – Goal was to establish the boundary
between what was and was not computable.
• 1940s/1950s : In an attempt to model “Brain
function” researchers defined finite state
machines.
• Late 1950s : Linguist Noam Chomsky began the
study of Formal Grammars.
• 1960s : A convergence of all this into a formal
theory of computer science, with very deep
philosophical implications as well as practical
applications (compilers, web searching, hardware,
A.I., algorithm design, software engineering,…)
 Computation

CPU memory

13
temporary memory

input memory
CPU
output memory

Program memory

14
 3
Example: f ( x)  x

temporary memory

input memory
CPU
output memory
Program memory
compute xx
2
compute x x
15
 3
f ( x)  x

temporary memory
input memory
x2
CPU
output memory
Program memory
compute xx
2
compute x x
16
 3
temporary memory f ( x)  x
z  2*2  4
f ( x)  z * 2  8
input memory
x2
CPU
output memory
Program memory
compute xx
2
compute x x
17
 3
temporary memory f ( x)  x
z  2*2  4
f ( x)  z * 2  8
input memory
x2
CPU
f ( x)  8
Program memory output memory
compute xx
2
compute x x
18
 Automaton
temporary memory

Automaton
input memory
CPU
output memory

Program memory

19
 Different Kinds of Automata
Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

20
 Finite Automaton

temporary memory

input memory
Finite
Automaton
output memory

Example: Automatic Door, Vending Machines

(small computing power)
21
 Pushdown Automaton
Stack Push, Pop

input memory
Pushdown
Automaton
output memory

Example: Compilers for Programming Languages

(medium computing power)
22
 Turing Machine

Random Access Memory

input memory
Turing
Machine
output memory

Examples: Any Algorithm

(highest computing power)
23
 Power of Automata

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
24
 A simple computer
I TCH
SW

BATTERY

input: switch
output: light bulb
actions: flip switch
states: on, off
 A simple “computer”
I TCH
SW

BATTERY start off on

input: switch
bulb is on if and only if
output: light bulb
there was an odd number of
actions: f for “flip switch” flips
states: on, off
 Another “computer” 1
1 start off off
1

2 2 2 2
BATTERY
1
2
off on
1

inputs: switches 1 and 2

bulb is on if and only if
actions: 1 for “flip switch 1”
both switches were flipped
actions: 2 for “flip switch 2”
an odd number of times
states: on, off
 Mathematical Preliminaries

• Sets
• Functions
• Relations
• Graphs
• Proof Techniques

28
 SETS
A set is a collection of elements
A  {1, 2, 3}
B  {train, bus, bicycle, airplane}

We write
1 A
ship  B
29
 Set Representations

C = { a, b, c, d, e, f, g, h, i, j, k }

C = { a, b, …, k } finite set

S = { 2, 4, 6, … } infinite set

S = { j : j > 0, and j = 2k for some k>0 }

S = { j : j is nonnegative and even }

30
A = { 1, 2, 3, 4, 5 }
U
6 A
2 3 8
1
7 4 5
9
10

Universal Set: all possible elements

U = { 1 , … , 10 }
31
 Set Operations
A = { 1, 2, 3 } B = { 2, 3, 4, 5}
A B
• Union
2 4
1
3
A U B = { 1, 2, 3, 4, 5 } 5

• Intersection
U 2
A B = { 2, 3 } 3

• Difference
1
A-B={1}

B - A = { 4, 5 } Venn diagrams 5
32
• Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 } A = { 4, 5, 6, 7}

4
A
A 3 6
1
2
5 7

A=A
33
{ even integers } = { odd integers }

Integers

1 odd
even 5
2 6
0
4
3 7

34
DeMorgan’s Laws

U
AUB=A B

U
A B=AUB

35
 Empty, Null Set:
={}

SU =S
U
S = = Universal Set

S- =S

-S=

36
 Subset
A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }
A B

U
Proper Subset: A B

U
B
A

37
 Disjoint Sets
A = { 1, 2, 3 } B = { 5, 6}

U
A B=

A B

38
 Set Cardinality
• For finite sets
A = { 2, 5, 7 }

|A| = 3

(set size)

39
 Powersets
A powerset is a set of sets

S = { a, b, c }

Powerset of S = the set of all the subsets of S

2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }

Observation: | 2S | = 2|S| ( 8 = 23 )

40
 Cartesian Product
A = { 2, 4 } B = { 2, 3, 5 }

A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }

|A X B| = |A| |B|

Generalizes to more than two sets

AXBX…XZ
41
 FUNCTIONS
domain range
4 A B
f(1) = a a
1
2 b
3 c
5
f : A -> B
If A = domain
then f is a total function
otherwise f is a partial function
42
 RELATIONS
Let A & B be sets. A binary relation “R” from A to B
R = {(x1, y1), (x2, y2), (x3, y3), …}
Where xi  A and yi  B
R⊆AxB

xi R yi to denote (a, b)  R

e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1

43
 GRAPHS
A directed graph
e
b
node
a d
edge c
• Nodes (Vertices)
V = { a, b, c, d, e }
• Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
44
 Labeled Graph
2
6 e
b 2
1 3
a 6 d
5
c

45
 Walk

e
b
a d

Walk is a sequence of adjacent edges

(e, d), (d, c), (c, a)

46
 Path

e
b
a d

Path is a walk where no edge is repeated

Simple path: no node is repeated

47
 Cycle

base e
b
3
a 1 d
2
c

Cycle: a walk from a node (base) to itself

Simple cycle: only the base node is repeated

48
 Euler Tour
8 base
7 e
b 1
4 6
a 5 2 d
3
c

A cycle that contains each edge once

49
 Hamiltonian Cycle
5 base
e
b 1
4
a 2 d
3
c

A simple cycle that contains all nodes

50
 Trees
root

parent

leaf

child

Trees have no cycles

51
 root
Level 0

Level 1
leaf Height 3

Level 2

Level 3

52
Binary Trees

53