Theory of Automata

Shakir Ullah Shah

FAST National University of Computer and Emerging Sciences, Peshawar Campus

 Recommended Book(s)
Text Book
Introduction to Computer Theory, by Daniel I. A Cohen, John
Wiley and Sons, Inc., Second Edition
Reference Book(s)
John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman,
Introduction to Automata Theory, Languages and
Computation, Second Edition. Addison-Wesley, 2001.
John C. Martin. Introduction to Languages and the Theory of
Computation. McGraw-Hill, 2003.
Introduction to the Theory of Computation, Michael Sipser,
2nd edition, Thomson Course Technology, 2005.
2
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Objectives
To introduce the basic parts of formal languages
To describe the various methods to define a
language
To teach different formal models of computation
such as Finite Automata and Turing Machines
To build mathematical models and then to study
their limitations
To understand about the various types of
Languages, including Regular languages
3
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Outcomes
Understand a language and its basic parts
How to define languages
Finding the language
successful inputs of a machine
Language processing machines, including FSA ,
TG , Mealy & Moore machines
Comparison of various Languages

4
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Tentative Class Policy
Grading

Assignments 10%
Quizzes 10%
Class Participation 5%
Project 10%
Sessional Exams 25%
Final Exams. 40%

5
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 What does Theory of automata mean?

The word Theory means that this subject is a

more mathematical subject and less practical.
It is not like your other courses such as
programming. However, this subject is the
foundation for many other practical subjects.
Automata is the plural of the word Automaton
which means self-acting
In general, this subject focuses on the theoretical
aspects of computer science.
6
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Theory of Automa Applications
This subject plays a major role in:
Theory of Computation
Compiler Construction
Parsing
Formal Verification
Defining computer languages

7
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Background
In this course we will consider a mathematical
model of computing, called machines, and then
to study their limitations by analyzing the types
of inputs on which they can operate
successfully.
The collection of these successful inputs is
called the language of the machine.
These theoretical models helped/lead to the
construction of real computers
8
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Background (cont.)
Every time we introduce a new machine, we will
learn its language; and every time we develop a
new language, we will try to find a machine that
corresponds to it.

This interplay between languages and machines

will be our way of investigating problems and
their potential solutions by automatic
procedures, which we call algorithms.
9
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Background (cont.)
We will arrive at what we may believe to be the
most powerful machine possible. When we do,
we will be surprised to find tasks that even such
machine cannot perform.

Our ultimate result is that no matter what

machine we build, there will always be
questions that are simple to state and that the
machine can not answer.
10
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 What do automata mean?

It is the plural of automaton, and it means

something that works automatically

11
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Automaton
temporary memory

Automaton
input

output
transition

state
12
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Automaton
temporary memory

Automaton
input
CPU
output

Program memory

13
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Types of languages
There are two types of languages
Formal Languages are used as a basis for
defining computer languages
A predefined set of symbols and string
Formal language theory studies purely syntactical
aspects of a language (e.g., word abcd)
Informal Languages such as English has many
different versions.

14
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Basic Element of a Formal Language
Alphabets
Definition:
A finite non-empty set of symbols (letters), is
called an alphabet. It is denoted by Greek
letter sigma .
Example:
={1,2,3}
={0,1} //Binary digits
={i,j,k}
15
FAST National University of Computer and Emerging Sciences, Peshawar Campus
Basic Element of a Formal Language
Alphabets
Alphabet:
Symbol String
Alphabet Symbols Name Name
binary 01 bit Bit string
abcdefghijklmnop
qrstuvwxyz
Eng .Alph. letter word
ABCDEFGHIJKLMN
OPQRSTUVWXYZ
decimal 0123456789 digit integer

special ~!@#$%^&*()_-
+={[}]|\:;"'<,>.?/
Eng .Alph.
keyboard + decimal + keystroke typescript
special+..
ASCII .. byte String
UNICODE .. char String
16
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 String
A string over the alphabet means a string all
of whose symbols are in
Example:
If = {a,b} then
a, abab, aaabb, ababababababababab
The set of all strings of length 2 over the
alphabet {a,b} is
{aa, ab, ba, bb}
17
FAST National University of Computer and Emerging Sciences, Peshawar Campus
What is an EMPTY or NULL String

A string with no symbol is denoted by (Small Greek

letter Lambda) or (Capital Greek letter Lambda)
. It is called an empty string or null string.
We will prefer in this course. Please dont
confuse it with logical operator and.
One important thing to note is that we never allow
to be part of alphabet of a language

18
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Substring, prefix and suffix
Substring:
any consecutive sequence of symbols that occurs anywhere in a string. For
example,
ab and bc are substrings in abc while cb or ac are not.
Prefix and Suffix:
A beginning of a string (upto any symbol) is called prefix and ending is called
suffix, if w=xy with |x|,|y| >=0, then x is prefix and y is a suffix of w. Example
W=abaab
,a, aba, and abaab are some prefixes
, abaab, aab, and baab are some suffixes.
Note:
A string is prefix and suffix of itself
is a prefix and suffix of any string

19
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Word
Words are strings belonging to some language.

Example:
If = {a} then a language L can be defined as
L={an : n=1,2,3,..} or L={a,aa,aaa,.}
Here a,aa, are the words of L but not ab.

All words are strings, but not all strings are words.

20
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Ambiguity (Contd)
Example: an alphabet may contain letters
consisting of group of symbols for example
1= {A, aA, bab, d}.

Now consider an alphabet

2= {A, Aa, bab, d} and a string AababA.

21
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Ambiguity (Contd)

This string can be factored in two different ways

(Aa), (bab), (A)
(A), (abab), (A)
Which shows that the second group cannot be
identified as a string, defined over = {a, b}.
This is due to ambiguity in the defined alphabet
2

22
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Ambiguity (Contd)
Why Ambiguity comes: A computer program
first scans A as a letter belonging to 2, while for
the second letter, the computer program would
not be able to identify the symbols correctly.
Ambiguity Rule:- The Alphabets should be
defined in a way that letters consisting of more
than one symbols should not start with a letter,
already being used by some other letter.

23
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Ambiguity Examples

1= {A, aA, bab, d}

2= {A, Aa, bab, d}
1 is a valid alphabet while 2 is an in-valid alphabet.
Similarly,
1= {a, ab, ac}
2= {a, ba, ca}
In this case, 1 is a invalid alphabet while 2 is a valid
alphabet.
24
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 String Operations
Length
We define the function length of a string to be the number of
letters in the string s, denoted by |s|.
Example:
={a,b}
s=ababa
|s|=5
In any language that includes the null word , then length() = 0

For any word w in any language, if length(w) = 0 then w = .

25
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Word Length Example
Example:
= {A, aA, bab, d}
s=AaAbabAd
Factoring=(A), (aA), (bab), (A), (d)
|s|=5
One important point to note here is that aA has a
length 1 and not 2.

26
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Length of strings over n alphabets
Formula: Number of strings of length m defined
over alphabet of n letters is nm
Examples:
The language of strings of length 2, defined
over ={a,b}is L={aa, ab, ba, bb} i.e. number
of strings = 22
The language of strings of length 3, defined
over ={a,b} is L={aaa, aab, aba, baa, abb,
bab, bba, bbb} i.e. number of strings = 23
27
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 String Operations
Concatenation: wy, wK
Let w = w1 . . .wk and y = y1 . . . yk be two strings over some
alphabet . Then the concatenation of w and y (in symbols
w y, or just wy) is the string w1 . . .wky1 . . . Yk,
If w = a1. an and y= b1.bm then w.y (or wy)
= a1anb1bm
Notation: by wK we denote w concatenated with itself k
times, i.e.
= ={0,1} ; 05 = 00000
Note: For any x, x = x = x
28
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 String Operations
The reverse of a string s denoted by Rev(s) or sR, is defined as as
follows:
If s= then
sR=
otherwise
If s=s1sk then
sR=sks1
sR is obtained by writing the letters of s in reverse order.
Example 1:
If s=abc is a string defined over ={a,b,c} then Rev(s) or sR = cba

29
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 String Operations
Example:
= {A, aA, bab, d}
s=AaAbabAd
Rev(s)=dAbabaAA or
Rev(s)= dAbabAaA

Which one is correct?

30
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 About Null
The empty set is a language which has no strings.
Let L= then It is not true that is a word in the language since
this language has no words at all.

The set { } is a language which has one string, namely . So it is not

empty.
If a certain language L does not contain the word and we wish to
add it to L, we use the operation + to form L + {}. This language is
not the same as L.

However, the language L + is the same as L since no new words

have been added.
31
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Defining Languages
The rules for defining a language can be of
two kinds:

They can tell us how to test if a string of alphabet

letters is a valid word, or

They can tell us how to construct all the words in

the language by some clear procedures.

32
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Defining Languages (contd.)
The languages can be defined in different ways, such as
1. Descriptive definition,
2. Recursive definition,
3. Regular Expressions(RE)
4. Finite Automaton(FA) etc.

Descriptive Definition:
The language is defined by describing the conditions
imposed on its words.

33
FAST National University of Computer and Emerging Sciences, Peshawar Campus
Descriptive definition of language
Examples
1. The language L of strings of odd length, defined over
={a}, can be written as
L1={a, aaa, aaaaa,..}
2. The language L of strings that does not start with a,
defined over ={a,b,c}
L2 ={, b, c, ba, bb, bc, ca, cb, cc, }
3. The language L of strings of length 2, defined over
={0,1,2},
L3={00, 01, 02,10, 11,12,20,21,22}
34
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Descriptive definition of language
4. The language L of strings ending in 0, defined over
={0,1}, can be written as
L4={0,00,10,000,010,100,110,}
5. The language EQUAL, of strings with number of as equal
to number of bs, defined over ={a,b}
L5={ ,ab,aabb,abab,baba,abba,}
6. The language EVEN-EVEN, of strings with even number of
as and even number of bs, defined over ={a,b}
L6={, aa, bb, aaaa,aabb,abab, abba, baab, baba, bbaa,
bbbb,}
35
FAST National University of Computer and Emerging Sciences, Peshawar Campus
 Descriptive definition of language
7. The language {anbn}, of strings defined over
={a,b}, as {anbn: n=1,2,3,}
L7={ab, aabb, aaabbb,aaaabbbb,}

8. The language FACTORIAL, of strings defined

over ={a}, as {an! : n=1,2,3,}, can be written as
L8={a,aa,aaaaaa,}.

36
FAST National University of Computer and Emerging Sciences, Peshawar Campus