Damascus University

Formal Languages and Automata

Theory

Dr. Mohammad Ahmad

What is a utomata theory
• Automata theory is the study of a bstra ct
computationa l devices
- Note: A “device” need not even be a phys ica l
hardware !
• Abstra ct devices are (s implified) models of rea l
computations
• Computations ha ppen everywhere: On your
la ptop, on your cell phone, in nature, …
• Why do we need a bstra ct models ?
3
Alan Turing (1912-1954) &
Noam Chomsky (born December 7, 1928)
(A pioneer of automata theory)
• F ather of Modern Computer S cience
• E ng lis h mathematicia n
• Stu died a bstra ct ma chines ca lled Turing
machines even before computers existed
• Heard of the Turing test?
Noam Chomsky (born December 7, 1928) is an
American linguist, philosopher, cognitive
scientist, historian, political activist, and social
critic. Sometimes called "the father of modern
linguistics", Chomsky is also a major figure
in analytic philosophy and one of the founders of
the field of cognitive science. He holds a joint
appointment as Institute Professor Emeritus at MIT
and laureate professor at the University of Arizona
A s imple computer

BATTERY

input: s witch
output: lig ht bulb
actions: flip s witch
states: on, off
A s imple “computer”

BATTERY start off on

input: s witch
output: lig ht bulb bulb is on if a nd only if
there wa s a n odd
actions: f for “flip s witch” number of flips
states: on, off
Thes e devices ca n model ma ny thing s
• They ca n des cribe the operation of a ny “sma ll
computer”, like the control component of a n
a larm clock or a microwa ve
• They are a ls o us ed in lexica l a na lyzers to
recog nize well formed expres s ions in
progra mming la ng ua g es :
ab1 is a leg a l na me of a varia ble in C
5u= is not
Different kinds of a utomata
• This wa s only one exa mple of a computationa l
device, a nd there are others
• We will look at different devices , a nd look at the
following questions :
– What ca n a g iven type of device compute, a nd what
are its limitations ?
– Is one type of device more powerful tha n a nother?
S ome devices we will s ee
finite automata Devices with a finite amount of memory.
Used to model “small” computers.

push-down automata Devices with infinite memory that can be accessed

in a restricted way.
Used to model parsers, etc.

Turing Machines Devices with infinite memory.

Used to model any computer.

time-bounded Turing Infinite memory, but bounded running time.

Machines Used to model any computer program that runs in
a “reasonable” amount of time.
S ome hig hlig hts of the cours e
• F inite a utomata
– We will understa nd what kinds of thing s a device with
finite memory ca n do, a nd what it ca nnot do
– Introduce s imulation: the a bility of one device to
“imitate” a nother device
– Introduce nondeterminis m: the a bility of a device to
ma ke arbitrary choices
• P us h-down a utomata
– Thes e devices are relate d to gra mmars , which
des cribe the structure of progra mming (a nd natura l)
la ng ua g es
S ome hig hlig hts of the cours e
• Turing Ma chines
– This is a g enera l model of a computer, ca pturing
a nything we could ever hope to compute
S ome hig hlig hts of the cours e
• Time-bounded Turing Ma chines
– Ma ny problems are pos s ible to s olve on a computer in
principle, but ta ke too much time in pra ctice
– Tra veling s a les ma n: Given a list of cities , find the
s hortest wa y to vis it them a nd come ba ck home
Beijing

Xian
Chengdu Shanghai
Guangzhou
Hong Kong

– E a s y in principle: Try the cities in every pos s ible order

– Hard in pra ctice: F or 100 cities , this would ta ke 100+
years even on the fa stest computer!
Problems
• E xa mples of problems we will cons ider
– Given a word s, does it conta in the s ubword “fool”?
– Given a number n, is it divis ible by 7?
– Given a pa ir of words s a nd t, are they the s a me?
– Given a n expres s ion with bra ckets , e.g . (()()), does
every left bra cket match with a s ubs equent rig ht
bra cket?
• All of thes e ha ve “yes /no” a ns wers .
Alpha bets a nd string s
• A common wa y to ta lk a bout words , number,
pa irs of words , etc. is by repres enting them a s
string s
• To define string s , we start with a n a lpha bet
An a lpha bet is a finite, non-empty s et of s ymbols .

• E xa mples
S1 = {a, b, c, d, …, z}: the s et of letters in E ng lis h
S2 = {0, 1, …, 9}: the s et of (ba s e 10) dig its
S3 = {a, b, …, z, #}: the s et of letters plus the
s pecia l s ymbol #
S4 = {(, )}: the s et of open a nd clos ed bra ckets
1

String s
4

A s tring or wo rd is a finite s e que nc e of s y mbo ls

c ho s e n fro m ∑
• Empty string is  (or “eps ilon”)

• Length of a string w, denoted by “|w|”, is equa l to

the numbe r of (no n - ) c harac te rs in the s tring
– E .g ., x = 010100 |x| = 6
– x = 01  0  1  00  |x| = ?

– xy = c oncate ntation of two string s x a nd y

1

P owers of a n a lpha bet

5

Let ∑ be a n a lpha bet.

– ∑k = the s et of a ll string s of length k

– ∑* = ∑0 U ∑1 U ∑2 U …

– ∑+ = ∑1 U ∑2 U ∑3 U …
1

La ng ua g es
6

L is a s aid to be a lang uag e ov e r alphabe t ∑, only if L  ∑*

➔ this is beca us e ∑* is the s et of a ll string s (of a ll

pos s ible length including 0) over the g iven a lpha bet ∑
E xa mples :
1. Let L be the la ng ua g e of a ll string s cons isting of n 0’s
followed by n 1’s :
L = {, 01, 0011, 000111,…}
2. Let L be the la ng ua g e of a ll string s of with equa l
number of 0’s a nd 1’s :
L = {, 01, 10, 0011, 1100, 0101, 1010,
1001,…}
Definition: Ø denotes the Empty language
• Let L = {}; Is L=Ø?
1

The Members hip Problem

7

G iv e n a s tring w ∑*and a lang uag e L ov e r ∑,

de c ide whe the r or no t w L .

E xa mple:
Let w = 100011
Q) Is w  the la ng ua g e of string s with equa l
number of 0s a nd 1s ?
F inite Automata
E xa mple of a finite a utomaton

off on

• There are state s off a nd on, the a utomaton starts

in off a nd tries to rea ch the “g ood state” on
• What s equences of fs lea d to the g ood state ?
• Ans wer: {f, fff, fffff, …} = {f n: n is odd}
• This is a n exa mple of a deterministic finite
a utomaton over a lpha bet {f}
Deterministic finite a utomata
• A deterministic finite a utomaton (DFA) is a 5-tuple
(Q, S, d, q0, F) where
– Q is a finite s et of state s
– S is a n a lpha bet
– d: Q × S → Q is a tra ns ition function
– q0  Q is the initia l state
– F  Q is a s et of a ccepting state s (or fina l state s ).
• In dia gra ms , the a ccepting state s will be denoted
by double loops
E xa mple

0 1 0,1

q0 1 q1 0 q2

a lpha bet S = {0, 1} tra ns ition function d:

start state Q = {q0, q1, q2} inputs
initia l state q0 0 1
a ccepting state s F = {q0, q1} q0 q0 q1

state s
q1 q2 q1
q2 q2 q2
La ng ua g e of a DFA

The la ng ua g e of a DFA (Q, S, d, q0, F) is the s et of

a ll string s over S that, starting from q0 a nd
following the tra ns itions a s the string is rea d left
to rig ht, will rea ch s ome a ccepting state .

M: off on

• La ng ua g e of M is {f, fff, fffff, …} = {f n: n is odd}

E xa mples
0 0
1
q0 q1
1

0 1
1
q0 q1
0

0 1 0,1

q0 1 q1 0 q2

What are the la ng ua g es of thes e DFAs ?

E xa mples
• Construct a DFA that a ccepts the la ng ua g e

L = {010, 1} ( S = {0, 1} )
E xa mples
• Construct a DFA that a ccepts the la ng ua g e

L = {010, 1} ( S = {0, 1} )

• Ans wer
q0 1 q01 0 q010
0
0 1
q 0, 1
1 0, 1
q1 qdie
0, 1
E xa mples
• Construct a DFA over a lpha bet {0, 1} that a ccepts
a ll string s that end in 101
E xa mples
• Construct a DFA over a lpha bet {0, 1} that a ccepts
a ll string s that end in 101

• Hint: The DFA must “remember” the la st 3 bits of

the string it is rea ding
E xa mples
• Construct a DFA over a lpha bet {0, 1} that a ccepts
a ll string s that end in 101
• S ketch of a ns wer:
0
0 q000
0 q00 1
1
q0 q001
1
0 q01 …

…
q
1 q101
0 q10
1
q1 …

…
1
q11 1
q111 1
2
9
What does a DFA do on rea ding a n input string ?

• Input: a word w in ∑*
• Question: Is w a ccepta ble by the DFA?
• Ste ps :
– Start at the “start state” q 0
– F or every input s ymbol in the s equence w do
• Compute the next state from the current state , g iven the
current input symbol in w a nd the tra ns ition function
– If a fter a ll s ymbols in w are cons umed, the current
state is one of the a ccepting state s (F ) then ac c e pt w;
– Otherwis e, re je c t w.
3

R eg ular La ng ua g es
0

• Let L(A) be a la ng ua g e re c o g nize d by a DFA A.

– Then L(A) is ca lled a “R e g ular L ang uag e ”.

• Locate reg ular la ng ua g es in the Choms ky

Hierarchy
3

The Choms ky Hiera chy

1

• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) free Context-
Recursively-
(PDA) sensitive
enumerable
(LBA)
(TM)
3

E xa mple
2

• Build a DFA for the following la ng ua g e:

– L = {w | w is a binary string that conta ins 01 a s a substring }
• Ste ps for building a DFA to recog nize L:
– ∑ = {0,1}
– Decide on the state s : Q
– Des ig nate start state a nd fina l state (s )
– δ: Decide on the tra ns itions :
• “F ina l” state s == sa me a s “a ccepting state s”
• Other state s == sa me a s “non-a ccepting state s”
3

DFA for string s conta ining 01

3

Regular expression: (0+1)*01(0+1)*

• What makes this DFA deterministic? • Q = {q0,q1,q2}

1 0,1 • ∑ = {0,1}
0
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Accepting • Transition table
state symbols
0 1
q0 q1 q0
• What if the language allows
states

q1 q1 q2
empty strings? *q2 q2 q2
3

F inite Automaton (FA)

4

• Informa lly, a state dia gra m that comprehens ively ca ptures

a ll pos s ible state s a nd tra ns itions that a ma chine ca n ta ke
while res ponding to a strea m or s equence of input
s ymbols
• R ecog nizer for “R eg ular La ng ua g es”

• Deterministic F inite Automata (DFA)

– The ma chine ca n exist in only one state at a ny g iven time
• Non-deterministic F inite Automata (NFA)
– The ma chine ca n exist in multiple state s at the s a me time