Automata theory

What is automata theory

• Automata theory is the study of abstract

computational devices
• Abstract devices are (simplified) models of
real computations
• Computations happen everywhere: On your
laptop, on your cell phone, in nature, …
• Why do we need abstract models?
 What does Automata mean?
• It is the plural of automaton, and it means
“something that works automatically”.
• Automata theory is the study of abstract
computational devices and the computational
problems that can be solved using them.
• Abstract devices are (simplified) models of
real computations.
• Helps in design and construction of different
software's and what we can expect from our
software's.
• Automata play a major role in theory of
computation, compiler design, artificial
intelligence.
 Introduction to languages

Kinds of languages:

– Talking language
– Programming language
– Formal Languages (Syntactic languages)
 EMPTY STRING or NULL STRING

• Sometimes a string with no symbol at all is

used, denoted by (Small Greek letter Lambda)
λ or (Capital Greek letter Lambda) Λ, is called
an empty string or null string.

• The capital lambda will mostly be used to

denote the empty string, in further discussion.
 Words
• Definition:
Words are strings belonging to some language.

• Example:
If Σ= {a} then a language L can be defined as
L={an : n=1,2,3,…..} or L={a,aa,aaa,….}

Here a,aa,… are the words of L

All words are strings, but not all strings are words
 These devices can model many things

• They can describe the operation of any “small

computer”, like the control component of an
alarm clock or a microwave
• They are also used in lexical analyzers to recognize
well formed expressions in programming
languages:

ab1 is a legal name of a variable in C

5u= is not
 Some devices we will see
finite automata Devices with a finite amount of memory.
Used to model “small” computers.

push-down Devices with infinite memory that can be

automata accessed in a restricted way.
Used to model parsers, etc.

Turing Machines Devices with infinite memory.

Used to model any computer.

time-bounded Infinite memory, but bounded running time.

Turing Machines Used to model any computer program that
runs in a “reasonable” amount of time.
 Alphabets and strings

• A common way to talk about words, number, pairs

of words, etc. is by representing them as strings
• To define strings, we start with an alphabet

An alphabet is a finite set of symbols.

S1 = {a, b, c, d, …, z}: the set of letters in English
• Examples
S2 = {0, 1, …, 9}: the set of (base 10) digits
S3 = {a, b, …, z, #}: the set of letters plus the
special symbol #
S4 = {(, )}: the set of open and closed brackets
 Strings

A string over alphabet S is a finite sequence

of symbols in S.
• The empty string will be denoted by e
• Examples
abfbz is a string over S1 = {a, b, c, d, …, z}
9021 is a string over S2 = {0, 1, …, 9}
ab#bc is a string over S3 = {a, b, …, z, #}
))()(() is a string over S4 = {(, )}
 Languages

A language is a set of strings over an alphabet.

• Languages can be used to describe problems
with “yes/no” answers, for example:
L1 = The set of all strings over S1 that contain
the substring “fool”
L2 = The set of all strings over S2 that are divisible by 7
= {7, 14, 21, …}
L3 = The set of all strings of the form s#s where s is any
string over {a, b, …, z}
L4 = The set of all strings over S4 where every ( can be
matched with a subsequent )
Finite Automata
 Example of a finite automaton
f

off on

• There are states off and on, the automaton starts

in off and tries to reach the “good state” on
• What sequences of fs lead to the good state?
• Answer: {f, fff, fffff, …} = {f n: n is odd}
• This is an example of a deterministic finite
automaton over alphabet {f}
 Deterministic finite automata
• A deterministic finite automaton (DFA) is a 5-
tuple (Q, S, d, q0, F) where
– Q is a finite set of states
– S is an alphabet
– d: Q × S → Q is a transition function
– q0  Q is the initial state
– F  Q is a set of accepting states (or final states).
• In diagrams, the accepting states will be
denoted by double loops
 Example
0 1 0,1

q0 1 q1 0 q2

alphabet S = {0, 1} transition function d:

start state Q = {q0, q1, q2} inputs
initial state q0 0 1
accepting states F = {q0, q1} q0 q0 q1

states
q1 q2 q1
q2 q2 q2
 Language of a DFA

The language of a DFA (Q, S, d, q0, F) is the set of

all strings over S that, starting from q0 and
following the transitions as the string is read left
to right, will reach some accepting state.
f

M: off on

• Language of M is {f, fff, fffff, …} = {f n: n is odd}

 Examples
0 0
1
q0 q1
1

0 1
1
q0 q1
0

0 1 0,1

q0 1 q1 0 q2

What are the languages of these DFAs?

 Examples

• Construct a DFA that accepts the language

L = {010, 1} ( S = {0, 1} )
 Examples
• Construct a DFA that accepts the language
L = {010, 1} ( S = {0, 1} )

• Answer
q0 1 q01 0 q010
0
0 1
qe 0, 1
1 0, 1
q1 qdie
0, 1
 Examples
• Construct a DFA over alphabet {0, 1} that
accepts all strings that end in 101
 Examples
• Construct a DFA over alphabet {0, 1} that
accepts all strings that end in 101

• Hint: The DFA must “remember” the last 3 bits

of the string it is reading
 Examples
• Construct a DFA over alphabet {0, 1} that
accepts all strings that end in 101
0
• Sketch of answer:
0 q000
0 q00 1
1
q0 q001
1
0
q01 …

…
qe
1 q101
0 q10
1
q1 …

…
1
q11 1
q111 1
 Grammar ?

•Describes underlying rules (syntax) of

programming languages
Compilers (parsers) are based on such
descriptions
•More expressive than regular
expressions/finite automata
•Context-free grammar (CFG) or just
grammar

24
 Grammar and its Chomsky
Classification
• We’ll cover three types of structures used in modeling computation:
• Grammars
• Used to generate sentences of a language and to determine if a given
sentence is in a language
• Formal languages, generated by grammars, provide models for
programming languages (Java, C, etc) as well as natural language ---
important for constructing compilers
• Finite-state machines (FSM)
• FSM are characterized by a set of states, an input alphabet, and
transitions that assigns a next state to a pair of state and an input. We’ll
study FSM with and without output. They are used in language
recognition (equivalent to certain grammar)but also for other tasks such
as controlling vending machines
• Turing Machine – they are an abstraction of a computer; used to
compute number theoretic functions

25
 Intro to Languages

• English grammar tells us if a given combination of words is a

valid sentence.
• The syntax of a sentence concerns its form while the semantics
concerns
• its meaning.
• e.g. the mouse wrote a poem

• From a syntax point of view this is a valid sentence.

• From a semantics point of view not so fast…perhaps in Disney

land

• Natural languages (English, French, Portguese, etc) have very

complex rules of syntax and not necessarily well-defined.
26
 Formal Language

• Formal language – is specified by well-defined set of rules of

syntax

• We describe the sentences of a formal language using a

grammar.

• Two key questions:

• 1 - Is a combination of words a valid sentence in a formal
language?
• 2 – How can we generate the valid sentences of a formal
language?

• Formal languages provide models for both natural languages

and programming languages. 27
 Grammars
• A formal grammar G is any compact, precise
mathematical definition of a language L.
– As opposed to just a raw listing of all of the language’s
legal sentences, or just examples of them.
• A grammar implies an algorithm that would generate
all legal sentences of the language.
– Often, it takes the form of a set of recursive definitions.
• A popular way to specify a grammar recursively is to
specify it as a phrase-structure grammar.
 Grammars (Semi-formal)

• Example: A grammar that generates a

subset of the English language
sentence  noun _ phrase predicate

noun _ phrase  article noun

predicate  verb 29
 article  a
article  the
•
noun  boy
noun  dog

verb  runs
verb  sleeps
30
• A derivation of “the boy sleeps”:

sentence  noun _ phrase predicate

 noun _ phrase verb
 article noun verb
 the noun verb
 the boy verb
 the boy sleeps

31
• Language of the grammar:
L = { “a boy runs”,
“a boy sleeps”,
“the boy runs”,
“the boy sleeps”,
“a dog runs”,
“a dog sleeps”,
“the dog runs”,
“the dog sleeps” }

32
 Notation
• noun  boy
noun  dog

Variable Terminal
or Production
Symbols of
Non-terminal rule
the vocabulary

Symbols of
the vocabulary

33
 Basic Terminology

► A vocabulary/alphabet, V is a finite nonempty set of elements

called symbols.
• Example: V = {a, b, c, A, B, C, S}
► A word/sentence over V is a string of finite length of elements
of V.
• Example: Aba
► The empty/null string, λ is the string with no symbols.
► V* is the set of all words over V.
• Example: V* = {Aba, BBa, bAA, cab …}
► A language over V is a subset of V*.
• We can give some criteria for a word to be in a language.
 Analytical Definition of grammar
A grammar is a 4-tuple G = (V,T,P,S)
• V: set of variables or nonterminals
• T: set of terminal symbols (terminals)
• P: set of productions
– Each production: head  body, where head is a
variable, and body is a string of zero or more
terminals and variables
• S: a start symbol from V

35
Example 1:
Assignment statements
• V = { S, E }, T = { i, =, +, *, n }
• Productions:
Si=E
En
Ei
EE+E
EE*E

36
Example 3: 0n1n
• V = { S }, T = { 0, 1 }
• Productions:
Se
S  0S1

37
 Derivation
• Definition

• Let G=(V,T,S,P) be a phrase-structure grammar.

• Let w0=lz0r (the concatenation of l, z0, and r) w1=lz1r be strings over V.

• If z0  z1 is a production of G we say that w1 is directly derivable from w0

and we write wo => w1.

• If w0, w1, …., wn are strings over V such that w0 =>w1,w1=>w2,…, wn-1 => wn,
then we say that wn is derivable from w0, and write w0=>*wn.

• The sequence of steps used to obtain wn from wo is called a derivation.

L(G): Language of a grammar
• Definition: Given a grammar G, and a string w over
the alphabet T, S *G w if there is a sequence of
productions that derive w
• L(G) = { w in T* | S *G w },
the language of the grammar G

40
 Leftmost vs rightmost derivations
• Leftmost derivation: the leftmost variable is always
the one replaced when applying a production
– Example: S  i = E  i = E + E
i=n+Ei=n+n
• Rightmost derivation: rightmost variable is replaced
– Example: S  i = E  i = E + E
i=E+ni=n+n
 Sentential forms
• In a derivation, assuming it begins with S, all
intermediate strings are called sentential forms of
the grammar G
• Example: i = E and i = E + n are sentential forms of
the assignment statement grammar
• The sentential forms are called leftmost (rightmost)
sentential forms if they are a result of leftmost
(rightmost) derivations
 Parse trees
• Recall that a tree in graph theory is a set of nodes
such that
– There is a special node called the root
– Nodes can have zero or more child nodes
– Nodes without children are called leaves
– Interior nodes: nodes that are not leaves
• A parse tree for a grammar G is a tree such that the
interior nodes are non-terminals in G and children of
a non-terminal correspond to the body of a
production in G
 Yield of a parse tree
• Yield: concatenation of leaves from left to right
• If the root of the tree is the start symbol, and all
leaves are terminal symbols, then the yield is a string
in L(G)
• A derivation always corresponds to some parse tree
 Types of Grammars -
Chomsky hierarchy of languages
• Venn Diagram of Grammar Types:

Type 0 – Phrase-structure Grammars

Type 1 –
Context-Sensitive
Type 2 –
Context-Free
Type 3 –
Regular
 Classifying grammars

• Given a grammar, we need to be able to find the

smallest class in which it belongs. This can be
determined by answering three questions:
• Are the left hand sides of all of the productions
single non-terminals?
• If yes, does each of the productions create at most
one non-terminal and is it on the right?
• Yes – regular No – context-free
• If not, can any of the rules reduce the length of a
string of terminals and non-terminals?
• Yes – unrestricted No – context-sensitive
• Context-Free Languages

n n R
{a b : n  0} {ww }

Regular Languages

a *b * ( a  b) *

Fall 2006
 Definition: Context-Free Grammars

Grammar G  (V , T , S , P)

Vocabulary Terminal Start

symbols variable

Productions of the form:

A x
Non-Terminal String of variables
and terminals
 Derivation Tree of A Context-free Grammar

► Represents the language using an ordered rooted tree.

► Root represents the starting symbol.

► Internal vertices represent the nonterminal symbol that
arise in the production.
► Leaves represent the terminal symbols.

► If the production A → w arise in the derivation, where w

is a word, the vertex that represents A has as children
vertices that represent each symbol in w, in order from
left to right.
Ambiguity

Fall 2006
Costas Busch - RPI 51
Grammar for mathematical expressions

E  E  E | E  E | (E) | a

Example strings:

(a  a )  a  (a  a  (a  a ))

Denotes any number

Fall 2006
Costas Busch - RPI 52
 E  E  E | E  E | (E) | a

E  E  E  a E  a EE
E
 a  a E  a  a*a
E  E A leftmost derivation
for

a  aa
a E  E

a a Fall 2006
Costas Busch - RPI 53
 E  E  E | E  E | (E) | a

E  EE  E  EE  a EE

E
 a  aE  a  aa
Another E  E
leftmost derivation
for

a  aa E  E a

a a Fall 2006
Costas Busch - RPI 54
 E  E  E | E  E | (E) | a

Two derivation trees

for

E a  aa E

E  E E  E

a E  E E  E a

a a a a Fall 2006
Costas Busch - RPI 55
 take
a2

a  a a  2  22
E E

E  E E  E

2 E  E E  E 2

2 2 2 2 Fall 2006
Costas Busch - RPI 56
 Good Tree Bad Tree

2  22  6 2  22  8
6 Compute expression result 8
using the tree
E E
2 4 4 2
E  E E  E
2 2 2 2
2 E  E E  E 2

2 2 2 2 Fall 2006
Costas Busch - RPI 57
Ambiguous Grammar:
A context-free grammar
if there is a string
is ambiguous
which has: G
two different derivation trees w L(G )
or
two leftmost derivations

(Two different derivation trees give two

different leftmost derivations and vice-versa)

Fall 2006
Costas Busch - RPI 58
 • Context-Sensitive Languages

• The language { anbncn | n  1} is context-sensitive

but not context free.
• A grammar for this language is given by:
• S  aSBC | aBC
• CB  BC
• aB  ab
Terminal • bB  bb
and
non-terminal
• bC  bc
• cC  cc
• A derivation from this grammar is:-
• S  aSBC
•  aaBCBC (using S  aBC)
•  aabCBC (using aB  ab)
•  aabBCC (using CB  BC)
•  aabbCC (using bB  bb)
•  aabbcC (using bC  bc)
•  aabbcc (using cC  cc)
• which derives a2b2c2.
 Deterministic Finite State Automata (DFA)
……..
0 1 1 0 0

Finite
Control

• One-way, infinite tape, broken into cells

• One-way, read-only tape head.
• Finite control, i.e.,
– finite number of states, and
– transition rules between them, i.e.,
– a program, containing the position of the read head, current symbol being
scanned, and the current “state.”
• A string is placed on the tape, read head is positioned at the left end,
and the DFA will read the string one symbol at a time until all symbols
have been read. The DFA will then either accept or reject the string.
61
• The finite control can be described by a transition diagram or table:

• Example #1:

0
q0 q1 1
0

1 0 0 1 1
q0 q0 q1 q0 q0 q0

• One state is final/accepting, all others are rejecting.

• The above DFA accepts those strings that contain an even number of
0’s, including the null string, over Sigma = {0,1}
L = {all strings with zero or more 0’s}
• Note, the DFA must reject all other strings 62
• Example #2:

a a a/b/c

c c
q0 q1 q2

b b

a c c c b accepted
q0 q0 q1 q2 q2 q2

a a c rejected
q0 q0 q0 q1

• Accepts those strings that contain at least two c’s

63
 a a a/b/c
q0 c q1 c q2

b b
Inductive Proof (sketch): that the machine correctly accepts strings with at least two c’s
Proof goes over the length of the string.

Base: x a string with |x|=0. state will be q0 => rejected.

Inductive hypothesis: |x|= integer k, & string x is rejected - in state q0 (x must have zero c),
OR, rejected – in state q1 (x must have one c),
OR, accepted – in state q2 (x has already with two c’s)

Inductive steps: Each case for symbol p, for string xp (|xp| = k+1), the last symbol p = a, b or c

xa xb xc

x ends in q0 q0 =>reject q0 =>reject q1 =>reject

(still zero c => should (still zero c => should (still zero c => should
reject) reject) reject)
x ends in q1 q1 =>reject q1 =>reject q2 =>accept
(still one c => should (still one c => should (two c now=> should
reject) reject) accept)
x ends in q2 q2 =>accept q2 =>accept q2 =>accept
(two c already => (two c already => (two c already =>
should accept) should accept) should accept)
64
 Formal Definition of a DFA

• A DFA is a five-tuple:

M = (Q, Σ, δ, q0, F)

Q A finite set of states

Σ A finite input alphabet
q0 The initial/starting state, q0 is in Q
F A set of final/accepting states, which is a subset of Q
δ A transition function, which is a total function from Q x Σ to Q

δ: (Q x Σ) –> Q δ is defined for any q in Q and s in Σ, and

δ(q,s) = q’ is equal to some state q’ in Q, could be q’=q

Intuitively, δ(q,s) is the state entered by M after reading symbol s while in

state q.
65
• Revisit example #1:
1
Q = {q0, q1}
0
Σ = {0, 1}
q0 q1 1
Start state is q0
F = {q0} 0

δ:
0 1
q0 q1 q0

q1 q0 q1

66
• Revisit example #2:

a a a/b/c
Q = {q0, q1, q2}
Σ = {a, b, c} c c
q0 q1 q2
Start state is q0
F = {q2} b b

δ: a b c
q0 q0 q0 q1

q1 q1 q1 q2

q2 q2 q2 q2

• Since δ is a function, at each step M has exactly one option.

• It follows that for a given string, there is exactly one computation.

67
 Nondeterministic Finite State
Automata (NFA)

• An NFA is a five-tuple:

M = (Q, Σ, δ, q0, F)

Q A finite set of states

Σ A finite input alphabet
q0 The initial/starting state, q0 is in Q
F A set of final/accepting states, which is a subset of Q
δ A transition function, which is a total function from Q x Σ to 2Q

δ: (Q x Σ) –> 2Q :2Q is the power set of Q, the set of all subsets of Q

δ(q,s) :The set of all states p such that there is a transition
labeled s from q to p

δ(q,s) is a function from Q x S to 2Q (but not only to Q)

68
• Example #1: one or more 0’s followed by one or more 1’s

0 1 0/1
Q = {q0, q1, q2}
Σ = {0, 1} 0 1
q0 q1 q2
Start state is q0
F = {q2}

δ: 0 1
q0 {q0, q1} {}

q1 {} {q1, q2}

q2 {q2} {q2}

69
• Example #2: pair of 0’s or pair of 1’s as substring

0/1 0/1
Q = {q0, q1, q2 , q3 , q4}
Σ = {0, 1} 0 0
q0 q3 q4
Start state is q0
F = {q2, q4} 1 0/1

1
δ: 0 1 q1 q2
q0 {q0, q3} {q0, q1}

q1 {} {q2}

{q2} {q2}
q2

q3 {q4} {}

q4 {q4} {q4}
70
• Question: Why non-determinism is useful?
– Non-determinism = Backtracking
– Compressed information
– Non-determinism hides backtracking
– Programming languages, e.g., Prolog, hides backtracking => Easy to
program at a higher level: what we want to do, rather than how to do it
– Useful in algorithm complexity study

– Is NDA more “powerful” than DFA, i.e., accepts type of languages that
any DFA cannot?

71
 Equivalence of DFAs and NFAs

• Do DFAs and NFAs accept the same class of languages?

– Is there a language L that is accepted by a DFA, but not by any NFA?
– Is there a language L that is accepted by an NFA, but not by any DFA?

• Observation: Every DFA is an NFA, DFA is only restricted NFA.

• Therefore, if L is a regular language then there exists an NFA M such

that L = L(M).

• It follows that NFAs accept all regular languages.

• But do NFAs accept more?

72
• Consider the following DFA: 2 or more c’s

a a a/b/c
Q = {q0, q1, q2}
Σ = {a, b, c} c c
q0 q1 q2
Start state is q0
F = {q2} b b

δ: a b c
q0 q0 q0 q1

q1 q1 q1 q2

q2 q2 q2 q2

73
• An Equivalent NFA:

a a a/b/c
Q = {q0, q1, q2}
Σ = {a, b, c} c c
q0 q1 q2
Start state is q0
F = {q2} b b

δ: a b c
q0 {q0} {q0} {q1}

q1 {q1} {q1} {q2}

q2 {q2} {q2} {q2}

74
Real-life Uses of DFAs
Grep

Coke Machines

Thermostats (fridge)

Elevators

Train Track Switches

Lexical Analyzers for Parsers

Presentation Outline

• Introduction
• Chomsky normal form
• Preliminary simplifications
• Final steps
• Greibach Normal Form
• Algorithm (Example)
• Summary

May 27, 2009 77

Introduction

Grammar: G = (V, T, P, S)

Terminals T = { a, b }

Variables V = A, B, C

Start Symbol S

Production P=S→A

May 27, 2009 78

Introduction

Grammar example

S → aBSc
S → abc
L = { anbncn | n ≥ 1 }
Ba → aB
Bb → bb

S aBSc aBabcc aaBbcc aabbcc

May 27, 2009 79

Introduction

Context free grammar

The head of any production contains only one

non-terminal symbol

S→P
P → aPb L = { anbn | n ≥ 0 }
P→ε

May 27, 2009 80

Chomsky Normal Form

A context free grammar is said to be in Chomsky

Normal Form if all productions are in the following
form:
A → BC
A→α

• A, B and C are non terminal symbols

• α is a terminal symbol

May 27, 2009 81

Preliminary Simplifications

There are three preliminary simplifications

1 Eliminate Useless Symbols

2 Eliminate ε productions

3 Eliminate unit productions

May 27, 2009 82

Preliminary Simplifications

Eliminate Useless Symbols

We need to determine if the symbol is useful by
identifying if a symbol is generating and is reachable

• X is generating if X 
* ω for some terminal string ω.
• X is reachable if there is a derivation X 
* αXβ
for some α and β

May 27, 2009 83

Preliminary Simplifications

Example: Removing non-generating symbols

S → AB | a
A→b Initial CFL grammar

S → AB | a
Identify generating symbols
A→b

S→a
Remove non-generating
A→b

May 27, 2009 84

Preliminary Simplifications

Example: Removing non-reachable symbols

S→a
Identify reachable symbols
A→b

S→a Eliminate non-reachable

May 27, 2009 85

Preliminary Simplifications

The order is important.

Looking first for non-reachable symbols and then

for non-generating symbols can still leave some
useless symbols.

S → AB | a S→a
A→b A→b

May 27, 2009 86

Preliminary Simplifications

Finding generating symbols

If there is a production A → α, and every symbol

of α is already known to be generating. Then A is
generating

We cannot use S → AB because

S → AB | a B has not been established to
A→b be generating

May 27, 2009 87

Preliminary Simplifications

Finding reachable symbols

S is surely reachable. All symbols in the body of a

production with S in the head are reachable.

S → AB | a In this example the symbols

A→b {S, A, B, a, b} are reachable.

May 27, 2009 88

Preliminary Simplifications

Eliminate ε Productions

• In a grammar ε productions are convenient but

not essential
• If L has a CFG, then L – {ε} has a CFG

A 
* ε

Nullable variable

May 27, 2009 89

Preliminary Simplifications

If A is a nullable variable

• Whenever A appears on the body of a production

A might or might not derive ε

S → ASA | aB
A→B|S Nullable: {A, B}
B→b|ε

May 27, 2009 90

Preliminary Simplifications

Eliminate ε Productions

• Create two version of the production, one with

the nullable variable and one without it
• Eliminate productions with ε bodies

S → ASA | aB S → ASA | aB | AS | SA | S | a
A→B|S A→B|S
B→b|ε B→b

May 27, 2009 91

Preliminary Simplifications

Eliminate ε Productions

• Create two version of the production, one with

the nullable variable and one without it
• Eliminate productions with ε bodies

S → ASA | aB S → ASA | aB | AS | SA | S | a
A→B|S A→B|S
B→b|ε B→b

May 27, 2009 92

Preliminary Simplifications

Eliminate ε Productions

• Create two version of the production, one with

the nullable variable and one without it
• Eliminate productions with ε bodies

S → ASA | aB S → ASA | aB | AS | SA | S | a
A→B|S A→B|S
B→b|ε B→b

May 27, 2009 93

Preliminary Simplifications

Eliminate unit productions

A unit production is one of the form A → B where both

A and B are variables

Identify unit pairs

A 
* B

A → B, B → ω, then A → ω

May 27, 2009 94

Preliminary Simplifications

Example:
T = {*, +, (, ), a, b, 0, 1} Pairs Productions
( E, E ) E→E+T
I → a | b | Ia | Ib | I0 | I1 ( E, T ) E→T*F
F → I | (E) ( E, F ) E → (E)
T→F|T*F ( E, I ) E → a | b | Ia | Ib | I0 | I1
E→T|E+T ( T, T ) T→T*F
( T, F ) T → (E)
( T, I ) T → a | b | Ia |Ib | I0 | I1
Basis: (A, A) is a unit pair ( F, F ) F → (E)
of any variable A, if ( F, I ) F → a | b | Ia | Ib | I0 | I1
A  * A by 0 steps. ( I, I ) I → a | b | Ia | Ib | I0 | I1

May 27, 2009 95

Preliminary Simplifications

Example:
Pairs Productions
… …
( T, T ) T→T*F
( T, F ) T → (E)
( T, I ) T → a | b | Ia |Ib | I0 | I1
… …

I → a | b | Ia | Ib | I0 | I1
E → E + T | T * F | (E ) | a | b | la | lb | l0 | l1
T → T * F | (E) | a | b | Ia | Ib | I0 | I1
F → (E) | a | b | Ia | Ib | I0 | I1

May 27, 2009 96

Final Simplification

Chomsky Normal Form (CNF)

Starting with a CFL grammar with the preliminary

simplifications performed

1. Arrange that all bodies of length 2 or more to

consists only of variables.
2. Break bodies of length 3 or more into a cascade of
productions, each with a body consisting of two
variables.

May 27, 2009 97

Final Simplification

Step 1: For every terminal α that appears in a body

of length 2 or more create a new variable that has
only one production.
E → E + T | T * F | (E ) | a | b | la | lb | l0 | l1
T → T * F | (E) | a | b | Ia | Ib | I0 | I1
F → (E) | a | b | Ia | Ib | I0 | I1
I → a | b | Ia | Ib | I0 | I1
E → EPT | TMF | LER | a | b | lA | lB | lZ | lO
T → TMF | LER | a | b | IA | IB | IZ | IO
F → LER | a | b | IA | IB | IZ | IO
I → a | b | IA | IB | IZ | IO
A→a B→b Z→0 O→1
P→+ M→* L→( R→)

May 27, 2009 98

Final Simplification

Step 2: Break bodies of length 3 or more adding

more variables

E → EPT | TMF | LER | a | b | lA | lB | lZ | lO

T → TMF | LER | a | b | IA | IB | IZ | IO
C1 → PT
F → LER | a | b | IA | IB | IZ | IO
C2 → MF
I → a | b | IA | IB | IZ | IO
C3 → ER
A→a B→b Z→0 O→1
P→+ M→* L→( R→)

May 27, 2009 99

Greibach Normal Form

A context free grammar is said to be in Greibach

Normal Form if all productions are in the following
form:

A → αX

• A is a non terminal symbols

• α is a terminal symbol
• X is a sequence of non terminal symbols.
It may be empty.

May 27, 2009 100

Greibach Normal Form

Example:

S → XA | BB S = A1 A 1 → A2 A 3 | A 4 A 4
B → b | SB X = A2 A4 → b | A1A4
X→b A = A3 A2 → b
A→a B = A4 A3 → a

CNF New Labels Updated CNF

May 27, 2009 101

Greibach Normal Form

Example:

A 1 → A2 A 3 | A 4 A 4 First Step Ai → AjXk j > i

A 4 → b | A1 A 4
A2 → b Xk is a string of zero
A3 → a or more variables

A 4 → A1 A 4

May 27, 2009 102

Greibach Normal Form

Example:

First Step Ai → A j Xk j > i

A 4 → A1 A 4 A 1 → A2 A 3 | A 4 A 4
A4 → A2A3A4 | A4A4A4 | b A4 → b | A1A4
A2 → b
A4 → bA3A4 | A4A4A4 | b
A3 → a

May 27, 2009 103

Greibach Normal Form

Example:

A 1 → A2 A 3 | A 4 A 4 Second Step
A4 → bA3A4 | A4A4A4 | b
Eliminate Left
A2 → b
Recursions
A3 → a

A4 → A4A4A4

May 27, 2009 104

Greibach Normal Form

Example:
Second Step
Eliminate Left
Recursions

A4 → bA3A4 | b | bA3A4Z | bZ A1 → A2A3 | A4A4

Z → A4A4 | A4A4Z A4 → bA3A4 | A4A4A4 | b
A2 → b
A3 → a

May 27, 2009 105

Greibach Normal Form

Example:

A1 → A2 A 3 | A 4 A 4
A4 → bA3A4 | b | bA3A4Z | bZ A → αX
Z → A4A4 | A4A4 Z
A2 → b GNF
A3 → a

May 27, 2009 106

Greibach Normal Form

Example:
A1 → A2A3 | A4A4
A4 → bA3A4 | b | bA3A4Z | bZ
Z → A4A4 | A4A4 Z
A2 → b
A3 → a

A1 → bA3 | bA3A4A4 | bA4 | bA3A4ZA4 | bZA4

Z → bA3A4A4 | bA4 | bA3A4ZA4 | bZA4 | bA3A4A4 | bA4 | bA3A4ZA4 | bZA4

May 27, 2009 107

Greibach Normal Form

Example:

A1 → bA3 | bA3A4A4 | bA4 | bA3A4ZA4 | bZA4

A4 → bA3A4 | b | bA3A4Z | bZ
Z → bA3A4A4 | bA4 | bA3A4ZA4 | bZA4 | bA3A4A4 | bA4 | bA3A4ZA4 | bZA4
A2 → b
A3 → a

Grammar in Greibach Normal Form

May 27, 2009 108

Regular Expressions
 Definition of a Regular Expression
• R is a regular expression if it is:
1. a for some a in the alphabet , standing for the language {a}
2. ε, standing for the language {ε}
3. Ø, standing for the empty language
4. R1+R2 where R1 and R2 are regular expressions, and + signifies
union (sometimes | is used)
5. R1R2 where R1 and R2 are regular expressions and this signifies
concatenation
6. R* where R is a regular expression and signifies closure
7. (R) where R is a regular expression, then a parenthesized R is
also a regular expression

This definition may seem circular, but 1-3 form the basis
Precedence: Parentheses have the highest precedence, followed by *(iteration),
concatenation, and then union(ICU)
 RE Examples
• L(001) = {001}
• L(0+10*) = { 0, 1, 10, 100, 1000, 10000, … }
• L(0*10*) = {1, 01, 10, 010, 0010, …} i.e. {w | w has exactly a single 1}
• L()* = {w | w is a string of even length}
• L((0(0+1))*) = { ε, 00, 01, 0000, 0001, 0100, 0101, …}
• L((0+ε)(1+ ε)) = {ε, 0, 1, 01}
• L(1Ø) = Ø ; concatenating the empty set to any set yields the empty set.
• Rε = R
• R+Ø = R

• Note that R+ε may or may not equal R (we are adding ε to the language)
• Note that RØ will only equal R if R itself is the empty set.
 Regular Expressions
• Regular expressions
• describe regular languages

(a  b  c) *
• Example:
a, bc*   , a, bc, aa, abc, bca,...
• describes the language

Fall 2006 Costas Busch - RPI 112

 Languages of Regular Expressions
L r  : language of regular expression r

• Example

L(•a  b  c) *   , a, bc, aa, abc, bca,...

Fall 2006 Costas Busch - RPI 113

 Equivalence of FA and RE
• Finite Automata and Regular Expressions are
equivalent. To show this:
– Show we can express a DFA as an equivalent RE
– Show we can express a RE as an ε-NFA. Since the
ε-NFA can be converted to a DFA and the DFA to
an NFA, then RE will be equivalent to all the
automata we have described.
 DFARE Example
• Convert the following
to a RE 0 0,1

1 1
Start 3 1 2

• First convert the edges 0

to RE’s: 0 0+1

1 1
Start 3 1 2

0
 Converting a RE to an Automata
• We have shown we can convert an automata to a RE.
To show equivalence we must also go the other
direction, convert a RE to an automaton.
• We can do this easiest by converting a RE to an ε-NFA
– Inductive construction
– Start with a simple basis, use that to build more complex
parts of the NFA
 RE to ε-NFA
• Basis:

a
R=a

R=ε ε

R=Ø

Next slide: More complex RE’s

 ε S ε
R=S+T

ε ε
T

ε
R=ST
S T

ε ε
R=S*
S

ε
 RE to ε-NFA Example
• Convert R= (ab+a)* to an NFA
– We proceed in stages, starting from simple
elements and working our way up

a
a

b
b

a ε b
ab
 RE to ε-NFA Example (2)

ab+a
a ε b
ε ε

a
ε ε

(ab+a)*
a ε b
ε ε
ε ε

a
ε ε

ε
Pushdown Automata
 Formal Definition of a PDA

• A pushdown automaton (PDA) is a seven-tuple:

M = (Q, Σ, Г, δ, q0, z0, F)

Q A finite set of states

Σ A finite input alphabet
Г A finite stack alphabet
q0 The initial/starting state, q0 is in Q
z0 A starting stack symbol, is in Г // need not always remain at the bottom of stack
F A set of final/accepting states, which is a subset of Q
δ A transition function, where

δ: Q x (Σ U {ε}) x Г –> finite subsets of Q x Г*

122
 Pushdown Automaton
• A pushdown automaton (PDA) is an abstract model machine
similar to the FSA

• It has a finite set of states. However, in addition, it has

a pushdown stack. Moves of the PDA are as follows:
• 1. An input symbol is read and the top symbol on the stack
is read.
• 2. Based on both inputs, the machine enters a new state and
writes zero or more symbols onto the pushdown stack.
• 3. Acceptance of a string occurs if the stack is ever
empty. (Alternatively, acceptance can be if the PDA is in a
final state. Both models can be shown to be equivalent.)

123
 Power of PDAs

• PDAs are more powerful than FSAs.

• anbn, which cannot be recognized by an FSA, can
easily be recognized by the PDA.
• Stack all a symbols and, for each b, pop an a off
the stack.
• If the end of input is reached at the same time
that the stack becomes empty, the string is
accepted.

• It is less clear that the languages accepted by

• PDAs are equivalent to the context-free languages.

124
 NDPDAs are different from DPDAs

• What is the relationship between deterministic

• PDAs and nondeterministic PDAs? They are different.

• Consider the set of palindromes, strings reading the same

forward and backward, generated by the grammar
• S  0S0 | 1S1 | 2
• We can recognize such strings by a deterministic PDA:
– 1. Stack all 0s and 1s as read.
– 2. Enter a new state upon reading a 2.
– 3. Compare each new input to the top of stack, and pop
stack.
• However, consider the following set of palindromes:
• S  0S0 | 1S1 | 0 | 1
• In this case, we never know where the middle of the string is.
To recognize these palindromes, the automaton must guess where
the middle of the string is (i.e., is nondeterministic).

125
• The PDA can be represented by
M = (Q, Σ, Γ, δ, s, F)
where Σ is the alphabet of input symbols and Γ
is the alphabet of stack symbols.
• The set of all strings accepted by a PDA M is
denoted by L(M). We also say that the
language L(M) is accepted by M.
• The transition diagram of a PDA is an alternative way
to represent the PDA.
• For M = (Q, Σ, Γ, δ, s, F), the transition diagram of M
is an edge-labeled digraph G=(V, E) satisfying the
following:

V = Q (s = ,f= for f F) 

E={q a, v/u p | (p,u) δ(q, a, v) }.

Example 1. Construct PDA to accept
n n
L= {0 1 | n > 0}

Solution 1.

1, 0/ε

0, ε/0 1, 0/ε
Solution 2.

Consider a CFG
G = ({S}, {0,1}, {S → ε | 0S1}, S).

ε, S/1
ε, ε/S
ε, ε/S
ε, ε/0
ε, S/ε

0, 0/ε
1, 1/ε
• TMs model the computing capability of a general purpose computer, which
informally can be described as:
– Effective procedure
• Finitely describable
• Well defined, discrete, “mechanical” steps
• Always terminates
– Computable function
• A function computable by an effective procedure

• TMs formalize the above notion.

• Church-Turing Thesis: There is an effective procedure for solving a problem if

and only if there is a TM that halts for all inputs and solves the problem.
– There are many other computing models, but all are equivalent to or subsumed by
TMs. There is no more powerful machine (Technically cannot be proved).

• DFAs and PDAs do not model all effective procedures or computable

functions, but only a subset.

130
 Deterministic Turing Machine (DTM)
…….. B B 0 1 1 0 0 B B ……..

Finite
Control

• Two-way, infinite tape, broken into cells, each containing one symbol.
• Two-way, read/write tape head.
• An input string is placed on the tape, padded to the left and right infinitely with
blanks, read/write head is positioned at the left end of input string.
• Finite control, i.e., a program, containing the position of the read head, current
symbol being scanned, and the current state.
• In one move, depending on the current state and the current symbol being
scanned, the TM 1) changes state, 2) prints a symbol over the cell being
scanned, and 3) moves its’ tape head one cell left or right.
• Many modifications possible, but Church-Turing declares equivalence of all.
131
 Formal Definition of a DTM

• A DTM is a seven-tuple:

M = (Q, Σ, Γ, δ, q0, B, F)

Q A finite set of states

Σ A finite input alphabet, which is a subset of Γ– {B}
Γ A finite tape alphabet, which is a strict superset of Σ
B A distinguished blank symbol, which is in Γ
q0 The initial/starting state, q0 is in Q
F A set of final/accepting states, which is a subset of Q
δ A next-move function, which is a mapping (i.e., may be undefined) from
Q x Γ –> Q x Γ x {L,R}

Intuitively, δ(q,s) specifies the next state, symbol to be written, and the direction of
tape head movement by M after reading symbol s while in state q.

132
• Example #1: {w | w is in {0,1}* and w ends with a 0}

0
00
10
10110
Not ε

Q = {q0, q1, q2}

Γ = {0, 1, B}
Σ = {0, 1}
F = {q2}
δ:

0 1 B
->q0 (q0, 0, R) (q0, 1, R) (q1, B, L)
q1 (q2, 0, R) - -
q2* - - -

– q0 is the start state and the “scan right” state, until hits B
– q1 is the verify 0 state
– q2 is the final state
133
• Example #2: {0n1n | n ≥ 1}

0 1 X Y B
->q0 (q1, X, R) - - (q3, Y, R)0’s finished -
q1 (q1, 0, R)ignore1 (q2, Y, L) - (q1, Y, R) ignore2 - (more 0’s)
q2 (q2, 0, L) ignore2 - (q0, X, R) (q2, Y, L) ignore1 -
q3 - - (more 1’s) - (q3, Y, R) ignore (q4, B, R)
q4* - - - - -

• Sample Computation: (on 0011), presume state q looks rightward

q00011BB.. |— Xq1011
|— X0q111
|— Xq20Y1
|— q2X0Y1
|— Xq00Y1
|— XXq1Y1
|— XXYq11
|— XXq2YY
|— Xq2XYY
|— XXq0YY
|— XXYq3Y B…
|— XXYYq3 BB…
|— XXYYBq4
134
• Same Example #2: {0n1n | n ≥ 1}

0 1 X Y B
q0 (q1, X, R) - - (q3, Y, R) -
q1 (q1, 0, R) (q2, Y, L) - (q1, Y, R) -
q2 (q2, 0, L) - (q0, X, R) (q2, Y, L) -
q3 - - - (q3, Y, R) (q4, B, R)
q4 - - - - -

Logic: cross 0’s with X’s, scan right to look for corresponding 1, on finding it cross it with Y, and scan
left to find next leftmost 0, keep iterating until no more 0’s, then scan right looking for B.
– The TM matches up 0’s and 1’s
– q1 is the “scan right” state, looking for 1
– q2 is the “scan left” state, looking for X
– q3 is “scan right”, looking for B
– q4 is the final state

Can you extend the machine to include n=0?

How does the input-tape look like for string epsilon?
• Other Examples:
000111 00
11 001
011

135
 Formal Definitions for DTMs
• Let M = (Q, Σ, Г, δ, q0, B, F) be a TM.

• Definition: An instantaneous description (ID) is a triple α1qα2, where:

– q, the current state, is in Q

– α1α2, is in Г*, and is the current tape contents up to the rightmost non-blank symbol, or the
symbol to the left of the tape head, whichever is rightmost
– The tape head is currently scanning the first symbol of α2
– At the start of a computation α1= ε
– If α2= ε then a blank is being scanned

• Example: (for TM #1)

q00011 Xq1011 X0q111 Xq20Y1 q2X0Y1

Xq00Y1 XXq1Y1 XXYq11 XXq2YY Xq2XYY

XXq0YY XXYq3Y XXYYq3 XXYYBq4

136
• Suppose the following is the current ID of a DTM

x1x2…xi-1qxixi+1…xn

Case 1) δ(q, xi) = (p, y, L)

(a) if i = 1 then qx1x2…xi-1xixi+1…xn |— pByx2…xi-1xixi+1…xn

(b) else x1x2…xi-1qxixi+1…xn |— x1x2…xi-2pxi-1yxi+1…xn

– If any suffix of xi-1yxi+1…xn is blank then it is deleted.

Case 2) δ(q, xi) = (p, y, R)

x1x2…xi-1qxixi+1…xn |— x1x2…xi-1ypxi+1…xn

– If i>n then the ID increases in length by 1 symbol

x1x2…xnq |— x1x2…xnyp

137
L is Recursively enumerable:
TM exist: M0, M1, …
They accept string in L, and do not accept any string outside L

L is Recursive:
at least one TM halts on L and on ∑*-L, others may or may not

L is Recursively enumerable but not Recursive:

TM exist: M0, M1, …
but none halts on all x in ∑*-L
M0 goes on infinite loop on a string p in ∑*-L, while M1 on q in ∑*-L
However, each correct TM accepts each string in L, and none in ∑*-L

L is not R.E:
no TM exists

138
 Modifications of the Basic TM Model

• Other (Extended) TM Models:

– One-way infinite tapes
– Multiple tapes and tape heads
– Non-Deterministic TMs
– Multi-Dimensional TMs (n-dimensional tape)
– Multi-Heads
– Multiple tracks

All of these extensions are equivalent to the basic DTM model

139
 The Halting Problem - Background
• Definition: A decision problem is a problem having a yes/no answer (that one
presumably wants to solve with a computer). Typically, there is a list of parameters on
which the problem is based.
– Given a list of numbers, is that list sorted?
– Given a number x, is x even?
– Given a C program, does that C program contain any syntax errors?
– Given a TM (or C program), does that TM contain an infinite loop?

From a practical perspective, many decision problems do not seem all that interesting.
However, from a theoretical perspective they are for the following two reasons:
– Decision problems are more convenient/easier to work with when proving complexity results.
– Non-decision counter-parts can always be created & are typically at least as difficult to solve.

• Notes:
– The following terms and phrases are analogous:

Algorithm - A halting TM program

Decision Problem - A language(will show shortly)
(un)Decidable - (non)Recursive
140
A language is called Turing-recognizable or recursively enumerable (r.e.) if
some TM recognizes it.

A language is called decidable or recursive if some TM decides it.

r.e. recursive
languages languages
 THE HALTING PROBLEM
HALTTM = { (M,w) | M is a TM that halts on string w }

Theorem: HALTTM is undecidable

Proof: Assume, for a contradiction, that TM H decides HALTTM

We use H to construct a TM D that decides ATM

On input (M,w), D runs H on (M,w)

If H rejects then reject

If H accepts, run M on w until it halts:

Accept if M accepts and

Reject if M rejects
 References

• http://www.cse.cuhk.edu.hk/~andrejb/csc3130
• cs.www.duke.edu › courses › fall07 › cps102 › lectures › lecture02
• ie.technion.ac.il › courses › verification › C4.1_NFA+Buchi.ppt
• curry.ateneo.net › ~jpv › cs130-L6-Grammars
• montefiore.www.ulg.ac.be › cours › psfiles › calc-chap3
• cit.courses.cornell.edu › PPT › INFO-2950-Languages-and-Grammars
• cs.www.rpi.edu › ~moorthy › Courses › modcomp › slides › PDA
• cs.fit.edu › ~dmitra › FormaLang › Lectures › PushdownAutomata
• cs.www.cmu.edu › ~emc › flac09 › lectures
• cse.www.iitd.ernet.in › ~naveen › courses › COL352 › slides
• cs.www.rpi.edu › ~moorthy › Courses › modcomp › slides › NFA
• cs.fit.edu › ~dmitra › FormaLang › Lectures › FiniteAutomata