Great Theoretical Ideas In Computer Science

John Lafferty CS 15-251 Fall 2005

Lecture 10 Sept 29 2005 Carnegie Mellon University

One Minute To Learn Programming:

Finite Automata
a
b
b
a a

b a
a
b
b
 Let me teach
you a
programming
language so
simple that
you can learn
it in less
than a
minute.

Steven Rudich: www.cs.cmu.edu/~rudi

Meet “ABA” The Automaton!
a
b
b
a a

b a
a
b b
Input String Result
aba Accept
aabb Reject
aabba Accept
 Accept
Steven Rudich: www.cs.cmu.edu/~rudi
The Simplest Interesting Machine:

Finite State Machine

OR
Finite Automaton

Steven Rudich: www.cs.cmu.edu/~rudi

 Finite Automaton

Finite set of
states Q  {qo , q1 , q2 , , qk }
A start state qo
A set of
accepting F   qi1 , qi2 , , qir 
states
A finite a b #
alphabet
x 1

State a  :Q   Q
transition qi qj
instructions  ( qi , a )  q j
Steven Rudich: www.cs.cmu.edu/~rudi
 How Machine M operates.
M “reads” one letter at a time from
the input string (going from left to
right)

M starts in state q0.

If M is in state qi reads the letter a then

If (qi. a) is undefined then CRASH.

Otherwise M moves to state (qi,a)

Steven Rudich: www.cs.cmu.edu/~rudi

Let M=(Q,,F,) be a finite automaton.

M accepts the string x if when

M reads x it ends in an accepting
state.

M rejects the string x if when

M reads x it ends in a non-
accepting state.

M crashes on x if M crashes
while reading x.
Steven Rudich: www.cs.cmu.edu/~rudi
 The set (or language) accepted by M is:

LM   x  * | M accepts x

  All length k strings over the alphabet 

k

 *
         ...
0 1 2 3

Notice that this is 

Steven Rudich: www.cs.cmu.edu/~rudi

What is the language accepted by
this machine?

a,b

L = {a,b}* = all finite strings of a’s and b’s

Steven Rudich: www.cs.cmu.edu/~rudi

What is the language accepted by
this machine?

a,b

a,b

Steven Rudich: www.cs.cmu.edu/~rudi

What is the language accepted by
this machine?

a,b

a,b

L = all even length strings of a’s and b’s

Steven Rudich: www.cs.cmu.edu/~rudi
What machine accepts this
language?

L = all strings in {a,b}* that

contain at least one a
b
a,b
a

Steven Rudich: www.cs.cmu.edu/~rudi

 What machine accepts this
language?

L = strings with an odd number of b’s

and any number of a’s

a b a

Steven Rudich: www.cs.cmu.edu/~rudi

What is the language accepted by
this machine?

a b b

L = any string ending with a b

Steven Rudich: www.cs.cmu.edu/~rudi

What is the language accepted by
this machine?

b b
a a a,b

Steven Rudich: www.cs.cmu.edu/~rudi

What is the language accepted by
this machine?

b b
a a a,b

L = any string with at least two a’s

Steven Rudich: www.cs.cmu.edu/~rudi

 What machine accepts this
language?

L = any string with an a and a b

Steven Rudich: www.cs.cmu.edu/~rudi

 What machine accepts this
language?

L = any string with an a and a b

a
a b
a,b

b a
b
Steven Rudich: www.cs.cmu.edu/~rudi
 What machine accepts this
language?
L = strings with an even number of ab pairs
a
b a b
b

b a
a
Steven Rudich: www.cs.cmu.edu/~rudi
L = all strings containing ababb as a consecutive
substring

Steven Rudich: www.cs.cmu.edu/~rudi

 L = all strings containing ababb as a consecutive
substring
b a
a,b
a b a b b

a
b a

 a ab aba abab
Invariant: I am state s exactly when s is the
longest suffix of the input (so far) that forms a
prefix of ababb. Steven Rudich: www.cs.cmu.edu/~rudi
 The “grep” Problem
Input: Problem:
• text T of length t Does the string
• string S of length n S appear inside
the text T ?
n symbols
       
Naïve method:

a1 , a2 , a3 , , at
Cost: O(nt) comparisons
Steven Rudich: www.cs.cmu.edu/~rudi
 Automata Solution

•Build a machine M that accepts any string with

S as a consecutive substring.

•Feed the text to M.

•Cost: t comparisons + time to build M.

•As luck would have it, the Knuth, Morris, Pratt

algorithm builds M quickly.

Steven Rudich: www.cs.cmu.edu/~rudi

Real-life uses of finite state machines

•grep

•coke machines

•thermostats (fridge)

•elevators

•train track switches

Steven Rudich: www.cs.cmu.edu/~rudi
Any L   is defined to be a
language.

L is just a set of strings. It is

called a language for historical
reasons.

Steven Rudich: www.cs.cmu.edu/~rudi

Let L   be a language.

L is called a regular language

if there is some finite
automaton that accepts L.

In this lecture we have seen

many regular languages.

• 
• even length strings

• strings containing ababb

Steven Rudich: www.cs.cmu.edu/~rudi
Theorem: Any finite langage is regular.
Proof: Make a machine with a
“path” for each string in the
language, sharing prefixes
Example: L = {a, bcd, ac, bb}

a b

c b c

d
Steven Rudich: www.cs.cmu.edu/~rudi
 Are all
languages
regular?

Steven Rudich: www.cs.cmu.edu/~rudi

 Consider the language

a b    , ab, aabb, aaabbb,

n n

i.e., a bunch of a’s

followed by an equal
number of b’s
No finite automaton accepts this language.

Can you prove this?

Steven Rudich: www.cs.cmu.edu/~rudi

 anbn is not regular.
No machine has
enough states to keep
track of the number
of a’s it might
encounter.

Steven Rudich: www.cs.cmu.edu/~rudi

That is a fairly weak
argument. Consider the
following example…

Steven Rudich: www.cs.cmu.edu/~rudi

L = strings where the # of
occurrences of the pattern ab is
equal to the number of occurrences
of the pattern ba

Can’t be regular. No machine has

enough states to keep track of
the number of occurrences of
ab.

Steven Rudich: www.cs.cmu.edu/~rudi

 Remember “ABA”?

a
b
b
a a

b a
a
b b

ABA accepts only the strings

with an equal number of ab’s and
ba’s! Steven Rudich: www.cs.cmu.edu/~rudi
 Let me show you a
professional strength proof
that anbn is not regular….

Steven Rudich: www.cs.cmu.edu/~rudi

 Pigeonhole principle:
Given n boxes and m > n
objects, at least one box
must contain more than
one object.

Letterbox principle: If
the average number of
letters per box is a, then
some box will have at
least a letters. (Similarly,
some box has at most a.)

Steven Rudich: www.cs.cmu.edu/~rudi

Theorem: anbn is not regular.

Steven Rudich: www.cs.cmu.edu/~rudi

 Professional Strength Proof
Theorem: anbn is not regular.
Proof: Assume that it is. Then  M with k
states that accepts it.
For each 0  i  k, let Si be the state M is
in after reading ai.
i,j  k s.t. Si = Sj, but i  j

M will do the same thing on aibi and ajbi .

But a valid M must reject ajbi and accept
aibi.

Steven Rudich: www.cs.cmu.edu/~rudi
 MORAL:

Finite automata can’t count.

Steven Rudich: www.cs.cmu.edu/~rudi

 Advertisement

You can learn much more about these

creatures in the FLAC course.

Formal Languages, Automata, and

Computation

• There is a unique smallest

automaton for any regular language

• It can be found by a fast

algorithm.
Steven Rudich: www.cs.cmu.edu/~rudi
 Cellular Automata
•Line up a bunch of identical finite automata in a
straight line.

qi qk qj

•Transitions are based on the states of the

machine’s two neighbors or an indicator that a
neighbor is missing. (There is no other input.)

Q Q Q  Q qi , q j
( qk , qi , q j )  ql qk ql

•All cells move to their next states at the same time:

synchronous transition
Steven Rudich: www.cs.cmu.edu/~rudi
 The Firing Squad Problem
•Five “soldiers” all start in the sleep state.
You change the one on the left to the wake
state.
•All five must get to the fire state at the
same time (for the first time).
sleep sleep sleep sleep sleep
wake wake wake wake wake
fire fire fire fire fire

sleep sleep sleep sleep sleep

wake wake wake wake wake
fire fire fire fire fire
Steven Rudich: www.cs.cmu.edu/~rudi
 Shorthand

Q ,  a, b, d 

Means use this transition when

your left neighbor is in any state
at all and your right neighbor is in
state a,b, or d.

Steven Rudich: www.cs.cmu.edu/~rudi

{sleep,fire,end},Q

sleep
{w},Q {w3},Q {w4},Q
{w2},Q
wake wake2 wake3 wake4
Q,Q Q,Q Q,Q
Q,Q
fire!

1 2 3 4 5
sleep sleep sleep sleep sleep
sleep sleep sleep sleep sleep
sleep sleep sleep sleep sleep
sleep sleep sleep sleep sleep
sleep sleep sleep sleep sleep
Steven Rudich: www.cs.cmu.edu/~rudi
{sleep,fire,end},Q

sleep
{w},Q {w3},Q {w4},Q
{w2},Q
wake wake2 wake3 wake4
Q,Q Q,Q Q,Q
Q,Q
fire!

1 2 3 4 5
wake sleep sleep sleep sleep

Steven Rudich: www.cs.cmu.edu/~rudi

{sleep,fire,end},Q

sleep
{w},Q {w3},Q {w4},Q
{w2},Q
wake wake2 wake3 wake4
Q,Q Q,Q Q,Q
Q,Q
fire!

1 2 3 4 5
wake sleep sleep sleep sleep
wake2 wake2 sleep sleep sleep

Steven Rudich: www.cs.cmu.edu/~rudi

{sleep,fire,end},Q

sleep
{w},Q {w3},Q {w4},Q
{w2},Q
wake wake2 wake3 wake4
Q,Q Q,Q Q,Q
Q,Q
fire!

1 2 3 4 5
wake sleep sleep sleep sleep
wake2 wake2 sleep sleep sleep
wake3 wake3 wake3 sleep sleep

Steven Rudich: www.cs.cmu.edu/~rudi

{sleep,fire,end},Q

sleep
{w},Q {w3},Q {w4},Q
{w2},Q
wake wake2 wake3 wake4
Q,Q Q,Q Q,Q
Q,Q
fire!

1 2 3 4 5
wake sleep sleep sleep sleep
wake2 wake2 sleep sleep sleep
wake3 wake3 wake3 sleep sleep
wake4 wake4 wake4 wake4 sleep

Steven Rudich: www.cs.cmu.edu/~rudi

{sleep,fire,end},Q

sleep
{w},Q {w3},Q {w4},Q
{w2},Q
wake wake2 wake3 wake4
Q,Q Q,Q Q,Q
Q,Q
fire!

1 2 3 4 5
wake sleep sleep sleep sleep
wake2 wake2 sleep sleep sleep
wake3 wake3 wake3 sleep sleep
wake4 wake4 wake4 wake4 sleep
fire fire fire fire fire
Steven Rudich: www.cs.cmu.edu/~rudi
 Question

Can you build the soldier’s finite

automaton brain before you know
how many soldiers will be in the
line?
No. Finite automata
can’t count!

Steven Rudich: www.cs.cmu.edu/~rudi

Don’t jump to conclusions!
It is possible to design a
single cellular automaton
that works for any number
of soldiers!

Steven Rudich: www.cs.cmu.edu/~rudi