Great Theoretical Ideas In Computer Science

John Lafferty CS 15-251 Fall 2005

Lecture 10 Sept 29 2005 Carnegie Mellon University
b
b
a
b
a
a
a
b
a
b
One Minute To Learn Programming:
Finite Automata
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Let me teach
you a
programming
language so
simple that
you can learn
it in less
than a
minute.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Meet ABA The Automaton!
b
b
a
b
a
a
a
b
a
b
Input String Result
aba Accept
aabb Reject
aabba Accept
c
Accept
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
The Simplest Interesting Machine:

Finite State Machine
OR
Finite Automaton
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Finite set of
states
A start state
A set of
accepting
states
A finite
alphabet
a b #
x 1
State
transition
instructions
1 2
{ , , , , }
o k
Q q q q q =
Finite Automaton
o
q
{ }
1 2
, , ,
r
i i i
F q q q =

:
( , )
i j
Q Q
q a q
c E
c =
i
q
j
q
a
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
How Machine M operates.

M reads one letter at a time from
the input string (going from left to
right)

M starts in state q
0
.

If M is in state q
i
reads the letter a then

If o(q
i
. a) is undefined then CRASH.

Otherwise M moves to state o(q
i
,a)

Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
M accepts the string x if when
M reads x it ends in an accepting
state.

M rejects the string x if when
M reads x it ends in a non-
accepting state.

M crashes on x if M crashes
while reading x.
Let M=(Q,E,F,o) be a finite automaton.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
The set (or language) accepted by M is:
{ }
*
M
k
* 0 1 2 3
L x | M accepts x
All length k strings over the alphabet
...
= e


Notice that this is {c}
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What is the language accepted by
this machine?
L = {a,b}* = all finite strings of as and bs
a,b
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What is the language accepted by
this machine?
a,b
a,b
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What is the language accepted by
this machine?
L = all even length strings of as and bs
a,b
a,b
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What machine accepts this
language?
L = all strings in {a,b}* that
contain at least one a
a
a,b
b
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What machine accepts this
language?
L = strings with an odd number of bs
and any number of as
b
a
a
b
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What is the language accepted by
this machine?
L = any string ending with a b
b
b
a
a
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What is the language accepted by
this machine?
b b
a,b
a a
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What is the language accepted by
this machine?
L = any string with at least two as
b b
a,b
a a
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What machine accepts this
language?
L = any string with an a and a b
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What machine accepts this
language?
L = any string with an a and a b
b
b
a,b
a
a
b
a
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
What machine accepts this
language?
L = strings with an even number of ab pairs
a
b
b
a
a
b
a
b
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
L = all strings containing ababb as a
consecutive substring
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
L = all strings containing ababb as a
consecutive substring
b
a,b
a
a
b
b
b b
a
a
a
a
ab
aba
abab

Invariant: I am state s exactly when s is the
longest suffix of the input (so far) that forms a
prefix of ababb.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Problem:
Does the string
S appear inside
the text T ?
The grep Problem
Input:
text T of length t
string S of length n
Cost: O(nt) comparisons
1 2 3
, , , ,
t
a a a a
symbols



n
Nave method:
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Automata Solution
Build a machine M that accepts any string with
S as a consecutive substring.

Feed the text to M.

Cost: t comparisons + time to build M.

As luck would have it, the Knuth, Morris, Pratt
algorithm builds M quickly.

Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Real-life uses of finite state machines
grep

coke machines

thermostats (fridge)

elevators

train track switches

lexical analyzers for parsers
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Any L

is defined to

be a
language.

L is just a set of strings. It is
called a language for historical
reasons.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Let L

be a language.

L is called a regular language
if there is some finite
automaton that accepts L.
In this lecture we have seen
many regular languages.

even length strings

strings containing ababb
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Theorem: Any finite langage is regular.
Proof: Make a machine with a
path for each string in the
language, sharing prefixes
Example: L = {a, bcd, ac, bb}
b
d
a
c
b
c
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Are all
languages
regular?
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
{ }
, , , ,
n n
a b ab aabb aaabbb c =
Consider the language
i.e., a bunch of as
followed by an equal
number of bs
No finite automaton accepts this language.

Can you prove this?
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
a
n
b
n
is not regular.
No machine has
enough states to
keep track of the
number of as it
might encounter.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
That is a fairly weak
argument. Consider the
following example
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
L = strings where the # of
occurrences of the pattern ab is
equal to the number of occurrences
of the pattern ba
Cant be regular. No machine has
enough states to keep track of the
number of occurrences of ab.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Remember ABA?
b
b
a
b
a
a
a
b
a
b
ABA accepts only the strings
with an equal number of abs and
bas!
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Let me show you a
professional strength proof
that a
n
b
n
is not regular.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Pigeonhole principle:
Given n boxes and m > n
objects, at least one box
must contain more than
one object.


Letterbox principle: If
the average number of
letters per box is a, then
some box will have at
least a letters. (Similarly,
some box has at most a.)
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Theorem: a
n
b
n
is not regular.

Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Professional Strength Proof
Theorem: a
n
b
n
is not regular.

Proof: Assume that it is. Then - M with k
states that accepts it.

For each 0 s i s k, let S
i
be the state M is
in after reading a
i
.

-i,j s k s.t. S
i
= S
j
, but i = j

M will do the same thing on a
i
b
i
and a
j
b
i
.


But a valid M must reject a
j
b
i
and accept
a
i
b
i
.
:
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
MORAL:
Finite automata cant count.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Advertisement
You can learn much more about these
creatures in the FLAC course.

Formal Languages, Automata, and
Computation

There is a unique smallest
automaton for any regular language

It can be found by a fast
algorithm.

Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Cellular Automata
Line up a bunch of identical finite automata in a
straight line.
( , , )
k i j l
Q Q Q Q
q q q q

=
k
q
l
q
,
i j
q q
i
q
j
q
k
q
Transitions are based on the states of the
machines two neighbors or an indicator that a
neighbor is missing. (There is no other input.)
All cells move to their next states at the same time:
synchronous transition
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
The Firing Squad Problem
Five soldiers all start in the sleep state.
You change the one on the left to the wake
state.
All five must get to the fire state at the
same time (for the first time).
sleep
wake
fire
sleep
wake
fire
sleep
wake
fire
sleep
wake
fire
sleep
wake
fire
sleep
wake
fire
sleep
wake
fire
sleep
wake
fire
sleep
wake
fire
sleep
wake
fire
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Shorthand
{ }
, , , Q a b d
Means use this transition when
your left neighbor is in any state
at all and your right neighbor is in
state a,b, or d.
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
sleep
wake
wake2 wake3
wake4
fire!
{w2},Q
{w},Q
{w3},Q
Q,Q
{w4},Q
{sleep,fire,end},Q
Q,Q
Q,Q
Q,Q
1 2 3 4 5
sleep sleep sleep sleep sleep
sleep sleep sleep sleep sleep
sleep sleep sleep sleep sleep
sleep sleep sleep sleep sleep
sleep sleep sleep sleep sleep
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
sleep
wake
wake2 wake3
wake4
fire!
{w2},Q
{w},Q
{w3},Q
Q,Q
{w4},Q
{sleep,fire,end},Q
Q,Q
Q,Q
Q,Q
1 2 3 4 5
wake sleep sleep sleep sleep
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
sleep
wake
wake2 wake3
wake4
fire!
{w2},Q
{w},Q
{w3},Q
Q,Q
{w4},Q
{sleep,fire,end},Q
Q,Q
Q,Q
Q,Q
1 2 3 4 5
wake sleep sleep sleep sleep
wake2 wake2 sleep sleep sleep
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
sleep
wake
wake2 wake3
wake4
fire!
{w2},Q
{w},Q
{w3},Q
Q,Q
{w4},Q
{sleep,fire,end},Q
Q,Q
Q,Q
Q,Q
1 2 3 4 5
wake sleep sleep sleep sleep
wake2 wake2 sleep sleep sleep
wake3 wake3 wake3 sleep sleep
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
sleep
wake
wake2 wake3
wake4
fire!
{w2},Q
{w},Q
{w3},Q
Q,Q
{w4},Q
{sleep,fire,end},Q
Q,Q
Q,Q
Q,Q
1 2 3 4 5
wake sleep sleep sleep sleep
wake2 wake2 sleep sleep sleep
wake3 wake3 wake3 sleep sleep
wake4 wake4 wake4 wake4 sleep
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
sleep
wake
wake2 wake3
wake4
fire!
{w2},Q
{w},Q
{w3},Q
Q,Q
{w4},Q
{sleep,fire,end},Q
Q,Q
Q,Q
Q,Q
1 2 3 4 5
wake sleep sleep sleep sleep
wake2 wake2 sleep sleep sleep
wake3 wake3 wake3 sleep sleep
wake4 wake4 wake4 wake4 sleep
fire fire fire fire fire
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Question
Can you build the soldiers finite
automaton brain before you know
how many soldiers will be in the
line?
No. Finite automata
cant count!
Steven Rudich:
www.cs.cmu.edu/~rudich
rudich0123456789
Dont jump to conclusions!
It is possible to design a
single cellular automaton
that works for any number
of soldiers!