AUTOMATA

Thus far we have been concerned with two major topics: the discovery of an
appropriate model for computation and an examination of the intrinsic
properties of computation in general. We found that since Turing machines are
equivalent to prograns, they form an appropriate model for computation. But
we also discovered that in some sense they possessed far too much
computational power. Because of this we ran into great difficulty when we tried
to ask questions about them, and about computation in general. Whenever we
wished to know something nontrivial, unsolvability sprang forth. And even in
the solvable or recursive realm, we found intractability.

Now we shall attempt to overcome this lack of information about computation

by restricting the power or our computational model. We hope that this will
force some of the decision problems in which we are interested into the zone of
solvability.

The sections include:

Finite Automata
Closure Properties
Nondeterministic Operation
Regular Sets and Expressions
Decision Problems for Finite Automata
Pushdown Automata
Unsolvable Problems for Pushdown Automata
Linear Bounded Automata

Historical Notes and References

Problems
 Finite Automata
Let us begin by removing almost all of the Turing machine's power! Maybe then
we shall have solvable decision problems and still be able to accomplish some
computational tasks. Also, we might be able to gain insight into the nature of
computation by examining what computational losses we incur with this loss of
power.

If we do not allow writing or two-way operation of the tape head, we have what
has been traditionally called a finite automaton. This machine is only allowed to
read its input tape and then, on the basis of what it has read and processed,
accept or reject the input. This restricted machine operates by:

a) Reading a symbol,
b) Transferring to a new instruction, and
c) Advancing the tape head one square to the right.

When it arrives at the end of its input it then accepts or rejects depending upon
what instruction is being executed.

This sounds very simple. It is merely a one-way, semi-literate Turing machine

that just decides membership problems for a living! Let us examine one. In
order to depict one, all we need to do is jot down Turing machine instructions
in one large table, leave out the write part (that was not a pun!), and add a note
which indicates whether the machine should accept. Here is an example:

Instruction Read Goto Accept?

0 same
1 no
1 next
0 same
2 yes
1 next
0 same
3 no
1 same

Look closely at this machine. It stays on instruction one until it reads a one.
Then it goes to instruction two and accepts any input unless another one arrives
(the symbol 1 - not another input). If two or more ones appear in the input,
then it ends up executing instruction three and does not accept when the input
is over. And if no ones appear in the input the machine remains on instruction
one and does not accept. So, this machine accepts only inputs that contain
exactly one 1.
Finite Automata 2

(N.B. Endmarkers are not needed since the machine just moves to the right.
Accepting happens when the machine finishes the input while executing an
instruction that calls for acceptance. Try this machine out on the inputs 000,
0100, 1000100, etc.)

Traditionally these machines have not had instructions but states. (Recall A. M.
Turing's states of mind.) Another way to represent this same machine is to put
the next instruction or next state or goto portion under the input in a table like
that in figure 1.

There is another traditional method to describe finite automata which is

extremely intuitive. It is a picture called a state graph. The states of the finite
automaton appear as vertices of the graph while the transitions from state to
state under inputs are the graph edges. The state graph for the same machine
also appears in figure 1.

1 1
1 2 3

0 0 0,1

Input
State Accept?
0 1
1 1 2 no
2 2 3 yes
3 3 3 no

Figure 1 - Finite Automaton Representations

(Note that the two circles that surround state two mean acceptance.)

Before continuing let's examine the computation of a finite automaton. Our first
example begins in state one and reads the input symbols in turn changing
states as necessary. Thus a computation can be characterized by a sequence of
states. (Recall that Turing machine configurations needed the state plus the
tape content. Since a finite automaton never writes, we always know what is on
the tape and need only look at a state as a configuration.) Here is the sequence
for the input 0001001.

Input Read: 0 0 0 1 0 0 1
States: 1 → 1 → 1 → 1 → 2 → 2 → 2 → 3
Finite Automata 3

Our next example is an elevator controller. Let's imagine an elevator that serves
two floors. Inputs are calls to a floor either from inside the elevator or from the
floor itself. This makes three distinct inputs possible, namely:

0 - no calls
1 - call to floor one
2 - call to floor two

The elevator itself can be going up, going down, or halted at a floor. If it is on a
floor it could be waiting for a call or about to go to the other floor. This
provides us with the six states shown in figure 2 along with the state graph for
the elevator controller.

0,2 2

W2 D2
W1 Waiting on first floor 0,2 0,1
U1 About to go up 1 1
UP Going up UP DN
DN Going down
0,2 0,1
W2 Waiting - second floor
D2 About to go down 2 2
W1 D1

0,1 1

Figure 2 - Elevator Control

A state table for the elevator is provided below as table 1.

Input
State
none call to 1 call to 2
W1 (wait on 1) W1 W1 UP
U1 (start up) UP U1 UP
UP W2 D2 W2
DN W1 W1 U1
W2 (wait on 2) W2 DN W2
D2 (start down) DN DN D2

Table 1 - Elevator Control

Accepting and rejecting states are not included in the elevator design because
acceptance is not an issue. If we were to design a more sophisticated elevator,
it might have states that indicated:
Finite Automata 4

a) power failure,
b) overloading, or
c) breakdown

In this case acceptance and rejection might make sense.

Let us make a few small notes about the design. If the elevator is about to move
(i.e. in state U1 or D2) and it is called to the floor it is presently on it will stay.
(This may be good - try it next time you are in an elevator.) And if it is moving
(up or down) and gets called back the other way, it remembers the call by going
to the U1 or D2 state upon arrival on the next floor. Of course the elevator does
not do things like open and close doors (these could be states too) since that
would have added complexity to the design. Speaking of complexity, imagine
having 100 floors.

That is our levity for this section. Now that we know what a finite automaton is,
we must (as usual) define it precisely.

Definition. A finite automaton M is a quintuple M = (S,I,δ,s0,F) where:

S is a finite set (of states)
I is a finite alphabet (of input symbols)
δ: S × I → S (next state function)
s0 ∈ S (the starting state)
F ⊆ S (the accepting states).

We also need some additional notation. The next state function is called the
transition function and the accepting states are often called final states. The
entire machine is usually defined by presenting a state table or a state graph. In
this way, the states, alphabet, transition function, and final states are
constructively defined. The starting state is usually the lowest numbered state.
Our first example of a finite automaton is:

M = ({s1 , s2 , s3}, {0,1}, δ, s1, {s2})

where the transition function δ, is defined explicitly by either a state table or a

state graph.

At this point we must make a slight detour and examine a very important yet
seemingly insignificant input string called the empty string. It is a string
without any symbols in it and is denoted as ε. It is not a string of blanks. An
example might make this clear. Look between the brackets in the picture below.
Finite Automata 5

A Blank Empty String

[] []

Let's look again at a computation by our first finite automaton. For the input
010, our machine begins in s1, reads a 0 and goes to δ(s1,0) = s1, then reads a 1
and goes to δ(s1,1) = s2, and ends up in δ(s2,0) = s2 after reading the final 0. All
of that can be put together as:

δ(δ(δ(s1,0),1),0) = s2

We call this transition on strings δ* and define it as follows.

Definition. Let M = (S,I,δ,s0,F). For any input string x, input symbol a, and
state si, the transition function on strings δ* takes the values:
δ*(si,(*e) = si
δ*(si,a) = δ(si,a)
δ*(si,xa) = δ(δ*(si,x),a).

That certainly was terse. But, δ* is really just what one expects it to be. It
merely applies the transition function to the symbols in the string. Let's look at
this for the example in figure 3.

b
s0 s1

a a b

s3 s2

a,b a,b

This machine has a set of states = {s0 , s1 , s2 , s3} and operates over the input
alphabet {a, b}. Its starting state is s0 and its set of final or accepting states, F =
{s2}. The transition function is fully described twice in figure 3; once in figure
3a as a state table and once in figure 3b as a state graph.
Finite Automata 6

Input
State Accept?
a b
0 3 1 no
1 3 2 no
2 2 2 yes
3 3 3 no

Figure 3 - Finite Automaton

If the machine receives the input bbaa it goes through the sequence of states:

s0, s1, s2, s2, s2

while when it gets an input such as abab it goes through the state transition:

s0, s3, s3, s3, s3

Now we shall become a bit more abstract. When a finite automaton receives an
input string such as:
x = x1 x2 ... xn

where the xi are symbols from its input alphabet, it progresses through the
sequence:
s k 1 , sk 2 , ... , sk n+1

where the states in the sequence are defined as:

sk1 = s0
sk2 = δ(sk1 , x1) = δ(s0 , x1)
sk3 = δ(sk2 , x2 ) = δ ∗ (s0 , x1x 2 )
!
skn+1 = δ(skn , xn ) = δ ∗ (s0 , x1x2 ...xn )

Getting back to a more intuitive reality, the following table provides an

assignment of values to the symbols used above for an input of bbaba to the
finite automaton of figure 3.

i 1 2 3 4 5 6
xi b b a b a
sk i s0 s1 s2 s2 s2 s2
Finite Automata 7

We have mentioned acceptance and rejection but have not talked too much
about it. This can be made precise also.

Definition. The set (of strings) accepted by the finite automaton M =

(S,I,δ,s0,F) is: T(M) = { x | δ*(s0,x) ∈ F }

This set of accepted strings (named T(M) to mean Tapes of M) is merely all of
the strings for which M ended up in a final or accepting state after processing
the string. For our first example (figure 1) this was all strings of 0's and 1's that
contain exactly one 1. Our last example (figure 3.1.3) accepted the set of strings
over the alphabet {a, b} which began with exactly two b's.
 Closure Properties
Removing power from Turing machines provided us with a new machine. We
also found a new class of sets. Now it is time to examine this class. Our first
questions concern operations on sets within the class.

Set complement is our first operation to examine. Since we are dealing with
strings (rather than numbers), we must redefine this operation. Thus the
definition of complement will slightly different. This is to be expected because
even though 0100 and 100 are the same number, they are different strings.
Here is the definition. If a set contains strings over an alphabet, then its
complement contains all of the strings (over the alphabet) not in the set.

Theorem 1. If a set is accepted by a finite automaton then its complement

can be accepted by a finite automaton.

Proof. Let the finite automaton M = (S, I, δ, s0, F) accept the set T(M). We
must now show that there is a finite automaton which accepts the set of
strings over the alphabet I which M does not accept. Our strategy will be
to look at the operation of M, accepting when it rejects and rejecting
when M accepts. Since we know that strings which take M to a state of F
are accepted by M and those which take M into S-F are rejected, then our
course is fairly clear. Consider the machine:

M' = (S, I, δ, s0, S-F).

It is exactly the same as M except for its final or accepting states. Thus it
should accept when M rejects. When we precisely examine this, we find
that for all strings x over the alphabet I:

x ∈ T(M) if and only if δ*(s0,x) ∈ F

if and only if δ*(s0,x) ∉ S-F
if and only if x ∉ T(M')

and so T(M' ) = T(M) .

The last proof contained an example of our first method of dealing with finite
automata: rearranging an existing machine. Now we shall employ another
strategy: combining two machines. This is actually going to be parallel
Closure Properties 2

processing. Suppose we take the two machines whose state graphs are in figure
1.
a

b
s0 s1 q0

a a b a
b
a

s3 s2 q q2
1 b

a,b a,b b

(a) (b)
Figure 1 - Finite Automaton Examples

We have seen the machine (M1) of figure 1a before, it accepts all strings (over {a,
b}) which begin with two b's. The other machine (M2) accepts strings which end
with two b's. Let's try to combine them into one machine which accepts strings
which either begin or end with two b's.

Why not run both machines at the same time on an input? We could keep track
of what state each machine is in by placing pebbles upon the appropriate states
and then advancing them according to the transition functions of each machine.
Both machines begin in their starting states, as pictured in the state graphs
below

b
s1

a a b a
b
a

s3 s2 q q2
1 b

a,b a,b b

with pebbles on s0 and q0. If both machines now read the symbol b on their
input tapes, they move the pebbles to new states and assume configurations
like these
Closure Properties 3

b
s0 q0

a a b a
b
a

s3 s2 q2
b

a,b a,b b

with pebbles on s1 and q1. The pebbles have advanced according to the
transition functions of the machines. Now let's have them both read an a. At
this point, they both advance their pebbles to the next state and enter the
configurations

b
s0 s1

a a b a
b
a

s2 q q2
1 b

a,b a,b b

with pebbles on s3 and q0.

With this picture in mind, let's trace the computations of both machines as they
process several input strings. Pay particular attention to the pairs of states the
machines go through. Our first string is bbabb, which will be accepted by both
machines.

Input: b b a b b
M1's states s0 s1 s2 s2 s2 s2
M2's states q0 q1 q2 q0 q1 q2

Now let us look at an input string neither will accept: babab.

Input: b a b a b
M1's states s0 s1 s3 s3 s3 s3
M2's states q0 q1 q0 q1 q0 q1

And finally, the string baabb which will be accepted by M2 but not M1.
Closure Properties 4

Input: b a a b b
M1's states s0 s1 s3 s3 s3 s3
M2's states q0 q1 q0 q0 q1 q2

If we imagine a multiprocessing finite automaton with two processors (one for

M1 and one for M2), it would probably look just like the pictures above. Its
state could be a state pair (one from each machine) corresponding to the pebble
positions. Then, if a pebble ended up on an accepting state for either machine
(that is, either s2 or q2), our multiprocessing finite automaton would accept.

This is not difficult at all! All we need to do is to define a new class of

machines and we can accept several things at once. (Note that the new
machines accept unions of sets accepted by finite automata.) Or, if we think for
a bit, we find that we do not need to define a new class for this. The next result
shows that we already have this facility with finite automata. The proof of the
theorem demonstrates this by careful manipulation of the symbolic definition
of finite automata.

Theorem 2. The class of sets accepted by finite automata is closed under

union.

Proof Sketch. Let M1 = (S, I, δ, s0, F) and M2 = (Q, I, γ, q0, G) be two

arbitrary finite automata. To prove the theorem we must show that there
is another machine (M3) which accepts every string accepted by M1 or M2.

We shall try the multiprocessing pebble machine concept and work it into
the definition of finite automata. Thus the states of M3 are pairs of states
(one from M1 and one from M2). This works out nicely since the set of
pairs of states from S and Q is known as the cross product (written S×Q)
between S and Q. The starting state is obviously <s0 , q0>. Thus:

M3 = (S×Q, I, ξ, <s0, q0>, H)

where ξ and H will be described presently.

The transition function ξ is just a combination of δ and γ, since it

simulates the advance of pebbles on the state graphs. It uses δ to change
the states in S and γ, to change the states in Q. In other words, if a is a
symbol of I:

ξ(<si , qi >, a) = <δ(si, a), γ(qi, a)>.

Closure Properties 5

Our final states are all of the pairs which contain a state from F or a state
from G. In cross product notation this is:

H = (F×Q) ∪ (S×G).

We have now defined M3 and know that it is indeed a finite automaton

because it satisfies the definition of finite automata. We claim it does
accept T(M1) ∪ T(M2) since it mimics the operation of our intuitive
multiprocessing pebble machine. The remainder of the formal proof
(which we shall leave as an exercise) is merely an induction on the length
of input strings to show that for all strings x over the alphabet I:

x ∈ T(M1) ∪ T(M2) iff δ*(s0,x) ∈ F or γ*(q0,x) ∈ G

iff ξ*(<s0 , q0 >,x) ∈ H.

Thus by construction we have shown that the class of sets accepted by

finite automata is closed under union.

By manipulating the notation we have shown that two finite automata can be
combined. Let's take a look at the result of applying this construction to the
machines of figure 1. (This is pictured in figure 2.)

a
b
a sq sq
sq 3 0 3 1
0 0
a
b a a b

sq sq sq
1 1 2 1 32

b
b b
a b

sq sq
2 2 2 0
a

b a

Figure 2 - Union of M1 and M2

Note that not all pairs of states are included in the state graph. (For example,
<s0, q1> and <s1, q2> are missing.) This is because it is impossible to get to
these states from <s0 , q0>.
Closure Properties 6

This is indeed a complicated machine! But, if we are a bit clever, we might

notice that if the machine enters a state pair containing s2, then it remains in
pairs containing s2. Thus we can combine all of these pairs and get the smaller
but equivalent machine of figure 3.

a
b
a sq sq
sq 3 0 3 1
0 0
a
b a a b

sq sq sq
1 1 2i 32
b

a,b b

Figure 3 - Reduced Union Machine

This is very likely what anyone would design if they had to build a machine
from scratch rather than use the construction detailed in the theorem. Lest
anyone think that finding smaller, equivalent machines is always this simple, we
must admit that this was an elementary example and we did some prior
planning to provide such clean results.

The final Boolean operation is set intersection. We state this here and leave the
proof as an exercise in Boolean algebra identities (De Morgan's Laws) or
machine construction - as you wish!

Theorem 3. The class of sets accepted by finite automata is closed under

intersection.
 Nondeterministic Operation
So far, every step taken by a finite automaton has been exactly determined by
the state of the machine and the symbol read. No choices have existed. This
mode of operation is called determinism and machines of this ilk are known as
deterministic finite automata. Finite automata need not be so unambiguous. We
could have defined them to have some choices of which state to advance to on
inputs. Examine figure 1, it provides the state graph of such a machine.

a
s0 S2 s1
b

b b a a

s3 s4
b

a,b a

Figure 1 - Nondeterministic Finite Automaton

If the input to this machine begins with the string ab, it will progress from s0 to
s1 to s2. Now, if the next input symbol is an a, it has a choice of whether to go
back to s1 or to move to s4. So, now what does it do? Well, it transfers to the
state which will eventually take it into a final state if possible. Simple as that!
But, how does it know when it has not seen the remainder of the input yet? Do
not worry about that - it always chooses the right state!

The above explanation of the operation of the machine of figure 1 is a slight bit
mystical. But we can (if we wish) define machines with choices in their
transition functions. And we can (if we wish) define acceptance to take place
whenever a correct sequence of states under the input will end up in a final
state. In other words, if the machine can get to a final state in some proper
manner, it will accept. Now for a formal definition.

Definition. A nondeterministic finite automaton is the quintuple

M = (S, I, δ, S0, F) where S, I, and F are as before but:

S0 ∈ S (a set of starting states), and

δ(s,a) ⊆ S for each s ∈ S and a ∈ I .
Nondeterministic Operation 2

Now instead of having a starting state and a transition function, we have a

starting state set and a set of transition states. More on this later. For now, note
that the only differences in the finite automaton definitions was that the
machine now has a choice of states to start in and a choice of transitions under
a state-symbol pair. A reduced version of the last nondeterministic machine is
presented in figure 2.

b
s0 s1
a

b a
a

s2 s3
b

a,b a

Input Accept?
State
a b
0 1,3 2 yes
1 3 0 no
2 2 2 no
3 3 2 yes

Figure 2 - Reduced Nondeterministic Machine

(N.B. We must note that the transition indicator δ is not a function any more.
To be precise about this we must state that it has become a relation. In other
words, since δ(s, a) is a set, it indicates which states are members of δ(s, a). If
that is confusing then forget it and consider δ(s, a) to be a set.) Our next step is
to define acceptance for nondeterministic finite automata. We could extend the
transition indicator so that it will handle strings. Since it provides a set of next
states, we will just note the set of states the machine is in after processing a
string. Let's look at a picture. Think about the last machine (the one in figure
2). Now imagine what states it might go through if it processed all possible
strings of length three. Now, look at figure 3.
Nondeterministic Operation 3

s0

a a b

s1 s3 s2
a b a b a,b
s3 s0 s3 s2 s2

a b a a b a b a,b a,b

s3 s2 s1 s3 s2 s3 s2 s2 s2

Figure 3- Computation Tree

In processing the string abb, the machine ends up in s2, but it can get there in
two different ways. These are:

s0 → s1 → s0 → s2
s0 → s3 → s2 → s2

Likewise, the machine can end up in 3 after processing the string aaa. But since
the automaton is nondeterministic, it can have the option of ending up in
several states after processing an input. For example, after aba, the set of states
{s1 , s2 , s3} is reached.

In fact, a set of states is reached by the machine after processing an input. This
set depends upon the choices the automaton was offered along the way. And, if
a final state was in the set reached by the machine, we accept. In the above
example only the strings aba and aaa can be accepted because they were the
only strings which took the automaton into s3.

This gives a definition of δ* as the set of states reached by the automaton and
the tapes accepted as:

T(M) = { x | δ*(s0,x) ∩ F ≠ ∅ }

On the other hand, instead of extending δ to strings as with the deterministic

case, we shall discuss sequences of state transitions under the state transition
indicator δ. The following formal definition of acceptance merely states that a
string is accepted by a nondeterministic finite automaton if there is a sequence
of states (or path through the computation tree) which leads from a starting
Nondeterministic Operation 4

state to a final state under the input string. (Note that we do not discuss how a
machine might find this sequence - that does not matter! We just say that the
input is accepted if there exists such a sequence of states.)

Definition. The input x = x1 ... xn is accepted by the nondeterministic

finite automaton M = (S, I, δ, S0, F) if and only if there is a sequence of
states: sk1 , sk2 ,..., skn +1 where:
a ) sk1 ∈ S 0
b ) for each i ≤ n: ski +1 ∈ δ (ski , x i )
c ) skn +1 ∈ F .

With this definition of acceptance in mind, we define the set of tapes accepted
by a nondeterministic finite automaton (denoted T(M) as before) as exactly
those inputs for which there is a sequence of states leading from a starting
state to an accepting state.

Nondeterminism is useful because it allows us to define very simple machines

which perform certain operations. As an example, let's revisit the union closure
problem. As usual, suppose we have the two finite automata M1 = (S, I, δ, s0, F)
and M2 = (Q, I, γ, q0, G) and wish to build a machine which accepts the union of
the sets they accept. Our new union machine contains the states of both M1 and
M2 plus a new starting state. This new starting state leads into the states of
either M1 or M2 in a nondeterministic manner. That is, under the first input
symbol, we would advance to an appropriate state in M1 or an appropriate state
in M2. Formally, if I = {0,1}, the transition indicator of the union machine is ξ,
and its starting state s' then:

ξ(s', 0) = {δ(s0, 0), γ(q0, 0)}

ξ(s', 1) = {δ(s0, 1), γ(q0, 1)}

The rest of the transition relation ξ is just a combination of δ and γ, and the
state graph for the new union machine might resemble:

S'

M 1 's M 2's
state state
graph graph
Nondeterministic Operation 5

Thus the union machine is M = (S∪Q∪{s'}, I, ξ, s', F∪G). Acceptance takes place
whenever there is a sequence to an accepting state through either the state
graph of M1 or that of M2.

Let's try another closure property and build a nondeterministic machine which
realizes it. This will be a string property called concatenation or juxtaposition.
For strings this is easy, the concatenation of x and y is xy. If we have the sets A
and B then the concatenation of them is defined as:

AB = { xy | x ∈ A and y ∈ B }.

If A and B can be accepted by the deterministic finite automata M1 = (S, I, δ, s0,

F) and M2 = (Q, I, γ, q0, G) we need to try and construct a machine M3 which will
accept AB. Let us look at:

M3 = (S∪Q, I, ξ, {s0}, G)

and define ξ so that M3 accepts AB. Our strategy will be to start out processing
as M1 might and then when M1 wishes to accept a part of the input, switch to
M2 and continue on. With any luck we will end up in G, the final state set of
M2. Nondeterminism will be used to make the change from M1 to M2. The
transition relation ξ will operate just like δ on the state set S and like γ on the
state set Q except for the final states F ⊆ S. There it will include a transition to
the starting state q0 of M2. One might picture this as:

q0
S-F

Q
F

and define the transition relation precisely as:

ξ(si, a) = {δ(si, a)} for si ∈ S-F

ξ(si, a) = {δ(si, a), q0} for si ∈ F
ξ(qi, a) = {γ(qi, a)} for qi ∈ Q

By the definition of acceptance for nondeterministic machines, M3 will accept a

string if and only if there is a sequence of states (under the direction of ξ) which
leads from s0 to a state in G. Suppose z is a member of AB. Then:
Nondeterministic Operation 6

z ∈ AB iff z = xy where x ∈ A and y ∈ B

iff x ∈ T(M1) and y ∈ T(M2)
iff δ*(s0,x) ∈ F and γ*(q0,y) ∈ G

This means that there is a sequence of states

sk1 , sk2 ,..., sk n

in S from s0 = s k1 to sk n ∈ F under δ and x. Also, there is a sequence of states

in Q
q k1 , q k2 ,..., q k m

from q0 = q k1 to q k m ∈ G under γ and y. Since ξ is defined to be just like δ on

S and like γ on Q, these sequences of states in S and Q exist under the influence
of ξ and x and ξ and y. We now note that instead of going to the last state sk n
in the sequence in S, ξ could have directed transferred control to q0. Thus there
is a sequence:

s0 , sk2 ,..., sk n-1 , q 0 , q k2 ,..., q k m

under ξ and z = xy which proceeds from s0 to G. We can now claim that

T(M3) = AB = T(M1)T(M2)

noting that the only way to get from s0 to a state in G via ξ is via a string in AB.

A final note on the machine M3 which was just constructed to accept AB. If A
contains the empty word then of course s0 is a final state and thus q0 also must
be a member of the starting state set. Also, note that if B contains the empty
word, then the final states of M3 must be F∪G rather than merely G.

Now we shall move along and look at multiple concatenation. Suppose we

concatenated the same set together several times. Putting this more formally
and in superscript notation, let:

A0 = {ε}
A1 = A
A2 = AA
A3 = AA2 = AAA
Nondeterministic Operation 7

and so forth. The general scheme for this is:

A0 = {ε}
Ai+1 = AiA

To sum everything up, we consider the union of this infinite sequence of

concatenations and call it the Kleene closure of the set A. Here is the definition.

Definition. The Kleene Closure (written A*) of the set A is the union of Ak
over all integer values of k.

This operator is known as the Kleene Star Operator and so A* is pronounced A

star. One special case of this operator's use needs to be mentioned. If A is the
set {a,b} (in other words: an alphabet), then A* is the set of all strings over the
alphabet. This will be handy.

To accept the Kleene closure of a set accepted by a deterministic finite

automaton a construction similar to that used above for concatenation works
nicely. The strategy we shall use is to allow a reset to the starting state each
time a final state is entered. (That is, start over whenever a string from the
original set could have ended.) We shall present the construction below and
leave the proof of correctness as an exercise.

For a deterministic finite automaton M = (S, I, δ, s0, F) we must build a

(nondeterministic) machine which accepts [T(M)]*. As we mentioned, our
strategy will be to allow the transition relation to reset to s0 whenever a final
state is reached. First we shall introduce a new starting state s' and the
following transition relation γ for all a U I:

γ(s',a) = {δ(s0,a)}
γ(si,a) = {δ(si,a)} for si ∈ S-F
γ(si,a) = { s0, δ(si,a)} for si ∈ F

The machine which accepts [T(M)]* is M' = (S∪{s'},I,γ,{s'},F∪{s'}). Now that we have
seen how useful nondeterministic operation can be in the design of finite
automata, it is time to ask whether they have more power than their
deterministic relatives.
Nondeterministic Operation 8

Theorem 3. The class of sets accepted by finite automata is exactly the

same class as that accepted by nondeterministic finite automata.

Proof Sketch. Two things need to be shown. First, since deterministic

machines are also nondeterministic machines (which do not ever make
choices) then the sets they accept are a subclass of those accepted by
nondeterministic automata.

To show the other necessary relation we must demonstrate that every set
accepted by a nondeterministic machine can be accepted in a
deterministic manner. We shall bring forth again our pebble automaton
model.

Let Mn = (S, I, δ, S0, F) be an arbitrary nondeterministic finite automaton.

Consider its state graph. Now, place a pebble on each state in S0. Then,
process an input string under the transition relation δ. As δ calls for
transitions from state to state, move the pebbles accordingly. Under an
input a, with a pebble on si, we pick up the pebble from si and place
pebbles on every state in δ(Si, a). This is done for all states which have
pebbles upon them. Indeed, this is parallel processing with a vengeance!
(Recall the computation tree of figure 3, we are just crawling over it with
pebbles - or processors.) After the input has been processed, we accept if
any of the final states have pebbles upon them.

Since we moved pebbles on all of the paths Mn could have taken from a
starting state to a final state (and we did not miss any!), we should accept
whenever Mn does. And, also, if we accept then there was indeed a path
from a starting state to a final state. Intuitively it all seems to work.

But, can we define a deterministic machine which does the job? Let's try.
First let us define some states which correspond to the pebbled states of
Mn. Consider:
q1 means a pebble upon s0
q2 means a pebble upon s1
q3 means pebbles upon s0 and s1
q4 means a pebble upon s2
q5 means pebbles upon s0 and s2
•
•
•
q means pebbles upon s0, s1, ... ,sn
2n+1 −1
Nondeterministic Operation 9

This is our state set Q. The starting state is the qk which means pebbles
on all of S0. Our set of final states (G) includes all qi which have a pebble
on a state of F. All that is left is to define the transition function γ and we
have defined a deterministic finite automaton Md = (Q, I, γ, qk, G) which
should accept the same set as that accepted by Mn.

This is not difficult. Suppose qi means pebbles upon all of the states in s'
⊆ S. Now we take the union of δ(sk,a) over all sk ∈ s' and find the qj
which means pebbles on all of these states. Then we define γ(qi, a) = qj.

The remainder of the proof involves an argument that Mn and Md accept

the same set.

With this theorem and our previous constructions firmly in mind, we now state
the following.

Corollary. The class of sets accepted by finite automata is closed under

concatenation and Kleene closure.

A cautionary remark must be made at this time. It is rather wonderful to be

able to use nondeterminism in our machine constructions. But, when these
machines are converted to deterministic finite automata via the above
construction, there is an exponential state explosion! A four state machine
would have fifteen states when converted to a deterministic machine. Imagine
what a 100 state machine would look like when converted! Fortunately most of
these machines can be reduced to a more manageable size.
 Regular Sets and Expressions
Finite automata are important in science, mathematics, and engineering.
Engineers like them because they are superb models for circuits (And, since the
advent of VLSI systems sometimes finite automata are circuits!) Computer
scientists adore them because they adapt very nicely to algorithm design, for
example the lexical analysis portion of compiling and translation.
Mathematicians are intrigued by them too due to the fact that there are several
nifty mathematical characterizations of the sets they accept. This is partially
what this section is about.

We shall build expressions from the symbols 0, 1, +, and & using the operations
of union, concatenation, and Kleene closure. Several intuitive examples of our
notation are:

a) 01 means a zero followed by a one (concatenation)

b) 0+1 means either a zero or a one (union)
c) 0* means ^ + 0 + 00 + 000 + ... (Kleene closure)

With parentheses we can build larger expressions. And we can associate

meanings with our expressions. Here's how:

Expression Set Represented

(0+1)* all strings over {0,1}.
0*10*10* strings containing exactly two ones.
(0+1)*11 strings which end with two ones.

That is the intuitive approach to these new expressions or formulas.

Now for a precise, formal view. Several definitions should do the job.

Definition. 0, 1, ε, and ∅ are regular expressions.

Definition. If α and β are regular expressions, then so are

(αβ), (α + β), and (α)*.

OK, fine. Regular expressions are strings put together with zeros, ones,
epsilons, stars, plusses, and matched parentheses in certain ways. But why did
we do it? And what do they mean? We shall answer this with a list of what
various general regular expressions represent. First, let us define what some
specific regular expressions represent.
Regular Sets 2

a) 0 represents the set {0}

b) 1 represents the set {1}
c) ε represents the set {ε} (the empty string)
d) ∅ represents the empty set

Now for some general cases. If α and β are regular expressions representing the
sets A and B, then:

a) (αβ) represents the set AB

b) (α + β) represents the set A∪B
c) (α)* represents the set A*

The sets which can be represented by regular expressions are called regular
sets. When writing down regular expressions to represent regular sets we shall
often drop parentheses around concatenations. Some examples are 11(0 + 1)*
(the set of strings beginning with two ones), 0*1* (all strings which contain a
possibly empty sequence of zeros followed by a possibly null string of ones),
and the examples mentioned earlier. We also should note that {0,1} is not the
only alphabet for regular sets. Any finite alphabet may be used.

From our precise definitions of the regular expressions and the sets they
represent we can derive the following nice characterization of the regular sets.
Then, very quickly we shall relate them to finite automata.

Theorem 1. The class of regular sets is the smallest class containing the
sets {0}, {1}, {ε}, and ∅ which is closed under union, concatenation, and
Kleene closure.

See why the above characterization theorem is true? And why we left out the
proof? Anyway, that is all rather neat but, what exactly does it have to do with
finite automata?

Theorem 2. Every regular set can be accepted by a finite automaton.

Proof. The singleton sets {0}, {1}, {ε}, and ∅ can all be accepted by finite
automata. The fact that the class of sets accepted by finite automata is
closed under union, concatenation, and Kleene closure completes the
proof.

Just from closure properties we know that we can build finite automata to
accept all of the regular sets. And this is indeed done using the constructions
Regular Sets 3

from the theorems. For example, to build a machine accepting (a + b)a*b, we

design:

Ma which accepts {a},

Mb which accepts {b},
Ma+b which accepts {a, b} (from Ma and Mb),
M which accepts a*,
a*
and so forth

until the desired machine has been built. This is easily done automatically, and
is not too bad after the final machine is reduced. But it would be nice though to
have some algorithm for converting regular expressions directly to automata.
The following algorithm for this will be presented in intuitive terms in language
reminiscent of language parsing and translation.

Initially, we shall take a regular expression and break it into subexpressions. For
example, the regular expression (aa + b)*ab(bb)* can be broken into the three
subexpressions: (aa + b)*, ab, and (bb)*. (These can be broken down later on in
the same manner if necessary.) Then we number the symbols in the expression
so that we can distinguish between them later. Our three subexpressions now
are: (a1a2 + b1)*, a3b2, and (b3b4)*.

Symbols which lead an expression are important as are those which end the
expression. We group these in sets named FIRST and LAST. These sets for our
subexpressions are:

Expression FIRST LAST

(a a + b )*
1 2 1
a1 , b1 a2 , b1
a3b2 a3 b2
(b3b4)* b3 b4

Note that since the FIRST subexpression contained a union there were two
symbols in its FIRST set. The FIRST set for the entire expression is: {a1 , a3 , b1}.
The reason that a3 was in this set is that since the first subexpression was
starred, it could be skipped and thus the first symbol of the next subexpression
could be the first symbol for the entire expression. For similar reasons, the
LAST set for the whole expression is {b2 , b4}.

Formal, precise rules do govern the construction of the FIRST and LAST sets.
We know that FIRST(a) = {a} and that we always build FIRST and LAST sets from
the bottom up. Here are the remaining rules for FIRST sets.
Regular Sets 4

Definition. If α and β are regular expressions then:

a) FIRST(α + β) = FIRST(α) ∪ FIRST(β)
b) FIRST(α*) = FIRST(α) ∪ {ε}
FIRST(α) if ε ∉ FIRST(α)
c) FIRST(αβ) =
FIRST(α) ∪ FIRST(β) otherwise

Examining these rules with care reveals that the above chart was not quite what
the rules call for since empty strings were omitted. The correct, complete chart
is:

Expression FIRST LAST

(a a + b )*
1 2 1 a1, b1, ε a2 , b1, ε
a3b2 a3 b2
(b3b4)* b3, ε b4, ε

Rules for the LAST sets are much the same in spirit and their formulation will
be left as an exercise.

One more notion is needed, the set of symbols which might follow each symbol
in any strings generated from the expression. We shall first provide an example
and explain in a moment.

Symbol a1 a2 a3 b1 b2 b3 b4
FOLLOW a2 a1, a3, b b2 a1, a3, b1 b3 b4 b3

Now, how did we do this? It is almost obvious if given a little thought. The
FOLLOW set for a symbol is all of the symbols which could come next. The
algorithm goes as follows. To find FOLLOW(a), we keep breaking the expression
into subexpressions until the symbol a is in the LAST set of a subexpression.
Then FOLLOW(a) is the FIRST set of the next subexpression. Here is an example.
Suppose that we have αβ as our expression and know that a ∈ LAST(α). Then
FOLLOW(a) = FIRST(β). In most cases, this is the way it we compute FOLLOW
sets.
Regular Sets 5

But, there are three exceptions that must be noted.

1) If an expression of the form aγ* is in α then we must also include the

FIRST set of this starred subexpression γ.

2) If α is of the form β* then FOLLOW(a) also contains α's FIRST set.

3) If the subexpression to the right of α has an ε in its FIRST set, then we

keep on to the right unioning FIRST sets until we no longer find an ε
in one.

Another example. Let's find the FOLLOW set for b1 in the regular expression (a1
+ b a *)*b *(a + b ). First we break it down into subexpressions until b is in a
1 2 2 3 3 1
LAST set. These are:

(a1 + b1 a2 * )* b2* (a3 + b3)

Their FIRST and LAST sets are:

Expression FIRST LAST

(a + b a *)*
1 1 2 a1, b1, ε a1, b1, a2, ε
b2* b2, ε b2, ε
(a3 + b3) a3, b3 a3, b3

Since b1 is in the LAST set of asubexpression which is starred then we place

that subexpression's FIRST set {a , b } into FOLLOW(b ). Since a * came after
1 1 1 2
b1 and was starred we must include a2 also. We also place the FIRST set of the
next subexpression (b2*) in the FOLLOW set. Since that set contained an ε , we
must put the next FIRST set in also. Thus in this example, all of the FIRST sets
are combined and we have:

FOLLOW(b1) = {a1 , b1 , a2 , b2 , a3 , b3}

Several other FOLLOW sets are:

FOLLOW(a1) = {a1 , b1 , b2 , a3 , b3}

FOLLOW(b2) = {b2 , a3 , b3}

After computing all of these sets it is not hard to set up a finite automaton for
any regular expression. Begin with a state named s0. Connect it to states
Regular Sets 6

denoting the FIRST sets of the expression. (By sets we mean: split the FIRST set
into two parts, one for each type of symbol.) Our first example (a1a2 +
b )*a b (b b )* provides:
1 3 2 3 4

a
s0 a1,3

b1

Next, connect the states just generated to states denoting the FOLLOW sets of
all their symbols. Again, we have:

a b
s0 a1,3 b
2

b a a

b1 a2

Continue on until everything is connected. Any edges missing at this point

should be connected to a rejecting state named sr. The states containing
symbols in the expression's LAST set are the accepting states. The complete
construction for our example (aa + b)*ab(bb)* is:

a b b2
s0 a1,3

b a a a a b

b1 a2 b3
b
a
b
b b

sr b4
a

a,b
Regular Sets 7

This construction did indeed produce an equivalent finite automaton, and in

not too inefficient a manner. Though if we note that b2 and b4 are basically the
same, and that b1 and a2 are similar, we can easily streamline the automaton to:

a b b2,4
s0 a1,3

b a a a b b

a2b sr b3
1 a

b a,b

Our construction method provides:

a a1,3
s0

b b

a a
b 123 123
b
b a

for our final example. There is a very simple equivalent machine.

Try to find it!

We now close this section with the equivalence theorem concerning finite
automata and regular sets. Half of it was proven earlier in the section, but the
translation of finite automata into regular expressions remains. This is not
included for two reasons. First, that it is very tedious, and secondly that nobody
ever actually does that translation for any practical reason! (It is an interesting
demonstration of a correctness proof which involves several levels of iteration
and should be looked up by the interested reader.)

Theorem 3. The regular sets are exactly those sets accepted by finite
automata.
 Decision Problems for Finite Automata
Now we wish to examine decision problems for the sets accepted by finite
automata (or the regular sets). When we tried to decide things concerning the
r.e. sets we were disappointed because everything nontrivial seemed to be
unsolvable. Our hope in defining finite automata as a much weaker version of
Turing machines was to gain solvability at the expense of computational power.
Let us see if we have succeeded. Our first result indicates that we have.

Theorem 1. Membership is solvable for the regular sets.

Proof. This is very easy indeed. With a universal Turing machine we can
simulate any finite automaton. To decide whether it accepts an input we
need just watch it for a number of steps equal to the length of that input.

So far, so good. In order to decide things a bit more intricate than membership
though, we need a very useful technical lemma which seems very strange
indeed until we begin to use it. It is one of the most important results in finite
automata theory and it provides us with a handle on the finiteness of finite
automata. It tells us that only the information stored in the states of a finite
automaton is available during computation. (This seems obvious, but the proof
of the following lemma points this out in a very powerful manner.)

Lemma (Pumping). Let M = (S, I, δ, s0, F) be a finite automaton. Then for

any string x accepted by M whose length is no less than the size of S, there
are strings u, v, and w (over the alphabet I) such that:

a) x = uvw,
b) v is not the empty string, and
c) for all k ≥ 0, uvkw ∈ T(M).

Proof. Let x = x1 ... xn. Suppose that x is accepted by the finite

automaton M and has length n where n is not smaller than the size of S,
the state set of M. Then as M processes x, it goes through the sequence of
states:

s j ,...,s j
1 n +1
Finite Automata Decision Problems 2

where M is in state s j as it reads xi, and:

i

a) s j = s0,
1
b) for each i ≤ n, δ(s j ,xi) = s j , and
i i+1
c) s j ∈ F.
n+1

Since M has no more than n states (our initial assumption about the
length of x not being less than the number of M’s states), at least one of
the states in the sequence must be repeated because there are n+1 states
in the sequence. We shall assume that this repeat occurs at s ja and s j
b
where a < b.

Now let's consider the string xa ... xb-1, the portion of the input which is
processed by M as it goes through the sequence of states s ja , ... ,s j . We
b
shall say that v is this substring of x and note that since a < b, v ≠ ε. Now
we shall assign the remaining characters of x to u and w. The following
picture illustrates this.

← u → ← v → ← w →

x1 ... xa-1 xa ... xb-1 xb ... xn

It is obvious that uvw = x. And, when M processes x:

a) δ*(s j ,u) = δ*( s0,u) = s ja [since s j = s0]

1 1
*
b) δ (s ja ,v) = s j = s ja [since s j = s ja ]
b b
c) δ*( s0,uv) = s = s
jb ja [same reason]
d) δ*(s j ,w) = δ*(s ja ,w) = s j ∈F [same again]
b n+1

In other words, M enters and leaves the substring v in the same state. Now
we shall examine exactly what this means. If we were to omit v and
process uw, M would leave u in s ja = s j and finish w in s j just as
b n+1
before. Thus uw is in T(M). If we were to make M process uvvw then M
would leave uv in s j = s ja , leave uvv in s j and finish w in the same
b b
state as before. Thus uvvw ∈ T(M). In fact, no matter how many times we
add another v between u and w, M always leaves and enters each v in s ja
and therfore finishes the entire input in the same final state. Thus for
any k ≥ 0, uvkw ∈ T(M).
Finite Automata Decision Problems 3

If we go back and examine our proof of the pumping lemma, we find that we
can prove something a little more powerful. In fact, something that will come
in handy in the future. Something which will make our lives much more
pleasing. Here it is.

Corollary. The substring v of x can be forced to reside in any portion of x

which is at least as long as the number of states of the automaton.

Proof. Just note that in any substring of x which is no shorter than the
number of states, we can find a repeated state while processing. This
provides a v and the proof proceeds as before.

This technical lemma (referred to as the pumping lemma from now on) is one of
the most useful results in theoretical work involving finite automata and the
regular sets. It is the major tool used to

a) detect non-regular sets, and

b) prove decision problems solvable for finite automata.

The usefulness of the pumping lemma comes from the fact that it dramatically
points out one of the major characteristics of finite automata, namely that they
have only a finite amount of memory. Another way to state this is to say that if
a finite automaton has n states, then it can only remember n different things! In
fact, if δ*(s0, x) = δ*(s0, y) then the machine with the transition function δ
cannot tell the difference between x and y. They look the same to the machine
since they induce the same last state in computations involving them. And, if a
finite automaton accepts a very long string, then chances are that this string
contained repetitive patterns.

Our first use of the pumping lemma will be to present a non-regular set. This is
the favorite example for computer science theorists and illustrates the method
almost always used to prove sets non-regular.

Theorem 2. The set of strings of the form {0n1n} for any n ≥ 0 is not a
regular set.

Proof. Assume that the set of strings of the form 0n1n is a regular set
and that the finite automaton M = (S, I, δ, s0, F) accepts it. Thus every
string of the form 0k1k for k larger than the size of the state set S will be
accepted by M.

If we take one of these strings for some k > |S| then the pumping lemma
assures us that there are strings u, v, and w such that:
Finite Automata Decision Problems 4

a) uvw = 0k1k,
b) v ≠ ε, and
c) for all n ≥ 0, uvnw ∈ T(M).

Invoking the corollary to the pumping lemma assures us that the

substring v can be in the midst of the 0's if we wish. Thus v = 0m for
some (nonzero) m ≤ k. This makes uw = 0k-m1k and the pumping lemma
states that uw ∈ T(M). Since uw is not of the form 0n1n we have a
contradiction. Thus our assumption that strings of the form 0n1n can be
accepted by finite automata and be regular set is incorrect.

As we mentioned earlier, almost all of the proofs of non-regularity involve the

same technique used in the proof of the last theorem. One merely needs to
examine the position of v in a long string contained in the set. Then either
remove it or repeat it several times. This will always produce an improper
string!

Next, we shall use the deflation aspect of the pumping lemma in order to show
that emptiness is solvable for regular sets.

Theorem 3. If a finite automaton accepts any strings, it will accept one of

length less than the size of its state set.

Proof. Let M be a finite automaton which accepts the string x and that
the length of x is no less than the size of M's state set. Assume further
that M accepts no strings shorter than x. (This is the opposite of our
theorem.)

Immediately the pumping lemma asserts that there are strings u, v, and w
such that uvw = x, v ≠ ε, and uw ∈ T(M). Since v ≠ ε, uw is shorter than
uvw = x. Thus M accepts shorter strings than x and the theorem follows.

Here is a sequence of corollaries which follow from the last theorem. In each
case the proof merely involves checking membership for all strings of length
less than the size of the state set for some finite automaton or the machine
which accepts the complement of the set it accepts. (Recall that the class of
regular sets, namely those accepted by finite automata is closed under
complement.)

Corollary (Emptiness Problem). Whether or not a finite automaton

accepts anything is solvable.

Corollary (Emptiness of Complement). Whether or not a finite automaton

rejects anything is solvable.
Finite Automata Decision Problems 5

Corollary. Whether or not a finite automaton accepts everything is

solvable.

Another cautionary note about these decision problems is in order. It is quite

refreshing that many things are solvable for the regular sets and it is wonderful
that several solvable decision problems came immediately from one lemma and
a theorem. But, it is very expensive to try and solve these problems. If we need
to look at all of the input strings of length less than the size of the state set
(let's say that it is of size n) then we are looking at almost 2n strings! Imagine
how long this takes when n is equal to 100 or so!

Flushed with success we shall attempt (and succeed) at another decision

problem which is unsolvable for the class of recursively enumerable sets.

Theorem 4. Whether a regular set is finite is solvable.

Proof Sketch. We know that if a finite automaton accepts any strings at

all then some will be of length less than the size of the state set. (This
does not help directly, but it gives a hint as to what we need for this
theorem.) The deletion aspect [uw Υ T(M)] of the pumping lemma was
used to prove this. Let's use the inflation aspect [uvnw Υ T(M)] of the
lemma to look for an infinite set.

For starters, if we were to find a string accepted by a finite automaton M

which was longer than or equal to the size of it’s state set, we could use
the aforementioned inflation aspect of the pumping lemma to show that
machine M must accept an infinite number of strings. This means that:

a finite automaton accepts only strings of length less than the

size of its set of states, if and only if it accepts a finite set.

Thus, to solve the finiteness problem for M = (S, I, δ, s0, F), we need to
determine whether or not:

T(M) - {strings of length < |S|} = ∅.

A question now arises as to how many inputs we must examine in order

to tell if M will accept an input longer than or equal to the size of its state
set. The answer is that we only need to consider input strings up to twice
the size of the state set. (The proof of this is left as an exercise, but it is
much the same as the proof of the emptiness problem.) This ends our
proof sketch.
Finite Automata Decision Problems 6

The next decision problem is included because it demonstrates another

technique; using old problems to solve a new one. We see this in the areas of
unsolvability and complexity when we use reducibility to show problems
unsolvable or intractable. Here we show that the problem of set equivalence is
reducible to the emptiness problem and thus solvable.

Theorem 5. Whether two regular sets are identical is solvable.

Proof. Let's take two finite automata (M1 and M2) and examine a picture
of the sets they accept.

T(M1 ) T(M2 )

If the intersection is the same as both sets then indeed they are identical.
Or, on the other hand, if the areas outside the intersection are empty then
both sets are identical. Let's examine these outside areas.

T(M1 ) T(M2 )

T (M1) ∩ T(M 2 ) T(M 2 ) ∩ T(M1 )

The picture on the left represents the set accepted by M1 and rejected by
M2 while that on the right is the set which M2 accepts and M1 rejects.

If these two areas (or sets) are empty then the sets accepted by M1 and
M2 are exactly the same. This means that the equivalence problem for
T(M1) and T(M2) is exactly the same as the emptiness problem for:
[ T(M1) ∩ T(M2 ) ] ∪ [ T(M2 ) ∩ T(M1) ]

So, if we can solve the emptiness problem for the above set, then we can
solve the equivalence problem for T(M1) and T(M2). Since the regular sets
are closed under union, complement, and intersection; the above set is a
regular set. And, we know that emptiness is solvable for the class of
regular sets.

Here is yet another cautionary note. The last proof was quite slick and elegant,
but one should not do the construction in an attempt to prove that two finite
Finite Automata Decision Problems 7

automata accept the same set. We know that the complexity of any algorithm
which attempts to do this is very large since it would take a while to do
emptiness for a machine formed from the unions and intersections of four
different machines.

We shall close this section on a cheerful note with an intuitive, imprecise

statement about finite automata and the class of regular sets.

Folk Theorem. Just about everything concerning finite automata or the

regular sets is solvable!
 Pushdown Automata
In the last section we found that restricting the computational power of
computing devices produced solvable decision problems for the class of sets
accepted by finite automata. But along with this ability to solve problems came
a rather sharp decrease in computational power. We discovered that finite
automata were far too weak to even tell if an input string was of the form anbn.
In this section we shall extend the power of finite automata a little so that we
can decide membership in sets which cannot be accepted by finite automata.

Let's begin. In fact, let's provide a finite automaton with a data structure which
will allow it to recognize strings of the form anbn. To tell if a string is of the
form anbn we need to match the a's with the b's. We could use a counter for
this, but thinking ahead a bit, there is a computer science way to do this. We
shall allow the machine to build a pile of discs as it processes the a's in its
input. Then it will unpile these disks as it passes over the b's. Consider the
following algorithm for a machine of this kind.

place the input head on the leftmost

input symbol

while symbol read = a

advance head
place disc on pile

while symbol read = b and pile contains discs

advance head
remove disc from pile

if input has been scanned

and pile = empty then accept

n n
Figure 1 - a b Recognition Algorithm

It is clear exactly what happens when the algorithm of figure 1 is used on the
input aaabbb. The machine reads the a's and builds a pile of three discs. Then
it reads the b's and removes the discs from the pile one by one as each b is
read. At this point it has finished the input and its pile is empty so it accepts.
If it was given aabbb, it would place two discs on the pile and then remove them
Pushdown Automata 2

as it read the first two b's. Then it would leave the second while loop with one
b left to read (since the pile was empty) and thus not accept. For aaabb it would
end with one disk on the pile and not accept that input either. When given the
input string aabbab, the machine would finish the second loop with ab yet to be
read. Now, try the strings aaa and bbb as exercises. What happens?

We now have a new data structure (a pile) attached to our old friend, the finite
automaton. During the last algorithm several conventions implicitly arose.
They were:

• The tape head advanced on each step,

• Discs were placed on top of the pile, and
• An empty pile means acceptance.

Let us now attempt something a bit more difficult. Here's where we shall use a
structure more powerful than a counter. Why not try to recognize strings of the
form w#wR where w is a string over the alphabet {a, b} and wR is the reversal of
the string w? (Reversal is just turning the string around end for end. For
example, abaaR = aaba.) Now we need to do some comparing, not just counting.
Examine the algorithm of figure 2.

place input head upon leftmost input symbol

while symbol being scanned ≠ #

if symbol scanned = a, put red disk on pile
if symbol scanned = b, put blue disk on pile
advance input head to next symbol

advance input head past #

repeat
if (symbol scanned = a and red disk on pile)
or (symbol scanned = b and blue disk on pile)
then remove top disk; advance input head
until (pile is empty) or (no input remains)
or (no disk removed)

if input has been read and pile is empty then accept

Figure 2 - Accepting Strings of the Form w#wR

Pushdown Automata 3

We will now look at what happens to the disc pile when this machine processes
the input string abaa#aaba. Here is a picture:

Input: a b a a # a a b a
red red
Pile: red red red red
blue blue blue blue blue blue
red red red red red red red red

At the right end of the picture, the machine reads the a and removes the red
disk from the stack. Since the stack is empty, it accepts.

Our first machine (figure 1) used its discs to count the a's and match them
against the b's. The second machine (figure 2) used the pile of discs to full
advantage in that it compared actual symbols, not just the number of them.
Note how this machine recorded the symbols before the marker (#) with discs
and then matched them against the symbols following the marker. Since the
input was completely read and the pile was empty, the machine accepted.

Now, here is what happens when the string abaa#aaab is processed:

Input: a b a a # a a a
red red
Pile: red red red red
blue blue blue blue blue blue
red red red red red red red

In this case, the machine stopped with ab yet to read and discs on the pile since
it could not match an a with the blue disc. So, it rejected the input string. Try
some more examples as exercises.

The machines we designed algorithms for above in figures 1 and 2 are usually
called pushdown automata. All they are is finite automata with auxiliary
storage devices called stacks. (A stack is merely a pile. And symbols are
normally placed on stacks rather than various colored discs.) The rules
involving stacks and their contents are:

a) Symbols must always be placed upon the top of the stack.

b) Only the top symbol of a stack can be read.
c) No symbol other than the top one can be removed.

We call placing a symbol upon the stack a push operation and removing one
from the top of the stack a pop operation.
Pushdown Automata 4

Figure 3 provides a picture of one of these machines.

i n p u t t a p e

f in it e
s
contro l
t
a
c
k

Figure 3 - A Pushdown Automaton

Pushdown automata can be presented as state tables in very much the same
way as finite automata. All we need to add is the ability to place (or push)
symbols on top of the stack and to remove (or pop) symbols from the top of the
stack. Here is a state table for our machine of figure 1 which accepts strings of
the form anbn.

state read pop push goto

a A 1
1
b A 2
2 b A 2

Note that this machine operates exactly the same as that of the algorithm in
figure 1. During operation, it:

a) reads a's and pushes A's on the stack in state 1,

b) reads b's and pops A's from the stack in state 2, and
c) accepts if the stack is empty at the end of the input string.

Thus, the states of the pushdown machine perform exactly the same as the
while loops in the algorithm presented in figure 1.

If a pushdown machine encounters a configuration which is not defined (such

as the above machine being in state 2 reading an a, or any machine trying to
pop a symbol from an empty stack) then computation is terminated and the
machine rejects. This is very similar to Turing machine conventions.

A trace of the computation for the above pushdown automaton on the input
aaabbb is provided in the following picture:
Pushdown Automata 5

Input Remaining: aaabbb aabbb abbb bbb bb b

State: s1 s1 s1 s1 s2 s2 s2

A
Stack: A A A
A A A A A

and a trace for the input aaabb is:

Input Remaining: aaabb aabb abb bb b

State: s1 s1 s1 s1 s2 s2

A
Stack: A A A
A A A A A

In the first computation (for input aaabbb), the machine ended up in state s2
with an empty stack and accepted. The second example ended with an A on the
stack and thus the input aaabb was rejected. If the input was aaabba then the
following would take place:

Input Remaining: aaabba aabba abba bba ba a

State: s1 s1 s1 s1 s2 s2

A
Stack: A A A
A A A A A

In this case, the machine terminates computation since it does not know what
to do in s2 with an a to be read on the input tape. Thus aaabba also is rejected.

Our second machine example (figure 2) has the following state table.

state read pop push goto

a A 1
1 b B 1
# 2
a A 2
2
b B 2

Note that it merely records a's and b's on the stack until it reaches the marker
(#) and then checks them off against the remainder of the input.
Pushdown Automata 6

Now we are prepared to precisely define our new class of machines.

Definition. A pushdown automaton (pda) is a quintuple

M = (S, Σ, Γ, δ, s0), where:

S is a finite set (of states),

Σ is a finite (input) alphabet,
Γ is a finite (stack) alphabet,
δ: SxΣxΓ ∪ {ε} → Γ*xS (transition function),
and s0 ∈ S (the initial state).

In order to define computation we shall revert to the conventions used with

Turing machines. A configuration is a triple <s, x, α> where s is a state, x a
string over the input alphabet, and α a string over the stack alphabet. The
string x is interpreted as the input yet to be read and α is of course the content
of the stack. One configuration yields another (written Ci → Ck ) when applying
the transition function to it results in the other. Some examples from our first
machine example are:

<s1, aaabbb, ε> → <s1, aabbb, A>

<s1, aabbb, A> → <s1, abbb, AA>
<s1, abbb, AA> → <s1, bbb, AAA>
<s1, bbb, AAA> → <s2, bb, AA>
<s2, bb, AA> → <s2, b, A>
<s2, b, A> → <s2, ε, ε>

Note that the input string decreases in length by one each time a configuration
yields another. This is because the pushdown machine reads an input symbol
every time it goes through a step.

We can now define acceptance to take place when there is a sequence of

configurations beginning with one of the form <s0, x, ε> for the input string x
and ending with a configuration <si, ε, ε>. Thus a pushdown automaton accepts
when it finishes its input string with an empty stack.

There are other conventions for defining pushdown automata which are
equivalent to that proposed above. Often machines are provided with an initial
stack symbol Z0 and are said to terminate their computation whenever the stack
is empty. The machine of figure 2 might have been defined as:
Pushdown Automata 7

state read pop push goto

a Z0 A 1
0 b Z0 B 1
# Z0 2
a A 1
1 b B 1
# 2
a A 2
2
b B 2

if the symbol Z0 appeared upon the stack at the beginning of computation.

Including it in our original definition makes a pushdown automaton a sextuple
such as:

M = (S, Σ, Γ, δ, s0, Z0).

Some definitions of pushdown automata require the popping of a stack symbol

on every move of the machine. Our example might now become:

state read pop push goto

a Z0 A 1
0 b Z0 B 1
# Z0 2
a A AA 1
a B AB 1
b A BA 1
1
b B BB 1
# A A 2
# B B 2
a A 2
2
b B 2

where in state s1 the symbols which were popped are placed back upon the
stack.

Another very well known convention is to have pushdown machines accept by

final state. This means that the automaton must pass the end of the input in an
accepting or final state. Just like finite automata. Now we have a final state
subset and our machine becomes:

M = (S, Σ, Γ, δ, s0, Z0, F)

and our tuples get larger and larger.

Pushdown Automata 8

Converting this example to this format merely involves detecting when the
stack has only one symbol upon it and changing to an accepting state if things
are satisfactory at this point. We do this by placing special sentinels (X and Y)
on the bottom of the stack at the beginning of the computation. Here is our
example with s3 as an accepting state. (Note that the machine accepts by empty
stack also!).

state read pop push goto

a X 1
0 b Y 1
# 3
a A 1
1 b B 1
# 2
a A 2
a X 3
2
b B 2
b Y 3

All of these conventions are equivalent (the proofs of this are left as exercises)
and we shall use any convention which seems to suit the current application.

Now to get on with our examination of the exciting new class of machines we
have defined. Our first results compare them to other classes of automata we
have studied.

Theorem 1. The class of sets accepted by pushdown automata properly

includes the regular sets.

Proof. This is very easy indeed. Since finite automata are just pushdown
machines which do not ever use their stacks, all of the regular sets can be
accepted by pushdown machines which accept by final state. Since
strings of the form an bn can be accepted by a pushdown machine (but
not by any finite automaton), the inclusion is proper.

Theorem 2. All of the sets accepted by pushdown automata are recursive.

Proof. For the same reason that the regular sets are recursive. Pushdown
machines are required to process an input symbol at each step of their
computation. Thus we can simulate them and see if they accept.

Corollary. The class of recursively enumerable sets properly contains the

class of sets accepted by pushdown automata.
Pushdown Automata 9

As you may have noticed (and probably were quite pleased about), the
pushdown machines designed thus far have been deterministic. These are
usually called dpda's and the nondeterministic variety are known as npda's. As
with nondeterministic finite automata, npda's may have several possible moves
for a configuration. The following npda accepts strings of the form wwR by
nondeterministicly deciding where the center of the input string lies.

state read pop push goto

a A 0
a A 1
0
b B 0
b B 1
1 a A 1
b B 1

Note that in s0 there are choices of either to remain in s0 and process the w
part of wwR or to go to s1 and process the wR portion of the input. Here is an
example computation of the machine on the string abbbba.

Input Remaining: abbbba bbbba bbba bba ba a

State: s0 s0 s0 s0 s1 s1 s1

B
Stack: B B B
A A A A A

One should note that the machine does in fact change from s0 to s1 when bba
remains to be read and a B is on top of the stack. We shall not prove it here, but
nondeterminism does indeed add power to pushdown machines. In fact, the set
of strings of the form wwR cannot be accepted by a deterministic pushdown
automaton because it would have no way to detect where the center of the input
occurs.

Our last example of a set which can be accepted by a pushdown automaton is a

set of simple arithmetic expressions. Let's take v as a variable, + as an operator,
and put them together (in the proper manner) with parentheses. We get
expressions such as:

v+v+v, or v+(v+v), or (v+v)+(v+v)

but not expressions like:

v+v+v+, or (v+v)(v+v), or (v+(v+(v+v)).

Pushdown Automata 10

The strategy for the machine design is based on the method in which these
simple expressions are generated. Since an expression can be:

a) a variable,
b) a variable, followed by a plus, followed by an expression, or
c) an expression enclosed in parentheses.

Here is a simple but elegant, one state, nondeterministic machine which decides
which of the above three cases is being used and then verifies it. The machine
begins computation with the symbol E upon its stack.

read pop push

v E OE
v E
( E EP
+ O
) P

With a little effort this nondeterministic machine can be turned into a

deterministic machine. Then, more arithmetic operators (such as subtraction,
multiplication, etc.) can be added. At this point we have a major part of a
parser for assignment statements in programming languages. And, with some
output we could generate code exactly as compilers do. This is discussed in the
treatment on formal languages.
 Unsolvable Problems for Pushdown Automata
Soon after introducing pushdown automata we proved that their membership
problem was solvable. Well, this was not quite true since we did it only for
deterministic machines. But, it should not be too difficult to extend this result
to the nondeterministic variety. Soon we shall when we examine formal
languages. Also at that time we shall show that the emptiness and finiteness
problems for pushdown automata are solvable too. Unfortunately most of the
other decision problems for these machines are unsolvable. So, it would appear
that stacks produce some unsolvability.

As usual we shall use reducibility. We shall map some undecidable Turing

machine (or actually r.e. set) problems into pushdown automata problems. To
do these reductions we need to demonstrate that pushdown automata can
analyze Turing machine computations. Recall from before that a Turing
machine configuration is a string that describes exactly what the machine is
doing at the moment. It includes:

a) what is written on the tape,

b) which symbol the machine is reading, and
c) the instruction the Turing machine is about to execute.

If we dispense with endmarkers and restrict ourselves to the two symbol

alphabet {0, 1, b} then a configuration is a string of the form x(Ik)y or x where x
and y are strings over the alphabet {0, 1, b}. (We interpret the configuration
x(Ik)y to mean that the string xy is on the tape, the machine is about to execute
instruction Ik, and is reading the first symbol of the string y. A configuration of
the form x with no instruction in it means that the machine has halted with x
on its tape.) Let us prove some things as we develop the ability of pushdown
machines to examine Turing machine computations.

Lemma. The set of strings that are valid Turing machine configurations is
a regular set.

Proof. This is simple because a Turing machine configuration is of the

form [0+1+b]* or of the form [0+1+b]*(I1[0+1]*)[0+1+b]*. (Note that we
used square brackets in our expressions because the parentheses were
present in the Turing machine configurations.)

Since these are regular expressions and represent a regular set.

Unsolvable Problems for PDA 2

A computation is merely a sequence of configurations (Ci) separated by markers.

It can be represented by a string of the form:

C1#C2# ... #Cn

Since the regular sets are closed under concatenation and Kleene star these
kinds of strings form a regular set also.

Now let us examine just what makes a sequence of valid Turing machine
configurations not a valid halting computation for a particular machine. Either:

1) C1 is not an initial configuration,

2) Cn is not a halting configuration,
3) one of the Ci contains an improper instruction, or
4) for some i < n, some Ci does not yield Ci+1.

Parts (1) and (2) are easy to recognize. An initial configuration is merely a string
of the form (I1)[0+1+b]* and a halting configuration is a string of the form
[0+1+b]*. These are both regular sets. Part (3) is regular also because Turing
machine instructions are required to be members of a sequence beginning at I1
and there can be only a finite number of them for a particular Turing machine.
Thus this is regular also because a finite automata can recognize finite sets.

Let us now put this in the form of a lemma that we shall use later when we wish
to build a pushdown machine which check Turing machine computations.

Lemma. For each Turing machine there are finite automata that are able
to detect:
a) initial configurations,
b) halting configurations, and
c) configurations with improper instructions.

The final situation that leads to a string not being a valid Turing machine
computation is part (4) of our previous list. This when a configuration does not
yield the next in the sequence. We shall show now that a nondeterministic
pushdown machine can detect this.

Lemma. For each Turing machine, there is a pushdown automaton that

accepts pairs of configurations (for that Turing machine) such that the
first does not yield the second.

Proof. Let C1 and C2 be valid configurations for a particular Turing

machine. We shall build a pushdown automaton which accepts the input
C1#C2 whenever it is not the case that C1 → C2. The automaton will
Unsolvable Problems for PDA 3

operate in a nondeterministic manner by first selecting the reason that C1

does not yield C2, and then verifying that this happened. Let us analyze
the reasons why C1 might not yield C2.

a) No configuration can follow C1. There are three possible reasons for
this, namely:

1) C1 is a halting configuration.
2) The instruction in C1 is not defined for the symbol read.
3) C1 = (Ik)x and the Turing machine wishes to move left.

Since these three conditions are finite, a finite automaton could detect
them, and so can our pushdown machine.

b) The instruction in C2 is incorrect. In other words it does not follow

from the situation in C1. This can also be detected by a finite
automaton.

c) The tape symbols in C2 are wrong. Suppose that

C1 = x a ( I k ) c y

and instruction Ik calls for the Turing machine to write the symbol d
and transfer to instruction Im if it reads the symbol c. Then it should
be the case that

C2 = x ( I m ) a d y

if the Turing machine moved left, and

C2 = x a d ( I m ) y

if the Turing machine moved a square to the right. Suppose that

instead of things being as they should, we end up with the following
configuration:

C2 = u ( I m ) v
Unsolvable Problems for PDA 4

The errors that could take place have been tabulated in the following
table.

left move error: u≠x or v ≠ ady

right move error: u ≠ xad or v≠y

The pushdown machine first nondeterministicly selects which of the four

errors has taken place and then verifies it. Here is how this works. Our
pushdown automaton first scans xacy (the tape in C1) and uses its stack
as a counter to select the position of where the error will be in C2. It then
remembers (in its finite control) exactly what symbol should be in that
location on C2. Now it counts off squares of C2 until it reaches the site of
the error and verifies that the wrong symbol is in that location.

To recap, our pushdown automaton first selects the error that makes C1
not yield C2 and then verifies that it did indeed take place.

Now we possess all of the tools necessary to show that pushdown machines can
analyze strings and detect whether or not they are Turing machine
computations. This is done by enumerating what could be making the string
not a proper computation, and then verifying that something of this sort took
place.

Theorem 1. For every Turing machine, there is a pushdown automaton

that accepts all strings that are not valid halting computations for the
Turing machine.

Proof Sketch. The pushdown machine in question merely selects

(nondeterministicly) the reason that its input cannot be a valid halting
computation for the Turing machine. Then it verifies that this error took
place. The preceding lemmas provide the list of errors and the methods
needed to detect them.

This theorem is very interesting in its on right. as well as being a nice example
of nondeterministic computation. In fact, the guess and verify strategy cannot
be carried out on a deterministic device unless the choices remain constant no
matter how long the input gets.

It also indicates that pushdown automata can in some sense analyze the
computations of Turing machines. (By the way, npda cannot detect valid halting
computations for Turing machines. This should become clear after observing
linear bounded automata that is a more powerful device that is able to
recognize valid computations.) The major use of this theorem, however, is to
Unsolvable Problems for PDA 5

prove that several decision problems involving pushdown machines are

unsolvable. This will be done by reducing unsolvable problems for the r.e. sets
to problems for pushdown automata.

Theorem 2. Whether or not a pushdown automaton accepts every string

over its alphabet is unsolvable.

Proof. We shall reduce the emptiness problem for Turing machines (is Wi
= ∅?) to this problem (known as the Σ* problem since Σ is the input
alphabet) for pushdown machines.

Let Mi be a Turing machine and let Pg(i) be the pushdown automaton that
accepts the set of strings that are not valid halting computations for
Turing machine Mi. Then note that:

Wi = ∅ iff ∀x[Mi(x) never halts]

iff ∀ax[Mi(x) has no halting computations]
iff Pg(i) accepts every input

That was the reduction from the emptiness problem for Turing machines
to the Σ* problem for pushdown machines. Recalling our results on
reducibilities indicates that both must be unsolvable. (Also they are not
r.e.)

Acceptance of strings that are not valid halting computations of Turing

machines leads to several other unsolvability results concerning pushdown
automata. Two which appear in the exercises are the equivalence problem
(whether two machines accept the same set) and the cofiniteness problem
(whether the complement of the set accepted by a pushdown machine is finite).
They are proven in the same manner.
 Linear Bounded Automata
The last machine model of computation which we shall examine is the linear
bounded automaton or lba. These were originally developed as models for
actual computers rather than models for the computational process. They have
become important in the theory of computation even though they have not
emerged in applications to the extent which pushdown automata enjoy.

Here is the motivation for the design of this class of machines. Computers are
finite devices. They do not have unending amounts of storage like Turing
machines. Thus any actual computation done on a computer is not as extensive
as that which could be completed on a Turing machine. So, to mimic (or maybe
model) computers, we must restrict the storage capacity of Turing machines.
This should not be as severely as we did for finite automata though. Here is the
definition.

Definition. A linear bounded automaton (lba) is a multi-track Turing

machine which has only one tape, and this tape is exactly the same length
as the input.

That seems quite reasonable. We allow the computing device to use just the
storage it was given at the beginning of its computation. As a safety feature, we
shall employ endmarkers (* on the left and # on the right) on our lba tapes and
never allow the machine to go past them. This will ensure that the storage
bounds are maintained and help keep our machines from leaving their tapes.

At this point, the question of accepting sets arises. Let's have linear bounded
automata accept just like Turing machines. Thus for lba halting means
accepting.

For these new machines computation is restricted to an area bounded by a

constant (the number of tracks) times the length of the input. This is very
much like a programming environment where the sizes of values for variables is
bounded.

Now that we know what these devices are, let's look at one. A set which cannot
be accepted by pushdown machines (this is shown in the material on formal
languages) is the set of strings whose length is a perfect square. In symbols this
is:

{an | n is a perfect square }.

Linear Bounded Automata 2

Here is the strategy. We shall use a four track machine with the input written
on the first track. The second and third tracks are used for scratch work while
the fourth track is holds strings of square length which will be matched against
the input string.

To do this we need to generate some strings of square length. The second and
third tracks are used for this. On the second track we will build strings of length
k = 1, 2, 3, and so forth. After each string is built, we construct (on the fourth
track) a string whose length is the square of the length of the string on the
second track by copying the second track to the fourth exactly that many times.
The third track is used to count down from k. Here is a little chart which
explains the use of the tracks.

track content
1 an (input)
2 ak
3 ak-m
4 amk

Then we check to see if this is the same length as the input. The third track is
used for bookkeeping. The algorithm is provided as figure 1.

repeat
clear the 3 rd and 4th tracks
add another a to the 2 nd track
copy the 2nd track to the 3rd track
while there are a’s written on the 3 rd track
delete an a from the 3rd track
add the 2 nd track's a's to those on 4th track
until overflow takes place or 4th track = input
if there was no overflow then accept

Figure 1 - Recognition of Perfect Square Length Inputs

Now we've seen something of what can be done by linear bounded automata.
We need to investigate some of the decision problems concerning them. The
first problem is the halting problem.

Theorem 1. The halting problem is solvable for linear bounded automata.

Proof. Our argument here will be based upon the number of possible
configurations for an lba. Let's assume that we have an lba with one track
(this is allowed because can use additional tape symbols to simulate
tracks as we did with Turing machines), k instructions, an alphabet of s
Linear Bounded Automata 3

tape symbols, and an input tape which is n characters in length. An lba

configuration is the same as a Turing machine configuration and consists
of:

a) an instruction,
b) the tape head's position, and
c) the content of the tape.

That is all. We now ask: how many different configurations can there be?
It is not too difficult to figure out. With s symbols and a tape which is n
squares long, we can have only sn different tapes. The tape head can be
on any of the n squares and we can be executing any of the k instructions.
Thus there are only

k*n*sn

possible different configurations for the lba.

Let us return to a technique we used to prove the pumping lemma for

finite automata. We observe that if the lba enters the same configuration
twice then it will do this again and again and again. It is stuck in a loop.

The theorem follows from this. We only need to simulate and observe the
lba for k*n*sn steps. If it has not halted by then it must be in a loop and
will never halt.

Corollary. The membership problems for sets accepted by linear bounded

automata are solvable.

Corollary. The sets accepted by linear bounded automata are all

recursive.

Let's pursue this notion about step counting a bit more. We know that an lba
will run for no more than k&n&sn steps because that is the upper bound on the
number of configurations possible for a machine with an input of length n. But,
let us ask: exactly how many configurations are actually reached by the
machine? If we knew (and we shall soon) we would have a sharper bound on
when looping takes place. Thus to detect looping, we could count steps with an
lba by using an extra track as a step counter.

Now let's make the problem a little more difficult. Suppose we had a
nondeterministic linear bounded automaton (nlba). We know what this is;
merely a machine which has more than one possible move at each step. And, if
it can achieve a halting configuration, it accepts. So we now ask: how many
configurations can an nlba reach for some input? We still have only k*n*sn
Linear Bounded Automata 4

possible configurations, so if we could detect them we could count them using

n tape squares. The big problem is how to detect them. Here is a rather nifty
result which demonstrates nondeterminism in all of its glory. We start with a
series of lemmata.

Lemma. For any nondeterministic linear bounded automaton there is

another which can locate and examine m configurations reachable (by the
first lba) from some input if there are at least m reachable configurations.

Proof. We have an nlba and an integer m. In addition we know that there

are at least m configurations reachable from a certain input. Our task is
to find them.

If the nlba has k instructions, one track, s symbols, and the input is
length n, then we know that there are at most k&n&sn possible
configurations (Ci) reachable from the starting configuration (which we
shall call C0). We can enumerate them and check whether the nlba can
get to them. Consider:

x = 0
for i = 1 to k*n*sn
generate C i
guess a path from C0 to Ci
verify that it is a proper path
if Ci is reachable then x = x + 1
verify that x ≥ m (otherwise reject)

This is a perfect example of the guess and verify technique used in

nondeterministic operation. All we did was exploit our definition of
nondeterminism. We looked at all possible configurations and counted
those which were reachable from the starting configuration.

Note also that every step above can be carried out using n tape squares
and several tracks. Our major problem here is to count to k*n*sn. We
need to first note that for all except a few values of n, this is smaller than
(s+1)n and we can count to this in base s+1using exactly n tape squares.

Since we verify that we have indeed found at least m configurations our

algorithm does indeed examine the appropriate number of reachable
configurations if they exist.
Linear Bounded Automata 5

Lemma. For any nondeterministic linear bounded automaton there is

another which can compute the number of configurations reachable from
an input.

Proof. As before we begin with an arbitrary machine which has k

instructions, one track, s symbols, and an input of length n. We shall
iteratively count the number of configurations (ni) reachable from the
initial configuration. Consider:

n0 = 1
i = 0
repeat
i = i + 1
ni = 0
m := 0
for j = 1 to k*n*sn
generate Cj
guess whether C j can be reached in i steps or less
if path from C0 to C j is verifiable then
ni = ni + 1
if reached in less than i steps then m = m + 1
verify that m = n i-1 (otherwise reject)
until n i = n i-1

The guessing step is just the algorithm of our last lemma. We do it by

finding all of the configurations reachable in less than i steps and seeing
if any of them is Cj or if one more step will produce Cj. Since we know ni-
1, we can verify that we have looked at all of them.

The remainder is just counting. We do not of course have to save all of

the ni, just the current one and the last one. All of this can be done on n
squares of tape (and several tracks). Noting that we are done when no
more reachable configurations can be found finishes the proof.

Theorem 2. The class of sets accepted by nondeterministic linear bounded

automata is closed under complement.

Proof. Most of our work has been done. To build a machine which
accepts the complement of the set accepted by some nlba involves
putting the previous two together. First find out exactly how many
configurations are reachable. Then examine all of them and if any halting
configurations are encountered, reject. Otherwise accept.
Linear Bounded Automata 6

Our final topic is decision problems. Unfortunately we have seen the only
important solvable decision problem concerning linear bounded automata. (At
least there was one!) The remaining decision problems we have examined for
other classes of machines are unsolvable. Most of the proofs of this depend
upon the next lemma.

Lemma. For every Turing machine there is a linear bounded automaton

which accepts the set of strings which are valid halting computations for
the Turing machine.

The proof of this important lemma will remain an exercise. It should not be too
hard to see just how an lba could check a string to see if it is a computation
though. After all, we did a rather careful analysis of how pushdown machines
recognize invalid computations.

Theorem 3. The emptiness problem is unsolvable for linear bounded

automata.

Proof. Note that if a Turing machine accepts no inputs then it does not
have any valid halting computations. Thus the linear bounded automaton
which accepts the Turing machine's valid halting computations accepts
nothing. This means that if we could solve the emptiness problem for
linear bounded automata then we could solve it for Turing machines.

In the treatment of formal languages we shall prove that the class of sets
accepted by linear bounded automata properly contains the class of sets
accepted by pushdown machines. This places this class in the hierarchy

fa ⊂ pda ⊂ lba ⊂ TM

of classes of sets computable by the various machine models we have been

examining.

(By the way, we could intuitively indicate why lba's are more powerful than
pushdown machines. Two observations are necessary. First, a tape which can
be read and written upon is as powerful a tool as a stack. Then, note that a
pushdown machine can only place a bounded number of symbols on its stack
during each step of its computation. Thus its stack cannot grow longer than a
constant times the length of its input.)

The other relationship we need is not available. Nobody knows if

nondeterministic linear bounded automata are more powerful than ordinary
ones.