10 1 1 37 307
10 1 1 37 307
10 1 1 37 307
Finite Automata
and Regular Languages
3.1 Introduction
3.1.1 States and Automata
A finite-state machine or finite automaton (the noun comes from the Greek;
the singular is automaton, the Greek-derived plural is automata,
although automatons is considered acceptable in modern English) is a
limited, mechanistic model of computation. Its main focus is the notion
of state. This is a notion with which we are all familiar from interaction
with many different controllers, such as elevators, ovens, stereo systems,
and so on. All of these systems (but most obviously one like the elevator)
can be in one of a fixed number of states. For instance, the elevator can be
on any one of the floors, with doors open or closed, or it can be moving
between floors; in addition, it may have pending requests to move to certain
floors, generated from inside (by passengers) or from outside (by would-be
passengers). The current state of the system entirely dictates what the system
does nextsomething we can easily observe on very simple systems such as
single elevators or microwave ovens. To a degree, of course, every machine
ever made by man is a finite-state system; however, when the number of
states grows large, the finite-state model ceases to be appropriate, simply
because it defies comprehension by its usersnamely humans. In particular,
while a computer is certainly a finite-state system (its memory and registers
can store either a 1 or a 0 in each of the bits, giving rise to a fixed number
of states), the number of states is so large (a machine with 32 Mbytes of
43
44
q1
q2
q3
q1
0,1
q2
q3
Figure 3.1
3.1 Introduction
q1
q2
q3
0
q2
q3
q3
1
q2
q2
Figure 3.2 The transition function for the automaton of Figure 3.1(b).
45
46
Figure 3.3 An automaton that accepts strings with an even number of 1s.
0,1
0,1
0,1
while that of Figure 3.4(b) accepts everything except the empty string. This
last construction may suggest that, in order to accept the complement of a
language, it suffices to flip the labels assigned to states, turning rejecting
states into accepting ones and vice versa.
Exercise 3.1 Decide whether this idea works in all cases.
0
1
0
1
1
0,1
3.1 Introduction
has entered this state, it cannot leave it. This particular trap is a rejecting
state; the automaton of Figure 3.4(b) had an accepting trap.
We are now ready to give a formal definition of a finite automaton.
Definition 3.1 A deterministic finite automaton is a five-tuple, (6, Q, q0 , F,
), where 6 is the input alphabet, Q the set of states, q0 Q the start state,
F Q the final states, and : Q 6 Q the transition function.
h
Our choice of the formalism for the transition function actually makes
the automaton deterministic, conforming to the examples seen so far.
Nondeterministic automata can also be definedwe shall look at this
distinction shortly.
Moving from a finite automaton to a description of the language that it
accepts is not always easy, but it is always possible. The reverse direction
is more complex because there are many languages that a finite automaton
cannot recognize. Later we shall see a formal proof of the fact, along with
an exact characterization of those languages that can be accepted by a finite
automaton; for now, let us just look at some simple examples.
Consider first the language of all strings that end with 0. In designing
this automaton, we can think of its having two states: when it starts or
after it has seen a 1, it has made no progress towards acceptance; on the
other hand, after seeing a 0 it is ready to accept. The result is depicted in
Figure 3.6.
Consider now the set of all strings that, viewed as natural numbers in
unsigned binary notation, represent numbers divisible by 5. The key here is
to realize that division in binary is a very simple operation with only two
possible results (1 or 0); our automaton will mimic the longhand division
by 5 (101 in binary), using its states to denote the current value of the
remainder. Leading 0s are irrelevant and eliminated in the start state (call it
A); since this state corresponds to a remainder of 0 (i.e., an exact division by
5), it is an accepting state. Then consider the next bit, a 1 by assumption. If
the input stopped at this point, we would have an input value and thus also
a remainder of 1; call the state corresponding to a remainder of 1 state Ba
rejecting state. Now, if the next bit is a 1, the input (and also remainder)
0
1
1
Figure 3.6
47
48
D
1
0
A
0
1
1
B
3.1.3
In all of the fully worked examples of finite automata given earlier, there
was exactly one transition out of each state for each possible input symbol.
That such must be the case is implied in our formal definition: the transition
function is well defined. However, in our first example of transitions
(Figure 3.2), we looked at an automaton where the transition function
remained undefined for one combination of current state and current input,
that is, where the transition function did not map every element of
its domain. Such transition functions are occasionally useful; when the
automaton reaches a configuration in which no transition is defined, the
standard convention is to assume that the automaton aborts its operation
and rejects its input string. (In particular, a rejecting trap has no defined
transitions at all.) In a more confusing vein, what if, in some state, there
had been two or more different transitions for the same input symbol?
Again, our formal definition precludes this possibility, since (qi , a) can
have only one value in Q; however, once again, such an extension to our
mechanism often proves useful. The presence of multiple valid transitions
leads to a certain amount of uncertainty as to what the finite automaton
will do and thus, potentially, as to what it will accept. We define a finite
automaton to be deterministic if and only if, for each combination of state
and input symbol, it has at most one transition. A finite automaton that
3.1 Introduction
allows multiple transitions for the same combination of state and input
symbol will be termed nondeterministic.
Nondeterminism is a common occurrence in the worlds of particle
physics and of computers. It is a standard consequence of concurrency:
when multiple systems interact, the timing vagaries at each site create an
inherent unpredictability regarding the interactions among these systems.
While the operating system designer regards such nondeterminism as
both a boon (extra flexibility) and a bane (it cannot be allowed to
lead to different outcomes, a catastrophe known in computer science
as indeterminacy, and so must be suitably controlled), the theoretician
is simply concerned with suitably defining under what circumstances a
nondeterministic machine can be termed to have accepted its input. The
key to understanding the convention adopted by theoreticians regarding
nondeterministic finite automata (and other nondeterministic machines) is
to realize that nondeterminism induces a tree of possible computations for
each input string, rather than the single line of computation observed in a
deterministic machine. The branching of the tree corresponds to the several
possible transitions available to the machine at that stage of computation.
Each of the possible computations eventually terminates (after exactly n
transitions, as observed earlier) at a leaf of the computation tree. A stylized
computation tree is illustrated in Figure 3.8. In some of these computations,
the machine may accept its input; in others, it may reject iteven though it is
the same input. We can easily dispose of computation trees where all leaves
correspond to accepting states: the input can be defined as accepted; we
can equally easily dispose of computation trees where all leaves correspond
to rejecting states: the input can be defined as rejected. What we need to
address is those computation trees where some computation paths lead
to acceptance and others to rejection; the convention adopted by the
branching point
leaf
Figure 3.8
49
50
(evidently optimistic) theory community is that such mixed trees also result
in acceptance of the input. This convention leads us to define a general
finite automaton.
Definition 3.2 A nondeterministic finite automaton is a five-tuple, (6, Q,
q0 , F, ), where 6 is the input alphabet, Q the set of states, q0 Q the start
state, F Q the final states, and : Q 6 2 Q the transition function. h
Note the change from our definition of a deterministic finite automaton:
the transition function now maps Q 6 to 2 Q , the set of all subsets of Q,
rather than just into Q itself. This change allows transition functions that
map state/character pairs to zero, one, or more next states. We say that
a finite automaton is deterministic whenever we have |(q, a)| 1 for all
q Q and a 6.
Using our new definition, we say that a nondeterministic machine
accepts its input whenever there is a sequence of choices in its transitions
that will allow it to do so. We can also think of there being a separate
deterministic machine for each path in the computation treein which
case there need be only one deterministic machine that accepts a string for
the nondeterministic machine to accept that string. Finally, we can also
view a nondeterministic machine as a perfect guesser: whenever faced with
a choice of transitions, it always chooses one that will allow it to accept the
input, assuming any such transition is availableif such is not the case, it
chooses any of the transitions, since all will lead to rejection.
Consider the nondeterministic finite automaton of Figure 3.9, which
accepts all strings that contain one of three possible substrings: 000, 111,
or 1100. The computation tree on the input string 01011000 is depicted
in Figure 3.10. (The paths marked with an asterisk denote paths where
the automaton is stuck in a state because it had no transition available.)
There are two accepting paths out of ten, corresponding to the detection
of the substrings 000 and 1100. The nondeterministic finite automaton thus
accepts 01011000 because there is at least one way (here two) for it to do
0,1
B
0
A
0
F
1
1
D
0,1
3.1 Introduction
A
A
A
A
A
A
A
B
D *
B * *
D
* * *
D B E * * *
B * C *
* * *
B C * F *
* * *
A B C F * F *
* * *
Figure 3.10 The computation tree for the automaton of Figure 3.9 on
input string 01011000.
so. For instance, it can decide to stay in state A when reading the first three
symbols, then guess that the next 1 is the start of a substring 1100 or 111
and thus move to state D. In that state, it guesses that the next 1 indicates
the substring 1100 rather than 111 and thus moves to state B rather than E.
From state B, it has no choice left to make and correctly ends in accepting
state F when all of the input has been processed. We can view its behavior
as checking the sequence of guesses (left, left, left, right, left, , , ) in the
computation tree. (That the tree nodes have at most two children each is
peculiar to this automaton; in general, a node in the tree can have up to |Q|
children, one for each possible choice of next state.)
When exploiting nondeterminism, we should consider the idea of
choice. The strength of a nondeterministic finite automaton resides in its
ability to choose with perfect accuracy under the rules of nondeterminism.
For example, consider the set of all strings that end in either 100 or in
001. The deterministic automaton has to consider both types of strings
and so uses states to keep track of the possibilities that arise from either
suffix or various substrings thereof. The nondeterministic automaton can
simply guess which ending the string will have and proceed to verify the
guesssince there are two possible guesses, there are two verification paths.
The nondeterministic automaton just gobbles up symbols until it guesses
that there are only three symbols left, at which point it also guesses which
ending the string will have and proceeds to verify that guess, as shown in
Figure 3.11. Of course, with all these choices, there are many guesses that
51
52
Figure 3.11
lead to a rejecting state (guess that there are three remaining symbols when
there are more, or fewer, left, or guess the wrong ending), but the string
will be accepted as long as there is one accepting path for it.
However, this accurate guessing must obey the rules of nondeterminism:
the machine cannot simply guess that it should accept the string or guess that
it should reject itsomething that would lead to the automaton illustrated
in Figure 3.12. In fact, this automaton accepts 6 , because it is possible for
it to accept any string and thus, in view of the rules of nondeterminism, it
must then do so.
Figure 3.12
3.1.4
3.1 Introduction
53
54
6 = 6
Q = 2Q
F = {s Q | s F 6= }
q0 = {q0 }
The key idea is to define one state of the deterministic machine for each
possible combination of states of the nondeterministic onehence the 2|Q|
possible states of the equivalent deterministic machine. In that way, there
is a unique state for the deterministic machine, no matter how many computation paths exist at the same step for the nondeterministic machine. In
order to define , we recall that the purpose of the simulation is to keep
track, in the state of the deterministic machine, of all computation paths of
the nondeterministic one. Let the machines be at some step in their computation where the next input symbol is a. If the nondeterministic machine can
be in any of states qi1 , qi2 , . . . , qik at that stepso that the corresponding
deterministic machine is then in state {qi1 , qi2 , . . ., qik }then it can move to
any of the states contained in the sets (qi1 , a), (qi2 , a), . . ., (qik , a)so
that the corresponding deterministic machine moves to state
({qi1 , qi2 , . . ., qik }, a) =
k
[
(qi j , a)
j =1
(a, 0) = {a, b}
(b, 0) = {b}
(a, 1) = {b}
(b, 1) = {a}
0
1,0
b
1
{b}
{a,b}
0,1
0,1
55
56
6 = {0, 1}, Q = {, {a}, {b}, {a, b}}, F = {{a}, {a, b}}, q0 = {a},
:
(, 0) =
({a}, 0) = {a, b}
({b}, 0) = {b}
({a, b}, 0) = {a, b}
(, 1) =
({a}, 1) = {b}
({b}, 1) = {a}
({a, b}, 1) = {a, b}
Thus the conversion of a nondeterministic automaton to a deterministic one creates a machine, the states of which are all the subsets of the
set of states of the nondeterministic automaton. The conversion takes a
nondeterministic automaton with n states and creates a deterministic automaton with 2n states, an exponential increase. However, as we saw briefly,
many of these states may be useless, because they are unreachable from the
start state; in particular, the empty state is always unreachable. In general,
the conversion may create any number of unreachable states, as shown in
Figure 3.14, where five of the eight states are unreachable. When generating a deterministic automaton from a given nondeterministic one, we can
avoid generating unreachable states by using an iterative approach based
on reachability: begin with the initial state of the nondeterministic automaton and proceed outward to those states reachable by the nondeterministic
automaton. This process will generate only useful statesstates reachable
from the start stateand so may be considerably more efficient than the
brute-force generation of all subsets.
0,1
0,1
Figure 3.14
0
0
B
1
1
1
0
AB
0
0
ABC
1
AC
0,1
1
BC
3.2.2 Transitions
An transition is a transition that does not use any inputa spontaneous
transition: the automaton simply decides to change states without reading any symbol.
Such a transition makes sense only in a nondeterministic automaton: in
a deterministic automaton, an transition from state A to state B would
have to be the single transition out of A (any other transition would induce
a nondeterministic choice), so that we could merge state A and state B,
simply redirecting all transitions into A to go to B, and thus eliminating
the transition. Thus an transition is essentially nondeterministic.
Example 3.2 Given two finite automata, M1 and M2 , design a new finite
automaton that accepts all strings accepted by either machine. The new
machine guesses which machine will accept the current string, then sends
the whole string to that machine through an transition.
h
The obvious question at this point is: Do transitions add power to
finite automata? As in the case of nondeterminism, our answer will be
no.
Assume that we are given a finite automaton with transitions; let its
transition function be . Let us define (q, a) to be the set of all states that
can be reached by
1. zero or more transitions; followed by
2. one transition on a; followed by
3. zero or more transitions.
This is the set of all states reachable from state q in our machine while
reading the single input symbol a; we call the -closure of .
In Figure 3.15, for instance, the states reachable from state q through
the three steps are:
1. {q, 1, 2, 3}
2. {4, 6, 8}
3. {4, 5, 6, 7, 8, 9, 10}
so that we get (q, a) = {4, 5, 6, 7, 8, 9, 10}
Theorem 3.2 For every finite automaton with transitions, there exists an
equivalent finite automaton without transitions.
h
We do not specify whether the finite automaton is deterministic or nondeterministic, since we have already proved that the two have equivalent
power.
57
58
a
b
a
q
2
b
a
a
b
a
b
8
b
a
10
59
60
61
62
For the regular expression denoting the set {}, the corresponding finite
automaton is
For the regular expression a denoting the set {a}, the corresponding finite
automaton is
a
MP
MQ
The transitions at the end are needed to maintain a unique accepting state.
MP
MQ
MP
Again, the extra transitions are here to maintain a unique accepting state.
It is clear that each finite automaton described above accepts exactly the
set of strings described by the corresponding regular expression (assuming
inductively that the submachines used in the construction accept exactly
the set of strings described by their corresponding regular expressions).
Since, for each constructor of regular expressions, we have a corresponding
constructor of finite automata, the induction step is proved and our proof
is complete.
Q.E.D.
We have proved that for every regular expression, there exists an equivalent
nondeterministic finite automaton with transitions. In the proof, we
chose the type of finite automaton with which it is easiest to proceed
the nondeterministic finite automaton. The proof was by constructive
induction. The finite automata for the basic pieces of regular expressions (,
, and individual symbols) were used as the basis of the proof. By converting
the legal operations that can be performed on these basic pieces into finite
automata, we showed that these pieces can be inductively built into larger
and larger finite automata that correspond to the larger and larger pieces of
the regular expression as it is built up. Our construction made no attempt
to be efficient: it typically produces cumbersome and redundant machines.
For an efficient conversion of regular expressions to finite automata, it
is generally better to understand what the expression is conveying, and
then design an ad hoc finite automaton that accomplishes the same thing.
However, the mechanical construction used in the proof was needed to
prove that any regular expression can be converted to a finite automaton.
63
64
3.3.3
2
0
1
0
1
3
Figure 3.16
65
66
Path Sets
Regular Expression
0
R11
= {}
0
R12 = {0}
0
R13
= {1}
0
R21 = { }
0
R22 = {, 1}
0
1
+1
...
+1
0
R33
Figure 3.17
...
= {, 1}
through state k without passing through any state higher than k 1, even
though k is higher than k 1. We can circumvent this difficulty by breaking
any path through state k every time it reaches state k, effectively splitting
the set of paths from i to j through k into three separate components, none
of which passes through any state higher than k 1. These components
are:
k1
Rik
, the paths that go from i to k without passing through a state
higher than k 1 (remember that entering the state at the end of the
path does not count as passing through the state);
k1
Rkk
, one iteration of any loop from k to k, without passing through
a state higher than k 1 (the paths exit k at the beginning and enter
k at the end, but never pass through k); and
k1
, the paths that go from state k to state j without passing through
Rkj
a state higher than k 1.
k1
The expression Rkk
describes one iteration of a loop, but this loop could
occur any number of times, including none, in any of the paths in Rikj .
The expression corresponding to any number of iterations of this loop
k1
therefore must be (Rkk
) . We now have all the pieces we need to build up
the inductive step from k 1 to k:
k1
k1 k1
Rikj = Rik1
+ Rik
(Rkk
) Rkj
j
k1 k1
k1
) Rkj .
(Rkk
Figure 3.18 illustrates the second term, Rik
With this inductive step, we can proceed to build all possible paths in
the machine (i.e., all the paths between every pair of nodes i and j for each
k
no k
Figure 3.18
j
no k
k from 1 to n) from the expressions for the base cases. Since the Rk s are built
from the regular expressions for the various Rk1 s using only operations
that are closed for regular expressions (union, concatenation, and Kleene
closurenote that we need all three operations!), the Rk s are also regular
expressions. Thus we can state that Rikj is a regular expression for any
value of i, j , and k, with 1 i, j, k n, and that this expression denotes
all paths (or, equivalently, strings that cause the automaton to follow these
paths) that lead from state i to state j while not passing through any state
numbered higher than k.
67
68
In the larger picture, this proof completes the proof of the equivalence
of regular expressions and finite automata.
proceed to paths that can pass (arbitrarily often) through state 1, then to
paths that can pass through states 1 and 2, and so on. This process looks
good until we remember that we want paths from the start state to an
accepting state: we may not be able to find such a path that also obeys our
requirements. Thus we should look not just at paths from the start state to
an accepting state, but at paths from any vertex to any other. Once we have
regular expressions for all source/target pairs, it will be simple enough to
keep those that describe paths from the start state to an accepting state.
Now we can formalize our induction: at step k of the induction, we
shall compute, for each pair (i, j ) of vertices, all paths that go from vertex i
through vertex j and that are allowed to pass through any of the vertices
numbered from 1 to k. If the starting vertex for these paths, vertex i, is
among the first k vertices, then we allow paths that loop through vertex i;
otherwise we allow the path only to leave vertex i but not see it again on
its way to vertex j . Similarly, if vertex j is among the first k vertices, the
path may go through it any number of times; otherwise the path can only
reach it and stop.
In effect, at each step of the induction, we define a new, somewhat larger
finite automaton composed of the first k states of the original automaton,
together with all transitions among these k states, plus any transition from
state i to any of these states that is not already included, plus any transition
to state j from any of these states that is not already included, plus any
transition from state i to state j , if not already included. Think of these
states and transitions as being highlighted in red, while the rest of the
automaton is blue; we can play only with the red automaton at any step of
the induction. However, from one step to the next, another blue state gets
colored red along with any transitions between it and the red states and
any transition to it from state i and any transition from it to state j . When
the induction is complete, k equals n, the number of states of the original
machine, and all states have been colored red, so we are playing with the
original machine.
To describe with regular expressions what is happening, we begin
by describing paths from i to j that use no intermediate state (no state
numbered higher than 0). That is simple, since such transitions occur either
under (when i = j ) or under a single symbol, in which case we just look
up the transition table of the automaton. The induction step simply colors
one more blue node in red. Hence we can add to all existing paths from
i to j those paths that now go through the new node; these paths can go
through the new node several times (they can include a loop that takes
them back to the new node over and over again) before reaching node j .
Since only the portion that touches the new node is new, we simply break
69
70
any such paths into segments, each of which leaves or enters the new node
but does not pass through it. Every such segment goes through only old red
nodes and so can be described recursively, completing the induction.
q0
x1
Figure 3.19
q1
x2
x|z|
qk
y
y
x
no loop
y
loop
t
tail
path through the automaton into three parts: an initial part that does not
contain any loop, the first loop encountered, and a final part that may or
may not contain additional loops. Figure 3.20 illustrates this partition. We
used x, y, and t to denote the three parts and further broke the loop into
two parts, y and y , writing y = y y y , so that the entire string becomes
x y y y t. Now we can go through the loop as often as we want, from zero
times (yielding x y t) to twice (yielding x y y y y y t) to any number of times
(yielding a string of the form x y (y y ) t); all of these strings must be in the
language. This is the spirit of the pumping lemma: you can pump some
string of unknown, but nonzero length, here y y , as many times as you
want and always obtain another string in the languageno matter what
the starting string z was (as long, that is, as it was long enough). In our case
the string can be viewed as being of the form uvw, where we have u = x y ,
v = y y , and w = t. We are then saying that any string of the form uv w is
also in the language. We have (somewhat informally) proved the pumping
lemma for regular languages.
Theorem 3.4 For every regular language L, there exists some constant n
(the size of the smallest automaton that accepts L) such that, for every string
z L with |z| n, there exist u, v, w 6 with z = uvw, |v| 1, |uv| n,
and, for all i N, uvi w L.
h
Writing this statement succinctly, we obtain
L is regular (nz, |z| n, u, v, w, |uv| n, |v| 1, i, uvi w L)
so that the contrapositive is
(nz, |z| n, u, v, w, |uv| n, |v| 1, i, uvi w
/ L)
L is not regular
71
72
Thus to show that a language is not regular, all we need to do is find a string
z that contradicts the lemma. We can think of playing the adversary in a
game where our opponent is attempting to convince us that the language
is regular and where we are intent on providing a counterexample. If our
opponent claims that the language is regular, then he must be able to provide
a finite automaton for the language. Yet no matter what that automaton is,
our counterexample must work, so we cannot pick n, the number of states
of the claimed automaton, but must keep it as a parameter in order for our
construction to work for any number of states. On the other hand, we get to
choose a specific string, z, in the language and give it to our opponent. Our
opponent, who (claims that he) knows a finite automaton for the language,
then tells us where the first loop used by his machine lies and how long
it is (something we have no way of knowing since we do not have the
automaton). Thus we cannot choose the decomposition of z into u, v, and
w, but, on the contrary, must be prepared for any decomposition given to
us by our opponent. Thus for each possible decomposition into u, v, and w
(that obeys the constraints), we must prepare our counterexample, that is, a
pumping number i (which can vary from decomposition to decomposition)
such that the string uvi w is not in the language.
To summarize, the steps needed to prove that a language is not regular
are:
1.
2.
3.
4.
Failure to proceed through these steps invalidates the potential proof that L
is not regular but does not prove that L is regular! If the language is finite,
the pumping lemma is useless, as it has to be, since all finite languages are
regular: in a finite language, the automatons accepting paths all have length
less than the number of states in the machine, so that the pumping lemma
holds vacuously.
Consider the language L 1 = {0i 1i | i 0}. Let n be the constant of the
pumping lemma (that is, n is the number of states in the corresponding
deterministic finite automaton, should one exist). Pick the string z = 0n 1n ;
it satisfies |z| n. Figure 3.21 shows how we might decompose z = uvw to
ensure |uv| n and |v| 1. The uv must be a string of 0s, so pumping v
all 0s
all 1s
n
u
Figure 3.21 Decomposing the string z into possible choices for u, v, and w.
will give more 0s than 1s. It follows that the pumped string is not in L 1 ,
which would contradict the pumping lemma if the language were regular.
Therefore the language is not regular.
As another example, let L 2 be the set of all strings, the length of
which is a perfect square. (The alphabet does not matter.) Let n be the
constant of the lemma. Choose any z of length n 2 and write z = uvw with
|v| 1 and |uv| n; in particular, we have 1 |v| n. It follows from the
pumping lemma that, if the language is regular, then the string z = uv2 w
must be in the language. But we have |z | = |z| + |v| = n 2 + |v| and, since we
assumed 1 |v| n, we conclude n 2 < n 2 + 1 n 2 + |v| n 2 + n < (n + 1)2,
or n 2 < |z | < (n + 1)2, so that |z | is not a perfect square and thus z is not
in the language. Hence the language is not regular.
As a third example, consider the language L 3 = {a i b j ck | 0 i < j < k}.
Let n be the constant of the pumping lemma. Pick z = a n bn+1cn+2 , which
clearly obeys |z| n as well as the inequalities on the exponentsbut is
as close to failing these last as possible. Write z = uvw, with |uv| n
and |v| 1. Then uv is a string of as, so that z = uv2 w is the string
a n+|v| bn+1cn+2 ; since we assumed |v| 1, the number of as is now at least
equal to the number of bs, not less, so that z is not in the language. Hence
L is not regular.
As a fourth example, consider the set L 4 of all strings x over {0, 1} such
that, in at least one prefix of x, there are four more 1s than 0s. Let n be the
constant of the pumping lemma and choose z = 0n 1n+4 ; z is in the language,
because z itself has four more 1s than 0s (although no other prefix of z does:
once again, our string z is on the edge of failing membership). Let z = uvw;
since we assumed |uv| n, it follows that uv is a string of 0s and that, in
particular, v is a string of one or more 0s. Hence the string z = uv2 w, which
must be in the language if the language is regular, is of the form 0n+|v| 1n+4 ;
73
74
but this string does not have any prefix with four more 1s than 0s and so is
not in the language. Hence the language is not regular.
As a final example, let us tackle the more complex language L 5 =
{a i b j ck | i 6= j or j 6= k}. Let n be the constant of the pumping lemma and
choose z = a n bn!+n cn!+n the reason for this mysterious choice will become
clear in a few lines. (Part of the choice is the now familiar edge position:
this string already has the second and third groups of equal size, so it
suffices to bring the first group to the same size to cause it to fail entirely.)
Let z = uvw; since we assumed |uv| n, we see that uv is a string of as and
thus, in particular, v is a string of one or more as. Thus the string z = uvi w,
which must be in the language for all values of i 0 if the language is regular,
is of the form a n+(i1)|v|bn!+n cn!+n . Choose i to be (n!/|v|) + 1; this value is
a natural number, because |v| is between 1 and n, and because n! is divisible
by any number between 1 and n (this is why we chose this particular value
n! + n). Then we get the string a n!+n bn!+n cn!+n , which is not in the language.
Hence the language is not regular.
Consider applying the pumping lemma to the language L 6 = {a i b j ck |
i > j > k 0}. L 6 is extremely similar to L 3 , yet the same application of
the pumping lemma used for L 3 fails for L 6 : it is no use to pump more
as, since that will not contradict the inequality, but reinforce it. In a
similar vein, consider the language L 7 = {0i 1 j 0 j | i, j > 0}; this language is
similar to the language L 1 , which we already proved not regular through a
straightforward application of the pumping lemma. Yet the same technique
will fail with L 7 , because we cannot ensure that we are not just pumping
initial 0ssomething that would not prevent membership in L 7 .
In the first case, there is a simple way out: instead of pumping up, pump
down by one. From uvw, we obtain uw, which must also be in the language
if the language is regular. If we choose for L 6 the string z = a n+2 bn+1 , then
uv is a string of as and pumping down will remove at least one a, thereby
invalidating the inequality. We can do a detailed case analysis for L 7 , which
will work. Pick z = 01n 0n ; then uv is 01k for some k 0. If k equals 0, then
uv is just 0, so u is and v is 0, and pumping down once creates the string
1n 0n , which is not in the language, as desired. If k is at least 1, then either
u is , in which case pumping up once produces the string 01k 01n 0n , which
is not in the language; or u has length at least 1, in which case v is a string
of 1s and pumping up once produces the string 01n+|v| 0n , which is not in
the language either. Thus in all three cases we can pump the string so as to
produce another string not in the language, showing that the language is
not regular. But contrast this laborious procedure with the proof obtained
from the extended pumping lemma described below.
75
76
3.4.2
To see that regular sets are closed under this operation, we shall use regular
expressions. Since each regular set can be written as a regular expression,
each of the f (a) for a 6 can be written as a regular expression. The
77
78
0 10 /10 = 0
0 10+ /0 1 =
(1 + 10+ )/(0+ + 11) = 1 + 10
79
80
3.4.3
a,c
a,b
c
a,b
b
a,c
b
c
ac,bc,cb
ac,bb,cc
aa,ab,ba,bb,ca,cc
Figure 3.22
a
a
b
c
c
Figure 3.23
Our automaton for swap(L) will have a four-state block for each state of
the pair-grouped automaton for L, as illustrated in Figure 3.23. We can
formalize this construction as followsalbeit at some additional cost in
the number of states of the resulting machine. Our new machine M has
state set Q (Q 6), where Q is the state set of M; it has transitions of
the type (q, a) = [q, a] for all q Q and a 6 and transitions of the type
([q, a], b) = ((q, b), a) for all q Q and a, b 6; its start state is q0 , the
start state of M; and its accepting states are the accepting states of M.
h
Example 3.5 The approach used in the previous example works when
trying to build a machine that reads strings of the same length as those
read by M; however, when building a machine that reads strings shorter
than those read by M, nondeterministic transitions must be used to guess
the missing symbols.
Define the language odd(L) to be
{a1 a3 a5 . . . a2n1 | a2 , a4 , . . ., a2n , a1 a2 . . . a2n1 a2n L}
When machine M for odd(L) attempts to simulate what M would do, it
gets only the odd-indexed symbols and so must guess which even-indexed
symbols would cause M to accept the full string. So M in some state q
corresponding to a state of M reads a symbol a and moves to some new
state not in M (call it [q, a]); then M makes an transition that amounts to
guessing what the even-indexed symbol could be. The replacement block
of states that results from this construction is illustrated in Figure 3.24.
Thus we have q ([q, a], ) for all states q with q = ((q, a), b) for any
choice of b; formally, we write
([q, a], ) = {((q, a), b) | b 6}
81
82
ab
a
b
c
Figure 3.24
In this way, M makes two transitions for each symbol read, enabling it to
simulate the action of M on the twice-longer string that M needs to verify
acceptance.
As a specific example, consider the language L = (00 + 11), recognized
by the automaton of Figure 3.25(a). For this choice of L, odd(L) is just
6 . After grouping the input symbols in pairs, we get the automaton of
Figure 3.25(b). Now our new nondeterministic automaton has a block
of three states for each state of the pair-grouped automaton and so six
states in all, as shown in Figure 3.26. Our automaton moves from the start
state to one of the two accepting states while reading a character from the
0,1
1
00,11
01,10
Figure 3.25
0
1
0
0
1
83
84
3.5 Conclusion
3.5 Conclusion
Finite automata and regular languages (and regular grammars, an equivalent mechanism based on generation that we did not discuss, but that is
similar in spirit to the grammars used in describing legal syntax in programming languages) present an interesting model, with enough structure
to possess nontrivial properties, yet simple enough that most questions
about them are decidable. (We shall soon see that most questions about
universal models of computation are undecidable.) Finite automata find
most of their applications in the design of logical circuits (by definition,
any chip is a finite-state machine, the difference from our model being
simply that, whereas our finite-state automata have no output function,
finite-state machines do), but computer scientists see them most often in
parsers for regular expressions. For instance, the expression language used
to specify search strings in Unix is a type of regular expression, so that the
Unix tools built for searching and matching are essentially finite-state automata. As another example, tokens in programming languages (reserved
words, variables names, etc.) can easily be described by regular expressions
and so their parsing reduces to running a simple finite-state automaton
(e.g., lex).
However, finite automata cannot be used for problem-solving; as we
have seen, they cannot even count, much less search for optimal solutions.
Thus if we want to study what can be computed, we need a much more
powerful model; such a model forms the topic of Chapter 4.
85
86
3.6 Exercises
Exercise 3.4 Give deterministic finite automata accepting the following
languages over the alphabet 6 = {0, 1}:
1. The set of all strings that contain the substring 010.
2. The set of all strings that do not contain the substring 000.
3. The set of all strings such that every substring of length 4 contains at
least three 1s.
4. The set of all strings that contain either an even number of 0s or at
most three 0s (that is, if the number of 0s is even, the string is in
the language, but if the number of 0s is odd, then the string is in the
language only if that number does not exceed 3).
5. The set of all strings such that every other symbol is a 1 (starting at the
first symbol for odd-length string and at the second for even-length
strings; for instance, both 10111 and 0101 are in the language).
This last problem is harder than the previous four since this automaton
has no way to tell in advance whether the input string has odd or even
length. Design a solution that keeps track of everything needed for
both cases until it reaches the end of the string.
Exercise 3.5 Design finite automata for the following languages over {0, 1}:
1. The set of all strings where no pair of adjacent 0s appears in the last
four characters.
2. The set of all strings where pairs of adjacent 0s must be separated by
at least one 1, except in the last four characters.
Exercise 3.6 In less than 10 seconds for each part, verify that each of the
following languages is regular:
1. The set of all C programs written in North America in 1997.
2. The set of all first names given to children born in New Zealand in
1996.
3. The set of numbers that can be displayed on your hand-held calculator.
Exercise 3.7 Describe in English the languages (over {0, 1}) accepted by the
following deterministic finite automata. (The initial state is identified by a
short unlabeled arrow; the final statethese deterministic finite automata
have only one final state eachis identified by a double circle.)
3.6 Exercises
1.
0
0,1
0,1
0,1
2.
1
0
0
0
1
1
0,1
3.
0
0
0,1
1
0
87
88
2. The set of all strings such that the fifth symbol from the end of the
string is a 1.
3. The set of all strings over the alphabet {a, b, c, d} such that one of the
three symbols a, b, or c appears at least four times in all.
Exercise 3.10 Devise a general procedure that, given some finite automaton
M, produces the new finite automaton M such that M rejects , but
otherwise accepts all strings that M accepts.
Exercise 3.11 Devise a general procedure that, given a deterministic finite
automaton M, produces an equivalent deterministic finite automaton M
(i.e., an automaton that defines the same language as M) in which the start
state, once left, cannot be re-entered.
Exercise 3.12 Give a nondeterministic finite automaton to recognize the
set of all strings over the alphabet {a, b, c} such that the string, interpreted
as an expression to be evaluated, evaluates to the same value left-to-right
as it does right-to-left, under the following nonassociative operation:
a
b
c
a
a
b
c
b
b
c
a
c
b
a
b
Then give a deterministic finite automaton for the same language and
attempt to prove a nontrivial lower bound on the size of any deterministic
finite automaton for this problem.
Exercise 3.13 Prove that every regular language is accepted by a planar
nondeterministic finite automaton. A finite automaton is planar if its
transition diagram can be embedded in the plane without any crossings.
Exercise 3.14 In contrast to the previous exercise, prove that there exist
regular languages that cannot be accepted by any planar deterministic finite
automaton. (Hint: Exercise 2.21 indicates that the average degree of a
node in a planar graph is always less than six, so that every planar graph
must have at least one vertex of degree less than six. Thus a planar finite
automaton must have at least one state with no more than five transitions
leading into or out of that state.)
Exercise 3.15 Write regular expressions for the following languages over
{0, 1}:
1. The language of Exercise 3.5(1).
3.6 Exercises
2.
3.
4.
5.
6.
7.
8.
9.
10.
{x {0, 1} | x 6= x R }
{x {0, 1, 2} | x = w2w, with w {0, 1} }
{x {0, 1} | x = w R wy, with w, y {0, 1}+ }
{x {0, 1} | x
/ {01, 10} }
The set of all strings (over {0, 1}) that have equal numbers of 0s and
1s and such that the number of 0s and the number of 1s in any prefix
of the string never differ by more than two.
{0l 1m 0n | n l or n m, l, m, n N}
{0 n | n N}
The set of all strings x (over {0, 1}) such that, in at least one substring
of x, there are four more 1s than 0s.
The set of all strings over {0, 1} that have the same number of
occurrences of the substring 01 as of the substring 10. (For instance,
we have 101 L and 1010
/ L.)
{0i 1 j | gcd(i, j ) = 1} (that is, i and j are relatively prime)
89
90
3.6 Exercises
91
92
3.7 Bibliography
The first published discussion of finite-state machines was that of McCulloch and Pitts [1943], who presented a version of neural nets. Kleene [1956]
formalized the notion of a finite automaton and also introduced regular expressions, proving the equivalence of the two models (Theorem 3.3 and Section 3.3.3). At about the same time, three independent authors, Huffman
[1954], Mealy [1955], and Moore [1956], also discussed the finite-state
model at some length, all from an applied point of viewall were working on the problem of designing switching circuits with feedback loops,
or sequential machines, and proposed various design and minimization
methods. The nondeterministic finite automaton was introduced by Rabin
and Scott [1959], who proved its equivalence to the deterministic version
(Theorem 3.1). Regular expressions were further developed by Brzozowski
[1962, 1964]. The pumping lemma (Theorem 3.4) is due to Bar-Hillel et
al. [1961], who also investigated several closure operations for regular languages. Closure under quotient (Theorem 3.5) was shown by Ginsburg
and Spanier [1963]. Several of these results use a grammatical formalism
instead of regular expressions or automata; this formalism was created in
a celebrated paper by Chomsky [1956]. Exercises 3.31 and 3.32 are examples of proportional removal operations; Seiferas and McNaughton [1976]
characterized which operations of this type preserve regularity. The interested reader should consult the classic text of Hopcroft and Ullman [1979]
for a lucid and detailed presentation of formal languages and their relation
to automata; the texts of Harrison [1978] and Salomaa [1973] provide
additional coverage.