CS154

Finishing Minimization,
The Myhill-Nerode Theorem,
and Streaming Algorithms

1
 DFA Minimization Theorem:

For every regular language L’, there is a

unique (up to re-labeling of states)
minimal-state DFA M* such that
L(M*) = L’.

Furthermore, there is an efficient

algorithm which, given any DFA M, will
output this unique M*.

2
 Extending transition function δ to strings
Given M = (Q, Σ, δ, q0, F), we can extend δ to a
function ∆ : Q × Σ* → Q that works on strings:
∆(q, ε) = q
∆(q, σ) = δ(q, σ)
∆(q, σ1 …σk+1 ) = δ( ∆(q, σ1 …σk ), σk+1 )
∆(q, w) = the state of M reached after reading in w,
starting from state q
Note: ∆(q0, w) ∈ F ⇔ M accepts w
Def. w ∈ Σ* distinguishes states q1 and q2 iff
∆(q1, w) ∈ F ⇔ ∆(q2, w) ∉ F
3
 Extending transition function δ to strings
Given M = (Q, Σ, δ, q0, F), we can extend δ to a
function ∆ : Q × Σ* → Q that works on strings:
∆(q, ε) = q
∆(q, σ) = δ(q, σ)
∆(q, σ1 …σk+1 ) = δ( ∆(q, σ1 …σk ), σk+1 )
∆(q, w) = the state of M reached after reading in w,
starting from state q
Note: ∆(q0, w) ∈ F ⇔ M accepts w
Def.ww∈∈Σ*
Def. Σ*distinguishes
distinguishesstates
statesqq1 1and
andqq2 2 iff
iff
exactlyone
exactly oneof ∆(q11,,w),
of∆(q ∆(q22,,w)
w),∆(q w)isisaafinal
finalstate
state
4
 Fix M = (Q, Σ, δ, q0, F) and let p, q ∈ Q

Definition:
State p is distinguishable from state q
iff there is w ∈ Σ* that distinguishes p and q
iff there is w ∈ Σ* so that
exactly one of ∆(p, w), ∆(q, w) is a final state

State p is indistinguishable from state q

iff p is not distinguishable from q
iff for all w ∈ Σ*, ∆(p, w) ∈ F ⇔ ∆(q, w) ∈ F
Pairs of indistinguishable states are redundant…

5
 Fix M = (Q, Σ, δ, q0, F) and let p, q, r ∈ Q
Define a binary relation ∼ on the states of M:
p ∼ q iff p is indistinguishable from q
p ≁ q iff p is distinguishable from q

Proposition: ∼ is an equivalence relation

p ∼ p (reflexive)
p ∼ q ⇒ q ∼ p (symmetric)
p ∼ q and q ∼ r ⇒ p ∼ r (transitive)

6
 Fix M = (Q, Σ, δ, q0, F) and let p, q, r ∈ Q

Proposition: ~ is an equivalence relation

As a consequence, the relation ~ partitions Q

into disjoint equivalence classes

[q] := { p | p ~ q }
Q
q
q0

7
 Algorithm: MINIMIZE-DFA
Input: DFA M
Output: DFA MMIN such that:
L(M) = L(MMIN)
MMIN has no inaccessible states
MMIN is irreducible
||
For all states p ≠ q of MMIN, p and q are distinguishable
Theorem: MMIN is the unique minimal DFA
that is equivalent to M
8
 The Table-Filling Algorithm
Input: DFA M = (Q, Σ, δ, q0, F)
Output: (1) DM = { (p, q) | p, q ∈ Q and p ~
/ q}
(2) EQUIVM = { [q] | q ∈ Q }
q0
Base Case: For all (p, q) such that
q1
p accepts and q rejects ⇒ p /~ q
Iterate: If there are states p, q and
qi D D symbol σ ∈ Σ satisfying:
δ (p, σ) = p′
qn D ~/ ⇒ p ~/ q
q0 q1 qi qn δ (q, σ) = q′
Repeat until no more D’s can be added9
 Algorithm MINIMIZE
Input: DFA M
Output: Equivalent minimal-state DFA MMIN
1. Remove all inaccessible states from M
2. Run Table-Filling algorithm on M to get:
EQUIVM = { [q] | q is an accessible state of M }
3. Define: MMIN = (QMIN, Σ, δMIN, q0 MIN, FMIN)
QMIN = EQUIVM, q0 MIN = [q0], FMIN = { [q] | q ∈ F }
δMIN( [q], σ ) = [ δ( q, σ ) ]
Claim: L(MMIN) = L(M)
10
 1 0 0,1
0
q0 q1 q4
1 1

0,1 0 0 q5
q0

q1 q2 1
D q3
q3 D D
q4 D D
q5 D D D D
q0 q1 q3 q4 q5 11
 1 0 0,1
0
q0 q1 q4
1 1

0 0 q5
q0

q1 D q3
q3 D D
q4 D D
q5 D D D D
q0 q1 q3 q4 q5 12
Thm: MMIN is the unique minimal DFA equivalent to M
Claim: Suppose L(M′)=L(MMIN) and M′ has no
inaccessible states and M’ is irreducible.
Then there is an isomorphism between M′ and MMIN

Suppose for now the Claim is true.

If M’ is a minimal DFA, then M’ has no inaccessible
states and is irreducible (why?)
So the Claim implies:
Let M’ be a minimal DFA for M.
Then, there is an isomorphism between M’ and
the DFA MMIN that is output by MINIMIZE(M).
Therefore the Thm holds!
13
Thm: MMIN is the unique minimal DFA equivalent to M
Claim: Suppose L(M′)=L(MMIN) and M′ has no
inaccessible states and M’ is irreducible.
Then there is an isomorphism between M′ and MMIN

Proof: We recursively construct a map from

the states of MMIN to the states of M′
Base Case: q0 MIN a q0′
Recursive Step: If p a p′
σ σ Then q a q′
q q′

14
Base Case: q0 MIN a q0′
Recursive Step: If p a p′
σ σ Then q a q′
q q′

15
 Base Case: q0 MIN a q0′
Recursive Step: If p a p′
σ σ Then q a q′
q q′

Goal: Show this is an isomorphism. Need to prove:

The map is defined everywhere
The map is well defined
The map is a bijection
The map preserves all transitions:
If p a p′ then δMIN(p, σ) a δ’(p’, σ)
(this follows from the definition of the map!)
16
 Base Case: q0 MIN a q0′
Recursive Step: If p a p′
σ σ Then q a q′
q q′

The map is defined everywhere

That is, for all states q of MMIN
there is some state q′ of M′ such that q a q′
If q ∈ MMIN, there is a string w such that
∆MIN(q0 MIN,w) = q (Why?)
Let q′ = ∆′(q0′,w). Then q a q′
(proof by induction on |w|)
17
 Base Case: q0 MIN a q0′
Recursive Step: If p a p′
σ σ Then q a q′
q q′
The map is well defined
Proof by contradiction.
Suppose there are states q′ and q′′ such that
q a q′ and q a q′′
We show that q′ and q′′ are indistinguishable,
so it must be that q′ = q′′

18
 Suppose there are states q′ and q′′ such that
q a q′ and q a q′′
Now suppose q′ and q′′ are distinguishable…

MMIN M′
Accept

Accept
u w u w
q0 MIN q q0′ q′
Contradiction!
v w v w

Reject
Reject

q0 MIN q q0′ q′′

19
 Base Case: q0 MIN a q0′
Recursive Step: If p a p′
σ σ Then q a q′
q q′
The map is onto
Want to show: For all states q′ of M′ there is a
state q of MMIN such that q a q′
For every q′ there is a string w such that
M’ reaches state q’ after reading in w

Let q be the state of MMIN after reading in w

Claim: q a q′ (proof by induction on |w|)
20
 The map is one-to-one
Proof by contradiction. Suppose there are
states p ≠ q such that p a q′ and q a q′
If p ≠ q, then p and q are distinguishable
MMIN M′
Accept

Accept
u w u w
q0 MIN p q0′ q′
Contradiction!
v w v w

Reject
Reject

q0 MIN q q0′ q′
21
How can we prove that two regular
expressions are equivalent?

22
The Myhill-Nerode Theorem

23
 In DFA Minimization, we defined
an equivalence relation between states.
We can also define a similar equivalence relation
over strings and languages:

Let L ⊆ Σ* and x, y ∈ Σ*
x ≡L y iff for all z ∈ Σ*, [xz ∈ L yz ∈ L]

Define: x and y are indistinguishable to L iff x ≡L y

Claim: ≡L is an equivalence relation
Proof?
24
 Let L ⊆ Σ* and x, y ∈ Σ*
x ≡L y iff for all z ∈ Σ*, [xz ∈ L yz ∈ L]
The Myhill-Nerode Theorem:
A language L is regular if and only if
the number of equivalence classes of ≡L is finite.
Proof (⇒) Let M = (Q, Σ, δ, q0, F) be a min DFA for L.
Define the relation: x ~M y ∆(q0,x) = ∆(q0,y)
Claim: ~M is an equivalence relation with |Q| classes
Claim: If x ~M y then x ≡L y
Proof: x ~M y implies for all z ∈ Σ*, xz and yz reach
the same state of M. So xz ∈ L yz ∈ L, and x ≡L y
Corollary: Number of equiv. classes of ≡L is at most
the number of equiv. classes of ~M (which is |Q|) 25
 Let L ⊆ Σ* and x, y ∈ Σ*
x ≡L y iff for all z ∈ Σ*, [xz ∈ L yz ∈ L]

(⇐) If the number of equivalence classes of ≡L is k

then there is a DFA for L with k states

Idea: Build a DFA using equivalence classes of ≡L!

Define a DFA M where
Q is the set of equivalence classes of ≡L
q0 = [ε] = {y | y ≡L ε}
δ([x], σ) = [x σ]
F = {[x] | x ∈ L}

Claim: M accepts x if and only if x ∈ L

26
 The Myhill-Nerode Theorem gives us a new way
to prove that a given language is not regular:

L is not regular
if and only if
there are infinitely many equiv. classes of ≡L

L is not regular Distinguishing set for L

if and only if
There are infinitely many strings w1, w2, … so that
for all wi ≠ wj, wi and wj are distinguishable to L:
there is a z ∈ Σ* such that
exactly one of wi z and wj z is in L
27
 The Myhill-Nerode Theorem gives us a new way
to prove that a given language is not regular:

Theorem: L = {0n 1n | n ≥ 0} is not regular.

Proof: Consider the infinite set of strings
S = {0, 00, 000, …, 0n, …}
Take any pair (0m, 0n) of distinct strings in S
Let z = 1m
Then 0m 1m is in L, but 0n 1m is not in L
That is, all pairs of strings in S are distinguishable
Hence there are infinitely many equivalence
classes of ≡L , and L is not regular.

28