Year: 2024-2025

Spring Semester

Natural Language
Processing
Dr. Wafaa Samy
Dr. Hanaa Eissa
 Finite State Machines (Part 1)

Lecture (4)

2
 Contents
• Finite State Machines (FSMs)
o Deterministic Finite State Machines.
o Non-Deterministic Finite State Machines.

3
 Finite State Automata (FSA)
• Finite State Automata (FSA):
o Recognizer for “Regular Languages”.
o Regular expressions can be viewed as a textual way of
specifying the structure of finite-state automata.
o FSA are needed when studying morphology and parts of
syntax.
• Finite State Automata (FSAs) are Graphs.
• Example: Modeling recognition of the word “then”.

1 2 3 4 5

4 Transition Intermediate
Start state Final state
state
 Types of Finite State Automata (FSA)
• Two Types:
1. Deterministic Finite Automata (DFA)
 The machine can exist in only one state at any given
time.
2. Non-deterministic Finite Automata (NFA)
 The machine can exist in multiple states at the same time.

• Let L(A) be a language recognized by a DFA A.

o Then L(A) is called a “Regular Language”.

5
 Deterministic Finite Automata -
Definition
• A Deterministic Finite Automaton (DFA)
consists of:
1. Q : a finite set of states.
2. ∑ : a finite set of input symbols (alphabet).
3. q0 : a start state.
4. F : a set of accepting states.
5. δ : a transition function, which is a mapping
between Q x ∑ ==> Q
• A DFA is defined by the 5-tuple:
{ Q , ∑ , q0 , F , δ }
6
 DFA Notation

DFA

7
 Example (1)

8
 What does a DFA do on reading an
input string?
• Input: a string (or word) w in ∑*
• Question: Is w acceptable by the DFA?
• Steps:
o Start at the “start state” q0.
o For every input symbol in the sequence w do:
 Compute the next state from the current state, given the
current input symbol in w and the transition function.
o If after all symbols in w are consumed, the current
state is one of the accepting states (F) then accept w.
o Otherwise, reject w.

9
 Example (1) (Cont.)

10
 Example (1) (Cont.)

11
 Example (1) (Cont.)

12
 Example (1) (Cont.)

13
 Example (1) (Cont.)

14
 Example (1) (Cont.)

15
 Example (2)
• What is the DFA for the sheep language?
{baa!, baaa!, baaaa!, …}

• Using Regular Expression: /baa+!/ or /baaa*!/

• Sheep DFA, this machine has the following 5 parts:
1. 5 states, Q = { q0 , q1 , q2 , q3 , q4 }. (Finite set of states).
2. b, a, and ! are in its alphabet, ∑ = {b , a, !}.
3. q0 is the start state.
4. q4 is an accept state.
5. 5 transitions.

16
 Example (2) (Cont.)
• The input can be written on a long tape broken up into cells,
with one symbol written in each cell.

• Can we reach the accept (or final) state if we read input cell
by cell from the tape?

• This input cannot be accepted.

17
 Example (2) (Cont.)
• An automaton can be represented by a state-transition table
(i.e. equivalent to the FSA graph).
Alphabet

q0

18
 Example (2) (Cont.)
• Note that, we can view any machine with empty
transition as if it is augmented with a fail (or dead)
state.
• For example, adding a fail state to sheep DFA:

19
 Example (3)
• DFA for the numbers 1-99.
Note: in NLP, we
may use words
instead of
characters for
the alphabet.

20
 Example (4)
• Design a finite state machine that accepts only valid web
addresses.
Σ = { http, ://, www, a, b, …, z, . , com, org, / }
• Example: http://www.facebook.com

q2

q0 q1 q3 q4 q5 q6 q7 q8

21
 Language of a DFA
• A DFA A accepts string w if there is a path
from q0 to an accepting (or final) state that is
labeled by w.

• i.e. L(A) = { w | δ(q0 ,w)  F }

• i.e. L(A) = all strings that lead to an accepting

state from q0.

22
 Example (5)
• Build a DFA for the following language:
L = {w | w is a binary string that contains 01 as a substring}
• Steps for building a DFA to recognize L:
o ∑ = {0,1}
o Decide on the states: Q
o Designate start state and final state(s).
o δ: Decide on the transitions:
• “Final” states == same as “accepting states”.
• Other states == same as “non-accepting states”.

23
Example (5): DFA for strings containing
01
• L = {w | w is a binary string that contains 01 as a substring}
• Regular expression: (0|1)*01(0|1)*
• Q = {q0,q1,q2}
1 0 0,1
• ∑ = {0,1}
start 0 1 • start state = q0
q0 q1 q2
• F = {q2}
Accepting state
• Transition table

• What makes this DFA deterministic? symbols

0 1
states q0 q1 q0
q1 q1 q2
24 *q2 q2 q2
 Non-deterministic Finite Automata
(NFA)
• A Non-deterministic Finite Automaton (NFA)
o is of course “non-deterministic”
• Implying that the machine can exist in more than one state
at the same time.
• Transitions could be non-deterministic.
• Can have ε-moves.
1 qj
qi Each transition function therefore
…
1 maps to a set of states.
qk

• Advantages of NFA:
o Great for modeling regular expressions.
o String processing (e.g. lexical analyzer).
25
 Non-deterministic Finite Automata
(NFA)
• A Non-deterministic Finite Automaton (NFA)
consists of:
1. Q : a finite set of states.
2. ∑ : a finite set of input symbols (alphabet).
3. q0 : a start state.
4. F : a set of accepting states.
5. δ : a transition function, which is a mapping
between Q x ∑ ==> subset of Q.

• An NFA is also defined by the 5-tuple:

{ Q , ∑ , q0 , F , δ }
26
 How to use an NFA?
• Input: a string (or word) w in ∑*.
• Question: Is w acceptable by the NFA?
• Steps:
o Start at the “start state” q0.
o For every input symbol in the sequence w do:
 Determine all possible next states from all current
states, given the current input symbol in w and the
transition function.
o If after all symbols in w are consumed and if at least
one of the current states is a final state then accept w.
o Otherwise, reject w.
27
Example (5): NFA for strings containing
01
• Regular expression: (0|1)*01(0|1)*
• Q = {q0,q1,q2}
0,1 0,1 •  = {0,1}
• start state = q0
start 0 1
q0 q1 q2
• F = {q2}
Final state • Transition table
Why is this non-deterministic?
symbols
What will happen if at state q1 an input of 0 1
0 is received?
q0 {q0,q1} {q0}
Transition to dead state, so the input states
string is rejected. q1 Φ {q2}
28 *q2 {q2} {q2}
 Language of an NFA
• An NFA N accepts w if there exists at least one
path from the start state to an accepting (or
final) state that is labeled by w.

• L(N) = { w | δ(q0,w) ∩ F ≠ Φ }

29
 Deterministic and Nondeterministic
Automata
• Two types of finite automata:
1. Deterministic Finite Automata (DFA) qi
1 qj
…
o The machine can exist in only one state at any given time. 2
qk
 Exactly one transition per input per state.
o No ε-moves.
o DFAs are faster to execute because there are no choices to consider.
o A DFA can take only one path through the state graph.
 Completely determined by input.
2. Non-deterministic Finite Automata (NFA)
o The machine can exist in multiple states at the same time.
 Can have zero, one, or multiple transitions for one input in a given state.
o Can have ε-moves. 1 qj
o For a given language, NFA can be simpler than DFA. qi …
1
o NFAs can choose:
qk
 Whether to make ε-moves. ɛ
30  Which of multiple transitions for a single input to take. qm