Automata theory

and formal languages

What is automata theory
 Automata theory is the study of abstract computational
devices
 Abstract devices are (simplified) models of real computations
 Computations happen everywhere: On your laptop, on your
cell phone, in nature, …
 Why do we need abstract models?
 A simple computer

BATTERY

input: switch
output: light bulb
actions: flip switch
states: on, off
 A simple “computer”

BATTERY start off on

input: switch
output: light bulb bulb is on if and only if
there was an odd number
actions: f for “flip switch” of flips
states: on, off
 Another “computer” 1
1 start off off
1

2 2 2 2
BATTERY
1
2
off on
1

inputs: switches 1 and 2

actions: 1 for “flip switch 1” bulb is on if and only if
actions: 2 for “flip switch 2” both switches were flipped
an odd number of times
states: on, off
 A design problem
1 4

?
5
BATTERY

Can you design a circuit where the light is on if and only

if all the switches were flipped exactly the same number
of times?
A design problem
 Such devices are difficult to reason about, because they can
be designed in an infinite number of ways
 By representing them as abstract computational devices, or
automata, we will learn how to answer such questions
 These devices can model many things
 They can describe the operation of any “small computer”, like the
control component of an alarm clock or a microwave
 They are also used in lexical analyzers to recognize well formed
expressions in programming languages:

ab1 is a legal name of a variable in C

5u= is not
Different kinds of automata
 This was only one example of a computational device, and
there are others
 We will look at different devices, and look at the following
questions:
 What can a given type of device compute, and what are its
limitations?
 Is one type of device more powerful than another?
Some devices we will see
finite automata Devices with a finite amount of memory.
Used to model “small” computers, lexical
analyzer etc.
push-down Devices with infinite memory that can be
automata accessed in a restricted way.
Used to model parsers, etc.

Turing Machines Devices with infinite memory.

Used to model any computer.

Time-bounded Infinite memory, but bounded running time.

Turing Machines Used to model any computer program that
runs in a “reasonable” amount of time.
Some highlights of the course
 Finite automata
 We will understand what kinds of things a device with finite
memory can do, and what it cannot do
 Introduce simulation: the ability of one device to “imitate”
another device
 Introduce nondeterminism: the ability of a device to make
arbitrary choices
 Push-down automata
 These devices are related to grammars, which describe the
structure of programming (and natural) languages
Some highlights of the course
 Turing Machines
 This is a general model of a computer, capturing anything we
could ever hope to compute
 Surprisingly, there are many things that we cannot compute, for
example:
Write a program that, given the code of another
program in C, tells if this program ever outputs
the word “hello”

 It seems that you should be able to tell just by looking at the

program, but it is impossible to do!
Some highlights of the course
 Time-bounded Turing Machines
 Many problems are possible to solve on a computer in principle,
but take too much time in practice
 Traveling salesman: Given a list of cities, find the shortest way
to visit them and come back home
Beijing

Xian
Chengdu Shanghai
Guangzhou
Hong Kong
 Easy in principle: Try the cities in every possible order
 Hard in practice: For 100 cities, this would take 100+ years
even on the fastest computer!
Preliminaries of automata theory
 How do we formalize the question

Can device A solve problem B?

 First, we need a formal way of describing the problems that

we are interested in solving
Problems
 Examples of problems we will consider
 Given a word s, does it contain the subword “good”?
 Given a number n, is it divisible by 7?
 Given a pair of words s and t, are they the same?
 Given an expression with brackets, e.g. (()()), does every
left bracket match with a subsequent right bracket?
 All of these have “yes/no” answers.
 There are other types of problems, that ask “Find this” or
“How many of that” but we won’t look at those.
 Alphabets and strings
 A common way to talk about words, number, pairs of words, etc.
is by representing them as strings
 To define strings, we start with an alphabet

An alphabet is a finite set of symbols.

 Examples

S1 = {a, b, c, d, …, z}: the set of letters in English

S2 = {0, 1, …, 9}: the set of (base 10) digits
S3 = {a, b, …, z, #}: the set of letters plus the
special symbol #
S4 = {(, )}: the set of open and closed brackets
 Strings
A string over alphabet S is a finite sequence
of symbols in S.

 The empty string will be denoted by e

 Examples

abfbz is a string over S1 = {a, b, c, d, …, z}

9021 is a string over S2 = {0, 1, …, 9}
ab#bc is a string over S3 = {a, b, …, z, #}
))()(() is a string over S4 = {(, )}
 Languages
A language is a set of strings over an alphabet.

 Languages can be used to describe problems with “yes/no”

answers, for example:

L1 = The set of all strings over S1 that contain

the substring “good”
L2 = The set of all strings over S2 that are divisible by 7
= {7, 14, 21, …}
L3 = The set of all strings of the form s#s where s is any
string over {a, b, …, z}
L4 = The set of all strings over S4 where every left
parenthesis”(“ can be matched with a subsequent
right parenthesis ”)”
Finite Automata
 Example of a finite automaton
f

off on

 There are states off and on, the automaton starts in off and tries
to reach the “final state” on
 What sequences of fs lead to the final state?
 Answer: {f, fff, fffff, …} = {f n: n is odd}
 This is an example of a deterministic finite automaton over
alphabet Σ={f}
Deterministic finite automata

 A deterministic finite automaton (DFA) is a 5-tuple machine,

M=(Q, S, d, q0, F)
where
 Q is a finite set of states
 S is an alphabet
 d: Q × S → Q is a transition function
 q0  Q is the initial state
 F  Q is a set of accepting states (or final states).
 In diagrams, the accepting states will be denoted by double
loops
Example
0 1 0,1

q0 1 q1 0 q2

alphabet S = {0, 1} transition function d:

start state Q = {q0, q1, q2} inputs
initial state q0 0 1
accepting states F = {q0, q1} q0 q0 q1

states
q1 q2 q1
q2 q2 q2
 Language of a DFA
The language of a DFA M=(Q, S, d, q0, F) is the set of
all strings over S that, starting from q0 and
following the transitions as the string is read from left
to right, will reach some accepting state.

M: off on

 Language of M is {f, fff, fffff, …} = {f n: n is odd}

Examples
0 0
1
q0 q1
1

0 1
1
q0 q1
0

0 1 0,1

q0 1 q1 0 q2

What are the languages of these DFAs?

 Examples
 Construct a DFA that accepts the language

L = {010, 1} ( S = {0, 1} )
 Examples
 Construct a DFA that accepts the language

L = {010, 1} ( S = {0, 1} )

 Answer

q0 1 q01 0 q010
0
0 1
qe 0, 1
1 0, 1
q1 qdie
0, 1
Examples
 Construct a DFA over alphabet {0, 1} that accepts all strings
that end in 101
Examples
 Construct a DFA over alphabet {0, 1} that accepts all strings
that end in 101

 Hint: The DFA must “remember” the last 3 bits of the string
it is reading
Examples
 Construct a DFA over alphabet {0, 1} that accepts all strings
that end in 101
 Sketch of answer: 0
1
0 0 q2 1
q3
1 q1
q0
0 1