Computation Theory

Felicity Ni
July 2024

§1 Introduction
Computation theory studies three areas: automata, computability, and complexity. When
you hear the word “computation,” you might think of computers. What can a computer
do? Here are two famous problems.
Problem 1.1 (Eulerian Path). The beautiful state of Ohio has n towns and m roads,
each connecting two towns (assume you can get to one town from any other town). A
path is a sequence of roads connected by towns. Is there a path to visit every road exactly
once?
Problem 1.2 (Hamiltonian Path). Given n towns and m roads, each connecting two
towns, is there a path to visit every town exactly once?
The first problem is easy, proved by Euler, that if you define a town with an odd
number of roads coming out from it as an odd town, then the desired path exists if and
only if the number of odd towns ≤ 2. However, the second problem is hard, in which you
need to search every permutation between the vertices. When there are 20 towns, in a 2
gigahertz computer which runs 2 billion cycles per second, it takes less than a milisecond
to solve the first problem, but 100 years to solve the second one. What leads to this
difference? This is the central question of complexity theory– what makes some questions
computationally hard and others easy? Some problems might not even be solvable, as
suggested by Godel’s Incompleteness Theorem, which leads to computability theory.
More fundamentally, what is a computer? A computer has a screen, a motherboard,
a CPU, a harddrive, a cooling fan, etc, so many parts that makes it difficult to study.
However, mathematicians create models called automata and turing machines to make
the definition precise and simple.

§2 Automata
§2.1 Finite Automaton
Consider the light here, which has two states– on and off. When you toggle the switch,
it transitions from on to off, or off to on. Otherwise, when the switch is idle, the light
doesn’t change its state unless there is a power failure just like the wifi failure today.
This is a finite automaton. A finite automaton, or finite state machine, consists of a
set of states connected by transitions which are the arrows here. Automaton means it
operates with pre-written instructions and knows what to do at each step.

1
Felicity Ni — July 2024 Computation Theory

§2.2 Strings
To have a more formal definition of DFAs, we need to first know what are strings.
Definition 2.1 (string). A string is a finite sequence of characters. The empty string
ε has 0 characters.
Definition 2.2 (character). A character is a unit of information that corresponds to a
symbol, like letters, digits, whitespace, etc.
Definition 2.3 (alphabet). An alphabet, denoted as Σ, is a finite, nonempty set of
characters. The set of all strings composed from characters in Σ is denoted as Σ∗ .
Definition 2.4 (language). A language over Σ is a set of strings over Σ, i.e. L ⊂ Σ∗ .
A finite automaton is a “computer” that determines whether a string is contained
within some language. It takes a string and either accepts or rejects.

The above diagram shows a finite automaton where the circles are states and the
arrows are transitions. The double circle indicates it’s an accepting state. For example, ε
and 0010 are rejected, and 01 and 0111 are accepted. This DFA rejects any string with
an odd number of 0 or contains no 1.
Definition 2.5 (DFA). A deterministic finite automaton (DFA) is a 5-tuple
(Q, Σ, δ, q0 , F ), where

1. Q is a finite set called the states,

2. Σ is a finite set called the alphabet,

3. δ : Q × Σ → Q is the transition function,

4. q0 ∈ Q is the start state, and

5. F ⊂ Q is the set of accept states.

Since δ is a function, each pair of state and character corresponds to exactly one state,
which is the “deterministic” part. The below examples are not DFAs because at some
point, out of a state on some input, there’s either no transition or multiple transitions.

§2.3 Constructing DFAs

Definition 2.6 (language of an automaton). The language L(M ) of a DFA M is the set
of strings accepted by M .
At each point, the DFA only remembers the current state and has no previous life.
Two strings are equivalent in a DFA if they end in the same state. To build a DFA, we
make each state some information we want to remember.
Problem 2.7. Construct a DFA with L1 = {w ∈ {0, 1}∗ | w has an even number of 1}.

2
Felicity Ni — July 2024 Computation Theory

What about the strings whose number of 1s is a multiple of 3? We can generalize the
construction to the set of strings whose number of 1s is a multiple of n.
Problem 2.8. Construct a DFA with L2 = {w ∈ {0, 1}∗ | w has 01 as its substring}.
Problem 2.9. Construct a DFA with L3 = {w ∈ {0, 1}∗ | w ends in 010 or 101}.

Problem 2.10. Construct a DFA that accepts strings with a prime number of 1s.
Definition 2.11 (regular language). A language is called a regular language if exists
a DFA which accepts all strings in that language and rejects all other strings.
If a language L is regular, is its complement L̄ = Σ∗ − L also regular? Yes, because we
can flip each state. Regular languages are closed under many operations. We define the
following three operations on languages, called regular operations:

1. Union: A ∪ B = {x | x ∈ A or x ∈ B}.

2. Concatenation: A ◦ B = {xy | x ∈ A, y ∈ B}.

3. Kleene star: A∗ = {x1 x2 . . . xk | k ≥ 0, xi ∈ A}.

The Kleene (KLAY-nee) star is a unary operation, defining the smallest superset of A
containing ε and closed under concatenation.

Theorem 2.12
Regular languages are closed under the regular operations.

We first prove they’re closed under union. Let DFAs M1 = (Q1 , Σ, δ1 , q1 , F1 ) and
M2 = (Q2 , Σ, δ2 , q2 , F2 ) recognize languages L1 and L2 respectively. We want to build a
DFA M that accepts if either M1 or M2 accepts, which needs to simulate M1 and M2
simultaneously.
Let M = (Q, Σ, δ, q0 , F ) where

1. Q = Q1 × Q2 is the Cartesian product of states in M1 and M2 ,

2. δ : Q × Σ → Q is defined as δ((r1 , r2 ), a) = (δ(r1 , a), δ(r2 , a)),

3. q0 = (q1 , q2 ), and

4. F = {(r1 , r2 ) | r1 ∈ F1 or r2 ∈ F2 }. Notice if we change or to and, L(M ) will be

L(M1 ) ∩ L(M2 ).

To show the regular languages are closed under concatenation or star, we need to chop a
string into parts and see whether each part is accepted. However, since we don’t know
from where we should chop up the string, we need a stronger tool: nondeterminism.

3
Felicity Ni — July 2024 Computation Theory

§2.4 Nondeterminism
An NFA can have missing transitions or multiple transitions based on the same input
symbol. When it reads a symbol, it splits into multiple parallel universes, each going
to a branch of a possible transition. It accepts if any possible series of choices leads to
an accept state. When it needs to make a transition and no such transition exists, that
particular path is rejected. There are many versions of you, and if any version of you is
good, you are accepted. However, if all versions of you suck, you suck.
NFAs also have a special type of transition: the ε transition, which doesn’t consume
any input.
Definition 2.13 (NFA). A nondeterministic finite automaton (NFA) is a 5-tuple
(Q, Σ, δ, q0 , F ), where

1. Q is a finite set called the states,

2. Σ is a finite set called the alphabet,

3. δ : Q × Σ → P (Q) is the transition function,

4. q0 ∈ Q is the start state, and

5. F ⊂ Q is the set of accept states.

When designing NFAs, we use a guess and check mechanism.

Problem 2.14. Construct an NFA with L3 = {w ∈ {0, 1}∗ | w ends in 010 or 101}.
We use nondeterminism guess when the string ends, and deterministically check if the
condition is satisfied.

By definition, every DFA is an NFA. The NFAs seem much stronger. Can they recognize
languages that aren’t regular? Surprisingly, every language recognized by an NFA is
regular.
Definition 2.15 (equivalence of finite automata). Two finite automata are equivalent
if they recognize the same language.

Theorem 2.16
Every NFA has an equivalent DFA.

Proof: Left for the reader as exercise! Notice the codomain of an NFA is the power set of
Q, so what should the states of the DFA correspond to?

§2.5 Nonregular Languages

What languages are nonregular? The set of strings with a prime number of 1s is
nonregular, due to the limit that a DFA can only have finitely many states, just like
our computers have limited memory. Our computers seem to be able to identify prime,
only because the strings we usually input aren’t long enough. Normal computers use 16

4
Felicity Ni — July 2024 Computation Theory

9
gigabytes of RAM to process daily tasks, which are 128 billion bits and 2128·10 possible
states. If you are really bored and want to wait until the universe goes to heat death,
you can try to build a DFA that corresponds to this. However, there’s a model for the
ideal computer with infinite memory, known as Turing Machine, which we will introduce
next time.