Formal Language

and
Automata Theory
Outline
• Introduction
• Alphabets and Strings
• What is a Formal Language?
• What is Automata Theory?
• General Model of Automaton
• Introduction to Finite Automata
Introduction – cont.…
• Formal Language theory mainly deals with the mathematical
properties of strings and collection of strings.
• Soon after the advent of modern electronic computers, people
realized that all forms of information – whether numbers, names,
pictures, or sound waves can be represented as strings.
• Then collection of strings known as languages became central to
computer science.
Introduction – cont.…
• Real computers are modeled with a very simple abstract
mathematical model – called Turing Machine.
– Using it we are able to prove that whether some computational problems
can be algorithmically solvable (decidable) or cannot be algorithmically
solvable (i.e. undecidable).
• Automata theory is the study of abstract machines (computational
devices) and automata as well as the computational problems that
can be solved using them.
Introduction – cont.…
• An automaton is an abstract model of machines (computing device)
that perform computation on an input.
• Automata theory and formal language theory are closely related, where
an automaton is a finite representation of a formal language that may
be an infinite set.
• Throughout the course, we deal with the relationship between formal
languages and automaton by constructing accepting devices for four
types of formal languages:
– Regulars language and Finite automata
– Context free languages and Pushdown automata
– Context sensitive languages and Linear-bounded automata
– Recursively enumerable languages and Turing machines.
Definitions
• Symbol – An atomic unit, such as a digit, character, lower-case letter,
etc. Sometimes a word. [Formal language does not deal with the
“meaning” of the symbols.]

• Alphabet – A finite set of symbols, usually denoted by Σ.

Σ = {0, 1} Σ = {0, a, 4} Σ = {a, b, c, d}

• String – A finite set sequence of symbols, Chosen from some alphabet.

Alphabets and strings
• A common way to talk about words, number, pairs of words, etc.
is by representing them as strings
• To define strings, we start with an alphabet
An alphabet is a finite set of symbols.

• Examples
1 = {a, b, c, d, …, z}: the set of letters in English
2 = {0, 1, …, 9}: the set of (base 10) digits
3 = {a, b, …, z, #}: the set of letters plus the
special symbol #
4 = {(, )}: the set of open and closed brackets
Strings

A string over alphabet  is a finite sequence

of symbols in .

• The empty string will be denoted by (epsilon)

• Examples
abfbz is a string over 1 = {a, b, c, d, …, z}
9021 is a string over 2 = {0, 1, …, 9}
ab#bc is a string over 3 = {a, b, …, z, #}
))()(() is a string over 4 = {(, )}
Strings
Let: u=ε w = 0110 y = 0aa x = aabcaa z = 111
concatenation: wz = 0110111
length: |w| = 4 |x| = 6 but |u| = 0
reversal: yR = aa0
Some special sets of strings
• Σ* All strings of symbols from Σ
• Σ+ Σ* - {ε}
Example: Σ = {0, 1}
• Σ* = {ε, 0, 1, 00, 01, 10, 11, 000, 001,…}
• Σ+ = {0, 1, 00, 01, 10, 11, 000, 001,…}
What is Formal Language ?

A (formal) language is a set of strings over an alphabet.

i.e. any subset L of Σ*

• Classes of formal languages (e.g., regular, context-free,

context-sensitive, recursive, recursively enumerable).
• Formal languages are related to programming languages and
natural languages
Formal Language Examples:
Σ = {0, 1} 
L = {x | x is in Σ* and x contains an even number of 0’s}
= {00, 010,11100,00100,…}


\
Σ = {a, b, c,…,
 z, A, B,…, Z}
L = {x | x is in Σ* and x is a CPP reserved word}
= {while, for, if, int, …}


Formal Language Examples:
Σ = {CPP reserved words} U { (, ), ., :, ;,…} U {Legal CPP
identifiers}


L = {x | x is in Σ* and x is a syntactically correct CPP

program}


Σ = {English words}
L = {x | x is in Σ* and x is a syntactically correct English
sentence}

What is automata theory ?
• Automata theory is the study of abstract computational devices
• Abstract devices are (simplified) models of real computations
• Computations happen everywhere: On your laptop, on your cell
phone, in nature, …
• Why do we need abstract models?
A simple “computer”
H
W ITC
S

BATTERY

input: switch
output: light bulb
actions: flip switch
states: on, off
A simple “computer”
H
W ITC
S

BATTERY start off on

input: switch
bulb is on if and only if there is
output: light bulb
an odd number of flips
actions: f for “flip switch”
states: on, off
Another “computer”
1
1 start off off
1

2 2 2 2
BATTERY
1
2
off on
1

inputs: switches 1 and 2

bulb is on if and only if both
actions: 1 for “flip switch 1”
switches were flipped an odd
actions: 2 for “flip switch 2”
number of times
states: on, off
A design problem
1 4

?
5
BATTERY

Can you design a circuit where the light is on if and only if all the
switches were flipped exactly the same number of times?
A design problem
• Such devices are difficult to reason about, because they can be
designed in an infinite number of ways
• By representing them as abstract computational devices, or
automata, we will learn how to answer such questions
These abstract devices can model many things
• They can describe the operation of any “small computer”, like the
control component of an alarm clock or a microwave
• They are also used in lexical analyzers to recognize well formed
expressions in programming languages:

ab1 is a legal name of a variable in C++

5u= is not
Different kinds of automata
• This was only one example of a computational device, and there
are others
• We will look at different devices, and look at the following
questions:
– What can a given type of device compute, and what are its limitations?
– Is one type of device more powerful than another?
Some devices we will see
finite automata Devices with a finite amount of memory.
Used to model “small” computers.

push-down automata Devices with infinite memory that can be accessed in

a restricted way.
Used to model parsers, etc.

Turing Machines Devices with infinite memory.

Used to model any computer.
General Model of Automaton
● In computer science the term 'automaton'
means 'discrete automation' and is defined in a
more abstract way as shown in figure below.

22
General Model of Automaton

23
General Model of Automaton
● The characteristics of an automaton are:
4) State relation: The next state of an automaton at
any instant of time is determined by the present
state and input.
5) Output relation: The output is related to either state
only or to both the input and the state.
• It should be noted that at any instant of time the automaton
is in some state.
• On 'reading' an input symbol, the automaton moves to a next
state which is given by the state relation.

24
Finite Automata
Finite Automata
• Is an abstract computing device.
• It is a mathematical model of system with discrete input, output
and set of transitions from state to the state that occurs on input
symbols from alphabet.
• It represented by three models;
 Graphical (transition diagram)
 Tabular (transition table)
 Mathematical (transition function or mapping function)
Example of a finite automaton

off on

• There are states off and on, the automaton starts in off and tries to
reach the “good state” on
• What sequences of fs lead to the good state?
• Answer: {f, fff, fffff, …} = {f n: n is odd}
• This is an example of a deterministic finite automata over alphabet
{f}
Deterministic finite automata
• A deterministic finite automata (DFA) is a 5-tuple (Q, , , q0, F)
where
– Q is a finite set of states
–  (sigma) is an alphabet
– : Q ×  → Q is a transition function
– q0  Q is the initial state
– F  Q is a set of accepting states (or final states).
• In diagrams, the accepting states will be denoted by double loops
Deterministic finite automata
• At the initial time, it is assumed to be in the initial state q0,
with its input mechanism on the leftmost symbol of the input
string.
• During each move of the automation, the input mechanism
advances one position to the right, so each move consumes one
input symbol.
• When the end of the string is reached, the string is accepted if the
automation is in one of its final states; otherwise the string is
rejected.
Example

0 1 0,1

q0 1 q1 0 q2

State transition diagram

alphabet  = {0, 1} Transition table

Set of states Q = {q0, q1, q2} inputs
initial state q0 0 1
accepting states F = {q0, q1} q0 q0 q1

states
q1 q2 q1
For every transition rule (qi,a)= qj, the graph
q2 q2 q2
has an edge (qi, qj) labeled a.
The vertex associated with q0 is called the initial
vertex, while those labeled with qf are the final
vertices.
Language of a DFA

The language of a DFA (Q, , , q0, F) is the set of

all strings over  that, starting from q0 and
following the transitions as the string is read left
to right, will reach some accepting state.

M: off on

• Language of M is {f, fff, fffff, …} = {f n: n is odd}

Examples
0,1
0 1
1
q0 q1
1
q0 q1
0 0

q2
0,1

0 1 0,1

q0 1 q1 0 q2

What are the languages of these DFAs?

Examples
• Construct a DFA that accepts the language

L = {010, 1} (  = {0, 1} )

• Answer
q0 1 q01 0 q010
0

q

1 q1
Examples
• Construct a DFA that accepts the language

L = {010, 1} (  = {0, 1} )

• Answer
q0 1 q01 0 q010
0
0 1
q 0, 1
1 q1 0, 1 qdie
0, 1
Examples
• Construct a DFA over alphabet {0, 1} that accepts all strings that
end in 101
Exercise
• Construct a DFA over alphabet {0, 1} that accepts all strings that
end in 101

• Hint: The DFA must “remember” the last 3 bits of the string it is
reading