Formal Languages

and
Theory of Automata
Formal Languages and Theory of Automata

Chapter 1– Introduction
Basic Concepts of Finite Automata and
Languages
Outline
• Introduction
• Alphabets and Strings
• What is a Formal Language?
• What is Automata Theory?
• General Model of Automaton
• Introduction to Finite Automata
Introduction
• The term Formal Languages refers to languages that can be
described by a body of systematic rules – simple and precise
mathematical rules.
• The formal language theory as a discipline started from the work
of linguist Noam Chomsky in the 1950s –
– When he attempted to give a precise characterization of the structure of
natural languages.
– His goal was to define the syntax of languages using simple and precise
mathematical rules.
– Later it was found that the syntax of programming languages can be
described using one of Chomsky’s grammatical models called Context
Free Grammars.
Introduction – cont.…
• Formal Language theory mainly deals with the mathematical
properties of strings and collection of strings.
• Soon after the advent of modern electronic computers, people
realized that all forms of information – whether numbers, names,
pictures, or sound waves can be represented as strings.
• Then collection of strings known as languages became central to
computer science.
• Also theoretical computer science studies the mathematical
foundation of computer science
– which investigates the power and limitations of computing devices using
abstract mathematical models of computers.
Introduction – cont.…
• Models should be as simple as possible so that they can be easily
analyzed; yet they have to be powerful enough to be able to perform
relevant computation process.
– The model ignores the implementation details of individual computers and
concentrates on the actual computation process.
• Real computers are modeled with a very simple abstract mathematical
model – called Turing Machine.
– Using it we are able to prove that whether some computational problems can be
algorithmically solvable (decidable) or cannot be algorithmically solvable (i.e.
undecidable).
• Automata theory is the study of abstract machines and automata as well
as the computational problems that can be solved using them.
Introduction – cont.…
• An automaton is an abstract model of machines that perform
computation on an input.
• Automata theory and formal language theory are closely related, where
an automaton is a finite representation of a formal language that may
be an infinite set.
• Throughout the course, we deal with the relationship between formal
languages and automaton by constructing accepting devices for four
types of formal languages:
– Regulars language and Finite automata
– Context free languages and Pushdown automata
– Context sensitive languages and Linear-bounded automata
– Recursively enumerable languages and Turing machines.
Definitions
• Symbol – An atomic unit, such as a digit, character, lower-case
letter, etc. Sometimes a word. [Formal language does not deal with
the “meaning” of the symbols.]

• Alphabet – A finite set of symbols, usually denoted by Σ.

Σ = {0, 1} Σ = {0, a, 4} Σ = {a, b, c, d}

• String – A finite length sequence of symbols, presumably from some

alphabet.
Alphabets and strings
• A common way to talk about words, number, pairs of words, etc.
is by representing them as strings
• To define strings, we start with an alphabet
An alphabet is a finite set of symbols.

• Examples
1 = {a, b, c, d, …, z}: the set of letters in English
2 = {0, 1, …, 9}: the set of (base 10) digits
3 = {a, b, …, z, #}: the set of letters plus the
special symbol #
4 = {(, )}: the set of open and closed brackets
Strings

A string over alphabet  is a finite sequence

of symbols in .

• The empty string will be denoted by (epsilon)

• Examples
abfbz is a string over 1 = {a, b, c, d, …, z}
9021 is a string over 2 = {0, 1, …, 9}
ab#bc is a string over 3 = {a, b, …, z, #}
))()(() is a string over 4 = {(, )}
Strings
Let: u=ε w = 0110 y = 0aa x = aabcaa z = 111
concatenation: wz = 0110111
length: |w| = 4 |x| = 6 but |u| = 0
reversal: yR = aa0
Some special sets of strings
• Σ* All strings of symbols from Σ
• Σ+ Σ* - {ε}
Example: Σ = {0, 1}
• Σ* = {ε, 0, 1, 00, 01, 10, 11, 000, 001,…}
• Σ+ = {0, 1, 00, 01, 10, 11, 000, 001,…}
What is Formal Language ?

A (formal) language is a set of strings over an alphabet.

i.e. any subset L of Σ*

• Classes of formal languages (e.g., regular, context-free,

context-sensitive, recursive, recursively enumerable).
• Formal languages are related to programming languages and
natural languages
Formal Language Examples:
Σ = {0, 1}

L = {x | x is in Σ* and x contains an even number of 0’s}
= {00, 010,11100,00100,…}


Σ = {0, 1, 2,…, 9, .}
L = {x | x is in Σ* and x forms a finite length real number}
= {0, 1.5,

9.326,…}
Σ = {a, b, c,…, z, A, B,…, Z}
L = {x | x is in Σ* and x is a CPP reserved word}
= {while, for, if, int, …}


Formal Language Examples:


Σ = {CPP reserved words} U { (, ), ., :, ;,…} U {Legal CPP

identifiers}


L = {x | x is in Σ* and x is a syntactically correct CPP

program}


Σ = {English words}
L = {x | x is in Σ* and x is a syntactically correct English
sentence}

What is automata theory ?
• Automata theory is the study of abstract computational devices
• Abstract devices are (simplified) models of real computations
• Computations happen everywhere: On your laptop, on your cell
phone, in nature, …
• Why do we need abstract models?
A simple “computer”
H
I TC
SW

BATTERY

input: switch
output: light bulb
actions: flip switch
states: on, off
A simple “computer”
H
I TC
SW

BATTERY start off on

input: switch
bulb is on if and only if there is
output: light bulb
an odd number of flips
actions: f for “flip switch”
states: on, off
Another “computer”
1
1 start off off
1

2 2 2 2
BATTERY
1
2
off on
1

inputs: switches 1 and 2

bulb is on if and only if both
actions: 1 for “flip switch 1”
switches were flipped an odd
actions: 2 for “flip switch 2”
number of times
states: on, off
A design problem
1 4

?
5
BATTERY

Can you design a circuit where the light is on if and only if all the
switches were flipped exactly the same number of times?
A design problem
• Such devices are difficult to reason about, because they can be
designed in an infinite number of ways
• By representing them as abstract computational devices, or
automata, we will learn how to answer such questions
These abstract devices can model many things
• They can describe the operation of any “small computer”, like the
control component of an alarm clock or a microwave
• They are also used in lexical analyzers to recognize well formed
expressions in programming languages:

ab1 is a legal name of a variable in C++

5u= is not
Different kinds of automata
• This was only one example of a computational device, and there
are others
• We will look at different devices, and look at the following
questions:
– What can a given type of device compute, and what are its limitations?
– Is one type of device more powerful than another?
Some devices we will see
finite automata Devices with a finite amount of memory.
Used to model “small” computers.

push-down automata Devices with infinite memory that can be accessed in

a restricted way.
Used to model parsers, etc.

Turing Machines Devices with infinite memory.

Used to model any computer.
Preliminaries of automata theory
• How do we formalize the question

Can device A solve problem B?

• First, we need a formal way of describing the problems that we

are interested in solving
Problems
• Examples of problems we will consider
– Given a word s, does it contain the subword “fool”?
– Given a number n, is it divisible by 7?
– Given a pair of words s and t, are they the same?
– Given an expression with brackets, e.g. (()()), does every left bracket
match with a subsequent right bracket?
• All of these have “yes/no” answers.
• There are other types of problems, that ask “Find this” or “How
many of that” but we won’t look at those.
General Model of Automaton
● An automaton is defined as a system where
energy, materials and information are transformed,
transmitted and used for performing some
functions without direct participation of man.

● Examples are
● automatic machine tools,
● automatic packing machines, and
● automatic photo printing machines, etc.

26
General Model of Automaton
● In computer science the term 'automaton'
means 'discrete automation' and is defined in a
more abstract way as shown in figure below.

27
General Model of Automaton

28
General Model of Automaton
● The characteristics of an automaton are:
4) State relation: The next state of an automaton at
any instant of time is determined by the present
state and input.
5) Output relation: The output is related to either state
only or to both the input and the state.
• It should be noted that at any instant of time the automaton
is in some state.
• On 'reading' an input symbol, the automaton moves to a next
state which is given by the state relation.

29
General Model of Automaton
● We can note the following with regard to output relation:
i. An automaton in which the output depends only on the
input is called an automaton without a memory.

ii. An automaton in which the output depends on the states

as well, is called automaton with a finite memory.

iii. An automaton in which the output depends only on the

states of the machine is called a Moore machine.

iv. An automaton in which the output depends on the state

as well as on the input at any instant of time is called a
Mealy machine.
30
Finite Automata
Example of a finite automaton

off on

• There are states off and on, the automaton starts in off and tries to
reach the “good state” on
• What sequences of fs lead to the good state?
• Answer: {f, fff, fffff, …} = {f n: n is odd}
• This is an example of a deterministic finite automata over alphabet
{f}
Deterministic finite automata
• A deterministic finite automata (DFA) is a 5-tuple (Q, , , q0, F)
where
– Q is a finite set of states
–  (sigma) is an alphabet
– : Q ×  → Q is a transition function
– q0  Q is the initial state
– F  Q is a set of accepting states (or final states).
• In diagrams, the accepting states will be denoted by double loops
Deterministic finite automata
• At the initial time, it is assumed to be in the initial state q0,
with its input mechanism on the leftmost symbol of the input
string.
• During each move of the automation, the input mechanism
advances one position to the right, so each move consumes one
input symbol.
• When the end of the string is reached, the string is accepted if the
automation is in one of its final states; otherwise the string is
rejected.
Example

0 1 0,1

q0 1 q1 0 q2

State transition diagram

alphabet  = {0, 1} Transition table

Set of states Q = {q0, q1, q2} inputs
initial state q0 0 1
accepting states F = {q0, q1} q0 q0 q1

states
q1 q2 q1
For every transition rule (qi,a)= qj, the graph
q2 q2 q2
has an edge (qi, qj) labeled a.
The vertex associated with q0 is called the initial
vertex, while those labeled with qf are the final
vertices.
Language of a DFA

The language of a DFA (Q, , , q0, F) is the set of

all strings over  that, starting from q0 and
following the transitions as the string is read left
to right, will reach some accepting state.

M: off on

• Language of M is {f, fff, fffff, …} = {f n: n is odd}

Examples
0 0
1
q0 q1
1

0 1
1
q0 q1
0

0 1 0,1

q0 1 q1 0 q2

What are the languages of these DFAs?

Examples
• Construct a DFA that accepts the language

L = {010, 1} (  = {0, 1} )

• Answer
q0 1 q01 0 q010
0

q

1
q1
Examples
• Construct a DFA that accepts the language

L = {010, 1} (  = {0, 1} )

• Answer
q0 1 q01 0 q010
0
0 1
q 0, 1
1 0, 1
q1 qdie
0, 1
Examples
• Construct a DFA over alphabet {0, 1} that accepts all strings that
end in 101
Examples
• Construct a DFA over alphabet {0, 1} that accepts all strings that
end in 101

• Hint: The DFA must “remember” the last 3 bits of the string it is
reading