1

Theoretical Computer Science

Matteo Pradella
Email: [email protected]

DEIB, building 22, floor 3; phone: 3495

 2

Objectives and motivations

Why Theoretical Computer Science in an Engineering Curriculum?
Theory is stimulated by practice,
practice is helped by theory:
generality, rigor, insight, “control”

• Engineer is a person who:

– Applies theory to practice
– Extracts theory from practice
 3

• Deeply and critically understand the principles of

Computer Science (careful re-view of basic CS notions)
• Build solid ground to understand and master innovation
(e.g.: multimedia, modeling of concurrent and wireless
computation)
• Theory as an antidote to overspecialization and as a
support to interdisciplinary studies
 4

The program (1/2)

• Modeling in Computer Science

(How to describe a problem and its solution):
Do not go deep into specific models, rather
provide ability to understand models and invent new ones
• Theory of Computation:
what can I do by means of a computer
(which problems can I solve)?
 5

The program (2/2)

• Complexity theory:
how much does it cost to solve a problem through a
computer?
• Only the basics: further developments in follow-up
courses
 6

Organization (1/3)
• Requirements :
– Basics of Computer science (CS 1)
– Elements of discrete mathematics (Algebra and Logic)
• Lesson and practice classes (all rather classical style…)
– Student-teacher interaction is quite appreciated:
• In classroom
• In my office (when possible)
• By Email (for administrative matters and to fix face-to-face meetings)
 7

Organization (2/3)

• Exam tests ability to apply, not to repeat:

mainly written
open book (you can consult textbooks and notes)
not repetitive, challenging (hopefully)
• Standard written exams on the whole program
 8
Organization (3/3)
• Teaching material:
– Official text :
• Ghezzi/Mandrioli: Theoretical Foundations of Computer Science [Wiley]
– Not re-printed by the publisher: you get it, possibly, in a legal way, see the WeBeep course
website
• (In Italian: Dino Mandrioli, Paola Spoletini, Informatica Teorica, 2nd Ed. Città Studi, 2011.)
– Dino Mandrioli, Paola Spoletini, Mathematical logic for computer science: an Introduction,
Esculapio Società Editrice, 2010.
– D. Mandrioli, D. Martinenghi, A. Morzenti, M. Pradella, M. Rossi: Lecture Notes on Monadic
First- and Second-Order Logic on Strings, lecture notes, 2019, on the course website
– English translation of the Italian Exercise book: Mandrioli, Lavazza, Morzenti, San Pietro,
Spoletini, Esercizi di informatica teorica, 3rd edition, Esculapio, on the WeBeep course
website
– Previous exam texts with solutions, on the course website
– Lesson slides (NOT to use as a substitute of the text book! ), on the course website
 9

Use of formal models

1 Formalization of the problem: from reality to mathematical

abstraction
2 Solving of the formalized problem
3 Interpretation of the results =>
evaluation of the design choices

A model is adequate if the results obtained from it correspond to

the properties we want to study in the physical system (within
acceptable approximations)
 10

Models for Computer Science

Main phases of Software Engineering:

1 Requirement analysis => specification document
2 Design => sw architecture
3 Implementation => code

Often natural language or semi-formal diagrams are used.

Nowadays, the usage of formal languages and models is

becoming stronger.
=>
Better reliability and flexibility; automatic tools
 11

Models in Computer Science

• Not only discrete vs. continuous

(bits & bytes vs. real numbers and continuous functions)

• Operational models
(abstract machines, dynamic systems, …)
based on the concept of a state and of means (operations) to
represent its evolution (chronologically, i.e., w.r.t. “time”)

• Descriptive models
aimed at expressing properties (desired or feared) of the
modeled system, rather than its functioning as an evolution in
time through states
 12

Examples

• Operational model of an ellipsis (how you can draw it):

• descriptive model of it (property of the coordinates of its points):

ax2+by2=c
 13

• Operational definition of sorting:

– Find minimum element and put it in first position;
– Find minimum element among the remaining ones and put it in
second position;
–…
• Descriptive definition of sorting:
– A permutation of the sequence such that

i, a[i]  a[i+1]

 14

• In fact, differences between operational and descriptive

models are not so sharp
– (grammars are somewhere in between)
• It is however a very useful classification of models
– Corresponds closely to different “attitudes” that people adopt
when reasoning on systems
 15

A first, fundamental, “meta”model:

the notion of language

• Italian, French, English, …

• C, Pascal, Ada, …
but also:
• Graphics
• Music
• Multimedia, ...
 16

Elements of a language

• Alphabet or vocabulary
(from a mathematical viewpoint these are synonyms):
Finite set of basic symbols
{a,b,c, …z}
{0,1}
{Do, Re, Mi, …}
ASCII char set
...
 17

• String (over an alphabet A):

ordered finite sequence of elements of A, possibly
with repetitions
a, b, aa, alpha, john, leaves of grass, …
• Length of a string:
|a| = 1, |ab| = 2
• Null string, or empty string, e: |e| = 0
• A* = set of all strings, including e, over A
A = {0,1}, A* = {e, 0, 1, 00, 01, 10, …}
 18

• Operations on strings:
concatenation (product): x  y

x = abb, y = baba, xy = abbbaba

x = Quel ramo, y = del lago di Como,
x  y = Quel ramo del lago di Como
“” is an associative, non-commutative operation

• A* is called
free monoid over A built through “”
• e is the unit w.r.t. “” : for any x,   x = x   = x
 19

Language

• L is any set of strings, i.e. subset of A*: (L A*)

Italian, C, Pascal, … but also:
sequences of 0 and 1 with an even number of 1
the set of musical scores in F minor
symmetric square matrices
…
• It is a very broad notion, somehow universal
 20

Operations on languages

• Set theoretic operations:

, , L1 - L2, ¬L = A* - L, ( L  L )

• Concatenation (of languages):

L1  L2 = { x y | x  L1, y  L2 }

L1 = {0, 1}*, L2 = {a, b}* ;

L1  L2 = {e, 0, 1, 0a, 11b, abb, 10ba, ….} NB: ab1 not included!
 21
21

●
L0 = {}, Li = Li-1.L

●
L = U Ln
*
n=0
●
NB: {}  !
{} . L = L;
.L=
●
+ = “* - 0”: A+ = set of all strings on A.
A = {0,1}, A+ = {0, 1, 00, 01, 10, …}

●
L = U Ln
+
n=1
 22

Some practical examples

• let
– L1 : set of “MS Office” documents, and
– L2 : set of “LibreOffice” documents; then
– L1  L2 : set of documents that are “compatible MSOffice - LibreOffice”
• Composition of a message over the net:
– x  y  z:
– x = header (address, …)
– y = text
– z = “tail”
•
– L1 : set of e-mail messages
– L2 : set of SPAM messages
– Filter: L1 - L2
 23
• A language can be a means of expression … of a problem
• x  L?
– Is a message correct? Is a program correct?
– y = x2 ?
– z = Det(A)? (is z the determinant of matrix A?)

• Every string can be encoded in binary

– every binary string can be interpreted as a natural number ...
– … hence every problem can be recast (by suitable encoding) into an inclusion
problem for a set of natural numbers
 24

Translation

• y = t (x)
t : translation is a function from L1 to L2
– t 1 : double the “1” (1  11):
t 1(0010110) = 0011011110, …
– t 2 : change a with b and viceversa (a  b):
t 2(abbbaa) = baaabb, …
– Other examples:
• File compression
• Self-correcting protocols
• Computer language compilation
• translation Italian  English
 25
Conclusion
• The notion of language and the basic associated operations
provide a very general and expressive means to describe any
type of systems, their properties and related problems:
– Compute the determinant of a matrix;
– Determine if a bridge will collapse under a certain load;
– ….
• Notice that, after all, in a computer any information is a string of
bits (hence a string of some language…)
 26

Operational Models
(state machines, dynamical systems)

• Finite State Machines (or automata) (FSA, or FA):

– A finite state set:
{ on, off }, ….
{1, 2, 3, 4, …k}, {TV channels}, …

Graphic representation:

On Off
 27

Commands (input) and state transitions

• Two very simple flip-flops:

T
On Off

S
R R
On Off

S
Turning a light on and off, ...
 28

A first formalization

• A finite state automaton is (made of):

– A finite state set: Q
– A finite input alphabet: I
– A transition function (partial, in general):
d: Q  I  Q
 29

Automata as language recognizers (or acceptors)

(x  L?)
• A move sequence starts from an initial state and it is
accepting if it reaches a final (or accepting) state.

L = {strings with an even number of “1” and any number of “0”}

0 0
1
q0 q1

1
 30

Formalization of the notion of acceptance of language L

• Move sequence for an input string (not just one symbol):

– d*: Q  I*  Q
d* defined inductively from d, by induction on the string length
d* (q, e) = q
d* (q, yi) = d( d* (q,y), i)
• Initial state: q0  Q
• Set of final, or accepting states: F  Q
• Acceptance: x  L  d* (q0, x)  F
 31
Accepting Pascal identifiers
0
1
<letter> 2
<letter> <digit>
q0 ...
q1 9
a
<digit> b
c
qE <letter>
<digit> ...
<digit> Z
<letter>
NB: the red part might be omitted

a useful notation/shorthand that is occasionally adopted

* a
means for all a  alphabet I
 32
Automata as language translators
y = t(x)

Transitions with output: notice that w is a string (possibly )

i/w
q q’

t: doubles the number of “1” and halves the “0” (the input must have an even number of “0”)

1/11 1/11
0/e
q0 q1

0/0
 33

Formalization of translating automata (transducers)

• T = <Q, I, d, q0, F, O, h>

– <Q, I, d, q0, F>: just like for acceptors
– O: output alphabet
– h : Q  I  O* (NB: the output is a string)
• h* : Q  I*  O*, h* defined inductively as usual
h*(q, e) = e
h*(q, yi) = h*(q, y)  h(d* (q,y), i)
• definition of the translation:
t(x) = h*(q0, x) iff d* (q0, x)  F
 34

Analysis of the finite state model

(for the synthesis refer to other courses - e.g. on digital circuits)

• Very simple, intuitive model, applied in various applicative

domains, also outside computer science
• Is there a price for this simplicity?
• …
• A first, fundamental property: the cyclic behavior of finite
automata
 35
b a b
q0 q1 q2 q9

b b a a

q5 q4 q3 q8
a b
a
b
q6 q7

There is a cycle q1 ----aabab---> q1

If, when reading a string, one goes through the cycle once, then
one can also go through it 2, 3, …, n, … times
 36

Formally:

• If x  L and |x| > |Q| then there exists a q  Q and a w  I + such that:
x = ywz
d* (q, w) = q

and then y wn z  L,  n ≥ 0

• This is known as the Pumping Lemma

 37
Several properties of FA –both good and bad ones–
follow from the pumping lemma and other properties of the graph of 

• Let A be an FA, and L=L(A) the language accepted by A

• |L| = ? |L| =  iff there exists a cycle on the graph
of  with a node on a path from the initial
state q0 to a final state qF
• L = ?  x  L   y  L, |y| < |Q|:
proof: Eliminate all cycles from A, then look for a path from
initial state q0 to a final state qF
• Notice that being able to answer the question “x  L?” for every
possible given string x (i.e., for a generic string) does not enable us
to answer other questions, such as the emptiness of the accepted
language
 38
A “negative” consequence of the Pumping Lemma (PL)

• Is the language L = {anbn | n > 0} recognized by any FA?

– Put another way: can we count using a FA?
• Let us assume so (by contradiction):
• Consider x = ambm, with m > |Q| and apply the PL: then x = ywz and
ywnzL n
• There are 3 possible cases:
– x = ywz, w = ak , k > 0 ====> am+r.kbm  L, r : this cannot be (it
contradicts the assumption)
– x = ywz, w = bk , k > 0 ====> idem
– x = ywz, w = ak bs , k,s > 0 ====> am-kakbs akbsbm-s  L: this
cannot be (it contradicts the assumption)
 39

• Intuitive conclusion: to “count” any n one needs an infinite memory!

• NB: strictly speaking any computer is a FA: so a computer cannot count?
– But this is a wrong abstraction of (way of looking at) a computer!
– It is important to consider a different, less immediate notion of infinity!
– We consider the memory of a computer infinite as long as all the strings that it
manipulates (confortably) fit into its (finite) memory.
• Going from the toy example {anbn} to more concrete ones:
– Recognizing parenthetical structures like those of the programming languages
cannot be done with a finite memory
• Therefore we need “more powerful” models
 40

Closure properties of FA

• The mathematical notion of closure:

– Natural numbers are closed under the sum operation
– But not under subtraction
– Integer numbers are closed under sum, subtraction,
multiplication, but not …
– Rational numbers …
– Real numbers …
– Closure (under operations and relations) is a very important
notion
 41

In the case of Languages:

• L = {Li}: a family of languages

• L is closed under the binary operator OP if and only if (iff)
for every L1 , L2  L , L1 OP L2  L
• R : regular languages, those accepted by an FA
• R is closed under set theoretic operations, concatenation,
“*”, … and virtually “all others”
 42

Intersection

b a a
A q0 q1 q2 q9

b a a
B p0 p1 p2 p9

One can simulate the “parallel run” of A and B by simply

“coupling them”:

b a a
<A,B> q0 p0 q1 p1 q2 p2 q9 p9
 43
Example
a
a
S0 S1 b,c c
T0 T1
a b,c b
c

S0 T0
a c
c
S0 T1 b a S1 T0

c b
S1 T1

c
 44

Formally:

• Given A1 = <Q1, I, d1, q01, F1> and

A2 = <Q2, I, d2, q02, F2>
• The automaton < A1, A2 > is defined as:
<Q1  Q2, I, d, <q01 , q02 >, F1  F2 >
d(<q1 , q2 > , i) = <d1(q1, i), d2(q2,i)>
• One can show by a simple induction that
L(< A1, A2 >) = L(A1)  L( A2 )
 45
Union
• A similar construction …
otherwise ... exploit identity A  B = (A  B)
 need a FA for the complement language

Complement: example automaton accepting

binary strings x with odd number of 1’s

0 0
1
q0 q1

1
An idea: F^ = Q - F:
Yes, it works for the automaton above, but ….
 46

0 0
1 L: strings with
exactly one ‘1’
q0 q1

The “complement F construction”, i.e., turning F into Q – F, doesn’t work

Why? because d is partial

0 0
1 L: strings with no
q0 1 q1 ‘1’ or with more
than one ‘1’
qE
0
1
 47

Some general remarks on the complement

• If the analysis of the string terminates for all possible strings then it
suffices the “turn a yes into a no” (F into Q-F)
• If, for some string, the analysis of the string does not terminate (it
gets blocked or continues forever) then turning F into Q-F does not
work
• With FA the problem is easily solved …
• In general if we are unable to provide a positive answer to a
problem (e.g., xL), this does not necessarily mean we can provide
a positive answer for the complement problem (e.g., xL)
 48

Let us increase the power of the FA

by increasing its memory
• Consider a more “mechanical” view of the FA:

Input tape

Control device
(finite state)

Output tape
 49

• Now let us “enrich it” :

“stack” memory
Input tape
a

Control device
(finite state) q p

A
x
B

Output tape
Z0
Z0 stack bottom symbol
 50
The move of the pushdown automaton (with a stack memory):
• Depending on :
– the symbol read on the input tape (but it could also ignore the input …)
– the symbol read on top of the stack
– the state of the control device:
• the pushdown automaton
– changes its state
– moves ahead the scanning head (or it does not if input was ignored)
– changes the symbol A read on top of the stack with a string a of
symbols (a may be empty: this amounts to a pop of A)
– (if translator) it writes a string (possibly empty) on the output tape
(advancing the writing head consequently)
 51
• The input string x is recognized (accepted) if
– The automaton scans it completely (the scanning head reaches
the end of x), and
– Upon reaching the end of x it is in an acceptance state (just like
for the FA)
• If the automaton is also a translator, then
t(x) is the string on the output tape after x has been
accepted (if x is accepted, otherwise t(x) is undefined:
t(x) = )

NB:  is the “undefined” symbol, but not in any alphabet:

t(x) =  is just a shorthand for y (t(x) = y)
 52

A first example: accepting {anbn | n > 0}

a,A/AA b,A/e

a,Z0/ B Z0
q0 q1 q2 q3
b,A/e
b,B /e

a,B/AB
b,B /e

}
the B in the stack marks
the first symbol of x, to
match the last symbol A
n - 1 A’s
A
B
Z0
 53

Another one:

a,A/AA b,A/e

a,Z0/ A Z0
q0 q1 q2 q3
b,A/e e, Z0 / Z0

e-move (AKA spontaneous move)

the ‘’ symbol is «overloaded»: used as an additional input symbol
but it is not a symbol: I
 54
A (classical) pushdown automaton-translator
It reverses a string: (wc)=wR, w{a,b}*

e,A/ e, a

a,Z0/ AZ0,e
a,A/ AA,e

c,A/A, e,Z0 / Z0,e

c,B/B,
c,Z0/Z0,
b,Z0/ BZ0,e q1 q2
q0
a,B/ AB,e

b,B/ BB,e e,B/ e, b

b,A/ BA,e
 55
Now we formalize ...
• Pushdown automaton [transducer]: <Q, I, G, d, q0, Z0 , F [, O, h]>
• Q, I, q0, F [O] just like FA [FST]
• G stack alphabet (disjointed from other ones for ease of definition)
• Z0 : initial stack symbol (not essential, useful to simplify definitions)
• d: Q  (I  {})  G  Q  G* d is partial just as in
FSA
• h: Q  (I  {})  G  O* ( h defined where d is)
Graphical notation: <p,a> = d(q,i, A)
w = h(q,i, A)

i,A/a,w
q p
 56

• Configuration (a generalization of the notion of state):

c = <q, x, g, [z]>:
– q: state of the control device
– x: unread portion of the input string (the head is positioned
on the first symbol of x)
– g : string of symbols in the stack
(convention: <top-left, bottom-right>)
– z: string written (up to now) on the output tape
 57
• Transition relation ‘├’ among configurations:

• c = <q, i.y, A, [z]> ├ c' = <q', x', , [z.w]>

Case 1: (q, i, A) = <q', > [(q, i, A) = w]

– x' = y

Case 2: (q, , A) = <q', > [(q, , A) = w]

– x' = i.y
●
NB: q,A,i (d(q, i, A)    d(q, e, A) = ) (hence d(q, e, A)    d(q, i, A) = )
– i.e., -moves are alternative to all input-consuming ones
• Otherwise … nondeterminism!
 58

• Acceptance [and translation] of a string

• ├* : reflexive, transitive closure of the relation ├
– i.e., ├* denotes a number 0 of “steps” of the relation├
x  L [z = t(x)] 
c0 = <q0, x, Z0, [e]> ├* cF = <q, e, g, [z]>, q  F

stack last
input string
content state
completely
immaterial final
scanned

Pay attention to e-moves, especially at the end of the string!

We assume that the PDA executes all possible final e-moves
 59

Pushdown automata in practice

• They are the heart of compilers

• Stack memory (LIFO) suitable to analyze nested
syntactic structures (arithmetical expressions,
compound instructions, …)
• Abstract run-time machine for programming languages
with recursion
• ….
Occur very frequently in the course of Languages and
translators
 60
Properties of pushdown automata
(especially as acceptors)
• {anbn | n > 0} is accepted by a pushdown automaton (not by a FA)
– However {anbncn | n > 0} ….
– NOT: after counting –using the stack- n a’s and “de-counting” n b’s
how can we remember n to count the c’s?
The stack is not a read-only memory: to read it, one must destroy it!
This limitation of the pushdown automaton can be proved formally
through a generalization of the pumping lemma.
 61
Properties of pushdown automata
(especially as acceptors)

• {anbn | n > 0} accepted by a pushdown automaton;

{anb2n | n > 0} accepted by a pushdown automaton

• However {anbn | n > 0}  {anb2n | n > 0} …

– Reasoning intuitively similar to the previous one:
• If I empty all the stack with n b’s then I am unable to count other b’s
• If I empty only half the stack and I do not find any more b I cannot know
if I am halfway in the stack
– The formalization of this reasoning is however not trivial ….
 62

Some consequences
• LP = class of languages accepted by pushdown automata
• LP is not closed under union nor intersection
• Why?

• Considering the complement …

The same principle as with FA: change the accepting states into
non accepting states.
There are however new difficulties
 63

• Function d must be made complete (as with FA) with an error state.
Pay attention to the nondeterminism caused by e-moves!
• The e-moves can cause cycles  never reach the end of the string
 the string is not accepted, but it is not accepted either by the
automaton with F^ = Q-F.
• There exists however a construction that associates to every
automaton an equivalent loop-free automaton
• Not finished yet: what if there is a sequence of e-moves at the end of
the scanning with some states in F and other ones not in F?
 64

<q1, , 1> ├ <q2, , 2> ├ <q3, , 3> ├ …

q1 F, q2  F, …. ?
• Then we must “force” the automaton to accept only at the end of a
(necessarily finite, as the automaton is loop-free) sequence of e-moves.
• This is also possible through a suitable construction.

Once more, rather than the technicalities of the construction/proof we are

interested in the general mechanism to accept the complement of a language:
sometimes the same machine that solves the “positive instance” of the
problem can be adapted to solve the “negative instance”: this can be trivial or
difficult, but we must be sure to be able to complete the construction
 65

Pushdown automata [as acceptors (PDA) or translators (PDT)] are more

powerful than finite state ones (a FA is a trivial special case of a PDA;
more, PDA have an unlimited counting ability that FA lack)
However also PDA/PDT have their limitations …
… a new and “last” (for us) automaton:

the Turing Machine (TM)

Historical model of a “computer”, simple and conceptually important
under many aspects.

We consider it as an automaton; then we will derive from it some

important, universal properties of automatic computation.

For now we consider the “k-tape” version, slightly different from the
(even simpler) original model. This choice will be explained later.
 66
k-tape TM
First memory tape
Input tape A

a
Second memory tape
B

q p
…..
x

Output tape k-th memory tape

D
 67

Informal Description and Partial Formalization of the TM

• States and alphabets as with other automata (input, output, control device, memory
alphabet)
• For historical reasons and due to some “mathematical technicalities” the tapes are
represented as infinite cell sequences [0,1,2, …] rather than finite strings. There
exists however a special symbol “blank” ( “ ”, or “barred b” or “_” ) and it is
assumed that every tape contains only a finite number of non-blank cells.
– The equivalence of the two ways of representing the tape content is obvious.
• A special symbol Z0 in the first cell of all memory tapes (as in the PDA)
• Scanning (input) and output heads are also as in previous models
 68

The move of the TM:

• Reading:
– one symbol on the input tape
– k symbols on the k memory tapes (one for each tape)
– state of the control device
• Action:
– State change: q ----> q’
– Write a symbol in place of the one read on each of the k memory tapes:
Ai ----> Ai’, 1 <= i <= k
– [Write a symbol on the output tape]
– Move of the k + 2 heads:
• memory and scanning heads can move one position right (R) or left (L) or
stand still (S)
• The output head can move one position right (R) or stand still (S)
 69

As a consequence:

  , [ ] : Q  I   k  Q   k  {R, L, S }k 1 [ O  {R, S }]

(partial!)
Graphical notation:

i,<A1,A2, …Ak>/[o], <A’1, A’2 …A’k> , <M0, M1 …Mk , [Mk+1 ]>

q q’

i[0..k] Mi  {R,L,S} Mk+1  {R,S}

Why do we not loose generality having O rather than O* for the output?
 70

• Initial configuration:
• Z0 followed by all blanks in the memory tapes
• [output tape all blank]
• Heads in the 0-th position on every tape
• Initial state of the control device q0
• Input string x starting from the 0-th cell of the input tape,
followed by all blanks
 71

• Final configurations:
– Accepting states F  Q
– For ease of notation, convention:
<d,[h]> (q, …) =   q  F: no further action from a final state
– The TM stops when <d,[h]> (q, …) = 
– Input string x is accepted iff:
• After a finite number of moves the TM stops (hence it is in a
configuration where <d,[h]> (q, …) =  )
• When it stops, its state q  F
 72

Formalization of acceptor TM (for simplicity we set k = 1)

Transition beween configurations:
• <q, xiy, A>├ <q’, x’y’, ’’> where:
– q, q’Q, iI, x,y,x’,y’I*, ,,a’,b’*, A
– (q, i, A) = <q’, A’, M, N>, M, N{S, R, L}
• M = S: x’ = x and y’ = iy
• M = R: x’ = xi, if y = e then y’ = _ else y’ = y
• M = L: x = x’a, y’ = aiy

• N = S: ’ = , ’ = A’
• N = R: ’ = A’, if  = e then ’ = _ else ’ = 
• N = L:  = ’B, ’ = BA’.
 73

Language of a TM:

A string x is accepted by M iff

• c = <q0, x, Z0>├* <qF, x’y’, ’’> = cF
(├* is the usual reflexive and transitive closure of├)
• where qF  F and c’: cF├ c’ (i.e. the machine halts)
• L(M) = {xI* | x accepted by M}

• Note: as a consequence
– x is not accepted if:
• The TM stops in a state q  F; or
• The TM never stops (NB: this case is very important)
– There is a similarity with the PDA (a non-loop-free PDA might also not accept
because of a “non stopping run”), however … does there exist a loop-free TM?
 74

Some examples

• A TM accepting {anbncn | n > 0}

Input tape
a a a b b b c c c
Memory tape
Z0 M M M

CD
 75

a, _ / M, <R, R>
b, M / M, <R, L>

a, Z0/Z0, <S, R>

q0 q1 b, _ / _, <S, L>
q2

c, Z0/Z0, <S, R>

_, _ / _, <S, S>
qF q3

c, M / M, <R, R>
 76
Computing the successor of a number n coded with decimal digits
two memory tapes T1 and T2

• M copies all digits of n on T1, to the right of Z0, while it moves head T2
by the same number of positions.
• M scans the digits of T1 from right to left. It writes on T2 from right to
left changing the digits as needed (9’s become 0’s, first digit9
increased by one, then all other ones to its left are unchanged, …)
• M copies T2 on the output tape.

• Notation: d : any decimal digit

• _ : blank
• # : any digit  9
• ^ : successor of a digit denoted as # (in the same transition)

Purpose of the example: to show that the TM can compute any function
 77

Input tape
1 3 4 9 9 9
T1
Z0 1 3 4 9 9 9

CD
T2
Z0
Output tape
 78
copy input on T1 Change rightmost 9’s into 0’s

d,_,_/_,<d,_>,<R,R,R,S>
d,Z0 ,Z0 /_,<Z0,Z0>,<S,R,R,S> _,9,_/_,< 9,0>,
q0 q1 q2
_,_,_/_,< _,_>,<S,L,L,S> <S,L,L,S>

_,d,_/_,<d,d>,<S,L,L,S> _,#,_/_,< #,^>,<S,L,L,S>

first digit <9 is increased
_,Z0,Z0/_,<Z0,Z0>,<S,R,R,S>
q3 q4

_,_,_/_,< _,_>,<S,S,S,S> _,d,d’/d’,<d,d’>,<S,R,R,R>

q5
copy rest of T1 on T2 copy all T2 on output
_,d, 0 / 0,<d, 0 >,<S,R,R,R>
then all others are 0
left end reached (all
9’s): output an initial 1
q6 q7
_,Z0,Z0/1,<Z0,Z0>,<S,R,R,R> _,_,_/_,< _,_>,<S,S,S,S>
 79
Closure properties of TM
•  : OK (a TM can easily simulate two other ones,
both “in series” and “in parallel”)
•  : OK (idem)
• Idem for other operations (concatenation, *, ….)

• What about the complement?

Negative answer! (Proof later on)

If there existed loop-free TM’s, it would be easy: it would
suffice to define the set of halting states (it is easy to make it
disjoint from the set of non-halting states) and partition it into
accepting and non-accepting states. ==========>
It is therefore apparent that the problem arises from
nonterminating computations
 80
Equivalent TM models
• Single tape TM (NB:  TM with 1 memory tape)

A single tape (usually unlimited in both directions):

serves as input, memory, and output

CD
 81
• Bidimensional tape TM

CD

• TM with k heads for each tape

• …..
 82

All versions of the TM are equivalent, w.r.t. their accepting or

translating ability, for instance:

x * Content # i-th tape *

Encodes the position

of the i-th head

CD
The state encodes the
symbols in the k + 1
cells under the heads
 83

What relations exist among the various automata (TM in

particular) and more traditional/realistic computing models?

•The TM can simulate a Von Neumann machine (which is also “abstract”)

•The main difference is in the way the memory is accessed: sequential rather than
“random” (direct)
•This does not influence the machine for what concerns computational power (i.e., the
class of problems it can solve)
•There can be a (profound) impact for what concerns the complexity of the
computations, i.e., the time and memory needed
• We will consider the implications in both cases
 84
Nondeterministic (operational) models

• Usually one thinks of an algorithm as a uniquely determined sequence of operations: in

a certain configuration there is no doubt on what the next “step” will be
• Are we sure that this is desirable?

Let us compare

if x > y then max := x else max := y

with

if x >= y then max := x

y >= x then max := y
fi
 85
• Is it only a matter of elegance?

• Let us consider the case construct of Pascal & others:

why not having something like the following?

case
– x=y then S1
– z > y +3 then S2
– …. then …
endcase
 86
Another form of nondeterminism which is usually “hidden”:
blind search

?
 87

• In fact, the search algorithms are a “simulation” of “basically nondeterministic”

algorithms:
• Is the searched element in the root of the tree?
• If yes, OK. Otherwise
– Search the left subtree
or
– Search the right subtree
• Choice of priority among various paths is often arbitrary
• If we were able to assign two tasks in parallel to two distinct machines ---->
• Nondeterminism as a model of computation or at least a model of design of parallel
computing
(For instance Ada and other concurrent languages exploit nondeterminism)
 88

Among the numerous nondeterministic (ND) models:

ND version of known models

• ND FA (we will soon see how handy it is)

q2
a

q1

a
q3

Formally: d(q1,a) = {q2, q3}

d : Q  I  (Q)
 89
d* : formalization of a move sequence

b q4
a q2
b
q1
q5
a b
q3

b
q6

d(q1,a) = {q2, q3}, d(q2,b) = {q4, q5}, d(q3,b) = {q6, q5}

d*(q1,ab) = {q4, q5 , q6}

 * (q,  )  {q}

 * (q, y.i )    (q' , i )

q ' * ( q , y )
 90
How does a ND FA accept?

xL   ( q0 , x )  F  
*

Among the various possible runs (with the same input) of the ND FA it
suffices that one of them (that is, there exists one that) succeeds and
accepts the input string

Another, alternative interpretation of nondeterminism:

universal nondeterminism (the previous one is existential):
all runs of the automaton accept

( (q0 , x)  F )
*
 91
nondeterministic PDA (NPDA)

• In fact PDA are “natural born” ND :

i, A/a q2

q1

e, A/b
q3
 92

• We might as well remove the deterministic constraint and generalize:

i, A/a q2
q1

i, A/b
q3

 : Q  ( I  { })    F (Q  * )

• Why index F? (finite subsets: we do not want an infinite set of moves)

• As usual, the NPDA accepts x if there exists a sequence
c0├* <q, e, g>, q  F
• in case of nondeterminism, the relation ‘├’ is not unique (i.e., functional) any more
 93
A “trivial” example: accepting {anbn | n>0} {anb2n | n>0}
a,A/AA b,A/e

a,Z0/ Z0 A
q0’ q1’ b,A/e
q2’ q3’
e, Z0 /e

e,Z0/ Z0

q0

e,Z0/ Z0 q3
a,A/AA b,A/e
e, Z0 /e
a,Z0/ Z0 A b,A/A
q0” q1 b,A/A
q2 q4
 94

Some immediate significant consequences

• NPDA can accept a language that is not accepted by deterministic PDA ---->
they are more powerful
• The previous construction can be easily generalized to obtain a constructive proof
of closure under union of the NPDA
-a property that deterministic PDA do not enjoy
• The closure under intersection still does not hold ({a nbncn} = {anbnc*}  {a*bncn}
cannot be accepted by a PDA, not even ND)
-the two cited examples, {anbncn} and {anbn} {anb2n}, are in fact not so similar…
 95

• If a language family is closed under union and not w.r.t. intersection

it cannot be closed under complement (why?)
• Hence the family of lang. accepted by NPDA is not closed under complement
• This highlights a deep change caused by nondeterminism concerning the
complement of a problem -in general-:
if the way of operating of a machine is deterministic and its computation finishes
it suffices to change the positive answer into a negative one to obtain the solution
of the “complement problem” (for instance, presence rather than absence of
errors in a program)
 96

• In the case of NPDA, though it is possible, like for PDA, to make a

computation always finish, there can be two computations
– co ├* <q1, e, g1>
– co ├* <q2, e, g2>
– q1  F, q2  F
• In this case x is accepted
• However, if F turned into Q - F, x is still accepted:
with nondeterminism changing a yes into a no does not work!

• …and for other kinds of automata?

does nondeterminism increase the power of the model?
 97
Nondeterministic Finite-state Automata (NFA)

b q4
a q2
b
q1
q5
a b
q3

b
q6

Starting from q1 and reading ab the automaton reaches a state that

belongs to the set {q4,q5,q6}
Let us call again “state” the set of possible states in which the
NFA can be during a run.
Formally ...
 98

• Given a NFA an equivalent deterministic one can be automatically computed 

• NFA are not more powerful than their deterministic relatives (this is different than
with PDA)
(so what is their use? We’ll see in a moment…)
• Let AND = <QN,I, dN, q0N, FN> the NFA from which we build a FA
• Let AD = <QD,I, dD, q0D, FD> the FA we intend to build
– QD = (QN )

 D ( qD , i )    N ( q N , i )
– qN qD -- recall that N (qN, i ) is a set

– q0D = {q0N}

– FD = { Q  QN | Q  FN   } -- set of states that include a final state

 99

• Though it is true that for all NFA one can find (and build) an equivalent
deterministic one
• This does not mean that using NFA is useless:
– It can be easier to “design” a NFA and then obtain from it automatically an
equivalent deterministic one, just to skip the (painful) job of build it ourselves
deterministic from the beginning (we will soon see an application of this idea)
– For instance, from a NFA with 5 states one can obtain, in the worst case, one with
25 states!
• Consider NFA and FA for languages L1=(a,b)*a(a,b) (i.e., strings over {a,b}
with ‘a’ as the symbol before the last one) and L2=(a,b)*a(a,b)4 (i.e., ‘a’ as the
fourth symbol before the last...)

• We still have to consider the TM ...

 100

Nondeterministic TM

  , [ ] : Q  I   k  (Q   k  {R, L, S }k 1 [ O  {R, S}])

• Configurations, transitions, transition sequences and acceptance

are defined as usual
• Does nondeterminism increment the power of TM’s?
• It does not if for every ND Turing Machine there exists a
deterministic one that can simulate it
 101
Computation tree
C accepting
c0
C halt but not
accepting

Unfinished
(possibly
c11 c12 c13 nonterminating)
computations

c21 c22 c23 c24 c25 c26

c32
c31 ckj cim
 102
• x is accepted by a ND TM iff there exists a computation that terminates in an
accepting state
• Can a deterministic TM establish whether a “sister” ND TM accepts x, that is,
accept x if and only if the ND TM accepts?
• This amounts to build the computation tree of the NDTM and “visit” it to
establish whether it contains a path that finishes in an accepting state
• This is a (almost) trivial, well known problem of tree visit, for which there are
classical algorithms
• The problem is therefore reduced to implementing an algorithm for visiting
trees through TM’s: a boring, but certainly feasible exercise … but beware the
above “almost” ...
 103

• Everything is easy if the computation tree is finite

• But it could be that some paths of the tree are infinite (they describe
nonterminating –hence non-accepting– computations)
• In this case a depth-first visit algorithm (for instance leftmost preorder) might
“get stuck in an infinite path” without ever discovering that another branch is
finite and leads to acceptance.
• The problem can however be easily overcome by adopting a breadth-first visit
algorithm (it uses a queue data structure rather than a stack to manage the nodes
still to be visited).
• Hence nondeterminism does not increase the power of the TM
 104
Conclusions
• Nondeterminism: a useful abstraction to describe search problems and algorithms; or situations
where there are no elements of choice or they are equivalent; or parallel computations
• In general it does not increase the computing power, at least in the case of TM’s (which are the
most powerful automaton seen so far) but it can provide more compact descriptions
• It increases the power of pushdown automata
• It can be applied to various computational models (to every one, in practice); in some cases
“intrinsically nondeterministic” models were invented to describe nondeterministic phenomena
• For simplicity we focused only on (D and ND) acceptors but the notion applies also to
translator automata
• NB: the notion of ND must not be confused with that of stochastic (there exist stochastic
models -e.g. Markov chains- that are completely different from the nondeterministic ones)