Models of Computation Jeff Erickson PDF Download
Models of Computation Jeff Erickson PDF Download
https://ebookbell.com/product/models-of-computation-jeff-
erickson-9947804
https://ebookbell.com/product/models-of-computation-an-introduction-
to-computability-theory-maribel-fernndez-fernndez-23397276
https://ebookbell.com/product/models-of-computation-in-context-7th-
conference-on-computability-in-europe-cie-2011-sofia-bulgaria-
june-27-july-2-2011-proceedings-1st-edition-pablo-arrighi-2453806
https://ebookbell.com/product/models-of-computation-in-context-7th-
conference-on-computability-in-europe-cie-2011-sofia-bulgaria-
june-27-july-2-2011-proceedings-1st-edition-pablo-arrighi-4142670
https://ebookbell.com/product/models-of-computation-roberto-bruni-ugo-
montanari-5768410
Models Of Computation 1st Ed 2017 Roberto Bruni Ugo Montanari
https://ebookbell.com/product/models-of-computation-1st-
ed-2017-roberto-bruni-ugo-montanari-6981950
https://ebookbell.com/product/models-of-computation-for-big-data-1st-
ed-rajendra-akerkar-7322674
https://ebookbell.com/product/unconventional-models-of-computation-
umc2k-proceedings-of-the-second-international-conference-on-
unconventional-models-of-computation-umc2k-1st-edition-luigi-
accardi-4199236
https://ebookbell.com/product/unconventional-models-of-computation-
third-international-conference-umc-2002-kobe-japan-
october-1519-2002-proceedings-1st-edition-manuel-lameiras-campagnolo-
auth-4605364
https://ebookbell.com/product/interactive-models-of-computation-and-
program-behavior-pierrelouis-curien-6821434
Models of Computation
Jeff Erickson
http://jeffe.cs.illinois.edu/teaching/algorithms/ • http://algorithms.wtf/
© Copyright 2014–2017 Jeff Erickson. Last update December 28, 2018.
Caveat Lector!
These notes are best viewed as an unfinished first draft. You should assume the notes
contain several major errors, in addition to the usual unending supply of typos, fencepost errors,
off-by-one errors, and brain farts. Before Fall 2014, I had not taught this material in more than
two decades. Moreover, the course itself is still very new—Lenny Pitt and I developed the
course and offered the first pilot in Spring 2014 (with Lenny presenting the formal language
material)—so even the choice of which material to emphasize, sketch, or exclude is still very
much in flux.
I would sincerely appreciate feedback of any kind, especially bug reports.
Thanks, and enjoy!
— Jeff
Contents
1 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Regular Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3 Finite-State Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4 Nondeterministic Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5 Context-Free Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6 Turing Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7 Undecidability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
8 Universal Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
9 Nondeterministic Turing Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
1 Strings
Throughout this course, we will discuss dozens of algorithms and computational models that
manipulate sequences: one-dimensional arrays, linked lists, blocks of text, walks in graphs,
sequences of executed instructions, and so on. Ultimately the input and output of any algorithm
must be representable as a finite string of symbols—the raw contents of some contiguous portion
of the computer’s memory. Reasoning about computation requires reasoning about strings.
This note lists several formal definitions and formal induction proofs related to strings. These
definitions and proofs are intentionally much more detailed than normally used in practice—most
people’s intuition about strings is fairly accurate—but the extra precision is necessary for any
sort of formal proof. It may be helpful to think of this material as part of the “assembly language”
of theoretical computer science. We normally think about computation at a much higher level
of abstraction, but ultimately every argument must “compile” down to these (and similar)
definitions.
But the actual definitions and theorems are not the point. The point of playing with this
material is to get some experience working with formal/mechanical definitions and proofs,
especially inductive definitions and recursive proofs. Or should I say recursive definitions and
inductive proofs? Whatever, they’re the same thing. Strings are a particularly simple and
induct
convenient playground for recurs ion; we’ll see many more examples throughout the course.
When you read this note, don’t just look at the content of the definitions and proofs; pay close
attention to their structure and the process for creating them.
1.1 Strings
Fix an arbitrary finite set Σ called the alphabet; the individual elements of Σ are called symbols
or characters. As a notational convention, I will always use lower-case letters near the start of
the English alphabet (a, b, c, . . . ) as symbol variables, and never as explicit symbols. For explicit
symbols, I will always use fixed-width upper-case letters (A, B, C, . . . ), digits (0, 1, 2, . . . ),
1
Models of Computation Lecture 1: Strings [Sp’18]
or other symbols (, $, #, •, . . . ) that are clearly distinguishable from variables. For further
emphasis, I will almost always typeset explicit symbols in RED.
A string (or word) over Σ is a finite sequence of zero or more symbols from Σ. Formally, a
string w over Σ is defined recursively as either
We normally write either a · x or simply ax to denote the ordered pair (a, x). Similarly, we
normally write explicit strings as sequences of symbols instead of nested ordered pairs; for
example, STRING is convenient shorthand for the formal expression (S, (T, (R, (I, (N, (G, ")))))).
As a notational convention, I will always use lower-case letters near the end of the English
alphabet (. . . , w, x, y, z) for string variables, and SHOUTY RED MONOSPACED TEXT to typeset
explicit (non-empty) strings.
The set of all strings over Σ is denoted Σ∗ (pronounced “sigma star”). It is very important to
remember that every element of Σ∗ is a finite string, although Σ∗ itself is an infinite set containing
strings of every possible finite length.
For example, the string FIFTEEN has length 7, the string SEVEN has length 5, and the string 5
has length 1. Although they are formally different objects, we do not normally distinguish
between symbols and strings of length 1.
The concatenation of two strings x and y, denoted either x • y or simply x y, is the
unique string containing the characters of x in order, followed by the characters in y in
order. For example, the string NOWHERE is the concatenation of the strings NOW and HERE;
that is, NOW • HERE = NOWHERE. (On the other hand, HERE • NOW = HERENOW.) Formally,
concatenation is defined recusively as follows:
¨
z if w = ",
w • z :=
a · (x • z) if w = ax.
(Here I’m using a larger dot • to formally distinguish the operator that concatenates two arbitrary
strings from from the syntactic sugar · that builds a string from a single character and a string.)
When we describe the concatenation of more than two strings, we normally omit all dots
and parentheses, writing wxyz instead of (w • (x • y)) • z, for example. This simplification is
justified by the fact (which we will prove shortly) that the function • is associative.
2
Models of Computation Lecture 1: Strings [Sp’18]
• The case structure of the proof should mirror the case structure of the recursive defin-
ition. For example, if you are proving something about all strings, your proof should have
two cases: Either w = ", or w = ax for some symbol a and string x.
• The inductive hypothesis should be as strong as possible. The (strong) inductive hypoth-
esis for statements about natural numbers is always “Assume there is no counterexample k
such that k < n.” I recommend adopting a similar inductive hypothesis for strings: “Assume
there is no counterexample x such that |x| < |w|.” Then for the case w = a x, we have
|x| = |w| − 1 < |w| by definition of |w|, so the inductive hypothesis applies to x.
Thus, string-induction proofs have the following boilerplate structure. Suppose we want to prove
that every string is perfectly cromulent, whatever that means. The white boxes hide additional
proof details that, among other things, depend on the precise definition of “perfectly cromulent”.
• Suppose w = " .
Here are three canonical examples of this proof structure. When developing proofs in this
style, I strongly recommend first mindlessly writing the green text (the boilerplate) with lots of
space for each case, then filling in the red text (the actual theorem and the induction hypothesis),
and only then starting to actually think.
Many students are confused (or at least bored and distracted) by the fact that we are proving
mind-bogglingly obvious facts. If you’re one of these students, try to remember that the lemmas
themselves are not the point. Pay close attention to the structure of the proofs. Notice how each
proof follows the boilerplate described above. Notice how every sentence of the proof follows
mechanically from earlier sentences, definitions, and the rules of standard logic and arithmetic.
Lemma 1.1. Adding nothing does nothing: For every string w, we have w • " = w.
3
Models of Computation Lecture 1: Strings [Sp’18]
Proof: Let w be an arbitrary string. Assume that x • " = x for every string x such that |x| < |w|.
There are two cases to consider:
• Suppose w = ".
Lemma 1.2. Concatenation adds length: |w • x| = |w| + |x| for all strings w and x.
Proof: Let w and x be arbitrary strings. Assume that | y • x| = | y| + |x| for every string y such
that | y| < |w|. (Notice that we are using induction only on w, not on x.) There are two cases to
consider:
• Suppose w = ".
|w • x| = |a y • x| because w = a y
= |a · ( y • x)| by definition of •
= 1 + | y • x| by definition of | · |
= 1 + | y| + |x| by the inductive hypothesis
= |a y| + |x| by definition of | · |
= |w| + |x| because w = a y
Proof: Let w, x, and y be arbitrary strings. Assume that (z • x) • y = z • (x • y) for every string
z such that |z| < |w|. (Again, we are using induction only on w.) There are two cases to consider.
4
Models of Computation Lecture 1: Strings [Sp’18]
• Suppose w = ".
(w • x) • y = (" • x) • y because w = "
=x•y by definition of •
= " • (x • y) by definition of •
= w • (x • y) because w = "
Proof: Let w, x, and y be arbitrary strings. Assume that (z • x 0 ) • y 0 = z • (x 0 • y 0 ) for all strings
x 0 , y 0 , and z such that |z| < |w|. (We need a stronger induction hypothesis here than in the
previous proofs!) There are three cases to consider.
• Suppose w = ".
(w • x) • y = (" • x) • y because w = "
=x•y by definition of •
= " • (x • y) by definition of •
= w • (x • y) because w = "
5
Models of Computation Lecture 1: Strings [Sp’18]
1The same bracket notation is also used for bibliographic references, instead of the traditional footnote/endnote
superscripts, for exactly the same reasons.
2A typewriter is an obsolete mechanical device loosely resembling a computer keyboard. Pressing a key on a
typewriter moves a lever (called a “typebar”) that strikes a cloth ribbon full of ink against a piece of paper, leaving the
image of a single character. Many historians believe that the ordering of letters on modern keyboards (QWERTYUIOP)
evolved in the late 1800s, reaching its modern form on the 1874 Sholes & Glidden Type-WriterTM , in part to separate
many common letter pairs, to prevent typebars from jamming against each other; this is also why the keys on most
modern keyboards are arranged in a slanted grid. (The common folk theory that the ordering was deliberately
intended to slow down typists doesn’t withstand careful scrutiny.) A more recent theory suggests that the ordering
was influenced by telegraph3 operators, who found older alphabetic arrangements confusing, in part because of
ambiguities in American Morse Code.
6
Models of Computation Lecture 1: Strings [Sp’18]
A proper substring of w is any substring other than w itself. For example, LAUGH is a proper
substring of SLAUGHTER. Whenever y is a (proper) substring of w, we also call w a (proper)
superstring of y.
A prefix of w[1 .. n] is any substring of the form w[1 .. j]. Equivalently, a string p is a prefix
of another string w if and only if there is a string x such that px = w. A proper prefix of w is
any prefix except w itself. For example, DIE is a proper prefix of DIET.
Similarly, a suffix of w[1 .. n] is any substring of the form w[i .. n]. Equivalently, a string s is a
suffix of a string w if and only if there is a string x such that xs = w. A proper suffix of w is
any suffix except w itself. For example, YES is a proper suffix of EYES, and HE is both a proper
prefix and a proper suffix of HEADACHE.
A subsequence of a string w is a string obtained by deleting zero or more symbols from
anywhere in w. More formally, z is a subsequence of w if and only if
• z = ", or
• w = a x for some symbol a and some string x such that z is a subsequence of x.
• w = a x and z = a y for some symbol a and some strings x and y, and y is a subsequence
of x.
Exercises
Most of the following exercises ask for proofs of various claims about strings. Here “prove” means
give a complete, self-contained, formal proof by inductive definition-chasing, using the boilerplate
structure recommended in Section 1.3. Feel free to use Lemmas 1.1, 1.2, and 1.3 without proof,
but don’t assume any other facts about strings that you have not actually proved. (Some later
exercises rely on results proved in earlier exercises.) Do not appeal to intuition, and do not use
the words “obvious” or “clearly” or “just”. Most of these claims are in fact obvious; the real
exercise is understanding and formally expressing why they’re obvious.
7
Models of Computation Lecture 1: Strings [Sp’18]
Note to instructors: Do not assign any of these problems before solving them
yourself, especially on exams. It’s very easy to underestimate the difficulty of these
problems, or at least the lengths of their solutions, which for exams is a reasonable proxy
for difficulty. Also, several later exercises rely implicitly on identities like #(a, x • y) =
#(a, x) + #(a, y) that are only proved in earlier exercises. It’s unfair to assign these problems
to students without telling them which earlier results they can use.
1. Let w be an arbitrary string, and let n = |w|. Prove each of the following statements.
2. Prove the following useful identities for all strings w, x, y, and z directly from the definition
of •, without referring to the length of any string.
3. Prove the following useful fact about substrings. An arbitrary string x is a substring of
another arbitrary string w = u • v if and only if at least one of the following conditions
holds:
• x is a substring of u.
• x is a substring of v.
• x = yz where y is a suffix of u and z is a prefix of v.
4. Let w be an arbitrary string, and let n = |w|. Prove the following statements for all indices
1 ≤ i ≤ j ≤ k ≤ n.
Recursive Functions
5. For any symbol a and any string w, let #(a, w ) denote the number of occurrences of a
in w. For example, #(A, BANANA) = 3 and #(X, FLIBBERTIGIBBET) = 0.
8
Models of Computation Lecture 1: Strings [Sp’18]
(b) Prove that #(a, x y) = #(a, x) + #(a, y) for every symbol a and all strings x and y.
Your proof must rely on both your answer to part (a) and the formal recursive
definition of string concatenation.
(c) Prove the following identity for all alphabets Σ and all strings w ∈ Σ∗ :
X
|w| = #(a, w)
a∈Σ
7. For any string w and any non-negative integer n, let w n denote the string obtained by
concatenating n copies of w; more formally, we define
(
" if n = 0
wn :=
n−1
w•w otherwise
(a) Prove that wm • wn = wm+n for every string w and all non-negative integers n and m.
(b) Prove that #(a, wn ) = n · #(a, w) for every string w, every symbol a, and every
non-negative integer n. (See Exercise 5.)
(c) Prove that (wR )n = (wn )R for every string w and every non-negative integer n.
(d) Prove that for all strings x and y that if x • y = y • x, then x = wm and y = wn for
some string w and some non-negative integers m and n. [Hint: Careful with "!]
9
Models of Computation Lecture 1: Strings [Sp’18]
9. For any string w ∈ {0, 1, 2}∗ , let w + denote the string obtained from w by replacing each
symbol a in w by the symbol corresponding to (a + 1) mod 3. for example, 0102101+ =
1210212. This function can be defined more formally as follows:
" if w = "
1· x +
if w = 0 x
w+ :=
2· x +
if w = 1 x
0· x +
if w = 2 x
(a) Prove that |w| = |w+ | for every string w ∈ {0, 1, 2}∗ .
(b) Prove that (x • y)+ = x + • y + for all strings x, y ∈ {0, 1, 2}∗ .
(c) Prove that #(1, w+ ) = #(0, w) for every string w ∈ {0, 1, 2}∗ .
(d) Prove that (w+ )R = (wR )+ for every string w ∈ {0, 1, 2}∗ .
10. For any string w ∈ {0, 1}∗ , let swap(w) denote the string obtained from w by swapping the
first and second symbols, the third and fourth symbols, and so on. For example:
swap(101) = 011
swap(100111) = 011011
swap(10 11 00 01 10 1) = 01 11 00 10 01 1.
The swap function can be formally defined as follows:
" if w = "
swap(w) := w if w = 0 or w = 1
ba • swap(x) if w = ab x for some a, b ∈ {0, 1} and x ∈ {0, 1}∗
11. For any string w ∈ {0, 1}∗ , let sort(w) denote the string obtained by sorting the characters
in w. For example, sort(010101) = 000111. The sort function can be defined recursively
as follows:
" if w = "
sort(w) := 0 · sort(x) if w = 0 x
sort(x) • 1 if w = 1 x
10
Models of Computation Lecture 1: Strings [Sp’18]
(a) Prove that merge(x, y) ∈ 0∗ 1∗ for all strings x, y ∈ 0∗ 1∗ . (The regular expression
0∗ 1∗ is shorthand for the language {0a 1 b | a, b ≥ 0}.)
(b) Prove that sort(x • y) = merge(sort(x), sort( y)) for all strings x, y ∈ {0, 1}∗ . (The
sort function is defined in the previous exercise.)
11
Models of Computation Lecture 1: Strings [Sp’18]
For example, faro(0011, 0101) = 00011011. (A "faro shuffle" splits a deck of cards into
two equal piles and then perfectly interleaves them.)
(a) Prove that |faro(x, y)| = |x| + | y| for all strings x and y.
(b) Prove that faro(w, w) = stutter(w) for every string w.
(c) Prove that faro(odds(w), evens(w)) = w for every string w.
16. For any string w, let declutter(w) denote the string obtained from w by deleting any symbol
that equals its immediate successor. For example, declutter(MISSISSIPPI) = MISISIPI,
and declutter(ABBCCCAAAACCCBBA) = ABCACBA.
12
Exploring the Variety of Random
Documents with Different Content
Let my luggage be brought up, and Haga V. subir mi equipaje, y no
don't forget the towels. olvide las toallas.
We want dinner at six o'clock. Queremos comer a las seis.
Will you dine at table d'hôte at ¿Quiere V. comer en la mesa redonda
seven, Sir? a las siete?
No, I shall dine with some friends No, comeré con algunos amigos en
in town and return late. la cuidad y volveré tarde.
It is very warm in this country. Hace mucho calor en este país.
Have you any cooling drinks? ¿Tiene V. algunos refrescos?
Please reserve us two good seats at Sírvase V. reservarnos dos buenos
table d'hôte. asientos en la mesa redonda.{300}
There is a bell at the side of the Hay campanilla al lado de la
chimney-piece. chimenea.
Is there no electric bell? ¿No hay timbre eléctrico?
No, Sir, but we have electric light No, Señor, pero tenemos luz
in all the rooms. eléctrica en todos los cuartos.
I notice that there is neither soap Veo que no hay ni jabón ni agua
nor fresh water. fresca.
The servant will put everything in La criada pondrá todo en orden
order directly. inmediatamente.
Please let me have my bill. Hágame V. el favor de la cuenta.
Here is your account, Sir. Aquí tiene V. su cuenta, Señor.
Very well, please receipt it. Muy bien, sírvase V. poner el recibí.
Waiter here is something for you. Mozo, aquí tiene V. para sí.
The Money-changer. El Cambista.
Where could I get my money ¿Donde podría cambiar mi dinero?
changed?
Where is the nearest money- ¿Donde está la casa de cambio más
changer's? próxima?{301}
Is there a money-changer's close ¿Hay algún cambista cerca de aquí?
by?
I wish to change some English Quiero cambiar algún dinero inglés.
money.
What do you give for a sovereign? ¿A cómo paga V. la libra esterlina?
Can you change me a five-pound ¿Puede V. cambiarme un billete de
note? banco de cinco libras?
What is the rate of exchange to- ¿A qué tipo está hoy el cambio?
day?
I will take half in silver. Tomaré la mitad en plata.
I should like a peseta's worth of Quisiera una peseta en calderilla.
coppers.
Could you cash me this cheque on ¿Podría V. hacerme efectivo este
London? cheque sobre Londres?
The Theatre. El Teatro.
Is there a good theatre (music-hall, ¿Hay en ésta un buen teatro (teatro
picture-house) here? de variedades, cine)?
Which is the best theatre? ¿Cual es el mejor teatro?
Is there a performance to-night? ¿Hay función esta noche? {302}
What are they playing? ¿Qué representan?
What time does the performance ¿A qué hora principia (es) la
begin? función?
What time is the performance ¿A qué hora acaba (termina) la
over? función?
Is there a good orchestra? ¿Hay buena orquesta?
What are the prices? ¿Cuáles son los precios (de las
localidades)?
The boxes, the stalls, the circle, the Los palcos, las butacas, el anfiteatro,
gallery (the gods). la galería (el paraíso).
What time does the box-office ¿A qué hora se abre la taquilla?
open?
A ticket, a pass, a pass-out check. Un billete, un billete de favor, una
contraseña.
Let me have a programme, please, Hágame V. el favor de un programa,
and the book of words. y el libreto.
Can I hire a pair of opera glasses? ¿Puedo alquilar unos gemelos?
The interval. El entreacto.
The curtain. El telón.
The Seasons. Las Estaciones.
Winter is over at last. Al fin se acabó el invierno.
Do you like winter? ¿Le gusta a V. el invierno?
I like it as much as summer. Me gusta tanto como el verano.
You are perhaps the only one of V. será tal vez el único de esa
that opinion. opinión.
In winter one is not comfortable En invierno no se está bien sino al
except by the fireside. lado del fuego.
Can you skate? ¿Sabe V. patinar?
Yes, a little. Sí, un poco.
Have you skated this winter? ¿Ha patinado V. este invierno?
The winter has been very severe El invierno ha sido muy riguroso
this year. este año.
I am glad to see the spring. Me alegro de ver la primavera.
It is the season I like best. Es la estación que me gusta más.
It is the most pleasant of all Es la más agradable de todas las
seasons. estaciones.
The season is very backward. La estación está muy atrasada.{304}
I fear we shall have a very hot Temo que tendremos un verano muy
summer. caluroso.
One would think that the order of Se diría que las estaciones están
the seasons had been reversed. todas trastornadas.
Summer is the season of the El verano es la estación de la
harvest. cosecha.
Summer is over. Ya pasó el verano.
After summer comes the autumn. Después del verano viene el otoño.
Autumn is the season of vintages. El otoño es la estación de las
vendimias.
Towards the middle of autumn the Hacia mediados del otoño las
mornings and evenings are cool. mañanas y las tardes son frescas.
Which season do you like best, ¿Cuál estación le gusta a V. más,
Sir? Señor?
Unquestionably, I like the spring Sin duda alguna, me gusta más la
best. primavera.
Is it as hot in England as in Spain ¿Hace tanto calor en Inglaterra como
in the summer? en España en el verano?{305}
Never. It is not too warm in Nunca. No hace demasiado calor en
summer nor too cold in winter. el verano ni demasiado frío en el
invierno.
The winter in the north of Spain is En el norte de España el invierno es
almost as cold as in England. casi tan frío como en Inglaterra.
No doubt, but for my part I should Sin duda, pero por mi parte preferiría
prefer the winter of Northern Spain el invierno del norte de España al de
to that of England, especially Inglaterra, especialmente al de
London, as it is very foggy there. Londres, pues allí hace mucha
niebla.
You are right; it must be V. tiene razón; se debe mencionar,
mentioned, however, that the fog is sin embargo, que la niebla no se
not so much due to the climate of debe tanto al clima de aquel país
that country as to the immense como al enorme consumo de carbón
quantities of coal consumed daily que tiene lugar todos los días en las
in the dwellings and numerous casas{306} particulares y en las
manufactories. muchas fábricas.
I have read that as soon as a means He leído que tan pronto como se
of consuming the smoke is haya inventado un modo de
invented, London will have a very consumir el humo, Londres tendrá
agreeable climate. un clima muy agradable.
Let us hope that this will soon be Esperemos que sea pronto.
the case.
ebookbell.com