All Questions Answered

Donald Knuth

On October 5, 2001, at the Technische Universität (1979), the Adelsköld Medal from the Royal Swedish
München, Donald Knuth presented a lecture entitled Academy of Sciences (1994), the Harvey Prize from
“All Questions Answered”. The lecture drew an au- the Technion of Israel (1995), the John von Neumann
dience of around 350 people. This article contains Medal from the Institute of Electrical and Electron-
the text of the lecture, edited by Notices senior ics Engineers (1995), and the Kyoto Prize from the
writer and deputy editor Allyn Jackson. Inamori Foundation (1996). Since 1968 Knuth has
Originally trained as a mathematician, Donald been on the faculty of Stanford University, where
Knuth is renowned for his research in computer sci- he currently holds the title of Professor Emeritus of
ence, especially the analysis of algorithms. He is a The Art of Computer Programming.
prolific author, with 160 entries in MathSciNet. —Allyn Jackson
Among his many books is the three-volume series
The Art of Computer Programming [TAOCP], for Knuth: In every class that I taught at Stanford,
which he received the AMS Steele Prize for Exposi- the last day was devoted to “all questions an-
tion in 1986. The citation for the prize stated that swered”. The students didn’t have to come to class
TAOCP “has made as great a contribution to the if they didn’t want to, but if they did, they could
teaching of mathematics for the present generation ask any question on any subject except religion or
of students as any book in mathematics proper in politics or the final exam. I got the idea from
recent decades.” The long awaited fourth volume is Richard Feynman, who did the same thing in his
in preparation and some parts are available through classes at Caltech, and it was always interesting to
Knuth’s website, http://www-cs-faculty. see what the students really wanted to know. Today
stanford.edu/~knuth/. I’ll answer any question on any subject. Do we
Knuth is the creator of the TE X and METAFONT allow religion or politics? I don’t know. But there
languages for computer typesetting, which have is no final exam to worry about. I’ll try to answer
revolutionized the preparation and distribution of without taking too much time so that we can get a
technical documents in many fields, including math- lot of questions in.
ematics. In 1978 he presented the AMS Gibbs Lecture So, who wants to ask the first question?… Well,
entitled “Mathematical Typography”. The lecture if there are no questions…[Knuth makes as if to
was subsequently published in the Bulletin of the leave.]
AMS [MT]. Question: There was a special report to the Amer-
Knuth earned his Ph.D. in mathematics in 1963 ican president, the PITAC report [PITAC], contain-
from the California Institute of Technology under ing some recommendations. I am wondering
the direction of Marshall Hall. He has received the whether you would be willing to comment on the pri-
Turing Award from the Association for Computing orities outlined in these recommendations:
Machinery (1974), the National Medal of Science better software engineering, building a teraflop

318 NOTICES OF THE AMS VOLUME 49, NUMBER 3

computer, improvements in the Internet including you don’t make a test and then have that determine
higher security and higher bandwidth, and the what you do next. A lot of you have seen the movie
socio-economic impacts of managing information Lola rennt (called Run Lola Run in the U.S.), in which
available via computer networks. the plot is played out three different times, with the
Knuth: I think that’s a brilliant solution of the outcome taking three different branches. Quantum
problem of what to present to a president. But in computing is something like that: The world goes
fact what I would like to see is thousands of com- into many different branches, and we’re interested
puter scientists let loose to do whatever they want. in the one where the outcome is the nicest.
That’s what really advances the field. From my ex- I’m good at nonquantum computing myself, so
perience writing The Art of Computer Programming, it’s quite possible that if quantum computing takes
if you asked me any year what was the most im- over, I won’t be able to do the new stuff. My life’s
portant thing that happened work is with computers not

NON SEQUITUR © 2001 Wiley Miller. Dist. By UNIVERSAL PRESS SYNDI-

in computer science that year, because I’m interested so
I probably would have no an- much in computation, but be-
swer for the question, but over cause I happen to be good at
five years’ time the whole field this kind of computing. For-
changes. Computer science is tunately for me, I found that
a tremendous collaboration the thing I could do well was

CATE. Reprinted with permission. All rights reserved.

of people from all over the interesting to other people. I
world adding little bricks to a didn’t develop an ability to
massive wall. The individual think about algorithms be-
bricks are what make it work, cause I wanted to help people
and not the milestones. solve problems. Somehow, by
Next question? the time I was a teenager, I
Question: Mathematicians had a peculiar way of think-
say that God has the “Book of ing that made me good at pro-
Proofs”, where all the most gramming. But I might not be
aesthetic proofs are written. good at quantum program-
Can you recommend some ming. It seems to be a differ-
algorithms for the “Book of Al- ent world from my own.
gorithms”? I’ll take a question from
Knuth: That’s a nice ques- the back.
tion. It was Paul Erdős who Question: I am working in
promulgated the idea that God has a book con- theorem proving, and one of the most important pa-
taining the best mathematical proofs, and I guess pers is your paper “Simple word problems in uni-
my friend Günter Ziegler in Berlin has recently versal algebra” [KB] from 1970, written with
written about it [PFB]. P. B. Bendix. I have two questions. The first is, do you
I remember that mathematicians were telling still follow this area and what do you think of it? And
me in the 1960s that they would recognize com- the second is, who is and what became of P. B. Ben-
puter science as a mature discipline when it had dix?
1,000 deep algorithms. I think we’ve probably Knuth: This work was published in 1970, but I
reached 500. There are certainly lots of algorithms actually did it in 1967 while I was at Caltech. It
that I think have to be considered absolutely beau- was a simple idea, but fortunately it’s turned out
tiful and immortal, in some sense. Two examples to be very widely applicable. The idea is to take a
are the Euclid algorithm and a corresponding one set of mathematical axioms and find all the
that works in binary notation and that may have implications of those axioms. If I have a certain
been developed independently in China, almost as set of axioms and you have a possible theorem,
early as Euclid’s algorithm was invented in Greece. you ask, does this theorem follow from those
In my books I am mostly concerned with the algo- axioms or not? I called my paper “Simple word
rithms that are classical and that have been around problems in universal algebra”, and I said a
for a long time. But still, every year we find brand problem was “simple” if my method could
new ideas that I think are going to remain forever. handle it. Now people have extended the method
Question: Do you have thoughts on quantum quite a lot, so that a lot more problems are
computing? “simple”. I think their work is beautiful.
Knuth: Yes, but I don’t know a great deal about The year 1967 was the most dramatic year of
it. It’s quite a different paradigm from what I’m used my life by far. I had no time for research. I had
to. It has lots of things in common with the kind two children less than two years old; I had been
of computing I know, but it’s also quite mysterious scheduled to be a lecturer for ACM (Association
in that you have to get all the answers at the end; for Computing Machinery) for three weeks; I had

MARCH 2002 NOTICES OF THE AMS 319

 Question: It seems to me it’s easier to revise a
book than the huge software programs we see day
to day. How can we apply theory to improve soft-
ware?
Knuth: Certainly errors in software are more dif-
ficult to fix than errors in books. In fact, my main
conclusion after spending ten years of my life work-
ing on the TEX project is that software is hard. It’s
harder than anything else I’ve ever had to do. While
I was working on the TEX program, I was unable to
do full-time teaching. Although I love teaching, I
had to take a year off from it because there was just
too much to keep in my head at one time. Writing a
book is a little more difficult than writing a techni-
cal paper, but writing software is a lot more difficult
than writing a book.
to give lectures in a
NATO summer school In my books I offer rewards for the first person
in Copenhagen; I had to who finds any particular error, and I must say that
speak in a conference at I’ve written more checks to people in Germany
Oxford; and so on. And than in any other country in the world. I get letters
I was getting the page from all over, but my German readers are the best
proofs for The Art of nitpickers that I’ve ever had! In software I similarly
Computer Program- pay for errors in the TEX and METAFONT programs.
ming, of which the first The reward was doubling every year: It started out
volume was being at $2.56, then it went to $5.12, and so on, until it
published in 1968. All reached $327.68, at which time I stopped dou-
of this was in addition bling. There has been no error reported in TEX since
to the classes I was 1994 or 1995, although there is a rumor that some-
teaching, and an attack body has recently found one. I’m going to have to
of ulcers that put me in look at it again in a year or two. I do everything in
the hospital, and being batch mode, by the way. I am going to look again
an editor for twelve at possible errors in TEX in, say, the year 2003.
journals. That year I I think letting users know that you welcome re-
thought of two little ports of errors is one important technique that
ideas. One has become could be used in the software industry. I think
known as the Knuth-Bendix algorithm; the other Microsoft should say, “You’ll get a check from Bill
one is known as attribute grammars [AG]. That Gates every time you find an error.”
was the most creative year of my life, and it was Question: What importance do you give to the de-
also the most hectic. sign of efficient algorithms, and what emphasis do
You asked about Peter Bendix. He was a sopho- you suggest giving this area in the future?
more in a class I taught at Caltech, “Introduction Knuth: I think the design of efficient algorithms
to Algebra”. Every student was supposed to do a is somehow the core of computer science. It’s at
class project, and Peter did his term paper on the the center of our field. Computers are incredibly
implementation of the algorithm. He was a physics fast now compared to what they were before, so
major. This was the time of the Vietnam War, and for many problems there is no need to have an ef-
he became an objector. He went to Canada and ficient algorithm. I can write programs that are in
worked as a high school teacher for about five some sense extremely inefficient, but if it’s only
years and later got a degree in physics. I found he going to take one second to get the answer, who
was living near Stanford a couple of years ago, so cares? Still, some things we have to do millions or
I called him up and found out that he has had a billions of times, and just knowing that the num-
fairly happy life in recent years. ber of times is finite doesn’t tell us that we can han-
In the 1960s, if I wrote a joint paper with my ad- dle it. So the number of problems that are in need
visor Marshall Hall, it meant that he did the theory of efficient algorithms is huge. For example, many
and I did the programming. But if I wrote a paper problems are NP complete, and NP complete is
with anybody else, it meant that I did the theory and just a small level of complexity. Therefore I see an
the other person did the programming. So Pete almost infinite horizon for the need for efficient
Bendix was a good programmer who implemented algorithms. And that makes me happy because
the method. those are the kinds of problems I like the best.

320 NOTICES OF THE AMS VOLUME 49, NUMBER 3

 Question: You have a big interest in puzzles, in- Question: You
cluding the “Tower of Hanoi” puzzle on more than spent a lot of time on
3 pegs. I won’t ask a harder question—what is the computer typesetting.
shortest solution?—because I am not sure everyone What are your reflec-
knows this puzzle. But I will ask a more philosoph- tions on the impact of
ical question: Is it possible to show this can never be this work?
solved? Knuth: I am ex-
Knuth: Do people know the “Tower of Hanoi” tremely happy that
problem? You have 3 pegs, and you have disks of my work was in the
different sizes. You’re supposed to transfer the disks public domain and
from one peg to another, and the disks have to be made it possible for
sorted on each peg so that the biggest is always on people on all plat-
forms to communi-
the bottom. You can move only one disk at a time.
cate with each other
Henry Dudeney invented the idea of generalizing
via the Internet. Espe-
this puzzle to more than 3 pegs, and the task of find-
cially now I’m thrilled
ing the shortest solution to the 4-peg problem has
by some recent pro-
been an open question for more than a hundred jects. Two weeks ago
years. The 3-peg problem is very simple; we teach it I heard a great lecture
to freshmen. by Bernd Wegner from
But take another, more famous problem, the the Technical Univer-
Goldbach conjecture in mathematics: Every even in- sity of Berlin about
teger is the sum of two odd primes. Now, I think the plans for online
that’s a problem that’s never going to be solved. I journals by the Euro-
think it might not even have a proof. It might be pean Mathematical So-
one of the unprovable theorems that Gödel showed ciety. Such things
exist. In fact, we now know that in some sense al- would simply have
most all correct statements about mathematics are been impossible with-
unprovable. Goldbach’s conjecture is just, sort of, out the open source
true because it can’t be false. There are so many software that came
ways to represent an even number as the sum of out of my work. So I’m
two odd numbers, that as the numbers grow the extremely delighted
number of representations grows bigger and big- this is helping to ad-
10
ger. Take a 1010 -digit even number, and imagine vance science.
how many ways there are to write that as the sum I’m happy to see
of two odd integers. For an n-bit odd number, the many books that look
chances are proportional to 1/n that it’s prime. How pretty good. Before I
are all of those pairs of odd numbers going to be started my work,
books on mathemat-
nonprime? It just can’t happen. But it doesn’t fol-
ics were looking worse
low that you’ll find a proof, because the definition
and worse from year
of primality is multiplicative, while Goldbach’s con-
to year. It took a lot
jecture pertains to an additive property. So it might
of skilled handwork
very well be that the conjecture happens to be
to do it in the old sys-
true, but there is no rigorous way to prove it. tem. The people who
In the case of the 4-peg “Tower of Hanoi”, there could do that were
are many, many ways to achieve what we think is dying out, and high
the minimum number of moves, but we have no priority did not go to
good way to characterize all those solutions. So mathematical books.
that’s why I personally came to the conclusion that I never expected that
I was never going to be able to solve it, and I TEX would take over the entire world of publishing.
stopped working on it in 1972. But I spent a solid I’m not a very competitive person, and I did not
week working on it pretty hard. want to take jobs away from anybody who was
Question: What are the five most important prob- doing another way of printing. But I found that no-
lems in computer science? body wanted to do mathematical publishing well,
Knuth: I don’t like this “top ten” business. It’s so math was something I could improve without
the bottom ten that I like. I think you’ve got to getting anybody upset about me being an upstart.
go for the little things, the stones that make up The downside is that I’m too sensitive to things
the wall. now. I can’t go to a restaurant and order food

MARCH 2002 NOTICES OF THE AMS 321

 because I keep looking at the fonts on the menu. solve a problem, we can prove that we’ve solved it.
Five minutes later I realize that it’s also talking No astronomer will ever know whether his theories
about food. If I had never thought about computer of astronomy are correct. You can’t go up to the sun
typesetting, I might have had a happier life in some and measure it.
ways. So these are my first thoughts on that connec-
Question: Can you give us an outline for com- tion. But there is a difference between mathemat-
puter science, some milestones for the next ten or ics and computer programming, and sometimes I
twenty years? can feel when I’m putting on one hat or the other.
Knuth: You’re asking for milestones again. Some parts of me like mathematics, and some
There is a lot of interest in applications to biology parts of me like emacs hacking. I think they go
because so many things have opened up in that together okay, but I don’t see that they’re the same
domain, with chances to cure diseases. The fact paradigm.
that human beings are based on a discrete code Question: What is the relationship between God
means that people like you and me, who are good and computers?
at discrete problems, are able to do relevant work Knuth: In one of my books, 3:16 Bible Texts
for this area. The problems are very difficult and Illuminated [BTI], I used random sampling to study
challenging, and that’s why I foresee an important sixty different verses of the Bible and what people
future there. from all different religious persuasions and dif-
But in all aspects of our field, I really don’t see ferent centuries have said about those verses. I did
any slowing down. Every time I think I’ve discov- the study at first on my own, and then I found it
ered something interesting, I look on the Internet was interesting enough that I ought to make a book
and find that somebody else has done it too. So we about it. I got sixty of the best artists in the world
have a field that at the moment still seems to be to illustrate the book, many of them in Germany.
like a boiling kettle, where you can’t keep the lid The artwork was exhibited twice in Germany, and
on. in other countries around the world. It was also
In the field of biology, I think we can confidently shown in the National Cathedral in Washington,
predict that it’s going to have rich problems to DC. In that book I used methodology that com-
solve for at least 500 years. I can’t make that claim puter scientists often use for understanding a
for computer science. complicated subject, to see if that method would
Question: What is the connection between math- give some insight into the Bible, which is a com-
ematics and computer programming viewed as an plicated subject. In the book, I don’t give answers.
art? I just say I think it’s good that life should be an
Knuth: Art is Kunst. The American movie ongoing search. The journey is more important
Artificial Intelligence is called Kunstlicher Intelligenz than the destination.
in Germany—that is, artificial as well as artistic. I Question: Do you know whether “P equals NP”
think of programming with beauty in mind, as has been proved? I heard a rumor that it has.
being something elegant, something that you can Knuth: Which rumor did you hear?
be proud of for the way it fits together. Mathematics Question: One from Russia.
in the same way has elegance. Both fields, com- Knuth: From Russia? That’s new to me. Well, I
puting and mathematics, are different from don’t think anybody has proved that P equals NP
other sciences because they are artificial; they yet. But I know that Andy Yao has retired and
are not in nature. They’re totally under our own hopes to solve the problem in the next five to
control. We make up the axioms, and when we ten years. He is inspired by Andrew Wiles, who

322 NOTICES OF THE AMS VOLUME 49, NUMBER 3

devoted several years to proving Fermat’s Last by stealing my bank accounts or whatever. So I am
Theorem. They’re both Princeton people. Andy supportive of a high level of secrecy. But whether
can do it if anybody can. it should be impossible for the authorities to
Three or four years ago, there was a paper in a decode things even in criminal investigations, in
Chinese journal of computer science and technol- extreme cases—there I tend to come down on the
ogy by a professor who claimed he could solve an side of wanting to have some way to break some
NP-hard problem in polynomial time. The problem keys sometimes.
was about cliques, and he had a very clever way to Question: Will we have intelligent machines some-
represent cliques. The method was supposedly time in the future? Should we have them?
polynomial time, but it actually took something like Knuth: There have always been inflated esti-
n12 steps, so you couldn’t even check it when n mates as to how soon we are going to have a
equals 5. So it was very hard to see the bug in his machine that’s intelligent. I still see no signs of
proof. I went to Stanford and sat down with our getting around the central problem of under-
graduate students, and we needed a couple of standing what is cognition, what it means to think.
hours before we found the flaw. I wrote the author Neurologists are making better measurements
a letter pointing out the error, and he wrote back than they ever have before, but we are still so far
a couple of months later, saying, “No, no, there is from finding an answer that I can’t yet rank
no error.” I decided not to pursue it any further. I neuroscience as one of the most active fields of
had done my part. But I don’t believe it’s been current work. Biology has been getting answers,
solved. That’s the most mind-boggling problem with DNA and stem cells and so on. But with cog-
facing theoretical computer science, and maybe nition we are still looking for the secret.
all of science at the moment. Some very thought-provoking books came out
Question: What do you think of research in a year or two ago, one by Hans Moravec [Mo], and
cryptographic algorithms? And what do you one by Ray Kurzweil [Ku]. Both of them are saying
think of efforts by politicians today to put limits on that in twenty or thirty years we are going to have
cryptography research? machines smarter than humans. Some people were
Knuth: Certainly the whole area of cryptographic worried about that. My attitude is, if that’s true,
algorithms has been one of the most active and ex- more power to them. If they are smarter than us,
citing areas in computer science for the past ten so what? Then we can learn from them. But I see
years, and many of the results are spectacular and no signs that there are any breakthroughs around
beautiful. I can’t claim that I’m good at that par- the corner.
ticular subject, though, because I can’t think of Two weeks ago in Greece I was at the inaugura-
sneaky attacks myself. But the key problem is, tion of a new book by Christos Papadimitriou, who
what about the abuse of secure methods of com- is chairman of computer science at Berkeley. He
munication? I don’t want criminals to use these published a novel in the Greek language called The
methods to become better criminals. Smile of Turing [Pa]. I don’t want to give away the
I’m a religious person, and I think that God story, but when it gets published in German or
knows all my secrets, so I always feel that whatever English, you’ll find that it has a very nice discus-
I’m thinking is public knowledge in some way. I sion of artificial intelligence and the Turing test for
come from this kind of background. I don’t feel intelligence.
I have to encrypt everything I do. On the other The most promising model of how the brain
hand, I would certainly feel quite differently if works that I’ve seen says that the brain is a dynamic
somebody started to use such openness against me, genetic algorithm that operates all the time. As I

MARCH 2002 NOTICES OF THE AMS 323

 am talking to you now, your brains have a lot of integer really discovered by man? Or is it something
competing theories about what I’m going to say. It’s that is God given? When we start thinking of com-
the survival of the fittest, a continual plexity issues, we have to change our viewpoint as
battle among the competing theories. to what is in nature and what is invented.
Some come to the surface and actually Question: You have been writing checks to peo-
enter your consciousness, but the ple who point out errors in your books. I have never
others are all there. Some kind of mat- heard of anyone cashing these checks. Do you know
ing of concepts might be going on in our how much money you would be out of, if everyone
heads all the time. This model seems to suddenly cashed the checks?
have the right properties to account for Knuth: There’s one man who lives near Frank-
how we can do what we do with the furt who would probably have more than $1,000
relatively slow response time that our if he cashed all the checks I’ve sent him. There’s a
neurons have. But I am by no means an man in Los Gatos, California, whom I’ve never met,
expert on this. who cashes a check for $2.56 about once a month,
Question: What is your thinking about and that’s been going on for some years now.
software patents? There is a big discus- Altogether I’ve written more than 2,000 checks
sion going on in Europe right now about over the years, and the average amount exceeds
whether software should be patentable. $8.00 per check. Even if everybody cashed their
Knuth: I’m against patents on things checks, it would still be more than worth it to me
that any student should be expected to to know that my books are getting better.
discover. There have been an awful lot
References
of software patents in the U.S. for ideas
[TAOCP] The Art of Computer Programming, by Donald E.
that are completely trivial, and that
Knuth. Volume 1: Fundamental Algorithms (third
bothers me a lot. There is an organiza- edition, Addison-Wesley, 1997). Volume 2: Semi-
tion that has worked for many years to numerical Algorithms (third edition, Addison-Wesley,
make patents on all the remaining triv- 1997). Volume 3: Sorting and Searching (second
ial ideas and then make these available edition, Addison-Wesley, 1998). Volume 4: Combina-
to everyone. The way patenting had torial Algorithms (in preparation).
been going was threatening to make [MT] Mathematical typography, by Donald E. Knuth. Bull.
the software industry stand still. Amer. Math. Soc. (N.S.) 1 (1979), no. 2, 337–372.
Algorithms are inherently mathe- Reprinted in Digital Typography (Stanford, Califor-
matical things that should be as un- nia: CSLI Publications, 1998), pp. 19–65.
[PITAC] President’s information technology advisory com-
patentable as the value of π . But for
mittee. See http://www.itrd.gov/ac/.
something nontrivial, something like [PFB] Proofs from The Book, by Martin Aigner and Gün-
the interior point method for linear pro- ter Ziegler. Second edition, Springer Verlag, 2001.
gramming, there’s more justification [KB] Simple word problems in universal algebras, by
for somebody getting a right to license Peter B. Bendix and Donald Knuth. Computational
the method for a short time, instead of Problems in Abstract Algebra, J. Leech, ed. (Oxford:
keeping it a trade secret. That’s the Pergamon, 1970), pp. 263–297. Reprinted in Au-
whole idea of patents; the word patent tomation of Reasoning, Jörg H. Siekmann and Graham
means “to make public”. Wrightson, eds. (Springer, 1983), pp. 342–376.
[BTI] 3:16 Bible Texts Illuminated, by Donald E. Knuth.
I was trained in the culture of mathematics, so
A-R Editions, Madison, Wisconsin, 1990.
I’m not used to charging people a penny every time
[AG] Semantics of context-free languages, by Donald E.
they use a theorem I proved. But I charge somebody Knuth. Mathematical Systems Theory 2 (1968),
for the time I spend telling them which theorem 127–145. See also The genesis of attribute gram-
to apply. It’s okay to charge for services and mars, in Lecture Notes in Computer Science 461
customization and improvement, but don’t make (1990), 1–12.
the algorithms themselves proprietary. [Pa] TO XAMOGELO TOY TOYRINGK (The Smile of Tur-
There’s an interesting issue, though. Could you ing), by Christos Papadimitriou. Livani Publishers,
possibly have a patent on a positive integer? It is Athens, Greece, 2001.
not inconceivable that if we took a million of the [Ku] The Age of Spiritual Machines: When Computers Ex-
greatest supercomputers today and set them going, ceed Human Intelligence, by Ray Kurzweil. Penguin
USA, 2000.
they could compute a certain 300-digit constant
[Mo] Robot: Mere Machine to Transcendent Mind, by
that would solve any NP-hard problem by taking Hans P. Moravec. Oxford University Press, 2000.
the GCD of this constant with an input number, or
by some other funny combination. This integer
would require massive amounts of computation
Photographs used in this article are courtesy of
time to find, and if you knew that integer, then you
Andreas Jung, Technische Universität München.
could do all kinds of useful things. Now, is that

324 NOTICES OF THE AMS VOLUME 49, NUMBER 3