Incompleteness and Randomness
Incompleteness and Randomness
Beyond
Cristian S. Calude
Department of Computer Science
University of Auckland
Auckland, New Zealand
Email: [email protected]
Gödel’s Incompleteness Theorems have the same scientific status as Einstein’s princi-
ple of relativity, Heisenberg’s uncertainty principle, and Watson and Crick’s double helix
model of DNA. Our aim is to discuss some new faces of the incompleteness phenomenon
unveiled by an information-theoretic approach to randomness and recent developments
in quantum computing.
After proving the completeness of the system of predicate logic in his doctoral dis-
sertation (1929), Gödel has continued the investigation of the completeness problem for
more comprehensive formal systems, especially systems encompassing all known methods
of mathematical proof. In 1931 (see [44]) Gödel proved his famous First Incompleteness
Theorem, which in modern terms reads:
The system is computably enumerable if its ‘theorems’ can be listed by a Turing machine.
Informally, the set of axioms and deduction rules generates all ‘theorems’; for example,
we cannot take as axioms all true statements about natural numbers as this set is not
computably enumerable. The condition that the system contains the elementary arith-
metic is also essential. For example, the Euclidean geometry which makes statements
only about points, circles and lines in general does not satisfy this condition, hence it
might be complete; and, indeed, it is complete as Tarski has proved. The flat nature of
the Euclidean geometry plays no role here, non-Euclidean geometries are also complete.
This result together with the Second Incompleteness Theorem (which states that the
consistency of the axioms cannot be proved within the system) ended a hundred years
of attempts to establish axioms to put mathematics on an axiomatic basis. Gödel’s In-
completeness Theorem does not destroy the fundamental idea of formalism, but suggests
that a) mathematics will be described by many formal systems instead of a universal one,
b) a more sophisticated and comprehensive form of formal system than that envisaged
by Hilbert is required (see also Post [65]).
Anticipating resistance to his conclusions Gödel wrote his papers very carefully. Spec-
ulating on his extreme caution, Feferman [43] stated that Gödel “could have been more
centrally involved in the development of the fundamental concepts of modern logic –
truth and computability – than he was.” Gödel took pain to convince various people
about the validity of his assertions and results, but he avoided any public debate and
considered his results to have been accepted by those whose opinion mattered to him.
For example, P. Finsler, E. Post and E. Zermelo were concerned with priority issues,
while C. Perelman, M. Barzin, J. Kuczyński asserted that Gödel had in fact discovered
another antinomy; see [37]. Unlike the others, Post expressed “the greatest admiration”
for Gödel’s work, conceding that “after all it is not ideas but the execution of ideas that
constitute[s] . . . greatness”. Gödel’s result provoked Hilbert’s anger, but he apparently
accepted its correctness (cf. [37]). Hilbert never cited Gödel’s work.
2
The reactions of two great philosophers are also of interest. Wittgenstein’s negative
comments (dated 1938 and posthumously published in “Remarks on the foundations of
mathematics” in [86]) are now generally considered an embarrassment in the work of a
great philosopher. Russell realized the importance of Gödel’s work, but expressed his
continuous puzzlement in a rather ambiguous way in a letter from 1 April 1963 (addressed
to L. Henkin; see [37]): Are we to think that 2+2 is not 4, but 4.001?. Following the same
source, Gödel remarked (in a letter addressed to A. Robinson) that “Russell evidently
misinterprets my result; however he does so in a very interesting manner . . . ”.
In 1936 Turing [80] showed the undecidability of the Halting Problem, the question
of whether a given computer program will eventually halt:
These two results have very deep connections. To understand them we need to
examine a very delicate notion: randomness.
2 Randomness
What is randomness? Are there random events in nature? Are there laws of randomness?
Even today, these few questions stir controversy.
I am convinced that the vast majority of my readers, and in fact the vast
majority of scientists and even nonscientists, are convinced that they know
what ‘random’ is. A toss of a coin is random; so is a mutation, and so is the
emission of an alpha particle. . . . Simple, isn’t it? said Kac in [51].
Well, no! Kac knew very well that randomness, the very stuff of life, could be
called many things, but not simple. The fact that maintaining perfect order is difficult
surprises no one, but it may come as something of a “revelation” that perfect disorder
is beyond reach. People, even experts, perform poorly when dealing with randomness.
The “gambler fallacy” is a classical example: the common belief that after a sequence of
losses in a game of chance there will probably follow a sequence of gains is false. Various
explanations have been suggested: according to one of them, the human cognitive and
psychological constitution, trained over the years to look for patters and trends (even
where there are none) is “blind” when it comes to see randomness.
Randomness is a most troubling concept – it is hard not only to attain but also to
define or even to imagine in spite of the fact that have been heroic efforts to understand
randomness (cf. Efron (cited in Kolata [55]).
Books on probability theory do not even attempt to define it: It’s like the concept of
a point in geometry books. According to Beltrami [5]):
3
The subject of probability begins by assuming that some mechanism of un-
certainty is at work giving rise to what is called randomness, but it is not
necessary to distinguish between chance that occurs because of some hidden
order that may exist and chance that is the result of blind lawlessness. This
mechanism, figuratively speaking, churns out a succession of events, each in-
dividually unpredictable, or it conspires to produce an unforeseeable outcome
each time a large ensemble of possibilities is sampled.
There are various equivalent ways to define the notion of (algorithmic) ran-
dom sequence: measure-theoretical definitions (Martin-Löf [63, 62] and Solovay [74]),
information-theoretical definitions (Chaitin [23] and Schnorr), topological definition
(Hertling and Weihrauch [50]). The information-theoretic characterizations read:
4
An infinite sequence x = x1 x2 . . . xn . . . is Chaitin random if
limn→∞ HU (x1 x2 . . . xn ) − n = ∞; x is Schnorr random if there is a con-
stant c > 0 such that HU (x1 x2 . . . xn ) > n − c, for every integer n > 0.
• global disorder contrasts with local total order (any pattern appears).
3 Information-Theoretic Incompleteness
Is there any relation between randomness and incompleteness? The answer is affirmative
and one possibility to reveal such relations is to look at a special class of reals – the
computable enumerable reals (see Soare [72]).
In 1975 a more modern version of the Halting Problem emerged. Chaitin [23] intro-
duced the probability that an arbitrary universal Chaitin machine will eventually halt:
X
ΩU = 2−|x| .
U (x) stops
5
Each ΩU depends on the choice of U , so there is not just one Omega (as there is
only one π), but a class of Omegas. This observation leads to Solovay’s question ([74]):
Are there random and computably enumerable real numbers other than Omegas? The
answer is negative, and the proof is constructive, cf. Calude, Hertling, Khoussainov,
Wang [14], Slaman [71] (see also Kučera and Slaman [58]):
To make the discussion more concrete we will formulate all results relative to ZF C,
Zermelo-Fraenkel set theory with choice; all theorems hold true under more general
conditions. The First Information-theoretic Incompleteness Theorem (Chaitin [23]) is:
In fact, there is a constant c > 0 such that ZF C cannot prove the statement “HU (x) >
m” if m > HU (ZF C) + c. So, all true statements “HU (x) > m” (an infinite set) are
unprovable in ZF C. Recognizing high complexity is a difficult task even for ZF C. The
difficulty depends upon the choice of U : some U ’s are worse than others. Raatikainen [66]
has shown that there exists a universal Chaitin machine U so that ZF C, if arithmetically
sound, can prove no statement of the form “HU (x) > 0”. It follows that ZF C, if
arithmetically sound, can prove no (obviously, true) statement of the form “HU (x) > 0”.
For every c.e. and random real α we can construct a universal Chaitin machine U
such that α = ΩU and ZF C is able to determine finitely (but as many as we want) bits
of ΩU . Solovay [75] went into the opposite direction by showing that:
6
Chaitin’s Second Information-theoretic Incompleteness Theorem holds true for any
universal Chaitin machine while Solovay constructed a specific machine. A Chaitin ma-
chine for which Peano Arithmetic can prove its universality and ZF C cannot determine
more than the initial block of 1’s of the binary expansion of its halting probability will
be called Solovay machine. Which c.e. and random reals are halting probabilities of
Solovay machines? Calude [10] proved the following result:
Assume that ZF C is arithmetically sound. Then, every c.e. and random real
is the halting probability of a Solovay machine.
For example, if α ∈ (3/4, 7/8) is c.e. and random, then in the worst case ZF C
can determine its first two bits (11), but no more. Assume that ZF C is arithmetically
sound. Then, every c.e. and random real α ∈ (0, 1/2) is the halting probability of a
Solovay machine which cannot determine any single bit of α. No c.e. and random real
α ∈ (1/2, 1) has the above property.
7
. . . besides mathematical intuition there exists another (though only probable)
criterion of truth of mathematical axioms, namely their fruitfulness in math-
ematics, and one may add, possibly also in physics . . . The simplest case of
an application of the criterion under discussion arises when some . . . axiom
has number-theoretical consequences verifiable by computation up to any given
integer.
Do these results have any impact on mathematics and/or the philosophy of mathe-
matics? Opinions vary dramatically. H. Weyl described incompleteness in a pessimistic
way, as a constant drain on the enthusiasm of pursuing scientific research; F. Dyson sees
it in an optimistic way, as an insurance policy that science will go on for ever. And, of
course, some would argue that the work of the overwhelming majority of mathematicians
and philosophers has been quite unaffected by the incompleteness results. One thing is
certain: incompleteness has captured the interest of many. Many books and thousands
of technical papers discuss it and and its implications and the March 29 1999 issue of
TIME magazine has included Gödel and Turing in its list of the twenty greatest twenty
scientists and thinkers of the twentieth century.
4 Beyond
Classically, there are two equivalent ways to look at the mathematical notion of proof:
a) as a finite sequence of sentences strictly obeying some axioms and inference rules, b)
as a specific type of computation. Indeed, from a proof given as a sequence of sentences
one can easily construct a machine producing that sequence as the result of some finite
computation and, conversely, giving a machine computing a proof we can just print all
sentences produced during the computation and arrange them in a sequence. A proof
is an explicit sequence of reasoning steps that can be inspected at leisure; in theory, if
followed with care, such a sequence either reveals a gap or mistake, or can convince a
skeptic of its conclusion, in which case the theorem is considered proven.
This equivalence has stimulated the construction of programs which perform like ar-
tificial mathematicians.1 From proving simple theorems of Euclidean geometry to the
proof of the four-color theorem, these “theorem provers” have been very successful. Of
course, this was a good reason for sparking lots of controversies. Artificial mathemati-
cians are far less ingenious and subtle than human mathematicians, but they surpass
their human counterparts by being infinitely more patient and diligent. What about
making errors? Are human mathematicians less prone to errors? This is a difficult
question which requires more attention.
If a conventional proof is replaced by a “quantum computational proof” (or a proof
produced as a result of a molecular experiment), then the conversion from a computation
to a sequence of sentences may be impossible, e.g., due to the size of the computation.
For example, a quantum machine could be used to create some proof that relied on
quantum interference among all the computations going on in superposition. The quan-
tum machine would say “your conjecture is true”, but there will be no way to exhibit
1
Other types of “reasoning” such as medical diagnosis or legal inference have been successfully modeled
and implemented; see, for example, the British National Act which has been encoded in first-order logic
and a machine has been used to uncover its potential logical inconsistencies.
8
all trajectories followed by the quantum machine in reaching that conclusion. In other
words, the quantum machine has the ability to check a proof, but it may fail to reveal
any “trace” of how it did it. Even worse, any attempt to watch the inner working of the
quantum machine (e.g. by “looking” at any information concerning the state of the on
going proof) may compromise for ever the proof itself!
These facts may not affect the essence of mathematical objects and constructions
(which have an autonomous reality quite independent of the physical reality), but they
seem to have an impact of how we learn/understand mathematics (which is thorough
the physical world). Indeed, our glimpses of mathematics seem to be “revealed” through
physical objects, i.e. human brains, silicon computers, quantum Turing machines, etc.,
hence, according to Deutsch [41], they have to obey not only the axioms and the inference
rules of the theory, but the laws of physics as well.
The question of trespassing Turing’s barrier, i.e. the possibility to solve a Turing
undecidable problem, to compute an uncomputable function has been considered by
various authors, for example, [70, 34, 35]. Is there any hope for quantum (or DNA)
computing to challenge the Turing barrier, i.e. to solve an undecidable problem, to
compute an uncomputable function? According to Feynman’s argument (see [45], a paper
reproduced also in [46]) any quantum system can be simulated with arbitrary precision
by a (probabilistic) Turing machine, so the answer seems to be negative. However,
some recent tentative approaches promise a positive answer: for quantum approaches
see [12, 13, 18, 52] and for DNA methods see [19].
Is incompleteness affected? We need more understanding of the quantum world to
be able to answer this question. One step toward a possible answer to this question is
too look at the quantum version of Ω, the number Ωq invented in 1995 by G. Chaitin, K.
Svozil and A. Zeilinger (see [78, 85]; see also [52, 83]). The number Ωq is the probability
amplitude with which a random quantum program halts on a self-delimiting universal
quantum machine (hence, the halting probability of a self-delimiting universal quantum
machine is |Ωq |2 ).2 For computing Ωq only the quantum versions of classical bits in the
domain of the quantum machine are allowed as inputs, so from the computability point
of view Ωq is an Ω, hence all information-theoretic results remain unchanged. However,
if some “quantum machine” would be able to solve the Halting Problem (for classical
Turing machines), then its halting probability will be an α number (as introduced in [4]),
a random, but not c.e. real; the “incompleteness” derived from such a number has not
(yet) been studied.
As is pointed out in [13], even if theoretically one could show that Turing’s barrier
can be trespassed by a quantum machine, the impact on computer technology would be
very, very low. So, when reality is so far way from theory, why are we concerned with the
later? According to Landauer [60] the answer is: Because it is at the very core of science.
. . . Information, numerical or otherwise, is not an abstraction, but it is inevitable tied to
a physical representation. . . . the handling of information is inevitable tied to the physical
universe, its contents and its laws.
2
Things are more complicated as the halt bit of the quantum machine might enter a superposition
state and remain there while other parts of the output state describing the quantum machine continue
to change. Finally, to settle the matter one has to perform a measurement.
9
5 Bibliographical Comments
Acknowledgment
We thank Greg Chaitin, Fred Kroon, Sergiu Rudeanu, Jerry Seligman and Karl Svozil
for useful comments and criticism.
Bibliography
[1] D. Auburn. Proof. A Play, Faber & Faber, New York, 2001.
[2] J. D. Barrow. Impossibility—The Limits of Science and the Science of Limits, Ox-
ford University Press, Oxford, 1998.
[3] J. D. Barrow. Mathematical jujitsu: Some informal thoughts about Gödel and
physics, Complexity, 5, 5 (2000), 28-34.
[4] Verónica Becher, S. Daicz and G. Chaitin. A highly random number, in C. S. Calude,
M. J. Dinneen, S. Sburlan (eds.). Combinatorics, Computability and Logic, Proceed-
ings of DMTCS’01, Springer-Verlag, London, 2001, 55–68.
[5] E. Beltrami. What is Random? Chance and Order in Mathematics and Life,
Springer-Verlag, New York, 1999.
[6] C. H. Bennett, M. Gardner. The random number omega bids fair to hold the mys-
teries of the universe, Scientific American 241 (1979) 20–34.
10
[10] C. S. Calude, Chaitin Ω numbers, Solovay machines and incompleteness, Theoret.
Comput. Sci., to appear in 2002.
[12] C.S. Calude, M. J. Dinneen, K. Svozil. Counterfactual Effect, the Halting Problem,
and the Busy Beaver Function (Preliminary Version), CDMTCS Research Report
107, 1999, 8 pp.
[17] C. Calude, A. Nies. Chaitin Ω numbers and strong reducibilities, J. Univ. Comput.
Sci. 3 (1997), 1161–1166.
[19] C. S. Calude, G. Păun. Computing with Cells and Atoms, Taylor & Francis Pub-
lishers, London, 2001.
[20] J. L. Casti. Five More Golden Rules—Knots, Codes, Chaos, and Other Great The-
ories of 20th-Century Mathematics, Wiley, New York, 2000.
[22] G. J. Chaitin. On the length of programs for computing finite binary sequences, J.
Assoc. Comput. Mach. 13(1966), 547-569. (Reprinted in: [26], 219-244.)
11
[27] G. J. Chaitin. Information-Theoretic Incompleteness, World Scientific, Singapore,
1992.
[32] M. Chow. The Omega man, New Scientist 10 March (2001), 29-31.
[33] G. P. Collins. Computing with light, Scientific American, Aug. (2001), 12.
[36] J. W. Dawson, Jr. Kurt Gödel in sharper focus, The Mathematical Intelligencer 6
(1984), 9-17.
[37] J. W. Dawson, Jr. Logical Dilemmas. The Life and Work of Kurt Gödel, A K Peters,
Wellesley, Massachusetts, 1997.
[41] D. Deutsch. Quantum theory, the Church-Turing principle and the universal quan-
tum computer, Proceedings of the Royal Society London, A 400 (1985), 97–119.
[42] A. Doxiadis. Uncle Petros & Goldback’s Conjecture. A Novel about Mathematical
Obsession, Bloomsbury, New York, 2000.
[43] S. Feferman. Kurt Gödel: Conviction and caution, Philos. Natur. 21 (1984), 546–
562.
[45] R.P. Feynman, Simulating physics with computers, International Journal of Theo-
retical Physics, 11 (1985), 11–20.
[46] J.G. Hey (ed.). Feynman and Computation. Exploring the Limits of Computers,
Perseus Books, Reading, Massachusetts, 1999.
12
[47] K. Gödel. Russell’s mathematical logic, in P. Benacerref, H. Putnam (eds.). Philos-
ophy of Mathematics, Prentice-Hall, Englewood Cliffs, New Jersey, 1964, 211–232.
[52] T. D. Kieu. Hilbert’s incompleteness, Chaitin’s Ω number and quantum physics, Los
Alamos preprint archive http://arXiv:quant-ph/0111062, v1, 10 November 2001.
[53] T. D. Kieu. A reformulation of the Hilbert’s tenth problem through quantum me-
chanics, Los Alamos preprint archive http://arXiv:quant-ph/0111063, v1, 12
November 2001.
[54] S. C. Kleene. The work of Kurt Gödel, J. Symbolic Logic 41 (4) (1976), 761–778.
An addendum to “The work of Kurt Gödel”, J. Symbolic Logic 43 (3) (1978), 613.
[56] A. N. Kolmogorov. Three approaches for defining the concept of “information quan-
tity”, Problems Inform. Transmission 1(1965), 3-11.
[57] G. Kreisel. Kurt Gödel, Biographical Memoirs of Fellows of the Royal Society of
London 26 (1980), 149-224, 27, 697, 28, 718.
[63] P. Martin-Löf. The definition of random sequences, Inform. and Control 9 (1966),
602–619.
[64] E. Nagel, J. R. Newman. Gödel’s Proof, New York University Press, New York, 1958
(second printing 1986).
13
[65] E. Post. Absolutely unsolvable problems and relatively undecidable propositions:
account of an anticipation, in M. Davis (ed.). The Undecidable, Raven Press, New
York, 1965, 340–433.
[67] H. Rice. Recursive reals, Proc. Amer. Math. Soc. 5 (1954), 784–791.
[69] R. Rucker. Infinity and the Mind, Bantam Books, New York, 1982.
[70] H. Siegelmann. Computation beyond the Turng limit, Science 268 (1995), 545-548.
(April 28).
[72] R. I. Soare. Recursion theory and Dedekind cuts, Trans. Amer. Math. Soc. 140
(1969), 271–294.
[73] R. J. Solomonoff. A formal theory of inductive inference, Part 1 and Part 2, Inform.
and Control 7(1964), 1-22 and 224-254.
[74] R. M. Solovay. Draft of a paper (or series of papers) on Chaitin’s work ... done for
the most part during the period of Sept.–Dec. 1974, unpublished manuscript, IBM
Thomas J. Watson Research Center, Yorktown Heights, New York, May 1975, 215
pp.
[75] R. M. Solovay. A version of Ω for which ZF C can not predict a single bit, in
C.S. Calude, G. Păun (eds.). Finite Versus Infinite. Contributions to an Eternal
Dilemma, Springer-Verlag, London, 2000, 323-334.
[76] E. Specker. Nicht konstruktiv beweisbare Sätze der Analysis, J. Symbolic Logic 14
(1949), 145–158.
[79] L. Staiger. The Kolmogorov complexity of real numbers, in G. Ciobanu and Gh.
Păun (eds.). Proc. Fundamentals of Computation Theory, Lecture Notes in Comput.
Sci. No. 1684, Springer–Verlag, Berlin, 1999, 536-546.
14
[83] P. M. Vitányi. Quantum Kolmogorov complexity based on classical descriptions,
IEEE Trans. Inform. Theory 47,6 (2001), 2464-2479.
[85] C. P. Williams, S. H. Clearwater. Ultimate Zero and One, Copernicus, New York,
2000.
15