Computational Depth and Reducibility

David W. Juedes, James I. Lathrop, and Jack H. Lutz

Department of Computer Science
Iowa State University
Ames, IA 50011

Abstract
This paper reviews and investigates Bennett's notions of strong
and weak computational depth (also called logical depth) for innite
binary sequences. Roughly, an innite binary sequence x is dened to
be weakly useful if every element of a non-negligible set of decidable
sequences is reducible to x in recursively bounded time. It is shown
that every weakly useful sequence is strongly deep. This result (which
generalizes Bennett's observation that the halting problem is strongly
deep) implies that every high Turing degree contains strongly deep
sequences. It is also shown that, in the sense of Baire category, almost
every innite binary sequence is weakly deep, but not strongly deep.

Contents
1 Introduction 2
2 Preliminaries 5
3 Measure and Category 7
4 Algorithmic Information and Randomness 14
5 Strong Computational Depth 22
6 Weak Computational Depth 37
7 Conclusion 40
This research was supported in part by National Science Foundation Grant CCR-
9157382, with matching funds from Rockwell International and Microware Systems
Corporation.

1
1 Introduction
Algorithmic information theory, as developed by Solomono 51],
Kolmogorov 21, 22, 23], Chaitin 9, 10, 11, 12], Martin-Lof 39, 40], Levin
26, 27, 28, 29, 30, 31, 55], Schnorr 47], Gacs 15], Shen 48, 49], and others,
0

gives a satisfactory, quantitative account of the information content of in-

dividual binary strings (nite) and binary sequences (innite). However, a
given quantity of information may be organized in various ways, rendering it
more or less useful for various computational purposes. In order to quantify
the degree to which the information in a computational, physical, or bio-
logical object has been organized, Bennett 4, 5] has extended algorithmic
information theory by dening and investigating the computational depth of
binary strings and binary sequences.
Roughly speaking, the computational depth (called \logical depth" by
Bennett 4, 5]) of an object is the amount of time required for an algorithm to
derive the object from its shortest description. (Precise denitions appear
in the sections to follow.) Since this shortest description contains all the
information in the object, the depth thus represents the amount of \compu-
tational work" that has been \added" to this information and \stored in the
organization" of the object. (Depth is closely related to Adleman's notion
of \potential" 1] and Koppel's notion of \sophistication" 24, 25].)
One way to investigate the computational usefulness of an object is to
investigate the class of computational problems that can be solved eciently,
given access to the object. When the object is an innite binary sequence,
i.e., a sequence x 2 f0 1g , this typically amounts to investigating the
1

class of binary strings y 2 f0 1g that are Turing reducible to x in some

1

recursive time bound s : N ! N. This condition, written y DTIME( s) x,

means that there is an oracle Turing machine M that, on input n 2 N with
T

oracle x, computes y n], the nth bit of y , in at most s(l) steps, where l is the
number of bits in the binary representation of n. For example, consider the
diagonal halting problem
K 2 f0 1g , whose nth bit
K n] is 1 if and only
1

if Mn , the nth Turing machine, halts on input n. It is well-known that
K is

useful, in the sense that every recursive sequence (in fact, every recursively
enumerable sequence) y 2 f0 1g is Turing reducible to
K in polynomial
1

time.
An interesting feature of this example is that
K has relatively low in-
formation content. In fact, an n-bit prex of
K , denoted
K 0::n ; 1],
contains only O(log n) bits of algorithmic information 3]. Intuitively, this
is because
K 0::n ; 1] is completely specied by the number of indices

2
i 2 f0 : : : n ; 1g such that the ith Turing machine Mi halts on input i. Once
this O(log n)-bit number is known, direct simulation of M0 M1    Mn 1 ;

on inputs 0 1 : : : n ; 1, respectively, will eventually determine all n bits of

K 0::n ; 1].
In contrast, consider a sequence z 2 f0 1g that is algorithmically ran-
1

dom in the equivalent senses of Martin-Lof 39], Levin 26], Schnorr 47],
Chaitin 11], Solovay 52], and Shen 48, 49]. (See section 4 below for a
0

precise denition and basic properties of algorithmic randomness.) An n-bit

prex z 0::n ; 1] of an algorithmically random sequence z contains approxi-
mately n bits of algorithmic information 39], so the information content of
z is exponentially greater than that of
K . On the other hand, z is much
less useful than
K , in the following sense. While every recursive sequence is
Turing reducible to
K in polynomial time, a recursive sequence y 2 f0 1g 1

is Turing reducible to z in polynomial time if and only if y is in the com-

plexity class BPP 5, 8]. (The class BPP, dened by Gill 17], consists of
those sequences y 2 f0 1g such that there is a randomized algorithm that
1

decides y n], the nth bit of y , with error probability less than n1 , using time
that is at most polynomial in the number of bits in the binary representation
of n.) Since BPP contains only the simplest recursive sequences, this means
that, for the purpose of eciently deciding recursive sequences,
K is much
more useful than an algorithmically random sequence z .
Bennett has argued that the computational usefulness of
K derives not
from its algorithmic information content (which is relatively low), but rather
from its computational depth. In support of this thesis, Bennett 5] has
proven that
K is strongly deep, while no algorithmically random sequence
can even be weakly deep. (Precise denitions of these terms appear in sec-
tions 5 and 6 below.)
This paper furthers Bennett's investigation of the computational depth
of innite binary sequences. We pay particular, quantitative attention to
interactions between computational depth and time-bounded Turing reduc-
tions.
In order to further investigate the above-discussed notion of the compu-
tational usefulness of a sequence x 2 f0 1g , we quantify the size of the set
1

of recursive sequences that are Turing reducible to x within some recursive

time bound. For this purpose, let REC be the set of all recursive (i.e., decid-
able) sequences, and, for a recursive time bound s : N ! N, let DTIMEx (s)
be the set of all sequences y 2 f0 1g such that y DTIME(
1 s) x. We are
T
interested in the size of DTIMEx (s) \ REC as a subset of REC. To quantify

3
this, we use a special case of the resource-bounded measure theory of Lutz
37, 36]. (A detailed description of the relevant special case appears in sec-
tion 3 below.) Intuitively, this theory, a generalization of classical Lebesgue
measure theory, denes a set X of innite binary sequences to have measure
0 in REC if X \ REC is a negligibly small subset of REC.
In this paper, we dene a sequence x 2 f0 1g to be weakly useful
1

if there exists a recursive time bound s : N ! N such that DTIMEx (s)

does not have measure 0 in REC. Returning to the two examples discussed
earlier,
K is weakly useful because every element of REC is in DTIME K (s),
provided that s is superpolynomial, e.g. if s(n) = nlog n . On the other hand,
if z is algorithmically random, then z is not weakly useful, by the following
two facts.
(i) For every recursive time bound s : N ! N there exists a recursive time
bound sb : N ! N such that, for all algorithmically random sequences
z , DTIMEz (s) \ REC  DTIME(sb) 5, 8, 7].
(ii) For every recursive time bound sb : N ! N, DTIME(sb) has measure 0
in REC 37].
Our main result, Theorem 5.11 below, establishes that every weakly
useful sequence is strongly deep. This implies that every high Turing degree
contains strongly deep sequences (Corollary 5.15). Since the Turing degree
of
K is one of many high Turing degrees, our main result thus generalizes
Bennett's result 5] that
K is strongly deep.
More importantly, our main result rigorously conrms Bennett's intuitive
arguments relating the computational usefulness of
K to its depth. The
fact that the useful sequence
K is strongly deep is no coincidence. Every
sequence that is even weakly useful must be strongly deep.
Bennett 5] also denes the class of weakly deep binary sequences. (As
noted by Bennett, this class has been investigated in other guises by Levin
and V'jugin 28, 31, 32, 53, 54, 55].) A sequence x 2 f0 1g is weakly deep
1

if there do not exist a recursive time bound s : N ! N and an algorithmi-

cally random sequence z such that x DTIME( s) z . Bennett 5] notes that
T
every strongly deep sequence is weakly deep, but that there exist weakly
deep sequences that are not strongly deep. In section 6 below we strengthen
the separation between these two notions by proving that, in the sense of
Baire category, almost every sequence x 2 f0 1g is weakly deep, but not
1

strongly deep. (A self-contained discussion of Baire category appears in

4
section 3.) Intuitively, this means that weakly deep sequences are \topolog-
ically abundant." (They \cannot be avoided" by one player in a two-person
game described in section 3.) In contrast, weakly deep sequences are \prob-
abilistically scarce," in the sense that, with respect to Lebesgue measure,
almost every sequence x 2 f0 1g is algorithmically random 39], hence not
1

weakly deep.
In order to provide a basis for further investigation of Bennett's funda-
mental ideas, this paper also includes a self-contained mathematical treat-
ment of the weak and strong computational depth of innite sequences. In
section 2 we introduce our basic terminology and notation. In section 3
we review fundamental ideas of Baire category and measure that are used
in our work. In section 4 we give a similar review of algorithmic informa-
tion and randomness. Section 5 is the main section of the paper. In this
section, we present the strong computational depth of innite binary se-
quences in a unied, self-contained framework using a convenient family of
parametrized depth classes, Dtg . This framework is used to prove our main
result (Theorem 5.11), that every weakly useful sequence is strongly deep.
In the course of our development, we prove several results, some of which
were already proven by Bennett 5], giving precise, quantitative relationships
among depth, randomness, and recursiveness. We also prove (Theorem 5.16)
that strongly deep sequences are extremely rare, in that they form a meager,
measure 0 subset of f0 1g . In section 6 we give a brief discussion of weak
1

computational depth, including a proof that, in the sense of Baire category,

almost every sequence is weakly deep, but not strongly deep. In section 7
we mention possible directions for further research.

2 Preliminaries
We work primarily in the set f0 1g of all (innite, binary) sequences. We
1

also use the set f0 1g of all (nite, binary) strings. We write jxj for the


length of a string x, and  for the empty string. The standard enumeration
of f0 1g is the sequence s0 s1 : : :, in which shorter strings precede longer


ones and strings of the same length are ordered lexicographically.

Given a sequence x 2 f0 1g and m n 2 N with m  n, we write
1

xm::n] for the string consisting of the mth through nth bits of x. In par-
ticular, x0::n ; 1] is the string consisting of the rst n bits of x. We write
xn] for xn::n], the nth bit of x.

5
 We write  '] for the Boolean value of a condition ', i.e.,
(
 '] = 10 ifif '' isis true
false
The characteristic sequence of a set A  N is then the sequence
A 2 f0 1g 1

dened by
A n] =  n 2 A] for all n 2 N.

We say that a condition '(n) holds in
nitely often (i.o.) if it holds
for innitely many n 2 N. We say that a condition '(n) holds almost
everywhere (a.e.) if it holds for all but nitely many n 2 N.
All logarithms in this paper are base-2 logarithms.
Given a function f : Nn  f0 1g ! Y and an n-tuple ~k 2 Nn , we


dene the function f~k : f0 1g ! Y by f~k (x) = f (~k x) for all x 2 f0 1g .

 

This enables us to regard the function f as a \uniform enumeration" of the

functions f~k .
Although we introduce a very specic Turing machine model to dene
algorithmic information, algorithmic probability, and algorithmic depth in
sections 4 and 5, we assume that the reader is already familiar with the
general ideas of Turing machine computation, including computation by
oracle Turing machines. (Discussion of such machines may be found in
many texts, e.g., 2, 19, 44, 50].)
Given a recursive time bound s : N ! N, we say that an oracle Turing
machine M is s-time-bounded if, given any input n 2 N and oracle y 2
f0 1g , M outputs a bit M y (n) 2 f0 1g in at most s(l) steps, where l
1

is the number of bits in the binary representation of n. In this case, if

x 2 f0 1g satises xn] = M y (n) for all n 2 N, then we say that x is
1

Turing reducible to y in time s via M , and we write x DTIME( s) y via M .

T
We say that x is Turing reducible to y in time s, and we write x DTIME( s) y ,
T
if there is some oracle Turing machine M such that x DTIME( s) y via M .
For y 2 f0 1g and s : N ! N, we write
T
1

n
o
DTIMEy (s) = x 2 f0 1g

x DTIME(s) y :
1
T

(Note that the time bound here is \sharp" there is no \big-O.") The1 un-
relativized complexity class DTIME(s) is then dened to be DTIME0 (s),
where 0 is the sequence consisting entirely of 0's.
1

A sequence x 2 f0 1g is truth-table reducible to a sequence y 2 f0 1g ,

1 1

and we write x tt y , if there exists a recursive time bound s : N ! N

such that x DTIME( s) y . (This denition is easily seen to be equivalent to
T

6
standard textbook denitions of truth-table reducibility 44, 50].) Given a
set Y  f0 1g , we write
1

RECtt (Y ) = fx 
2 f0 1g j (9y 2 Y ) x tt y g
1

= DTIMEy (s):
recursive s y Y
2

We write REC for the set of all recursive (i.e., decidable) sequences x 2
f0 1g . Note that REC Y  RECtt (Y ) for all sets Y  f0 1g . A
1 1

sequence x 2 f0 1g is Turing reducible to a sequence y 2 f0 1g , and we

1 1

write x T y , if there is an oracle Turing machine M such that M y (n) = xn]

for every n 2 N. Two sequences x y 2 f0 1g are Turing equivalent, and
1

we write x
T y , if x T y and y T x. A Turing degree is an equivalence
class of f0 1g under the equivalence relation
T .
1

The complement of a set X  f0 1g is X c = f0 1g ; X .

1 1

3 Measure and Category

Three dierent senses in which a set X of binary sequences may or may not
be \small" are used in this paper. A set X  f0 1g may have measure
1

0, in which case it is small \in the sense of Lebesgue measure." A set

X  f0 1g may have measure 0 in REC, in which case X \ REC is a small
1

subset of REC, \in the sense of resource-bounded measure." Finally, a set

X  f0 1g may be meager (also known as
rst category), in which case it
1

is small \in the sense of Baire category." This section reviews the basic ideas
from Lebesgue measure, resource-bounded measure, and Baire category that
are involved in our use of these three notions of \smallness." The interested
reader may consult 6, 18, 36, 37, 43, 45] for further discussion of these
notions, but the material in the present section is sucient for following the
arguments of this paper.
Resource-bounded measure 36, 37] is a generalization of classical
Lebesgue measure. As such it has classical Lebesgue measure and measure
in REC as special cases. We use this fact to present the notions \measure
0" and \measure 0 in REC" more or less simultaneously.
Consider the random experiment in which a binary sequence x 2 f0 1g 1

is chosen probabilistically, using an independent toss of a fair coin to decide

each bit of x. Intuitively, a set X  f0 1g has (Lebesgue) measure 0|a
1

condition dened precisely below|if Prx 2 X ] = 0, where Prx 2 X ] is

the probability that x, the outcome of the coin-tossing experiment, is an

7
element of X . In this case, we write (X ) = 0 (\X has measure 0"). We
now develop the necessary denitions.
A string w 2 f0 1g is a pre
x of a string or sequence x 2 f0 1g
 

f0 1g , and we write w v x, if there exists y 2 f0 1g f0 1g such that

1  1

x = wy. The cylinder generated by a string w 2 f0 1g is 

Cw = fx 2 f0 1g j w v xg
1

i.e., the set of all in
nite binary sequences beginning with the string w.
Denition 37]. A density function is a function d : f0 1g ! 0 1) 

satisfying
d(w) = d(w0) +2 d(w1) (3:1)
for all w 2 f0 1g . The global value of a density function d is d(). The set


covered by a density function d is


S d] = Cw : (3:2)
w2f0 1g
d( w )1

An n-dimensional density system (n-DS ) is a function

d : Nn  f0 1g ! 0 1)


such that, for all ~k 2 Nn , the function d~k is a density function. (Recall that
d~k (w) = d(~k w) for all ~k 2 Nn and w 2 f0 1g .) 

Taken together, parts (3.1) and (3.2) of the above denition imply that
Prx 2 S d]]  d()
in our coin-tossing random experiment. We thus intuitively regard d as a
\detailed verication" that Prx 2 X ]  d() for all X  S d]. With this
intuition in mind, we present the central idea of resource-bounded measure
0 sets.
Denition 37]. A null cover of a set X  f0 1g is a 1-DS d that satises
1

the following two conditions for all k 2 N.

(i) X  S dk ].

8
 (ii) dk ()  2 k .
;

Denition 37]. A set X  f0 1g has (Lebesgue) measure 0, and we write

1

(X ) = 0, if it has a null cover. A set X  f0 1g has (Lebesgue) measure

1

1, and we write (X ) = 1, if (X c ) = 0. In this latter case, we say that X

contains almost every sequence x 2 f0 1g . 1

It is a routine exercise to check that this denition is equivalent to \stan-

dard textbook" denitions 6, 18, 43, 45] of measure 0 and measure 1 sets.
The main advantage of the above denition is that it naturally yields
analogous notions of measure in REC and various complexity classes. To
specify the analogous measure in REC, we need to dene the computability
of density systems. Since density systems are real-valued, they must be
computed via approximations. For this purpose, it is natural to use the set
D = fm2 n j m 2 Z n 2 Ng
;

of dyadic rationals. These are real numbers whose standard binary repre-
sentations are nite.
Denition 37]. An n-DS d is computable if there is a total recursive
function db : Nn+1  f0 1g ! D such that, for all ~k 2 Nn , r 2 N, and


w 2 f0 1g ,






db~kr (w) ; d~k (w)

 2 r :
;

Note that the above denition is uniform, in the sense that it requires
a single total recursive function db to compute approximations for all the
density functions d~k (given ~k, a precision parameter r, and the input to d~k
as inputs to db).
Denition 37]. A recursive null cover of a set X  f0 1g is a null cover
1

of X that is computable. A set X  f0 1g has recursive measure 0, and

1

we write rec(X ) = 0, if X has a recursive null cover. A set X  f0 1g 1

has recursive measure 1, and we write rec (X ) = 1, if rec(X c) = 0. A set

X  f0 1g has measure 0 in REC, and we write (X j REC) = 0, if
1

rec(X \ REC) = 0. A set X  f0 1g has measure 1 in REC, and we

1

write (X j REC) = 1, if (X c j REC) = 0. In this latter case, we say that
X contains almost every recursive sequence x 2 REC.
Note that the implications
9
 µrec(X)= 0 µrec (X)= 1
and

µ (X ) = 0 µ (X | REC ) = 0 µ (X ) = 1 µ (X | REC ) = 1

all follow immediately from the above denitions. It is easy to see that
every subset of a recursive measure 0 set has recursive measure 0, that every
nite subset of REC has recursive measure 0, and that every nite union
of recursive measure 0 sets has recursive measure 0. In fact, the recursive
measure 0 sets enjoy a stronger closure property, which we now dene.
Denition 37]. Let Z Z Z : : :  f0 1g . Then Z is a recursive union
1

S
0 1

of the sets Z0 Z1 : : : of measure 0 in REC if Z = Zj and there exists a

1

j =0
computable 2-DS d such that, for all j 2 N, dj is a recursive null cover of
Zj \ REC.
Theorem 3.1 (Lutz 37]). If Z  f0 1g is a recursive union of sets of
1

measure 0 in REC, then Z has measure 0 in REC.

On the other hand, the following result shows that not every set has
measure 0 in REC.
Theorem 3.2 (Lutz 37]). No cylinder Cw has measure 0 in REC. In par-
ticular, REC does not have measure 0 in REC.
Taken together, the above facts justify the intuition that, if X has mea-
sure 0 in REC, then X \ REC is a negligibly small subset of REC. Further
discussion of this intuition may be found in 37, 43].
Other formulations of measure in REC have been investigated by Freid-
zon 14], Mehlhorn 41], and others. The advantage of the formulation here
is that it uniformly yields Lebesgue measure, measure in REC, and measure
in various complexity classes 37]. It is easy to show that, if X has \measure
0 in REC" in the sense of 14], then X has measure 0 in REC in our sense.
We now turn to the fundamentals of Baire category. Baire category gives
a topological notion of smallness, usually dened in terms of \countable
unions of nowhere dense sets" 42, 43, 45]. Here it is more convenient to
dene Baire category in terms of certain two-person, innite games of perfect
information, called Banach-Mazur games.
10
 Informally, a Banach-Mazur game is an innite game in which two play-
ers construct a sequence x 2 f0 1g by taking turns extending a prex of
1

x. There is a \payo set" X  f0 1g such that Player I wins a play of the

1

game if x 2 X and Player II wins otherwise.

More formally, a strategy for a Banach-Mazur game is a function  :
N  f0 1g ! f0 1g with the property that w <= m(w), i.e., w is a proper
 

prex of m (w) for all m 2 N and w 2 f0 1g . A play of a Banach-Mazur

6


game is an ordered pair (  ) of strategies. The result of the play (  )

T
is the unique sequence R(  ) 2 Cwk , where the strings w0 w1 : : : are
1

k=0
dened by the following recursion.
(i) w0 = .
(ii) For all m 2 N, w2m+1 = m (w2m).
(iii) For all m 2 N, w2m+2 = m (w2m+1).
Intuitively, Player I uses strategy  , Player II uses strategy  , and wk is
the prex of R(  ) that has been constructed when the two players have
moved a total of k times. For example, if  and  are dened by
m (w) = w0m+1 m (w) = w1
then
w0 =  w1 = 0 w2 = 01 w3 = 0100 : : :
so
R(  ) = 01001000100001000001    :
We write GX ] for the Banach-Mazur game with payo set X  f0 1g . 1

A winning strategy for Player I in GX ] is a strategy  such that, for all
strategies  , R(  ) 2 X . A winning strategy for Player II in GX ] is a
strategy  such that, for all strategies  , R(  ) 2= X .
Denition. A set X  f0 1g is meager if there exists a winning strat-
1

egy for Player II in the Banach-Mazur game GX ]. A set X  f0 1g is 1

comeager if X c is meager. (A meager set is sometimes called a \set of rst

category.")
As an easy example, let FIN be the set of all characteristic sequences of
nite subsets of N, i.e.,
FIN = fx 2 f0 1g j x has only nitely many 1's g:
1

11
Then the strategy  dened by m (w) = w1 is a winning strategy for Player
II in GFIN], so FIN is meager.
The proof that the above denition is equivalent to the \standard text-
book" denition of the meager sets is due to Banach and may be found in
42] or 43]. It is clear that every subset of a meager set is meager and that
every countable set X  f0 1g is meager. In fact, it is well-known that
1

every countable union of meager sets is meager 43]. On the other hand, for
every w 2 f0 1g , the strategy


(
u <= w
m (u) = wu0 ifotherwise
6

is a winning strategy for Player I in GCw ], so no cylinder is meager. (This

is the Baire Category Theorem 43].) These facts justify the intuition that
meager sets are \topologically small," or (negligibly) small in the sense of
Baire category. Thus, if a set X  f0 1g is comeager, we say that its
1

elements are \topologically abundant," or that X is large in the sense of

Baire category, or that X contains almost every sequence in the sense of
Baire category.
The proofs of our Baire category results, Theorems 5.16 and 6.2 below,
are easy, given some elementary properties of the Cantor topology on the
set f0 1g . For completeness, we review these properties. Further details
1

may be found in a number of texts, e.g., 20, 42].

A set X  f0 1g is open, or
01 , if it can be expressed as a (countable)
1

union of cylinders. A set X  f0 1g is closed, or 01 , if X c is open. For

1

each positive integer k, a set X  f0 1g is

0k+1 if it can be expressed as
1

a countable union of 0k sets. For each positive integer k, a set X  f0 1g 1

is 0k+1 if X c is
0k+1 . (The \boldface" classes
01 01
02 02 : : : are col-
lectively known as the
nite Borel hierarchy. This hierarchy is closely analo-
gous to the \lightface" arithmetical hierarchy 01 01 02 02 : : : of recursion
theory 42].)
A
nite variation of a sequence x 2 f0 1g is a sequence y 2 f0 1g
1 1

such that y n] = xn] for all but nitely many n 2 N. A set X  f0 1g is 1

closed under
nite variations if every nite variation of every element of X

is an element of X .
A function f : f0 1g ! f0 1g is continuous if, for every x 2 f0 1g
1 1 1

and n 2 N, there exists k 2 N such that f (Cx0::k 1])  Cf (x)0::n 1].

; ;

We use the following two facts. For completeness, we sketch proofs.

Further details may be found in standard texts, e.g., 20, 42].

12
Fact 3.3.
1. Let X and Y be disjoint subsets of f0 1g . If X is
02, Y 6= , and
1

Y is closed under nite variations, then X is meager.

2. If X = f0 1g is
02 and closed under nite variations, then X is

6
1

meager.
Proof. To prove part 1, assume the hypothesis and x a sequence z 2 Y .
Since X is
, there exist closed sets X X : : :  f0 1g such that X =
0 1

S X . To see that X is meager, it suces to exhibit a winning strategy

2 0 1
1

k
k=0
for Player II in the Banach-Mazur game GX ]. Player II's strategy uses
z as a source of bits. To specify this strategy, let wk 2 f0 1g be the

string constructed by the game play prior to move k of Player II, where
k 2 N. Let wk ==z be the sequence obtained from z by putting wk in place
of the rst jwk j bits of z . Since z 2 Y and wk ==z is a nite variation of
z , it must be the case that wk ==z 2 Y . In particular, this implies that
wk ==z 2= Xk . Since Xk is closed, it follows that there exists n > jwk j such
that C(wk ==z)0::n 1] \ Xk = . Player II's strategy in move k is to extend
;

wk to (wk ==z)0::n ; 1] for this value of n. The nal sequence x 2 f0 1g 1

constructed by the game play is now guaranteed to satisfy x 2= Xk . Since

Player II eventually establishes this for every k 2 N, it follows that x 2= X .
Hence this is a winning strategy for Player II in GX ], so X is meager.
To prove part 2, take Y = X c in part 1. 2
Fact 3.4. If X  f0 1g is
and f : f0 1g ! f0 1g is continuous,
1 0
2
1 1

then the image f (X ) is also

. 0
2

Proof. Assume the hypothesis. Then there exist closed sets Y Y : : : 

S
f0 1g such that X = Yk . Each Yk is a closed subset of the compact
1
1
0 1

k=0
Hausdor space f0 1g , so each Yk is compact. Since f is continuous, it
1

S
follows that each f (Yk ) is compact, hence closed. Since f (X ) = f (Yk ),
1

k=0
this implies that f (X ) is
02 . 2
We have described three notions of smallness in this section. It should
be noted that no two of them coincide. Although some sets (e.g. nite
sets) are small in all three senses, it is possible for a set to be small in any
one of these senses without being small in the other two. For example, in
section 4 below, we dene the set RAND, consisting of all algorithmically

13
random sequences. Consider also the set REC of all recursive sequences. It
is well-known 39] that REC \ RAND = , that RAND is meager, and that
RAND has measure 1. (See also Theorems 4.7 and 6.2 below.) Also, since
REC is countable, REC is meager and has measure 0. The following three
things follow easily from these observations.
(a) RAND REC is meager, but has measure 1 and measure 1 in REC.
(b) RECc has measure 0 in REC but is comeager and has measure 1.
(c) RANDc has measure 0, but is comeager and has measure 1 in REC.
As Oxtoby 43] has noted, \There is of course nothing paradoxical in the
fact that a set that is small in one sense may be large in some other sense."

4 Algorithmic Information and Randomness

In this section we review some fundamentals of algorithmic information the-
ory that are used in this paper. We are especially concerned with self-
delimiting Kolmogorov complexity and algorithmic randomness. The inter-
ested reader is referred to 33, 35] for more details, discussion, and proofs.
Kolmogorov complexity, also called program-size complexity, was discov-
ered independently by Solomono 51], Kolmogorov 21], and Chaitin 9].
Self-delimiting Kolmogorov complexity is a technical improvement of the
original formulation that was developed independently, in slightly dierent
forms, by Levin 26, 27], Schnorr 47], and Chaitin 11]. The advantage of the
self-delimiting version is that it gives precise characterizations of algorithmic
probability and randomness.
Self-delimiting Kolmogorov complexity employs a slightly restricted
model of (deterministic) Turing machine computation. In this model, a
Turing machine M has a program tape, an output tape, and some number
k of worktapes. (For some purposes it is also advantageous to have a special
input tape, but we do not need one here.) Only 0's, 1's and blanks can ever
appear on a tape. The program tape and the output tape are innite to the
right, while the worktapes are innite in both directions. Each tape has a
scanning head. The program and output tape heads cannot move left, but
the worktape heads can move left or right. The program tape is read-only,
the output tape is write-only, and the worktapes are read/write. The output
tape head can only write 0's and 1's it cannot write blanks.

14
 A Turing machine M starts in the initial state with a program  2 f0 1g 

on its program tape, the output tape blank, and the worktapes blank. The
leftmost cell of the program tape is blank, with the program tape head
initially scanning this cell. The program  lies immediately to the right of
this cell. The rest of the program tape is blank. The output tape head
initially scans the leftmost cell of the output tape.
If, after nitely many steps, M halts with the program tape head scan-
ning the last bit of  , then the computation is deemed to be a success, we
write M ( )#, and the output of the computation is the string M ( ) 2 f0 1g

that has been written on the output tape. Otherwise, the computation is a
failure, we write M ( )", and there is no output (i.e., we disregard the con-
tents of the output tape). If M ( )#, then timeM ( ) denotes the number of
steps executed in this computation. If M ( )", then we write timeM ( ) = 1.
It should be emphasized that a successful computation must end with
the program tape head scanning the last bit of the program. Since the
program tape head is read-only and cannot move left, this implies that, for
every Turing machine M , the set
PROGM = f 2 f0 1g j M ( )#g 

must be an instantaneous code, i.e., must be a set of nonempty strings, no

one of which is a prex of another. (It is this feature of the model that the
adjective \self-delimiting" describes.) It follows by Kraft's inequality (see
13], for example) that, for all Turing machines M ,
X
2   1:
;j j

 2PROGM
It is well-known that there are Turing machines U that are universal, in
the sense that, for every Turing machine M , there exists a program prex
M 2 f0 1g such that, for all  2 f0 1g ,
 

U (M  ) = M ( ):
(This condition means that M ( ) # if and only if U (M  ) #, in which case
U (M ) = M ().) Furthermore, there are universal Turing machines U
that are ecient, in the sense that, for each Turing machine M there is a
constant c 2 N (which depends on M ) such that, for all  2 f0 1g , 

timeU (M )  c(1 + timeM ( ) log timeM ()):

15
Notational Convention. Throughout this paper, U is a xed, ecient,
universal Turing machine.

The set of programs for a string x 2 f0 1g relative to a Turing machine



M is
PROGM (x) = f 2 f0 1g j M ( ) = xg :


Similarly, given a time bound t : N ! N, the set of t-fast programs for x

relative to M is
PROGtM (x) = f 2 PROGM (x) j timeM ( )  t(jxj)g :
(Note that the time bound here is computed in terms of the output length.)
We write PROG, PROG(x), and PROGt (x) for PROGU , PROGU (x), and
PROGtU (x), respectively.
We dene the probability of an instantaneous code I  f0 1g to be


X
Pr(I ) = 2 w : ;j j

w I
2

Intuitively, if we choose a sequence x 2 f0 1g probabilistically, using an

1

independent toss of aSfair coin to decide each bit of x, then Pr(I ) is the
probability that x 2 Cw , i.e., the probability that some element of I is
w I
a prex of x.
2

We now come to the central ideas of algorithmic information theory. (See

33] for a history of the development of these denitions.)
Denition. Let x 2 f0 1g , let t : N ! N be a time bound, and let M be


a Turing machine.
1. The (self-delimiting) Kolmogorov complexity of x relative to M is
n
o
KM (x) = min jj

 2 PROGM (x) :
(Here we use the convention that min = 1.) The (self-delimiting)
Kolmogorov complexity of x is
K (x) = KU (x)
The quantity K (x) is also called the algorithmic entropy, or algorithmic
information content, of x.

16
 2. The t-time-bounded (self-delimiting) Kolmogorov complexity of x rela-
tive to M is
n
o
KMt (x) = min j j

 2 PROGtM (x) :
The t-time-bounded (self-delimiting) Kolmogorov complexity, or t-time-
bounded algorithmic entropy, of x is
K t (x) = KUt (x):
3. The algorithmic probability of x relative to M is
mM (x) = Pr(PROGM (x)):
The algorithmic probability of x is
m(x) = mU (x):
4. The t-time-bounded algorithmic probability of x relative to M is
mtM (x) = Pr(PROGtM (x)):
The t-time-bounded algorithmic probability of x is
mt(x) = mtU (x):
In general, we omit the adjective \self-delimiting", since this is the only
type of Kolmogorov complexity in this paper.
We now present some basic properties of Kolmogorov complexity and
algorithmic probability that are used in this paper. The rst is obvious,
well-known, and useful.
Lemma 4.1. There is a constant c 2 N such that, for all x 2 f0 1g and
0


all  2 PROG(x),
K (x)  K ( ) + c0:
The next two important theorems express the fundamental relationship
between Kolmogorov complexity and algorithmic probability.
Theorem 4.2 (Levin 26, 27], Chaitin 11]). There is a constant ec 2 N such
that, for all x 2 f0 1g ,


; log m(x)  K (x) < ; log m(x) + ce:

17
 A straightforward modication of the proof of Theorem 4.2 yields the
following time-bounded version. (This result also follows immediately from
Lemma 3 of 34].)
Theorem 4.3. Let t : N ! N be recursive.
1. For all x 2 f0 1g ,


; log mt (x)  K t(x):

2. There exist a recursive function t1 : N ! N and a constant c1 2 N
such that, for all x 2 f0 1g , 

K t1 (x) < ; log mt(x) + c1 :

In addition to the above facts, we need the following lemma and corollary,
due to Bennett. For the lemma, say that a string  2 f0 1g computes a 

nite instantaneous code I if U ( ) = x0 : : : xn 1 ] is a binary string that

;

encodes an enumeration of the elements x0 : : : xn 1 of I in some standard

;

fashion.
Lemma 4.4 (Bennett 5]). There is a constant c 2 N such that, for all
0

 2 f0 1g , if  computes a nite instantaneous code I , then for all x 2 I ,



K (x)  jxj + log Pr(I ) + j j + c : 0

(Note that ;jxj  log Pr(I )  0, so that the bound becomes tighter as Pr(I )
becomes smaller.)
Proof. Let M be a Turing machine that performs as indicated in Figure 1
with program  b , where  computes a nite instantaneous code and b 2
f0 1g . (If the program for M is not of this form, then the computation is


a failure.) Since U is a universal Turing machine, there is a program prex

M 2 f0 1g such that, for all  2 f0 1g , U (M  ) = M (). Let
 

c = jM j + 1:
0

To see that c has the desired property, let  2 f0 1g compute a nite

0 

instantaneous code I . If I = , then the lemma is armed vacuously, so

assume that I 6= . Let x0 : : : xn 1 and k0 : : : kn 1 be as in Figure 1.
; ;

Dene real numbers r0 <    < rn by the recursion

r0 = 0 ri+1 = ri + 2 ; ki

18
begin
simulate U ( ) to obtain I (on a worktape) in the form
I = fx0 : : : xn 1g, ;

where x0 : : : xn 1 are in standard order

;

 := 
0

for 0  i < n do
begin
if i = 0 then w := 0ki else w := next(w ki),
where ki = jxi j ; b; log Pr(I )c and next(w ki) is the
immediate lexicographic successor of the string w1ki w 
;j j

while  v w do
0

if  = w then output xi and halt

0

else  :=  b, where b is the

0 0

next bit on the program tape

end
end M (b).

Figure 1: The Turing Machine M used in the proof of Lemma 4.4.

19
and note that
nX1
;
X X
rn = 2 ki = 2
; b; I
log Pr( )c;jx  Pr(I )
j ;1
2 x = 1:
;j j

i=0 x I
2 x I 2

Dene strings b : : : bn 1 2 f0 1g by
;


b0 = 0k0 bi+1 = next(bi ki+1)

where the function next is dened as in Figure 1. A routine induction on i
shows that each bi is the standard ki -bit binary representation of the natural
number ri  2ki . (The key point in the induction step is that, for 0  i < n ; 1,
we have ri + 2 ki = ri+1  rn 1 < rn  1, so ri  2ki < 2ki ; 1. By the
;
;

induction hypothesis, this means that bi does not consist entirely of 1's, so
bi+1 = next(bi ki+1) contains only ki+1 bits.) Moreover, it is easily checked
that, for all 0  i < n, bi is the value assigned to w by M during iteration
i of the for-loop, and that
U (M  bi ) = M (bi ) = xi
whence
K (xi )  jM bi j = ki + j j + c ; 1 0

 jxi j + log Pr(I ) + j j + c : 0

2
Corollary 4.5. For every recursive function t : N ! N there exists a
constant c 2 N such that, for all y 2 f0 1g and all  2 PROGt (y ),
 

K ()  jj + log mt(y ) + K (y ) + c : 

(Note that ;j j  log mt(y )  0, so the bound becomes tighter as the

time-bounded algorithmic probability of y becomes smaller.)
Proof. Let t : N ! N be recursive. Let M be a Turing machine that,
with program  2 f0 1g , does the following. First M simulates U ( ). If


this computation does not succeed, then M ( ) ". Otherwise, if U ( ) = y ,

then M simulates U ( ) for t(jy j) steps for every string  2 f0 1g t( y ),
0 0  j j

and uses the result of this simulation to output an (encoded) enumeration

0 : : : n 1 ] of the nite instantaneous code PROGt (y ).
;

20
 Since U is a universal Turing machine, there is a program prex M 2
f0 1g such that, for all  2 f0 1g , U (M  ) = M ( ). Let
 

c = jM j + c
 0

where c is the constant given by Lemma 4.4. For y 2 f0 1g , let y be a

0 

shortest element of PROG(y ). Then, for all y , the string M y computes

the nite instantaneous code PROGt (y ). It follows by Lemma 4.4 that, for
all y 2 f0 1g and  2 PROGt (y ),


K ()  jj + log Pr(PROGt (y )) + jM y j + c 0

= j j + log mt(y ) + K (y ) + c : 

2
In this paper we are especially interested in the Kolmogorov complexi-
ties of initial segments of innite binary sequences. In this regard, given a
function g : N ! 0 1) and a recursive time bound t : N ! N, we dene
the classes
K : :< g(n)] = fx 2 f0 1g j K (x0::n ; 1]) < g(n) i.o.g
io
1

and
n o
Kt: :< g(n)] = x 2 f0 1g j K t(x0::n ; 1]) < g(n) i.o. :
io
1

Thus we are using g (n) as a \threshold value" for the Kolmogorov complexity
of the n-bit prex of a sequence x 2 f0 1g . These classes contain those
1

sequences for which this Kolmogorov complexity is below the threshold value
for innitely many prexes.
The following theorem, which is used in proving our main result, says
that almost every recursive sequence has very high time-bounded
Kolmogorov complexity almost everywhere.
Theorem 4.6 (Lutz 37]). For every recursive bound t : N ! N and every
real number 0 < < 1,


(Kti:o: < n]

REC) = 0:
(In fact, Corollary 4.9 of 37] is stronger than this in several respects.)

21
 We conclude this section with a brief discussion of the algorithmic ran-
domness of innite binary sequences. Algorithmic randomness was origi-
nally dened by Martin-Lof 39], using constructive versions of ideas from
measure theory. Subsequently, Levin 26, 27], Schnorr 47], and Chaitin
11] showed that algorithmic randomness could be characterized in terms
of self-delimiting Kolmogorov complexity. (Indeed, this was an important
motivation for developing the self-delimiting formulation.) For the purposes
of the present paper, it is convenient to use this characterization as the
denition.
Denition. A sequence x 2 f0 1g is algorithmically random, and we write
1

x 2 RAND, if there is a constant k 2 N such that K (x0::n ; 1]) > n ; k a.e.

That is,
 1

RAND = K : :< n ; k]c:

io
k=0
The following theorem summarizes some elementary properties of RAND
that are used in this paper.
Theorem 4.7 (Martin-Lof 39]). RAND is a
, measure 1 subset of
0
2
f0 1g that is closed under nite variations and does not contain the char-
1

acteristic sequence of any recursively enumerable set.

5 Strong Computational Depth

In this section, we investigate Bennett's notion of strong computational
depth for innite binary sequences. This notion can be dened in several
equivalent ways. We start with the denition most convenient for our pur-
poses. Subsequently, in Theorem 5.4 below, we prove the equivalence of this
denition with others that have appeared in the literature.
Denition. For t g : N ! N and n 2 N, we dene the sets


Dtg (n) = fx 2 f0 1g

(8 2 PROGt (x0::n ; 1]))K ( )  j j ; g (n)g
1

and
 \ t
1 1

Dtg = Dg (n)
m=0 n=m

= fx 2 f0 1g

x 2 Dtg (n) a:e:g:
1

22
A sequence x 2 f0 1g is strongly deep, and we write x 2 strDEEP, if for
1

every recursive time bound t : N ! N and every constant c 2 N, it is the

case that x 2 Dtc .
Intuitively, then, a sequence x 2 f0 1g is in Dtg (n) if every t-fast pro-
1

gram  for x0::n ; 1] can be compressed by at least g (n) bits. Note that, if
t(n)  tb(n) and g (n)  gb(n), then Dbbtg (n)  Dtg (n). Thus, if t(n)  bt(n) a:e:
and g (n)  gb(n) a:e:, then Dbbtg  Dtg . In particular, if g (n) = c and gb(n) = cb
are constant, then we have the situation depicted in Figure 2.

{0,1}∞

t
Dc

t
D
c

strDEEP

Figure 2: The classes Dtc , Dbbtc , in the case where t(n)  bt(n) a.e. and c  cb.

We start by examining the relationship between randomness and strong

depth. We use the following technical lemma.
Lemma 5.1. If x 2 RAND, then there exist a sequence k k : : : of natural
0 1

23
numbers and a sequence 0 1 : : : of programs satisfying the following three
conditions for all i 2 N.
(1) For all n  ki , K (x0::ki ; 1]) ; ki  K (x0::n ; 1]) ; n.
(2) U (i) = x0::ki ; 1] and jij = K (x0::ki ; 1]).
(3) ki+1 > ki + timeU (i ).
Proof. Let x 2 RAND. Dene f : N ! Z by f (n) = K (x0::n ; 1]) ; n. For
each i 2 N, x the least argument ni  i such that f (ni )  f (n) for all n  i.
(Since x 2 RAND, f is bounded below, so ni exists.) Dene the sequences
k0 k1 : : : and 0 1 : : : recursively as follows. Let k0 = n0 and let 0 be a
minimal program for x0::k0 ; 1]. Given ki and i , let ki+1 = nki +timeU (i )+1
and let i+1 be a minimal program for x0::ki+1 ; 1]. It is easily veried
that the sequences k0 k1 : : : and 0 1 : : : satisfy conditions (1), (2), and
(3). 2
Bennett 5] has noted that no algorithmically random sequence is
strongly deep. We now prove this fact. Moreover, we show that it holds
in a very strong way. Intuitively, we show that every algorithmically ran-
dom sequence lies \very near the top" of the diagram in Figure 2.
Theorem 5.2 (Bennett 5]). RAND \ strDEEP = . In fact, there exist
a recursive function t(n) = O(n log n) and a constant c 2 N such that
RAND \ Dtc = .
Proof. Let M be a Turing machine that, with program y, does the follow-
ing. The machine M simulates U ( ), recording timeU ( ) while doing so.
If the simulated computation succeeds, M then reads and outputs the rst
timeU () bits of y (appended to the string U ( ) already produced as out-
put) and halts. Note that if jy j = timeU ( ), then the computation of M (y )
succeeds, with M (y ) = U ( )y . Otherwise, the computation of M (y ) is a
failure.
On successful computations, the Turing machine M takes O(jy j) steps
to produce U ( )y . Thus there exist a program prex M and a recursive,
nondecreasing time bound t(n) = O(n log n) such that, for all successful
computations U ( ) and all strings y with jy j = timeU ( ), the following two
conditions hold.
(i) U (M y ) = U ( )y .
24
 (ii) timeU (M y )  t(jy j).
Let c = jM j + c0, where c0 is the constant from Lemma 4.1. We prove that
RAND \ Dtc = .
Let x 2 RAND. Fix sequences k0 k1 : : : and 0 1 : : : as in Lemma 5.1.
For each i 2 N, let ni = ki + timeU (i ). Note that the sequence n0 n1 : : :
is strictly increasing. We prove that x 62 Dtc by showing that, for all i 2 N,
x 62 Dtc(ni).
Conditions (i) and (ii) above imply that the following conditions hold
for all i 2 N.
(iii) U (M i xki::ni ; 1]) = x0::ni ; 1].
(iv) timeU (M i xki::ni ; 1])  t(ni ; ki )  t(ni ).
Then, for all i 2 N
M ixki::ni ; 1] 2 PROGt (x0::ni ; 1])
and Lemma 5.1 tells us that
K (x0::ki ; 1])  K (x0::ni ; 1]) ; ni + ki
= K (x0::ni ; 1]) ; timeU (i )
whence
K (M i xki ::ni ; 1])  K (x0::ni ; 1]) ; c0
 K (x0::ki ; 1]) + timeU (i ) ; c0
= ji j + ni ; ki ; c0
= ji xki::ni ; 1]j ; c0
= jM i xki::ni ; 1]j ; c:
Thus x 62 Dtc (ni ) for all i 2 N, so x 62 Dtc . 2
We next show that strong computational depth can be characterized in
several equivalent ways. For this, we need some notation and a lemma. We
rst recall Bennett's denition of the computational depth of nite strings.
Denition 5]. Let w 2 f0 1g and c 2 N. Then the computational depth


of w at signi
cance level c is
depthc (w) = minft 2 N j (9 2 PROGt (w)) j j < K ( ) + cg:
25
 That is, the depth of a nite string at signicance level c is the minimum
time required to compute w from a program that is not compressible by c
or more bits.
Our alternate characterizations of strong depth also use the following
classes.
Denition. For t g : N ! N and n 2 N, we dene the sets

Db tg (n) = fx 2 f0 1g j K (x0::n ; 1])  K t(x0::n ; 1]) ; g (n)g

1

De tg (n) = fx 2 f0 1g j m(x0::n ; 1])  2g(n) mt(x0::n ; 1])g

1

 \
Db tg = Db tg (n)
1 1

m=0 n=m
 \ et
De tg =
1 1

Dg (n):
m=0 n=m
The following lemma shows that the classes Db tg and De tg are, in a quanti-
tative sense, \minor variants" of the classes Dtg . This result was proven in
a slightly dierent form in 5].
Lemma 5.3 (Bennett 5]). If t : N ! N is recursive, then there exist
constants c c c 2 N and a recursive function t : N ! N such that the
0 1 2 1
following six conditions hold for all g : N ! N and all n 2 N.
1. Dtg+c0 (n)  Db tg (n) 4. Dtg+c0  Db tg
2. Db tg1+c1 (n)  De tg (n) 5. Db tg1+c1  De tg
3. De tg+c2 (n)  Dtg (n) 6. De tg+c2  Dtg
Proof. It suces to prove 1, 2, and 3, since 4, 5, and 6 then follow imme-
diately.
1. Let c0 be as in Lemma 4.1 and assume that x 2 Dtg+c0 (n). Let 
be a shortest element of PROGt (x0::n ; 1]). Since x 2 Dtg+c0 (n), we have
K ()  jj ; g(n) ; c0 . It follows that
K (x0::n ; 1])  K () + c0
 j j ; g (n)
= K t (x0::n ; 1]) ; g (n)

26
whence x 2 Db tg (n).
2. Choose c1 and t1 for t as in Theorem 4.3 and assume that x 2
bDtg1+c1 (n). Then K (x0::n ; 1])  K t1 (x0::n ; 1]) ; g (n) ; c1. It follows by
Theorems 4.2 and 4.3 that
m(x0::n ; 1])  2 K (x0::n 1])
; ;

t
 2g(n)+c1 K 1 (x0::n 1])
; ;

> 2g(n)mt (x0::n ; 1])

whence x 2 De tg (n).
3. Let ce be as in Theorem 4.2, choose c for t as in Corollary 4.5, let


c2 = ce + c , and assume that x 2 De tg+c2 (n). Then



K (x0::n ; 1])  ; log m(x0::n ; 1]) + ce

 ; log mt(x0::n ; 1]) ; g (n) ; c2 + ec
= ; log mt(x0::n ; 1]) ; g (n) ; c : 

Thus, for all  2 PROGt (x0::n ; 1]),

K ()  jj + K (x0::n ; 1]) + log mt(x0::n ; 1]) + c 

 j j ; g (n)
whence x 2 Dtg (n). 2
We now prove the equivalence of several characterizations of strong com-
putational depth.
Theorem 5.4 (Bennett 5]). For x 2 f0 1g , the following four conditions
1

are equivalent.
(1) x is strongly deep.
(2) For every recursive time bound t : N ! N and every constant c 2 N,
depthc (x0::n ; 1]) > t(n) a:e:
(3) For every recursive time bound t : N ! N and every constant c 2 N,
x 2 Db tc.
(4) For every recursive time bound t : N ! N and every constant c 2 N,
x 2 De tc.

27
Proof. The equivalence of (1) and (2) follows immediately from the deni-
tions. The equivalence of (1), (3), and (4) follows immediately from Lemma
5.3. 2
In 5], Bennett uses condition (2) of Theorem 5.4 above as the denition
of strong computational depth. As noted above, this is trivially equivalent
to condition (1), i.e., to our denition in terms of the classes Dtc . Bennett
5] also considers denitions in terms similar to those used in dening the
classes Db tc and De tc and implicitly proves the equivalence of conditions (1),
(3), and (4). The discussions of depth by Li and Vitanyi in the Handbook of
Theoretical Computer Science 33] and their recent book 35] essentially use
condition (4) as the denition. In any case, a sequence x is strongly deep if,
for every recursive t and constant c, almost every prex x0::n ; 1] is \more
than t deep at signicance level c," in the sense that more than t(n) time is
required to derive x0::n ; 1] from any description whose length is within c
bits of the minimum possible length.
We next prove a technical lemma on the quantitative relationship be-
tween computational depth and time-bounded Turing reducibility. This can
be regarded as a quantitative, innitary version of Bennett's deterministic
slow-growth law 5]. We need two special notations for this lemma. First,
for any function s : N ! N, we dene the function s : N ! N by


s (n) = 2s( log n )+1:

 d e

Second, for any unbounded, nondecreasing function f : N ! N, we dene

the special-purpose \inverse" function f 1 : N ! N by
;

f 1 (n) = maxfm j f (m) < ng:

;

Also, for this lemma, say that a function s : N ! N is time-constructible if

there exist a constant cs 2 N and a Turing machine that, given the stan-
dard binary representation w of a natural number n, computes the standard
binary representation of s(n) in at most cs s(jwj) steps. Using standard tech-
niques 2, 19], it is easy to show that, for every recursive function r : N ! N,
there is a strictly increasing, time-constructible function s : N ! N such
that, for all n 2 N, r(n)  s(n).
Lemma 5.5. Let s : N ! N be strictly increasing and time-constructible,
with the constant cs 2 N as witness. For each s-time-bounded oracle Turing
machine M , there is a constant cM 2 N with the following property. Given
28
nondecreasing functions t g : N ! N, dene the functions  bt gb : N ! N
by
 (n) = t(s (n + 1)) + 4s (n + 1) + 2(n + 1)css(l) + 2ns (n + 1)s(l)
  

bt = cM (1 +  (n)dlog  (n)e)
gb = g(s (n + 1)) + cM


where l is the number of bits in the binary representation of n. For all

x y 2 f0 1g , if y DTIME( s) x via M and y 2 Dbt , then x 2 Dt .
bg g
1
T

Proof. Let s and M be as in the statement of the lemma. Let M be 0

a Turing machine that, with program  2 f0 1g , operates as in Figure 3.



Since U is an ecient universal Turing machine, there exist a program prex

M 0 2 f0 1g and a constant cM 0 2 N such that, for all  2 f0 1g ,
 

U (M 0  ) = M ( ) 0

and
timeU (M 0 )  cM 0 (1 + timeM 0 () log timeM 0 ()):
Let M be a Turing machine that, with program  2 f0 1g , simulates
00  

U ( ) and outputs  if and only if U ( ) = M 0  . Since U is universal,

 

there is a program prex M 00 2 f0 1g such that, for all  2 f0 1g ,

  

U (M 00  ) = M ( ). Let
 00 

cM = max fcM 0 jM 0 j + jM 00 jg :
Fix m0 2 N such that (s ) 1(m) > 0 for all m  m0 .
 ;

Now dene  , bt, and gb as in the statement of the lemma and assume that
x y 2 f0 1g satisfy y DTIME( s) x via M and y 2 Dbt . Fix n 2 N such
bg
1
T 0
b
that y 2 Dbtg (n) for all n  n0 and let
m1 = max fm0 s (n0) + 1g :


The following two claims are veried at the end of this proof.
Claim 1. For all m  m and  2 f0 1g , if  2 PROGt(x0::m ; 1]),
0


then M 0  2 PROGbt (y 0::n ; 1]), where n = (s ) 1 (m).  ;

Claim 2. For all m  m and all  2 PROGt(x0::m ; 1]),

1

K ()  j j ; gb(n) + cM
29
begin
u := U ()
n := (s ) 1(juj)
 ;

for 0  i < n do
append the bit M u01 (i) to the output
halt
end M ().
0

Figure 3: The Turing machine M used in the proof of Lemma 5.5.

0

where n = (s ) 1 (m).
 ;

To nish proving the lemma, let m  m1 and let  2 PROGt(x0::m;1]).

Then, by Claim 2 and the monotonicity of g ,
K ()  jj ; gb((s ) 1 (m)) + cM
 ;

= j j ; g (s ((s ) 1 (m) + 1))

  ;

 j j ; g (m):
Thus x 2 Dtg (m). Since this holds for all m  m1 , it follows that x 2 Dtg ,
arming the lemma. All that remains, then, is to prove the two claims.
To prove Claim 1, assume that m  m0 and  2 PROGt (x0::m;1]). Let
u = x0::m ; 1] and n = (s ) 1(m). Since m  m0, we
 ;
must have s (n) < m.


Since M is s-time-bounded, this implies that M (i) = M x (i) = y i] for

u01

all 0  i < n. (All queries in these computations must be made to bits xj ]
for j < juj.) Thus
U (M 0  ) = M () = y 0::n ; 1]:
0

With program  , M requires at most t(m) steps to compute u, at most

0

4m additional steps to compute juj in binary, at most 2(n + 1)css(l) steps

to compute n, and at most 2nms(l) steps to execute the for-loop. Since
s (n + 1)  m, and t is nondecreasing, it follows that timeM 0 ()   (n), so


timeU (M 0 )  bt(n):
Thus M 0  2 PROGbt (y 0::n ; 1]). This proves Claim 1.

30
 Finally, to prove Claim 2, let m  m1, let  2 PROGt (x0::m ; 1]),
and let n = (s ) 1 (m). Since m > s (n0 ), it must be the case that n =
 ; 

(s ) 1 (m)  n0 , whence y 2 Dbbtg (n). Since m  m0 , Claim 1 tells us that

 ;

M 0  2 PROGbt(y0::n ; 1]). Since y 2 Dbbtg , it follows that

K (M 0  )  jM 0 j ; gb(n) = jj ; gb(n) + jM 0 j:
Now let  be a shortest element of PROG(M 0  ). Then U ( ) = M 0  , so
 

U (M 00  ) = M ( ) = 
 00 

so
K ()  jM 00  j 

= K (M 0  ) + jM 00 j
 j j ; gb(n) + cM :
This proves Claim 2 and completes the proof of Lemma 5.5
2
Using Lemma 5.5, we prove that a strongly deep sequence cannot be
truth-table reducible (equivalently, reducible in recursively bounded time)
to a sequence that is not also strongly deep. This implies the fact, noted by
Bennett 5], that strong depth is invariant under truth-table equivalence.
Theorem 5.6. Let x y 2 f0 1g . If y  x and y is strongly deep, then x
1
tt
is strongly deep.
Proof. Assume the hypothesis. To see that x is strongly deep, x a recursive
function t : N ! N and a constant c 2 N. It suces to prove that x 2 Dtc .
Since y tt x, there exist a strictly increasing time-constructible function
s : N ! N and an s-time-bounded oracle Turing machine M such that
y DTIME( s) x via M . Choose a constant c for M as in Lemma 5.5 and
M
dene g : N ! N by g (n) = c for all n 2 N. Then, in the notation of
T

Lemma 5.5, tb is recursive and gb is constant. Since y is strongly deep, it

follows that y 2 Dbbtg . It follows by Lemma 5.5 that x 2 Dtc . 2
We now note that no recursive sequence is strongly deep.
Corollary 5.7 (Bennett 5]). REC \ strDEEP = .
31
Proof. Let x 2 REC it suces to show that x 62 strDEEP. Fix z 2 RAND.
Then, trivially, x tt z . By Theorem 5.2, z 62 strDEEP, so by Theorem 5.6,
x 62 strDEEP. 2
Up to this point, this section has largely followed the line of Bennett's
work. We now build on this work to prove some new results. Our rst such
result says, roughly, that every recursive sequence is either somewhat deep
or somewhat compressible. It is convenient to use the classes Db tg for this
result.
Theorem 5.8. If t : N ! N is recursive and 0 < <  < 1, then
REC  Db tn Kt: :< n]:io

Proof. Assume the hypothesis and let

x 2 REC ; Kt: :< n]: io

It suces to prove that x 2 Db tn .

Since x 62 Kti:o:< n], we have
K t (x0::n ; 1])  n a:e:
Since x is recursive, it follows that there is a constant c 2 N such that, for
all suciently large n,
K (x0::n ; 1]) < 2 log n + c
< n ; n
 K t(x0::n ; 1]) ; n
whence x 2 Db tn . 2
Corollary 5.9. For every recursive function t : N ! N and every 0 <  < 1,
the set Dtn has measure 1 in REC.
Proof. Let t : N ! N be recursive and let 0 <  < <  < 1. Choose a
recursive function t : N ! N and constants c c 2 N for t as in Lemma
1 1 2
5.3, so that
Db tn1 +c2 +c1 (n)  De tn+c2 (n)  Dtn (n)
for all n 2 N. For all suciently large n,
Db tn1 (n)  Db tn1 +c2 +c1 (n)
32
so it follows that Db tn1  Dtn .
By Theorem 4.6, Kbti:o:< n] has measure 0 in REC. By Theorem 5.8,
this implies that Db tn1 has measure 1 in REC. Since Db tn1  Dtn , it follows
that Dtn has measure 1 in REC. 2
Corollary 5.10. For every recursive function t : N ! N and every constant
c 2 N, Dtc has measure 1 in REC.
It is instructive to compare RAND with REC in light of Theorem 5.2,
Corollary 5.7, and Corollary 5.10. Neither RAND nor REC contains a
strongly deep sequence. However, referring to Figure 2, Corollary 5.10 says
that REC \reaches arbitrarily close to" strDEEP, in the sense that each
class Dtc (for t recursive and c constant) contains almost every sequence in
REC. In contrast, if t and c are xed as in Theorem 5.2, then every element
of RAND lies above (i.e. outside of) Dtc in Figure 2. In this sense, intuitively,
REC is much deeper than RAND.
We have now developed enough machinery to examine the computational
depth of computationally useful sequences. We use the following denition.
Denition. A sequence x 2 f0 1g is weakly useful if there is a recursive
1

time bound s : N ! N such that DTIMEx (s) does not have measure 0 in
REC.
That is, x is weakly useful if it can be used to \eciently" (i.e., in some
recursive time s) solve all the problems in a non-negligible subset of REC.
If x 2 REC, then for every recursive time bound s, there is a recur-
sive time bound t such that DTIMEx (s)  DTIME(t): Since every such
set DTIME(t) has measure 0 in REC by Theorem 4.6, this shows that no
recursive sequence is weakly useful.
The following result, which is the main theorem of this paper, shows
that much more is true.
Theorem 5.11. Every weakly useful sequence is strongly deep.
Proof. Let x 2 f0 1g be weakly useful. To see that x is strongly deep,
1

let t : N ! N be a recursive time bound, and let c 2 N. It suces to prove

that x 2 Dtc .
Since x is weakly useful, there is a recursive time bound s : N ! N such
that DTIMEx (s) does not have measure 0 in REC. Since every recursive

33
function is bounded above by a strictly increasing, time-constructible func-
tion, we can assume without loss of generality that s is strictly increasing
and time-constructible.
Let et(n) = n  (1 +  (n)dlog  (n)e), where  is dened from t and s as
in Lemma 5.5, and let  = 12 . Since te is recursive, Corollary 5.9 tells us
that Detn has measure 1 in REC. Since DTIMEx (s) does not have measure
0 in REC, it follows that Detn \ DTIMEx (s) 6= . Fix a sequence y 2
Detn \ DTIMEx (s). Then there is an s-time-bounded oracle Turing machine
M such that y DTIME( s) x. Fix a constant c for M as in Lemma 5.5.
M
Dene g (n) = c for all n 2 N and dene the functions  bt, and gb from t and
T

g as in Lemma 5.5. Since gb and cM are constant, we have te(n) > tb(n) a.e.
and n > gb(n) a.e., so y 2 Detn  Dbbtg . It follows by Lemma 5.5 that x 2 Dtc .
2
Notation. Let
H and
K be the characteristic sequences of the halting
problem and the diagonal halting problem, respectively. That is, the se-
quences
H
K 2 f0 1g are dened by
1

H hi ni] = 1 , Mi(n) halts

K n] = 1 , Mn (n) halts
where M0 M1 : : : is a standard enumeration of all deterministic
; Turing
 ma-
chines and h i is a standard pairing function, e.g., hi ni = i+n2+1 + n:
Corollary 5.12 (Bennett 5]). The sequences
H and
K are strongly deep.
Proof. It is well-known that H and K are polynomial-time complete for
the set of all recursively enumerable subsets of N, so
H and
K are weakly
useful. Thus
H and
K are strongly deep by Theorem 5.11. 2
Note that Theorems 5.2 and 5.11 also provide a new proof of the fact,
noted in the introduction, that no algorithmically random sequence is weakly
useful.
To see that Theorem 5.11 is actually stronger than Corollary 5.12, we
use two known facts concerning high Turing degrees. We rst review the
relevant denitions. (More detailed discussion can be found in a standard
recursion theory text, e.g. 50].)
Recall from section 2 that the characteristic sequence of a set A  N is
the sequence
A 2 f0 1g such that A = fn 2 N j
A n] = 1g. A sequence
1

34
x 2 f0 1g is recursively enumerable (r.e.) if x =
A for some r.e. set
1

A  N. The diagonal halting problem relative to a sequence x 2 f0 1g is 1

the set

K x = fn 2 N

M x (n) haltsg
n
where Mn is the nth oracle Turing machine in a standard enumeration. The
jump of a sequence x 2 f0 1g is the sequence
1

jump(x) =
Kx :
A sequence x 2 f0 1g is high if x T
K and jump(x)
T jump(
K ). A
1

Turing degree is high if it contains a high sequence. It is clear that
K and

its Turing degree are high.
A set X  f0 1g is uniformly recursive in a sequence x 2 f0 1g if
1 1

there is a sequence y 2 f0 1g with the following two properties.

1

(i) y T x.
(ii) X  fyk j k 2 Ng, where each yk 2 f0 1g is dened by yk n] =
1

yhk ni] for all; n 2 N  . (Here we are using the standard pairing func-
k +n+1
tion hk ni = 2 + n:)
We use the following two known facts.
Theorem 5.13 (Sacks 46]). There exist r.e. sequences that are high and
not Turing equivalent to
K .
Theorem 5.14 (Martin 38]). A sequence y 2 f0 1g satises jump(
K ) 
1
T
jump(y) if and only if there exists x
T y such that REC is uniformly re-
cursive in x.
Corollary 5.15. Every high Turing degree contains a strongly deep se-
quence.
Proof. The key observation, pointed out to the third author by Stuart
Kurtz, is that every high Turing degree contains a weakly useful sequence.
To see this, let a be a high Turing degree. By Theorem 5.14, there is a
sequence x 2 a such that REC is uniformly recursive in x. Then there is a
sequence y T x such that REC  fyk j k 2 Ng. Dene z 2 f0 1g by 1

( k
z k] = yx k2 ] 1 ] ifif kk isis even
;
odd
2

35
Then z
T x, so z 2 a. Also, there is a constant c 2 N such that
REC  fyk j k 2 Ng  DTIMEz (cn2 + c)
so z is weakly useful. This conrms that every high Turing degree contains a
weakly useful sequence. By Theorem 5.11, the corollary follows immediately.
2
Taken together, Theorem 5.13 and Corollary 5.15 show that Theorem
5.11 does indeed strengthen Bennett's result, Corollary 5.12.
We conclude this section by proving that strongly deep sequences are
extremely rare, both in the sense of Lebesgue measure and in the sense of
Baire category.
Theorem 5.16. The set strDEEP is meager and has measure 0. In fact, if
t and c are as in Theorem 5.2, then Dtc is meager and has measure 0.
Proof. Let t and c be as in Theorem 5.2. Then RAND \ Dtc = . Since
RAND has measure 1, it follows that Dtc has measure 0.
For each n 2 N, the complement of Dtc (n) can be written as a (nite)
union of cylinders Cw , with each jwj = n. (This is because membership or
nonmembership of a sequence x in Dtc (n) depends only upon x0::n ; 1].)
Thus, for each n 2 N, the set Dtc (n) is closed. It follows that, for each
m 2 N, the set T Dtc (n) is closed, whence the set Dtc = S T Dtc(n)
1 1 1

n=m m=0 n=m

is
02 . By Theorems 4.7 and 5.2, RAND is nonempty, closed under nite
variations, and disjoint from Dtc . It follows by Fact 3.3 that Dtc is meager.
2
If we combine the proofs of Fact 3.3 and Theorem 5.16 to form a direct
proof of Theorem 5.16, then Player II's strategy in this proof is to play an
appropriate number of \random bits" (bits from a sequence z 2 RAND)
during each turn. Intuitively, it is only the \shallowness" of these random
bits that is relevant to the argument. For example, let FIN be the set of all
characteristic sequences of
nite subsets of N, i.e.,
FIN = fx 2 f0 1g j xn] = 0 a:e:g
1

If t and c are as in Theorem 5.2, then it is not dicult to show that FIN \
Dtc = . It follows that Player II could use the sequence 0 in place of z in
1

the above strategy. That is, Player II could win by playing an appropriate
number of 0's, instead of random bits, during each turn.
36
6 Weak Computational Depth
In Theorem 5.16, we saw that strongly deep sequences are very rare, both
in the sense of Lebesgue measure and in the sense of Baire category. In
this brief section, we show that the situation is dierent for weakly deep
sequences. We rst recall the denition.
Denition (Bennett 5]). A sequence x 2 f0 1g is weakly deep, and we
1

write x 2 wkDEEP, if there is no sequence z 2 RAND such that x tt z .

We use the notation
RECtt (RAND) = fx 2 f0 1g j (9z 2 RAND)x tt z g:
1

We thus have
wkDEEP = RECtt(RAND)c :
Since REC RAND  RECtt(RAND), it follows immediately that
wkDEEP \ REC = wkDEEP \ RAND =
i.e., that no weakly deep sequence can be recursive or algorithmically ran-
dom.
As the terminology suggests, every strongly deep sequence is weakly
deep.
Theorem 6.1 (Bennett 5]). strDEEP  wkDEEP.
Proof. Assume that x 2 strDEEP and x  y. To see that x 2 wkDEEP,
tt
it suces to show that y 62 RAND. But this follows immediately from
Theorems 5.2 and 5.6. 2
In particular, Theorems 5.11 and 6.1 imply that weakly deep sequences
exist. It should be noted that Gacs 16] has proven that, for every sequence
x 2 f0 1g , there exists a sequence z 2 RAND such that x T z . Thus
1

T -reducibility cannot be used in place of tt-reducibility in the denition

of wkDEEP.
We have already noted that wkDEEP \ RAND = . Since RAND has
Lebesgue measure 1, it follows that wkDEEP, like strDEEP, has Lebesgue
measure 0. The situation for Baire category is quite dierent. While
strDEEP is meager by Theorem 5.16, wkDEEP is comeager by the following
result.
37
Theorem 6.2. The set wkDEEP is comeager.
Proof. Each  -reduction can be interpreted as a continuous function f :
tt
f0 1g ! f0 1g . (The condition y = f (x) means that y tt x via the
1 1

tt -reduction f .) If we let F be the set of all tt-reductions, then F is

countable and 
RECtt (RAND) = f (RAND):
f 2F

We noted in section 4 that RAND is

02 . It follows by Fact 3.4 that
f (RAND) is
02 for every f 2 F . Since f is countable, this implies that
RECtt (RAND) is
02 .
It is clear that RECtt (RAND) is closed under nite variations. Also,
by Corollary 5.12 and Theorem 6.1, RECtt (RAND) = f0 1g . Thus, by
 1

Fact 3.3, RECtt(RAND) is meager, whence wkDEEP = RECtt (RAND)c is

6

comeager. 2
Bennett 5] noted that there exist sequences that are weakly deep, but
not strongly deep. The following corollary shows that such sequences are,
in the sense of Baire category, commonplace.
Corollary 6.3. The set wkDEEP ; strDEEP is comeager.
Proof. This follows immediately from Theorems 5.16 and 6.2. 2
Thus, in the sense of Baire category, almost every sequence x 2 f0 1g1

is weakly deep, but not strongly deep.

Corollary 6.4 (Bennett 5]). strDEEP wkDEEP.

6=

Proof. This follows immediately from Theorem 6.1 and Corollary 6.2. 2
Figure 4 summarizes the relationships among REC, RAND, wkDEEP,
and strDEEP. In the sense of Lebesgue measure, almost every binary se-
quence is in RAND. On the other hand, in the sense of Baire category,
almost every binary sequence is in wkDEEP ; strDEEP.

38
 RAND

REC tt (RAND)

REC wkDEEP

strDEEP

Figure 4: A classication of binary sequences. RAND has measure 1, while

wkDEEP ; strDEEP is comeager.

39
7 Conclusion
We have shown that every weakly useful sequence is strongly deep. This
result generalizes Bennett's observation that
K is strongly deep, and gives
support to Bennett's thesis that the computational usefulness of
K is re-
lated to its computational depth. We mention two open questions that are
suggested by this result.
Recall that a sequence x 2 f0 1g is weakly useful if there is a recursive
1

time bound s : N ! N such that DTIMEx (s) does not have measure 0
in REC. Dene a sequence x 2 f0 1g to be strongly useful if there is a
1

recursive time bound s : N ! N such that REC  DTIMEx (s). Clearly,

every strongly useful sequence is weakly useful.
Question 7.1. Do there exist sequences that are weakly useful, but not
strongly useful? (We conjecture in the armative.)
Our main result implies that every high Turing degree contains a strongly
deep sequence. A well-known generalization of high sequences and degrees
denes a sequence x 2 f0 1g to be highn (n 2 N) if x T
K and
1

jump(n)(x)
T jump(n) (
K ), where jump(n) is the n-fold iteration of the
jump operation. A Turing degree a is then highn if it contains a highn
sequence. (See 50], for example.) If a sequence or degree is highn , then it
is clearly highn+1 . The Turing degree of
K is clearly the only high0 degree.
It is also clear that a sequence or degree is high1 if and only if it is high.
Thus, by Corollary 5.15, every high1 Turing degree contains a strongly deep
sequence.
Question 7.2. For n > 1, is it necessarily the case that every highn Turing
degree contains a strongly deep sequence?
Answers to Question 7.1 and 7.2 may well improve our understanding
of computational depth vis-a-vis computational usefulness. More generally,
further investigation of Bennett's fundamental notions may yield profound
insights into the role of depth in the organization of computational, physical,
and biological information.

Acknowledgments
The third author thanks Charles Bennett for several helpful discussions,
and Stuart Kurtz for pointing out Theorem 5.14. We also thank Ron Book,
40
Josep D"az, and two anonymous referees for suggestions that have improved
the exposition of this paper.

References
1] L. Adleman. Time, space, and randomness. Technical Report
MIT/LCS/79/TM-131, Massachusettes Institute of Technology, Lab-
oratory for Computer Science, March 1979.
2] J. L. Balcazar, J. D"az, and J. Gabarro. Structural Complexity I.
Springer-Verlag, 1988.
3] Y. M. Barzdin . Complexity of programs to determine whether natural
0

numbers not greater than n belong to a recursively enumerable set.

Soviet Mathematics Doklady, 9:1251{1254, 1968.
4] C. H. Bennett. Dissipation, information, computational complexity and
the denition of organization. In D. Pines, editor, Emerging Syntheses
in Science, Proceedings of the Founding Workshops of the Santa Fe
Institute, pages 297{313, 1985.
5] C. H. Bennett. Logical depth and physical complexity. In R. Herken,
editor, The Universal Turing Machine: A Half-Century Survey, pages
227{257. Oxford University Press, 1988.
6] P. Billingsley. Probability and Measure, second edition. John Wiley and
Sons, 1986.
7] R. V. Book. On languages reducible to algorithmically random lan-
guages. SIAM Journal on Computing, 1993. to appear.
8] R. V. Book, J. H. Lutz, and K. W. Wagner. An observation on proba-
bility versus randomness with applications to complexity classes. Math-
ematical Systems Theory. to appear.
9] G. J. Chaitin. On the length of programs for computing nite bi-
nary sequences. Journal of the Association for Computing Machinery,
13:547{569, 1966.
10] G. J. Chaitin. On the length of programs for computing nite binary
sequences: statistical considerations. Journal of the ACM, 16:145{159,
1969.
41
11] G. J. Chaitin. A theory of program size formally identical to information
theory. Journal of the Association for Computing Machinery, 22:329{
340, 1975.
12] G. J. Chaitin. Incompleteness theorems for random reals. Advances in
Applied Mathematics, 8:119{146, 1987.
13] Thomas M. Cover and Joy A. Thomas. Elements of Information The-
ory. John Wiley & Sons, Inc., 1991.
14] R. I. Freidzon. Families of recursive predicates of measure zero. trans-
lated in Journal of Soviet Mathematics, 6(1976):449{455, 1972.
15] P. Gacs. On the symmetry of algorithmic information. Soviet Mathe-
matics Doklady, 15:1477, 1974.
16] P. Gacs. Every sequence is reducible to a random one. Information and
Control, 70:186{192, 1986.
17] J. Gill. Computational complexity of probabilistic Turing machines.
SIAM Journal on Computing, 6:675{695, 1977.
18] P. R. Halmos. Measure Theory. Springer-Verlag, 1950.
19] J. E. Hopcroft and J. D. Ullman. Introduction to Automata Theory,
Languages, and Computation. Addison-Wesley, 1979.
20] J. L. Kelley. General Topology. Van Nostrand, 1955.
21] A. N. Kolmogorov. Three approaches to the quantitative denition of
`information'. Problems of Information Transmission, 1:1{7, 1965.
22] A. N. Kolmogorov. Logical basis for information theory and probabil-
ity theory. IEEE Transactions on Information Theory, IT-14:662{664,
1968.
23] A. N. Kolmogorov and V. A. Uspenskii. Algorithms and randomness.
translated in Theory of Probability and its Applications, 32:389{412,
1987.
24] M. Koppel. Complexity, depth, and sophistication. Complex Systems,
1:1087{1091, 1987.

42
25] M. Koppel. Structure. In R. Herken, editor, The Universal Turing
Machine: A Half-Century Survey, pages 435{452. Oxford University
Press, 1988.
26] L. A. Levin. On the notion of a random sequence. Soviet Mathematics
Doklady, 14:1413{1416, 1973.
27] L. A. Levin. Laws of information conservation (nongrowth) and as-
pects of the foundation of probability theory. Problems of Information
Transmission, 10:206{210, 1974.
28] L. A. Levin. On the principle of conservation of information in intu-
itionistic mathematics. Soviet Mathematics Doklady, 17:601{605, 1976.
29] L. A. Levin. Uniform tests of randomness. Soviet Mathematics Doklady,
pages 337{340, 1976.
30] L. A. Levin. Various measures of complexity for nite objects (ax-
iomatic description). Soviet Mathematics Doklady, 17:522{526, 1976.
31] L. A. Levin. Randomness conservation inequalities information and in-
dependence in mathematical theories. Information and Control, 61:15{
37, 1984.
32] L. A. Levin and V. V. V'jugin. Invariant properties of informational
bulks. Proceedings of the Sixth Symposium on Mathematical Founda-
tions of Computer Science, pages 359{364, 1977.
33] M. Li and P. M. B. Vitanyi. Kolmogorov complexity and its applica-
tions. In J. van Leeuwen, editor, Handbook of Theoretical Computer
Science, Volume A, pages 187{254. Elsevier, 1990.
34] M. Li and P. M. B. Vitanyi. Learning simple concepts under simple
distributions. SIAM Journal on Computing, 20:911{935, 1991.
35] M. Li and P. M. B. Vitanyi. An Introduction to Kolmogorov Complexity
and its Applications. Springer, 1993.
36] J. H. Lutz. Resource-bounded measure. in preparation.
37] J. H. Lutz. Almost everywhere high nonuniform complexity. Journal
of Computer and System Sciences, 44:220{258, 1992.

43
38] D. A. Martin. Classes of recursively enumerable sets and degrees of
unsolvability. Z. Math. Logik Grundlag. Math., 12:295{310, 1966.
39] P. Martin-Lof. On the denition of random sequences. Information and
Control, 9:602{619, 1966.
40] P. Martin-Lof. Complexity oscillations in innite binary sequences.
Zeitschrift fur Wahrscheinlichkeitstheory und Verwandte Gebiete,
19:225{230, 1971.
41] K. Mehlhorn. The \almost all" theory of subrecursive degrees is decid-
able. In Proceedings of the Second Colloquium on Automata, Languages,
and Programming, pages 317{325. Springer Lecture Notes in Computer
Science, vol. 14, 1974.
42] Y. N. Moschovakis. Descriptive Set Theory. North-Holland, 1980.
43] J. C. Oxtoby. Measure and Category. Springer-Verlag, 1980. second
edition.
44] H. Rogers, Jr. Theory of Recursive Functions and Eective Computabil-
ity. McGraw - Hill, 1967.
45] H. L. Royden. Real Analysis, third edition. Macmillan Publishing
Company, 1988.
46] G. E. Sacks. Degrees of Unsolvability. Princeton University Press, 1966.
47] C. P. Schnorr. Process complexity and eective random tests. Journal
of Computer and System Sciences, 7:376{388, 1973.
48] A. Kh. Shen . The frequency approach to dening a random sequence.
0

Semiotika i Informatika, 19:14{42, 1982. (In Russian.).

49] A. Kh. Shen . On relations between dierent algorithmic denitions of
0

randomness. Soviet Mathematics Doklady, 38:316{319, 1989.

50] R. I. Soare. Recursively Enumerable Sets and Degrees. Springer-Verlag,
1987.
51] R. J. Solomono. A formal theory of inductive inference. Information
and Control, 7:1{22, 224{254, 1964.
52] R. M. Solovay, 1975. reported in 12].
44
53] V. V. V'jugin. On Turing invariant sets. Soviet Mathematics Doklady,
17:1090{1094, 1976.
54] V. V. V'jugin. The algebra of invariant properties of nite sequences.
Problems of Information Transmission, 18:147{161, 1982.
55] A. K. Zvonkin and L. A. Levin. The complexity of nite objects and the
development of the concepts of information and randomness by means
of the theory of algorithms. Russian Mathematical Surveys, 25:83{124,
1970.

45