at i

Remarks on simplicity and complexiq

by the Nobel Prize- winning author of The Quark and the Jaguar

‘hat is complexity? A great many quantities Iiave been proposed as measures

ofsomething like complexity. In fact, a variety of different measures would be
required to capture all our intuitive ideas about what is meant by complexity
and by its opposite, simplicity.
Some of the quantities, like computational complexity, are time (or space) mea-
sures. They are concerned with how long it would take (or how much capacity would
be needed), at a minimum, for a standard universal computer to perform a particular
task. Computational complexity itself is related to the least time (or number of steps)
needed to carry out a certain computation.
Other suggested quantities are information measures, referring, roughly spealting,
to the length of the shortest message conveying certain information. For example, the
algorithmic information content (or AIC) of a string of bits is defined as tlie length of
the shortest program that will cause a standard universal computer to print out the
string of bits and then halt.
As measures of something like complexity for an entity in the real world, all such
quantities are to some extent context-dependent or even subjective. They depend on
the coarse graining (level of detail) of the description of the entity, on tlie previous
MURRAY EEL%-M4NI knowledge and understanding of the world that is assumed, on the language employed,
~ ~~ ~ ~~ on the coding method used for conversion from that language into a string of bits, and
on the particular idealized computer chosen as a standard. However, if one is consid-
Nobel lairreate Murray Gell-Marin ering a sequence of similar entities of increasing size and complexity, and one is inter-
has rnariy intellectiral passions ested only in how the measure behaves as the size becomes large, then of course many
including natirral history, liriguistics, of tlie arbitrary features become negligible. Students of computational complexity are
a rcheo logy, his tory, depth psych ology typically concerned with whether a sequence of larger and larger problems can be solved
and creatiue thinking. As Professor in a time that grows as a polynomial in the size of the problem (rather than an expo-
arid Co-Chair-nian of the Science Board nential or something worse). It is probably safe to say that any measure of complexity
oftlie Santa Fe Institute, he works with is most useful for comparisons between things at least one of which has high complex-
rnarzy researchers i n their search ity by that measure.
for corznectioris betiueen basic laita Many of the candidate quantities are uncomputable. For example, the algorithmic
of physics and the cornplexiqi arid information content of a long bit string can readily be shown to be less than or equal to
diversity of the nati.ira1 world. some value. But for any such value there is no way of excluding the possibility that the
AIC could be lower, reduced by some as yet undiscovered theorem revealing a hidden
regularity in the string. A bit string that is incompressible has no such regularities and
is defined as “random.”A random bit string has maximal AIC for its length, since the
shortest program that will cause the standard computer to print it out and then halt is
just the one that says PRINT followed by the string itself.
This property of AIC, which leads to its being called, on occasion, “algorithmic ran-
domness,” reveals the unsuitability of the quantity as a measure of complexity, since

116 C O M P L E X i T Y 0 1995 John Wiley & Sons, Inc.

CCC 1076-2787/95/010016-04
the works of Shakespeare have a lower from one member of the set to another.) ior of the system. The package of infor-
AIC than random gibberish of the same The effective complexity of the regulari- mation or “schema” is subject to varia-
length that would typically be typed by ties can then be defined as the AIC of the tion, in such a way that there is competi-
the proverbial roomful of monkeys. description of the set of entities and their tion among different schemata. Each
A measure that corresponds much probabilities. (Specifying a given entity, schema can be used, along with some of
better to what is usually meant by com- such as the original one, requires addi- the data, to describe the system and its
plexity in ordinary conversation, as well tional information.) environment, to predict the future, and
as in scientific discourse, refers not to the Some authors have tried to character- to prescribe behavior for the system. But
length of the most concise description ot ize complexity by using the nnzo~iii~ of the description and prediction can be
an entity (which is roughly what AIC is), mutual algorithmic information rather checked against further data, with the
but to the length of a concise description than the length of a concise description comparison feeding back to influence the
of a set of the entity’s regularities. Thus of the corresponding regularities. Such a competition among schemata. Likewise
behavior conforming to a prescription
has real world consequences, which can
also affect the competition. In this way
the schemata evolve, with a general ten-
dency to favor better description and pre-
diction as well ;IS behavior conforming
more or less to the selection pressures in
completely regular, such as a bit string usually meant by complexity. Take, as a the real world.
consisting entirely of zeroes. Effective simple example, any string of bits consist- Examples on Earth of the operation of
complexity can be high only in a region ing entirely of pairs 00 and 11. Such a complex adaptive systems include bio-
i n t e r in ed i a t e bet ween total o rd e r and string possesses an obvious regularity, logical evolution, learning and thinking
complete disorder. but one that can be very briefly described: in animals (including people), the func-
the sequences of odd-numbered and tioning of the immune system in mam-
here can exist no procedure for find- even-numbered bits are identical. The mals and other vertebrates, the operation
ing all regularities of an entity. But quantityofmutual AIC between those se- of the human scientific enterprise, and
classes of regularities can be iden- quences is enormous, however, for a long the behavior of computers that are built
tified. Finding regularities typically refers string. Evidently the complexity here is or programmed to evolve strategies-for
to taking the available data about the en- better represeiited by the length of the example by means of neural nets or ge-
tity, processing them in some manner brief description than by the amount of netic algorithms. Clearly, complex adap-
into, say, a bit string, and then dividing nititual algorithmic information. tive systems have a tendency to give rise
that string into parts in a particular way Since it is impossible to find all regu- to other complex adaptive systems.
and looking for mutual AIC among the larities of an entity, the question arises as
parts. If a string is divided into two parts, to who or what determines the class of t is worth remarking for readers of
for example, the mutual AIC can be taken regularities to be identified. One answer this journal that John Holland, for ex-
to be the sum of the AIC’s of the parts is to point to a most important set of sys- ample, uses il different set of terms to
minus the AIC of the whole. An amount tems, each of which functions precisely describe some of the same ideas. He uses
of mutual algorithmic information coti- by identifying certain regularities in the “adaptive agent” for a complex adaptive
tent above a certain threshold can be con- data stream reaching it and compressing system as defined above, reserving the
sidered diagnostic of a regularity. Given those regularities into a concise package name “complex adaptive system” for a
t h e identified regularities, the corre- of information. The data stream includes composite complex adaptive system
sponding effective complexity is the AIC information about the system itself, its (such as an economyor an ecological sys-
of a description of those regularities. environment, and the interaction be- tem) consisting of many adaptive agents
More precisely, any particular regu- tween the environment and the behav- making predictions of o n e another’s
larities may be regarded as embedding
the entity in question in a set of entities
sharing the regularities and differingonly
in other respects. In general, the regulari-
ties associate a probability with each en-
tity in the set. (The probabilities are in
many cases all equal but they may differ

01995 John Wiley & Sons, Inc. C O M P L E X I T Y 17

behavior.What I call a schema he calls an of that) rather than effective complexity. Heisenberg’s uncertainty principle.
internal model. Both of us are conform- In contemplating na t ti ral phenomena, Of course, in many cases the quan-
ing to the old saying that a scientist would we frequently have to distinguish be- tum-mechanical probabilities are very
rather use someone else’s toothbrush tween effective complexity and logical close to certainties, so that deterministic
than another scientist’s nomenclature. depth. For example, the apparently com- classical physics is a good approximation.
Any complex adaptive system can, of plicated pattern of energy levels of atomic But even in the classical limit, and even
course, make mistakes in spotting regu- nuclei might easily be misattributed to when the laws and initial condition are
larities.\Ve human beings, who are prone some complex law at the fundamental exactly specified, indeterminacy can still
to superstition and often engage in de- level, btit i t is now believed to follow be introduced by any ignorance ofprevi-
nial of the obvious, are all too familiar from a simple underlying theory of ous history. Moreover, the effects of
with such errors. quarks, gluons, and photons, although such ignorance can be magnified by the
Besides the possibility of error, we lengthy calculations would be required phenomenon of chaos in nonlinear dy-
should also consider the difficulty of to deduce the detailed pattern from the namics, whereby future outcomes are
computation. How much time is involved basic equations. T ~ UtheS pattern has a arbitrarily sensitive to tiny changes in
in deducing practical predictions from a good deal of logical depth and very little present conditions.
highly compressed schema, say a scien- effective complexity. We can think of the alternative pos-
tific theory, together with some specific sible coarse-grained histories of the uni-
additional data such as boundary condi- verse as forming a branching tree, with
tions? Here we encounter time measures probabilities at each branching. Note
of “complexity,” for instance logical these are n priori probabilities rather
depth, which for a bit string is related to than statistical ones, unless we engage in
the time required for a standard univer- the exercise of treating the universe as
sal computer to compute the string, print one of a huge set of alternative universes,
it out, and then halt. That time is aver- forming a “tnultiverse.” Of course, even
aged over the various programs that will within a single universe cases arise of re-
acromplish the task, with an averaging producible events (such as physics ex-
procedure that weights shorter programs I t now seems likely that the funda- periments). and for those events the (I
more heavily. We can then consider the mental law governing the behavior of all priori probabilities of the quantum me-
logical depth of any entity it a suitably matter in the universe-the unified quan- chanics of the universe yield conven-
coarse-grained description of it is en- tum field theory ofall the elementary par- tional statistical probabilities.
coded into a bit string. ticles and their interactions-is quite A n y entity in the world around us,
simple. (In fact, we already have a plau- such as a n individual h u m a n being,
kind of inverse concept to logical sible candidate in the form ofsuperstring owes its existence not only to the simple
depth is crypticity, \vhich measures theory.) It also appears that the bound- fundamental law of physics a n d t h e
the time needed for a computer to ary condition specifiing the initial COII- boundary condition on the early universe
reverse the process and go from a bit dition of the universe around the begin- but also to the outcomes of an inconceiv-
string to one of the shorter programs ning of its expansion may be simple as ably long s e q u e n c e of probabilistic
that will generate i t . I n t h e h u m a n well. If both of these propositions are events, each of which could have turned
scientific enterprise, we can identify true, does that mean that there is hardly out differently.
crypticity roughly with the difficulty of any effective complexity in the universe? Now a great many of those accidents,
constructing a good theory from a set of Not at all, because of the relentless op- for instance niost cases of the bouncing
data, while logical depth is a crude mea- eration of chance. of a particular molecule in a gas to the
sure of the difficulty of making predic- right rather than the left in a molecular
tions from the theory. iven the basic law and the initial collision, have few ramifications for the
It is often hard to tell whether some- condition, the history of the uni- future coarse-grained histories. Some-
thing that is apparently complex really verse is by no means determined, times, however, an accicient can have
possesses a great deal of effective com- because the law is qiiantutn-mechanical, widespread consequences for the future,
plexity or reflects instead underlying sim- thus yielding only probabilities for alter- although those are typically restricted to
plicity combined with a certain amount native histories. Moreover, histories can particular regions of space and time. Such
of logical depth. Faced with a fairly de- be assigned probabilities only if they are a “frozen accident” produces a great deal
tailed diagram of Mandelbrot’s famous sufficiently coarse-grained to display of m u t u a l algorithmic information
fractal set, for example, we might at- decoherence (the absence of interference among various parts or aspects of a fu-
tribute to it a high effective complexity terms between them). Thus quantum ture coarse-grained history of the uni-
until we learn that it can be generated mechanics introduces a great deal of in- verse, for many such histories and for
from a very simple formula. It has logical determinacy, going far beyond the rather various ways of dividing them up.
depth (and not even a gigantic amount trivial in d e t e r in i n acy as so c i a t e d with But such a situation, in which there is

18 C O M P L E X I T Y 01995 John Wiley & Sons, Inc.

a great deal of mutual algorithmic infor- time, relative to a fixed time, say the
mation generated, corresponds precisely present. The new quantity is the effective
to what we have called a regularity. Thus, complexity of the entity at a future time,
as time goes by in the history of the uni- averaged over the various coarse-grained
verse and accidents (with probabilities histories of the universe between the
for various outcomes) accumulate, so do present and that time, weighted accord-
frozen accidents, giving rise to regulari- ing to their probabilities.
ties. Most of the effective complexity of The era may not last forever in which
the universe lies in the AIC of a descrip- more and more complex forms appear
tion of those frozen accidents and their the possibility arises of a selective advan- as time goes on. If, in the very distant
consequences, while only a small part tage being associated under certain cir- future, virtually all nuclei in the universe
comes from the simple fundamental laws cumstances with increased complexity. decay into electrons and positrons, neu-
of the universe, (the law of the elemen- trinos and antineutrinos, and photons,
tary particles and the condition at the he second law of thermodynamics, then the era characterized by fairly well-
beginning of the expansion). For a given which requires average entropy (or defined individual objects may draw to
entity in the universe, it is of course only disorder) to increase, does not in an end, while self-organization becomes
the frozen accidents leading up to its own any way forbid local order from arising rare and the envelope of complexity
regularities that contribute, along with through various mechanisms of self-or- begins to shrink.
the basic laws, to its effective complexity. ganization, which can turn accidents into
-As the universe grows older and fro- frozen ones producing extensive regulari- ACKNOWLEDGMENTS
zen accidents pile up, the opportunities ties. Again, such mechanisms are not re- These remarks summarize some of the
for effective complexity to increase keep stricted to complex adaptive systems. material in my book, The Quark and the
accumulating as well. Hence there is a Different entities may have different Jaguar, which is irztendedfor the lay reader
tendency for the envelope of complexity potentialities for developing higher com- interested in science. A more precise and
to expand, even though any given entity plexity. Something that is not particularly mathematical version will be presented
may either increase or decrease its com- distinguished from similar things by its elsewhere, with proper references to ear-
plexity during a given time period. effective complexity can nevertheless be lier work.
The appearance of more and more remarkable for the complexity it may It is a pleasure to acknowledge thegreat
complex forms is not a phenomenon re- achieve in the future. Therefore it is im- ualue of conversations with Charles H.
stricted to the evolution of complex adap- portant to define a new quantity, “poten- Bennett, James Crutchfield, James B.
tive systems, although for those systems tial complexity,” as a function of future Hartle, John Holland, and Seth Lloyd.

ledesls aled A ~ h o ~ i s m s
by David Lane

01995 John Wiley & Sons, Inc. C O M P L E X I T Y ’19