Life’s longing for itself

Speculations on propagation, prediction,

purpose and progress
Amsterdam, October 2004
(Updated version of Ulam Lectures given Sept.
2002)
J. Doyne Farmer
Santa Fe Institute
Your children are not your children.
They are the sons and daughters of Life’s
longing for itself.
They come through you but not from you,
And though they are with you yet they
belong not to you.
…
For life goes not backward nor tarries with
yesterday.

Kahlil Gibran, The Prophet

 Lecture 1: Propagation
• What are complex systems?
• The history of the mechanistic view
• Entropy and information
• What is a machine?
• Organisms and artifacts evolve
• How might the first copy machine have been built?
• The symbiosis of human population and technology
 Main points of lecture 1
• Why is the world populated with functional
structures?
– Propagation implies prevalence.
• Not reducing universe to “just mechanics”
– Through self-organization, machines are capable of
far more than previously thought.
• Biological life, human artifacts, and human
societies all evolve.
– Relationship becoming increasingly intimate.
 Lecture 2: Prediction
• Prediction, action and survival
• Methods of prediction
• Limits to prediction and their loopholes
• Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
 Lecture 3: Purpose and progress
• The progress debate
• The purposeful arrow of time
• What is progress?
• Happiness vs. purpose
• A few speculations about the future
 Prediction, action, and survival
• Prediction is a prerequisite for purposeful
behavior.
• Purposeful behavior consists of three parts
– Sensation
– Prediction
– Action
• Purposeful behavior (and therefore prediction)
exists because it is useful for propagation.
Prediction, action and survival

QuickTime™ and a
Sorenson Video 3 decompressor
are needed to see this picture.
 Lecture 2: Prediction
• Prediction, action and survival
• Methods of prediction
• Limits to prediction and their loopholes
• Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
 Methods of prediction
• How is the model constructed?
– First principles vs. empirical
• What does the model predict?
– Dynamical vs. contemporaneous predictions
• Modeling paradigm
– Deterministic vs. random processes
Astrological prediction of stock prices
Fibonnaci predicts social trends!
 How is the model constucted?
• First principles. Based on an understanding
of the world.
– Requires high degree of sophistication
• Empirical. Build a model automatically by
fitting historical data.
– Simple organisms: Hard-wired models, tuned by
evolution
– Complex organisms: Further tuning by
experience
 What does the model predict?
• Dynamical systems:
– Predict the future based on the past
• Contemporaneous models:
– Relate one property of the world to another
property of the world at the same time.
– Useful in simplifying description of the world.
 Modeling paradigm
• Deterministic
– World is described by a single point in state
space. Completely determines future. Rule
that does this is called a dynamical system.
• Random
– Evolution of future states is not determined by
present states.
 Dynamical prediction
• Key idea is state space. A state is a list of
numbers that gives the information needed
to determine the future. If you have N such
numbers, it is useful to think of them as
defining an N-dimensional space.
• E.g. from Newton’s laws, knowing forces,
position and velocity are sufficient to
determine future motion. Position and
velocity are the state.
 How do bacteria do it?
• Don’t know in detail.
– Must predict concentration
– Involves measuring the concentration at
different points in time and comparing.
– If concentration is increasing, keep swimming
– Otherwise tumble and/or eat
• State space:
– One number: Rate of change of concentration
– (concentration now – concentration earlier)
 Lecture 2: Prediction
• Prediction, action and survival
• Methods of prediction
• Limits to prediction and their loopholes
• Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
 Limits to prediction
• Limits to prediction come from complicated
geometry of state space, which causes
nearby states to diverge rapidly.
• Produce behavior that looks random, even
in purely deterministic setting.
• Small uncertainties in initial measurements
are amplified, limiting predictability even
when model is known.
 On Hurricane Charlie
You can’t plan for the unforeseen. God
doesn’t follow the linear directions of
computer models. And these are powerful
storms that don’t behave in any kind of way
that you can say with certainty where they
are going.
Jeb Bush
 Poincare’ on Fortuitous phenomena
A very small cause which escapes our notice determines a
considerable effect that we cannot fail to see, and then we say
that the effect is due to chance. If we knew exactly the laws
of nature and the situation of the universe at the initial
moment, we could predict exactly the situation of that same
universe at a succeeding moment. But even if it were the
case that the natural laws had no longer any secret for us, we
could still only know the initial situation approximately. If
that enabled us to predict the succeeding situation with the
same approximation, that is all we require, and we should say
that the phenomenon had been predicted, that it is governed
by laws. But it is not always so; it may happen that small
differences in initial conditions produce very great ones in
the final phenomena. A small error in the former will
produce an enormous error in the latter. Prediction becomes
impossible, and we have the fortuitous phenomenon.
 Chaos is a double-edged sword
• On one hand, long-term behavior is effectively
random (though dynamics are deterministic)
• On the other hand, short-term behavior is
predictable if model is known.
– Systems otherwise believed random become
predictable in the short term.
– Simple mechanical oscillators, transition fluid flows,
sunspots, timing of ice ages, …

(joint work with John “Sid” Sidorowich).

Rolling ball on a circular track with
counter-spinning inside track
 Roulette
• Classical physics problem Newton could have solved.
– Measuring position and velocity at a given time determines
future motion. Wind resistance is main force
2
rate of change of velocity constant (velocity)
– complication due to tilt
• Prediction is difficult because of:
– circularity of wheel (like taking remainder in division)
– imperfections in track and ball creates “turbulence”
– Chaotic bouncing on cups
Shoe computer
 Shoe +
computer
Histogram + battery boat
Copy machines
 Illustration of two
methods of prediction
• Roulette provided a good illustration of two
methods of prediction:
– Version (1) based on first principles
– Version (2) based on empirical method
– (1) was more accurate but less robust
 Making predictions can alter the future
• After the book The Eudeamonic Pie was published
in 1984
– Nevada passed a law against using computers to predict
the outcome of “a game”.
– Huxley roulette wheel company designed a new
roulette wheel with lower cups and more elastic balls.
– Winning players who place bets at the last minute are
immediately asked to take their business elsewhere.
– Altered the rest of my life
 Limits to short term prediction
• Limits to short-term prediction come from

The curse of dimensionality

Data needed to build a good model increases
exponentially with dimension of the state space
• Even worse: High dimensionality, chaos, ability to
measure only some variables, means that some
systems are fundamentally random, even for very
short term prediction
– Casdagi, Eubank, Farmer, Gibson (1991)
• Weather, the economy, …
– high dimensional + chaotic
 What about free will?

The brain as a dynamical system

 Prediction can make the world less predictable
• Market efficiency: most economists believe that future
price movements are fundamentally unpredictable.
– If there are patterns in prices, profit-seeking behavior of
participants will eliminate them.
– E.g. if people think the price is going to rise, more people
will buy, which drives the price up, so the price rise happens
before it is possible to take advantage of it.
– The future becomes the present
• Effect of predictions complicates dynamics.
• Result: unpredictable prices -- “market efficiency”
• To first approximation a good model
 Prediction Company
(cofounded in 1991 with Norman Packard)

• (Empirical, dynamical prediction, random process)

• Manages money under exclusive relationship with
United Bank of Switzerland (Warburg Dillon Reed)
• “Cerebellar” approach to market forecasting
– empirically searches for patterns in historical data
– keys are feature extraction, law of large numbers
– little understanding of origin of patterns
– relies on abundant past data, stationary conditions.
• Trading is fully automated (no human decisions).
Harvard Business Review, 1992
 Nonetheless
• Profits are limited
– Market has friction -- trading changes prices
– Particularly felt with frequent trading
– Market is “pretty efficient” - like any business
 Lecture 2: Prediction
• Prediction, action and survival
• Methods of prediction
• Limits to prediction and their loopholes
• Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
 Mechanistic properties of markets

• Market institutions shape our behavior

• Neoclassical economic models assume
perfect rationality of agents
• We explore alternative: Random behavior
– Make a physics style model
– Agents make random orders at random times
 Order driven market
• Two fundamental types of orders
– Market order: buy or sell given number shares
at best available price
– Limit order: buy or sell given number shares at
a specified price. Does not guarantee execution!
• Patient traders use limit orders; impatient
traders use market orders
 Patient trading
• Patient traders place non-marketable
limit orders that do not lead to an
Limit Order
immediate transaction
• Non-marketable limit orders accumulate BUY / SELL
• Limit order book is a storage device # OF SHARES
LIMIT PRICE
VOLUME

BID

ASK
price ($)
NEW ASK
 Impatient trading
Market order:
• An order to buy or sell up to a given volume
Market Order
• No limit price is defined
BUY / SELL
• Executed immediately
# OF SHARES
• Often causes unfavorable price impact
VOLUME

BID

ASK
price ($)
NEW ASK
 Order cancellation

Limit order cancellations:

• Limit orders can be cancelled by the owner
• Market defined expiration
VOLUME

price ($)
 The Basic Model
Limit order arrival: unit size, in time & price; 
Market order arrival: unit size, random in time; 
Cancellation: Random in time (like radioactive decay); 
Assume prices are continuous.

Depth of the book np,t tells how many shares are in the book at
price p at a given time t.
SELL LIMIT ORDERS 
( p, t )
n( p, t ) BID
BUY MARKET
ORDERS


0 SELL MARKET
 p
ORDERS
ASK

BUY LIMIT ORDERS 

 Parameters of model
Three fundamental dimensional quantities:
shares, price, time
Five parameters:
 limit order rate, shares/(price time)
 market order rate, (shares/time)
 order cancellation rate, (1/time)
 typical order size (shares)
dp tick size (price)
Results in simple formulas predicting volatility, liquidity, spread, ..
London Stock Exchange data set
 Continuous double auction
Execution priority of limit orders:
• Price priority: lower sell / higher buy limit prices
• Time priority: applicable only for limit orders with same price

PRIORITY
VOLUME

VOLUME LIMIT ORDERS

SELL

SPREAD

(BEST) BID

price ($)
(BEST) ASK
BUY

PRIORITY
 QuickTime™ and a
Video decompressor
are needed to see this picture.
Predicted
price
diffusion
rate
 1/ 2
  
5 / 2 1/ 2

 2
Top 10 Russian jokes, Oct. 23, 2003
с сайта "Немецкая волна"
http://www.dw-world.de/russian/0,3367,2212_A_985770_1_A,00.html
Ученые-экономисты давно стараются понять закономерности, которым
подчиняются биржевые курсы, и используют для этого математические
модели. На протяжении многих десятилетий такие модели исходили из
представлений о брокерах как об аналитиках с выдающимися умственными
способностями, обладающих исчерпывающей информацией о рынке и
действующих исключительно рационально. Однако удовлетворительно описать
реальные изменения биржевых курсов эти модели оказались не в состоянии.
Значительно успешнее справляется с этой задачей новая модель,
предложенная Дойном Фармером (J. Doyne Farmer), сотрудником Института
Санта-Фе в штате Нью-Мексико. Она базируется на предположении, что
брокеры Ц полные Ђидиотыї, действующие совершенно случайно и к тому же
лишенные какой бы то ни было информации. Сравнив данные, рассчитанные на
основе этой модели, с реальными курсами лондонской фондовой биржи за
период с 1998-го по 2000-й годы, ученые выявили очень высокую степень
совпадения
 Lecture 2: Prediction
• Prediction, action and survival
• Methods of prediction
• Limits to prediction and their loopholes
• Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
The 2nd millenium technology bubble
(NASDAQ)
Second millenium tech bubble (CISCO)
 Wheat price in Munich (1815-1820)

from: Hidden Collective

Factors in Speculative
Trading, by Bertrand M.
Roehner (2001)
 Monthly wheat price in Paris (1691-1694)

from: Hidden Collective

Factors in Speculative
Trading, by Bertrand M.
Roehner (2001)
 Price of Bananas

from: Hidden Collective

Factors in Speculative
Trading, by Bertrand M.
Roehner (2001)
Fortune teller problem

Predicting the future

influences the future
Feedback between prediction and reality
… if a dream can tell the future it can also thwart that
future. For God will not permit that we shall know
what is to come. He is bound to no one that the
world shall unfold just so upon its course and those
who by some sorcery or by some dream might
come to pierce the veil that lies so darkly over all
that is before them may serve by just that vision to
cause that God should wrench the world from its
heading and set it upon another course altogether
and then where stands the sorcerer? Where the
dreamer and his dream?
Cormac McCarthy, The Crossing
Dmitriy Cherkashin

Seth Lloyd
 A simple game with feedback between
perception and reality
• Assume N agents bet on event at time t.
– e.g. a horse race.
• Odds based on net wager on each outcome.
• Allow outcome to be influenced by odds.
– financial markets provide a good example.
– supply and demand are inherently subjective.
• Simplest case is coin flip
– only two outcomes
– coin can be biased -- bias can depend on odds

probability tails (1 - probability heads)

 Examples where perception can
influence reality
• Gambling
• Economics
• Politics
• Global climate change
• Personal achievement
 Agent strategies
• Assume N+ 1 players bet
(0, 1/N, 2/N, …, 1) of their money on heads
• Give them all equal wealth to start with.
Purely objective reality
Self-defeating prophesy
Self-defeating prophesy

Bias of coin vs. time

Weak self-fulfilling prophesy

Bias of coin vs. bias of predictions

Weak self-fulfilling prophesy

Bias of coin vs. time

Perfectly self-fulfilling
Perfectly self-fulfilling prophesy

Bias of coin vs. time

Over self-fulfilling
Over self-fulfilling

Bias of coin vs. time

Perfectly self-fulfilling prophesy

Bias of coin vs. time

 Feedback and reality
• If you are able to create reality, reality can
become unstable due to feedback.
• You must make yourself part of the
prediction; must also model others.
• Danger of schizophrenia
 Creating reality
Guys like you are ‘in what we called the
reality-based community’, defined as people
who ‘believe that solutions emerge from your
judicious study of discernable reality. That’s
not the way the world really works anymore.
We’re an empire now, and when we act we
create our own reality.’
A senior advisor to G.W. Bush, to Ron Susskind
 Summary
• Prediction is a key element of purposeful
behavior coming out of propagation.
• Predictions come in many shapes and sizes
• As the sophistication of predictions
increases, their affect on the environment
can make prediction more difficult, and
make dynamics unstable.
Thanks

Else Neeft