Robotics and Autonomous Systems 53 (2005) 177–193

Quantitative description of robot–environment

interaction using chaos theory
Ulrich Nehmzow a,∗ , Keith Walker a,b
a Department of Computer Science, University of Essex, Colchester CO4 3SQ, UK
b Department of Physics and Engineering, Point Loma Nazarene University, San Diego, CA 92106, USA

Received 3 July 2003; received in revised form 22 September 2005; accepted 28 September 2005

Abstract

Mobile robotics research to date is still largely reliant on trial-and-error procedures, rather than exploiting established the-
ories describing robot–environment interaction in a formal manner, making falsifiable predictions and allowing quantitative
descriptions of a robot’s behaviour.
We argue that quantitative performance measures are the first step towards a theory of robot–environment interaction, and
present the theoretical background to such measures, as well as their practical application to mobile robotics research.
Results obtained with a Pioneer II mobile robot, executing a number of different tasks in a range of environments, are presented.
© 2005 Elsevier B.V. All rights reserved.

Keywords: Mobile robotics; Dynamical systems theory; Chaos theory; Quantitative description of robot–environment interaction; Scientific
robotics; Analytical robotics

1. Introduction observations made using the model and the robot. This
process is iterated until the robot’s behaviour resembles
Research in mobile robotics to date has, with very the desired behaviour to a sufficient degree of accu-
few exceptions, been based on trial-and-error exper- racy. Typically, the results of these iterative refinement
imentation and the presentation of existence proofs. processes are valid within a very narrow band of appli-
Task-achieving robot control programs are obtained cation scenarios, they constitute “existence proofs”. As
through a process of iterative refinement, typically such, they demonstrate that a particular behaviour can
involving the use of computer models of the robot, be achieved, but not, how that particular behaviour can
the robot itself, and program refinements based on in general be achieved for any experimental scenario.
The purpose of this paper is to lay foundations for a
∗ Corresponding author. Tel.: +44 1206 87 3586; theory of robot–environment interaction. We begin by
fax: +44 1206 87 2788. defining what we mean by “robot–environment inter-
E-mail address: [email protected] (U. Nehmzow). action” and “theory”. We then outline a possible theory

0921-8890/$ – see front matter © 2005 Elsevier B.V. All rights reserved.
doi:10.1016/j.robot.2005.09.009
178 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193

of robot–environment interaction, and in part II of this robot can be made successful by either (i) changing
paper present results obtained through experimentation the robot’s control code (“task”), (ii) modifying the
with Pioneer II mobile robots, applying the principles environment (for example by adding sensor-reflective
outlined in part I of this paper. strips, i.e. changing the “environment”), or (iii) by
adding extra sensors (“robot”).
1.1. Robot–environment interaction This triangle of robot, task and environment consti-
tutes a non-linear system, whose analysis is the purpose
When we speak of “robots” in this paper, we mean of any theory of robot–environment interaction.
situated, embodied mobile robots operating in a real- Rather than speaking solely of a robot’s behaviour,
world environment. “Situated” refers to an agent (in it is therefore necessary to speak of robot–environment
this case a mobile robot) operating within an embracing interaction, and the robot’s behaviour resulting
framework, a “world”, that it cannot leave—the agent thereof.
cannot “step outside” its environment, and actions per-
formed can affect the environment, and change it for 1.2. Theory
the robot. For example, a situated agent has to oper-
ate within the time frame made available for it, e.g. By “theory of robot–environment interaction” we
the clock cannot be stopped for lengthy computations. mean a coherent body of hypothetical, conceptual
In contrast to artificial, simulated environments, where and pragmatic generalisations and principles that form
time or the results of physical actions (e.g. collisions) the general frame of reference within which mobile
may be ignored by the agent, the physical, real-world robotics research is conducted.
is always “there” and imposes its restrictions on the There are two key elements that make a theory of
situated agent, whatever the agent’s actions, etc. robot–environment useful, and therefore desirable for
“Embodied” refers to physical agents whose mor- research:
phology does matter with respect to the agent’s inter-
action with the environment. For instance, the modality (1) A theory will allow the formulation of hypotheses
and physical positioning of sensors does affect a robot’s for testing,
behaviour, and it is a known fact that different robot (2) a theory will make predictions (for instance regard-
behaviour can be obtained by simply repositioning sen- ing the outcome of experiments).
sors, without ever re-programming the robot.
A theory retains, in abstraction and generalisa-
It follows from these considerations that the
tion, the essence of what it is that the triple
behaviour of a mobile robot cannot be discussed in
of robot–task–environment does. This generalisa-
isolation: it is the result of properties of the robot itself
tion is essential: it highlights the important aspects
(physical aspects—the “embodiment”), the environ-
of robot–environment interaction, while suppressing
ment (“situatedness”), and the control program (the
unimportant ones. Finally, the validity of a theory (or
“task”) the robot is executing (see Fig. 1). The fol-
otherwise) can then be established by evaluating the
lowing illustration may help to appreciate this point: it
predictions made applying the theory.
is well known to robotics practioners that a “failing”

1.2.1. The role of quantitative performance

measures
Measurement is the backbone of science, and there-
fore the backbone of any theory of robot–environment
interaction.
If we assume (at this point of this paper for argu-
ment’s sake—we will show later how this can be
achieved) that the behaviour resulting from the inter-
Fig. 1. The fundamental triangle of robot–environment interaction:
a robot’s behaviour has to be analysed in the context of robot, task action of agent, task and environment can be described
and environment. quantitatively, a number of possibilities arise:
 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193 179

(1) Two of the three elements in Fig. 1 are kept robot itself, its hardware, physical makeup, inertia, sen-
unchanged, and the third one is modified in a sor and motor characteristics, etc.; (ii) the environment
principled manner. The quantitative performance the robot is operating in, the colour, texture and struc-
measure then characterises that third component. ture of walls, floors and ceilings, the temperature and
• For instance, to characterise two environments humidity, speed of sound in the environment, noise,
quantitatively, the same robot and control pro- etc.; and (iii) the task the robot is executing, i.e. the
gram can be used in either environment, and control program that is currently being executed by the
the quantitative description of behaviour used robot (Fig. 1).
to identify (in the sense of system identification) Therefore, we argue, a mobile robot, interacting
the environments. with its environment, could be viewed as an analog
• Likewise, by principled modification of just one computer. Similar to an optical lens, which takes light
of the three components shown in Fig. 1, optimal rays as its input and “computes” the Fourier transfor-
parameter settings (with respect to some desired mation of the input image as its output (thus acting as an
behaviour) can be determined in a systematic analog computer), or a cylindrical lens taking the visual
way. image of the environment as its input and “computing”
(2) Experimental results can be stated quantitatively. the positions of vertical edges in the image as its out-
This allows replication and verification of experi- put (again acting as an analog computer), the mobile
mental results, which is currently hardly possible robot, executing some control program in its environ-
in mobile robotics research. ment “computes” behaviour as its output from the three
(3) Predictions made by the theory of robot–envi- input components shown in Fig. 1, i.e. robot-specific
ronment interaction can be made quantitatively, components, environment-specific components and the
and tested against the actual experimental results. task (see Fig. 2).
In this paper we present quantitative descriptions of
robot behaviour that can be used in the manner outlined 2.2. Three theses
above.
Based on the aforementioned considerations we
argue that
2. Elements of a theory of robot–environment
interaction
Thesis 1. A mobile robot, interacting with its envi-
2.1. Robot–environment interaction as analog ronment, is essentially an analog computer that “com-
computation putes” behaviour (the output) from the three inputs
robot morphology, environmental characteristics and
To repeat the point made earlier: the behaviour of a executed task, by means of an (unknown) control func-
mobile robot is governed by three main factors: (i) the tion f (Fig. 2).

Fig. 2. A mobile robot interacting with its environment can be viewed as an analog computer, taking environmental, morphological and task-
related data as input, and “computing” behaviour as output (see also Fig. 1).
180 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193

How exactly is the output of this analog computer of robot–environment interaction, and to use these to
(“behaviour”) defined? It is obvious that the manifes- characterise what it is that a robot does, as well as high-
tations of the output variable “behaviour” are manifold lighting differences between different tasks or environ-
and complex; however, we argue that ments quantitatively.
Based on Thesis 3 (stated above) we treated logged
Thesis 2. The behaviour exhibited by a robot, inter- robot trajectories (x- and y-coordinate separately) as a
acting with its environment while executing some par- time series, and analysed them with respect to chaotic
ticular task, is encapsulated, as a first approximation, properties. In particular, we determined the largest Lya-
in the robot’s trajectory. punov exponent and the correlation dimension of each
time series to obtain quantitative descriptions of the
The reason for the second thesis is that the final, robot’s behaviour. These quantitative descriptions of
observed trajectory is the result of all processes affect- chaotic systems are described in detail below.
ing robot perception and action, environmental influ- A dynamical system is considered to exhibit deter-
ences, and the effects of the control program. Any noise ministic chaos if it has the following four properties:
or variability encountered in perceiving or acting, any
(1) the signal is stationary,
assumption or error made in the control program affects
(2) the signal is deterministic,
the robot’s trajectory in some way. Arguably, there-
(3) the signal is aperiodic,
fore, the trajectory of a robot retains the essence of that
(4) the system exhibits sensitivity to initial conditions.
robot’s interaction with its environment.
There is a second, more pragmatic justification for A discussion of our methods to investigate each of these
the second thesis. For many real-world applications of four aspects follows.
mobile robots it is actually the final trajectory that deter-
mines whether the robot achieves its task or not. It is 3.1. Stationarity
therefore sensible to start with the analysis of the tra-
jectory. For analysis, the dynamics of the system investi-
gated need to be bounded, i.e. staying within a finite
Thesis 3. One suitable method to describe a robot’s range and never approaching ∞ or −∞. As any signal
trajectory quantitatively is to analyse the trajectory derived from a physical system will in fact stay within a
taken, using methods of time-series analysis and finite range, the concept of boundedness is impractical
dynamical systems theory. to use, and a related concept, that of stationarity is used
instead.
The second thesis argued that the essence of robot– A signal is defined as stationary if it shows
environment interaction is captured in the robot’s tra- similar behaviour throughout its duration. “Similar
jectory. The third thesis follows from that, making the behaviour”, in turn, is defined as having similar
assumption that measures from dynamical systems the- mean, standard deviation and autocorrelation structure
ory will identify the dominant features in a time-series, throughout the time series [1, p. 314]. To establish sta-
and can therefore be used to describe it quantitatively. tionarity, we divided the each of the measured time
If these theses are correct, the first and arguably most series into three equal parts, and tested the means for
important step towards a theory of robot–environment each third for significant differences, using the t-test. If
interaction is the quantitative description of robot the means of all three thirds do not differ significantly
trajectories. It was the purpose of the experiments (P < 0.01), then the signal is deemed to be stationary.
presented in this paper to achieve exactly that. An alternative method for establishing stationarity
is the non-parametric runs test, discussed in [2].

3. Characterising chaotic systems 3.2. Determinism

The main goal of the research presented in this As part of our procedure to establish the characteris-
paper, then, was to determine quantitative descriptions tics of the robot’s interaction with its environment, we
 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193 181

establish whether the time series (in our investigations tion dimension indicates whether data is aperiodic or
the robot’s trajectory) is deterministic, i.e. causally not, and to what degree: periodic data has a correlation
dependent on past events, or not. To do this, we use dimension of zero, chaotic attractors have a non-integer
the following method, described by Kaplan and Glass correlation dimension [1, p. 321].
[1, p. 324ff] (see also [3]).
The underlying assumption in determining whether 3.4. Sensitivity to initial conditions
the signal is deterministic or not is that in a determinis-
tic signal D of length 2T, the first half of the signal One of the most distinctive characteristics of a
should be usable as a “good” predictor for the 2nd chaotic system is its sensitivity to a variation in the
half—in a purely stochastic system this assumption system’s variables: two trajectories in phase space that
would not hold. started close to each other will diverge from one another
We therefore split the time series D into two halves as time progresses, the more chaotic the system, the
of length T each, and construct the three-dimensional greater the divergence.
embedding given in the following equation: Consider some state So of a deterministic dynamical
system and its corresponding location in phase space.
D(T + i) = [D(T + i), D(T + i − 1), D(T + i − 2)],
As time progresses the state of the system follows a
∀i = 3, . . . , T. (1) deterministic trajectory in phase space. Let another
state S1 of the system lie arbitrarily close to So , and
To make a prediction of D(tk + 1) (T < tk ≤ 2T), we follows a different trajectory, again fully determinis-
determine the closest point Dc (tc ) (0 < tc ≤ T) to D(tk ) tic. If do is the initial separation of these two states in
in Euclidean space, and select D(tc + 1) as the pre- phase space at time t = 0, then their separation dt after
diction of D(tk + 1). In this fashion all points of the t seconds can be expressed as dt = do eλt .
second half are predicted (one-step-ahead predictions Or, stated differently, consider the average loga-
only). rithmic growth of an initial error E0 (the distance
We then compute the mean squared prediction error |S0 − (S0 +
)|, where
is some arbitrarily small value)

. In order to decide whether this error is “large” or between two points in phase space [4, p. 709]. If
“small”, we set it in relation to a baseline prediction
b , Ek is the error at time step k, and Ek−1 the error
of simply using the average of the first half of the signal at the previous time step, then the average logarith-
as a prediction of the second. In a highly deterministic mic error growth can be expressed by the following
signal the ratio
/
b is close to zero, indicating that past equation:
data is a very good predictor of future data. A value of  
n

/
b close to (or exceeding) 1 indicates that the signal 1
 Ek 
λ = lim lim log  . (2)
is stochastic in nature, and predicting the mean is the n→∞ E0 →0 n Ek−1 
k=1
best prediction possible.
λ (which is measured in Hz or s−1 ) is known as the Lya-
3.3. Aperiodicity punov exponent. For an m-dimensional phase space,
there are m values for λ, one for each dimension. If any
The third main characteristic of a dynamical sys- one or more of those components are positive, then the
tem exhibiting chaos is that the state variables never trajectories of nearby states diverge exponentially from
return to their exact previous values: the trajectory each other in phase space, and the system is deemed
in phase space lies on a strange attractor. There is, chaotic. Since any system’s variables of state are sub-
however, variation from system to system in how ject to uncertainty, a knowledge of what state the system
close state variables return to previous values, and is in can quickly become unknown if chaos is present.
it is therefore desirable to quantify this degree of The larger the positive Lyapunov exponent, the quicker
“proximity”. knowledge about the system is lost. One only knows
The measure to quantify the degree of aperiodicity that the state of the system lies somewhere on one of the
is the correlation dimension ν of the attractor, which trajectories traced out in phase space, i.e., somewhere
is discussed in detail in Section 5.1.2. The correla- on the strange attractor.
182 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193

The Lyapunov exponent is one of the most use- Consider a sequential set of N + 1 measurements
ful quantitative measures of chaos, since it will reflect x(t), x(t + τ), x(t + 2τ), . . ., x(t + Nτ ), where the time
directly whether the system is indeed chaotic, and will between successive measurements is τ. If x(tn ) rep-
quantify the degree of that chaos. Also, knowledge of resents the x(t + nτ) time measurement then a delay
the Lyapunov exponents becomes imperative for any reconstruction of the strange attractor in m-dimensional
analysis on prediction of future states. phase space can be traced out by a series of lag vectors.
The practical aspects of estimating λ from a time Each lag vector can be written as
series are discussed in Section 5.2.
Dn = (x(tn − (p − 1)h), x(tn − (p − 2)h),
. . . , x(tn − h), x(tn )), (3)
4. Experiments and results
where h is the delay (or lag) time, which is usually some
4.1. Introduction whole multiple of τ. In such procedures, the two most
crucial decisions are the choice of h and the embedding
4.1.1. Experimental goals dimension p. A complete discussion of this approach
Most complex systems rarely present themselves is available in several books [1,10,11].
in a closed-form mathematical model from which This new spatial representation of the attractor is
one can deduce its dynamics. Experimental data must related to the ‘true’ representation of the system’s
be obtained which can be analysed to determine the states through a smooth change of coordinates. Hence,
nature of the dynamics. To verify that the system is those characteristics associated with the original spa-
chaotic the data must show, as previously mentioned, tial representation of the attractor are insensitive to
that the system’s dynamics are stationary, aperiodic, the reconstruction. This means the necessary informa-
deterministic, and sensitive to initial conditions. On tion describing the system’s attractor is still present
the surface this seems a formidable task that would for analysis in the new reconstruction. The correlation
require the measurement of numerous parameters. For- dimension and Lyapunov exponents mentioned above
tunately, techniques of dynamical data analysis have are two such characteristics which remain invariant in
been devised whereby a sequential time series of a this reconstruction.
single observable of the system is sufficient for com-
plete reconstruction of the system’s states. The only 4.1.3. Experimental procedure
condition is that the state of the system can be rep- In all experiments reported in this paper, we used a
resented as a function of the observed parameter. A Pioneer II mobile robot (see Fig. 3), executing a number
number of excellent books have been written over of different control programs in a number of different
state reconstruction via time series measurements, e.g.
[5,6,1].

4.1.2. Time series analysis

For linear systems, there are a variety of techniques
to analyse time series in order to find the associ-
ated dynamical models. For non-linear systems, which
include those suspected of chaos, the task is much
more thorny and requires the reconstruction of the sys-
tem’s states in phase space using a technique known
as delay coordinate embedding [7–9]. These recon-
structed states in phase space develop a structure which
is called, for chaotic systems, the strange attractor. The
geometric unfolding of the attractor is obtained from
lag vectors which have been generated from the mea- Fig. 3. The Pioneer II mobile robot used. Six of the robot’s eight
sured sequential time series. sonar sensors are visible at the front. The camera was not used.
 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193 183

environments. The Pioneer II is a fully autonomous

mobile robot, controlled by its on-board computer. The
robot is equipped with eight sonar sensors, mounted at
the front of the robot. The camera visible in Fig. 3 was
not used in the experiments reported here. The robot
was situated in a walled arena, walls were 25 cm high.
The arena was square with 2.5 m sides. Fitted to the top
of the robot was a super-bright LED to allow automated
trajectory logging using an overhead camera.
Directly above the centre of the walled arena and at
a height of 3 m was located a stationary video camera
whose field of view was adjusted to the arena’s border.
Fig. 4 shows an image of the experimental arena, taken
by the overhead camera. Fig. 4. View of the experimental arena, taken by the overhead camera
As the robot wandered in the arena its location was used for logging the robot’s trajectory. The robot is visible in the
recorded by capturing a frame of video and through bottom right hand corner.
image processing the pixel position of the LED on the
robot was determined. This pixel location was stored would create a time series of over 30,000 points. Longer
in a text file. The process was repeated at a frequency runs jeopardised the output of the robot’s power sup-
of 4 Hz. Hence, one obtains a sequential time series ply whereby the robots behaviour could start to be
of the robot’s x- and y-coordinates (see Fig. 5 for an affected. By keeping the battery within 0.2 V of its full
example). A run could last up to 3 h which at 4 frames/s charge, we ensured that the systems parameters were

Fig. 5. x- and y-coordinate of dataset 240601 (part).

184 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193

Fig. 6. ‘Billiard ball’ behaviour in square arena—entire trajectory (left) and 150 data points (right).

not drastically altered over the experiment and thus lying attractor (and therefore quantitative descriptions
were not a contributor to any changes in the robot’s of the robot’s interaction with its environment).
trajectory. The “size” of the arena was approximately The trajectory taken by the robot exhibiting this
384 pixels × 288 pixels. behaviour is shown in Fig. 6.
In order to exclude significant distortions of our
results due to data logging noise we conducted an 5.1. Reconstructing the attractor
analysis of image noise (logging a stationary robot for
about 6 min) for the four corners and the centre of the In order to reconstruct the attractor and perform
arena, and found that pixel jitter was in almost all areas subsequent analysis it is crucial to determine (i) the
between ±1 or ±2 pixels, and nowhere exceeded 4% of embedding time lag, h, and (ii) the embedding dimen-
the true value (this worst cast was a jitter of ±4 pixels). sion, p. The values associated with these two quantities
Variations in the experiment consisted of modifying will influence all subsequent analysis of the attractor.
the arena’s layout (the environment) and the behaviour Hence, for each we utilised two different methods of
of the robot (the task)—cp. Fig. 1. There were two calculation in order to affirm the values obtained.
tasks examined in this work: (i) wall following and (ii)
quasi-billiard ball behaviour. 5.1.1. Determining the embedding time lag h
Fig. 6 gives an example of the kind of trajectory To determine the correct embedding lag h we used
logged (dataset 240601), the dataset in that figure con- the following two methods:
tains just over 26000 data points, recording 109 min of
robot operation. Using those data points, we then gen- (1) The autocorrelation method presented in [1, p.
erated time series from the data, using the x-coordinate 353]: According to Kaplan and Glass “a good
of the trajectory versus time, or the y-coordinate versus choice for the embedding lag is [that value]
time, or both. Fig. 5 shows a section of 500 data points at which the autocorrelation function falls to
of the x- and y-coordinate of dataset 240601. e−l ≈ 0.37”. Fig. 7 shows the autocorrelation func-
tion for the x-coordinate of dataset 240601, it can
be seen that for h ≈ 29 the autocorrelation has
5. Detailed example: analysis of the billiard fallen below 0.37. We therefore select an embed-
ball behaviour (dataset 240601) ding lag of h = 29 to reconstruct the attractor of
dataset 240601.
We will now analyse the x-coordinate time series (2) The mutual information (MI) method: This proce-
of the 240601 dataset in detail (quasi-billiard ball dure is associated with non-linear statistics while
behaviour, see Fig. 5, top). This analysis will establish the autocorrelation method is based on linear statis-
whether (a) the underlying attractor is chaotic or not, tics. In essence, mutual information is analogously
and (b) provide quantitative descriptions of the under- a non-linear autocorrelation function [11]. MI has
 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193 185

Fig. 8. Mutual information of the x-coordinate of dataset 240601.

The first minimum gives a lag of h ≈ 29–30.
Fig. 7. Autocorrelation of the x-coordinate of dataset 240601. For a
lag of h ≈ 29 the autocorrelation has dropped below e−l . 5.1.2. Determining a suitable embedding
dimension (computing the correlation dimension)
As with the embedding time lag h, there were two
its origin from the treatment of chaos as a source of methods to determine the correct embedding dimen-
information [12]. The MI essentially measures the sion p:
degree to which knowledge of one measurement
determines another measurement h seconds later. (1) Correlation dimension procedure: It is known [7]
Fraser and Swinney [13] argue that one should that the dynamics of a reconstructed attractor are
take the first minimum that occurs in the mutual geometrically identical to the original dynamics
information function as the ideal time lag for con- (for both continuous and discrete systems), if the
structing an embedded attractor. This minimum in relationship p = 2ν + 1 holds (ν is the dimension
the MI function corresponds to the time separa- of the original attractor). In order to establish a
tion between two measured values so that there is suitable embedding dimension p to reconstruct the
a minimum of redundancy of information connect- attractor, it is therefore necessary to determine the
ing the two measurements in the time series. dimension ν of the original attractor. One common
measure of the dimension of an attractor is the cor-
To calculate the time lag, one uses an expression relation dimension ν.
based on Shannon’s entropy concepts [14]. The mutual The dimensionality of an attractor is related to
information I(τ) is given in the following equation: its aperiodicity: the more aperiodic the dynamics,
the greater the dimension of the attractor. In order



I(τ) = Pij (τ) ln Pij (τ) − 2 Pi ln Pi . (4) to measure how periodic a trajectory through phase
i,j i space is, one uses the correlation integral C(r),
defined in the following equation:
If one generates a histogram for the data distribu- θ
tion, then Pi is the probability a given measurement C(r) = , (5)
N(N − 1)
falls within the ith bin of the distribution. Pij (τ) repre-
sents the probability that if x(t) is in bin i then x(t + τ) where θ is the number of times that |Di − Dj | < r, i
is in bin j. τ is the time lag against which the mutual and j are two different times at which the embed-
information is computed, Fig. 8 shows the mutual infor- ding D is taken, and r is an arbitrary distance
mation curve for the x-coordinate of the 240601 dataset measure, the “correlation distance”. N(N − 1) the
versus τ. The first minimum of this curve is obvious maximum number of cases where |Di − Dj | < r
around τ ≈ 29–30, so again the time lag h is set equal is theoretically possible (the trivial case i = j is
to 29. excluded).
186 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193

The important measure is not the correlation

integral C(r) itself, but how this integral changes
with respect to r, i.e. the slope of the curve C(r)
versus r. This slope is the correlation dimension ν.
The mechanism to determine a suitable embed-
ding dimension to reconstruct the attractor is based
on these considerations. Obviously, the computa-
tion of the correlation dimension is dependent upon
the chosen embedding dimension p and the corre-
lation distance r. To compute both p and ν from
the same process is an ill-defined problem, and the
goal is to find a range of parameters p and r for
which ν is computed virtually identically (a so- Fig. 10. Ave correlation dimension vs. embedding dimension for
called “scaling region”). In other words, one aims dataset 240601, for scaling region between r = 103–127. Embedding
to find a region where the computation of the corre- lag h = 30. The computed correlation dimension is ν ≈ 1.9.
lation dimension ν is not critically dependent upon
the choice of embedding dimension p and correla- longer changes the computed correlation dimen-
tion distance r. sion. This is also visible from Fig. 10.
To find a scaling region, we plot the correlation Fig. 10 shows a “levelling off” of the com-
dimension ν as a function of correlation distance r puted correlation dimension ν for an embedding
for all embedding dimensions p between 1 and 10 dimension between 6 and 10. The computed cor-
(see [1, p. 323]). Fig. 9 shows ν versus correlation relation dimension ν at this point is ν ≈ 1.9. As
distance r for the x-coordinate of dataset 240601. stated above, Takens’ theorem [7] states that an
Choosing a large correlation distance r is some- embedding dimension p = 2ν + 1 will be adequate
what like looking at an object from a great dis- to reconstruct the attractor, indicating the embed-
tance: it will have dimension zero [1, p. 323]. The ding dimension we should use to reconstruct the
other extreme, choosing too small an r, will result attractor to be on the “safe side”. Therefore an
in signal noise being amplified, which is equally embedding dimension of p = 5 should be appro-
undesirable. Fig. 9 reveals that a scaling region priate for dataset 240601.
exists around r = 125, and shows that increasing (2) False neighbour method: An excellent presenta-
the embedding dimension above 6 in this region no tion of this method is in Abarbanel’s book [11].
It is based on the fact that if you have a multi-
dimensional graph and all points are collapsed onto
a single axis (say the x-axis) then it is impossi-
ble to determine if two nearby points on the x-
axis are true neighbours on the original graph. To
determine true neighbours one first measures the
distances between a point xo on the axis and all
of its nearby neighbours. Another axis is added.
Some of those nearby neighbours will unfold. Now
distances between xo and the original neighbours
are recalculated. If it is a true neighbour, the dis-
tance between the two will stay small. If it is a
false neighbour, the distance between the two will
increase significantly. One continues to increase
Fig. 9. Correlation dimension vs. correlation distance for dataset
the number of axes (the embedding dimension)
240601, for various embedding dimensions and various correlation until the percentage of false neighbours becomes
distances (h = 29). zero. Fig. 11 illustrates this for the 240601 dataset.
 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193 187

5.2. Estimating the Lyapunov exponent from a

time series

Determination of the Lyapunov exponents from a

time series such as the robot trajectories used in this
research or, more importantly, the existence and value
of any positive exponents, has been discussed exten-
sively in the literatures [15,4,10,11,1,16]. Because the
Lyapunov exponent is formally defined as the expo-
nential separation over time between two trajectories
in phase space, but only one time series is available, a
“second” trajectory has to be estimated from the one
time series. This is not a trivial problem, and a full
Fig. 11. Percentage of false neighbours for the x-coordinate of
dataset 240601. Zero false neighbours occurs at an embedding discussion can be found for instance in [4].
dimension of 5. Several academic and commercial software pack-
ages are available to compute Lyapunov exponents
from time series, for example [15,17,18]. In our anal-
It is clear from the figure that an embedding dimen- ysis we have used two of those methods, the method
sion of five should unfold the attractor. proposed by Wolf et al. [15], as well as Abarbanel’s
The time lag and the embedding dimension have commercially available software package [17] to deter-
each been determined by two entirely different mine the largest Lyapunov exponent. Fig. 13 shows an
methods. From these methods, it is established that example of the computation, applying the algorithm
a time lag h = 29 and embedding dimension p = 5 described in [15] to dataset 240601.
are needed to reconstruct the attractor underlying It can be seen that in this calculation after 100 itera-
the observed time series, using Eq. (3). tions of the algorithm the computed value converges to
The reconstructed attractor is shown in Fig. 12 λ ≈ 0.2 bit/s (note that results shown in Table 1 are aver-
(note that for graphical presentation obviously a age values over both Abarbanel’s and Wolf’s method,
three-dimensional embedding had to be chosen, applied to both the x- and y-coordinate data). Calcula-
but that for all calculations a five-dimensional tion of the same Lyapunov exponent using Abarbanel’s
embedding was used). software package results in λ ≈ 0.2 bit/s as well.
We are now ready to analyse the 240601 dataset
with respect to the four main characteristics of deter-

Fig. 12. Phase space reconstruction of robotic billiard ball experi-

ment (dataset 2), using the x-coordinate, embedding dimension 3 and Fig. 13. Computation of the Lyapunov exponent of the x-coordinate
an embedding lag of 29. of dataset 240601, using the method outlined by Wolf et al. [15].
188 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193

Table 1
Summary of experimental results
Behaviour Environment Set Embedding lag h Embedding dimension p λ̄ (bit/s) CD
Wall following Square arena (no obstacles) 140601 40 4 0.1 1.4
Billiard ball Square arena (no obstacles) 200601 30 5 0.2 1.9
240601 1.9
Billiard ball Square arena (centre barrier) 010701 20 5 0.2 1.6
040702 1.7
Billiard ball Square arena (off-centre barrier) 050701 20 5 0.2 ≈2.4

Table 2 the actual signal. As can be seen, the predictions of

Mean and standard deviation of the 1st, 2nd and 3rd third of dataset the second half of dataset 240601 follow the true
240601
signal so closely that the difference is hardly visible.
1st third 2nd third 3rd third
/
b = 0.046, which confirms clearly that 240601 is
Mean 199.2 198.9 197.9 a deterministic signal.
S.D. 65.8 64.0 65.2 • Aperiodicity. A correlation dimension of ν ≈ 1.9 was
obtained thereby indicating the signal is aperiodic
and the dynamics are chaotic.
ministic chaos: stationarity, aperiodicity, determinism
• Sensitivity to initial conditions. The estimation
and sensitivity to initial conditions.
of the Lyapunov exponent of dataset 240601 as
λ ≈ 0.2 bit/s, using the method described by Wolf
5.3. Dynamics of the robotic quasi-billiard ball et al. [15], is shown in Fig. 13.
experiment (dataset 240601)
The existence of a positive Lyapunov exponent and a
If the dynamics of this experiment are indeed non-integer correlation dimension confirm that dataset
chaotic, then the characteristics introduced in Section 240601 exhibits deterministic chaos.
3 must be shown to be true:
• Stationarity. The means of all three thirds of the time 6. Application: experiments and results
series for the x-component of the 240601 dataset are
presented in Table 2. They do not differ significantly Based on the hypothesis stated earlier that a robot’s
(t-test, P < 0.01). This confirms that the signal is sta- behaviour emerges through the interaction of robot,
tionary. task and environment (Fig. 1), and provided a quanti-
• Determinism. In Fig. 14, the first 500 data points tative description (such as Lyapunov exponent or cor-
of the data were used for making predictions. The relation dimension) exists, any experiment in which
second 500 points show two lines, the prediction and one of the three components of Fig. 1 is changed in

Fig. 14. Prediction of the x-coordinate of dataset 240601, using the method outlined by Kaplan and Glass [1, p. 326].
 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193 189

Fig. 15. Wall following behaviour of the pioneer robot in square arena—entire trajectory (left) and 100 consecutive positions (right).

a systematic manner would reveal the influence of for the wall following behaviour was λ̄ ≈< 0.1 bit/s,
that component upon the robot’s overall behaviour. We i.e. a very small number, indicating good predictability.
therefore conducted two sets of experiments: For the billiard ball behaviour the computation yielded
(1) Using the same robot, we let the robot perform λ̄ ≈ 0.2 bit/s, i.e. a considerably higher degree of
different tasks in the same environment, sensitivity to initial conditions, indicating a much
(2) Again using the same robot, we let the robot per- shorter prediction horizon. This result is entirely within
form the same task in different environments. our expectations, as the wall following behaviour
is considerably more orderly, and should result in a
6.1. Experimental scenario 1: same environment, longer prediction horizon. Under ideal circumstances
different tasks (for instance in simulation), wall following behaviour
should yield a Lyapunov exponent very close to
In the first set of two experiments, the robot operated zero, because the wall acts as a “physical attractor”,
in a square laboratory environment, containing smooth, resulting in predictable and repeatable trajectories.
light coloured walls. Apart from the boundary walls, no
other objects were present. This environment is shown 6.2. Experimental scenario 2: different
in Fig. 4. environments, same task
The first task of the robot was to act like an “active
billiard ball” (dataset 240601, Fig. 6), moving ran- In a second set of experiments, we let the Pioneer
domly through the environment and avoiding obstacles. II execute the “billiard ball” program in three differ-
The second task was that of wall following, performed ent environments: the square environment introduced
in the same environment, using the same robot (dataset in experimental scenario 1 (Section 6.1), the same envi-
140601, Fig. 15—see also Table 3). ronment with an additional central obstruction (dataset
Stationarity and determinism were confirmed for all 040702), and the same environment again with an
datasets. The average Lyapunov exponent computed off-centre obstruction (dataset 050701). The two tra-
Table 3
Datasets analysed in this paper
Robot behaviour Dataset No. of data points Run time (min)
Wall following 140601 13041 54
Quasi-billiard ball 200601 13005 54
Quasi-billiard ball 240601 26196 109
QB/ball with central obstruction 010701 12558 52
QB/ball with central obstruction 040702 29996 125
QB/ball with an off-centre obstruction 050701 32794 136
190 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193

Fig. 16. ‘Billiard ball’ behaviour in square arena with centre obstacle—entire trajectory (left) and 200 consecutive positions (right).

Fig. 17. ‘Billiard Ball’ behaviour in square arena with off-centre obstacle—entire trajectory (left) and 200 consecutive positions (right).

jectories obtained with obstructions present are shown The Lyapunov exponents given in Table 1 were the
in Figs. 16 and 17, the trajectory obtained when no averages of the four values obtained applying the meth-
obstruction is present is shown in Fig. 6. ods given in [15,17] to the logged x- and y-coordinate
Again, stationarity and determinism were confirmed respectively. Correlation dimensions were computed
first. The computed Lyapunov exponents were very using the methods of [1,17].
similar in all cases, and computed in all three cases as Calculations of dimensions of attractors are noto-
λ̄ ≈ 0.2 bit/s. This is a highly interesting observation, riously difficult, and correlation dimension values
as it demonstrates quantitatively that in our experiments given in Table 1 should be treated with healthy
the robot’s behaviour is governed much more by the scepticism. However, we obtained similar dimensions,
task the robot is executing than by the environment the irrespective of coordinate analysed, or method used.
robot is operating in. Furthermore, the false neighbour method gives
an independent estimate of a suitable embedding
dimensions, and (by means of Takens’ theorem) adds
7. Summary and conclusions credibility to the correlation dimensions shown in the
7.1. Experimental results table.
In all experiments we obtained positive Lyapunov
Table 1 gives an overview over the results of all exponents, and fractal correlation dimensions. This
experiments. leads us to the conclusion that the observed interac-
 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193 191

tion between robot and environment indeed exhibits est correlation dimension. This is to be expected. The
deterministic chaos. correlation dimensions of the other behaviours, how-
ever, should be interpreted with caution, this is subject
7.2. Interpretation of results to ongoing research.
7.2.1. Lyapunov exponents
The positive Lyapunov exponents obtained in all 7.3. Summary
experiments indicate that the attractors are sensitive to
initial conditions, and that two neighbouring trajecto- Mobile robotics research to date is largely based
ries in phase space diverge over time. This obviously on trial-and-error refinements, and the presentation of
means that any prediction of the robot’s trajectory will existence proofs. Replication and independent verifi-
only stay within a given error bound for a limited num- cation of experimental results are not common, and
ber of time steps. no reliable tools exist that allow the design of mobile
In the simplest experiment – wall following, in robot control programs before real-world experiments
which the wall effectively acted as a “physical attrac- are conducted.
tor” to the robot – unsurprisingly we obtained the low- In this paper, we argue that the main reason for
est Lyapunov exponent. If the robot’s behaviour was this situation is that no theory of robot–environment
changed to quasi-billiard ball behaviour, the Lyapunov interaction exists, and that therefore the development
exponent increased by a factor of 3. of such a theory should be pursued vigorously. This
An interesting observation here is that the Lyapunov argument is in agreement with a small, but growing
exponent appears to be much stronger correlated with number of researchers in the field (e.g. [23,24]).
the control program than with the environment in which Any theory must be based on quantitative measures,
the robot operates. Wherever the quasi-billiard ball in the case of mobile robotics quantitative measures
behaviour was executed, the same λ̄ of around 0.2 bit/s of robot–environment interaction. In this paper we
was obtained. This suggests that the sensitivity to initial demonstrate how dynamical systems theory (chaos the-
conditions is far more dependent on the control pro- ory) can be used to provide these measures. We were
gram being executed than on the environment in which able to show that robot–environment interaction can be
the robot operates. Because the control program is the chaotic, and to what degree. We were furthermore able
parameter the user has more control over, this is good to demonstrate quantitatively that the control program
news, and raises hopes that eventually it will be possible executed on a robot has more influence on the behaviour
to establish a theory of robot–environment interaction, than the environment in which the robot operates.
modelling this phenomenon more faithfully than is cur- We argued earlier that quantitative measures of
rently possible. behaviour form the bedrock on which a theory of
robot–environment interaction has to be founded. The
7.2.2. Correlation dimensions two measures introduced in this paper – Lyapunov
The (correlation) dimension of the attractor is exponent and correlation dimension – are just exam-
a measure of the periodicity and repeatability of ples of such quantitative measures. Obviously, they
the trajectory in phase space. Periodic trajectories only describe certain aspects of the robot’s dynami-
should have a correlation dimension of zero; the more cal behaviour, and will eventually have to be comple-
erratic (unperiodic) the trajectory gets, the higher the mented by additional quantitative measures.
dimension of the attractor. Chaotic attractors have
non-integer dimensions. 7.4. Future work
The computation of the correlation dimension is
difficult [19–22], and numerical results must be inter- There is a direct relationship between the Lyapunov
preted as estimates. The fractal dimensions we found exponent and the “predictability horizon”, that is that
confirm our findings that deterministic chaos is present length of time by which a predicted trajectory deviates
in our robot’s interaction with its environment. more than a defined threshold from the actual trajectory
It is perhaps interesting to note that the most periodic of the robot. One aspect of our future research, there-
of all behaviours – wall following – has indeed the low- fore, will be to investigate the relationship between that
192 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193

threshold (which is defined in physical space) and the [13] A. Fraser, H. Swinney, Independent coordinates for strange
Lyapunov exponent (which is a measure taken in phase attractors from mutual information, Phys. Rev. A 33 (1986)
1134–1140.
space).
[14] C. Shannon, W. Weaver, The Mathematical Theory of Informa-
Second, a theory of robot–environment interaction tion, University of Illnois Press, Urbana, IL, 1949.
will be manifest in a numerical model of some kind. [15] A. Wolf, J. Swift, H. Swinney, J. Vastano, Determining Lya-
Ongoing work at our laboratories investigates numeri- punov exponents from a time series, Physica D 16 (1995).
cal models of robot–environment interaction. Clearly, [16] H. Abarbanel, R. Brown, J. Sidorowich, L. Tsimring, The anal-
ysis of observed chaotic data in physical systems, Rev. Mod.
the quantitative measures of robot behaviour presented
Phys. 65 (1993) 1331–1392.
in this paper will be used to establish whether these [17] Applied Nonlinear Sciences, Last access May 2003. Tools for
numerical models are a faithful resemblance of robot Dynamics. http://www.zweb.com/apnonlin.
behaviour or not. [18] H. Kantz, T. Schreiber, Last access May 2003. TISEAN—
Third, how this approach scales up to more com- Nonlinear Time Series Analysis. http://www.mpipks-
dresden.mpg.de/tisean/.
plex robot behaviours is subject to ongoing research.
[19] J. Theiler, Estimating fractal dimension, J. Opt. Soc. Am. A 7
First results [25] suggest that the proposed methods do (1990) 1055–1073.
indeed scale up. [20] L. Smith, Intrinsic limits on dimension calculations, Phys. Lett.
A 133 (1988) 283–288.
[21] D. Ruelle, Comments on deterministic chaos: the science and
Acknowledgement the fiction, Proc. Roy. Soc. London A 427 (1990) 241–248.
[22] C. Essex, M. Nerenberg, Comments on ‘deterministic chaos: the
science and the fiction’ by D. Ruelle, Proc. Roy. Soc. London
We are grateful for support received from the British
A 435 (1991) 287–292.
Engineering and Physical Sciences Research Council [23] T. Smithers, On quantitative performance measures of robot
under grant GR/S30955/01. behaviour, in: L. Steels (Ed.), The Biology and Technology of
Intelligent Autonomous Agents, Springer Verlag, 1995.
[24] G. Schöner, M. Dose, A dynamical systems approach to task-
References level system integration used to plan and control autonomous
vehicle motion, Robot. Auton. Syst. 10 (1992).
[25] R. Iglesias, T. Kyriacou, U. Nehmzow, S. Billings, Pro-
[1] D. Kaplan, L. Glass, Understanding Nonlinear Dynamics,
gramming through system identification and training, in: A.
Springer-Verlag, London, 1995.
Borkowski, W. Burgard, P. Zingaretti (Eds.), Proceedings of the
[2] U. Nehmzow, Scientific Methods in Mobile Robotics—
Second European Conference on Mobile Robots, Ancona, 2005.
Quantitative Analysis of Agent Behaviour, Springer Verlag,
Berlin, Heidelberg, New York, London, 2006.
[3] M. Kennel, S. Isabelle, Method to distinguish possible chaos Ulrich Nehmzow is Reader in Analytical
from colored noise and to determine embedding parameters, and Cognitive Robotics in the Department
Phys. Rev. A 46 (1992) 3111–3118. of Computer Science at the University of
[4] H.-O. Peitgen, H. Jürgens, D. Saupe, Chaos and Fractals, Essex. He obtained his Diplom in Electrical
Springer Verlag, 1992. Engineering and Information Science from
[5] A.S. Weigend, N.A. Gershenfeld (Eds.), Time Series Prediction, the University of Technology at Munich
Addison-Wesley, 1994. in 1988 and his PhD in Artificial Intelli-
[6] D. Peña, G.C. Tiao, R.S. Tsay (Eds.), A Course in Time Series gence from the University of Edinburgh in
Analysis, Wiley–InterScience, 2001. 1992. He is a Chartered Engineer (CEng)
[7] F. Takens, Detecting Strange Attractors in Turbulence. Lec- and a member of the IEE. After a postdoc-
ture Notes in Mathematics, vol. 898, Springer-Verlag, Berlin, toral position at Edinburgh University he
1981. became Lecturer in Artificial Intelligence at the University of Manch-
[8] T. Sauer, J. Yorke, M. Casdagli, Embedol. J. Stat. Phys. 65 ester in 1994, and Senior Lecturer in Robotics at the University of
(1991) 579–616. Essex in 2001. His research interests comprise autonomous mobile
[9] E. Ott, T. Sauer, J.A. Yorke, Coping with Chaos, robotics, with emphasis on robot learning, robot navigation, simula-
Wiley–InterScience, 1994. tion and modelling and novelty detection for industrial applications
[10] H. Kantz, T. Schreiber, Nonlinear Time Series Analysis, Cam- of robotics. His current research focuses on scientific methods in
bridge University Press, 1997. mobile robotics, using chaos theory and dynamical systems theory
[11] H. Abarbanel, Analysis of Observed Chaotic Data, Springer- to obtain quantitative measures of robot–environment interaction.
Verlag, New York, 1996. Ulrich Nehmzow is the founder and co-chair of the British confer-
[12] R. Gallager, Information Theory and Reliable Communication, ence on mobile robotics (TAROS), member of the editorial board of
John Wiley and Sons, New York, 1968. Journal of Connection Science and the AISB journal, and member of
 U. Nehmzow, K. Walker / Robotics and Autonomous Systems 53 (2005) 177–193 193

the Engineering and Physical Sciences Research Council College for He is presently chair of that department. Dr. Walker has several
peer review. He is Secretary of the International Society for Adaptive active research interests. He has conducted extensive research into
Behaviour. electron–atom collision cross sections with particular emphasis on
the inert gases. National Science Foundation, Research Corpora-
Keith Walker received his MS degree in tion, and the Air Force Office of Scientific Research have supported
Physics from Ohio State University and this research. He has received research assignments with Wright-
his PhD in Physics from the University of Patterson Research Labs, the University of Arkansas, Geophysics
Oklahoma. He accepted a teaching position Laboratories at Hanscom Air Force Base, and the University of Wis-
in the Department of Physics at Southern consin. Since 1997, he has been involved in autonomous mobile
Nazarene University. During his tenure at robotics research. He is presently doing collaborative research into
SNU, he received a National Teaching Fel- describing the behaviour of mobile robots using chaos theory. Dr.
lowship and spent 1 year at the University Walker enjoys traveling with his wife, Maxine, and in his spare time
of Surrey in Britain. In 1986, he moved he crafts replicas of 18th century furniture in his woodworking work-
to San Diego to join the Physics Depart- shop.
ment of Point Loma Nazarene University.