Unconstrained Evolution and

Hard Consequences
CSRP 397
Adrian Thompson, Inman Harvey and
Philip Husbands

December 1995

To appear in: E. Sanchez and M. Tomassini, editors, Towards Evolv-

able Hardware , Springer-Verlag Lecture Notes in Computer Science,
1996. Presented at the workshop of the same name, EPFL, Lausanne,
2{3rd October 1995.
1
 Unconstrained Evolution and Hard
Consequences
Adrian Thompson, Inman Harvey and Philip Husbands
School of Cognitive and Computing Sciences,
University of Sussex, Brighton BN1 9QH, UK.
E-mail: adrianth, inmanh, philh @cogs.susx.ac.uk

Abstract. Articial evolution as a design methodology for hardware

frees many of the simplifying constraints normally imposed to make de-
sign by humans tractable. However, this freedom comes at some cost,
and a whole fresh set of issues must be considered. Standard genetic
algorithms are not generally appropriate for hardware evolution when
the number of components need not be predetermined. The use of sim-
ulations is problematic, and robustness in the presence of noise or hard-
ware faults is important. We present theoretical arguments, and illustrate
with a physical piece of hardware evolved in the real-world (`intrinsically
evolved' hardware). A simple asynchronous digital circuit controls a real
robot, using a minimal sensorimotor control system of 32 bits of RAM
and a few ip- ops to co-ordinate sonar pulses and motor pulses with
no further processing. This circuit is tolerant to single-stuck-at faults in
the RAM. The methodology is applicable to many types of hardware,
including Field-Programmable Gate Arrays (FPGA's).

1 Introduction
An evolutionary approach to hardware design makes possible the relaxation of
several constraints which other more orthodox techniques require. Human de-
signers conventionally need a prior rigorous analysis of the problem, and a de-
composition of a complex system into separate parts of manageable size; using
articial evolution this is not necessary. Simplifying design constraints are often
applied to hardware so as to make it behave in an easily analysable fashion |
for instance, strict synchronisation to a global clock. This is no longer necessary
with evolution, and such constraints can be relaxed.
However, this freedom comes at some cost; there are a whole new set of is-
sues relating to evolution that must be considered, and many of these will be
unfamiliar to those schooled in conventional design methods. Evolution of hard-
ware systems often cannot be tted into the constrained optimisation framework
which standard genetic algorithms assume. This means that these algorithms
need some crucial changes.
The main cost of an evolutionary approach is the large number of trials that
are required. Adequate simulations may take time comparable to doing the trials
for real, or may not be feasible; e.g. when vision in complex environments, or the
modelling of detailed semiconductor physics is involved, as will be shown later.

2
Under many circumstances robustness in the presence of noise or hardware faults
is a crucial factor, which can increase the number of trials needed. Noise is not
always a problem, indeed it may have advantageous evolutionary eects.
We discuss the constraints that can be relaxed and the hard consequences
that must be recognised, initially at a theoretical level. Then a real example of
evolved hardware will be presented in the light of these discussions. A simple
asynchronous digital circuit directly takes echo pulses from a pair of left/right
sonars, and drives the two motors of a real robot, so that it exhibits a wall-
avoidance behaviour in the real world. The complete sensorimotor control system
(no pre- or post-processing) consists of just 32 bits of RAM and a few ip- ops,
and is even tolerant to single-stuck-at faults in the RAM. The remarkable ef-
ciency of this circuit can be attributed to the facts that it was evolved as a
physical piece of hardware in the real world, and that many of the constraints
on its dynamics were under evolutionary control. The rationale behind this ex-
periment applies to many other kinds of system, including Field-Programmable
Gate Arrays (FPGA's) [2].
The paper proceeds thus: Sections 2{11 discuss various aspects of articial
evolution. Sections 12{14 cover issues of noise, the relationship between sim-
ulation and reality, and fault tolerance. Sections 15{17 discuss the theory of
Intrinsic Hardware Evolution. Sections 18 and 19 give a case study of a physical
piece of hardware, intrinsically evolved in the real world as a robot controller. A
nal section summarises the discussion.

2 Evolution not Design

Human beings nd it dicult to design complex systems, and the main heuris-
tic used to make design easier is \Divide and Conquer". The complex whole
is decomposed into smaller semi-independent parts or modules, which can be
tackled one at a time. For this to work the modules must have minimal interac-
tions between them, to allow any one to be tackled independently of the others.
However, there are many complex systems which either do not have any such
decomposition, or do not obviously show how they should be carved up.
If, however, some objective tness function can be derived for any given
complex system which is intended to carry out some specic task, there is the
possibility of automatic evolution of the system without explicit design. Natural
evolution is the existence proof for the viability of this approach, given appropri-
ate resources. Genetic Algorithms (GAs) [3] use ideas borrowed from evolution
in order to solve problems in highly complex search spaces, and by suitably ex-
tending GAs they can be used to search through design space whilst evading the
problems of decomposition faced by human designers.
The articial evolution approach maintains a population of viable genotypes
(chromosomes), coding for designs or architectures, which are inter-bred and
mutated according to a selection pressure. This pressure is controlled by a task-
oriented evaluation function: the better the system performs its task the more
ospring it has in succeeding generations. Ospring inherit genetic material from

3
their parents. Rather than attempting to hand-design a system to perform a
particular task or range of tasks well, the evolutionary approach should allow
their gradual emergence.

3 GAs for Optimisation

Here we will discuss the framework within which GAs are typically used, before
going on to suggest that for many hardware design problems a signicantly
dierent framework is needed.
The majority of published GA work, both applications and theoretical analy-
sis, refers to optimisation problems which can be seen as search problems in some
high-dimensional search space, of known (usually enormous) size. The compo-
nents to be optimised could be parameters which need to be set at appropriate
real values; or they could be discrete values which need to be chosen. In the
former case the real values needed are usually coded to some desired degree of
precision, so that they can be specied in binary form with a dened number of
bits (although some evolutionary algorithms work directly with real values).
What such optimisation problems share is the well-dened nite dimension-
ality of the search space. This allows the choice of some genotype coding, such
that a genotype (often binary) of xed length can encode any potential solution
within the space of possibilities. In the context of hardware design, this approach
is appropriate where the optimal attributes for a predetermined number of com-
ponents is to be found; also where the ordering of any given set of components
needs to be determined.
GA theory has in general assumed such a xed-dimensional search space.
The optimisation problem has typically been seen as starting with a population
of random points eectively spanning and coarsely sampling the whole search
space. Successive rounds of selection, reproduction and mutation are intended
to focus the population of sample points towards tter regions of the space,
homing in on an optimum or near-optimal region. Theorems such as the Schema
Theorem, intended to show the circumstances under which GAs can be expected
to produce the desired results, rely on these assumptions. One consequence of
this approach has been the primary reliance on recombination as the genetic
operator, which combines information from dierent samples in order to move
towards regions of expected higher tness; mutation is typically treated as a
background operator.
GAs can be considered a compromise between a weak and a strong form of
search. While far stronger than random search, which typically is infeasible in
large search spaces, GAs are not strong enough for one to be able to demonstrate
conclusively that the global optimum has been reached. Once recombination has
brought the population, over a number of generations, into a focused region of
search space | i.e. it is genetically converged | then with only background
rates of mutation further exploration cannot be expected, and search can be
terminated. If this convergence happens too rapidly, it implies that only a sketchy
sampling of the search space has been done, and any local optimum reached may

4
be far from the global optimum. Much GA analysis is directed towards setting
up the GA parameters so as to avoid this premature convergence.
However, some problems | including perhaps most hardware design prob-
lems | do not fall into this convenient picture of a xed-dimensional search
space. If there is no predetermined number of components to be used in the
design then standard GA theory will not apply.

4 GAs for Structure Evolution

There are at least two possible scenarios in which one might be uncertain initially
how many components might be needed. The rst case is incremental evolution:
where a sequence of increasingly complex tasks is posed, requiring the evolution
in succession of ever more complex hardware systems. The second is evolution
for parsimony: where the number of components is to be reduced to a minimum.
If there is no predened number of components in a structure to be designed
by GA, then any encoding of potential solutions onto a genotype will use an
amount of information which varies from case to case, given a xed interpretation
process. In other words, genotypes will have to be variable in length. But if the
genotype is potentially unbounded in length, then the nature of the search space
is such that it is impossible for an initial random population of nite size to
eectively sample from all parts of it. Any nite population inevitably spans
only a constricted region of the whole. Hence any GA search in an unbounded
space must work with a relatively converged nite population from the very start.
Possible ways in which systems of variable size can be encoded on a genotype
will be discussed below, but this convergence property holds regardless of how
the encoding is done.
Something very similar holds true if there is an upper bound to the number
of components. Suppose that a maximum of 20 components is allowed for some
hardware system, and an initial population contains sampling points to be eval-
uated with varying numbers of components between 0 and this maximum of 20.
Since the subspace of designs with 10 or less components is of a radically dif-
ferent nature from that with 20 components, samples from the former subspace
have no useful correlation in tness with corresponding points from the latter
(corresponding in the sense that the extra components have been added without
altering the existing ones). But the underlying theory of standard GAs relies on
there being some such correlation.
Evolutionary search can, however, operate in domains of varying dimension-
ality | indeed evolution in the natural world has done just that. Relatively
complex species, with lengthy genotypes, have evolved from simpler ancestors
with smaller genotypes, but a present-day animal should not be considered as
a solution to a problem posed 4 billion years ago, with a search space of xed
dimensionality.
GAs when applied to search spaces of varying dimensionality need a dif-
ferent framework from those used for standard optimisation problems. Species
Adaptation Genetic Algorithms (SAGA) were developed as this framework.

5
5 Species Evolution
In articial evolution problems of varying dimensionality one should expect to
have a genetically converged population, in eect a species , at all times. This
contrasts with optimisation problems, where convergence in a GA signals the end
of the search process. Using the example given above, where a hardware design
being evolved could potentially have any number of components up to 20, during
any single generation one should expect all the members of the population to
have the same or a very similar number of components, for instance 10. What
counts as similarity will be qualied later.
The conceptual framework of SAGA was introduced by Harvey in 1991 in
order to try to understand the dynamics of a GA when genotype lengths are
allowed to increase [4]. It was shown, using concepts of epistasis and tness land-
scapes drawn from theoretical biology [5], that progress through such a genotype
space will only be feasible through relatively gradual increases in information in
the genotype (typically, in genotype length). A general trend towards increase
in length is associated with the evolution of a species rather than global search.
Such evolutionary search in the space of hardware designs would be from ini-
tially simple designs for simple tasks, towards more complex designs for more
complex tasks; although in natural evolution there is no externally provided
sense of direction, in articial evolution this can be provided.
Throughout such articial evolution, a species will be relatively t, in the
sense that most members of the population will be tter than most of their
neighbours in the tness landscape. Evolutionary search can be thought of as
searching around the current focus of a species for neighbouring regions which
are tter (or in the case of neutral drift, not less t) while being careful not to
lose gains that were made in achieving the current status quo . In the absence of
any mutation (or change-length) genetic operator, selection will concentrate the
population at the current best. The smallest amount of mutation will hill-climb
this current best to a local optimum. As mutation rates increase, the population
will spread out around this local optimum, searching the neighbourhood; but if
mutation rates become too high then the population will disperse completely,
losing the hill-top, and the search will become random. If an ideal balance is
achieved between selective forces and those of mutation (as modied by recom-
bination), then some elements of the population can crawl down the hill far
enough to reach a ridge of relatively high selective values. As discussed in [6],
this results in a signicant proportion of the population working their way along
this ridge under selection, and making possible the reaching of outliers ever fur-
ther in Hamming-distance in that particular direction from the current ttest.
The term `ridge' is used here to t in with intuitive notions of tness landscapes;
in fact in high-dimensional search spaces such ridges may form complex neutral
networks, percolating long distances through genotype space.
If any such outliers reach a second hill that climbs away from the ridge, then
parts of the population can climb this hill. Depending on the dierence in tness
and the spread of the population, it will either move en masse to the new hill
as a better local optimum, or share itself across both of them.

6
 So in a SAGA setup of evolution of a converged species, we want to encourage
through the genetic operators such exploration along ridges to new hills, subject
to the constraint that we do not want to lose track of the current hill. Eigen
and co-workers use the concept of a quasi-species to refer to a similar genetically
converged population in the study of early RNA evolution. To quote from [6]:
In conventional natural selection theory, advantageous mutations drove
the evolutionary process. The neutral theory introduced selectively neu-
tral mutants, in addition to the advantageous ones, which contribute to
evolution through random drift. The concept of quasi-species shows that
much weight is attributed to those slightly deleterious mutants that are
situated along high ridges in the value landscape. They guide populations
toward the peaks of high selective values.

6 SAGA and Mutation Rates

Although progress of a species through a tness landscape is not discussed in
the standard GA literature, in theoretical biology there is relevant work in the
related eld of molecular quasi-species [7, 6]. In particular, analysis of the `error
catastrophe' shows that, subject to certain conditions, there is a maximum rate
of mutation which allows a quasi-species of molecules to stay localised around
its current optimum. Selection and mutation are opposing forces, the former
tending to increase numbers of the ttest members of the population, while the
latter tends to drag ospring down in tness away from any local optimum. A
zero mutation rate allows for no further local search beyond the current species,
and other things being equal increased mutation rates will increase the rate of
evolution. Hence if mutation rates can be adjusted, it would be a good idea to
use a rate close to but less than any critical rate which causes the species to fall
apart. A further possibility, in the spirit of simulated annealing, is to temporarily
allow the rate to go slightly above the critical rate | to allow exploration | and
then cut it back again to consolidate any gains thus made.
For an innite asexual population, in the particular context of molecular evo-
lution, Eigen and Schuster show [7] that these forces just balance for a mutation
rate
m = ln l
where l is the genotype length and  is the superiority parameter of the master
sequence (the ttest member of the population) | the factor by which selection
of this sequence exceeds the average selection of the rest of the local tness
landscape, and hence the rest of the population. The diagrams they show for
the very sharp cuto at the critical rate refer to a tness landscape with a single
`needle' peak for the master sequence, taking all the rest of the population to
be equally (un-)t; where the hill slopes more gently from the master sequence,
the cuto is less abrupt. For typical values of  between 2 and 20, the upper
limit of mutation before a quasi-species `loses its grip' on the current hill would
be between 0:7=l and 3=l. For nite population size, there is some reduction

7
in this critical mutation rate (the `error threshold') [8], but for genotypes of
length order 100, and populations of size order 100, the error threshold will be
extremely close to that for an innite population.
Since it is the natural logarithm of  which enters into the equation for m,
variations in  of an order of magnitude do not aect ln() (and hence the error
threshold) as signicantly as variations in genotype length. In conventional GAs,
choice of mutation rates tends to be a low gure, typically 0.01 or 0.001 per bit as
a background operator, decided upon without regard to the genotype length. The
SAGA framework means that mutation rates of the order of 1 per genotype are
required when using linear rank selection or tournament selection as discussed
below, subject to some qualications concerning recombination and `junk DNA'.
These qualications tend to increase the recommended rate to somewhere in
the range 1 to 5 mutations per genotype, the idiosyncratic nature of tness
landscapes for dierent problems making it dicult to be more specic.
When applying such mutation rates in a GA, it is essential that the prob-
ability of mutation is applied independently at each locus on the genotype.
This gives a binomial distribution (approximating a Poisson distribution for
long genotypes) for the number of mutations per string, so that genotypes with
an expected m mutations have this as the average value with a wide variance
(including the possibility of zero mutations).

7 Neutral Sequences and Drift

Mutations in a genotype encoding a t phenotype are often deleterious, and
occasionally advantageous. There is a third possibility, that a mutation is neutral
and leaves the tness unchanged.
Neutral mutations can in turn be subdivided into two kinds, with a rather
grey area between them. They can be in parts of `junk DNA', such that the
decoding of the genotype ignores the values in that part. In this case it is only
the functional part of the genotype (the part which is capable of causing some
dierence in the tness) that counts towards eective genotype length when
deciding upon mutation rates. For example, if a genotype of length 1000 is 90%
junk, then a mutation rate set at the rate of one per eective genotype length
should be implemented at the rate of 1/100 per locus, rather than 1/1000. It
is often dicult to estimate what proportion of a genotype is junk, however, as
this shades into the second class of neutral mutation.
This second type of mutation may leave the phenotype unchanged, yet open
the possibility of a further mutation making some dierence. As a simple exam-
ple, a binary genotype with two loci, whose tness is given by the logical AND
of the alleles at each locus, retains a tness of 0 during mutation from 00 to 01;
yet this opens up the possibility of a further single point mutation reaching 11,
with a tness of 1 which was not achievable from the starting point. Such neutral
mutations can in a high-dimensional space allow extended neutral paths which
can percolate through vast areas of sequence space. Neutral drift of a population
through such pathways means that it is much more dicult to get stuck on a

8
local optimum than one's intuition based on 3-D landscapes might lead one to
think. In addition, the percolation of such paths through sequence space tends
to mean that it does not matter too much where in sequence space a converged
population starts; under many circumstances it is possible to reach all possible
t regions from most starting points.
The SAGA selection and mutation rates encourage just such exploration
through neutral drift in sequence space.

8 Selection
Selective forces need to be maintained at the same level throughout an evolu-
tionary run, so as to balance mutational forces and maintain a similar degree
of genetic convergence throughout. Basing selection directly on absolute tness
values does not achieve this, and some system based on ranking of the population
must be used. This implies that the ttest member of the population has the
same expected number of ospring whether it is far better than the rest, or only
slightly better. Truncation selection (reproducing only from a top slice of the
population) is one way of achieving this, but generally is too severe in restrict-
ing exploration by the less t mutants. Less severe methods are recommended
such as linear ranking; for instance giving the top ranker twice the average ex-
pected number of ospring, and reducing this amount linearly as one goes down
the ranks, towards zero.
One way to achieve an eect comparable to linear ranking in a steady state
GA is through tournament selection. Rather than replacing the whole population
by a similar number of ospring at each generation, only one new ospring at a
time replaces a fatality in an otherwise unchanged population. Two parents for
the ospring can each be chosen by picking the ttest of a randomly picked pair
(the tournament), and the fatality chosen at random from the whole population;
alternatively, the parents can be picked at random from the whole population,
and the fatality selected as the loser of a tournament.
Elitism is often advocated in GAs when used for optimisation. This is the
requirement that the current ttest member of the population is never deleted to
make way for another that is less t. In real world applications such as hardware
design, however, evaluations are likely to be noisy. Since in this case one can
never be certain which is the ttest, elitism cannot be relied upon.

9 Recombination
With a genetically converged population, sections of genotype that are swapped
in recombination are likely to be fairly similar. With species evolution, recombi-
nation does not have the prime signicance it has in standard GAs | asexual
evolution is indeed feasible | but nevertheless it is a useful genetic operator.
There are two r^oles recombination has which are opposite sides of the same
coin. On the one hand, it allows two fortunate mutations which happen to have

9
occurred independently in two dierent lineages within the population to be
combined into one which has both: something not possible with asexual repro-
duction. On the other hand, it allows parents with a detrimental mutation to
produce an ospring which does not have it: also impossible asexually, in the
absence of highly improbable back-mutations. This latter eect in general allows
higher mutation rates to be used with recombination than were suggested above
for asexual populations, thus promoting exploration without risking loss of a
currently achieved local optimum.
Recombination is particularly powerful when combined with a distributed
GA. Here each member of the population is allocated a dierent position in
some notional geographical space, often a two-dimensional toroidal grid. Re-
combination between individuals is only allowed for pairs within a certain dis-
tance of each other on this grid, which thus comprises a number of overlapping
neighbourhoods. This combines the virtues of small and large populations; small
interrelated local populations allow through random drift more extended search
through genotype space, but the overlapping nature of such localities means that
any improvement found percolates through the whole population.
Recombination can run into problems with genotypes of diering lengths; it
may not be clear, given a crossover or recombination point in one parent, where
a corresponding crossover should be made in the other parent. Whatever system
is used should ensure that homologous segments of the genotype are swapped.
Often this may need domain-specic GA program code; a general algorithm for
long binary strings is given in [9].

10 Change in Genotype Length

If systems of dierent complexity are encoded by genotypes of dierent length,
then the genetic operators must allow changes in genotype length to take place
in the process of going from parent to ospring. How this is done will neces-
sarily depend to a large extent on the form of genotype encoding. One lesson
spelt out in [4] is that any genetic operator which allows a change in genotype
length should be restricted to small changes only | to be precise, to changes
which in general can be expected to produce a small phenotypic change. This
last qualication allows for genetic operators such as gene duplication where the
encoding used means that such duplications are neutral. Neutral gene duplica-
tion, followed by mutations in one of the copies, is a potentially powerful method
for creating variants on useful substructures.

11 Morphogenesis
The preceding discussion has been deliberately general insofar as very few con-
straints have been laid down as to how the genotype should encode for the system
to be evolved. To some extent such an encoding is always domain-specic, but
nevertheless there are some general rules that come into play as a system grows
more complex.

10
 With simple designs where there are no symmetries or repetitions to be
expected, then the straightforward method is for the genotype to encode in
sequence the type and parameters of each component part, together with the
interconnections between such parts. However, as systems become more complex,
then symmetries and repetitions can be expected to play a signicant r^ole in
many circumstances. For instance, when designing a control system for a robot
with bilateral symmetry, then there may well be good reason to expect this
bilateral symmetry to be re ected to some extent in the controller. If designing
an articial retina, then repetitions of similar local structure across a 2-D array
can be expected.
If the genotype constitutes in eect a linear description of each component
of the system in turn, with for instance the left and right halves separately so
described, then bilateral symmetry can only be achieved by separate independent
evolution of each half of the genotype towards the same target. In a stochastic
process such as evolution this is improbable and dicult; however it could be
achieved relatively simply if the genotype described just one half, together with a
routine which `called' the description twice with appropriate parameters, in the
sense that a program can call a sub-routine. In the case of multiple repetitions
as in a retinal array, such a process becomes even more ecient.
Earlier it was suggested that evolution was a method of avoiding the con-
straints of human design, which seems to require the decomposition of a system
into semi-independent modules. The repetitions and symmetries now being dis-
cussed dier from such human decomposition in two critical ways. Firstly, any
such decomposition can emerge from evolutionary choice, rather than being pre-
determined by fallible human prejudice; and secondly, such repeated modules can
be intimately connected with each other, as with neighbourhood relationships
on a retina, and need not be semi-independent.
In the natural world the genotype does not constitute a description of the
organism. Instead it acts as a constraint on the way in which a multi-cellular
organism develops from a single cell, in such a fashion that symmetries and rep-
etitions can naturally emerge. Attempts to replicate such emergence in articial
evolution are currently ad hoc and domain-specic.

12 The Use of Noise

Stochastic noise can be actually be advantageous to the evolutionary process.
Such noise alters the behaviour of the hardware, as compared to its behaviour
in some idealised non-noisy world; hence it alters the tness of this hardware
at the task on which it is being evaluated. These alterations are in general such
as to blur the tness landscape, to decrease the dierence in behaviour (and
tness) between two pieces of hardware which are neighbours in genotype space.
Rugged regions in the tness landscape tend to get blurred into more rolling
hills, on which the evolutionary process nds progression towards higher tness
much more easy.

11
 This eect is related distantly to that of stochastic resonance. The arguments
are similar to those for the `Baldwin eect' [10]. Practical examples of this phe-
nomenon can be seen in [11, 12]. The r^ole of noise in smoothing the transition
from simulations to reality is under investigation [12].

13 The Use of Simulation

Articial evolution requires the evaluation of members of a population over the
course of many generations. Virtually all the time and expense is in performing
these evaluations rather than in the genetic operations; optimising these latter
operations has little practical use. The number of evaluations may be reduced
by setting up the GA appropriately. Each individual evaluation should also not
be unnecessarily long.
Simulations are often seen as attractive, oering the possibility of performing
evaluations in faster than real time. Such simulations would need to be validated
by testing evolved architectures in realised form at regular intervals. However
there are a number of possible problems, depending on what precisely is to be
simulated. Consider in turn the cases where simulations are of just the hardware,
of both hardware and environment, and of just the environment.
Simulation of the hardware alone might be attempted if appropriate recong-
urable hardware was not available | simulations are likely to run more slowly
than the real hardware. This is `extrinsic' evolution, as opposed to `intrinsic'
evolution to be discussed later. Often detailed simulations of physical properties
of hardware are too computationally expensive to be practical; evolution can
then only be eective with the real hardware.
When both the hardware and the environment are to be simulated, it is
frequently the latter that is much more complex. Often such simulations are
simply not practical. For instance, when evolving control systems for visually
guided robots [11], the visual world of a robot takes so much computing power
to adequately simulate that testing in the real world is simpler and faster.
Consider nally the use of real hardware in a simulated environment. It has
been suggested by de Garis [13] that where a hardware device such as a robot
controller must be evaluated within some environment, then an environment
simulation could be implemented in electronics situated next to the evolving
hardware control system on a VLSI chip. It is suggested that this will allow the
speeding up of each evaluation by perhaps many orders of magnitude. There are
two serious doubts to be raised about this proposal.
The rst is scepticism about the complexity of the simulation. For many real
tasks (and that of a robot controller in a human environment is one example) the
complexity of those crucial aspects of the environment which must be adequately
simulated makes it infeasible; e.g., when the environment includes other entities
of a complexity and speed comparable to that of the hardware itself.
A second doubt arises when an adequate evaluator circuit running faster than
real-time can be built (perhaps possible for simpler situations than robotics). The
same hardware that has been evolved within a high-speed environment must then

12
adjust to cope with normal timescales in the real world. This could be done with
clocked hardware, by simply adjusting the clock-rate. However, the imposition
of adjustable clocking on a piece of hardware is a severe constraint, limiting the
range of dynamics available. Later sections will show the benet of removing
such constraints. More subtle ways of controlling timescales may be possible,
but only those aspects of the dynamics which are controlled may be exploited
during accelerated evolution. Thus, for this technique to be useful, any benets
from the increased rate of evolution must outweigh the costs of constraints. Note
that real-time simulations are not prone to this diculty.

14 SAGA and Fault Tolerance

When considering the mutation rate for SAGA (Section 6), we saw that there is
an `error catastrophe' mutation rate above which the species disintegrates and
loses its local tness peak. For practical SAGA mutation rates, the population
will never completely converge upon the single locally-most-t sequence, but
will converge around it, in an equilibrium of being driven away by mutation and
pulled back by selection. Most of the individuals will not have the locally-most-
t genotype, but will be a small number of mutations away in genotype space.
This has the eect that selection is not able to hold the population at an isolated
`needle in the haystack' peak as well as it can hold it on a `smoother' hillside,
where tness falls away gradually with increasing Hamming distance from the
peak. In fact, Thompson [14] adapts one of Eigen & Schuster's experiments
[15] to SAGA, illustrating that an initially completely converged population can
abandon an isolated peak in favour of a less t smoother one.
The result is that, under certain conditions, the evolving species tends to
move to a high tness region of the landscape at which mutations have only
a gradual eect on tness on average (if such a region exists). In particular,
most single mutations will tend not to decrease tness dramatically, although a
few critical ones might. What does this have to do with fault tolerance? Take an
example: say that part of the genotype directly encodes the connectivity between
some components of the phenotype circuit, with a 1 indicating a connection and
a 0 no connection. The preceding argument predicts that single mutations will
tend to have only small eects on tness, implying that the performance of the
circuit will tend not to be drastically degraded by the creation or removal of
connections. Therefore the evolved circuits will tend to be tolerant to hardware
faults that cause connections to be broken or spuriously created.
This general phenomenon applies whenever the genetic encoding is such that
a genetic mutation has the same eect on the phenotype as would a certain type
of fault; there will be a tendency for the evolved systems to be less sensitive
to that type of fault than equivalent systems produced by non-evolutionary
means. Current research aims to characterise the eect using the NK model
of tness landscapes [16] over the full range of possible SAGA parameters and
selection schemes: for some settings, evolved individuals have been seen to be
10% less sensitive to single mutations than equivalent individuals found through

13
exhaustive search, on average. Thompson [14] deals with the evolution of fault
tolerance in greater detail than here, also suggesting that the use of a co-evolving
antagonistic population of emulated faults could force tolerance to a large set of
faults in cases where the above technique is inapplicable or insucient.

15 The Relationship Between Intrinsic Hardware

Evolution and Conventional Design Techniques
Intrinsic 1 evolution of hardware means that the individuals of the evolving pop-
ulation receive tness scores according to their performance when instantiated
as real physical pieces of electronics. Evolution proceeds by taking note of the
overall behavioural eect of variations made to the real circuits; this is very
dierent from conventional design techniques, which proceed by manipulating
abstract models.
The use of abstract models simplies design by allowing some aspects of
reality to be ignored, but the properties of the real hardware that have been
`abstracted away' must be suppressed: they must not be allowed to in uence
the nal behaviour of the designed circuit. For example, a designer engaged
in digital design does not need to think about the analogue behaviour of the
transistors, but considers them as ON/OFF switches. To allow circuits designed
at this level of abstraction to work in reality, the transistors must always be
kept in either the ON state or the OFF state except during short transient
periods while they are switching between them. Steps must be taken to ensure
that these transients do not in uence the overall behaviour of the system as
predicted by the designer's digital model; this is manifest in the use of a global
clock in synchronous design, and the more local co-ordination mechanisms of
asynchronous design methodologies. Hence, the use of an abstract model at the
design stage imposes constraints on what circuits can be produced, in order to
ensure that the model is valid.
Intrinsic hardware evolution, by observing the consequences of variations
made to the real hardware, avoids the need for design abstractions and the ac-
companying constraints. Our notion of the nature of electronic systems is heavily
biased by our design methodologies and the constraints applied to facilitate their
abstractions, so evolvable hardware demands a radical rethink of what electronic
circuits can be. Both the spatial structure (modularity) and the temporal struc-
ture (synchronisation and the r^ole of phase in general) need to be considered.

15.1 Unconstrained Spatial Structure

We saw above that with digital design, care must be taken to prevent switch-
ing transients (a feature absent from the designer's model) from aecting the
system's overall behaviour. This is done by making sure that one part of the
system does not in uence the rest until any transient dynamics have died down
1
This term is due to Hugo de Garis.

14
and a steady state is reached (all transistors stable in the ON or OFF states).
Usually, this means that the circuit is broken into modules, the internal tran-
sient dynamics of which are hidden from each-other. (Here, the word `module' is
used in a general sense to mean a cohesive sub-assembly within a larger system.)
The spatial or topological structure of the circuit has thus been constrained to
facilitate a design abstraction.
Even if recongurable hardware intended for use by digital designers is used
(eg. most current FPGAs) then design abstractions, such as the digital model,
are not required for intrinsic hardware evolution. Is modularity,then, a designer's
constraint that can be abandoned, or is it necessary or useful for all complex
systems whether designed or evolved? Are the kinds of modules appropriate for
an evolving circuit dierent from those used by a designer?
These questions are currently dicult to answer fully. Certainly, we have seen
that evolution does not need modular structures to support abstract designer's
models, because intrinsic evolvable hardware (hereafter `intrinsic EHW') does
not use such models. However, the modularity of a system can also be caused
by the nature of the problem-solving or adaptive process that derived it. Hu-
mans typically use some sort of \divide and conquer" strategy, by which the
problem is successively decomposed. Whether the decomposition is a functional
one or a behavioural one [17], the nal structure arrived at usually has modules
corresponding to that decomposition. Wagner [18] argues that the evolutionary
process also requires a kind of modularity: that there should be an \independent
genetic representation of functionally distinct character complexes." The idea is
that such a genotype-phenotype mapping prevents small mutational variations
applied at one point from having large-scale ramications throughout the whole
phenotype, so that parts of it can be improved semi-independently. However, it
is not clear to what extent this consideration necessarily implies modules in the
structure of an EHW circuit, because the \distinct character complexes" of the
phenotype are components of the behaviour of the circuit, not of its physical
organisation2. A related r^ole for modules in an evolved circuit was given above
(Section 11) when considering the part which repeated structures and symme-
tries can play in the morphogenesis and evolution of complex systems.
It seems that if modular circuits are desirable in EHW, it is because modu-
larity may aid the evolution of complex systems, rather than because modularity
is essential to the operation of a complex electronic circuit. For this reason, the
kinds of modules appropriate to EHW will be tuned to the characteristics of
the evolutionary process (particularly the genetic encoding and morphogenetic
development) and the way in which this interacts with the detailed properties
of the particular type of recongurable hardware being used. It remains to be
seen what such modules may look like, but the important message is that they
may be radically dierent from what is seen in circuits produced by traditional
design methods.
2
For example, a character complex might inhere in a particular basin of attraction of
the system, which could be a property of the whole circuit.

15
15.2 Unconstrained Temporal Structure
Real physical electronic circuits are continuous-time dynamical systems. They
can display a broad range of dynamical behaviour, of which discrete-time sys-
tems, digital systems and even computational systems are but subsets. These
subsets are much more amenable to design techniques than dynamical electronic
systems in general, because the restrictions to the dynamics that each subset
brings support design abstractions, as described above. Intrinsic EHW does not
require abstract models, so there is no need to constrain articially the dynamics
of the recongurable hardware being used.
In particular, there no longer needs to be an enforced method of control-
ling the phase (temporal co-ordination) in recongurable hardware originally
intended to implement digital designs. The phase of the system does not have to
be advanced in lock-step by a global clock, nor even the local phase-controlling
mechanisms of asynchronous digital design methodologies imposed. The suc-
cess of pulse-stream neural networks [19, 20], where analogue operations are
performed using binary pulse-density signals, gives a clue that allowing the sys-
tem's phase to unfold in real-time in a way useful to the problem at hand can
add a powerful new dimension to electronic systems: time. Mead's highly suc-
cessful analogue neural VLSI devices (eg. the `silicon retina') [21], exploiting the
continuous-time dynamics of networks of analogue components (with the tran-
sistors mostly operating in the sub-threshold region), show how protable an
excursion into the space of general dynamical electronic systems can be.
In some applications, dynamics on more than a single timescale are needed
in an EHW circuit. For example, a real-time control system needs to behave on a
timescale suited to the actuators (typically in the range milliseconds to seconds),
while the underlying dynamics of the controller's electronic components might be
measured in nanoseconds. Being able to have dierent parts of a circuit behaving
at dierent timescales can also be signicant in other ways; indeed, learning can
be thought of as a dynamic on a slower timescale than individual task-achieving
behaviours.
There are several ways in which high-speed electronic components can give
rise to much slower behaviour:
{ The phase can be governed by one or more external signals, a digital clock
being the prime example.
{ Large time-constant resources can be provided. Large capacitors or induc-
tors cannot be made in VLSI, so either external components can be made
available, or special techniques can be used to make the most of smaller
on-chip components [22].
{ The high-speed components can somehow be assembled to give rise to slower
dynamics, without explicitly providing large time-constant resources or slow-
speed clocks.
Is the last of these possibilities feasible in an EHW framework? To nd out,
a simulation experiment was performed to see if a network of high-speed logic
gates could be evolved to oscillate at a much slower timescale. The number of

16
logic nodes available was xed at 100, and the genotype determined which of the
boolean functions of Table 1(a) was instantiated by each node, and how the nodes
were connected. The nodes were analogous to the recongurable logic blocks of
an FPGA, but an input could be connected to the output of any node without
restriction. The linear bit-string genotype consisted of 101 segments (numbered
0::100 from left to right), each of which directly coded for the function of a node
and the sources of its inputs, as shown in Table 1(b). (Node 0 was a special
`ground' node, the output of which was always clamped at logic zero.) This
encoding is based on that used in [23]. The source of each input was specied by
counting forwards/backwards along the genotype (according to the `Direction'
bit) a certain number of segments (given by the `Length' eld), either starting
from one end of the string, or starting from the current segment (dictated by
the `Addressing Mode' bit). When counting along the genotype, if one end was
reached, then counting continued from the other.

Name Symbol
Bits Meaning
BUFFER 0-4 Junk
NOT 5-7 Node Function
AND Pointer to First Input
OR
8 Direction
9 Addressing Mode
XOR 10-15 Length
NAND Pointer to Second Input
NOR 16 Direction
17 Addressing Mode
NOT-XOR 18-23 Length
(a) (b)
Table 1. (a) Node functions, (b) Genotype segment for one node.

At the start of the experiment, each node was assigned a real-valued propa-
gation delay, selected uniformly randomly from the range 1.0 to 5.0 nanoseconds,
and held to double precision accuracy. These delays were to be the input-output
delays of the nodes during the entire experiment, no matter which functions the
nodes performed. There were no delays on the interconnections. To commence
a simulation of a network's behaviour, all of the outputs were set to logic zero.
From that moment onwards, a standard asynchronous event-based logic simu-
lation was performed [24], with real-valued time being held to double precision
accuracy. An equivalent time-slicing simulation would have had a time-slice of
10 24 seconds, so the underlying synchrony of the simulating computer was only
manifest at a time-scale 15 orders of magnitude smaller than the node delays,
allowing the asynchronous dynamics of the network to be seen in the simula-
tion. A low-pass lter mechanism meant that pulses shorter than 0.5ns never
happened anywhere in the network.

17
 The objective was for node number 100 to produce a square wave oscillation
of 1kHz, which means alternately spending 0:5  10 3 seconds at logic 1 and at
logic 0. If k logic transitions were observed on the output of node 100 during
the simulation, with the nth transition occurring at time tn seconds, then the
average error in the time spent at each level was calculated as :
k 
average error = k 1 1
X 
 (tn tn 1) 0:5  10 3 
(1)
n=2

For the purpose of this equation, transitions were also assumed to occur at
the very beginning and end of the trial, which lasted for 10ms of simulated
time. The tness was simply the reciprocal of the average error. Networks that
oscillated far too quickly or far too slowly (or not at all) had their evaluations
aborted after less time than this, as soon as a good estimate of their tness had
been formed. The genetic algorithm used was a conventional generational one [3],
with elitism and linear rank-based selection. At each breeding cycle, the 5 least
t of the 30 individuals were killed o, and the 25 remaining individuals were
ranked according to tness, the ttest receiving a fecundity rating of 20:0, and
the least t a fecundity of 1:0. The linear function of rank dened by these end
points determined the fecundity of those in-between. The ttest individual was
copied once without mutation into the next generation, which was then lled
by selecting individuals with probability proportional to their fecundity, with
single-point crossover probability 0.7 and mutation rate 6:0  10 4 per bit.3
Fig. 1 shows that the output of the best individual in the 40th generation
(Fig. 2) was approximately 4 21 thousand times slower than the best of the ran-
dom initial population, and was six orders of magnitude slower than the propa-
gation delays of the nodes. In fact, tness was still rising at generation 40 when
the experiment was stopped because of the excessive processor time needed to
simulate this kind of network. This result suggests that it is possible for evolution
to arrange for a network of high-speed components to generate much slower be-
haviour, without having to provide explicit `slowing-down' resources with large
time constants, and without the need for a clock (though these could still be
useful).
The evolved oscillators produced spike trains rather than the desired square
wave. Probing internal nodes indicated that this was because beating between
spike trains of slightly dierent frequencies was being used to generate a much
lower frequency; beating only works for spikes, not for square waves.4 This does
not mean that the task was easy: it is dicult for beats to reduce the frequency
by the massive factor required and yet produce an output as regular as that seen
in Fig. 1.
3
This per-bit mutation rate was crudely derived by trial and error, but turns out to
be on the order of one signicant mutation per genotype for the nal evolved circuit,
in line with SAGA theory.
4
Of course, the output spike train could be converted into a square wave by passing
it through a toggling ip- op, though this did not evolve here.

18
 logic ‘1’
~
~ 18MHz
logic ‘0’
Two millionths of a second

logic ‘1’
~
~ 4kHz
logic ‘0’
Two thousandths of a second

Fig.1. Output of the evolving oscillator. (Top) Best of the initial random population of
30 individuals, (Bottom) best of generation 40. Note the dierent time axes. A visible
line is drawn for every output spike, and in the lower picture each line represents a
single spike.

30

74

52
16
87
35 98 10

5
2

72 86
53
58
91 17 62

51 24
25 34
78 15 44

89

85
92 57 75 47 61
55

4 3
67

Output
23 100
66 68

56 41 1 42
11
69
79 21 9

19

94

81 37
71
83 90 38 13
48 45
46
64
8

43 36

97

32 33
12

Fig.2. The evolved 4kHz oscillator (unconnected gates removed, leaving the 68 shown).

The simulation was quite an unrealistic model of the evolution of the con-
guration of a real FPGA. No analogue eects were modelled apart from time,
but they would be a big part of the way a real chip would behave. Nevertheless,
the result lends strength to the concept of evolving unconstrained dynamical
systems, because beating evolved: a highly eective solution which is essentially
a continuous-time phenomenon. Beating does not just occur at one node but

19
throughout the network, which does not have any signicant modules5.

16 Exploiting the Hardware versus Tolerance to Device

Variations
In the evolved oscillator experiment of the previous section, each node had a
slightly dierent input ) output time delay, crudely modelling the propaga-
tion delays of the recongurable blocks of an FPGA. The delays were initially
randomly chosen but then held xed during the experiment. If the delays of
the nal evolved circuit were re-randomised then the behaviour was totally de-
stroyed, and the circuit was no better than a randomly generated one: it relied
on the particular time delays present during its evolution. Extrapolating to real
intrinsic hardware evolution, it can be expected that all of the detailed physics
of the hardware will be brought to bear on the problem at hand: time delays,
parasitic capacitances, cross-talk, meta-stability constants and other low-level
characteristics might all be used in generating the evolved behaviour.
The exploitation of all of the hardware's physical properties must be traded
against sensitivity to variations in them. Some tolerance is essential because of
the inevitable changes over time due to uctuations in the temperature or power
supply, for example, or to the various on-chip ageing phenomena. In addition,
if an evolved design is to be implemented on more than one device (as in a
commercial application), then it must not be aected by properties that vary
from chip to chip. The required tolerance can be evolved by making sure that
the properties in question actually do vary while the tnesses are being eval-
uated. This might involve purposely varying the temperature or power supply
during evaluations and/or carrying out multiple evaluations on dierent (nomi-
nally identical) recongurable devices. An alternative to evaluating over several
separate devices is to use the same recongurable hardware to instantiate the
circuit in dierent ways: translating or rotating it on an FPGA for example.
Extending that idea, a genetic encoding could be used which forces the use of
repeated structures, so that each structure must cope with the characteristics of
all of the places in which it occurs. Another alternative would be to use further
adaptation each time the evolved circuit is transferred from one recongurable
device to another.
The introduction of a fault can be seen as an extreme form of variation in
the device's properties: Section 19 will describe a real application of the `further
adaptation' approach to coping with a fault. It may also be possible to use the
evolutionary fault tolerance mechanisms described in Section 14 to give tolerance
to normal device variations.
Even when forced to produce a circuit with tolerance to some range of varia-
tions, there is still room for intrinsic EHW to exploit detailed hardware charac-
teristics much more than conventional design does. Traditional design methods
5
A heuristic graph partitioning algorithm was used to search for non-trivial cohesive
sub-assemblies, but none were found.

20
cannot proceed far with precise descriptions of the physics of the individual
components (eg. transistors) before abstraction and modularisation need to be
invoked to make the problem tractable, as discussed above. The `try it and see'
opportunistic nature of intrinsic EHW is not subject to this diculty, so the de-
tailed behaviours of the components can be integrated usefully to give rise to the
required overall performance. This paper largely concentrates on the use of digi-
tal recongurable devices because these are currently available o-the-shelf, but
we can now see that the advantages of intrinsic EHW over conventional design
are even greater for analogue systems. Analogue FPGAs are being developed
[25] and will be a fruitful avenue for EHW research in the future.

17 The Danger of Inheriting Inappropriate Constraints

from Natural Evolution
Evolvable Hardware is a combination of electronics and evolution. We have dis-
cussed the error of adhering too closely to the conventional principles of electron-
ics, but there are also potential pitfalls in blindly applying ideas from natural
evolution.
Consider biological neural networks. Compared to electronics, the neuron
response and signal propagation times are extremely slow. On the other hand,
there is very high connectivity in three dimensions, contrasting with the highly
restricted planar wiring in VLSI. The two media | biological cell based and sili-
con VLSI based | provide very dierent resources. A structure evolved to exploit
the former may not eciently utilise the latter. It may be possible to evolve par-
allel distributed architectures better tailored to the opportunities provided by
VLSI than models of biological neural networks are. Such an architecture might
use the high speed of VLSI to compensate for limited connectivity in a more
sophisticated way than the multiplexing schemes commonly seen in VLSI im-
plementations of neural nets. Hence it would be unwise to rigidly limit EHW to
a neuro-mimetic structure when intrinsic EHW can oer a more unconstrained
exploitation of hardware resources. For engineering purposes, `VLSI-plausible'
architectures are required, not `biologically-plausible' ones.
In the same way that the architecture of natural nervous systems evolved to
be suited to the restrictions and opportunities of biology, so did the process of
natural evolution itself adapt to the resources available (\the evolution of evolv-
ability"). The large timescale, highly parallel, distributed co-evolution found in
nature is somewhat dierent that possible in present-day implementations of ar-
ticial evolution. It is thus justiable to use biologically-implausible mechanisms
where these are eective, for example in the setting of the mutation rate or in the
morphogenesis process. The aim is to arrive at an implementation of articial
evolution that is inspired by nature, but suited to the facilities available.
We have now proposed a synthesis of genetic algorithms, natural evolution
and electronics that adapts each in the formation of a new eld: Intrinsic Evolv-
able Hardware. The next section begins to put some of the ideas into practice.

21
18 Case Study: An evolved hardware sensorimotor
control structure
This experiment takes a standard electronic architecture, removes some of the
dynamical constraints used to make conventional design tractable, and subjects
the resulting dynamical electronic system to intrinsic hardware evolution. The
result was the rst evolved hardware control system for a real robot, reported
in [26].
The EHW circuit was the on-board controller for the robot shown in Fig. 3.
This two-wheeled autonomous mobile robot has a diameter of 46cm, a height
of 63cm, and was required to display simple wall-avoidance behaviour in an
empty 2.9m4.2m rectangular arena. For this scenario, the d.c. motors were not
allowed to run in reverse and the robot's only sensors were a pair of time-of-
ight sonars rigidly mounted on the robot, one pointing left and the other right.
The sonars re simultaneously ve times a second; when a sonar res, its output
changes from logic 0 to logic 1 and stays there until the rst echo is sensed at
its transducer, at which time its output returns to 0.
Conventional electronic design would tackle the control problem along the
following lines: For each sonar, a timer would measure the length of its output
pulses | and thus the time of ight of the sound | giving an indication of the
range to the nearest object on that side of the robot. These timers would provide
binary-coded representations of the two times of ight to a central controller. The
central controller would be a hardware implementation of a nite-state machine
(FSM), with the next-state and output functions designed so that it computes
a binary representation of the appropriate motor speed for each wheel. For each
wheel, a pulse-width modulator would take the binary representation of motor
speed from the central controller and vary the mark:space ratio of pulses sent to
the motor accordingly.
It would be possible to evolve the central controller FSM as intrinsic EHW
by implementing the next-state and output functions as look-up tables held
in an o-the-shelf random access memory (RAM) chip.6 The FSM would then
be specied by the bits held in the RAM, which could be recongured under
the control of each individual's genotype in turn. There would be no benet
in evolving this architecture as hardware, however, because the electronics is
constrained to behave in accordance with the FSM design abstraction: all of
the signals are synchronised to a global clock to give clean, deterministic state-
transition behaviour as predicted by the model. Consequently, the hardware
would behave identically to a software implementation of the same FSM.
What if the constraint of synchronisation of all signals is relaxed and placed
under evolutionary control? Although supercially similar to the FSM imple-
mentation, the result (shown in Fig. 4), is a machine of a fundamentally dier-
ent nature. Not only is the global clock frequency placed under genetic control,
but the choice of whether each signal is synchronised (latched) by the clock or
whether it is asynchronous is also genetically determined. These relaxations of
6
This is the well known `Direct Addressed ROM' implementation of an FSM [27].

22
 Virtual
World
Simulator

Sonar
Emulator

Evolvable
Hardware

Sonars

Wheels

Rotation
Sensors

Fig. 3. The robot known as \Mr Chips."

temporal constraints | constraints necessary for a designer's abstraction but not

for intrinsic EHW | endow the system with a rich range of potential dynamical
behaviour, to the extent that the sonar echo pulses can be fed directly in, and
the motors driven directly by the outputs, without any pre- or post-processing:
no timers or pulse-width modulators. (The sonar ring cycle is asynchronous to
the evolved clock).
Let this new architecture be called a Dynamic State Machine (DSM). It is not
a nite-state machine because a description of its state must include the temporal
relationship between the asynchronous signals, which is an real-valued analogue
quantity. In the conventionally designed control system there was a clear sen-
sory/control/motor decomposition (timers/controller/pulse-width-modulators),
communicating in atemporal binary representations which hid the real-time dy-
namics of the sensorimotor systems, and the environment linking them, from the
central controller. Now, the evolving DSM is intimately coupled to the real-time

23
 Sonars Evolved RAM Contents

1k by 8 bits RAM
10 Address inputs 8 Data outputs

1 1 10 6 1 1
G.L.

G.L.

Evolved M M
Clock
Motors

Fig. 4. The hardware implementation of the evolvable DSM. `G.L.' stands for a bank
of genetic latches: it is under genetic control whether each signal is passed straight
through asynchronously, or whether it is latched according to the global clock of evolved
frequency.

dynamics of its sensorimotor environment, so that real-valued time can play an

important r^ole throughout the system. The evolving DSM can explore special-
purpose tight sensorimotor couplings because the temporal signals can quickly
ow through the system being in uenced by, and in turn perturbing, the DSM
on their way.
For the simple wall-avoidance behaviour, only two of the possible eight feed-
back paths seen in Fig. 4 were enabled. The resulting DSM can be viewed as
the fully connected, recurrent, mixed synchronous/asynchronous logic network
shown in Fig. 5, where the bits stored in the RAM give a look-up table imple-
menting any pair of logic functions of four inputs. This continuous-time dynam-
ical system cannot be simulated in software, because the eects of the asyn-
chronous variables and their interaction with the clocked ones depend upon the
characteristics of the hardware: meta-stability and glitches will be rife, and the
behaviour will depend upon physical properties of the implementation, such as
propagation delays and meta-stability constants. Similarly,a designer would only
be able to work within a small subset of the possible DSM congurations | the
ones that are easier to analyse.
The genetic algorithm was the same as that described in Section 15.2, with
the contents of the RAM (only 32 bits required for the machine with two feedback

24
 LEFT RIGHT

MOTORS M M

LOGIC LOGIC
FUNCTION FUNCTION

LEFT RIGHT

SONARS

Fig.5. An alternative representation of the evolvable Dynamic State Machine, as used

in the experiment. Each is a `Genetic Latch' (see previous gure).

paths), the period of the clock (16 bits, giving a clock frequency from around
2Hz to several kHz) and the clocked/unclocked condition of each signal all being
directly encoded onto the linear bit-string genotype. The population size was 30,
probability of crossover 0.7, and the mutation rate was set to be approximately
1 bit per genotype. If the distance of the robot from the centre of the room in
the x and y directions at time t was cx (t) and cy (t), then after an evaluation for
T seconds, the robot's tness was a discrete approximation to the integral:
tness = T1
Z T  
e kx cx (t)2 + e ky cy (t)2 s(t) dt (2)
0

kx and ky were chosen such that their respective Gaussian terms fell from their
maximum values of 1.0 (when the robot was at the centre of the room) to a

25
minimum of 0.1 when the robot was actually touching a wall in their respective
directions. The function s(t) has the value 1 when the robot is stationary, oth-
erwise it is 0: this term is to encourage the robot always to keep moving. Each
individual was evaluated for four trials of 30 seconds each, starting with dierent
positions and orientations. The worst of the four scores was taken as the tness
[28]. For the nal few generations, the evaluations were extended to 90 seconds,
to nd controllers that were not only good at moving away from walls, but also
staying away from them.
For convenience, evolution took place with the robot in a kind of `virtual
reality.' The real evolving hardware controlled the real motors, but the wheels
were just spinning in the air. The wheels' angular velocities were measured, and
used by a real time simulation of the motor characteristics and robot dynamics
to calculate how the robot would move. The sonar echo signals were then ar-
ticially synthesised and supplied in real time to the hardware DSM. Realistic
levels of noise were included in the sensor and motor models, both of which were
constructed by tting curves to experimental measurements, including a proba-
bilistic model for specular sonar re ections. The photograph of Fig. 3 was taken
during an evolutionary run of this kind.
Fig. 6 shows the excellent performance which was attained after 35 genera-
tions, with a good transfer from the virtual environment to the real world. The
robot is drawn to scale at its starting position, with its initial heading indicated
by the arrow; thereafter only the trajectory of the centre of the robot is drawn.
The bottom-right picture is a photograph of behaviour in the real world, taken
by double-exposing a picture of the robot at its starting position, with a long
exposure of a light xed on top of the robot, moving in the darkened arena. If
started repeatedly from the same position in the real world, the robot follows a
dierent trajectory each time (occasionally very dierent), because of real-world
noise. The robot displays the same qualitative range of behaviours in the virtual
world, and the bottom pictures of Fig. 6 were deliberately chosen to illustrate
this.
When it is remembered that this miniscule electronic circuit receives the raw
echo signals from the sonars and directly drives the motors (one of which happens
to be more powerful than the other), then this performance in surprisingly good.
It is not possible for the DSM directly to drive the motors from the sonar inputs
(in the manner of Braitenberg's `Vehicle 2' [29]), because the sonar pulses are
too short to provide enough torque. Additionally, such nave strategies would
fail in the symmetrical situations seen at the top of Fig. 6. One of the evolved
wall-avoiding DSMs was analysed (see below), and was found to be going from
sonar echo signals to motor pulses using only 32 bits of RAM and 3 ip- ops
(excluding clock generation): highly ecient use of hardware resources, made
possible by the absence of design constraints.
Fig. 7 illustrates the state-transition behaviour of one of the wall avoiders.
This particular individual used an evolved clock frequency of 9Hz (about twice
the sonar pulse repetition rate). Both sonar inputs evolved to be asynchronous,
and both motor outputs clocked, but the internal state variable that was clocked

26
Fig.6. Wall avoidance in virtual reality and (bottom right) in the real world, after 35
generations. The top pictures are of 90 seconds of behaviour, the bottom ones of 60.

to become the left motor output was free-running (asynchronous), whereas that
which became the right output was clocked. In the diagram, the dotted state
transitions occur as soon as their input combination is present, but the solid
transitions only happen when their input combinations are present at the same
time as a rising clock edge. Since both motor outputs are synchronous, the state
can be thought of as being sampled by the clock to become the motor outputs.
This state-transition representation is misleadingly simple in appearance, be-
cause when this DSM is coupled to the input waveforms from the sonars and its
environment, its dynamics are subtle, and the strategy being used is not at all
obvious. It is possible to convince oneself that the diagram is consistent with the
behaviour, but it would have been very dicult to predict the behaviour from
the diagram, because of the rich feedback through the environment and sensori-
motor systems on which this machine seems to rely. The behaviour even involves
a stochastic component, arising from the probabilities of the asynchronous echo
inputs being present in certain combinations at the clocking instant, and the

27
 Left motor ON Left motor ON
Right motor ON Right motor OFF

01 00,10

11 01 01,11

01,11 11

RESET Left motor OFF 00 Left motor OFF

Right motor OFF Right motor ON

00,01

Fig. 7. A representation of one of the wall-avoiding DSMs. Asynchronous transitions

are shown dotted, and synchronous transitions solid. The transitions are labelled with
(left, right) sonar input combinations, and those causing no change of state are not
shown. There is more to the behaviour than is seen immediately in this state-transition
diagram, because it is not entirely a discrete-time system, and its dynamics are tightly
coupled to those of the sonars and the rest of the environment.

probability of the machine being in a certain state at that same instant (remem-
ber that one of the feedback loops is unclocked).
Even this small system is non-trivial, and performs a dicult task with min-
imal resources, by means of its rich dynamics and exploitation of the real hard-
ware. After relaxing the temporal constraints necessary to support the designer's
nite-state model, a tiny amount of hardware has been able to display rather
surprising abilities7. FPGAs are undoubtedly very much more powerful than
the DSM. It is left to the reader to speculate on the potential capabilities of
an evolvable chip containing many hundreds of logic gates, bearing in mind the
power released by unconstrained evolution from the equivalent of only two gates
in the DSM described above.

19 The Fault Tolerance of The Evolved Hardware

Control System
To end the paper, we brie y use the evolved DSM robot controller to consolidate
two of the points made earlier pertaining to fault tolerance.
Firstly, notice that the contents of the RAM were directly encoded bit-for-bit
onto the genotype. A genetic mutation in the RAM coding region causes one of
7
If all of the genetic latches were constrained to be synchronous, so that we have
a FSM, then three control experiments failed to produce successful systems. If all
of the latches were constrained to be asynchronous, then three control experiments
each produced a recognisable wall-avoider, but much inferior to those evolved when
the genetic latches were under evolutionary control.

28
 No Faults
2.60
Mean Faulty

Mean
Fitness

Mean Random
1.60

32 different SSA faults

Fig. 8. Sensitivity to adverse SSA faults.

the bits in the RAM to be inverted: the same eect as a single-stuck-at (SSA)
fault in the RAM's memory array. By the argument of Section 14, there should
therefore be a tendency for the evolved controller to be tolerant to such SSA
faults. Fig. 8 shows that the evolved wall-avoider DSM is indeed quite robust
to adverse SSA faults | observation of the robot's qualitative behaviour bears
this out | but it is not known how much is due to the action of evolution,
and how much is simply a property of the DSM architecture. The 32 possible
adverse SSA faults were each emulated in turn by writing the opposite value
to that specied by the genotype to the RAM bit in question. For each fault,
the DSM was then used to control the robot (in the virtual environment) for
sixteen 90-second runs from the same starting position, and the average tness
was measured to give the data in the gure. The results are in accord with the
theory, but gathering sucient data from the real robot actually to verify the
theory would be prohibitively time-consuming: hence the ongoing study using
NK landscapes mentioned in Section 14.

The second experiment was to introduce the SSA fault marked with an arrow
in Fig. 8 as a permanent feature in the DSM, and then to allow the already-
evolved population to evolve further. At rst, the tness of the population was
signicantly lowered, with none of the individuals performing as well as the best
of the population used to, but after 10 generations the mean and best tnesses
of the population had recovered to their previous values. This approach to fault-
tolerance (proposed in Section 16 above) would be useful when transferring the
evolved controller from one piece of hardware to another, or to cope with long-
lasting faults within one.

29
20 Conclusion
We have formulated Evolvable Hardware | and in particular intrinsic EHW
| as a synthesis of genetic algorithms, inspiration from natural evolution and
electronics. In forming this synthesis, none of the three components has been left
unaltered. For the evolution of complex structures, genetic algorithms need to be
extended to incorporate the notions of converged species evolution and incremen-
tally increasing complexity: the SAGA framework. When drawing inspiration
from nature, it is necessary to recognise the signicantly dierent constraints and
opportunities associated with the articial evolution of VLSI circuits. Finally,
the whole concept of what electronics can be needs to be re-thought, because
until now it has been governed by what is amenable to design techniques. Step-
ping into the wider space of dynamical electronic systems, the rst intrinsically
evolved hardware robot controller has been presented, showing remarkable levels
of eciency and robustness. There is every reason to expect that this new eld
will go far, but extravagant projections are not appropriate at this early stage.
Acknowledgements
Special thanks to Dave Cli. The research is funded by a D.Phil. scholarship from the
School of Cognitive & Computing Sciences and by EPSRC.
References
1. For an overview of EHW literature, see A.J. Hirst: Notes on the evolution of
adaptive hardware. To appear in: Proc. of 2nd Int. Conf. on Adaptive Computing
in Engineering Design and Control (ACEDC96), University of Plymouth UK, 26{
28th March 1996. http://kmi.open.ac.uk/~monty/evoladaphwpaper.html
2. Trevor A. York. Survey of eld programmable logic devices. Microprocessors and
Microsystems, 17(7):371{381, 1993.
3. David E. Goldberg. Genetic Algorithms in Search, Optimization and Machine
Learning. Addison-Wesley, Reading MA, 1989.
4. Inman Harvey. Species adaptation genetic algorithms: The basis for a continuing
SAGA. In F. J. Varela and P. Bourgine, editors, Toward a Practice of Autonomous
Systems: Proc. of 1st Eur. Conf. on Articial Life, pp346{354. MIT Press/Bradford
Books, Cambridge, MA, 1992.
5. Stuart Kauman. Adaptation on rugged tness landscapes. In Daniel L. Stein,
editor, Lectures in the Sciences of Complexity, pp527{618. Addison Wesley: Santa
Fe Institute Studies in the Sciences of Complexity, 1989.
6. M. Eigen, J. McCaskill, and P. Schuster. Molecular quasi-species. Journal of Phys-
ical Chemistry, 92:6881{6891, 1988.
7. M. Eigen and P. Schuster. The Hypercycle: A Principle of Natural Self-
Organization. Springer-Verlag, 1979.
8. M. Nowak and P. Schuster. Error thresholds of replication in nite populations,
mutation frequencies and the onset of Muller's ratchet. Journal of Theoretical
Biology, 137:375{395, 1989.
9. Inman Harvey. The SAGA cross: the mechanics of crossover for variable-length
genetic algorithms. In R. Manner and B. Manderick, editors, Parallel Problem
Solving from Nature 2, pp269{278. North-Holland, 1992.
10. G.E. Hinton and S.J. Nowlan. How learning can guide evolution. Complex Systems,
1:495{502, 1987.

30
11. I. Harvey, P. Husbands, and D. T. Cli. Seeing the light: Articial evolution, real
vision. In D. Cli, P. Husbands, J.-A. Meyer, and S. Wilson, editors, From Ani-
mals to Animats 3: Proc. of 3rd Int. Conf. on Simulation of Adaptive Behaviour
(SAB94), pp392{401. MIT Press/Bradford Books, Cambridge MA, 1994.
12. N. Jakobi, P. Husbands, and I. Harvey. Noise and the reality gap: The use of sim-
ulation in evolutionary robotics. In F. Moran et al., editors, Advances in Articial
Life: Proc. of 3rd Eur. Conf. on Articial Life (ECAL95)., pp704-720. LNAI 929,
Springer-Verlag, 1995.
13. Hugo de Garis. Evolvable hardware: Genetic programming of a Darwin Machine.
In C.R. Reeves et al., editors, Articial Neural Nets and Genetic Algorithms - Proc.
of the Int. Conf. in Innsbruck, Austria, pp441{449. Springer-Verlag, 1993.
14. Adrian Thompson. Evolving fault tolerant systems. In Proc. of 1st IEE/IEEE Int.
Conf. on Genetic Algorithms in Engineering Systems (GALESIA95), IEE Confer-
ence Publication No. 414, pp524{529, 1995.
15. M. Eigen. New concepts for dealing with the evolution of nucleic acids. In Cold
Spring Harbor Symposia on Quantitative Biology, volume LII, 1987.
16. Stuart A. Kauman. The Origins of Order. Oxford University Press, 1993.
17. R. Brooks. Intelligence without representation. Articial Intelligence, (47):139{
159, 1991.
18. Gunter P. Wagner. Adaptation and the modular design of organisms. In F Moran
et al., editors, Advances in Articial Life: Proc. of 3rd Eur. Conf. on Articial Life
(ECAL95), pp317{328. LNAI 929, Springer-Verlag, 1995.
19. A. F. Murray et al. Pulsed silicon neural networks - following the biological leader.
In Ramacher and Ruckert, editors, VLSI Design of Neural Networks, pp103{123.
Kluwer Academic Publishers, 1991.
20. Alan F. Murray. Analogue neural VLSI: Issues, trends and pulses. Articial Neural
Networks, (2):35{43, 1992.
21. Carver A. Mead. Analog VLSI and Neural Systems. Addison Wesley, 1989.
22. P. Kinget, M. Steyaert, and J. van der Spiegel. Full analog CMOS integration of
very large time constants for synaptic transfer in neural networks. Analog Inte-
grated Circuits and Signal Processing, 2(4):281{295, 1992.
23. Dave Cli, Inman Harvey, and Phil Husbands. Explorations in evolutionary
robotics. Adaptive Behaviour, 2(1):73{110, 1993.
24. Alexander Miczo. Digital Logic Testing and Simulation. Wiley New York, 1987.
25. IMP, Inc. IMP50E10 EPAC Electronically Programmable Analog Circuit: Prelim-
inary product information sheet, November 1994.
26. Adrian Thompson. Evolving electronic robot controllers that exploit hardware
resources. In F. Moran et al., eds., Advances in Articial Life: Proc. of 3rd Eur.
Conf. on Articial Life (ECAL95), pp640{656. LNAI 929, Springer-Verlag, 1995.
27. D.J. Comer. Digital Logic & State Machine Design. Holt, Rinehart & Winston,
1984.
28. I. Harvey, P. Husbands, and D. Cli. Genetic convergence in a species of evolved
robot control architectures. In S. Forrest, editor, Proc. of 5th Int. Conf. on Genetic
Algorithms, p636. Morgan Kaufmann, 1993.
29. V. Braitenberg. Vehicles : Experiments in Synthetic Psychology. MIT Press, 1984.
30. D.P.M. Northmore and J.G. Elias. Evolving synaptic connections for a silicon
neuromorph. In Proc of 1st IEEE Conf. on Evolutionary Computation, IEEE
World Congress on Computational Intelligence, volume 2, pp753{758. IEEE, New
York, 1994.

31