TAKING A PAGE FROM DARWIN'S ON THE ORIGIN OF THE

L
QI SPECIfS, COMPUTER SCIENTISTS HAVE FOUND WAYS TO
> EVOLVE SOLUTIONS TO COMPLEX PROBLEMS
0
U

What is evolutionary
computation?
D AV I D B . F 0G E L THE PRINCIPLE OF EVOLUTION is the primary unifying concept of biol-
~~t~ I ogy, linking every organism together in a historical chain of events. Every
creature in the chain is the product of a series of "accidents" that have been
S e l e c t i o n Inc.
sorted out thoroughly under selective pressure from the environment. Over
many generations, random variation and natural selection shape the behav-
iors of individuals and species to fit the demands of their surroundings.
This fit can be quite extraordinary and compelling [Fig.11, a clear indication
that evolution is creative. While evolution has no intrinsic purpose-it is merely
the effect of physical laws acting on and within populations and species-it
is capable of engineering solutions to the problems of sutvival that are unique
to each individual's circumstance and, by any measure, quite ingenious.
Imagine what harnessing the evolutionary process within a computer might
do. It could provide a means for addressing complex engineering problems-
ones involving chaotic disturbances, randomness, a n d complex nonlinear
dynamics-that our traditional algorithms have been unable to conquer.
Indeed, the field of evolutionary computation is one of the fastest growing
areas of computer science and engineering for just this reason; it is address-
ing many problems that were previously beyond reach, such as rapid design
of medicines, flexible solutions to supply-chain management problems, and
rapid analysis of battlefield tactics for defense. Potentially, the field may
fulfill the dream of artificial intelligence: a computer that can learn on its own
and become an expert in any chosen area.

26 0018~92351001'110.0002000IEEE IEEESPECTRUM FEBRUARY 2000

 In the most general terms, evolution initial population of creatures, the algo- dom variation. The resultant solutions are
can b e described as a two-step iterative rithmic approach begins by selectingan ini- evaluated for their effectiveness-their "fit-
process, consisting of random variation tial set of contending solutions for a par- ness'-and undergo selection. Just as nature
followed by selection. T h e link between ticular problem. The set may be chosen imposes the rule of "sumival of the fittest,"
this description of evolution and the opti- by generating solutions randomly or by uti- those solutions that are the least fit are
mizing algorithms that are the hallmark lizing any available knowledge about the removed from further consideration, and
of evolutionary computation is concep- problem. the process is repeated over successive gen-
tually simple. These "parent" solutions then generate erations [Fig. 21
Just as natural evolution starts from an "offsprinc by a preselected means of ran-
THE ASSUMPTION PROBLEM
To be useful, traditional algorithms for
[rl Evolution copes with pro- discovering the most appropriate solu-
blems in ways that might tions-optimization algorithms-require
never occur to people, who that their users make many assumptions
tend to thinklinearly Witness about how to evaluate the fitness of a solu-
the leafy sea dragon. whose tion. (These traditional means of evaluation
limbs and back take on t h e go by many names: the fitness or cost func-
form of the surrounding vege- tion, the response surface, or, in engineer-
tation, concealing it from ing, the performance index.) For example,
predators. the so-called linear programming algorithms
demand that the cost functions also be lin-
(Source: A Natural Selection: ear--a sum of weighted individual cost
Marvels and Oddities of the terms. Another traditional approach-gra-
Natural World, R. Morecroft. dient-based search, in which the point of
Simon and Schuster. 1993) zero gradient, hopefully, the maxima o r
minima, is sought-requires a smooth, dif-
ferentiable cost function; it is unable to deal
with sudden, discontinuous change.
But evolutionary algorithms require no
such assumptions. Fundamentally, the per-
formance index need only be able to rank
two competing solutions; that is, it must
determine that one solution is, in some way,
better than another. This makes a broad
range of problems that are outside the range
of conventional engineering amenable to
the evolutionary approach. In short, evo-
lutionary algorithms can often resolve prob-
lems that do not yield to common numer-
ical techniques.
In the real world, an evolutionary ap-
proach to solving engineering problems
offers considerable advantages. O n e such
advantage is adaptability to changing sit-
uations. For instance, suppose a manager
must find the best schedule for operating
a factory Doubtless, there will be many
constraints: availability of personnel, the
number of machines, the time required to
change machine settings, and so forth. Even
if the manager could find an optimum
schedule that would be most profitable, he
or she would still need to consider that
machines may break down and personnel
may not arrive at work on time, In daily life,
the problem facedat any moment may have
diverged significantly from the problem
anticipated originally.
Unfortunately, in many traditional opti-
mization procedures, the calculation must
be restarted from the beginning if any vari-
able in the problem changes. This is com-
putationally expensive. With an evolution-
[21 An evolutionary algorithm begins by initializing a population of candidate solutions to ary algorithm, on the other hand, the
a problem. New solutions are then created by randomly varying those of the initial popu- current population selves as a memoir of
lation. All solutions are measured with respect to how well they address the task. Finally, stored knowledge that can be applied on
a selection criterion is applied to weed out those that are below par. The process is iter- the fly to a dynamic environment. Re-
ated using t h e selected ret of solutions until a specific criterion is met. starting from scratch is not necessary.

28 IEEE SPECTRUM FEBRUARY 2000

 Another advantage of an evolutionary passes. Each of these steps is explained in 3r only after other customers are visited. If
approach to problem solving comes in being greater detail in the following paragraphs. io, evaluating candidate solutions might
able to generate good enough solutions become much more complex. Complexity
quickly enough for them to be of use. This EVOLUTIONARY STEPS only makes the application of an evolu-
ability is perhaps best illustrated by the clas- To represent any possible solution within tionaly algorithm more pertinent, because
sic traveling salesman problem, to wit, the confinesof a computer, a sttucture must it rapidly removes the problem from the
"Suppose a salesman must visit clients in dif- be defined for the data that will encode domain of most traditional optimization
ferent cities, and then return home. What evely possible solution that it might be techniques. For the sake of this example, let
is the shortest tour through those cities, vis- desirable to evaluate. Here, there is no sin- the simple total distance of the tour be the
iting each one once and only once?" gle best choice for the representation (this measure of the quality of a solution.
This problem is simple to state,. .but dif- is provable mathematically) so a little inge- T h e second step is to devise a random
ficult to solve. It belongs to the class of nuity is called for. variation operator (or operators) that can
problems referred to as NP-hard, where NP One potential data representation for the be used to generate offspring solutions from
stands for nondeterministic polynomial. For traveling salesman problem is to identify parent solutions. Many options exist. In
such problems, no known algorithms are each different possibility in a permutation. nature, there are two general forms of re pro^
able to generate the best answer in an To simplify, if there were six cities (includ- duction: sexual and asexual. In sexual repro-
amount of time that grows only as a poly- ing the salesman's home base), then one pos- duction, two parents within a species ex-
nomial function of the number of elements sible solution might he [ I 2 3 4 5 61. which change genetic material that is recombined
in the problem. would indicate an order of progression. to form an offspring. Asexual reproduction
In fact, the number of possible tours in (Note that, as this problem requires a round is essentially cloning, but mutations of var-
any traveling salesman problem increases as trip-that is, a closed loop-the first ele- ious forms can creep into the genetic in for^
a factorial function of the number of cities. ment in the series is also the last. Also note mation passed along from parent to off-
So for 100 cities, there are over 10'55 dif- that, since this is a loop, it does not matter spring. These operators are worth modeling
ferent possible paths through all the cities which city the salesman actually starts in an evolutionaly algorithm.
one could try. Considering that there are from.) Any permutation of these cities Thinking even more broadly, why not
only an estimated 10'8 seconds in the his- would be another more or less desirable devise other variation schemes not found in
tory of the universe, simply applying brute solution to the problem. nature? Examples include recombining gen-
force to search all possible solutions to a Suppose that this representation is cho- etic material from three or more parents and
traveling salesman problem of even modest sen. Then the cost function, that is, the allowing a democratic vote between the par-
size is doomed to fail. means to evaluate any candidate solution, ents involved in reproduction. There is vir-
Instead, consider an evolutionary ap- must also be determined. Here the task is tually no limit to the types of variation oper-
proach to discovering a useful solution to straightforward: travel as short a distance as ators that can be devised, nor any reason to
such a problem. The approach has four pre- possible. So the "cost" of any solution can be constrained by nature far inspiration.
requisite steps: he made equal to the distance of the tour, The ultimate success of an evolutionaly
Choosing the solution representation. with shorter tours being favored over longer algorithm depends strongly on how well the
Devising a random variation operator. ones. Of course, things could be made more variation operator(s), the representation,
Determining a rule for solution sumival. complicated, incorporating additional as- and the evaluation function are matched.
Initializing the population. pects of real-world problems such as mini- Different operators will vary in usefulness
Once these are taken, the evolutionary algo- mizing the traveling required during peak- with the situation. Just as with the repre-
rithm will start to generate solutions, and traffic hours or requiring that certain sentation, there is no single best variation
will continue to generate better ones as time customers be visited, say, in the afternoon operator for all problems (and this, too, is
provable mathematically).
Consider two possibilities for the travel-
131The traveling sales-
Parent tour Offspring tour man problem is: find
ing salesman problem represented using per^
mutations. One choice is to use a single par-
the sho~testmute for ent and generate an offspring by randomly
visiting all the cities choosing two positions along the parent and
selected. With six cities inverting the list of cities in that segment
[l2 3 4 5 61, a single-
[Fig. 31. Another is to use two parents,
parent operator could
choose a random point along the permu-
create an offspring tation, then take the first segment of cities
solution by inverting from the first parent and the subsequent seg-
the visiting order ment of cities from the second parent.
between two random- The first operator looks a bit like asexual
ly chosen points [top]. reproduction, while the second looks some-
With a two.parent thing like sexual reproduction. But whereas
operator. two tour5 the first variation will always generate a legal
could be split at some tour (each city visited once and only once),
random point and the the second variation might generate ille-
pieces joined to create gal tours, because some offspring may con-
an offspring [bottom].
tain more than one copy of some cities and
-[ 1 2 3 4 3 51 The tour here, though, no copies of others. That does not Nk out
is incomplete and addi- using semal reproduction, it merely dictates
[1642351 tional ruler are needed the inclusion of additional operations to
to repair it or to pre- "repaii'such Solutions so that they can be
vent it from occuring. fixed before they are evaluated. T h e rem-
edy might be to locate any city that is rep-

FOGEL I WHAT IS EVOLUTIONARY COMPUTATION?

29
 Generation 1 Generatio 500
I '

0 20 40 60 80 100 0 20 40 60 80 100
Unit distance, x Unit distance, x
[41 A simple iterative evolutionary algorithm war used to solve a The best 100 (in terms of shortest mute) were selected, with the tap
lmitytravelingsalesman pmblem.Atthestart,the reseanherselected solutionfrom this f i m generation shown at far 1eft.The best solutions
100 possible solutions, each of which was used to generate one off- after 500,lOW. and 4000 iterations indicatethe progress made as the
spring by means of inversion [Fig. 31, yielding a total of 200 solutions. process was repeated.

resented more than once in one solution, chosen completely at random from the Granted, there are now many betteralgo-
then replace that city with one not repre- space of all possible solutions. In the case of rithms for solving the traveling salesman
sented at all. Once this is done for each the traveling salesman problem here, that problem than the above evolutionary pro-
duplicated city, the offspring will have been would mean randomly generating a number cedure, since mathematicians have already
repaired and become a viable contending of permutations of integers, each permuta- spent much time and effort trying to solve
solution. Many variation operaton could be tion representing one possible solution. it. But all these other techniques rely on
considered for the traveling salesman prob- Alternatively, there may he some hints at some specific knowledge about the prob-
lem but, for illustration, the inversion oper- good solutions available-perhaps because lem to improve their performance. They
ator described above will be used here. some other algorithm or prior knowledge sacrifice generality in order to gain perfor-
The third step is determining a rule for can be used to generate some reasonable mance. Just like Deep Blue, the worldcham-
selecting which solutions will survive to head start-which can be incorporated in pion chess program, they shine in their nar-
become parents of the next generation. As the initial population. If those solutions row domain of application, but do poorly
with variation, many forms of selection can prove worthwhile, they will survive and pro^ outside that domain. Imagine Deep Blue
be considered. O n e simple rule is survival duce new variants; if they are false leads, playing a game of checkers it couldn't make
of the fittest: only the handful of very best then they will perish along with other, even the first move.
solutions in the population is retained while weaker solutions. For the example here, sup- The key point is that, while it is possible
all the othen are killed off, so to speak. An pose the initial population is chosen com- to incorporate any problem~specificknowl-
alternative is to use a sort of tournament pletely at random. edge available and thereby take advantage
amroach. where randomly oaired solutions A tvoical run of a n evolutionarv alzo- of it when using evolutionam algorithms. it

times win because they get a lucky draw in uted at random but in a uniform manner, has tionary algorithms can tackle an enormously
the tournament, weaker solutions in a pop^ the results shown in Fig. 4. T h e improve- broad range of problems.
dation sometimes survive a few generations ment at successive stages in the evolution
under this format. This can he a plus in com- of the tours is evident (The program for IN THE FIELD
plex problems, where it may he easier to this example, written in Matlab, is avail- Evolutionary algorithms are already
find new improved solutions by making able at wwwmatural-selection.com under being used to solve a wide variety of real-
variations of weaker ones than to do SO by the publications section.) world problems that pose significant chal-
relying only on the very best. T h e possi- The evolutionary algorithm has searched lenges. O n e such problem involves the dis-
bilities abound, but any rule that generally the space of possible toun and has discov- covery of new drugs. In the case of docking
favon better solutions over weaker solutions eredavery goodone. While it is probably a small molecule of a potential drug (called
for suwival is reasonable. For simplicity,only not the perfect answer, it is of high qual- a 1ija.d. meaning"something that has to be
the most basic approach-survival of the ity. In all, only 400 000-or one out of bound") into a target protein's binding site,
fittest-will be considered here. every IO'So-possible solutions were the result depends in large part on the three-
examined, an infinitesimally small frac- dimensional shapes of both elements. It is
GENESIS tion of the whole. This is in line with the analogous to a lock and kev _iust, as onlv the
The final step is selecting the initial pop- operating credo of evolutionary practition- right-shaped key will open a particularlock,
dation. If nothing is known about how to e n : "Solutions should be good enough and so only an appropriately shaped ligand will
solve the problem, then solutions can be generated fast enough to he useful." he able to bind to the target protein.
30 IEEE SPECTRUM FEBRUARY 2000
 Generation 1WO Generation 4000

0 20 40 60 80 100 0 20 40 60 80 100
Unit distance, x Unit distance, x

[51 To test i l l applicability to drug

Finding that ligand is extremely difficult.
design, an evolutionary algorithm
Not only are there tens of thousands of can-
war wed to select candidate mol-
didates, hut each one has many bonds that
ecules,or ligands, thatwoulddlxk
can be rotated into many different positions.
tothereceptorsiteofanHIV-1 p m
Essentially, every ligand has infinitely many
tease protein [shown, far left top,
shapes. It is certainly m e that many of these
asa biochemist's ribbon diagram].
possible conformations are extremely un-
The best 150 out of 1000 possible
likely owing to the physical and chemical
solutions in the 2nd generation
properties of the molecules. Nevertheless,
lookmessy[nearleft,topl. Butthe
the number of possible profiles that each
algorithm maker progress in the
ligand can present is often enormous. This 70th [far left, middle]. 92nd [near
makes predicting the best conformation of
left, middle]. and 149th [far left,
a ligand as it binds to a target protein quite
bottom1 generations.
challenging.
The best structure obtained
In a study published in Cbanirtry &Biology,
[green, near left, bottom 1 closely
researchers at Agouron Pharmaceuticals Inc.
matches one found experimentally
indicated how an evolutionary algorithm
[white]. indicating the usefulness
could he used for predicting the manner in of evolutionary techniques.
which a ligand would dock into HIV- I pro-
tease as a potential drug against AIDS. The
(Images from "Molecular recog-
task was to find the most energetically favor- nition of the inhibitor AG-1343
able conformation for the ligand when it by HIV-1 protease: Conformation-
attached to the binding site of the protein. ally flexible docking by evolu-
Several factors were believed to be imDor- tionary programming." by Daniel
tant, including: K. Gehlhaar e t al., Chemistry &
The steric (three-dimensional) fit of the Biology, vol. 2, 1995.)
ligand, bared on how well it complements
the surface structure of the binding site on
the protein. ers. Figure 5 shows a successful evolution of saves time and labor, both of which are typ-
Electrostatic interactions between atoms. the ligand, named A G 1343, docking into ically in short supply. When a pharmaceu-
Van der Waal's forces, which dictate essen~ HIV-1 protease after completion of the evo- tical company targets a protein, there are
tially that two atoms cannot he in one and lutionaly algorithm and after post-process- many more potential ligands to consider
the same place. ing by a gradient minimization on each than could be examined enumeratively.Each
In light of these criteria, any potential con- rotation angle, Comparing it to the experi- possible ligand may have several rotatable
formation of a ligand can he scored in terms mentally determined crystal structure i n d i ~ bonds, offering a multidimensional opti-
of how well it hinds to the target protein. cates that there is very little discrepancy mization problem of finding the hest over-
An evolutionary algorithm was devised between the prcdicted conformation de- a11 conformation. Rathcr than trying to look
to operate on a population of over 1000 dif- rived from the evolutionary algorithm and at each rotation individually, an evolution-
ferent potential conformations of a candi- the actual ohsewation gleaned from nature. ary algorithm can quickly discover which
date ligand, with variation randomly alter- In summary, the experiments give indepen- ligands best fit the target protein's binding
ing the rotation angles of the ligand, and dent verification that the evolutionary algo- site and therefore have potential as candi-
selection sorting out which shapes appeared rithm did indeed find the correct answer. date drugs. Chemists can then focus their
to be more energetically favorable than o t h ~ This application is importani because it attention on these candidates. T h e result

FOGEL I WHAT IS EVOLUTIONARY COMPUTATION?

31
161 In a recent game of ary approaches. With natural evolution,
checkers played on-line,
individuals are always being evaluated in
DavidllOl'r mover were
parallel by their environment. So b y rely-
rewetly determined by an
ing on a cluster of computers (a so-called
evolved neural network.
"pile of PCS"),a practitioner can take less
whereas Deceplicon12345
time t o solve difficult problems; evaluate
was a human competitor
individual solutions i n parallel rather than
with an expen rating.
in series; and have them migrate from com-
At the point in the
puter to computer. The v e y large popula-
game shown. the neural
tions this protocol allows yield better solu-
network. playing white,
tions faster than do small ones.
has just earneda king and
The products that will emerge from these
the human expert has
massively parallel designs remain a matter
acknowledged the move
of speculation. Some even foresee the use
by typing "gm," meaning
of evolutionay algorithms to design surro-
gate artificial brains that w i l l supplant
human cognition, as did Ray Kurzweil in
Thr&r ajSpintual Computing (Viking, 1999).
Whether or not this conjecture becomes
is an improved, more cost-effective proce- and I are poor players). Aher 100 genera- reality, there i s no doubt that evolutionaly
dure for screening drug leads. tions, the best-evolved network played algorithms will become a mainstay of prob-
Other real applications of evolutionary against people a t the Microsoft Network lem solving in the coming yean. e
algorithms are being made i n scheduling, Gaming Zone Web site (ww.zone.cam)
supply-chain management, and medical and, based on the number of games played TO PROBE FURTHER
diagnosis. They may even be used as the and the network's performance, earned a An introduction t o the a r t of designing and
basis for combat simulations t o train mili- "Class A" ratinz-ne level below the des- applying evolutionary algorithms t o real-
tary personnel; an evolutionary algorithm ignation of "expert" following the United world problems is provided in How to Solve
can serve as an interactive adversary that States Chess Federation rating system. Of it: Modern Heuristics byzbigniew Michalewicz
learns and adapts its tactics as the strategic course, opponents were not told that they and David B. Fogel (Springer, 2000). The book
situation changes. Moreover, evolution i s were playing against a program, nor did any also details Classic optimization techniques.
being used as a principle for creating new of them guess that their rival was a program.
hardware designs, in terms of both physical In fact, some people praised the apparent Recent advances in evolutionary computation
devices and electronic circuits. ingenuity of their competitor [Fig. 61. are found in many conferences and symposia
In the longer run, evolutionary algo- The crucial achievement of this work is internationally. including the annual Congress
rithms may play a pivotal role in devising that the neural networks did not assess their on EvolutionaryComputation, co-sponsoredby
truly intelligent machines--computers that position by using human expertise-piece the IEEE Neural Networks Council. the IEE, and
can learn on their own. The real challenge mobility, control of the center of the board, the Evolutionary Programming Society. Infor-
i s not so much to make a computer that can having a path that can earn a king, or any mation on the upcoming conference can be
compete with people; in many narrowly other sophisticated technique a human found a t ha~Jlpcgipseca.cee.hw.ac.u~~e~O~l
focused applications, such as games, this i s checkers player would use. Instead, they
easily accomolished b v translating human used only the actual Dositions of the Dieces "Evolution, neural networks, games, and intel-
ligence" are discussed in the paper of that title
by Kumar Chellapilla and David 8. Fogel in the
Proceedings of the IEEE, Vol. 87. no. 9,
pp. 1471-96, September, 1999.

The relationship between evolutionary algo-

rithmrand machine intelligence is explored in
Evolutionary Computation: Toward a New
Philosophy of Machine intelligence by D.B.
Fogel (2nd edition, IEEE Press, Piscataway, N.J.,
ers Experiments are continuing in an effort 2000). Many of the foundational papers in the
to achieve a n expert-level rating, which field. a s far back as 1956. are reprinted In
would place the evolved strategy in the top EvolutionaryComputation: The Fossil Record
1 percent of all players who have registered (D.B. Fogel, editor, IEEE Press, 1998).
a t the Internet gaming site.
The ability for a computer to gain pro- ABOUT THE AUTHOR
ficiency at a game of skill such as checkers David B. Fogel is executive vice president and
simply b y teaching itself opens up a far chief scientist of Natural Selection Inc., located
greater possibility: having computers learn in La Joila, Calif. He is the founding editor-
about other facets of the real world without in-chief of the IEEE Transactions on EYO~U-
relying on people t o program in all the req- tlanary Computationand the author of more
uisite knowledge. As the cost of computing than 180 publications In the field, including
keeps declining, the amenability of evolu- four books. He was elected a Fellow of the
tionary algorithms t o solving real-world IEEE in 1999.
problems will soar. This i s particularly so
because of'the parallel nature of evolution- Spectrum editor: Richard Camerford

IEEE SPECTRUM FEBRUARY 2000