Lecture IX: Evolution

Markus M. Mbius o March 15, 2003

Introduction

For models of learning we typically assume a xed number of players who nd out about each others intentions over time. In evolutionary models the process of learning is not explicitly modeled. Instead, we assume that strategies which do better on average are played more often in the population over time. The biological explanation for this is that individuals are genetically programmed to play one strategy and their reproduction rate depends on their tness, i.e. the average payo they obtain in the game. The economic explanation is that there is social learning going on in the background people nd out gradually which strategies do better and adjust accordingly. However, that adjustment process is slower than the rate at which agents play the game. We will focus initially on models with random matching: there are N agents who are randomly matched against each other over time to play a certain game. Frequently, we assume that N as innite. We have discussed in the last lecture that random matching gives rise to myopic play because there are no repeated game concerns (Im unlikely to ever encounter my current opponent again). We will focus on symmetric n by n games for the purpose of this course. In a symmetric game each player has the same strategy set and the payo matrix satises ui (si , sj ) = uj (sj , si ) for each player i and j and strategies si , sj Si = Sj = {s1 , .., sn }. Many games we encountered so far in the 1

course are symmetric such as the Prisoners Dilemma, Battle of the Sexes, Chicken and all coordination games. In symmetric games both players face exactly the same problem and their optimal strategies do not depend on whether they play the role of player 1 or player 2. An important assumption in evolutionary models is that each agent plays a xed pure strategy until she dies, or has an opportunity to learn and about her belief. The game is fully specied if we know the fraction of agents who play strategy s1 , s2 , .., sn which we denote with x1 , x2 , .., xn such that n i=1 xi = 1.

Mutations and Selection

Every model of evolution relies on two key concepts - a mutation mechanism and a selection mechanism. We have already discussed selection - strategies spread if they give above average payos. This captures social learning in a reduced form. Mutations are important to add noise to the system (i.e. ensure that xi > 0 at all times) and prevent it from getting stuck. For example, in a world where players are randomly matched to play a Prisoners Dilemma mutations make sure that it will never be the case that all agents cooperate or all agents defect because there will be random mutations pushing the system away from the two extremes.1

Replicator Dynamics

The replicator dynamics is one particular selection mechanism which captures the notion that strategies with above average payo should spread in the population. Typically, the replicator dynamics is modelled without allowing for mutations - the dynamics therefore becomes deterministic. Since the stage game is symmetric we know that u (si , sj ) = u1 (si , sj ) = u2 (sj , si ) . The average payo of strategy si for a player is u (si , x) since he is randomly matched with probability xj against agents playing sj (x =
If we start from an all-cooperating state, mutating agents will defect and do better. Hence they spread, and nally take over. Without mutations the system might be stuck in the all-cooperating state.
1

n (x1 , x2 , .., xn )). The average payo of all strategies is i=1 xi u (si , x) = u (x, x). Strategy si does better than the average strategy if u (si , x) > u (x, x), and worse otherwise. A minimum requirement for a selection mechanism is that sgn (xi (t)) = sgn [u (si , x) u (x, x)]. The share xi increases over time if and only if strategy si does better than average. The replicator dynamics is one particular example:

Denition 1 In the replicator dynamics the share xi of the population playing strategy si evolves over time according to: xi = u (si , x) u (x, x) xi If xi = 0 at time 0 then we have xi = 0 at all subsequent time periods: if nobody plays strategy si then the share of population playing it can neither decrease not increase. The next proposition makes sure that the denition is consistent (i.e. population share always sum up to 1). Proposition 1 The population shares always sum up to 1. Proof: We can write: xi = xi u (si , x) xi u (x, x) = u (x, x) u (x, x) = 0

This establishes that xi = const. The constant has to be 1 because the population shares sum up to 1 initially.

3.1

Steady States and Nash Equilibria

Denition 2 The strategy is a steady state if for xi = i (si ) we have dx = 0. dt Proposition 2 If is the strategy played by each player in a symmatric mixed NE then it is a steady state. Proof: In a NE each player has to be indierent between the strategies in her support. Therefore, we have u (si , x) = u (x, x). Note, that the reverse is NOT true. If all players cooperate in a Prisoners Dilemma this will be a steady state (since there are no mutations). 3

3.1.1

Example I

In 2 by 2 games the replicator dynamics is easily understood. Look at the following game:
A B

0,0

1,1

1,1

0,0

There are only two types in the population and x = (xA , xB ). Its enough to keep track of xA . xA = xB 2xA xB = (1 xA ) (1 2xA ) xA (1)

1 Its easy to see that xA > 0 for 0 < xA < 1 and xA < 0 for 1 > xA > 2 . This 2 makes xA = 1/2 a stable equilibrium (see below for a precise denition).

3.1.2

Example II

Now look at the New York game.

1,1

0,0

0,0

1,1

We can show: xE = xE (xE xE + xC xC ) = (1 xE ) (2xE 1) xE Now the steady state xE =

(2)

is unstable.

3.2

Stability

Denition 3 A steady state is stable if for all > 0 there exists > 0 such that if the process starts a distance away from the steady state it will never get further away than . The mixed equilibrium is stable in example I and unstable in example II. Denition 4 A steady state is asymptotically stable if there exists some > 0 such that the process converges to if it starts from a distance at most away from the steady state. The mixed equilibrium in example I is asymptotically stable. Denition 5 A steady state is globally stable if the process converges to the steady state from any initial state where xi > 0 for all si . Stability can be interpreted as a form of equilibrium selection. Theorem 1 If the steady state is stable then it is a NE. 5

Proof: Assume not. Then there exists a protable deviation si such that u (si , ) u (, ) = b > 0. Because the linear utility function is uniformly continuous there exists some such that for all x a distance less b than away we have |u (si , x) u (si , )| < 4 and |u (x, x) u (, )| < b b . This implies that |u (si , x ) u (x, x)| > 2 . Because is stable 4 we know that for x close enough to (less than a distance the dy1 namics converges to . So take x (0) = 1 1 x + 2 si . Then we 2
b have dxi = xi (u (si , x) u (x, x)) xi 2 dt x (t) which is a contradiction.
b 2

. But this implies that

3.2.1

Example III

This part is harder and is NOT required for the exam. The mixed equilibrium in the RPS game is stable but not asymptotically stable. The RPS game is harder to analyze because each player has three possible strategies. This implies that there are two dierential equations to keep track of.

0,0

1,1

1,1

1,1

0,0

1,1

1,1

1,1

0,0

We can concentrate on xR and xP . The average payo is u (x, x) = xR [xP + xS ] + xP [xR xS ] + xS [xR + xP ] = 0 (3) (4)

We then get: xR = xP + xS xR xP = xR xS xP (5) (6)

The unique mixed equilibrium 1 , 1 , 1 is the only steady state of the system. 3 3 3 To see whether it is stable we have to linearize the system around the steady state. We dene xi = xi 1 and obtain: 3 3xR = R 2P x x 3xP = 2R + xP x (7) (8)

We have to nd the eigenvalues of the system which are = 3i. This implies that the population shares circle around the steady state.

ESS-Evolutionary Stable States

The ESS concept concentrates on the role of mutations. Intuitively, the stability notion in the replicator dynamics is already a selection criterion because if the system is disturbed a little bit, it moves away from unstable steady states. The ESS concept expands on this intuition. Denition 6 The state x is ESS if for all y = x there exists u (x, (1 ) x + y) > u (y, (1 ) x + y) for all 0 < < . such that

This denition captures the idea that a stable state is impervious to mutations since they do worse than the original strategy. Proposition 3 x is ESS i for all y = x either (a) u (x, x) > u (y, x) or (b) u (x, x) = u (y, x) and u (x, y) > u (y, y). The following results establish the link between Nash equilibrium and ESS. Proposition 4 If x is ESS then it is a NE. Proposition 5 If x is a strict NE then it is an ESS. 7

Proposition 6 If x is a totally mixed ESS then it is the unique ESS. The next theorem establishes the link between ESS and stability under the replicator dynamics. Theorem 2 If x is an ESS then it is asymptotically stable under the Replicator Dynamics.

Stochastic Stability

Stochastic stability combines both mutations and a selection mechanism. It provides a much more powerful selection mechanism for Nash equilibria than ESS and the replicator dynamics. The ESS concept gives us some guidance as to which Nash equilibria are stable under perturbations or experimentation by agents. The replicator dynamics tells us how a group of agents can converge to some steady state starting from initial conditions. However, selection relies in many cases on the initial conditions (take the coordination game, for example). In particular, we cannot select between multiple strict equilibria. For this section we concentrate on generic coordination games:
A B

a,a

b,c

c,b

d,d

We assume, that a > c and d > b such that both (A, A) and (B, B) are NE of the game. We assume that a + b > c + d such that A is the risk-dominant 8

strategy for each player. Note, that the risk-dominant NE is not necessarily the Pareto optimal NE. We have seen in experiments that agents tend to choose A. Can we justify this with an evolutionary story? YES! Assume that there is a nite number n of agents who are randomly matched against each other in each round. Assume that agents choose the best response to whatever strategy did better in the population in the last period. This is called the BR dynamics. Clearly, all agents will choose A if db 1 xA > q = ab+dc . Because A is risk-dominant we have q < 2 . There are also mutations in each period. Specically, with probability each agent randomizes between both strategies. We dene the basin of attraction of xA = 1 (everybody plays A) to be BA = [q , 1] and the basin of attraction of xA = 0 to be [0, q ]. Whenever the initial state of the system is within the basin of attraction it converges to all A/ all B for sure if there are no mutations. We dene the radius of the basin BA to be the number of mutations it takes to jump out of the state all A. We get RA = (1 q ) n. Similarly, we dene the co-radius CRA as the number of mutations it takes at most to jump into the basin BA . We get CRA = q n. Theorem 3 If CRA < RA then the state all A is stochastically stable. The waiting time to reach the state all A is of the order CRA . Therefore, the risk-dominant equilibrium is stochastically stable as q < 1 . 2

5.1

The Power Local Interaction

Local interaction can signicantly speed up the evolution of the system. Assume, that agents are located on the circle and play the BR to average play of their direct neighbors in the previous period. It can be shown that CRA = 2 and CRA = n . The convergence is a lot faster than under global interaction. 2