BELIEFS & EVOLUTIONARY GAME THEORY

Heinrich H. Nax Bary S. R. Pradelski

&
[email protected] [email protected]

May 15, 2017: Lecture 10

1 / 32
 Lecture 6: Beliefs and evolutionary game theory

Plan

Normal form games

Equilibrium invariance
Equilibrium refinements
Dynamic games
Extensive form games
Incomplete information
Sub-game perfection
Beliefs and Bayes
Sequential and perfect Bayesian equilibria
Evolutionary game theory

2 / 32
 Lecture 6: Beliefs and evolutionary game theory

Selten’s horse
0 3 0 4 1
0 2 0 4 1
0 2 1 0 1

L R L R
3
R
L

L R

s = (L, R, R) is a subgame perfect equilibrium.

Player 2’s move is inconsequential and hence does not need to be examined.
But why does player 2 play R if his best reply to s is L?
Moreover if player 1 expects this then he should play R.
But if player 3 realizes all that he should deviate and play L.

The subgame perfect equilibrium is not “self-enforcing”.

3 / 32
 Lecture 6: Beliefs and evolutionary game theory

Beliefs

Definition: Beliefs (Kreps and Wilson 1982)

A belief system in an extensive form game is a function µ that maps all

actions in each information set to a probability distribution, that is:
X
µ(a) = 1 ∀D ∈ D
a∈D

A belief system thus encodes players’ expectations of each others’ play in the
whole game (not just along one path).

4 / 32
 Lecture 6: Beliefs and evolutionary game theory

Weakly consistent and sequentially rational

Definition: weakly consistent

A belief system µ is weakly consistent with a behavior strategy profile y

if µ agrees with the conditional probability distribution µ(·|y) induced
by y over the information sets on its path.
Suppose we are in information set I, then:

P[a, y]
µ(a|y) = ∀a ∈ I
P[I, y]

That is, the beliefs are computed via Bayes’ rule along the path of play.

We shall assume that all other probabilities are chosen freely.

5 / 32
 Lecture 6: Beliefs and evolutionary game theory

Perfect Bayes equilibrium

Definition: sequentially rational

A behavior strategy y∗ is sequentially rational under a belief system µ

if for every player i ∈ I and information set D ∈ Di :
X
y∗i ∈ arg max µ(a)Uia (yi , y∗−i )
yi ∈Yi a∈D

That is, each player chooses optimally given his beliefs at each infor-
mation set and the others’ equilibrium strategies.

Definition: Perfect Bayesian equilibrium

A behavior strategy profile y∗ is a perfect Bayesian equilibrium if there

exists a weakly consistent belief system under which y∗ is sequentially
rational for every player.

6 / 32
 Lecture 6: Beliefs and evolutionary game theory

Back to Selten’s horse

0 3 0 4 1
0 2 0 4 1
0 2 1 0 1

L R L R
3
R
L

L R

Observation 1: (L, R, R) is not a perfect Bayes equilibrium:

If player 2 gets to move then by playing R he is not sequentially rational.

7 / 32
 Lecture 6: Beliefs and evolutionary game theory

Back to Selten’s horse

0 3 0 4 1
0 2 0 4 1
0 2 1 0 1

L R L R
3
R
L

L R

Observation 2: A perfect Bayes Nash equilibrium is given by (R, R, L).

Suppose that player 3 holds belief µ̃ = µplayer 1 plays L ≤ 1/3 then his expected
payoff from R is less than that from L:
2
µ̃ · 2 + (1 − µ̃) · 0 ≤ ≤ µ̃ · 0 + (1 − µ̃) · 1
3
⇒ As player 3 chooses L, both other players are better off playing R.
8 / 32
 Lecture 6: Beliefs and evolutionary game theory

Quick aside on Bayes’ rule

Bayes’ rule is central to game theory in its treatment of imperfect information.

Consider the following example:

The prior probability of a disease is 0.9%.

If the disease is present then the test is positive with 90% chance.
The test also gives a false positive with 7% probability.

How likely is it that the patient is ill if the test is positive?

In a study by Gigerenzer (Simon & Schuster 2002), 95% of American

medical doctors guessed the answer to be ca. 75%.

9 / 32
 Lecture 6: Beliefs and evolutionary game theory

Quick aside on Bayes’ rule

Bayes’ Theorem

Let A and B be events and P(B) 6= 0. Then

P(B|A)P(A)
P(A|B) =
P(B)

P(disease) =: P(A) = 0.9%

P(test positive) =: P(B)
P(B|A) = 90%
P(B) = P(B|A) · P(A) + P(B|¬A)P(¬A) =
90% · 0.9% + 7% · 99.1 ≈ 7.7%
P(A|B) = 10.5%

10 / 32
 Lecture 6: Beliefs and evolutionary game theory

Quick aside on Bayes rule: Intuition

A frequencies approach:

Imagine 1000 patients, 9 sick, 991 healthy

Of the 9 sick, 90% · 9 ≈ 8 test positive
Of the 991 healthy, 7% · 991 ≈ 69 test positive.
Of the total 77 with positive tests, 8/77 ≈ 10.5% are ill.

Bayes’ Rule is a great tool for everyday life!

11 / 32
 Lecture 6: Beliefs and evolutionary game theory

Entry deterrence game

Entry deterrence
I unaware if sane/crazy
The set of PBE includes
(Out, F) supported by I’s
beliefs µs = 0, µc = 1.

But E does not observe Nature’s move, so how could his action convey
information about it and change I’s prior beliefs?
12 / 32
 Lecture 6: Beliefs and evolutionary game theory

Consistency and sequential equilibrium

Definition: consistent
A belief system µ is consistent with a behavior strategy profile y∗ if
there exists a sequence of interior behavior strategy profiles yt → y∗
such that µ(a|yt ) → µ∗ (a) for all a where µ(·|yt ) is the belief system
induced by Bayes’ law from yt .

Definition: Sequential equilibrium

A behavior strategy profile y∗ is a sequential equilibrium if there exists

a consistent belief system under which y∗ is sequentially rational for
every player.

13 / 32
 Lecture 6: Beliefs and evolutionary game theory

Existence and inclusions

Theorem (Fudenberg & Tirole 1991)

Every finite game has a perfect Bayesian equilibrium.

Theorem (Kreps & Wilson 1982)

Every finite game has a sequential equilibrium.

Sequential equilibrium ⊆ perfect Bayesian equilibria ⊆ subgame per-
fect equilibria

Proposition

Every subgame perfect equilibrium is a Nash equilibrium of the associ-

ated strategic form game.

14 / 32
 Lecture 6: Beliefs and evolutionary game theory

Key learnings

Dynamic games take timing and information into consideration

Equilibrium refinements are needed to make “reasonable” predictions

When converting to reduced normal form, remember that strategies are

complete contingent plans

Perfect Bayesian equilibrium and sequential equilibrium are not only

strategy profiles but also (Bayesian or consistent) beliefs at every
information set

15 / 32
 Lecture 6: Beliefs and evolutionary game theory

Common knowledge of rationality and the game

Suppose that players are rational decision makers and that mutual rationality
is common knowledge, that is:
I know that she knows that I will play rational
She knows that “I know that she knows that I will play rational”
I know that “She knows that “I know that she knows that I will play
rational””
...
Further suppose that all players know the game and that again is common
knowledge.

16 / 32
 Lecture 6: Beliefs and evolutionary game theory

Rationality and the “as if” approach

The rationalistic paradigm in economics (Savage, The Foundations of

Statistics, 1954)
A person’s behavior is based on maximizing some goal function (utility)
under given constraints and information

The “as if” approach (Friedman, The methodology of positive economics,

1953)
Do not theorize about the intentions of agents’ actions but consider only
the outcome (observables)
Similar to the natural sciences where a model is seen as an approximation
of reality rather than a causal explanation (e.g., Newton’s laws)

But is the claim right? Do people act (as if) they where rational?

17 / 32
 Lecture 6: Beliefs and evolutionary game theory

Nash’s mass-action interpretation (Nash, PhD thesis,1950)

“We shall now take up the “mass-action” interpretation of equilibrium

points. In this interpretation solutions have no great significance. It is
unnecessary to assume that the participants have full knowledge of the total
structure of the game, or the ability and inclination to go through any
complex reasoning processes. But the participants are supposed to
accumulate empirical information on the relative advantages of the various
pure strategies at their disposal.

...

Thus the assumption we made in this “mass-action” interpretation lead to the

conclusion that the mixed strategies representing the average behavior in
each of the populations form an equilibrium.”
(bold text added for this presentation)

18 / 32
 Lecture 6: Beliefs and evolutionary game theory

Nash’s mass-action interpretation (Nash, PhD thesis,1950)

A large population of identical individuals represents each player role in

a game
The game is played recurrently (t = 0, 1, 2, 3, ...):
In each period one individual from each player population is drawn
randomly to play the game
Individuals observe samples of earlier behaviors in their own population
and avoid suboptimal play

Nash’s claim: If all individuals avoid suboptimal pure strategies and the
population distribution is stationary then it constitutes a [Nash] equilibrium

Almost true! Evolutionary game theory formalizes these questions and

provides answers.

19 / 32
 Lecture 6: Beliefs and evolutionary game theory

The folk theorem of evolutionary game theory

Folk theorem
If the population process converges from an interior initial state,
then for large t (in the limit) the distribution is a Nash equilibrium
If a stationary population distribution is stable, the it coincides
with a Nash equilibrium

Charles Darwin: “Survival of the fittest”

The population which is best adapted to environment (exogenous) will
reproduce more

Evolutionary game theory

The population which performs best against other populations (endogenous)
will survive/reproduce more

20 / 32
 Lecture 6: Beliefs and evolutionary game theory

Domain of analysis

Symmetric two-player games

A symmetric two-player normal form game G = hN, {Si }i∈N , {ui }i∈N i
consists of three object:
1 Players: N = {1, 2}, with typical player i ∈ N.
2 Strategies: S1 = S2 = S with typical strategy s ∈ S.
3 Payoffs: A function ui : (h, k) → R mapping strategy profiles to a
payoff for each player i such that for all h, k ∈ S:

u2 (h, k) = u1 (k, h)

21 / 32
 Lecture 6: Beliefs and evolutionary game theory

Battle of the Sexes

Cafe Pub
Cafe 4, 3 0, 0
Pub 0, 0 3, 4

Not symmetric since:

u1 (Cafe, Cafe) 6= u2 (Cafe, Cafe)

22 / 32
 Lecture 6: Beliefs and evolutionary game theory

Prisoner’s dilemma

Cooperate Defect
Cooperate −1, −1 −8, 0
Defect 0, −8 −5, −5

Symmetric since:

u1 (Cooperate, Cooperate) = u2 (Cooperate, Cooperate) = −1

u1 (Cooperate, Defect) = u2 (Defect, Cooperate) = −8
u1 (Defect, Cooperate) = u2 (Cooperate, Defect) = 0
u1 (Defect, Defect) = u2 (Defect, Defect) = −5

23 / 32
 Lecture 6: Beliefs and evolutionary game theory

Symmetric Nash equilibrium

Definition: Symmetric Nash Equilibrium

A symmetric Nash equilibrium is a strategy profiles σ ∗ such that for

every player i,
ui (σ ∗ , σ ∗ ) ≥ ui (σ, σ ∗ ) for all σ

In words: If no player has an incentive to deviate from their part in a

particular strategy profile, then it is Nash equilibrium.

Proposition

In a symmetric normal form game there always exists a symmetric Nash

equilibrium.

Note: Not all Nash equilibria of a symmetric game need to be symmetric.

24 / 32
 Lecture 6: Beliefs and evolutionary game theory

Evolutionarily stable strategy (Maynard Smith and Price,

1972)

Definition: Evolutionarily stable strategy (ESS)

A mixed strategy σ ∈ ∆(S) is an evolutionarily stable strategy (ESS)

if for every strategy τ 6= σ there exists ε(τ ) ∈ (0, 1) such that for all
ε ∈ (0, ε(τ )):

U(σ, ετ + (1 − ε)σ) > U(τ, ετ + (1 − ε)σ)

Let ∆ESS be the set of evolutionarily stable strategies.

25 / 32
 Lecture 6: Beliefs and evolutionary game theory

Alternative representation

Note that an ESS needs to be a best reply to itself, thus ∆ESS is a subset of the
set of Nash equilibria.

Proposition

A mixed strategy σ ∈ ∆(S) is an evolutionarily stable strategy (ESS)

if:

U(τ, σ) ≤ U(σ, σ) ∀τ
U(τ, σ) = U(σ, σ) ⇒ U(τ, τ ) < U(σ, τ ) ∀τ 6= σ

26 / 32
 Lecture 6: Beliefs and evolutionary game theory

Prisoner’s dilemma

Cooperate Defect
Cooperate −1, −1 −8, 0
Defect 0, −8 −5, −5

∆ESS = {Defect}

27 / 32
 Lecture 6: Beliefs and evolutionary game theory

Coordination game

A B
A 4, 4 0, 0
B 0, 0 1, 1

Nash equilibria:
(A, A), (B, B), (0.2 · A + 0.8 · B, 0.2 · A + 0.8 · B)

All Nash equilibra are symmetric.

But the mixed Nash equilibrium is not ESS:

L performs better against it!

Note that the mixed Nash equilibrium is trembling-hand perfect.

28 / 32
 Lecture 6: Beliefs and evolutionary game theory

Existence of ESS not guaranteed

Example: Rock, paper, scissors

R P S
R 0, 0 −1, 1 1, −1
P 1, −1 0, 0 −1, 1
S −1, 1 1, −1 0, 0

Unique Nash equilibrium and thus symmetric:

σ = ( 13 R, 13 P, 13 S)

All pure strategies are best replies and do as well against themselves as σ
does against them ⇒ Not an ESS!

29 / 32
 Lecture 6: Beliefs and evolutionary game theory

Relations to normal form refinements

Propositions

If σ ∈ ∆(S) is weakly dominated, then it is not evolutionarily

stable.
If σ ∈ ∆ESS , then (σ, σ) is a perfect equilibrium.
If (σ, σ) is a strict Nash equilibrium, then σ is evolutionarily
stable.

30 / 32
 Lecture 6: Beliefs and evolutionary game theory

Summary

Evolutionary game theory studies mutation (ESS) and selection process

(not in this lecture)
The stable states often coincide with solution concepts from the
“rational” framework
Evolutionary game theory does not explain how a population arrives at
such a strategy
⇒ Learning in games and behavioral game theory

The “best” textbook: Weibull, Evolutionary game theory, 1995

31 / 32
 Lecture 6: Beliefs and evolutionary game theory

THANKS EVERYBODY
See you next week!
And keep checking the website for new materials as we progress:
http://www.coss.ethz.ch/education/GT.html

32 / 32