SF2972: Game theory

Mark Voorneveld, [email protected]

February 2017

Topic: extensive form games.

Purpose: (1) explicitly model situations in which players move
sequentially and their information; (2) formulate appropriate
equilibrium notions.
Textbook (Peters): chapters 4, 5, 14.
Notes (Weibull): chapter 3
Reading guide at end of these slides.
Defining games and strategies

Drawing a game tree is usually the most informative way to represent

an extensive form game. Here is one with an initial (c)hance move:

Mark Voorneveld Game theory SF2972, Extensive form games 1/64

Extensive form game: formal definition
A (directed, rooted) tree; i.e. it has a well-defined initial node.
Nodes can be of three types:
1 chance nodes: where chance/nature chooses a branch
according to a given/known probability distribution;
Let τ assign to each chance node a prob distr over feasible
branches.
2 decision nodes: where a player chooses a branch;
3 end nodes: where there are no more decisions to be made and
each player i gets a payoff/utility given by a utility function ui .
A function P assigns to each decision node a player i in player
set N who gets to decide there.
Decision nodes P −1 (i) of player i are partitioned into
information sets.
Nodes in an information set of player i are ‘indistinguishable’
to player i; this requires, for instance, the same actions in
each decision node of the information set.
If h is an information set of player i, write P(h) = i and let
A(h) be the feasible actions in info set h.
Mark Voorneveld Game theory SF2972, Extensive form games 2/64
Extensive form game: formal definition

In summary, an extensive form game is a tuple (T , N, P, H, A, τ, (ui )i∈N )

with
T the tree,
N the set of players,
P a mapping from decision nodes to the players,
H the collection of information sets,
A the collection of actions in these information sets,
τ the prob distr over actions in nature’s/chance’s nodes,
(ui )i∈N the players’ (vNM) utility functions.

Mark Voorneveld Game theory SF2972, Extensive form games 3/64

Notational conventions
p. 253: “Clearly, this formal notation is quite cumbersome
and we try to avoid its use as much as possible. It is only
needed to give precise definitions and proofs.” Draw tree!
Nodes in same information set: dotted lines between them
(Peters’ book) or enclosed in an oval (my drawings).
Since nodes in an information set are indistinguishable,
information sets like

are not allowed: since there are two branches in the left node
and three in the right, they are easily distinguishable.
Mark Voorneveld Game theory SF2972, Extensive form games 4/64
We call an extensive form game finite if it has finitely many nodes.
An extensive form game has
perfect information if each information set consists of only
one node.
perfect recall if each player recalls exactly what he did in the
past.
Formally: on the path from the initial node to a decision node
x of player i, list in consecutive order which information sets
of i were encountered and what i did there. Call this list the
experience Xi (x) of i in node x. The game has perfect recall if
nodes in the same information set have the same experience.
otherwise, the game has imperfect information/recall.
Convention: we often characterize nodes in the tree by describing
the sequence of actions that leads to them. For instance:
the initial node of the tree is denoted by ∅;
node (a1 , a2 , a3 ) is reached after three steps/branches/actions:
first a1 , then a2 , then a3 .
Mark Voorneveld Game theory SF2972, Extensive form games 5/64
Imperfect recall: absentminded driver
Two crossings on your way home. You need to (C)ontinue on the
first, (E)xit on the second. But you don’t recall whether you already
passed a crossing.

Only one information set, {∅, C }, but with different experiences:

in the first node: X1 (∅) = ({∅, C })
in the second node:
X1 (C ) = ( {∅, C } , C , {∅, C } )
| {z } |{z} | {z }
1’s first info set choice there resulting info set
X1 (∅) 6= X1 (C ): imperfect recall!

Mark Voorneveld Game theory SF2972, Extensive form games 6/64

Second example of imperfect recall

Player 1 forgets the initial choice:

Different experiences in the two nodes of information set {L, R}:

in the left node: X1 (L) = ( ∅ , L , {L, R} )
|{z} |{z} | {z }
initial node choice there resulting info set
in the right node: X1 (R) = (∅, R, {L, R}).
X1 (L) 6= X1 (R): imperfect recall!

Mark Voorneveld Game theory SF2972, Extensive form games 7/64

Third example of imperfect recall
Player 1 knew the chance move, but forgot it:

Different experiences in the two nodes of information set {(L, C ), (R, C )}:

in the left node:

X1 ((L, C )) = ( {L} , C , {(L, C ), (R, C )})
|{z} |{z} | {z }
1’s first info set choice there resulting info set
in the right node: X1 ((R, C )) = ({R}, C , {(L, C ), (R, C )}).
X1 ((L, C )) 6= X1 ((R, C )): imperfect recall!
Mark Voorneveld Game theory SF2972, Extensive form games 8/64
Pure, mixed, and behavioral strategies

A pure strategy of player i is a function si that assigns to each

information set h of player i a feasible action si (h) ∈ A(h).
A mixed strategy of player i is a probability distribution σi
over i’s pure strategies.
σi (si ) ∈ [0, 1] is the prob assigned to pure strategy si .
‘Global randomization’ at the beginning of the game.
A behavioral strategy of player i is a function bi that assigns
to each information set h of player i a probability distribution
over the feasible actions A(h).
bi (h)(a) is the prob of action a ∈ A(h).
‘Local randomization’ as play proceeds.
Let us consider the difference between these three kinds of strategies
in a few examples.

Mark Voorneveld Game theory SF2972, Extensive form games 9/64

The difference between mixed and behavioral strategies

Imperfect recall; 4 outcomes with payoffs a, b, c, and d.

Four pure strategies, abbreviated AC , AD, BC , BD.
Mixed strategies: probability distributions over the 4 pure
strategies. A vector (pAC , pAD , pBC , pBD ) of nonnegative
numbers, adding up to one, with px the probability assigned
to pure strategy x ∈ {AC , AD, BC , BD}.

Mark Voorneveld Game theory SF2972, Extensive form games 10/64

Behavioral strategies assign to each information set a
probability distribution over the available actions. Since pl. 1
has 2 information sets, each with 2 actions, it is summarized
by a pair (p, q) ∈ [0, 1] × [0, 1], where p ∈ [0, 1] is the
probability assigned to action A in the initial node (and 1 − p
to B) and q is the probability assigned to action C in
information set {A, B} (and 1 − q to D).
Mixed strategy (1/2, 0, 0, 1/2) assigns probability 1/2 to each
of the outcomes a and d. There is no such behavioral
strategy:
reaching a with positive probability requires that p, q > 0;
reaching d with positive probability requires p, q < 1;
hence also b and c are reached with positive probability.

Mark Voorneveld Game theory SF2972, Extensive form games 11/64

A trickier example: the absentminded driver revisited

Pure strategies: C with payoff 1 and E with payoff 0.

Mixed: let p ∈ [0, 1] be the prob of choosing pure strategy C
and 1 − p the prob of pure strategy E . Expected payoff: p.
Behavioral: let q ∈ [0, 1] be the prob of choosing action C in
the info set and 1 − q the prob of choosing E in the info set.
Expected payoff:
0 · (1 − q) + 4 · q(1 − q) + 1 · q 2 = q(4 − 3q).
No behavioral strategy is outcome-equivalent with p = 1/2
(why?)
No mixed strategy is outcome-equivalent with q = 1/2 (why?)
Mark Voorneveld Game theory SF2972, Extensive form games 12/64
Outcome-equivalence under perfect recall

Conclude: under imperfect recall, mixed and behavioral strategies

might generate different probability distributions over end nodes.
Perfect recall helps to rule this out. We need a few definitions:
Each profile b = (bi )i∈N of behavioral strategies induces an outcome
O(b), a probability distribution over end nodes.
How to compute O(b) in finite games?
The probability of reaching end node x = (a1 , . . . , ak ), described by
the sequence of actions/branches leading to it, is simply the product
of the probabilities of each separate branch:
k−1
Y
bP(a1 ,...,a` ) (a1 , . . . , a` )(a`+1 ).
`=0

Mark Voorneveld Game theory SF2972, Extensive form games 13/64

Likewise, each profile σ = (σi )i∈N of mixed strategies induces an
outcome O(σ), a probability distribution over end nodes.
How to compute O(σ) in finite games?
Let x = (a1 , . . . , ak ) be a node, described by the sequence of
actions/branches in the game tree leading to it.
Pure strategy si of player i is consistent with x if i chooses
the actions described by x: for each initial segment
(a1 , . . . , a` ) with ` < k and P(a1 , . . . , a` ) = i:
si (a1 , . . . , a` ) = a`+1 .
The prob of i choosing a pure strategy si consistent with x is
X
πi (x) = σi (si ),
with summation over the si consistent with x.
Similar for nature, whose behavior is given by function τ .
The probability of reaching end node x is
Y
πi (x).
i∈N∪{c}

Mark Voorneveld Game theory SF2972, Extensive form games 14/64

A mixed strategy σi and a behavioral strategy bi of player i are
outcome-equivalent if — given the pure strategies of the remaining
players — they give rise to the same outcome:

for all s−i : O(σi , s−i ) = O(bi , s−i ).

Theorem (Kuhn, outcome equivalence under perfect recall)

In a finite extensive form game with perfect recall:
(a) each behavioral strategy has an outcome-equivalent mixed
strategy,
(b) each mixed strategy has an outcome-equivalent behavioral
strategy.

Mark Voorneveld Game theory SF2972, Extensive form games 15/64

Proof sketch:
(a) Given beh. str. bi , assign to pure strategy si the probability
Y
σi (si ) = bi (h)(si (h)),
h
with the product taken over all info sets h of pl i.
Intuition: si selects action si (h) in information set h. How
likely is that?
(b) Given mixed str. σi . Consider an info set h of pl i and a
feasible action a ∈ A(h). How should we define bi (h)(a)?
Consider any node x in info set h. The probability of choosing
consistent with x is πi (x).
Perfect recall: πi (x) = πi (y ) for all x, y ∈ h.
Define
πi (x, a)
bi (h)(a) = if πi (x) > 0 (and arbitrarily otherwise)
πi (x)
Intuition: conditional on earlier behavior that is consistent
with reaching information set h, how likely is i to choose
action a?
Mark Voorneveld Game theory SF2972, Extensive form games 16/64
Example of outcome equivalent strategies

Question: Which behavioral strategies are outcome-equivalent with

mixed strategy (pAC , pAD , pBC , pBD )?
In 1’s first information set, the prob that A is chosen is pAC + pAD .
In 1’s second information set, the prob that C is chosen is computed
as the probability of choosing C conditional on earlier behavior that
is consistent with this information set being reached:
pAC
. (arbitrary if pAC + pAD = 0)
pAC + pAD
Mark Voorneveld Game theory SF2972, Extensive form games 17/64
Example of outcome equivalent strategies

Question: Find a mixed strategy that is outcome equivalent with

the behavioral strategy choosing A with prob p and C with prob q.
(pAC , pAD , pBC , pBD ) = (pq, p(1 − q), (1 − p)q, (1 − p)(1 − q))
If p = 0, the 2nd info set is not reached: end node ‘B’ is reached
with prob 1. Only pure strategies BC and BD are consistent with
this node being reached. All mixed strategies with pBC + pBD = 1
are then outcome equivalent.
Mark Voorneveld Game theory SF2972, Extensive form games 18/64
Homework exercise 1

(a) Show that the game above has perfect recall.

(b) For each mixed strategy σ1 of player 1, find the
outcome-equivalent behavioral strategies.
(c) For each behavioral strategy b1 of player 1, find the
outcome-equivalent mixed strategies.

Mark Voorneveld Game theory SF2972, Extensive form games 19/64

On the definition of strategies

For later, think about the following: pure, mixed, and behavioral
strategies specify what happens in all information sets of a player.
Even in those information sets that cannot possibly be reached if
those strategies are used. Why do you think that is the case?

Mark Voorneveld Game theory SF2972, Extensive form games 20/64

Nash equilibrium

We can compute, for each profile of pure strategies, the

corresponding (expected) payoffs: every extensive form game
has a corresponding strategic/normal-form game.
Terminology: Jörgen used ‘normal-form game’, the book of
Peters uses ‘strategic form game’.
A pure/mixed Nash equilibrium of the extensive form game is
then simply a pure/mixed Nash equilibrium of the
corresponding strategic form game.
Nash equilibria in behavioral strategies are defined likewise: a
profile of behavioral strategies is a Nash equilibrium if no
player can achieve a higher expected payoff by unilaterally
deviating to a different behavioral strategy.

Mark Voorneveld Game theory SF2972, Extensive form games 21/64

Example: from extensive to strategic game

a b
This game (from previous lecture) has strategic form: A 4, −4 0, 0
B 3, −3 1, −1
Dominance solvable, unique Nash equilibrium (B, b).
Mark Voorneveld Game theory SF2972, Extensive form games 22/64
On the definition of strategies

I asked you to think about the following: pure, mixed, and behavioral
strategies specify what happens in all information sets of a player.
Even in those information sets that cannot possibly be reached if
those strategies are used. Why do you think that is the case?
Main reason: Nash equilibrium: does each player choose a best reply
to the others’ strategies?
If a player were to deviate, ending up in a different part of the game
tree, we need to know what happens there!

Mark Voorneveld Game theory SF2972, Extensive form games 23/64

Theorem (Equilibrium existence)
Every finite extensive form game with perfect recall has a Nash
equilibrium in mixed/behavioral strategies.

1 For mixed strategies: finite extensive form game gives finite

strategic game, which has a Nash equilibrium in mixed
strategies.
2 For behavioral strategies: by outcome-equivalence, we can
construct a Nash equilibrium in behavioral strategies.

Mark Voorneveld Game theory SF2972, Extensive form games 24/64

Strategic form analysis of extensive form games
The extensive form game

has corresponding strategic form

L R
A 0, 0 2, 1
B 1, 2 1, 2
Pure Nash equilibria: (B, L) and (A, R).
But if pl. 2 is called upon to play, would 2 choose L? This is an
implausible choice in the ‘subgame’ that starts at node A!
To rule out such implausible equilibria, require that an equilibrium
is played in each subgame: ‘subgame perfect equilibrium’
Mark Voorneveld Game theory SF2972, Extensive form games 25/64
Subgame perfect equilibrium

In an extensive form game with perfect information, let x be a

node of the tree that is not an end node. The part of the
game tree consisting of all nodes that can be reached from x
is called a subgame.
Each game is a subgame of itself. A subgame on a strictly
smaller set of nodes is called a proper subgame.
A subgame perfect equilibrium is a strategy profile that
induces a Nash equilibrium in each subgame.
In the game on the previous slide, only (A, R) is subgame perfect.

Mark Voorneveld Game theory SF2972, Extensive form games 26/64

Subgame perfect equilibria via backward induction

Subgame perfect equilibria are typically found by backward induc-

tion:
1 Start with subgames with only one decision left. Determine
the optimal actions there.
2 Next, look at subgames with at most two consecutive
decisions left. Conditioning on the previous step, the first
player to choose (say i) knows what a ‘rational’ player will do
in the subgame that starts after i’s choice, so it is easy to find
i’s optimal action.
3 Continue with subgames of at most 3 consecutive moves, etc.
This is the game-theoretic generalization of the dynamic program-
ming algorithm in optimization theory.

Mark Voorneveld Game theory SF2972, Extensive form games 27/64

Backward induction: example 1

Strategic form:
C D
AE 1, 3 1, 3
AF 1, 3 1, 3
BE 2, 1 0, 2
BF 2, 1 1, 0
Pure Nash equilibria: (AE , D), (AF , D), and (BF , C ).
Subgame perfect equilibrium: (BF , C )

Mark Voorneveld Game theory SF2972, Extensive form games 28/64

Backward induction: example 2

Dividing 2 indivisible objects. Pl. 1 proposes, pl. 2 accepts or

rejects.

How many pure strategies for player 1? 3

How many pure strategies for player 2? 23 = 8
Subgame perfect equilibria? ((2, 0), yyy ) and ((1, 1), nyy )

Mark Voorneveld Game theory SF2972, Extensive form games 29/64

Backward induction: example 3 (the ‘rotten kid’ game)
A child’s action a from some nonempty, finite set A affect
both her own payoff c(a) and her parents’ payoff p(a); for all
a ∈ A we have 0 ≤ c(a) < p(a).
The child is selfish: she cares only about the amount of
money she receives.
Her loving parents care both about how much money they
have and how much their child has. Specifically, model the
parents as a single player whose utility is the smaller of the
amount of money the parents have and the amount the child
has. The parents may transfer money to the child (pocket
money, trust fund, etc).
First the child chooses action a ∈ A.
Then the parents observe the action and decide how much
money x ∈ [0, p(a)] to transfer to the child. The game ends
with utility c(a) + x for the child and min{c(a) + x, p(a) − x}
to the parents.
Mark Voorneveld Game theory SF2972, Extensive form games 30/64
Show: in a subgame perfect equilibrium, the child takes an action
that maximizes the sum of her private income c(a) and her parents’
income p(a). Not so selfish after all!
In the subgame after action a ∈ A, the parents maximize
min{c(a) + x, p(a) − x} over x ∈ [0, p(a)].
This is done by choosing x such that c(a) + x = p(a) − x,
i.e., by x ∗ (a) = 12 (p(a) − c(a)).
Anticipating this, the child knows that action a ∈ A leads to
transfer x ∗ (a) and consequently utility
c(a) + x ∗ (a) = 12 (c(a) + p(a)). Maximizing this expression is
equivalent with maximizing c(a) + p(a).

Mark Voorneveld Game theory SF2972, Extensive form games 31/64

Finite trees: existence of subgame perfect equilibria

Using backward induction, if there are only finitely many nodes, the
first player to move — conditioning on the optimal behavior in the
smaller subgames — is optimizing over a finite set: an optimum will
always exist. Using this and induction on the ‘depth’ of the tree,
one can show:
Theorem (Existence of subgame perfect equilibria)
In a finite extensive form game with perfect information, there is
always a subgame perfect equilibrium in pure strategies.

That’s a pretty nice result:

1 no need to consider randomization
2 no implausible behavior in subgames
As an aside: what if there are infinitely many nodes?

Mark Voorneveld Game theory SF2972, Extensive form games 32/64

Subgame perfection and backward induction coincide

Theorem
In a finite extensive form game with perfect information, subgame
perfect equilibria and those found by backward induction are
identical.
Difficult! Main step is the ‘one-deviation property’: a strategy profile
is subgame perfect if and only if for each subgame the first player to
move cannot obtain a better outcome by changing only the initial
action.

Mark Voorneveld Game theory SF2972, Extensive form games 33/64

Centipede games
Although subgame perfect equilibria were introduced to rule out
implausible behavior in subgames, there are examples where such
equilibria lead to outcomes that some people find counterintuitive.
This is sometimes corroborated with experimental support. One
well-known example consists of Rosenthal’s centipede games, char-
acterized by the following properties:
Players 1 and 2 take turns during at most 2T rounds (T ∈ N).
At each decision node, the player can choose to (S)top or
(C)ontinue.
The game ends (i) if one of the players decides to stop, or (ii)
if no player has chosen stop after 2T periods.
For each player, the outcome when he stops the game in
period t is:
better than the outcome if the other player stops in period
t + 1 (or the game ends),
worse than any outcome that is reached if in period t + 1 the
other player continues.
Mark Voorneveld Game theory SF2972, Extensive form games 34/64
Centipede games
Here is an example of a centipede game with 6 periods:

It is tempting to continue the game if you can be sure that the other
player does so as well: the longer the game goes on, the higher the
payoffs.
But in the unique subgame perfect equilibrium, players choose (S)top
in each node. In particular, the game ends immediately in the initial
node.
Reason: in the final node, player 2’s best reply is to (S)top. Given
that 2 (S)tops in the final round, 1’s best reply is to stop one period
earlier, etc.
There are other Nash equilibria, but they all lead to the same out-
come: player 1 ends the game immediately.
Mark Voorneveld Game theory SF2972, Extensive form games 35/64
Subgame perfect equilibrium in games with imperfect
information
Subgame perfect equilibria in games with perfect information
require each player to play a best reply to other players’
strategies in each subgame — regardless of whether that
subgame is reached or not.
It is possible to extend the notion of subgame perfect
equilibria to games with imperfect information. But the
definition of subgames is trickier: information sets must lie
entirely outside the subgame or entirely inside the subgame.
Formally, let x be a (non-end) node and let V x be the nodes
of the tree that can be reached from x. A well-defined
subgame starts at x if and only if each information set h of
the original game is a subset of V x or is a subset of its
complement.
Since extensive form games with imperfect information need
not have proper subgames, the notion of subgame perfection
typically has little ‘bite’.
Mark Voorneveld Game theory SF2972, Extensive form games 36/64
Homework exercise 2

In the game of homework exercise 1:

(a) Find the corresponding strategic form game.
(b) Find all pure-strategy Nash equilibria.
(c) What is the outcome of iterated elimination of weakly
dominated (pure) strategies?
(d) Find all subgame perfect equilibria (in behavioral strategies).

Mark Voorneveld Game theory SF2972, Extensive form games 37/64

Implausible behavior in subgame perfect equilibria

(D, c, L) is a subgame perfect equilibrium. But in 2’s information

set, c is not a best reply to the others’ strategies.
In game-theoretic folklore this game is known as ‘Selten’s horse’.

Mark Voorneveld Game theory SF2972, Extensive form games 38/64

Question: Can we find a suitable equilibrium refinement for imper-
fect information games that
1 makes sense even if there are no subgames and
2 still insists that players choose ‘rationally’ even in information
sets that are reached with zero probability?
First attempt: require best responses in each information set.
Problem: the best response depends on where in the information
set the player believes to be!

Mark Voorneveld Game theory SF2972, Extensive form games 39/64

Beliefs and optimal strategies affecting each other
The following game has no proper subgames:

Beliefs affect optimal strategies: consider pl 2 in info set {M, R}. A

is a best response if and only if the player assigns at most prob 1/2
to being in node M.
Strategies affect reasonable

beliefs: If pl 1 assigns to actions (L, M, R)
1 3 6
probabilities 10 , 10 , 10 , pl 2 is twice as likely to end up in node
R than in node M. Bayes’ law requires that he assigns conditional
prob 1/3 to M and 2/3 to R.
Question: What are reasonable beliefs if 1 chooses L with prob 1?
Mark Voorneveld Game theory SF2972, Extensive form games 40/64
Assessments
We consider two requirements on beliefs that give different answers
to the final question:
1 Weak consistency: in information sets that are reached with

positive probability, beliefs are determined by Bayes’ law. In

information sets reached with zero probability, beliefs are
allowed to be arbitrary.
2 Consistency: beliefs are determined as a limit of cases where

everything happens with positive probability and —

consequently — where Bayes’ law can be used.
In particular, in both of these notions, we need to define two things:
strategies and beliefs over the nodes in the information sets. The
difference will lie in the constraints that are imposed.
Formally, consider a finite extensive form game with perfect recall.
An assessment is a pair (b, β), where
b = (bi )i∈N is a profile of behavioral strategies and
β is a belief system, assigning to each information set h a
probability distribution βh over its nodes.
Mark Voorneveld Game theory SF2972, Extensive form games 41/64
Two belief requirements
Given node x and behavioral strategies b, let Pb (x) be the probability
that node x is reached using b: it is the product of the probabilities
assigned to the branches leading to x. Similarly,
P if h is an information
set, it is reached with probability Pb (h) = x∈h Pb (x).
Assessment (b, β) is:
weakly consistent if beliefs in information sets reached with
positive probability are determined by Bayes’ law:

βh (x) = Pb (x)/Pb (h)

for every info set h with Pb (h) > 0 and every node x ∈ h.
consistent if there is a sequence of weakly consistent
assessments (b m , β m )m∈N with each b m completely mixed (all
actions in all info sets have positive prob) and
limm→∞ (b m , β m ) = (b, β).
Note: (b, β) consistent ⇒ (b, β) weakly consistent.
Mark Voorneveld Game theory SF2972, Extensive form games 42/64
Consistency: example

In the game above, where payoffs are omitted since they are irrele-
vant to the question:
(a) Find all weakly consistent assessments (b, β).
(b) Find all consistent assessments (b, β).

Mark Voorneveld Game theory SF2972, Extensive form games 43/64

Summarize an assessment (b, β) by a 4-tuple (p, q, α1 , α2 ) ∈ [0, 1]4 ,
where
p is the probability that 1 chooses In,
q is the probability that 2 chooses In,
α1 is the probability that the belief system assigns to the left
node in 1’s info set,
α2 is the probability that the belief system assigns to the left
node in 2’s info set.
(a) Distinguish two cases:
1 If p ∈ (0, 1], 2’s information set is reached with positive
probability. In that case, Bayes’ Law dictates that
α1 = α2 = 21 . Conclude: all
(p, q, α1 , α2 ) ∈ (0, 1] × [0, 1] × 12 × 12 are weakly
 

consistent.
2 If p = 0, 2’s information set is reached with zero probability
and 2 is allowed any belief α2 ∈ [0, 1] over the nodes in the
1
information set. Bayes’ Law only dictates that  1 α1 = 2 .
Conclude: all (p, q, α1 , α2 ) ∈ {0} × [0, 1] × 2 × [0, 1] are
weakly consistent.
Mark Voorneveld Game theory SF2972, Extensive form games 44/64
(b) Every completely mixed profile of behavioral strategies leads
to α1 = α2 = 12 .
Indeed, in 2’s information set, both nodes are reached with
equal probability 21 p.
Conclude: consistent are all
(p, q, α1 , α2 ) ∈ [0, 1] × [0, 1] × 12 × 12 .
 

Mark Voorneveld Game theory SF2972, Extensive form games 45/64

Terminology

In the game theoretic literature there are many versons of consis-

tency requirements on assessments. My version of weak consistency
is the same as Definition 27 in Jörgens notes, but is a bit less de-
manding than Definition 14.12 (Bayesian consistency) in Hans Pe-
ters’ book. (They coincide whenever all actions are chosen with
positive probability, so for defining consistency, either can be used)

Mark Voorneveld Game theory SF2972, Extensive form games 46/64

Expected payoffs in information sets
Fix assessment (b, β) and an information set h of player i. To
formalize the requirement that i plays a best response in info set h,
we need to specify i’s expected payoff:
1 Conditional on i being in his info set h, belief system β

assigns probability βh (x) to being in node x ∈ h.

2 Given such a node x, the probability P(e | b, x) that an end

node e is reached, conditional on starting in x and using

strategies b is
zero if e cannot be reached from x;
the product of the probabilities of the corresponding branches
from x to e otherwise.
3 In end node e, the payoff to i equals ui (e).
4 So the expected payoff to agent i in his information set h,
given assessment (b, β) is
!
X X
ui (bi , b−i | h, β) = βh (x) P(e | b, x)ui (e) .
x∈h e

Mark Voorneveld Game theory SF2972, Extensive form games 47/64

Sequential rationality

Assessment (b, β) is sequentially rational if each player i in each of

his information sets h chooses a best response to the belief system
β and the strategies of the other players:

ui (bi , b−i | h, β) ≥ ui (bi0 , b−i | h, β)

for all other behavioral strategies bi0 of player i.

Note:
1 consistency requirements say that beliefs have to make sense
given the strategies, without requirements on the strategies;
2 sequential rationality says that strategies have to make sense
given the beliefs, without requirements on the beliefs.
Putting the two together, we have:

Mark Voorneveld Game theory SF2972, Extensive form games 48/64

Weakly perfect Bayesian and sequential equilibrium
An assessment (b, β) is:
a weakly perfect Bayesian equilibrium if it is weakly consistent
and sequentially rational,
a sequential equilibrium if it is consistent and sequentially
rational.
Sequential equilibria are weakly perfect Bayesian equilibria; the latter
are Nash, but not necessarily subgame perfect: they impose no con-
straints on players’ beliefs in subgames off the path of the strategy
profile in question.
Theorem (Relations between solution concepts for extensive form
games)
(a) Each finite extensive form game with perfect recall has a
sequential equilibrium.
(b) If assessment (b, β) is a sequential equilibrium, then b is a
subgame perfect equilibrium and (hence) a Nash equilibrium.
Mark Voorneveld Game theory SF2972, Extensive form games 49/64
Proof sketch

(a) Via perfect equilibria of an auxiliary ‘agent-strategic form

game’: extensive-form perfect equilibria!
Each player i is split up into agents, one agent for each of i’s
information sets;
Agents of i have the same preferences as i;
A mixed strategy in this agent-strategic form game is a
behavioral strategy in the original game;
Consider a completely mixed seq b m → b making b a perfect
equilibrium
For each b m , Bayes’ law gives a belief system β m .
Drawing a convergent subsequence if necessary, we can show
that limm→∞ (b m , β m ) = (b, β) is a sequential equilibrium.
(b) Suppose not. Let i have a profitable deviation bi0 in a
subgame starting at some node x. Hence, in this subgame
there has to be an information set that is reached with
positive probability and where i has a profitable deviation,
contradicting sequential rationality and correctness of beliefs.
Mark Voorneveld Game theory SF2972, Extensive form games 50/64
Compute the sequential equilibria of the game below:

Intuition: What should it be? Player 1 chooses between sure payoffs

(6, 0) or the strategic game
C D
E 8, 0 0, 8
F 0, 8 8, 0
Mark Voorneveld Game theory SF2972, Extensive form games 51/64
Behavioral strategies b = (b1 , b2 ) can be summarized by three prob-
abilities:
1 p, the prob that 1 chooses A in the initial node;
2 q, the prob that 2 chooses C in his information set {A};
3 r , the prob that 1 chooses E in information set
{(A, C ), (A, D)}.
Belief system β can be summarized by one probability α, the prob as-
signed to the left node (A, C ) in the information set {(A, C ), (A, D)}.
Consistency: completely mixed beh. str. have p, q, r ∈ (0, 1).
Bayes’ law then gives
pq
α= = q,
pq + p(1 − q)

So for each consistent assessment (b, β), it follows that α = q.

Which of these assessments also satisfies sequential rationality?

Mark Voorneveld Game theory SF2972, Extensive form games 52/64

Distinguish 3 cases:
1 If q = 0, then α = 0, so r = 0 is 1’s unique best reply in the

final info set. But if r = 0, then q = 0 is not a best reply in

2’s info set. Contradiction.
2 If q = 1, then α = 1, so r = 1 is 1’s unique best reply in the

final info set. But if r = 1, then q = 1 is not a best reply in

2’s info set. Contradiction.
3 If q ∈ (0, 1), rationality in 2’s info set {A} dictates that both

C and D must be optimal. C gives 8(1 − r ), D gives 8r , so

r = 1/2.
In the info set {(A, C ), (A, D)} of pl. 1, his expected payoff is
α[8r ] + (1 − α)[8(1 − r )] |{z}
= 8 − 8q + 8r (2q − 1).
α=q

Choosing r = 1/2 is rational only if q = 1/2.

Finally, in the initial node, A gives expected payoff 4 and B
gives expected payoff 6, so p = 0.
Conclude: there is a unique sequential equilibrium with p = 0, q =
r = α = 1/2.
Mark Voorneveld Game theory SF2972, Extensive form games 53/64
Homework exercise 3

Find the sequential equilibria (b, β) of the game in homework exer-

cise 1.

Mark Voorneveld Game theory SF2972, Extensive form games 54/64

Peters, p. 263: “There is hardly any general method available
to compute sequential equilibria: it depends very much on the
game at hand what the best way is.”
Some potential approaches are:
1 Method 1: first find all consistent assessments, then find which
of these are sequentially rational.
2 Method 2: first find all sequentially rational assessments, then
find which of these are consistent.
3 Method 3: by the previous theorem: if (b, β) is a sequential
eq, then b is a (subgame perfect) NE. So first find all
(subgame perfect) NE. This is easier (no belief system) and
often rules out many candidates. Then verify which can be
turned into sequential equilibria.
Methods 1 and 2 are rarely used in that strict order. Often
you will discuss sequential rationality and consistency
together. If you anyway have to find the subgame perfect
equilibria, method 3 can save you a lot of work.

Mark Voorneveld Game theory SF2972, Extensive form games 55/64

Normal versus extensive form

Given an extensive form game, we defined its Nash equilibria in pure,

mixed, behavioral strategies in the corresponding normal-form game.
Likewise, it is possible to consider perfect and proper equilibria in
this normal-form.
On the other hand, we introduced variants of Nash equilibria (sub-
game perfect, weakly perfect Bayesian, sequential) that were appli-
cable directly to the extensive form.
We introduced one very important relation: existence of sequential
equilibria in the extensive form was derived from the existence of
perfect equilibria in the agent-normal form.
There are other links between extensive-form and normal-form anal-
ysis, but this is outside the scope of the course; if this interests
you, have a look at sections 3.11 till 3.14 of Jörgen’s notes and the
references there.

Mark Voorneveld Game theory SF2972, Extensive form games 56/64

Signalling games: examples

1 Michael Spence, 2001 Nobel Memorial Prize in Economics,

job-market signalling model
A prospective employer can hire an applicant.
The applicant has high or low ability, but the employer doesn’t
know which.
Applicant can give a signal about ability, for instance via
education.
2 Language, according to some evolutionary biologists, evolved
as a way “to tell the other monkeys where the ripe fruit is.”
Sometimes it makes sense to signal what your private information
is, sometimes not.

Mark Voorneveld Game theory SF2972, Extensive form games 57/64

Signalling games: model

1 Chance chooses a type t from some nonempty finite set T

according to known prob distr P with P(t) > 0 for all t ∈ T .
2 Pl. 1 (the sender) observes t and chooses a message m ∈ M
in some nonempty finite set of messages M.
3 Pl. 2 (the receiver) observes m (not t) and chooses an action
a ∈ A in some nonempty finite set of actions A.
4 The game ends with utilities (u1 (t, m, a), u2 (t, m, a)).
A pure strategy for player 1 is a function s1 : T → M and a pure
strategy for player 2 is a function s2 : M → A.

Mark Voorneveld Game theory SF2972, Extensive form games 58/64

Separating and pooling equilibria in signalling games
In signalling games, it is common to restrict attention to equilibria
(s1 , s2 , β), where
s1 and s2 are pure strategies;
assessment (s1 , s2 , β) is weakly consistent;
assessment (s1 , s2 , β) is sequentially rational.
Sometimes it is in the sender’s interest to try to communicate her
type to the receiver by sending different messages for different types

s1 (t) 6= s1 (t 0 ) for all t, t 0 ∈ T .

In such cases we call the equilibrium (s1 , s2 , β) a separating equilib-

rium.
In other cases, the sender might want to keep her signal a secret to
the receiver and send the same message for each type:

s1 (t) = s1 (t 0 ) for all t, t 0 ∈ T .

In such cases we call the equilibrium (s1 , s2 , β) a pooling equilibrium.

Mark Voorneveld Game theory SF2972, Extensive form games 59/64
Signalling games: example

In the signalling game above:

(a) Find the corresponding strategic form game and its
pure-strategy Nash equilibria.
(b) Determine (if any) the game’s separating equilibria.
(c) Determine (if any) the game’s pooling equilibria.
Mark Voorneveld Game theory SF2972, Extensive form games 60/64
Answer (a):
Pl. 1’s pure strategies are pairs in {L, R} × {L, R}, denoting
the action after t and t 0 , respectively.
Pl. 2’s pure strategies are pairs in {u, d} × {u, d}, denoting
the action after message L and R, respectively.
Strategic form:
(u, u) (u, d) (d, u) (d, d)
1 ∗ 1 ∗ 5∗ 1 5 1
(L, L) 2, 1 2, 1 2 ,2 2, 2
(L, R) 1∗ , 1 3 ∗
2 , 2 3
2 , 0 2, 1
3 ∗ ∗, 1
(R, L) 0, 12 3
2 , 0 2 , 1 3 2
1 1 5∗ ∗ 1 1 5 ∗
(R, R) ,
2 2 2 , 1 ,
2 2 2 , 1
Payoffs corresponding with best replies are starred, so there is
a unique pure-strategy Nash equilibrium ((R, R), (u, d)).
Answer (b): Separating equilibria must be Nash equilibria; but the
only candidate ((R, R), (u, d)) is of the pooling type: pl. 1 sends the
same message R for both types. Conclude: no separating equilibria.

Mark Voorneveld Game theory SF2972, Extensive form games 61/64

Answer (c):
In (a), we found the candidate strategy profile ((R, R), (u, d)).
But what should the belief system be? Let α1 , α2 ∈ [0, 1]
denote the prob assigned to the top node in the left and right
info set, respectively.
Bayesian consistency: requires that α2 = 12 , but imposes no
constraints on α1 .
Sequential rationality:
1 Both info sets of pl. 1 and the right info set of pl. 2 are
reached with positive prob. Since ((R, R), (u, d)) is a NE, the
players choose a best reply in those information sets.
2 The left info set of pl. 2 is reached with zero prob. But the
beliefs should be such that 2’s action u is a best reply there.
3 Pl. 2’s payoff from u is 2α1 + 0(1 − α1 ) and from d is
0α1 + 1(1 − α1 ), so seq. rat. requires α1 ≥ 13 .
Conclude: Assessments (s1 , s2 , β) with strategies
(s1 , s2 ) = ((R, R), (u, d)) and belief system
β = (α1 , α2 ) ∈ [1/3, 1] × {1/2} are the game’s pooling
equilibria.
Mark Voorneveld Game theory SF2972, Extensive form games 62/64
Homework exercise 4

In the signalling game above:

(a) Find the corresponding strategic form game and its
pure-strategy Nash equilibria.
(b) Determine (if any) the game’s pooling equilibria.
(c) Determine (if any) the game’s separating equilibria.
Mark Voorneveld Game theory SF2972, Extensive form games 63/64
Reading guide
definition extensive form games: slides 1–5, book §4.1, §14.1
examples (im)perfect recall: slides 6–8, book 53–54, 253
pure, mixed, behavioral strategies: slides 9–12, book §4.2,
§14.2
outcome equivalence of mixed and behavioral strategies under
perfect recall: slides 13–19, book §14.2
Nash equilibrium: slides 21–24, book §4.2, 256–258
subgame perfect equilibrium and backward induction: slides
25–38, book §4.3, §14.3.1
assessments: consistency of beliefs: slides 39–46, book §4.4
(partly), §14.3.2 (partly), Jörgen’s notes §3.9
assessments: sequential rationality: slides 47–48, book p. 259
weakly perfect Bayesian and sequential equilibrium: slides
49–55, book §4.4, §14.3.2, Jörgen’s notes §3.9
signalling games: slides 57–63, book §5.3
Mark Voorneveld Game theory SF2972, Extensive form games 64/64