Dynamic or Sequential Games

…Static vs Dynamic Games

…Games in Extensive form
… Rollback equilibrium
…Conversion of Normal form to Extensive form
…Conversion of Extensive form to Normal form
…Subgame Perfect Nash Equilibrium
 In this chapter

Sequential Move Games

Models of strategic situations where there is a strict order of play

One-off Sequential Games Repeated Games

One off game but inherently sequential A static game being played repeatedly
interaction between players

Infinitely Repeated Finitely Repeated

With Perfect With Imperfect Games Games
Information Information

Sequential move games are most easily represented in the

form of a game tree.
 Dynamic or Sequential Games

• Many situations involve player choosing actions sequentially

(over time), rather than simultaneously
… In a simultaneous game there is no concept of time and order of play.
… Instead of playing simultaneously,
… the rules dictate when players play,
… and what they know about the past when they play.
… Represented in the form of a tree.
• A sequential game involves:
… a list of players
… for each player, a set of actions at each stage
… each player’s information at each stage
… each player’s preferences over all possible combination of actions =
payoffs
 Example: New Product Launch

… Two Firms A and B

… Entering market with a new product
… Market is limited. If both enter, both will incur loss.
… The game is static if both make their decision simultaneously
without knowing the decision of the other party.

Firm B

Enter Don’t

Enter -10, -10 50, 0

Firm A

Don’t 0, 50 0, 0

… What if B decided to wait until A has made a decision?

 New Product Launch as Dynamic Game
… Suppose B decided to wait until A has made a decision.
… This can be represented as the following extensive form
representation.

(-10, -10)

(50, 0)
A (0, 50)

B
(0, 0)
 Normal and Extensive Form Games

• Normal Form Game

… Identify Players, Strategies and Payoffs
… Represent in the form of a matrix
• Extensive Form Games
… Alternative representation in which attention is paid to timing
and information available.
… Represented as a tree
… Node represents a particular player’s ’turn’
… Branches represent different choices for that player
… Each node has at least one branch pointing out of it and at most
one branch going into it.
 Check your understanding…
Valid extensive form games?
 Dynamic games of perfect information
… These are games where all players observe the entire
history of the game, and the game terminates in finitely
many steps.
… This is an Extensive form game
with perfect information.
(-10, -10)
… Perfect information implies that
B all players observe the entire
history of the game
(50, 0) … That is every player is at any point
A (0, 50) aware of the previous choices of
all other players.
B … Furthermore, only one player
moves at a time, so that there are
(0, 0)
no simultaneous moves.
 Example: A sharing game

• Suppose players 1 and 2 are two children

• Someone offers them two cookies, but only if they can
agree how to share them
• Player 1 chooses one of the following options:
… Player 1 gets 2 cookies, player 2 gets 0 cookies (2-0)
… They each get 1 cookie (1-1)
… Player 1 gets 0 cookies, player 2 gets 2 cookies (0-2)
• Player 2 chooses to accept or reject the split:
… Accept => they each get their cookies(s)
… Otherwise, neither gets any
A sharing game: tree representation

(2, 0)

(0, 0)
(1, 1)
1-1
1 2
(0, 0)
(0, 2)

2
(0, 0)
How to Solve Finite Games with Perfect Information

• The game tree is also called the extensive form of the

game.
• Finite games of perfect information can be solved using
backward induction.
• Idea: find optimal decisions for players from the bottom
of the tree to the top.
• Any finite dynamic game of perfect information has a
backward induction solution.
 Backward Induction
• Finite games of perfect information can be solved using
backward induction
• Find “optimal” decisions for players from the bottom
(right) of the tree to the top (left).

(-10, -10)

(50, 0)
A (0, 50)

B
(0, 0)

(50, 0) is called the Rollback equilibrium

 Another Example: Backward Induction

A B

2 2

C D E F

(3, 8) (8, 3) (5, 5) 1

G H

(3, 8) is the Rollback

equilibrium (2, 10) (1, 0)
Another Example: Backward Induction

2,4,6

2
middle
3,8,1

4,4,4
1
10,1,7
up 3

2
6,8,9
3,1,2

(6, 8, 9) is the Rollback equilibrium

  Game: Centipede Game

• Two players A and B.

• Game starts with a coin worth 10 in the centre pile.
• Player A can take it or pass.
… If A takes the pile, the game is over, with A getting Rs 10
and B getting nothing.
… If A passes, one more coin gets added to the pile, and now
B has the choice of taking Rs 20 or passing.
… If B passes, one more coin gets added to the pile, and now
A has the choice of taking Rs 30 or passing.
… And so on…
• The pile of money grows until reaching Rs 1000 after
which no more coins will be added to the pile.
 Rosenthal’s Centipede Game
• Two Players.
• Two possible moves (Pass/Take).
• Players move alternately.
• Across : Game continues
• Down : Game ends

1 2 1 2 1 1 2
P P P P P P P
(0,0)
T T T T T T T
(1,0) (0,2) (3,0) (0,4) (5,0) 99,0 0,100
 Rosenthal’s Centipede Game
1 2 1 2 1 1 2
P P P P P P P
(0,0)
T T T T T T T
(1,0) (0,2) (3,0) (0,4) (5,0) 99,0 0,100
• Playing across is direct evidence of irrationality.
• (1,0) is the outcome based on 100 iterations of “I believe that you
believe that I believe that you believe….”
• What if each player has a slight doubt about the rationality of other
player?
• Based on this doubt, 2 may also play across, leading 1 to doubt the
rationality of 2.
• For this game, the assumption of common knowledge of rationality
is inappropriate.
• Assume incomplete information that Players are unsure if the
opponent is rational.
• It may be rational to initially pretend to be irrational.
• Pretending to be irrational is a rational thing to do.
 Real Life Board Games
• Board games are, almost by definition, games of perfect information, since everything is in
the open provided no cards are part of the game. They are also very often sequential games,
where the players take turns to move.
• Well-known example of board games are Chess, Checkers, Backgammon with a very long
history, or the more recent games of Monopoly, Hex, Tic-tac-toe or others.
• Some, like Backgammon, and Monopoly, are played with dice or even with cards that are
revealed, and contain therefore randomness.
• But Chess, Checkers, Tic-tac-toe are all two-players zero-sum finite sequential games without
randomness and with perfect information. So shouldn't people play the backward-induction
strategy?
… The simple and quick answer is: Yes, they should, but in some cases backward induction analysis has
just not yet been performed.
… Thus the backward induction strategy, the optimal strategy, is unknown in many cases.
• Tic-tac-toe is the simplest of these games, with the smallest game tree. The game tree has
just 5478 positions. The game has been analyzed, and the expected outcome is a draw. You
can play tic-tac-toe against the computer.
• Checkers will always result in a draw when played correctly ([von Nievergelt and Gasser
1994].
• On the other hand, for Chess, the number of possible positions is so large that even with the
help of the most advanced computers a full analysis is not expected for soon. Chess has
about 1043 positions, a number with about 43 digits.
Extensive Form Games with imperfect information

• Players may not always have full access to all the information which
is relevant to their choices.
• Extensive games with imperfect information model exactly which
information is available to the players when they make a move.

(-10, -10)

B
• What if player B does not
observe player A’s decision.
(50, 0) • Player B can not distinguish
A (0, 50) between the two nodes
• Represented by a dotted line
B
(0, 0)

It is important to note that it is same as a simultaneous move game

 Information Set

• We represent imperfect information by combining nodes

into information sets.
• An information set h is
… a subset of nodes of the game tree
… all identified with the same player
… with the same actions available at each node in h
• Information Set:
… Two nodes of a game tree are said to be part of the same
information set if the player at that node cannot differentiate
between them.
• All information sets are singletons implies that
… This is a game with perfect information
 Check your understanding…
Valid extensive form games?
 Representing extensive form into the normal form

• Every extensive form game, has one corresponding normal

form game.
• To convert an extensive form game to the normal form, pure
strategy profiles of each player are needed.
… A pure strategy for a player is a complete plan of action that
specifies the choice to be made at each decision node.
… complete specification of which deterministic action to take at every
node of the that player.
… The number of (pure) strategies of a player can be calculated by
multiplying the number of actions she has at each information set.
… The list of pure strategies is the cross product of information sets.
 Example

# of pure strategies of P1: 2

# of pure strategies of P2: 6

# of pure strategies of P1: 4

# of pure strategies of P2: 2
 Representing extensive form into the normal form

Strategies of Player 1: 3
(2, 0)
{2-0, 1-1, 0-2}
2 Strategies of Player 2: 8
{AAA, AAR, ARA, RAA, ARR, RAR,
(0, 0) RRA, RRR}
(1, 1)
1-1
1 2
(0, 0)
(0, 2)

2
(0, 0)
 Another Example
1 CE CF DE DF

AG 3, 8 3, 8 8, 3 8, 3
AH 3, 8 3, 8 8, 3 8, 3
A B
BG 5, 5 2, 10 5, 5 2, 10

2 2 BH 5, 5 1, 0 5, 5 1, 0

C D E F
Player 1: four pure strategies
{(A, G), (A, H), (B, G), (B, H)}
(3, 8) (8, 3) (5, 5) 1
(Must include (A, G) and (A, H),
even though action A makes the
G H G-versus-H choice moot)

Player 2: four pure strategies

(2, 10) (1, 0)
{(C, E), (C, F), (D, E), (D, F)}
Check your understanding…
Solution:
Exercise: Convert the game to normal form
Solution:
 Extensive Form Games
• For every extensive form game,
There is only one corresponding normal form game.
• For every normal form game,
There are, in general, several corresponding
extensive form games.
 Sequential New Product Launch in Normal Form
(-10, -10)

(50, 0)
A (0, 50)

B
(0, 0)

Firm B
Always enter Always stay Same as A Opposite of
(EE) out (DD) (ED) A (DE)
Enter -10, -10 50, 0 -10, -10 50, 0
Firm A

Stay out 0, 50 0, 0 0, 0 0, 50
 Firm B
Always Always Same as Opposite
enter stay out A of A

Firm A
Enter -10, -10 50, 0 -10, -10 50, 0
Stay out 0, 50 0, 0 0, 0 0, 50

• There are three Pure Strategy Nash Equilibria.

… How many of them will actually be realised?
• Understand these in terms of Threats/Promises made by Player B.
… Firm B threatens always to enter the market.
… Firm B promises to always stay out of the market.
… Firm B promises to do the opposite of what A does.
• All promises/threats may not be credible.
• Is it in the interest of B to enter the market, even after A has done so? NO.
• Is it in the interest of B to stay out, even if A chooses to stay out? NO.
• First two do not stand to this scrutiny, only the third one does.
• A should ignore incredible threats/promises.
Sub-Game
… Nash equilibrium does not distinguish between credible and non-
credible threats. Hence there is a need for stronger concept.
… SUBGAME: A proper subgame is the entire game that remains starting
from any non-terminal node. It can be viewed as an independent game.
 How many subgames?

P1

A B

P2 P2

C D E F

P1
G H
 Subgame Perfect Nash Equilibrium (SPNE)

• A strategy profile of a sequential game is a subgame perfect

equilibrium if it is a Nash equilibrium for every subgame of the
original game. In other words, the strategy is perfect even if
the play never goes to that part of the tree.
• Hence, The set of subgame perfect equilibria is a subset of set
of Nash equilibria.
• Subgame perfect Nash equilibria are Nash equilibria which
prove to be optimal for the players in every part of the game,
i.e., after every possible history
• Rollback Equilibrium is the SPNE as it always chooses the
optimal option for the player in that subgame.
• If the payoffs are distinct across terminal nodes, then there is a
unique subgame perfect equilibrium.
Extensive-Form Games with Imperfect Information

• In games of perfect information,

… the subgame perfect equilibrium coincides with the backward-induction
solution.

• But subgame-perfect equilibrium is a more general concept,

… defined also for games without perfect information.

• In some strategic interactions, players make decisions without

knowing prior decisions of other players.
• This formulation includes the possibility of simultaneous
moves, and the possibility of uncertainty about the opponents’
types.
 Example: Entry Game
• Consider an entry game in which the entrant may or not be
ready for a fight, and this is private information.

R U O
II II
2, 5
S F S F

3, 3 1, 1 4, 3 0, 2

• The two decision subhistory of player 2 are linked, to account

for the fact that she does not know player 1’s prior action.
 Transforming into normal form
• The strategic form representation of the entry game is:

F S
R 1, 1 3, 3
U 0, 2 4, 3
O 2, 5 2, 5

• There are 2 pure strategy Nash Equilibria (U, S) and (O, F).
• Both equilibria are subgame perfect, because there are no proper
subgames in the extensive form representation.
• Only (U, S) is reasonable.
• (O, F) contains a non-credible threat.
• To account that only some Subgame Perfect Equilibria are
reasonable, the solution concept of Perfect Bayesian Equilibrium is
used.
 Golden Balls
• https://www.youtube.com/watch?v=S0qjK3TWZE8
 Golden Balls
Abraham
Split Steal

Split 6800, 6800 0, 13600

Nick

Steal 13600, 0 0, 0
 Strategic Moves

• A strategic move is an action chosen by a player with the aim to

influence the actions of the other player to her own advantage.
The idea behind strategic moves by a player is to change the
behaviour of the other player in a way that is conducive to her.
• A strategic move may be unconditional such as a commitment
that is irreversible or conditional where a player’s move is
dependent on the actions of other players.
• A threat is a conditional strategic move wherein a player acts in a
way that harms the other player if her interests are not met.
• A promise is a conditional strategic move wherein a player
rewards the other player for promoting her interests.
 Example: Nim's Game

• Two players A and B

• 21 coins in a box
• Players alternate in taking turns to remove coins
• At each turn, the player must remove 1, 2 or 3 coins
• The player who removes the last coins (whether as
the sole remaining coin or one of the last surviving
set of 2 or 3 coins) is the winner
 Analysis: Nim's Game

• If player 1 leaves player 2 with four coins

… Player 2 must remove 1, 2, or 3,
… Player 1 can take the rest and win.
• To make sure that player 1 leaves player 2 with four
coins, player 1 must leave player 2 facing eight coins on
the immediately preceding turn.
• The logical sequence then is to leave 12, 16, and 20 on
previous turns.
• Therefore, starting with 21 coins, player 1 should remove
one and proceed to take four minus whatever player 2
takes at the immediately preceding turn.
 First Mover Advantage?

• Games like the battle of the sexes or chicken have first-

mover advantages in their sequential move versions
• The cricket game example has a second-mover
advantage in its sequential-move version
• Many other games show no change in equilibrium as a
result of the change in rules
… Games like the prisoners’ dilemma, in which both players have
dominant strategies, fall into this category

45