Journal of Mathematical Economics 1097 (2003) 1–19

Deterministic multi-player Dynkin games

Eilon Solan a,b,∗ , Nicolas Vieille c
aMEDS Department, Kellogg School of Management, Northwestern University,
2001 Sheridan Road, Evanston, IL 60208, USA
b School of Mathematical Sciences, Tel Aviv University, Tel Aviv, Israel
c Département Finance et Economie, HEC, 1 rue de la Libération, 78 351 Jouy-en-Josas, France
Received 3 September 2002; received in revised form 22 January 2003; accepted 28 January 2003

Abstract
A multi-player Dynkin game is a sequential game in which at every stage one of the players is
chosen, and that player can decide whether to continue the game or to stop it, in which case all
players receive some terminal payoff.
We study a variant of this model, where the order by which players are chosen is deterministic,
and the probability that the game terminates once the chosen player decides to stop may be strictly
less than 1.
We prove that a subgame-perfect -equilibrium in Markovian strategies exists. If the game is not
degenerate this -equilibrium is actually in pure strategies.
© 2003 Elsevier Science B.V. All rights reserved.
JEL classification: C73

Keywords: Dynkin game; Stochastic game; Equilibrium; n-player games

1. Introduction

Dynkin (1969) introduced the following zero-sum game of optimal stopping. The game
involves two players, and two stochastic processes: (in )n∈N N is a {1, 2}-valued process,
which indicates which player is active at stage n, and (rn )n∈NN is a R -valued process, which
2

indicates the terminal payoff.

At every stage n, the two players are informed of past and current values of the two
processes. Player in , the active player at stage n, decides whether he continues or stops. The
game stops at the first stage θ in which the active player chooses to stop. The payoff (paid
∗ Corresponding author.

E-mail addresses: [email protected], [email protected] (E. Solan), [email protected]

(N. Vieille).

0304-4068/03/$ – see front matter © 2003 Elsevier Science B.V. All rights reserved.
doi:10.1016/S0304-4068(03)00021-1
2 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

by player 2 to player 1) is rθ if θ < +∞ and zero otherwise. A pure strategy of player i is

a stopping time that is consistent with the rules of the game.
N |rn | ∈ L , and constructed pure
Dynkin proved that this game has a value if supn∈N 1

-optimal strategies for the two players. Dynkin’s -optimal strategies are subgame-perfect
in the sense that after every finite history, the continuation strategy is -optimal in the
subgame defined by that history.
An extensive literature developed from this seminal work. In a discrete time framework,
much attention was paid to the case where the players are allowed to stop simultaneously.
In the zero-sum case, several authors, including Kiefer (1971) and Neveu (1975), pro-
vided sufficient conditions for the existence of the value, when players are restricted to
stopping times. Rosenberg et al. (2001) proved (under a minimal boundedness condition)
that the value always exists, provided the players are allowed to use randomized stop-
ping times. In the two-player non-zero-sum case, Shmaya and Solan (2002) proved that an
ε-equilibrium always exists in randomized stopping times (again, under some boundedness
condition).
Dynkin’s (1969) result implies that in every multi-player Dynkin game (without simulta-
neous moves) an ε-equilibrium exists. Indeed, let σ i be a pure ε-optimal strategy of player i
in the zero-sum game in which player i maximizes his expected payoff, and all other players
try to minimize player i’s payoff. Let σi−i be a pure ε-optimal strategy of i’s opponents in
this game. One can verify that the strategy profile in which each player i follows σ i until
a deviation occurs (since each σ i is pure, a deviation is detected immediately), and upon
−j
deviation of player j all his opponents switch to σj , is a 2ε-equilibrium.
The model of multi-player Dynkin games offers a stylized framework to analyze various
issues of timing games. For example, in situations of shrinking markets (see, e.g. Ghemawat
and Nalebuff, 1985; Fine and Li, 1989), n firms have to decide when to exit a shrinking
market. Once a firm exits, we remain with a market with n − 1 firms, which can be solved
inductively, hence the overall game reduces to a Dynkin game.
A similar situation occurs in takeover games, where n firms strategically decide to make
a takeover attempt on opponent firms.
Another related model is that of multi-player duels, or n-uels (see, e.g. Kilgour, 1975,
1977 or Kilgour and Brams, 1997). In this model, n gunners alternately have the option to
shoot one of their opponents or to abstain. Since once a gunner hits one of his opponents
we are left with a game with n − 1 players, which can be solved inductively, the game is
essentially reduced to a Dynkin game where players have several stop actions.
As the ε-equilibrium we presented above involves threats of punishment, which might
be non-credible, it is desirable to know whether a subgame-perfect ε-equilibrium exists for
every ε > 0. To this day, it is still not known whether every multi-player Dynkin game has
an ε-equilibrium.
When |I| = 2, the proof of Shmaya and Solan (2002) can be used to show the existence
of a subgame-perfect ε-equilibrium. Solan (2002) uses the theory of differential inclusions
to prove the existence of a subgame-perfect ε-equilibrium when (i) the sequence (in ) is
i.i.d., and (ii) rn depends only on in (so that the terminal payoff depends only on the identity
of the player who terminates the game).
In the present paper, we analyze the following class of I-player games. A deterministic
sequence (in , pn , rn ) ∈ I × [0, 1] × R I is given. At each stage n, player in chooses whether
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 3

to continue or to stop. If he continues, the game continues to the next stage, while if he
stops a lottery is performed. With probability pn the game terminates, yielding the payoff
rn , while with probability 1 − pn the game continues.
The assumption that the order of players is deterministic is restrictive but sometimes
relevant. On the other hand, allowing the probability of termination to be strictly less than
1 is quite natural: a takeover attempt is not always successful, and the accuracy of a gunner
is not always perfect.
Our main result states that if the sequence (rn ) of payoffs is bounded, a subgame-perfect
ε-equilibrium in Markovian strategies exists. Moreover, unless the game is degenerate,
this ε-equilibrium is in pure strategies. However, in degenerate cases, a subgame-perfect
0-equilibrium need not exist. Since the subgame-perfect -equilibrium we identify is in
Markovian strategies, it is robust to the information players receive along the game; all they
need to know is the stage of the game. Translated to the n-uel model, this means that there
is a subgame-perfect -equilibrium which is also a subgame-perfect -equilibrium in the
silent n-uel, in which players do not observe missed shots.
In degenerate cases, there need not be subgame-perfect ε-equilibria in pure strategies. This
is to be contrasted with (i) finite games of perfect information and (ii) two-player zero-sum
Dynkin games, where a subgame perfect (ε-) equilibrium in pure strategies always exists.
We hope that the combination of the arguments we use here with the techniques presented
by Shmaya and Solan (2002) and Solan (2002) can be used to further study multi-player
Dynkin games.
Another motivation to our study is linked to the observation that deterministic Dynkin
games form a simple class of stochastic games. By now, some results are available on
the existence of equilibrium payoffs in multi-player stochastic games, see Solan (1999)
and Vieille (2000). By contrast, apart from few classes of games, there are no results on
the existence of subgame-perfect equilibrium payoffs and useful techniques are yet to be
found. We hope that this paper will contribute to this emerging literature.
The paper is arranged as follows. In Section 2 we present the model and the main result.
Several examples appear in Section 3. The proof of the main result appears in Section 4.

2. The model and the main result

2.1. Deterministic multi-player Dynkin games

A deterministic multi-player Dynkin game Γ = (I, (in , pn , rn )n∈N

N ) is given by

• a finite set I of players;

• for every n ∈ N , a triplet (in , pn , rn ) ∈ I × [0, 1] × R I .
The triplet (in , pn , rn ) specifies who is allowed to stop at stage n, the probability that the
game terminates if player in decides to stop, and the terminal payoff if the game terminates
at stage n, respectively.
The game is played in stages. At each stage n ∈ N , provided the game has not terminated
yet, player in has to choose whether to Continue or Stop. If he decides to continue, the
game continues to stage n + 1. If he decides to stop, a lottery takes place (all lotteries in
4 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

the game, including random choices by the players, are independent). With probability pn
the game terminates, and the terminal payoff for the players is given by the vector rn . With
probability 1 − pn the game continues to stage n + 1. If the game never terminates, the
payoff is zero for all players.1
We denote by θ the termination stage of the game, i.e. the first stage in which a player
decides to stop and the game terminates. Thus, the payoff to player i ∈ I is rθi 1 θ<∞ .

2.2. Strategies and results

A strategy of player i ∈ I maps the set of information sets of player i to the set of mixed
moves of player i. We let Ni = {n ∈ N |in = i} be the set of stages in which player i is
active.
We are going to restrict the players to Markovian strategies; namely, strategies that depend
only on the stage, and not on the history. We will prove below that the game admits a
subgame-perfect ε-equilibrium in Markovian strategies. By a general observation (see,
e.g. Fudenberg and Tirole (1991, p. 501)), this subgame-perfect ε-equilibrium remains a
subgame-perfect ε-equilibrium without the restriction to Markovian strategies.
In the present context, a (behavior Markovian) strategy of player i is a function σ i : Ni →
[0, 1], where σ i (n) is the probability assigned by player i to stop at stage n, provided the
game does not terminate before that stage. We denote the set of strategies of player i by Σ i .
A strategy profile (or simply a profile) is a vector σ = (σ i )i∈I of strategies, one for each
player.
Every strategy profile σ ∈ ×Σi∈I i induces a probability distribution P over the space of
σ
plays, or infinite histories. The corresponding expectation operator is E σ . Thus, the expected
payoff to player i given a strategy profile σ is

γ i (σ) := E σ [rθi 1 θ<∞ ].

Before we state our result, we first recall standard equilibrium notions.

Definition 1. Let ε ≥ 0. A strategy profile σ is an ε-equilibrium if for every player i ∈ I

and every strategy τ i ∈ Σ i ,

γ i (σ) ≥ γ i (σ −i , τ i ) − ε.

We mention that, for any ε > ε, an ε-equilibrium is a uniform ε -equilibrium; that is, it is
an ε -equilibrium (a) in every discounted game, provided the discount factor is sufficiently
small, and (b) in every N-stage game, provided N is sufficiently large. Indeed, the proof
provided in Solan and Vieille (2001, Proposition 2.13) does adapt to the present framework.
For n ∈ N , we denote by γn (σ) the expected payoff induced by the strategy profile σ in
the subgame starting at stage n.
A strategy profile is a subgame-perfect (ε-) equilibrium of a game if it induces an (ε-)
equilibrium in any subgame. In the present context, this amounts to the following definition.

1 Equivalently, we may assume that, with probability p , player i is given the opportunity to stop for sure. For
n n
each strategy profile, the payoff is the same under both interpretations of the game.
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 5

Definition 2. Let ε ≥ 0. A strategy profile σ is a subgame-perfect ε-equilibrium if for every

n ∈ N , every player i ∈ I, and every τ i ∈ Σ i ,

γni (σ) ≥ γni (σ −i , τ i ) − ε.

Our main result is the following.

Theorem 1. Let Γ = (I, (in , pn , rn )n∈NN ) be a deterministic Dynkin game. If the se-
quence (rn )n∈N
N is bounded, then for every ε > 0 the game Γ admits a subgame-perfect
ε-equilibrium in Markovian strategies.

We conclude this section with two comments.

As will be clear from the proof, in most cases, there is a pure subgame-perfect ε-equi-
librium. However, this is not always true (see Example 3 below). This is in sharp contrast with
finite extensive games of perfect information and with two-player zero-sum Dynkin games.
Our proof is valid as long as γ(σ) is uniformly bounded, for every profile σ (which is
the case when the sequence (rn )n∈N N is bounded). If this does not hold, there are strategies
σ such that the corresponding payoff for at least one player is infinite, so that the payoff
function of the game is not well-defined.

3. Examples

In the present section we provide several examples that illustrate the main features of the
model.

Example 1. Take I = {1, 2, 3} and



 (1, 1, (1, 0, 3)), n = 1 modulo 3,

(in , pn , rn ) = (2, 1, (3, 1, 0)), n = 2 modulo 3,


 (3, 1, (0, 3, 1)), n = 0 modulo 3.

In words, at the first stage, player 1 can stop the game, thereby yielding the payoff vector
(1, 0, 3). If player 1 chooses to continue, at the second stage player 2 can stop the game,
yielding the terminal payoff (3, 1, 0). If player 2 chooses to continue as well, at the third
stage player 3 can stop the game, yielding the terminal payoff (0, 3, 1). The process then
repeats itself cyclically. This game is a variation upon a game studied by Flesch et al. (1997).
We will characterize all pure subgame-perfect 0-equilibrium profiles of that game, using
backward induction.
Let σ be such a 0-equilibrium. Assume that at stage 3n, for some n ≥ 2, player 3 stops
with probability 1; that is, σ 3 (3n) = 1. In particular, γ3n (σ) = (0, 3, 1).
Consider the subgame starting at stage 3n−1. In that subgame, player 2 receives γ3n 2 (σ) =

3 if he chooses to continue at stage 3n − 1, while he receives only 1 if he chooses to stop. By

the subgame-perfect equilibrium condition, player 2 continues at stage 3n − 1, that is,
σ 2 (3n − 1) = 0. Hence, γ3n−1 (σ) = γ3n (σ) = (0, 3, 1).
6 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

We repeat this argument with the subgame starting at stage 3n − 2. By continuing at

stage 3n − 2 player 1 receives 0, as the game will be terminated at stage 3n, while by
stopping he receives 1. By the subgame-perfect equilibrium condition, σ 1 (3n − 2) = 1 and
γ3n−2 (σ) = (1, 0, 3).
Applying this backward induction argument repeatedly, we get that σ 3 (3n − 3) = 0,
σ (3n−4) = 1, σ 1 (3n−5) = 0 and σ 3 (3n−6) = 1. The cycle of length 6 then repeats itself.
2

On the other hand, if σ 3 (3n) = 0 for some n ≥ 2, then σ 3 (3n − 3) = 1 and the previous
analysis holds.
Thus, there are two pure subgame-perfect 0-equilibria: (a) at odd stages the active player
stops, and at even stages the active player continues, and (b) at even stages the active player
stops, and at odd stages the active player continues.
In each pure equilibrium, the players agree on who shoots first. We believe that the
interpretation of these two equilibria is quite appealing: suppose there are three gunners. If
gunner 1 thinks that gunner 2 is going to shoot tomorrow gunner 3 (or gunner 1 if gunner
3 is already dead), he has no reason to shoot today: he is better off by letting gunner 2 be
done with gunner 3, and shoot gunner 2 the next time he can. On the other hand, if gunner
1 thinks that gunner 2 is not going to shoot tomorrow if gunner 3 is still alive, but shoot
gunner 1 if gunner 3 is already dead, and that gunner 3 is going to shoot him the day after,
he is indifferent between shooting and not shooting gunner 2, as he is going to die anyway,
so he can as well shoot gunner 2 today.

Remark 1. This game admits other subgame-perfect equilibria. In particular, the profile in
which each player stops with probability 1/2 whenever active, is a subgame-perfect equili-
brium. In a sense, it corresponds to the cyclic equilibrium constructed by Flesch et al. (1997).

In the next example, we allow for probabilities of success below 1.

Example 2. Consider the following modification of Example 1, where I = {1, 2, 3}, and


 (1, 1, (1, 0, 3)),
 n = 1 modulo 3,
(in , pn , rn ) = (2, 1/2, (3, 1, 0)), n = 2 modulo 3,


 (3, 1/2, (0, 3, 1)), n = 0 modulo 3.

Thus, when player 1 stops the game terminates with probability 1, while when either player
2 or player 3 stops the game terminates with probability 1/2.
As we did in Example 1, we characterize the set of subgame-perfect 0-equilibrium in
pure strategies. Let σ be such a strategy profile. Let n > 0 and i be the active player at stage
n. By the subgame-perfect equilibrium condition, σ i (n) = 1 if γn+1i (σ) < 1 and σ i (n) = 0
i
if γn+1 (σ) > 1.
Let n ≥ 3, and assume that σ 1 (3n + 1) = 1. Then γ3n+1 (σ) = (1, 0, 3), and therefore
σ 3 (3n) = 0. This implies that γ3n (σ) = γ3n+1 (σ) = (1, 0, 3), and therefore σ 2 (3n−1) = 1.
It follows that
γ3n−1 (σ) = 21 (3, 1, 0) + 21 (1, 0, 3) = (2, 21 , 23 ),
and therefore σ 1 (3n − 2) = σ 3 (3n − 3) = 0 and σ 2 (3n − 4) = 1.
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 7

Then,
γ3n−4 (σ) = 21 (3, 1, 0) + 21 (2, 21 , 23 ) = ( 25 , 43 , 43 ),

and therefore σ 1 (3n − 5) = 0 and σ 3 (3n − 6) = 1.

One therefore has
γ3n−6 (σ) = 21 (0, 3, 1) + 21 ( 25 , 43 , 43 ) = ( 45 , 15
8 , 8 ),
7

and therefore σ 2 (3n − 7) = 0, σ 1 (3n − 8) = 0 and σ 3 (3n − 9) = 1.

Finally,
γ3n−9 (σ) = 21 (0, 3, 1) + 21 ( 45 , 15
8 , 8 ) = ( 8 , 16 , 16 ),
7 5 39 15

and therefore σ 2 (3n − 10) = 0, and σ 1 (3n − 11) = 1.

Therefore, any pure subgame-perfect 0-equilibrium must repeat the sequence of actions
(starting with player 1) (S, C, S; C, C, S; C, S, C; C, S, C). Along this cycle, player
1 first stops, then player 3 stops twice in a row, then player 2 stops twice in a row. This
difference with the subgame-perfect 0-equilibrium of Example 1 arises since the probability
of termination is here below 1. By further decreasing the probabilities pn for n = 2 or 3
modulo 3, while keeping pn = 1 for n = 1 modulo 3, one can create examples in which all
pure subgame-perfect equilibria have cycles of arbitrary length.
This example highlights one effect of low values for (pn ). Note indeed that the expected
payoff, starting from some stage n, is a convex combination of rn and of the continuation
payoff (the expected payoff, starting from stage n + 1). The weight of rn depends on the
probability of termination, but cannot exceed pn . In particular, when the probability of
termination is low, the expected payoff is close to the continuation payoff. Therefore, if
some player has an incentive to stop only once the continuation payoff reaches a certain
threshold, many stages may be required so that this threshold is reached. Thus, if the game
has a periodic equilibrium, lowering the probabilities of termination often results in periodic
equilibria with longer and longer periods.
We next introduce a two-player game that has no subgame-perfect 0-equilibrium and no
pure subgame-perfect ε-equilibrium.

Example 3. Take I = {1, 2}, and


(1, 1, (−1, 2)), n is odd,
(in , pn , rn ) =
(2, 1, (−2, 1)), n is even.

Fix ε ∈ (0, 1), and let σ be the strategy profile defined by σ 1 (2n+1) = 1 and σ 2 (2n+2) =
ε for every n ≥ 0. We claim that σ is a subgame-perfect ε-equilibrium. One should verify
that player 1 (respectively player 2) cannot profit by deviating in the subgames that start
at odd (respectively even) stages. Consider first the subgame that starts at stage 2n + 1,
for some n ≥ 0. By stopping at stage 2n + 1 player 1 receives −1, while, since player 2
eventually stops with probability 1, player 1’s payoff is at most −1, whatever he plays. In the
subgame starting at stage 2n + 2, player 2’s expected payoff under σ is ε + 2(1 − ε) = 2 − ε,
whereas the maximal payoff to player 2 in the game is 2.
8 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

We next prove that the game has no subgame-perfect ε-equilibrium in pure strategies.
Assume to the contrary that there exists such a profile σ.
We first claim that there is an infinite set of even stages in which player 2 chooses to
stop. Otherwise, let N be the maximal integer such that player 2 stops at stage 2N (set
N = 0 if player 2 never stops). Consider now the subgame that starts at stage 2N + 2.
By the definition of N, player 2 never stops in this subgame. Since σ is a subgame-perfect
ε-equilibrium, this implies that under σ player 1 never stops in this subgame: by never
stopping he receives 0, while by stopping he receives −1. But this leads to a contradiction,
as it implies that player 2 can profit 1 by deviating: by never stopping he receives 0, while
by stopping he receives 1.
We next claim that there is at most one even stage in which player 2 chooses to stop. To-
gether with the previous paragraph, this shows that there cannot be a subgame-perfect
ε-equilibrium. Assume that player 2 stops at stage 2N, with N > 1. Since σ induces an
ε-equilibrium in the subgame that starts at stage 2N − 1, and since player 2 stops at stage
2N, under σ player 1 stops at stage 2N − 1. However, since player 1 stops at stage 2N − 1,
under σ player 2 continues in all stages 2k for k < N: by continuing in all these stages he
receives 2, while his payoff upon stopping is 1.
This example shows that pure subgame-perfect ε-equilibria need not exist. Such a case
may arise when there is a player i who by stopping gives everyone else high payoff, but
he himself receives low payoff. It is then in the interest of his opponents to threaten him
that if he does not stop, one of them will eventually stop and punish player i. The punisher,
however, stops with low probability, so that player i has a chance to correct his behavior
and stop the game at a later stage.
We finally prove that there is no subgame-perfect 0-equilibrium. We argue by contra-
diction, and we let σ be a subgame-perfect 0-equilibrium. For i = 1, 2, we denote by ci
the strategy that always continues, i.e. ci (n) = 0 for each n ∈ Ni . Note first that, for each
n ∈ N , one has

P σ (θ < +∞|θ ≥ n) = 1. (1)

Indeed, the sequence (PP σ (θ < +∞|θ ≥ n))n∈N N would otherwise decrease to zero, hence
the sequence (γn (σ))n∈NN would converge to zero, and player 2 would have a profitable
deviation in the subgame starting at stage n, for n large enough. By (1) the game terminates
with probability 1, hence at least one of the players eventually stops with probability 1:

P σ 1 ,c2 (θ < +∞|θ ≥ n) = 1 for each n ∈ N , (2)

or

P c1 ,σ 2 (θ < +∞|θ ≥ n) = 1 for each n ∈ N . (3)

If (2) holds, then c2 is the best reply to σ 1 in all subgames, hence σ 2 = c2 . Since the unique
best reply of player 1 to c2 is c1 , one gets σ = (c1 , c2 )—a contradiction to (1).
If (3) holds, there are infinitely many even integers n such that σ 2 (n) > 0. By optimality
of σ 1 , and since (3) holds, one has σ 1 (n − 1) = 1 for any such n. Therefore, (2) holds—a
contradiction.
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 9

4. The Proof of Theorem 1

In the present section we prove Theorem 1.

4.1. Preliminaries

In this subsection, we analyze few degenerate cases, and slightly rephrase the problem.
The core of the proof of Theorem 1 is in Section 4.4.
Let Γ = (I, (in , pn , rn )n∈N N ) be a deterministic Dynkin game. Since the sequence (rn )n∈N N
is bounded, we can assume w.l.o.g. that payoffs are bounded by 1.
Let Γ̃ = (I, (in , pn , r̃n )n∈N
N ) be another game with the same sequence of active players
and the same probabilities of success. Since the payoff functions of the two games differ
by at most supn∈N N rn − r̃n , any subgame-perfect ε-equilibrium of Γ̃ is a subgame-perfect
ε -equilibrium of Γ , where ε = ε + supn∈N N rn − r̃n .
Since we are looking for an ε-equilibrium, and since payoffs are bounded, there is no
loss of generality in assuming that the range of the sequence (rn )n∈N N is finite, and that if
(i, r) and (j, r̃) are two distinct elements in that range then r k = r̃k for every k ∈ I.
Notice now that Theorem 1 will follow if we prove that there is a subgame-perfect
ε-equilibrium in some subgame of Γ . Indeed, the conclusion for Γ will then follow by
applying backward induction to the first stages of the game. Moreover, since finite extensive
games with perfect information have pure subgame-perfect equilibria, the resulting profile
will be pure when the subgame-perfect ε-equilibrium of the subgame is pure.
Let IR be the finite range of the sequence (in , rn )n∈N N . For each (i, r) ∈ IR define

π(i, r) = {pn |n ∈ N , (in , rn ) = (i, r)}

and set IR∞ = {(i, r) ∈ IR|π(i, r) = +∞}. If π(i, r) = +∞ then if player i stops whenever
(in , rn ) = (i, r), and all players continue in all other stages, the game will eventually
terminate, and the terminal payoff will be r.
We now argue that we may assume w.l.o.g. that
π(i, r) = +∞ for each (i, r) ∈ IR. (4)
As a first step, we prove that we may assume w.l.o.g. that
π(i, r) = 0 for each (i, r) ∈
/ IR∞ . (5)
Choose first N ∈ N large enough such that

pn < ε/|IR| for each (i, r) ∈
/ IR∞ . (6)
n≥N:(in ,rn )=(i,r)

Such an N exists since IR is a finite set. Denote by ΓN the subgame that starts at stage
N. Let Γ̃N = (I, (in , p̃n , rn )n∈N
N ) be the game that coincides with ΓN except that p̃n = 0
whenever (ĩn , r̃n ) ∈
/ IR∞ .
By (6), the payoff functions of the two games ΓN and Γ̃N differ by at most 2ε. Therefore,
any subgame-perfect ε-equilibrium of Γ̃N is a subgame-perfect 3ε-equilibrium of ΓN , and,
by backward induction, yields a subgame-perfect 3ε-equilibrium of Γ .
10 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

As the game Γ̃N satisfies (5), one can assume w.l.o.g. that (5) holds.
Thus, we are led to analyze games such that, for each (i, r) ∈ IR, either π(i, r) = 0
or π(i, r) = +∞. If π(i, r) = 0 for each (i, r) ∈ IR (so that IR∞ = ∅), the payoff
function of the game is identically zero and the conclusion of Theorem 1 follows tri-
vially.
Assume now that IR∞ = ∅. Consider the game obtained by dropping all stages n such that
π(in , rn ) = 0 (and by relabeling stages). Since there are infinitely many stages n such that
π(in , rn ) = +∞, the resulting game is again a deterministic multi-player Dynkin game.
Plainly, any subgame-perfect ε-equilibrium of this new game is also a subgame-perfect
ε-equilibrium of the initial game (with the proper identification of stages, and with an
arbitrary behavior in the stages that have been dropped).
It follows that we can assume w.l.o.g. that (4) holds.

4.2. A partition into blocks

In the present section we fix ε < 1/40. Given ε, we define a partition of the set N of
stages into blocks. This partition will be used in the sequel to prove Theorem 1.
We will use the following technical result.

Lemma 1 (Rosenberg et al.,  2002, Lemma 18). Let n ∈ N , and let p1 , . . . , pn be non-
negative reals that satisfy ni=1 pi < 1/20. Then,
n
 n 2 n i−1 n
   
pi − 20 pi ≤ pi (1 − pj ) ≤ pi .
i=1 i=1 i=1 j=1 i=1


Observe that ni=1 pi i−1j=1 (1 − pj ) is the probability that the result of at least one out
of n coins with parameters p1 , . . . , pn is Head. In particular it is equal to 1 − ni=1 (1 − pi ).

Corollary 1. Let ε
 < 1/40, n ∈ N , and p1 , . . . , pn be non-negative reals that satisfy
n n i−1
p
i=1 i ≥ ε. Then i=1 pi j=1 (1 − pj ) ≥ ε/2.

Proof.
 The proof is divided into three cases.
If ni=1 pi ≤ 1/20 the claim follows from Lemma 1 and since ε < 1/40.
If there is i such that pi ≥ ε the claim holds trivially. 
Otherwise, there is a subset I ⊂ {1, . . . , n} such that 1/20 − ε ≤ i∈I pi ≤ 1/20. Then
n
 i−1 n
pi (1 − pj ) = 1 − (1 − pi ) ≥ 1 − (1 − pi )
i=1 j=1 i=1 i∈I
2
1 1
≥ − ε − 20 −ε ≥ ε − 20ε2 ≥ ε/2,
20 20
where the second inequality follows from Lemma 1, and the third one holds since the
function x − 20x2 is monotonic decreasing for x < 1/20 and since ε < 1/40. 䊐
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 11

We are now ready to define the partition of N into blocks. Set n0 = 1 and, for l ∈ N ,
define the initial stage nl of block l to be
 
  
nl = min n > nl−1 | pk ≥  ∀(i, r) ∈ IR .
 
nl−1 ≤k<n,(ik ,rk )=(i,r)

Since π(i, r) = +∞ for each (i, r) ∈ IR, all nl , l ∈ N , are finite.

By Corollary 1, in each block all players have a probability at least ε/2 to terminate the
game with any vector they choose.

4.3. A simple case

Under the assumption that π(i, r) = +∞ for each (i, r) ∈ IR, the proof proceeds by
induction over the number of elements in IR. The conclusion is easy if |IR| = 1, and is left
to the reader.
We now analyze a somewhat degenerate case that generalizes Example 2. This is the only
place in the proof where we use the induction hypothesis.

Lemma 2. Assume that there exists (i, r) ∈ IR such that

r j ≥ r̃j for every (j, r̃) ∈ IR.

Then for each ε > 0 there is a subgame-perfect -equilibrium.

The lemma states that if there is a terminal payoff r that is preferred by each player i to all
terminal payoffs i controls, then a subgame-perfect ε-equilibrium exists.

Proof. We assume w.l.o.g. that ε < 1/40, and we split the discussion into three cases.

Case 1. ri ≥ 0.
Let σ be the pure strategy profile in which player i stops whenever (in , rn ) = (i, r), and
all players continue in all other stages, i.e.

σ in (n) = 1 if and only if (in , rn ) = (i, r).

Fix n ∈ N . We prove that σ induces a 0-equilibrium in the subgame that starts at stage
n. Since π(i, r) = +∞, the game eventually terminates, and therefore the expected payoff
is r. Player i cannot gain by deviating, since his payoff is at most ri if he terminates the
game, and 0 ≤ ri if he always continues. Every player j = i cannot gain by deviating either,
since his payoff under σ is r j , while if he deviates his payoff is in the convex hull of r j and
{r̃j , (j, r̃) ∈ IR}, hence at most r j .

Case 2. r i < 0, and there is (j, r̃) ∈ IR such that i = j and r̃i < ri .
In this case, we elaborate upon the construction in Example 3. We will have player i stop
at all stages in {n ∈ N : (in , rn ) = (i, r)}, and player j stop with some small probability at
stages in {n ∈ N : (in , rn ) = (j, r̃)}. The choices of the corresponding probabilities should
12 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

fulfill two conditions: (i) these values should be small enough so that γn (σ) − r < ε,
for each n ∈ N and (ii) they should be high enough so that, if player i were to continue at
all stages, the game would still stop a.s. in finite time. These two conditions relate to the
two dual aspects of the threat. By condition (i) the threat will be used on the equilibrium
path with small probability. By condition (ii) it will provide incentives to player i to act as
required.
Recall the partition of N into blocks that was defined in Section 4.2, and that nl is the
first stage of block l, l ≥ 0. 
Since for every l ≥ 0 one has nl ≤n<nl+1 :(in ,rn )=(j,r̃) pn ≥ ε, there is a function x:

N → [0, 1] such that for every l ≥ 0 one has nl ≤n<nl+1 :(in ,rn )=(j,r̃) xn pn = ε2 .
We let σ be the strategy profile in which player i stops whenever (in , rn ) = (i, r), player j
stops with probability xn whenever (in , rn ) = (j, r̃), and all players continue otherwise, i.e.


 1, if (in , rn ) = (i, r),

in
σ (n) = xn , if (in , rn ) = (j, r̃),


 0, otherwise.

We prove that σ is a subgame-perfect 2ε-equilibrium. Let n ∈ N and consider the subgame

that starts at stage n.
The definition of σ, Lemma 1 and Corollary 1 imply that (a) the probability that player
i stops under σ in each block l, conditioned that the game reaches stage nl , is at least
(1 − ε2 )ε/2, and (b) the probability that player j stops under σ in each block l, conditioned
that the game reaches stage nl , is between (1 − ε)ε2 /2 and ε2 .
This implies that γn (σ) − r ≤ 2ε. Furthermore, (a) and (b) imply that under any
unilateral deviation the game terminates with probability one.
Since for every player k and every (k, r  ) ∈ IR one has rk ≤ rk ≤ γnk (σ) + 2ε, no player
k = i can profit more than 2ε by deviating from σ in the subgame that starts at stage n.
Since r̃i ≤ ri ≤ γni (σ) + 2ε, the same applies to player i.

Case 3. r i < 0, and r̃ i ≥ ri for every (j, r̃) ∈ IR with i = j.

In that case, by the assumption of the lemma, the strategy of player i that always continues
is a weakly dominant strategy.
Consider the modified game where one sets pn = 0 whenever in = i, or, alternatively,
one drops all stages in which in = i. Note that player i is a dummy in the modified game.
By the induction hypothesis, the modified game admits a subgame-perfect ε-equilibrium
σ  . Extend σ  to a profile σ in the original game, by instructing player i to continue at all
stages n. Then σ is a subgame-perfect ε-equilibrium.

4.4. The general case

In view of Lemma 2, Theorem 1 will follow from Proposition 1 below.

Proposition 1. Let Γ be a deterministic multi-player Dynkin game. Assume that for every
(i, r) ∈ IR, (i) π(i, r) = +∞, and (ii) there is (j, r̃) ∈ IR such that r̃j > rj . Then, for every
ε > 0, the game Γ has a subgame-perfect ε-equilibrium in pure Markovian strategies.
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 13

Note that Example 3 does not fit into Proposition 1. We do not know whether a subgame-
perfect 0-equilibrium exists or not. The rest of this section is devoted to the proof of the
proposition.
As remarked at the beginning of Section 4.1, we can assume w.l.o.g. that for every
(i, r), (j, r̃) ∈ IR, either (i, r) = (j, r̃), or r k = r̃k for every k.
For every i ∈ I set
mi = max{ri |(i, r) ∈ IR}.
This is the maximal terminal payoff player i can receive when stopping alone. Let ρi ∈ R I
be the unique vector r such that (i, r) ∈ IR and r i = mi (uniqueness is guaranteed by the
preceeding paragraph).
Finally, set
W = {w ∈ R N |wi ≤ mi for some i ∈ I}.
This is the set of all payoff vectors w such that at least one player is better off by stopping
at some stage rather than continuing forever and receiving w.
An important property of the set W is that if the continuation payoff at stage n is w ∈
W, and if player in prefers to stop rather than continue (that is, win ≤ rnin ), then the
expected payoff if player in stops at stage n, (1 − pn )w + pn rn , is in W. Formally, for every
n ∈ N,
w ∈ W and win ≤ rnin imply (1 − pn )w + pn rn ∈ W. (7)
Indeed, under the assumptions, (1 − pn )win + pn rnin ≤ rnin ≤ min , and (7) follows.
We will prove the existence of a subgame-perfect ε-equilibrium. We assume w.l.o.g. that
ε < 1/40, and that furthermore ε < 21 min(i,r)=(j,r̃) |r i − r̃ i |.
Let l ∈ N be given. We will define a pure profile σl up to stage nl . We will simultaneously
nl
construct a sequence (wl (n))n=1 of vectors in W. As a first approximation, the vector wl (n)
may be interpreted as the expected payoff under σ from stage n onwards.
As for now, we fix l ∈ N and we write σ and w instead of σl and wl , respectively.
We define both σ and w backwards. We let w(nl ) be an arbitrary point in W ∩ [−1, 1]I .
We deal with each of the blocks inductively (starting with the lth one). Let k ≤ l. Assuming
w(nk ) ∈ W is already defined, we define now σ and w over the stages n = nk−1 , . . . ,
nk − 1.
Given w(n + 1) and σ in (n), we set w(n) = σ in (n)pn rn + (1 − σ in (n)pn )w(n + 1), so
that we need only define σ in (n). Thus, if w(n + 1) is the expected payoff from stage n + 1
onwards, w(n) is the expected payoff from stage n onwards. We will define σ in (n) such
that (i) σ in (n) is pure, and (ii) σ in (n) = 1 implies rnin ≥ win (n + 1). Since w(nl ) ∈ W and
by (7), this implies that w(n) ∈ W for every n ≤ nl .

Case 1. wi (nk ) ≤ mi −  for some i ∈ I.

We define σ by backward induction, with an appropriate tie-breaking rule. Set σ in (n) = 1

if rnin ≥ win (n + 1), and σ in (n) = 0 otherwise.
Thus, at stage n, player in compares his continuation payoff win (n + 1) to the payoff rnin
he would get by stopping, and he continues or stops accordingly.
14 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

Case 2. wi (nk ) > mi −  for each i ∈ I.

Fix i∗ ∈ I such that wi∗ (nk ) ≤ mi∗ . Since w(nk ) ∈ W, such a player exists. We will
define σ so that at the final stages of the block only player i∗ will possibly stop. In earlier
stages, σ will be defined using backward induction as in Case 1.
Formally, let nk−1 ≤ n < nk . Assume that σ has been defined for stages q = n +
1, . . . , nk − 1. We define σ at stage n as follows. Denote by π(n + 1, nk ) the probability
under σ that, starting from stage n + 1, the game terminates under σ before stage nk , i.e.

π(nk , nk ) = 0, and
π(q, nk ) = σ iq (q)pq + (1 − σ iq (q))π(q + 1, nk ) for n + 1 ≤ q < nk .

Then:
• if π(n + 1, nk ) < ε, we set σ in (n) = 1 if both in = i∗ and rni∗ ≥ wi∗ (n) hold. We set
σ in (n) = 0 otherwise;
• if π(n + 1, nk ) ≥ ε, we set σ in (n) = 1 if rnin ≥ win (n), and σ in (n) = 0 otherwise.
We now prove that under σ, the probability of termination in any single block is bounded
away from zero.

Lemma 3. For each k such that 0 ≤ k < l, one has

P σ (θ < nk+1 |θ ≥ nk ) ≥ 13 ε.

Proof. We will prove that π(nk , nk+1 ) ≥ ε/3. We consider Cases 1 and 2 in turn.
We first assume that Case 1 holds, and we let i∗ ∈ I be a player such that wi∗ (nk+1 ) ≤
i
m ∗ − .
1. If σ i∗ (n) = 1 whenever (in , rn ) = (i∗ , ρi∗ ), one has by Corollary 1 π(nk , nk+1 ) ≥ ε/2.
2. If σ i∗ (n) = 0 for some n such that (in , rn ) = (i∗ , ρi∗ ), then wi∗ (n + 1) > mi∗ . Observe
now that, since payoffs are bounded by one, one has

wi∗ (n + 1) ≤ π(n + 1, nk+1 ) + (1 − π(n + 1, nk+1 ))wi∗ (nk+1 ).

By the choice of i∗ one has wi∗ (nk+1 ) ≤ mi∗ − ε, so that

ε ε
π(n + 1, nk+1 ) ≥ ≥ .
1 − mi∗ + ε 3
Since π(nk , nk+1 ) ≥ π(n + 1, nk+1 ), the conclusion also follows in that case.
We next assume that Case 2 holds and we let i∗ ∈ I be the player distinguished in the
definition of σ.
1. Assume first that σ j (n) = 1 for some n and some player j = i∗ . By definition of the
profile σ, one then has π(n+1, nk+1 ) ≥ ε/2, hence π(nk , nk+1 ) ≥ π(n+1, nk+1 ) ≥ ε/2.
2. Assume now that σ in (n) = 0 whenever in = i∗ . In that case, wi∗ (n) ≤ mi∗ for each
n. Indeed, only player i∗ stops, and his payoff is the average of wi∗ (nk+1 ) ≤ mi∗
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 15

and ρii∗∗ = mi∗ . Therefore, σ i∗ (n) = 1 whenever (in , rn ) = (i∗ , ρi∗ ), and one gets
π(nk , nk+1 ) ≥ ε/2, as in Case 1, item 1. 䊐

We will now let l vary and we denote by σl and wl the objects that were defined above.
The pure strategy profile σl may be identified with a point in {0, 1}N (the nth component
being the behavior at stage n of the active player in ). Since the product space {0, 1}N is
compact (and metrizable), the sequence (σl )l≥0 has a subsequence that converges to some
pure strategy profile σ∗ . For notational convenience, we still denote this subsequence by
(σl )l≥0 . Since σl is a pure strategy for every l ∈ N , for every fixed n ∈ N the first n
components of σ∗ coincide with the first n components of σl , provided l is sufficiently large.
For such l’s, the behavior in the first n stages of the game under the two strategy profiles σ∗
and σl coincide.
Our goal is to prove that σ∗ is a subgame-perfect ε-equilibrium. We first prove that the play
terminates P σ∗ -a.s. in each subgame. We will then relate the payoff γ(σ∗ ) to the sequence
(wl )l∈N
N (Lemma 4) and prove that no player has a profitable one-stage deviation (Lemma 5)
under σ∗ . The conclusion follows (Proposition 2), after we prove that no single player is
responsible for the termination of the game (Lemma 6).

Corollary 2. For each k ∈ N , one has

P σ∗ (θ < nk+1 |θ ≥ nk ) ≥ 13 ε.

Proof. Let l > k be large enough so that σ∗ coincides with σl up to stage nk+1 , and apply
Lemma 3. 䊐

Lemma 4. For each n ∈ N , one has

γn (σ∗ ) = lim wl (n).
l→∞

Proof. We prove the result for n = 1. The proof is similar for the subgame that starts at
any stage n ∈ N .
Let k ∈ N be given. For each l ≥ k, one has
γ(σ∗ ) = E σ∗ [rθ 1θ<nk ] + P σ∗ (θ ≥ nk )γnk (σ∗ ),
and
wl (1) = E σl [rθ 1θ<nk ] + P σl (θ ≥ nk )wl (nk ).
For l large enough, the two profiles σl and σ∗ coincide up to stage nk . In particular,
E σ∗ [rθ 1θ<nk ] = E σl [rθ 1θ<nk ] and P σ∗ (θ ≥ nk ) = P σl (θ ≥ nk ). By Corollary 2

γ(σ∗ ) − wl (1) ≤ 2(1 − 31 ε)k

provided l is large enough, and the result follows. 䊐

The next lemma says in substance that no player can increase his payoff by more than
3ε by modifying his strategy in a single stage.
16 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

Lemma 5. Let n ∈ N be given. The following implications hold.

• If σ∗in (n) = 0 then γn+1
in
(σ∗ ) ≥ rnin − 3ε.
• If σ∗in (n) = 1 then γn+1
in
(σ∗ ) ≤ rnin .

Proof. Let n ∈ N be given. Let l ∈ N be sufficiently large so that nl > n. We first prove a
related statement for the strategy profile σl . Let k < l be determined by nk ≤ n < nk+1 .
By construction, σlin (n) is defined using backward induction, except in one case where
σl (n) is required to be zero. In Case 1 one has σlin (n) = 1 if rnin ≥ wiln (n+1) and σlin (n) = 0
in

otherwise. In Case 2 one has wiln (nk+1 ) ≥ min − ε and π(n + 1, nk+1 ) < ε. Therefore,
|wiln (nk+1 ) − wiln (n + 1)| < 2ε, which yields wiln (n + 1) ≥ min − 3ε ≥ rnin − 3ε. Hence,
in both cases, one has

wiln (n + 1) ≤ rnin , if σlin (n) = 1, and

wiln (n + 1) ≥ rnin − 3ε, if σlin (n) = 0.

The conclusion follows by taking the limit l → +∞ and using Lemma 4. 䊐

We now prove that the play terminates a.s., even if a single player chooses to continue
whenever active. Recall that ci is the strategy of player i that always continues.

Lemma 6. For every i ∈ I and every n ∈ N , one has

P ci ,σ∗−i (θ < +∞|θ ≥ n) = 1.

Proof. We argue by contradiction, and we assume that, for some player i ∈ I, the sequence
P ci ,σ∗−i (θ < +∞|θ ≥ n) converges to zero when n goes to +∞. By Corollary 2 the game
eventually terminates, so that P σ∗ (θ < +∞|θ ≥ n) = 1 for every n. Therefore, it must be
the case that player i terminates the game: P σ∗i ,c−i (θ < +∞|θ ≥ n) = 1 for every n, and
limn→+∞ γn (σ∗ ) − γn (σ∗i , c−i ) = 0.
We first prove that limn→+∞ γn (σ∗ ) = ρi , and then deduce a contradiction with the basic
assumption made on Γ .

Step 1. The sequence (γni (σ∗ ))n∈NN has a limit.

i (σ ). On the
Let n ∈ N be arbitrary. If n ∈ Ni then Lemma 5 implies that γni (σ∗ ) ≥ γn+1 ∗
i i in
other hand, for n ∈
/ Ni , one has γn (σ∗ ) = γn+1 (σ∗ ) if σ∗ (n) = 0, and

|γni (σ∗ ) − γn+1

i
(σ∗ )| = pn |rni − γn+1
i
(σ∗ )| ≤ 2pn

if σ∗in (n) = 1. Therefore, for every two positive integers n ≥ m, one has

i
γm (σ∗ ) ≥ γni (σ∗ ) − 2 pq1 iq . (8)
σ∗ (q)=1
m≤q<n;q∈N
/ i
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 17

Let ε̃ ∈ (0, 1/40) be given. Choose Nε̃ ∈ N sufficiently large so that P ci ,σ∗−i (θ <

+∞|θ ≥ Nε̃ ) < ε̃/2. For such Nε̃ , one has by Corollary 1 Nε̃ ≤q<+∞;q∈N
/ i pq 1 iq ≤
σ∗ (q)=1
ε̃. Therefore, by (8),
i
γm (σ∗ ) ≥ γni (σ∗ ) − 2ε̃, for every n ≥ m ≥ Nε̃ .

This implies the convergence of (γni (σ∗ ))n∈N

N , since it is a bounded sequence.

Step 2. limn→+∞ γn (σ∗ ) = ρi .

Denote λ := limn→+∞ γn (σ∗ ). We first prove that λi = mi .
Fix δ > 0 sufficiently small, and take k sufficiently large so that (i) |γnk (σ∗ ) − λ| < δ,
(ii) |γnk+1 (σ∗ ) − λ| < δ, and (iii) P ci ,σ∗−i (θ < +∞|θ ≥ nk ) < δ.
By Lemma 4 and since σ∗ = liml→+∞ σl , there is l > k sufficiently large such that
(i) |wl (nk ) − λ| < δ, (ii) |wl (nk+1 ) − λ| < δ, and (iii) P ci ,σl (θ < nk+1 |θ ≥ nk ) < δ.
Consider now the block that is played between stages nk and nk+1 under σl . By (iii), the
probability that the game terminates by a player j = i is smaller than δ. Therefore, player
i never stops at a stage n such that in = i and rni < λi − 2δ. However, the probability that
player i stops at a stage n such that in = i and rni = mi is at least ε/2. Therefore,

2δ > wil (nk ) − wil (nk+1 ) > 21 ε(mi − λi ) − 3δ and − δ < mi − λi ,

so that −δ < mi − λi < 10δ/ε. As δ is arbitrary, the first claim follows.

Hence, limn→+∞ γni (σ∗ ) = mi . This yields limn→+∞ γni (σ∗i , c−i ) = mi . Since ρi ∈ R I
is the unique vector such that (i, r) ∈ IR and r i = mi , and since γn (σ∗i , c−i ) is in the
convex hull of {r̃: (i, r̃) ∈ IR}, one has limn→+∞ γn (σ∗i , c−i ) = ρi . Finally, this implies
limn→+∞ γn (σ∗ ) = ρi .

Step 3. The contradiction.

j
By assumption, there exists (j, r̃) ∈ IR such that r̃j > ρi . Since P ci ,σ∗−i (θ < +∞|θ ≥
m) < 1 for some m ∈ N , and since π(j, r̃) = +∞, there are infinitely many stages n such
j j
that (in , rn ) = (j, r̃) and σ∗ (n) = 0. For each such n, by Lemma 5, one has r̃ j = rn <
j j j j j j
γn+1 (σ∗ ). Therefore, lim supn→+∞ γn (σ∗ ) ≥ r̃ . Since r̃ > ρi = limn→+∞ γn (σ∗ ), we
get a contradiction.

Proposition 2. σ∗ is a subgame-perfect 3ε-equilibrium.

Proof. Let i ∈ I be given. We prove that player i cannot gain more than 3ε by deviating from
σ∗ . The same proof will hold in any subgame, thereby showing the subgame-perfectness
property.
Define the sequence (Xn )n∈N i i
N of random variables by Xn = rθ if θ < n and Xn = γn (σ∗ )
i
if θ ≥ n. Let τ be an arbitrary strategy of player i. By Lemma 6, the sequence (Xn )n∈N N
converges P τ i ,σ∗−i -a.s. to X∞ := rθ 1 θ<+∞ , hence

E
lim E i −i [Xn ] = E τ i ,σ∗−i [rθ 1 θ<+∞ ] = γ i (τ i , σ∗−i ). (9)
n→+∞ τ ,σ∗
18 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19

On the other hand, let n ∈ N , and denote by Hn the past play up to stage n. We shall
prove that
E τ i ,σ∗−i [Xn+1 |Hn ] ≤ Xn + 3ε11θ=n , a.s. (10)

On the event θ < n, both Xn and Xn+1 are equal to rθi . Consider now the event θ ≥ n. If
in = i, one has
Xn = γni (σ∗ ) = E σ∗ [Xn+1 |Hn ] = E τ i ,σ∗−i [Xn+1 |Hn ],

where the last equality follows since the two profiles (τ i , σ∗−i ) and σ∗ coincide at stage n.
In both cases, (10) follows trivially. Finally, if in = i, one has Xn = Xn+1 = γn+1 i (σ ) if
∗
in i (σ ) otherwise. Inequality (10)
σ∗ (n) = 0 and E τ i ,σ∗−i [Xn+1 |Hn ] = pn rni + (1 − pn )γn+1 ∗
then follows by Lemma 5.
By taking expectations in (10), and by summing over n, one obtains limn→+∞E τ i ,σ∗−i [Xn ]
≤ X1 + 3ε which yields, by (9),
γ i (τ i , σ∗−i ) ≤ γ i (σ∗ ) + 3ε. 䊐

Acknowledgements

We acknowledge the financial support of the Arc-en-Ciel/Keshet program for 2001/2002.

The research of the first author was supported by the Israel Science Foundation (Grant No.
03620191).

References

Dynkin, E.B., 1969. Game variant of a problem on optimal stopping. Soviet Mathematics Doklady 10, 270–274.
Fine, C.H., Li, L., 1989. Equilibrium exit in stochastically declining industries. Games and Economic Behavior
1, 40–59.
Flesch, J., Thuijsman, F., Vrieze, K., 1997. Cyclic Markov equilibria in stochastic games. International Journal of
Game Theory 26, 303–314.
Fudenberg, D., Tirole, J., 1991. Game Theory. MIT Press, Cambridge, MA.
Ghemawat, P., Nalebuff, B., 1985. Exit. RAND Journal of Economics 16, 184–194.
Kiefer, Y.I., 1971. Optimal stopped games. Theory of Probability and its Application 16, 185–189.
Kilgour, D.M., 1975. The sequential truel. International Journal of Game Theory 4, 151–174.
Kilgour, D.M., 1977. Equilibrium points in infinite sequential truels. International Journal of Game Theory 6,
167–180.
Kilgour, D.M., Brams, S.J., 1997. The truel. Mathematics Magazine 70, 315–325.
Neveu, J., 1975. Discrete-Parameter Martingales. North-Holland, Amsterdam.
Rosenberg, D., Solan, E., Vieille, N., 2001. Stopping games with randomized strategies. Probability Theory and
Related Fields 119, 433–451.
Rosenberg, D., Solan, E., Vieille, N., 2002. On the maxmin value of stochastic games with imperfect monitoring.
Discussion Paper 1337. The Center for Mathematical Studies in Economics and Management Science, Kellogg
School of Management, Northwestern University.
Shmaya, E., Solan, E., 2002. Two-player non-zero-sum stopping games in discrete time. Discussion Paper 1347.
The Center for Mathematical Studies in Economics and Management Science, Kellogg School of Management,
Northwestern University.
 E. Solan, N. Vieille / Journal of Mathematical Economics 1097 (2003) 1–19 19

Solan, E., 1999. Three-person absorbing games. Mathematics of Operations Research 24, 669–698.
Solan, E., 2002. Subgame perfection in quitting games with perfect information and differential equations.
Discussion Paper 1356. The Center for Mathematical Studies in Economics and Management Science, Kellogg
School of Management, Northwestern University.
Solan, E., Vieille, N., 2001. Quitting games. Mathematics of Operations Research 26, 265–285.
Vieille, N., 2000. Equilibrium in 2-person stochastic games. II. The case of recursive games. Israel Journal of
Mathematics 119, 93–126.