Int J Game Theory (2016) 45:321–352

DOI 10.1007/s00182-015-0507-5

ORIGINAL PAPER

Continuous-time limit of dynamic games with

incomplete information and a more informed player

Fabien Gensbittel1

Accepted: 17 September 2015 / Published online: 24 September 2015

© Springer-Verlag Berlin Heidelberg 2015

Abstract We study a two-player, zero-sum, dynamic game with incomplete informa-

tion where one of the players is more informed than his opponent. We analyze the
limit value as the players play more and more frequently. The more informed player
observes the realization of a Markov process (X, Y ) on which the payoffs depend,
while the less informed player only observes Y and his opponent’s actions. We show
the existence of a limit value as the time span between two consecutive stages goes to
zero. This value is characterized through an auxiliary optimization problem and as the
unique viscosity solution of a second order Hamilton–Jacobi equation with convexity
constraints.

Keywords Incomplete information · Repeated games · Hamilton–Jacobi

1 Introduction

This paper contributes to the literature on zero-sum dynamic games with incomplete
information, by considering the case where one player is always more informed than
his opponent.
A key feature appearing in recent contributions to the field of zero-sum dynamic
games is the interplay between discrete-time and continuous-time dynamic models,
as in Cardaliaguet et al. (2012), Neyman (2013) or Cardaliaguet et al. (2013), where
the authors consider sequences of discrete-time dynamic games in which the players
play more and more frequently. Such an analysis is related to the study of a sequence

B Fabien Gensbittel
[email protected]; [email protected]

1 Toulouse School of Economics (GREMAQ, Université Toulouse 1 Capitole),

Manufacture des Tabacs, MF213, 21, Allée de Brienne, 31015 Toulouse Cedex 6, France

123
322 F. Gensbittel

of discretizations in time of a given continuous-time dynamic game. In the present

work, we adopt this method in order to study a continuous-time zero-sum dynamic
game where one player is always more informed than his opponent and where the state
variable evolves according to an exogenous Markov process. Precisely, we consider
a model with two payoff-relevant variables (X t , Yt )t≥0 which are evolving over time:
X is a Markov chain with finite state space and Y is a diffusion process whose drift
parameter depend on the current value of X . The process X is privately observed
by the more informed player (say player 1) while Y is publicly observed, allowing
the less informed player (player 2) to learn information about the variable X during
the game. We analyze the sequence of discrete-time games indexed by n ≥ 1 with
incomplete information and perfect observation of actions, where stages occur at times
q
n for q ≥ 0. At each stage, player 1 observes a pair of signals (X n , Y n ) while player
q q

2 only observes Y q . The stage payoff function is assumed to depend on actions of

n
both players and on (X q , Y q ). The global payoff is a discounted sum of the stage
n n
 1/n
payoffs with discount factor λn = 0 r e−r t dt = 1 − e−r/n , where r > 0 is a given
continuous-time discount rate. We assume that the stage payoffs are not observed and
we study the limit value of these games as the players play more and more frequently.
We provide two characterizations for the limit value of these games as n goes to
infinity. The first one is a probabilistic representation formula where the optimization
variable is the set of admissible belief processes for the less informed player. Such
a formula already appears in Sorin (2002) as an illustration of the classical Cav(u)
theorem of Aumann and Maschler (1995). A similar discrete-time formula was intro-
duced by De Meyer (2010) in order to obtain a continuous-time limit value in a class of
financial games and this approach led to several extensions in continuous-time mod-
els [see Cardaliaguet and Rainer (2009, 2012), Grün (2013, 2012), Gensbittel (2015,
2013), and more recently Cardaliaguet et al. (2013) and Gensbittel and Grün (2014)].
This representation formula is important as it provides a characterization of optimal
processes of revelation (martingales of posteriors induced by optimal strategies).
The second one is a variational characterization, the limit value is shown to be
the unique viscosity solution of a second-order Hamilton–Jacobi equation with con-
vexity constraints as introduced by Cardaliaguet (2007, 2009) and generalized in
Cardaliaguet and Rainer (2009), Grün (2013), Cardaliaguet et al. (2013) and Gensbit-
tel and Grün (2014).

2 Main results

Notation 2.1 For any topological space E, (E) denotes the set of Borel probability
distributions on E endowed with the weak topology and the associated Borel σ -
algebra. δx denotes the Dirac measure on x ∈ E. Finite sets are endowed with the
discrete topology and Cartesian products with the product topology. D([0, ∞), E)
denotes the set of càdlàg trajectories taking values in E, endowed with the topology
of convergence in Lebesgue measure. The notations ,  and |.| stand for the canonical
scalar product and the associated norm in Rm .

123
Continuous-time limit of dynamic games... 323

Let us at first describe the continuous-time game we will approximate. This descrip-
tion is incomplete as we do not define strategies in continuous-time. Rather, we define
below strategies in the different time-discretizations of this game. The notion of value
for this game will therefore be the limit value along a sequence of discretizations when
the mesh of the corresponding partitions goes to zero.
We assume that (X t )t∈[0,∞) is a continuous-time homogeneous Markov chain with
finite state space K , infinitesimal generator R = (Rk,k  )k,k  ∈K and initial law p ∈
(K ). We identify (K ) with the canonical simplex of R K , i.e.:
 

(K ) = p ∈ R |∀k ∈ K , p(k) ≥ 0,
K
p(k) = 1 .
k∈K

Then, we define the real-valued process (Yt )t∈[0,∞) as the unique solution of the
following stochastic differential equation (SDE)
 t  t
∀t ≥ 0, Yt = y + b(X s , Ys )ds + σ (Ys )dWs , (2.1)
0 0

where (Wt )t∈[0,∞) is a standard Brownian motion independent of X and y ∈ R is a

given initial condition. The process Y may be seen as some noisy observation of the
process X .
We assume that the functions b and σ in (2.1) are bounded and Lipschitz, and that
there exists  > 0 such that for all y ∈ R, σ (y) ≥ . The state process Z := (X, Y )
with values in K × R is a well defined Feller Markov process, with semi-group of
transition probabilities denoted (Pt )t≥0 .
Let I, J denote finite action sets for the two players (players 1 and 2), g : (K ×
R) × I × J → R a bounded payoff function which is Lipschitz with respect to the
second variable, and r > 0 a fixed discount rate.
We consider the following (heuristic) zero-sum game, played on the time interval
[0, ∞):
• Player 1 observes the trajectory of Z = (X, Y ).
• Player 2 observes only the trajectory of Y .
• They play the game G( p, y) with total expected payoff for player 1:
 +∞ 
E r e−r t g(X t , Yt , i t , jt )dt ,
0

where i t (resp. jt ) denote the action of player 1 at time t (resp. of player 2).
• Actions are observed during the game (and potentially convey relevant informa-
tion).
We aim at studying the value function of this game and how information is used by
the more informed player when playing optimally. In order to achieve this goal, we
introduce a sequence of time-dicretizations of the game. For simplicity, and without
loss of generality, let us consider the uniform partition of [0, +∞) of mesh 1/n. The
corresponding discrete-time game, denoted G n ( p, y) proceeds as follows:

123
324 F. Gensbittel

• The variable Z q = (X q , Y q ) is observed by player 1 before stage q for q ≥ 0.

n n n
• The variable Y q is observed by player 2 before stage q for q ≥ 0.
n
• At each stage, both players choose simultaneously a pair of actions (i q , jq ) ∈ I × J .
• Chosen actions are observed after each stage.
• Stage payoff of player 1 equals g(Z q , i q , jq ) (realized stage payoffs are not
n
observed).
• The total expected payoff of player 1 is
⎡ ⎤

E ⎣ λn (1 − λn )q g Z q , i q , jq ⎦ ,
n
q≥0

with λn = 1 − e−r/n

Remark 2.2 When σ is constant and b depends only on X , the observation of player
 q+1
2 correspond to a normally distributed random variable with mean q n b(X s )ds and
n
σ2
variance n . It may therefore be interpreted as a noisy observation of X .

The description of the game is common knowledge and we consider the game played
in behavior strategies: at round q, player 1 and player 2 select simultaneously and
independently an action i q ∈ I for player 1 and jq ∈ J for player 2 using some
lotteries depending on their past observations.
Formally, a behavior strategy σ for player 1 is a sequence (σq )q≥0 of transition
probabilities:

σq : ((K × R) × I × J )q × (K × R) → (I ),

where σq (Z 0 , i 0 , j0 , . . . , Z q−1 , i q−1 , jq−1 , Z q ) denotes the lottery used to select the
n n
action i q played at round q by player 1 when past actions played during the game are
(i 0 , j0 , . . . , i q−1 , jq−1 ) and the sequence of observations of player 1 is (Z 0 , . . . , Z q ).
n
Let  denote the set of behavior strategies for player 1. Similarly, a behavior strategy
τ for player 2 is a sequence (τq )q≥0 of transition probabilities depending on his past
observations

τq : (R × I × J )q × R → (J ).

Let T denote the set of behavior strategies for player 2.

Let P(n, p,y,σ,τ ) ∈ (D([0, ∞), K × R) × (I × J )N ) denote the probability on the set
of trajectories of Z and actions induced by the strategies σ, τ . The payoff function in
G n ( p, y) is defined by
⎡ ⎤

γn (ν, σ, τ ) := EP(n, p,y,σ,τ ) ⎣λn (1 − λn )q g Z q , i q , jq ⎦ .
n
q≥0

123
Continuous-time limit of dynamic games... 325

It is well known that the value of the game exists, i.e.

Vn ( p, y) := sup inf γn ( p, y, σ, τ ) = inf sup γn ( p, y, σ, τ ).

σ ∈ τ ∈T τ ∈T σ ∈

We also need to consider the value function u of the non-revealing one-stage game
( p, y), which is a finite game with payoff g in which player 1 cannot use his private
information. Precisely,
 
u( p, y) := sup inf p(k)σ (i)τ ( j)g(k, y, i, j),
σ ∈(I ) τ ∈(J ) i∈I j∈J k∈K

and the value exists (i.e. the sup and inf commute in the above formula) as it is a finite
game. It follows from standard arguments that u is Lipchitz in ( p, y).
The main results proved in Sects. 3 and 4 are two different characterizations for the
limit of the sequence of value functions Vn .
Let us now introduce some notations.
Notation 2.3
• The natural filtration F A of a process (At )t∈[0,∞) is defined by FtA = σ (As , s ≤
t). The associated right-continuous filtration is denoted F A,+ with FtA,+ :=
∩s>t FsA .
• For any topological space E, D([0, ∞), E) denotes the set of E-valued càdlàg
trajectories.
• For all ( p, y) ∈ (K ) × R, P p,y ∈ (D([0, +∞), K × R)) denotes the law of
the process Z = (X, Y ) with initial law p ⊗ δ y .
Our first main result is the following probabilistic characterization.
Theorem 2.4 For all ( p, y) ∈ (K ) × R,
 +∞ 
Vn ( p, y) −→ V ( p, y) := max E r e−r t u(πt , Yt )dt , (2.2)
n→∞ (Z t ,πt )t≥0 ∈B( p,y) 0

where B( p, y) ⊂ (D([0, ∞), (K × R) × (K ))) denotes the set of laws of càdlàg
processes (Z t , πt )t∈[0,∞) such that:
• (Z t )t≥0 has law P p,y and is an F (Z ,π ) -Markov process.
(π,Y )
• For all t ≥ 0, for all k ∈ K , πt (k) = P(X t = k|Ft ).
Let us comment briefly this result. We generalize here the idea that the problem the
informed player is facing can be decomposed into two parts: at first he may decide how
information will be used during the whole game, and then maximize his payoff under
this constraint. To apply this method of decomposition, we need to identify precisely
the set B( p, y) of achievable processes of posterior beliefs on X of the less informed
player. The filtration F (π,Y ) represents the information of player 2, which observes
the process Y (a lower bound on information). The condition that Z is F (Z ,π ) -Markov
reflects the fact that player 2 cannot learn any information on the process X which

123
326 F. Gensbittel

is not known by player 1 (an upper bound on information) and the second condition
simply says that π represents the process of beliefs of player 2 on X t . Maximizers
of the right-hand side of Eq. (2.2) represent optimal processes of revelation for the
informed player and induce asymptotically optimal strategies for the informed player
in the sequence of discretized games [see the proof of Theorem 2.4].
We now turn to the second characterization. Define b(y) := (b(k, y))k∈K ∈ R K
and for all k ∈ K and t ≥ 0, define the optional projection1 :

χt (k) := P X t = k|FtY,+ .

Using Theorem 9.1 in Lipster and Shiryaev (2001) (see also the Note p. 360 about
the Markov property), the process ψ := (χ , Y ) with values in R K × R is a diffusion
process satisfying the following stochastic differential equation:
 t  t
∀t ≥ 0, ψt = ψ0 + c(ψs )ds + κ(ψs )d W̄s , (2.3)
0 0

where W̄ is a standard F Y,+ -Brownian motion and the vectors c( p, y) and κ( p, y) in

R K +1 = R K × R are defined by

c( p, y) := Rp,  p, b(y) ,
T
 
pk
κ( p, y) := (b(k, y) − b(y), p , σ (y)),
σ (y) k∈K

where TR denotes the transpose of the matrix R and probabilities are seen as col-
umn vectors. We deduce from standard properties of diffusion processes that for any
function f ∈ C 2 (R K +1 ) with polynomial growth (say) and for all 0 ≤ s ≤ t:
 t 
E[ f (ψt )|Fsψ ] = f (ψs ) + E A( f )(ψu )du|Fsψ
s

where A( f ) is the differential operator defined by (using the notation z = ( p, y))

1
A f (z) = D f (z), c(z) + κ(z), D 2 f (z)κ(z).
2
In order to state our second main result, we need to define precisely the notion of
weak solution we will use. Let p ∈ (K ), we define the tangent space at p by
 
T(K ) ( p) := x ∈ R K | ∃ε > 0, p + εx, p − εx ∈ (K ) .

1 In all the proofs, we consider only natural or right-continuous filtrations, but we adopt the same convention
as in Jacod and Shiryaev (2003) and do not complete the filtrations to avoid complex or ambiguous notations.
Note that optional projections of càdlàg processes are well-defined and have almost surely càdlàg paths
[see appendix 1 in Dellacherie and Meyer (1982)].

123
Continuous-time limit of dynamic games... 327

Let S m denote the set of symmetric matrices of size m. For S ∈ S K and p ∈ (K ),
we define
 
x, Sx
λmax ( p, S) := max | x ∈ T(K ) ( p) \ {0}
x, x

and by convention λmax ( p, X ) = −∞ whenever T(K ) ( p) = {0}.

Theorem 2.5 V is the unique continuous viscosity solution of
 
min r V + H (z, DV (z), D 2 V (z)); −λmax p, D 2p V (z) = 0 (2.4)

where for all (z, ξ, S) ∈ ((K ) × R) × R K +1 × S K +1 :

1
H (z, ξ, S) := −ξ, c(z) − κ(z), Sκ(z) − r u(z),
2

and where DV, D 2 V denote the gradient and the Hessian matrix of V and D 2p V (z)
the Hessian matrix of the function V with respect to the variable p.
Let us recall the definitions of sub and super-solutions.
Definition 2.6 We say that a bounded lower semi-continuous function f is a (viscos-
ity) supersolution of the Eq. (2.4) on (K ) × R if for any test function φ, C 2 in a
neighborhood of (K ) × R (in R K × R) such that φ ≤ f on (K ) × R with equality
in ( p, y) ∈ (K ) × R, we have

λmax p, D 2p φ( p, y) ≤ 0 and r φ( p, y) − A(φ)( p, y) − r u( p, y) ≥ 0.

We say that a bounded upper semi-continuous function f is a (viscosity) subsolution of

the Eq. (2.4) on (K )×R if for any test function φ, C 2 in a neighborhood of (K )×R
(in R K × R) such that φ ≥ f on (K ) × R with equality in ( p, y) ∈ (K ) × R, we
have

λmax ( p, D 2p φ( p, y)) < 0 ⇒ r φ( p, y) − A(φ)( p, y) − r u( p, y) ≤ 0.

The proof of Theorem 2.5 is based on Theorem 2.4 and on dynamic programming.

2.1 Possible extensions and open problems

We list below miscellaneous remarks.

• In comparison to Cardaliaguet et al. (2013), in the statement of Theorem 2.4, we
maximize over a set of joint distributions (Z , π ) rather than on the set of induced
distributions for (π, Y ), which are the only relevant variables for the computation
of the objective functional. The latter set of distributions is exactly the set of joint

123
328 F. Gensbittel

laws of càdlàg processes (π, Y ) such that for all bounded continuous function φ
on (K ) × R which are convex with respect to the first variable, we have:
 
∀ 0 ≤ s ≤ t, E φ(πt , Yt )|Fs(π,Y ) ≥ Q t−s (φ)(πs , Ys ),

where Q is the semi-group of the diffusion process ψ. We do not prove this claim
but it follows quite easily from Strassen’s Theorem and the same techniques used
in Lemma 4 in Cardaliaguet et al. (2013) and Lemma 5.11 in [18]. However, such
a proof would not be constructive (due to Strassen’s theorem) and therefore, we do
not think that this result would be more interesting stated this way. Indeed, in order
to construct asymptotically optimal strategies following the proof of Theorem 2.4,
player 1 has to compute the joint law of (Z , π ) anyway (precisely the conditional
law of π given Z at times q/n for q ≥ 0).
• One may generalize all the present results for the lower value functions to the
case of infinite actions spaces I, J (even if the value u does not exist) by adapting
the method developed in Gensbittel (2015). Note that the proof of the same kind
of results for the upper value functions may rely on different tools as shown in
Gensbittel (2015), and that the extension of these results in the present model
remains an open question.
• It can be shown directly (with classical arguments) that the functions Vn and V
are continuous. However, this does not simplify nor shorten the proofs.
• It is reasonable to think that Theorem 2.4 can be extended to the case of a more
general Feller processes (X, Y ), at least for diffusions with smooth coefficients.
However, such an extension leads to the following open question: is it possible to
write an Hamilton–Jacobi equation in the case of a diffusion process Z = (X, Y )
taking values in Rm ×R p ? Note that such an equation would be stated in an infinite
dimensional space of probability measures.
• It would be interesting to try to find explicit solutions for simple examples with two
states for X and with simple payoff functions and simple diffusion parameters for
Y . Such an analysis and the comparison with the examples studied in Cardaliaguet
et al. (2013) is left for future research.

3 Proof of Theorem 2.4

Recall the definition of conditional independence.

Definition 3.1 Let (, A, P) a probability space and F, G, H three sub σ -fields of
A. We say that F and G are conditionally independent given H if

∀F ∈ F, ∀G ∈ G, P(F ∩ G|H) = P(F|H)P(G|H).


This relation is denoted F H G and the definition extends to random variables by
considering the σ -fields they generate.
The next definition is related to the characterization of the Markov property in terms
of conditional independence and will be useful in the sequel.

123
Continuous-time limit of dynamic games... 329

Definition 3.2 Given two random processes (Aq , Bq )q≥0 (with values in some Polish
spaces) defined on (, A, P). We say that (Aq )q≥0 is non-anticipative with respect to
(Bq )q≥0 if

∀q ≥ 0, (A0 , . . . , Aq ) (Bm )m≥0 .
B0 ,...,Bq

The next result is a classical property of conditional independence and its proof is
postponed to the appendix.
Lemma 3.3 Given two random processes (Aq , Bq )q≥0 (with values in some Polish
spaces), the process (Aq )q≥0 is non-anticipative with respect to (Bq )q≥0 if and only
if there exists (on a possibly enlarged probability space) a sequence of independent
random variables (ξq )q≥0 uniformly distributed on [0, 1] and independent of (Bq )q≥0 ,
and a sequence of measurable functions f q (defined on appropriate spaces) such that
for all q ≥ 0

Aq = f q (Bm , ξm , m ≤ q).

The proof of Theorem 2.4 is divided in two steps and relies on the technical Lemma
3.8, whose proof is postponed to the next subsection.
Step 1: We prove that lim inf Vn ≥ V .
Let σ ∗ ( p, y) and τ ∗ ( p, y) be measurable selections of optimal strategies for player 1
and 2 respectively, in the game ( p, y) with value u( p, y).
We start with a continuous-time process (Z t , πt )t≥0 in B( p, y). We consider the
discrete-time process (Z q , π q )q≥0 . Using the Markov property at times qn , we deduce
n n
that (π q )q≥0 is non-anticipative with respect to (Z q )q≥0 . We now construct a strategy
n n
σ in G n ( p, y) depending on the process (Z , π ). Using the conditional independence
property (see Lemma 3.3), there exists a sequence (ξq )q≥0 of independent random vari-
ables uniformly distributed on [0, 1] and independent from (Z q )q≥0 , and a sequence
n
of measurable functions ( f q )q≥0 such that

π q = f q ((Z mn , ξm )m≤q ) for all q ≥ 0.

n

We define player 1’s strategy σ as follows:

σ q Z 0 , . . . , Z q , ξ0 , . . . , ξq := σ ∗ π q , Y q .
n n n

This does not define formally a behavior strategy but these transition probabilities
induce a joint law for (Z q , i q )q≥0 which can always be disintegrated in a behavior
n
strategy (that does not depend on player 2’s actions) since the induced process (i q )q≥0
is by construction non-anticipative with respect to (Z q )q≥0 (using again Lemma 3.3).
n
By taking the conditional expectation given (Y  , π  , i  , j )=0,...,q , the payoff at stage
n n
q against any strategy τ is such that:

123
330 F. Gensbittel

 
  
En, p,σ ,τ g X q , Y q , i q , jq = En, p,σ ,τ π q (k)g k, Y q , i q , jq
n n n n
k∈K
 
≥ En, p,σ ,τ u π q , Y q .
n n

Therefore, σ is such that

  
Vn ( p, y) ≥ inf γn ( p, y, σ , τ ) ≥ λn (1 − λn )q E u π q , Y q .
τ n n
q≥0

Define (π̃ n , Z̃ n ) as the piecewise-constant process equal to (Z , π ) at times qn for

q ≥ 0. Then (π̃ n , Z̃ n ) converges in probability to (Z , π ) [see e.g. Lemma VI.6.37 in
Jacod and Shiryaev (2003)] and therefore

    ∞ 
−r t
λn (1 − λn ) E u π q , Y q
q
= E re u π̃tn , Ỹtn dt
n n
q≥0 0
 ∞ 
−→ E r e−r t u (πt , Yt ) dt
n→∞ 0

As (Z , π ) ∈ B( p, y) was chosen arbitrarily, we deduce that:

lim inf Vn ( p, y) ≥ V ( p, y)
n→∞

Step 2: We prove that lim sup Vn ≤ V .

Let us fix ( p, y) and let (εn )n≥1 a positive sequence going to zero. For all n ≥ 1, let
σ n be an εn -optimal behavior strategy for player 1 in G n ( p, y). We will construct a
strategy τ n for player 2 by induction such that for all q ≥ 0 the expected payoff at
round q is not greater than
 
En, p,y,σ n ,τ n u p̂q , Y q + C| pq − p̂q |1 , (3.1)
n

for some constant C independent of n, where |.|1 denotes the 1 -norm and where
for all q ≥ 0, p̂q and pq denote respectively the conditional laws of X q given the
n
information of player 2 before and after playing round q. Precisely, for all k ∈ K :

p̂q (k) := P(n, p,y,σ n ,τ n ) X q = k | Y0 , i 0 , j0 , . . . , Y q−1 , i q−1 , jq−1 , Y q ,

n n n

pq (k) := P(n, p,y,σ n ,τ n ) X q = k | Y0 , i 0 , j0 , . . . , Y q , i q , jq .

n n

Note that the computation of p̂q does not depend on τqn . We can therefore define
by induction τqn := τ ∗ ( p̂q , Y q ). Then, inequality (3.1) follows directly from Lemmas
n
V.2.5 and V.2.6 in Mertens et al. (1994). We now suppress the indices (n, p, y, σ n , τ n )

123
Continuous-time limit of dynamic games... 331

from the probabilities and expectations. Using that u is Lipschitz with respect to p,
we have
   
E u p̂q , Y q + C| pq − p̂q |1 ≤ E u pq , Y q + 2C| pq − p̂q |1
n n

Define also:
1T
p̃q+1 := P X q+1 = k|Y0 , i 0 , j0 , . . . , Y q , i q , jq = e n R
pq (k).
n n

Note that for all q ≥ 0, the sequence ( p̃q+1 , p̂q+1 , pq+1 ) is a martingale so that using
Jensen’s inequality.
 
E ( p̃q+1 )2 ] ≤ E[( p̂q+1 )2 .

On the other hand, using the previous equality, we can choose the constant C so that
almost surely
C
∀q ≥ 0, | p̃q+1 − pq | ≤ .
n
Mimicking the proof of Cardaliaguet et al. (2013), we have
⎡ ⎤
   
E⎣ λn (1 − λn )q | pq − p̂q |1 ⎦ = λn (1 − λn )q E | pq (k) − p̂q (k)|
q≥0 k∈K q≥0
⎛ ⎞1/2
   
≤ ⎝ λn (1 − λn )q E | pq (k) − p̂q (k)|2 ⎠
k∈K q≥0
⎛ ⎞1/2
   
= ⎝ λn (1 − λn )q E ( pq (k))2 − ( p̂q (k))2 ⎠
k∈K q≥0

which is also equal to

⎛
  
⎝ λn (1 − λn )q E ( pq (k))2 − ( p̃q+1 (k))2
k∈K q≥0
⎞1/2

+( p̃q+1 (k))2 − ( p̂q+1 (k))2 + ( p̂q+1 (k))2 − ( p̂q (k))2 ⎠

and therefore is bounded from above by

⎛ ⎞1/2
    2C  
⎝ ⎠ 2C 1/2
λn (1 − λn ) E ( p̂q+1 (k)) − ( p̂q (k)) +
q 2 2
≤ K λn +
n n
k∈K q≥0

123
332 F. Gensbittel

We proved that:

    1/2
2C
Vn ( p, y) ≤ λn (1 − λn ) E u π q , Y q
q
+ K λn + + εn
n n n
q≥0

In order to conclude the proof, we consider the continuous-time process ( Z̃ n , π̃ n )

which is piecewise-constant and equal to (Z q , pq ) at times q/n. Let us at first extract
n
a subsequence of Vn ( p, y) which converges to lim sup Vn ( p, y). Then, using Lemma
3.8, there exists a further subsequence of ( Z̃ n , π̃ n ) which converges in law to some
process (Z , π ) in B( p, y). We have therefore along this subsequence
    ∞ 
−r t
λn (1 − λn ) E u π , Y
q q q = E re u(π̃tn , Ỹtn )dt
n n
q≥0 0
 ∞ 
−r t
−→ E re u(πt , Yt )dt ,
0

so that
 ∞ 
−r t
lim sup Vn ( p, y) ≤ E re u(πt , Yt )dt ≤ V ( p, y).
n→∞ 0

3.1 A technical Lemma

In reference to the paper of Meyer and Zheng (1984), we will denote M Z the following
topology on the set of càdlàg paths.
Notation 3.4 For a separable metric space (E, d), the M Z -topology on the set
D([0, ∞), E) of càdlàg functions is the topology of convergence in measure when
[0, ∞) is endowed with the measure e−x d x. The associated weak topology over the
set (D([0, ∞), E)) when D([0, ∞), E) is endowed with the M Z -topology will be
denoted L(M Z ).
Remark 3.5 In contrast to the Skorokhod topology (Sk hereafter), if E = F × F  is
a product of separable metric spaces, the M Z topology is a product topology, i.e. (as
topological spaces)

(D([0, ∞), F × F  ), M Z ) = (D([0, ∞), F), M Z ) × (D([0, ∞), F  ), M Z ).

The following remark will be used in the proofs.

Remark 3.6 If E is a Polish space, the space (D([0, ∞), E), M Z ) is a separable metric
space which is not topologically complete. However, its Borel σ -algebra is the same as
the one generated by the Sk topology and its topology is weaker than the Sk topology
for which the space is Polish, implying that all the probability measures are M Z -tight.
Therefore, all the results about disintegration and measurable selection usually stated
for Polish spaces and which depend only on the Borel structure apply to this space.

123
Continuous-time limit of dynamic games... 333

Recall that the transition probabilities of Z are denoted (Pt )t≥0 , i.e. for any bounded
measurable function φ on K × R, we have

Pt (φ)(z) := Ez [φ(Z t )] = φd Pt (z),
Z

and that P is a Feller semi-group implying that (z, t) → Pt (φ)(z) is continuous for
any bounded continuous function φ.

Notation 3.7 Given a process (Z t )t∈[0,∞) of law P p,y , we define the process
(!
Z tn )t∈[0,∞) ∈ D([0, ∞), K × R) by

∀t ≥ 0, !
Z tn := Z nt
n

where a denotes the greatest integer lower or equal to a.

Lemma 3.8 Let ( p, y) be given, and let us consider a sequence of càdlàg processes
(Z n , π n ) that are piecewise constant on the partition {[ qn , q+1
n )}q≥0 and such that

• Z n has the same law as !

Z n (see the above notation).
• (π )q≥0 is non-anticipative with respect to (Z q )q≥0 .
q
n n
(π n ,Y n )
• For all t ≥ 0, for all k ∈ K , πtn (k) = P(X tn = k|Ft ).
Then, the sequence (Z n , π n ) admits an L(M Z )-convergent subsequence and all the
limit points belong to B( p, y).

Proof Let Qn denote a sequence of laws of processes (Z tn , πtn )t∈[0,∞) . It follows from
Proposition VI.6.37 in Jacod and Shiryaev (2003) that Z n L(M Z )-converges to Z of
law P p,y . On the other hand, Theorem 4 in Meyer and Zheng (1984) together with
a diagonal extraction implies that the set of possible laws for (Z tn , πtn )t≥0 is M Z -
relatively sequentially compact, and we may extract some convergent subsequence2 .
Let us now prove that the limit belongs to B( p, y). Assume (without loss of gen-
erality) that the sequence of processes (Z tn , πtn )t≥0 L(M Z )-converges to (Z t , πt )t≥0 .
Note at first that the law of (Z t )t≥0 is P p,y since the projection of the trajectories on
the first coordinate is continuous (see Remark 3.5).
Using Skorokhod’s representation Theorem for separable metric spaces [see The-
orem 11.7.31 in Dudley (2002)], we can assume that the processes are defined on
MZ
the same probability space and that (Z tn , πtn )t≥0 → (Z t , πt )t≥0 almost surely. Up
to extracting a subsequence,
 we can also assume that there exists a subset I of full
measure in [0, ∞) (i.e. I e−x d x = 1) such that for all t ∈ I , (Z tn , πtn ) → (Z t , πt )
almost surely.

2 Precisely, for all T > 0 we may first apply this result to each coordinate of the processes (Z n , π n )
t∧T t∧T t≥0 .
Then, since convergent sequences are tight [see Theorem 11.5.3 in Dudley (2002) and remark 3.6], we apply
Lemma 5.3 to deduce that the set of laws {Qn , n ≥ 1} is tight. Applying the direct part of Prohorov’s theorem,
which is valid for separable metric spaces, we may extract some convergent subsequence.

123
334 F. Gensbittel

We now prove that for all t ≥ 0 and all k ∈ K

(Y,π )
πt (k) = P X t = k|Ft .

For any t ∈ I , any finite family (t1 , . . . , tr ) in I ∩ [0, t] and any bounded continuous
function φ defined on (R × (K ))r , we have
" # " #
E πtn (k) − 1 X tn =k φ Ytn1 , πtn1 , . . . , Ytnr , πtnr = 0.

It follows by bounded convergence that

 
E (πt − 1 X t =k )φ(Yt1 , πt1 , . . . , Ytr , πtr ) = 0.

We deduce that

(Y,π )
πt (k) = P X t = k|Ft .

Given an arbitrary t, we take a decreasing sequence in I with limit t and applying

Lemma 5.6 (see apendix), we obtain:

(Y,π ),+
πt (k) = P X t = k|Ft ,

which implies the result using the tower property of conditional expectations.
It remains to prove the Markov property. Let t1 ≤ . . . ≤ tm ≤ s ≤ t in I , and φ, φ 
some bounded continuous functions defined on ((K × R) × (K ))m and K × R, we
claim that

E[φ  (Z t )φ(Z t1 , πt1 , . . . , Z tm , πtm )] = E[Pt−s (φ  )(Z s )φ(Z t1 , πt1 , . . . , Z tm , πtm )].

Indeed, for all n, we have

 
E φ  (Z tn )φ(Z tn1 , πtn1 , . . . , Z tnm , πtnm )
 
= E P nt−ns (φ  )(Z sn )φ(Z tn1 , πtn1 , . . . , Z tnm , πtnm ) ,
n

and the conclusion follows by bounded convergence. The property extends to arbitrary
t1 ≤ · · · ≤ tm ≤ s ≤ t by taking decreasing sequences in I and we conclude as above
that Z is an F (Z ,π ) Markov process. 


Let us end this section with a second technical lemma whose proof is similar to
Lemma 3.8.

Lemma 3.9 The set-valued map ( p, y) → (B( p, y), L(M Z )) has a closed graph
with compact values.

123
Continuous-time limit of dynamic games... 335

Proof Since Z is a Feller process, the map ( p, y) → P p,y is L(Sk)-continuous [hence

L(M Z )-continuous, see e.g. Ethier and Kurtz (1986)]. We omit the rest of the proof as
it follows exactly from the same arguments as Lemma 3.8 with obvious modifications.



4 The variational characterization

We state at first some properties of the function V .

Proposition 4.1 V is upper-continuous and for all y ∈ R, p → V ( p, y) is concave

on (K ).

Proof That V is upper semi-continuous follows directly from Lemma 3.9.

Concavity follows from the same method as the well-known splitting Lemma [see
e.g. Chapter V.1 in Mertens et al. (1994)]. Given y ∈ R, p1 , p2 ∈ (K ) and λ ∈ [0, 1],
P1 ∈ B( p1 , y) and P2 ∈ B( p2 , y), let us construct Pλ ∈ B(λp1 + (1 − λ) p2 , y) as
follows. Assume that (Z 1 , π 1 ) and (Z 2 , π 2 ) are independent and of respective laws
P1 and P2 . Let ξ be a random variable independent of (Z 1 , π 1 ) and (Z 2 , π 2 ) and such
that P(ξ = 1) = λ and P(ξ = 2) = 1 − λ. Define (Z , π ) as the process equal to
(Z i , π i ) on {ξ = i}. It follows easily by conditioning on ξ that

 ∞   ∞ 
E r e−r t u(πt , Yt )dt = λE r e−r t u πt1 , Yt1 dt
0 0
 ∞ 
+(1 − λ)E r e−r t u πt2 , Yt2 dt .
0

If we assume that (Z , π ) has a law Pλ ∈ B(λp1 + (1 − λ) p2 , y), then for any ε > 0,
we can choose P1 and P2 as ε-optimal probabilities so that

 ∞ 
V (λp1 + (1 − λ) p2 , y) ≥ E r e−r t u(πt , Yt )dt
0
 ∞ 
= λE r e−r t u πt1 , Yt1 dt
0
 ∞ 
+ (1 − λ)E r e−r t u πt2 , Yt2 dt
0
≥ λV ( p1 , y) + (1 − λ)V ( p2 , y) − ε,

and this proves that V is concave with respect to p as ε can be chosen arbitrarily small.
In order to conclude, it remains therefore to prove that (Z , π ) has a law Pλ ∈
B(λp1 + (1 − λ) p2 , y). Note at first that (Z t )t≥0 has law Pλp1 +(1−λ) p2 ,y by construc-
tion. Moreover, Ftπ,Y is included in σ (ξ ) ∨ Ftπ
1 ,Y 1
∨ Ftπ
2 ,Y 2
. Using independence,
we have therefore for all k ∈ K and all t ≥ 0:

123
336 F. Gensbittel

1 ,π 1 2 ,π 2
P X t = k|FtY , FtY ,ξ
1 ,π 1 2 ,π 2 1 ,π 1 2 ,π 2
= P X t1 = k|FtY , FtY , ξ 1ξ =1 + P X t2 |FtY , FtY , ξ 1ξ =2
Y 1 ,π 1 Y 2 ,π 2
= P X t1 = k|Ft 1ξ =1 + P X t2 = k|Ft 1ξ =2
= πt1 (k)1ξ =1 + πt2 (k)1ξ =2 = πt (k),

and using the tower property of conditional expectations, we deduce that

(π,Y )
πt (k) = P X t = k|Ft .

To prove the Markov property, let s ≥ t and φ some bounded continuous function on
(K ) × R. As above, we have:
    
E φ(Z s )|FtZ ,π , FtZ ,π , ξ = 1ξ =i E φ(Z si )|FtZ ,π , FtZ ,π , ξ
1 1 2 2 1 1 2 2

i
 i
= 1ξ =i E[φ(Z si )|FtZ ]
i

= 1ξ =i Ps−t (φ)(Z ti ) = Ps−t (φ)(Z t ).
i

The conclusion follows by using the tower property of conditional expectation with
the intermediate σ -field FtZ ,π . 


4.1 Dynamic programming

Notation 4.2 In the following, we will use the notation E p,y to denote the expectation
associated to the diffusion process ψ starting at time 0 with initial position ψ0 =
( p, y).

We now state a dynamic programming principle which will be the key element for
the proof of Theorem 2.5.

Proposition 4.3 For all ( p, y) ∈ (K ) × R, for all h ≥ 0, we have

 h 
−r t −r h
V ( p, y) = max E re u(πt , Yt )dt + e V (πh , Yh ) . (4.1)
(π,Y )∈B( p,y) 0

As a consequence,
 h 
V ( p, y) ≥ E p,y r e−r t u(ψt )dt + e−r h V (ψt ) . (4.2)
0

123
Continuous-time limit of dynamic games... 337

Moreover, if (π, Y ) is an optimal process for V ( p, y), then for all h ≥ 0:

 h 
V ( p, y) = E r e−r t u(πt , Yt )dt + e−r h V (πh , Yh ) . (4.3)
0

Proof We prove at first that the maximum is reached in the right-hand side of (4.1).
Let us define the M Z -topology on the set D([0, h], K ×R×(K )) as the convergence
in Lebesgue measure of the trajectories together with the convergence of the value of
the process at time h. Note that this topology coincides (up to an identification) with
the induced topology on the subset of D([0, ∞), K × R × (K )) made by trajectories
that are constant on [h, ∞). Using this identification and adapting the arguments of
Lemma 3.8, the set of laws of the restrictions of the processes (Z , π ) ∈ B( p, y) to
the time interval [0, h] is L(M Z )-sequentially relatively compact in (D([0, h], K ×
R × (K ))). The existence of a maximum follows since the map
 h 
−r t −r h
P ∈ (D([0, h], K × R × (K ))) −→ EP re u(πt , Yt )dt + e V (πh , Yh ) ,
0

is L(M Z ) upper-semi-continuous.
We now prove (4.1). We begin with a measurableselection argument. 
∞
The function P ∈ B( p, y) → J (P) := E 0 r e−r t u(πt , Yt )dt is L(M Z )-
continuous, and the set-valued map ( p, y) → B( p, y) is L(M Z ) upper-semi-
continuous. We deduce that the subset O of the space (K ) × R × (D([0, ∞), (K ×
R) × (K ))) defined by

O := {( p, y, P)|P ∈ B( p, y), J (P) ≥ V ( p, y)}

is Borel-measurable (see Remark 3.6). Moreover, Lemma 3.8 implies that for any
( p, y), there exists some P such that ( p, y, P) ∈ O. It follows therefore from Von
Neumann’s selection Theorem [see e.g. Proposition 7.49 in Bertsekas and Shreve
(1978)] that there exists an optimal universally-measurable selection φ from (K )×R
to B( p, y) such that for all ( p, y) ∈ (K ) × R, φ( p, y) ∈ O.
Let (Z , π ) ∈ B( p, y) and h ≥ 0 and let μh denote the joint law of (πh , Yh ). By
construction, φ is μh -almost surely equal to a Borel map φ̃. Using Lemma 5.5, we can
construct a process (π̃s )s≥h (on some extension of the probability space) such that the
conditional law of (Z h+s , π̃h+s )s≥0 given Fh(Y,π ) is precisely φ̃(πh , Yh ) and such that
there exists a variable U , uniformly distributed on [0, 1] and independent of (Z , π ),
and a measurable map  such that

(π̃s )s≥h = ((Z s )s≥h , (πh , Yh ), ξ ). (4.4)

Let us consider the process (Z , π̂ ) where π̂ is equal to π on [0, h) and to π̃ on [h, ∞).
Using the preceding construction, if we assume that the process (Z , π̂ ) has a law in
B( p, y), we deduce that:

123
338 F. Gensbittel

 ∞ 
−r t
V ( p, y) ≥ E re u(π̂t , Yt )dt
0
 h    ∞ 
−r t −r h −r t (Y,π )
=E re u(πt , Yt )dt + e E E re u(π̃h+t , Yh+t )dt|Fh
0 0
 h 
≥E r e−r t u(πt , Yt )dt + e−r h E [V (πh , Yh )]
0

which would prove that

 h 
V ( p, y) ≥ max E r e−r t u(πt , Yt )dt + e−r h V (πh , Yh ) . (4.5)
(Z ,π )∈B( p,y) 0

To conclude the proof of (4.5), we now check that the process (Z , π̂ ) has a law in
B( p, y).
At first, note that (Z t )t≥0 is a Markov process with initial law p⊗δ y by construction.
(Y,π̂)
Let us prove that for all t ≥ 0, π̂t (k) = P(X t = k|Ft ]. The result is obvious by
construction for t < h. For t ≥ h, let us consider two finite families (t1 , . . . , tm ) in
[h, t] and (t1 , . . . , t ) in [0, h) and two bounded continuous function φ, φ  defined on
((K ) × R)m and ((K ) × R) . Then:

 
E 1 X t =k φ(π̂t1 , Yt1 , . . . , π̂tm , Ytm )φ  (π̂t1 , Yt1 , . . . , π̂t , Yt )
   
= E E 1 X t =k φ(π̃t1 , Yt1 , . . . , π̃tm , Ytm )|Fh(π,Y ) φ  (πt1 , Yt1 , . . . , πt , Yt )
 
= E π̃t (k)φ(π̃t1 , Yt1 , . . . , π̃tm , Ytm )φ  (πt1 , Yt1 , . . . , πt , Yt )
 
= E π̂t (k)φ(π̂t1 , Yt1 , . . . , π̂tm , Ytm )φ  (π̂t1 , Yt1 , . . . , π̂t , Yt ) .

This property extends to bounded measurable functions of any finite family (ti ) in
(Y,π̂)
[0, t] by monotone class and we deduce that π̂t (k) = P(X t = k|Ft ).
We now prove the Markov property. For t ≥ 0, we have to prove that


(Z s )s≥t (π̂s )s∈[0,t] . (4.6)
(Z s )s∈[0,t]

The case t < h follows directly by construction. Let us consider the case t ≥ h.
At first, since the conditional law of (Z s , π̃s )s≥h given (πh , Yh ) belongs to B(πh , Yh ),
we have:


(π̃s )s∈[h,t] (Z s )s≥t . (4.7)
(Z s )s∈[h,t] ,(πh ,Yh )

123
Continuous-time limit of dynamic games... 339

Using (4.4) and that Z is an F Z ,π -Markov process, we also have


(π̃s )s∈[h,t] (Z s , πs )s∈[0,h] , (4.8)
(Z s )∈[h,t] ,(πh ,Yh )

(π̃s )s∈[h,t] (Z s , πs )s∈[0,h] . (4.9)
(Z s )s≥t ,(Z s )s∈[h,t] ,(πh ,Yh )

From the characterization of conditional independence in terms of conditional laws

recalled in Lemma 5.2, properties (4.7), (4.8) and (4.9) together imply that:

(π̃s )s∈[h,t] (Z s )s≥t (4.10)
(Z s ,πs )s∈[0,h] ,(Z s )s∈[h,t] ,(πh ,Yh )

Using again the fact that Z is an F Z ,π -Markov process, we also have


(Z s )s≥t (πs )s∈[0,h] (4.11)
(Z s )s∈[0,t]

Finally (4.10) and (4.11) imply

 " #
(Z s )s≥t (πs )s∈[0,h] , (π̃s )s∈[h,t] , (4.12)
(Z s )s∈[0,t]

" #
from which we deduce (4.6) since (π̂s )s∈[0,t] is a function of (πs )s∈[0,h] , (π̃s )s∈[h,t] .
This concludes the proof of (4.5).
In order to conclude the proof of (4.1), we now prove the reverse inequality.
Let (Z , π ) be an admissible process and h > 0. We check easily that the conditional
(π,Y )
law of (Z h+s , πh+s )s≥0 given Fh belongs almost surely to B(πh , Yh ). It follows
that
 ∞   h 
E r e−r t u(πt , Yt )dt = E r e−r t u(πt , Yt )dt
0 0
  ∞ 
(π,Y )
+ e−r h E E r e−r t u(πh+t , Yh+t )dt|Fh
0
 h 
≤E r e−r t u(πt , Yt )dt + e−r h E[V (πh , Yh )].
0

The conclusion follows by taking the supremum over all admissible processes (Z , π ).
The inequality (4.2) follows directly from (4.1). Precisely, given a process Z with initial
law p ⊗ δ y , define π by πt (k) := χt (k) = P[X t = k|FtY,+ ] (optional projection).
As explained before, (Z , π ) has a law in B( p, y) and (π, Y ) is a diffusion process of
semi-group Q.

123
340 F. Gensbittel

We finally prove (4.3). If (Z , π ) ∈ B( p, y) is an optimal process (existence of a

maximum follows from Lemma 3.9), then using the same arguments as above, we
have for all h ≥ 0:
 ∞   h 
−r t −r t
V ( p, y) = E re u(πt , Yt )dt ≤ E re u(πt , Yt )dt
0 0
−r h
+e E[V (πh , Yh )],

and the conclusion follows from (4.1). 



4.2 Proof of Theorem 2.5

Proof of theorem 2.5 The proof is divided in two parts showing respectively that the
lower semicontinuous envelope V∗ of V is subsolution and that V is supersolution
of (2.4). Uniqueness and continuity will follow from the comparison result (Theorem
5.8) whose proof is postponed to the appendix.
Part 1: We prove that the lower semicontinuous envelope of V , denoted V∗ , is a
supersolution of (2.4).
Let φ be any smooth test function such that φ ≤ V∗ with equality in ( p, y) ∈ (K )×R.
As V∗ is bounded, we may assume without loss of generality that φ is bounded.
Consider a sequence ( pn , yn ) → ( p, y) such that V ( pn , yn ) → V∗ ( p, y). From (4.2),
we deduce that
 h 
V ( pn , yn ) − e−r h E pn ,yn [φ(ψh )] − E pn ,yn r e−r s u(ψs )ds ≥ 0.
0

Letting n → ∞, we obtain that (recall that ψ is a Feller process):

 h 
−r h −r s
φ( p, y) − e E p,y [φ(ψh )] − E p,y re u(ψs )ds ≥ 0.
0

Applying Itô’s formula, we have

 h 
E p,y [φ(ψh )] = φ( p, y) + E p,y A(φ)(ψs )ds .
0

Dividing by h and letting then h → 0, it follows from usual arguments that

r φ( p, y) − A(φ)( p, y) − r u( p, y) ≥ 0. (4.13)

Let us prove that V∗ is concave with respect to p. Let y ∈ R and p = λp1 + (1 − λ) p2

for some p, p1 , p2 ∈ (K ) and λ ∈ [0, 1]. Let ( p n , y n ) a sequence converging to
( p, y) such that V ( p n , y n ) → V∗ ( p, y). Then, there exists p1n , p2n ∈ (K ) such that

123
Continuous-time limit of dynamic games... 341

p n = λp1n + (1 − λ) p2n and ( p1n , p2n ) → ( p1 , p2 ) (it is for example a consequence of

Lemma 8.2 in Laraki (2004)). It follows that

V ( p n , y n ) ≥ λV ( p1n , y n ) + (1 − λ)V ( p2n , y n ).

By letting n → ∞ and using the definition of V∗ , we deduce that

V∗ ( p, y) ≥ λV∗ ( p1 , y) + (1 − λ)V∗ ( p2 , y),

which proves that V∗ is concave. We deduce that λmax ( p, D 2 φ p ( p, y)) ≤ 0, and

together with (4.13) this concludes the proof of the supersolution property.
Part 2: We prove that V is subsolution of (2.4).
Let φ be smooth test function such that φ ≥ V with equality at z̄ = ( p̄, ȳ). We have
to prove that if λmax ( p̄, D 2p φ(z̄)) < 0, then r V (z̄) − A(φ)(z̄) − r u(z̄) ≤ 0.
Using Proposition 4.3, let (Z , π ) ∈ B(z̄) be an optimal process, so that for all h ≥ 0,
we have
 h 
V (z̄) = E r e−r s u(πs , Ys )ds + e−r h V (πh , Yh ) . (4.14)
0

Since λmax ( p̄, D 2p φ(z̄)) < 0 [see e.g. the proof of Theorem 3.3. in Cardaliaguet
(2009)], there exists δ > 0 such that for all z = ( p, ȳ) with p ∈ (K ) such that
p − p̄ ∈ T(K ) ( p̄), we have:

V (z) ≤ V (z̄) + D p φ(z̄), p − p̄ − δ| p − p̄|2 .

As E[π0 ] = p̄, the variable π0 belongs almost surely to the smallest face of (K )
containing p̄ so that π0 − p̄ ∈ T(K ) ( p̄). On the other hand, Y0 = ȳ so that (4.14)
with h = 0 implies

V (z̄) = E[V (π0 , ȳ)] ≤ V (z̄) − δE[|π0 − p̄|2 ].

We deduce that π0 = p̄ almost surely.

Recall the definition of the process χ as an optional projection:

∀k ∈ K , ∀s ≥ 0, χs (k) = P X s = k|FsY,+ .

Lemma 5.6 implies that πs (k) = P(X s = k|Fs(π,Y ),+ ), and we deduce that
E[πs |FsY,+ ] = χs using the tower property of conditional expectations. Coming back
to (4.14), Jensen’s inequality implies:
 h 
V (z̄) = E r e−r s u(πs , Ys )ds + e−r h V (πh , Yh )
0
 h 
≤E r e−r s u(πs , Ys )ds + e−r h V (χh , Yh ) .
0

123
342 F. Gensbittel

Since V ≤ φ, we obtain
 h 
−r t −r h
V (z̄) = φ(z̄) ≤ E re u(πs , Ys )ds + e φ(χh , Yh ) .
0

Dividing the above inequality by h, and letting h go to zero, it follows from the usual
arguments (using that πs → π0 when s → 0, and Itô’s formula) that:

r V (z̄) − A(φ)(z̄) − r u(z̄) ≤ 0.




Acknowledgments The author gratefully acknowledges the support of the Agence Nationale de la
Recherche, under grant ANR JEUDY, ANR-10-BLAN 0112. The author is grateful to the editor and to
an anonymous referee for carefully reading this work and making useful remarks.

5 Appendix A: Technical Proofs and auxiliary tools

5.1 Proofs of Lemma 3.3

Let us now recall some properties of conditional independence. As we will manipulate

conditional laws, we introduce a specific notation in order to shorten statements and
proofs.
Notation 5.1 Let E be a Polish space and A be an E-valued random variable defined
on some probability space (, A, P).
• A denotes the law of A.
• Given a σ -field F ⊂ A, A | F denotes a version of the conditional law of A
given F, hence an F-measurable random variable with values in (E) (see e.g.
Bertsekas and Shreve (1978) Proposition 7.26 for this last point).

Lemma 5.2 • Let A, B, C be three random variables (with values in some Polish
space) defined on the same probability space. A is independent of B conditionally
onC if and only if B|C = B|C, A.
• A C B if and only if there exists (on a possibly enlarged probability space) a
random variable ξ uniform on [0, 1] independent of (A, C), and a measurable
function f such that B = f (C, ξ ).

Proof See Proposition 5.6 and 5.13 in Kallenberg (2002). 



Proof of Lemma 3.3 The “if” part is obvious. Let us prove the “only if” part. For
q = 0, this is just Lemma 5.2. However, we need to be more precise on how to construct
this variable. We assume that there exists a family of independent variables (ζ0 , . . . , ζn )
uniformly distributed on [0, 1] and independent of (A0 , B0 , . . . , An , Bn ). Then, the
variable ξ0 given by Lemma 5.2 can be constructed as a function of (A0 , B0 , ζ0 ) (see
the proof of Proposition 5.13 in Kallenberg (2002)). Let us now proceed by induction

123
Continuous-time limit of dynamic games... 343

and assume the above property is true for p ≤ q and that ξ p is measurable with respect
to (A0 , B0 , ζ0 , . . . , A p , B p , ζ p ). Since

(A0 , . . . , Aq+1 ) (B0 , . . . , Bn ),
(B0 ,...,Bq+1 )

we have

B0 , . . . , Bn |B0 , . . . , Bq+1 , A0 , . . . , Aq+1  = B0 , . . . , Bn |B0 , . . . , Bq+1 .

We deduce that

B0 , . . . , Bn |B0 , . . . , Bq+1 , Aq+1  = B0 , . . . , Bn |B0 , .., Bq+1 .

Using now the induction hypothesis and independence, we also have

B0 , . . . , Bn |B0 , . . . , Bq+1 , ξ0 , . . . , ξq  = B0 , . . . , Bn |B0 , . . . , Bq+1 ,

B0 , . . . , Bn |B0 , . . . , Bq+1 , ξ0 , . . . , ξq , Aq+1  = B0 , . . . , Bn |B0 , . . . , Bq+1 , Aq+1 .

Finally, we deduce that Aq+1 (ξ0 ,...,ξq ,B0 ,...,Bq+1 ) (B0 , . . . , Bn ) and the result follows
then by applying Lemma 5.2. 


5.2 Auxiliary tools

The following lemma is classical.

Lemma 5.3 Let E, E  be two separable metric spaces and A,A two tight (resp.
closed, convex) subsets of (E) and (E  ). Then the set P(A, A ) of probabilities on
E × E  having marginals in the sets A and A is itself tight (resp. closed, convex).
Proof Let us prove the tightness property. Let μ ∈ A, ν ∈ A and π ∈ P(μ, ν). By
assumption, for any ε > 0 there is a compact K ε of E, independent of the choice of
μ in A, such that μ(E/K ε ) ≤ ε, and a compact K ε , independent of the choice of ν in
A such that ν(E  /K ε ) ≤ ε. Then for any pair of random variables (U, V ) of law π :

P[(U, V ) ∈
/ K ε × L ε ] ≤ P[U ∈
/ K ε ] + P[V ∈
/ L ε ] ≤ 2ε

The closed and convex properties follow directly from the continuity and linearity of
the application mapping π to its marginals. 

The following theorem is well-known and allows to construct variables with pre-
scribed conditional laws.
Theorem 5.4 (Blackwell-Dubins Blackwell and Dubins (1983)) Let E be a polish
space with (E) the set of Borel probabilities on E,and ([0, 1], B([0, 1]), λ) the unit
interval equipped with Lebesgue’s measure. There exists a measurable mapping

 : (E) × [0, 1] −→ E

123
344 F. Gensbittel

such that for all μ ∈ (E), the law of (μ, U ) is μ where U is the canonical element
in [0, 1].

In the Proof of Proposition 4.3, we use indirectly this result together with a disinte-
gration theorem. Precisely:

Lemma 5.5 Let E, F be Polish spaces, (, A, P) be some probability space, Y be an

E-valued random variable defined on , and F a sub-σ -field of A. Assume that f is
an F-measurable map from  to (E × F) such that the marginal f 1 (ω) ∈ (E) of
f (ω) on the first coordinate is a version of the conditional law of Y given F. Then, (up
to enlarging the probability space, there exists a random variable Z such that f (ω)
is a version of the conditional law of (Y, Z ) given F.

Proof Up to enlarging the probability space, we may assume that there exists some
random variable U uniformly distributed on [0, 1] and independent of (Y, F). One
can define using Theorem 5.4 a variable (Ỹ , Z̃ ) = ( f (ω), U ) having the property
that f 1 (ω) is a version of the conditional law of Ỹ given F. Let g(ω, Ỹ ) be a version
of the conditional law of Z̃ given (F, Ỹ ), it follows easily that Z = (g(ω, Y ), U )
fulfills the required properties. 


The next Lemma is a generalized martingale backward convergence theorem

directly adapted from the corresponding result for classical forward martingales that
can be found in chapter III of Mertens et al. (1994).

Lemma 5.6 Let (X n )n≥0 be an uniformly bounded sequence of real-valued random

variables defined on some probability space (, A, P). Let (Fn )n≥0 be a decreasing
sequence of sub σ -fields of A. Assume that (X n )n≥0 converges almost
$ surely to some
variable X , then (E[X n |Fn ])n≥0 converges almost surely to E[X | n≥0 Fn ].

Proof Define X n+ = supm≥n X m and Yn+ = E[X n+ |Fn ]. The sequence X n+ is non-
increasing with limit X and we have

+ +
Yn+1 = E[X n+1 |Fn+1 ] ≤ E[X n+ |Fn+1 ] = E[Yn+ |Fn+1 ].

Yn+ is therefore a backward sub-martingale and converges almost surely to some

variable Y + [see e.g. Theorem 30 p. 24 in Dellacherie and Meyer (1982)] which is
$ − − −
n≥0 Fn -measurable. Similarly, define X n = inf m≥n X m , and Yn = E[X n |Fn ].
Then Yn is a backward supermartingale which converges almost surely to Y − . To
−

conclude, note that

Yn− ≤ E[X |Fn ] ≤ Yn+ ,

and that E[Yn+ − Yn− ] = E[X n+ − X$

−
n ] → 0 by bounded convergence. Since E[X |Fn ]
converges almost surely$ to E[X | n≥0 Fn ], we deduce that E[X n |Fn ] converges
almost surely to E[X | n≥0 Fn ] as Yn− ≤ E[X n |Fn ] ≤ Yn+ . 


123
Continuous-time limit of dynamic games... 345

5.3 Comparison

In this section we adapt the comparison principle given in Cardaliaguet et al. (2013)
for super solutions and sub solutions of (2.4).

Remark 5.7 Note that the process χ takes values in (K ), and that our assumptions
on b and σ imply that the functions c and κ are Lipschitz continuous and bounded
on (K ) × R. In the following, we will assume without loss of generality that the
functions c and κ are bounded and Lipschitz on the whole space R K +1 (the explicit
formula cannot be used directly since the resulting functions would be unbounded
and only locally Lipschitz). Similarly, we assume that the function u is bounded and
Lipschitz on the whole space R K +1 .

With our assumptions on c and κ, it is well known (see e.g. Crandall et al. (1992),
p. 19) that there exists a constant C (depending on the Lipschitz constants of c, κ, u)
such that for any η > 0, z, z  ∈ (K ) × R, ξ ∈ R K +1 and symmetric matrices
S, S  ∈ S K +1 with
   
S 0 I −I
≤η ,
0 S −I I

we have
|u(z) − u(z  )| ≤ C|z − z  |
|b(z), ξ  − b(z  ), ξ | ≤ C|ξ ||z − z  |,
1 1
− κ(z  ), −S  κ(z  ) ≤ − κ(z), Sκ(z) + Cη|z − z  |2 .
2 2
Let us state the comparison principle.
Theorem 5.8 Let w1 be a subsolution and w2 be a supersolution of (2.4), then w1 ≤
w2 .
The rest of this subsection is devoted to the proof of this result. Let w1 be a subso-
lution and w2 be a supersolution of (2.4) (recall that w1 , w2 are bounded functions).
Our aim is to show that w1 ≤ w2 . We argue by contradiction, and assume that
M := sup {w1 (z) − w2 (z)} > 0 . (5.1)
z∈(K )×R

%
Because of the lack of compactness, let β > 0 and g(y) := (1 + y 2 ). Define
M  := sup {w1 (z) − w2 (z) − 2βg(y)} .
z∈(K )×R

We choose β sufficiently small so that M  > 2Cr1 β > 0 with C1 = κ2∞ + b∞ .
We first regularize the maps w1 and w2 by quadratic sup and inf-convolution respec-
tively. This technique is classical (see Crandall et al. (1992) for details), for δ > 0 and
z ∈ R K +1 we define:

123
346 F. Gensbittel

 
1
w1δ (z) := max w1 (z  ) − |z − z  |2
z  ∈(K )×R 2δ

and
 
1
w2,δ (z) := min w2 (z  ) + |z − z  |2 .
z  ∈(K )×R 2δ

Note that w1δ and w2,δ are defined on the whole space R K +1 and that w1δ is semi-
convex while w2,δ is semiconcave. Moreover, we have the following growth property
(uniformly in y)
lim | p|−1 w1δ ( p, y) = −∞, lim | p|−1 w2,δ ( p, y) = +∞.
| p|→+∞ | p|→+∞

Define (with z i = ( pi , y i )):

 
δ 1 1 1
Mδ := sup w1 (z ) − w2,δ (z ) − β(g(y ) + g(y )) − |z − z | .
2 1 2 2 2
z 1 ,z 2 ∈R K +1 2δ
(5.2)

The following result is classical.

Lemma 5.9 For any δ > 0, the problem (5.2) has at least one maximum point. If
(z δ1 , z δ2 ) is such a maximum point and if (z 1 )δ ∈ (K ) × R and (z 2 )δ ∈ (K ) × R
are such that
1 1
w1δ (z δ1 ) = w1 ((z 1 )δ ) − |z − (z 1 )δ |2 and
2δ δ (5.3)
1
w2,δ (z δ2 ) = w2 ((z 2 )δ ) + |z δ2 − (z 2 )δ |2
2δ

then, as δ → 0, Mδ → M  while

|z δ1 − z δ2 |2 |z 1 − (z 1 )δ |2 |z 2 − (z 2 )δ |2
+ δ + δ → 0.
2δ 2δ 2δ
We first prove that the regularized sub/supersolutions are sub/supersolutions of
sligthly modified equations.
Lemma 5.10 Assume that w1δ has a second order Taylor expansion at a point z. Then

min{r w1 (z) + H (z  , Dw1δ (z), D 2 wδ1 (z)) ; −λmax ( p  , D 2p w1δ (z))} ≤ 0, (5.4)

where z  ∈ (K ) × R is such that w1δ (z) = w1 (z  ) − 2δ

1
|z − z  |2 .
Similarly, if w2,δ has a second order Taylor expansion at a point z, then

r w2 (z) + H (z  , Dw2,δ (z), D 2 w2,δ (z)) ≥ 0, (5.5)

123
Continuous-time limit of dynamic games... 347

where z  ∈ (K ) × R is such that w2,δ (z) = w2 (z  ) + 2δ |z

1
− z  |2 .

Proof We do the proof for w1δ , the second part being similar. Assume that w1δ has a
second order Taylor expansion at a point z̄ and set, for γ > 0 small,

1 γ
φγ (z) := Dw1δ (z̄), z − z̄ + z − z̄, D 2 w1δ (z̄)(z − z̄) + |z − z̄|2 .
2 2

Let z̄  denote a point in (K ) × R such that w1δ (z̄) = w1 (z̄  ) − 2δ

1
|z̄ − z̄  |2 .
Then w1δ − φγ has a maximum at z̄, which implies, by definition of w1δ , that

1 
w1 (z  ) − |z − z|2 ≤ φγ (z) − φγ (z̄) + w1δ (z̄) ∀z ∈ R K +1 , ∀ z  ∈ (K ) × R,
2δ

with an equality for (z, z  ) = (z̄, z̄  ). If we choose z = z  − z̄  + z̄ in the above formula,

we obtain:

1 
w1 (z  ) ≤ φγ (z  − z̄  + z̄) + |z̄ − z̄|2 − φγ (z̄) + w1δ (z̄) ∀z  ∈ (K ) × R,
2δ

with an equality at z  = z̄  . As w1 is a subsolution, we obtain therefore, using the

right-hand side of the above inequality as a test function,
& '
min r w1 (z̄  ) + H (z̄  , Dφγ (z̄), D 2 φγ (z̄)) ; −λmax (z̄  , D 2p φγ (z̄)) ≤ 0. (5.6)

By construction, we have Dφγ (z̄) = Dw1δ (z̄), D 2 φγ (z̄) = D 2 w1δ (z̄) + γ I and
w1 (z̄  ) ≥ w1δ (z̄). The conclusion follows therefore by letting γ → 0. 


In order to use inequality (5.4), we have to produce points at which w1δ is strictly
concave with respect to the first variable. For this reason, as in Cardaliaguet et al.
(2013), we introduce a additional penalization. For σ > 0 and z i = ( pi , y i ) ∈ R K +1 ,
we consider

Mδ,σ := sup w1δ (z 1 ) − w2,δ (z 2 ) − β(g(y 1 ) + g(y 2 ))
(z 1 ,z 2 )∈(R K +1 )2

1 1
+ σ g(| p 1 |) − |z − z 2 |2 .
2δ

One easily checks that there exists a maximizer (ẑ 1 , ẑ 2 ) to the above problem. In
order to use Jensen’s Lemma (Lemma A.3 in Crandall et al. (1992)), we also need this
maximum to be strict. For this we modify the penalization: we set for i = 1, 2:

ξ1 ( p 1 ) = g(| p 1 |) − σ g(| p 1 − p̂ 1 |), ζi (y i ) = −βg(y i ) − σ g(y i − ŷ i ).

123
348 F. Gensbittel

We choose σ > 0 sufficiently small so that ξ1 has a positive second order derivative.
By definition,


Mδ,σ = sup w1δ (z 1 ) − w2,δ (z 2 ) + ζ1 (y 1 ) + ζ2 (y 2 )
(z 1 ,z 2 )∈(R K +1 )2

1 1
+ σ ξ1 (| p 1 |) − |z − z 2 |2 ,
2δ

and the above problem has a strict maximum at (ẑ 1 , ẑ 2 ). As the map (z 1 , z 2 ) →
w1δ (z 1 ) − w2,δ (z 2 ) + ζ1 (y 1 ) + ζ2 (y 2 ) + σ ξ1 ( p 1 ) − 2δ
1 1
|z − z 2 |2 is semiconcave,
Jensen’s Lemma (together with Alexandrov theorem) states that, for any ε > 0, there
is vector aε ∈ (R K +1 )2 with |aε | ≤ ε, such that the problem


Mδ,σ,ε := sup w1δ (z 1 ) − w2,δ (z 2 ) + ζ1 (y 1 ) + ζ2 (y 2 )
z 1 ,z 2 ∈(R K +1 )2

1 1
+ σ ξ1 (| p |) − |z − z | + aε , (z , z ) ,
1 2 2 1 2
2δ

has a maximum point (z δ,σ,ε

1 , z δ,σ,ε
2 ) ∈ (R K +1 )2 at which the maps w1δ and w2,δ have
a second order Taylor expansion. From Lemma 5.10, we have

&
min r w1 (z δ,σ,ε
1
) + H ((z 1 )δ,σ,ε , Dw1δ (z δ,σ,ε
1
), D 2 w1δ (z δ,σ,ε
1
))
1  2 δ 1
'
− λmax ((z )δ,σ,ε , D p w1 (z δ,σ,ε )) ≤ 0, (5.7)

and

r w2 (z δ,σ,ε
2
) + H ((z 2 )δ,σ,ε , Dw2,δ (z δ,σ,ε
2
), D 2 w2,δ (z δ,σ,ε
2
)) ≥ 0, (5.8)

where (z 1 )δ,σ,ε and (z 2 )δ,σ,ε are points in (K ) × R at which one has

1 1
w1δ (z δ,σ,ε
1
) = w1 ((z 1 )δ,σ,ε ) − |z − (z 1 )δ,σ,ε |2 and w2,δ (z δ,σ,ε
2
)
2δ δ,σ,ε
1 2
= w2 ((z 2 )δ,σ,ε ) + |z δ,σ,ε − (z 2 )δ,σ,ε |2 .
2δ

Using the properties of inf and sup-convolutions, we have:

1" 1 # 1" 2 #
Dw1δ (z δ,σ,ε
1
)=− z δ,σ,ε − (z 1 )δ,σ,ε and Dw2,δ (z δ,σ,ε
2
)= z δ,σ,ε − (z 2 )δ,σ,ε .
δ δ
(5.9)

123
Continuous-time limit of dynamic games... 349

By definition of Mδ,σ,ε we have for all (z 1 , z 2 ) ∈ (R K +1 )2 :

w1δ (z 1 ) − w2,δ (z 2 ) + ζ1 (y 1 ) + ζ2 (y 2 ) + σ ξ1 ( p1 )
1
≤ Mδ,σ,ε + |z 1 − z 2 |2 − aε , (z 1 , z 2 ),
2δ

with an equality at (z δ,σ,ε

1 , z δ,σ,ε
2 ). Hence
 
σ Dξ1 ( pδ,σ,ε
1 ) 1 1
Dw1δ (z δ,σ,ε
1
)+ (z − z δ,σ,ε
= 2
) − aε1
−βg (yδ,σ,ε ) − σ g  (yδ,σ,ε
 1 1 − ŷ 1 )δ δ,σ,ε
(5.10)
 
0 1
−Dw2δ (z δ,σ,ε
2
)+ = (z δ,σ,ε
2
− z δ,σ,ε
1
) − aε2
−βg  (yδ,σ,ε
2 ) − σ g  (yδ,σ,ε
2 − ŷ 1 ) δ
(5.11)

while
   
S 0 1 I −I
≤ (5.12)
0 S δ −I I

with
 
σ D 2 ξ1 ( pδ,σ,ε
1 ) 0
S := D 2 w1δ (z δ,σ,ε
1
)+
0 −βg  (yδ,σ,ε
1 ) − σ g  (yδ,σ,ε − ŷ 1 )
 
 0 0
S := −D w2,δ (z δ,σ,ε ) +
2 2
0 −βg  (yδ,σ,ε
2 ) − σ g  (yδ,σ,ε
2 − ŷ 2 )

This implies that S ≤ −S  (see Crandall et al. (1992) p. 19) and therefore

D 2p w1δ (z δ,σ,ε
1
) ≤ D 2p w2,δ (z δ,σ,ε
2
) − σ D 2 ξ1 ( pδ,σ,ε
1
). (5.13)

We now check that λmax ((( p 1 )δ,σ,ε ), D 2p w1δ (z δ,σ,ε

1
)) < 0. Using the definition of w2,δ ,

for all p ∈ R and p ∈ (K ),
K

1
w2,δ ( p, yδ,σ,ε
2
) ≤ w2 ( p  , (y 2 )δ,σ,ε ) + | p − p  |2 + |yδ,σ,ε
2
− (y 2 )δ,σ,ε |2 ,
2δ

with an equality at ( p, p  ) = ( pδ,σ,ε

2 , ( p 2 )δ,σ,ε ). If m ∈ T(K ) (( p 2 )δ,σ,ε ) with |m|
small enough, taking p := pδ,σ,ε2 + m and p  = ( p 2 )δ,σ,ε + m gives

w2,δ ( pδ,σ,ε
2
+ m, yδ,σ,ε
2
) ≤ w2 (( p 2 )δ,σ,ε + m, (y 2 )δ,σ,ε )
1
+ | pδ,σ,ε
2
− ( p 2 )δ,σ,ε |2 + |yδ,σ,ε
2
− (y 2 )δ,σ,ε |2 , (5.14)
2δ

123
350 F. Gensbittel

with equality for m = 0. As w2 is concave with respect to the first variable

[see e.g. Lemma 3.2 in Cardaliaguet (2009)], the above inequality implies that
λmax (( p 2 )δ,σ,ε , D 2p w2,δ (z δ,σ,ε
2
)) ≤ 0. In view of (5.13) we get therefore

λmax (( p 1 )δ,σ,ε , D 2p w1δ (z δ,σ,ε

1
)) ≤ −σ λmin (( p 1 )δ,σ,ε , D 2 ξ1 ( pδ,σ,ε
1
)) < 0,

because D 2 ξ1 > 0 by construction. So (5.7) becomes

r w1 (z δ,σ,ε
1
) + H ((z 1 )δ,σ,ε ), Dw1δ (z δ,σ,ε
1
), D 2 w1δ (z δ,σ,ε
1
)) ≤ 0 (5.15)

We compute the difference of the two inequalities (5.15) and (5.8) above:

r (w1δ (z δ,σ,ε
1
) − w2,δ (z δ,σ,ε
2
)) + H ((z 1 )δ,σ,ε , Dw1δ (z δ,σ,ε
1
), D 2 w1δ (z δ,σ,ε
1
))
− H ((z 2 )δ,σ,ε , Dw2,δ (z δ,σ,ε
2
), D 2 w2,δ (z δ,σ,ε
2
)) ≤ 0,

where, in view of (5.9) and the definitions of S, S  (and using that g  , g  and |Dξ1 |,
|D 2 ξ1 | are bounded by 1)

1 1
H ((z 1 )δ,σ,ε , Dw1δ (z δ,σ,ε
1
), D 2 w1δ (z δ,σ,ε
1
)) ≥ H ((z 1 )δ,σ,ε , (z δ,σ,ε − z δ,σ,ε
2
), S)
δ
− C1 (β + ε + σ ),
1 1
H ((z 2 )δ,σ,ε , Dw2,δ (z δ,σ,ε
2
), D 2 w2,δ (z δ,σ,ε
2
)) ≤ H ((z 2 )δ,σ,ε , (z δ,σ,ε − z δ,σ,ε
2
), −S  )
δ
+ C1 (β + ε + σ ),

Next, we have:
( ( ( (
( ( ( (
(u((z 1 )δ,σ,ε ) − u((z 2 )δ,σ,ε )( ≤ C ((z 1 )δ,σ,ε − (z 2 )δ,σ,ε ( ,
() * ) *(
( (
( b((z 1 ) ), 1 (z 1 − z 2
) − b(z 2 
) ),
1 1
(z − z 2
) (
( δ,σ,ε
δ δ,σ,ε δ,σ,ε δ,σ,ε
δ δ,σ,ε δ,σ,ε (

C (( ((
(( 1
(
(
≤ ((z 1 )δ,σ,ε − (z 2 )δ,σ,ε ( (z δ,σ,ε − z δ,σ,ε
2
(,
δ
1 + , 1+ ,
− κ((z 2 )δ,σ,ε ), −S  κ((z 2 )δ,σ,ε ) ≤ − κ((z 1 )δ,σ,ε ), Sκ((z 1 )δ,σ,ε )
2 2
C (( 1  2 
(2
(
+ ((z )δ,σ,ε − (z )δ,σ,ε ( .
δ
We deduce that:
" #
r w1δ ( pδ,σ,ε ) − w2,δ ( pδ,σ,ε )
 ( (2 ( (
1( 1  2  ( ( 1  2  (
≤C ((z )δ,σ,ε − (z )δ,σ,ε ( + ((z )δ,σ,ε − (z )δ,σ,ε (
δ
C (( ((
(( 1
(
(
+ ((z 1 )δ,σ,ε − (z 2 )δ,σ,ε ( (z δ,σ,ε − z δ,σ,ε
2
( + 2C1 (β + ε + σ ).
δ

123
Continuous-time limit of dynamic games... 351

As σ and ε tend to 0, the z δ,σ,ε 1 2

, z δ,σ,ε , (z 1 )δ,σ,ε and (z 2 )δ,σ,ε converges (up to a
subsequence) to z δ1 , z δ2 , (z 1 )δ and (z 2 )δ , where (z δ1 , z δ2 ) is a maximum in (5.2) and
where (z 1 )δ and (z 2 )δ satisfy (5.3). The above inequality together with the definition
of Mδ implies:
 ( (2 ( (
1( 1  ( ( (
r Mδ ≤ r w1δ (z δ1 ) − w2,δ (z δ2 ) ≤ C ((z )δ − (z 2 )δ ( + ((z 1 )δ − (z 2 )δ (
δ
C (( ((
((
(
(
+ ((z 1 )δ − (z 2 )δ ( (z δ1 − z δ2 ( + 2C1 β.
δ

We finally let δ → 0: in view of Lemma 5.9 the above inequality yields to r M  =

limδ→0 r Mδ ≤ 2C1 β, which contradicts our initial assumption. Therefore w1 ≤ w2
and the proof is complete.

References
Aumann RJ, Maschler M (1995) Repeated games with incomplete information, with the collaboration of
R Stearns. MIT Press, Cambridge
Bertsekas DP, Shreve SE (1978) Stochastic optimal control: the discrete time case. Academic Press, New
York
Blackwell D, Dubins LE (1983) An extension of Skorohod’s almost sure representation theorem. Proc Am
Math Soc 89:691–692
Cardaliaguet P (2007) Differential games with asymmetric information, SIAM. J Control Optim 46:816–838
Cardaliaguet P (2009) A double obstacle problem arising in differential game theory. J Math Anal Appl
360:95–107
Cardaliaguet P, Rainer C (2009) Stochastic differential games with asymmetric information. Appl Math
Optim 59:1–36
Cardaliaguet P, Rainer C (2009) On a continuous-time game with incomplete information. Math Oper Res
34:769–794
Cardaliaguet P, Rainer C (2012) Games with incomplete information in continuous time and for continuous
types. Dyn Games Appl 2:206–227
Cardaliaguet P, Rainer C, Rosenberg D, Vieille N (2013) Markov games with frequent actions and incomplete
information, Preprint, arXiv:1307.3365
Cardaliaguet P, Laraki R, Sorin S (2012) A continuous time approach for the asymptotic value in two-person
zero-sum repeated games. SIAM J Control Optim 50:1573–1596
Crandall MG, Ishii H, Lions PL (1992) User’s guide to viscosity solutions of second order partial differential
equations. Bull Am Math Soc 27:1–67
De Meyer B (2010) Price dynamics on a stock market with asymmetric information. Games Econ Behav
69:42–71
Dellacherie C, Meyer J-P (1982) Probabilities and potential. B. North-Holland Mathematics Studies. North-
Holland Publishing Co., Amsterdam
Dudley RM (2002) Real analysis and probability. Cambridge University Press, Cambridge
Ethier SN, Kurtz TG (1986) Markov processes: characterization and convergence. Wiley, New York
Gensbittel F (2015) Extensions of the Cav(u) theorem for repeated games with one-sided information. Math
Oper Res 40:80–104
Gensbittel F (2013) Covariance control problems over martingale arising from game theory. SIAM J Control
Optim 51:1152–1185
Gensbittel F, Grün C (2014) Zero-sum stopping games with asymmetric information, Preprint,
arXiv:1412.1412
Grün C (2013) On Dynkin games with incomplete information. SIAM J Control Optim 51:4039–4065
Grün C (2012) A BSDE approach to stochastic differential games with incomplete information. Stoch
Process Appl 122:1917–1946

123
352 F. Gensbittel

Jacod J, Shiryaev AN (2003) Limit theorems for stochastic processes, 2nd edn. Springer, Berlin
Kallenberg O (2002) Foundations of modern probability. Springer, Berlin
Laraki R (2004) On the regularity of the convexification operator on a compact set. J convex Anal 11:209–
234
Lipster RS, Shiryaev AN (2001) Statistics of random processes. I: general theory, 2nd edn. Springer, Berlin
Mertens JF, Sorin S, Zamir S (1994) Repeated games, CORE Discussion Papers 9420, 9421 and 9422.
Université Catholique De Louvain, Belgium
Meyer PA, Zheng WA (1984) Tightness criteria for laws of semimartingales. Ann de l’Institut Henri Poincaré
(B) Probab Stat 20:353–372
Neyman A (2013) Stochastic games with short-stage duration. Dyn Games Appl 3:236–278
Sorin S (2002) A first course on zero-sum repeated games. Springer, Berlin

123
Reproduced with permission of the copyright owner. Further reproduction prohibited without
permission.