Adaptive Trading Strategies Across Liquidity Pools

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/343735473
Adaptive trading strategies across liquidity pools
Preprint · August 2020
CITATIONS READS
0 1,591
2 authors:
Bastien Baldacci Manziuk Iuliia

École Polytechnique Université de Paris 1 Panthéon-Sorbonne
24 PUBLICATIONS 39 CITATIONS 11 PUBLICATIONS 65 CITATIONS
SEE PROFILE SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Contract theory View project
All content following this page was uploaded by Bastien Baldacci on 24 August 2020.
The user has requested enhancement of the downloaded file.

Adaptive trading strategies across liquidity pools∗
Bastien Baldacci† Iuliia Manziuk‡
August 19, 2020
arXiv:2008.07807v1 [q-fin.TR] 18 Aug 2020
Abstract
In this article, we provide a flexible framework for optimal trading in an asset listed on dif-
ferent venues. We take into account the dependencies between the imbalance and spread of the
venues, and allow for partial execution of limit orders at different limits as well as market orders.
We present a Bayesian update of the model parameters to take into account possibly changing
market conditions and propose extensions to include short/long trading signals, market impact
or hidden liquidity. To solve the stochastic control problem of the trader we apply the finite
difference method and also develop a deep reinforcement learning algorithm allowing to consider
more complex settings.
Keywords: cross-platform trading, optimal trading, Bayesian learning, adaptive trading strate-
gies, deep reinforcement learning, stochastic control.
1 Introduction
A vast majority of quantitative trading strategies are based on cross-platform arbitrage. These strate-
gies involve cross-listed stocks, that are assets traded on two or more liquidity venues. In [21], the
authors investigate the prices of cross-listed stocks in different venues and provide evidence of price
deviations for the majority of the 600 cross-listed stocks they studied. In [3], the authors highlight
mispricing that can exist between a domestic stock and its ADR (American Deposit Receipt) coun-
terpart. The study conducted in [26] for US-UK cross-listed stocks shows that markets for cross-listed
securities are among the most heavily arbitraged. In particular, higher potential of arbitrage can be
exploited for cross-listed stocks from emerging markets, see [24].
Usually, the trader builds an execution curve targeting, for example, an Implementation Shortfall
or volume-weighted average price (VWAP). Then, he buys or sells shares of the asset following the
∗
This work benefits from the financial support of the Chaires Analytics and Models for Regulation, Financial Risk
and Finance and Sustainable Development. Bastien Baldacci gratefully acknowledge the financial support of the ERC
Grant 679836 Staqamof. The authors would like to thank Joffrey Derchu (Ecole Polytechnique), Mathieu Rosenbaum
(Ecole Polytechnique) and Olivier Guéant (Université Paris-1 Panthéon-Sorbonne) for numerous fruitful discussions.
In particular, Mathieu Rosenbaum deserves warm thanks for his careful reading of the paper and his many suggestions
to improve its quality.
†
École Polytechnique, CMAP, 91128, Palaiseau, France, [email protected].
‡
École Polytechnique, CMAP, 91128, Palaiseau, France, [email protected].
1
execution curve by sending limit and market orders to the different venues. But how to find the
best splitting of orders between the venues? The trader splits his orders depending mainly on the
imbalance and spread of the different venues, which of course depend on each other. For example, a
higher imbalance on the ask side of one venue can indicate a potential imminent price change and a
lower probability to have an ask limit order executed, so it may be more profitable to send the order to
another venue. The problem of optimal trading across liquidity pools has been treated, for instance,
in [1, 13, 22, 23]. In [1], the authors develop a dynamic estimate of the hidden liquidity present on
several venues and use this information to make order splitting decisions (Smart Order Routing).
The paper [13] solve a general order placement problem and provide explicit solution for the optimal
split between limit and market orders on different venues. Finally, in [22, 23], the authors build a
stochastic algorithm to find the optimal splitting between liquidity pools, including dark venues.
Building a good model for optimal trading cross-listed assets requires to take into account the cross-
dependence between the imbalance and spread of each venue, as well as the probability and the
proportion of execution of limit orders. However, the quality of the model will mainly rely on the
estimation of the market parameters. If one assumes constant parameters over the trading period,
he believes in the quality of his parameters’ estimation. In this case, his strategy is not robust to
changes in price dynamics or the platforms’ behavior. For example, the trading period may occur
when another market participant is executing a buy (or sell) metaorder on one or several venues. This
participant will consume the vast majority of the liquidity available on the sell (or buy) side of those
venues. If the trader does not adjust his market parameters, the algorithm will keep sending limit
orders on the platform where the metaorder is being split, without an execution opportunity. That
is why it is essential to update model parameter estimations with new information obtained from
observing the market dynamics. Here, we treat the updates in a Bayesian manner.
In this paper, we formulate the problem of a trader dealing in a stock, listed on several venues, by
placing limit and market orders. Trader’s activity can be formulated as a stochastic control problem.
The controls are the splitting of volumes between limit and market orders on each venue, and the
limits chosen by the trader. The optima are obtained from a classical Hamilton-Jacobi-Bellman (HJB)
quasi-variational inequality, which, for a parsimonious model, can be easily solved by grid methods.
Then we propose a Bayesian update of each market parameter, decoupled from the control prob-
lem. One of the advantages of this method is the simplicity of the formulae for each parameter’s
posterior estimate. In particular, we do not need to use Markov chain Monte-Carlo. This method’s
choice comes from the fact that updating the market parameters continuously in the control problem
increases the number of state variables drastically, leading to high computation time. A continuous
Bayesian update would require first to compute the conditional expectation of the value function given
the market parameters and then to integrate it over their posterior distribution. This last integration
brings multiple non-linearities in the equation, making this fully Bayesian control problem hard to
solve numerically.
The proposed Bayesian procedure is easier to apply in practice: we divide the trading period into time
slices of about several seconds up to a few minutes long, assuming that market conditions do not vary
drastically throughout the slice. For each slice, we keep track of all the market events. Specifically,
on each venue, we count the number of executed limit orders, the executed proportions (for example,
2
50% or 100% of the order volume) given the couple spread-imbalance on each venue at the time of
the execution. We also keep track of the price dynamics. At the end of each slice, we update our
view on the market parameters and recompute the optimal trading strategy for the next slice. This
application of the Bayesian updates on slices of execution is time-inconsistent. However, we see it
as a first step toward a more integrated Bayesian learning framework for cross-listed trading. By
using finite difference schemes or deep reinforcement learning methods (which could also be mixed)
for high-dimensional PDE resolution, we can compute in a couple of minutes the optimal trading
strategy on a slice given new market conditions.
This paper aims at giving a useful and applicable model for practitioners who work on cross-trading
strategies. For a quantitative firm, the control model is flexible enough to reproduce the main stylized
facts about the market and to design trading strategies taking into account real signals. Moreover,
the procedure for Bayesian updates of market parameters in the control problem enables to reevaluate
the optimal strategy when the market conditions may differ from the prior empirical estimation of
the trader.
The article has the following structure: in Section 2, we describe the framework for cross-platform
trading and formulate the trader’s optimization problem. In Section 2.2, we derive the Hamilton-
Jacobi-Bellman quasi-variational inequality (HJBQVI) associated with the trader’s optimal trading
problem. We introduce a change of variable to reduce the dimensionality of the problem and prove the
existence and uniqueness of the viscosity solution of the initial HJBQVI in Appendix A. In Section 3,
we first define the conjugate Bayesian update of all market parameters. Then, we describe the update
procedure in practice and its link to the control problem of the trader. Section 4 is dedicated to some
extensions of the model and their impact on the dimensionality of the resulting HJBQVI. We devote
Section 5 to numerical results, for the sake of clarity of interpretations considered in the case of limit
orders only. Finally in Appendix Appendix B.2, we present an application of the Bayesian update of
the market parameters to the problem of an OTC market maker.
2 Optimal trading on several liquidity pools

The model presented in this section is a generalization of the classic optimal trading framework,
developed notably in [5, 17, 19, 25] and in the reference books [12, 16], to the case of several liquidity
venues.
2.1 Framework
We consider a trader acting on N liquidity platforms operating with limit order books over time
interval [0, T ]. He trades continuously on each venue by sending limit and market orders. For n ∈
{1, . . . , N}, the n-th venue is characterized by the following continuous-time Markov chains:
n
• the bid-ask spread process (ψtn )t∈[0,T ] taking values in the state space ψ = {δ n , . . . , Jδ n },
n
• the imbalance process (Itn )t∈[0,T ] taking values in the state space I = {I1n , . . . , IK
n
},
where J, K ∈ N denote the number of possible spreads and imbalances respectively and δ n stands for
the tick size of the n-th venue. We define the sets Ψ = {Ψ1 , . . . , Ψ#Ψ }, I = {I1 , . . . , I#I } of disjoint
intervals, representing different market regimes of interest in terms of spreads and imbalances.
3
n o
Example 2.1. Assume for all n ∈ {1, . . . , N} that δ n = δ. The set Ψ = δ, {2δ, 3δ}, {4δ, 5δ} denotes
three spread regimes: low (one tick), medium (two or three ticks), and high (four or five ticks).
n Assume for all n ∈ {1, . . . , N} and k ∈ {1, . . . , K} thato Ik = Ik . In this case

n
Example 2.2.
the set I = [−1, −0.66], (−0.66, −0.33], (−0.33, 0.33], (0.33, 0.66], (0.66, 1] denotes five regimes of
imbalance: low (−33% to 33%), medium on the ask (resp. bid) from 33% to 66% (resp. from −66%
to −33%) and high on the ask (resp. bid) from 66% to 100% (resp. from −100% to −66%).
Whenever the spread and the imbalance of each venue enter the state k = (kψ , kI ) ∈ K where
n n
K= N n=1 ψ × n=1 I , they remain in this state for a time exponentially distributed with mean νk .
1
Q QN
We define a transition matrix P = (pkk′ ), (k, k′ ) ∈ K, and corresponding intensity vectors ν = (νk )T
k.
We assume that pkk = 0, meaning that we cannot come to the same state twice in a row. The
infinitesimal generator of the processes can be written as
rkk′ = νk pkk′ if k 6= k′
rkk = − rkk′ = −νk , otherwise.
X
k′ 6=k
Remark 2.3. This general formulation allows us a full coupling between the spread and imbalance
of all venues. If one wants a more parsimonious model, the following simplifications could be made.
When the spread (imbalance) of the n-th venue enters the state k, it remains there for an exponentially
1 1
distributed time with mean ν n,ψ ( ν n,I for the imbalance). Therefore, we define a transition matrix
k k
n
Pn,ψ = (pn,ψ n,ψ
kk ′ ), n ∈ {1, . . . , N}, (k, k ) ∈ ψ such that pkk = 0, and corresponding intensity vectors
′
ν n,ψ = (ν1n,ψ , . . . , νK
n,ψ T
) . Similarly we define a transition matrix Pn,I for the imbalance. Then, the
infinitesimal generator of the processes can be written as
n,ψ n,ψ n,ψ
rkk ′ = νk pkk′ if k 6= k ′
n,ψ n,ψ n,ψ
=− ′ = −νk
X
rkk rkk otherwise.
k ′ 6=k
This framework will be used in Section 5, where we present the numerical results.
In what follows, the trader designs his strategy on the ask side of the market (optimal liquidation
problem). The extension to trading on both sides of the market is straightforward and does not cause
an increase in the problem’s dimensionality.
The number of, possibly partially, filled askorders in the venue n is modeled by a Cox process denoted
by N n , n ∈ {1, . . . , N} with intensities λn ψt , It , pnt , ℓt where pnt ∈ Qnψ represent the limit at which
the trader sends a limit order of size ℓnt , and
Qnψ = {0, 1} if ψ n = δ n , and {−1, 0, 1} otherwise,

N
A = (ℓt )t∈[0,T ] , F − predictable, s.t for all t ∈ [0, T ], 0 ≤ ℓnt
X
≤ qt ,
n=1
where (qt )t∈[0,T ] is defined in Equation (2.1). Practically for n ∈ {1, . . . , N}, when the spread is equal
to the tick size, the trader can post at the first best limit (pn = 0) or the second best limit (if pn = 1).
4
When the spread is equal to two ticks or more, the trader can either create a new best limit (pn = −1)
or post at the best or the second best limit as previously. The arrival intensity of a buy market order
at time t on the venue n ∈ {1, . . . , N} at the limit p ∈ Qnψ , given a couple (ψt , It ) = m of spread and
imbalance on each venue, is equal to λn,m,p > 0. When the trader posts limit orders of volume ℓnt
on the n-th venue for n ∈ {1, . . . , N}, the probability that it is executed is equal to f λ (ℓt ), where
f λ (·) ∈ [0, 1] is a continuously differentiable function, decreasing with respect to each of its coordinate.
Therefore, the arrival intensity of an ask market order filling the buy limit order of the trader on the
n-th venue at the limit pnt , given spread and imbalance (ψt , It ) is a multi-regime function defined by
λn (ψt , It , pnt , ℓt ) = f λ (ℓt ) λn,m,p1{(ψt ,It )∈m,pnt =p} ,
X
m∈M,p∈Qn
ψ
where M = ΨN × I N . Moreover, we allow for partial execution, the fact of which we represent
by random variables ǫnt ∈ [0, 1]. The proportion of executed volume for limit orders in each venue
depends on the spread and the imbalance in all N venues, as well as the volume and the limit of
the order chosen by the trader. We assume a categorical distribution with R > 0 different execution
proportions ω r , r ∈ {1, . . . , R} for each venue with P(ǫnt = ω r ) = ρn,r (ψt , It , pnt , ℓt ), where
ρn,r (ψt , It , pnt , ℓt ) = f ρ (ℓt ) ρn,m,p,r 1{(ψt ,It )∈m,pnt =p} ,
X
m∈M,p∈Qn
ψ
where f ρ (·) is a continuously differentiable function, decreasing with respect to each of its coordinate.
Remark 2.4. The estimation of this kind of parameters for executed proportions can be quite intricate
in practice. To simplify, one can assume that ρn,r (ψt , It , pnt , ℓt ) = ρn,r ∈ [0, 1]. In practice, this means
that there are different execution proportion probabilities inherent by each venue, depending on its
toxicity.
Finally, we allow for the execution of market orders (denoted by a point process (Jtn )t∈[0,T ] ) on each
venue of size (mnt )t∈[0,T ] ∈ [0, m] where m > 0 and Jtn = Jtn− + 1. We assume that market orders are
always fully executed.
The cash process of the trader at time t ∈ [0, T ] is

N
ψtn ψn

dXt = ℓnt St + + pnt δ n ǫnt dNtn + mnt St − t dJtn ,
X
n=1 2 2
where
dSt = µdt + σdWt , (µ, σ) ∈ R × R+ ,
is the dynamics of the mid-price process. The inventory process of the trader at time t ∈ [0, T ] is
defined by
N Z t Z t
qt = q0 − ℓnu ǫnu dNun + mnu dJun (2.1)
X
.
n=1 0 0
We also assume that the trader has a pre-computed trading curve q ⋆ that he wants to follow (Almgren-
Chriss trading curve or VWAP strategy, for example). Then the trader’s optimization problem is
Z T
sup E XT + qT ST − g(qt − qt⋆ )dt , (2.2)
(p,ℓ,m)∈Qψ ×A×[0,m]N 0
where the function g penalizes deviation from the pre-computed optimal trading curve.
5
2.2 The Hamilton-Jacobi-Bellman quasi-variational inequality
The HJBQVI associated with the optimization problem of the trader (2.2) is the following:

 1
0 = min − ∂t u(t, x, q, S, ψ, I) + g(q − qt⋆ ) − µ∂S u − σ 2 ∂SS u
 2

r(ψ,I),(kψ ,kI ) u(t, x, q, S, kψ , kI ) − u(t, x, q, S, ψ, I)
X
−
k∈K
N (2.3)
ψn

− sup λ (ψ, I, p , ℓ)E u t, x + ǫ ℓ (S +
n n
+ pn δ n ), q − ℓn ǫn , S, ψ, I
n n
X
p∈Qψ ,ℓ∈A n=1 2

N n
ψ

− u(t, x, q, S, ψ, I) ; u(t, x, q, S, ψ, I) − sup u t, x + mn (S − ), q − mn , S, ψ, I ,
X
n=1 mn ∈[0,m] 2
with terminal condition
u(t, x, q, S, ψ, I) = x + qS,
where ψ = (ψ 1 , . . . , ψ N ), I = (I 1 , . . . , I N ). The expectation in (2.3) is taken over the variables ǫn , n ∈

{1, . . . , N}. We prove the following theorem in Appendix A:
Theorem 1. There exists a unique viscosity solution to the HJBQVI (2.3), which coincides with the
value function of the control problem of the trader (2.1).
The proof of existence and uniqueness of the viscosity solution mainly relies on adaptations of the
theory of the second order viscosity solution with jumps, see [9], for example.
The value function has to be linear with respect to the cash process and the mark-to-market value
of the trader’s inventory due to the form of the terminal condition. Therefore we use the following
ansatz:
u(t, x, q, S, ψ, I) = x + qS + v(t, q, ψ, I).
The HJBQVI then becomes a system of ODEs with 2N + 1 state variables:



0 = min − ∂t v(t, q, ψ, I) + g(q − qt⋆ ) − µq


r(ψ,I),(kψ ,kI ) v(t, q, S, kψ , kI ) − v(t, q, S, ψ, I)
X
−
k∈K
N (2.4)
ψn

sup λn (ψ, I, pn , ℓ)E ǫn ℓn ( + pn δ n ) + v t, q − ℓn ǫn , ψ, I
X
−

N
ψn

− v(t, q, ψ, I) ; v(t, q, ψ, I) − sup −mn + v t, q − mn , ψ, I ,
X
n=1 mn ∈[0,m] 2 
with terminal condition v(T, q, ψ, I) = 0.
6
Conditionally on the market parameters such as the transition matrix of both the spread and the
imbalance processes, the drift and volatility of the underlying asset and the execution proportion
probabilities, solving Equation (2.4) is done using simple finite difference schemes and the optimal
splitting of volumes as well as the optimal limits can be computed in advance.
If one want to incorporate directly Bayesian learning of the parameters in the control problem, the
result would be a very high number of state variables, which makes the problem intractable in practice.
For example, if we want to update continuously the value of the processes λn for n ∈ {1, . . . , N} we
need to add the counting processes (Ntn )t∈[0,T ] to the state variables, which increases the dimension
of the HJBQVI (2.4) by N. What we propose in the following section is a practical way to update
the market parameters according to trader’s observations in a Bayesian way. This method, which is
performed separately from the optimization procedure, allows to update, at the end of a slice, the
trading strategy according to changing market conditions.
3 Adaptive trading strategies with Bayesian update

The framework presented in the above section allows to choose generic parametric forms for the state
variables prior distributions (transition matrix of spreads and imbalances, intensities of orders’ arrival
on each venue) suitable to the use of conjugate Bayesian updates.
3.1 Bayesian update of the model parameters

In this section, we present the conjugate Bayesian update of the market parameters and how to choose
the prior distributions.
3.1.1 Update of the intensities

Let us recall the form of the intensities for counterpart market orders’ arrival:
λn (ψt , It , pnt , ℓt ) = f λ (ℓt ) λn,m,p1{(ψt ,It )∈m,pnt =p} .

X
m∈M,p∈Qn
ψ
In the vast majority of optimal liquidation models, the probability of execution λn,m,p is estimated
empirically. We propose to put a prior law Γ(αn,m,p, β n,m,p) on the arrival rate, and to update a prior
belief at the end of each slice of execution. The parameters αn,m,p , β n,m,p are chosen by the trader
according to his vision of the market before he starts to trade. Up to time t ∈ [0, T ] the trader
observes the processes
Z t
Ntn,m,p = 1{(ψs ,Is )∈m,pns=p} dNsn ,
0
which represent the number of executed orders on each venue for every spread-imbalance zone m.
The posterior distribution of λn,m,p for n ∈ {1, . . . , N} is then given by
Z t
λn,m,p|Ntn,m,p ∼ Γ αn,m,p + Ntn,m,p , β n,m,p + f λ (ℓs )ds ,
0
7
and at time t, our best estimate of the filling ratio becomes
αn,m,p + Ntn,m,p

λ n,m,p
(t, Ntn,m,p , ℓt ) =E λ n,m,p
|Ntn,m,p = n,m,p R t λ .
β + 0 f (ℓs )ds
The posterior estimate of the intensity λn (ψt , It , pnt , ℓt ) becomes
αn,m,p + Ntn,m,p
λ̂n (ψt , It , pnt , ℓt ) = f λ (ℓt )
X
1 n
n,m,p + t f λ (ℓ )ds {(ψt ,It )∈m,pt =p}
R .
m∈M,p∈Qn , β
ψ
0 s
As the convergence of the prior parameters toward the true market specification follows from the
central limit theorem, the convergence rate equals to √1om where om is the number of observations of
filled limit orders on the spread-imbalance zone m. If we consider even a quite parsimonious model,
for example two venues, two regimes of spread and three regimes of imbalance, we have #M = 36
different zones. This means that we need a sufficiently large amount of observations (large number of
executed orders) to get an accurate approximation of the market behavior.
If the trader anticipates that the number of observations he will have is not adequate to obtain
a suitable approximation of the “true” market parameters (in the case of a mid to low frequency
strategy with only a few number of trades throughout the day), he might choose at the beginning
n,m,p N n,m,p
the couples (αn,m,p, β n,m,p) such that αβ n,m,p >> R t tλ . That way, his prior will not be sensitive
0
f (ℓs )ds
to a small number of observations, and with sufficient number of observations the prior will have less
influence and the estimation will be less biased.
3.1.2 Update of the executed proportion

We propose to use the Dirichlet prior distribution on the executed proportion parameters so that
ρn,m,p ∼ Dirichlet(αǫ,n,m,p) where αǫ,n,m,p = (αǫ,n,m,p,1, . . . , αǫ,n,m,p,R) for all (n, m, p, r) ∈ {1, . . . , N}×
M × Qψ × {1, . . . , R}. Given observations of ǫnt , the executed proportion parameters have Dirichlet
posterior distribution
ρn,m,p ∼ Dirichlet(αǫ,n,m,p + cn,m,p

t ),
where cn,m,p = (cn,m,p,1 , . . . , cn,m,p,R ) and cn,m,p,r = s≤t 1{ǫns =ωr ,(ψs ,Is )∈m,pns =p,Nsn−N n− =1} is the number
P
t t t t s
of observations before time t in zone m for a limit p in the venue n. Therefore, the ǫit have the
following posterior distribution:
αǫ,n,m,p,r + cn,m,p,r
t
ρ̂n,r (ψt , It , pnt , ℓt ) = f ρ (ℓt )
X
1 n
ǫ,n,m,p,r + cn,m,p,r ) {(ψt ,It )∈m,p =p}
.
(α
PR
m∈M,p∈Qψ r=1 t
This Bayesian update is linked to the filling of limit orders of the trader: the proportion executed is
updated only if the limit order is (partially) executed. If one chooses the parametrization independent
of the spread-imbalance zones and the order volume, that is execution proportion depends only on
the venue, the speed of convergence is much faster as the same amount of gathered information is
used to update a much smaller number of parameters. Using this more parsimonious parametrization
the trader can rely on the observations more than on his prior.
8
3.1.3 Update of the characteristics of the venues
We observe the states of the Markov chains ψtd , Itd , d ∈ {0, . . . , D} and the times td of the D > 0
transitions. The likelihood function for the spread and the imbalance processes is
D
L(P, ν|ψt≤tD , It≤tD ) = νtd−1 exp − νtd−1 (td − td−1 ) p(ψt d−1 ,Itd−1 )(ψt d ,Itd )
Y
d=1
(νk )nk· exp(−νk Tk ) (pkk′ )nkk′ ,

Y Y
∝
k∈K k′ ∈K
where nkk′ is the number of observed transitions from state k to k′ for (k, k′ ) ∈ K, Tk is the total
time spent in state k, and nk· = k′ ∈K nkk′ is the total number of transitions out of state k.
P
Given independent prior distributions for P, ν, the posterior distributions will also be independent.
We can carry out Bayesian inference separately on the probability matrix and the intensity vectors
of the Markov chains. We assume the following priors:
νk ∼ Γ(ak , bk ),
pk = (pkk′ )k′ ∈K ∼ Dirichlet(αk ), where αk = (αkk′ )k′ ∈K .
Given these conjugate priors, our best estimators of νk , pk are
ak + nk· − 1
ν̂k = ,
bk + Tk
αkk′ + nkk′
p̂kk′ =P .
l6=k (αkl + nkl )
Then the posterior transition matrix is

r̂kk′ = ν̂k p̂kk′ , k 6= k′ ,
r̂kk = −ν̂k .
This update aims at finding the “true” behavior of the imbalance and spread processes of each venue.
This is of particular importance if an event (for instance, an announcement or news) happens in the
market. More specifically, if one event occurs in a particular platform (if a metaorder is executed in
one specific platform, for example), this helps to discriminate one venue from the others and to redirect
the orders to the less toxic liquidity platforms. Given the large number of observations (transitions
from one state of imbalance or spread to another occur fast), the trader does not necessarily need to
be confident about his prior distributions.
Remark 3.1. If one wants to use a more parsimonious model as in Remark 2.3, the same methodology
n
applies. In particular for k ∈ ψ , we assume the following prior:
νkn,ψ ∼ Γ(an,ψ n,ψ
k , bk ),
pn,ψ
k = (pn,ψ n,ψ n,ψ
kk ′ )k ′ ∈ψ ∼ Dirichlet(αk ), where αk
n
n,ψ
= (αkk ′ )k ′ ∈ψ n .
Given these conjugates priors, our best estimators of νkn,ψ , pn,ψ

k are
an,ψ + nn,ψ
k· − 1
ν̂kn,ψ = k
,
bk + Tkn,ψ
n,ψ
9
αkk′ + nn,ψ
p̂n,ψ
kk ′ = P
kk ′
n,ψ .
l6=k (αkl + nkl )
The posterior transition matrix is given by

n,ψ n,ψ n,ψ
r̂kk ′ = ν̂k p̂kk′ , k 6= k ′ ,
n,ψ
r̂kk = −ν̂kn,ψ .
Similar formulae apply for νkn,I , pn,I

k .
3.1.4 Update of the mid-price

We recall that the price process has the following dynamics:
dSt = µdt + σdWt ,
so that (St − S0 |µ, σ) ∼ N (µt, σ 2t). We assume that the couple (µ, σ 2 ) follows a Normal-Inverse-
Gamma prior distribution NIG(µ0 , ν, αs , β s ), where (µ0 , ν, αs , β s ) ∈ R × R3+ . Therefore the posterior
distribution has the following form:
(St − S0 ) + µ0 ν t tν ( St −S 0
− µ 0 )2

2
(µ, σ |St − S0 ) ∼ NIG , ν + t, αs + , β s + t
.
ν +t 2 ν+t 2
Given our observations of the stock price up to time t, the best approximation of the drift and volatility
are given by
St −S0
( −µ0 )2
(St − S0 ) + µ0 ν 2 2
β s + ν+t
tν t
2
µ(t, St ) = E[µ|St − S0 ] = , σ (t, St ) = E[σ |St − S0 ] = .
ν +t αs + t
2
−1
The volatility σ does not appear explicitly in the HJBQVI (2.3). However, it is taken into account
when the trader computes his trading curve q ⋆ .
In the case where the trader is confident with his estimation of σ, one can use a Normal prior
distribution on µ such that µ ∼ N (µ0, ν 2 ). Then, the best approximation of the drift is given by
µ0 σ 2 + ν 2 (St − S0 )
µ(t, St ) = E[µ|St − S0 ] = . (3.1)
σ2 + ν 2 t
If the trader firmly believes in the a priori parameter estimation, he can set ν close to 0 so that he
mostly relies on his prior. On the contrary, if he sets ν high enough, his estimation comes mostly
from market information. Given the large amount of data coming from the market (each time step
corresponding to one new observation), convergence to the real value of the drift is fast.
Remark 3.2. One can argue about the use of a frequentist estimator of the model parameters, which
would actually lead to quite similar formulae. However the original problem, that is continuous update
of market parameters in the control problem, is of Bayesian nature. Moreover, in our approach, the
formulae for posterior distribution of market parameters are as explicit as in the frequentist approach.
10
3.2 Algorithm description
We now present the use of the Bayesian updates in order to obtain adaptive trading strategies in
practice. We emphasize that the procedure is decoupled from the optimization problem (2.4), so
that we do not perform Bayesian optimization but rather a Bayesian update of the parameters of an
optimization problem.
Number of time steps is an important parameter of the optimization problem because its choice is a
trade-off between computation time and computation precision. To address this problem, we use the
trading algorithm with fixed market parameters over a short period of time (a couple of seconds up
to a few minutes), which we call a slice. Let us consider V > 0 slices Tv = [Tv , Tv+1 ], v = 0, . . . , V − 1,
such that T0 = 0, TV = T . We define for each slice v ∈ V a set of market parameters
θvm = (r, ρn , λn,m,p, µ, σ){n∈{1,...,N },m∈M,p∈Qψ } .
At each time slice v ∈ {0, V − 1} starting from v = 0 we perform the following algorithm:
1. Take the best estimation of market parameters θvm from the prior distribution for the current
slice v.
2. Compute the optimal trading strategy on Tv using the set of parameters θvm .
3. Observe market events during the current slice (executions, changes of the state).
4. At Tv+1 , update the parameters θv+1
m
following the Bayes rules described in Section 3.
To summarize, we use the output of the control model (the optimal volumes and limits in each venue)
over a slice of execution and then run the model again with the updated market parameters. This
method, which is clearly time inconsistent, is common practice when one applies optimal control with
online parameter estimation, see for example [7]. We now present some possible extensions of the
presented model.
4 Model extensions
In this section we describe different potential model extensions and their impact on the problem’s
dimensionality.
4.1 Extension 1: Incorporation of signals in the price process

4.1.1 Short-term price signals
The two main sources of signals at the microstructural level are the imbalance and the bid-ask spread.
Therefore, one can assume a parametric dependence f short (ψt , It ) of the price process on these two
sources, such that the price process becomes

dSt = µ + f short (ψt , It ) dt + σdWt .
In a modified stochastic control problem the term µq in the HJBQVI is replaced by (µ + f short(ψ, I)),
which causes no increase in the dimensionality of the state process.
11
4.1.2 Mid/Long term and path-dependent price signals
When trading on longer time horizon, one can incorporate mid- or long-term signals such as Bollinger
bands, moving average or cointegration ratio. For example, consider a signal taking into account the
moving average and the maximum of the price process St , that is
1 t
Z
St = St dt, St⋆ = max Ss .
t 0 s≤t
The triplet (S t , St⋆ , St ) is Markovian. Therefore, we can add a long term signal f long (St , S t , St⋆ ) into
the asset’s drift:

dSt = µ + f long (St , S t , St⋆ ) dt + σdWt .
The HJBQVI then becomes:

 S −S 1
0 = min − ∂t u(t, q, S, S, S ⋆ , ψ, I) + g(q − qt⋆ ) − µ + f long (S, S, S ⋆ ) ∂S u − ∂S u − σ 2 ∂SS u
 t 2

r(ψ,I),(kψ ,kI ) u(t, q, S, S, S ⋆ , kψ , kI ) − u(t, q, S, S, S ⋆ , ψ, I)
X
−
k∈K
N
ψn

sup λ (ψ, I, p , ℓ)E ǫn ℓn (S +
n n
+ pn δ n ) + u t, q − ℓn ǫn , S, S, S ⋆ , ψ, I
X
−

N
ψn
X 
− u(t, q, S, S, S ⋆ , ψ, I) ; u(t, q, S, S, S ⋆ , ψ, I) − sup mn (S − ) + u t, q − mn , S, S, S ⋆ , ψ, I ,
n=1 mn ∈[0,m] 2 
for S ≤ S ⋆ , with ∂S u = 0 for S = S ⋆ . To obtain this equation we just use a change of variable
v(t, x, q, S, S, S ⋆ , ψ, I) = x + u(t, q, S, S, S ⋆ , ψ, I), linear with respect to the cash process Xt . We end
up with a 2N + 4 dimensional HJBQVI, that we can still solve using our deep reinforcement learning
algorithm (but unlikely with finite differences).
More generally, adding a path-dependent state variable that gives information on the price trend adds
one dimension to the HJBQVI (in the example above (S t , St⋆ , St ) add one dimension each).
4.2 Extension 2: Market impact

So far we assumed no market impact on the price process. It is common knowledge that cost of
market impact can cut down a large proportion of the trading strategy’s profit. Therefore, we can
use a simple permanent-temporary market impact model, inspired by [2].
The impacted mid-price process can be modeled as follows:

N
dSt = µ + h(ℓt ) dt + σdWt + ξ n,l (t, ℓnt )dNtn + ξ n,m(t, ℓnt )dJtn ,
X
n=1
where the functions h, ξ , ξ are the permanent and temporary market impact functions. Follow-
n,l n,m
ing [15], we assume linear permanent market impact, that is

N
h(ℓt ) = κn,per ℓnt , κn,per > 0 for all n ∈ {1, . . . , N}.
X
n=1
12
For the temporary market impact, we can follow the well-known “square-root law” and set
n,l n,m
ξ n,l (t, ℓnt ) = κn,l (ℓnt )γ , ξ n,m (t, ℓnt ) = κn,m (ℓnt )γ ,
where κn,l , κn,m , γ n,l , γ n,m > 0 and γ n,l , γ n,m ≈ 1/2. On the other hand, in order to take into account
the transient part of the impact, we can set the following form for St :
Z t N Z t
St = S0 + µ + h(ℓs )ds + σWt + ξ n,l (t − s)ξñ,l (ℓis )dNsn + ξ n,m(t − s)ξñ,m(ℓs )dJsn , (4.1)
X
0 n=1 0
where ξ n,l , ξ n,m are decreasing kernels, and ξñ,l , ξñ,m are decreasing functions of the posted volume. It
is well known that by taking an exponentially decreasing kernel, Equation (4.1) admits a Markovian
R t n,{l,m}
representation as the couples Nt , 0 ξn
(t − s)dNsn are Markovian. Practically, this will
t∈[0,T ]
add 2N dimension to the HJBQVI.
Functions h, ξ n,l , ξ n,m could also be approximated by neural networks. Determination of a cross-impact
function between liquidity pools can lead to possible arbitrage detection across liquidity venues.
4.3 Extension 3: Hidden liquidity

Hidden liquidity represents a great proportion of the liquidity especially in the US markets, see for
example [21]. Therefore, if one wants to design trading tactics for assets cross-listed in a European
and an American market, taking into account the hidden part of the liquidity is crucial.
Assume that the n-th venue is a US liquidity pool. Borrowing the notations of [4], we denote by
H n the hidden liquidity of the n-th venue at the first limit of the order book. Therefore, the corre-
sponding imbalance process represented by the continuous-time Markov chain I n can be rewritten as
Ntn,a,m −Ntn,b,m
Ntn,a,m
+Ntn,b,m +2H n
, where Ntn,b,m , Ntn,a,m are the bid and ask market order flow processes on the n-th
venue. Empirical estimation of the prior parameters for the transition matrix of I n have to take into
account this additional term in the imbalance processes. Furthermore, incorporating the imbalance
process with hidden liquidity into trading signals allows to detect arbitrage opportunities between
different venues. This does not increase the dimensionality of Equation (2.3).
5 Numerical results
5.1 Global parameters
We take the example of a trader acting on a stock cross-listed on 2 different venues (N = 2), with
the following global parameters:
n
• ψ = {δ, 2δ}: the processes (ψtn )t∈[0,T ] can take two values, which correspond to a low or high
spread regimes, and the tick size is δ = 0.05.
n
• I = {−0.5, 0, 0.5}: the processes (Itn )t∈[0,T ] can take three values, which correspond to a nega-
tive, neutral or positive imbalance regime.
13
• R = 2, (ω 1, ω 2 ) = (0.5, 1): the processes (ǫnt )t∈[0,T ] can take two values, which correspond to a
total or half-execution of the posted volume (ℓnt )t∈[0,T ] .
• q0 = 5 × 104 : initial inventory of the trader.
• Tv = [v, v + ∆v ], where ∆v = 1 min, which means that each slice lasts one minute, with V = 10
slices and T = 10 min.
• ∆t = 0.1: we take 10 time steps in each slice, that is the agent takes 10 trading decisions during
each slice.
The pre-computed trading curve is borrowed from an implementation shortfall execution using market
orders, that is:
q
γσ2 V
sinh 2η
(T − t)
qt⋆ = q0 q .
γσ2 V
sinh 2η
T
with the following set of parameters
• η = 0.1: coefficient of quadratic costs.
• V = 1 × 108 : average market volume.
• γ = 1 × 10−6 : risk aversion of the trader using a CARA utility function.
• σ = 0.05: volatility of the asset.
• f λ (ℓt ) = exp(−κ N
n=1 ℓt ) with κ = 2.5 × 10 : sensitivity of the execution with respect to the
n −5
P
total volume posted.
• f ρ (ℓt ) = 1: no sensitivity of the executed proportion with respect to the total volume posted.
For this numerical experiment for the sake of clarity of interpretations we consider the trader sending
only limit orders.
5.2 Numerical methods

5.2.1 Finite differences
To find optimal strategy for limit orders we consider the following equation:
0 = − ∂t v(t, q, ψ, I) + g(q − qt⋆ ) − µq

N X
J N X
K
n,ψ −n n,I
v(t, q, ψ, II−n ) − v(t, q, ψ, I)
X X
− rψ,jδ v(t, q, ψjδ , I) − v(t, q, ψ, I) − rI,I k k
n=1 j=1 n=1 k=1
N
ψn

− sup λ (ψ, I, p , ℓ)E ǫ ℓ (
n n
+ pn δ n ) + v t, q − ℓn ǫn , ψ, I − v(t, q, ψ, I) ,
n n
X
14
where
−n
ψjδ = (ψ 1 , . . . , ψ n−1 , jδ, ψ n+1 , . . . ), II−n
k
= (I 1 , . . . , I n−1 , Ik , I n+1 , . . . ).
In order to apply the finite difference method we introduce the discretization of time and state space.
For inventories we have Q = {q1 = 0 < . . . < q#Q = q0 }. Time discretization in the slice is
T = {t0 = 0 < t1 = t0 + ∆t < . . . < t#T = ∆v }. We also discretize the order volumes the trader can
send L = {l1 = 0 < . . . < l#L = q0 }.
Using the first difference for the value function derivative with respect to time we can rewrite the
above equation as ∀i ∈ {0, . . . , #T − 1}, ∀q ∈ Q, ∀(ψ, I) ∈ M

v(ti+1 , q, ψ, I) =v(ti , q, ψ, I) − ∆t g(q − qt⋆ ) − µq
N X
J N X
K
n,ψ −n n,I
v(t, q, ψ, II−n ) − v(t, q, ψ, I)
X X
− rψ,jδ v(t, q, ψjδ , I) − v(t, q, ψ, I) − rI,I k k
n=1 j=1 n=1 k=1

N
ψn

− sup λn (ψ, I, pn , ℓ)E ǫn ℓn ( + pn δ n ) + v t, q − ℓn ǫn , ψ, I − v(t, q, ψ, I) ,
X
p∈{−1,0,1}N ,ℓ∈LN n=1 2
with terminal condition v(T, q, ψ, I) = 0.
In terms of calculations the most demanding part is obviously the search of the supremum which is
needed to be performed on the dimension 3N × #Q × #LN × #M for each time step. From what
follows that finite differences can be applied to solve the problem of optimal orders posting for the
stock cross-listed in N = 2 venues with reasonable precision and calculation time. However, if we
introduce more venues finite differences are not going to be any more efficient because the complexity
is growing exponentially.
For our numerical example, we used the discretization with #Q = 101 and #L = 51 which assures
the calculation time (on a simple PC) around 1min for the whole slice.
5.2.2 Neural networks

In this section, we briefly introduce the method using neural networks to solve HJB equations. In this
paper, we used a method which can be referred to as Actor-Critic method to approximate optimal
controls and corresponding value function for the problem. Applications of this approach have shown
to be fruitful, especially when we talk about equations in high dimension, more elaborate description
of the method can be found for example in [6, 8, 18, 20].
The core of this approach is to represent the strategy of the trader with a neural network as well as
the corresponding value function. Then one needs to formalize the target functions for both neural
networks and to perform the gradient descent on the parameters (weights) of these networks. This
procedure needs to be done for every time step, and so one ends up with 2#T networks.
15
Let us start from the description of the value function approximation. We consider the neural networks
taking as an input the spreads and the imbalances in the venues of interest and the inventory of the
trader giving as an output the value function at this point. As in the finite difference method we solve
our problem backward, starting from t#T−1 = ∆v −∆t , because the value function at the end of the slice
is known from the terminal condition. To calculate the value function at time ti , ∀i ∈ {0, . . . , #T − 1}
we use the minimization of the mean-squared error between values given by the neural network and
the target values calculated with the use of the value function approximation for time ti+1 and the
network for the controls at the current step. Let us assume that we have the controls ℓ∗ , p∗ (obtained
via neural networks, for example) for time ti , then the target for the value function can be found as
v target (ti−1 , q, ψ,I) = v[θiv ](ti , q, ψ, I) + ∆t g(q − qt⋆ ) − µq

N X
J
n,ψ −n
v[θiv ](t, q, ψjδ , I) − v[θiv ](t, q, ψ, I)
X
− rψ,jδ
n=1 j=1
N X
K
n,I
v[θiv ](t, q, ψ, II−n ) − v[θiv ](t, q, ψ, I)
X
− rI,I k k
n=1 k=1
N
ψn
!
λ (ψ, I, p∗n , ℓ∗ )E ǫn ℓ∗n (
n
+ p∗n δ n ) + v[θiv ] t, q − ℓ∗n ǫn , ψ, I − v[θiv ](t, q, ψ, I)
X
− ,
n=1 2
with v[θ#T
v
](t#T , q, ψ, I) = 0 and where [θiv ] stands for the weights of the neural network for the value
function at time ti .
The trader’s inventory is of continuous nature, however, spread and imbalance are categorical, so we
need to verify if we should use some special techniques to ensure better fitting in this case.
Target for value function for random spreads and imbalances
20000 target value
−20000
−40000
−60000
−80000
0 10000 20000 30000 40000 50000
Figure 1: Target value function for increasing inventory and random market states.
Let us see first in Figure 1 the example of the target value function of the trader for q ∈ [0, q0 ] at
different spreads and imbalances. We see considerable changes in the value function level depending
on the market state which we would like to capture by our approximation.
Now, let us compare the fitting of the value function parametrization taking as inputs raw spread
and imbalance values with the parametrization working with encoded values of the spread and the
imbalance. Here we are going to use the so-called one-hot encoding for categorical variables, which
consists in the representation of different values of the variable by a one-hot vector eiψ ∈ {0, 1}#Ψ
16
for the spread and eiI ∈ {0, 1}#I for the imbalance. And ei (both for eiψ and eiI ) are such that that
eij = 0, ∀j 6= i, and eii = 1 otherwise.
Fitting of target value function with raw values for states Fitting of target value function with encoded values for states
20000 target value target value
20000
predicted value predicted value
0 0
−20000 −20000
−40000 −40000
−60000 −60000
−80000 −80000
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 2: Comparison of the target value with ap- Figure 3: Comparison of the target value with
proximation continuous in spread and imbalance. approximation discrete in spread and imbalance.
In Figures 2 and 3 we see the comparison between values predicted by two parametrizations with
target values for the same number of learning epochs. There is a considerable gain in precision when
the parametrization takes into account the categorical nature of market states. Therefore we apply
it for both value function network approximation and the strategy neural network approximation.
Now, let us describe the learning procedure for the strategy. First of all, the inputs of the strategy
network are the same as for the value function network, i.e. the trader’s inventory, spreads and imbal-
ances for both venues. As an output, we need to have volumes of the orders and limits on which the
trader needs to send his orders. Volumes to send to each venue are bounded by the current inventory
because we do not want the trader to execute more shares than he possesses. Limits should equal
−1, 0, or 1, but as soon as we want to use the tools of automatic differentiation, we need to represent
them by differentiable function. The softmax activation function serves well to this purpose, so we
represent the limits for each venue by the probabilities to send an order to each precise limit. In
practice, the trader can choose the maximum of the three to perform his action.
The optimization criterium used for the strategy neural network is the function under supremum from
the HJBQVI (2.4), with limit probabilities taken into account (let us denote them by P(p = a), for
a ∈ {−1, 0, 1}) we need to maximize with respect to θiℓ , i ∈ {0, . . . , #T − 1}:
N
ψn

P[θiℓ ](pn = a)λn (ψ, I, a, ℓ[θiℓ ])E ǫn ℓ[θiℓ ]n + aδ n
X X
n=1 a∈{−1,0,1} 2

+ ℓ
v[θi+1 ] ti , q − ℓ[θiℓ ]n ǫn , ψ, I − ℓ
v[θi+1 ](ti , q, ψ, I) ,
where θiℓ stand for the weights of the neural network of controls at time ti . So we want to maximize this
function for all possible values of market states and inventories. To avoid the dimensionality trap we
need to optimize this function on some subset of possible values, which we are going to draw randomly.
When optimizing neural networks approximations, it is important to normalize the data, to have if
possible a universal set of hyperparameters. First of all, the inventory entering as an input of the
17
value function neural network and of the strategy neural network is normalized by q0 to always stay
in [0, 1]. Also, we are going to learn not the target value function itself, but the target value function
normalized by q0 , which sufficiently reduces the order of values. For strategy network, we are going to
learn the proportion of the inventory to be sent and not the volume itself. And finally, we can notice
that for high inventories the difference between value functions (which are quadratic in the inventory)
in the supremum can become much more important than the profit of the trader coming from the tick
(which is not more than linear in inventory). This fact can hinder us from finding optimal values for
the limit to which the trader should send his order, especially for small inventories. We normalize the
values of the optimized function for different inventories to make small inventories more important
by multiplying all values by 1q . However, this latter normalization is used when we optimize over the
part of the strategy responsible for the limits only, leaving volume updates untouched.
To summarize in Figures 4 and 5, we presented the structures of the neural networks used to represent
the approximators for the strategy and the value function. Another feature worth mentioning here
is the separation of market state and inventory inputs for some layers, both for the strategy and the
value function. This allows capturing features of the market state independently of the inventory.
Also, we separated some layers preceding the outputs of the strategy network to be able to perform
the learning process with different learning rates for volumes and limits of limit orders.
imbalances: InputLayer spreads: InputLayer inventory: InputLayer
imbalances: InputLayer spreads: InputLayer inventory: InputLayer

spreads_and_imbalances: Concatenate all_inputs: Concatenate
spreads_and_imbalances: Concatenate all_inputs: Concatenate

hidden_layer_1.1: Dense hidden_layer_1.2: Dense
hidden_layer_2.1: Dense hidden_layer_2.2: Dense hidden_layer_1.1: Dense hidden_layer_1.2: Dense
outputs_2.1_and_2.2: Concatenate
outputs_1.1_and_1.2: Concatenate
hidden_layer_3.1: Dense hidden_layer_3.2: Dense

hidden_layer_2: Dense
hidden_layer_4.1: Dense limits_venue_2: Dense limits_venue_1: Dense

value_function: Dense
volumes: Dense
Figure 5: Neural network structure for the

value function.
Figure 4: Neural network structure for the
trader’s strategy.
While the finite difference schemes must complete the entire recalculation of values for the whole
grid every time the trader wants to adapt his strategy using the updated market parameters, neural
networks can be adapted progressively, starting from some pre-trained strategy, for example, the one
corresponding to the previous parameters. In practice, a pre-trained model can be reused for different
problem settings due to normalization. Therefore a long and elaborate training procedure should
be done only once. The resulting model can be ameliorated by small adjustment trainings which
18
take only 1 minute on the simplest instance of the AWS platform (2CPU, no GPU), and have great
speed-up potential when performed on more complex infrastructures.
5.3 Two identical venues

We assume that the trader is confident about his estimation of σ. Therefore he uses Bayesian update
only on the drift µ of the asset. The venues share identical parameters, which will be inferred by the
trader through time.
5.3.1 Value function

We first plot in Figures 6 and 7 the evolution through time of the value function of the trader in the
state ψ 1 = ψ 2 = 1 and I 1 = I 2 = 0 during a slice of execution, obtained through finite difference
method.
Value function at different time steps ψ1 = ψ2 = 1 and I1 = I2 = 0 Value function at different time steps ψ1 = ψ2 = 1 and I1 = I2 = 0
0 0
−10000
−10000
−20000
−30000
−20000
−40000
−50000 t=0.4 −30000 t=0.9
t=0.3 t=0.8
−60000 t=0.2 t=0.7
t=0.1 −40000 t=0.6
−70000 t=0.0 t=0.5
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 6: Value function with respect to the Figure 7: Evolution of the value function v be-
inventory between t = 0 and t = 0.4. tween t = 0.5 and t = 0.9.
The parabolic form of the value function comes from the term g(q − qt⋆ ) in (2.3). The maximum value
indicates the optimal inventory for the next step in the slice. When t increases, the maximum shifts
toward zero, which means that the trader wants to finish the execution at the end of the slice.
We plot in Figures 8 and 9 the value function of (2.3) obtained using neural networks. We can see
that the neural networks approximate accurately the value function.
0 0
−10000
−10000
−20000
−30000 −20000
−40000
t=0.4 t=0.9
t=0.3 −30000 t=0.8
−50000
t=0.2 t=0.7
−60000 t=0.1 t=0.6
t=0.0 −40000 t=0.5
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 8: Evolution of the value function v be- Figure 9: Evolution of the value function v be-
tween t = 0 and t = 0.4 using neural networks. tween t = 0.5 and t = 0.9 using neural networks.
19
Next, we plot the strategy (in terms of limits and volumes) of the trader in both venues, using finite
difference schemes.
5.3.2 Strategy: limit orders and volumes with finite difference schemes
In Figures 10 and 11, we plot the limits at which the trader posts his limit orders in the two venues,
given equal spread and imbalance processes.
Limits at different time steps when ψ1 = ψ2 = 1 and I1 = I2 = 0 venue 1 Limits at different time steps when ψ1 = ψ2 = 1 and I1 = I2 = 0 venue 2
1.00 t=0.9 1.00 t=0.9
t=0.6 t=0.6
0.75 0.75
t=0.3 t=0.3
0.50 t=0.0 0.50 t=0.0
0.25 0.25
0.00 0.00
−0.25 −0.25
−0.50 −0.50
−0.75 −0.75
−1.00 −1.00
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 10: Limit strategy in the first venue, Figure 11: Limit strategy in the second venue,
ψ 1 = ψ 2 = δ, I 1 = I 2 = 0. ψ 1 = ψ 2 = δ, I 1 = I 2 = 0.
As the trader has the same prior distribution in the two venues, his strategy is the same in both venues.
At the beginning of the slice, i.e. at t = 0, the maximum of the value function is near q = 32000.
Therefore, if the trader has a lower inventory, he does not post any orders and wait for the next time
step. If he has a higher inventory, he tries to reach q = 32000 inventory. For q ∈ [32000, 34000], being
sufficiently close to the next step optimal inventory, he posts limit orders on the second best limit
to collect an additional tick. For q ∈ [34000, 40000], he posts at the first best limit to increase his
probability of execution. If he has q > 40000, he creates a new best limit and accepts to loose one
tick in order to be executed faster and reach the optimal inventory at the following time step. We
can see in this behavior the trade-off between the possibility of being executed at a more favorable
price and the necessity to complete the execution.
For the sake of homogeneity (for all M market states, the trader faces similar trade-off), we considered
the same set of controls for the limit where the trader can send his order. For this reason, we can
see that even for the spread equal to δ the trader can submit an order to the limit p = −1, which in
practice can obviously be treated as p = 0 due to piecewise monotonous nature of the optimal limit
strategy (which is, in fact, monotonous, though it cannot be reflected by finite differences when the
optimal volume equals to 0).
When the trader is near the end of the slice, he starts posting limit orders earlier (can be seen if both
volumes and limits are considered). For example if t = 0.6, he begins to trade at the second best limit
when q ∈ [8000, 11000], at the first best limit when q ∈ [11000, 19000], and creates a new best limit
when q ∈ [19000, 50000]. Therefore, if the trader still has a very positive inventory at the end of the
slice, he prefers to sacrifice one tick at the first best limit in order to complete his execution at this step.
20
It is important to highlight the fact that, when t = 0.9, the trader does not rush to liquidate his
inventory completely. This comes from the absence of a terminal penalty, often used in optimal liq-
uidation problem to guarantee the complete execution of the inventory. It enables in some sense to
“relax” the optimal execution framework on a slice, as the part of the inventory that has not been
executed during one slice is split between the remaining ones.
We plot in Figures 12 and 13 the volumes posted in both venues, for the same spread and imbalance.
We see that, at the beginning of the slice, the trader begins to post a nonzero volume only when
q > 32000. Moreover, he posts a higher volume when he is near the end of the slice.
Volumes at different time steps when ψ1 = ψ2 = 1 and I1 = I2 = 0 venue 1 Volumes at different time steps when ψ1 = ψ2 = 1 and I1 = I2 = 0 venue 2
20000 t=0.9 t=0.9
t=0.6 17500 t=0.6
t=0.3 t=0.3
t=0.0 15000 t=0.0
15000
12500
10000
10000
7500
5000 5000
2500
0 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 12: Volume sent to the first venue, Figure 13: Volume sent to the second venue,
ψ 1 = ψ 2 = δ, I 1 = I 2 = 0. ψ 1 = ψ 2 = δ, I 1 = I 2 = 0.
s When the second venue has a higher spread, we plot the strategy of the trader in both venues in
Figures 14 and 15.
Limits at different time steps when ψ1 = 1, ψ2 = 2 and I1 = I2 = 0 Volumes at different time steps when ψ1 = 1, ψ2 = 2 and I1 = I2 = 0
1.00 t=0.5, venue 1 t=0.5, venue 1
t=0.1, venue 1 t=0.1, venue 1
0.75
t=0.5, venue 2
20000 t=0.5, venue 2
0.50 t=0.1, venue 2 t=0.1, venue 2
0.25 15000
0.00
10000
−0.25
−0.50
5000
−0.75
−1.00 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 14: Limit strategy, ψ 1 = δ, ψ 2 = 2δ, Figure 15: Volume strategy, ψ 1 = δ, ψ 2 = 2δ,
I 1 = I 2 = 0. I 1 = I 2 = 0.
For t = 0.5, we see in Figure 14 that the trader starts to post at the second best limit in the second
venue when q = 10000 and in the first when q = 11000. For q ∈ [18000, 21000], he creates a new best
limit in the second venue to execute his inventory faster but keeps posting at the best limit in the
first venue in order to collect a higher spread. Finally, for q ∈ [30000, 50000], he stops posting in the
second venue in order to consume more liquidity in the first one where the probability of getting his
order filled is higher. Similar interpretations apply for t = 0.1.
21
In Figure 15, we see that the trader posts a higher volume in the first venue compared to the second
one. For t = 0.5, he starts to trade at q = 10000 for the second venue and at q = 11000 for the first
one. The volume posted in the first venue increases almost linearly with respect to the inventory. In
contrast, the volume posted in the second venue increases until an inventory of q = 22000, then stays
constant until q = 30000 and decreases to zero afterward. This means that for q ∈ [10000, 30000], the
trader prefers to collect the spread from both venues. When q > 30000, he prefers to stop posting in
the second venue, the one with a higher spread, in order to maximize his chances of being executed
in the first one. Similar interpretations apply for t = 0.1.
In Figures 16 and 17, we show the choice of limits and volumes of the trader if the imbalance is more
favorable in the second venue. in Figure 16, we observe for t = 0.5 that the trader posts in the first
venue at the second best limit for q ∈ [12000, 15000], at the first limit for q ∈ [15000, 20000] and at
a new best limit for q ∈ [20000, 32000]. At the same time, he posts in the first limit of the second
venue when q ∈ [12000, 22000] and at a new best limit for q ∈ [22000, 50000]. We see that the trader
prefers to post at a higher limit in the second venue because of the higher probability of execution
due to a more favorable imbalance. For large inventories, he stops posting in the first venue in order
to increase his probability of execution using limit orders in the second venue at a new best limit.
Same results hold for t = 0.1.
In Figure 17, we see that the trader posts a majority of his volume in the second venue due to a more
favorable imbalance. When his inventory is not too high, he collects the spread from both venues.
However, when his inventory is relatively high, he sends all the volume to the first venue in order to
increase the probability of filling.
Limits at different time steps when ψ1 = ψ2 = 1 and I1 = − 0.5, I2 = 0.5 Volumes at different time steps when ψ1 = ψ2 = 1 and I1 = 0.5, I2 = − 0.5
1.00 t=0.5, venue 1 t=0.5, venue 1
20000
0.75
0.50 t=0.1, venue 2 t=0.1, venue 2
15000
0.25
0.00 10000
−0.25
−0.50 5000
−0.75
−1.00 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 16: Limit order strategy, ψ = ψ = δ, 1 2

Figure 17: Volume strategy, ψ = ψ = δ, 1 2
I 1 = −0.5, I 2 = 0.5. I 1 = −0.5, I 2 = 0.5.
We now describe the strategies on the limits and the volumes obtained by a reinforcement learning
approach.
5.3.3 Strategy: limit orders and volumes with neural networks

We plot in Figures 18 and 19 the strategies on the limits used by the trader. As soon as limits are
represented by probabilities to send an order to each precise limit, for graphical representation, we
plot the limit corresponding to the highest of the three probabilities. We see that the choice of the
limits is in line with the ones of Figures 10 and 11 up to states where optimal order volume is at 0 (in
22
this case limit values are indistinguishable for finite differences). When the trader is at the beginning
of the slice, for a small inventory, he prefers to collect a higher spread by being executed at the second
best limit. When he is near the end of the slice, he prefers to be filled at a less favorable price, at
the best or new best limit, in order to lower his execution risk. We can also see that neural networks
preserve the monotonicity of the optimal limit function.
1.00 t=0.9 1.00 t=0.9
t=0.6 t=0.6
0.75 0.75
t=0.3 t=0.3
0.50 t=0.0 0.50 t=0.0
0.25 0.25
0.00 0.00
−0.25 −0.25
−0.50 −0.50
−0.75 −0.75
−1.00 −1.00
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 18: Limit order strategy in the first Figure 19: Limit order strategy in the second
venue, ψ 1 = ψ 2 = δ, I 1 = I 2 = 0 using neural venue, ψ 1 = ψ 2 = δ, I 1 = I 2 = 0 using neural
networks. networks.
In Figures 20 and 21, we plot the posted volumes of the trader in both venues for the same spread
and imbalance. We see that the strategy is a smoothed approximation of the one obtained using finite
differences in Figures 12 and 13. We see that at the very beginning of the slice, the trader is not going
to trade if his inventory is already small enough. The strategy in both venues is the same up to some
negligible numerical effects.
t=0.9 25000 t=0.9
20000 t=0.6 t=0.6
t=0.3 t=0.3
t=0.0 20000 t=0.0
15000
15000
10000
10000
5000 5000
0 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 20: Volume posted in the first venue, Figure 21: Volume posted in the second venue,
ψ 1 = ψ 2 = δ, I 1 = I 2 = 0 using neural net- ψ 1 = ψ 2 = δ, I 1 = I 2 = 0 using neural net-
works. works.
If the spread of the second venue is higher, we see in Figure 22 that the strategy with the limits
is the same as in Figure 14. It is interesting to note in Figure 23 that the trader does not stop
posting in the second venue, as in Figure 15, again because of the approximation coming from neural
networks. However, this behavior enables to perform some exploration of the venue parameters. For
example, if the trader follows the strategy given by finite differences in Figure 15, he posts a volume
equal to 0 in the second venue when q > 32000 for t = 0.5. However, if the trader underestimates
23
the prior on the filling probability in the second venue λ̂2 , he will keep sending orders in the first
venue, neglecting the possibility of splitting his orders which can potentially improve his execution.
Moreover, Figures 8 and 9 show that this slight difference in the obtained controls does not change
drastically the performance of the trader in terms of the value function.
1.00 t=0.5, venue 1 t=0.5, venue 1
0.75
t=0.5, venue 2 20000 t=0.5, venue 2
0.50 t=0.1, venue 2 t=0.1, venue 2
0.25 15000
0.00
10000
−0.25
−0.50 5000
−0.75
−1.00 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 22: Limit order strategy, ψ 1 = δ, ψ 2 = 2δ, Figure 23: Volume strategy, ψ 1 = δ, ψ 2 = 2δ,
I 1 = I 2 = 0 using neural networks. I 1 = I 2 = 0 using neural networks.
The same comments apply to Figures 24 and 25, where we see that the trader posts a small but
nonzero volume in the first venue with a less favorable imbalance which potentially allows to perform
exploration in this venue and faster improve parameter estimations.
1.00 t=0.5, venue 1 20000 t=0.5, venue 1
0.75
t=0.5, venue 2
17500 t=0.5, venue 2
0.50 t=0.1, venue 2 15000 t=0.1, venue 2
0.25 12500
0.00 10000
−0.25 7500
−0.50 5000
−0.75 2500
−1.00 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 24: Limit order strategy, ψ 1 = ψ 2 = δ, Figure 25: Volume strategy, ψ 1 = ψ 2 = δ,

I 1 = −0.5, I 2 = 0.5 using neural networks. I 1 = −0.5, I 2 = 0.5 using neural networks.
5.4 Two different venues

In this section, we analyze the behavior of the trader believing that the first venue is better than the
second venue in terms of filling rate. We compare the solutions obtained via finite difference schemes
and neural networks.
5.4.1 Value function

We show in Figures 26 and 27 the evolution of the value function of the trader during a slice of
execution, obtained through the finite difference method.
24
0 0
−10000
−10000
−20000
−30000 −20000
−40000
−30000
t=0.4 t=0.9
−50000
t=0.3 t=0.8
−60000 t=0.2 −40000 t=0.7
t=0.1 t=0.6
−70000 t=0.0 t=0.5
−50000
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 26: Evolution of the value function v Figure 27: Evolution of the value function v
between t = 0 and t = 0.4. between t = 0.5 and t = 0.9.
One can see that the value function deteriorates compared to the previous example, which is pre-
dictable in view of the fact that one of the venues is exactly like in the above example, and another
one is worse in terms of filling ratio. For example in Figure 27, the minimum of the function v at
t = 0.5 when q = 50000 is −49000 compared to a minimum of −45000 in the example above. This
is a natural consequence of a worse prior distribution on the filling ratio of the second venue while
keeping the prior on the first venue unchanged.
0 0
−10000
−10000
−20000
−20000
−30000
−40000 −30000
t=0.4 t=0.9
t=0.3 t=0.8
−50000 t=0.2 t=0.7
−40000
t=0.1 t=0.6
−60000 t=0.0 t=0.5
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
Figure 28: Evolution of the value function v Figure 29: Evolution of the value function v
between t = 0 and t = 0.4 using neural net- between t = 0.5 and t = 0.9 using neural net-
works. works.
We check in Figures 28 and 29 that we obtain a similar shape for the value function using neural
networks.
We now describe the strategy of the trader on the limits and the posted volumes and compare it to
the case of two identical venues.
5.4.2 Strategy: limit orders and volumes with finite difference schemes
In Figures 30 and 31, we show the limit order strategy of the trader in the two venues for the same
spreads and imbalances. As the second venue is less favorable for execution, the trader prefers to
create a new best limit for smaller inventories. For example, when t = 0.6, he posts an order on the
new best limit starting from q = 19000, and in the second venue, he prefers to create a new limit
25
starting from q = 18000. Generally, either at the beginning or at the end of the slice, the trader
prefers to post at a lower limit in the second venue in order to increase his execution rate there,
sacrificing the spread that could have been collected.
1.00 t=0.9 1.00 t=0.9
t=0.6 t=0.6
0.75 0.75
t=0.3 t=0.3
0.50 t=0.0 0.50 t=0.0
0.25 0.25
0.00 0.00
−0.25 −0.25
−0.50 −0.50
−0.75 −0.75
−1.00 −1.00
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
venue, ψ 1 = ψ 2 = δ, I 1 = I 2 = 0. venue, ψ 1 = ψ 2 = δ, I 1 = I 2 = 0.
The strategy of the trader differs drastically in terms of order volumes. In Figures 32 and 33, we
see that the trader posts the majority of his volume in the first venue. Especially when at t = 0.9
the trader stops posting in the second venue to reduce his liquidity consumption and maximize his
probability of execution in the first venue.
40000
t=0.9 t=0.9
35000 t=0.6 10000 t=0.6
t=0.3 t=0.3
30000 t=0.0 t=0.0
8000
25000
20000 6000
15000 4000
10000
2000
5000
0 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
ψ 1 = ψ 2 = δ, I 1 = I 2 = 0. ψ 1 = ψ 2 = δ, I 1 = I 2 = 0.
In Figures 34 and 35, we see the limits and the volumes recommended to the trader when the second
venue has a higher spread, and the imbalances are equal. The trader posts an even smaller volume in
the second venue, compared to Figure 15. As the filling rate is lower in the second venue, the trader
decreases his liquidity consumption in this venue, because of the smaller probability of collecting a
higher spread.
The strategy on the limits in Figure 34 is also different from the one in Figure 14. When t = 0.5
and the two venues are the same, the trader posts at the second best limit in the second venue when
q ∈ [11000, 13000], then at the first best limit when q ∈ [13000, 18000] and at a new best limit for
q ∈ [18000, 30000]. When the venues are different, the trader posts at the second best limit in the
26
second venue for q ∈ [10000, 12000], at the first best limit for q ∈ [12000, 17000] and at a new best
limit when q ∈ [17000, 19000]. Therefore, when the second venue has a worse filling rate, the trader
posts in the second venue earlier (for a higher inventory) and less compared to the case with two
equivalent venues.
1.00 t=0.5, venue 1 t=0.5, venue 1
0.75
t=0.5, venue 2 20000 t=0.5, venue 2
0.50 t=0.1, venue 2 t=0.1, venue 2
0.25 15000
0.00
10000
−0.25
−0.50 5000
−0.75
−1.00 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
I 1 = I 2 = 0. I 1 = I 2 = 0.
If the imbalance is more favorable in the second venue, we see in Figures 36 and 37 that the strategy is
very different from the one in Figures 16 and 17 where the two venues shared the same characteristics.
As the second venue has a more favorable imbalance, the trader posts a higher volume in it. However,
he posts a nonzero volume in the first venue, because of the overall better filling ratio. This contrasts
with Figure 17 where at some sufficiently high inventories, the trader stops sending orders to the first
venue. Due to the trade-off between an overall higher filling ratio in the first venue and a more favor-
able imbalance in the second venue, the trader splits his liquidity consumption between the two venues.
The strategy on the limits in Figure 36 also differs from the one with two identical venues in Fig-
ure 16. For t = 0.5 in Figure 16, the trader posts in the first venue at the second best limit
for q ∈ [10000, 13000], at the first best limit for q ∈ [13000, 20000] and at a new best limit for
q ∈ [20000, 32000]. In Figure 36, the trader posts in the first venue at the second best limit for
q ∈ [10000, 12000], at the first best limit for q ∈ [12000, 18000] and at a new best limit for q > 18000.
Therefore, he posts at a more favorable limit in terms of filling rate in the first venue in order to
compensate for the unfavorable imbalance compared to the second venue.
1.00 t=0.5, venue 1 t=0.5, venue 1
t=0.1, venue 1 17500 t=0.1, venue 1
0.75
t=0.1, venue 2 15000 t=0.1, venue 2
0.50
0.25
12500
0.00 10000
−0.25 7500
−0.50 5000
−0.75 2500
−1.00 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000

I 1 = −0.5, I 2 = 0.5. I 1 = −0.5, I 2 = 0.5.
27
Before moving to the analysis of the effectiveness of the Bayesian update of market parameters, we
conclude with a comparison of the strategies obtained via neural networks optimization.
5.4.3 Strategy: limit orders and volumes with neural networks

We observe in Figures 38 and 39 that the strategy of the trader on the limits is in line with the one
in Figures 30 and 31 up to the states where the optimal volume of the order equals 0.
1.00 t=0.9 1.00 t=0.9
t=0.6 t=0.6
0.75 0.75
t=0.3 t=0.3
0.50 t=0.0 0.50 t=0.0
0.25 0.25
0.00 0.00
−0.25 −0.25
−0.50 −0.50
−0.75 −0.75
−1.00 −1.00
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
venue, ψ 1 = ψ 2 = δ, I 1 = I 2 = 0 using neural venue, ψ 1 = ψ 2 = δ, I 1 = I 2 = 0 using neural
networks. networks.
In Figures 40 and 41, we see that the strategy of the trader on the posted volumes is well approximated
and smoothed by neural networks.
50000 t=0.9 t=0.9
t=0.6 10000 t=0.6
t=0.3 t=0.3
40000
t=0.0 t=0.0
8000
30000
6000
20000
4000
10000 2000
0 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
ψ 1 = ψ 2 = δ, I 1 = I 2 = 0 using neural net- ψ 1 = ψ 2 = δ, I 1 = I 2 = 0 using neural net-
works. works.
In Figures 42 and 43, we see in the case of a higher spread in the second venue that, because of
neural network parametrization of the strategy, the trader posts a nonzero volume in the second
venue leaving the possibility to better explore the filling ratios. Results are in line with the ones in
Figures 34 and 35: the trader posts the majority of his volume in the first venue because of a lower
spread and a more favorable filling ratio.
28
1.00 t=0.5, venue 1 t=0.5, venue 1
0.75 20000
0.50 t=0.1, venue 2 t=0.1, venue 2
0.25 15000
0.00
10000
−0.25
−0.50 5000
−0.75
−1.00 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000
I 1 = I 2 = 0 using neural networks. I 1 = I 2 = 0 using neural networks.
Finally, we show in Figures 44 and 45 a similar behavior compared to the finite difference schemes
in Figures 36 and 37: the trader posts a higher volume in the second venue due to a more favorable
imbalance, and keeps posting in the first venue due to an overall more favorable filling ratio.
1.00 t=0.5, venue 1 17500 t=0.5, venue 1
0.75
t=0.5, venue 2 15000 t=0.5, venue 2
0.50 t=0.1, venue 2 t=0.1, venue 2
12500
0.25
10000
0.00
7500
−0.25
5000
−0.50
−0.75
2500
−1.00 0
0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 50000

I 1 = −0.5, I 2 = 0.5 using neural networks. I 1 = −0.5, I 2 = 0.5 using neural networks.
5.5 Bayesian update

In this section, we analyze the effectiveness of the Bayesian update framework through several exe-
cution slices.
5.5.1 Market simulation on a slice

We first show an example of a market simulation of one slice and demonstrate the trading strategy
through the slice, which are illustrated in Figure 46.
At t = 0.2, the spread in both venues is equal to δ, with an unfavorable imbalance in both venues. In
that case, as two venues share the same characteristics, and the inventory is sufficiently close to the
optimal for the next step, so the trader sends the same quantity to both venues, which is close to zero.
29
When t = 0.5, the first venue has an unfavorable imbalance, and the second venue has a higher spread.
In this configuration, the trader sends a higher volume in the first venue, in order to get a better
filling rate due to a lower spread.
Finally, at t = 0.7, the first venue has a higher spread and a more favorable imbalance compared to
the second venue. This leads to a higher volume in the first venue at the second best limit and a lower
volume in the second venue at first best limit. The favorable imbalance in the first venue indicates
a higher probability of execution for an order at a higher limit, because the price may move in this
direction. Therefore, even if the spread is equal to two ticks, the trader posts in this venue in order to
be executed at a more favorable price. As the spread in the second venue is lower, but the imbalance
is less favorable, he posts at the first best limit to benefit from the trade-off between execution and
profit through collecting the spread.
Figure 46: Market simulation: spreads (upper left), imbalances (upper right), volumes (lower left)
and limits (lower right) in both venues.
The corresponding execution trajectory is shown in Figure 47, where we can see the typical Imple-
mentation Shortfall execution shape, coming from the pre-computed trading curve q ⋆ .
Number of shares q
50000
40000
30000
20000
10000
0.0 0.2 0.4 0.6 0.8 1.0
t
Figure 47: Evolution of the inventory of the trader on a slice of execution.
30
5.5.2 Update of the execution proportion
We show how the trader updates the market parameters through observations and trading. The
update of the execution proportion is quite fast, as it can be seen in Figures 48 and 49 the good
estimation can be achieved after completing 1-2 slices. In this example we started from the correct
prior for the second venue and the inaccurate one for the first:
h i h i
ρ1 = 0.1 0.9 , ρ2 = 0.1 0.9 .
Bayesian updates of parameters ρ 1

ρ 1, 1 ρ 1, 2
estimation
real value
0.20 0.95
0.15 0.90
0.10 0.85
0.05 0.80
estimation
real value
0 250 500 750 1000 1250 1500 1750 2000 0 250 500 750 1000 1250 1500 1750 2000
Figure 48: Bayesian update of the executed pro- Figure 49: Bayesian update of the executed pro-
portion in the first venue. portion in the second venue.
5.5.3 Update of the imbalance and the spread dynamics

We plot the convergence of the estimated transition matrices r 1,ψ , r 2,ψ in Figures 50 and 51. We
observe quite fast convergence to a good approximation of the spread dynamics parameters. The
prior values are respectively:
−5 5 −5 5
" # " #
r 1,ψ = , r 2,ψ = .
5 −5 5 −5
Bayesian updates of parameters r ψ for venue 1 Bayesian updates of parameters r ψ for venue 2
ψ ψ ψ ψ
rδ, δ rδ, 2δ rδ, δ rδ, 2δ
−3.25 5.00 −3.25
5.00 estimation estimation
−3.50 real value 4.75 real value −3.50
4.75
−3.75 4.50 4.50 −3.75
−4.00 4.25 4.25 −4.00
−4.25 4.00 4.00 −4.25
−4.50 3.75 3.75 −4.50
−4.75 estimation 3.50 −4.75 estimation
3.50
−5.00 real value real value
3.25 3.25 −5.00
0 250 500 750 1000 1250 1500 1750 2000 0 250 500 750 1000 1250 1500 1750 2000 0 250 500 750 1000 1250 1500 1750 2000 0 250 500 750 1000 1250 1500 1750 2000
ψ ψ ψ ψ
r2δ, δ r2δ, 2δ r2δ, δ r2δ, 2δ
5.00 −3.25 −3.25 5.00
estimation estimation
4.75 real value −3.50 −3.50 4.75 real value
4.50 −3.75 −3.75 4.50
4.25 −4.00 −4.00 4.25
4.00 −4.25 −4.25 4.00
3.75 −4.50 −4.50 3.75
3.50 −4.75 estimation −4.75 estimation 3.50

real value real value 3.25
3.25 −5.00 −5.00
0 250 500 750 1000 1250 1500 1750 2000 0 250 500 750 1000 1250 1500 1750 2000 0 250 500 750 1000 1250 1500 1750 2000 0 250 500 750 1000 1250 1500 1750 2000
Figure 50: Bayesian update of the transition ma- Figure 51: Bayesian update of the transition ma-
trix r 1,ψ . trix r 2,ψ .
We perform the same study for the transition matrices r 1,I , r 2,I of the imbalance processes through
the observed one in Figures 52 and 53. We see that we need just a couple of slices to have a quite good
approximation and only a dozen of slices (less for more granular slices) to achieve the right estimation.
31
Figure 52: Bayesian update of the transition matrix r 1,I .
Figure 53: Bayesian update of the transition matrix r 2,I .
32
In the examples in Figures 52 and 53 we started from the following prior parameters:
−5 2.8 2.2 −5 2.8 2.2

   
r 1,I 2.2 −5. 2.8 ,

= 
r 2,I =
 2.2 −5. 2.8
.
2.2 2.8 −5 2.2 2.8 −5
5.5.4 Update of the long term drift of the asset

As we observe the increments of the price process St continuously, it is easy to converge toward a real
market drift, the example is in Figure 54, we find µtrue = −0.5, starting from a prior of µ = 0.1. It
took 20 slices to find a real value even if in the considered example we supposed to be sure in our
prior estimation ν = 0.02, which appeared to be incorrect.
Bayesian updates of parameter μ
0.1 estimation
real value
0.0
−0.1
−0.2
−0.3
−0.4
−0.5
0 20 40 60 80 100
Figure 54: Bayesian update of the drift of the asset.
5.5.5 Update of the intensity of limit orders

The hardest parameter to update quickly is obviously the intensity of filling which depends on states
of both venues. In our numerical setting we have 32 possible states, so during one slice of 10 time
steps we have no possibility to even visit all the states. The results of convergence of the parameters
λ can be found in Figures 55 and 56. We see that full convergence requires a lot of observations,
however we should keep in mind that to have a strategy close to the optimal one we do not necessitate
an excessive precision.
In this example, we started from the priors same for both venues, whereas the real parameters are
different. The priors are:
5.35 6.52 7.11 8.28 10.03 10.9

   
λ1δ,δ = λ2δ,δ = 2.75 3.4 3.79 , λ1δ,2δ = λ2δ,2δ = 4.38 5.35 5.9  ,
   
1.5 1.86 2.1 2.5 3.05 3.4

1.81 2.27 2.5 2.96 3.65 4.
   
λ12δ,δ = λ22δ,δ = 0.78 1.04 1.19


, λ12δ,2δ = λ22δ,2δ = 1.42 1.81 2.04

.
0.29 0.43 0.53 0.68 0.9 1.04
33
Bayesian updates of parameters λ 1
λ 1, (δ, δ), (−0.5, − 0.5) λ 1, (δ, δ), ( 0.5, 0.0) λ 1, (δ, δ), ( 0.5, 0.5) λ 1, (δ, δ), (0.0, 0.5) λ 1, (δ, δ), (0.0, 0.0) λ 1, (δ, δ), (0.0, 0.5)
4.2
5.8 7.25 7.6 3.25
4.0 4.2
5.6 7.00 7.4 3.00 3.8
6.75 4.0
5.4 7.2 2.75 3.6
estimation estimation estimation estimation estimation estimation
real value 6.50 real value real value 2.50 real value 3.4 real value 3.8 real value
5.2 7.0
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 1, (δ, δ), (0.5, 0.5) λ 1, (δ, δ), (0.5, 0.0) λ 1, (δ, δ), (0.5, 0.5) λ 1, (δ, 2δ), ( 0.5, 0.5) λ 1, (δ, 2δ), ( 0.5, 0.0) λ 1, (δ, 2δ), ( 0.5, 0.5)
8.8 10.50
2.0
11.4
2.2 2.6 8.6
10.25
1.8 8.4 11.2
2.0 2.4 10.00
8.2 11.0
1.6
estimation estimation estimation estimation 9.75 estimation estimation
real value 1.8 real value 2.2 real value real value real value 10.8 real value
8.0
1.4
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 1, (δ, 2δ), (0.0, 0.5) λ 1, (δ, 2δ), (0.0, 0.0) λ 1, (δ, 2δ), (0.0, 0.5) λ 1, (δ, 2δ), (0.5, 0.5) λ 1, (δ, 2δ), (0.5, 0.0) λ 1, (δ, 2δ), (0.5, 0.5)
5.0 6.25 estimation 6.6 3.6 4.0
real value 3.0
4.8 6.00 6.4
3.4 3.8
6.2 2.8
4.6 5.75
3.2 3.6
4.4 estimation 6.0 estimation 2.6 estimation estimation estimation
5.50
real value real value real value real value real value
5.8 2.4 3.4
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 1, (2δ, δ), ( 0.5, 0.5) λ 1, (2δ, δ), ( 0.5, 0.0) λ 1, (2δ, δ), ( 0.5, 0.5) λ 1, (2δ, δ), (0.0, 0.5) λ 1, (2δ, δ), (0.0, 0.0) λ 1, (2δ, δ), (0.0, 0.5)
2.4
1.8
2.8 3.0 1.2
2.2 1.4
2.6 1.6
2.8
2.0 1.0
2.4 1.2 1.4
2.6
1.8 real value 2.2 real value real value real value real value 1.2 real value
2.4 0.8 1.0
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 1, (2δ, δ), (0.5, 0.5) λ 1, (2δ, δ), (0.5, 0.0) λ 1, (2δ, δ), (0.5, 0.5) λ 1, (2δ, 2δ), ( 0.5, 0.5) λ 1, (2δ, 2δ), ( 0.5, 0.0) λ 1, (2δ, 2δ), ( 0.5, 0.5)
1.0 1.0 5.00
3.6 4.5
0.8 4.75
0.8 3.4
0.8 4.50
0.6 3.2 4.0
0.6 4.25
0.4 estimation estimation 0.6 estimation 3.0 estimation estimation estimation
real value real value real value real value 3.5 real value 4.00 real value
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 1, (2δ, 2δ), (0.0, 0.5) λ 1, (2δ, 2δ), (0.0, 0.0) λ 1, (2δ, 2δ), (0.0, 0.5) λ 1, (2δ, 2δ), (0.5, 0.5) λ 1, (2δ, 2δ), (0.5, 0.0) λ 1, (2δ, 2δ), (0.5, 0.5)
2.0 1.4 1.6
1.4
1.8 2.2 1.2
2.4 1.2 1.4
1.6 1.0
2.0 2.2 1.0
1.2
estimation estimation estimation 0.8 estimation estimation estimation
1.4 real value real value real value real value real value real value
1.8 0.8
2.0
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
Figure 55: Bayesian update of the intensity of limit orders in the first venue.
34
Bayesian updates of parameters λ 2
λ 2, (δ, δ), (−0.5, 0.5) λ 2, (δ, δ), (0.0, 0.5) λ 2, (δ, δ), (0.0, 0.0) λ 2, (δ, δ), (0.0, 0.5) λ 2, (δ, δ), (0.5, 0.5) λ 2, (δ, δ), (0.5, 0.0)
4.2
7.6 3.25 2.0
4.0 4.2
2.2
7.4 3.00 3.8 1.8
4.0 2.0
7.2 2.75 3.6 1.6
real value 2.50 real value 3.4 real value 3.8 real value real value 1.8 real value
7.0 1.4
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 2, (δ, δ), (0.5, 0.5) λ 2, (δ, 2δ), ( 0.5, 0.5) λ 2, (δ, 2δ), ( 0.5, 0.0) λ 2, (δ, 2δ), ( 0.5, 0.5) λ 2, (δ, 2δ), (0.0, 0.5) λ 2, (δ, 2δ), (0.0, 0.0)
8.8 10.50 5.0 6.25 estimation
11.4
2.6 8.6 real value
10.25 4.8 6.00
8.4 11.2
2.4 10.00 4.6 5.75
8.2 11.0
estimation estimation 9.75 estimation estimation 4.4 estimation 5.50
2.2 real value real value real value 10.8 real value real value
8.0
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 2, (δ, 2δ), (0.0, 0.5) λ 2, (δ, 2δ), (0.5, 0.5) λ 2, (δ, 2δ), (0.5, 0.0) λ 2, (δ, 2δ), (0.5, 0.5) λ 2, (2δ, δ), ( 0.5, 0.5) λ 2, (2δ, δ), ( 0.5, 0.0)
6.6 2.4
3.6 4.0
3.0 2.8
6.4 2.2
3.4 3.8 2.6
6.2 2.8
2.0 2.4
3.2 3.6
6.0 estimation 2.6 estimation estimation estimation estimation estimation
real value real value real value real value 1.8 real value 2.2 real value
5.8 2.4 3.4
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 2, (2δ, δ), ( 0.5, 0.5) λ 2, (2δ, δ), (0.0, 0.5) λ 2, (2δ, δ), (0.0, 0.0) λ 2, (2δ, δ), (0.0, 0.5) λ 2, (2δ, δ), (0.5, 0.5) λ 2, (2δ, δ), (0.5, 0.0)
1.0
1.8
3.0 1.2
1.4 0.8
1.6 0.8
2.8
1.0 0.6
2.6 1.2 1.4 0.6
estimation estimation estimation estimation 0.4 estimation estimation
2.4 real value 0.8 real value real value 1.2 real value real value real value
1.0
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 2, (2δ, δ), (0.5, 0.5) λ 2, (2δ, 2δ), ( 0.5, 0.5) λ 2, (2δ, 2δ), ( 0.5, 0.0) λ 2, (2δ, 2δ), ( 0.5, 0.5) λ 2, (2δ, 2δ), (0.0, 0.5) λ 2, (2δ, 2δ), (0.0, 0.0)
5.00 2.0
1.0 3.6 4.5
4.75 1.8 2.2
3.4
0.8 4.50
3.2 4.0 1.6 2.0
4.25
0.6 estimation 3.0 estimation estimation estimation estimation estimation
real value real value real value 4.00 real value 1.4 real value real value
3.5 1.8
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
λ 2, (2δ, 2δ), (0.0, 0.5) λ 2, (2δ, 2δ), (0.5, 0.5) λ 2, (2δ, 2δ), (0.5, 0.0) λ 2, (2δ, 2δ), (0.5, 0.5) λ 2, (δ, δ), ( 0.5, 0.5) λ 2, (δ, δ), ( 0.5, 0.0)
1.4 1.6
1.4 6.25 3.0
1.2 6.00
2.4 1.2 1.4 2.8
1.0 5.75
2.2 1.0 2.6
1.2
estimation 0.8 estimation estimation estimation 5.50 estimation estimation
real value real value 0.8 real value real value real value 2.4 real value
2.0 5.25
0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000 0 5000 10000 15000 20000
Figure 56: Bayesian update of the intensity of limit orders in the second venue.
35
Appendix A Proof of Theorem 2.3
It can be show with the dynamic programming principle that the HJBQVI (2.3) does not depend on
˜

the cash variable x. We set q̃, ψ̃, I ∈ D = Q × K and (ti , Si ) ∈ [0, T ) × R such that
ti →i→+∞ t̂,
Si →i→+∞ Ŝ,
˜ →i→+∞ v⋆ (t̂, q̃, Ŝ, ψ̃, I).
v(ti , q̃, Si , ψ̃, I) ˜
We begin with t̂ = T . By taking ℓn = 0 for all n ∈ {1, . . . , N} we get

Z T
˜ ≥E g(qt⋆ − qt )dt .
v(ti , q̃, Si , ψ̃, I) ti ,q̃,Si ,ψ̃,I˜ QT ST −
0
˜ ≥ q̃ Ŝ.
By dominated convergence, we get v⋆ (T, q̃, Ŝ, ψ̃, I)
Assume now that t̂ < T and that the minimum in the HJBQVI is given by the first term. We
take φ : [0, T ) × R × D → R be C 1 in time, C 2 in Ŝ and such that 0 = min[0,T ]×R×D (v⋆ − φ) =
˜ If there exists η > 0 such that
(v⋆ − φ)(t̂, q̃, Ŝ, ψ̃, I).
˜ 1
˜

2η <∂t φ(t̂, q̃, Ŝ, ψ̃, I)−g(q − qt⋆ )+µ∂S φ + σ 2 ∂SS φ+ ψ ,kI ) φ(t̂, q̃, Ŝ, k , k )−φ(t̂, q̃, Ŝ, ψ̃, I)
ψ I
X
r(ψ̃,I),(k
˜
2 k∈K
N
ψ̃ n

˜ pn , ℓ)E ǫn ℓn (Ŝ + + pn δ n ) + φ t̂, q̃ − ℓn ǫn , Ŝ, ψ̃, I˜ − φ(t̂, q̃, Ŝ, ψ̃, I)
˜ ,

+ sup λn (ψ̃, I,
X
we should have
˜ − g(q − q ⋆ ) + µ∂S φ + 1 ∂SS φ+ r ˜ ψ I φ(t, q̃, S, kψ , kI )−φ(t, q̃, S, ψ̃, I)
˜

0 ≤∂t φ(t, q̃, S, ψ̃, I)
X
t (ψ̃,I),(k ,k )
2 k∈K
N n
˜ pn , ℓ)E ǫn ℓn (S + ψ̃ + pn δ n ) + φ t, q̃ − ℓn ǫn , S, ψ̃, I˜ − φ(t, q̃, S, ψ̃, I)

˜ ,

+ sup λn (ψ̃, I,
X

for all (t, S) ∈ B = (t̂ − r, t̂ + r) ∩ [0, T ) × Ŝ − r, Ŝ + r for a given r ∈ (0, T − t̂). We can assume
without loss of generality that B contains the sequences (ti , Si )i and, by taking η arbitrarily small
˜ + η ≤ v⋆ (t, q̃, S, ψ̃, I)
φ(t, q̃, S, ψ̃, I) ˜ ≤ v(t, q̃, S, ψ̃, I)
˜
on the boundary of B, denoted by ∂p B. Without loss of generality we can also assume that
φ(t, q, S, ψ, I) + η ≤ v⋆ (t, q, S, ψ, I) ≤ v(t, q, S, ψ, I),
for (t, q, S, ψ, I) ∈ B̃ where

B̃ = (t, q, S, ψ, I) : (t, S) ∈ B, q ∈ {q̃ − min ǫn , q̃, q̃ + min ǫn },
n n
N N
{ψ̃ n − δ n , ψ̃ n , ψ̃ n + δ n }, I ∈ {I˜−1
n ñ ñ ˜ .
, I , I+1 }, (q, ψ, I) 6= (q̃, ψ̃, I)
Y Y
ψ∈
n=1 n=1
36
We introduce the set

˜ : (t, S) ∈ B ,
n
BD = (t, q̃, S, ψ̃, I)
˜
and denote by τi the first exit time of (t, qt , St , ψt , It )t≥ti from BD , with qti = q̃, Sti = Ŝ, ψti = ψ̃, Iti = I,
and the processes are controlled by the optimal controls (ℓ , p )n∈{1,...,N } ∈ A × Qψ . By Ito’s formula,
n n
we get
Z τi
φ(τi , qτi ,Sτi , ψτi , Iτi ) = φ(ti , qti , Sti , ψti , Iti ) + ∂t φ(s, qs , Ss , ψs , Is ) + µ∂S φ
ti
1
+ σ 2 ∂SS φ + r(ψs ,Is ),(kψ ,kI ) φ(s, qs , Ss , kψ , kI ) − φ(s, qs , Ss , ψs , Is )
X
2 k∈K
N
+ n
(ψs , Is , pns , ℓs )E ℓns ǫns , Ss , ψs , Is − φ(s, qs , Ss , ψs , Is ) ds + M(τi , ti ),
X
λ φ s, qs −
n=1
where M(τi , ti ) is a martingale. This can be rewritten as

Z τi
φ(τi , qτi ,Sτi , ψτi , Iτi ) = φ(ti , qti , Sti , ψti , Iti ) + ∂t φ(s, qs , Ss , ψs , Is ) + µ∂S φ − g(qs − q ⋆ (s))
ti
1
+ σ 2 ∂SS φ + r(ψs ,Is ),(kψ ,kI ) φ(s, qs , Ss , kψ , kI ) − φ(s, qs , Ss , ψs , Is )
X
2 k∈K
N
ψn

+ n
(ψs , Is , pns , ℓs )E ǫns ℓns (Ss + s + pns δ n )+φ s, qs −ℓns ǫns , Ss , ψs , Is −φ(s, qs , Ss , ψs , Is ) ds
X
λ
n=1 2
N Z τi h ψsn i
+ M(τi , ti ) − λn (ψs , Is , pns , ℓs )E ǫns ℓns (Ss + + pns δ n ) + g(qs − q ⋆ (s))ds.
X
n=1 ti 2
We derive
φ(τi , qτi , Sτi , ψτi , Iτi ) ≥φ(ti , qti , Sti , ψti , Iti )
N Z τi h ψsn i
+ M(τi , ti ) − λn (ψs , Is , pns , ℓs )E ǫns ℓns (Ss + + pns δ) + g(qs − q ⋆ (s))ds.
X
n=1 ti 2
As the martingale term vanishes with the expectation, we get

φ(ti , qti , Sti , ψti , Iti ) ≤E φ(τi , qτi , Sτi , ψτi , Iτi )
N Z τi ψn
h i
+ n
(ψs , Is , pns , ℓs )E ǫns ℓns (Ss + s + pns δ) − g(qs − q ⋆ (s))ds .
X
λ
n=1 ti 2
and thus

φ(ti , qti , Sti , ψti , Iti ) ≤ − η + E v(τi , qτi , Sτi , ψτi , Iτi )
N Z τi ψsn
h i
+ λn (ψs , Is , pns , ℓs )E ǫns ℓns (Ss + + pns δ) − g(qs − q ⋆ (s))ds .
X
n=1 ti 2
37
For i sufficiently large, we deduce
˜ ≤ − η + E v(τi , qτ , Sτ , ψτ , Iτ )

v(ti , q̃, Sti , ψ̃, I)
2 i i i i
N Z τi ψn
h i
+ n
(ψs , Is , pns , ℓs )E ǫns ℓns (Ss + s + pns δ) − g(qs − q ⋆ (s))ds ,
X
λ
n=1 ti 2
which contradicts the dynamic programming principle. In conclusion, we necessarily have
˜ 1
˜

0 ≥∂t v(t̂, q̃, Ŝ, ψ̃, I)−g(q − qt⋆ )+µ∂S v+ σ 2 ∂SS v+ ψ ,kI ) v(t̂, q̃, Ŝ, k , k )−v(t̂, q̃, Ŝ, ψ̃, I)
ψ I
X
r(ψ̃,I),(k
˜
2 k∈K
N n
˜ pn , ℓ)E ǫn ℓn (Ŝ + ψ̃ + pn δ n ) + v t̂, q̃ − ℓn ǫn , Ŝ, ψ̃, I˜ − v(t̂, q̃, Ŝ, ψ̃, I)

˜ .

+ sup λ (ψ̃, I,
n
X
The second part of the HJBQVI being straightforward, we prove that v is a viscosity supersolution
of the HJBQVI on [0, T ) × R × D. The proof for the subsolution is identical, except that we need to
prove
N
ψn
sup mn (S − ) + v t, q − mn , S, ψ, I − v(t, q, S, ψ, I) ≥ 0,
X
n
n=1 m ∈[0,m] 2
which is direct by choosing the constant controls mn = 0 for all n ∈ {1, . . . , N}.
For the proof of the uniqueness, we recall the definition of subjet and superjet.
Definition Appendix A.1. Let v : [0, T ) × R × D → R be l.s.c (resp u.s.c) with respect to (t̂, Ŝ).
˜ ∈ [0, T ) × R × D we say that (y, p, A) ∈ R3 is in the subjet P − v(t̂, q̃, Ŝ, ψ̃, I)
For (t̂, q̃, Ŝ, ψ̃, I) ˜ (resp.
+ ˜
the superjet P v(t̂, q̃, Ŝ, ψ̃, I) if
˜ + y(t − t̂) + p(S − Ŝ) + 1 A(S − Ŝ)2 + o(|t − t̂| + |S − Ŝ|2 ),

˜ ≥ v(t̂, q̃, Ŝ, ψ̃, I)
v(t, q̃, S, ψ̃, I)
2
˜ ≥ v(t̂, q̃, Ŝ, ψ̃, I)
(resp. v(t, q̃, S, ψ̃, I) ˜ + y(t − t̂) + p(S − Ŝ) + 1 A(S − Ŝ)2 + o(|t − t̂| + |S − Ŝ|2 )), for
2
˜ ∈ [0, T ) × R × D. We also define P − (t̂, q̃, Ŝ, ψ̃, I)
all (t, S) such that (t, q̃, S, ψ̃, I) ˜ as the set of points
(y, p, A) ∈ R such that there exists a sequence (tI , q̃, Si , ψ̃, I,
3 ˜ yi, pi , Ai ) satisfying
˜ yi , pi , Ai ) →i→+∞ (t̂, q̃, Ŝ, ψ̃, I,

(ti , q̃, Si , ψ̃, I, ˜ y, p, A).
+
˜ is defined similarly.
The set P (t̂, q̃, Ŝ, ψ̃, I)
We now introduce an analogous of the Ishii’s lemma, whose proof can be found in [11].
Lemma Appendix A.2. A l.s.c (resp u.s.c) function v is a supersolution (resp. subsolution) of the
˜ ∈ [0, T ) × R × D, and all (ŷ, p̂, Â) ∈
HJBQVI on [0, T ) × R × D if and only if for all (t̂, q̃, Ŝ, ψ̃, I)
− ˜ + ˜
P (t̂, q̃, Ŝ, ψ̃, I) (resp. P (t̂, q̃, Ŝ, ψ̃, I)), we have

 1
˜

0 ≤ min − ŷ + g(q̃ − q ⋆ (t̂)) − µp̂ − σ 2 Â − ψ I
)
X
r(ψ̃,I),(k
˜ ψ I
,k ) v( t̂, q̃, Ŝ, k , k − v( t̂, q̃, Ŝ, ψ̃, I)
 2 k∈K
38
N
ψ̃ n

sup ˜ pn , ℓ)E ǫn ℓn (Ŝ +
λn (ψ̃, I, ˜
+ pn δ n ) + v(t̂, q̃ − ℓn ǫn , Ŝ, ψ̃, I)
X
−

N n
˜ − sup mn (Ŝ − ψ̃ ) + v t̂, q̃ − mn , Ŝ, ψ̃, I˜ ,

˜ ;
X 
− v(t̂, q̃, Ŝ, ψ̃, I) v(t̂, q̃, Ŝ, ψ̃, I)
n=1 mn ∈[0,m] 2 
(resp. ≤ 0).
We now prove the following comparison principle:
Proposition Appendix A.3. Let u (resp. v) be a l.s.c supersolution (resp. u.s.c subsolution) with
polynomial growth of the HJBQVI on [0, T ) × R × D. If u ≥ v on {T } × R × D, then u ≥ v on
[0, T ) × R × D.
Proof. For ρ > 0 we introduce the following change of variables:
ũ(t, q, S, ψ, I) = eρt u(t, q, S, ψ, I), ṽ(t, q, S, ψ, I) = eρt v(t, q, S, ψ, I).
Then, ũ and ṽ are respectively supersolution and subsolution of the following equation:

 1
0 = min − ∂t w(t, q, S, ψ, I) + ρw(t, q, S, ψ, I) + g(q − qt⋆ ) − µ∂S w − σ 2 ∂SS w
 2

r(ψ,I),(kψ ,kI ) w(t, q, S, kψ , kI ) − w(t, q, S, ψ, I)
X
−
k∈K
N
ψn

sup λ (ψ, I, p , ℓ)E ǫn ℓn eρt (S +
n n
+ pn δ n ) + w t, q − ℓn ǫn , S, ψ, I
X
−

N
ψn

− w(t, q, S, ψ, I) ; w(t, q, S, ψ, I) − sup mn eρt (S − ) + w t, q − mn , S, ψ, I ,
X
n=1 mn ∈[0,m] 2 
on [0, T ) × R × D, with ũ ≥ ṽ on {T } × R × D. In order to prove the proposition, we only have to

show that ũ ≥ ṽ on [0, T ) × R × D. Assume that the minimum is given by the first term and that
sup[0,T )×R×D ṽ − ũ > 0. We fix p ∈ N⋆ such that
|ũ(t, q, S, ψ, I)| + |ṽ(t, q, S, ψ, I)|

lim sup = 0.
kSk2 →+∞ t∈[0,T ] 1 + kSk2p 2
(q,ψ,I)∈D
˜ ∈ [0, T ] × R × D such that

Then, there exists (t̂, q̃, Ŝ, ψ̃, I)
˜ − ũ(t̂, q̃, Ŝ, ψ̃, I)
0 < ṽ(t̂, q̃, Ŝ, ψ̃, I) ˜ − φ(t̂, q̃, Ŝ, Ŝ, ψ̃, I)
˜
= max ṽ(t, q, S, ψ, I) − ũ(t, q, S, ψ, I) − φ(t, q, S, S, ψ, I),
(t,q,S,ψ,I)
where ǫ > 0 is small enough and

φ(t, S, R) = ǫ exp(−κ̃t) 1 + kSk2p 2p
2 + kRk2 , κ̃ > 0.
39
Since ũ ≥ ṽ on {T } × R × D, we directly have t̂ < T .
For all i ∈ N, we can find a sequence (ti , Si , Ri ) such that

˜ − ũ(ti , q̃, Ri , ψ̃, I)
˜ − φ(ti , Si , Ri ) − i|Si − Ri |2 − |ti − t̂|2 + |Si − Ŝ|4

0 < ṽ(ti , q̃, Si , ψ̃, I)
˜ − ũ(t, q̃, R, ψ̃, I)
˜ − φ(t, S, R) − i|S − R|2 − |t − t̂|2 + |S − Ŝ|4 .

= max ṽ(t, q̃, S, ψ̃, I)
(t,S,R)
Then we have:
(ti , Si , Ri ) −→ (t̂, Ŝ, Ŝ)
i→+∞
up to a subsequence, and
˜ − ũ(ti , q̃, Ri , ψ̃, I)
˜ − φ(ti , Si , Ri ) − i|Si − Ri |2 − |ti − t̂|2 + |Si − Ŝ|4

ṽ(ti , q̃, Si , ψ̃, I)
˜ − ũ(t̂, q̃, Ŝ, ψ̃, I)
−→ ṽ(t̂, q̃, Ŝ, ψ̃, I) ˜ − φ(t̂, Ŝ, Ŝ)
n→+∞
Let us then denote for i ∈ N∗

ϕi (t, S, R) = φ(t, S, R) + i|S − R|2 + |t − t̂|2 + |S − Ŝ|4 ∀ (t, S, R) ∈ [0, T ] × R2 .
˜
Then Ishii’s Lemma (see [9, 14]) guarantees that for all η > 0, we can find (yi1 , p1i , A1i ) ∈ P̄ + ṽ(ti , q̃, Si , ψ̃, I)
˜ such that:
and (yi2, p2i , A2i ) ∈ P̄ − ũ(ti , q̃, Ri , ψ̃, I)
yi1 − yi2 = ∂t ϕi (ti , Si , Ri ), (p1i , p2i ) = (∂S ϕi , −∂R ϕi ) (ti , Si, Ri ),
and
0
!
A1i
≤ HSR ϕi (ti , Si , Ri ) + η (HSR ϕn (ti , Si, Ri ))2 ,
0 −A2i
where HSR ϕi (ti , ., .) denotes the Hessian matrix of ϕi (ti , ., .). Applying Lemma Appendix A.2, we get
˜ ≤ y 1 − y 2 + 1 σ 2 (A1 − A2 ) + µ(p1 − p2 )
˜ − ũ(ti , q̃, Ri , ψ̃, I)

ρ ṽ(ti , q̃, Si , ψ̃, I) i i i i i i
2
˜

+ r ˜ ψ I ṽ(ti , q̃, Si, kψ , kI ) − ṽ(ti , q̃, Si , ψ̃, I)
X
(ψ̃,I),(k ,k )
k∈K
N n
˜ pn , ℓ)E ǫn ℓn eρti (Si + ψ̃ + pn δ n ) + ṽ ti , q̃ − ℓn ǫn , Si , ψ̃, I˜ − ṽ(ti , q̃, Si , ψ̃, I)

˜

+ sup λ (ψ̃, I,
n
X
˜

r ˜ ψ I ũ(ti , q̃, Ri , kψ , kI ) − ũ(ti , q̃, Ri , ψ̃, I)
X
− (ψ̃,I),(k ,k )
k∈K
N
ψ̃ n

˜ pn , ℓ)E ǫn ℓn eρti (Ri + + pn δ n ) + ũ ti , q̃ − ℓn ǫn , Ri , ψ̃, I˜ − ũ(ti , q̃, Ri , ψ̃, I)
˜ .

sup λn (ψ̃, I,
X
−
Moreover, we have
!
∂ 2 φ(t , S , R ) + 2i + 12(Si − Ŝ)2 ∂SR
2
φ(ti , Si, Ri ) − 2i
HSR ϕi (ti , Si , Ri ) = SS i 2 i i ,
∂SR φ(ti , Si , Ri ) − 2i ∂SR φ(ti , Si , Ri ) + 2i
2
and
∂S ϕi (ti , Si , Ri ) = ∂S φ(ti , Si , Ri ) + 2i|Si − Ri | + 4|Si − Ŝ|3 ,
40
∂R ϕi (ti , Si , Ri ) = ∂R φ(ti , Si , Ri ) − 2i|Si − Ri |,
so from what precedes we can write

˜ − ũ(ti , q̃, Ri , ψ̃, I)
˜ ≤ ∂t φ(ti , Si, Ri ) + 2(ti − t̂) + µ ∂S φ(ti , Si , Ri ) + ∂R φ(ti , Si , Ri )

ρ ṽ(ti , q̃, Si , ψ̃, I)

+ 4(Si − Ŝ)3
1 2
+ σ 2 ∂SS φ(ti , Si , Ri ) + ∂RR 2 2
φ(ti , Si , Ri ) + 2∂SR φ(ti , Si , Ri ) + 12(Si − Ŝ) + ηCi
2
˜

+ ψ ,kI ) ṽ(ti , q̃, Si , k , k ) − ṽ(ti , q̃, Si , ψ̃, I)
ψ I
X
r(ψ̃,I),(k
˜
k∈K
N n
˜ pn , ℓ)E ǫn ℓn eρti (Si + ψ̃ + pn δ n ) + ṽ ti , q̃ − ℓn ǫn , Si, ψ̃, I˜ − ṽ(ti , q̃, Si , ψ̃, I)

˜

+ sup λ (ψ̃, I,
n
X
˜

r ˜ ψ I ũ(ti , q̃, Ri , kψ , kI ) − ũ(ti , q̃, Ri , ψ̃, I)
X
− (ψ̃,I),(k ,k )
k∈K
N
ψ̃ n

˜ pn , ℓ)E ǫn ℓn eρti (Ri + + pn δ n ) + ũ ti , q̃ − ℓn ǫn , Ri , ψ̃, I˜ − ũ(ti , q̃, Ri , ψ̃, I)
˜ ,

sup λn (ψ̃, I,
X
−
where Ci does not depend on η. As ṽ is u.s.c., ũ is l.s.c. and (ti , Si , Ri )i is convergent, when η → 0 it
is clear, that, when i → +∞, for a certain constant M we get
˜ ˜ ≤ ∂t φ(t̂, Ŝ, Ŝ) + µ ∂S φ(t̂, Ŝ, Ŝ) + ∂R φ(t̂, Ŝ, Ŝ)

ρ ṽ(t̂, q̃, Ŝ, ψ̃, I)−ũ( t̂, q̃, Ŝ, ψ̃, I)
1 2
+ σ 2 ∂SS 2
φ(t̂, Ŝ, Ŝ) + ∂RR 2
φ(t̂, Ŝ, Ŝ) + 2∂SR φ(t̂, Ŝ, Ŝ) + M.
2
For κ̃ > 0 large enough, the right-hand side is strictly negative, and as ρ > 0 we get
˜ − ũ(t̂, q̃, Ŝ, ψ̃, I)
ṽ(t̂, q̃, Ŝ, ψ̃, I) ˜ < 0,
which yields to a contradiction. The proof for the other part of the HJBQVI is direct.
With the two above propositions, it is easy to conclude the proof of the theorem. Indeed, as v⋆
is a supersolution such that v⋆ ≥ v on {T } × R × D, and v ⋆ is a subsolution such that v ⋆ ≤ v on
{T }×R×D, we can apply the maximum principle to get v⋆ ≥ v ⋆ on [0, T ]×R×D. But by definition of
v⋆ and v ⋆ , we must have v⋆ ≤ v ≤ v ⋆ on [0, T ] × R × D, which proves that we have v⋆ = v = v ⋆ and v is
continuous. The maximum principle implies that if two continuous viscosity solutions of the HJBQVI
satisfy the same terminal condition, they are equal on [0, T ] × R × D, hence the uniqueness.
41
Appendix B Application to OTC market making
Appendix B.1 Framework
The model we present in this article is designed for trading in cross-listed stocks in limit order books.
However, it can be adapted straightforwardly to handle the problem of an OTC market maker, who
often deals with a large number of assets driven by a few factors. We borrow here the factorial method
market making framework of [10] (we are also going to keep their notation only for this section). We
consider a market maker who is in charge of providing bid and ask quotes on d assets, whose dynamics
are
dSti = µi dt + σ i dWti , i ∈ {1, . . . , d},
where µi is the drift of the i-th asset, σ i is its volatility and (Wt1 , . . . , Wtd ) is a d-dimensional Brownian
motion. We consider a non-singular variance-covariance matrix Σ = (ρi,j σ i σ j )i,j∈{1,...,d} for the vector
of assets (St1 , . . . , Std ). The market maker sets bid and ask prices on every asset:
S i,b (t, z) = Sti − δ i,b(t, z), S i,a (t, z) = Sti + δ i,a (t, z), z ∈ R,
where δ = (δ i,a , δ i,b)i∈{1,...,d} are the (predictable and uniformly lower bounded) bid and ask quotes
around the mid-price of each asset. The volume of transactions on the bid and ask sides are modeled
by marked point processes N i,b (dt, dz), N i,a (dt, dz) of intensity νti,b (dz), νti,a (dz) defined by

νti,j (dt, dz) = Λi,j δ i,j (t, z) η i,j (dz), i ∈ {1, . . . , d},
where Λi,j is a sufficiently regular function (exponential, logistic, SU Johnson etc.) modeling the
probability to trade on the asset i, on the side j for a given spread δ i,j (t, z) and a size z. The
functions η i,j (dz) are probability densities over R+ modeling the distribution of a trade size. The
market maker manages his inventory process qt = (qt1 , . . . , qtd ) of dynamics given by
Z Z
dqti = zN (dt, dz) −
i,b
zN i,a (dt, dz), i ∈ {1, . . . , d}.
R+ R+
The market maker manages his cash process given at time t by

d Z Z
dXt = zS i,a (t, z)N i,a (dt, dz) − zS i,b (t, z)N i,b (dt, dz).
X
i=1 R+ R+
Its optimization problem is defined as

d Z T
sup E XT + qTi STi φ(qt )dt ,
X
−
δ i=1 0
where φ is a running penalty preventing from too large positions and di=1 qTi STi is the marked-to-
P
market value of the market maker’s portfolio at time t. The corresponding HJB equation is given
by
d d Z v(t, q) − v(t, q + zei )
0 =∂t v(t, q) + q µ − φ(q) +
i i i,b
η i,b (dz)
X X
zH
i=1 i=1 R+ z
d Z v(t, q) − v(t, q − zei )
+ zH i,a η i,a (dz),
X
i=1 R+ z
with terminal condition v(T, q) = 0, H i,j (p) = supδ Λi,j (δ)(δ − p), and (e1 , . . . , ed ) is the canonical
basis of Rd .
42
Appendix B.2 Bayesian update for OTC market makers
Usually, the functions Λi,j are of the form

Λi,j δ i,j (t, z) = λi,j
RFQ f δ (t, z) ,
i,j

where λi,j
RFQ is the constant intensity of arrival of requests for quote, and f δ (t, z) gives the prob-
i,j
ability that a request will result in a transaction given the quote δ proposed by the market maker.
The estimation of the quantity λi,j
RFQ is of particular importance for the market maker so that he can
adjust his quotes depending on his view on the number of request for a certain asset and a certain
side. In the same spirit as in Section 3.1.1, we assume the following prior distribution:
λi,j
RFQ ∼ Γ(α , β ),
i,j i,j
(αi,j , β i,j ) > 0.
αi,j
For an asset i ∈ {1, . . . , d} on the side j ∈ {a, b}, this corresponds to an average intensity of β i,j
,
αi,j
with variance equal to (β i,j )2
. If the market maker is confident in his estimation of the intensity
λi,j
RFQ ,he can choose a large β so that the variance of his Bayesian estimator is small. Given all the
i,j
information accumulated up to time t, its best estimation of the quantity λi,j

RFQ , is given by
αi,j +
R
R+ N(t, dz)
h i
E λi,j
RFQ |N(t, dz) = Rt . (Appendix B.1)
β i,j + f (δ i,j (s, z))ds η i,j (dz)
R
R+ 0
By the law of large numbers, when the market maker has accumulated a sufficiently large number
of observations, his best estimation of λi,j
RFQ converges to the “real” intensity of the market. As time
passes, the prior parameters (α , β ) of the market maker are less important as the estimation will
i,j i,j
rely mostly on the observations.
Another important parameter of the model is the size of transactions, which impacts the quotes of the
market maker as well as his inventory risk. In [10], the authors choose in their numerical experiments
a Γ(ai,j , bi,j ) distribution for η i,j . The trader can choose between Bayesian updates (revise only ai,j ,
only bi,j , or both), depending on his confidence on parameters’ estimation. If he is confident with
respect to the shape parameter ai,j , that is he knows approximately the average size of a request but
not the standard deviation, he sets bi,j ∼ Γ(ai,j i,j
0 , b0 ). Given n observations of size z , . . . , z , the best
1 n
Bayesian estimate of bi,j (the scale parameter of the size of the request) is
ai,j + nai,j
E[b |(z , . . . , z )] = i,j0 Pn i .
i,j 1 n
(Appendix B.2)
b0 + i=1 z
The use of different prior distribution to take into account the uncertainty on the shape parameter
ai,j (if bi,j is known) or on both (ai,j , bi,j ) can be done in the same way.
Another sensitive parameter, especially for the multi-asset market making, is the variance-covariance
matrix Σ. This quantity is usually estimated on a long run, but parameters are subject to a brutal
change. For example, let us assume that the market maker is in charge of d assets on 2 different
sectors (for instance, technology and aerospace). Following the factorial approach, the market making
problem’s dimension will be reduced from d to 3. The three factors mainly correspond to the three
43
highest eigenvalues of the variance-covariance matrix Σ, and will drive the quotes of the market
maker. However, in case of a sectorial tail event, for example the bankruptcy of one of the companies
of the tech sector, it is likely that all the correlations between the assets of this sector will rise to
one. This will impact the eigenvalue related to the technology sector, and change the quotes of the
market maker as he has to avoid long inventory positions on assets whose values are decreasing. To
design adaptive market making strategy based on Bayesian update of the correlation matrix and the
drift of the assets, we define the Normal-Inverse-Wishart prior on (µ, Σ) ∼ NIW(µ0 , κ0 , ν0 , ψ), where
(µ0 , κ0 , ν0 , ψ) ∈ Rd × R⋆+ × (d − 1, +∞) × Md (R). This distribution is built as follows:
1
µ|(µ0 , κ0 , Σ) ∼ N µ0 , Σ , Σ|(ψ, ν0 ) ∼ W −1 (ψ, ν0 ), then (µ, Σ) ∼ NIW(µ0 , κ0 , ν0 , ψ),
κ0
where W −1 is the standard inverse Wishart distribution. In other words, the drift vector µ of the
assets follows a multivariate Gaussian distribution whereas the variance-covariance matrix Σ follows
a standard inverse Wishart distribution. At time t, if we note St = (St1 , . . . , Std ) the prices observed
up to time t, the Bayesian update of (µ, Σ) is
κ0 µ0 + (St − S0 )

(µ, Σ|St − S0 ) ∼ NIW , κ0 + t, ν0 + t,
κ0 + t
St St T κ0 t St St T

ψ + (St − )(St − ) + (µ0 − )(µ0 − ) .
t t κ0 + t t t
Following the law of large numbers, as t → +∞ we have a larger number of information and we
converge toward the drift and variance-covariance of the market maker’s portfolio. Therefore, the
market maker can recompute his factors derived from the updated variance-covariance matrix and
adjust his quotes.
This extension deserves several remarks. First, the problems encountered by an OTC market maker
are quite different from a high-mid frequency trader in an order book. The model is more parsimo-
nious, especially for the intensity functions. Therefore, the convergence toward the “true” market
parameters will be faster than in order book model. The objective of the Bayesian update on the
quantities λi,j
RFQ is to determine the average behavior or the counterparts of the market maker. If he
observes a large number of requests on the ask (resp. bid) side of the asset i, the Bayesian update
(Appendix B.1) enables the market maker to adjust his quotes to set a higher ask (resp. bid) price
for this asset. If the market maker observes a higher discrepancy than expected for the transaction
sizes, the Bayesian update (Appendix B.2) helps to adjust his quotes. Finally, the Bayesian learning
on the drift and covariance of the assets enables to update the factors from which the market maker
chooses his quotes.
44
References
[1] R. Almgren and B. Harts. A dynamic algorithm for smart order routing. White paper StreamBase,
2008.
[2] R. Almgren, C. Thum, E. Hauptmann, and H. Li. Equity market impact. Risk, July, pages
58–62, 2005.
[3] H. Alsayed and F. McGroarty. Arbitrage and the law of one price in the market for ameri-
can depository receipts. Journal of International Financial Markets, Institutions and Money,
22(5):1258–1276, 2012.
[4] M. Avellaneda, J. Reed, and S. Stoikov. Forecasting prices from level-i quotes in the presence of
hidden liquidity. Algorithmic Finance, 1(1):35–43, 2011.
[5] M. Avellaneda and S. Stoikov. High-frequency trading in a limit order book. Quantitative
Finance, 8(3):217–224, 2008.
[6] A. Bachouch, C. Huré, N. Langrené, and H. Pham. Deep neural networks algorithms for stochastic
control problems on finite horizon, part 2: Numerical applications, 2018.
[7] B. Baldacci, P. Bergault, and O. Guéant. Algorithmic market making: the case of equity deriva-
tives. arXiv preprint arXiv:1907.12433, 2019.
[8] B. Baldacci, I. Manziuk, T. Mastrolia, and M. Rosenbaum. Market making and incentives
design in the presence of a dark pool: a deep reinforcement learning approach. arXiv preprint
arXiv:1912.01129, 2019.
[9] G. Barles and C. Imbert. Second-order elliptic integro-differential equations: viscosity solutions’
theory revisited. In Annales de l’Institut Henri Poincare/Analyse non lineaire, volume 3, pages
567–585, 2008.
[10] P. Bergault and O. Guéant. Size matters for otc market makers: viscosity approach and dimen-
sionality reduction technique. arXiv preprint arXiv:1907.01225, 2019.
[11] B. Bouchard. Introduction to stochastic control of mixed diffusion processes, viscosity solutions
and applications in finance and insurance. Lecture Notes Preprint, 2007.
[12] Á. Cartea, S. Jaimungal, and J. Penalva. Algorithmic and high-frequency trading. Cambridge
University Press, 2015.
[13] R. Cont and A. Kukanov. Optimal order placement in limit order markets. Quantitative Finance,
17(1):21–39, 2017.
[14] M. G. Crandall, H. Ishii, and P.-L. Lions. User’s guide to viscosity solutions of second order
partial differential equations. Bulletin of the American mathematical society, 27(1):1–67, 1992.
[15] J. Gatheral. No-dynamic-arbitrage and market impact. Quantitative finance, 10(7):749–759,

2010.
45
[16] O. Guéant. The Financial Mathematics of Market Liquidity: From optimal execution to market
making, volume 33. CRC Press, 2016.
[17] O. Guéant, C.-A. Lehalle, and J. Fernandez-Tapia. Dealing with the inventory risk: a solution
to the market making problem. Mathematics and financial economics, 7(4):477–507, 2013.
[18] O. Guéant and I. Manziuk. Deep reinforcement learning for market making in corporate bonds:
beating the curse of dimensionality. arXiv preprint arXiv:1910.13205, 2019.
[19] F. Guilbaud and H. Pham. Optimal high-frequency trading with limit and market orders. Quan-
titative Finance, 13(1):79–94, 2013.
[20] C. Huré, H. Pham, A. Bachouch, and N. Langrené. Deep neural networks algorithms for stochastic
control problems on finite horizon, part i: convergence analysis. arXiv preprint arXiv:1812.04300,
2018.
[21] A. Jain and C. Jain. Hidden liquidity on the us stock exchanges. The Journal of Trading,
12(3):30–36, 2017.
[22] S. Laruelle, C.-A. Lehalle, and G. Pages. Optimal split of orders across liquidity pools: a
stochastic algorithm approach. SIAM Journal on Financial Mathematics, 2(1):1042–1076, 2011.
[23] S. Laruelle, C.-A. Lehalle, and G. Pagès. Optimal posting price of limit orders: learning by
trading. Mathematics and Financial Economics, 7(3):359–403, 2013.
[24] R. Rabinovitch, A. C. Silva, and R. Susmel. Returns on adrs and arbitrage in emerging markets.
Emerging Markets Review, 4(3):225–247, 2003.
[25] S. Stoikov and M. Sağlam. Option market making under inventory risk. Review of Derivatives
Research, 12(1):55–79, 2009.
[26] I. M. Werner and A. W. Kleidon. Uk and us trading of british cross-listed stocks: An intraday
analysis of market integration. The Review of Financial Studies, 9(2):619–664, 1996.
46
View publication stats

Adaptive Trading Strategies Across Liquidity Pools

Uploaded by

Copyright:

Available Formats

Adaptive Trading Strategies Across Liquidity Pools

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Adaptive Trading Strategies Across Liquidity Pools

Uploaded by

Copyright:

Available Formats

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Adaptive trading strategies across liquidity pools

Preprint · August 2020

Bastien Baldacci Manziuk Iuliia

SEE PROFILE SEE PROFILE

Contract theory View project

The user has requested enhancement of the downloaded file.

2 Optimal trading on several liquidity pools

n Assume for all n ∈ {1, . . . , N} and k ∈ {1, . . . , K} thato Ik = Ik . In this case

Qnψ = {0, 1} if ψ n = δ n , and {−1, 0, 1} otherwise,

The cash process of the trader at time t ∈ [0, T ] is

with terminal condition

where ψ = (ψ 1 , . . . , ψ N ), I = (I 1 , . . . , I N ). The expectation in (2.3) is taken over the variables ǫn , n ∈

u(t, x, q, S, ψ, I) = x + qS + v(t, q, ψ, I).

The HJBQVI then becomes a system of ODEs with 2N + 1 state variables:

with terminal condition v(T, q, ψ, I) = 0.

3 Adaptive trading strategies with Bayesian update

3.1 Bayesian update of the model parameters

3.1.1 Update of the intensities

λn (ψt , It , pnt , ℓt ) = f λ (ℓt ) λn,m,p1{(ψt ,It )∈m,pnt =p} .

The posterior estimate of the intensity λn (ψt , It , pnt , ℓt ) becomes

3.1.2 Update of the executed proportion

ρn,m,p ∼ Dirichlet(αǫ,n,m,p + cn,m,p

(νk )nk· exp(−νk Tk ) (pkk′ )nkk′ ,

Then the posterior transition matrix is

Given these conjugates priors, our best estimators of νkn,ψ , pn,ψ

The posterior transition matrix is given by

Similar formulae apply for νkn,I , pn,I

3.1.4 Update of the mid-price

dSt = µdt + σdWt ,

4.1 Extension 1: Incorporation of signals in the price process

4.2 Extension 2: Market impact

The impacted mid-price process can be modeled as follows:

ing [15], we assume linear permanent market impact, that is

4.3 Extension 3: Hidden liquidity

• q0 = 5 × 104 : initial inventory of the trader.

with the following set of parameters

• η = 0.1: coeﬃcient of quadratic costs.

• V = 1 × 108 : average market volume.

• γ = 1 × 10−6 : risk aversion of the trader using a CARA utility function.

• σ = 0.05: volatility of the asset.

total volume posted.

5.2 Numerical methods

0 = − ∂t v(t, q, ψ, I) + g(q − qt⋆ ) − µq

v(ti+1 , q, ψ, I) =v(ti , q, ψ, I) − ∆t g(q − qt⋆ ) − µq

p∈{−1,0,1}N ,ℓ∈LN n=1 2

with terminal condition v(T, q, ψ, I) = 0.

5.2.2 Neural networks

v target (ti−1 , q, ψ,I) = v[θiv ](ti , q, ψ, I) + ∆t g(q − qt⋆ ) − µq

0 10000 20000 30000 40000 50000

imbalances: InputLayer spreads: InputLayer inventory: InputLayer

imbalances: InputLayer spreads: InputLayer inventory: InputLayer

spreads_and_imbalances: Concatenate all_inputs: Concatenate

hidden_layer_2.1: Dense hidden_layer_2.2: Dense hidden_layer_1.1: Dense hidden_layer_1.2: Dense

hidden_layer_3.1: Dense hidden_layer_3.2: Dense

hidden_layer_4.1: Dense limits_venue_2: Dense limits_venue_1: Dense

Figure 5: Neural network structure for the

5.3 Two identical venues

5.3.1 Value function

Figure 16: Limit order strategy, ψ = ψ = δ, 1 2