Optimal trading of an asset driven by a hidden Markov

process in the presence of fixed transaction costs

G. W. P. Thompson
Centre for Financial Research,
Judge Institute of Management,
University of Cambridge

Abstract

We consider the problem of the optimal trading of an asset in the presence of fixed
transaction costs where the asset price satisfies an SDE of the form dSt = dBt +h(Xt ) dt
where Bt is a Brownian motion, h is a known function and Xt is a Markov Chain. We
look at two versions of the problem, maximising the long term gain per unit time and
maximising a form of discounted gain. It is well known that the optimal trading strategy
for such a problem is the solution of a free-boundary problem; we present an intuitive
derivation by viewing the optimal trading problem as a pair of simultaneous optimal
stopping problems. We also give explicit solutions for a range of examples, and give
bounds on the transaction cost above which it is optimal never to buy the asset at
all. We show that in the case where Markov Chain Xt is independent of the Brownian
motion and has a finite statespace, this critical transaction cost has a simple form.

Keywords: Optimal stopping problem, trading problem, hidden Markov model

1 Introduction
We consider the problem of optimally trading an asset in the presence of transaction costs.
Rt
We assume that the asset price process has the form St = Bt + 0 h(Xu ) du where B is a
standard Brownian motion and X is a Markov process. The only information available at
time t is the past history of the asset; the Markov process X is not directly observable.
At each time t, t ∈ [0, ∞) we assume that we are allowed to hold 0 or 1 units of the asset
and that we incur a transaction cost of 21 c > 0 whenever we buy or sell the asset. Letting
ξt denote our holding at time t and {Ti }, i ≥ 1 the times when ξ is discontinuous, we look
R∞
for strategies which maximise E( 0 e−ρt ξt dSt − 12 c e−ρTi ) where ρ > 0. For tractability
P

we will frequently just consider the limiting strategy obtained as ρ → 0. This is often also

1
 Rt
the strategy which maximises lim inf t→∞ 1t E( 0 ξu dSu − 12 c I{Ti ≤t} ). We will refer to the
P
Rt
quantity lim inf t→∞ 1t E( 0 ξu dSu − 12 c I{Ti ≤t} ) as the (long run) average gain.
P

In Morton & Pliska (n.d.) and Pliska & Selby (1994) a similar type of problem is con-
sidered. The authors try to maximise the long term log return per unit time in a framework
with proportional transaction costs where the asset price follows an n-dimensional geomet-
ric Brownian motion with constant drift. Here we look at a rather different class of models
by introducing an unobserved Markov Chain, and allow ourselves the luxury of a somewhat
simpler transaction cost structure and optimality criteria. This approach also introduces
the element of filtering the past history of the asset to estimate the current state of Xt ,
a feature also considered in Mandarino (1990), where the Kalman filter is used. More ex-
amples of the filtering of hidden Markov models can be found in Elliot, Aggoun & Moore
(1995) and in its references.
The outline of this paper is as follows: In the following section, Section 2, we derive a
condition for optimality closely related to the Hamilton-Jacobi-Bellman equation, by con-
sidering the optimal trading problem as a pair of simultaneous optimal stopping problems.
In Section 3 we consider the case where the instantaneous expected drift is a Markov
process of a particular type. We derive the limiting optimal strategy as ρ → 0 and prove
that it does maximise the average gain in certain cases. As an example, we derive the
optimal trading strategy when the asset follows an Ornstein-Ulhenbeck price process.
In Section 4 we consider the case where X has a finite statespace. We derive conditions
on c which determine whether it is ever optimal to hold the asset. We examine both the
discounted and average gain cases and as an example consider a simple 2-state model.
Finally, in Section 5 we consider a model where the asset drifts towards a level which
follows a Brownian motion; we show that this model is essentially the same as the OU
model considered in Section 3.

2 Optimality Equations
Rt
We assume that the asset price process has the form St = Bt + 0 hu du, where Bt is
a Brownian motion adapted to a filtration F and hu = h(Xu ) for some Markov process
Xt also adapted to F. We denote by Y the complete filtration generated by S and let
ht = E(ht |Yt ).
b
Rt Rt
We define the innovations process Nt = St − 0 b hu du. Assuming E( 0 h2u du < ∞) for
each t, Nt is a Y-Brownian motion (see Section VI.8 of Rogers & Williams (1987)). We
R∞
shall also assume E( 0 e−ρt |b
ht | dt)2 < ∞. Note that if h is bounded these conditions are
certainly met.

2
 A trading strategy ξ is a Y-previsible process with values in {0, 1} and with ξ0 given,
such that E( e−ρTi )2 < ∞ where {Ti }, i ≥ 1, are the discontinuities of ξ. Our aim is to
P

maximise the expected discounted gain, which we define by

Z ∞ X 
E e−ρt ξt dSt − 12 c e−ρTi Y0 . (2.1)
0

We assume that the conditional distribution of Xt given Yt is is described by a diffusion

pt ∈ E ⊆ Rn with generator G, and that pt satisfies the SDE dpt = σ(pt ) dNt + µ(pt ) dt.
For example, if Xt has a finite statespace we could let pt (i) = P(Xt = i|Yt ), and if the
process (St , Xt ) is jointly Gaussian we could set pt = (E(Xt |Yt ), Var(Xt |Yt )). Write Ep for
expectation under the law of (pt ) with p0 = p.
For an arbitrary function f on the statespace of X, we define fb(p) to be the value Ef (X)
under the distribution for X corresponding to p.
RT RT
Note that since E( 0 e−ρt ξt dNt )2 ≤ 0 e−2ρt dt < (2ρ)−1 the collection of random
RT Rt
variables { 0 e−ρt ξt dNt : T ≥ 0} is bounded in L2 and the process 0 e−ρu ξu dNu is a UI
martingale. Using the Optional Stopping Theorem and the fact that dSt = dNt + b ht dt, the
expected discounted gain, (2.1) equals
Z ∞ X 
−ρt b 1 −ρTi
E e ξt ht dt − 2 c e Y0 .
0

For simplicity we will restrict attention to Markov strategies which specify ξt in terms of
ξt− and pt . Thus a strategy amounts to the specification of two subsets B and S of E, the
asset, if it is not already held, being bought at the moment pt enters B and then sold when
pt enters S. We look for Markov strategies which are optimal for all initial ξ0 and p0 .

Proposition 2.1 Suppose that for fixed subsets B, S ⊆ E the strategy ξ which buys when
pt ∈ B and sells when pt ∈ S is optimal. Then there is a function w on E such that

(G − ρ)w = −b
h on B c ∩ S c (2.2)
w = + 21 c on B (2.3)
w = − 12 c on S (2.4)
σ · ∇w = 0 on ∂B, ∂S (2.5)

Proof
Define the function g : E → R as follows: for p ∈ S, set g(p) = 0, otherwise set
RH
g(p) = Ep [ 0 S e−ρtb
ht dt − 21 c(1 + e−ρHS )], where HS is the first hitting time of S by pt .
First consider the problem: find a stopping time τ to maximise Ep e−ρτ g(pτ ∧HS ). Let B ∗
be the optimal stopping set; we will show that B = B ∗ .

3
 Define the sequence of stopping times σn , τn , n ≥ 0 by

τ0 = 0
σn = inf{t > τn : pt ∈ B ∪ B ∗ }
τn+1 = inf{t > σn : pt ∈ S}

Note that ξ buys at most once in [σj−1 , τj ), j ≥ 1 and sells at τj . Define Tj0 = T if ∆ξT = 1,
T ∈ [σj−1 , τj ) and Tj0 = ∞ otherwise. Note that
 
Z ∞ X  Z σ0 X
E e−ρt ξtb
ht dt − 21 c e−ρTi =E  e−ρt ξtb
ht dt − 12 c e−ρTi 
0 0 Ti ≤σ0
 !
∞ Z τj 0
ht dt − 12 c(e−ρTj
X
+E  I{T 0 <∞} j
e−ρtb + e−ρτj ) 
j=1 Tj0

Rσ
Since E( 0 j e−ρt |b
ht | dt)2 and E( Ti ≤σj e−ρTi )2 are both increasing in j and bounded
P

above, we use Dominated Convergence to interchange the order of summation and expec-
tation in the final term of the equation above. Conditioning on Yσj−1 and using the strong
Markov property gives
∞
X
E[e−ρσj−1 Epσj−1 I{HB <HS } e−ρHB g(pHB )],
j=1

where the inner expectation is just Epσj e−ρHB g(pHB ∧HS ). Thus by the optimality of ξ, and
that fact that B ∗ is the optimal stopping set, ξ buys at time σj−1 if and only if pσj−1 ∈ B ∗ .
Thus B = B ∗ .
We now define v(p) = Ep e−ρτ g(pτ ∧HS ), so that v satisfies (G−ρ)v = 0 in S c ∩B c , v(p) = 0
on S, v(p) = g(p) on B and σ·∇v = σ·∇g on ∂B. (The final condition is the ‘smooth pasting’
condition on the decision boundary, see Shiryayev (1978) page 161 or Öksendal (1994) page
RH
202.) Similarly we define g̃(p) = 0 for p ∈ B and g̃(p) = −Ep [ 0 B e−ρtb ht dt + 21 c(1 + e−ρHB )]
for p 6∈ B. We let S ∗ be the stopping set for the problem ‘find a stopping time τ̃ to maximise
Ep e−ρτ̃ g̃(pτ̃ ∧HB )’, and we define the sequence of stopping times σ̃n , τ̃n , n ≥ 0 by

τ̃0 = 0
σ̃n = inf{t > τ̃n : pt ∈ S ∪ S ∗ }
τ̃n+1 = inf{t > σ̃n : pt ∈ B}

We see that ξ sells at most once in [σ̃j−1 , τ̃j ), j ≥ 1, and buys at τ̃j . Defining T̃j0 = T if

4
∆ξT̃ 0 = −1, T ∈ [σ̃j−1 , τ̃j ) and T̃j0 = ∞ otherwise, we have
 
Z ∞ X  Z σ̃0 X
E e−ρt ξtb
ht dt − 12 c e−ρTi = E e−ρt ξtb
ht dt − 12 c e−ρTi 
0 0 Ti ≤σ̃0
 !
∞ Z T̃j0
−ρT̃j0
X
+E  I {T̃j0 <∞} e−ρtb
ht dt − 12 c(e + e−ρτ̃j ) 
j=1 σ̃j−1
 
X∞ Z τ̃j
+E  I {T̃j0 =∞} e−ρtb
ht dt
j=1 σ̃j−1

where the final two terms equal

   !
∞ Z τ̃j ∞ Z τ̃j 0
ht dt − 12 c(e−ρT̃j + e−ρτ̃j ) 
X X
E e−ρtb
ht dt + E  I {T̃j0 <∞} − e−ρtb
j=1 σ̃j−1 j=1 T̃j

Again we can interchange the order of summation and expectation in the final term. Con-
ditioning on Yσ̃j and using the strong Markov property, the this becomes
∞
pσ̃j−1
X
E[e−ρσ̃j−1 E I{HS <HB } e−ρHS g̃(pHS )],
j=1
p
where the inner expectation is now E σ̃j−1 e−ρHS g̃(pHS ∧HB ). Thus S = S ∗ .
Similarly to before, define ṽ(p) = Ep e−ρτ̃ g̃(pτ̃ ∧HB ), then ṽ satisfies (G − ρ)ṽ = 0 in
S c ∩ B c , ṽ(p) = 0 on B, ṽ(p) = g̃(p) on B and σ · ∇ṽ = σ · ∇g̃ on ∂B. We now have two
functions
Z HS 
v(p) = Ep e−ρtb
ht dt − 12 cI{HB <HS } (e−ρHS + e−ρHB )
HS ∧HB
Z HB 
p −ρt b 1 −ρHS −ρHB
ṽ(p) = −E e ht dt + 2 cI{HS <HB } (e +e ) .
HS ∧HB
RH RH R H ∧H
Since 0 = σ·∇(v−g) on ∂B, noting that HSS∧HB e−ρtb
ht dt = 0 S e−ρtbht dt− 0 S B e−ρtb ht dt,
we see that
 Z HS ∧HB 
p −ρt b 1 −ρHS −ρHB −ρHS
0 = σ · ∇E − e ht dt − 2 c(I{HB <HS } (e +e )−e )
0
 Z HS ∧HB 
p −ρt b 1 −ρHS −ρHS
= σ · ∇E − e ht dt − 2 c(I{HB <HS } e − I{HS <HB } e )
0
Similarly, since 0 = σ · ∇(ṽ − g̃) on ∂S
Z HS ∧HB 
0 = σ · ∇Ep e−ρtb
ht dt − 12 c(I{HS <HB } (e−ρHB + e−ρHS ) − e−ρHB )
0
Z HS ∧HB 
= σ · ∇Ep e−ρtb
ht dt − 12 c(I{HS <HB } e−ρHB − I{HB <HS } e−ρHS )
0

5
Defining the function w on E by
Z HS ∧HB 
p −ρt b 1 −ρHB −ρHS
w(p) = E e ht dt − 2 c(I{HS <HB } e − I{HB <HS } e )
0

h in S c ∩ B c , w = + 12 c in B, w = − 12 c in S and σ · ∇w = 0 on ∂B,
we have (G − ρ)w = −b
∂S.

Remark To see the connection between (2.2)–(2.5) and the HJB equation, define
Z ∞ X 
p −ρt b −ρTi
V (p, ξ) = sup E e ξt ht dNt − e
0

to be the maximal expected discounted gain over trading strategies such that ξ0 = ξ, where
the conditional distribution of X0 given Y0 is given by p. The HJB equation in this case is

h + GV (p, ξ) − ρV (p, ξ), − 12 c + V (p, 1 − ξ) − V (p, ξ)) = 0

max(ξb (2.6)

where, as before, G is the generator of p acting on V (·, ξ). We then have

h + GV (p, ξ) − ρV (p, ξ) =
ξb 0 on B c ∩ S c ,
1
V (p, 1) − V (p, 0) = 2c on B,
1
V (p, 0) − V (p, 1) = 2c on S,

Setting w(p) = V (p, 1) − V (p, 0), these conditions become

(G − ρ)w = −b
h on B c ∩ S c ,
w(p) = + 12 c on B,
w(p) = − 12 c on S,

which are just (2.2), (2.3) and (2.4) again.

3 Markovian expected drift

We now consider the special case where b
ht is a recurrent diffusion on an interval I with
SDE

db ht ) dNt − γ b
ht = σ(b ht dt,

where γ > 0. We will show that the limiting optimal strategy as ρ → 0 is to buy when
ht ≥ b ≥ 0 and to sell when b
b ht ≤ s ≤ 0 where b and s satisfy:

s0 (b) = s0 (s), (3.1)

0
b − s − cγ = [s(b) − s(s)]/s (s). (3.2)

6
 xR −2
h, which we define by s0 (x) = e2γ uσ(u) du ,
Here the function s(x) is the scale function of b
determined up to a positive affine transformation. As an example we will solve the problem
in the case when the asset price follows an Ornstein-Ulhenbeck process. Finally, we will
show that if bht is positive recurrent, then, as suggested in the introduction, this strategy
Ru
also maximises lim inf t→∞ 1t E( 0 ξt dSu − 12 c I{Ti ≤t} )
P

Our first two lemmas will show that the limiting optimal strategy for the discounted
version of the problem has the stated form.

Lemma 3.1 Any optimal strategy never buys when b

ht < 0 and never sells when b
ht > 0.

Proof Let ξ be an arbitrary strategy and define the sequence of interleaved stopping times
σn , τn , n ≥ 0 as follows:

σ0 = 0
τn = inf(t > σn : ∆ξt = −1, b
ht > 0 or ∆ξt = +1, b
ht < 0)
σn+1 = inf(t > τn : b
ht = 0).

Now define a new strategy ξ˜ by ξ˜0 = ξ0 and for t > 0,


 ξ on ∪n≥0 (σn , τn )

˜
ξ = 0 on ∪n≥0 [τn , σn+1 ] ∩ {∆ξτn = +1}

1 on ∪n≥0 [τn , σn+1 ] ∩ {∆ξτn = −1}


The strategy ξ˜ just mimics ξ until ξ either buys when b ht < 0 or sells when b ht > 0. It then
takes no action until ht hits 0 when it changes back to ξ.
b
Since ξ˜ is continuous at τn , it is clear that ξ˜ is previsible. In addition we have that
T̃i ≥ Ti and so E( e−ρT̃i )2 < ∞. We also have ξ˜ub hu for all times u, and so ξ˜ has a
P
hu ≥ ξ u b
larger expected discounted gain than ξ. 

Lemma 3.2 Let b, b0 ∈ I with b0 > b and suppose it is optimal to buy when b ht = b ≥ 0.
0 0 ht = b0 .
Then there exists ρ(b ) > 0 such that for ρ ≤ ρ(b ) it is also optimal to buy whenever b

Proof Suppose b h0 = b0 > b and ξ0 = 0, and let T denote the first time b h hits b. Let ξ
be a Markov strategy which buys when ht = b, but not when ht ∈ U , where U is an open
b b
interval containing b0 . Let ξ˜ be the strategy identical to ξ except that ξ˜ buys immediately.
Thus T̃i = Ti for i ≥ 2 and T̃1 , T1 ≤ T . The difference between the expected discounted
gains under ξ˜ and ξ is given by
Z ∞ 
E e−ρt (ξ˜t − ξt )b
ht dt − 12 cE(e−ρT̃1 − e−ρT1 ) Y0 .
0

7
We note that ξ˜t ≥ ξt and b ht > 0 for t ∈ [0, T ], so that b ht (ξ˜t − ξt ) > 0 for t ∈ [0, T ]. Thus,
by Monotone convergence on each term separately as ρ → 0, and noting that T̃1 and T1 are
a.s. finite since b
ht is recurrent, we have
Z ∞ 
−ρt ˜ 1 −ρ T̃ −ρT
lim E e (ξt − ξt )b
ht dt − 2 c(e 1
−e 1
) Y0 > 0.
ρ→0 0

Thus for ρ sufficiently small it is better to buy immediately. 

Similarly if it is optimal to sell when b 0
ht = s ≤ 0, s ∈ I, then for any s < s, s ∈ I and
ht = s0 .
for ρ sufficiently small, it is optimal to sell whenever b
Thus the limiting optimal strategy has the form: buy if b ht ≥ b ≥ 0 and sell if b
ht ≤ s ≤ 0.
It only remains to show s and b satisfy (3.1) and (3.2).

Proposition 3.3 The limiting optimal strategy as ρ → 0 is to buy when b

ht ≥ b ≥ 0 and to
ht ≤ s ≤ 0, where b and s satisfy:
sell when b

s0 (b) = s0 (s),
b − s − cγ = (s(b) − s(s))/s0 (s),

and the function s(x) is the scale function of b

h, defined on page 7.

Proof From the optimality condition derived in Section 2, Equations (2.2)–(2.5), the
limiting strategy as ρ → 0 gives rise to a function w which satisfies 21 σ(x)2 w00 − γxw0 = −x
in (s, b); w(x) = 12 c on [b, ∞); w(x) = − 12 c on (−∞, s], and w(x) is differentiable at x = s
and x = b.
We first define f (x) = w(x) − x/γ so that f satisfies 12 σ(x)2 f 00 − γxf 0 = 0 on (s, b).
Rx −2
Integrating once and using s0 (x) = e2γ uσ(u) du gives f 0 /s0 = K, a constant, so using
the fact that w0 (b) = w0 (s) = 0 we have s0 (s) = s0 (b) = − Kγ1
. Integrating again and using
0
w(b) − w(s) = c gives c − (b − s)/γ = −(s(b) − s(s))/(γs (s)). Rearranging gives the result.

Exmaple If the price process is an Ornstein-Ulhenbeck process, so dSt = σ dBt − γSt dt,
then db ht = −γσ dBt − γ b ht dt. The optimal strategy is to buy if St ≤ −b/γ and to sell if
St ≥ b/γ, where b satisfies
Z b
2 2 2 2
2b − γc = 2e−b /(γσ ) eu /(γσ ) du. (3.3)
0

We now show that this equation has a unique solution in b ≥ 0 and thus determines the
optimal trading strategy. Let φ(b) denote the right hand side of (3.3), so φ(b) ≥ 0 and
φ(b) = 0 only at b = 0. Since φ0 (b) = 2(1 − γσb 2 φ) and the left hand side of (3.3) has
derivative 2 (considered as a function of b), there can be at most one solution. To prove

8
existence we note that φ(b) → 0 as b → 0; we now show φ(b) → 0 as b → ∞. Writing
u = bv, we have
Z 1 Z
b2 (v 2 −1)/(γσ 2 ) 2 2 2
φ(b) = 2 be dv = 2 beb (v −1)/(γσ ) dv.
0 [0,1)
2x √
For x ≥ 0 the maximum of be−b occurs at b = 1/ 2x, so
2 (v 2 −1)/(γσ 2 ) 1 p
beb ≤ e− 2 / 2(1 − v 2 )/(γσ 2 ), b ≥ 0, v ∈ [0, 1)
2 2 2
which is integrable on [0, 1). Since beb (v −1)/(γσ ) → 0 as b → ∞ on [0, 1), by Dominated
Convergence, φ(b) → 0 as b → ∞.
Average gain case. We now consider the problem of maximising
Z t X
lim inf t E[ ξu dSu − 12 c
1
I{Ti ≤t} |Y0 ],
t→∞ 0

which we shall refer to as the (long run) average gain. Our method is essentially the same
as that of the proof of Proposition 1, and we will deduce the same condition (Equations
(2.2)–(2.5)) but for ρ equal to 0. Thus if an optimal strategy exists for this second problem,
and the equation Gw = −b h, with the same boundary conditions as before, has a unique
solution, then the limiting optimal strategy as ρ → 0 also maximises the average gain. Let
Ex denote expectation under the law of b ht started from x. In this Section we will assume
that bht is positive recurrent (so Py (Hx < ∞) = 1 implies Ey (Hx ) < ∞ where Hx denote
ht ), that σ is bounded on I and that 1t E(|b
the first hitting time of x by b ht |) → 0 as t → ∞.
First we state and prove a useful lemma:
Lemma 3.4 Let σ < τ be two Y-stopping times with E(τ − σ) < ∞. Then E(Sτ − Sσ ) =
−γ −1 E(b
hτ − b
hσ ).
Proof We have
Z τ
E(Sτ − Sσ + γ −1 (b
hτ − b
hσ )) = E [1 + γ −1 σ(b
hu )] dNu .
σ

Since 1 + γ −1 σ(·) is bounded and E(τ − σ) < ∞, we have that the family of random
R τ ∧t Rτ
variables { σ∧t (1 + γ −1 σ(bhu )) dNu : t ≥ 0} is L2 -bounded and hence UI. Thus E σ (1 +
γ −1 σ(b
hu )) dNu = 0 by a version of the Optimal Stopping Theorem. 
Note that a corollary of this is that the optimal strategy is to buy when bht ≥ b ≥ 0 and
to sell when b ht ≤ s ≤ 0, for some b and s (by very similar arguments to those given earlier).

Proposition 3.5 Suppose that, for fixed s ≤ 0 ≤ b, the strategy ξ which buys when b
ht ≥ b
and sells when b ht ≤ s is optimal. There is a function w such that Gw(x) = −x on (s, b),
w = + 2 c on [b, ∞), w = − 12 c on (−∞, s] and w0 = 0 at b and s, where G denotes the
1

generator of b
ht .

9
 RH
Proof Define g(x) = 0 for x ≤ s, and g(x) = Ex 0 s b ht dt − c for x > s, where Hs is
∗
the first hitting time of s by ht . Let B be the optimal stopping set for the problem:
b
max Ex g(bhτ ∧HS ). Let b̄ = b ∧ inf{x ∈ B, x > s}; we will show b = b̄.
Define the sequence of stopping times σn , τn , n ≥ 0 by

τ0 = 0
ht ≥ b̄}
σn = inf{t > τn : b
ht ≤ s}
τn+1 = inf{t > σn : b
Rt
hu du− 21 c
P
and define Gt = 0 ξu b I{Ti ≤t} . Let N (t) be the greatest index such that τN (t) ≤ t
and write
N (t)+1
X
Gt = Gt − GN (t)+1 + Gτ1 + (Gτn − Gτn−1 ).
n=2

We will show limt→∞ 1t E(Gt − GN (t)+1 ) = 0, limt→∞ 1t E(Gτ1 ) = 0, and then finally that
PN (t)
limt→∞ 1t E n=2 (Gτn+1 − Gτn ) = K −1 Eb̄ g(b hHb ∧Hs ) for some constant K. This will imply
b = b̄.
For the two first statements, note that ξ can buy and sell at most once in each of the
intervals [0, τ1 ) and [t, τN (t)+1 ). Thus we have

E(Gτ1 ) ≤ γ −1 (|b
ht | + |b| + |s|),
E(Gt − GτN (t)+1 ) ≤ γ −1 (|b
ht | + |b| + |s|).

Since 1t E(|b
ht |) → 0 as t → ∞, limt→∞ 1t E(Gt − GN (t)+1 ) = 0 and limt→∞ 1t E(Gτ1 ) = 0.
For the third statement, E(τn+1 − τn ) > 0 so E[N (t)] < ∞ (see Grimmett & Stirzaker
(1992), 10.5.1(b)). As the random variables Gτn+1 − Gτn , n ≥ 1, are IID, using Wald’s
equation (Grimmett & Stirzaker (1992), page 211) and setting K = E(τn+1 − τn ) gives
N (t)+1
X
1 x
lim t E [ (Gτn+1 − Gτn )] = lim 1 Ex [N (t)]Ex (Gτn+1 − Gτn )
t→∞ t→∞ t
n=2

= K −1 Eb̄ (Gτn+1 − Gτn )

= K −1 Eb̄ [I{Hb <Hs } g(b
hHb )]
= K −1 Eb̄ [g(b
hHs ∧Hb )].

Thus b = inf{x ∈ B ∗ , x > s}. The remainder of the proof is very similar to the proof of
RH
Proposition 1: defining g̃(x) = 0 for x ≥ b and g̃(x) = −Ex ( 0 b b
ht dt + c) for x < b, we can
∗ ∗
show that s = sup{x ∈ S , x < b} where S is the optimal stopping set for the stopping

10
problem: max Ex g̃(bhHs ∧Hb ). These are just the same stopping problems met in Proposition
1 but with ρ replaced with 0. Thus the points s and b satisfy (2.2)–(2.5) with ρ = 0, and
the result follows.


4 Finite statespace models

We consider the model

dSt = dBt + h(Xt ) dt

where Xt is an irreducible Markov Chain on {1, . . . , n} with Q-matrix Q, and independent

of B. In this section we will consider the problem of finding a transaction cost above which
it is optimal never to buy the asset, in both the discounted and average gain versions of the
optimal trading problem. At the end of the section we will consider a simple 2-state model.
Letting pt (i) = P(Xt = i|Yt ), where Y is as usual the filtration generated by the asset
price process, we have (see Section VI.11 of Rogers & Williams (1987))

ht ) dNt + (Q> pt )(i) dt.

dpt (i) = pt (i)(h(i) − b (4.1)

From Section 2 the optimal strategy, which buys when pt ∈ B and sells when pt ∈ S,
P
gives rise to a function w on {x1 , . . . , xn : i xi = 1, xi ≥ 0}, such that (G −ρ)w = −bh. Here
P
G is the generator of pt and the function h is defined by h(x) = i xi h(i), with boundary
b b
conditions w = + 21 c in B, w = − 12 c in S, and i xi (h(i) − b
P
h)∂w/∂xi = on ∂B and ∂S.
R ∞ −ρt
Define ψ(i) = E( 0 e h(Xt ) dt|X0 = i). Note that (ρ − Q)ψ = h so (G − ρ)ψb =
−bh. Suppose we are about to buy at time 0 and subsequently sell at the random time τ .
Rτ
This will not be an optimal decision if E( 0 e−ρt h(Xt ) dt − 12 c(1 + e−ρτ )) < 0. Note that
Rτ
E( 0 e−ρt h(Xt ) dt − 21 c(1 + e−ρτ )) = ψb0 − 12 c − Ee−ρτ (ψbτ + 12 c) using the Strong Markov
property at τ .

Proposition 4.1 Define ψ M = maxi ψ(i), ψ m = mini ψ(i). Then, i) if ψ M > −ψ m and pt
P
is irreducible on {x1 , . . . , xn : xi = 1, xi > 0} (with respect to Lebesgue measure), it is
optimal never to buy the asset if c > 2ψ M , and to buy at some point with probability one if
c < 2ψ M ; ii) if ψ M ≤ −ψ m , a sufficient condition to ensure that it is optimal never to buy
the asset is that c > ψ M − ψ m .

Proof i) If ψ M > −ψ m and c > 2ψ M , then ψbτ + 21 c > 0 and ψb0 − 12 c < 0, so ψb0 − 12 c −
Ee−ρτ (ψbτ + 12 c) < 0. If c < 2ψ M , choose k such that 21 c < k < ψ M , buy when ψb ≥ k, which
happens with probability one, and sell at some sufficiently large deterministic time τ .

11
 ii) If ψ M ≤ −ψ m then ψ M + ψ m ≤ 0. Write 2c = ψ M − ψ m + 2c0 , with c0 > 0. Now

ψb0 − 12 c − Ee−ρτ (ψbτ + 12 c) ≤ ψ M − 12 c − Ee−ρτ (ψ m + 12 c)

= 1
2 (ψ
m + ψ M − 2c0 ) − Ee−ρτ 12 (ψ m + ψ M + 2c0 )
= 1
2 (ψ
m + ψ M )(1 − Ee−ρτ ) − c0 (1 + Ee−ρτ )
< 0


Remark In Proposition 4.1 we ignore the case where ψM
= ψm since then h = (ρ − Q)ψ
is constant and the asset has constant drift.
Average gain case. We now suppose that we are considering buying at time 0 and
subsequently selling at time τ , which we will assume satisfies E(τ ) < ∞.
R∞
Define ψ ρ (i) = E( 0 e−ρt h(Xt ) dt|X0 = i) (the function ψ of the previous section) and
let π be the stationary distribution of X. Note that as ρ → 0, ψ ρ (i) − π · h/ρ converges,
R∞
since ψ ρ (i) − π · h/ρ = 0 (e−ρt (E(h(Xt )|X0 = i) − π · h) dt) and E(h(Xt )|X0 = i) − π · h
converges to 0 exponentially fast as t → ∞. Let ψ̃ ρ (i) = ψ ρ (i) − π · h/ρ and ψ̃ = limρ→0 ψ̃ ρ ,
so that (ρ − Q)ψ̃ ρ = h − π · h. Now using E(τ ) < ∞ we have
Z ∞ 
−ρt 1 −ρτ
E(Sτ − S0 ) = lim E e h(Xt ) dt − 2 c(1 + e )
ρ→0 0
= cρ − 1 c − E[e−ρτ (ψ
lim (ψ cρ + 1 c)])
0 2 τ 2
ρ→0

= ψ 0 − E(ψ τ ) − c + (π · h)E(1 − e−ρτ )/ρ

b̃ b̃

= ψ 0 − E(ψ τ ) − c + (π · h)E(τ )
b̃ b̃

Thus if π · h > 0 it is always optimal to buy at some point, and we can ensure a positive
expected profit by simply selling at a sufficiently large deterministic future time. If π ·h < 0,
a sufficient condition to ensure it is optimal never to buy the asset is c > maxi ψ̃(i) −
mini ψ̃(i). If π · h = 0 it is optimal to buy the asset at some point if and only if c <
c∗ = maxi ψ̃(i) − mini ψ̃(i). In the case where π · h = 0 note that this amounts to the
assumption that the asset has no long-term drift; in this case we also have −Qψ̃ = h and
thus G ψ = −b h. These both still hold if we add a constant to ψ̃, so without loss of generality
b̃
assume mini ψ̃(i) = − 21 c. We can now write down an explicit solution to (2.2)–(2.5) in the
case c = c∗ by setting w(p) = ψ.
b̃
Exmaple We consider a model for the asset price with the form

dSt = dBt + Xt dt

where X is a Markov chain on {−1, +1}, independent of the Brownian motion B, so St

alternates between being a Brownian motion with drift +1 and a Brownian motion with

12
drift −1. We assume that X jumps between ±1 at a rate q > 12 ρ. This model fits into the
previous framework through the choice h(x) = x. Since b
ht = E(h(Xt )|Yt ), from (4.1) we
have

db h2t ) dNt − 2qb

ht = (1 − b ht dt.

This is of the form considered in Section 2. Here the scale function s(x) satisfies s0 (x) =
Rx 2
exp(4q u/(1 − u2 )2 du) = e2q/(1−u ) . Thus, using results from Section 2, the optimal
strategy is to buy when b ht ≥ b and to sell when b ht ≤ −b, where b satisfies
Z b
−2q/(1−b2 ) 2
b − qc = e e2q/(1−u ) du.
0

This equation has a unique positive solution provided c < 1/q. Note that ψ(1) = ρ/((q +
ρ)2 − ρ2 ) = (2q − ρ)−1 and ψ(−1) = −ρ/((q + ρ)2 − ρ2 ) = −(2q − ρ)−1 . If c > 1/q then for
ρ sufficiently small, c > ψ M − ψ m and it is optimal never to buy the asset. If c = 1/q, then
ψ M > 12 c and it is optimal to buy when ψb hits a level b(ρ) where b(ρ) → 1 as ρ → 0.

5 Reversion to a moving level

In this section we assume the asset price obeys the SDE

dSt = dBt + h(Xt ) dt,

where h(x) = −γx, γ > 0. The Markov process X is defined by

Xt = St − σBt0 ,

where B 0 is a Brownian motion independent of B, σ ≥ 0 is known, and, conditional on Y0 ,

X0 is Gaussian with mean x b0 and variance v0 . This model is very similar to that considered
in Mandarino (1990).
Since the process (St , Xt ) is jointly Gaussian, the conditional distribution of Xt given
Yt is also Gaussian and we need only consider the evolution of the conditional mean, x bt
and variance vt of Xt . Introduce the notation ft = E(f (Xt )|Yt ) for an arbitrary function f .
b
Since X is a diffusion, the evolution of fbt is given by (see Section VI.8 of Rogers & Williams
(1987))
Z t Z t
fbt = fb0 + (fchu − fbub
hu + α
bu , dNu ) + Gf
c du,
u (5.1)
0 0

where G is the generator of X as usual. The process αt is defined by dαt = d[B, M ]t , where
Rt
Mt denotes the F-martingale f (Xt ) − 0 Gfu du.

13
 Applying (5.1) to the function f (x) = x, and using that fact that dXt = dBt − γXt dt −
σ dBt0 , we obtain (supressing some indicies)

db
x = (−γ x x2 + 1) dNt − γb
c2 + γb x dt
= (1 − γvt ) dNt − γb
xt dt.

Similarly, applying (5.1) to the function f (x) = x2 , giving

c2 = (−γ x
dx c3 + γ x
c2 x
b + 2b c2 + 1 + σ 2 ) dt.
x) dNt + (−2γ x

x2 ,
c2 − db
Thus, since dvt = dx

c3 + 3γ x
dvt = (−γ x c2 x x2 ) dNt + (−2γ x
b − 2γb c2 + 1 + σ 2 + 2γb
x2 − (1 − γvt )2 ) dt.

The second term simplifies to (σ 2 −γ 2 vt2 ) dt, and as Xt given Yt is Gaussian, x

c3 −3b x3 =
c2 +2b
xx
0. Thus the conditional distribution of Xt evolves according to

xt = (1 − γvt ) dNt − γb
db xt dt
dvt = (σ 2 − γ 2 vt2 ) dt.

If we are in the steady state, with vt ≡ σ/γ, we have

ht = −γ(1 − σ) dNt − γ b
db ht dt.

This OU form for bht has already been considered in Section 2, and the optimal strategy
ht hits −b, where b is the unique positive solution to
is: buy when ht hits b and sell when b
b
Z b
2 2 2 2
2b − γc = 2eb /(γ|1−σ| ) eu /(γ|1−σ| ) du.
0

References
Elliot, R. J., Aggoun, L. & Moore, J. B. (1995), Hidden Markov Models, number 29 in
‘Applications of Mathematics’, Springer-Verlag.

Grimmett, G. R. & Stirzaker, D. R. (1992), Probability and Random Processes, Oxford

Science Publications, second edn, Oxford.

Mandarino, J. V. (1990), ‘The trader’s problem’, Comm. Statist. Stochastic Models 6(1), 69–
85.

Morton, A. & Pliska, S. R. (n.d.), ‘Optimal portfolio management with fixed transaction
costs’, Math. Finance 5, 337–356.

14
Öksendal, B. (1994), Stochastic Differential Equations, fourth edn, Springer.

Pliska, S. R. & Selby, M. J. P. (1994), ‘On a free boundary problem that arises in portfolio
management’, Phil. Trans. R. Soc. Lond. A 347(1684), 555–561.

Rogers, L. C. G. & Williams, D. (1987), Diffusions, Markov Processes and Martingales:

Volume 2, Itô Calculus, Wiley Series in Probability and Mathematical Statistics, Wi-
ley.

Shiryayev, A. N. (1978), Optimal Stopping Rules, number 8 in ‘Applications of Mathemat-

ics’, Springer.

15