Method to Find a -Optimal Control Non-Markovian Systems

• The strong Markov property yields the family (Tnk )n≥0 is a se- • The random vectors {(∆Tik , σik ) : i = 1, 2, 3, · · · } are indepen-
Introduction
quence of F-stopping times where the increments {∆Tnk ; n ≥ 1} dents and identically distributed, where
is an i.i.d sequence with the same distribution as T1k .
n o 1Z
We present a general methodology for control problems dri- P∞ k ν k (E) = P (∆T1k , σ1k ) ∈ E = fk (s)ds, E = G × j k , (10)
k k
• Then, we define A ; k ≥ 1 A (t) := 2 −k 2 G
ven by the Brownian motion filtration including non-Markovian and n=1 σn11{Tnk ≤t}; t ≥ 0.
non-semimartingale state processes controlled by mutually singu- The jumps {σnk ; n ≥ 1} are given by where fk is the probability density function related to ∆T1k ,
lar measures. The main result is the development of a concrete
( G ∈ B((0, ∞)) and j k ∈ Ik .
pathwise method for characterizing and computing near-optimal 1; if ∆A k (T k ) > 0
controls for abstract controlled Wiener functionals. The theory σnk := k
n
k (5) • The whole dynamics will take place in the history space Hk :=
−1; if ∆A (Tn ) < 0, A × Sk . We denote Hk,n and Ink as the n-fold Cartesian product
does not require ad hoc functional differentiability assumptions on
the value process and elipticity conditions on the diffusion com- of Hk and Ik , respectively.
ponents. The analysis is pathwise over suitable finite dimensional • The elements of Hk,n will be denoted by
spaces and it is based on the weak differential structure introduced  
by Leão, Ohashi and Simas [1] jointly with measurable selection okn := (ak0 , sk1 , j1k ), . . . , (akn−1, skn, jnk ) (11)
arguments. The theory is applied to stochastic control problems
based on path-dependent SDEs where both drift and possibly de- where (ak0 , . . . , akn−1) ∈ An, (sk1 , . . . , skn) ∈ (0, +∞)n and
generated diffusion components are controlled. Optimal control of (j1k , . . . , jnk ) ∈ Ink.
drifts for path-dependent SDEs driven by fractional Brownian mo- k,e(k,T )
tion is also discussed. We finally provide an application in the con- • Every control uk ∈ U0 is completely determined by a list of
text of financial mathematics. Namely, we construct near-optimal Borel measurable functions gnk : Snk → A; 0 ≤ n ≤ (e(k, T ) − 1)
controls in a non-Markovian portfolio optimization problem. in the sense that
e(k,T )
uk (t) = ukj−11{T k <t≤T k }; 0 ≤ t ≤ T and ukn−1 = gn−1
k (Ak ).
X
The Problem j−1 j n−1
j=1
Fix a filtered probability space (Ω, F, P) equipped with a Brow- • One can easily check that {σnk ; n ≥ 1} is an i.i.d sequence of 12 -
nian motion B where F := (Ft)t≥0 is the usual P-augmentation of Bernoulli random variables for each k ≥ 1 and, it is independent where g0k is constant a.s.
the filtration generated by B under a fixed probability measure P. from {∆Tnk ; n ≥ 1} for each k ≥ 1. By construction • For a given list of functions g k = (gn−1
k ) e(k,T )
representing a con-
n=1
Let DnT := {h : [0, T ] → Rn with càdlàg paths}, let ξ : DnT → R k,g k
sup |Ak (t) − B(t)| ≤ 2−k a.s k ≥ 1. (6) trol, we define Ξn : Snk → Hk,n as follows
be a Borel functional and let UtT ; 0 ≤ t ≤ T be the class of admissi- t≥0
ble control processes: For each pair (M, N ) of a.s finite F-stopping k,g k k k k k

Ξn s1 , j1 , . . . , sn, jn := (12)
times such that M < N a.s, we denote • Let Fk := {Ftk ; t ≥ 0} be the natural filtration generated by
{Ak (t); t ≥ 0}. One should notice that Fk is a filtration of discrete
 
N := {the set of all F − predictable processes u : ]]M, N ]] → A;
UM (g0k , sk1 , j1k ), . . . , (gn−1
k (sk , j k , . . . , sk , j k ), sk , j k )
1 1 n−1 n−1 n n
u(M +) exists}. type in the sense that
n[∞ o where 1 ≤ n ≤ e(k, T ).
The action space is the compact set A := [0, ā] for some 0 < ā < Ftk = D` ∩ {T`k ≤ t < T`+1
k }; D ∈ F k for ` ≥ 0 , t ≥ 0,
` Tk
N have the following properties ` • The Doob-Dynkin’s theorem guarantees the existence of a
+∞. The control space UM `=0 k
Borel function γe(k,T : Hk,e(k,T ) → D [0, T ]; Rq such that


N ⇒u| P for M < P ≤ N a.s. )
• Restriction: u ∈ UM ]]M,P ]] ∈ UM where F0k = {Ω, ∅} and FTk k = σ(T1k , . . . , Tm
k , ∆Ak (T k ), . . . , ∆Ak (T k )) k k,m k,u k
m
1 m γm(o )(0) = x = X (0) and
• Concatenation: If u ∈ UM N and v ∈ U P for M < N < P a.s, then for m ≥ 1.  k

N k,g k
k k
γe(k,T ) Ξe(k,T )(Ae(k,T )(ω)) (t) = X k,u (t, ω) (13)
(u ⊗N v)(·) ∈ UMP , where
• The number of periods is given by m := e(k, T ) := d22k T e, where
 dxe is the smallest integer greater or equal to x ≥ 0. for ω and all t ∈ [0, T ].
u(r); if M < r ≤ N
(u ⊗N v)(r) := (1) k,Tmk
v(r); if N < r ≤ P. • For n < m, let UT k be the set of Fk -predictable processes of • We start with the map Vkm : Hk,m → R defined by
n
N and G ∈ F , we have the form m
• Finite Mixing: For every u, v ∈ UM
 
v k (t) = k 1 k < t ≤ Tk, Vkm(okm) := ξ γm
k (ok ) ; ok ∈ Hk,m.
X
M vj−1 ; T (7)
k k
{Tj−1<t≤Tj } n m m m
N.
u1G + v1Gc ∈ UM j=n+1
By construction, Vkm is a continuous function.
k
where, for each j = n + 1, . . . , m, vj−1 is an A-valued FTk k -
• Countable mixing on deterministic times: Given a sequence j−1
• Lemma: Let Vki−1 : Hk,i−1 → R be the function defined by
of controls u1, u2, . . . in Ust for s < t and a sequence of disjoint measurable random variable. To keep notation simple, we use Z
k,m k,Tmk k
Vi−1(o k,i−1 ) := sup k
Vi o k,i−1 k k k
k k k
, ai−1, si , ji ν (dsi , dji )
sets D1, D2, . . . in Fs, we have the shorthand notation Un := UT k ; 0 ≤ n < m.
∞ n aki−1∈A Sk
k,m k,m
ui11Di ∈ Ust. • For uk ∈ U0 and v k ∈ Un with n < m, we write
X
ok,i−1 ∈ Hk,i−1. Then, Vki−1 is upper semianalytic and for every
i=1 
(uk ⊗n v k ) := (uk0 , . . . , ukn−1, vnk , . . . , vm−1
k ).  > 0, there exists an analytically measurable function Ck,i−1 :
We are interested in the stochastic optimal control problem Hk,i−1 → A which realizes
our goal is to find an -optimal policy u∗, ∈ U0T (when exists) • For every control u ∈ U0 there exist a sequence {uk } ⊂ Z
S k,m k
Vi−1(ok,i−1 )≤ k k,i−1  k,i−1 k k


Vi o ,Ck,i−1(o ),sm,ji ν k (dski ,djik )+
U ⊂ U0 such that
∗, 
h i
E ξ(X u ) = sup E ξ X φ − .


(2) k≥1 0 Sk
Z T
φ∈U0T
E | uk (s) − u(s) |2 ds → 0, k ↑ ∞. for every ok,i−1 ∈ Hk,i−1.
when  = 0, we say that u0 is an optimal control and where 0
• Proposition: For each j = m − 1, . . . , 0 and a control uk ∈
{X φ; φ ∈ U0T } is a given family of F-adapted controlled processes. • The Euler-Maruyama scheme related to the SDE driven by the k,m
Example, U0 associated with a list of universally measurable functions
Brownian motion is given by
Case (A) dX u(t) = α(t, X u, u(t))dt + σ(t, X u, u(t))dB(t), (gjk )m−1
j=0 , we have
X k (Tnk ) = X k (Tn−1
k ) + α(T k , X k , uk )∆T k
n−1 k n−1 n
Tn−1 k k k k k,g k k
Case (B) dX u(t) = α(t, X u, u(t))dt + σdBH (t), k , X k , uk )∆Ak (T k ), (8) V (Tj , u ) = Vj (Ξj (Aj )) a.s. (14)
+ σ(Tn−1 k
Tn−1 n−1 n
k,
We assume the following regularity properties on the payoff Moreover, for every  > 0, there exists a control uj defined by
with n = 1, · · · , m − 1, X k (0) = 0 and
functional:
• The discrete-time value process V k (·, uk ) is the unique process k,  k,g k k
uj := Ck,j (Ξj (Aj )); j = m − 1, . . . , 0 (15)
• (A1): The payoff ξ : DnT → R is bounded and there exists
satisfying the Dynamic Programming Equation (DPE),
γ ∈ (0, 1] and a constant C > 0 such that " # which realizes
|ξ(f ) − ξ(g)| ≤ C( sup kf (t) − g(t)kRn )γ
 
(3) V k (Tnk , uk ) = ess sup E V k Tn+1 k , uk,n−1 ⊗ θ k | G k ;
0≤t≤T n n n k k k  k k k k, k 
k,n+1
θnk ∈Un (9) V (Tj , u ) ≤ E V (Tj+1, u ⊗j uj )|Gj +  a.s (16)
for every f, g ∈ DnT . k k k k,uk
for every j = m − 1, . . . , 0.
V (Tm, u ) = ξ(X ) q.c.
• For a given controlled Wiener functional u 7→ X(·, u), we define
 k k k

h i • Theorem Convergence lim E V (Te(k,t), u ) − V (t, u) =

0 ≤ n ≤ m − 1, V k (Tnk , uk ) := ess supφk ∈U k,m E ξX k (uk ⊗n φk )Gnk . k→+∞

h i
V (t, u) := ess sup E ξX (u ⊗t v)|Ft ; 0 ≤ t < T, u ∈ U0, (4) k,e(k,T )
n
0, 0 ≤ t ≤ T, for every sequence uk ∈ U0 such that
v∈UtT After this discretization for find the optimal control we need the
pathwise form of the dynamic programming equation with this limk→+∞ uk = u in L2a(P × Leb).
where V (T, u) := ξ(X u) a.s and the process V (·, u) has to be and using the select measurable theorem we can solve the pro- • Theorem (Optimal Control for Original Problem) Then for any
viewed backwards. blem.  > 0, the control φ∗,k constructed via V k (15) satisfies: φ∗,k ∈ U0
• Moreover, V is an U0-supermartingale, in the sense that V (·, u) and h i h i
E ξX φ∗,k, ≥ sup E ξX u − 



is an F-supermartingale for each u ∈ U0 and, it satisfies the Dy-
Pathwise Dynamic Programming Equation u∈U0T
namic Programming Equation (DPE)
for every k sufficiently large.
V (t, u) = ess sup E [V (t + h, (u ⊗t θ)) | Ft] , h > 0, • Let us define This solve the equation 2 and we have the -optimal control.
θ∈Utt+h
Ik := {−1, 1} and Sk := (0, +∞) × Ik .
with terminal condition V (T, u) = ξ(X u) for every u ∈ U0. To
solve the problem in the non-Markovian case, we discretized the The n-fold Cartesian product of Sk is denoted by Snk and a gene- Referências
problem as in the following section. ric element of Snk will be denoted by bkn := (sk1 , j1k , . . . , skn, jnk ) ∈ Snk
where (skr , jrk ) ∈ (0, +∞) × Ik for 1 ≤ r ≤ n. Let us define [1] Leão,D. Ohashi, A. and Simas, A. B. (2017). A weak version
of path-dependent functional Itô calculus. To appear in Annals
Discretization
 
Akn := ∆T1k , σ1k , . . . , ∆Tnk , σnk ∈ Snk a.s of Probability. arXiv: 1707.04972.
[2] Leão, D., Ohashi, A. and Souza, F. Optimal mean-variance
• For discretize we start setting T0k := 0 and Tnk := inf{Tn−1
k <t< One should notice that FTk k = (Akn)−1(B(Snk)), where B(Skn) is the hedging strategies with rough stochastic volatility. In prepara-
n
k )| = 2−k }, n ≥ 1.
∞; |B(t) − B(Tn−1 Borel sigma algebra generated by Snk; n ≥ 1. tion.