STOCHASTIC PROCESSES, JANUARY 10th 2022

3 hours. No documents allowed.

Please write the answers to the Course Questions and the solution of two exercises on three different
sheets.

Course Questions. Please justify all answers by giving a fair amont of details.
(1) What does it mean that the sequence of random variables (Xn ) is tight?
(a) (Xn ) converges in law: does this mean that (Xn ) is tight?
(b) limn ϕXn (t) exists for every t ∈ R: does this mean that (Xn ) is tight? Here of course
ϕX (t) := E[exp(itX)].
(2) What does it mean that a family of random variables is Uniformly Integrable (UI)?
(a) Y ∈ L1 is defined on the probability space (Ω, F, P): is the family {E[Y |G] : G sub-σ-algebra
of F} UI?
(b) limn Xn exists in L1 . Is (Xn ) UI?
(c) (Yn ) is UI and Yn → Y in law. Is it true that Yn → Y in L1 ? Is it true that E[Yn ] → E[Y ]?
(3) (Mn ) is a martingale with bounded increments (i.e., supn kMn+1 − Mn k∞ < ∞) and τ is a
stopping time (we are on a filtered space).
(a) P(τ < ∞) = 1: is it true that E[Mτ ] = E[M0 ] ?
(b) E[τ ] < ∞: is it true that E[Mτ ] = E[M0 ] ? P∞
(4) X is a Q-MC on the state space E and, for x ∈ E, Nx := n=0 1Xn =x . Show that either
Px (Nx = ∞) = 1 or there exists c > 0 such that Ex [exp(cNx )] < ∞.
(5) (Xn ) is an irreducible Q-MC on E. Explain why the existence of a non contant Q-superharmonic
function u (i.e. Qu ≤ u) which is bounded below implies that (Xn ) is transient.

Exercise 1. We work on the state space E = Z and we consider the stochastic matrix defined for
n ∈ Z \ {0} by Q(n, n + 1) = p(n) ∈ (0, 1) and Q(n, n − 1) = q(n) = 1 − p(n) and Q(0, 0) = r ∈ (0, 1),
Q(0, +1) = Q(0, −1) = (1 − r)/2. We assume that p(n) = q(−n) for every n 6= 0 and that there exists
β ∈ R and c > 1 such that for n → ∞
 
p(n) β 1
= 1− +O .
q(n) n nc
X := (Xj )j=0,1,... is a Q-MC.

(1) Show that X is irreducible an aperiodic.

(2) Show that a bounded non constant Q-harmonic function u (i.e., Qu = u) exists if and only if
β < −1.
Obs.: note that u(n + 1) − u(n) = (u(1) − u(0)) nj=1 (q(j)/p(j)) for n = 1, 2, . . .
Q

(3) Consider, for l < r, the set Dl,r := {l, l + 1, . . . , r} and τl,r := inf{j = 0, 1, . . . : Xj 6∈ Dl,r }.
Express Pn (Xτl,r = l − 1) in terms of a Q-harmonic function u.
Obs.: this is almost a course question, please give details for each step.
(4) Note that, by the course question (5), (2) implies that X is transient for β < −1. Show that X
is transient if and only if β < −1.
Hint: this is of course a matter of showing recurrence for β ≥ −1. And for this it may be of help
to exploit the answer to the previous question.
1
2 STOCHASTIC PROCESSES, 10-01-2022

(5) Remark that for n = 1, 2, . . .

p(0)p(1) . . . p(n − 1)
v(n) := ,
q(1)q(2) . . . q(n)

satisfies vQ(n) = v(n) for n = 1, 2, . . . and build an invariant measure µ for the Q-MC.
(6) Show that the chain is positive recurrent if and only if β > 1 (equivalently, that the chain is null
recurrent if and only if β ∈ [−1, 1]).
Obs.: Hence, by the course question (5), there cannot exist bounded (sub/super) harmonic func-
tions for β ≥ −1.
(7) Show that for β > 1 there does not exist P a non constant function u which is subharmonic (or
superharmonic) for Q and that satisfies x∈E |u(x)|µ(x) < ∞.
(8) Nevertheless, build a Q-harmonic function for β ≥ −1.
Obs.: of course this should be unbounded. And particularly so if β > 1.

Exercise 2. In this exercise (Xn )n=1,2,... is a sequence of IID random variables. We use the standard
notation Sn = X1 + . . . + Xn . The aim is showing that
 
|Sn |  
E sup < ∞ ⇐⇒ E |X1 | log+ |X1 | < ∞ . (?)
n n

In what follows you can use without proof the following generalization of a result that we have proven
in the course: if X and Y are non negative random variables defined on the same probability space then
if λP(Y ≥ λ) ≤ E[X; Y ≥ λ] for every positive λ, then E[Y ] ≤ 2(1 + E[X log+ X]).
Without loss of generality we make the assumption that P(|X1 | ≤ 1) > 0 (this will be used only at
the very end of the exercise).

(1) (Course question). Show that if (Zn ) is a submartingale with Zn ≥ 0 for every n, then for every
λ > 0 we have
! " #
λP sup Zk ≥ λ ≤ E Zn ; sup Zk ≥ λ .
k∈{0,...n} k∈{0,...n}

(2) Show that

" #


E sup Zk ≤ 2 1 + E Zn log+ Zn .
k∈{0,...n}

Obs.: this of course is the generalization of Doob’s Lp , p > 1, inequality to p = 1.

(3) (Course question). Explain in detail why, if X1 ∈ L1 , we have Snn = E[X1 |Sn , Sn+1 , . . .] for every
n = 1, 2, . . ..
(4) Show that, for every n, the process
Sn Sn−1 S2
, , . . . , , X1 , X1 , X1 , . . .
n n−1 2
is a martingale for an adequate choice of the filtration (please, give it explicitly).
(5) Show the ⇐= part of (1).
(6) We set now
τ := inf {n = 1, 2, . . . : |Xn | > n} .
 STOCHASTIC PROCESSES, 10-01-2022 3

Show that
    ∞
|Xn | |Xτ | X 1
E sup ≥ E ;τ < ∞ = P (τ > n) E [|Xn |; |Xn | > n]
n n τ n=1
n
∞
X 1
≥ P(τ = ∞) E [|X1 |; |X1 | > n] .
n=1
n
Obs.: three questions.
(7) Show that E[|X1 |] < ∞ implies P(τ = ∞) P > 0.
Hint: show first that E[|X1 |] < ∞ implies n P(|Xn | > n) < ∞.
(8) By putting (6) and (7) together show that E[supn |Xn |/n] < ∞ implies E[|X1 | log+ |X1 |] < ∞.
(9) Therefore, to complete the proof of (1) (i.e., to complete the proof of =⇒), it suffices to show
that for X1 ∈ L1 we have that
" τ −1 #
1X E[|X1 |]
E |Xk | ; 1 < τ < ∞ ≤ . (A)
τ k=1 P(|X1 | ≤ 1)
Prove (A).
Hint: first show that E[|Xk | | τ = n] = E[|Xk | | |Xk | ≤ k] for k < n.
4 STOCHASTIC PROCESSES, 10-01-2022

Solution.

Exercise 1.

(1) The chain is irreducible since one can go from m to p in m − p consecutive steps of positive
probability. And from m to m in two steps. The chain is aperiodic since it is irreducible and
Q(0, 0) > 0.
(2) Let u be harmonic for Q. ThenQ one directly verifies that the formula suggested is correct and
u(n + 1) − u(n) = (u(1) − u(0)) nj=1 (q(j)/p(j)) for n = 1, 2, . . .. and by symmetry we have
n
Y n
Y
u(−n + 1) − u(−n) = (u(−1) − u(0)) (p(−j)/q(−j)) = (u(−1) − u(0)) (q(j)/p(j))
j=1 j=1

for the same values of n. So we have functions that are harmonic in the positive and negative
semi axes, but the crucial point is to verify that we can match these two functions so that they are
harmonic the full line. The harmonicity condition in 0 is (u(1) + u(−1))(1 − r)/2 + u(0)r = u(r),
i.e. u(1) + u(−1) − 2u(r) = 0, i.e. u(1) − u(0) = −(u(−1) − u(0)). Therefore every harmonic
function is equal to a + bu where a and b are constants and we have chosen u(1) − u(0) = 1 and
u(0) = 0 so u(−n) = −u(n) for every n, u(1) = 1 and for n ≥ 2
n k−1
X Y q(j)
u(n) = .
j=1 j=1
p(j)

Since from the hypothesis we have that (we assume without loss of generality that c ≤ 2)
 
q(n) β 1
= 1+ +O .
p(n) n nc
Qk Pk c β
j=1 (q(j)/p(j)) = exp( j=1 (β/j + O(1/j )) which behaves as k → ∞ as Ck for a C > 0.
Hence u is bounded if and only if β < −1.
(3) Since the transition rates are uniformly bounded from below for a finite number of values, the
chain exits from this segments almost surely. Thus τl,r < ∞ almost surely. Choose u a non
constant Q-harmonic function (we built them in the previous question). We use now the fact
that (u(Xn )) is a bounded martingale and apply the optional stopping theorem at the first time
X· touches l − 1 or r + 1 In particular, the optional stopping theorem shows that for n ∈ Dl,r
u(n) = u(l − 1)Pn (Xτl,r = l − 1) + u(r + 1)(1 − Pn (Xτl,r = l − 1) .
u(n)−u(r+1)
This gives Pn (Xτl,r = l − 1) = u(l−1)−u(r+1) . Obs.: some people decided to show that n 7→
Pn (Xτl,r = l − 1) is harmonic inside the interval. This is correct (and accepted as an answer to
this question), but it is of no help for the next question. The point was to use one of the non
trivial u built in the previous question.
(4) For β ≥ −1 we have that limn u(n) = ∞ (we are using here precisely the choice of u made above,
in particular u(0) = 0, u(1) = 1 and u is increasing). So P1 (Xτ1,r = 0) = u(1)−u(r+1)
u(0)−u(r+1)
→ 1 as
r → ∞. Note also that {Xτ1,r = 0} is an event that increases in r and converges to the event
{there exists n such that Xn = 0}, i.e. the event T0 < ∞ (T0 first hit time to 0 for positive
times). Hence P1 (T0 = ∞) = 0. But P0 (T0 = ∞) = P1 (T0 = ∞) by the strong Markov property
(condition of the first time of hitting ±1 and use the symmetry), which means recurrence for
β ≥ −1.
(5) By direct checking one sees that if we define µ(n) = v(|n|) for n 6= 0 and µ(0) = 2p(0)/(1 − r),
then µ is an invariant measure. Once again: one needs to solve µQ(x) = µ(x) for every x,
included x = 0!
 STOCHASTIC PROCESSES, 10-01-2022 5

(6) The chain in recurrent for β ≥ −1, so in this case µ is the unique invariant measure up to a
multiplicative constant. By exploiting the asymptotic behavior of p(n)/q(n) one sees that µ can
be normalized iff β > 1, hence the chain is positive recurrent iff β > 1.
(7) Let (Xn ) be chosen with X0 ∼ µ (µ is now normalized to be a probability.
P So the law of u(Xn )
does not depend on n and therefore E[|u(Xn )|] = E[|u(X0 )|] = x |u(x)|µ(x) < ∞. Note that
this P that |Qu(x)| < ∞ for every x: in fact Q|u(x)| < ∞ for every x because
P implies in particular
x Q|u(x)|µ(x) = x |u(x)|µ(x) < ∞. With this and the hypothesis on u we verify that
(u(Xn )) is a submartingale. Since it is bounded in L1 it converges a.s.. But this is impossible
because there exist x and y such that u(x) 6= u(y) and both x and y are visited infinitely often
by recurrence.
(8) We have already built explicitly all the harmonic functions in the solution to question (2), for
every β. For β ≥ −1, the non trivial harmonic functions are unbounded and one readily sees
also that, for β > 1, they are not in L1 (µ). Note that building explicitly a non trivial harmonic
function is not an obvious matter: in particular if we were working on E = {0, 1, . . .} (by choosing
the transition probabilities from 0 in such a way one stays on E) the harmonicity condition in 0
implies that such an harmonic function is constant.
Exercise 2. In this exercise (Xn )n=1,2,... is a sequence of IID random variables. We use the standard
notation Sn = X1 + . . . + Xn . The aim is showing that
 
|Sn |  
E sup < ∞ ⇐⇒ E |X1 | log+ |X1 | < ∞ . (1)
n n
In what follows you can use without proof the following generalization of a result that we have proven
in the course: if X and Y are non negative random variables defined on the same probability space then
if λP(Y ≥ λ) ≤ E[X; Y ≥ α] for every positive λ, then E[Y ] ≤ 2(1 + E[X log+ X]).
Without loss of generality we make the assumption that P(|X1 | < 1) > 0 (this will be used only at
the very end of the exercise).

(1) See course.

(2) Apply the above given inequality to Y = supk∈{0,...n} Zk and X = Zn .
(3) See course.
(4) Fix n and write
Sn Sn−1 S2
M0 := , M1 := , . . . , Mn−2 := , Mn−1 := X1 , Mn := X1 , Mn+1 := X1 , . . .
n n−1 2
so, by the previous question, (Mk ) is a martingale with respect to (Fk ), with Fk = σ(Sn−k , Sn−k+1 , . . .)
for k ≤ n − 1 and Fk = Fn−1 for every larger k.
(5) We now apply the result of question (2) for a fixed n to the positive submartingale (|Mn |) and
then use (MON) when sending n to infinity.
(6) First and third inequality are immediate. As for the second one, it is enough to see (using
independence) that E[Xn 1τ =n ] = E[X Pn 1Xn >n 1X1 6 1 . . . 1Xn−1 6 n−1 ] = E[Xn 1Xn >n ]P(τ > n)
(7) It is classical that E[|X1 |] 6Q∞ iff P(|X1 | > n) < ∞. Then P one can write explicitly P(τ =
∞) = ∞ ∞
Q
n=1 P(|X 1 | ≤ n) = n=1 (1 − P(|X
P 1 | > n)) = exp( n log(1 − P(|X1 | > n))). But the
sum in the exponent is finite because P(|X1 | > n) < ∞ so P(τ = ∞) > 0.
(8) It is enough to note the existence of a universal constant C such that one has
∞
E[X1 n1 1X1 >n ] > CE[X1 log+ (X1 )]. This is simply an application of Fubini-Tonelli and of the
P
n=1 P
fact that n<x (1/n) ∼ log x for x → ∞.
(9) The equality E[|Xk | | τ = n] = E[|Xk | | |Xk | 6 k] comes from the independence of the variables
Xi . The second bound comes from the fact that
E[|Xk |||Xk | 6 k] = E[Xk 1Xk 6 k ]/P(|Xk | 6 k) 6 E[|X1 |]/P(|X1 | ≤ 1). To conclude, write
6 STOCHASTIC PROCESSES, 10-01-2022

 1 Pτ −1 ∞ Pn−1
1
 P 
E τ k=1 |X k | ; 1 < τ < ∞ = E n k=1 |Xk | 1τ =n and then use the fact that
n=1
E[E[|Xk | | τ = n]] 6 E[|X1 |]/P(|X1 | < 1) < ∞.