Advanced Econometrics III

Lecture 10 Part One: Unit Root Processes

Katerina Petrova
Random Walk
Consider an AR(1) process with a unit root: ρ = 1
Xt = ρXt 1 + εt = X0 + St (1)
where
St = ∑tj=1 εj , εj iid 0, σ2 (2)
is called a random walk1 . Elementary properties (exercise):
1 cov (St , Sk ) = σ2 min (t, k ). (St ) is nonstationary with
V (St ) = σ2 t.
2 St is a martingale w.r.t. Ft = σ (εt , ..., ε1 )
3 St has independent increments: St2 St1 and Sk2 Sk1 are
independent for all k1 < k2 < t1 < t2
4 CLT: t 1/2 St !d N 0, σ2 , so St σN (0, t ) as t ! ∞
1 Nomeclature: a drunk walks in a straight line taking random steps forward
εj = 1 or backwards εj = 1 with probability equal to 1/2. Then St is his
position after t steps.
Random Walk

Consider the denominator of the OLS estimator of ρ in (1):

∑nt=1 Xt2 1 = ∑nt=1 St2 1 + 2X0 ∑nt=1 St 1 + nX02 .

Rates of convergence:
Z n 1
σ2 2
E ∑nt=1 St2 1 = σ2 ∑nt =11 t σ2 tdt n ,
1 2
so ∑nt=1 St2 1 = Op n 2
Random Walk

By the triangle and Lyapounov inequalities:

k∑nt=1 St 1 kL 1 ∑nt=1 kSt 1 kL 2 = σ ∑tn=11 t 1/2

Z n 1
σ t 1/2 dt = O n3/2 .
1

We conclude that
1 n 1 n 1
∑ X2 = ∑ S2 + Op p . (3)
n 2 t =1 t 1
n 2 t =1 t 1
n
Random Walk

Let’s informally analyse the leading term of (3): informally,

the CLT gives
2
1 n 1 t 1
2 ∑t =1 St2 1 = ∑nt=1 p ∑tj =11 εj
n n n t
1 n t
∑ (σN (0, 1))2 (t ! ∞)
n t =1 n
2
1 t
=d σ2 ∑nt=1 N 0, .
n n

This is op (1) if t/n ! 0, so we need to consider t that is a

fraction of n:

t = bnr c for some r 2 [0, 1] .

Random Walk

With this insight, using bnr c /n ! r , we expect the limit

above to depend on

bnr c
N 0, N (0, r )
n

a Gaussian process W (r ) =d N (0, r ) for r 2 [0, 1].

We will characterise this Gaussian process W (r ) as a
standard Brownian motion (BM) or Wiener process.
R1
Since n 1 ∑nt=1 f nt ! 0 f (u ) du, so we may expect a
limit of the form
Z 1
1 n
∑ S2 !d σ 2
W 2 (r ) dr . (4)
n 2 t =1 t 1
0
Random Walk

We now formalise the algebra on the left side of (4):

Z n Z 1
1 n 1 1
∑ S2 = Sb2t c dt = Sb2nr c dr
n 2 t =1 t 1
n2 1 n 1/n
Z 1 2
2 1
= σ p Sbnr c dr
1/n σ n
Z 1
= σ2 Wn (r )2 dr
1/n

where
1 1 bnr c
Wn (r ) := p Sbnr c = p ∑j =1 εj , r 2 [0, 1] (5)
σ n σ n

is called the partial sum process of (εj ).

Random Walk
It is easy to see that EWn (r )2 1, showing that
Z 1/n Z 1/n
1
Wn (r )2 dr = EWn (r )2 dr .
0 L1 0 n
We conclude that
Z 1
1 n 1
∑ S2 =σ 2
Wn (r )2 dr + Op (6)
n 2 t =1 t 1
0 n
where Wn ( ) is the partial sum process in (5).
In order to deduce (4) from (6) we need two major steps:
1. To make sense of the limit on the RHS of (4) by de…ning a
Brownian motion W (r ) on [0, 1] and discuss its properties.
2. To discuss a functional CLT Wn (r ) !d W (r ) that is
uniform on r 2 [0, 1] in order to ensure
R1 2 R1
0
W n ( r ) dr ! d 0
W (r )2 dr .
Brownian motion axioms

A continuous time process fW (t ) : t 0g on a probability space

(Ω, F , P), is a standard Brownian motion (or Wiener process) if:
BM1 W (0) = 0 a.s.
BM2 W (t ) has a.s. continuous sample paths

P (fω : W (ω, .) continuousg) = 1

BM3 For each t 0, W (t )

N (0, t ).
BM4 W (t ) has independent increments, i.e.

W (t2 ) W (t1 ) is independent of W (s2 ) W ( s1 )

for all s1 s2 t1 t2 .
Brownian motion properties

Consider the following conditions:

BM5 E [W (t ) W (s )] = t ^ s.
BM6 W (t ) is a martingale w.r.t. Ft = σ (fW (s ) : s 2 [0, t ]g)

Lemma
If BM3 holds, then conditions BM4, BM5 and BM6 are equivalent.
Brownian motion properties
Proof BM4)BM6.
For s < t

E ( W (t )j Fs ) = E ( (W (t ) W (s ))j Fs ) + W (s )
= E (W (t ) W (s )) + W (s ) = W (s )

by independence of W (t ) W (s ) from Fs , showing the MG

property.

Proof BM6)BM5.

E [W (t ) W (s )] = E [W (t ^ s ) E ( W (t _ s )j Ft ^s )]
h i
= E W (t ^ s )2 = t ^ s.
Brownian motion properties

Proof BM5)BM4.
s 1 s 2 < t1 t2

E (W (t2 ) W (t1 )) (W (s2 ) W (s1 ))

= EW (t2 ) W (s2 ) EW (t2 ) W (s1 )
EW (t1 ) W (s2 ) + EW (t1 ) W (s1 )
= s 2 s 1 s2 + s 1 = 0

so cov (W (t2 ) W (t1 ) , W (s2 ) W (s1 )) = 0. Since

W (t2 ) W (t1 ) and W (s2 ) W (s1 ) are Gaussian random
variables, they are independent.

This completes the proof of the lemma.

Brownian motion properties

Exercise: Suppose that W (t ) is a standard BM and

Ft = σ (fW (s ) : s 2 [0, t ]g) . Show the following:
(i) For any 0 s < t, W (t ) W (s ) N (0, t s ).
p
(ii) For any c 6= 0, B (t ) := c W (t/c ) is a standard BM.
(iii) For any …xed T > 0, B (t ) := W (t + T ) W (T ) is a
standard BM independent of FT .
(iv) M (t ) := W (t )2 t is an Ft martingale.
(v) Strong Markov property of BM: (iii) continues to hold for a
random time T , provided that T is measurable w.r.t.
σ (fW (t ) : t 0g) .
Existence of Brownian motion

It is not at all obvious that a process W (t ) satisfying

BM1-BM3 and one of the equivalent conditions BM4-BM6
actually exists.
The existence of such a process is shown by the following
theorem.
The Fourier series approach below is similar to Wiener’s
original 1923 approach.
Existence of Brownian motion

Theorem
Let ξ j j 1
be i.i.d. N (0, 1) random variables. The process
p
2 ∞ sin [(j 1/2) πt ]
W (t ) = ∑ ξj t 2 [0, 1] , (7)
π j =1 j 1/2
where the in…nite series converges a.s., satis…es BM1-BM3 and
BM5. Hence, W (t ) is a standard BM on [0, 1] .

The random series representation of a BM in (7) is sometimes

called the Karhunen-Loève representation.
Proof modulo continuity
Consider the partial sum of the series in (7):
p
2 n sin [(j 1/2) πt ]
wn ( t ) = ∑ ξj
π j =1 j 1/2
and note that wn ( ) is a martingale w.r.t. σ (ξ n , ξ n 1 , ..., ξ 1 ) with

8 n sin2 [(j 1/2) πt ]

Ewn (t )2 = ∑
π 2 j =1 (2j 1)2
8 ∞ 1 8 ∞ 1
∑ ∑ < ∞.
π 2 j =1 (2j 1) 2 π 2 j =1 j 2

The martingale convergence theorem implies that:

fwn (t ) : n 2 Ng converges a.s. and in L2 so
W (t ) = lim wn (t ) a.s. t 2 [0, 1] .
n !∞
Proof modulo continuity

Putting t = 0 in (7) implies that W (0) = 0 a.s.; also since ξ j is

a Gaussian sequence, W (t ) is Gaussian for each t. Since
E ξ j = 0, E [W (t )] = limn !∞ Ewn (t ) = 0.
We now show that E [W (t ) W (s )] = t ^ s
This will complete the proof of BM1, BM3 and BM5.
We need the trigonometric identity
1
sin x sin y = [cos (x y) cos (x + y )] (8)
2
and the Fourier series of the function f (x ) = jx j:

π 4 ∞ cos [(2j 1) x ]
jx j = ∑ , x 2 ( π, π ) . (9)
2 π j =1 (2j 1)2
Proof modulo continuity
Since W (t ) in (7) converges in L2 , EW (t ) W (s )
2 ∞ sin [(j 1/2) πt ] sin [(j 1/2) πs ]
= ∑ E ξ 2j
π 2 j =1 (j 1/2) 2

8 ∞ sin (2j 1) πt
2 sin (2j 1) πs
2
= ∑
π 2 j =1 (2j 1)2
Using (8) followed by (9): EW (t ) W (s )
( π (t s ) π (t +s )
)
4 ∞ cos (2j 1) 2 ∞ cos (2j 1) 2
= 2 ∑ j =1 ∑ j =1
π (2j 1)2 (2j 1)2
4 π2 π π (t s ) π2 π π (t + s )
=
π2 8 4 2 8 4 2
t +s jt s j
= = t ^s
2 2
completing the proof of BM5.
Proof modulo continuity

It remains to show a.s. continuity of W (t ); this is more

technical and we omit it (see Khoshnevisan (2007) p.166-167
if interested).
The issue is that the continuity of wn (t ) (immediate by
continuity of sin) only guarantees the continuity of its a.s.
limit W (t ) if convergence holds uniformly in t 2 [0, 1], and
proving this uniform a.s. convergence is tricky.