Outline

1. Objectives of time series analysis. Examples.

2. Overview of the course.

3. Time series models.

4. Time series modelling: Chasing stationarity.

49
 A Time Series

400

350

300

250

200

150

100

50

0
0 1000 2000 3000 4000 5000 6000 7000

4
 A Time Series

400

350

300

250

200

150

100

50

0
1960 1965 1970 1975 1980 1985 1990
year

5
 A Time Series

400

350

300

250

200
$

150

100

50

0
1960 1965 1970 1975 1980 1985 1990
year

6
 A Time Series

SP500: 1960−1990
400

350

300

250

200
$

150

100

50

0
1960 1965 1970 1975 1980 1985 1990
year

7
 A Time Series

SP500: Jan−Jun 1987

340

320

300

280
$

260

240

220
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year

8
 A Time Series

SP500 Jan−Jun 1987. Histogram

30

25

20

15

10

0
240 250 260 270 280 290 300 310
$

9
 A Time Series

SP500: Jan−Jun 1987. Permuted.

340

320

300

280
$

260

240

220
0 20 40 60 80 100 120

10
 Objectives of Time Series Analysis

1. Compact description of data.

2. Interpretation.
3. Forecasting.

4. Control.
5. Hypothesis testing.

6. Simulation.

11
 Classical decomposition: An example

Monthly sales for a souvenir shop at a beach resort town in Queensland.

(Makridakis, Wheelwright and Hyndman, 1998)

4
x 10
12

10

0
0 10 20 30 40 50 60 70 80 90

12
 Transformed data

12

11.5

11

10.5

10

9.5

8.5

7.5

7
0 10 20 30 40 50 60 70 80 90

13
 Trend

12

11.5

11

10.5

10

9.5

8.5

7.5

7
0 10 20 30 40 50 60 70 80 90

14
 Residuals

1.5

0.5

−0.5

−1
0 10 20 30 40 50 60 70 80 90

15
 Trend and seasonal variation

12

11.5

11

10.5

10

9.5

8.5

7.5

7
0 10 20 30 40 50 60 70 80 90

16
 Objectives of Time Series Analysis

1. Compact description of data.

Example: Classical decomposition: Xt = Tt + St + Yt .

2. Interpretation. Example: Seasonal adjustment.

3. Forecasting. Example: Predict sales.

4. Control.
5. Hypothesis testing.

6. Simulation.

17
 Unemployment data

Monthly number of unemployed people in Australia. (Hipel and McLeod, 1994)

5
x 10
8

7.5

6.5

5.5

4.5

4
1983 1984 1985 1986 1987 1988 1989 1990

18
 Trend
5
x 10
8

7.5

6.5

5.5

4.5

4
1983 1984 1985 1986 1987 1988 1989 1990

19
 Trend plus seasonal variation

5
x 10
8

7.5

6.5

5.5

4.5

4
1983 1984 1985 1986 1987 1988 1989 1990

20
 Residuals
4
x 10
8

−2

−4

−6
1983 1984 1985 1986 1987 1988 1989 1990

21
Predictions based on a (simulated) variable

5
x 10
8

7.5

6.5

5.5

4.5

4
1983 1984 1985 1986 1987 1988 1989 1990

22
 Objectives of Time Series Analysis

1. Compact description of data:

Xt = Tt + St + f (Yt ) + Wt .

2. Interpretation. Example: Seasonal adjustment.

3. Forecasting. Example: Predict unemployment.

4. Control. Example: Impact of monetary policy on unemployment.

5. Hypothesis testing. Example: Global warming.

6. Simulation. Example: Estimate probability of catastrophic events.

23
 Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis

4. State space models(?)

24
 Overview of the Course

1. Time series models

(a) Stationarity.
(b) Autocorrelation function.
(c) Transforming to stationarity.
2. Time domain methods

3. Spectral analysis
4. State space models(?)

25
 Overview of the Course

1. Time series models

2. Time domain methods
(a) AR/MA/ARMA models.
(b) ACF and partial autocorrelation function.
(c) Forecasting
(d) Parameter estimation
(e) ARIMA models/seasonal ARIMA models

3. Spectral analysis
4. State space models(?)

26
 Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis
(a) Spectral density
(b) Periodogram
(c) Spectral estimation
4. State space models(?)

27
 Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis

4. State space models(?)

(a) ARMAX models.
(b) Forecasting, Kalman filter.
(c) Parameter estimation.

28
 Time Series Models

A time series model specifies the joint distribution of the se-

quence {Xt } of random variables.
For example:

P [X1 ≤ x1 , . . . , Xt ≤ xt ] for all t and x1 , . . . , xt .

Notation:
X1 , X2 , . . . is a stochastic process.
x1 , x2 , . . . is a single realization.
We’ll mostly restrict our attention to second-order properties only:
EXt , E(Xt1 , Xt2 ).

29
 Time Series Models

Example: White noise: Xt ∼ W N (0, σ 2 ).

i.e., {Xt } uncorrelated, EXt = 0, VarXt = σ 2 .
Example: i.i.d. noise: {Xt } independent and identically distributed.

P [X1 ≤ x1 , . . . , Xt ≤ xt ] = P [X1 ≤ x1 ] · · · P [Xt ≤ xt ].

Not interesting for forecasting:

P [Xt ≤ xt |X1 , . . . , Xt−1 ] = P [Xt ≤ xt ].

30
 Gaussian white noise
Z xt
1 −x2 /2
P [Xt ≤ xt ] = Φ(xt ) = √ e dx.
2π −∞
2.5

1.5

0.5

−0.5

−1

−1.5

−2

−2.5
0 5 10 15 20 25 30 35 40 45 50

31
 Gaussian white noise

2.5

1.5

0.5

−0.5

−1

−1.5

−2

−2.5
0 5 10 15 20 25 30 35 40 45 50

32
 Time Series Models

Example: Binary i.i.d. P [Xt = 1] = P [Xt = −1] = 1/2.

1

0.8

0.6

0.4

0.2

−0.2

−0.4

−0.6

−0.8

−1

0 5 10 15 20 25 30 35 40 45 50

33
 Random walk
Pt
St = i=1 Xi . Differences: ∇St = St − St−1 = Xt .
8

−2

−4
0 5 10 15 20 25 30 35 40 45 50

34
 Random walk

ESt ? VarSt ?
10

−5

−10

−15
0 5 10 15 20 25 30 35 40 45 50

35
 Random Walk

Recall S&P500 data. (Notice that it’s smooth)

SP500: Jan−Jun 1987
340

320

300

280
$

260

240

220
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year

36
 Random Walk

Differences: ∇St = St − St−1 = Xt .

SP500, Jan−Jun 1987. first differences
10

0
$

−2

−4

−6

−8

−10
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year

37
 Trend and Seasonal Models
P
Xt = Tt + St + Et = β0 + β1 t + i (βi cos(λi t) + γi sin(λi t)) + Et
6

5.5

4.5

3.5

2.5
0 50 100 150 200 250

38
 Trend and Seasonal Models

Xt = Tt + Et = β0 + β1 t + Et
6

5.5

4.5

3.5

2.5
0 50 100 150 200 250

39
 Trend and Seasonal Models
P
Xt = Tt + St + Et = β0 + β1 t + i (βi cos(λi t) + γi sin(λi t)) + Et
6

5.5

4.5

3.5

2.5
0 50 100 150 200 250

40
Trend and Seasonal Models: Residuals

0.5

0.4

0.3

0.2

0.1

−0.1

−0.2

−0.3

−0.4

−0.5
0 50 100 150 200 250

41
 Time Series Modelling

1. Plot the time series.

Look for trends, seasonal components, step changes, outliers.
2. Transform data so that residuals are stationary.
(a) Estimate and subtract Tt , St .
(b) Differencing.
√
(c) Nonlinear transformations (log, ·).
3. Fit model to residuals.

42
 Nonlinear transformations

Recall: Monthly sales. (Makridakis, Wheelwright and Hyndman, 1998)

4
x 10 12
12

11.5

10 11

10.5
8
10

9.5
6

4
8.5

8
2

7.5

0 7
0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90

43
 Time Series Modelling

1. Plot the time series.

Look for trends, seasonal components, step changes, outliers.
2. Transform data so that residuals are stationary.
(a) Estimate and subtract Tt , St .
(b) Differencing.
√
(c) Nonlinear transformations (log, ·).
3. Fit model to residuals.

44
 Differencing

Recall: S&P 500 data.

SP500: Jan−Jun 1987 SP500, Jan−Jun 1987. first differences
340 10

320
6

4
300

280 0
$

$
−2

260
−4

−6
240

−8

220 −10
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year year

45
 Differencing and Trend

Define the lag-1 difference operator, (think ‘first derivative’)

∇Xt = Xt − Xt−1 = (1 − B)Xt ,

where B is the backshift operator, BXt = Xt−1 .

• If Xt = β0 + β1 t + Yt , then

∇Xt = β1 + ∇Yt .
Pk
• If Xt = i=0 βi ti + Yt , then

∇k Xt = k!βk + ∇k Yt ,

where ∇k Xt = ∇(∇k−1 Xt ) and ∇1 Xt = ∇Xt .

46
 Differencing and Seasonal Variation

Define the lag-s difference operator,

∇s Xt = Xt − Xt−s = (1 − B s )Xt ,

where B s is the backshift operator applied s times, B s Xt = B(B s−1 Xt )

and B 1 Xt = BXt .
If Xt = Tt + St + Yt , and St has period s (that is, St = St−s for all t), then

∇s Xt = Tt − Tt−s + ∇s Yt .

47
 Time Series Modelling

1. Plot the time series.

Look for trends, seasonal components, step changes, outliers.
2. Transform data so that residuals are stationary.
(a) Estimate and subtract Tt , St .
(b) Differencing.
√
(c) Nonlinear transformations (log, ·).
3. Fit model to residuals.

48