Subject: Statistics

Paper: Stochastic Processes and Time Series

Analysis
Module: Time Series Analysis - The Classical
Models

1 / 27
Development Team

Principal investigator: Prof. Bhaswati Ganguli,

Department of Statistics, University of Calcutta
Paper co-ordinator: Dr. Arindam Sengupta,
Department of Statistics, University of Calcutta
Content writer: Mr. Samopriya Basu & Prof. Sugata Sen Roy,
Department of Statistics, University of Calcutta
Content reviewer: Dr. Indranil Mukhopadhyay,
Indian Statistical Institute, Kolkata

2 / 27
Analysis in the Time Domain

Let Yt be a time series observed at time points t, t = 1, 2, . . . , n.

Time series models are primarily concerned with modelling the

mean and/or the variance of the series, i.e. the focus is on
identifying the average level (devoid of noise) over which the series
runs and the assignable fluctuations from these average levels.

However, in this discussions we will be concerned only with the

modelling of the mean of the series and not of its variability, i.e. in
all the models that will be considered, we will assume that the
variance is a constant.

3 / 27
Analysis in the Time Domain

Let Yt be a time series observed at time points t, t = 1, 2, . . . , n.

Time series models are primarily concerned with modelling the

mean and/or the variance of the series, i.e. the focus is on
identifying the average level (devoid of noise) over which the series
runs and the assignable fluctuations from these average levels.

However, in this discussions we will be concerned only with the

modelling of the mean of the series and not of its variability, i.e. in
all the models that will be considered, we will assume that the
variance is a constant.

3 / 27
Analysis in the Time Domain

Time domain studies of Yt can be broadly divided into three

categories :
I The Classical models
I The Box-Jenkins models
I The Volatility models.

Of course, these models are not mutually exclusive and for a given
time series more than one of these models my be applied to study
its different components.

4 / 27
Analysis in the Time Domain

Time domain studies of Yt can be broadly divided into three

categories :
I The Classical models
I The Box-Jenkins models
I The Volatility models.

Of course, these models are not mutually exclusive and for a given
time series more than one of these models my be applied to study
its different components.

4 / 27
Analysis in the Time Domain

Time domain studies of Yt can be broadly divided into three

categories :
I The Classical models
I The Box-Jenkins models
I The Volatility models.

Of course, these models are not mutually exclusive and for a given
time series more than one of these models my be applied to study
its different components.

4 / 27
Analysis in the Time Domain

Time domain studies of Yt can be broadly divided into three

categories :
I The Classical models
I The Box-Jenkins models
I The Volatility models.

Of course, these models are not mutually exclusive and for a given
time series more than one of these models my be applied to study
its different components.

4 / 27
The Models

I The classical models are usually used to study the trend and
the seasonal pattern of a series.
I The Box-Jenkins models are primarily focussed on the
stationary component of a series.
Although these models also do account for trend and
seasonality, very often a better undersatnding of these
components can be had from the classical model.
I Volatility models are designed to capture the changing
variability or heteroscedasticity in the data and often comes as
a follow-up to the Box-Jenkins models.

5 / 27
The Models

I The classical models are usually used to study the trend and
the seasonal pattern of a series.
I The Box-Jenkins models are primarily focussed on the
stationary component of a series.
Although these models also do account for trend and
seasonality, very often a better undersatnding of these
components can be had from the classical model.
I Volatility models are designed to capture the changing
variability or heteroscedasticity in the data and often comes as
a follow-up to the Box-Jenkins models.

5 / 27
The Models

I The classical models are usually used to study the trend and
the seasonal pattern of a series.
I The Box-Jenkins models are primarily focussed on the
stationary component of a series.
Although these models also do account for trend and
seasonality, very often a better undersatnding of these
components can be had from the classical model.
I Volatility models are designed to capture the changing
variability or heteroscedasticity in the data and often comes as
a follow-up to the Box-Jenkins models.

5 / 27
Components of a Time Series

To understand the nature of a time series, we need to understand

two very important aspects of the series :
Stationary component
A time series (Xt ) is said to be stationary if its characteristics,
particularly its mean and variance, do not change over time.

Evolutionary component
A time series (Et ) whose mean and variance evolve over time is
said to be evolutionary. Such series look different during different
periods.

6 / 27
Components of a Time Series

To understand the nature of a time series, we need to understand

two very important aspects of the series :
Stationary component
A time series (Xt ) is said to be stationary if its characteristics,
particularly its mean and variance, do not change over time.

Evolutionary component
A time series (Et ) whose mean and variance evolve over time is
said to be evolutionary. Such series look different during different
periods.

6 / 27
Evolutionary Component

A series is primarily evolutionary because of the presence of

I Trend (Tt )
I Seasonality (St ).

The trend, Tt , is a smooth long-term movement.

The seasonality, St , is a cyclical movement with period less than a

year.

Cyclical movements with periods greater than a year are not

seasonal and can be incorporated either as a non-linear trend or in
the stationary component.

7 / 27
Evolutionary Component

A series is primarily evolutionary because of the presence of

I Trend (Tt )
I Seasonality (St ).

The trend, Tt , is a smooth long-term movement.

The seasonality, St , is a cyclical movement with period less than a

year.

Cyclical movements with periods greater than a year are not

seasonal and can be incorporated either as a non-linear trend or in
the stationary component.

7 / 27
Evolutionary Component

A series is primarily evolutionary because of the presence of

I Trend (Tt )
I Seasonality (St ).

The trend, Tt , is a smooth long-term movement.

The seasonality, St , is a cyclical movement with period less than a

year.

Cyclical movements with periods greater than a year are not

seasonal and can be incorporated either as a non-linear trend or in
the stationary component.

7 / 27
Evolutionary Component

A series is primarily evolutionary because of the presence of

I Trend (Tt )
I Seasonality (St ).

The trend, Tt , is a smooth long-term movement.

The seasonality, St , is a cyclical movement with period less than a

year.

Cyclical movements with periods greater than a year are not

seasonal and can be incorporated either as a non-linear trend or in
the stationary component.

7 / 27
Trend

8 / 27
Seasonality

9 / 27
Structure

Let Xt be the stationary component. In the classical model, the

series Yt is decomposed either as a
Additive model
Yt = ET + Xt = Tt + St + Xt

or as a
Multiplicative model
Yt = Et × Xt = Tt × St × Xt

10 / 27
Structure

Let Xt be the stationary component. In the classical model, the

series Yt is decomposed either as a
Additive model
Yt = ET + Xt = Tt + St + Xt

or as a
Multiplicative model
Yt = Et × Xt = Tt × St × Xt

10 / 27
Structure

In most situations, a multiplicative model with a log

transformation is used, thus reducing the model to an additive one,

log Yt = log Tt + log St + log Xt .

This is often justified in time series with an exponential trend,

where the log transformtion reduces the trend to a linear function
of time.
In this and subsequent lectures, We will confine ourselves to the
additive model only.

11 / 27
Trend

Trend is the slow long-term movement of a series, which can

I increase with time.
I decrease with time.
I take various other shapes.
It does not include short-term movements, like minor dips or
oscillatory movements in a generally increasing or decreasing series.
Methods of estimating trend

I mathematical curve fitting .

I method of moving averages.

12 / 27
Trend

Trend is the slow long-term movement of a series, which can

I increase with time.
I decrease with time.
I take various other shapes.
It does not include short-term movements, like minor dips or
oscillatory movements in a generally increasing or decreasing series.
Methods of estimating trend

I mathematical curve fitting .

I method of moving averages.

12 / 27
Trend

Trend is the slow long-term movement of a series, which can

I increase with time.
I decrease with time.
I take various other shapes.
It does not include short-term movements, like minor dips or
oscillatory movements in a generally increasing or decreasing series.
Methods of estimating trend

I mathematical curve fitting .

I method of moving averages.

12 / 27
Trend

Trend is the slow long-term movement of a series, which can

I increase with time.
I decrease with time.
I take various other shapes.
It does not include short-term movements, like minor dips or
oscillatory movements in a generally increasing or decreasing series.
Methods of estimating trend

I mathematical curve fitting .

I method of moving averages.

12 / 27
Trend

Trend is the slow long-term movement of a series, which can

I increase with time.
I decrease with time.
I take various other shapes.
It does not include short-term movements, like minor dips or
oscillatory movements in a generally increasing or decreasing series.
Methods of estimating trend

I mathematical curve fitting .

I method of moving averages.

12 / 27
Trend

Trend is the slow long-term movement of a series, which can

I increase with time.
I decrease with time.
I take various other shapes.
It does not include short-term movements, like minor dips or
oscillatory movements in a generally increasing or decreasing series.
Methods of estimating trend

I mathematical curve fitting .

I method of moving averages.

12 / 27
An Increasing Trend

13 / 27
Method of Curve Fitting

I Here the trend is modelled using some mathematical function, e.g.,

I Tt = α + βt,
I Tt = α + βt + γt2 ,
I More generally, Tt = β0 + β1 t + β2 t2 + · · · + βm tm ,
I log Tt = α + βγ t , γ > 0,
 
I Tt = α/ β + exp(−γt) .

I The parameters are then estimated by the least squares method,

i.e., by minimizing with respect to the bj ’s
X
2
Yt − β0 − β1 t − β2 t2 − · · · − βm tm
t
.
14 / 27
Method of Curve Fitting

I Here the trend is modelled using some mathematical function, e.g.,

I Tt = α + βt,
I Tt = α + βt + γt2 ,
I More generally, Tt = β0 + β1 t + β2 t2 + · · · + βm tm ,
I log Tt = α + βγ t , γ > 0,
 
I Tt = α/ β + exp(−γt) .

I The parameters are then estimated by the least squares method,

i.e., by minimizing with respect to the bj ’s
X
2
Yt − β0 − β1 t − β2 t2 − · · · − βm tm
t
.
14 / 27
Method of Curve Fitting

I Here the trend is modelled using some mathematical function, e.g.,

I Tt = α + βt,
I Tt = α + βt + γt2 ,
I More generally, Tt = β0 + β1 t + β2 t2 + · · · + βm tm ,
I log Tt = α + βγ t , γ > 0,
 
I Tt = α/ β + exp(−γt) .

I The parameters are then estimated by the least squares method,

i.e., by minimizing with respect to the bj ’s
X
2
Yt − β0 − β1 t − β2 t2 − · · · − βm tm
t
.
14 / 27
Method of Curve Fitting

I Here the trend is modelled using some mathematical function, e.g.,

I Tt = α + βt,
I Tt = α + βt + γt2 ,
I More generally, Tt = β0 + β1 t + β2 t2 + · · · + βm tm ,
I log Tt = α + βγ t , γ > 0,
 
I Tt = α/ β + exp(−γt) .

I The parameters are then estimated by the least squares method,

i.e., by minimizing with respect to the bj ’s
X
2
Yt − β0 − β1 t − β2 t2 − · · · − βm tm
t
.
14 / 27
Method of Curve Fitting

I Here the trend is modelled using some mathematical function, e.g.,

I Tt = α + βt,
I Tt = α + βt + γt2 ,
I More generally, Tt = β0 + β1 t + β2 t2 + · · · + βm tm ,
I log Tt = α + βγ t , γ > 0,
 
I Tt = α/ β + exp(−γt) .

I The parameters are then estimated by the least squares method,

i.e., by minimizing with respect to the bj ’s
X
2
Yt − β0 − β1 t − β2 t2 − · · · − βm tm
t
.
14 / 27
Method of Curve Fitting

I Here the trend is modelled using some mathematical function, e.g.,

I Tt = α + βt,
I Tt = α + βt + γt2 ,
I More generally, Tt = β0 + β1 t + β2 t2 + · · · + βm tm ,
I log Tt = α + βγ t , γ > 0,
 
I Tt = α/ β + exp(−γt) .

I The parameters are then estimated by the least squares method,

i.e., by minimizing with respect to the bj ’s
X
2
Yt − β0 − β1 t − β2 t2 − · · · − βm tm
t
.
14 / 27
Method of Curve Fitting

I Here the trend is modelled using some mathematical function, e.g.,

I Tt = α + βt,
I Tt = α + βt + γt2 ,
I More generally, Tt = β0 + β1 t + β2 t2 + · · · + βm tm ,
I log Tt = α + βγ t , γ > 0,
 
I Tt = α/ β + exp(−γt) .

I The parameters are then estimated by the least squares method,

i.e., by minimizing with respect to the bj ’s
X
2
Yt − β0 − β1 t − β2 t2 − · · · − βm tm
t
.
14 / 27
Example

Consider
Tt = α + βt + γt2 ,

Equating the derivatives (w.r.t. α, β and γ) of

2 2 to 0 leads to
P

t Yt − α + βt + γt
P P P 2
t Yt = nα + β tt+γ tt
I
P P P 2 P 3
t tYt = α tt+β tt +γ tt
I
P 2 P 2 P 3 P 4
t t Yt = α tt +β tt +γ tt
I

Solving the three equations in the three unknowns gives the trend
as T̂t = α̂ + β̂t + γ̂t2 ,

15 / 27
Example

Consider
Tt = α + βt + γt2 ,

Equating the derivatives (w.r.t. α, β and γ) of

2 2 to 0 leads to
P

t Yt − α + βt + γt
P P P 2
t Yt = nα + β tt+γ tt
I
P P P 2 P 3
t tYt = α tt+β tt +γ tt
I
P 2 P 2 P 3 P 4
t t Yt = α tt +β tt +γ tt
I

Solving the three equations in the three unknowns gives the trend
as T̂t = α̂ + β̂t + γ̂t2 ,

15 / 27
Example

Consider
Tt = α + βt + γt2 ,

Equating the derivatives (w.r.t. α, β and γ) of

2 2 to 0 leads to
P

t Yt − α + βt + γt
P P P 2
t Yt = nα + β tt+γ tt
I
P P P 2 P 3
t tYt = α tt+β tt +γ tt
I
P 2 P 2 P 3 P 4
t t Yt = α tt +β tt +γ tt
I

Solving the three equations in the three unknowns gives the trend
as T̂t = α̂ + β̂t + γ̂t2 ,

15 / 27
Example

Consider
Tt = α + βt + γt2 ,

Equating the derivatives (w.r.t. α, β and γ) of

2 2 to 0 leads to
P

t Yt − α + βt + γt
P P P 2
t Yt = nα + β tt+γ tt
I
P P P 2 P 3
t tYt = α tt+β tt +γ tt
I
P 2 P 2 P 3 P 4
t t Yt = α tt +β tt +γ tt
I

Solving the three equations in the three unknowns gives the trend
as T̂t = α̂ + β̂t + γ̂t2 ,

15 / 27
Example

Consider
Tt = α + βt + γt2 ,

Equating the derivatives (w.r.t. α, β and γ) of

2 2 to 0 leads to
P

t Yt − α + βt + γt
P P P 2
t Yt = nα + β tt+γ tt
I
P P P 2 P 3
t tYt = α tt+β tt +γ tt
I
P 2 P 2 P 3 P 4
t t Yt = α tt +β tt +γ tt
I

Solving the three equations in the three unknowns gives the trend
as T̂t = α̂ + β̂t + γ̂t2 ,

15 / 27
Method of Moving Averages

Here a (weighted) mean of the observations around a time-point is

taken as the trend-value at that point. The averaging actually
eliminates the noise as positive and negative deviations cancel each
other out.
Linear filtering : Use a weighted local mean
Pg Pm
Tt = j=1 at−j Yt−j + at Yt + j=1 at+j Yt+j .

The method is termed as filtering since it is like passing the inputs

Yt into a (black) box and getting the resultant output Tt . Since
the transformation is linear in the Yt , it is called linear filtering.

16 / 27
Simple Moving Averages

Usually, g = m (symmetric filters).

1
Simple moving average: Take, at+j = 2m+1 , ∀ j. Then,
I

Yt−m + · · · + Yt−1 + Yt + Yt+1 + · · · + Yt+m

Tt = .
2m + 1

This is typically the method used to identify trend.

A problem is that the trend values for the m time-points at the
beginning and the end cannot be obtained.

17 / 27
Detrending

Detrending is eliminating the trend from the observed series.

In an additive model this can be done simply by subtracting the
estimated trend from the original series i.e.

DTt = Yt − Tt = St + Xt ,

gives a series without trend.

An alternative way of removing trend is by differencing
I first difference: ∆Yt = Yt − Yt−1
I second difference: ∆2 Yt = ∆Yt − ∆Yt−1 .

18 / 27
Detrending

Detrending is eliminating the trend from the observed series.

In an additive model this can be done simply by subtracting the
estimated trend from the original series i.e.

DTt = Yt − Tt = St + Xt ,

gives a series without trend.

An alternative way of removing trend is by differencing
I first difference: ∆Yt = Yt − Yt−1
I second difference: ∆2 Yt = ∆Yt − ∆Yt−1 .

18 / 27
Detrending

Detrending is eliminating the trend from the observed series.

In an additive model this can be done simply by subtracting the
estimated trend from the original series i.e.

DTt = Yt − Tt = St + Xt ,

gives a series without trend.

An alternative way of removing trend is by differencing
I first difference: ∆Yt = Yt − Yt−1
I second difference: ∆2 Yt = ∆Yt − ∆Yt−1 .

18 / 27
Seasonality

I A seasonal component of a time series is a cyclical, periodic

movement with period less than a year.
I Can be the variations over a day, a week, over months or over
quarters.
I Methods of estimating a seasonal component:
I differnce-to-trend method.
I difference-to-moving averages method.

19 / 27
Seasonality

I A seasonal component of a time series is a cyclical, periodic

movement with period less than a year.
I Can be the variations over a day, a week, over months or over
quarters.
I Methods of estimating a seasonal component:
I differnce-to-trend method.
I difference-to-moving averages method.

19 / 27
Seasonality

I A seasonal component of a time series is a cyclical, periodic

movement with period less than a year.
I Can be the variations over a day, a week, over months or over
quarters.
I Methods of estimating a seasonal component:
I differnce-to-trend method.
I difference-to-moving averages method.

19 / 27
Seasonality

I A seasonal component of a time series is a cyclical, periodic

movement with period less than a year.
I Can be the variations over a day, a week, over months or over
quarters.
I Methods of estimating a seasonal component:
I differnce-to-trend method.
I difference-to-moving averages method.

19 / 27
Seasonality

I A seasonal component of a time series is a cyclical, periodic

movement with period less than a year.
I Can be the variations over a day, a week, over months or over
quarters.
I Methods of estimating a seasonal component:
I differnce-to-trend method.
I difference-to-moving averages method.

19 / 27
A Seasonal Pattern

20 / 27
Estimating Seasonality

If the original series Xt has a trend, find it and use the detrended
series

DTt = Yt − Tt = St + Xt .

Note : the terms difference-to-trend and difference-to-moving

average comes from the method by which trend is estimated, by
mathematical curve fitting in case of the former and by moving
average in case of the latter.

21 / 27
Estimating Seasonality (contd.)

For each season, there is a single seasonal value, i.e., if there are h
seasons,

· · · = St−2h = St−h = St = St+h = St+2h = · · · .

Example : The seasonality for January is different from that of

February, but is the same for the different years.

22 / 27
Estimating Seasonality (contd.)

I Treat Xt as some random noise.

I For each season, average over the different years to eliminate the
noise Xt , i.e.
X∞
St = (St∓jh + Xt∓jh ).
j=0

I This gives the h indices St , St+1 , . . . , St+h−1 .

I Adjust these indices so that their sum is zero.
I These give the adjusted seasonal indices St .
I Note : The adjustment allows easier comparison between the
different seasons since the zero now acts as the reference point.

23 / 27
Estimating Seasonality (contd.)

I Treat Xt as some random noise.

I For each season, average over the different years to eliminate the
noise Xt , i.e.
X∞
St = (St∓jh + Xt∓jh ).
j=0

I This gives the h indices St , St+1 , . . . , St+h−1 .

I Adjust these indices so that their sum is zero.
I These give the adjusted seasonal indices St .
I Note : The adjustment allows easier comparison between the
different seasons since the zero now acts as the reference point.

23 / 27
Estimating Seasonality (contd.)

I Treat Xt as some random noise.

I For each season, average over the different years to eliminate the
noise Xt , i.e.
X∞
St = (St∓jh + Xt∓jh ).
j=0

I This gives the h indices St , St+1 , . . . , St+h−1 .

I Adjust these indices so that their sum is zero.
I These give the adjusted seasonal indices St .
I Note : The adjustment allows easier comparison between the
different seasons since the zero now acts as the reference point.

23 / 27
Estimating Seasonality (contd.)

I Treat Xt as some random noise.

I For each season, average over the different years to eliminate the
noise Xt , i.e.
X∞
St = (St∓jh + Xt∓jh ).
j=0

I This gives the h indices St , St+1 , . . . , St+h−1 .

I Adjust these indices so that their sum is zero.
I These give the adjusted seasonal indices St .
I Note : The adjustment allows easier comparison between the
different seasons since the zero now acts as the reference point.

23 / 27
Estimating Seasonality (contd.)

I Treat Xt as some random noise.

I For each season, average over the different years to eliminate the
noise Xt , i.e.
X∞
St = (St∓jh + Xt∓jh ).
j=0

I This gives the h indices St , St+1 , . . . , St+h−1 .

I Adjust these indices so that their sum is zero.
I These give the adjusted seasonal indices St .
I Note : The adjustment allows easier comparison between the
different seasons since the zero now acts as the reference point.

23 / 27
Estimating Seasonality (contd.)

I Treat Xt as some random noise.

I For each season, average over the different years to eliminate the
noise Xt , i.e.
X∞
St = (St∓jh + Xt∓jh ).
j=0

I This gives the h indices St , St+1 , . . . , St+h−1 .

I Adjust these indices so that their sum is zero.
I These give the adjusted seasonal indices St .
I Note : The adjustment allows easier comparison between the
different seasons since the zero now acts as the reference point.

23 / 27
Deseasonalized, Detrended Series

I Removing both the trend and the seasonal effect from the model.
I In an additive model, Dt = DTt − St = Yt − Tt − St = Xt .
I Xt or, eqivalently, Dt is a series without trend and seasonality.
I Can expect a detrended, deseasonalized series to be mean
stationary .

24 / 27
Deseasonalized, Detrended Series

I Removing both the trend and the seasonal effect from the model.
I In an additive model, Dt = DTt − St = Yt − Tt − St = Xt .
I Xt or, eqivalently, Dt is a series without trend and seasonality.
I Can expect a detrended, deseasonalized series to be mean
stationary .

24 / 27
Deseasonalized, Detrended Series

I Removing both the trend and the seasonal effect from the model.
I In an additive model, Dt = DTt − St = Yt − Tt − St = Xt .
I Xt or, eqivalently, Dt is a series without trend and seasonality.
I Can expect a detrended, deseasonalized series to be mean
stationary .

24 / 27
Deseasonalized, Detrended Series

I Removing both the trend and the seasonal effect from the model.
I In an additive model, Dt = DTt − St = Yt − Tt − St = Xt .
I Xt or, eqivalently, Dt is a series without trend and seasonality.
I Can expect a detrended, deseasonalized series to be mean
stationary .

24 / 27
Alternative Method of Deseasonalizing

I Differencing:
I first, difference to remove trend

∆Yt = Yt − Yt−1

I then, difference to remove seasonality

∆s ∆Yt = ∆Yt − ∆Yt−s .

I Both differencing can involve higher order differences.

25 / 27
Alternative Method of Deseasonalizing

I Differencing:
I first, difference to remove trend

∆Yt = Yt − Yt−1

I then, difference to remove seasonality

∆s ∆Yt = ∆Yt − ∆Yt−s .

I Both differencing can involve higher order differences.

25 / 27
Alternative Method of Deseasonalizing

I Differencing:
I first, difference to remove trend

∆Yt = Yt − Yt−1

I then, difference to remove seasonality

∆s ∆Yt = ∆Yt − ∆Yt−s .

I Both differencing can involve higher order differences.

25 / 27
Alternative Method of Deseasonalizing

I Differencing:
I first, difference to remove trend

∆Yt = Yt − Yt−1

I then, difference to remove seasonality

∆s ∆Yt = ∆Yt − ∆Yt−s .

I Both differencing can involve higher order differences.

25 / 27
Summary

I Different types of time series models have been introduced.

I Distinction between the different components of the classical
models has been made.
I Methods of estimating trend and seasonality have been discussed.

26 / 27
Summary

I Different types of time series models have been introduced.

I Distinction between the different components of the classical
models has been made.
I Methods of estimating trend and seasonality have been discussed.

26 / 27
Summary

I Different types of time series models have been introduced.

I Distinction between the different components of the classical
models has been made.
I Methods of estimating trend and seasonality have been discussed.

26 / 27
Thank You

27 / 27