Stochiastic TIME SERIES

ANALYSIS
INTRODUCTION

1
 TIME SERIES
• Whether we wish to predict the trend in
financial markets or electricity consumption,
time is an important factor that must be
considered in our models. 
• A time series is simply a series of data points
ordered in time. In a time series, time is often
the independent variable and the goal is
usually to make a forecast for the future.

2
 DEFINITION

y 
• A stochastic process t t   is a collection of
random variables or a process that develops in
time according to probabilistic laws.

• The theory of stochastic processes gives us a

formal way to look at time series variables.

3
 DEFINITION
Y w, t  : Stochastic Process

belongs to sample space Indexed set

• For a fixed t, Y(w, t) is a random variable

• For a given w, Y(w, t) is called a sample
function or a realization as a function of t.

4
 DEFINITION
• Time series is a realization or sample function
from a certain stochastic process.
• A time series is a set of observations generated
sequentially in time. Therefore, they are
dependent to each other. This means that we do
NOT have a random sample.
• We assume that observations are equally spaced
in time.
• We also assume that closer observations might
have stronger dependency.
5
• Let us understand this with an example. Let’s say India expanded
its military strength of 1.5 active military personals in 2018 to 2
million in 2019.  That is, of course, an increase in power, sure
increase in military strength is not all to determine power but
let’s suppose it is. We know that in just one-year India
strengthened its military with half a million soldiers.
• A closer look shows how much Indian military has strengthened
in a year. Clearly, we can see something is dependent on time.
And say we have reasonable records proving the change in the
number over different points of time and then we call it dataset
and we can use it to predict the number for future. Now, this is
Time Series Analysis.
• We cannot predict the result of a war with the number of
soldiers but we sure can predict the number of soldiers with the
data. And that is the power of data.

6
 DEFINITION
• Discrete time series is one in which the set T0
at which observations are made in a discrete
set. Continuous time series are obtained when
observations are recorded continuously over
some time interval.

7
 EXAMPLES
• Data in business, economics, engineering,
environment, medicine, earth sciences, and other
areas of scientific investigations are often
collected in the form of time series.
• Hourly temperature readings
• Daily stock prices
• Weekly traffic volume
• Annual growth rate
• Seasonal ice cream consumption
• Electrical signals

8
• There is a company X which has been keeping a
record of monthly sales of shampoo for the
past 3 years. Company X wants to forecast the
sale of the shampoo for the next 4 months so
that the demand and supply gap can be
managed by the organization. Our main job
here is to simply predict the sales of the
shampoo for the next 4 months.
• Dataset comprises of only two columns. One is
the Date of the month and other is the sale of
the shampoo in that month.

9
EXAMPLES

10
EXAMPLES

11
EXAMPLES

12
EXAMPLES

13
EXAMPLES

14
EXAMPLES

15
EXAMPLES

16
EXAMPLES

17
EXAMPLES

18
EXAMPLES

19
EXAMPLES (SUNSPOT NUMBERS)

The Wolf number (also known

as the International sunspot
number, relative sunspot
number, or Zürich number) is a
quantity that measures the
number of sunspots and
groups of sunspots present on
the surface of the sun. The
Sunspot Number is a crucial
tool used to study the solar
dynamo, space weather and
climate change. 
20
OBJECTIVES OF TIME SERIES ANALYSIS
• Understanding the dynamic or time-
dependent structure of the observations of a
single series (univariate analysis)
• Forecasting of future observations
• Ascertaining the leading, lagging and feedback
relationships among several series
(multivariate analysis)

21
 STEPS IN TIME SERIES ANALYSIS
• Model Identification
– Time Series plot of the series
– Check for the existence of a trend or seasonality
– Check for the sharp changes in behavior
– Check for possible outliers
• Remove the trend and the seasonal component to get stationary
residuals.
• Estimation
– MME
– MLE
• Diagnostic Checking
– Normality of error terms
– Independency of error terms
– Constant error variance (Homoscedasticity)
• Forecasting
– Exponential smoothing methods
– Minimum MSE forecasting
22
 CHARACTERISTICS OF A SERIES
• For a time series Yt , t  0,1,2,
THE MEAN FUNCTION:
t  EYt  Exists iff E Yt  .
The expected value of the process at time t.

THE VARIANCE FUNCTION:

   0  Var Yt   E Yt  t   E Yt 2  t2  
t
2 2

0  0
23
 CHARACTERISTICS OF A SERIES
• THE AUTOCOVARIANCE FUNCTION:
 t , s  Cov Yt , Ys   EYt  t Ys   s 
 E YtYs   t  s ; t , s  0,  1,  2,
Covariance between the value at time t and the value at time s of a stochastic process Yt.

• THE AUTOCORRELATION FUNCTION:

 t ,s
t ,s  Corr Yt , Ys   ,1  t , s  1
 t s
The correlation of the series with itself

24
 EXAMPLE
• Moving average process: Let ti.i.d.(0, 1),
and
Xt = t + 0.5 t−1

25
 EXAMPLE
• RANDOM WALK: Let e1,e2,… be a sequence of
2
i.i.d. rvs with 0 mean and variance  e . The
observed time series
Yt , t  1,2,, n
is obtained as
Y1  e1
Y2  e1  e2  Y2  Y1  e2
Y3  e1  e2  e3  Y3  Y2  e3

Yt  e1    et  Yt  Yt 1  et 26
 A RULE ON THE COVARIANCE
• If c1, c2,…, cm and d1, d2,…, dn are constants
and t1, t2,…, tm and s1, s2,…, sn are time points,
then
m
 i 1
n  m n

Cov  ciYti ,  d jYs j     ci d j Cov Yti , Ys j
j 1  i 1 j 1

 m

 i 1
 m

 i 1
2
n i 1

i  2 j 1

Var   ciYti    ci Var Yti   2   ci c j Cov Yti , Yt j 

27
 JOINT PDF OF A TIME SERIES
• Remember that
FX1 x1  : the marginal cdf
f X 1 x1  : the marginal pdf
FX 1 , X 2 ,, X n x1 , x2 ,, xn  : the joint cdf
f X 1 , X 2 ,, X n x1 , x2 ,, xn  : the joint pdf

28
 JOINT PDF OF A TIME SERIES
• For the observed time series, say we have
two points, t and s.
• The marginal pdfs: fYt  yt  and fYs  ys 

• The joint pdf: fYt ,Ys  yt , ys   fYt  yt . fYs  ys 

29
 JOINT PDF OF A TIME SERIES
• Since we have only one observation for each
r.v. Yt, inference is too complicated if
distributions (or moments) change for all t (i.e.
change over time). So, we need a simplification.
r.v.

30
 JOINT PDF OF A TIME SERIES
• To be able to identify the structure of the
series, we need the joint pdf of Y1, Y2,…, Yn.
However, we have only one sample. That is,
one observation from each random variable.
Therefore, it is very difficult to identify the joint
distribution. Hence, we need an assumption to
simplify our problem. This simplifying
assumption is known as STATIONARITY.

31
 STATIONARITY
• The most vital and common assumption in
time series analysis.
• The basic idea of stationarity is that the
probability laws governing the process do not
change with time.
• The process is in statistical equilibrium.

32
 TYPES OF STATIONARITY
• STRICT (STRONG OR COMPLETE) STATIONARY
PROCESS: Consider a finite set of r.v.s.
t 1 t2 tn 
Y , Y ,, Y from a stochastic process

•
Y w, t ; t  0,1,2,.
The n-dimensional distribution function is defined
by

FYt ,Yt
1 2
 yt , yt ,, yt   P w : Yt  y1 ,, Yt
,,Ytn
where y , i=1, 2,…, n are any real numbers.
1 2 n 1 n
 yn 
i

33
 STRONG STATIONARITY
• A process is said to be first order stationary in
distribution, if its one dimensional distribution
function is time-invariant, i.e.,
FYt  y1   FYt  k  y1  for any t1 and k.
1 1

• Second order stationary in distribution if

FYt ,Y
1 t2
 y1, y2   FYt1 k ,Yt2 k  y1, y2  for any t1 ,t2 and k.
• n-th order stationary in distribution if
FYt ,,Yt  y1,, yn   FYt  k ,,Yt  k  y1,, yn  for any t1 , ,tn and k.
1 n 1 n

34
 STRONG STATIONARITY
n-th order stationarity in distribution = strong stationarity

 Shifting the time origin by an amount “k” has

no effect on the joint distribution, which must
therefore depend only on time intervals
between t1, t2,…, tn, not on absolute time, t.

35
 STRONG STATIONARITY
• So, for a strong stationary process
fYt ,,Yt  y1,, yn   fYt  k ,,Yt  k  y1,, yn 
i) 1 n 1 n

ii) EYt   EYt  k   t  t  k   ,t , k

Expected value of a series is constant over time, not a function of time

iii)Var Yt   Var Yt k    2

t   2
t k   2
, t , k
The variance of a series is constant over time, homoscedastic.

iv) CovYt , Ys   CovYt  k , Ys  k    t , s   t  k , s  k , t , k

  t  s   t  k  s k   h
Not constant, not depend on time, depends on time interval, which we call “lag”, k
36
 STRONG STATIONARITY
Yt

Y1 Y2 Y3 ……………………………………….... Yn

   
2 2 2 2
t

CovY2 , Y1    21   1
CovY3 , Y2    3 2   1
CovYn , Yn 1    n ( n 1)   1 Affected from time lag, k

CovY3 , Y1    31   2
CovY1, Y3    13    2
37
 STRONG STATIONARITY
v) CorrYt , Ys   CorrYt  k , Ys  k   t , s  t  k , s  k , t , k
  t  s   t  k  sk  h
Let t=t-k and s=t,
t ,t  k  t  k ,t   k , t , k
• It is usually impossible to verify a distribution
particularly a joint distribution function from
an observed time series. So, we use weaker
sense of stationarity.
38
 WEAK STATIONARITY
• WEAK (COVARIANCE) STATIONARITY OR
STATIONARITY IN WIDE SENSE: A time series is
said to be covariance stationary if its first and
second order moments are unaffected by a
change of time origin.
• That is, we have constant mean and variance
with covariance and correlation beings
functions of the time difference only.

39
 WEAK STATIONARITY
E Yt    , t
Var Yt    2  , t
CovYt , Yt  k    k , t
Corr Yt , Yt  k    k , t

From, now on, when we say “stationary”, we

imply weak stationarity.

40
 EXAMPLE
• Consider a time series {Yt} where
Yt=et
and eti.i.d.(0,2). Is the process stationary?

41
 EXAMPLE
• MOVING AVERAGE: Suppose that {Yt} is
constructed as
et  et 1
Yt 
2
and eti.i.d.(0,2). Is the process {Yt} stationary?

42
 EXAMPLE
• RANDOM WALK
Yt  e1  e2    et
where eti.i.d.(0,2). Is the process {Yt}
stationary?

43
 EXAMPLE
• Suppose that time series has the form
Yt  a  bt  et
where a and b are constants and {et} is a
weakly stationary process with mean 0 and
autocovariance function k. Is {Yt} stationary?

44
 EXAMPLE
t
Yt   1 et
where eti.i.d.(0,2). Is the process {Yt}
stationary?

45
STRONG VERSUS WEAK STATIONARITY
• Strict stationarity means that the joint distribution only
depends on the ‘difference’ h, not the time (t1, . . . , tk).
• Finite variance is not assumed in the definition of strong
stationarity, therefore, strict stationarity does not
necessarily imply weak stationarity. For example, processes
like i.i.d. Cauchy is strictly stationary but not weak
stationary.
• A nonlinear function of a strict stationary variable is still
strictly stationary, but this is not true for weak stationary.
For example, the square of a covariance stationary process
may not have finite variance.
• Weak stationarity usually does not imply strict stationarity
as higher moments of the process may depend on time t.

46
STRONG VERSUS WEAK STATIONARITY
• If process {Xt} is a Gaussian time series, which
means that the distribution functions of {Xt}
are all multivariate Normal, weak stationary
also implies strict stationary. This is because a
multivariate Normal distribution is fully
characterized by its first two moments.

47
 STRONG VERSUS WEAK STATIONARITY
• For example, a white noise is stationary but may
not be strict stationary, but a Gaussian white
noise is strict stationary. Also, general white
noise only implies uncorrelation while Gaussian
white noise also implies independence. Because
if a process is Gaussian, uncorrelation implies
independence. Therefore, a Gaussian white
noise is just i.i.d. N(0, 2).

48
 STATIONARITY AND
NONSTATIONARITY
• Stationary and nonstationary processes are
very different in their properties, and they
require different inference procedures. We
will discuss this in detail through this course.

49