Introduction to time

series and
stationarity
F ORECA S TIN G US IN G A RIMA MODELS IN P YTH ON

James Fulton
Climate informatics researcher
Motivation
Time series are everywhere

Science

Technology

Business

Finance

Policy

FORECASTING USING ARIMA MODELS IN PYTHON

Course content
You will learn

Structure of ARIMA models

How to t ARIMA model

How to optimize the model

How to make forecasts

How to calculate uncertainty in predictions

FORECASTING USING ARIMA MODELS IN PYTHON

Loading and plotting
import pandas as pd
import matplotlib as plt

df = pd.read_csv('time_series.csv', index_col='date', parse_dates=True)

date values
2019-03-11 5.734193
2019-03-12 6.288708
2019-03-13 5.205788
2019-03-14 3.176578

FORECASTING USING ARIMA MODELS IN PYTHON

Trend
fig, ax = plt.subplots()
df.plot(ax=ax)
plt.show()

FORECASTING USING ARIMA MODELS IN PYTHON

Seasonality

FORECASTING USING ARIMA MODELS IN PYTHON

Cyclicality

FORECASTING USING ARIMA MODELS IN PYTHON

White noise
White noise series has uncorrelated values

Heads, heads, heads, tails, heads, tails, ...

0.1, -0.3, 0.8, 0.4, -0.5, 0.9, ...

FORECASTING USING ARIMA MODELS IN PYTHON

Stationarity
Stationary Not stationary

Trend stationary: Trend is zero

FORECASTING USING ARIMA MODELS IN PYTHON

Stationarity
Stationary Not stationary

Trend stationary: Trend is zero

Variance is constant

FORECASTING USING ARIMA MODELS IN PYTHON

Stationarity
Stationary Not stationary

Trend stationary: Trend is zero

Variance is constant

Autocorrelation is constant

FORECASTING USING ARIMA MODELS IN PYTHON

Train-test split
# Train data - all data up to the end of 2018
df_train = df.loc[:'2018']

# Test data - all data from 2019 onwards

df_test = df.loc['2019':]

FORECASTING USING ARIMA MODELS IN PYTHON

 Let's Practice!
F ORECA S TIN G US IN G A RIMA MODELS IN P YTH ON
 Making time series
stationary
F ORECA S TIN G US IN G A RIMA MODELS IN P YTH ON

James Fulton
Climate informatics researcher
Overview
Statistical tests for stationarity

Making a dataset stationary

FORECASTING USING ARIMA MODELS IN PYTHON

The augmented Dicky-Fuller test
Tests for trend non-stationarity

Null hypothesis is time series is non-stationary

FORECASTING USING ARIMA MODELS IN PYTHON

Applying the adfuller test

from statsmodels.tsa.stattools import adfuller

results = adfuller(df['close'])

FORECASTING USING ARIMA MODELS IN PYTHON

Interpreting the test result
print(results)

(-1.34, 0.60, 23, 1235, {'1%': -3.435, '5%': -2.913, '10%': -2.568}, 10782.87)

0th element is test statistic (-1.34)

More negative means more likely to be stationary

1st element is p-value: (0.60)

If p-value is small → reject null hypothesis. Reject non-stationary.

4th element is the critical test statistics

FORECASTING USING ARIMA MODELS IN PYTHON

 Interpreting the test result
print(results)

(-1.34, 0.60, 23, 1235, {'1%': -3.435, '5%': -2.863, '10%': -2.568}, 10782.87)

0th element is test statistic (-1.34)

More negative means more likely to be stationary

1st element is p-value: (0.60)

If p-value is small → reject null hypothesis. Reject non-stationary.

4th element is the critical test statistics

1 https://www.statsmodels.org/dev/generated/statsmodels.tsa.stattools.adfuller.html

FORECASTING USING ARIMA MODELS IN PYTHON

The value of plotting
Plotting time series can stop you making wrong assumptions

FORECASTING USING ARIMA MODELS IN PYTHON

The value of plotting

FORECASTING USING ARIMA MODELS IN PYTHON

Making a time series stationary

FORECASTING USING ARIMA MODELS IN PYTHON

Taking the difference

Difference: Δyt = yt − yt−1

FORECASTING USING ARIMA MODELS IN PYTHON

Taking the difference
df_stationary = df.diff()

city_population
date
1969-09-30 NaN
1970-03-31 -0.116156
1970-09-30 0.050850
1971-03-31 -0.153261
1971-09-30 0.108389

FORECASTING USING ARIMA MODELS IN PYTHON

Taking the difference
df_stationary = df.diff().dropna()

city_population
date
1970-03-31 -0.116156
1970-09-30 0.050850
1971-03-31 -0.153261
1971-09-30 0.108389
1972-03-31 -0.029569

FORECASTING USING ARIMA MODELS IN PYTHON

Taking the difference

FORECASTING USING ARIMA MODELS IN PYTHON

Other transforms
Examples of other transforms

Take the log

np.log(df)

Take the square root

np.sqrt(df)

Take the proportional change

df.shift(1)/df

FORECASTING USING ARIMA MODELS IN PYTHON

 Let's practice!
F ORECA S TIN G US IN G A RIMA MODELS IN P YTH ON
 Intro to AR, MA and
ARMA models
F ORECA S TIN G US IN G A RIMA MODELS IN P YTH ON

James Fulton
Climate informatics researcher
AR models
Autoregressive (AR) model

AR(1) model :
yt = a1 yt−1 + ϵt

FORECASTING USING ARIMA MODELS IN PYTHON

AR models
Autoregressive (AR) model

AR(1) model :
yt = a1 yt−1 + ϵt

AR(2) model :
yt = a1 yt−1 + a2 yt−2 + ϵt

AR(p) model :
yt = a1 yt−1 + a2 yt−2 + ... + ap yt−p + ϵt

FORECASTING USING ARIMA MODELS IN PYTHON

MA models
Moving average (MA) model

MA(1) model :
yt = m1 ϵt−1 + ϵt

MA(2) model :
yt = m1 ϵt−1 + m2 ϵt−2 + ϵt

MA(q) model :
yt = m1 ϵt−1 + m2 ϵt−2 + ... + mq ϵt−q + ϵt

FORECASTING USING ARIMA MODELS IN PYTHON

ARMA models
Autoregressive moving-average (ARMA) model

ARMA = AR + MA

ARMA(1,1) model :
yt = a1 yt−1 + m1 ϵt−1 + ϵt

ARMA(p, q)

p is order of AR part

q is order of MA part

FORECASTING USING ARIMA MODELS IN PYTHON

Creating ARMA data
yt = a1 yt−1 + m1 ϵt−1 + ϵt

FORECASTING USING ARIMA MODELS IN PYTHON

Creating ARMA data
yt = 0.5yt−1 + 0.2ϵt−1 + ϵt

from statsmodels.tsa.arima_process import arma_generate_sample

ar_coefs = [1, -0.5]

ma_coefs = [1, 0.2]

y = arma_generate_sample(ar_coefs, ma_coefs, nsample=100, sigma=0.5)

FORECASTING USING ARIMA MODELS IN PYTHON

Creating ARMA data
yt = 0.5yt−1 + 0.2ϵt−1 + ϵt

FORECASTING USING ARIMA MODELS IN PYTHON

Fitting and ARMA model
from statsmodels.tsa.arima_model import ARMA

# Instantiate model object

model = ARMA(y, order=(1,1))

# Fit model
results = model.fit()

FORECASTING USING ARIMA MODELS IN PYTHON

 Let's practice!
F ORECA S TIN G US IN G A RIMA MODELS IN P YTH ON