11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

Open in app Sign up Sign in

Search

Guide to Time Series Analysis with Python

— 4: ARIMA and SARIMA
Buse Köseoğlu · Follow
9 min read · Jan 6, 2024

Listen Share

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 1/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

In previous articles, we examined analysis techniques, moving average

process and autoregressive process. In this article, we will examine ARIMA
(Auto Regressive Integrated Moving Average) and SARIMA (Seasonal ARIMA).

You can access previous articles here.

Guide to Time Series Analysis with Python — 1: Analysis Techniques and

Baseline Model

Guide to Time Series Analysis with Python — 2: Moving Average Process

Guide to Time Series Analysis with Python — 3: Autoregressive Process

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 2/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

You can find full code of this article on GitHub. If you are ready, let’s get
started.

What is ARIMA(p,d,q)?
As the name suggests, ARIMA is a combination of AR and MA models and
order of integration.

AR: autoregressive process says that the past values in the time series
affect the present.

MA: Moving average process indicates that the current value depends on
the current and past error rates

I: Differentiation is applied to obtain a stationary time series that does not

show trend or seasonality.

All of these components create the parameters to be used by ARIMA(p,d,q).

p: This p-value decides how far back we go. It is the lag order.

d: It is equal to the number of times a series is differenced until it becomes

stationary.

q: This parameter determines the number of historical error terms that

affect the current value

In fact, apart from the AR and MA models, the only new parameter here is the
d parameter. In the diagram below, you can see how the data should be
examined after receiving it.

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 3/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 4/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

First of all, we need to pay attention to whether the data is stationary or not.
After applying the transformations to make the data stationary (you can find
them in the first article), we need to determine the d parameter of ARIMA.
This parameter can also be determined by how many differencing we receive
when making the data stationary. Later, in order to find the optimum values of
the p and q parameters, we can create separate lists for both and try every
combination of the 3 parameters with iteration. The model with the smallest
AIC value can be selected as the best model. There is an important point here:
if we want to compare models according to the AIC metric, the d parameter
must be constant. The same d should be used in every model. After choosing
the model, we should not say that this model is the best and leave it at that.
Here we need to do a residual analysis for the model we chose. Residual
analysis helps assess whether the model adequately captures the underlying
patterns in the time series data.

What is SARIMA(p,d,q)(P,D,Q)m?
The SARIMA model includes P, D, Q, m parameters in addition to ARIMA.
These parameters help us capture seasonality.

P: order of seasonal AR(P)

D: seasonal order of integration

Q: order of seasonal MA(Q)

m: Number of observations per cycle (frequency)

Note that a SARIMA(p,d,q)(0,0,0)m model is equivalent to an ARIMA(p,d,q)

model.

CODE PRACTICE

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 5/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

Now let’s do all this in practice. We will use auto_arima from the pmdarima
library to determine the optimal model.

For examples we will use the air passengers dataset available on Kaggle.

This data set contains the number of passengers on a monthly basis. The data
consists of 144 rows and 2 columns. There are monthly time periods between
1949–1960.

When we examine the graph of the number of passengers over time, we can
clearly see that there is seasonality even here.

df = pd.read_csv("AirPassengers.csv")
df.rename(columns={"Month":"month","#Passengers":"passengers"}, inplace=T
df.head()

plt.figure(figsize=(15,4))
plt.plot(df["month"],df["passengers"]);
plt.xlabel('Timesteps');
plt.ylabel('Value');
plt.xticks(df['month'][::10]);

To see this more closely, we need to decompose the data.

When examining the graphs, an increasing trend can be observed.
Additionally, there is seasonality in the data. If there were no seasonality in
the data, the “Seasonal” graph would continue as a straight line starting from
0.
https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 6/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

# Decomposition

advanced_decomposition = STL(df.passengers, period=4).fit()

fig, (ax1, ax2, ax3, ax4) = plt.subplots(nrows=4, ncols=1, sharex=True)

ax1.plot(advanced_decomposition.observed)
ax1.set_ylabel('Observed')

ax2.plot(advanced_decomposition.trend)
ax2.set_ylabel('Trend')

ax3.plot(advanced_decomposition.seasonal)
ax3.set_ylabel('Seasonal')

ax4.plot(advanced_decomposition.resid)
ax4.set_ylabel('Residuals')

fig.autofmt_xdate()
plt.tight_layout()

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 7/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

We must apply the Dickey-Fuller test to decide whether the data is stationary
or not.

Null Hypothesis: Data is non-stationary

Alternative Hypothesis: Data is stationary

def adfuller_test(y):
adf_result = adfuller(y)

print("ADF Statistic:", adf_result[0])

print("P-Value:", adf_result[1])

adfuller_test(df.passengers)

When we apply the Dickey-Fuller test to the “Passengers” variable, we can see
that the p-value is greater than 0.05, meaning the data is not stationary. We
can perform differencing to make the data stationary.

df_diff = np.diff(df['passengers'], n=1)

adfuller_test(df_diff)

print("*"*50)

df_diff2 = np.diff(df_diff, n=1)

adfuller_test(df_diff2)

# d = 2

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 8/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

First order differencing did not make the data stationary. P-value is still
greater than 0.05 so we perform differencing again. The second time the data
became stationary. Thus, we determined that our d parameter should be 2.

train = df[:-12]
test = df[-12:]

We divide the data into two: train and test. We put 12 months of data to the
test.

Now we will create an ARIMA model using auto_arima. auto_arima makes it

easy for us to find the optimum parameters. It reduces the workload because
it performs the iterations itself. If you wish, you can create your own function
with for lobs without using this library.

ARIMA_model = auto_arima(train['passengers'],
start_p=1,
start_q=1,
test='adf', # use adftest to find optimal 'd'
tr=13, max_q=13, # maximum p and q
m=1, # frequency of series (if m==1, seasonal is se
d=2,
seasonal=False, # No Seasonality for standard ARIMA
trace=True, #logs
error_action='warn', #shows errors ('ignore' silenc
suppress_warnings=True,

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 9/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

stepwise=True)
ARIMA_model.summary()

Model parameters:

Training data (train[“passengers”]: First, we give the data we want to train.

Since the data we will be predicting is passengers, we give the passengers
variable in the train data to the model.

start_p: At what number should we start searching for the p value for the
AR(p) part?

start_q: At what number should we start searching for the q value for the
MA(q) part?

test=’adf’: use adftest to find optimal ‘d’

max_p: The maximum value that the p parameter can take.

max_q: The maximum value that the q parameter can take.

m:frequency. If it is 1, we say there is no seasonality.

d: order of integration

seasonal: is there seasonality or not?

When we run the model, we can see that the best parameters it finds are
SARIMAX(4, 2, 0). The reason why it is shown as SARIMAX is because
auto_arima can also run the SARIMAX model. Since these models are already
separated by parameters, the SARIMAX(4, 2, 0) representation is equal to the
ARIMA(4, 2, 0) representation.

We can visualize the residual analysis of the resulting model as follows.

Standardized Residual: There is no obvious pattern in the residuals, with

values having zero mean and uniform variance.

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 10/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

Histogram plus: The KDE curve should be very similar to the normal
distribution (labeled as N(0,1) in the plot)

Q-Q Plot: Most of the data points should lie on the straight line

Correlogram (ACF plot): There should be no significant autocorrelation

coefficients after lag 0, but we have.

ARIMA_model.plot_diagnostics(figsize=(10,7))
plt.show()

The next step is to run the Ljung-Box test on the residuals to make sure that
they are independent and uncorrelated.

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 11/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

ARIMA_model = SARIMAX(train["passengers"], order=(4,2,0), simple_differen

ARIMA_model_fit = ARIMA_model.fit(disp=False)

residuals = ARIMA_model_fit.resid
acorr_ljungbox(residuals, np.arange(1, 11, 1))

Here we expect all p-value values to be greater than 0.05. But the values we
obtained are not like this.

To compare the ARIMA and SARIMA models, we will add the predictions of
both to the test data.

test['naive_seasonal'] = df['passengers'].iloc[120:132].values
ARIMA_pred = ARIMA_model_fit.get_prediction(132, 143).predicted_mean
https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 12/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

test['ARIMA_pred'] = ARIMA_pred

“naive_seasonal” treats the values of the last 12 months as the same as the 12
months in the test data we want to predict. So we assume the last 2 years are
the same. ARIMA_pred contains the values of the last 12 months predicted by
the model.

In the SARIMA model, we first need to look at differencing. As we know from

ARIMA, first order differencing was not enough to make the data stationary.

df_diff = np.diff(df['passengers'], n=1)

adfuller_test(df_diff)

print("*"*50)

df_diff2 = np.diff(df_diff, n=12)

adfuller_test(df_diff2)

Here we first take first order differencing. Since the data is not stationary, we
do this process again, but we write 12 (frequency of the data — m parameter)
instead of 1 in the n parameter. As a result, we can see that the data is
stationary and the parameters are:

d=1

D=1

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 13/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

# Seasonal - fit stepwise auto-ARIMA

SARIMA_model = auto_arima(train["passengers"], start_p=1, start_q=1,
test='adf',
max_p=5, max_q=5,
m=12, #12 is the frequncy of the cycle
start_P=1, start_Q=1,
max_P = 5, max_Q=5,
seasonal=True, #set to seasonal
d=1,
D=1,
trace=False,
error_action='ignore',
suppress_warnings=True,
stepwise=True)
SARIMA_model.summary()

We use some extra parameters when using SARIMA.

start_P, start_Q: Initial value of P and Q parameters to be used in SARIMA

max_P, max_Q: Maximum value of P and Q parameters to be used in

SARIMA

D: differencing order for SARIMA

m: Since we have seasonality, we must replace this parameter with the

frequency of the data.

Seasonal: must be True since there is seasonality

As a result, the best model was found to be SARIMAX(1, 1, 0)x(0, 1, 0, 12).

When we examine the residual analysis of this;

SARIMA_model = SARIMAX(train["passengers"], order=(1,1,0), seasonal_order

SARIMA_model_fit = SARIMA_model.fit(disp=False)
SARIMA_model_fit.plot_diagnostics(figsize=(10,8));

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 14/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

Standardized Residual: shows that the residuals do not exhibit a trend or a

change in variance.

Histogram plus: residuals’ distribution is very close to a normal

distribution.

Q-Q Plot: Most of the data points should lie on the straight line, which
displays a fairly straight line

Correlogram (ACF plot): hows no significant coefficients after lag 0.

Therefore, everything leads to the conclusion that the residuals resemble
white noise.

To verify this, we will also perform the Ljung-Box test.

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 15/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

residuals = SARIMA_model_fit.resid
acorr_ljungbox(residuals, np.arange(1, 11, 1))

The returned p-values are all greater than 0.05. Therefore, we do not reject the
null hypothesis, and we conclude that the residuals are independent and
uncorrelated, just like white noise.
Our model has passed all the tests from the residuals analysis, and we are
ready to use it for forecasting.

SARIMA_pred = SARIMA_model_fit.get_prediction(132, 143).predicted_mean

test['SARIMA_pred'] = SARIMA_pred

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 16/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

We can visualize the results of both models and observe which one gives
results closer to reality.

fig, ax = plt.subplots()

ax.plot(df['month'], df['passengers'])
ax.plot(test['passengers'], 'b-', label='actual')
ax.plot(test['naive_seasonal'], 'r:', label='naive seasonal')
ax.plot(test['ARIMA_pred'], 'k--', label='ARIMA')
ax.plot(test['SARIMA_pred'], 'g-.', label='SARIMA')

ax.set_xlabel('Date')
ax.set_ylabel('Number of air passengers')
ax.axvspan(132, 143, color='#808080', alpha=0.2)
ax.legend(loc=2)
plt.xticks(np.arange(0, 145, 12), np.arange(1949, 1962, 1))
ax.set_xlim(110, 143)
fig.autofmt_xdate()
plt.tight_layout()

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 17/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

The gray area in the graph above is our test data. The blue line shows the
actual values. When we examine the results, we can say that ARIMA’s success
is very low, but SARIMA is successful in its predictions. The addition of
seasonality has been successful for the model.

We can also evaluate the success of the model with MAPE (Mean Absolute
Percentage Error). Here we expect MAPE to be small.

def mape(y_true, y_pred):

return np.mean(np.abs((y_true - y_pred) / y_true)) * 100

mape_naive_seasonal = mape(test['passengers'], test['naive_seasonal'])

mape_ARIMA = mape(test['passengers'], test['ARIMA_pred'])
mape_SARIMA = mape(test['passengers'], test['SARIMA_pred'])
fig, ax = plt.subplots()

x = ['naive seasonal', 'ARIMA', 'SARIMA']

y = [mape_naive_seasonal, mape_ARIMA, mape_SARIMA]
ax.bar(x, y, width=0.4)
ax.set_xlabel('Models')
ax.set_ylabel('MAPE (%)')

for index, value in enumerate(y):

plt.text(x=index, y=value + 1, s=str(round(value,2)), ha='center')
plt.tight_layout()

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 18/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

When we examine the metric, we can see that the best model is the SARIMA
model.

Thank you for reading.

Time Series Analysis Time Series Forecasting Arima Sarima

Time Series Model

Follow

Written by Buse Köseoğlu

485 Followers · 58 Following

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 19/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

BAU Software Engineering | Data Science | https://www.linkedin.com/in/busekoseoglu/

No responses yet

What are your thoughts?

Respond

More from Buse Köseoğlu

Buse Köseoğlu

One Layer Neural Network From Scratch — Classification

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 20/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

Classification is a supervised learning method. In supervised learning, we have labels for

our data. Algorithms learn from these labels…

Jul 29, 2023 225 2

Buse Köseoğlu

LLMs In Context Learning (ICL)— Hands On Example

ICL is formally introduced by GPT-3. It ensures that the desired output is obtained by
giving examples of language models for one or more…

Apr 17 4 1

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 21/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

In BAU Yazılım ve Bilişim Kulübü by Buse Köseoğlu

Model Performansını Değerlendirmek: Regresyon

Regresyonda bağımlı X değişkenlerini alarak bağımsız Y değişkenini tahmin etmeye
çalışırız. Tahmin sonucu elde ettiğimiz sonuçlar genelde…

Feb 11, 2021 136 2

Buse Köseoğlu

Guide to Time Series Analysis with Python — 1: Analysis Techniques

and Baseline Model
Time series analysis is a method used in various industries, such as stock market
prediction, product price forecasting, weather…

Jul 1, 2023 323

See all from Buse Köseoğlu

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 22/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

Recommended from Medium

Irina (Xinli) Yu, Ph.D.

Mastering Time Series Forecasting with ARIMA Models

Time series forecasting is a critical component in many domains, including finance,
economics, environmental science, and operations…

Jun 7 2

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 23/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

In Stackademic by Ganesh Bajaj

Time Series Analysis: Interpretation of ACF and PACF Plots

Autocorrelation (ACF) and Partial Autocorrelation (PACF) plots are powerful tools for
uncovering hidden patterns in time series data…

Sep 24 50

Lists

Practical Guides to Machine Learning

10 stories · 2058 saves

Predictive Modeling w/ Python

20 stories · 1691 saves

ChatGPT prompts
50 stories · 2298 saves

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 24/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

Pape

Time Series Forecasting with GANs: A Comprehensive Guide

Time series forecasting is essential in various fields such as finance, weather prediction,
and demand forecasting. Traditional methods…

Jun 8 64

In Data Analytics Mastery by Nilimesh Halder, PhD

A Simple Guide on Time Series Analysis for Economic and Business

Forecasting

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 25/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

Time series analysis is essential for predicting future trends in sequential data, such as
sales figures, stock prices, or economic…

Oct 30

Palash Mishra

Predicting Time Series Data with Machine Learning, Generative AI,

and Deep Learning
Time series data prediction is a critical aspect of various industries, ranging from
finance and healthcare to marketing and logistics. The…

Jun 19 741 10

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 26/27
11/30/24, 8:15 PM Guide to Time Series Analysis with Python — 4: ARIMA and SARIMA | by Buse Köseoğlu | Medium

In Towards Data Science by Jonas Dieckmann

Getting Started Predicting Time Series Data with Facebook Prophet

This article aims to take away the entry barriers to get started with time series analysis
in a hands-on tutorial using Prophet

Jan 31 348 3

See more recommendations

https://buse-koseoglu13.medium.com/guide-to-time-series-analysis-with-python-4-arima-and-sarima-d62bcdcfb64a 27/27