Time Series Forecasting

Content
● Taxonomy of Time Series Forecasting Methods
● Time Series Decomposition
● Smoothening Methods
● Average Method, Moving Average smoothing, Time series analysis using
linear regression
● ARIMA Model
● Performance Evaluation - mean absolute error,root mean square error,
mean absolute percentage error,mean absolute scaled error
Time Series Analysis
● A time series is a series of observations recorded in the order of time.
● Time series analysis is the process of extracting meaningful patterns and
information from time series data.
● Time series forecasting is the process of predicting future values of a time
series based on past observations and other inputs.
● Time series forecasting is one of the oldest known predictive analytics
techniques and is widely used in every organizational setting.
Time Series Analysis
● Time series data often exhibit patterns and trends that are not present in
other types of data, such as seasonality, trends, and cyclical patterns.

● When building a time series forecasting model, the goal is to use historical
information to make predictions about future values of the same quantity.
Time Series Analysis
● There are a variety of modeling techniques that can be used for time series
forecasting, including ARIMA models, exponential smoothing, and machine
learning algorithms.

● Time series forecasting is a specialized area of predictive modeling that

requires a deep understanding of time series analysis techniques and
statistical modeling methods.
Time Series Data Analysis
Time Series Data Analysis
● Features of the Plot:
● There is no consistent trend (upward
or downward) over the entire time
span.
● The series appears to slowly wander
up and down. The horizontal line
drawn at quakes = 20.2 indicates the
mean of the series.
● Notice that the series tends to stay
on the same side of the mean
(above or below) for a while and then
wanders to the other side.

● There are no obvious outliers.

● It’s difficult to judge whether the
variance is constant or not.
Time Series Data Analysis

● In time series analysis, the focus is on forecasting a specific variable over time, based on
its past behavior.

● For example, in predicting house prices, variables such as location, square footage, number
of rooms, etc. are observed at a point in time, and the price is predicted at the same point in
time.
● it's important to take the time variable into account, as house prices can fluctuate over time
due to economic conditions, supply and demand, etc.
● Time series analysis is concerned with understanding and forecasting these fluctuations
over time.
 -Level (Avg)
- Trend ( up, down)
Time Series Decomposition - Seasonality
-Cyclic Pattern
-Noise
Time Series Decomposition
● A time series consists of three components:
● trend, seasonal, and random.
Time Series Decomposition

● Trend
● A trend is a long-term pattern in the data that shows its overall direction.
It can be upward, downward, or horizontal.
● Trends can be linear or nonlinear.
● A linear trend is a straight line that shows a constant increase or decrease
in the data.
● A nonlinear trend is a curve that shows a gradual increase or decrease in
the data.
Time Series Decomposition

● Seasonal
● A seasonal component is a pattern that repeats itself over a fixed period of
time.
● For example, sales of winter clothing are higher in the winter season
than in the summer season.
● The seasonal component is usually represented by a cycle that repeats
each year, quarter, month, or week.
Time Series Decomposition

● Random
● The random component is the part of the data that cannot be explained by
the trend or seasonal components.
● It represents the random fluctuations in the data that are not predictable.
Time Series Decomposition
● Features of the Plot:
● There is an upward trend, possibly
a curved one.
● There is seasonality – a regularly
repeating pattern of highs and lows
related to quarters of the year.
● There are no obvious outliers.
Time Series Decomposition
There are four types of time-series patterns:

● Trend: Long-term increase or decrease in the data. The trend can be any
function, such as linear or exponential, and can change direction over time.
● Seasonality: Repeating cycle in the series with fixed frequencies (hour of
the day, week, month, year, etc.). A seasonal pattern exists of a fixed
known period.
● Cyclicity: Occurs when the data rise and fall, but without a fixed
frequency and duration caused, for example, by economic conditions.
● Noise: The random variation in the series.
Time Series Decomposition
The trend and seasonality components are predictable (and are called systematic components), whereas, the noise, by definition, is random (and is

called the non-systematic component).

TAXONOMY OF TIME SERIES FORECASTING
1. TIME SERIES FORECASTING - Decomposition based
Method
TIME SERIES FORECASTING - Decomposition based
Method
● Time series decomposition involves breaking down a time series into its component parts, typically by separating it
into its trend, seasonal, and residual components.

● The trend component represents the long-term direction of the time series,

● Seasonal component captures the recurring patterns or cycles that occur within a time series.

● The residual component represents the random, unpredictable fluctuations that remain after the trend and seasonal
components have been accounted for.
TIME SERIES FORECASTING- Decomposition based
Method
● Time series decomposition has numerous applications

● Time series analysis for

○ Forecasting,

○ Anomaly detection,

○ Trend analysis.
TIME SERIES FORECASTING - Decomposition based
Method
● Additive Decomposition Method-
● Additive decomposition method. This involves simply adding the trend,
seasonal, and residual components together to obtain the original time
series:
● Y(t) = T(t) + S(t) + R(t)
● where Y(t) is the original time series at time t, T(t) is the trend
component at time t, S(t) is the seasonal component at time t, and R(t) is
the residual component at time t.
TIME SERIES FORECASTING - Decomposition based
Method
● To obtain the trend component, analysts typically use a moving average or a regression model to smooth out the
noise in the data and highlight the underlying long-term trend.
● The seasonal component can be obtained by averaging the values of the time series at each corresponding time
point across multiple cycles (e.g., days, weeks, or months).
● The residual component is then obtained by subtracting the trend and seasonal components from the original time
series.

link
Multiplicative decomposition of time series
● In the case of multiplicative decomposition the components are decomposed in the such a way that when they are
multiplied together, the original timeseries can be derived back.
● Time series = Trend X Seasonality X Noise
● Both additive and multiplicative time series decomposition can be represented by these equations

● where Tt, St, and Et are trend, seasonal, and error components respectively.
● The original time series yt is just an additive or multiplicative combination of components.
● If the magnitude of the seasonal fluctuation or the variation in trend changes with the level of the time series,
then multiplicative time series decomposition is the better model.
Additive and Multiplicative decomposition of time series
2. TIME SERIES FORECASTING - Smoothing based
forecasting
TIME SERIES FORECASTING - Smoothing based
forecasting

● Smoothing based forecasting methods involve smoothing

the past observations and projecting them to the future by
taking a weighted average of past observations.
TIME SERIES FORECASTING - Smoothing based
forecasting
● Smoothing based forecasting methods involve smoothing
the past observations and projecting them to the future by
taking a weighted average of past observations.
● It smooth out the noise in a time series and identify its underlying trend.
● The most commonly used smoothing-based forecasting techniques are
moving averages and exponential smoothing.
TIME SERIES FORECASTING - Smoothing based forecasting
In the smoothing based approaches, an observation is a function of past few
observations.
Simple Forecasting Methods
Simple Forecasting Methods
Average Method

● E.g one has monthly data from January 2010 to December 2010
and they want to predict the next January 2011 value, they would
simply average the values from January 2010 to December 2010.
Simple Forecasting Methods
Moving Average Smoothing

● The obvious problem with a simple average is figuring out how many points to use in the
average calculation
● One can select a window of the last “k” periods to calculate the average, and as the actual
data grows over time, one can always take the last k samples to average, that is, n, n- 1, ..., n -
k +1.
● When the actual data from January comes in, the February 2021 value is forecasted using
January 2021 (n), December 2020 (n - 1) and November 2020 (n - 3 +1 or n - 2).
Simple Forecasting Methods
Weighted Moving Average Smoothing

● For some cases, the most recent value could have more influence than some of the earlier
values.
Simple Forecasting Methods
Exponential Smoothing

● Exponential smoothing is the weighted average of the past data, with the recent data
points given more weight than earlier data points.
● The weights decay exponentially towards the earlier data points, hence, the name. The
exponential smoothing is given by the equation

α is generally between 0 and 1

Simple Forecasting Methods
Exponential Smoothing

Where Actual Value is yn and forecasted value Fn

E.g. February 2011 forecast using not only the actual January 2011 value but also the
previously forecasted January 2011 value, the new forecast would have “learnt” the data
better. This is the concept behind basic exponential smoothing .

The model is suited only for time series without clear trend or seasonality.
Simple Forecasting Methods :Exponential Smoothing
Simple Forecasting Methods :Exponential Smoothing
● Limitations:
○ The forecasts cannot be made more than one-step ahead, because to make a forecast
for step (n 1 1), the data for the previous step, n, is needed.
○ It is not possible to make forecasts several steps ahead, that is, (n + h), using the
methods described (where it was simply assumed that Fn+h = Fn+1), where h is the
horizon.
○ For making longer horizon forecasts, that is, where h>=1, the trend and seasonality
information also needs to be considered.
Simple Forecasting Methods :Exponential Smoothing
● Holt’s Two-Parameter Exponential Smoothing
○ A trend is an averaged long-term tendency of a time series.
○ Model described earlier is not particularly effective at capturing trends.
○ If the series also has a trend, then an average slope of the series needs to be
estimated as well.
○ Holt’s two parameter smoothing does by means of another parameter, β. A smoothing
equation similar to Eq. (12.10) is constructed for the average trend at n + 1. With two
parameters, α and β, any time series with a trend can be modeled and, therefore,
forecasted.
○ The forecast can be expressed as a sum of these two components, average value or
“level” of the series, Ln, and trend, Tn, recursively as:
Simple Forecasting Methods :Exponential Smoothing
● Holt’s Two-Parameter Exponential Smoothing

The values of the parameter can be estimated based on the best fit with the training (past) data
Simple Forecasting Methods :Exponential Smoothing
● Holt-Winters’ Three-Parameter Exponential Smoothing
○ When a time series contains seasonality in addition to a trend, yet another parameter,
γ, will be needed to estimate the seasonal component of the time series .
○ The estimates for value (or level) are now adjusted by a seasonal index, which is
computed with a third equation that includes γ.
Simple Forecasting Methods :Exponential Smoothing
TIME SERIES FORECASTING - Smoothing based forecasting
1. Moving Averages based Smoothing:
● Moving averages involve taking the average of a fixed number of consecutive
data points in a time series, and then using this average to predict future
values of the time series.
● The size of the moving average window (i.e., the number of data points
included in the average) can be adjusted to capture different levels of trend
or seasonality in the data.

2. Exponential Smoothing:

● Exponential smoothing, assigns exponentially decreasing weights to the

historical values of the time series, with more recent values given greater
weight.
● This technique is particularly useful for time series with trend and seasonality
components that change over time.
TIME SERIES FORECASTING - Smoothing based
forecasting
● They are relatively simple to implement and do not require extensive
knowledge of advanced mathematical concepts.
● Additionally, they can be applied to a wide range of time series data,
including financial data, stock prices, and weather data.
3. TIME SERIES FORECASTING - Regression based
methods
3. TIME SERIES FORECASTING - Regression based
methods
● Regression-based forecasting methods are
○ Linear regression,
○ Multiple regression,
○ Autoregressive integrated moving average
(ARIMA) models.
TIME SERIES FORECASTING - Regression based method
● Linear regression is a simple form of regression-based forecasting, where the
target variable's value at time t (yt) is estimated using a linear function of
time (t) with coefficients a and b. The coefficients are estimated from a
training set, and future values can be predicted using this model. yt = a * t + b
● Linear regression involves fitting a straight line to the historical data of the
time series, with the slope and intercept of the line representing the trend and
intercept of the time series, respectively.
● This method is useful for time series that have a linear trend over time.
TIME SERIES FORECASTING - Regression based method
● The variable time is the predictor or independent variable and the time series
value is the dependent.
● Regression based methods are generally preferable when the time series
appears to have a global pattern.
TIME SERIES FORECASTING - Regression based method
● The linear regression model is able to capture the long-term
tendency of the series, but it does a poor job of fitting the data.
● Polynomial functions can be used to improve the fit. Polynomial
regression is similar to linear regression except that higher-degree
functions of the independent variable are used
TIME SERIES FORECASTING - Regression based method
● Regression-based techniques can become more complex with the use of
exponential, polynomial, or power law functions to model the relationship
between future values and time. These models can capture more complex
patterns in the data and produce more accurate predictions.
● It involves modeling the relationship between a time series and multiple
predictor variables, such as economic indicators or weather patterns. This
method is useful for time series that are influenced by external factors.
● Choice of method depends on the nature of the data and the specific
forecasting problem.
Autoregressive Integrated Moving Average(ARIMA)
● Autocorrelation: Correlation measures how two variables are dependent on each
other or if they have a linear relationship with each other.

- Prod column shows

simple time series
data
- 1-lag series - In the
third column, data are
lagged by one step.
1992 Q1 data is shown
in 1992 Q2.
- additional 2-lag, 3-lag,
..., n-lag series in the
dataset.

A strong correlation between the original time series “prod” and

4-lag “prod-4.” They tend to move together. This is Autocorelation
Autoregressive Integrated Moving Average(ARIMA)
1.Autocorrelation:
● As in a multivariate correlation
matrix , one can measure the
strength of correlation between
the original time series and all the
lag series.
● The plot of the resultant correlation
matrix is called an
Autocorrelation Function (ACF)
chart.
● The ACF chart is used to study
all the available seasonality in
the time series.
● From Fig. it can be concluded that
the time series is correlated with
the 4th, 8th, and 12th lagged
quarter due to the yearly
seasonality.
● It is also evident than Q1 is
negatively correlated with Q2 and
Q3.
ARIMA
ARIMA is an acronym that stands for AutoRegressive Integrated Moving Average. It is a generalization of the simpler
AutoRegressive Moving Average and adds the notion of integration.

Let’s decode the essence of ARIMA:

● AR (Autoregression): This emphasizes the dependent relationship between an observation and its preceding or ‘lagged’
observations.
● I (Integrated): To achieve a stationary time series, one that doesn’t exhibit trend or seasonality, differencing is applied. It
typically involves subtracting an observation from its preceding observation.
● MA (Moving Average): This component zeroes in on the relationship between an observation and the residual error from a
moving average model based on lagged observations.

Each of these components is explicitly specified in the model as a parameter. A standard notation is used for ARIMA(p,d,q)
where the parameters are substituted with integer values to quickly indicate the specific ARIMA model being used.
ARIMA
The parameters of the ARIMA model are defined as follows:

● p: The lag order, representing the number of lag observations incorporated in the model.
● d: Degree of differencing, denoting the number of times raw observations undergo differencing.
● q: Order of moving average, indicating the size of the moving average window.
Autoregressive Integrated Moving Average(ARIMA)
Autoregression:
● Autoregressive models are regression models applied on lag series generated using the
original time series.
● In multiple linear regression, the output is a linear combination of multiple input variables.
● In the case of autoregression models, the output is the future data point and it can be
expressed as a linear combination for past p data points. p is the lag window

where, l is the level in the dataset and e is the noise. α are the coefficients that need to be learned
from the data.
This can be referred to as an autoregressive model with p lags or an AR(p) model.
In an AR(p) model, lag series is a new predictor used to fit the dependent variable, which is still the
original series value, Yt.
Autoregressive Integrated Moving Average(ARIMA)
Stationary Data:
In a time series with trends or seasonality, the value is affected by time
A time series is called stationary when the value of time series is not dependent on time.

Random white noise is a stationary time

series.
 Autoregressive Integrated Moving Average(ARIMA)
Differencing:
● A non-stationary time series can be converted to a stationary time series through a
technique called differencing.
● Differencing series is the change between consecutive data points in the series.

This is called first order differencing.

Just differencing once will still yield

a nonstationary time series.
 Autoregressive Integrated Moving Average(ARIMA)
Differencing:
● Second order differencing is the change between two consecutive data points in a first
order differenced time series.
● Differencing of order d is used to convert non stationary time series to stationary time series.
● Seasonal differencing is the change between the same period in two different seasons.
Assume a season has period, m
 Autoregressive Integrated Moving Average(ARIMA)
Moving Average Error
● Along with Autoregressive Models we can also use forecast errors of past data to predict
the future values

Autoregressive Models

where ei is the forecast error of data point i.

(This makes sense for the past data points but not for data point t because it is still
being forecasted.)
et is assumed as white noise. The regression equation for yt can be understood as
the weighted (θ) moving average of past q forecast errors.

This is called Moving Average with q lags model or MA(q)

 Autoregressive Integrated Moving Average(ARIMA)
6.Autoregressive Integrated Moving Average
● The Autoregressive Integrated Moving Average (ARIMA) model is a combination of the
differenced autoregressive model with the moving average model. It is expressed as:

Autoregressive Models

+
The AR part of ARIMA shows that the time series is regressed on its own past data.
The MA part of ARIMA indicates that the forecast error is a linear combination of past
respective errors.
The I part of ARIMA shows that the data values have been replaced with differenced values of
d order to obtain stationary data, which is the requirement of the ARIMA model approach
Autoregressive Integrated Moving Average(ARIMA)
6.Autoregressive Integrated Moving Average

The AR part of ARIMA shows that the time series is regressed on its own past data.
The MA part of ARIMA indicates that the forecast error is a linear combination of past
respective errors.
The I part of ARIMA shows that the data values have been replaced with differenced values of
d order to obtain stationary data, which is the requirement of the ARIMA model approach

● The predictors are the lagged p data points for the autoregressive part
● The lagged q errors are for the moving average part,
● The prediction is the difference yt in the dth order.
● ARIMA(p,d,q) model.
○ Estimating the coefficients α and θ for a given p,d,q is what ARIMA does
when it learns from the training data in a time series.
Autoregressive Integrated Moving Average(ARIMA)
● ARIMA is a generalized model.
● Some of the models are special cases of an ARIMA model.
● For example:
Seasonal ARIMA
Seasonal ARIMA- How to Implement
● Decide : (p,d,q)(P,D,Q)m.
● The optimal parameters for the Beer production dataset is
ARIMA(1,0,0)(1,1,0)4. The seasonal ARIMA model is used to forecast the
future 12 data points using the forecast() function
TIME SERIES FORECASTING - Regression based method
● A more sophisticated technique is based on the concept of autocorrelation.
Autocorrelation refers to the fact that data from adjacent time periods are
correlated in a time series. The most well-known among these techniques is
ARIMA, which stands for Auto Regressive Integrated Moving Average.
● ARIMA models, are a type of time series model that combines autoregression
(i.e., predicting future values based on past values) and moving averages (i.e.,
smoothing out the noise in the data) to model the underlying trend and
seasonality in a time series.
● This method is useful for time series that exhibit non-linear trends or
seasonality.
TIME SERIES FORECASTING - Regression based method
● Advantages:
● They can capture the influence of external factors on the time series, making them useful for
forecasting in a variety of industries, including finance and economics.
● Additionally, they can handle time series with non-linear trends or seasonality, which may not be
captured by other forecasting methods.
● Limitations/Drawbacks:
● Regression-based forecasting methods also have some limitations. They assume that the
relationship between the time series and predictor variables remains constant over time, which may
not always be the case.
● Additionally, they can be computationally intensive, requiring large amounts of data and
computational resources.
● Applications
● They are commonly used in finance to forecast stock prices and other financial indicators.
● They are also used in marketing to forecast sales and customer demand.
4. TIME SERIES FORECASTING-Machine Learning Based
Methods
4. TIME SERIES FORECASTING-Machine Learning Based
Methods
Machine Learning Models
● The series is transformed into cross-sectional data using a technique
called windowing.
● This technique defines a set of consecutive time series data as a
window, where the latest record forms the target while other series data
points, which are lagged compared to the target, form the input variables.

• Sufficient number of
windows are extracted
from the dataset,
• A supervised model can
be learned based on the
inferred relationship
between the lagged input
variables and the target
variable.
Windowing
● The purpose of windowing is to transform the time series data into a
generic machine learning input dataset.
Windowing
Windowing
● The series data are now converted into a generic cross-sectional
dataset that can be predicted with learning algorithms like regression,
neural networks, or support vector machines.
How to Implement
● (1) conversion to cross-sectional data,
● (2) training an machine learning model, and
● (3) forecasting one data point at a time in a loop.
Step 1: Set Up Windowing
● The operator must be informed that one of the columns in the
dataset is a date and should be considered as an “id.” - Set Role operator.
• If the input data has multiple time series, Select
• Attributes operator can be used to select the one to be forecasted.
Step 1: Set Up Windowing
Step 2: Train the Model
● When training any supervised model using this data, the attributes
labeled input Yt-5 through input Yt-0 form the independent variables.
● linear regression is used to fit the dependent variable called label, using
the independent variables input Yt-5 through input Yt-0.
Step 3: Generate the Forecast in a Loop
● Note that given this configuration of the window size and horizon, one

can now only make the forecast for the next step.
The regression equation is be used to predict December 2011 value.
Step 3: Generate the Forecast in a Loop
Neural Network Autoregressive
Neural Network Autoregressive
Evaluation of Time Series Models
Mean Absolute Error
● The error of the individual data point may be positive or negative and may
cancel each other out. To derive the overall forecast for the model,
calculate the absolute error to aggregate all the residuals and average it.
●
Root Mean Squared Error
● In some cases it is advantageous to penalize the individual point error
with higher residues. Even though two models have the same MAE, one
might have consistent error and the other might have low errors for some
points and high error for other points. RMSE penalizes the latter.
●
Mean Absolute Percentage Error
Mean Absolute Scaled Error
Sliding Window Validation
● Sliding window validation is a process of backtesting time series models built
through machine learning based methods.
● The whole cross-sectional dataset is divided into different training windows by
specifying the window width.
● A model is trained using a training window and applied on the testing window
to compute the performance for the first run.
For the next run, the training
● window is slid to new set of training records and the process is repeated until
all the training windows are used.
● By this technique, an average performance metric can be calculated across the
entire dataset.
● The performance metric derived through sliding window validation is generally more
robust than split validation technique.
Revision - steps to analyze time series dataset
Data Preparation

Model Refinement
Visualization

Forecasting Decomposition

Model Evaluation Model Selection

Model Fitting
Revision - steps to analyze time series dataset
● Data Preparation - It involves cleaning, transforming and organizing
data including handling missing values and outliers.
● Visualization- It involves creating graphs and plots of the data to help
identify trends, patterns , relationships, stability and stationarity of the
data.
● Decomposition - breaking down the data into its component parts such
as trend, seasonality and residuals to better understand the structure
of underlying data.
● Model Selection - selecting appropriate model to use for the analysis
such as simple moving average, exponential smoothing model, ARIMA ,
ML models etc.
Revision - steps to analyze time series dataset
● Model Fitting - This involves fitting the selected model to the data,
adjusting its parameters to maximize the accuracy of the forecasts.
● Model evaluation - This involves evaluating the performance of the
model using measures such as mean absolute error etc.
● Forecasting- This involves using the fitted model to make predictions
about future values of the variable based on past patterns and
relationships.
● Model Refinement - This involves fine tuning the model, to improve the
accuracy and reliability and to incorporate new information as it becomes
available.
Forecasting Best Practices:
Forecasting Best Practices:
Questions
● What are the features of time series data set?
● What is difference between univariate and Multivariate time series
dataset?
● Given a time series dataset, how to perform time series decomposition ?
● Explain the steps for time series data analysis.
● What are the evaluation measures used for evaluating performance of
time series models?