Predictive Modelling

What is Predictive Modelling?

Predictive modeling is a statistical technique

used to predict the outcome of future events
based on historical data.
It involves building a mathematical model that
takes relevant input variables and generates a
predicted output variable.
Machine learning algorithms are
used to train and improve these
models to help you make better
decisions.

Predictive modeling is used in many

industries and applications and can
solve a wide range of issues, such
as fraud detection, customer
segmentation, disease diagnosis,
and stock price prediction.
 Benefits
In today's data-driven world, your organization
is likely inundated with massive amounts of
complex and rapidly changing data from
various sources.
Augmented analytics such as predictive
modeling, predictive or prescriptive analytics,
and prescriptive analytics can help you leverage
this big data to enhance your decision-making
processes and improve overall performance.
Whether it's optimizing revenue, streamlining
operations, or combating fraud, predictive
modeling empowers you to make data-driven
decisions that are less susceptible to human
bias and error.

This allows you to focus on executing your

plans instead of wasting time second-guessing
decisions.
Improved decision-making: Gain insights into
future trends and patterns, enabling you to make
informed decisions based on data-driven insights.

Increased efficiency: Automate processes and

streamline your operations, reducing the time and
effort required to perform complex analyses.

Enhanced accuracy: Use large amounts of data to

identify patterns and make predictions, resulting in
more accurate forecasts than traditional methods.
Better risk management: Get help identifying potential
risks and mitigate them before they occur, reducing the
likelihood of financial loss or other negative outcomes.

Increased customer satisfaction: Better understand your

customers' needs and preferences, leading to improved
products and services that better meet your customers'
needs.

Competitive advantage: Gain a competitive advantage by

identifying and acting on opportunities faster and more
effectively than your competitors.
 Challenges
While predictive modeling has numerous benefits, it also presents some key
challenges:

Poor quality data, such as data with missing values or outliers, can negatively impact
the accuracy of your models.

Overfitting occurs when your model is too complex and fits the training data too
closely. This can result in a model that performs well on the training data but fails to
generalize to new data.

Model interpretability can also be an issue if your model is too complex. This makes it
challenging for you to understand how it arrived at its predictions.

Selection bias can occur if your training data is not representative of the population
being studied. This can lead to inaccurate predictions and unfair outcomes.

Unforeseen changes in the future can render your model inaccurate since it is based on
historical data. Unexpected changes can be especially problematic for models that are
used for long-term predictions.
 Examples of predictive modeling

Predictive modeling is used across a wide range of

industries and job roles, and the following are some
examples of use cases in different industries.

In the financial services sector, it’s used to forecast the

likelihood of loan default, identify and prevent fraud,
and predict future price movements of securities.
Insurance companies use it to assess policy
applications based on the risk pool of similar
policyholders, in order to predict the likelihood
of future claims.

Energy and utilities use it to mitigate safety

risks by analyzing historical equipment failures,
and to predict future energy needs based on
previous demand cycles.
Healthcare companies use it to better manage
patient care by forecasting patient admissions
and readmissions.

The public sector uses it to analyze population

trends, and to plan infrastructure investments
and other public works projects accordingly.
Predictive Modeling vs Predictive Analytics

Predictive modeling is such an important

part of predictive analytics, the two terms
are often used interchangeably. However,
predictive modeling is a subset of
predictive analytics, and refers specifically
to the modeling stage of the overall
process.
Predictive analytics, is a broad term that
encompasses the entire process of using data,
statistical algorithms, and machine learning
techniques to make predictions about future
events or outcomes. This includes everything from
data preparation and cleansing, to data integration
and exploration, developing and deploying
models, and collaborating and sharing the findings.

As stated above, predictive modeling refers to the

process of using statistical algorithms and machine
learning techniques to build a mathematical model
that can be used to predict future outcomes based
on historical data.
 Model Types and Algorithms
Regression

Regression models are used to predict a continuous numerical value

based on one or more input variables.
The goal of a regression model is to identify the relationship between
the input variables and the output variable, and use that relationship to
make predictions about the output variable.
Regression models are commonly used in various fields, including
financial analysis, economics, and engineering, to predict outcomes such
as sales, stock prices, and temperatures.

Scatter plot with blue data points and a red linear regression line,
showing a positive correlation.
 Neural network
Neural network models are a type of predictive modeling
technique inspired by the structure and function of the
human brain.
The goal of these models is to learn complex
relationships between input variables and output
variables, and use that information to make predictions.
Neural network models are often used in fields such as
image recognition, natural language processing, and
speech recognition, to make predictions such as object
recognition, sentiment analysis, and speech transcription.
 Neural network model algorithims
Multilayer Perceptron (MLP) consists of multiple layers of nodes, including an input layer, one or
more hidden layers, and an output layer. The nodes in each layer perform a mathematical
operation on the input data, with the output of one layer serving as the input for the next layer.
The weights between the nodes are adjusted during training using backpropagation to minimize
the error between the predicted output and the actual output. MLP is a versatile algorithm that
can be used for a wide range of predictive modeling tasks, including classification, regression, and
pattern recognition.

Convolutional neural networks (CNN) are commonly used for image recognition tasks, with each
layer processing increasingly complex features of the image.

Recurrent neural networks (RNN) are used for sequential data, such as natural language
processing, and incorporate feedback loops that allow previous output to be used as input for the
next prediction.

Long Short-Term Memory (LSTM) is a type of RNN that addresses the vanishing gradient problem
and is particularly useful for learning long-term dependencies in sequential data.
Backpropagation is a common algorithm used to train neural networks
by adjusting the weights between nodes in the network based on the
error between the predicted output and the actual output.

Feedforward neural networks consist of layers of nodes that process

information from previous layers, with each node performing a
mathematical operation on the input data.

Auto encoder is used for unsupervised learning, where the network is

trained to reconstruct the input data and can be used for tasks such as
dimensionality reduction and anomaly detection.

Generative Adversarial Networks (GAN) involves two neural networks,

one that generates synthetic data and another that discriminates
between real and synthetic data, and is commonly used for tasks such
as image generation and data synthesis.
 Classification

Classification models are used to classify data into one or

more categories based on one or more input variables.
Classification models identify the relationship between the
input variables and the output variable, and use that
relationship to accurately classify new data into the
appropriate category.
Classification models are commonly used in fields like
marketing, healthcare, and computer vision, to classify
data such as spam emails, medical diagnoses, and image
recognition.
 Classification model algorithms:
• Decision trees are a graphical representation of a set of rules
used to make decisions based on a series of if-then statements.

• Random forests are an ensemble method that combines multiple

decision trees to improve accuracy and reduce errors.

• Naive Bayes is a probabilistic model that assumes independence

between input variables

• Support vector machines (SVM) and k-nearest neighbors (KNN)

are distance-based models that use mathematical algorithms to
classify data.
 Clustering
Clustering models are used to group data points
together based on similarities in their input variables.
The goal of a clustering model is to identify patterns
and relationships within the data that are not
immediately apparent, and group similar data points
into clusters.
Clustering models are typically used for customer
segmentation, market research, and image
segmentation, to group data such as customer
behavior, market trends, and image pixels.
 Clustering model algorithms:

K-means clustering is a popular method that partitions the

data into k clusters based on the distances between data
points.

Hierarchical clustering creates a tree-like structure of nested

clusters based on the distances between data points.

Density-based clustering groups data points based on their

density in a particular area.
 Time series
Time series models are used to analyze and
forecast data that varies over time.
Time series models help you identify patterns and
trends in the data and use that information to
make predictions about future values.
Time series models are used in a wide variety of
fields, including financial analytics, economics,
and weather forecasting, to predict outcomes such
as stock prices, GDP growth, and temperatures.
 Time series model algorithms:

• ARIMA (autoregressive integrated moving average) algorithms use

previous values of a time series to predict future values, taking
into account factors such as seasonality, trends, and stationarity.

• Exponential smoothing algorithms use a weighted average of past

observations to predict future values, and are particularly useful
for short-term forecasting.

• Seasonal decomposition algorithms decompose the time series

into seasonal, trend, and residual components, and then use
those components to make predictions.
 Decision Tree
Decision tree models use a tree-like structure to model decisions and their possible
consequences. The tree consists of nodes that represent decision points, with
branches representing the possible outcomes or consequences of each decision.
Each node corresponds to a predictor variable and each branch corresponds to a
possible value of that variable. The goal of a decision tree model is to predict the
value of a target variable based on the values of the predictor variables. The model
uses the tree structure to determine the most likely outcome for a given set of
predictor variable values.

Decision tree models can be used for both classification and regression tasks. In a
classification tree, the target variable is categorical, while in a regression tree, the
target variable is continuous. Decision tree models are easy to interpret and
visualize, making them useful for understanding the relationships between predictor
variables and the target variable. However, they can be prone to overfitting and may
not perform as well as other predictive modeling techniques on complex datasets.
 Decision tree model algorithms:

• CART (Classification and Regression Tree) can be used for both classification and regression tasks. It
uses Gini impurity as a measure of the quality of a split, aiming to minimize it. CART constructs
binary trees, where each non-leaf node has two children.

• CHAID (Chi-squared Automatic Interaction Detection) is used for categorical variables and constructs
trees based on chi-squared tests to determine the most significant associations between the
predictor variables and the target variable. It can handle both nominal and ordinal categorical
variables.

• ID3 (Iterative Dichotomiser 3) is used to build decision trees for classification tasks. It selects the
attribute with the highest information gain at each node to split the data into subsets. Information
gain is calculated based on the entropy of the subsets.

• C4.5 is an extension of the ID3 algorithm that can handle both categorical and continuous variables.
It uses information gain ratio to select the splitting attribute, which takes into account the number of
categories and their distribution in the subsets.

• These algorithms use various criteria to determine the optimal split at each node, such as
information gain, Gini index, or chi-squared test.
 Regression model algorithms

Linear regression models assume that there is a linear

relationship between the input variables and the output variable.

Polynomial regression models assume a non-linear relationship

between input and output.

Logistic regression models are used for binary classification

problems, where the output variable is either 0 or 1.
 Ensemble

Ensemble models combine multiple models to

improve their predictive accuracy and stability. By
combining multiple models, the errors and biases of
individual models are usually reduced, leading to
better overall performance.
Ensemble models can be used for both classification
and regression tasks and are well suited for data
mining. They’re often used in machine learning or AI
competitions and real-world applications where high
predictive accuracy is required.
 Ensemble model algorithms:

Bagging (Bootstrap Aggregating) involves creating multiple versions of the same

prediction model on different subsets of the training data, and then aggregating
their predictions to make the final prediction. Bagging is used to reduce the
variance of a single model and improve its stability.

Boosting involves creating multiple weak models sequentially, where each model
tries to correct the errors of the previous model. Boosting is used to reduce the
bias of a single model and improve its accuracy.

Stacking involves training multiple models and using their predictions as input to
a meta-model, which then makes the final prediction. Stacking is used to combine
the strengths of multiple models and achieve better performance.

Random Forest is an extension of bagging that uses decision trees as the base
models. Random Forest creates multiple decision trees on different subsets of the
training data, and then aggregates their predictions to make the final prediction.