SUPERVISED REGRESSION AND SUPERVISED CLASSIFICATION

Dr. Muhammad sajid khan

SUPERVISED MACHINE LEARNING
CLASSIFICATION

Learning goals
Understand why classification models

have a score / probability as output

and not a class
Log. Regr. Naive Bayes
Understand the difference between
scoring and probabilistic classifiers
0.100 0.100

0.075 0.075
Response Response

Know the concept of decision regions

0.050 M 0.050
V2

V2

R R

0.025 0.025

0.000
0.00 0.01 0.02 0.03
V1
0.04 0.05
0.000
0.00 0.01 0.02 0.03
V1
0.04 0.05 and boundaries
CART Tree SVM
0.100

0.075
0.100
Know the difference between
0.075

generative and discriminant approach

Response Response

0.050 M 0.050 M
V2

V2

R R

0.025 0.025

0.000 0.000
0.00 0.01 0.02 0.03 0.04 0.05 0.00 0.01 0.02 0.03 0.04 0.05
V1 V1
WORKING OF SUPERVISED LEARNING
ADVANTAGES OF SUPERVISED LEARNING:
 With the help of supervised learning, the model can
predict the output on the basis of prior experiences.
 In supervised learning, we can have an exact idea
about the classes of objects.
 Supervised learning model helps us to solve various
real-world problems such as fraud detection,
spam filtering, etc.
DISADVANTAGES OF SUPERVISED
LEARNING:
 Supervised learning models are not suitable for
handling the complex tasks.
 Supervised learning cannot predict the correct
output if the test data is different from the training
dataset.
 Training required lots of computation times.
 In supervised learning, we need enough knowledge
about the classes of object.
CLASSIFICATION
 Classification algorithms are used when the output
variable is categorical, which means there are two
classes such as Yes-No, Male-Female, True-false, etc.
 Spam Filtering,
 Random Forest
 Decision Trees
 Logistic Regression
 Support vector Machines
STEPS INVOLVED IN SUPERVISED
LEARNING:
 First Determine the type of training dataset
 Collect/Gather the labeled training data.
 Split the training dataset into training dataset, test dataset, and
validation dataset.
 Determine the input features of the training dataset, which should have
enough knowledge so that the model can accurately predict the output.
 Determine the suitable algorithm for the model, such as support vector
machine, decision tree, etc.
 Execute the algorithm on the training dataset. Sometimes we need
validation sets as the control parameters, which are the subset of
training datasets.
 Evaluate the accuracy of the model by providing the test set. If the
model predicts the correct output, which means our model is accurate
REGRESSION
 Regression algorithms are used if there is a relationship
between the input variable and the output variable. It is
used for the prediction of continuous variables, such as
Weather forecasting, Market Trends, etc. Below are some
popular Regression algorithms which come under
supervised learning:
 Linear Regression
 Regression Trees
 Non-Linear Regression
COMMON USE FOR MACHINE LEARNING
REGRESSION MODELS INCLUDE:
 Forecasting continuous outcomes like house prices,
stock prices, or sales.
 Predicting the success of future retail sales or marketing
campaigns to ensure resources are used effectively.
 Predicting customer or user trends, such as on
streaming services or e-commerce websites.
 Analyzing datasets to establish the relationships
between variables and an output.
 Predicting interest rates or stock prices from a variety of
factors.
 Creating time series visualizations.
INTRODUCTION TO LINEAR
REGRESSION
 Definition: Linear regression is a statistical method that
models the relationship between a dependent variable
(Y) and one or more independent variables (X).
 Purpose: To predict the value of Y based on X.
 Mathematical Equation:
 Simple Linear Regression:
 Where:
 YYY = dependent variable (target)
 XXX = independent variable (feature)
 b0b_0b0​= intercept
 b1b_1b1​= slope
 ϵ\epsilonϵ = error term
LINEAR REGRESSION ANALYSIS MODEL
 STEPS IN A REGRESSION ANALYSIS
1.Examine the scatterplot of the data.
• Does the relationship look linear?
• Are there points in locations they shouldn’t be?
• Do we need a transformation?
2. Assuming a linear function looks appropriate, estimate the
regression parameters.
• How do we do this? (Method of Least Squares)
3. Test whether there really is a statistically significant linear
relationship. Just because we assumed a linear function it does
not follow that the data support this assumption.
• How do we test this? (F-test for Variances)
 CLASSIFICATION APPROACHES
Two fundamental approaches exist to construct classifiers: The
generative approach and the discriminant approach.

They tackle the classification problem from different angles:

Generative classification approaches assume a data-generating process in
which the distribution of the features x is different for the various classes of
the output y , and try to learn these conditional distributions:
“Which y tends to have x like these?”

Discriminant approaches use empirical risk minimization

based on a suitable loss function:
“What is the best prediction for y given these x?”
©
CONTINUE
 4. If there is a significant linear relationship, estimate the
response, Y, for the given values of X, and compute the
residuals.
 5. Examine the residuals for systematic inadequacies in the
linear model as fit to the data.
• Is there evidence that a more complicated relationship
(say a polynomial) should be considered; are there
problems with the regression assumptions? (Residual
analysis).
• Are there specific data points which do not seem to
follow the proposed relationship? (Examined using
influence measures).
SUPERVISED CLASSIFICATION

 It
can be defines as the process of using
samples (training data) of known
identify to classify pixels of unknown
identify
SIMPLE LINEAR REGRESSION - EXAMPLE AND
THEORY
SITUATION: A company that repairs small Number Repair
computers needs to develop a better way of of components time
providing customers typical repair cost estimates. i xi yi
To begin this process, they compiled data on repair 1 1 23
2 2 29
times (in minutes) and the number of components
3 4 64
needing repair or replacement from the previous
4 4 72
week. The data, sorted by number of components 5 4 80
are as follows: 6 5 87
7 6 96
8 6 105
9 8 127
Paired Observations (xi, yi) 10 8 119
11 9 145
12 9 149
13 10 165
14 10 154
DATA ANALYSIS SOFTWARE (MINITAB)
SPECIFYING
MODEL AND
OUTPUT OPTIONS

Introd
uction
to
Machi
ne
Learni
ng –
 STA6166-RegBasics
Introd
uction
to
Machi
ne
Learni
ng –
REFERENCE
 https://www.wikihow.com/Use-Minitab
 https://www.seldon.io/machine-learning-regression-explain
ed#:~:text=Machine%20Learning%20Regression%20is%2
0a,used%20to%20predict%20continuous%20outcomes
.
THANK YOU