Scribd
Upload
33 views

2-Logistic Regression

Logistic regression is a classification algorithm that uses an S-shaped logistic function to predict a binary or categorical dependent variable. It is similar to linear regression but is used for classification problems rather than regression. Logistic regression models the logarithm of odds using an equation involving the independent variables to generate probabilities between 0 and 1, which can be used to classify samples.

Uploaded by

abdala sabry
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
33 views

2-Logistic Regression

Logistic regression is a classification algorithm that uses an S-shaped logistic function to predict a binary or categorical dependent variable. It is similar to linear regression but is used for classification problems rather than regression. Logistic regression models the logarithm of odds using an equation involving the independent variables to generate probabilities between 0 and 1, which can be used to classify samples.

Uploaded by

abdala sabry
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

Logistic

Regression
Classification
Machine Learning
Why use Logistic Regression?

• Logistic regression is one of the most popular Machine Learning algorithms,


which comes under the Supervised Learning technique. It is used for predicting
the categorical dependent variable using a given set of independent variable. It
can be either Yes or No, 0 or 1, true or False, etc. but instead of giving the exact
value as 0 and 1, it gives the probabilistic values which lie between 0 and 1.
• Logistic Regression is much similar to the Linear Regression except that how
they are used. Linear Regression is used for solving Regression problems,
whereas Logistic regression is used for solving the classification problems.
The below image is showing the logistic function:

Note: Logistic regression uses the concept of predictive modeling as regression;


therefore, it is called logistic regression, but is used to classify samples; Therefore, it
falls under the classification algorithm
principles of Logistic Regression.

Logistic Function (Sigmoid Function):


• The sigmoid function is a mathematical function used to map the predicted
values to probabilities.
• It maps any real value into another value within a range of 0 and 1.
• The value of the logistic regression must be between 0 and 1, which cannot
go beyond this limit, so it forms a curve like the "S" form. The S-form
curve is called the sigmoid function or the logistic function.
• In logistic regression, we use the concept of the threshold value, which
defines the probability of either 0 or 1. Such as values above the threshold
value tends to 1, and a value below the threshold values tends to 0.
• The dependent variable must be categorical in nature.
Logistic Regression Equation:

The Logistic regression equation can be obtained from the Linear Regression
equation. Themathematical steps to get Logistic Regression equations are given
below:
We know the equation of the straight line can be written as:

In Logistic Regression y can be between 0 and 1 only, so for this let's divide the
aboveequation by (1-y), and we need range between -[infinity] to +[infinity], then take
logarithm of the equation itwill become
Logistic Regression Equation:

In Logistic Regression y can be between 0 and 1 only, so for this let's divide the
aboveequation by (1-y), and we need range between -[infinity] to +[infinity], then take
logarithm of the equation itwill become

𝑝𝑖
𝑙𝑜𝑔𝑒 = 𝜃𝑇𝑥
1 − 𝑝𝑖
Logistic Regression Equation:
𝑝𝑖
𝑙𝑜𝑔𝑒 = 𝜃𝑖 𝑥
1 − 𝑝𝑖

𝑝𝑖
= 𝑒 𝜃𝑖 𝑥
1 − 𝑝𝑖

𝑝𝑖 = (1 − 𝑝𝑖 )𝑒 𝜃𝑖 𝑥

1
𝑝𝑖 =
1+𝑒 −𝜃𝑖 𝑥
types of Logistic Regression

Binomial: In binomial Logistic regression, there can be only two possible


types of the dependent variables, such as 0 or 1, Pass or Fail, etc.
Multinomial: In multinomial Logistic regression, there can be 3 or more
possible unordered types of the dependent variable, such as "cat", "dogs",
or "sheep“.
Ordinal: In ordinal Logistic regression, there can be 3 or more possible
ordered types ofdependent variables, such as "low", "Medium", or "High”.
advantages and limitations of the Logistic Regression

Advantages Limitations
Logistic regression is easier to implement, If the number of observations is lesser than
interpret, and very efficient to train. the number of features, Logistic
Regression should not be used; otherwise,
it may lead to overfitting.
It makes no assumptions about It can only be used to predict discrete
distributions of classes in feature space. functions.
Good accuracy for many simple data sets In Linear Regression independent
and it performs well when the dataset is and dependent variables are related
linearly separable. linearly. But Logistic Regression needs that
Logistic regression is less inclined to over- independent variables are linearly related
fitting but it can overfit in high to the log odds (log(p/(1-p)).
dimensional datasets.
Difference between logistic regression and linear regression
Linear regression Logistic regression
Linear regression is used to predict the Logistic Regression
continuous dependent variable using a given is used to predict the categorical dependent
set of independent variables. variable using a given set of independent
variables.
is used for solving Regression problem. is used for solving classification problem.
we predict the value of continuous we predict the value of categorical
variables. variables.
we find the best fit line, by which we can we find the S-curve by which
easily predict the output. we can classify the samples.
Here no threshold value is needed. a threshold value is added.
We plot the training datasets, a straight line Any change in the coefficient leads to a
can be drawn that touches maximum plots. change in the direction of the logistic
function. It means positive slopes result in
an S- shaped curve and negative slopes
result in a Z- shaped curve.
The Similarities between Linear Regression and Logistic Regression
• Linear Regression and Logistic Regression both are supervised
Machine Learningalgorithms.
• Linear Regression and Logistic Regression, both the models are
parametricregression i.e. both the models use linear equations for
predictions
That‘s all the similarities we have between these two models.
The Cost function of linear regression

Cost function of Logistic Regression

The above two functions can be compressed into


a single function :
Gradient Descent

Want :
Repeat

(simultaneously update all )

Algorithm looks identical to linear regression!

You might also like