Logistic Regression in Machine Learning

o Logistic regression is one of the most popular Machine Learning algorithms, which comes under the
Supervised Learning technique. It is used for predicting the categorical dependent variable using a
given set of independent variables.

o Logistic regression predicts the output of a categorical dependent variable. Therefore the outcome must
be a categorical or discrete value. It can be either Yes or No, 0 or 1, true or False, etc. but instead of
giving the exact value as 0 and 1, it gives the probabilistic values which lie between 0 and 1.

o Logistic Regression is much similar to the Linear Regression except that how they are used. Linear
Regression is used for solving Regression problems, whereas Logistic regression is used for solving
the classification problems.

o In Logistic regression, instead of fitting a regression line, we fit an "S" shaped logistic function, which
predicts two maximum values (0 or 1).

o The curve from the logistic function indicates the likelihood of something such as whether the cells are
cancerous or not, a mouse is obese or not based on its weight, etc.

o Logistic Regression is a significant machine learning algorithm because it has the ability to provide
probabilities and classify new data using continuous and discrete datasets.

o Logistic Regression can be used to classify the observations using different types of data and can easily
determine the most effective variables used for the classification. The below image is showing the
logistic function:

Note: Logistic regression uses the concept of predictive modeling as regression; therefore, it is called logistic
regression, but is used to classify samples; Therefore, it falls under the classification algorithm.

Logistic Function (Sigmoid Function):

 o The sigmoid function is a mathematical function used to map the predicted values to probabilities.

o It maps any real value into another value within a range of 0 and 1.

o The value of the logistic regression must be between 0 and 1, which cannot go beyond this limit, so it
forms a curve like the "S" form. The S-form curve is called the Sigmoid function or the logistic
function.

o In logistic regression, we use the concept of the threshold value, which defines the probability of either
0 or 1. Such as values above the threshold value tends to 1, and a value below the threshold values
tends to 0.

Assumptions for Logistic Regression:

o The dependent variable must be categorical in nature.

o The independent variable should not have multi-collinearity.

Logistic Regression Equation:

The Logistic regression equation can be obtained from the Linear Regression equation. The mathematical steps
to get Logistic Regression equations are given below:

o We know the equation of the straight line can be written as:

o In Logistic Regression y can be between 0 and 1 only, so for this let's divide the above equation by (1-
y):

o But we need range between -[infinity] to +[infinity], then take logarithm of the equation it will become:

The above equation is the final equation for Logistic Regression.

Type of Logistic Regression:

On the basis of the categories, Logistic Regression can be classified into three types:

o Binomial: In binomial Logistic regression, there can be only two possible types of the dependent
variables, such as 0 or 1, Pass or Fail, etc.
 o Multinomial: In multinomial Logistic regression, there can be 3 or more possible unordered types of
the dependent variable, such as "cat", "dogs", or "sheep"

o Ordinal: In ordinal Logistic regression, there can be 3 or more possible ordered types of dependent
variables, such as "low", "Medium", or "High".

Linear Regression vs Logistic Regression

Linear Regression and Logistic Regression are the two famous Machine
Learning Algorithms which come under supervised learning technique.
Since both the algorithms are of supervised in nature hence these
algorithms use labeled dataset to make the predictions. But the main
difference between them is how they are being used. The Linear
Regression is used for solving Regression problems whereas Logistic
Regression is used for solving the Classification problems. The description
of both the algorithms is given below along with difference table.

Linear Regression:
o Linear Regression is one of the most simple Machine learning
algorithm that comes under Supervised Learning technique and
used for solving regression problems.
o It is used for predicting the continuous dependent variable with the
help of independent variables.
o The goal of the Linear regression is to find the best fit line that can
accurately predict the output for the continuous dependent variable.
o If single independent variable is used for prediction then it is called
Simple Linear Regression and if there are more than two
independent variables then such regression is called as Multiple
Linear Regression.
o By finding the best fit line, algorithm establish the relationship
between dependent variable and independent variable. And the
relationship should be of linear nature.
o The output for Linear regression should only be the continuous
values such as price, age, salary, etc. The relationship between the
dependent variable and independent variable can be shown in
below image:
In above image the dependent variable is on Y-axis (salary) and
independent variable is on x-axis(experience). The regression line can be
written as:

y= a0+a1x+ ε

Where, a0 and a1 are the coefficients and ε is the error term.

Logistic Regression:

o Logistic regression is one of the most popular Machine learning

algorithm that comes under Supervised Learning techniques.
o It can be used for Classification as well as for Regression problems,
but mainly used for Classification problems.
o Logistic regression is used to predict the categorical dependent
variable with the help of independent variables.
o The output of Logistic Regression problem can be only between the
0 and 1.
o Logistic regression can be used where the probabilities between two
classes is required. Such as whether it will rain today or not, either 0
or 1, true or false etc.
o Logistic regression is based on the concept of Maximum Likelihood
estimation. According to this estimation, the observed data should
be most probable.
o In logistic regression, we pass the weighted sum of inputs through
an activation function that can map values in between 0 and 1. Such
activation function is known as sigmoid function and the curve
obtained is called as sigmoid curve or S-curve. Consider the below
image:
 o The equation for logistic regression is:

Difference between Linear Regression and Logistic Regression:

Linear Regression Logistic Regression

Linear regression is used to predict Logistic Regression is used to

the continuous dependent variable predict the categorical dependent
using a given set of independent variable using a given set of
variables. independent variables.

Linear Regression is used for solving Logistic regression is used for

Regression problem. solving Classification problems.

In Linear regression, we predict the In logistic Regression, we predict the

value of continuous variables. values of categorical variables.

In linear regression, we find the best In Logistic Regression, we find the S-

fit line, by which we can easily curve by which we can classify the
predict the output. samples.

Least square estimation method is Maximum likelihood estimation

used for estimation of accuracy. method is used for estimation of
accuracy.

The output for Linear Regression The output of Logistic Regression

must be a continuous value, such as must be a Categorical value such as
price, age, etc. 0 or 1, Yes or No, etc.
In Linear regression, it is required In Logistic regression, it is not
that relationship between required to have the linear
dependent variable and relationship between the dependent
independent variable must be and independent variable.
linear.

In linear regression, there may be In logistic regression, there should

collinearity between the not be collinearity between the
independent variables. independent variable.