Scribd
Upload
10 views7 pages

M2 Logistic Regression Classcopy 4

Logistic regression is a statistical method used for binary classification that employs a sigmoid function to predict probabilities between 0 and 1. It can be categorized into binomial, multinomial, and ordinal types based on the dependent variable's nature. Key concepts include independent and dependent variables, odds, log-odds, coefficients, and maximum likelihood estimation, with the aim of minimizing log loss through model parameter adjustments.

Uploaded by

debasmita2022
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
10 views7 pages

M2 Logistic Regression Classcopy 4

Logistic regression is a statistical method used for binary classification that employs a sigmoid function to predict probabilities between 0 and 1. It can be categorized into binomial, multinomial, and ordinal types based on the dependent variable's nature. Key concepts include independent and dependent variables, odds, log-odds, coefficients, and maximum likelihood estimation, with the aim of minimizing log loss through model parameter adjustments.

Uploaded by

debasmita2022
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

Logistic Regression

What is Logistic Regression?

Logistic regression is used for binary classification where we use sigmoid function, that takes input as
independent variables and produces a probability value between 0 and 1.

For example, we have two classes Class 0 and Class 1 if the value of the logistic function for an input
is greater than 0.5 (threshold value) then it belongs to Class 1 otherwise it belongs to Class 0. It’s
referred to as regression because it is the extension of linear regression but is mainly used for
classification problems.

Key Points:

 Logistic regression predicts the output of a categorical dependent variable. Therefore, the
outcome must be a categorical or discrete value.

 It can be either Yes or No, 0 or 1, true or False, etc. but instead of giving the exact value as 0
and 1, it gives the probabilistic values which lie between 0 and 1.

 In Logistic regression, instead of fitting a regression line, we fit an “S” shaped logistic
function, which predicts two maximum values (0 or 1).
Logistic Function – Sigmoid Function

 The sigmoid function is a mathematical function used to map the predicted values to
probabilities.

 It maps any real value into another value within a range of 0 and 1. The value of the logistic
regression must be between 0 and 1, which cannot go beyond this limit, so it forms a curve
like the “S” form.

 The S-form curve is called the Sigmoid function or the logistic function.

 In logistic regression, we use the concept of the threshold value, which defines the
probability of either 0 or 1. Such as values above the threshold value tends to 1, and a value
below the threshold values tends to 0.
Types of Logistic Regression

On the basis of the categories, Logistic Regression can be classified into three types:

1. Binomial: In binomial Logistic regression, there can be only two possible types of the
dependent variables, such as 0 or 1, Pass or Fail, etc.

2. Multinomial: In multinomial Logistic regression, there can be 3 or more possible unordered


types of the dependent variable, such as “cat”, “dogs”, or “sheep”

3. Ordinal: In ordinal Logistic regression, there can be 3 or more possible ordered types of
dependent variables, such as “low”, “Medium”, or “High”.

Terminologies involved in Logistic Regression

Here are some common terms involved in logistic regression:

 Independent variables: The input characteristics or predictor factors applied to the


dependent variable’s predictions.

 Dependent variable: The target variable in a logistic regression model, which we are trying
to predict.
 Logistic function: The formula used to represent how the independent and dependent
variables relate to one another. The logistic function transforms the input variables into a
probability value between 0 and 1, which represents the likelihood of the dependent
variable being 1 or 0.

 Odds: It is the ratio of something occurring to something not occurring. it is different from
probability as the probability is the ratio of something occurring to everything that could
possibly occur.

 Log-odds: The log-odds, also known as the logit function, is the natural logarithm of the
odds. In logistic regression, the log odds of the dependent variable are modeled as a linear
combination of the independent variables and the intercept.

 Coefficient: The logistic regression model’s estimated parameters, show how the
independent and dependent variables relate to one another.

 Intercept: A constant term in the logistic regression model, which represents the log odds
when all independent variables are equal to zero.

 Maximum likelihood estimation: The method used to estimate the coefficients of the
logistic regression model, which maximizes the likelihood of observing the data given the
model.
The Log Loss of Logistic Regression can be summarized with the following
formula:
where,
 m is the number of training examples
 yi is the true class label for the i-th example (either 0 or 1).
 hθ(xi) is the predicted probability for the i-th example, as calculated by
the logistic regression model.
 θ is the model parameters
The first term in the sum represents the cross-entropy for the positive class
(yi=1), and the second term represents the cross-entropy for the negative class
(yi=0). The goal of logistic regression is to minimize the cost function by
adjusting the model parameters θ.

Example 1:

Example 2:

You might also like