Scribd
Upload
12 views21 pages

Logistic Regression (Autosaved)

Logistic Regression for ML.

Uploaded by

7984789201abc
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
12 views21 pages

Logistic Regression (Autosaved)

Logistic Regression for ML.

Uploaded by

7984789201abc
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

Logistic Regression

Can we use linear regression for


Classification?
Can we use linear regression for
Classification?
Can we use linear regression for
Classification?
Can we use linear regression for
Classification?
Can we use linear regression for
Classification?
Can we use linear regression for
Classification?

Now for the new point with


weight 180 probability will be >
1.
Can we use linear regression for
Classification?

For few cases the probability


will be <1.
What is logistic regression?
• Logistic regression is the appropriate regression analysis to
conduct when the dependent variable is dichotomous (binary).
• Like all regression analyses, logistic regression is a predictive
analysis.
• Logistic regression is used to describe data and to explain the
relationship between one dependent binary variable and one or
more independent variables.
• The outcome can either be yes or no (2 outputs).
Applications
• Manufacturing  Find probability of failure
• Healthcare  Likelihood of disease
• Finance  Fraudulent or Non Fraudulent
• Marketing  if users will click on an advertisement or not
Assumptions
• The dependent variable is binary or dichotomous.
• There should be no, or very little, multicollinearity between the
predictor variables—in other words, the predictor variables (or
the independent variables) should be independent of each other.
This means that there should not be a high correlation between
the independent variables.
• The independent variables should be linearly related to the log
odds.
• Logistic regression requires fairly large sample sizes
What are the log Odds?
• In very simplistic terms, log odds are an alternate way of
expressing probabilities.
• In order to understand log odds, it’s important to understand a
key difference between odds and probabilities:
• odds are the ratio of something happening to something not happening,
• probability is the ratio of something happening to everything that could
possibly happen.
Logistic Function
Logistic function:

We all know the equation of the best fit line in linear regression is:

Let’s say instead of y we are taking probabilities (P).


But there is an issue here, the value of (P) will exceed
1 or go below 0 and we know that range of Probability
is (0-1).
To overcome this issue we take “odds” of P:

Here, it will take values in the range: (0, ∞)


Logistic Function
• To control this we take the log of odds which has a range from
(-∞,+∞).
Cost Function
In linear regression, we used the Mean squared error.

• In logistic regression Yi is a non-linear function


(Ŷ=1/1+ e-z).
• If we use this in the above MSE equation then it
will give a non-convex graph with many local
minima as shown
Cost Function
Cost Function

• When the true output value y= 1 (positive)


penalty = -(1*log(p) + (1–1)*log(1-p)) = -log(p)
• When the true output value y= 0 (negative)
penalty = -(0*log(p) + (1–0)*log(1-p)) = -log(1-p)

You might also like