Scribd
Upload
13 views16 pages

Logistic Regression

Uploaded by

asinghal2122003
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
13 views16 pages

Logistic Regression

Uploaded by

asinghal2122003
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 16

Logistic

Regression
What is Logistic
• Regression?
Learning

• A supervised algorithm that learns to separate training samples into two


categories

• Logistic regression is a statistical model used to predict the probability of a binary


outcome based on one or more independent variables.

• It is a type of regression analysis where the dependent variable is categorical,


typically representing two classes (e.g., yes/no, true/false, 0/1).

• Prediction

• Uses the learned line, plane or hyper-plane to predict the whether an


input sample results in a target of 0 or 1 (yes/no, true/false)
Logistic Regression is used when the dependent variable(target) is
categorical.

For example, To predict whether an email is spam (1) or (0)


Whether the tumor is malignant (1) or not (0)
Categorical Response
Variables
Whether or not a person  Non 
Y 
Smoker
smokes smoker
Binary Response
Success of a medical Survives
Y 
treatment  Dies
Opinion poll Agree
responses Y 
Ordinal Response

Neutral
Disagree
The main goal of logistic regression is to find the relationship between
the independent variables and the probability of the binary outcome.

It estimates the probability using a logistic function, also known as the


sigmoid function, which maps any real-valued number to a value
between 0 and 1.
Unlike, linear regression, which is used to make a prediction on the
numeric response , Logistic Regression is used to solve a classification
problem.

For example, when a person applies a loan from a bank, bank is interested
in predicting whether the applicant will default in future.

Due to the nature of the problem, the prediction will fall in [0,1].
By the rule of thumb, if the prob. is greater than equal to 0.5, then we can
label the applicant as default.

Range of Linear regression is -∞ to +∞, however range of logistic regression


is 0 to 1.
Types of Logistic
Regression

1.Binary Logistic regression

2.Multinomial Logistic Regression

3.Ordinal Logistic Regression


Hypothesis Representation

When using linear regression we used a formula of the hypothesis i.e.


hΘ(x) = β₀ + β₁X

For logistic regression we are going to modify it a little bit i.e.


σ(Z) = σ(β₀ + β₁X)

We have expected that our hypothesis will give values


between 0 and 1.
Sigmoid
function

Logistic Regression
Logistic
Regression
The outcome probability is modeled
using the logistic function, also known as
the sigmoid function.
Here's the logistic function:

• And has a graph like this:

• By applying this function we


end up with predictions that
are between zero and one
In logistic regression, the goal is to find the best-fitting line (or
hyperplane in higher dimensions) that separates the two classes in
the feature space.

The logistic function is used to transform the output of a linear


combination of input features into a probability score between 0 and
1.

This probability represents the likelihood that the input belongs to one
of the two classes.

The formula for logistic regression can be represented as:


During training, the logistic regression model learns the optimal
values for the coefficients (β) that best fit the training data. This is
typically done using optimization algorithms like gradient descent,
which minimize a cost function such as the cross-entropy loss.

Once trained, the logistic regression model can predict the probability
of new instances belonging to one of the two classes, and based on a
chosen threshold (often 0.5), it can make binary predictions.
Decision Boundary

We expect our classifier to give us a set of outputs or classes based on


probability when we pass the inputs through a prediction function and
returns a probability score between 0 and 1.

For Example, We have 2 classes, let’s take them like cats and dogs (1
— dog
, 0 — cats).

We basically decide with a threshold value above which we classify values


into Class 1 and of the value goes below the threshold then we classify it in
Class 2.
As shown in the above graph we have chosen the threshold as 0.5, if the
prediction function returned a value of 0.7 then we would classify this
observation as Class 1 (DOG).

If our prediction returned a value of 0.2 then we would classify the observation
as Class 2(CAT).

You might also like