Logistic

Regression
Overview of
Logistic Regression
● Extends idea of linear regression to situation where outcome variable is categorical
● Widely used, particularly where a structured model is useful to explain (=profilling ) or to predict
● We focus on binary classification i.e Y=o or Y=1.
What is
Logistic Regression
Logistic regression is one of the most popular Machine Learning algorithms, which
comes under the Supervised Learning technique. It is used for predicting the categorical
dependent variable using a given set of independent variable

Logistic regression predicts the output of a categorical dependent variable. Therefore the outcome must be a categorical or discrete value. It can be either

Yes or No, 0 or 1, true or False, etc. but instead of giving the exact value as 0 and 1, it gives the probabilistic values which lie between 0 and 1.

In Logistic regression, instead of fitting a regression line, we fit an "S" shaped logistic function, which predicts two maximum values (0 or 1).
Assumption in
Logistic Regression
● The dependent variable must be categorical in nature.

● absence of multicollinearity
Sigmoid
Function
● The sigmoid function is a mathematical function used to map the predicted values to probabilities.

● It convert the categorical value between 0 and 1.

● The value of the logistic regression must be between 0 and 1, which cannot go beyond this limit, so it forms a curve like the "S" form. The S-form

curve is called the Sigmoid function or the logistic function.

● In logistic regression, we use the concept of the threshold value, which defines the probability of either 0 or 1. Such as values above the threshold

value tends to 1, and a value below the threshold values tends to 0.

Logistic
Function
● Goal : find a function of the predictor variable that relates them to a 0/1 outcome .

● Instead of Y as outcome variable ,we use a function of Y called the logit

● Logit can be modeled as a linear function of the predictors

● The logit can be mapped back to a probability , which in turn , can be mapped to a class .

Step 1 -Logistic response function: P= probability of belonging to class 1.

Need to relate p to predictors with a function that guarantees 0 ≤ p ≤ 1.

Standard linear function does not work .

We use logistic response function :

Step 2. The odds :

The odds of an event are defined as :

P= probability of event

Or, given the odds of an event, the probability of the event can be
computed by :

odds We can also relate the odds to the predictors :

P = 1+ odds
Step 3 : take log on both side

This give us logit :

log(Odds) = logit

Example : Personal loan offer

Outcome variable accept bank loan (0/1)

Predictors : Demographic info , and info about their bank relationship .

Type of logistic regression :

On the basis of categorical variable in target variable we classify it into three types :

● Binomial: In Binomial logistic regression there can be two possible categorical dependent variable

● Such as “Profit” or “Loss “

● .Multinomial: In multinomial Logistic regression, there can be three or more possible unordered types of the dependent variable,

such as “Fast”, "Faster", or "Faster "

● Ordinal: In ordinal Logistic regression there can be three or more possible ordered types of dependent variables, such as "First",

“Second", or "Third".
Performance metrics for classification on ML
models
Evaluation is always used to evaluate the model.There are many types to evaluate the metrics.

Confusion metrics is useful machine learning which allows to measure the recall,precision,f1-score.

Let’s start with the example of confusion matrix for a binary classification.we can perform more than two classification also.

Here we have two classes with many cases like True positive,True negative,false positive and false negative.
Performance metrics for classification on ML
models
True positive:There is the cases where we predict true and the actual result is also true,

True negative:There is the cases where we predict no and the actual is also no.

False positive:we predicted yes but actual is not true

False negative:we predicted no but actual is yes.

Performance metrics for classification on ML
models
Precision :Precision is not limited to binary classification problems . precision is calculated as the sum of true positives across all classes divided by the
sum of true positives and false positives across all classes.

Recall :recall is calculated as the number of true positives divided by the total number of true positives and false negatives.
Performance metrics for classification on ML
models
F score : The F-score, also called the F1-score, is a measure of a model’s accuracy on a dataset. It is used to evaluate binary

classification systems, which classify examples into ‘positive’ or ‘negative’.

It combine the way of combine recall and precision of the model .

Thank
You...