Scribd
Upload
4 views

Logistic regression

Uploaded by

trilokmailangam6
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
4 views

Logistic regression

Uploaded by

trilokmailangam6
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

LOGISTIC

REGRESSION
Logistic Regression
Logistic regression is a machine learning algorithm used for classification tasks. It predicts the
probability that a given input belongs to a particular category (or class). The output is a value
between 0 and 1, which represents the probability of the input belonging to the positive class.
Based on this probability, the model can classify the input into one of the classes.

For binary classification (two classes, like "yes" or "no"), logistic regression uses the logistic
function (also called the sigmoid function) to map the output of a linear equation to a value
between 0 and 1.
Sigmoid Function
σ(x):This is the output of the sigmoid function. It gives a value between 0 and 1, which can be interpreted as a
probability.

x: This is the input to the sigmoid function, often referred to as the linear combination of features in the
context of logistic regression or neural networks. Mathematically, in logistic regression, this is represented
as:
x=w1​x1​+w2​x2​+⋯+wn​xn​+b

e: This is Euler's number, approximately equal to 2.718.


updating weights
1. Initial Weights Assignment:
At the start of training, weights (w1,w2,…,wn) are initialized. This is usually done with small random values or
sometimes set to zero.

2. Forward Pass:
The model makes a prediction using the initial weights. This is done by passing the input data through the model:

y^​=σ(w1​x1​+w2​x2​+⋯+wn​xn​+b)
Where y^​is the predicted output (a probability between 0 and 1) based on the current weights.

3. Compute Loss:
The loss function (like log loss for logistic regression) calculates how far off the model's prediction y^​is from the
actual target value y.
L(y,y^​)=−(y⋅log(y^​)+(1−y)⋅log(1−y^​))
4. Compute Gradients:
Gradient Descent is used to compute the gradients of the loss function with respect to each weight w1,w2,…,wn.

Mathematically, for each weight wi​, the gradient is:


∂L\∂wi
5. Update the Weights:
The weights are then updated using the gradients. This is done using the Gradient Descent update rule:

wi​←wi​−η⋅∂L\∂wi​​
6. Repeat:
The process of forward pass, computing loss, computing gradients, and updating weights is repeated for many
iterations (or epochs).
After each iteration, the weights are updated and gradually converge to values that minimize the loss.
Advantages: Disadvantages:
1. Simplicity and 1. Linear Decision Boundary
Interpretability 2. Sensitive to Outliers
2. Works Well for Linearly 3. Requires Meaningful Input
Separable Data Features
3. Probabilistic Output 4. Limited to Binary
4. Less Prone to Overfitting 5. Assumes Independence of
5. Efficiency Features
6. No Need for Scaling 6. May Require Large Sample
Size
Problem Statement:
In a college campus, the principal is keen to improve the placement process for students. Every year,
many students participate in campus recruitment, but not everyone secures a job offer. The principal
wants to understand which students are more likely to get placed based on their academic
performance and cognitive abilities.
To tackle this challenge, the principal approaches you, a data analyst, and asks for your help in creating
a predictive model. Your task is to develop a logistic regression model that predicts whether a student
will be placed during campus recruitment based on two key factors: IQ and CGPA (Cumulative Grade
Point Average).
Jupyter notebook
Hyper parameters in Logistic Regression
Regularization Parameter (C): Controls the strength of regularization. Smaller values specify stronger regularization.

Penalty: Type of regularization used (l1, l2, elasticnet).

Maximum Iterations (max_iter): The maximum number of iterations for the solver to converge.

Multi-class Strategy: For multiclass classification, defines whether to use ovr (one-vs-rest) or multinomial.

Solver: Algorithm used for optimization (e.g., liblinear, saga, lbfgs).


1. Key take ways
1. Logistic regression is primarily used for binary and multiclass classification tasks.

2.The sigmoid function converts the linear output into a probability, mapping values between 0 and 1.

3.The loss function for logistic regression is binary cross-entropy (or log loss) for binary classification.

4.For multiclass classification, categorical cross-entropy is used.

5.Regularization (L1, L2) can be applied to prevent overfitting by penalizing large coefficients. The C
parameter controls the strength of regularization.

6.Logistic regression can handle multiclass problems using the one-vs-rest (OvR) or multinomial
(softmax) approach.

You might also like