Scribd
Upload
15 views

3 - 1 Logistic Regression

Uploaded by

ayushverma1802
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
15 views

3 - 1 Logistic Regression

Uploaded by

ayushverma1802
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 9

Sigmoid function

Linear vs. Logistics Regression

we use sigmoid
to map
predictions to
probabilities in
the range (0,1)

Logistic regression is a machine learning algorithm that can be used to classify input data into discrete output (e.g., input
emails into spam or non-spam and tumour images into benign or malignant)
Gradient Descent For Logistic
Regression
Outline:
• Have cost function , where
• Start off with some guesses for
• Repeat until convergence {
}

• After learning the parameters "𝞱 = [" 𝜃_0, …,𝜃_𝑑], we can predict
the output of any new unseen 𝒙=[𝒙𝟎,…,𝒙_𝒅] as follows:
Training Phase
Let us apply logistic regression on the spam email recognition problem, assuming
𝞪 = 0.5 and starting with = [0, 0, 0] 2 words (or features) = [, ]

vaccine nigeria y We define 3 parameters (the first one, i.e., ,


Email a 1 1 1 1 is the intercept)
Email b 1 0 0 0
Email c 1 1 0 1 1 entails that a word (i.e., “vaccine”)
Email d 1 0 0 0 is present in an email (i.e., “Email a”)
Email e 1 0 1 1
Email f 1 0 0 0

Testing Phase
Based on the training, check whether the following email (feature vector) is spam or
not spam? =1 =1
Training Phase

Testing Phase
Exercise -1
• Suppose you are given the following training examples

• Build a Logistic regression classifier. Assume 𝞪 = 0.1 and starting with = [0, 0, 0]
• Report all the parameters of the model after 1 epoch.
• After 1 epoch = [0.017, 0.07, 0.084]
• Classify the new example
• 0.5633 >0.5  so predicted class label = 1
Exercise-2
Classify whether weight = 55kg as obese (1) or not(0) using Logistic Regression
model with the training dataset given below assuming 𝞪 = 0.5 and starting with
= [0, 0,0]

Age Weight y
12 56 1
45 65 0 Age:[12,45,30,23,17,32,33,34,21]
Weight:[56,65,79,98,45,58,81,50,49]
30 79 1 Y=[1,0,1,1,0,0,1,0,0]
23 98 1
17 45 0
32 58 0
33 81 1
34 50 0
21 49 0
x1 x2
x0 (age) (weight) y (1/(1+e^(-θTx) )-y)x_0 (1/(1+e^(-θTx) )-y)x_1 (1/(1+e^(-θTx) )-y)x_2
1 12 56 1 -0.5 -6 -28
1 45 65 0 0.5 22.5 32.5
1 30 79 1 -0.5 -15 -39.5
1 23 98 1 -0.5 -11.5 -49
1 17 45 0 0.5 8.5 22.5
1 32 58 0 0.5 16 29
1 33 81 1 -0.5 -16.5 -40.5
1 34 50 0 0.5 17 25
1 21 49 0 0.5 10.5 24.5
sum -> 0.5 25.5 -23.5

=(-(1/9))*0.5*0.5 = -0.02777778 Question is incomplete as it gives us 55 as


=(-(1/9))*0.5*25.5 = -1.41667 weight but no value for age…
=(-(1/9))*0.5*-23.5 = 1.305556
After 1 epoch = [-0.02777778 , -1.41667 , 1.305556]
Logistic regression
• Advantages:
• Makes no assumptions about distributions of classes in feature space
• Easily extended to multiple classes (multinomial regression)
• Quick to train
• Very fast at classifying unknown records
• Good accuracy for many simple data sets
• Disadvantages:
• Linear decision boundary

You might also like