Scribd
Upload
57 views7 pages

cs229 Livenotes Lecture3

1. Locally weighted regression is a nonparametric learning algorithm where the parameters grow linearly with the size of the data, unlike linear regression which uses a fixed set of parameters. 2. Logistic regression models the probability of the target variable using a logistic function of the linear combination of the features. It is used for binary classification problems by computing the maximum likelihood estimate of the parameters to minimize the log loss. 3. Newton's method can be used to optimize the logistic regression objective function more efficiently than gradient descent. It involves computing the derivative and Hessian matrix to estimate the second order derivative and find the minimum of the log loss function in fewer iterations due to its quadratic convergence rate.

Uploaded by

Alay camilo lopez
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
57 views7 pages

cs229 Livenotes Lecture3

1. Locally weighted regression is a nonparametric learning algorithm where the parameters grow linearly with the size of the data, unlike linear regression which uses a fixed set of parameters. 2. Logistic regression models the probability of the target variable using a logistic function of the linear combination of the features. It is used for binary classification problems by computing the maximum likelihood estimate of the parameters to minimize the log loss. 3. Newton's method can be used to optimize the logistic regression objective function more efficiently than gradient descent. It involves computing the derivative and Hessian matrix to estimate the second order derivative and find the minimum of the log loss function in fewer iterations due to its quadratic convergence rate.

Uploaded by

Alay camilo lopez
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Outline

Linear Regression recap


Locally Weighted Regression
Probabilistic interpretation
Logistic Regression
G Newton's Method
Recape features for
XM y Eh example XM ith training
example
N E IRDth C IR Xo L
y he prediction
n examples D features on EU
training
holx EX
ogx Xc
featuevedoegample
new example
T.co t.IE
Z Chow y T holx
g

sit
i
n

X Xz Ks

et
4

Locally Weighted Regression


Parametric learning algorithm
F it fixed set of parameters ou to date
Nonparametric learning algorithm
parameters grows linearly meh size of data

T.ie
j

To evaluate h at certain
Locally Weighted Regression
R Fcf 0 to Menominee fit 0 to minimise
Cgi 0 T an 5
EE wi Cg
Return Ota
wit expfllxitzf.LI
If XI small wi L
Ix H large wi r O

x
x x

An
Ti bandwidth
Probabilistic interpretation
why Least squares
Assume
ya OTW't E
A
error unmodded effects random
noise

E N O or

plea expc cE
Assumption E are ILD
ryaiydomd.ee mean
This implies that
P y In o Lor exec y zq3e
In art
parametrized by
y In io n N 0th or

P y In o

L O p Fl X o

II p y In o

FI oexrC CyHzotoI5
log likelihood
eco log Lco
log
7 exp
7
Eflag e log exp

n lag EE Gi x5
202
MLE Maximum Likelihood Estimation
Choose 0 to maximize LCO
Minimize
Ey OTHDZ TCO

classification
Make assumption P ylX O

Compute 0 by MLE

I
g EEO binary classification

o's

Logistic Regression
Want he E EO I

1
hoa gcoex
He 04
1
GG Ice
2
IEEE
at oa
sigmoid fn or logistic fr
P y L Ix O hocx
T A
sizeof
tumor tumor
malignant
I holx
Ply o IX o

P hfxY.CI h T
yl X o

Lfo pty IX O

pcy Ix o

hocxingty
II how

Ko log LCO

y log hock ta y toga h D

Batch Gradient Descent


0g i Oj Td eco Gradient Ascent
Descent
Og g azo Concave

tent

05 0 ta
E Cy hocxin x

Zo eco
J
Newton's Method
Have
Fundo f sa fCO 0

want maximize eco


v want e O 0
A
derivative

Maryann derivative 0

a slope flow

c o
09

ON OH O

f OG f O height
0 base
D f O
f COCO
Oath Oct f ft
f Oct
f o _e O
Oleh oh l'COct
l
Quadratic convergence
0 I 0.01 0.0001
Dxcd D
0 Vector 2 ftp.oh
Oath i get't H pole
in
vector HRD't
Hessian
ft Hy If
2920g

You might also like