Scribd
Upload
49 views11 pages

MLE MAP Regression PDF

The document discusses parameter estimation and maximum likelihood estimation for regression problems. It defines the mean squared error cost function and derives the normal equations to find the optimal weight vector that minimizes this cost function. It also introduces maximum a posteriori estimation, where prior distributions over the parameters are incorporated to address overfitting issues. The optimal parameters are estimated by maximizing the posterior distribution given the data.

Uploaded by

Alpha Wolf
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
49 views11 pages

MLE MAP Regression PDF

The document discusses parameter estimation and maximum likelihood estimation for regression problems. It defines the mean squared error cost function and derives the normal equations to find the optimal weight vector that minimizes this cost function. It also introduces maximum a posteriori estimation, where prior distributions over the parameters are incorporated to address overfitting issues. The optimal parameters are estimated by maximizing the posterior distribution given the data.

Uploaded by

Alpha Wolf
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

Parameter estimation for

regression

-
2 Regression : flashback

–
3

–
4
ML estimation for
the regression problem

–
5

–
6

1 N
E D ( w ) = ∑ (t n − yn ) 2
2 n =1
1
= (t − Φ w )T ( t − Φ w )
2
1
= (t − Φ w )T t − (t − Φ w )T Φ w
2
1 T
= (t t − w T Φ T t − t T Φ w + w T Φ T Φ w )
2

Using w T Φ T t = t T Φw
1 T
= (t t − 2 w T Φ T t + w T Φ T Φ w )
2
–
7

∂ED (w )
= Φ T t − Φ T Φw
∂w
At the optimal weight ∂ED (w ∗ )
vector , we have =0
∂w

⇒ Φ T t − Φ T Φw ∗ = 0

w ∗ = (Φ T Φ ) −1 Φ T t

–
8

–
9 Maximum a posterior estimation

p (D | θ). p (θ)
p (θ | D) =
∫ p(D | θ). p(θ)dθ
N
p ( D | θ) = ∏ p ( x k | θ)
k =1

θˆ = arg max θ
p(θ | D)

–
10
MAP estimation for
the regression problem

= + ln

–
11

You might also like