Scribd
Upload
50 views1 page

hw4 Theory Handout

Uploaded by

bfwang
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
50 views1 page

hw4 Theory Handout

Uploaded by

bfwang
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

CS 391L: Machine Learning Summer 2024

Homework 4 - Theory
Lecture: Prof. Qiang Liu

Note: Please typeset your answer (LATEX recommended) and upload it on edX.

1. Assume X is a discrete random variable that takes values in {1, 2, 3}, with probability defined
by

Pr(X = 1) = θ1
Pr(X = 2) = 2θ1
Pr(X = 3) = θ2 ,

where θ = [θ1 , θ2 ] is an unknown parameter to be estimated.


Now assume we observe a sequence D := {x(1) , x(2) , . . . , x(n) } that is independent and identi-
cally distributed (i.i.d.) from the distribution. We assume the number of observations of the
values: 1, 2, 3 in D are s1 , s2 , s3 , respectively.

(a) [5 points] To ensure that Pr(X = i) is a valid probability mass function, what constraint
should we put on θ = [θ1 , θ2 ]? Write your answers quantitatively as expressions that
include θ1 and θ2 .
(b) [5 points] Write down the joint probability of the data sequence
n o 
Pr (D | θ) = Pr x(1) , . . . , x(n) | θ ,

and the log probability log Pr(D | θ).


(c) [5 points] Calculate the maximum likelihood estimation θ̂ of θ based on the sequence
D.

2. [10 points] Let {x(1) , . . . , x(n) } be an i.i.d. sample from an exponential distribution, whose
the density function is defined as
 
1 x
f (x | β) = exp − , for 0 ≤ x < ∞.
β β
Please find the maximum likelihood estimator (MLE) of the parameter β. Show your work.
3. (a) [10 points] Assume that you want to investigate the proportion (θ) of defective items
manufactured at a production line. You take a random sample of 30 items and found 5
of them were defective. Assume the prior of θ is a uniform distribution on [0, 1]. Please
compute the posterior of θ. It is sufficient to write down the posterior density function
upto a normalization constant that does not depend on θ.
(b) [10 points] Assume an observation D := {x(1) , . . . , x(n) } is i.i.d. drawn from a Gaussian
distribution N (µ, 1), with an unknown mean µ and a variance of 1. Assume the prior
distribution of µ is N (0, 1). Please derive the posterior distribution p(µ | D) of µ given
data D.

You might also like