Scribd Logo
Upload

Welcome to Scribd!

  • 4 views

    hw4 Theory Handout

    Uploaded by

    bfwang

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    hw4 Theory Handout

    Uploaded by

    bfwang
    4 views1 page

    Original Title

    hw4_theory_handout

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    4 views1 page

    hw4 Theory Handout

    Uploaded by

    bfwang

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 1

    CS 391L: Machine Learning Summer 2024

    Homework 4 - Theory
    Lecture: Prof. Qiang Liu

    Note: Please typeset your answer (LATEX recommended) and upload it on edX.

    1. Assume X is a discrete random variable that takes values in {1, 2, 3}, with probability defined
    by

    Pr(X = 1) = θ1
    Pr(X = 2) = 2θ1
    Pr(X = 3) = θ2 ,

    where θ = [θ1 , θ2 ] is an unknown parameter to be estimated.


    Now assume we observe a sequence D := {x(1) , x(2) , . . . , x(n) } that is independent and identi-
    cally distributed (i.i.d.) from the distribution. We assume the number of observations of the
    values: 1, 2, 3 in D are s1 , s2 , s3 , respectively.

    (a) [5 points] To ensure that Pr(X = i) is a valid probability mass function, what constraint
    should we put on θ = [θ1 , θ2 ]? Write your answers quantitatively as expressions that
    include θ1 and θ2 .
    (b) [5 points] Write down the joint probability of the data sequence
    n o 
    Pr (D | θ) = Pr x(1) , . . . , x(n) | θ ,

    and the log probability log Pr(D | θ).


    (c) [5 points] Calculate the maximum likelihood estimation θ̂ of θ based on the sequence
    D.

    2. [10 points] Let {x(1) , . . . , x(n) } be an i.i.d. sample from an exponential distribution, whose
    the density function is defined as
     
    1 x
    f (x | β) = exp − , for 0 ≤ x < ∞.
    β β
    Please find the maximum likelihood estimator (MLE) of the parameter β. Show your work.
    3. (a) [10 points] Assume that you want to investigate the proportion (θ) of defective items
    manufactured at a production line. You take a random sample of 30 items and found 5
    of them were defective. Assume the prior of θ is a uniform distribution on [0, 1]. Please
    compute the posterior of θ. It is sufficient to write down the posterior density function
    upto a normalization constant that does not depend on θ.
    (b) [10 points] Assume an observation D := {x(1) , . . . , x(n) } is i.i.d. drawn from a Gaussian
    distribution N (µ, 1), with an unknown mean µ and a variance of 1. Assume the prior
    distribution of µ is N (0, 1). Please derive the posterior distribution p(µ | D) of µ given
    data D.

    You might also like