Scribd
Upload
69 views3 pages

MIT18.650. Statistics For Applications Fall 2016. Problem Set 3

This document contains instructions for Problem Set 3 of the Statistics for Applications course at MIT for Fall 2016. It lists 4 problems to solve involving maximum likelihood estimation, consistency of the maximum likelihood estimator, Kullback-Leibler divergence, and total variation distance. Specifically, it asks students to find maximum likelihood estimators for different probability density functions, show consistency of the MLE for a normal distribution, compute Kullback-Leibler divergences between normal and Bernoulli distributions, and compute or show convergence of total variation distances between different distributions like uniform, Bernoulli, and Poisson. The problem set is due on Friday, September 30th at noon.

Uploaded by

yilvas
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
69 views3 pages

MIT18.650. Statistics For Applications Fall 2016. Problem Set 3

This document contains instructions for Problem Set 3 of the Statistics for Applications course at MIT for Fall 2016. It lists 4 problems to solve involving maximum likelihood estimation, consistency of the maximum likelihood estimator, Kullback-Leibler divergence, and total variation distance. Specifically, it asks students to find maximum likelihood estimators for different probability density functions, show consistency of the MLE for a normal distribution, compute Kullback-Leibler divergences between normal and Bernoulli distributions, and compute or show convergence of total variation distances between different distributions like uniform, Bernoulli, and Poisson. The problem set is due on Friday, September 30th at noon.

Uploaded by

yilvas
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

18.650.

Statistics for Applications


Fall 2016. Problem Set 3
Due Friday, Sep. 30 at 12 noon

Problem 1 Maximum likelihood estimator


Let X1 , . . . , Xn be n i.i.d. random variables with density fθ with respect to the
Lebesgue measure. For each case below find the MLE of θ.
1. fθ (x) = θτ θ x−(θ+1) 1(x ≥ τ ), θ > 0, where τ > 0 is a known constant.
2. fθ (x) = τ θτ x−(τ +1) 1(x ≥ θ), θ > 0, where τ > 0 is a known constant.
√ √
3. fθ (x) = θx θ−1 1(0 ≤ x ≤ 1), θ > 0
4. fθ (x) = (x/θ2 ) exp(−x2 /2θ2 )1(x ≥ 0), θ > 0
5. fθ (x) = θτ xτ −1 exp(−θxτ )1(x ≥ 0), θ > 0, where τ > 0 is a known constant.

Problem 2 Consistency of the maximum likelihood estimator


Let X1 , . . . , Xn be n i.i.d. random variables with distribution N (θ, θ), for some un­
known θ > 0. Compute the maximum likelihood estimator of θ and show that it is
consistent.

Problem 3 Kullback-Leibler divergence


1. Compute the Kullback-Leibler divergence between P = N (a, σ 2 ) and Q = N (b, σ 2 )
for a, b ∈ IR, σ 2 > 0.
2. Compute the Kullback-Leibler divergence between the Bernoulli distributions P =
Ber(a) and Q = Ber(b) for a, b ∈ (0, 1) .

Problem 4 Total variation distance


1. Compute the total variation between the uniform probability measures on the
intervals [0, s] and [0, t], for some given real numbers s, t, with 0 < s ≤ t.
2. If p and q are two numbers in [0, 1], compute the total variation distance between
Ber(p) and Ber(q).
3. If X1 , . . . , Xn are n i.i.d. Bernoulli random variables with some parameter p ∈ [0, 1]
and X ¯ n is their empirical average, show that the total variation distance between
Ber(X̄n ) and Ber(p) converges to zero in probability.

1
4. Show that the Poisson distribution with parameter 1/n converges in total variation
distance to the Dirac distribution at zero (i.e., the distribution of the random
variable that is always equal to zero).

2
MIT OpenCourseWare
https://ocw.mit.edu

18.650 / 18.6501 Statistics for Applications


Fall 2016

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.

You might also like