The Hong Kong University of Science & Technology

MATH3423 - Statistical Inference

Take-home Examination in February

Answer ALL Questions Date: 20 February 2020

Full marks: 107 Time Allowed: 3 hours

Instructions

• I will release the exam paper through e-mail sharp at 8:30AM.

• You are required to upload your script (in pdf) via the link given in my e-mail or send your
file to me ([email protected]) at 11:30AM. Note that 10% of marks deduction per 15 min-
utes will be applied after 11:45AM until 12:15PM, i.e., 10% of marks deduction if your file
is uploaded at 11:46AM, 20% of marks deduction if your file is uploaded at 12:01PM and
ZERO mark if your file is submitted after 12:15PM.

• Write down your name and student ID on the first page. You may write down your student ID
on the top of every page for identification.

• Show all working steps of your answers for getting full marks.

• You must finish your work by yourself in HAND-WRITTEN form. NO TYPING WORK is
allowed. Neglecting this statement may lead to a ZERO mark.

• Write down the file name as yourname.pdf, e.g. CHENGHokLaam.pdf.

• Get your phone/scanning device and internet ready, so that you can upload the PDF promptly.
Please make sure your script is CLEARLY SCANNED and READABLE. Blurred/Blank/Problematic
script may not be marked.

• This exam will be open-book and open-notes. However, asking question in online social media
and finding/hiring someone to do the exam on your behalf are strictly prohibited.

• All academic works are central to HKUST Academic Honor Code. If students are caught
cheating, they will get an immediate F and sanctions will be imposed.

1
QUESTIONS

1. Suppose that (X1 , . . . , Xn ) are independently and identically distributed according to the uni-
form distribution U (0, θ] .

(a) (2 marks) Find the maximum likelihood estimator of θ.

(b) Define θ̂mle = δ. Consider the family of estimators δa = aδ, where a ∈ R.

i. (5 marks) Find a such that aδ is unbiased estimator of θ. Find its variance. Hence
or otherwise, show that it is a consistent estimator.

ii. (3 marks) Find a such that the mean squared error of aδ is minimized. Is it unbi-
ased? Explain.

(c) (4 marks) Find the sufficient statistic for θ and then prove that it is complete.

(d) (3 marks) Find the UMVUE of θn .

2. (a) Assume X1 ∼ N (1, 4), X2 ∼ N (3, 9) and Cov(X1 , X2 ) = 3.

( )
X1
i. (3 marks) Find the joint distribution of .
X2
( )
Y1
ii. (4 marks) Let Y1 = 2X1 + X2 and Y2 = X1 − X2 . Find the joint distribution of
Y2
and marginal distributions of Y1 and Y2 .

Hint: You may write down the distribution using the notation, e.g., N (µ, σ 2 ), χ2r ,
Gamma(α, λ).

(b) Let X1 , ..., Xn be a random sample from the following discrete distribution

2(1 − θ) θ
P (Xi = 1) = , P (Xi = 2) =
2−θ 2−θ
where θ ∈ (0, 1) is unknown.

i. (4 marks) Find a moment estimator θ̂ of the unknown parameter θ.

ii. (6 marks) Find the limiting distribution of the moment estimator θ̂ you obtained
above by Central Limit Theorem and Delta method.

2
3. Let X1 , . . . , Xn be i.i.d. r.v.’s from the Negative Binomial distribution with parameter θ ∈ Ω =
(0, 1), i.e. ( )
k+x−1
Pr(X = x) = θk (1 − θ)x x = 0, 1, 2, . . .
x

(a) (2 marks) Show that Negative Binomial distribution belongs to the exponential family.
Hence or otherwise, find the complete minimal sufficient statistic.

(b) (2 marks) Find the distribution of the complete minimal sufficient statistic in the previ-
ous part.

(c) (4 marks) Find the UMVU estimator of g(θ) = 1/θ and determine its variance.

(d) (4 marks) Find the Cramer Rao lower bound for unbiased estimators of θ. Is the variance
of the UMVUE for θ equal to Cramer Rao lower bound? Explain.

(e) (5 marks) Find the MLE and the UMVU estimator of θ.

(f) (5 marks) Find the UMVU estimator of ln(θ).

Hints
∑
∞
(1) (1 − a)−j = Cij+i−1 ai , where j is a positive integer.
i=0
∑
∞
(−1)ai
(2) ln(1 − a) = i
.
i=1

4. Suppose that X1 , ..., Xn are independently and identically distributed from normal distribution
N (µ, σ 2 ). Assume that both µ and σ 2 are unknown.

(a) (3 marks) Show that N (µ, σ 2 ) belongs to the exponential family. Hence or otherwise,
find a jointly complete minimal sufficient statistic for (µ, σ 2 ).
∑n
i=1 (Xi −X̄)
2
(b) (4 marks) Write down the distributions of X̄ and σ2
. Hence or otherwise, find
the UMVUE of µ and σ 2 .

(c) (3 marks) Find the UMVUE of µ2 .

(d) (5 marks) Find the UMVUE of 1/σ.

(e) (2 marks) Find the UMVUE of µ/σ.

3
5. (a) (5 marks) Let X1 , X2 , · · · , X10 be a random sample from the Bernoulli distribution with
density function f (x|θ) = θx (1 − θ)1−x for x = 0, 1, where 0 < θ < 1. To test H0 : θ = 0.5
∑
10
versus H1 : θ = 0.3, the critical region { Xi ≤ 3} is used. Find
i=1
i. the probability of committing type I error;
ii. the probability of committing type II error;
iii. the power of the test.

(b) Let X1 , ..., Xn1 and Y1 , ..., Yn2 be independent random samples from two normal popula-
tions N (µ1 , 1) and N (µ2 , 1) respectively. −∞ < µ1 < ∞, −∞ < µ2 < ∞.
i. (4 marks) Derive the likelihood ratio for testing H0 : µ1 = µ2 versus H1 : µ1 ̸= µ2 .

ii. (6 marks) Hence or otherwise, find the exact likelihood ratio test (LRT) for testing
H0 : µ1 = µ2 versus H1 : µ1 ̸= µ2 at significance level α.

iii. (2 marks) Given that the size of these two random samples are both 20. With the
summary statistic given below, conduct the LRT in (ii) at significance level 5%. Write
down your value of test statistic and your decision clearly.

∑
20 ∑
20 ∑
20 ∑
20
(Summary statistics: xi = 108.6631, x2i = 605.7094, yi = 74.4775, yi2 =
i=1 i=1 i=1 i=1
∑
20
299.8519, xi yi = 405.8938)
i=1

6. (a) Suppose that X1 , ..., Xn are independently and identically distributed from exponential(θ)
with density
f (x|θ) = θe−θx , x > 0
i. (6 marks) Find the UMP test for testing H0 : θ ≥ θ0 against H1 : θ < θ0 at α = 0.05.

ii. (2 marks) Determine the minimum sample size n required to obtain power at least
0.95 against the alternative θ0 = 25 when θ1 = 10 and α = 0.05.

(b) Let X1 , ..., Xn are independent Poisson random variable and Xi ∼ P oisson(λi ) where λi
are unknown.
i. (6 marks) Derive the approximate large sample likelihood ratio test at significance
level α for testing H0 : λ1 = λ2 = ... = λn vs H1 : otherwise.
∑
ii. (3 marks) By writing Xi = X̄ + Ui , where X̄ = Xi /n , and assuming that the
Ui ’s are
∑ small compared with X̄, show that the above test approximately compares
S = (Xi − X̄) /(n − 1) with X̄. Comment.
2 2

Hint: Take Log(1 + a) ≈ a if a is small.