Martin Luther University Halle-Wittenberg Statistics II

Chair of Econometrics
Prof. Dr. Christoph Wunder

Exercise Set 4 - Solutions

Winter Semester 2023/24

1. Let θ be the population parameter of interest and θ̂ the sample statistic or point estimator
of θ. Discuss the properties of good point estimators: unbiasedness, efficiency, consistency,
and sufficiency.

UNBIASEDNESS
The notation θ is the Greek letter theta, and the notation is pronounced
“theta-hat”. In general, θ represents any population parameter such as a
population mean, population standard deviation, population proportion, and so on;
θ̂ represents the corresponding sample statistic such as the sample mean, sample
standard deviation, and sample proportion.

If the expected value of the sample statistic is equal to the population parameter
being estimated, the sample statistic is said to be an unbiased estimator of
the population parameter. The sample statistic θ̂ is an unbiased estimator of the
population parameter θ if


E θ̂ = θ


where E θ̂ is the expected value of the sample statistic θ̂. Hence, the expected
value, or mean, of all possible values of an unbiased sample statistic is equal to
the population parameter being estimated.

Figure 1
Unbiasedness; Statistics for Business and Economics, p. 296

1
 EFFICIENCY
Assume that a simple random sample of n elements can be used to provide two
unbiased point estimators of the same population parameter. In this situation,
we would prefer to use the point estimator with the smaller standard error,
because it tends to provide estimates closer to the population parameter. The
point estimator with the smaller standard error is said to have greater relative
efficiency than the other.
The figure below
shows the distribution

of two unbiased point estimators, θ̂1 and
θ̂2 . Here, Var θ̂1 < Var θ̂2 ; hence, values of θ̂1 have a greater chance of being
close to the parameter θ than do values of θ̂2 . Because the variance of point
estimator θ̂1 is less than the variance of point estimator θ̂2 , θ̂1 is relatively more
efficient than θ̂2 and is the preferred point estimator.

Figure 2
Efficiency; Statistics for Business and Economics, p. 296

CONSISTENCY
A point estimator (θ̂) is consistent if the values of the point estimator tend to
become closer to the population parameter (θ) as the sample size becomes larger.
Consistency implies that,


• lim Var θ̂ → 0
n→∞

SUFFICIENCY
An estimator is sufficient if it uses all the sample information (i. e. the
entire data set). In particular, the median, because it considers only rank, is
not sufficient. The sample mean considers each member of the sample as well as
its size, so is a sufficient statistic.

2. Consider a random sample (xi , ..., xn ) of size n > 2 drawn from a distribution with expected
value µ and standard deviation σ. Check whether the following estimators for µ are unbiased.

a) θ̂1 = n1 ∑ni=1 xi

2
 x1 +xn
b) θ̂2 = 2 .
c) θ̂3 = x2

To prove that an estimator is unbiased, we need to prove that the expected value
of the estimator is equal to the population parameter it is estimating. In this
question the estimators a) θ̂1 , b) θ̂2 and c) θ̂3 are estimators for the population
parameter µ.


a) • Unbiasedness: E θ̂1 = µ

! !

1 n 1 n
1 n 1 n 1
E θ̂1 = E ∑ xi = E ∑ xi = ∑ E(xi ) = ∑ µ = nµ = µ
n i=1 n i=1 n i=1 n i=1 n

• θ̂1 is an unbiased estimator of µ.

b) • Unbiasedness: E θ̂2 = µ

 

x1 + xn x1 xn 1 1 µ+µ
E θ̂2 =E = E( ) + E( ) = E(x1 ) + E(xn ) = =µ
2 2 2 2 2 2

• θ̂2 is an unbiased estimator of µ.

c) • Unbiasedness: E θ̂3 = µ

E θ̂3 = E (x2 ) = µ

• θ̂3 is an unbiased estimator of µ.

3. Suppose that x1 and x2 are random samples of observations from a population with a mean
µ and variance σ2 . Consider the following three point estimators, X,Y, Z of µ:
X = 12 x1 + 12 x2 ; Y = 14 x1 + 43 x2 ; Z = 31 x1 + 23 x2

a) Show that all three estimators are unbiased

b) Which of the estimators is the most efficient?

a) • X = 12 x1 + 21 x2
 
1 1 1 1 1
E(X) = E x1 + x2 = E(x1 ) + E(x2 ) = · 2µ = µ
2 2 2 2 2

E(X) = µ, therefore X is an unbiased estimator of µ

• Y = 14 x1 + 34 x2
 
1 3 1 3 1
E(Y ) = E x1 + x2 = E(x1 ) + E(x2 ) = · 4µ = µ
4 4 4 4 4

E(Y ) = µ, therefore X is an unbiased estimator of µ

3
 • Z = 31 x1 + 23 x2

 
1 2 1 2 1
E(Y ) = E x1 + x2 = E(x1 ) + E(x2 ) = · 3µ = µ
3 3 3 3 3

E(Z) = µ, therefore X is an unbiased estimator of µ

b) • X = 12 x1 + 21 x2

σ2
 
1 1 1 1 1
Var(X) = Var x1 + x2 = Var(x1 ) + Var(x2 ) = · 2σ2 =
2 2 4 4 4 2

σ2
Var(X) = 2

• Y = 14 x1 + 34 x2
 
1 3 1 9 10 5
Var(Y ) = Var x1 + x2 = Var(x1 ) + Var(x2 ) = σ2 = σ2
4 4 16 16 16 8

Var(Y ) = 58 σ2

• Z = 31 x1 + 23 x2
 
1 2 1 4 5
Var(Y ) = Var x1 + x2 = Var(x1 ) + Var(x2 ) = σ2
3 3 9 9 9

Var(Z) = 95 σ2

• Var(X) < Var(Z) < Var(Y )

• Therefore, X is the most efficient estimator

4. Let x1 , ..., xn be a random sample of i.i.d. random variables with expected value µ and vari-
ance σ2 . Now consider the following estimator of µ:

θ̂4 = axi + (1 − a)x̄n−1 , a ∈ (0, 1),

where xi indicates the ith observation of the random variable and x̄n−1 is the sample mean
based on n − 1 observations, i. e. excluding the ith observation. Calculate the expected value
of θ̂4 . Is it an unbiased estimator of µ

4
 Method 1


E θ̂4 = E (axi + (1 − a)x̄n−1 )

= aE (xi ) + (1 − a)E (x̄n−1 )

= aµ + (1 − a)µ

=µ

Hence θ̂4 is an unbiased estimator of µ.

Method 2


E θ̂4 = E (axi + (1 − a)x̄n−1 )

= aE (xi ) + (1 − a)E (x̄n−1 )

1 n−1
= aE (xi ) + (1 − a)E( ∑ xi )
n − 1 i=1

1 n−1
= aE (xi ) + (1 − a) ∑ E(xi )
n − 1 i=1

1
= aµ + (1 − a) (n − 1)µ
n−1

= aµ + (1 − a)µ

=µ

Hence θ̂4 is an unbiased estimator of µ.

5. Let x1 , x2 , ..., xn denote a random sample of size n from a uniform population with lower
bound a = 0 and upper bound b. Show that the estimator θ̂ = 2X̄ is an unbiased estimator of
b. Use E(Xi ) = b−a 2 as the expected value of the uniform random variable.

5


E θ̂ = E (2X̄)
 
1
= E 2 · ∑ Xi
n

2
= E (Xi )
n∑
 
2 b−a
=
n∑ 2
 
2 b
=
n∑ 2

2·n·b
=
2·n

=b

Hence, θ̂ is an unbiased estimator of b.

b−a
• As discussed in class, you can also directly substitute E(X̄) = 2

6. Consider a continuous random variable Xi , where E(Xi ) = µ and Var(Xi ) = σ2 > 0 for all i.
1
θ̂ = n−1 ∑ni=1 Xi is an estimator for µ. Is θ̂ an unbiased estimator of µ? Provide detailed
calculations to support your answer.

• θ̂ is an unbiased estimator of µ if E(θ̂) = µ

! !

1 n 1 n
1 n 1 n 1
E θ̂ = E ∑ Xi = E ∑ Xi = ∑ E(Xi ) = ∑ µ = n − 1 nµ
n − 1 i=1 n−1 i=1 n − 1 i=1 n − 1 i=1

nµ
E θ̂ = ̸ µ
=
n−1

• Therefore, θ̂ is a biased estimator of µ.

6
 PRACTICE QUESTIONS

1. Suppose that you have an estimator with a relatively large bias. However, the estimator is
consistent and efficient. If you had a generous budget for your sampling survey would you
use this estimator? Explain.

A consistent estimator mean that n → ∞ the probability of getting close to the

parameter increases. A generous budget affords a large sample size (large n),
making this probability high.

2. What are the advantages of a sufficient statistics? Can you think of a possible disadvantage
of sufficiency?

Advantage: Uses all information in data. Disadvantage: May be too sensitive to

the influence of outliers