Case Study

NAME : M. Saubakeshwari
REGISTER NUMBER : 22STAU0345
CLASS : III – B.Sc. Statistics
COURSE TITLE : Theory of Estimation
COURSE CODE : 22USTEC2
TITLE OF THE STUDY : Rao-Blackwell Theorem
DATE : 14.10.2024

Course Outcome

CO1: Understand the significance of the Rao-Blackwell theorem in

the field of statistics.
CO2:Learn how to improve an estimator's efficiency using the Rao-
Blackwell process.
CO3:Apply the Rao-Blackwell theorem to create improved estimators
in simple scenarios.
CO4:Solve problems related to estimator efficiency and sufficiency
using theoretical and practical methods.
CO5:Develop an intuitive understanding of the connection between
sufficiency, completeness, and optimality in statistical estimators
 Rao-Blackwell Theorem

Introduction
In statistics, one of the main goals is to find estimators that provide the
most accurate estimates for unknown parameters in a population. An estimator's
quality is often measured by its variance, with lower variance being preferable.
The Rao-Blackwell theorem provides a method for improving estimators by
reducing their variance without introducing bias. Named after Calyampudi
Radhakrishna Rao and David Blackwell, the theorem highlights a structured
approach to improve estimators by using sufficient statistics.
The essence of the Rao-Blackwell theorem is that if we have an
unbiased estimator and a sufficient statistic, we can improve the estimator by
conditioning it on the sufficient statistic, resulting in an estimator with lower or
equal variance. This case study explores the theorem's concept, provides detailed
explanations, and applies it to solve example problems.
Explanation of the Rao-Blackwell Theorem
Theoretical Overview
The Rao-Blackwell theorem is a fundamental result in the field of statistics and
is associated with finding an improved estimator. The theorem states that if
T(X) is a sufficient statistic for a parameter θ and 𝜃̂(X) is an unbiased estimator
of θ, then the estimator
𝜃̂∗(X)=E[𝜃̂(X)∣T(X)]
(the conditional expectation of θ^(X ) given T(X)) is an improved estimator in
terms of lower variance, while still being unbiased.
Mathematically, the Rao-Blackwell theorem can be expressed as:
Var(𝜃̂∗(X)) ≤ Var(𝜃̂ (X))
where 𝜃̂∗(X) is the improved estimator and 𝜃̂(X) is the original unbiased
estimator. The improved estimator is guaranteed to have the same or lower
variance than the original.
Sufficient Statistic: A statistic T(X) is a sufficient for a parameter 𝜃 if it
contains all the information about 𝜃 that is available in the sample data X.
Unbiased Estimator: An estimator 𝜃̂ (X) is said to be unbiased for a parameter
θ if E[𝜃̂ (X)] = θ, meaning the expected value of the estimator equals the true
parameter value.
Example Problems
Example 1: Improving an Estimator for a Bernoulli Distribution
Suppose X1,X2,… Xn are independent and identically distributed random
variables from a Bernoulli distribution with parameter p. The goal is to estimate
p using an unbiased estimator.
Let 𝑃̂1 = 𝑋1 be a naive estimator. Use the Rao-Blackwell theorem to improve
this estimator.
Solution:
Step 1: Identify sufficient statistic.
The sum T(X) = ∑𝑛𝑖=1 𝑋i is a sufficient statistic for p by the factorization
theorem.
Step 2: Apply the Rao-Blackwell theorem.
To improve 𝑃̂1 = 𝑋1, we calculate the conditional expectation of 𝑃̂1 given the
sufficient statistic T(X).

𝑇(𝑋)
The improved estimator 𝑃̂∗ (X) = is simply the sample mean, which is the
𝑛
well-known best estimator for p with lower variance than the naive estimator 𝑃̂1 .
Example 2: Improving an Estimator for an Exponential Distribution
Let X1,X2,...,Xn be independent and identically distributed from an exponential
distribution with rate parameter λ. The goal is to estimate λ. Consider the naive
estimator λ^1=X1. Use the Rao-Blackwell theorem to find an improved
estimator.
Solution:
Step 1: Identify sufficient statistic.
The sum T(X) = ∑𝑛𝑖=1 𝑋i is a sufficient statistic for 𝜆.
Step 2: Apply the Rao-Blackwell theorem.
The conditional expectation of X1 given T(X) is calculated as:

𝑇(𝑋)
Therefore. the improved estimator 𝜆̂∗ (X) = is simply the sample mean,
𝑛
which is the well-known best estimator for p with lower variance than the naive
estimator 𝜆̂1 .
Conclusion
The Rao-Blackwell theorem is a powerful tool in statistics, allowing us
to improve estimators by reducing their variance while maintaining their unbiased
nature. Through the process of conditioning on a sufficient statistic, we can
generate improved estimators that perform better than the original naive
estimators. The examples discussed highlight the practical application of the Rao-
Blackwell theorem in real-world problems, showing how simple estimators can
be enhanced using statistical principles.
The significance of the Rao-Blackwell theorem lies in its broad
applicability, and its role in guiding statisticians toward more efficient and
effective estimators. Understanding and applying this theorem equips students
with a deeper appreciation for the relationship between sufficiency, unbiasedness,
and efficiency in estimation.
References:
[1] Casella, G., & Berger, R. L. (2001). Statistical Inference (2nd ed.). Duxbury
Press.
[2] Lehmann, E. L., & Casella, G. (1998). Theory of Point Estimation (2nd ed.).
Springer.
[3] Rao, C. R. (2002). Linear Statistical Inference and Its Applications (2nd ed.).
Wiley.
[4] Hogg, R. V., McKean, J., & Craig, A. T. (2018). Introduction to
Mathematical Statistics (8th ed.). Pearson.
[5] Mood, A. M., Graybill, F. A., & Boes, D. C. (1974). Introduction to the
Theory of Statistics (3rd ed.). McGraw-Hill.