Distributions of Sampling Statistics

Mahbub Latif, PhD

February 2025
2
Plan
Sampling distribution of sample mean
Central limit theorem
Sampling distribution of sample variance

3
Introduction

The science of statistics deals with drawing conclusions from observed data,
which is often a sample from a population of interest
To use sample data to make inferences about an entire population, it is
necessary to make some assumptions between the two
There is an underlying probability distribution
The sample data are independent values drawn from this population

4
Introduction

If X , … , X are independent random variables having a common

1 n

distribution F , i.e.
X1 , … , Xn is a random sample from a distribution with distribution
function F
Two types of methods
F is specified up to some unknown parameters (parametric inference)
Nothing is known about F except the type of the associated variable
(nonparametric inference)

5
Example 6.1a

Suppose that a new process has just been installed to produce computer
chips, and the successive chips produced by this new process will have
lifetimes that are independent with a common unknown distribution F
Physical reasons sometimes suggest the parametric form of the distribution
F (e.g. F is a normal distribution, etc., i.e. parametric inference)

For normal distribution, only μ and σ need to be estimated

2

In other situations, there might not be any physical justification for

supposing that F has any particular form (nonparametric inference)

6
The Sample Mean

7
The Sample Mean

Let X , … , X be a random sample from a population with mean μ and

1 n

variance σ
2

For any i, E(X ) = μ and

i V ar(Xi ) = σ
2

The sample mean is defined as

n
1 1
¯
X = (X1 + ⋯ + Xn ) = ∑ Xi
n n
i=1

Sample mean X̄ is a random variable because it is a function of random

variables
8
Properties of X̄

The expected value

X1 + ⋯ + Xn
¯
E[X] = E[ ] = μ
n

μ → population mean

9
Properties of X̄
The variance
2
X1 + ⋯ + Xn σ
¯
V ar[X] = V ar[ ] =
n n

σ
2
→ population variance
n → sample size

10
X1 , … , Xn is a random sample from N (0, 1)
¯ 1
X ∼ N (0, )
n

11
Suppose X 1, … , Xn is a random sample from N (10, 5)
¯
X ∼ N (10, 1) when n = 5

What would be the distribution of X

¯
when the population is not normal? 12
Central Limit Theorem

13
Central Limit Theorem

Let X , … , X is a random sample from a distribution with mean μ and

1 n

variance σ , for a large n

2

2
Y = (X1 + ⋯ + Xn ) ∼ N (nμ, nσ )

n
1
¯ 2
Y = ∑ Xi ∼ N (μ, σ /n)
n
i=1

¯ − μ
Y
⇒ Z = ∼ N (0, 1)
σ/√n

14
Sampling distribution of a sample mean

15
Sampling distribution of a sample mean

16
Normal distribution

17
Summary of central limit theorem

Let X , … , X be a random sample from a population with mean μ and

1 n

variance σ , and the corresponding sample mean is

2

n
1
¯
X = ∑ Xi
n
i=1

If the population is normal then for any n

¯ 2
X ∼ N (μ, σ /n)

If the population is non-normal then only for a large n

2
X̄ ∼ N (μ, σ /n)
18
Application of central limit theorem to binomial distribution

Let X , … , X be a random sample from a Bernoulli distribution with

1 n

parameter p
Define X = X1 + ⋯ + Xn and X ∼ B(n, p)

Using central limit theorem, for a large n

X ∼ N (np, np(1 − p))

E(X) = np and V ar(X) = np(1 − p)

19
Example 6.3c

The ideal size of a first-year class at a particular college is 150 students.

From the past experience college knows that, on the average, only 30
percent of those accepted for admission will actually attend
The college uses a policy of approving the applications of 450 students.
Compute the probability that more than 150 first-year students attend this
college.

20
Example 6.3c

X denotes the number of students that attend and X ∼ B(450, .3)

Using binomial formula

450
450
i 450−i
P (X > 150) = ∑ ( )(.3) (1 − .3)
i
i=151

Using normal approximation

150.5 − (450)(.3)
P (X > 150) = P (X > 150.5) = 1 − Φ( )
(450)(.3)(1 − .3)

= 1 − Φ(1.59) = 1 − 0.9441
21
Example 6.3d

The weights of a population of workers have mean 167 and standard

deviation 27.0
If a sample of 36 workers is chosen, approximate the probability that the
sample mean of their weights lies between 163 and 170.
Repeat the above question when the sample is of size 144.

22
Sample variance

Let X , … , X is a random sample from a distribution with mean μ and

1 n

variance σ2

Sample variance
n
1
2 ¯ 2
S = ∑(Xi − X)
n − 1
i=1

Sample standard deviation (SD): S = √S

2

It can be shown that E(S 2

) = σ
2

23
Sampling distribution of sample variance

Let X , … , X is a random sample from a distribution with mean μ and

1 n

variance σ 2

2
(n − 1)S
¯ 2 2
X ∼ N (μ, σ /n) and ∼ χ
2 n−1
σ

χ
2
n−1
→ Chi-square distribution with (n − 1) degrees of freedom

24
Chi-square distribution

Let Z 1, … , Zn be n independent standard normal random variables

The distribution of X = ∑ follows a chi-square distribution with n
n 2
Z
i=1 i

degrees of freedom, i.e.

2 2 2
X = Z + ⋯ + Zn ∼ χ n
1

25
Chi-square distribution

26
27
The t-distribution

Let X , … , X be a random sample from a population with mean μ and

1 n

variance σ2

For a large n, X
¯ 2
∼ N (μ, σ /n)

¯
X − μ
Z = ∼ N (0, 1)
σ/√n

28
The t-distribution

If σ is unknown, it is replaced by sample standard deviation s in Z statistic

The resulting statistic follows a t-distribution with (n − 1) degrees of
freedom
¯
X − μ
t = ∼ tn−1
s/√n

29
Comparison between t and standard normal distributions

30
31
Problems

1, 2, 5, 8, 9, 10, 11, 12, 13, 14, 18

32
Problem 1

Plot the probability mass function of the sample mean of X 1, … , Xn when

(i) n = 2 and (ii) n = 3.
Assume
P (X = 0) = .2, P (X = 1) = 0.3, P (X = 3) = 0.5

Calculate E(X) = μ and V (X) = σ 2

In both cases, determine E(X)

¯
and V (X)
¯

33
Problem 1 (for n = 2 )
Probability distribution of
X1 X2 X̄ P (X̄ = x)
sample mean
0 0 0.0 0.04 x̄ P (X̄ = x̄)

0 1 0.5 0.06 0.0 0.04

0 3 1.5 0.10 0.5 0.12
1 0 0.5 0.06 1.0 0.09
1 1 1.0 0.09 1.5 0.20
1 3 2.0 0.15 2.0 0.30
3 0 1.5 0.10 3.0 0.25
3 1 2.0 0.15
3 3 3.0 0.25

Obtain E(X)
¯
and V ar(X)
¯

34