STA255H1S: Statistical Theory

Week 12-1: Confidence Interval and hypothesis testing

Yaoming Zhen

Department of Statistical Sciences

University of Toronto

March 25, 2025

1 / 30
Roadmap

1. Confidence Interval

2. Concepts of Hypothesis testing

2 / 30
Roadmap

1. Confidence Interval

3 / 30
Normal Confidence Intervals for the Mean

To begin with, we assume that we have normally distributed

data: X1 , X2 , ..., Xn ∼ N(µ, σ 2 ). Our first objective is to estimate
the confidence interval around the mean µ. We will have two
different scenarios: when σ 2 is known, and when σ 2 is
unknown.

4 / 30
Critical Values

To develop confidence intervals when the variance is known,

we need values from the standard normal distribution to define
critical values.

5 / 30
Critical Values

The critical value zp of a N(0, 1) distribution is the number

that has right tail probability p. It is defined by P(Z ≥ zp ) =
p, where zp is the (1 − p) − th quantile of the standard nor-
mal distribution.

For example, P(Z ≥ 1.96) = 0.025 so the critical value is

z0.025 = 1.96.

By the symmetry of the standard normal density, P(Z ≤

−zp ) = P(Z ≥ zp ) = p, so P(Z ≥ −zp ) = 1 − p and there-
fore z1−p = −zp . For example z0.975 = −z0.025 = −1.96.

6 / 30
α-level

7 / 30
Confidence Intervals for µ: σ 2 known

Let X1 , X2 , ..., Xn ∼ N(µ, σ 2 ) where σ 2 is known and we wish

to construct a confidence interval for µ.
The estimator for the unknown parameter µ is X̄ and it has
distribution N(µ, σ 2 /n)
X̄ −µ
Use √
σ/ n
∼ N(0, 1) to construct the 1 − α confidence inter-
val
σ σ
[X̄ − √ z1−α/2 , X̄ + √ z1−α/2 ]
n n

8 / 30
Confidence Intervals for µ: σ 2 known

Example
Suppose we collect 100 data points from a N(µ, 32) distribution
and the sample mean is x̄ = 12. Give the 95 % confidence
interval for µ.

9 / 30
Confidence Intervals for µ: σ 2 unknown

When we do not know σ, we cannot use

X̄ − µ
√ .
σ/ n

Rather, we need an estimate for σ in order to construct the

confidence interval for µ. In this situation we use the estimate
S, and the fact that

X̄ − µ
√ ∼ T (n − 1)
S/ n

10 / 30
t-distribution
Let X ∼pN(0, 1), W ∼ χ2 (m). If X is independent with W ,
then X / W /m ∼ T (m).

11 / 30
Critical Values

We now need critical values for the t-distribution. Since it is

symmetric, we use the same logic as we did for z as the critical
value.

The critical value tm of a t-distribution is the number that

has right tail probability p. It is defined by P(T ≥ tm,p ) = p
where tm,p is the (1 − p) − th quantile of the t-distribution
with m degrees of freedom.

Because t is symmetric, tm,1−p = −tm,p

12 / 30
Confidence Intervals for µ: σ 2 unknown

Let X1 , X2 , ..., Xn ∼ N(µ, σ 2 ), where σ 2 is unknown and we

wish to construct a confidence interval for µ.
−µ
X̄ √
Use S n
to construct the 1 − α confidence interval

s s
[x̄ − √ tn−1,α/2 , x̄ + √ tn−1,α/2 ]
n n

13 / 30
Confidence Intervals for µ: σ 2 unknown

Example
Suppose the data 2.5, 5.5, 8.5, 11.5 was drawn from a N(µ, σ 2 )
distribution with µ and σ both unknown. Give interval estimates
for µ by finding the 95%, 80% and 50% confidence intervals.
(t3,0.025 = 3.1824, t3,0.05 = 2.3533, t3,0.1 = 1.6377,
t3,0.25 = 0.7649 )

14 / 30
Confidence Intervals for µ when σ 2 unknown: big n
The central limit theorem states that as n → ∞, things tend
to the normal distribution. In this case,

X̄ − µ d
√ → Z ∼ N(0, 1)
S/ n

That is, if n is large enough,

X̄ − µ
P(−zα/2 < √ < zα/2 ) ≈ 1 − α
S/ n

So, the interval

s s
[x̄ − √ zα/2 , x̄ + √ zα/2 ]
n n

approximates a 100(1-α)% confidence interval for µ.

15 / 30
Confidence Intervals for Non-normal distribution

Let X ∼ Bin(n, p). By the central limit theorem. For large n,

X − np
p
np(1 − p)

is approximately N(0,1)

Therefore,
X − np
P(−zα/2 < p < zα/2 ) ≈ 1 − α
np(1 − p)

16 / 30
Confidence Intervals for Binomial p
The confidence interval is characterized by

X − np
(p )2 < (zα/2 )2
np(1 − p)

17 / 30
Confidence Intervals for Binomial p

Example
Of a series of 100 i.i.d chemical experiments, 70 were
concluded successfully. Construct an approximating 90% CI for
the success probability of this type of experiment. (z0.05 = 1.64)

18 / 30
Sample Size

The width of an interval, which we can express as

w = 2 √σn zα/2 . If we use this width, we can express it in a way
such that we use it to find the smallest samples that satisfies
this width:
σ
2zα/2 √ ≤ w
n

and solve for n

2
2zα/2 σ

n≥
w

This tells us how many samples we need to satisfy a particular

confidence length.

19 / 30
Sample Size

Example
What sample size do we need if we want to be 99% confident
that the sample mean age of undergraduate U of T students is
within 2 years of the population mean age? Suppose we are
given that the standard deviation in ages is 4 years.
(z0.005 = 2.58)

20 / 30
One Sided Intervals

For one sided intervals, we have the following:

L(X) is a lower confidence bound when U(X)=∞

U(X) is an upper confidence bound when L(X)=−∞

21 / 30
One Sided CI: σ 2 unknown

If we are interested in the lower bound, for example, we would

be interested in saying that we are 95% confident that a mean
exceeds a certain value. In which case we would have:

X̄ − µ
P( √ < tn−1,α ) = 1 − α
S/ n

which gives us:

s
(x̄ − tn−1 √ , ∞)
n

Note we do not divide α by 2, and the upper limit of the interval

is ∞.

22 / 30
Roadmap

2. Hypothesis testing

23 / 30
Hypothesis Testing

We now turn to hypothesis testing whereby we form a state-

ment about a parameter θ and then perform a statistical test
to determine the correctness of the statement.

Hypothesis testing is similar to the scientific method: a sci-

entist formulates a theory and then tests this theory against
observation.

In statistics we pose a theory concerning one or more popu-

lation parameters (i.e. that they equal specified values), we
then sample the population and compare our observations
with our posed theory. If the observations disagree with the
theory then we reject it.

24 / 30
Null and Alternative Hypotheses
Elements of hypothesis testing:

H0 : the null hypothesis. This is the default assumption for

the model generating the data.

HA or H1 : the alternative hypothesis, which complements

the null hypothesis

Test statistic, computing from the data

Null distribution: the probability distribution of the test statis-

tics assuming H0 .

Rejection region (critical region): if X is in the rejection re-

gion we reject H0 in favor of HA .
25 / 30
Hypothesis Testing

A statistical hypothesis is a statement concerning an un-

known parameter for the population distribution f (x|θ), x ∈
R and θ ∈ Θ

The statistical hypothesis is a statement about θ and the

testing aims to prove its correctness.

One-sided hypothesis test takes on the form H0 : θ ≤ θ0

while H1 : θ > θ0 or H0 : θ ≥ θ0 while H1 : θ < θ0

Two-sided hypothesis test takes on the form H0 : |θ| ≤ θ0

while H1 : |θ| > θ0

26 / 30
Test Statistic

Test Statistic
For a dataset modeled on a realization of the random variable
X1 , ..., Xn , a test statistic is any sample statistic T = h(X1 , .., Xn )
whose numerical value is used to decide whether or not to
reject H0 .

The distributions that go with these statistics are always

conditioned on the null hypothesis. That is, we will compute
likelihoods such as f (z|H0 ).

27 / 30
Hypothesis Test Setup

Example
Suppose a mayoral candidate of Toronto claims she will get
more than 50% of the votes in an election, and thereby be the
winner. We do not believe this claim, so we would like to test
the candidate’s claim as a hypothesis test. Set up the elements
of a statistical test (hypothesis test):
null hypothesis
alternative hypothesis
test statistic
rejection/critical region

28 / 30
Type I and II Errors

Type I and Type II errors

Test of the null hypothesis H0 against an alternative hypoth-

esis H1 leads to either a correct decision or one of the fol-
lowing errors:

Type I error = rejecting H0 when it is true

Type II error = fail to reject H0 when it is false

Fail to reject H0 Reject H0

H0 is true correct decision type I error
H0 is false type II error correct decision

29 / 30
Significance

Significance Level:

α = P(reject H0 |H0 ) = P(type I error)

In testing a statistical hypothesis, a significance level α with

0 ≤ α ≤ 1 is the largest acceptable probability of commit-
ting a type I error.

Usually significance level is chosen to be α ≤ 0.05, but a

more conservative significance level is α ≤ 0.01.

30 / 30