Hypothesis testing

Junzhe Bao
Department of Biostatistics and Epidemiology

Email: [email protected]
1
 Outline

1 Introduction

2 Steps in hypothesis testing

3 Two types of errors and power of test

4 Test

2
 Introduction

• The interval estimation (confidence interval) introduced in the last lesson is a

type of statistical inference, and another common type is hypothesis testing.

• Suppose a school superintendent reads an article which states that the

overall mean score for the SAT is 910. Furthermore, suppose that, for a
sample of students, the average of the SAT scores in the superintendent’s
school district is 915. Can the superintendent conclude that the students in
his school district scored higher on average?

3
 Introduction

• At first glance, you might be inclined to say yes, since 915 is higher than 910.
But recall that the means of samples vary about the population mean when
samples are selected from a specific population.

• So the question arises, Is there a real difference in the means, or is the

difference simply due to chance (i.e., sampling error)? In this lesson, we will
learn how to answer that question by using statistics that explain hypothesis
testing.

4
 Introduction

Rationale for hypothesis testing: the Proof by Contradiction and the Principle
of Small Probability

Proof by Contradiction
The judge decides whether a person has committed a crime by first assuming
that he is "not guilty" (H0), and then searches for evidence through
reconnaissance, and if the evidence is sufficient then rejects the presumption
of "not guilty" (H0) and convicts the suspect; otherwise, the presumption of
"not guilty" (H0) can only be held to be valid.

Therefore, hypothesis testing always starts with the null hypothesis and then
looks for evidence to see if it can be rejected.

5
 Introduction

Principle of Small Probability

A small probability event is one that has such a small chance of occurring that
it can be considered virtually impossible for it to occur in a single trial (one
sample). It is up to the researcher to determine its probability level in advance,
and it is customary to consider p<0.05 or 0.01 as a small probability.

Rejection or non-rejection of the null hypothesis is based on the "principle that

small probability events are almost impossible to occur in a single trial".

6
 Introduction

• For example, a manufacturer claims that its products have a high pass rate
of 99%, and if 1 piece is randomly selected from a batch (assuming 100
pieces), the probability that this one piece happens to be an inferior
product is very small (1%).

• If the manufacturer's claims are true, it will almost never happen that a
randomly selected piece will be inferior.

• Assuming that it did happen, it would be reasonable to reject the original

hypothesis and a judgment could be made that the manufacturer's claim
was false.

7
 Steps in hypothesis testing

• Propose a hypothesis on a certain law of the population, and determine

whether to reject this hypothesis by inferring from sample data. Such a
statistical activity is called hypothesis testing.

• There are two types of statistical hypotheses: the null hypothesis and the
alternative hypothesis.

The null hypothesis, symbolized by H0, is a statistical hypothesis that states

that there is no difference between a parameter and a specific value, or that
there is no difference between two parameters (μ= μ0).

The alternative hypothesis, symbolized by H1, is a statistical hypothesis that

states the existence of a difference between a parameter and a specific value, or
states that there is a difference between two parameters (μ ≠ μ0 /  < 0 or   0 ).
8
 Steps in hypothesis testing
Distribution Plot
Normal, Mean=0, StDev=1

0.4

The level of significance

0.3
is a probability and, in
fact, is the probability of
Density

0.2

rejecting a true null

Rejection region hypothesis. Denoted by α
Rejection region0.1

𝛼/2 Nonrejection region 𝛼/2 (alpha). Generally, 0.05.

0.025 0.025
0.0
-1.960 0 1.960
X

μ0- μ

9
 Steps in hypothesis testing
Distribution Plot
Normal, Mean=0, StDev=1

0.4

The level of significance

0.3
is a probability and, in
fact, is the probability of
Density

0.2
Rejection region rejecting a true null
hypothesis. Denoted by α
0.1
𝛼
(alpha). Generally, 0.05.
0.05

0.0
0 1.645
X

10
 Steps in hypothesis testing

• Determine the appropriate test statistics

• For this example, suppose that the standard deviation of the overall SAT is 35, and the
sample size here is 100, then

ҧ μ
𝑥− 915−910
• Z=σ / = =1.429 p-value: 0.0764
𝑛 35/ 100

11
Steps in hypothesis testing

12
 Steps in hypothesis testing

The P value is the probability of the current statistic test and more extreme
conditions when H0 is true.

13
 Two types of errors and power of test
ҧ μ
𝑥−
Z= σ / 𝑛 Plot
Distribution
Normal, Mean=0, StDev=1

0.4

0.3
Density

0.2

Rejection region
Rejection region0.1

𝛼/2 Nonrejection region 𝛼/2

0.025 0.025
0.0
-1.960 0 1.960
X

μ0- μ

14
 Steps in hypothesis testing

Make statistical inferences:

If P≤α, we reject Ho and think that the two populations are different.

If P>α, we don`t reject Ho and think that two populations are not different.

15
Steps in hypothesis testing

16
 Two types of errors and power of test

Because the predictions in H0 and H1 are written so that they are mutually
exclusive and all inclusive, we have a situation where one is true and the
other is automatically false.
when H0 is true ,then H1 is false.
If we don’t reject H0, we have done the right thing.
If we reject H0 , we have made a mistake.

Type Ⅰ error is the error committed when a true null hypothesis

is rejected. The probability of type Ⅰ error is α.

17
 Two types of errors and power of test

when H0 is false, then H1 is true.

✓ If we don’t reject H0 , we have made a mistake.
✓ If we reject H0 , we have done the right thing.

Type Ⅱ error is the error committed when a false null hypothesis is not
rejected. The probability of type Ⅱ error is .

Power of test (1- ) is the probability of making the correct decision if

the alternative hypothesis is true.
18
 Two types of errors and power of test

The probability of rejecting the incorrect null hypothesis is called power of

test, and it equals to 1- . The higher the power of test, the more sensitive
the test is to detecting a real difference between parameters if there is a
difference.

19
 Two types of errors and power of test

Decision
Situation
Reject H0 Not reject H0
H0 is real Type I error (α) Correct (1-α)
H0 is false Correct (1-, power of test) Type II error (β)

20
Two types of errors and power of test

Tradeoff between α and 

21
 Two types of errors and power of test
Factors influencing power of test
ҧ μ
𝑥−
Z=σ / 𝑛
P< α, reject H0, detect difference.
|Z| P power of test
1. The greater the difference between the overall parameters,
the greater power of test

2. The smaller the individual difference (standard deviation),

the greater power of test

3. The larger the sample size, the greater power of test

4. The greater the level of significance (α), the greater power of test
22
 Test

Please describe the connection and difference between α and p in hypothesis testing.

23
 Test

The α value is a small probability value determined in advance by the

decision maker.

The P value is the probability of the current statistic test and more extreme
conditions when H0 is true.

If P≤α, we reject Ho and think that the two populations are different.

If P>α, we don`t reject Ho and think that two populations are not different.

24
 Test

The average value of hemoglobin in general adult men is 140g/L. A researcher

randomly selected 25 adult men in plateau areas for examination. The average
hemoglobin was 155g/L and the standard deviation was 25g/L.

Question: Is the hemoglobin of residents in plateau areas higher than that of

normal adult men?

25
 Test
1. Establish hypotheses test and determine the level of significance (α)
H0 : The hemoglobin of residents in plateau areas is equal to that of general adult men,
that is μ=μ0
H1 : The hemoglobin of residents in plateau areas is not equal to that of general adult men,
that is μ≠μ0
α =0.05
2. Selection of statistic methods and calculation of test statistics
Because the population standard deviation is unknown, a one-sample t test is used.

3. Determine the p value and provide the result

0.002<p<0.01 p< α 155>140
The average hemoglobin of residents in plateau areas is higher than the average hemoglobin
of general adult men.
26
 Test
Has practical meaning

May have practical meaning

The number of sample cases is too small

Meaningless
Not to reject H0
H0

Statistically significant Not statistically significant

27
Thanks！

28