Chapter 9 Testing Hypothesis and Assesing Goodness of Fit

9.1-9.3 The Neyman-Pearson Paradigm and Neyman Pearson Lemma

We begin by reviewing some basic terms in hypothesis testing.

Let H0 denote the null hypothesis to be tested against the alternative hypothesis HA.

Definition: Let C be the subset of the sample space which leads to rejection of
the hypothesis under consideration. C is called the critical region or rejection
region of the test.

Based on our analysis of the data and resulting conclusion, there are two types of
errors that can be made:

• type I error is where the null hypothesis is rejected when it is, in fact, true. This
is usually denoted by α and called the significance level of the test.

• type II error is where the null hypothesis is accepted (or not rejected) when it
is, in fact, false.
The probability tha H0 is rejected when it is false is called the power of the test and
is denoted by 1 − β.

We will first be concerned with testing a simple null hypothesis H0 : θ = θ0 against

a simple alternative hypothesis HA : θ = θ00.

Definition: Let C denote a subset of the sample space. Then C is called a best
critical region of size α for testing H0 : θ = θ0 against HA : θ = θ00 if, for every
subset A of the sample space for which P [A : H0] = α, the following are true

(a) P [C; H0] = α

and

(b) P [C; H1] ≥ P [A; H1].

Example 1 Consider a random variable X that has a binomial distribution with
n = 5 and p = θ. We wish to test H0 : θ = 1/2 versus HA : θ = 3/4, at significance
level α = 1/32.
X f(x;1/2) f(x;3/4) f(x;12)/f(x;34)
0 1/32 1/1024 32
1 5/32 15/1024 32/3
2 10/32 90/1024 32/9
3 10/32 270/1024 32/27
4 5/32 405/1024 32/81
5 1/32 243/1024 32/243
To satisfy the significance level α = 1/32 we select the critical region as either
A1 = {x : x = 0} or A2 = {x : x = 5}. In fact, these are the only two possible ways
of selecting a critical region that has probability no greater than 1/32 under the null
hypothesis H0 : θ = 1/2. Thus, at least one of them must be a best critical region.

1 243
However, note that P [A1; HA] = 1024 < 1024 = P [A2; HA]

Thus, A2 is the unique best critical region.

Lemma (Neyman-Pearson): Let X1, X2, ..., Xn, where n is a fixed positive
integer, denote a random sample from a distribution with pdf f (x; θ). Then the
joint pdf of X1, X2, ..., Xn is

L(θ; x1, ..., xn) = f (x1; θ)f (x2; θ) · · · f (xn; θ).

Let θ0 and θ00 be distinct values of the parameter space Ω = {θ : θ = θ0, θ00}, and let
k be a positive number.

Define C as the subset of the sample space such that

L(θ0 ;x1 ,...,xn )

(a) L(θ00 ;x1 ,...,xn ) ≤ k, for (x1, ..., xn) ∈ C.

L(θ0 ;x1 ,...,xn )

(b) L(θ00 ;x1 ,...,xn ) > k, for (x1, ..., xn) ∈ C ∗.

(c) α = P [C; H0].

Then C is a best critical region of size α for testing the simple null hypothesis
H0 : θ = θ0 against the simple alternative hypothesis HA : θ = θ00.
Example 2: Let X1, X2, ..., Xn denote a random sample from a distribution with
pdf
2
 
1  (x − θ) 
f (x; θ) = √ exp − 
2π 2
−∞ < x < ∞. We wish to test H0 : θ = 0 versus H1 : θ = 1.
√ n " P
n x2
#

L(0; x1, ..., xn) (1/ 2π) exp − 2 i=1 i 

n n


= # = exp  − xi 
X
√
L(1; x1, ..., xn) (1/ 2π)n exp − ni=1(xi−1)2
" P
2 i=1
2

For k > 0 the set of points (x1, ..., xn) satisfying

n Xn
 

exp  − xi ≤ k
2 i=1

is a best critical region. Notice that this inequality is equivalent to

n Xn
− xi ≤ ln(k)
2 i=1
or
 n n
xi ≥ − ln(k) = c.
X

i=1 2

Given a particular significance level α, we can find c. Note that under H0, Xi has
P

a N (0, n) distribution. Under H0,

√ √
α = P[ Xi ≥ c] = P [Z ≥ c/ n] = 1 − P [Z ≤ c/ n]
X

where Z is a standard normal random variable.

Thus,
√
P [Z ≤ c/ n] = 1 − α
and
√
c= nΦ−1(1 − α)

where Φ denotes the cdf of a standard normal distribution.

Now, we will extend the notion of best critical regions (most powerful tests), to the
case where the alternative hypothesis is a composite hypothesis.

Example 3: Suppose we know that the distribution of a random variableX has

the form
1
f (x; θ) = e−x/θ
θ
for 0 < x < ∞, where the parameter space Ω = {θ : θ ≥ 2}. We wish to test

H0 : θ = 2 versus HA : θ > 2.

We can see that X has an exponential distribution or equivalently a Gamma distri-

bution with α = 1 and β = θ. Thus, E[X; θ] = θ.

Based on this, we would have evidence in favor of HA for large values of the sample
mean or X1 + X2.

We will take a random sample X1, X2 of size 2 and test H0 versus HA according to
the critical region C = {(x1, x2) : 9.5 ≤ x1 + x2 < ∞}.
Find the significance level of this test.

significance level (size)=P [C; θ = 2].

When H0 is true, the joint pdf of X1 and X2 is

1
f (x1, x2; θ = 2) = f (x1; 2)f (x2; 2) = e−(x1+x2)/2
4
and
P [C; θ = 2] = 1 − P [C ∗; θ = 2]

Z 9.5 Z 9.5−x
2 1 −(x1+x2)/2
=1− 0 0
e dx1dx2 ≈ 0.05.
4

In fact, using the previous theory, we can see that for any θ00 > 2, C is a best critical
region of size 0.05 for testing the simple hypothesis H0 : θ = θ0 = 2, versus the
simple alternative HA : θ = θ00.
 
L(2; x1, x2) (1/2) exp − x1+x
2
2
2
+ 
x1 +x2
 ≤k
L(θ00; x1, x2) 00 2
(1/θ ) exp − θ00
This implies

2 ln(1/2) − 2 ln(1/θ00) + (x1 + x2)(−1/2 + 1/θ00)

≤ ln(k)
Equivalently,
(x1 + x2) ≥

[ln(k) − 2 ln(1/2) + 2 ln(1/θ00)]/(−1/2 + 1/θ00) = c

Thus, we would reject H0 when x1 + x2 is greater than c. For a given significance

level, we solve for c to find the best critical region C.

For example if, α = 0.05, we see that c = 9.5 is the correct choice, no matter the
value of θ00 > 2.

Because this is true for all θ00, it is also the best critical region of size 0.05, when the
alternative hypothesis is a composite hypothesis HA : θ > 2.
Let K(θ) = P [C; θ] denote the power function.
Z 9.5 Z 9.5−x
2 1 −(x1+x2)/θ
K(θ) = 1 − e dx1dx2
0 0 θ2
θ + 9.5  −9.5/θ
 

= e
θ
 Figure 1: Plot of K(θ) versus θ

1.0
0.8
0.6
power

0.4
0.2
0.0

5 10 15 20 25

theta
Definition: The critical region C is a uniformly most powerful critical re-
gion of size α for testing a simple null hypothesis H0 against a composite alternative
hypothesis HA if C is a best critical region of size α for testing H0 against each simple
hypothesis in HA.

Definition: A test defined by a uniformly most powerful critical region is called a

uniformly most powerful test, with significance level α, for testing the simple
null hypothesis H0 against the composite alternative hypothesis HA.

Uniformly most powerful tests do not always exist. However, when they do, using the
Neyman-Pearson Lemma as in the previous example is a useful method for finding
them.