PAS204: Lecture 16. The Neyman-Pearson Lemma: 16.1 Two Simple Hypotheses
PAS204: Lecture 16. The Neyman-Pearson Lemma: 16.1 Two Simple Hypotheses
The Neyman-Pearson
Lemma
In this lecture we show how to construct optimal tests comparing two simple
hypotheses.
H0 : = 0 ; H1 : = 1 :
In eect, we are supposing that can only have two possible values, 0
or 1 .
C ( 0) = P (X 2 C j = 0) ; C ( 1) = P (X 2 C j = 1) :
1
Remark 16.2 Notice the asymmetry between the two kinds of error. Mak-
ing the rst kind of error is thought to be more serious and so must be
controlled. This corresponds to the asymmetry in the hypotheses: the null
hypothesis is the one to stick with unless we are suitably convinced that the
alternative is more plausible.
Denition 16.1 (Likelihood ratio) The likelihood ratio (LR) for com-
paring two simple hypotheses is
L( 1 ; x) f (x j = 1)
(x) = = :
L( 0 ; x) f (x j = 0)
Ck = fx : (x) kg = fx : L( 1 ; x) k L( 0 ; x)g
for some k.
So the critical region includes all x for which is su ciently large.
2
Theorem 1 (Neyman-Pearson Lemma) Let Ck be the Likelihood Ra-
tio test of H0 : = 0 versus H1 : = 1 dened by
L( 1 ; x)
Ck = x: k ;
L( 0 ; x)
and with power function k ( ). Let C be any other test such that C ( 0)
k = k ( 0 ), where C ( ) is the power function of C. Then k ( 1)
C ( 1 ).
H0 : = 0 ; H1 : = 1 :
where
2 2 2 2
Q = (x 1) (x 0) = 2x( 0 1) +( 1 0) :
Therefore
Ck = fx : exp( 2n2 Q) kg
= fx : 2n2 Q log kg
2 2
= fx : 2x( 0 + ( 21
1)
2
0) n
log kg
= fx : x( 0 1) k g (16.1)
2
where k = n log k 12 ( 21 2
0 ). Now we are going to divide by ( 0 1 ),
but if this is negative we must change the direction of the inequality.
Therefore, if we now dene k = k =( 0 1 ), we have the following
form for the LR test Ck :
if 0 > 1, we reject H0 if x k ,
if 0 < 1, we reject H0 if x k .
3
Remark 16.6 You should always be very careful when dealing with in-
equalities!
Remark 16.7 Notice that the lines leading to (16.1) are all working to-
wards getting as simple as possible a function of the data on the left hand
side of the inequality and a constant on the right. The only way the data
appear in the LR is in the form of the sample mean x, so it is this that we
are trying to isolate on the left hand side, and the example ends by nishing
this task. It is important to recognise that on the right hand side we then
just have a constant, k .
k = P (X k j = 0) :
To derive this probability, we will use the fact that the distribution of X
is N ( ; 2 =n), and we will standardise so that we can work in terms of
standard normal probabilities.
Now
X k
k = P p 0 p 0j = 0 ;
= n = n
p
and we know that if = 0 then (X 0 )=( = n) has the standard normal
distribution. So if we let Z s N (0; 1) we have
k 0 k
k =P Z p =1 p 0 : (16.2)
= n = n
This equation links the test size to the critical value k for the test.
It immediately enables us to nd the P -value (observed signicance) for
4
the observed data. We simply equate the critical value k to the observed
vale x of X, and obtain
x x
P =1 p0 = 0
p ;
= n = n
using the general result that 1 (z) = ( z). Now suppose we wish to
choose k, or equivalently k , so as to obtain a test of specied size . This
means solving (16.2):
k
p 0 = Za :
= n
) k = 0 +p Z :
n
So we reject H0 if x 0 + n Z . Therefore the test of size
p formally has
critical region C = fx : x 0 + n Z g.
p
x 0
If 0 > 1 , we end up with P = p
= n
for observed signicance, and
for a xed size test we have k = 0 pn Z , and reject H0 if x 0
p Z .
n
[These are the one-sided tests for a normal mean that you will have met in
Level 1 statistics.]
There are several features to notice about this example.
Remark 16.8 First, note that we did not need to actually nd k. It was
enough to nd k . In Example 16.1 we found the general form of the LR
test. If 0 < 1 it is to reject H0 if x is su ciently large, and if 0 > 1
we reject H0 if x is su ciently small. In Example 16.2 we found just how
large or small x needed to be to get a test of a given size. We did not need
to have recorded what function k was of k, 2 , 0 and 1 , because we did
not use the information again.
5
general to know the relationship between k and k 0 . It is su cient to know
the form of the test, that it amounts to seeing if T is su ciently large or
su ciently small.
Then to identify the most powerful test of a given size , we need to
be able to derive the distribution of T (X) given = 0 , and hence to nd
k 0 such that = P (T (X) k 0 j = 0 ) or = P (T (X) k 0 j = 0 ), as
appropriate.
This gives us a general procedure for nding optimal tests, in two steps.
1. Find the likelihood ratio (x). Then manipulate the inequality (x)
k so as to express the test in terms of as simple a test statistic T (x)
as possible. In doing so, the objective is to nd the form of the LR
test in terms of T . If the LR is a monotone function of T , the form
of the test will be either to reject H0 if T is su ciently large or to
reject if T is su ciently small.
6
relationship between the gamma and chi squared distributions, so that
2 nX s Ga(n; 21 ) = 22n . We now use this result as follows.
= P (X k 0 j = 0 )
= P (2 0 nX 2 0 nk 0 j = 0)
= P (Y 2 0 nk 0 ) ; (16.3)
where Y has the 22n distribution. This gives us the observed signicance
P = P (Y 2 0 nx) for the observed value x of the test statistic X. How-
ever, this simple case illustrates why traditionally hypothesis testing was
done with certain conventional xed values of , because before the days
of computers it would have been impossible to calculate this probability
when required.
Consider, then, nding the critical value k 0 to obtain a test of size .
Denoting the upper 100p% point of the 2m distribution as in Lecture 2 by
2
m;p then we have
2
) k0 =
2n;1
2 0 nk 0 = 2
2n;1 :
2 0n
2
So the test of size rejects H0 if x 2n;1
2 0n
. We can get 22n;1 from
tables (e.g. Neave Table 3.2) for a range of values of , including the usual
5%, 1% etc.
With modern computer software, of course, we can readily nd the
P -value for any x, or the critical value k 0 for a test of any desired size.
7
A. OHagan
April 2008