6: THE POWER FUNCTION

The power function of a hypothesis test is the pro-

bability of rejecting H 0. This will be a function of

the true value of the parameter. For example, if the

parameter is the mean µ of a normal distribution,

then we write K 1(µ) for the power function, which

is the probability of rejecting H 0, given that the true

mean is µ.

We say that a statistical hypothesis is simple if it

completely specifies the distribution. An example of

a simple hypothesis is H 0 : µ = 5. Otherwise, the

hypothesis is said to be composite. For example,

H 0 : µ < 5 is composite. So far, we have always

 -2-
taken the null hypothesis to be simple, and we have

defined the significance level of the test to be the

probability of rejecting H 0 when H 0 is true. We

need to generalize this definition to allow for the

possibility of a composite null hypothesis. There-

fore, we now re-define the significance level of a

test to be the supremum of the power function of

the test, taken over all values of the parameter for

which H 0 is true. For a simple null, this definition

reduces to the one given earlier.

 -3-
g In general, the significance level provides a least

upper bound for the probability of a Type I error.

g If H 0 is true, the exact probability of a type I

error is given by the power function, evaluated at

the true parameter value.

g When H 1 is true, the power function is one minus

the probability of a Type II error.

g If evaluated at a parameter value θ for which H 1

is true, the power function gives the probability of

detecting that H 1 is true, given that the true param-

eter value is θ.
 -4-
Clearly, then, we would like the power function

of a level α test to be as close to 1 as possible as

the parameter ranges through the region describing

the alternative hypothesis, subject to the restriction

that the function not exceed α in the region describ-

ing the null hypothesis. For a given sample size,

however, we cannot expect high power for parame-

ter values "close" to the null.

Eg: Consider testing H 0 : µ = µ0 versus H 1 : µ > µ0

given a sample X 1 , . . . , Xn from a normal popula-

tion with unknown mean µ, and known variance,

σ2. If we use the test statistic Z = √ndd (Xd − µ0)/σ,

and reject H 0 if Z > z α, then the power function is

 -5-
K 1(µ) = Pr (Z > z α). Since the quantity

Z − √ndd (µ − µ0)/σ = √ndd (Xd − µ)/σ

has a standard normal distribution, we get

K 1(µ) = Pr (N (0 , 1) > z α − √ndd (µ − µ0)/σ)

=1 − Φ(z α − √ndd (µ − µ0)/σ) ,

where Φ(x ) = Pr (N (0 , 1) ≤ x ). If µ = µ0, we get

K 1(µ0) = 1 − Φ(z α) = α, as we must since we have a

simple null hypothesis, and the test has level α.

For any fixed value of µ which exceeds µ0, it is

seen that

lim K 1(µ) = 1 − Φ(−∞) = 1 ,

n →∞

so the power can be made arbitrarily close to 1 for

 -6-
any particular µ > µ0 by taking the sample size

sufficiently large. On the other hand, since

lim K 1(µ) = α, it follows that for a given sample

µ → µ0+

size, if µ is only slightly greater than µ0, the test

will have a low probability (roughly α) of detecting

that H 0 is false.

g We can use power functions to compare two tests

which have the same significance level.

Eg: In a test to see if a satellite is working, an

engineer sends a strong signal to the satellite for 10

seconds. If the satellite is working, it responds by

 -7-
sending back a signal which will register a 1.3mV

intensity on the engineer’s receiver, above and

beyond any noise received. If the satellite is not

working, it does not respond. The receiver repeat-

edly measures the voltage received from space at

equal intervals of one second. From a calibration

experiment which has already been performed, the

engineer knows that due to "noise" from space, and

also from the receiver, the voltage measured by the

receiver each second, in the absence of any signal

from the satellite, is normally distributed with mean

5mV and standard deviation 1mV. So the engineer

records the observations X 1 , . . . , X 10, where Xi is

 -8-
the voltage recorded by the receiver in the i ’th

second, minus the mean noise level of 5mV. It can

therefore be assumed that X 1 , . . . , X 10 are iid

N (µ,1), where µ = 1.3 if the satellite is working, and

µ = 0 if the satellite is not working. What is the best

way to test the hypothesis H 0 : µ = 0 versus

H 1 : µ = 1.3, at level α?

The answer, as we will show later, is the usual z -

test. It is easy to see that the test statistic for the z -

test is Z = √ndd (Xd − µ0)/σ, where σ = 1 and µ0 = 0,

and that we would reject H 0 in favor of H 1 if

Z > z α.
 -9-
But the z -test is not the only possibility. We could

also use a sign-test. The test statistic is simply Y ,

the number of observations Xi which are greater

than zero. Note that if H 0 is true, then each Xi will

be positive with probability .5, and Y will have a

binomial B (10 , .5) distribution. Using this informa-

tion, we could set up a level α sign-test.

g Which seems better for this problem, the z -test or

the sign-test? Why?

One way to answer this question is to examine the

power of the two tests. For the z -test, the power is

Pr (Reject H 0 e µ = 1.3) = Pr (√dnd (Xd − µ0)/σ > z α e µ = 1.3).

 - 10 -
Note that Z is distributed as N (√ndd (µ − µ0)/σ , 1),

ddd , 1) if the satellite is

which reduces to N (1.3 √10

working. Thus, the power of the z -test is

ddd
Pr (Z > z α) = Pr (Standard Normal > z α − 1.3√10)

ddd .
= 1 − Φ(z α − 1.3√10)

To facilitate comparisons with the sign test, we will

take α = 11/1024, which is .01 to two decimal

places. (We will henceforth ignore the difference).

Then the z -test has level .01, and power

ddd = 1 − Φ(−1.78) = Φ(1.78) = .9625.

1 − Φ(2.326 − 1.3√10)

g So if the satellite is working, the z -test has a 96%

chance of detecting this fact.

 - 11 -
What about the sign-test? Suppose we reject H 0

whenever Y ≥ 9. Then the level of the sign-test is

11/1024 ∼
∼ .01. Now, let’s compute its power. If H 1

is true, then Y has a binomial B (10 , p ) distribution,

where p = Pr (Xi > 0). Since Xi ∼ N (1.3 , 1) under

H 1, we get p = Pr (Std Norm > −1.3) = .9032.

Therefore, the power of the sign-test is

Pr (B (10 , .9032) ≥ 9) = 10 p 9(1−p ) + p 10 = .7485 .

So in this example, the z -test has much higher

power than a sign-test at the same level of

significance.