1

Stat 104: Quantitative Methods for Economists

Lecture 22: The Language of Hypothesis Testing
Introduction to Hypothesis Testing, Part I

Recall Proportion Confidence Intervals

n Called the Wald interval, assumes we have a

large sample and p is not close to 0 or 1.
Truth
p

Guess

95% Confidence Interval

1.96
p

(1 - p
)
p
n

https://www.predictit.org/Browse/Group/67/National

Stata Examples

n use
http://people.fas.harvard.edu/~mparzen/stat104/stat111_survey

5
ci proportion glasses
-- Binomial Exact -Variable |
Obs Proportion
Std. Err.
[95% Conf. Interval]
-------------+--------------------------------------------------------------glasses |
92
.6304348
.0503236
.5234493
.7287734
. ci proportion glasses if female==0

-- Binomial Exact -Variable |

Obs Proportion
Std. Err.
[95% Conf. Interval]
-------------+--------------------------------------------------------------glasses |
58
.5862069
.06467
.4492662
.7140367
. ci proportion glasses if female==1

-- Binomial Exact -Variable |

Obs Proportion
Std. Err.
[95% Conf. Interval]
-------------+--------------------------------------------------------------glasses |
34
.7058824
.0781425
.5252189
.8490163

Hypothesis Testing (intro)

We have discussed two methods of making inference

on parameters in a population based on a random
sample (in English, how do we figure out the true mean or proportion)

What is a Hypothesis?

q A hypothesis is a statement regarding a

characteristic of one or more populations.

Point estimates give us a single guess

Confidence intervals give us a region.

q In this section, we look at hypotheses

regarding a single population parameter.

Alternatively, we might be interested in using the information in a sample

to test hypothesis about parameters in the population.

Hypothesis Examples, Single Population

n In 2008, 62% of American adults regularly volunteered their time for

charity work. A researcher believes that this percentage is different
today.

n Using an old manufacturing process, the standard deviation of the

amount of wine put in a bottle was 0.23 ounces. With new
equipment, the quality control manager believes the standard
deviation has decreased.

n Allow us to use sample data to estimate a

population value, like the true mean or the true
proportion, and then put bounds on our estimate.
n Example: Give a 95% confidence interval for the
true average amount students spend weekly on
alcohol.

10

n We test these types of statements using sample data

because it is usually impossible or impractical to gain
access to the entire population.
n So theres always a chance of making a mistake L
n If population data are available, there is no need for
inferential statistics

n According to a study published in March, 2006 the mean length of a

phone call on a cellular telephone was 3.25 minutes. A researcher
believes that the mean length of a call has increased since then.

Recap : Confidence Intervals

Caution

11

Recap : Hypothesis Testing

n Allows us to use sample data to test a claim
about a population, such as testing whether a
population proportion or population mean equals
some number.
n Example: Is the true average amount that
students spent weekly on alcohol $20?

12

Parameter Identification

13

14

1. Make a statement regarding the nature of

the population.

n Hypothesis tests can be carried out on all the

population parameters (the Greeks), such as
the population mean, median, proportion or
variance.
n But in this section, we will conduct tests of
hypothesis only regarding the population mean m
or the population proportion p.

The Null Hypothesis

Hypothesis Testing Steps

2. Collect evidence (sample data) to test the

statement.
3. Analyze the data to assess the plausibility
of the statement.

15

nThe null hypothesis, denoted H0, is a

statement to be tested.
n The null hypothesis is a statement of no
change, no effect or no difference and is
assumed true until evidence indicates
otherwise.

The Alternative Hypothesis

16

nThe alternative hypothesis, denoted Ha,

is a statement that we are trying to find
evidence to support.
nSince we only have sample data, we can
really only disprove a theory, not prove it,
since we havent seen all the data.
H0: all cats have four legs

Setting up Ho and Ha
1.

2.

3.

Equal versus not equal hypothesis (two-tailed test)

H0: parameter = some value

Ha: parameter some value
Equal versus less than (left-tailed test)
H0: parameter = some value
Ha: parameter < some value
Equal versus greater than (right-tailed test)
H0: parameter = some value
Ha: parameter > some value

17

Status Quo or No Effect

n The null hypothesis is a statement of status
quo or no difference and always contains a
statement of equality.
n The null hypothesis is assumed to be true
until we have evidence to the contrary.
n We seek evidence that supports the
statement in the alternative hypothesis.

18

19

Forming Hypotheses
n Set up Ho and Ha for the following situation
n In 2008, 62% of American adults regularly
volunteered their time for charity work. A
researcher believes that this percentage is
different today.

n Set up Ho and Ha for the following situation

n Using an old manufacturing process, the
standard deviation of the amount of wine put
in a bottle was 0.23 ounces. With new
equipment, the quality control manager
believes the standard deviation has
decreased.

Recap-What can go wrong ?

23

Because the conclusion we will be making is

based on sample data, the possibility of making
an error always exists.
and we Claim that
Ho is true
Ho is true

IF

Correct Decision

Type I error

Ho is false
Type II error

Four Outcomes from Hyp Testing

1.

Reject the null hypothesis when the alternative hypothesis is true.

This decision would be correct.

2.

Do not reject the null hypothesis when the null hypothesis is true.
This decision would be correct.

3.

Reject the null hypothesis when the null hypothesis is true. This
decision would be incorrect. This type of error is called a Type I
error.

4.

Do not reject the null hypothesis when the alternative hypothesis

is true. This decision would be incorrect. This type of error is
called a Type II error.

Example: Type I and Type II Errors

q

In 2008, 62% of American adults regularly volunteered their time

for charity work. A researcher believes that this percentage is
different today.

A Type I error is made if the researcher concludes that

p 0.62 when the true proportion of Americans 18 years or older
who participated in some form of charity work is currently 62%.

A Type II error is made if the sample evidence leads the

researcher to believe that the current percentage of Americans 18
years or older who participated in some form of charity work is still
62% when, in fact, this percentage differs from 62%.

Ho is false

(no error)

Correct Decision
(no error)

20

n Set up Ho and Ha for the following situation

n According to a study published in March,
2006 the mean length of a phone call on a
cellular telephone was 3.25 minutes. A
researcher believes that the mean length of a
call has increased since then.

21

Forming Hypotheses

Forming Hypotheses

22

24

Example: Type I and Type II Errors

q

According to a study published in March, 2006 the mean length of

a phone call on a cellular telephone was 3.25 minutes. A
researcher believes that the mean length of a call has increased
since then.

A Type I error occurs if the sample evidence leads the researcher

to conclude that > 3.25 when, in fact, the actual mean call length
on a cellular phone is still 3.25 minutes.

A Type II error occurs if the researcher fails to reject the

hypothesis that the mean length of a phone call on a cellular
phone is 3.25 minutes when, in fact, it is longer than 3.25 minutes.

Which error is worse?

25

Notation for Type I and Type II Errors

26

= P(Type I Error)
= P(rejecting H0 when H0 is true)

= P(Type II Error)
= P(not rejecting H0 when H1 is true)

27

Example

28

Ho: Defendant is not guilty.

n The construction of the hypotheses are such that the

Type I error is considered the worse of the two.
n The probability of making a Type I error, , is chosen by
the researcher before the sample data is collected.
n The level of significance, , is the probability of making
a Type I error

Ha: Defendant is guilty.

What is the Type I Error ?

What is the Type II Error ?

Which error is more important ?

29

The milemaster tire company has decided that their

new tire must last more than 45,000 miles or they wont
market it.
Ho: tire lasts 45,000 (or less)
Ha: tire lasts more than 45,000.

What is the type I error ? The type II error ?

What is the cost of a type I error here ?

What is the cost of a type II error ?

The Significance Level

We define
a = Prob(Type I error) = P(reject H o | H o true)
We normally use a=.05 : called the significance
level
This is a pretty standard value to use, but not set in stone.
Usually the greater the cost of a type I error, the smaller
this number is.

30

Caution in Stating Your Conclusion

31

33

a)

Suppose the sample evidence indicates that the null hypothesis

should be rejected. State the wording of the conclusion.

b)

Suppose the sample evidence indicates that the null hypothesis

should not be rejected. State the wording of the conclusion.

Fail to Reject the Null

Suppose the sample evidence indicates that the null

hypothesis should be rejected. State the wording of
the conclusion.

Suppose the sample evidence indicates that the null

hypothesis should not be rejected. State the wording
of the conclusion.

The statement in the alternative hypothesis is that the

mean call length is greater than 3.25 minutes. Since
the null hypothesis ( = 3.25) is rejected, there is
sufficient evidence to conclude that the mean length of
a phone call on a cell phone is greater than 3.25
minutes.

Since the null hypothesis ( = 3.25) is not rejected,

there is insufficient evidence to conclude that the mean
length of a phone call on a cell phone is greater than
3.25 minutes. In other words, the sample evidence is
consistent with the mean call length equaling 3.25
minutes.

Actually Testing Hypotheses

n As you can see, there is a lot of terminology we had to
go through before we get to test any hypotheses
n There are 3 methods for actually doing the hypothesis
test:
n Confidence Interval Method (classical)
n Test statistic method (classical)
n P-values (using the computer)

32

According to a study published in March, 2006 the mean length of a

phone call on a cellular telephone was 3.25 minutes. A researcher
believes that the mean length of a call has increased since then.

n We never accept the null hypothesis, because, without

having access to the entire population, we dont know
the exact value of the parameter stated in the null.
n Rather, we say that we do not reject the null hypothesis.
This is just like the court system. We never declare a
defendant innocent, but rather say the defendant is not
guilty.

Reject the Null

Example: Stating Your Conclusion

35

Two Tailed Test Using a Confidence Interval

n The level of confidence, (1 - a) x 100%, in a confidence
interval represents the percentage of intervals that will
contain the unknown parameter if repeated samples are
obtained.
n When testing Ho:q=qo vs Ha:qqo, if a (1 - a) x 100%
confidence interval contains qo, we do not reject the null
hypothesis. If the confidence interval does not contain qo,
we conclude that qqo at the level of significance a.
n For now we are assuming n is large.

34

36

Testing a Two-Sided Hypothesis

37

q Step 1: Define Hypothesis

H :q = q
H :q q

q Step 2: Construct confidence interval

0

38

Example
A machine being used for packaging seedless golden raisins
has been set so that on the average 15 ounces of raisins will
be packaged per box. The quality control engineer wishes to
test the machine setting and selects a sample of 30 raisin
boxes. He finds =15.11 and s=0.4058
He wants to know if the mean weight per box is different from
15 ounces.

q Step 3: Accept or Reject

If qo falls within this interval, then we fail to
reject the null, otherwise we reject it.

H0 : m = 15

vs.

Ha : m 15

m0
38

39

Solution
. cii 30 15.11 .4058
Variable |
Obs
Mean
Std. Err.
[95% Conf. Interval]
-------------+--------------------------------------------------------------|
30
15.11
.0740886
14.95847
15.26153

n What can we conclude about

vs.

Ha : m 15

Whats the chance of making an error?

n How much confidence do we have in our
95% confidence interval?
n 95%!
n Or better stated (1-a)x100%
n We call a our level of significance; it is also
referred to as a Type I error

40

n Do we know the true value of m?

n However, we are pretty confident that the true value of m
is somewhere in the confidence interval!
n So when we ask if m=15, all we need to do is see if 15 is
in the confidence interval. If it is, its plausible that m=15.
But if 15 is not in the confidence interval, we would reject
the null hypothesis since then 15 is not a plausible value
for m.

n The 95% Confidence Interval is

H0 : m = 15

Logic

41

Example
n A manufacturer of sports equipment has developed a
new synthetic fishing line that the company claims has a
mean breaking strength of 8 kilograms.
n Test the hypothesis that =8 kilograms against the
alternative that 8 kilograms if a random sample of 50
lines is tested and found to have a mean breaking
strength of 7.8 kilograms with a standard deviation of 0.5
kilograms.

42

Fishing Line Example

43

n When a robot welder is in adjustment, its mean time to

perform its task is 1.3250 minutes.
n An incorrect mean operating time can disrupt the
efficiency of other activities along the production line.
n For a recent random sample of 80 jobs, the mean cycle
time for the welder was 1.3229 minutes with a std
deviation of 0.0396.
n Does the machine appear to be in need of adjustment?

n The 95% confidence interval is

. cii 50 7.8 .5
Variable |
Obs
Mean
Std. Err.
[95% Conf. Interval]
-------------+--------------------------------------------------------------|
50
7.8
.0707107
7.657902
7.942098

n Conclusion?

45

Example (cont)
n We want to test
H 0 : m = 1.325

vs.

44

Example

Example with Proportions

46

n The manufacturer of the Bic Extended Lighter

claims that it lights on the first time 75% of
the time. Test this claim.
n We want to test

H a : m 1.325

n The 95% CI is
.0396
1.325 1.96
= (1.316,1.334)
80

n Conclusion; fail to reject Ho-the robot is not in

need of adjustment.

Two sided Proportion Test

n Suppose we make 300 attempts and the
lighter lights on the first try 214 times.
n The resultant confidence interval is then
214
1.96
300

214
214
1
300
300
= (0.662, 0.764)
300

. cii 300 214,wald

-- Binomial Wald --Variable |
Obs
Mean
Std. Err.
[95% Conf. Interval]
-------------+--------------------------------------------------------------|
300
.7133333
.026108
.6621625
.7645041

47

Example with Proportions

n A new blog by politico.com says that 60% of
Americans trust the president. You want to
test this claim so you randomly survey 1000
adults and find that 637 trust the president.
What can you conclude?
n You want to test the claim
H o : p = 60%

H a : p 60%

48

Testing the claim

49

n We will be lazy and use Stata for the

calculations

n It is very easy and straightforward to test a

two sided hypothesis test using a confidence
interval.
n However, we would also like to be able to test
one-sided hypotheses.
n These can be done using one-sided
confidence intervals, or as we will see next
time, using what is called a test statistic.

. cii 1000 637,wald

-- Binomial Wald --Variable |

Obs
Mean
Std. Err.
[95% Conf. Interval]
-------------+--------------------------------------------------------------|
1000
.637
.0152063
.6071962
.6668038

n What can we conclude?

51

Things you should know

q Null and alternative hypothesis

q Type I and II error

q Using a confidence interval to test a large sample,
two sided hypothesis for the mean and
proportion.

One Sided Tests

50