#Statistics/Week 7

INTRODUCTION TO HYPOTHESIS
TESTING

PSY201 LEC 0201: Statistics I

Dr. Patricia Y. Sanchez
Week 7
 OVERVIEW

• Intro to Hypothesis Testing

• Weird Logic of Hypothesis Testing
• Outcomes of Hypothesis Testing
• Variations on the hypothesis test

2
INTRODUCTION TO HYPOTHESIS TESTING

3
 A RESEARCH PROBLEM

• Research problem:
• Infant touch intervention
• Designed to increase child growth/weight

• Children’s weight at age 2

• Known population: μ = 26, σ = 4
• Sample data (n=16): M = 30

• Did the intervention increase weight?

4
 THE LOGIC OF HYPOTHESIS TESTING

• Using sample data to evaluate a _____________________________

parameter
• Usually in the context of a research study
• Was a “treatment” effective?
• Usually comparing _______________
• Compare M to known population μ
• Did this sample come from this population?
• Compare two or more sample Ms to each other
• Did these samples come from the same population?
5
 THE LOGIC OF HYPOTHESIS TESTING

• Let’s begin with the simplest case

• Compare observed M to known μ

• Cannot take difference at face value
• Is this difference “____________________ significant”?
• Differences between M and μ expected simply on the basis of
chance (sampling __________________)
• Sampling distributions allow us to judge the likelihood that
difference is due to chance
6
 THE LOGIC OF HYPOTHESIS TESTING

• Back to our research problem

• Known: μ = 26, σ = 4
• Sample: M = 30
• Did the touch intervention work?

• Is the difference between 26 and 30 statistically significant or

due to chance (sampling variability)?
• How likely is it that a sample mean of 30 would come from a
population with μ = 26?
• Sounds like a z-score and normal curve problem, doesn’t it? ;) 7
 THE LOGIC OF HYPOTHESIS TESTING

• Assume the intervention _____________________________

• Sample means should be close to (should cluster around) the
population mean

!
• Here is what we know: Standard error = !! =
"
• μ = 26, σ = 4
• M = 30, n = 16
!
• So, σM = =1
"#
8
HYPOTHESIZED DISTRIBUTION OF SAMPLE MEANS

σM = 1 How likely are we

to get a sample
mean out here?

22 23 24 25 26 27 28 29 30 M

μ = 26 9
 THE LOGIC OF HYPOTHESIS TESTING

• Compare sample mean to known population

• How much does M deviate from μ?
• Compute a z-score:

# %&
z =
!!

• What is the probability of this occurrence? 10

HYPOTHESIZED DISTRIBUTION OF SAMPLE MEANS

How likely are we

σM = 1 to get a z-score out
here?

-4 -3 -2 -1 0 +1 +2 +3 +4 z-score

μ = 26 11
 THE LOGIC OF HYPOTHESIS TESTING

• Difference is “statistically significant” when…

• M is ____________________ to have occurred by chance
• M falls in the ____________________ of the sampling
distribution

12
 THE LOGIC OF HYPOTHESIS TESTING

• Bottom line
• We assume no difference between means
• This is the ” ___________________________”
• If true, then M should be close to μ
• If M is NOT close to μ, then our assumption of no
difference (the null hypothesis) must be wrong
• Therefore, we conclude that there is likely to be a
difference worth paying attention to

13
 THE LOGIC OF HYPOTHESIS TESTING

• The logic of hypothesis testing

• We assume no differences between means
• The “null hypothesis”
• BUT… we hope to discredit this assumption
• We hope to ______________ the null hypothesis

• Inferential statistics provides rules for deciding when we

can (and cannot) reject the null hypothesis

14
 THE LOGIC OF HYPOTHESIS TESTING

• There are many statistical tests…

• z-test
• t-test
• F-test

• Which test you choose depends on type of data you have,

and research design
• All tests are based on the logic of probability and sampling
distributions
15
THE (WEIRD) LOGIC OF HYPOTHESIS TESTING

16
 TWO BRANCHES OF STATISTICS

Statistics

Descriptive Inferential
Characterizes attributes Generalize from a
of samples and/or sample to an population
populations
17
 HYPOTHESIS TESTING

• A specific statistical procedure (distinct from more general

research methodology)
• Researcher states a specific hypothesis about a population
• Hypothesis makes statement predicting _________________
characteristics of sample, based on hypothesis
• Obtain random sample from population; compare obtained
sample to prediction made from hypothesis
• Consistent? Hypothesis is reasonable
• Discrepant? Hypothesis is rejected
18
 WE NEVER PROVE

• It is statistically impossible to demonstrate a phenomenon is

absolutely true (remember, it’s all about probability)
• Researchers instead “find ______________” or “do not find
_______________”
• Supporting evidence may not signal a theory is always
true; disconfirming evidence does not signal that a theory
is not always true

• So, we seek to find evidence that is is unlikely that our

hypothesis is false
19
 WE NEVER PROVE

• Process: Logic of the Null Hypothesis

• We determine what the population (distribution) would

look like if the null hypothesis were true
• Then, we see if our sample data are likely to have come
from this distribution
• In other words, we look for the likelihood that our data
are consistent with the idea that there is no effect

20
 HYPOTHESIS TESTING STEPS

• Step 1: State the statistical hypothesis

• Step 2: Create a decision rule
• Step 3: Collect data and compute “observed” test statistic
• Step 4: Make a decision
• Step 5: Report your results

22
 STEP 1: STATE THE STATISTICAL HYPOTHESES

• Null Hypothesis (H0)

• Predicts no difference, or no effect Hypotheses are about
populations; use population
• H0: μ = population value notation
• H0: μcontrol = μcondition

• Alternative Hypothesis (H1 or Ha) • Touch study example

• Predicts difference, or an effect
• H1: μ ≠ population value • H0: μ = 26
• H1: μcontrol ≠ μcondition • H1: μ ≠ 26
23
 DELICIOUS LEARNING CHECK

• Variable of Interest: Brown Sugar

• Null hypothesis?
• Alternative hypothesis?
24
https://handletheheat.com/the-ultimate-guide-to-chocolate-chip-cookies/
 STEP 2: CREATE DECISION RULE

• How will we decide if our data are consistent with the

hypothesis or not?
• Decision rules revolve around H0
• Will you reject H0 or retain H0?
• When will you “reject” H0?
• When values of M are unlikely given H0

• How do we decide what is “unlikely”?

• Look in the tails for low probability values
25
IF THE NULL HYPOTHESIS IS TRUE…

26
 STEP 2: CREATE DECISION RULE

• How do we decide what is “unlikely”?

• _____________ level (!) (aka, significance level)

• Convention: ! = .05
• Can also be .01, .001; this is an informed
decision made by the researcher

27
 STEP 2: CREATE DECISION RULE

! = .05

Likely
Unlikely p = .95 Unlikely
p = .025 p = .025

-z μ +z 28
 STEP 2: CREATE DECISION RULE

• ___________________ value
• Boundary between likely/unlikely outcomes
• Critical value of z for two-tailed hypothesis, when ! = .05
• z ± 1.96
• Critical region
• Area beyond the critical value
• Decision rule
• Reject H0 when observed test-statistic equals or exceeds critical
value (falls within critical region)
• Otherwise, retain H0 29
 STEP 2: CREATE DECISION RULE

Retain H0
Reject H0 Reject H0

Critical value
z = -1.96 μ z = +1.96
Critical region 30
 STEP 3: COLLECT DATA, COMPUTE TEST STATISTIC

• Important: Data ALWAYS collected after hypotheses are specified

and decision criteria are established

! %$!"# %
z =
%$
where !! = '

• In other words, z is the difference between the observed sample

mean and the hypothesized population mean divided by the
standard error of the mean 32
 STEP 4: MAKE A DECISION

• Two possibilities

• If sample statistic (z) is located in the critical region, the null

hypothesis is ___________________
• If the sample statistic (z) is not located in the critical region, the
researcher ___________________ the null hypothesis

33
 STEP 4: MAKE A DECISION

• If you reject the null

• Conclude that any difference you see is probably NOT due to
chance
• Called “statistically significant”
• If you fail to reject the null
• Conclude that any difference you see is unlikely due to chance
• Nothing special is happening
• Called “non-significant”

• We do not ACCEPT or PROVE the null 34

 LET’S PRACTICE

• Does touch increase weight?

• Population: μ = 26, σ = 4

• Step 1: Statistical hypotheses

• H0: μ = 26
Retain H0
• H1: μ ≠ 26 Reject H0 Reject H0

• Step 2: Decision rule

z = -1.96 μ z = +1.96
• ! = .05, two tailed test
• Critical value of z ± 1.96
35
 LET’S PRACTICE

• Step 3: Collect sample data: n = 16, M = 30

• Compute z-statistic
First compute standard error:

% '
!! = !! = !! = 1
' )*
Then compute z-score:

! %$ *+ % ,-
z =
%$
z =
.
z=4
36
 LET’S PRACTICE

• Step 4: Make a decision

• Compare observed z (+4.0) to critical value (± 1.96)
• Does observed z exceed critical value?

Retain H0
Reject H0 Reject H0

z = -1.96 μ z = +1.96 Our z = +4 falls

ways out here 37
 LET’S PRACTICE

• Step 4: Make a decision

• Compare observed z (+4.0) to critical value (± 1.96)
• Does observed z exceed critical value?
• Yes!
• Reject H0

38
 LET’S PRACTICE

• Step 5: Report and summarize

• Touch appears to increase child weight
• Difference between intervention sample and the known population
mean are not likely to be due to chance
• There is a statistically significant difference between the mean
weight for the intervention sample and the known population mean

• Note: These are NOT examples of formal APA style of reporting, we

will introduce that later

39
OUTCOMES OF HYPOTHESIS TESTING

40
 OUTCOMES OF HYPOTHESIS TESTING

• Four possible outcomes

True Status of H0
No Effect Effect
H0 true H0 false

Reject H0 ERROR Correct

Decision
Retain H0 Correct ERROR
(also called
“fail to reject”) 41
 OUTCOMES OF HYPOTHESIS TESTING

• Four possible outcomes

True Status of H0
No Effect Effect
H0 true H0 false

Reject H0 Type I
Correct
ERROR
Decision
Type II
Retain H0 Correct
ERROR
(also called
“fail to reject”) 42
 OUTCOMES OF HYPOTHESIS TESTING

• Type 1 error
• Rejecting H0 when it’s actually true
• A “______________________”
• Example: Claiming brown sugar affects your cookie recipe when it
really does not
• Type II error
• Retaining H0 when it’s actually false
• A “_______________”
• Example: Claiming brown sugar has no effect on your cookie recipe
when it really does 43
 OUTCOMES OF HYPOTHESIS TESTING

• We never know the “truth”

• We try our best to minimize possibility of mistake

• Zombie example:

You are one of the few survivors of a zombie

apocalypse. You roam the earth in search of
canned goods and Twinkies.

44
 OUTCOMES OF HYPOTHESIS TESTING

• Zombie example:

In this zombie apocalypse, kissing disables the

zombies and brings them back to human form

Your task to survive: know when (and when not)

to kiss!

45
 OUTCOMES OF HYPOTHESIS TESTING

• Zombie example:

This is obviously a zombie.

It is OKAY to kiss it.

46
 OUTCOMES OF HYPOTHESIS TESTING

• Zombie example:

This is probably a zombie.

However, it could be a student

after taking their last midterm
exam…

47
OUTCOMES OF HYPOTHESIS TESTING

Consider the following scenario…

You are walking down the road at

dusk and see a human figure.

Is it a zombie?
Should you kiss it?
48
 OUTCOMES OF HYPOTHESIS TESTING

Two different correct decisions:

1. You could not kiss the figure in the distance and later find out that
it was NOT, in fact, a zombie. Good job.
2. You could kiss the figure in the distance and later find out it was,
in fact, a zombie. Good job.

49
 OUTCOMES OF HYPOTHESIS TESTING

BUT ALSO,
Two different types of error :

1. You could choose not to kiss and find out that the figure in the
distance was a zombie. (How you find out is up to your
imagination…)
2. You could choose to kiss, only to find out that the figure in the
distance was a human named Bob, who knows the location of a
nearby Twinkie warehouse, but is creeped out and offended at
your kiss so he leaves you and doesn’t take you to the Twinkies.
50
 ZOMBIE HYPOTHESIS TESTING

Decision It was a human It was a zombie

Kiss Type I error Good job!
(False Alarm! You (You disabled a
offended Bob…) zombie!)

Don’t kiss Good job! Type II error

(Bob leads you to (Oops! Zombie eats
Twinkies) your brains…)

51
 OUTCOMES OF HYPOTHESIS TESTING

Decision Null hypothesis is Null hypothesis is

true false

Reject the null

Type I error Correct decision
hypothesis

Fail to reject the

Correct decision Type II error
null hypothesis

52
 THE TRADEOFF

• You could wait until you get closer to make your decision
• Risk brains being eaten
• You could go ahead and kiss
• Risk offending human

• Evaluate: which outcome is worse?

53
 ASSUME H 0 IS TRUE

• Only one mistake is relevant: Type I error

• ! = level of significance
• Probability of making a Type I error
• 1 - ! = level of confidence
• Probability of making a correct decision when H0 is TRUE

• Example: If ! = .05, 1 - ! = .95

• So, mistakes will be rare when H0 is true!

54
 ASSUME H 0 IS TRUE

• How do we minimize Type I error?

• We control error by choosing !
• If ! = .05, we have a 5% chance of making a Type I error
• A small, but reasonable risk

• If consequences of Type I error are serious, choose a more

stringent (smaller) ! level
• If ! = .01, we have 1% chance of Type I error

55
 ASSUME H 1 IS TRUE

• Only one mistake is relevant: Type II error

• β = probability of Type II error
• 1 - β = “power”
• Probability of making a correct decision when H0 is FALSE

56
 ASSUME H 1 IS TRUE

• How do we minimize Type II error (and increase “power”)?

• Not as easy to control
• Based primarily on:
• Effect size
• Sample size
• Alpha level
• One vs. two-tailed test

57
 MINIMIZING ERROR

• Can’t minimize both at once!

• Type I and Type II errors are linked
• Decreasing Type I increases Type II
• Reduce false positives, but increase false negatives

• Type I error typically considered more serious by researchers

• Saying something IS when it actually isn’t can be a serious
problem…
• Saying something ISN’T when it actually is usually just means more
research will be done on that subject 58
 LEGAL ANALOGY

CRIMINAL STATISTICAL
INVESTIGATION INVESTIGATION

Presumption of Assume the Null

Innocence Hypothesis is Correct

Collect & Analyze Collect & Analyze

Evidence Data

Decide to Convict (or Decide to Reject the

not) Null (or not) 59
 LEGAL ANALOGY

Judge’s Decision Statistical Test

Innocence: The defendant did not Null hypothesis: No association between flu shot and
commit the crime psychotic manifestations
Guilt: The defendant did commit the Alternative hypothesis: There is an association between flu
crime shot and psychosis
Standard for rejecting Innocence: Beyond Standard for rejecting null hypothesis: Level of statistical
a reasonable doubt significance (!)
Correct judgment: Convict a criminal Correct inference: Conclude there is an association when one
does exist in the population
Correct judgment: Acquit an innocent Correct inference: Conclude that there is no association
person between flu shot and psychosis when one does not exist in
the population
Incorrect judgment: Convict an innocent Incorrect inference (Type 1 error): Conclude that there is an
person association when there is actually none
Incorrect judgment: Acquit a criminal Incorrect inference (Type II error): Conclude that there is no
association when there actually is one
60
 SUMMARY: OUTCOMES OF HYPOTHESIS TESTING

True Status of H0
No Effect Effect
H0 true H0 false
Type I Error Correct
! 1–β
Reject H0
Decision Probability of T1 error “power”
Correct Type II Error
Retain H0 1-! Β
(also called
“fail to reject”) Level of confidence Probability of T2 error
61
ASSUMPTIONS FOR HYPOTHESIS TESTS WITH Z-SCORES

• Random sampling
• Independent observation
• Value of σ is not changed by the treatment
• Normally distributed sampling distribution

62
VARIATIONS ON THE HYPOTHESIS TEST

63
 DIRECTIONAL VS. NONDIRECTIONAL HYPOTHESES

• Directional hypothesis (___________________)

• Predicts direction of difference
• H0: μ0 ≤ μ1
• H1: μ0 > μ1

• Nondirectional hypothesis (___________________)

• Predicts only difference, not direction
• Often used in newer, exploratory research
• H0: μ0 = μ1
• H1: μ0 ≠ μ1 64
 DELICIOUS LEARNING CHECK

• Variable of Interest: Brown Sugar

• Directional hypothesis?
• Nondirectional hypothesis?
65
https://handletheheat.com/the-ultimate-guide-to-chocolate-chip-cookies/
 COMPARING ONE-TAILED AND TWO-TAILED TESTS

• Two-tailed tests are more __________________ than one-tailed tests

• A one-tailed test allows rejecting H0 with relatively small
difference provided the difference is in the predicted direction
• A two-tailed test requires a relatively large difference regardless
of the direction of the difference

• In general, two-tailed tests should be used unless there is a very

strong a priori justification for a directional prediction

66
 MORE ON EFFECT SIZE

Statistical significance Practical significance

• p-value • Many metrics (including

• Depends on sample size Cohen’s d, R2, eta-
squared, etc.)
• Related to !
• Independent of sample size
• More next class!

• Size of effect
• Reliability of effect

67
 COHEN’S D: MEASURE OF EFFECT SIZE

$ "#$% '())#*#%+# 3!"#$!%#&! 43&' !"#$!%#&!

!"#$% & ( = ,-$%'$*' '#.($-(/%
= 5

Magnitude of d Evaluation of effect size

d = 0.2 Small effect

d = 0.5 Medium effect
d = 0.8 Large effect
68
WHAT IS A “LARGE” EFFECT?

69
REAL WORLD CONSEQUENCES: MEN ARE FROM MARS,
WOMEN ARE FROM VENUS?

70
REAL WORLD CONSEQUENCES: MEN ARE FROM MARS,
WOMEN ARE FROM VENUS?

71
 REAL WORLD CONSEQUENCES:
INTERPRETING EFFECT SIZE

• Effect sizes from meta-analyses of sex differences (Hyde, 2005)

• Women’s verbal skills exceed men’s (d = -0.11)
• Men’s math skills exceed women’s (d = 0.15)
• Men can throw harder than women (d = 2.18)
• Leadership effectiveness (d = 0.02)
Effect size Cohen’s d
• Self-esteem (d = 0.20)
• Aggression (d = 0.50) Small .20
Medium .50
Conclusion? ”Men are from North
Dakota, women are from South Dakota” Large .80
72
TO-DO: PRACTICE FROM MEMORY

73
 FOR NEXT TIME

T-Tests!

Chapter 8 problem set (see Quercus and MindTap

for due dates)

74
TO-DO: PRACTICE FROM MEMORY

73