HYPOTHESISTESTING

HypothesisTesting Intro to Statistics

Hypothesis testing is
the starting point for
learning inferential
statistics. We want to
know something about
the world and we use
hypothesis testing to
evaluate statements
about how the world
works.

An Introduction to Hypothesis Testing

inferential statistics
Null Hypothesis Statistical Testing (NHST; a.k.a. will be some random variation, but not too much.
significance testing and Null Hypothesis Decision Inferential statistics are parametric procedures used to
Making) is a statistical method that uses sample data to decide whether a given sample mean differs statistically
evaluate assumptions (i.e. test a hypothesis) about a significantly from another mean. We test the hypothesis
population parameter. Asked more simply, “What can that the sample mean is really the same as the
we infer about the population based upon what we population mean. Then we use probability to
have learned from our sample?” determine whether the natural differences that we
Probability tells us what characteristics we can expect observe are more likely due to random variation or due
from a sample drawn from a given population. What isto an effect from the treatment. Generally, there are
true of the population (mean, SD) should also be true five steps to doing hypothesis testing.
of samples drawn from that population. Sure, there

Probability*
What*are*the*likely*characteris/cs*of*a*
sample*drawn*from*this*popula/on?*

μ"="100"
Typical(polar(bears( Sample"1"
Inferen/al*Sta/s/cs*
What*can*we*infer*about*the*popula/on*
based*on*what*we*know*from*this*sample?*

Todd Daniel PhD April 2021 1

 Hypothesis Testing
HypothesisTesting Intro to Statistics

A Five-Step Guide
Choose the Right Statistical test
Questions to Ask Before You Begin Use the diagrams for Step 1 to choose the right statistical
test. Choose your test based on (a) whether you are looking
What are your Variables? Step 1: for relationships or differences between groups, (b) the level
of the data (NOIR), and (c) whether the assumptions for the
• Independent variable (IV) test have been met.
Exploratory data analysis will help you know what test to choose
• Dependent variable (DV)

What do you want to know? Establish the Null and Alternative Hypothesis
The null hypothesis states that there is really no difference
• Relationships between groups? between means, so any apparent differences are simply due

• Differences between groups? Step 2: to random variation or chance. The alternative hypothesis
states that one group mean will be statistically significantly
different than the other.
The null hypothesis is written as (H0:) and the alternative is (H1:)
What do your data look like?
• What level are the DV data: NOIR?
Select a Criteria for Significance
• Have the assumptions been met?
After choosing a one-tailed vs. two-tailed test, you have three
• How many IV groups do you have? options for determining statistical significance:
1. Is the p-value of your test less than the level of
Step 3: significance (a.k.a. alpha level). The most common level
is ! = .05. (The level of significance sets your critical value.)
What type of test will you use?
2. Is the test statistic greater than a critical value (CV)?
• Directional or non-directional H1? 3. Does the confidence interval around the mean difference
include 0? All three options will agree with each other.
• Is your alpha level .05 or .01?
For most tests, assume α = .05 and a two-tailed test

Calculate the Statistical Test

When working by hand, use the test formula to calculate a
test statistic. This will be compared to the critical value from a
Step 4: table. When using software, the software will calculate a test
statistic, a probability (p) value, and a confidence interval.
In SPSS, the p value is in a box labeled “Sig.”

Interpret and Write Up the Findings

Each of the following indicate statistical significance:
1. The p-value is less than ! (i.e., p < .05)
Squirrels love this Statistical Hypothesis 2. The calculated statistic is in the critical region (region of
rejection); the test value exceeds the CV
Inference Testing
Step 5: 3. The confidence interval around the mean difference
includes 0 (i.e. one is negative and the other positive)
When each are true, you reject the null hypothesis. The null
says there is no difference between means; if you reject the
idea that there is no difference, you are inferring that there
there is a difference.
Write up your results in proper APA style

Todd Daniel PhD 2

 Looking for Differences Between Groups
HypothesisTesting Intro to Statistics

Step 1: Choosing the Right Statistical Test

What level is the
dependent variable (DV)?
DV data are at the Ordinal level
DV data are at the Nominal level DV data are at the Scale level but are not
DV data are at the Scale level
normally distributed and/or have outliers and normally distributed

2 2 3+ 2 3+
1 Groups 1 Groups 1 Groups
Groups Groups
Group Group Group

One-Way Chi Square test

Tests whether observed proportions in categorical
variable differ from a hypothesized value.

Two-Way Chi Square test

Are the data a “good fit” to what was expected? Single Samples
IV is nominal; DV is frequency
One-Sample t Test
Tests whether the mean (DV) of one sample (IV)
differs from a hypothesized mean
NP One-Sample t Test
Sign Test
Non-parametric (NP) Statistics

DV is simply positive vs. negative

2 Independent Samples
Independent Samples t Test

Parametric Statistics
NP Independent Samples t Test Compares the means (DV) of only two independent
Wilcoxin-Mann-Whitney Test groups (IV)
Compares ranks/median of two independent samples
Preferred when data have extreme outliers and/or have
sample sizes less than 20
2 Dependent Samples
Paired Samples t Test
NP Paired Samples t Test Compares the means (DV) of only two related
samples (IV); i.e. the sample people are measured twice
Wilcoxin Signed Rank Sum Test or subjects have been deliberately matched
Compares two related samples, DV is ranks with median
test
Related = repeated, paired, and/or dependent

3+ Independent Samples
NP One-Way ANOVA
One-Way ANOVA
Kruskal-Wallis Test Compares the means (DV) of a single IV with 3+
Compares a single IV with three independent levels/groups; DV independent levels/groups; analyzes variability
is ranks with median test
Generalized form of a Mann-Whitney test

NP Repeated Measures ANOVA 3+ Dependent Samples

Friedman Test Repeated Measures ANOVA
Compares a single IV with three dependent Compares the means (DV) of a single IV with 3+
levels/groups; DV is ranks with median test dependent levels/groups; analyzes variability
H0: distribution of the ranks of each score are the same

Todd Daniel PhD 3

HypothesisTesting Intro to Statistics

The Null Hypothesis

Step 2: Establish a Null and Alternative Hypothesis
Alpha Levels and Directional Tests
Hypotheses predict an outcome that we may or may not find in our experiment. The simplest strategy is
to say that there is a relationship between the IV and the DV, but we do not know which direction the DV
data will change when we manipulate the IV (two-tailed test). A more complicated strategy not only
predicts the existence of a relationship but also predicts the direction the DV data will change when we
manipulate the IV (one-tailed test).

Example Research Alternative

Type of Test Null Hypothesis Curve
Question Hypothesis

Will this drug affect

Two-Tailed H0: ! = 100 H0: ! ≠ 100
stamina?

One-Tail Right Will this drug increase

H0: ! ≤ 100 H0: ! > 100
(Positive) concentration?

One-Tail Left Will this drug decrease H0: ! ≥100

H0: ! < 100
(Negative) blood pressure?

How to Create a Null and Alternative Hypothesis

1. Is the research question one-tailed (directional) or two-tailed (nondirectional)?
2. What is the mean of the population? This number will come from the research question
Questions to 3. What direction does the dependent variable change? Should it increase or decrease?
If the DV increases, you will use > in the alternative hypothesis.
ask yourself…
If the DV decreases, you will use < in the alternative hypothesis.
If the DV simply changes, you will use ≠ in the alternative hypothesis.

Create the null hypothesis first: H0: ! = 100 * > greater than
Then create the alternative: H1: " ≠ 100 < less than
Example of a two tailed (nondirectional) hypothesis ≥ greater than or equal to
One tailed test The height of the average gnome is 28 inches tall. ≤ less than or equal to
H0: " = 28 ______________________
H1: " ≠ 28
3<5 & 7>1
“the alligator always eats more.”
Create the alternative hypothesis first: H1: ! > 100 * The equal sign always goes
Then create the null hypothesis what is left: H0: " ≤ 100 with the Null Hyp.
Example of possible null hypotheses: # = 100, # ≤ 100, or # ≥ 100
Remember that the null hypothesis will always include the equal sign.
Examples of one tailed (directional) hypotheses
Two tailed test The average grizzly bear eats more than 12 trout per day (one-tail - right)
H0: " ≤ 12
H1: " > 12
Most people wearing diapers are less than 30 months old. (one-tail - left)
H0: " ≥ 30
H1: " < 30

The number 100 Number is used as a place holder for the mean of the population. Do not use the number in your hypothesis
unless 50 happens to be the mean of the population. And don’t include the asterisk, either.

Todd Daniel PhD 4

HypothesisTesting Intro to Statistics

The Critical Value

Step 3: Level of Significance
Alpha Levels and Critical Values
You will compare two scores. Your goal is to set a mathematically-defined limit, “fence,” or “cut-off”
beyond which any differences between the two scores are so extreme that you must conclude that
they are truly different. The probability of an event occurring at random (by chance) is represented by
alpha: α or a
Alpha level: a level of significance, cut-off point, or “fence” that sets the critical region
(most commonly " = .05)
Critical Region: Area of rejection of the null hypothesis created by the alpha-level “fence”. In the picture
below, it is the purple shaded area of the normal curve. The critical region comprises extreme values
of a statistic that are unlikely to occur if the Null Hypothesis is true. When the calculated sample value
falls in the critical region (tail), you reject the null. This means that the difference did not occur by
chance.

Calculating the Critical Region (Inclusion and Exclusion)

Setting an alpha level at α = .05 means that 5% of the scores are in the “tail” (critical region) and the
remaining 95% are in the “body” of the curve. If we set α = .01, then 1% of the scores would be in the
tail(s).
In a one-tail test, the full 5% of the scores is in the right tail (if you are looking for an increase in scores)
or the full 5% of the scores is in the left tail (if you are looking for a decrease in scores)
Two-tail tests assume that the 5% exclusion is from both ends. This means that 2.5% of the scores
will be in the right tail and 2.5% of scores will be in the left tail.
For a two-tail z test, the critical value of z = ±1.96, (this will be different for a t-test or ANOVA.)
SPSS will give you a p-value in a box labeled “Sig. 2 Tail”: compare the p value to your α value.
• If p < " you reject the null hypothesis (although you cannot say that it is actually false.)
• If p > " you “accept” * that the null hypothesis is true. [p = .23 > " = .05]
*In truth, we never actually “prove” that the alternative hypothesis is correct and we never actually “accept” the
null hypothesis; we only “fail to reject” it. Statistics provide evidence.

Alpha Levels for a Normal Curve

For a z-test (these values differ for other distributions (i.e. t )

Alpha Tails left right

! = .05 two-tail ±1.96

z = -1.96 95% of scores z = +1.96
! = .05 one-tail -1.645 1.645 High-probability
Scores if H0 is true
! = .01 two-tail ±2..575
! = .01 one-tail -2.325 2.325

5% of scores Low-probability scores if H0 is true

Todd Daniel PhD 5

HypothesisTesting Intro to Statistics

Degrees of Freedom
Step 3: Level of Significance
Calculating the Critical region for Various Tests
For other tests (t-test, ANOVA, Chi-square), you can look up the critical value on a table. These tables are included at the
back of your class notes. Consult the class notes for the type of test you are using to find the appropriate table and
instructions on how to use it. Using most tables to calculate the critical region will require knowing the degrees of
freedom (df). In most cases, the degrees of freedom will be n-1.

Type of Test Degrees of Freedom Example

The z test does not require degrees of
One Sample z test none freedom because you are not
estimating population parameters

One-Sample t Test n-1 t(24) = 3.23, p = .022

Independent Samples t Test (n1 + n2) – 2 t(14) = 1.59, p = .135

Paired Samples t Test n – 1 (where n = number of pairs) t(7) = 2.76, p = .028

One-Way ANOVA dƒT = n – 1, dƒB = k – 1, dƒW = n – k F(3,16) = 10.49, p < .001

Repeated Measures ANOVA dƒT = n – 1, dƒB = k – 1, dƒW = n – k F(3,16) = 10.49, p < .001

Correlation n – 2 (where n = number of pairs) r(48) = -.65

Linear Regression not reported Report beta values and n

One-Way Chi-Square Test k–1 X2(2, N = 70) = 1.15, p = .563

One-Way Chi-Square Test (krow – 1) * (kcolumn – 1) X2(3, N = 350) = 46.03, p < .05

n = number of participants
k = number of IV categories

Todd Daniel PhD 6

HypothesisTesting Intro to Statistics

Calculate
Step 4: Run the Statistics
Analyze Menu
When you run a test in SPSS, you begin with the Analyze menu. You will not use all
of these commands at an introductory level, but you should become familiar with the
commands that you will need for this class. When you click on Analyze, a drop down
menu opens from which you can choose the appropriate test.

Descriptive Statistics
used for exploring data and calculating descriptive statistics. Both Frequencies
and Explore offer similar options but one may be more suitable to a particular
task, so you should know both. Descriptives menu is the fast track to
descriptive stats, although you might find Frequencies more usable. Crosstabs
is used for the two-way chi-square and cross-tabulation. Q-Q plots are used to
assess normality or compare any two distributions.

Compare Means
you will use this menu a lot. This is where you find the most commonly used
hypothesis testing tools. Here is where you will find three types of t tests, as well
as simple Analysis of Variance (ANOVA).

General Linear Model

You can also do an ANOVA from the GLM menu using the Univariate
command. GLM allows you to request effect sizes, power, and crate plots. You
can also do a two-way ANOVA or a repeated measures ANOVA. Repeated
Measures is not available in some student versions of SPSS.

Correlate
Simple correlation, both Pearson and Spearman, is performed from this menu
using the Bivariate command. Partial or Distances commands control for a third
variable.

Regression
Simple & multiple linear regressions are conducted using the Linear command.
You can feed multiple IVs into the regression model to create a predictive
equation for a single DV.

Nonparametric Statistics
When you have categorical data (or your scale data violate the assumptions of a
parametric test), you can use a nonparametric alternative. This menu is the
Legacy Dialogs menu. Chi-square can be conducted here (or in the Crosstabs
command in the Descriptive Statistics menu). Kolmogorov-Smirnov, Mann-
Whitney, Kruskal-Wallis, Wilcoxin, and Friedman are all conducted here.

Todd Daniel PhD 7

 Making the Decision
HypothesisTesting Intro to Statistics

Step 5: Decide Whether or Not to Reject the Null (H0)

Compare the Obtained value to the Critical Value

Obtained Value = this is the number that you calculated in your statistical test. It is the test result and will be z (for z
tests), t (for t tests), F (for ANOVA), X2 (for Chi-square tests) or another letter for other, less frequently used tests.
Critical Value = this is the value that you looked up in the table in Step 3. SPSS will not provide a critical value in the
output, but will compute a p value. In SPSS, the p value is in a box labeled “Sig.”

Rules for Determining and Reporting Significance

• If your obtained (calculated) value exceeds your critical value, then your test is significant: reject the null, p < .05
• For one-tailed tests, the obtained value must exceed the critical value in the one direction you are testing.
• If your obtained value is not in the critical region, your test is not significant: “accept” the null, p > .05, ns
We never really “accept” the null, we just fail to reject it. It is highly unlikely that we would have found these results if
the null hypothesis was true.

Sig
Results are Significant Results are Not Significant
(2-tailed)
p value given in SPSS p = .042 p = .45, ns
p value = .000 in SPSS p < .001 impossible
No p value (c.f. doing a test by p < .05 p > .05
hand; doing one-tailed test)

High Frequency Scores

# = .05
# = .05

Low Frequency Scores Low Frequency Scores

# < .05
Critical Region # > .05
M # > .05
# < .05
Critical Region X-Axis

Reject Accept (fail to reject the null) Reject

Reject the null hypothesis Fail to reject the null hypothesis Reject the null hypothesis
Use the alternative hypothesis No support for the alt. hypothesis Use the alternative hypothesis
Results are in the critical region Results are not in the critical region Results are in the critical region
Results are significant Results are not significant Results are significant
There was a significant difference No significant difference There was a significant difference
There was an effect There was not an effect There was an effect
The treatment worked The treatment made no difference The treatment worked
p < .05 p > .05 p < .05

Todd Daniel PhD 8

HypothesisTesting Intro to Statistics

Type I and Type II Statistical Errors

Errors in Hypothesis Testing vs. Pregnancy Testing
Pregnancy Test
Assumption is that you are NOT pregnant
True Condition
NOT PREGNANT PREGNANT

TYPE II ERROR
NEGATIVE CORRECT RESULT
(not pregnant) False Negative
Avoid an Error
Result of Pregnant But Don’t Know It
Pregnancy
Test TYPE I ERROR
POSITIVE CORRECT RESULT
(pregnant) False Positive
Avoid an Error
Not Really Pregnant After All
Type I Error: rejecting a null hypothesis that is actually true (false positive)
Concluding that a treatment has an effect when it does not. Type I errors are only a concern when you reject the
null hypothesis. The probability of a Type I error is alpha (#) (typically .05 or 5%).
To decrease the chance of a Type I error, you can change the alpha level from .05 to .01; however, this increases
the possibility of a Type II error.
Type II Error: “accepting” a null hypothesis that is actually false (false negative)
Failure to detect an effect that actually exists. Type II errors are only a concern when you fail to reject the null
hypothesis. The probability of a Type II error is beta ($).
Type II errors are most likely to happen:
• When effect size is small – you are looking for a very tiny effect and it is easy to miss; or,
• When the n is small – you do not have a large enough sample size to adequately detect the effect.
To decrease the chance of a Type II error, increase power by increasing sample size (n).
Memory Aids: Type I is like Pinocchio’s nose (lying); Type II is like a dunce cap (missed the effect)
Errors are always written with a Roman numeral (Type I and Type II, not Type One or Type 2).

Statistical Hypothesis Test

Null Hypothesis assumes that there is NO effect
True State of the World
H0 TRUE H0 FALSE
Type I situation Type II situation

CORRECT RESULT TYPE II ERROR

ACCEPT H0 Avoid a Type I Error ! False Negative "
Result of p=1–α p=β
Hypothesis
Test TYPE I ERROR CORRECT RESULT
REJECT H0 False Positive " Avoid a Type II Error #
p=α P = 1– β (a.k.a. power)

Todd Daniel PhD 9

HypothesisTesting Intro to Statistics

Reporting Your Results

Step 5: Write Up Results in APA Style
Six Steps to Writing the Write-Up in APA Style
When reporting your results, you should begin with a statement about why the test was performed then describe
the results. Finally, include the statistics from the test you conducted.
1. Test: what is the name of the test you used? (t, ANOVA, chi-square, correlation, regression)
2. Variables: what variables were you measuring (IV & DV) to test what research question?
3. Measurement: how were the variables measured/quantified?
4. Significance: was the finding significant?
5. Statistics: written in APA style, see General Guidelines for Writing Up Statistics in APA Style below
6. Summary: what do the results indicate and what are the implications of the findings?
• A (1) single sample t-test was conducted (2) to determine whether students in a statistics class scored higher
on the final exam than the class last year. The (3) average scores on the final (M = 76.4, SD = 9.34) were
were (4) significantly higher than last year’s class final average (M = 74.5), (5) t(29) = 3.14, p = .032. This
indicates that (6) the students this year performed significantly better on the final than the class last year.
• A (1) one-way ANOVA was conducted (2) to test which type of treated wood has superior durability for
outdoor decking. Samples of three types of treated wood (3), CCA (M = 1.13), MCP (M = 1.22), and ACQ (M
= 1.19) were exposed to artificial weather testing but (4) no significant differences were found between their
performances (5), F(2, 298) = 1.42, ns. These result show that (6) the three types of wood available for
outdoor decking have approximately equal durability.

General Guidelines for Writing Up Statistics in APA Style

Use italics for statistical symbols, but use standard typeface (no bold or italics) when writing (a) Greek letters, (b)
subscripts that function as identifiers, and (c) abbreviations that are not variables.
t, F, r, N, n, M, SD, p, ns but α = .05, H0: μ = 28, PTSD
Use parentheses to enclose statistical values.
...was statistically significant (p = .042) for all variables.
Use parentheses to enclose degrees of freedom.
t(45) = 4.35
F(3, 87) = 2.11
Use brackets to enclose limits of confidence intervals.
95% CIs [3.45, 2.7], [-6.0, 3.89], and [1.89, 7.23]
Use an italicized, uppercase N in reference to the total number of subjects or
participants in the sample.
N = 328 people participated in the study
Use an italicized, lowercase n in reference to only a portion of the sample.
n = 42 of participants were Pacific Islanders

Todd Daniel PhD 10

HypothesisTesting Intro to Statistics

The Normal Curve

a.k.a. Bell Curve, Gaussian Curve

High Frequency Scores

# = .05
# = .05

Low Frequency Scores Low Frequency Scores

Critical Critical
# < .05 Region # > .05 # > .05 Region # < .05
&̅
.0013 .1359 .3413 .3413 .1359 .0013 X-Axis

-3 -2 -1 0 1 2 3 Z scores (SD)

Number line
38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 M = 50, SD = 4
Proportions

The Normal Curve is a theoretical or hypothetical distribution of scores.

Most scores fall in the middle so mean = median = mode
Symmetrical - Each half of the curve is a mirror image of the other. It is perfectly symmetrical.
It has no skew and no excess kurtosis
Asymptotic - no matter how far tails go out, they never touch the X-axis (i.e., symptote)
Frequency - The normal curve is a frequency polygon.
Frequency of x values decreases the farther you move away from the mean (in either direction)
Scores with greater frequency are found in the middle; scores with less frequency are in the tails
Standardization - The curve has a constant, direct relationship with the standard deviation
Each raw score (number) corresponds to a standard deviation (or a fraction of a standard deviation)
A constant percentage of scores will fall under any area of bell curve
When marked in standard deviation units, it becomes a standard normal curve (mean = 0, SD = 1)
Z Scores and the Normal Curve - A z score distribution is in the shape of a normal curve
A z score distribution has a mean of 0 and a standard deviation of 1.00
In a normal distribution, the z scores are seldom greater than plus-or-minus 3 standard deviations
Convert a proportion to a decimal: multiply by 100 or move the decimal point right two
places: p = .3414 – > 34.13%

Todd Daniel PhD 11

HypothesisTesting Intro to Statistics

Hypothesis Test vs. Jury Trial

An Analogy to how Quantitative Research is Done
We want to know if lab rats who use cocaine run through mazes faster than lab rats who drink alcohol.
So, we give a small group of rats coke and another group of rats rum-and-Coke. We measure how
quickly each group of rats runs through a maze. But is one group significantly faster? Understanding the
logic behind hypothesis testing is easier if you compare it to a jury trial. The data are put “on trial” to see
if there is enough evidence to “prove” the treatment (cocaine) had an effect.

Hypothesis Testing A Jury Trial

Null Hypothesis: We assume that there is no treatment Presumption of Innocence: The jury is instructed to

effect unless there is enough evidence to prove otherwise. assume the defendant is innocent until proven guilty.

H0: rats1 = rats2 (no difference in group means) H0: no difference between this person and an innocent person

The alpha level: We set a standard of how different the Standard of Proof: The jury must be convinced beyond

groups must be before we will be willing to conclude that a reasonable doubt (“the standard”) before they find a

the treatment had an effect; otherwise, any differences are person guilty.

assumed to be simply due to random chance. Standard: beyond a reasonable doubt

α = .05, two-tail test (difference of 12 points)

The sample data: The research study is conducted to Evidence: The prosecutor presents evidence to

gather data (evidence) to demonstrate whether the demonstrate that the defendant is guilty.

treatment had an effect. Victim’s blood was on his clothes, his DNA was at the crime
scene, history of violence toward the victim
Run the rats through the maze and collect data

The critical region: The sample data fall in the critical Deliberation: Either there is enough evidence to meet

region (meet the burden of proof) or they fall outside the the burden for proof of guilt (reasonable doubt or

critical region (there is not enough evidence to reject the null) preponderance of evidence) or there is not.
A difference of more than 12 points will be significantly It is highly unlikely that he would have the victim’s blood on
different his clothes the same night as her murder if he was innocent.
He is different than an innocent person.

Conclusion: If the sample data (evidence) fall into the Verdict: If the evidence exceeds the burden of proof, the

critical region (standard of proof) we reject the null (convict). jury votes “guilty”. If the evidence falls short, the verdict is

If the sample data do not fall into the critical region, we fail “not guilty.” The verdict declares the defendant guilty but

to reject the null (acquit). does not prove the defendant is guilty. Being acquitted does
The means are 15 points apart. Our standard was 12. The not prove the defendant is innocent, either.
groups are statistically significantly different
This defendant is guilty.

Todd Daniel PhD 12