Quantitative Research

Methods
Lecture 3
ANOVA 1

Eric Jacobs
Hubert Korzilius

Chair Research and Intervention Methodology

Nijmegen School of Management
 Today
 Analysis of Variance, ANOVA
 background
 approach
 assumptions
 post hoc analysis
 more than one factor, factorial design
 interaction

2
 Objectives week 3
Students are able to adequately express and substantiate
choices made with regard to:
choosing analysis of variance (ANOVA) as type of analysis
for answering research questions
describing the procedure ANOVA
applying the procedure ANOVA by means of SPSS and
making adequate choices in the analysis process
correctly interpreting the outcomes of ANOVA

3
Empirical cycle (Vennix, 2016)

4
 ANOVA and MANOVA
Independent variables Dependent variable(s)
ANOVA one

MANOVA two or more

ANOVA one one

one-way
example discipline opinion about organization of program

ANOVA two one

two-way
example discipline, gender quality program organization

MANOVA two or more

example discipline, gender quality program organization
quality of exams
quality of staff 5
Concepts and measurement
levels ANOVA 1
one or more independent variables
controlled by the researcher
called treatment variables or factors (also
conditions or groups)
e.g., gender, research group, feedback manipulation
consist of two or more levels (categories)
e.g. male-female, experimental-control group,
receiving feedback: yes or no
nominal or ordinal measurement level
6
Concepts and measurement
levels ANOVA 2
one dependent variable
response variable
e.g., corporate reputation, grade. commitment to organization
interval / ratio measurement level
by means of experiment checking the differences
in the dependent variables for the levels of the
independent variable(s)

analysis of variance is a dependence technique

7
Testing differences group means of
two independent groups: t-test
 Hypotheses:
 H0: µ1 = µ2
 H1: µ1 ≠ µ2

 Test statistic:

 Test statistic follows t-distribution with (n1 + n2 - 2) degrees of

freedom

8
 Degrees of freedom, df (1)
 The t-distribution has many shapes dependent on the degrees
of freedom, df. Each t-distribution has its own critical values.
 df: the number of values that are free to vary

 Example
 4 observations (range scores 1-5), mean = 3.5

X1 = 3
X2 = 4
X3 = 5
Total = 12  X4 must be 2

 There are 4 - 1 free to estimate values, consequently df = 4 - 1 = 3

9
 Degrees of freedom, df (2)
 t-distribution comparing means of two groups in t-test
 df group 1: n1 - 1
 df group 2: n2 - 1
 df total: n1 + n2 – 2

 Degrees of freedom are also important for the F-distribution

(e.g., used in ANOVA) and the c2-distribution (e.g. used in
crosstabs)

10
 Difference between > 2 groups
 Testing differences between more than two groups:
 H0: µ1 = µ2 = µ3
 H1: at least one μ differs from the others

 You may be inclined to test this by means of three

separate t-tests.
 µ1 versus µ2
 µ1 versus µ3
 µ2 versus µ3

 This is NOT allowed because of alpha inflation (aka

familywise or experimentwise error rate):
 Three separate tests, each at a = .05, leads to a total a = .143.
 In general: for k combined tests: atot = 1 – (1- a )k 11
Experiment corporate reputation 1

 Experiment focusing on the effect of commercials on

corporate reputation

 Research hypothesis: Commercial type has an effect on

corporate reputation

12
Experiment corporate reputation 2
 Independent variable, three bank commercials

1. ABN AMRO: “ABN AMRO's Cultural Heart”

https://www.youtube.com/watch?v=Hr5DjDv1sdw
2. ING: orange is seeing opportunities
https://www.youtube.com/watch?v=aHmt2eUeG08
3. RABO bank: Invested in each other. The power of
cooperation
https://www.youtube.com/watch?v=bsIiZrLA6D4

  Independent factor A, nominal measurement level, 3

levels

13
Experiment corporate reputation 3
 Dependent variable Corporate reputation measured with
three indicators
 Items “This company has a good reputation on …”
1. Quality of products and services
2. Innovativeness
3. CSR: Corporate Social Responsibility

 5-point Likert scales (completely disagree - completely agree)

 Internal consistency measured by Cronbach’s alpha = .86;

qualification: good
 Scale scores were calculated, sum scores, range 3-15

 Dependent variable Y: interval measurement level 14

 Analysis of variance
 Goal: testing null hypothesis stating that the
means of more groups are equal

Group A (Independent variable)

Dependent
variable

Total

M
SD
15
 ANOVA: background

group a
group b
group c

group a
group b
group c

group a
group b
group c

group a
group b
group c
16
 Approach ANOVA
 Variance between groups is compared with variance
within groups

 If the variance between groups is small in

relation to the variance within groups there
is no statistically significant difference in
means

 If the variance between groups is large in

relation to the variance within groups there
is a statistically significant difference in
means. The mean of at least one group 17
differs
 Approach ANOVA
 In order to compare the Independent variable A

variance between groups, the

Depen-
total variance is divided into dent
variance between groups and variable
Y
variance within groups

M
SD

j = var. A
k = groups
i = observation
n = total observations

total 18
 Graphical representation:
total variance (SStotal)

Response, Y

5
4
3 Y
2
1
Comm. 1 Comm. 2 Comm. 3

19
 Graphical representation:
SSbetween

Response, Y

5
4 Y1 Y3
3 Y
2
Y2
1
Comm. 1 Comm. 2 Comm. 3

20
 Graphical representation:
SSwithin

Response, Y

5
4 Y1
3 Y3
2 Y2
1
Comm. 1 Comm. 2 Comm. 3

21
 Approach ANOVA
 SStotal = (3-8)² + (5-8)² + … + (5-8)² + (11-8)² + …+ (12-8)² + (11-8)² = 156

 SSbetween = 6 * (5-8)² + 6 * (8-8)² + 6 * (11-8)² = 108

 SSwithin = (3-5)² + (5-5)² + … + (5-5)²

(5-8)² + (11-8)² + … + (10-8)²
(13-11)² + (9-11)² + … + (11-11)² = 48

Independent variable A

 SSbetween is aka SSmodel Depen

 SSwithin is aka SSerror or SSresidual dent

variab
le
Y

M 22
SD
 Approach ANOVA
 Statistical test

 H0: µ1 = µ2 = µ3
 H1: at least one mean differs

 Test statistic F

 Test statistic F has an F-distribution with (k-1) and (N-k)

degrees of freedom
 Example: 108
54
F  3 1   16.875
48 3. 2 23

18  3
 SPSS
Menu Analyze, Compare Means, One-Way ANOVA …

24
 Results ANOVA: descriptives

Descriptives

y
95% Confidence Interval for
Mean
N Mean Std. Deviation Std. Error Lower Bound Upper Bound Minimum Maximum
1 6 5,00 1,414 ,577 3,52 6,48 3 7
2 6 8,00 2,366 ,966 5,52 10,48 5 11
3 6 11,00 1,414 ,577 9,52 12,48 9 13
Total 18 8,00 3,029 ,714 6,49 9,51 3 13

 Overview group means and standard

deviations

25
 Results ANOVA:
statistical testing

 one independent variable: called one-way ANOVA

 Reporting results: F(2, 15) = 16.88, p < .001
 After F between brackets df between and df within (or error)
 Reporting p-values dependent of results (in Column below Sig.):
 p < .05; p < .01; p < .001 26
 if p ≥ .05: n.s. (not significant) or report p-values, e.g., p = .35
 ANOVA: characteristics and
assumptions
 Goal: establish whether there are significant differences
between values on a dependent variable for the different
categories of one or more independent variables
 Assumptions:
 one dependent variable of metric measurement level
 (in case of more dependent variables: multivariate ANOVA = MANOVA)
 one or more independent variables (factors) of nominal or
ordinal measurement level (categorical)
 independent and randomly chosen experimental units
(often participants)
 populations are normally distributed
 equal variances within the categories of the factors
(homogeneity) 27
 Post Hoc analyses
 Back to example ANOVA
 If result of ANOVA is statistically significant, than the null
hypothesis is rejected.
 This means that at least one mean differs from the other
means. The question is which one?

28
 Levene’s test of homogeneity of
variances
 H0: group variances are equal
 H1: at least one of the group variances
differs

29
 Post Hoc analyses
 To determine which
means differ
 Testing differences of all
means
 Tukey
 if homogeneity can be
assumed and in case of
equal group sizes

 Hochberg
 if homogeneity can be
assumed and in case of
unequal group sizes

 Games-Howell
 if homogeneity cannot be
assumed 30
 Post hoc analyses
o m

De

e a
d
e r ee n n
e
e
SI r
-
r
((
ig
EJ
IJ B B) ) r
6
30
38T
1
82
2*u
6
30
308
23*
3
60
382
21
8*
6
30
3883
2*
3
60
303
21
8*
3
60
3822
8*
7
20
31H
1
62
4*o
7
20
306
43* Independent variable A
2
70
312
41
6*
7
20
3163
4*
2
70
303
41
6*Depen
2
70
3142
6*dent
2
20
55G
1
02
0 variab a
2
70
604
63*le
2
20
552
01
0 Y
2
20
5503
0
7
20
603
61
4*
2
20
5502
0
M 31
*
T. h
SD
 Reporting
 (After problem statement, conceptual model, description
experiment, data collection, etc.)
 The research hypothesis “Commercial type has an effect
on corporate reputation” received empirical evidence
(F(2, 15) = 16.88, p < .001)
Post hoc tests (Tukey’s HSD procedure)* showed that
the RABO bank commercial resulted in a higher score
for corporate reputation than the ABN AMRO and ING
commercials; the ING commercial had a higher score on
corporate reputation than the ABN AMRO commercial
 Possible recommendations: …
* Numerical details of the post hoc tests are omitted in your report
32
 Two-way ANOVA
 Two independent variables (A and B) and one
dependent variable (corporate reputation).
 A = commercial (1 = ‘ABN AMRO’, 2 = ‘ING’, 3 = ‘RABO bank’)
 B = gender (1 = male, 2 = female)
 Hypothesis “Commercial type, gender, and their
interaction have an effect on corporate reputation”
Independent variable A
M SD

Indepen-
dent
variable
B

M
SD 33
 Two-way ANOVA
 Three effects are possible:
 A main effect of factor A
 A main effect of factor B
 An interaction effect of A and B

 Always using the hypotheses:

 H0: all group means are equal
 H1: at least one mean differs

 The total sum of squares (SS) is subdivided into four

components

corresponding
degrees of freedom 34
(df)
 Two-way ANOVA in SPSS

Menu Analyze, General Linear

Model, Univariate … 35
 Output two-way ANOVA : descriptive
statistics + Levene’s test
Descriptive Statistics

Dependent Variable: y
a b Mean Std. Deviation N
1 1 4,33 1,155 3
2 5,67 1,528 3
Total 5,00 1,414 6
2 1 6,00 1,000 3
2 10,00 1,000 3
Total 8,00 2,366 6
3 1 10,00 1,000 3
2 12,00 1,000 3
Total 11,00 1,414 6
Total 1 6,78 2,682 9 Levene's Test of Equality of Error Variancesa
2 9,22 2,991 9 Dependent Variable: y
Total 8,00 3,029 18 F df1 df2 Sig.
,317 5 12 ,893
Tests the null hypothesis that the error variance of
the dependent variable is equal across groups.
36
a. Design: Intercept+a+b+a * b
 Output two-way ANOVA:
statistical tests
n -

D
III
q
SdF
S
S
u
i f
g
7
5
3
70a
C
0
1
05
0In
0
2
0
10A 
9
1
9
31B

8
2
9
3
2
8
1
7A
E 
0
8 T
0
7 C
a
R

 Main effect factor A, commercial type, (F(2,12) = 42.26, p

< .001)  elaborate by means of post hoc test
 Main effect factor B, gender, (F(1,12) = 21.04, p < .01) 
describe effect in terms of group means
 No interaction effect A*B (F(2,12) = 2.26, p = .15) 37
 Analysis of interaction effects
 Assume that interaction effect is significant. How to interpret
such an effect?
(illustrated by means of a manipulated example)
n -

D
II I
S
qu
dF
S
S
i f
g
0
5
0
00a
C
0
1
0
00In
0
2
0
00A
0
1
0
00B
0
2
0
00A
0
2
0 E
0
8 T
0
7 C
a
R

 Interpretation of significant interaction effects in terms of

plots and means. 38
 Analyses of interaction effects
 The main effects A and B
were not significant, but Estimated Marginal Means of Y
the interaction effect A*B 12
was significant
 The means combined with 11
the plot show a pattern
that enables to interpret 10
the interaction effect

Estimated Marginal Means

 The ‘ABN AMRO’ 9
commercial type (A1)
resulted in a higher value 8
on corporate reputation for
men than for women 7
 The ‘ING’ commercial type B
(A2) did not show any 6
differences
1
 The ‘RABO bank’ 5
commercial type (A3)
yielded a higher corporate 4 2
reputation score for 1 2 3
women than for men 39
A