Assumption Testing

Qin Gao

Contents
Testing Assumptions for Parametric Tests 2
Common assumptions of parametric tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Assessing Normality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Assessing Homogeneity of Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Data Transformation: Correcting Data Problems 15

Log transformation (log 𝑋𝑖 ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Square root transformation (√𝑋𝑖 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Reciprocal transformation (1/𝑋𝑖 ): . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Arcsine square root transformation for percentage numbers . . . . . . . . . . . . . . . . . . . . . . 16
To transform or not to transform, that is a question. . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Reporting assumption testing and data transformation 17

Links & Readings 17

1
Testing Assumptions for Parametric Tests

Common assumptions of parametric tests

Parametric tests based on the normal distribution assume:

• Normally distributed
– Sampling distribution
– Residuals

• Homogeneity of variance

• Interval or ratio level data

• Independent scores

Assessing Normality

Basic approaches

• Central limit theorem

– If N > 30, the sampling distribution is normal anyway

• Graphical displays

– Histogram
– Q-Q plot (or P-P plot)

• Descriptive statistics describing the distribution

– Skewness
– Kurtosis

• Statistics tests if data differ from a normal distribution

– Shapiro-Wilk
– Kolmogorov-Smirnov test

Histogram

Example data: students’ exam scores in R courses from two universities. The computer experience, the
percentage of lectures taken, and the level of numeracy prior to the class are collected.

library(dplyr)
library(ggplot2)
RexamData <- read.delim("RExam.dat", header = T)
str(RexamData)

2
## 'data.frame': 100 obs. of 5 variables:
## $ exam : int 18 30 40 30 40 15 36 40 63 31 ...
## $ computer: int 54 47 58 37 53 48 49 49 45 62 ...
## $ lectures: num 75 8.5 69.5 67 44.5 76.5 70 18.5 43.5 100 ...
## $ numeracy: int 7 1 6 6 2 8 3 7 4 6 ...
## $ uni : int 0 0 0 0 0 0 0 0 0 0 ...

Convert categorical variables to factors, create subsets that will be used

RexamData$uni <- factor(RexamData$uni, levels = c(0:1), labels = c("A", "B"))

RexamData.A <- filter(RexamData, uni == "A")

RexamData.B <- filter(RexamData, uni == "B")

Check the histgrams using qplot() from ggplot2

qplot(exam, data = RexamData, facets = uni ~., binwidth = 2)

A
2

4 B

0
25 50 75 100
exam

Check the density plot with ggplot2()

ggplot(RexamData.A, aes(exam))+geom_histogram(aes(y = ..density..), colour = "black", fill = "white")+st

3
 0.04
density

0.02

0.00

20 30 40 50 60 70
exam

ggplot(RexamData.B, aes(exam))+geom_histogram(aes(y = ..density..), colour = "black", fill = "white")+st

4
 0.06

0.04
density

0.02

0.00

60 70 80 90 100
exam

Please refer to cookbook for R for more information about ggplot2 package.

Skewness & Kurtosis

• The skewness of a data population is a measure of symmetry.

• Traditional nonparametric definition of skewness: (𝜇 − 𝜈)/𝜎, where 𝜇 is the mean, and 𝜈 is the median
• Modern definition of skewness: the third standardized moment 𝛾1 of a random variable (also called
the moment coefficient of skewness)

𝑋−𝜇 3 𝜇
𝛾1 = 𝐸[( ) ] = 33
𝜎 𝜎
where 𝜇3 is the third central moment of the random variable.
As a rule, negative skewness indicates that the mean of the data values is less than the median, and the data
distribution is left-skewed. Positive skewness would indicates that the mean of the data values is larger than
the median, and the data distribution is right-skewed.
Intuitively, the kurtosis is a measure of the peakedness of the data distribution. The kurtosis of a univariate
population is defined by the following formula.

𝑋−𝜇 4 𝜇
𝐾𝑢𝑟𝑡𝑜𝑠𝑖𝑠 = 𝐸[( ) ] = 44
𝜎 𝜎
where 𝜇4 is the fourth central moment of the random variable.
The kurtosis of a normal distribution is 3. To simplify the interpretation, in some R functions the excess
kurtosis formula is adopted.

5
 Figure 1: Postive and negative skewed distributions

𝑋−𝜇 4 𝜇
𝐸𝑥𝑐𝑒𝑠𝑠 𝐾𝑢𝑟𝑡𝑜𝑠𝑖𝑠 = 𝐸[( ) ] − 3 = 44 − 3
𝜎 𝜎

With this second formula, negative kurtosis would indicates a flat data distribution. Positive kurtosis would
indicates a peaked distribution.
Computing skewness & kurtosis
Skewness and kurtosis can be computed with skewness() and kurtosis() from moments package.

library(moments)
Runi <- group_by(RexamData, uni)
summarize(Runi, exam_skw = skewness(exam), exam_kur = kurtosis(exam), num_skw = skewness(numeracy), num_

## # A tibble: 2 x 5
## uni exam_skw exam_kur num_skw num_kur
## <fct> <dbl> <dbl> <dbl> <dbl>
## 1 A 0.300 2.37 0.496 2.45
## 2 B 0.264 2.64 0.769 3.12

Alternatively: describe() from the psych package produces both measures:

library(psych)
by(data = RexamData$exam, INDICES = RexamData$uni, FUN = describe)

## RexamData$uni: A
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 50 40.18 12.59 38 39.85 12.6 15 66 51 0.29 -0.72 1.78
## ------------------------------------------------------------
## RexamData$uni: B
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 50 76.02 10.21 75 75.7 8.9 56 99 43 0.26 -0.46 1.44

Alternatively: stat.desc() from the pastecs package produces even more descriptive information:

6
library(pastecs)
by(data = RexamData$exam, INDICES = RexamData$uni, FUN = stat.desc)

## RexamData$uni: A
## nbr.val nbr.null nbr.na min max range
## 50.0000000 0.0000000 0.0000000 15.0000000 66.0000000 51.0000000
## sum median mean SE.mean CI.mean.0.95 var
## 2009.0000000 38.0000000 40.1800000 1.7803210 3.5776890 158.4771429
## std.dev coef.var
## 12.5887705 0.3133094
## ------------------------------------------------------------
## RexamData$uni: B
## nbr.val nbr.null nbr.na min max range
## 50.0000000 0.0000000 0.0000000 56.0000000 99.0000000 43.0000000
## sum median mean SE.mean CI.mean.0.95 var
## 3801.0000000 75.0000000 76.0200000 1.4432079 2.9002348 104.1424490
## std.dev coef.var
## 10.2050208 0.1342413

Tip: Obtain descriptive statistics for multiple variables with these two functions, you can use
cbind() :

by(cbind(RexamData$exam,RexamData$numeracy), RexamData$uni, describe)

by(cbind(RexamData$exam,RexamData$numeracy), RexamData$uni, stat.desc, basic = F, norm = T)

The impact of sample size on skewness and kurtosis

The skewness and kurtosis statistics appear to be very dependent on the sample size.
The more data you have, the better you can describe the shape of the distribution.

Q-Q plot

The q-q plot provides a visual comparison of the sample quantiles to the corresponding theoretical quantiles.
In general, if the points in a q-q plot depart from a straight line, then the assumed distribution is called into
question.
Q-Q plots are useful to diagnose what kind of deviations from the normal distribution exist in the data.
QQ plots can be created with the qqnorm() function.

par(mfrow = c(1, 2))

qqnorm(RexamData.A$exam, col = "green")
qqnorm(RexamData.B$exam, col = "blue")

7
Figure 2: The impact of sample size on skewness and kurtosis

8
 Figure 3: Q-Q plots for data with flat or peaked dispersion

Figure 4: Q-Q plots showing various deviations from the normal distribution

9
 Normal Q−Q Plot Normal Q−Q Plot

100
60

90
Sample Quantiles

Sample Quantiles
50

80
40

70
30
20

−2 −1 0 1 2 60 −2 −1 0 1 2

Theoretical Quantiles Theoretical Quantiles

Shapiro-Wilk test

The Shapiro-Wilk test, proposed in 19651 , calculates a W statistic that tests whether a random sample,
x1,x2,…,xn comes from (specifically) a normal distribution. Small values of W are evidence of departure
from normality and percentage points for the W statistic, obtained via Monte Carlo simulations, were
reproduced by Pearson and Hartley (1972)2 .

𝑁
(∑ 𝑎𝑖 𝑌𝑖 )2
𝑊 = 𝑁 𝑖=1
∑ (𝑌𝑖 − 𝑌 ̄ )2
𝑖=1

• 𝑌𝑖 is the ith order statistic, i.e., the ith-smallest number in the sample;
• 𝑎𝑖 is given by

𝑚𝑇 𝑉 −1
(𝑎1 , … , 𝑎𝑛 ) =
((𝑚𝑇 𝑉 −1 )(𝑉 −1 𝑚))1/2

and 𝑚 = (𝑚1 , ..., 𝑚𝑛 )𝑇 are the expected values of the order statistics of independent and identically dis-
tributed random variables sampled from the standard normal distribution, and 𝑉 is the covariance matrix
of those order statistics.
1 Shapiro, S. S. and Wilk, M. B. (1965). “An analysis of variance test for normality (complete samples)”, Biometrika, 52, 3

and 4, pages 591-611.

2 Pearson, A. V., and Hartley, H. O. (1972). Biometrica Tables for Statisticians, Vol 2, Cambridge, England, Cambridge

University Press.

10
A simpler -but not fully technically correct- explanation is this: the Shapiro-Wilk test first quantifies the
similarity between the observed and normal distributions as a single number: it superimposes a normal curve
over the observed distribution as shown below. It then computes which percentage of our sample overlaps
with it: a similarity percentage.

Figure 5: A schematic interpretation of the Shapiro-Wilk test

Shapiro-Wilk test can be carried out using shapiro.test() in R.

by(RexamData$exam, RexamData$uni, shapiro.test)

## RexamData$uni: A
##
## Shapiro-Wilk normality test
##
## data: dd[x, ]
## W = 0.97217, p-value = 0.2829
##
## ------------------------------------------------------------
## RexamData$uni: B
##
## Shapiro-Wilk normality test
##
## data: dd[x, ]
## W = 0.98371, p-value = 0.7151

by(RexamData$numeracy, RexamData$uni, shapiro.test)

## RexamData$uni: A
##

11
## Shapiro-Wilk normality test
##
## data: dd[x, ]
## W = 0.94082, p-value = 0.01452
##
## ------------------------------------------------------------
## RexamData$uni: B
##
## Shapiro-Wilk normality test
##
## data: dd[x, ]
## W = 0.93235, p-value = 0.006787

Kolmogorov–Smirnov test

Kolmogorov–Smirnov test (K–S test or KS test) is a nonparametric test of the equality of continuous, one-
dimensional probability distributions that can be used to compare a sample with a reference probability
distribution (one-sample K–S test), or to compare two samples (two-sample K–S test).
The Kolmogorov–Smirnov statistic quantifies a distance between the empirical distribution function of the
sample and the cumulative distribution function of the reference distribution, or between the empirical
distribution functions of two samples.

Figure 6: Illustration of the Kolmogorov–Smirnov statistic

Kolmogorov–Smirnov test can be conducted using ks.test() in R. Details about the distribution to be com-
pared against (in this case, the theoretical normal distribution) need to be provided.

ks.test(RexamData.A$exam, "pnorm", mean(RexamData.A$exam), sd(RexamData.A$exam))

##
## Asymptotic one-sample Kolmogorov-Smirnov test
##
## data: RexamData.A$exam
## D = 0.1057, p-value = 0.6315
## alternative hypothesis: two-sided

12
ks.test(RexamData.B$exam, "pnorm", mean(RexamData.B$exam), sd(RexamData.B$exam))

##
## Asymptotic one-sample Kolmogorov-Smirnov test
##
## data: RexamData.B$exam
## D = 0.072777, p-value = 0.9538
## alternative hypothesis: two-sided

Comparing S-W test and K-S test

• S-W test is specifically for testing normality, whereas K-S test can be of more general usage.
• S-W test is more powerful than K-S test for testing normality.
• S-W test is suitable for small sample size (3 ≤ 𝑛 ≤ 50). For large sample, the S-W statistics can be
significant in any way.
• The two-sample K–S test is one of the most useful and general nonparametric methods for comparing
two samples, as it is sensitive to differences in both location and shape of the empirical cumulative
distribution functions of the two samples.

Assessing Homogeneity of Variance

Barlett’s test
𝑘
(𝑁−𝑘)𝑙𝑛𝑠2𝑝 −∑𝑖=1 (𝑁𝑖 −1)𝑙𝑛𝑠2𝑖
Barlett’s test: 𝑇 = 𝑘
1+(1/(3(𝑘−1)))((∑𝑖=1 1/(𝑁𝑖 −1))−1/(𝑁−𝑘))

Where 𝑁 = ∑𝑘𝑖=1 𝑛𝑖 , 𝑆𝑝2 = 1

𝑁−𝑘 ∑𝑖 (𝑛𝑖 − 1)𝑆𝑖2 𝑖

• Barlett’s test in its essence is modification of the corresponding likelihood ratio test designed to make
the approximation to the 𝜒2𝑘−1 distribution better.
• Barlett’s test is sensitive to normality.

You can perform Bartlett’s test with the bartlett.test function. If your data is in stacked form (with the values
for both samples stored in one variable), use the command: bartlett.test(values~groups, dataset)

bartlett.test(exam~uni, RexamData)

##
## Bartlett test of homogeneity of variances
##
## data: exam by uni
## Bartlett's K-squared = 2.122, df = 1, p-value = 0.1452

bartlett.test(numeracy~uni, RexamData)

##
## Bartlett test of homogeneity of variances
##
## data: numeracy by uni
## Bartlett's K-squared = 7.4206, df = 1, p-value = 0.006448

Tip: If your data is in unstacked form (with the samples stored in separate variables) nest the
variable names inside the list function as shown below. bartlett.test(list(dataset$sample1,
dataset$sample2, dataset$sample3))

13
Levene’s test
𝑘 ̄ −𝑍..
̄ )2
(𝑁−𝑘) ∑𝑖=1 𝑁𝑖 (𝑍𝑖.
Levene’s test: 𝑊 = (𝑘−1) ∑𝑘 𝑁 𝑖 ̄ )2
𝑖=1
∑𝑗=1 (𝑍𝑖𝑗 −𝑍𝑖.

|𝑌𝑖𝑗 − 𝑌𝑖⋅̄ |, 𝑌𝑖⋅̄ is a mean of the 𝑖-th group,

Where 𝑍𝑖𝑗 = {
|𝑌𝑖𝑗 − 𝑌𝑖⋅̃ |, 𝑌𝑖⋅̃ is a median of the 𝑖-th group.

𝑍𝑖⋅ = 1
𝑁𝑖 ∑𝑁 𝑖
𝑍 is the mean of the 𝑍𝑖𝑗 for group i
𝑗=1 𝑖𝑗
𝑁
1 𝑘 𝑖

𝑍⋅⋅ = ∑ ∑ 𝑍𝑖𝑗 is the mean of all 𝑍𝑖𝑗

𝑁 𝑖=1 𝑗=1
The test statistic 𝑊 is approximately F-distributed with (𝑘 − 1, 𝑁 − 𝑘) degrees of freedom.
When you are very sure about the normality of your data, use Barlett’s test; otherwise, use Levene’s test
instead. In particular, Levene’s test using the median (instead of the mean) should be performed when
the distribution is skewed.
You can perform Levene’s test with the leveneTest() function from the car package:
leveneTest(value variable, group, center = median/mean)

library(car)
leveneTest(RexamData$exam, RexamData$uni)

## Levene's Test for Homogeneity of Variance (center = median)

## Df F value Pr(>F)
## group 1 2.0886 0.1516
## 98

leveneTest(RexamData$numeracy, RexamData$uni)

## Levene's Test for Homogeneity of Variance (center = median)

## Df F value Pr(>F)
## group 1 5.366 0.02262 *
## 98
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Variance ratio (Hartley’s F)

As with the Shapiro-Wilk test, when the sample size is big, small differences in group variances can produce
a Leven’s test that is significant. A useful double check is to look at Hartley’s 𝐹𝑚𝑎𝑥 (also known as the
variance ratio, Pearson & Hartley, 19543 )

𝐹𝑚𝑎𝑥 = 𝑠2𝑙𝑎𝑟𝑔𝑒𝑠𝑡 /𝑠2𝑠𝑚𝑎𝑙𝑙𝑒𝑠𝑡

The resulting ratio, 𝐹𝑚𝑎𝑥 , is then compared to a critical value from a table of the sampling distribution of
𝐹𝑚𝑎𝑥 , as shown in the table. If the computed ratio is less than the critical value, the groups are assumed to
have similar or equal variances.
3 Pearson, E.S., & Hartley, H.O (1954). Biometrika tables for statistics, Volume I. New York: Cambridge University Press.

14
 Figure 7: The sampling distribution of Fmax

Data Transformation: Correcting Data Problems

Basic Approaches

• Log transformation
• Square root transformation
• Reciprocal transformation
• Arcsine square root transformation
• Box-cox transformation (The final desparate attempt!)

Log transformation (log 𝑋𝑖 )

• Reduce positive skew.

• Useful when SD is proportional to mean
• Dealing with zeros: log (𝑌 + 1)

Square root transformation (√𝑋𝑖

• Also reduces positive skew

• Can also be useful for stabilizing variance.
• Useful for count data or small whole numbers

Reciprocal transformation (1/𝑋𝑖 ):

• Dividing 1 by each score also reduces the impact of large scores.

15
 • This transformation reverses the scores; you can avoid this by reversing the scores before the transfor-
mation

1
𝑋𝑚𝑎𝑥 − 𝑋𝑖 + 1

Figure 8: Prototypical Histograms for Non-normal distributions

Arcsine square root transformation for percentage numbers

• Useful for percentage numbers

• The value of 0% should be substituted by (1/4n) and the value 100% by (100-1/4n), where n is the
number of units in which the percentage data were based (i.e. the denominator used in computing the
percentage).
• Some more guidelines for transforming percentage data
– For percentage data lying within the range of 20 - 80%, no transformation is needed.

– For percentage data lying within a range of either 0 - 20% or 80 – 100%, but not both, the square
root transformation could be useful.
– For percentage data that do not follow the ranges specified in either Rule 1 or Rule 2 (e.g. percent
control data), the Arc Sine square root transformation may be useful.

To transform or not to transform, that is a question.

• Sometimes transforming the data helps as often as it hinders the accuracy of F (Games & Lucas,
19664 ).
• Transforming the data changes the hypothesis being tested; Occasionally it reverse the difference in
means of the original data
4 Games, P.A., & Lucas, P.A. (1966). Power of the analysis of variance of independent groups on non-normal and normally

transformed data. Educational and Psychological Measurement, 26, 311-327.

16
 – E.g., when using a log transformation and comparing means, you change from comparing arith-
metic means to comparing geometric means
• The consequences for the statistical model of applying the ‘wrong’ transformation could be worse than
the consequences of analyzing the untransformed scores.

Reporting assumption testing and data transformation

Normally only the method and the fact of violation are reported in the main text in a journal paper. Detailed
statistics for assumption testing are normally not shown to the reader, but you are expected to be able to
provide the full details if being demanded.

Links & Readings

• Wikipedia explanation for Shapiro-Wilk and Kolmogorov-Smirnov tests
• Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests: A
simple guide to available normality tests
• Bartlett, M. S., (1947). The use of transformation. Biometric Bulletin, 3, 39-52.
• Zubin, J. (1935). Note on a transformation function for proportions and percentages. Journal of
Applied Psychology, 19, 213-220.
• Grissom, R. J. (2000). Heterogeneity of variance in clinical data. Journal of Consulting and Clinical
Psychology, 68, 155-165. Reporting a situation when the transformation reverse the difference of means

17