Recap

Sampling distribution of proportions

Lecture 15: Chi square tests Review question

Roughly 90% of individuals are right handed. The shape of the sam-
pling distribution of proportions of individuals that are right handed in
Statistics 101
random samples of size 80 will be
Prof. Rundel
(a) nearly normal
October 25, 2011 (b) left skewed
(c) right skewed
(d) not enough information to tell

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 1 / 39

Testing for goodness of fit using chi-square Testing for goodness of fit using chi-square

Waldon’s dice Labby’s dice

Walter Frank Raphael Weldon (1860 - In 2009, Zacariah Labby (U of

1906), was an English evolutionary biologist Chicago), repeated Weldon’s
and a founder of biometry. He was the joint experiment using a
founding editor of Biometrika, with Francis homemade dice-throwing, pip
Galton and Karl Pearson. counting machine.
In 1894, he rolled 12 dice 26,306 times, and The rolling-imaging process
recorded the number of 5s or 6s (which he took about 20 seconds per
considered to be a success). roll.

It was observed that 5s or 6s occurred more often than expected, Each day there were ∼150 images to process manually.
and Pearson hypothesized that this was probably due to the At this rate Weldon’s experiment was repeated in a little more
construction of the dice. Most inexpensive dice have than six full days.
hollowed-out pips, and since opposite sides add to 7, the face
with 6 pips is lighter than its opposing face, which has only 1 pip. http:// www.youtube.com/ watch?v=95EErdouO2w

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 2 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 3 / 39
 Testing for goodness of fit using chi-square Testing for goodness of fit using chi-square Creating a test statistic for one-way tables

Labby’s dice (cont.) Expected counts

Labby did not actually observe the same phenomenon that
Weldon observed (higher frequency of 5s and 6s).
Automation allowed Labby to collect more data than Weldon did Clicker question
in 1894, instead of recording “successes” and “failures”, Labby
recorded the individual number of pips on each die. Labby rolled 12 dice 26,306 times. If each side is equally likely to come
up, how many 1s, 2s, · · · , 6s would we expect to have observed?

1
(a) 6
12
(b) 6
26,306
(c) 6
12×26,306
(d) 6

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 4 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 5 / 39

Testing for goodness of fit using chi-square Creating a test statistic for one-way tables Testing for goodness of fit using chi-square Creating a test statistic for one-way tables

Summarizing Labby’s results Setting the hypotheses

The table below shows the observed and expected counts from
Labby’s experiment. Do these data provide convincing evidence to suggest an inconsis-
tency between the observed and expected counts?

Outcome 1 2 3 4 5 6 H0 : There is no inconsistency between the observed and the

Observed counts 53,222 52,118 52,465 52,338 52,244 53,285 expected counts. The observed counts follow the same
Expected counts 52,612 52,612 52,612 52,612 52,612 52,612 distribution as the expected counts.
HA : There is an inconsistency between the observed and the
expected counts. The observed counts do not follow the same
distribution as the expected counts. There is a bias in which side
At a first glance, does there appear to be an inconsistency between comes up on the roll of a die.
the observed and expected counts?

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 6 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 7 / 39
 Testing for goodness of fit using chi-square Creating a test statistic for one-way tables Testing for goodness of fit using chi-square The chi-square test statistic

Evaluating the hypotheses Anatomy of a test statistic

The general form of a test statistic is

To evaluate these hypotheses, we quantify how different the point estimate − null value
observed counts are from the expected counts. SE of point estimate
Large deviations from what would be expected based on
sampling variation (chance) alone provide strong evidence for This construction is based on
the alternative hypothesis.
1 identifying the difference between a point estimate and an
expected value if the null hypothesis was true, and
This is called a goodness of fit test since we’re evaluating how 2 standardizing that difference using the standard error of the point
well the observed data fit the expected distribution. estimate.
These two ideas will help in the construction of an appropriate
test statistic for count data.

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 8 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 9 / 39

Testing for goodness of fit using chi-square The chi-square test statistic Testing for goodness of fit using chi-square The chi-square test statistic

The standardized difference Calculating standardized differences for cells

The standardized difference
Clicker question
For each cell i
observed count − null count Which of the following is the correct calculation of the standardized
Zi = √ difference for the second cell, Z2 ?
null count
Outcome 1 2 3 4 5 6
Observed counts 53,222 52,118 52,465 52,338 52,244 53,285
in context... Expected counts 52,612 52,612 52,612 52,612 52,612 52,612

Outcome 1 2 3 4 5 6 52,118−52,612
(a) Z2 = √
52,612
= −2.15
Observed counts 53,222 52,118 52,465 52,338 52,244 53,285
52,612−52,118
Expected counts 52,612 52,612 52,612 52,612 52,612 52,612 (b) Z2 = √
52,612
= 2.15
52,612−52,118
↓ (c) Z2 = √
52,118
= 2.16
52,118−52,612
53, 222 − 52, 612 (d) Z2 = √
52,118
= −2.16
Z1 = √ = 2.66
52, 612
Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 10 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 11 / 39
 Testing for goodness of fit using chi-square The chi-square test statistic Testing for goodness of fit using chi-square The chi-square test statistic

The chi-square test statistic Why square?

The chi-square test statistic
Sum of squared differences for each cell, k = total # of bins

χ2 = Z12 + Z22 + · · · + Zk2

Squaring each standardized difference before adding them together
does two things:
in context...
Any standardized difference that is squared will now be positive.
Differences that already looked unusual – e.g. large standardized
Outcome 1 2 3 4 5 6 differences – will become much larger after being squared.
Observed counts 53,222 52,118 52,465 52,338 52,244 53,285
Expected counts 52,612 52,612 52,612 52,612 52,612 52,612

χ2 = 2.662 +(−2.15)2 +(−0.64)2 +(−1.19)2 +(−1.60)2 +2.932 = 24.67

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 12 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 13 / 39

Testing for goodness of fit using chi-square The chi-square distribution and finding areas Testing for goodness of fit using chi-square The chi-square distribution and finding areas

The chi-square distribution The chi-square distribution

Degrees of Freedom
2
In order to determine if the χ statistic we calculated is
2
4
9
considered unusually high or not we need to first describe its
distribution.
The chi-square distribution is sometimes used to characterize
data sets and statistics that are always positive and typically right
skewed.
The chi-square distribution has just one parameter called
degrees of freedom (df), which influences the shape, center, and 0 5 10 15 20 25

spread of the distribution.

Remember: The normal distribution had two parameters – mean and How does the center of the distribution change when the
standard deviation – that could be used to describe its exact degrees of freedom is larger?
characteristics.
What about the variability (spread)?
How does the shape change?
Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 14 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 15 / 39
 Testing for goodness of fit using chi-square The chi-square distribution and finding areas Testing for goodness of fit using chi-square The chi-square distribution and finding areas

Finding areas under the chi-square curve Finding areas under the chi-square curve (cont.)
Estimate the shaded area under the chi-square curve with df = 6.
We will calculate the p-value for the hypotheses we set earlier as
the tail area under the chi-square distribution.
For this we can use technology, or a chi-square probability table.
This table differs a bit from the normal probability table: df = 6

we identify a range for the area instead of a precise value

chi-square probability table only provides upper tail values as
opposed to the normal probability table, which shows lower tail 0 10

areas
Upper tail 0.3 0.2 0.1 0.05 0.02 0.01 0.005 0.001
df 1 1.07 1.64 2.71 3.84 5.41 6.63 7.88 10.83
2 2.41 3.22 4.61 5.99 7.82 9.21 10.60 13.82
Upper tail 0.3 0.2 0.1 0.05 0.02 0.01 0.005 0.001
df 1 1.07 1.64 2.71 3.84 5.41 6.63 7.88 10.83 3 3.66 4.64 6.25 7.81 9.84 11.34 12.84 16.27
2 2.41 3.22 4.61 5.99 7.82 9.21 10.60 13.82 4 4.88 5.99 7.78 9.49 11.67 13.28 14.86 18.47
3 3.66 4.64 6.25 7.81 9.84 11.34 12.84 16.27
4 4.88 5.99 7.78 9.49 11.67 13.28 14.86 18.47 5 6.06 7.29 9.24 11.07 13.39 15.09 16.75 20.52
5 6.06 7.29 9.24 11.07 13.39 15.09 16.75 20.52
6 7.23 8.56 10.64 12.59 15.03 16.81 18.55 22.46
6 7.23 8.56 10.64 12.59 15.03 16.81 18.55 22.46
0 5 10 15 20 25
7 8.38 9.80 12.02 14.07 16.62 18.48 20.28 24.32 7 8.38 9.80 12.02 14.07 16.62 18.48 20.28 24.32

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 16 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 17 / 39

Testing for goodness of fit using chi-square The chi-square distribution and finding areas Testing for goodness of fit using chi-square The chi-square distribution and finding areas

Finding areas under the chi-square curve (cont.) Finding areas under the chi-square curve (one more)
Clicker question Clicker question

Estimate the shaded area (above 17) under the χ2 curve with df = 9. Estimate the shaded area (above 30) under the χ2 curve with df = 10.

(a) 0.05 (a) greater than 0.3

(b) 0.02 (b) between 0.005 and 0.001
df = 9
(c) between 0.02 and 0.05 df = 10
(c) less than 0.001
(d) between 0.05 and 0.1 (d) greater than 0.001

0 17
(e) between 0.01 and 0.02 0 30
(e) cannot tell using this table

Upper tail 0.3 0.2 0.1 0.05 0.02 0.01 0.005 0.001 Upper tail 0.3 0.2 0.1 0.05 0.02 0.01 0.005 0.001
df 7 8.38 9.80 12.02 14.07 16.62 18.48 20.28 24.32 df 7 8.38 9.80 12.02 14.07 16.62 18.48 20.28 24.32
8 9.52 11.03 13.36 15.51 18.17 20.09 21.95 26.12 8 9.52 11.03 13.36 15.51 18.17 20.09 21.95 26.12
9 10.66 12.24 14.68 16.92 19.68 21.67 23.59 27.88 9 10.66 12.24 14.68 16.92 19.68 21.67 23.59 27.88
10 11.78 13.44 15.99 18.31 21.16 23.21 25.19 29.59 10 11.78 13.44 15.99 18.31 21.16 23.21 25.19 29.59
11 12.90 14.63 17.28 19.68 22.62 24.72 26.76 31.26 11 12.90 14.63 17.28 19.68 22.62 24.72 26.76 31.26

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 18 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 19 / 39
 Testing for goodness of fit using chi-square The chi-square distribution and finding areas Testing for goodness of fit using chi-square Finding a p-value for a chi-square test

Finding the tail areas in the year 2011 Back to Labby’s dice

The research question was: Do these data provide convincing

While probability tables are very helpful in understanding how evidence to suggest an inconsistency between the observed and
probability distributions work, and provide quick reference when expected counts?
computational resources are not available, they are somewhat The hypotheses were:
archaic. H0 : There is no inconsistency between the observed and the
expected counts. The observed counts follow the same
Using R: distribution as the expected counts.
HA : There is an inconsistency between the observed and the
> pchisq(q = 30, df = 10, lower.tail = FALSE)
expected counts. The observed counts do not follow the same
[1] 0.0008566412
distribution as the expected counts. There is a bias in which side
comes up on the roll of a die.
Using a web applet:
http:// www.socr.ucla.edu/ htmls/ SOCR Distributions.html We had calculated a test statistic of χ2 = 24.67.
All we need is the df and we can calculate the tail area (the
p-value) and make a decision on the hypotheses.

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 20 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 21 / 39

Testing for goodness of fit using chi-square Finding a p-value for a chi-square test Testing for goodness of fit using chi-square Finding a p-value for a chi-square test

Degrees of freedom for a goodness of fit test Finding a p-value for a chi-square test

The p-value for a chi-square test is defined as the tail area above the
calculated test statistic.

When conducting a goodness of fit test to evaluate how well the

observed data follow an expected distribution, the degrees of
p-value = P (χ2df =5 > 24.67)
freedom are calculated as the number of bins (k ) minus 1. df = 5
is less than 0.001
df = k − 1
0 24.67

For dice outcomes, k = 6, therefore

Upper tail 0.3 0.2 0.1 0.05 0.02 0.01 0.005 0.001
df = 6 − 1 = 5 df 1 1.07 1.64 2.71 3.84 5.41 6.63 7.88 10.83
2 2.41 3.22 4.61 5.99 7.82 9.21 10.60 13.82
3 3.66 4.64 6.25 7.81 9.84 11.34 12.84 16.27
4 4.88 5.99 7.78 9.49 11.67 13.28 14.86 18.47
5 6.06 7.29 9.24 11.07 13.39 15.09 16.75 20.52

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 22 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 23 / 39
 Testing for goodness of fit using chi-square Finding a p-value for a chi-square test Testing for goodness of fit using chi-square Finding a p-value for a chi-square test

Conclusion of the hypothesis test Turns out...

The 1-6 axis is consistently shorter than the other two (2-5 and
Clicker question 3-4), thereby supporting the hypothesis that the faces with one
We calculate a p-value less than 0.001. At 5% significance level, what and six pips are larger than the other faces.
is the conclusion of the hypothesis test? Pearson’s claim that 5s and 6s appear more often due to the
carved-out pips is not supported by these data.
(a) Reject H0 , the data provide convincing evidence that the dice are Dice used in casinos have flush faces, where the pips are filled in
fair. with a plastic of the same density as the surrounding material
(b) Reject H0 , the data provide convincing evidence that the dice are and are precisely balanced.
biased.
(c) Fail to reject H0 , the data provide convincing evidence that the
dice are fair.
(d) Fail to reject H0 , the data provide convincing evidence that the
dice are biased.

Suggested reading: http:// galton.uchicago.edu/ about/ docs/ labby09dice.pdf .

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 24 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 25 / 39

Testing for goodness of fit using chi-square Finding a p-value for a chi-square test Testing for goodness of fit using chi-square Finding a p-value for a chi-square test

Recap: p-value for a chi-square test Conditions for the chi-square test

The p-value for a chi-square test is defined as the tail area above
the calculated test statistic.
This is because the test statistic is always positive, and a higher
test statistic means a higher deviation from the null hypothesis.
1 Independence: Each case that contributes a count to the table
must be independent of all the other cases in the table.
2 Sample size / distribution: Just like for proportions, each
particular scenario (i.e. cell count) must have at least 10 cases.
Failing to check conditions may unintentionally affect the test’s error
rates.

p−value

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 26 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 27 / 39
 Testing for goodness of fit using chi-square Evaluating goodness of fit for a distribution Testing for goodness of fit using chi-square Evaluating goodness of fit for a distribution

2009 Iran Election Hypotheses

There was lots of talk of election fraud in the 2009 Iran election. We’ll
compare the data from a poll conducted before the election (observed
data) to the reported votes in the election to see if the two follow the
same distribution.
What are the hypotheses for testing if the distributions of reported and
Observed # of Reported % of polled votes are different?
Candidate voters in poll votes in election
Ahmedinajad 338 63.29%
Mousavi 136 34.10%
Minor candidates 30 2.61%
Total 504 100%

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 28 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 29 / 39

Testing for goodness of fit using chi-square Evaluating goodness of fit for a distribution Testing for goodness of fit using chi-square Evaluating goodness of fit for a distribution

Calculation of the test statistic Conclusion

Observed # of Reported % of Expected # of Clicker question

Candidate voters in poll votes in election votes in poll
Ahmedinajad 338 63.29% 504 × 0.6329 = 319 Based on these calculations what is the conclusion of the hypothesis
Mousavi 136 34.10% 504 × 0.3410 = 172 test?
Minor candidates 30 2.61% 504 × 0.0261 = 13
Total 504 100% 504 (a) p-value is low, H0 is rejected. The observed counts from the poll
do not follow the same distribution as the reported votes.
obs count − null count (b) p-value is high, H0 is not rejected. The observed counts from the
Zi = √
null count poll follow the same distribution as the reported votes.
(c) p-value is low, H0 is rejected. The observed counts from the poll
338 − 319 136 − 172 30 − 13 follow the same distribution as the reported votes
Z1 = √ = 1.06, Z2 = √ = −2.74, Z3 = √ = 4.71
319 172 13 (d) p-value is low, H0 is not rejected. The observed counts from the
poll do not follow the same distribution as the reported votes.
χ2df =2 = 1.062 + (−2.74)2 + 4.712 = 30.82

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 30 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 31 / 39
 Testing for independence in two way tables Testing for independence in two way tables

Popular kids Hypotheses

In the dataset popular, students in grades 4-6 were asked whether

good grades, athletic ability, or popularity was most important to them.
A two-way table separating the students by grade and by choice of
most important factor is shown below. Do these data provide evidence
to suggest that goals vary by grade?
H0 : Grade and goals are independent. Goals do not vary by grade.
HA : Grade and goals are dependent. Goals vary by grade.

Grades Popular Sports

4 63 31 25
5 88 55 33
6 96 55 32

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 32 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 33 / 39

Testing for independence in two way tables Expected counts in two-way tables Testing for independence in two way tables Expected counts in two-way tables

Expected counts in two-way tables Expected counts in two-way tables

Expected counts in two-way tables Clicker question

(row i total) × (column j total)
Expected Countrow i ,col j = What is the expected count for the highlighted cell?
table total
Grades Popular Sports Total
4 63 31 25 119
Grades Popular Sports Total 5 88 55 33 176
4 63 31 25 119 6 96 55 32 183
5 88 55 33 176 Total 247 141 90 478
6 96 55 32 183
Total 247 141 90 478
176×141
(a) 478
119×141
(b) 478
119 × 247 119 × 141 176×247
Erow 1,col 1 = = 61 Erow 1,col 2 = = 35 (c)
478 478 478
176×478
(d) 478

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 34 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 35 / 39
 Testing for independence in two way tables The chi-square test statistic for two-way tables Testing for independence in two way tables The chi-square test statistic for two-way tables

Calculating the test statistic in two-way tables Calculating the test statistic in two-way tables (cont.)

Expected counts are shown in (blue) next to the observed counts.

The calculation of the test statistic is exactly the same as before.
Grades Popular Sports Total
X (Oi ,j − Ei ,j )2
χ = 2 4 63 (61) 31 (35) 25 (23) 119
Ei ,j 5 88 (91) 55 (52) 33 (33) 176
{ let’s think for a second about why this works... } 6 96 (95) 55 (54) 32 (34) 183
Total 247 141 90 478
Then calculate the degrees of freedom

df = (R − 1) × (C − 1) X (63 − 61)2 (31 − 35)2 (32 − 34)2

χ2 = + + ··· + = 1.3121
where R is the number of rows, and C is the number of columns. 61
35 34
Note: We calculate df differently for one way and two-way tables. df = (R − 1) × (C − 1) = (3 − 1) × (3 − 1) = 2 × 2 = 4

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 36 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 37 / 39

Testing for independence in two way tables The chi-square test statistic for two-way tables Testing for independence in two way tables The chi-square test statistic for two-way tables

Calculating the p-value Conclusion

Clicker question

Which of the following is the correct p-value for this hypothesis test?

(a) more than 0.3

Do these data provide evidence to suggest that goals vary by grade?
(b) between 0.3 and 0.2
df = 4
(c) between 0.2 and 0.1 H0 : Grade and goals are independent. Goals do not vary by grade.
(d) between 0.1 and 0.05 HA : Grade and goals are dependent. Goals vary by grade.
0 1.3121 (e) less than 0.001

Upper tail 0.3 0.2 0.1 0.05 0.02 0.01 0.005 0.001
df 1 1.07 1.64 2.71 3.84 5.41 6.63 7.88 10.83
2 2.41 3.22 4.61 5.99 7.82 9.21 10.60 13.82
3 3.66 4.64 6.25 7.81 9.84 11.34 12.84 16.27
4 4.88 5.99 7.78 9.49 11.67 13.28 14.86 18.47
5 6.06 7.29 9.24 11.07 13.39 15.09 16.75 20.52

Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 38 / 39 Statistics 101 (Prof. Rundel) L15: Chi square October 25, 2011 39 / 39