Understanding

Statistical Inferences
Week 9

SSR3996 Research Project

Discussion Topics

Inferential statistics
Purpose
Error
Terminology
Hypothesis testing
Inferential tests
Criteria for evaluating the inferential
statistics reports in studies

SSR3996 Research Project

Inferential Statistics

The purpose of inferential statistics is to

draw inferences about a population on the
basis of an estimate from a sample
Inferential statistics - specific statistical
procedures that accomplish this purpose
The ultimate goal is to draw accurate
conclusions about the population

SSR3996 Research Project

Inferential Statistics
Two types of errors
Sampling errors
Without measuring the entire population, the results can
be inaccurate due to sampling error
The larger the proportion of the population that is sampled, the
lower the sampling error; the smaller the proportion of the
population that is sampled, the higher the sampling error
A sample of 99% of a population is likely to show results that
are very, very similar to those that would have been found if
everyone in the population was measured
A sample of 1% is likely to show results that are different from
those in the population - the question is how different are the
sample results

SSR3996 Research Project

Inferential Statistics
Measurement errors
Regardless of the sample size, the results can be
inaccurate due to measurement error
Lack of validity
Lack of reliability
Need to estimate the level of measurement error
relative to the inferences being drawn

SSR3996 Research Project

Inferential Statistics
Terminology
Null hypothesis
No differences between groups
No relationships between variables
Level of significance
Probability of being wrong in rejecting the null hypothesis
Known as alpha ()

Types of errors
Type I - rejecting the null hypothesis when it is true
Type II - not rejecting (i.e., accepting) the null hypothesis
when it is not true

SSR3996 Research Project

Inferential Statistics
Hypothesis testing exemplified with an experimental
control group comparison
The five stages of the process
State the null hypothesis - no difference between the mean
scores for the experimental and control groups
Assume the null hypothesis is true to establish a base from
which the statistician can work

SSR3996 Research Project

Inferential Statistics
The five stages of the process (continued)
Calculate the observed difference between the mean

scores for the two groups

Compare the observed difference between mean
scores to the sampling distribution of the test statistic
Accept or reject the null hypothesis based on this
comparison
If the observed difference is typical of the sampling
distribution, the null hypothesis is likely true and it is accepted
If the observed difference is atypical of the sampling
distribution, the null hypothesis is likely untrue and it is
rejected.

SSR3996 Research Project

Inferential Statistics
Issues related to statistical and practical
significance
Statistical significance
The typical or atypical nature of the comparison of the
observed difference to the sampling distribution can be
estimated using statistical theory
The estimate is the probability of being wrong in rejecting
the null hypothesis
It is stated as p = x where x is the specific probability of
the comparison (e.g., p = .001, p = .042, p = .56) or as p
< y where y is the alpha level (e.g., .10, .05, .01)

SSR3996 Research Project

Inferential Statistics
There is always the possibility of making a mistake
given that this is based on a probability model
Type I error - deciding to reject the null hypothesis when in
reality it is true
Type II error - accepting the null hypothesis when in reality it is
false
Typical levels of significance in education - .10, .05, and
.01
Factors affecting the level of significance
The actual differences between the groups
The degree to which sampling and measurement errors exist
The size of the sample

SSR3996 Research Project

Inferential Statistics

Null hypothesis Null hypothesis

(H0) is true (H0) is false
Reject null Type I error Correct outcome
hypothesis
Fail to reject null Correct outcome Type II error
hypothesis

SSR3996 Research Project

Inferential Statistics
Practical significance
Practical significance is related to the importance and
usefulness of the results
Estimates of practical significance
For correlations the coefficient of determination (i.e., r2) is
used
For comparisons an effect size is used
Effect size is the difference between two group means in
terms of the control group standard deviation
Evaluating effect sizes small (.30), moderate (.50), and
large (.75)

SSR3996 Research Project

Inferential Statistics
Each consumer of the research must judge the
balance between the statistical significance
and the practical significance of the statistical
results given the context in which the results
might be used

SSR3996 Research Project

Inferential Tests

Two types of inferential tests

Parametric - inferential procedures using
interval or ratio level data
Non-parametric - inferential procedures using
nominal or ordinal data

SSR3996 Research Project

Parametric Tests
T-test
A comparison of the means for two groups
Do the mean scores on the final exam differ for the
experimental and control groups?
Independent samples t-test - compares the means of
two separate groups on one variable
Posttest means for Group 1 and Group 2
Dependent sample t-test - compares the means of two
variables for one group
Pre-test and posttest means for Group 1

SSR3996 Research Project

Parametric Tests
Independent samples t-test
Group N Mean Std. t value Sig (2-
Deviation tailed)
Treatment 64 61.25 23.40 0.639 0.524
Control 63 58.57 23.82

Group N Mean Std. t value Sig (2-

Deviation tailed)
Treatment 64 46.01 29.94 2.295 0.024**
Control 63 35.90 18.44
= 0.05

SSR3996 Research Project

Parametric Tests
Dependent sample t-test
Konstruk Masa Min Sisihan Nilai t Aras
Piawai Signifikan
Motivasi dalaman Pra 3.762 0.563 - 0.907 0.366
Pasca 3.813 0.550
Motivasi luaran Pra 4.105 0.630 0.137 0.891
Pasca 4.097 0.622
Nilai tugasan Pra 3.806 0.714 - 0.520 0.604
Pasca 3.835 0.619
Kepercayaan Pra 4.126 0.519 1.570 0.119
Kawalan Pasca 4.039 0.584
pembelajaran
Keberkesanan Diri Pra 3.502 0.622 - 3.451 0.001**
untuk Pembelajaran Pasca 3.667 0.496
dan Prestasi
Kebimbangan Ujian Pra 3.262 0.897 -4.793 0.000**
Pasca 3.598 0.765
= 0.05
SSR3996 Research Project
Parametric Tests
T-test (continued)
A determination of whether a relationship exists
Does a correlation of +.63 between students math
attitudes and math achievement indicate a
relationship exists between these two variables?
Correlation t-test - compares the magnitude of the
difference between a correlation coefficient and 0.00

SSR3996 Research Project

Parametric Tests
Analysis of variance (ANOVA)
A comparison of the means for two or more
groups
Omnibus ANOVA - a procedure that indicates
whether one of more pairs of means are different
Do the mean scores differ for the groups using co-
operative group, lecture, or web-based
instruction?

SSR3996 Research Project

Parametric Tests
ANOVA (continued)
Multiple comparisons (i.e., post-hoc)
Procedures that indicate which specific pairs of means are
different as a follow-up to a significant omnibus ANOVA
result
Do the mean scores differ between the co-operative group

and lecture, co-operative group and web-based, and lecture

and web-based instruction?
Two common tests
Tukey
Scheffe

SSR3996 Research Project

Parametric Tests
Factorial ANOVA
A procedure that analyzes the difference between groups
across two or more independent variables
Do the mean scores differ for co-operative group, lecture,
and web-based instruction for males and females?
Effects
Main effects - differences between the levels of each
independent variable
Interaction effects - differences between combinations of
the levels of each independent variable

SSR3996 Research Project

Parametric Tests
Presenting result from One-way ANOVA with Post-Hoc Test
Sum of Squares df Mean Square F Sig.
Between groups 5.03 2 2.51 7.74 .000*
Within groups 392.23 1208 .33
Total 397.26 1210

SSR3996 Research Project

Parametric Tests
Dependent variable: Total Self-Esteem
Scheffe
(I) AGE (J) AGE Mean Std. Sig. 95% Confidence
Difference Error Interval
(I-J)
Lower Upper
Bound Bound

Young Middle - .10656* .04080 .033 - .2066 - .0066

Old - .15141* .03935 .001 - .2479 - .0550
Middle Young .10656* .04080 .033 .0066 .2066
Old - .04485 .04037 .540 - .1438 .0541
Old Young .15141* .03935 .001 .0550 .2479
Middle .04485 .04037 .540 - .0541 .1438
* The mean difference is significant at the .05 level.

SSR3996 Research Project

Parametric Tests
Analysis of covariance (ANCOVA)
A procedure that compares means after
statistically adjusting them for pretest differences
between groups
Very stringent assumptions that must be met to
use this procedure
Adjusts for small to moderate - not large - pretest
differences

SSR3996 Research Project

Parametric Tests
Multivariate statistics
Comparisons or relationships involving two or more
dependent variables
Comparison of means
Are there differences in the attitudes and performances of
students being taught with lecture or web-based
instruction?
Specific tests
Multivariate ANOVA (MANVOA)
Multivariate ANCOVA (MANCOVA)
Hotellings T

SSR3996 Research Project

Parametric Tests

Multivariate statistics (continued)

Relationships
Are students affective traits (e.g., attitudes, self-
esteem, preferences, etc.) predictive of their
knowledge (i.e., test scores) and skills (i.e.,
performances)?
Canonical correlation

SSR3996 Research Project

Parametric Tests
IV Category DV Category Parametric
test
1 (Gender) Male 1 (Test score) Independent
t-test
Female
1 (Group: 1 (Test score) Pretest Paired-
Treatment) sample t-
Posttest
test
1 (Ethnic) Malay 1 (Test score) One-way
ANOVA
Chinese
Indian
2 (Gender Male Malay 1 (Test score) Two-way
& Ethnic) ANOVA
Female Chinese
Indian

SSR3996 Research Project

Parametric Tests

IV Category DV Category Parametric

test
1 (Gender) Male 2 (Test score & One-way
motivation) MANOVA
Female
2 (Gender Male Control 2 (Test score & MANOVA
& Group) motivation)
Female Treatment

SSR3996 Research Project

Non-Parametric Tests
Chi-square - differences in frequencies
across different categories
Do male and female teachers differ in the grades
they teach?
Do the percentages of undergraduate, graduate,
and doctoral students differ in terms of their
support for the new class attendance policy?

SSR3996 Research Project

An example of how to
manually calculate chi-square
Place all data into crossbreak/contingency tables and calculate the
expected frequency for each cell
Grade 4 Grade 5 Grade 6 Grade 7 Total
O(E)
Male 10(20) 20(20) 20(20) 30(20) 80
Female 40(30) 30(30) 30(30) 20(30) 120
Total 50 50 50 50 200

O Observed frequency
E Expected frequency for each cell = (total for each column/total)
x total for each row

SSR3996 Research Project

Calculating Chi-square - continue
Calculate the difference between the expected and the
observed value (O-E) for each cell
Square the result (O-E)2
Divide the result by the expected value
(O-E)2
E
Add the result of all cells. This value is called chi-square value
(2)
To determine whether this is statistically significant, calculate
the degrees of freedom and look up the critical value using
chi-square probability table. If the value is more than the
critical value, then it is statistically significant.

SSR3996 Research Project

Calculating chi-square - continue
Example
O E OE (O E)2 (O E)2
E
10 20 -10 100 100/20 = 5.00

40 30 10 100 100/30 = 3.33

20 20 0 0 0
30 30 0 0 0
20 20 0 0 0
30 30 0 0 0
30 20 10 100 100/20 = 5.00

20 30 -10 100 100/30 = 3.33

2
16.66

SSR3996 Research Project

Calculating chi-square - continue
To calculate degrees of freedom (df)
No. of rows 1 = x
No. of columns 1 = y
Multiply (x)(y)
e.g. 2 1 = 1
4 1 = 3, so df = 3
To calculate the contingency coefficient, use the formula

C= = 16.66/(16.66 + 200) = .28

Contingency coefficient gives the strength of correlation which

shows the practical significance.
SSR3996 Research Project
SSR3996 Research Project
Non-Parametric Tests

Comparison of means
Mann Whitney U-test
Wilcoxon test
Kruskal-Wallis ANOVA
Relationships
Spearman r

SSR3996 Research Project

Evaluation Criteria
Basic descriptive statistics are needed to
evaluate the inferential results
Inferential analyses report statistical
significance, not practical significance
Inferential analyses do not indicate internal or
external validity
The results depend on sample sizes

SSR3996 Research Project

Evaluation Criteria

The appropriate statistical procedures are

used
The level of significance is interpreted
correctly
Caution is used to interpret non-parametric
results from studies with few subjects in
one or more groups or categories

SSR3996 Research Project

Problem solving activity
What is the inferential test that is needed for
a. testing whether there is a difference between the mean scores of
boys and girls groups within the same class
b. testing whether there is a difference between the mean scores of
two different groups of randomly selected samples
c. testing whether there is a difference between the mean scores of
the same group before and after a treatment
d. testing whether there is a difference between the mean scores of
three groups
e. testing whether there is a difference between the perception of
male and female teachers about a particular school policy
f. testing whether there is a relationship between attitudes towards
physics and physics achievement
g. testing whether there is a relationship between math scores and
physics scores

SSR3996 Research Project