Applied Statistics

Chapter 3
The story of Differences and comparisons
Instructor: Ms. Umme Siddiqa
 Comparing two means
 Sometimes researchers want to look at differences between groups of people or between
people in different treatment conditions.
 The simplest form of experiment is one in which we split the sample into an experimental
group and a control group that is identical to the experimental group in all respects except
the one expected to have an impact on the outcome.
 For example, we might also want to compare statistics knowledge before and after a
lecture. One group, the experimental group, has to sit through the lecture, while the other
group, the control group, gets to miss it and stay in bed.
Comparing two means

 Here are some other examples of scenarios where we’d compare two conditions:
 Is the movie Scream 2 scarier than the original, Scream? We could measure heart rates
(which indicate anxiety) during both films and compare them.
 Is your work better when you do it listening to some favorite music? You could get some
people to write an essay (or book) listening to their favorite music, and then write a
different one in silence (this is a control) and compare the grades.
 Do diet pills work? We take two groups of people and randomly assign one group to a
program of diet pills and the other group to a program of sugar pills (which they believe
will help them lose weight). If the people who take the diet pills lose more weight than
those on the sugar pills we can infer that the diet pills caused the weight loss.
The t test

When testing the difference between two means, researchers tend to apply the t-statistic also
called the t-test (Student,1908).
In this section we’ll look at the theoretical underpinnings of the test. There are two variants of
this test:
 Independent t-test: This test is used when you want to compare two means that come from
conditions consisting of different entities (this is sometimes called the independent-
measures or independent-means t-test).
 Paired-samples t-test: This test, also known as the dependent t-test, is used when you want
to compare two means that come from conditions consisting of the same or related entities
Rationale for t test

 Two samples of data are collected and the sample means calculated. These means might
differ by either a little or a lot.
 If the samples come from the same population, then we expect their means to be roughly
equal. Although it is possible for their means to differ because of sampling variation, we
would expect large differences between sample means to occur very infrequently.
Comparing more than two means

 Analysis of variance (ANOVA) is a hypothesis-testing procedure for studies with three or

more groups.
 One continuous variable is being studied across a categorical variable having three or more
categories.
 It differs from t-test in which the categorical variable has two categories.
 The mean difference can be calculated by analyzing variance.
One Way ANOVA

 Analysis of variance (ANOVA) is a hypothesis-testing procedure for studies with three or

more groups.
 One continuous variable is being studied across a categorical variable having three or more
categories.
 It differs from t-test in which the categorical variable has two categories.
 The mean difference can be calculated by analyzing variance.
 For example, if you want to check whether there is a difference in life satisfaction among
different groups of ethnic backgrounds. (i.e. Punjabi, sindhi, Balochi, pathan, Kashmiri,
Balti)
Two Way Independent design

 Two-way independent designs compare several means when there are two independent
variables and different entities have been used in all experimental conditions.
 For example, if you wanted to know whether different teaching methods worked better for
different topics, you could take students from four courses (Psychology, Geography,
Management, and Statistics) and assign them to either lecture-based or book-based
teaching. The two variables are topic and method of teaching. The outcome might be the
end-of-year mark (as a percentage).
Two Way Repeated Measures ANOVA

 When there is Two independent variables both measured using the same entities, it is a
Two way repeated measures ANOVA
 For example,
 Suppose we want to check whether combination of different types of Movies genre
(Comedy, Drama, Action) and location where you are watching (Home, Theatre) have an
impact on satisfaction of movie experience.
 It is repeated measure because same participants will go through all the conditions.
 Mixed Factorial ANOVA

 When Two independent variables: one measured using different entities and the other
measured using the same entities it is a Two way mixed ANOVA.
 For example, If we want to check whether Gender of the participant (Male, Female) and
birth order (First, middle, last) have an effect on the rebellious behaviors.
Different factorial designs

 Three independent variables all of which are measured using different entities. (three way
ANOVA)
 Comparisons using Factorial Designs
 The principles of the linear model extend to MANOVA in that we can use MANOVA when there
is one independent/predictor variable or several, we can look at interactions between outcome
variables, and we can do contrasts to see which groups differ.
 When we have only one outcome variable the model is known as univariate (meaning ‘one
variable’),
 When we include several outcome variables simultaneously the model is multivariate (meaning
‘many variables’).
 For example, we might be able to distinguish people who are married, living together or single
by their happiness. ‘
 Happiness’ is a complex construct, so we might want to measure their happiness with work, socially,
sexually and within themselves (self-esteem). It might not be possible to distinguish people who are
married, living together or single by only one aspect of happiness (which is what a univariate model
tests), but these groups might be distinguished by a combination of their happiness across all four
domains (which is what a MANOVA tests)
Planned Contrasts

 Comparison in which the particular means to be compared were decided in advance.

 Researchers use planned contrasts to look at some particular, focused differences between
groups that directly follow from a theory or that are related directly to some practical
application.
 Planned contrasts are also sometimes called a priori comparisons because they have been
planned in advance (a priori) of the study.
 They may also be called planned comparisons because they were planned in advance.
Post-Hoc Analysis

 Post-hoc (Latin, meaning “after this”) means to analyze the results of your experimental
data. 
 Multiple comparisons, not specified in advance; procedure conducted as part of an
exploratory analysis after an analysis of variance.
 In planned contrasts, researchers often plan specific comparisons based on theory or
practical considerations.
 In post hoc analysis, researchers take a more exploratory approach, for example,
comparing all the different pairings of means to discover which ones do and do not differ
significantly.
Post-hoc Analysis

 There are two ways to check post hoc analysis.

 Firstly if your data is normally distributed.
 Secondly if your data is not normally distributed.
 SPSS gives you 18 types of different post hoc analyses.
Bonferonni Procedure

 The idea of the Bonferroni procedure is that you use a more stringent significance level for
each contrast.
 if you are making several contrasts (comparisons), each at the .05 level, the chance of any one
of them coming out significant is more than .05.

 In fact, if you make two contrasts, each at the .05 significance level, there is about a .10 chance
that at least one will come out significant just by chance
 So in order to deal with this problem we use Bonferonni’s Procedure and divide the alpha level
with no of comparisons.
α
n
Scheffé Test

 Method of figuring the significance of post hoc comparisons that takes into account all
possible comparisons that could be made.
 It is used with unequal sample size.
Tukey’s Test

 The purpose of Tukey’s test is to figure out which groups in your sample differ.

 It uses the “Honest Significant Difference,” a number that represents the distance between
groups, to compare every mean with every other mean.
 It is used when sample size of each group is equal.
Other common tests

 If sample sizes are slightly different then use Gabriel’s,

 But if sample sizes are very different use Hochberg’s GT2.