ion

at
uc
Ed
n
BEHAVIORAL VARIABILITY

so
AND RESEARCH

ar
Pe
of
Variability and the Research Process emat Variance Across
Meta-Analysis: Systematic
Variance: An Index of Variability Studies

ion
Systematic and Error Variance on for Sys
The Question Systematic Variance
Effect Size: Assessing the Strength of Relationships

iss
rm
Psychologists use the word schema ma to refer
ref to a cognitive generalization that organizes
and guides the processing of information.
rmatio You have schemas about many categories of
nformation
pe

events, people, and other stimulimuli

uli that
tha you
y have encountered in life. For example, you
probably have a schema forr the the concept
con
co leadership. Through your experiences with
leaders of various sorts, you ou have
ha developed a generalization of what a good leader is.
ior

Similarly, you probably ly

y have a schema
s for big cities. What do you think of when I say,
“New York, Los Angeles,ngeles,
geles, and
an Atlanta”? Some people’s schemas of large cities include
pr

generalizations suchsuchh as “crowded

“cr and dangerous,” whereas other people’s schemas
tess such as “interesting and exciting.” We all have schemas about many
include attributes
t

categories off stimuli.

imuli
ou

archers
hers have
Researchers ha found that people’s reactions to particular stimuli and events are
stronglyy affected by the schemas they possess. For example, if you were a business
th

utive,
tive, your
executive, yo
you decisions about whom to promote to a managerial position would be
ected
ted by your schema for leadership. You would promote a very different kind of
affected
wi

employee
ployee to manager if your schema for leadership included attributes such as caring,
employee
involv d and people-oriented than if you saw effective leaders as autocratic, critical, and
involved,
n

aloof Similarly, your schema for large cities would affect your reaction to receiving a job
aloof.
tio

off in Miami or Dallas.

offer
Importantly, when people have a schema, they more easily process and organize
bu

information relevant to that schema. Schemas provide us with frameworks for organizing,
remembering, and acting on the information we receive. It would be difficult for
executives to decide whom to promote to manager if they didn’t have schemas for
tri

leadership, for example. Even though schemas sometimes lead us to wrong conclusions
dis

when they are not rooted in reality (as when our stereotypes about a particular group bias
our perceptions of a particular member of that group), they allow us to process
information efficiently and effectively. If we could not rely on the generalizations of our
or
tf
no

From Chapter 2 of Introduction to Behavioral Research Methods, Sixth Edition. Mark R. Leary. Copyright © 2012 by
le

Pearson Education, Inc. All rights reserved.

Fi


 ion
at
uc
Behavioral Variability and Research

Ed
schemas, we would have to painstakingly consider VARIABILITY AND THE RESEARCH
every new piece of information when processing PROCESS

n
information and making decisions.
All aspects of the research process revolve ve around
round

so
By now you are probably wondering how
schemas relate to research methods. Having taught the concept of variability. The concept off variability
variabil
courses in research methods and statistics for many runs through the entire enterprise of designing
esigning and

ar
years, I have come to the conclusion that, for most analyzing research. To show what I mean, let me
ability is central to
describe five ways in which variability

Pe
students, the biggest stumbling block to under-
standing behavioral research is their failure to the research process.
develop a schema for the material. Many students
1. Psychology and d other behavioral sci-

of
have little difficulty mastering specific concepts and
procedures, yet they complete their first course in ences involve the study y of beha
behavioral variability.
research methods without seeing the big picture. efined
Psychology is often defined ned aas the study of behavior

ion
They learn many concepts, facts, principles, designs, and mental processes. es. However,
Howeve what psychologists
Howe
analyses, and skills, but they do not develop an and other behavioralioral
ral rese
resear
researchers actually study is
iability.
bility. Th
behavioral variability. That is, they want to know

iss
overarching framework for integrating and organi-
zing all of the information they learn. Their lack of a how and why hy behavior
behavio varies across situations,
beha
schema impedes their ability to process, organize, ople,
among people, le, and over
ov time. Put differently, under-
rm
remember, and use information about research standingg behavior
behavio anda mental processes really means
methods. In contrast, seasoned researchers have a rstanding
tanding what
understanding w makes behavior, thought, and
otion
ion var
emotion vary.
pe

well-articulated schema for the research process that

facilitates their research activities and helps them to Th
Think about the people you interact with each
make methodological decisions. day aand
nd about
a the variation you see in their behav-
ior

The purpose of this chapter is to provide you ior. F

Fir
First, their behavior varies across situations.
with a schema for thinking about the research ch Peo
Peop
People feel and act differently on sunny days than
wh
when it is cloudy, and differently in dark settings
pr

process. In essence, this chapter will give you pegs egs

gs
on which to hang what you learn about behavioral avior
vio l than when it is light. College students are often
research. Rather than dumping all of the new ew infor- more nervous when interacting with a person of the
t

other sex than when interacting with a person of

ou

mation you learn in a big heap on the floor, r, we’ll pput

schematic hooks on the wall for you to use in or org
organ- their own sex. Children behave more aggressively
izing the incoming information. after watching violent TV shows than they did
th

The essence of this schema iss that,hat, at the most before watching them. A hungry pigeon that has
been reinforced for pecking when a green light
wi

basic level, all behavioral researchsearch

earch aattempts to
vioral
answer questions about behavioral ora variability—that
var is on pecks more in the presence of a green
is, how and why behavior varies aries aac
across situations, light than a red light. In brief, people and other
n

differs among individuals, ls, and cch

changes over time. animals behave differently in different situations.
tio

Behavioral researchers are interested in how and

why features of the situation cause this variability
bu

in behavior, thought, and emotion.

Second, behavior varies among individuals.
Even in similar situations, not everyone acts the
tri

same. At a party, some people are talkative and

dis

outgoing, whereas others are quiet and shy. Some

people are more conscientious and responsible than
others. Some individuals generally appear confident
or
tf
no
le
Fi



 ion
at
uc
Behavioral Variability and Research

Ed
and calm whereas others seem nervous. And certain research studies must be designed in a way tthat
animals, such as dogs, display marked differences in allows us to identify, as unambiguously sly as ppossible,

n
behavior depending on their breed. Thus, because of factors related to the behaviorall variabi variability we

so
differences in their biological makeup and previous ll-designe
-design study is
observe. Viewed in this way, a well-designed
experience, different people and different animals one that permits researchers to describe
escribe and account
behave differently. A great deal of behavioral havior
for the variability in the behaviorvior oof their research

ar
research focuses on understanding this variability participants. A poorly designedgned
ned stud
study is one in which

Pe
across individuals. researchers have difficulty ty answe
answer
answering questions about
Third, behavior also varies over time. A baby the source of variability y they ob
observe.
who could barely walk a few months ago can run Flaws in thee des design
design of a study can make

of
today. An adolescent girl who two years ago it impossible forr a researcher
rese to determine why
thought boys were “gross” now has romantic participants behav
behaved ed as they did. At each step
fantasies about them. A task that was interesting an of the design n and exe
ex
execution of a study, researchers

ion
hour ago has become boring. Even when the situa- must be sure that th their research will permit
tion remains constant, behavior may change as time them to o answer ttheir questions about behavioral

iss
passes. Some of these changes, such as develop- bility.
lity.
variability.
mental changes that occur with age, are permanent; 4. Th
The measurement of behavior involves the
other changes, such as boredom or sexual drive, are assessmen
ssessm
assessment of behavioral variability. All behav-
rm
temporary. Behavioral researchers are often inter- ioral rres
research involves the measurement of some
ested in understanding how and why behavior varies beha
behavi
behavior, thought, emotion, or physiological process.
pe

over time. O measures may involve the number of times a rat

Our
2. Research questions in all behavioral oral
ra pr
presses a bar, a participant’s heart rate, the score a child
sciences are questions about behaviorall vari- obtains on a memory test, or a person’s rating of how
ior

ability. Whenever behavioral scientistss design tired he or she feels on a scale of 1 to 7. In each case,
research, they are interested in answering ering
ng que
ques- we’re assigning a number to a person’s or animal’s
pr

tions about behavioral variability (whether hether

ether th
they behavior: 15 bar presses, 65 heartbeats per minute, a
think about it that way or not). Forr exa exam ple sup-
ple,
example, test score of 87, a tiredness rating of 5, or whatever.
t

pose we want to know the extent nt to whi

which sleep No matter what is being measured, we want
ou

deprivation affects performancee on cog cognitive tasks the number we assign to a participant’s behavior to
(such as deciding whether a blip on a radar screen correspond in a meaningful way to the behavior
th

ncoming
coming eenemy aircraft).
is a flock of geese or an incoming being measured. Put another way, we would like the
In essence, we are asking ng how the
th amount of sleep variability in the numbers we assign to various
wi

people get causes theirr performance

perfor
perform on the task to participants to correspond to the actual variability in
change or vary. Or imagin
magine tthat we’re interested in
imagine participants’ behaviors, thoughts, emotions, or
n

whether a particularular
lar form of counseling reduces physiological reactions. We must have confidence
family conflict.t.. Our re
res
research centers on the ques- that the scores we use to capture participants’ res-
tio

ether
tion of whether her co
cou
counseling causes changes or ponses reflect the true variability in the behavior we
variation in a fam
fami
family’s interactions. Any specific are measuring. If the variability in the scores does
bu

researchh question we might develop can be phrased not correspond, at least roughly, to the variability in
ms of behavioral
in terms beh variability. the attribute we are measuring, the measurement
tri

3. Re
Res
Research should be designed in a manner technique is worthless and our research is doomed.
dis

that
hat best aallows the researcher to answer questions 5. Statistical analyses are used to describe
about
bout bbehavioral variability. Given that all behav- and account for the observed variability in the
io ral research involves understanding variability,
ioral behavioral data. No matter what the topic being
or
tf
no


 ion
at
uc
Behavioral Variability and Research

Ed
investigated or the research strategy being used, In brief, the concept of variability accompanies
one phase of the research process always involves us through the entire research process: Our research earch
rch

n
analyzing the data that are collected. Thus, the questions concern the causes and correlates off behav-

so
study of research methods necessarily involves an ioral variability. We try to design studies that at best hel
help
introduction to statistics. Unfortunately, many stu- us to describe and understand variability in n a particu
partic
particular
dents are initially intimidated by statistics and behavior. The measures we use are ann attem attempt to

ar
sometimes wonder why they are so important. The capture numerically the variability y we ob obs
observe in

Pe
reason is that statistics are necessary for us to tistics hel
participants’ behavior. And our statistics help us to ana-
understand behavioral variability. lyze the variability in our data to answer
answ the questions
After a study is completed, all we have is a set we began with. Variability is trulyuly th
thee thread that runs

of
of numbers that represent the responses of our ocess.
throughout the research process. ess. Un
Understanding vari-
research participants. These numbers vary, and our ability will provide you with a sch
schema for understand-
goal is to understand something about why they vary. ing, remembering, and d applying
pplyin what you learn about

ion
The purpose of statistics is to summarize and answer behavioral research. For this rrea
reason, we will devote the
questions about the behavioral variability we observe hapter to the
remainder of this chapter t topic of variability.

iss
in our research. Assuming that the research was com-
petently designed and conducted, statistics help us
account for or explain the behavioral variability we VARIANCE:
NCE:
CE: AAN INDEX OF
rm
observed. Does a new treatment for depression cause
VARIABILITY
ABILITY
BILIT
an improvement in mood? Does a particular drug
pe

enhance memory in mice? Is self-esteem related to Given

venn the importance
im
imp of the concept of variability in
the variability we observe in how hard people try designing
des gning and
igning an analyzing behavioral research, researchers
when working on difficult tasks? We use statistics to need a w way to express how much variability there
ior

answer questions about the variability in our data. is in a set of data. Not only are researchers interested
Statistics serve two general purposes for or sim
simp
simply in knowing the amount of variability in their
pr

researchers. Descriptive statistics are used to da

data, but also they need a numerical index of the vari-
summarize and describe the behavior of partici rti
partici- ability in their data to conduct certain statistical analyses.
pants in a study. They are ways of reducing ducing
cing a Researchers use a statistic known as variance to indi-
t
ou

large number of scores or observations to interpr interpre- cate the amount of observed variability in partici-
table numbers such as averages and percentages. percenta pants’ behavior.
Imagine that you conducted a very simple
th

Inferential statistics, on the other han hand, are

used to draw conclusions about the reliability
iability and gen- study in which you asked 6 participants to describe
wi

eralizability of one’s findings. They hey

ey are uused to help their attitudes about capital punishment on a scale of
answer questions such as, How likely ke y is it
i that my find- 1 to 5 (where 1 indicates strong opposition and 5
aneous
ings are due to random extraneous eous fa
factors rather than indicates strong support for capital punishment).
n

to the variables of centrall interest in my study? How Suppose you obtained:

tio

ndings
dings of the larger population
representative are my findings
from which my sample ple of part
participants came? Participant Response
bu

Descriptive and infere

inferential statistics are simply
rchers
tools that researchershers use to interpret the behavioral 1 4
tri

lect.
data they collect.ct. Bey
Beyo
Beyond that, however, understand- 2 1
3 2
css provides
ing statistics provide insight into what makes some
dis

4 2
research studies
udies bebetter than others. As you learn about
5 4
tatistical
tical aanalyses are used to study behavioral
how statistical 6 3
ability,
bility, yyou’ll develop a keener sense of how to
variability,
or

esign
sign pow
design powerful, well-controlled studies.
tf
no
le
Fi



 ion
at
uc
Behavioral Variability and Research

Ed
For a variety of reasons, you may need to know how the variability in Figure 1 (b). That is, most of the
much variability there is in these data. Can you think scores in 1(a) are more tightly clustered tered
red tto
together

n
of a way of expressing how much these responses, than the scores in 1(b), which are more ore spre
spr
spread out.

so
or scores, vary from one person to the next? What we need is a way of expressing sing
ng varia
vari
variability that
includes information about all of the sc sco
scores.
When we talk about things ings
gs vary
varyi
varying, we usually

ar
do so in reference to somee standar
standard. A useful stan-
A Conceptual Explanation of Variance

Pe
dard for this purpose is the aver average or mean of the
One possibility is simply to take the difference between scores in our data set. t. Re
Resea
Researchers use the term
the largest and the smallest scores. In fact, this mean as a synonym m for w h you probably call the
what

of
number, the range, is sometimes used to express vari- average—the sum m of a seset of scores divided by the
ability. If we subtract the smallest from the largest number of scores res yyou
ou hhave.
score above, we find that the range of these data is The meaneann stands
stan as a fulcrum around which all

ion
3 (4 - 1 = 3). Unfortunately, the range has limitations as her
of the other er scores bbalance. So we can express the
an indicator of the variability in our data. The problem lity
variability y in ouour data in terms of how much the

iss
is that the range tells us only how much the largest and scoreses vary arou
ar
around the mean. If most of the scores in
smallest scores vary but does not take into account the ett of data are
a set ar tightly clustered around the mean (as
other scores and how much they vary from each other. in Figure 11[a]), then the variance of the data will be
rm
Consider the two distributions of data in small. IIf, however, our scores are more spread out (as
Figure 1. These two sets of data have the same in F
Figu
Figure 1[b]), they will vary a great deal around
pe

range. That is, the difference between the largest and d the m
mean, and the variance will be large. So, the vari-
smallest scores is the same in each set. However, th the an
ance is nothing more than an indication of how
variability in the data in Figure 1 (a) is smallerer than tightly or loosely a set of scores clusters around the
ior
pr

6 6
t

5 5
ou

4 4
th
Frequency

Frequency

3 3
wi

2 2
n
tio

1 1
bu

0 0
Scores Scores
(a) (b)
tri

FIGURE 1 Distributions with Low and High Variability. The two sets of data shown in these
dis

graphs contain the same number of scores and have the same range. However, the variability in
the scores in Graph (a) is less than the variability in Graph (b). Overall, most of the participants’
scores are more tightly clustered in (a)—that is, they vary less among themselves (and around
or

the mean of the scores) than do the scores in (b). By itself, the range fails to reflect the
difference in variability in these two sets of scores.
tf
no
le
Fi


 ion
at
uc
Behavioral Variability and Research

Ed
mean of the scores. As we will see, this provides a Step 3. By looking at these deviation scores, we can
very useful indication of the amount of variability in see how much each score varies or deviates viates
ates

n
a set of data. And, again, we need to know how much from the mean. Participant 2 scores furthest urthest

so
variability there is in our data in order to answer from the mean (1.67 units below thee mean mean)
mean),
questions about the causes of that variability. whereas Participant 6 scores closest losest
sest to the
mean (0.33 unit above it). Note otee that a posi-

ar
A Statistical Explanation of Variance tive number indicates that thee person
perso
person’s score

Pe
fell above the mean, whereas ereas a ne
neg
negative sign
You’ll understand more precisely what the variance
(-) indicates a score below low tthe mean. (What
tells us about our data if we consider how variance is
would a deviation scorecore
ore oof ze
zero indicate?)
expressed statistically. At this point in our discussion

of
You might think we could uld adadd these six devia-
of variance, the primary goal is to help you to better
tion scores to get a total tal vvariability
ariab score for the
understand what variance is from a conceptual
sample. However, if we sum th tthe deviation scores for

ion
standpoint, not to teach you how to calculate it. The
nts
ts in a set oof data, they always add
all of the participants
following statistical description will help you get a
up to zero. So wee need
eed to get
g rid of the negative signs.
clear picture of what variance tells us about our data.

iss
We do this by squaring
quaring eeach of the deviation scores.
We can see what the variance is by following
five simple steps. We will refer here to the scores or
rm
observations obtained in our study of attitudes on
capital punishment. Deviation Deviation Score
Participant
ticipant
ipant Score Squared
pe

Step 1. As we saw earlier, variance refers to how 1 1.33 1.77

spread out the scores are around the mean of 2 -1.67 2.79
the data. So to begin, we need to calculate the 3 -0.67 0.45
ior

mean of our data. Just sum the numbers (4 + 4 -0.67 0.45

1 + 2 + 2 + 4 + 3 = 16) and divide by thee 5 1.33 1.77
pr

number of scores you have (16/6 = 2.67). 2.67).

). 6 0.33 0.11
ymbol
Note that statisticians usually use the symbol
t

yq or xq to represent the mean of a set of data

da Step 4. Now we add the squared deviation scores.
ou

(although the symbol M is often used ed in sci-

s If we add all of the squared deviation scores
entific writing). In short, all wee do on the
th first obtained in Step 3, we get
th

step is calculate the mean off the six scores.

xpressing
Step 2. Now we need a way of expressing ressing how much 1.77 + 2.79 + 0.45 + 1.77 + 0.11 = 7.34.
wi

the scores vary aroundd the

the mean.
mean We do this
This number—the sum of the squared deviations of
by subtracting the mean from ffr each score.
n

the scores from the mean—is central to many statis-

This difference iss called a deviation score.
tical analyses. We have a shorthand way of referring
tio

Let’s do this for our data involving people’s

to this important quantity; we call it the total sum of
attitudes toward capitaltal punishment:
punish
squares. .
bu

Step 5. In Step 4 we obtained an index of the total

Participant
ant
nt Deviation Score variability in our data—the total sum of
tri

1 4 - 2.67 = 1.33 squares. However, this quantity is affected

dis

2 1 - 2.67 = -1.67 by the number of scores we have; the more

3 2 - 2.67 = -0.67 participants in our sample, the larger the
4 2 - 2.67 = -0.67 total sum of squares will be. However, just
because we have a larger number of partici-
or

5 4 - 2.67 = 1.33
pants does not necessarily mean that the
6 3 - 2.67 = 0.33
tf

variability of our data will be greater.

no
le
Fi



 ion
at
uc
Behavioral Variability and Research

Ed
Because we do not want our index of variability from each score, (3) squaring these differences erences or
to be affected by the size of the sample, we divide the deviation scores, (4) summing these squareduared
red deviation
de
d

n
sum of squares by a function of the number of partici- scores (this, remember, is the total sum um of squares),
s

so
pants in our sample. Although you might suspect that and (5) dividing by the number off scores
cores minus
m 1. By
we would divide by the actual number of participants following these steps, you should uld
d be able
ab
abl to see pre-
from whom we obtained data, we usually divide by one cisely what the variance is. Itt iss an index
ind
in of the aver-

ar
less than the number of participants. (Don’t concern age amount of variability inn a set of data expressed in

Pe
yourself with why this is the case.) This gives us the terms of how much the scorescores differ
diff from the mean in
dif
variance of our data, which is indicated by the symbol squared units. Again,, variance
varian is important because
s2. If we do this for our data, the variance (s2) is 1.47. ct of the
virtually every aspect th
t e research
r process will lead

of
To review, we calculate variance by (1) calculat- behav
behavi
to the analysis of behavioral variability, which is
ing the mean of the data, (2) subtracting the mean he statistic
expressed in the statistic known as variance.

ion
Developing Your Research Skills
Statistical Notation

iss
Statistical formulas are typically written using statistical notation.
sign (+) to indicate add and an equal sign (=) to indicate is equal
n Just as
a we commonly use symbols such as a plus
w be using special symbols—such as ∑, n,
ual to, we’ll
rm
and s2—to indicate statistical terms and operations. Although h some of these symbols may be new to you, they are
nothing more than symbolic representations of variables or math mathematical operations, all of which are elementary.
For example, the formula for the mean, expressed in sta statistical notation, is
pe

y = ©yi /n
ior

The uppercase Greek letter sigma (∑) is the statistic

statistical symbol for summation and tells us to add what follows.
The symbol yi is the symbol for each individual participant’s score. So the operation ∑yi simply tells us to add
dual pa
pr

up all of the scores in our data. That is,

gyi = y1 + y2 + y3 + p + yn
t
ou

participants. Then the formula for the mean tells us to divide ∑yi by n, the num-
where n is the number of partici
ber of participants. Thus, the formula qy = gy i /n indicates that we should add all of the scores and divide by the
th

number of participants.
wi

ce can be expressed in statistical notation as

Similarly, the variance

s2 = g(yi - qy)2 /( n - 1).

n
tio

ck at the steps for calculating the variance on the preceding pages and see whether you can inter-
Look back
ula for s2.
pret this formula
bu

Step 1 Calculate
Calcu the mean, qy.
Step
p 2 Subtract
Subt the mean from each participant’s score to 2obtain the deviation scores,( y i - qy).
tri

Step
ep 3 Square
Sq each participant’s deviation score, ( y i - qy) .
dis

2
Step 4 Sum the squared deviation scores, g 1yi - qy ) .
Ste 5 Divide by the number of scores minus 1, n – 1.
Step
or

Statistical notation will allow us to express certain statistical constructs in a shorthand and unambiguous
manner.
tf
no
le
Fi


 ion
at
uc
Behavioral Variability and Research

Ed
SYSTEMATIC AND ERROR VARIANCE participants’ behavior that is related systematically
matic
atic
to the variables under investigation is systematic
So far, our discussion of variance has dealt with the

n
variance. Two examples may help clarify the concept
oncept
total variance in the responses of participants in a

so
of systematic variance.
research study—the total variability in a set of data.
However, the total variance in a set of data can be TEMPERATURE AND AGGRESSION. ON. In
ION. I an

ar
split into two parts: experiment that examined the effects ts of temperature
temp
tem

Pe
on aggression, Baron and Bell (1976) (1 led
Total variance = systematic variance +
participants to believe that theyy wouldwo administer
error variance.
person
electric shocks to another person. erso (In ( reality, that

of
In fact, at one level, answering questions about other person was an accomplicemplice
lice of the experimenter
behavioral variability always involves distinguishing and was not actually ly
y shocked.)
shoc Participants
performed this task inn a room in which the ambient

ion
between the systematic and error variance in a set of
data and then figuring out what variables in our temperature was 73 degrees, 85 degrees, or 95 degrees
study are related to the systematic portion of the t
F. To determine whether temperature did, in fact,

iss
variance. Systematic and error variance are impor- ion,
affect aggression,n, the researchers
res
r had to determine
tant to the research process. how much off the variability
vvari in participants’ agg-
ression wass related
relate to temperature. That is, they
rm
needed to know how h much of the total variance in
Systematic Variance
the aggression
gressi scores (that is, the shocks) was
pe

Most research is designed to determine whether systematic

ematic variance
stematic v due to temperature. We
there is a relationship between two or more vari- wouldn’t
wouldn’t expect
ex all of the variability in participants’
ables. For example, a researcher may wish to test the aggression
aggr i to be a function of temperature. After all,
aggres
ior

hypothesis that self-esteem is related to drug use or parti

particip
participants entered the experiment already differing
that changes in office illumination cause systematic ic t
in their tendencies to respond aggressively. In
pr

changes in on-the-job performance. Put differently, tly,

y, ad
addition, other factors in the experimental setting
researchers usually are interested in whetherr va var
vari- may have affected aggressiveness. What the
ability in one variable (self-esteem, illumination)nation)
tion) is researchers wanted to know was whether any of the
t
ou

related in a systematic fashion to variability ityy in oth

other variance in how aggressively participants responded
variables (drug use, on-the-job performance). ance). was due to differences in the temperatures in the
three experimental conditions (73°, 85°, and 95°). If
th

Systematic variance is that part of the th total

variability in participants’ behaviorr that
hat is rrelated in systematic variance related to temperature was
wi

an orderly, predictable fashion to o the vvavariables the obtained, they could conclude that changes in
he ppa
researcher is investigating. If the rtici
tici
participants’ behav- temperature affected aggressive behavior. Indeed,
ior varies in a systematicc way as a certain other this and other research has shown that the likelihood
n

searcher has evidence that

variables change, the researcher of aggression is greater when the temperature is
tio

elated
those variables are relatedated to behavior. In other moderately hot than when it is cool, but that
words, when some of the tot total variance in partici- aggression decreases under extremely high
bu

pants’ behavior is found to bbe associated with certain temperatures (Anderson, 1989).
rderly, systematic fashion, we can
variables in an orderly,
tri

conclude thatt those

hose va
variables are related to partici- OPTIMISM AND HEALTH. In a correlational study
avior.
vior. The portion of the total variance in
pants’ behavior. of the relationship between optimism and health,
dis

Scheier and Carver (1985) administered to par-

ticipants a measure for optimism. Four weeks later,
the same participants completed a checklist on
or

which they indicated the degree to which they

had experienced each of 39 physical symptoms.
tf
no
le
Fi



 ion
at
uc
Behavioral Variability and Research

Ed
Of course, there was considerable variability in the behavior. In the Baron and Bell experiment,, not al all of
number of symptoms that participants reported. articipa
icipa was
the variability in aggression across participants

n
Some indicated that they were quite healthy, whereas due to temperature. And in the Scheier heier
eier and Carver

so
others reported many symptoms. Interestingly, study only 7% of the variance in the he sym
symp
symptoms that
participants who scored high on the optimism scale participants reported was related ated
d to oop
optimism; the
reported fewer symptoms than did less optimistic remaining 93% of the variance ce in sym
symptoms was due

ar
participants; that is, there was a correlation between to other things.

Pe
optimism scores and the number of symptoms that Clearly, then, other er factors are at work. Much
participants reported. In fact, approximately 7% of of the variance in thesee stu studie
studies was not associated
the total variance in reported symptoms was related with the primary variableriab s of interest (temperature
variables

of
to optimism; in other words, 7% of the variance in and optimism). Forr exam example, in the experiment on
symptoms was systematic variance related to aggression, some ome pparticipants
arti may have been in a
participants’ optimism scores. Thus, optimism and worse mood d than
han oth
oothers, leading them to behave

ion
symptoms were related in an orderly, systematic ressively
essively fo
more aggressively for reasons that had nothing to do
fashion. with roomomm tempera
tempe
temperature. Similarly, some participants

iss
In both of these studies, the researchers found may haveave cocome from aggressive homes, whereas
that some of the total variance was systematic vari- hers
rs may hhave been raised by parents who were
others
ance. Baron and Bell found that some of the total pacifists.
cifists. Th
The experimenter may have unintentionally
rm
variance in aggression was systematic variance treated ssome subjects more politely than others,
related to temperature; Scheier and Carver found ther
thereby lowering their aggressiveness. A few parti-
pe

that some of the total variance in physical symptoms ms cip

cipan
cipants may have been unusually hostile because they
was systematic variance related to optimism. sm ha
had just failed an exam. Each of these factors may
Finding evidence of systematic variance indicates dicates
ates have contributed to the total variability in partici-
ior

that variables are related to one another—that hat

at room pants’ aggression, but none of them is related to
temperature is related to aggression, and d optimism
ptimis the variable of interest in the experiment—the
pr

is related to physical symptoms, for or exam

exampl
example. temperature.
Uncovering relationships in research chh is alwa
alw
always a Even after a researcher has determined how
matter of seeing whether part of thee total va variance in much of the total variance is related to the variables
t
ou

participants’ scores is systematicc variance.

variance of interest in the study (that is, how much of the total
Researchers must designn their studies
st
s so that variance is systematic), some variance remains
they can tell how much of the total vvariance in par- unaccounted for. Variance that remains unaccounted
th

tematic
matic vvariance associated
ticipants’ behavior is systematic for is called error variance. Error variance is that
wi

with the variables they y are in investigating. If they portion of the total variance that is unrelated to
don’t, the study will ill faail
il tto detect relationships
fail the variables under investigation in the study (see
among variables that at are
are, in fact, related. Poorly Figure 2).
n

designed studiess do not permit researchers to con- Do not think of the term error as indicating
tio

ently
tly which
clude confidently whic variables were responsible errors or mistakes in the usual sense of the word.
ematic
matic va
for the systematic variance they obtained. Although error variance may be due to mistakes in
bu

recording or coding the data, more often it is simply

the result of factors that remain unidentified in a
Errorr Varian
Variance
tri

study. No single study can investigate every factor

Notot all of the
th total variability in participants’ behavior that is related to the behavior under investigation.
dis

iss system
systematic variance. Factors that the researcher is Rather, a researcher chooses to investigate the
notot inv
investigating may also be related to participants’ impact of only one or a few variables on the target
behavior. Baron and Bell chose to study temperature,
or

for example, and ignored other variables that might

influence aggression. Scheier and Carver focused on
tf
no
le
Fi


 ion
at
uc
Behavioral Variability and Research

Ed
Error variance due to all
other factors unidentified
in the study—personality

n
differences, mood, health,

so
recent experiences, etc.

ar
Pe
Systematic variance due to
the variable of interest in the
study—optimism.

of
FIGURE 2 Variability in Physical Symptoms. If we draw a circle to represent sen the tto
total
variability in the physical symptoms reported by participants in the Scheierer and Carver
Ca
C

ion
lated to
(1985) study, systematic variance is that portion of the variance that is related t the
t
hat portion
variable under investigation, in this case optimism. Error variance is that port
portio of the
d.
total variability that is not related to the variable(s) being studied.

optimism but not on other variables related to physi-

iss
allow them m not on
only to calculate how much of the
rm
cal symptoms. All of the other unidentified variables ariance
ance is sy
total variance systematic versus error variance but
that the researchers did not study contributed to the also to test whe
w
whether the amount of systematic vari-
pe

total variance in participants’ responses, and the ce in the da

ance data is large enough to conclude that the
variance that is due to these unidentified variables is effect
ct is real
effect re (as opposed to being due to random
called error variance. influ
influen ce For now, the important point to remem-
influences).
ior

ber iis that, in order to draw conclusions from their

da
data
data, researchers must statistically separate system-
Distinguishing Systematic from Error
pr

at
atic from error variance.
Variance
Unfortunately, error variance can mask or
To answer questions about behavioral variability, riability,
ability, obscure the effects of the variables in which
t
ou

researchers must determine whether any off the tot total researchers are primarily interested. The more error
variance in the data they collect is related in i a variance in a set of data, the more difficult it is to
systematic fashion to the variables they investi-
hey are iin
th

determine whether the variables of interest are

gating. If the participants’ behavior avior varies in a
or var related to variability in behavior. For example, the
wi

systematic way as certain otherr variables

variabl change, more participants’ aggression in an experiment is
systematic variance is present, nt, providing
pprovid evidence affected by extraneous factors, such as their mood
that those variables are relatedted to th
the behavior under or how the researcher treats them, the more difficult
n

investigation. it is to determine whether room temperature

tio

As they analyze theirheir data,

data researchers always affected their aggression.
inguishin
guishin the systematic vari-
face the task of distinguishing The reason that error variance can obscure the
bu

ance from the errorr variance in their data. In order to systematic effects of other variables is analogous to
determine whether ther
er vari
var
variables are related to one the way in which noise or static can cover up a song
tri

another, they must be able to tell how much of the that you want to hear on the radio. In fact, if the
bility in the behavior being studied is
total variability static is too loud (because you are sitting beside an
dis

systematicic variance
varianc versus error variance. This is the electrical device, for example), you might wonder
point at w hich statistics
which st are indispensable. Researchers whether a song is playing at all. Similarly, you can
ertain statistical analyses to partition the total
use certain think of error variance as noise or static—unwanted,
or

ariance
riance in their data into components that reflect
variance annoying variation that, when too strong, can mask
sy
ystemat
stema
systematic versus error variance. These analyses
tf
no
le
Fi



 ion
at
uc
Behavioral Variability and Research

Ed
the real “signal” produced by the variables in which the empirical relationships they discover by det deter-
the researcher is interested. mining the proportion of the total variab ariab li in
variability

n
In the same way that we can more easily hear a tematic
participants’ responses that is systematicmatic vvariance

so
song on the radio when the static is reduced, udy.
related to the variables under study. y. As w
we saw, the
researchers can more easily detect systematic variance total variance of a set of data ata
ta is cco
composed of
produced by the variables of interest when error systematic variance and error ror
or vari
varia
variance. Once we

ar
variance is minimized. They can rarely eliminate error calculate these types of variance
arianc we can easily
variance,

Pe
variance entirely, both because the behavior being stud- determine the proportion on of the total variance that
ied is almost always influenced by unknown factors is systematic (that is, s, th
the proportion of total
and because the procedures of the study itself can cre- ystema
tem ic
variance that is systematic i variance = systematic

of
ate error variance. But researchers strive to reduce error riance).
ance).
variance/total variance).
variance as much as possible. A good research design is Researchers
hers use measures of effect size to
one that minimizes error variance so that the researcher show them how strongly variables in a study are
w str
stro

ion
can detect any systematic variance that is present in the related to o one anoth
an
another. For now, it is enough to
data. and
understandnd that on
o index of the strength of the rela-
one

iss
To review, the total variance in a set of data hip
tionshipp between
betw
betwee variables involves the proportion
contains both systematic variance due to the variables of total
otal var
varian
variance that is systematic variance. That is,
of interest to the researcher and error variance due to wee can see
se how strongly two variables are related by
rm
everything else (that is, total variance = systematic calcula
calculatin
calculating the proportion of the total variance that is
variance + error variance). The analysis of data from syst
system
systematic variance. For example, we could calcu-
pe

a study always requires us to separate systematicc lat the proportion of the total variance in people’s
late
from error variance and thereby determine whether er a ra
ratings of how upset they are about losing their job
relationship between our variables exists. that is systematic variance related to their expecta-
ior

tions of finding a new one.

At one extreme, if the proportion of the total
EFFECT SIZE: ASSESSING THE
pr

variance that is systematic variance is .00, none of

STRENGTH OF RELATIONSHIPSPS the variance in participants’ responses in a study is
Researchers are interested not only inn wheth whether certain systematic variance. When this is the case, we know
t
ou

variables are related to participants’ ts’ respon

responses but also there is absolutely no relationship between the
in how strongly they are related. d. Somet
Someti
Sometimes variables variables under study and participants’ responses. At
th

are associated only weakly with part particular cognitive, the other extreme, if all of the variance in partici-
emotional, behavioral, or physiophysiol
physiological responses, pants’ responses is systematic variance (that is, if
wi

ariables
riables aare strongly related to
whereas at other times, variables systematic variance/total variance = 1.00), then all of
thoughts, emotions, and nd bbehavior.
eha For example, in a the variability in the data can be attributed to the
study of variables thathat pre
pred
predict workers’ reactions to variables under study. When this is the case, the
n

losing their jobs,, Prussia, Kinicki, and Bracker (1993) variables are as strongly related as they possibly can
tio

found that the degree

egree to which respondents were emo- be (in fact, this is called a perfect relationship).
tionally upsetsett about losing their jobs was strongly When the ratio of systematic to total variance is
bu

related to o how mumuch effort they expected they would between .00 and 1.00, the larger the proportion, the
have too exert to find a new job but only weakly related stronger the relationship between the variables.
tri

heirr expe
to their expect
expectations of actually finding a new job. When we view effect size as a proportion, we
Mea
Measures of the strength or magnitude of can compare the strength of different relationships
dis

relations
elation
relationships among variables show us how directly. For example, in the study of reactions to job
impor
mpor
important particular variables are in producing a loss described earlier, 26% of the total variance in
pparticular behavior, thought, emotion, or physio-
par emotional upset after being fired was related to how
or

lo
logical response. Researchers assess the strength of much effort the respondents expected they would
tf
no
le
Fi


 ion
at
uc
Behavioral Variability and Research

Ed
have to exert to find a new job. In contrast, only 5% ployment. In fact, by comparing the strength of asso-
of the variance in emotional upset was related to ciation for the two findings, we can see that at the

n
their expectations of finding a new job. Taken olved
person’s expectations about the effort involved ved in

so
together, these findings suggest that, for people who looking for work (which accounted for 26% 6%
% of th
the
lose their jobs, it is not the possibility of being total variance in distress) was over five times
imes mmore
forever unemployed that is most responsible for their pset
strongly related to their emotional upsetet than
tha their

ar
upset but rather the expectation of how difficult which
hich ac
expectation of finding a new job (which accounted

Pe
things will be in the short run while seeking reem- for only 5% of the variance).

In Depth

of
Types of Effect Size Indicators

ion
Researchers use several different statistics to indicate effect size depending on the nature of their data. Roughly
speaking, these effect size statistics fall into three broad categories. Some effectt size indindices, sometimes called d-
based effect sizes, are based on the size of the difference between the means of two groups, such as the differ-

iss
ence between the average scores of men and women on some measure or the diff differences in the average scores
that participants obtained in two experimental conditions. The larger the differendifference between the means, relative
to the total variability of the data, the stronger the effect and the largerr the ef effect size statistic.
rm
lation b
The r-based effect size indices are based on the size of the correlation between two variables. The larger the
correlation, the more strongly two variables are related and the more e of the total variance in one variable is system-
atic variance related to the other variable.
pe

A third category of effect sizes index involves the odds-ratio,-ratio, which tells us the ratio of the odds of an
event occurring in one group to the odds of the event occurring rin in aanother group. If the event is equally likely in
both groups, the odds ratio is 1.0. An odds ratio greater than 1 1.0 shows that the odds of the event is greater in
ior

one group than in another, and the larger the odds ratio, o, the st
stronger the effect. The odds ratio is used when the
variable being measured has only two levels. For example, ple, im
imagine doing research in which first-year students in
pr

college are either assigned to attend a special course se on h

how to study or not assigned to attend the study skills
course, and we wish to know whether the course rse reduces
duc the likelihood that students will drop out of college.
We could use the odds ratio to see how much off an eeffect the course had on the odds of students dropping out.
t
ou

statistic differences among these effect size indices, but you will
You do not need to understand the statistical
find it useful in reading journal articles to know what some of the most commonly used effect sizes are called.
These are all ways of expressing how strongly variables are related to one another—that is, the effect size.
th
wi

Sym
Symbol Name
d Cohen’s d
n

g Hedge’s g
h2 eta squared
tio

v2 omega squared
r or r2 correlation effect size
bu

OR odds ratio
tri
dis

Thee streng
strengt
strength of the relationships between variance may be quite large, sometimes (though
les varies a great deal across studies. In some
variables rarely) as high as 80%.
dies,
s, as little
studies, l as 1% of the total variance may be Generally, researchers prefer that their
or

ystematic
stematic variance, whereas in other contexts,
systematic research findings have relatively large effect sizes
thee propo
prop
proportion of the total variance that is systematic because a large effect size usually indicates that they
tf
no
le
Fi



 ion
at
uc
Behavioral Variability and Research

Ed
have identified an important correlate, predictor, or to a large portion of the total variance in thee pheno
phenom-
cause of the phenomenon they are studying. In real- enon being investigated. For example, think hink ooff all of

n
ity, however, studies in behavioral science rarely the factors that contribute to variability ility
ty in ha
happiness

so
account for more than 40% of the total variance with and unhappiness, such as a person’s n’ss healt
health
health, relation-
any single variable, and most effects are far smaller. ship satisfaction, family situation,uation,
tion, ffi
financial dif-
In fact, one study of three leading journals in psy- ifficulties
icultie at school, the
ficulties, job satisfaction, difficulties

ar
chology showed that the average effect sizes were in well-being of loved ones, legal gal pro
prob
problems, and so on.

Pe
the .10 to .20 range (Ward, 2002). And, we must laining eev
Viewed in this way, explaining even a small percent-
remember that published studies typically have age of the total variancee in a paparticular response, such
stronger effects than unpublished ones. as happiness, in terms mss of oonly
nly
l one variable may be an

of
Many students are initially surprised, and even important finding. g. Seemi
Seemingly small effects can be
troubled, to learn how “weak” many research findings interesting and impo p rtan
important.
are. For example, a national survey of a representative der anoth
Consider anothe
another example—the fact that peo-

ion
sample of nearly 5,000 adults by DeVoe and Pfeffer antic
ntic partn
ple’s romantic partner
partners tend to be about the same level
(2009) showed that people who had higher annual ical
of physicalal attractiv
attrac
attractiveness as they are. Highly attractive

iss
incomes reported being happier than people who peoplele tend to ha
have relationship partners who are high
made less money. But how much of the total variance in attractivene
ttractive
attractiveness, moderately attractive people tend to
in happiness do you think was accounted for by pair
ir with mmoderately attractive partners, and unattrac-
rm
income? Less than 3%! (That is, less than 3% of the tive pe
peop
people tend to have less attractive partners. But
total variance in happiness was systematic variance how m much of the total variance in the attractiveness of
pe

due to income.) That’s not a very large effect size. pe

peop
people’s relationship partners is systematic variance
Yet, perhaps we should not be surprised that anany re
related to the attractiveness of the people themselves?
hatever
ever
particular variable is only weakly related to whatever Research shows that it is only about 16% (Meyer et al.,
ior

phenomenon we are studying. After all, most psycho-

psycho 2001). That may not seem like a very strong associa-
mined—the
logical phenomena are multiply determined—the ned—th tion, yet the effect is strong enough to be seen easily in
pr

ht of this,
result of a large number of factors. In light this wwe everyday life and it shows that something involving
able
ble investigated
should not expect that any single variable vest
vestig physical appearance influences people’s choices of
ematicall
maticall related
in a particular study would be systematically relationship partners.
t
ou
th

In Depth
wi

Effect Sizes in Psychology,

hology,
ogy, M
Medicine, and Baseball

Behavioral researchersrs hav

have sometimes been troubled by the small effect sizes they often obtain in their research.
n

In fact, however, the siz

sizes of the effects obtained in behavioral research are comparable to those obtained in
tio

es. For ex
other disciplines. example, many effects in medicine that are widely regarded as important are smaller than
those typically obtain
obtained in psychological research (Meyer et al., 2001).
ch has
Research ha shown, for example, that taking aspirin daily helps to reduce the risk of death by heart attack,
bu

and manyny people regularly take aspirin for this purpose. But aspirin usage accounts for less than 1% of the risk of
having a heart attack. This should not deter you from taking aspirin if you wish; yours may be one of the lives that
tri

aved. But
are saved. Bu the effect is admittedly small. Similarly, many people take ibuprofen to reduce the pain of headaches,
ore muscles,
sore muscl and injuries, and ibuprofen’s effectiveness is well-documented. Even so, taking ibuprofen accounts for
dis

only abo
about 2% of the total variance in pain reduction. The effect of Viagra is somewhat more impressive; Viagra
accou
accounts for about 14% of the improvement in men’s sexual functioning.
To look at another well-known effect, consider the relationship between a major league baseball
or

player’s batting skill (as indicated by his RBI) and the probability that he will get a hit on a given instance at
(continued)
tf
no
le
Fi


 ion
at
uc
Behavioral Variability and Research

Ed
(continued)
bat. You might guess that RBI bears a very strong relationship to success-at-bat. A player with a higher RBI

n
surely has a much greater chance of getting a hit than one with a lower RBI. (Why else would players with

so
higher RBIs be paid millions of dollars more than those with lower RBIs?) But if we consider the question on from
the standpoint of variance, the answer may surprise you. RBI accounts for only .0036% of the total varian variance

ar
in a batter’s success at a given instance at bat! The small size of this effect stands in contrast to the im impor-
tance of RBI and makes an important point: Small effects can add up. Although a higher RBI gives a batter

Pe
mulative effects of
only a slight edge at any given time at bat, over the course of a season or a career, the cumulative
slight differences in batting average may be dramatic. (Hence, the large salaries.) The same me is true of certain
psychological variables as well.
er dom
My point is not to glorify the size of effects in behavioral research relative to other domains. Rather, my

of
point is twofold: The effects obtained in behavioral research are no smaller than those in n mo
most other fields, and
even small effects can be important.

ion
META-ANALYSIS: SYSTEMATIC
VARIANCE ACROSS STUDIES
iss
provides only lyy a rough
particularr relation
roug estimate of the strength of a
relationsh
relationship, averaging these estimates
rm
over manyany
ny studie
stud
studies that used different participants,
As we’ve seen, researchers are typically interested in sures,
res, and
measures, an procedures should provide a more
the strength of the relationships they uncover in their
pe

curate
rate indic
accurate ind
indication of how strongly the variables are
studies. However, any particular piece of research really” rrel
“really” related.
can provide only a rough estimate of the “true” pro- A procedure known as meta-analysis is used
ior

portion of the total variance in a particular behavior to an

ana
analyze and integrate the results from a large
that is systematically related to other variables. Thee se of individual studies (Cooper, 1990). When
set
effect size obtained in a particular study is only ya
pr

re
researchers conduct a meta-analysis, they examine
rough estimate of the true effect size because see tthe
he every study that has been conducted on a particular
strength of the relationship obtained in a study udy is topic to assess the relationship between whatever
t

affected not only by the relationship between tween

ween th the
ou

variables are the focus of their analysis. Using infor-

variables but also by the characteristics off the sstustudy mation provided in the journal article or report of
itself—the sample of participants who o were sstu
studied,
th

each study, the researcher calculates the effect size in

the particular measures used, and the he researc
research proce- that study, which, as we have seen, is an index of the
wi

dures, for example. Thus, although ough

gh Pru
Prussia et al. strength of the relationship between the variables.
(1993) found that 26% of the varia variance in their These effect sizes from different studies are then
respondents’ emotional upset sett was related to their statistically combined to obtain a general estimate of
n

expectations of how much h effort th

they would need to the strength of the relationship between the variables.
tio

exert to find a new job,, the stren

strength of the relation- By combining information from many individual
tations
ship between expectations tions and
an emotional upset in studies, researchers assume that the resulting esti-
bu

their study may have ve been aaffected by the particular mate of the average strength of the relationship will
easures,
asures, and procedures the res-
participants, measures, be more accurate than the estimate provided by any
tri

earchers used. We may find a somewhat stronger or particular study.

tionship iif we conducted a similar study
weaker relationship Let’s consider a meta-analysis of the psycho-
dis

ferent
erent pa
using different par
participants, measures, or methods. logical effects of punishment on children. Parents
Forr this reason,
r behavioral scientists have and psychologists have long debated the immediate
omee inc
become re
increasingly interested in examining the effectiveness and long-term impact of using corporal
or

rength of relationships
strength r between particular variables punishment, such as spanking, to discipline children.
cross
ross m
across many studies. Although any given study Some have argued that physical punishment is not
tf
no
le
Fi



 ion
at
uc
Behavioral Variability and Research

Ed
only effective but also desirable, but others have con- when particular experimental conditions ons wwere
cluded that it is ineffective if not ultimately harmful. present. For example, Gershoff (2002)) found ound that
h the

n
In an effort to address this controversy, Gershoff more girls that were included in a study, tthe less

so
(2002) conducted a meta-analysis of 88 studies that corporal punishment was associated tedd with aaggression
investigated various effects of corporal punishment. and antisocial behavior (suggesting sting
ng that
tha the effect of
These studies spanned more than 60 years (1938 to ggression
punishment on increased aggression ression is stronger for

ar
2000) and involved more than 36,000 participants. boys). Furthermore, although ough
gh corp
corporal punishment

Pe
Clearly, conclusions based on such a massive was associated with negative effects for all age
amount of data should be more conclusive than those groups, the negative effectsfects we
were strongest when the
obtained by any single study. Gershoff’s statistical rticipan
cip ts
mean age of the participants t was between 10 and 12,

of
analyses of these studies showed that, considered as orporal
suggesting that corporalporal ppunishment has a stronger
a whole, corporal punishment was associated with effect on middledle sc hool children than on other ages.
school
all of the 11 outcome behaviors she examined, which -analysis
alysis is used not only to document
Thus, meta-analysis

ion
included childhood aggression and antisocial hips acro
relationships across studies but also to explore
behavior, decreased quality of the relationship factors that
at affec
affect tthe strength of those relationships.

iss
between child and parents, poorer mental health dur- For ma
many years, researchers who conducted
ing both childhood and adulthood, and increased risk eta-analyses
a-analys were frustrated by the fact that many
meta-analyses
of later abusing a child or a spouse. authors
thors ddid not report information regarding the
rm
In most meta-analyses, researchers not only effect ssiz
sizes of their findings in journal articles and
determine the degree to which certain variables are other re
othe research reports. However, new guidelines from
pe

he
related (that is, the overall effect) but also explore the the AAmerican Psychological Association now require
factors that affect their relationship. For example, e, iin re
researchers to report effect sizes in their publications
looking across many studies, they may find that hatt the and papers (APA Publications and Communications
ior

relationship was generally stronger for male than for Board Working Group, 2008). With this information
female participants, that it was stronger when hen
n certa
certain more readily available, the quality and usefulness of
pr

kinds of measures were used, or that it was we weak

weaker meta-analyses will improve in the future.
t
ou

Behavioral Research Case

ase St
Study
Meta-Analyses of Gender Differenc
Differences in Math Ability
th

Meta-analyses have been conducted

onducte on many areas of the research literature, including factors that influence the
erapy, gender differences in sexuality, the effects of rejection on emotion and self-
effectiveness of psychotherapy,
wi

erence in prejudice, helping behavior, and employees’ commitment to their jobs.

esteem, personality differences
However, by far, the most popular topic for meta-analysis has been gender differences.
n

Although manyany studies

stud have found that men and women differ on a variety of cognitive, emotional, and
tio

ables, re
behavioral variables, researchers have been quick to point out that the differences obtained in these studies are
all (and typically smaller than popular stereotypes of men and women assume). Furthermore, some
often quite small
obtain differences between men and women, whereas others have not. This is fertile territory for
studies have obtained
bu

nalyses, w
meta-analyses, which can combine the findings of many studies to show us whether, in general, men and
women differ o on particular variables. Researchers have conducted meta-analyses of research on gender differ-
tri

encess to ans
answer the question of whether men and women really differ in regard to certain behaviors and, if so,
o document
to docume the strength of the relationship between gender and these behaviors. Using the concepts we have
dis

learned in this chapter, we can rephrase these questions as: Is any of the total variability in people’s behavior
relate
related to their gender, and, if so, what proportion of the total variance is systematic variance due to gender?
Hyde, Fennema, and Lamon (1990) conducted a meta-analysis to examine gender differences in
mathematics performance. Based on analyses of 100 individual research studies (that involved over 3 million
or

(continued)
tf
no
le
Fi


 ion
at
uc
Behavioral Variability and Research

Ed
(continued)
participants), these researchers concluded that, overall, the relationship between gender and math per- er-

n
formance is very weak. Put differently, the meta-analysis showed that very little of the total variance ce in

so
math performance is systematic variance related to gender. Analyses did show that girls slightly
outperformed boys in mathematic computation in elementary and middle school but that boys tended ended to

ar
outperform girls in math problem solving in high school. By statistically comparing the effect sizes es for stud-
ies that were conducted before versus after 1974, they also found that the relationship between tween gender

Pe
and math ability has weakened over time.
rence in mathemat-
More recently, Else-Quest, Hyde, and Linn (2010) conducted a meta-analysis of gender differences
ics achievement and attitudes using data from 69 countries. Their analysis, which was based sed onnn nearly 500,000
ifferen
students ranging in age from 14 to 16 years old, found that the average effect sizes for the differences between boys

of
and girls were very small, sometimes favoring one gender and sometimes the other. In fact, the he eff
effect sizes for gender
th ach
differences in the United States hovered around .00, showing no overall difference in math achievement between boys

ion
and girls. Further analyses showed that the effect size differed somewhat by country, but ove
overall, the data provided no
evidence for strong and consistent differences in the math abilities of boys and d girls. EEven so, the meta-analysis
showed that boys thought they were better at math than girls did.

iss
rm
THE QUEST FOR SYSTEMATIC rejectionn sensitiv
sensitivity have less satisfying relationships?
VARIANCE ddress
ress quest
To address qu
questions such as these, researchers design
pe

dies
es to ddete
studies determine whether certain variables relate
In the final analysis, virtually all behavioral research to the
he obobse
observed variability in the phenomenon of
is a quest for systematic variance. No matter what spe- inter
interes
interest in a systematic fashion. If so, they will explore
ior

cific questions researchers may want to answer, they preci

precise
precisely how the variables are related; but the first
are trying to account for (or explain) the variability y go is always to determine whether any of the total
goal
they observe in some thought, emotion, behavior, orr
pr

va
variance is systematic.
Does
physiological reaction that is of interest to them. Does Keeping this goal in mind as you move
the speed with which people process information rmation
mation forward in your study of research methods will give
t

decrease as they age? What effect does thee size of a

ou

you a framework for thinking about all stages of the

reward have on the extinction of a response nse once
onc the research process. From measurement to design to
reward is stopped? Are women more empathicempathi than data collection to analysis, a researcher must remember
th

men? What effect does alcohol havee on the aability to at each juncture that he or she is on a quest for
pay attention? Why do people who ho scor
sco
score high in
wi

systematic variance.
n

Summary
tio

1. Psychology andd other behavioral sciences attempt to capture numerically the variability
bu

involve the study of behavioral variability. in participants’ behavior; and (d) statistics
pects
Most aspectscts of behavioral research are are used to analyze the variability in our
tri

aimed at explai
expla
explaining variability in behavior: data.
(a) Research
esearch questions are about the causes 2. Descriptive statistics summarize and describe
dis

andd correla
correlates of behavioral variability; (b) the behavior of research participants. Inferential
res earche
arche try to design studies that will
researchers statistics analyze the variability in the data to
e
best explain the variability in a particular answer questions about the reliability and gen-
or

beha
behavior; (c) the measures used in research eralizability of the findings.
tf
no
le
Fi



 ion
at
uc
Behavioral Variability and Research

Ed
3. Variance is a statistical index of variability. 5. To examine the strength of the relationships
ationsh
Variance is calculated by subtracting the mean they study, researchers determine ne the propor-

n
of the data from each participant’s score, tion of the total variability inn behav behavior that

so
squaring these differences, summing the is systematic variance associated ociated with the
ssociated
squared difference scores, and dividing this variables under study. The larger the pro-
he lar
larg
sum by the number of participants minus 1. In portion of the total variance
riance that is systematic
ance th
tha

ar
statistical notation, the variance is expressed variance, the stronger onger
nger tthe relationship
as: s2 = g (yi - yq)2 /(n - 1).

Pe
between the variables.ables. St
Sta
Statistics that express
elatio
elations
the strength of relationships are called meas-
4. The total variance in a set of data can be
ures of effect size.
ize
broken into two components. Systematic

of
ysiss is uus
6. Meta-analysis used to examine the nature
variance is that part of the total variance in
ngth ooff rel
and strength relationships between variables
participants’ responses that is related in an
across manyany ind
in
individual studies. By averaging

ion
orderly fashion to the variables under investi-
ct sizes ac
effect acro
across many studies, a more accurate
gation in a particular study. Error variance is
stimate
mate of
estimate o tthe relationship between variables
variance that is due to unidentified sources

iss
can
an be obt
oobtained.
and, thus, remains unaccounted for in a study.
rm
Key Terms
pe

descriptive statistics meta-analysis

ysis total sum of squares
effect size range total variance
ior

error variance ical notat

statistical notatio
notation variability
inferential statistics tematic
matic vvar
systematic variance variance
pr

mean
t
ou

Questions for Review

th

1. Discuss how the concept ptt of beh

behavioral
beha variability 8. What does each of the following symbols mean in
relates to the following topics:
pics: statistical notation?
wi

a. the research questionsstions

tions th
that interest behavioral a. ∑
researchers b. xq
b. the design off research
esearch studies c. s2
n

rement of behavior
c. the measurement d. ∑yi/n
tio

lysis
ysis of be
d. the analysis behavioral data e. g (yi - yq)2
2. Why do researcher
searcher care how much variability exists
researchers 9. The total variance in a set of scores can be partitioned
bu

in a set of data? into two components. What are they, and how do they
inguish
nguish bet
3. Distinguish be
between descriptive and inferential statistics. differ?
onceptual what does the variance tell us about a
onceptuall
4. Conceptually, 10. What are some factors that contribute to error variance
tri

set
et of data
dat
data? in a set of data?
5. What iis the range, and why is it not an ideal index of 11. Generally, do researchers want systematic variance to
dis

vari
varia
variability? be large or small? Explain.
66. Gi
G
Give a definition of variance and then explain how you 12. Why are researchers often interested in knowing the
would calculate it. proportion of total variance that is systematic vari-
or

77. How does variance differ from the total sum of squares? ance?
tf
no
le
Fi


 ion
at
uc
Behavioral Variability and Research

Ed
13. What would the proportion of total variance that is 17. If the proportion of systematic variance to total variancee
systematic variance indicate if it were .25? .00? .98? mall,
is .08, would you characterize the relationship as small,

n
14. Why do researchers want the error variance in their medium, or large? What if the proportion were .72?72?? .00?
data to be small? 18. Why do researchers use meta-analysis? In what at way

so
15. Why is effect size important in scientific investigations? are meta-analyses more informative than thee results of
16. What are the three general types of effect size indica- a particular study?

ar
tors that researchers use? fect size indicate?
19. In a meta-analysis, what does the effect in

Pe
Questions for Discussion

of
1. Restate each of the following research questions as a umber
After analyzing the numbermbe off anagrams that partici-
question about behavioral variability. pants solved in the fourur conditions,
cond
conditi the researcher con-

ion
a. Does eating too much sugar increase children’s i fact, impede participants’
cluded that loud noise did, in
activity level? ability to solvee anagrams.
nagrams InI fact, the noise conditions
b. Do continuous reinforcement schedules result in accounted forr 23% of theth total variance in the number
faster learning than intermittent reinforcement
schedules? issof anagramsms that participants
partic
pa
his a small,
a. Is this sma medium,
m
solved.
or large effect?
rm
c. Do people who are depressed sleep more or less b. What hat proportion
propor
prop of the total variance was error
than those who are not depressed? ariance
variance?
d. Are people with low self-esteem more likely than le 10 things that might have contributed
c. List at least
pe

those with high self-esteem to join cults? th error variance in this study.
to the
e. Does caffeine increase the startle response to loud ever years ago, Mischel (1968) pointed out that, on
everal
4. Several
noise? a
ave
average, only about 10% of the total variance in a
ior

2. Simply from inspecting the following three data sets, par

particular behavior is systematic variance associated
which would you say has the largest variance? Which ch with another variable being studied. Reactions to
pr

has the smallest? Mischel’s observation were of two varieties. On one

a. 17, 19, 17, 22, 17, 21, 22, 23, 18, 18, 20 hand, some researchers concluded that the theories
b. 111, 132, 100, 122, 112, 99, 138, 134, 1166 and methods of behavioral science must somehow be
t

c. 87, 42, 99, 27, 35, 37, 92, 85, 16, 22, 500 flawed; surely, if our theories and methods were better
ou

3. A researcher conducted an experiment too examine

xamin the we would obtain stronger relationships. However,
effects of distracting noise on people’ss ability to solve others argued that accounting for an average of 10% of
th

unscram
unscra
anagrams (scrambled letters that can be unscrambled to the variability in a particular behavior with any single
make words). Participants worked edd on anagrams
ana for variable is not a bad track record at all. Where do you
wi

ound of jackhammers
10 minutes while listening to the sound stand on this issue? How much of the total variability
and dissonant music that was as played
pla
pl yedd at one of four in a particular phenomenon should we expect to
derate,
rate, loud,
volume levels (quiet, moderate, lo
lou or very loud). explain with some other variable?
n
tio
bu
tri
dis
or
tf
no