Alan Bryman - Social Research Methods

2
Research Designs
Chapter Overview
A research design is a framework for the collection and analysis of the data that will be used to answer
the research questions. It must satisfy certain criteria, and the form it takes depends on the research
questions being asked. This chapter will discuss four prominent research designs:
• experimental and related designs (such as the quasi-experiment)
• cross-sectional designs, including survey research, its most common form
• longitudinal designs, such as panel and cohort studies and various forms of qualitative research
• case study designs
“Would you tell me, please, which way I ought to go It’s not hard to feel a little like Alice when you start
from here?” out on the journey that is a research project, especially
“That depends a good deal on where you want to get if the voyage is one you are somewhat reluctant to
to,” said the Cat. undertake (this is not at all uncommon). Where should
“I don’t much care where—” said Alice. I be going with this study? How can I get there? How
“Then it doesn’t matter which way you go,” said can I tell if I’m on the right track? Such questions flood
the Cat. the mind at the beginning of the trip. As Alice learned,
“—so long as I get somewhere,” Alice added as an finding the right way starts with knowing where you
explanation. want to end up. In research, a lot depends on what you
“Oh, you’re sure to do that,” said the Cat, “if you only want to accomplish with the project, which in turn de-
walk long enough.” pends on the sort of research question you have posed.
Alice’s Adventures in Wonderland In this chapter we’ll explore what kinds of destination
(Carroll, 1865/2009, p. 56) are possible, some ways of getting to them, and some
techniques for determining whether you are on the
right path.
▲ Ditty_about_summer/Shutterstock
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

28 PART I Fundamental Issues in Social Research
Introduction watched changes, the level of violence would

have to change as well. You would have to ob-
Once a general research orientation has been chosen, serve that, in general, the greater the number
the next step is to select a research design. A re- of violent movies watched, the higher the level
search design is a broad structure that guides the of violence.
collection and analysis of data. Choosing a research 2. Time order. The proposed cause must precede
design involves decisions about what you want to the effect in time. Assume that you do find a
accomplish with the study. Is one of your goals to correlation between watching violent movies
describe the causal connections between variables? and real violence. You would also have to show
Will you explore how a social phenomenon changes that the increase in the watching of violent
over time, and how it may be linked to other events movies came before the increase in violence. If
and situations? Will it be important to search out the you found that the increase in the watching of
meanings that the research subjects attach to certain violent movies occurred after the level of vio-
things, and how they act on those meanings? Do you lence went up, your explanation would have to
want to be able to generalize your findings to people be rejected.
and groups that were not part of the study? How 3. Non-spuriousness. Alternative explanations
these questions are answered will affect your choice for the correlation observed have to be ruled
of research design. out. “Spurious” literally means false or illegit-
Another important consideration in choosing imate. A common source of spuriousness is
a research design is the kind of explanation you a variable that influences both the proposed
would like to develop. Quantitative researchers often cause and the effect. Suppose our data reveal
explain a phenomenon in terms of causes and ef- that people who are violent tend to watch a lot
fects that are expressed in laws and principles. Such of violent movies, and that people who reject
laws and principles are usually fairly general, and violence watch very few violent films, i.e., that
are meant to apply to people who were not part of the two variables are correlated. Can we con-
the study. For example, a study that finds that men clude from this that watching violent movies
are more likely than women to approve of pornog- causes people to be more violent? In order to
raphy may explain that result with a neo-Darwinian answer that question, we would try to deter-
theory of erotic attraction and sexual disgust. This mine whether some third factor is the cause of
sort of explanation is supposed to apply to human- the correlation between the number of violent
ity in general, not just to the people who participated movies watched and the level of violence. For
in the study. This approach to explanation is called instance, could it be that people who are vio-
nomothetic. lent to begin with seek out violent movies? If
Nomothetic explanations have to satisfy three cri- so, then pre-existing violent tendencies affect
teria of causation. In other words, three conditions both the watching of violent movies (your
have to be satisfied before the explanation is con- proposed cause) and the level of violence (the
sidered acceptable. Suppose you are trying to explain effect). If that is the case, the correlation be-
why some people are more violent than others, and tween watching violent movies and the level
your explanation involves the idea that the propen- of real violence is spurious, and your explan-
sity to be violent is influenced by watching violent ation does not hold up. As a further illustra-
movies. For this to be accepted as a causal explan- tion of spuriousness, imagine that you live in
ation, it would have to meet the following criteria: a summer resort community where there is
ample opportunity for swimming. Throughout
1. Correlation. The proposed cause and the the calendar year, there is probably a correla-
proposed effect have to vary together. In our tion between the number of ice-cream cones
example, as the number of violent movies sold on a particular day and the number of
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

2 | Research Designs 29
people swimming: as one increases the other you’ve selected a case to investigate, how will you
also increases. But does purchasing ice-cream get data on it? Do you use participant observation?
cones cause people to go swimming? Of course Do you observe from the sidelines? Do you conduct
not. Those two things are influenced by a third interviews? Do you examine documents? Do you ad-
factor: warm weather. minister questionnaires? Any or all of these methods
can be used.
Quantitative researchers choose research designs
that help to satisfy these three criteria. Qualitative
researchers, on the other hand, aim for different Research designs
kinds of explanations. They usually seek to produce
a rich description of a person or group based on Experimental design
the perceptions and feelings of the people studied, When people hear the word “research,” the use of
rather than to discover general laws and principles. experiments often comes to mind. However, true ex-
They may also try to get a sense of how those phe- periments are fairly rare in sociology, though they
nomena develop over time and how they can result can be found in areas such as social psychology and
from interpersonal interactions or one’s position in studies of organizations. Researchers in social policy
a power hierarchy. These sorts of accounts are called may also use them to assess the impact of reforms
idiographic explanations, and they usually involve a or new policies. The main reason for discussing ex-
detailed “story” or description of the people studied periments here is that a true experiment involves a
that is based on empathetic understanding. Bowen systematic comparison of what happens when one
(2015), for instance, used this approach to determine set of participants has a particular experience or
how off-street sex workers in Vancouver viewed the treatment while another does not, a logic that can
experience of transitioning from the sex industry also be found in other research designs. The greatest
into “square” (non-sex) occupations and lifestyles. strength of experiments is that they can be effective
On the basis of the insights gained from the study, in establishing causation (sometimes referred to as
she revealed that people who are in the process of internal validity), especially when they are used to
leaving the industry or who are contemplating leav- isolate the effects of a small number of factors. This
ing are often stigmatized and humiliated for their does not mean that the experimental method is ne-
past behaviours, and that the voices of such people cessarily the best design to use. As we shall see below,
must be heard if government transition programs experiments, like other methodological approaches,
are to be effective. Qualitative researchers will have their strengths and weaknesses when they are
choose research designs that will produce these sorts used in social research, and are better suited to some
of idiographic explanations. research topics than others.
Once a design has been selected, a specific method
for collecting data has to be chosen. There are many Manipulation
different ways of gathering data. One way is to use a If experiments are so strong in establishing caus-
preset instrument, such as a self-completion question- ation, why do social researchers not make more
naire or a structured interview schedule. Another is use of them? The reason is simple: to conduct a
to utilize a less formalized method like participant true experiment, it is necessary to do something to
observation or ethnography in which the researcher people and observe the effects. To put the matter
takes part in the activities of a group of people, some- more formally, an experiment manipulates an
times even living among them for a time. independent variable to determine its influence
Consider one of the research designs to be covered on a dependent variable. Typically, some subjects
in this chapter: the case study. It entails a detailed are allocated to a “treatment” group in which the
exploration of a specific case, which could be a com- independent variable is changed or manipulated,
munity, an organization, a person, or an event. Once while others are placed in a “control” group where
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

no manipulation takes place. The dependent vari- the rise of second-wave feminism in the 1960s?
able is then observed and measured. The problem is Unfortunately not: experiments are generally lim-
that many of the independent variables of concern ited to relatively simple, short-term manipulations
to social researchers cannot be manipulated. For of independent variables (see Brannigan, 2004).
example, some sociologists maintain that a coun- A further reason is that even where social scien-
try’s national character is affected by whether the tists are successful in using experiments to iden-
country came into existence through revolution tify causal variables, the perceptions and feelings
(for example, Lipset, 1990). In order to test this idea of the participants—which provide information
experimentally, revolution would have to be in- vital to a full understanding of the phenomenon
duced in some randomly selected areas to produce in question—are usually not examined in depth
new countries, while in others no revolution would (Dobash & Dobash, 2000).
be fomented. The researcher would then compare Before moving to a more complete discussion of
the national characters of the revolutionary coun- experimental designs, it is important to introduce
tries with the non-revolutionary ones. Needless a basic distinction between laboratory and field
to say, such experimental manipulations would experiments. The former take place in artificial set-
usually be impossible to carry out. tings, whereas the latter occur in real-life surround-
This example also illustrates a second reason ings such as classrooms and factories. The Rosenthal
why experiments are so rare in sociology: ethical and Jacobson (1968) study described below is a well-
concerns often preclude them. Suppose you are in- known example of a field experiment.
terested in the effect of poverty-induced poor nutri-
tion on children’s academic performance. In order Classic experimental design
to examine this issue experimentally, you could Rosenthal and Jacobson (1968) tried to determine
select a number of six-year-olds and randomly what effect teachers’ expectations had on their
assign them to one of two groups. Those in the first students’ academic performance. In addition to
group would be underfed for a month, while those illustrating the classical experimental design, their
in the second would be given ample, nutritious study is a good example of research that gets a lot
diets. Then you would test the two groups to see if of attention, is then subject to intense methodo-
they differed in academic ability. Although sound logical scrutiny (see Brannigan, 2004, pp. 80–89
from a methodological standpoint, such a study for critiques), and ultimately provides the impetus
would clearly raise serious ethical problems (and for further research designed to improve on the
did, when it was discovered that similar nutrition original.
studies were performed on Indigenous students at The research was conducted in a US school where
residential schools in British Columbia, Manitoba, many poor and minority-group children were en-
Alberta, Ontario, and Nova Scotia in the 1940s and rolled. In the spring all the students were given a test
1950s). In fact, as we will see in Chapter 3, ethical and told it was designed to identify “spurters”—that
issues may arise even when the experimental ma- is, students who were likely to experience a sudden
nipulation appears to be innocuous and is of very improvement in their academic performance. At the
short duration. beginning of the following academic year, the teach-
Another reason why experiments are uncom- ers were given the names of the spurters in their
mon in sociology is that many of the things of classes. But this was a ruse: the students identified as
interest to sociologists—gender roles, political spurters were simply selected at random; they were
preferences, the formation of social movements, not chosen on the basis of their test performance.
and so on—have complex, long-term causes that The test was re-administered eight months after the
cannot be easily simulated in experiments. Could original one, allowing the authors to compare the
the experimental method be used to explain so-called spurters with the other students on things
bry29440_ch02_027-049.indd 12/19/18 01:25 PM

such as IQ scores, reading ability, and intellectual To capture the essence of the classical experi-
curiosity. Since Rosenthal and Jacobson believed mental design, the following simple notation is used:
that there was no initial difference in ability between
the spurters and the others, any improvements were • Obs: an observation made of the dependent
attributed to the fact that the teachers had been led variable. There may be more than two obser-
to expect the spurters to perform better. The authors vations but, to simplify, the design shows the
report that the teachers’ expectations that the spurt- most common configuration: pre-test and
ers would show superior academic performance ac- post-test; here, IQ test scores before the experi-
tually caused those students to do better than the mental manipulation and after.
others, presumably because they received differen- • Exp: the experimental treatment (independ-
tial treatment from the teachers. ent variable); here, the creation of teacher
The Rosenthal and Jacobson study includes most expectancies. No Exp refers to the absence of
of the essential features of what is known as the an experimental treatment and represents the
classical experimental design. Subjects (in this case, experience of the control group.
students) are randomly assigned to two groups. The • T: the timing of the observations made in re-
experimental manipulation (different levels of the in- lation to the dependent variable; here, the time
dependent variable, here heightened teacher expect- when an IQ test is administered
ations) is carried out on the experimental group or
treatment group (the spurters). The other group is not What is the purpose of the control group? Surely
given the treatment and thus forms a control group. it is what happens to the spurters (the experimental
The dependent variable—academic performance—is group) that really matters. For this study to be a true
measured before the experimental manipulation to experiment, however, it must control for (in other
make sure that the two groups really are, on average, words, eliminate) rival explanations of its causal
equal at the start (see Figure 2.1). If they are equal, findings, leaving teacher expectations as the only
and because of random assignment they should be, factor that could have created any differences in per-
the researchers can feel confident that any differ- formance between the two groups. Of course, student
ences in student performance found between the two performance is a complex phenomenon, with many
groups after the manipulation were due to the treat- causes, but the present study wanted to examine only
ment. Everything else about the two groups is pre- one of those causes (teacher expectations). The pres-
sumed to be the same, leaving differences in teacher ence of a control group and the random assignment
expectations as the only possible explanation for any of subjects to the experimental and control groups
differences that are found between the spurters and help to eliminate rival explanations for differences
non-spurters. in academic performance, which in this case are any
explanations other than different teacher expecta-
tions. To see this, consider some threats that would
8 months
pose serious challenges to the study’s conclusions
Experimental T1 T2
group
if there were no control group or random assign-
Obs1 Exp Obs2
spurters IQ Teacher IQ
ment. The following list is based on a book written
expectancies by Cook and Campbell (1979). In each situation, the
Random
possibility of a rival interpretation of Rosenthal and
assignment
Jacobson’s findings is proposed, but the presence of
Control Obs3 No Exp Obs4 both a control group and random assignment greatly
group IQ No teacher IQ
non-spurters
reduces the threat. As a result, confidence in the
expectancies
finding that teacher expectations influenced student
FIGURE 2.1 Classical experimental design performance is enhanced.
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

• History. This refers to events other than the they got older, regardless of the effect of teacher
manipulation of teacher expectations that expectancies. But the control group would also
might have caused the spurters’ scores to rise. mature, so maturation effects cannot explain
For example, suppose that the school’s princi- the differences between it and the control group.
pal had taken steps to raise standards in the • Selection. When subjects are not assigned ran-
school. Without a control group, we could not domly to the experimental and control groups,
be sure whether it was the teachers’ expecta- variations between them in the post-test may
tions or the principal’s action that produced be due to pre-existing differences between the
the increase in spurters’ grades. With a control two groups. For example, if all the best stu-
group, we can say that the principal’s action dents were given the spurter label, it might
should have had an effect on the control sub- have been their pre-existing academic ability
jects too, and therefore that the differences rather than teacher expectations that caused
between the experimental and control groups them to do better. However, since a random
can be attributed to the effect of teacher ex- selection process was employed here, the “se-
pectations alone. lection” risk is greatly reduced. With random
• Testing. This threat refers to the possibility assignment, the top students would make up
that subjects may become more experienced roughly the same proportion of the spurter
at taking a test or sensitized to the aims of group as the non-spurter group, cancelling
the experiment as a result of the pre-test. The out the effect of pre-existing academic ability.
presence of a control group, which presumably However, even with the use of random assign-
would also experience the same things, dimin- ment, if the number of people in each group
ishes this possibility. is relatively small, there is still a risk of pre-
• Instrumentation. This threat refers to the possi- existing differences between the experimental
bility that changes in the way a test is adminis- and control groups.
tered can account for an increase (or decrease)
in scores between a pre-test and post-test; for Even if all these threats have been overcome, fur-
example, perhaps the teachers know their stu- ther issues may arise. First there is the question of
dents better or are more friendly the second whether the variables used in the study have been
time they give the test. Again, if there is a con- adequately measured. This refers to the matter of
trol group, the people in that group should be measurement validity, a topic addressed in detail in
affected as well. Chapter 4. In the case of the Rosenthal and Jacob-
• Mortality. A particular problem for studies that son (1968) study, there are potentially two aspects
span a long period of time is the risk that some to this question. First, has academic performance
subjects will leave the experiment before it is been adequately measured? Reading scores seem to
over: for example, some students might move correspond to what they are supposed to be measur-
to a different school, or experience a long-term ing. However, given the controversy surrounding IQ
illness. Since this problem is likely to affect tests and what they measure, we may feel unsure that
the control group too, it may not make a dif- gains in IQ test scores can be regarded as strongly
ference to the results. However, experimenters indicative of academic performance. Similarly, to
should try to determine whether mortality has take another of the authors’ measures, is intellec-
affected the experimental and control groups tual curiosity a valid measure of academic perform-
differently. ance? Does it really measure what it is supposed to
• Maturation. Quite simply, people change over measure?
time and the ways in which they change may Another question is whether the experimental
have implications for the dependent variable. manipulation really worked. In other words, was
The spurters might have improved anyway as the identification of some schoolchildren as spurters
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

enough to create the conditions needed for the hy- the result that their responses are affected as
pothesis about teacher expectations to be tested? The they become more test-wise. Consequently, the
study depended on the teachers’ remembering the findings may not be generalizable to groups
“information” they were given about the spurters that have not been pre-tested, and, of course,
for the duration of the experiment, but it is possible in the real world people are rarely pre-tested.
that as time went on some of them came to doubt or This may have occurred in the Rosenthal and
forget it; in that case, the manipulation would have Jacobson research, since all students were pre-
been contaminated. tested at the end of the previous academic year,
A second set of issues concerns the generaliz- so the students new to the school in the fall
ability of a study’s findings. In other words, do the (when the teachers were told about spurters)
study’s conclusions apply to other people, settings, presumably were not pre-tested.
or time periods? These matters pertain to the study’s • Reactive effects produced by the experimental
external validity. Cook and Campbell (1979) iden- arrangements. People are frequently, if not in-
tified five major threats to the external validity and variably, aware of the fact that they are par-
hence the generalizability of findings derived from ticipating in an experiment. Their awareness
an experiment: may influence how they respond to the experi-
mental treatment; for example, they may react
• The representativeness of the study participants. by behaving in a socially acceptable manner
To what social and psychological groups can a rather than sincerely and spontaneously,
finding be generalized? Can it be generalized to which could affect the generalizability of the
a wide variety of individuals who differ in eth- findings. Since Rosenthal and Jacobson’s sub-
nicity, social class, religion, gender, and type jects do not appear to have been aware that
of personality? In the Rosenthal and Jacobson they were participating in an experiment,
study, the students were largely from poorer this problem is unlikely to have been signifi-
groups and a large proportion from ethnic cant. The issue of reactivity and its potentially
minorities. This may limit the generalizability damaging effects is a recurring theme in many
of the findings. types of social research.
• The effects of the setting. Can the results of a
study be applied to other settings (in Rosenthal Then there is the question of replicability. A study
and Jacobson’s case, to other schools)? There is is replicable if others are able to repeat it and get the
also the wider issue of whether expectation ef- same results. Sometimes replications are conducted
fects can also be discerned in non-educational to make sure that the original research was carried
settings. out properly; this is especially important if a study’s
• History effects. This threat raises the question results do not match prior findings on the topic. For
of whether the findings can be generalized to replication to be possible, the initial researcher must
the past and into the future. The Rosenthal and spell out all the research procedures in great detail.
Jacobson research was conducted more than Rosenthal and Jacobson laid out their procedures
50 years ago. Would the findings still apply and measures in detail, and anyone carrying out a
today? Also, their investigation was conducted replication could obtain further information from
at a particular juncture in the school year. them. Consequently, their research is replicable, al-
Would the same results have been obtained if though there has never been an exact replication.
the research had been conducted at different Clairborn (1969) conducted one of the earliest rep-
points in the year? lications and followed a procedure very similar to
• The effects of pre-testing. As a result of being pre- Rosenthal and Jacobson’s, although the study was
tested, subjects in an experiment may become carried out in three middle-class suburban schools,
sensitized to the experimental treatment, with and the timing of the creation of teacher expectancies
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

was different from the original study. Clairborn also randomly assigned, were not told this. The data
failed to replicate Rosenthal and Jacobson’s findings, showed that women scored lower than men when
casting doubt on the external validity of the original informed of this “fact.” When told that the test was
research and suggesting that the first two threats to to compare Canadians and Americans, there was
external validity referred to earlier in this chapter no gender difference. Without the chance to ran-
may have limited the applicability of the findings. domly assign subjects, there would always be some
Virtually all experiments in the social sciences doubt that the treatment (being told that women
involve deception of some kind. In the Rosenthal score lower than men on the test) rather than a pre-
and Jacobson study, for example, the experimenters existing difference between the two sets of subjects
told the teachers that certain students were spurters, actually caused the women to do less well than the
which simply was not true. But could the experi- men. Also, laboratory experiments are more easily
ment have been carried out without any deception? replicated because they are less bound to a natural
For example, could the experimenters have told the milieu that could be difficult to reproduce.
teachers that they were testing the effects of teacher However, laboratory experiments also suffer from
expectations on student achievement? Could they a number of limitations. Many of those shortcomings
have given the teachers a list of students and said, arise from low external validity. For instance, the ex-
“Pretend that these students are gifted, although perimental setting may not mirror real-world experi-
they are not any more gifted, on average, than the ences and contexts, despite the fact that the subjects
students not on the list. We want to see whether you are very involved in most experiments and take them
would treat such children differently, and whether very seriously. In addition, the treatment effects may
differential treatment affects their academic per- be unique to the people in the study; others may not
formance.” Surely, that sort of approach would not react the same way. In the case of Fisher and Ma’s
have produced authentic behaviour on the part of the (2014) study described in Box 2.1, for example, the
teachers, and the teachers might well have refused to subjects were drawn from an online panel and hence
participate under those conditions for ethical and may not be representative of the general population,
practical reasons. Clearly, some form of deception if only because their willingness to go online and
was necessary for the experiment to work. their facility with the Internet may have made their
But deception raises ethical concerns—it is basic- responses to the experimental stimuli distinctive.
ally a form of lying. Chapter 3 will discuss the ethical They were also volunteers, who generally differ from
implications of using deception in social research, non-volunteers (Rosnow & Rosenthal, 1997, ch. 5). In
as well as other sorts of ethical dilemmas facing re- addition, the fact that they were given an incentive to
searchers in the social sciences. participate (they were paid a small fee) might have fur-
ther distinguished them from others since not every-
The laboratory experiment one would be equally motivated to earn the reward
One of the main advantages of laboratory over field given to those who participate. There was no effect
experiments is the researcher’s greater control over of pre-testing because, as in many experiments, the
the research environment. In particular, it is easier to participants described in Box 2.1 were not pre-tested.
randomly assign subjects to different experimental However, it is quite possible that reactive effects oc-
conditions in the laboratory than in a real-life curred: the subjects knew they were in an experiment,
situation, which enhances the researcher’s ability and that may have affected their behaviour.
to establish nomothetic causation. For example,
Walsh and colleagues (1999) were able to tell some Quasi-experiments
randomly assigned university students in Eastern Quasi-experiments have some characteristics of
Canada that previous results on the mathematics test the experimental model but lack some of the fea-
they were about to take showed that women perform tures that help the researcher establish causation.
less well than men on such tests. Other students, Several different types of quasi-experiments have
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

BOX 2.1 A laboratory experiment
It is commonly recognized, both in scientific circles from a village in Africa that had just been struck
and in everyday life, that there are tangible bene- by a mudslide, making her a homeless orphan.
fits associated with being physically attractive. It is People experiencing the “low severity” manipula-
also well known that attractive people are thought tion were told of the mudslide, but were informed
to possess more positive attitudes and behavioural that the young girl’s home and parents were not
traits than the less attractive; hence the “beauti- harmed by it. The results indicated that subjects
ful is good” stereotype. Yet two researchers at the in the low severity condition who were presented
University of Alberta (Fisher & Ma, 2014) found that with the “attractive” version of the girl’s photo-
under certain experimental conditions, partici- graph showed lower levels of empathy toward her
pants felt less empathy toward an “attractive” child. than those shown the less attractive image: the
The researchers wanted to know whether the opposite of the “beautiful is good” stereotype. In
attractiveness of children in need affects the em- the high severity condition, physical attractive-
pathy that non-relatives feel toward them. In one ness had no effect on the level of empathy. The
version of the study, the perceived attractiveness researchers reasoned that when severe harm is
of a young girl was manipulated by showing one not imminent, people perceive attractive chil-
group of participants a picture of her that had not dren to be more socially competent and hence
been altered. A second group was shown a version more capable of taking care of themselves. But
of the picture that had been digitally modified to when children are in grave danger or distress,
make the girl appear less attractive. A “high se- those superficial considerations are overridden by
verity” of need condition was created by telling strong feelings of compassion and a desire to help,
some participants that the child in the picture was regardless of how attractive the child may be.
been identified (Cook & Campbell, 1979). Not all groups may not have been equivalent on all relevant
of them can be covered here, but a particularly in- characteristics before the independent variable was
teresting type are “natural experiments,” in which introduced. For instance, in the earthquake example,
experiment-like conditions are produced by natur- there may have been some pre-existing differences
ally occurring phenomena or changes brought about between the two cities before the earthquake struck
by people not doing research. When that occurs, re- that contributed to post-earthquake differences. So
searchers can gather data in much the same way they if the stricken city showed an increase in civic pride,
do in experiments. For example, if an earthquake hit perhaps that was not because of the earthquake, but
a particular city, but a city of comparable size and because it had a more charismatic mayor than the
composition a short distance away was spared, the neighbouring city. However, the results of such stud-
conditions for a natural experiment would be present. ies are still compelling because they are real rather
The effects of the natural disaster on civic pride or at- than artificial interventions in social life, making
titudes toward local political leaders could be meas- them high in external validity.
ured by comparing the two cities on those variables. St Helena in the South Atlantic provided a fascin-
However, in natural experiments, it is usually im- ating natural laboratory for the examination of vari-
possible to randomly assign subjects to experimental ous claims regarding the effect of television violence
and control groups. The absence of random assign- on children when TV was introduced to the island for
ment casts doubt on any causal inferences, since the the first time in the mid-1990s. The findings—from
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

video footage showing young children at play during allows researchers to estimate the strength of environ-
school breaks, from diaries kept by about 300 of the mental and genetic influences on a variety of attitudes
children, and from ratings by teachers—suggested and behaviours (Plomin et al., 2013). The presence
that the introduction of television was not followed of greater behavioural or attitudinal similarities
by an increase in the number of aggressive acts ob- among monozygotic than dizygotic twins suggests
served (Charlton et al., 1998, 1999). The research- a genetic influence. For example, recent behavioural
ers suggest that in environments such as St Helena, genetic studies indicate that social and political atti-
where children are closely watched by the commun- tudes are affected not only by societal and situational
ity and are expected to avoid violence or aggression, factors, but also by genes (Alford et al., 2005; Bell &
television may have little effect on their behaviour. Kandler, 2017; Bell et al., 2009, 2012). Additional ex-
Another type of natural experiment can be found in amples of quasi-experiments are presented in Box 2.2.
twin studies, which are commonly used in behavioural Quasi-experimental designs have been particu-
genetic research. Monozygotic (popularly known as larly prominent in evaluation research studies, which
“identical”) twins share 100 per cent of their genetic examine the effects of organizational innovations such
structure, whereas dizygotic (“fraternal”) twins have as a longer school day or greater worker autonomy in a
in common only about 50 per cent of the genes that plant (see Box 2.3). Sometimes the results are surpris-
vary between human beings. This natural difference ing. A quasi-experimental investigation on the effect of
BOX 2.2 Quasi-experiments
A common type of quasi-experiment compares real social world, and as such have to meet goals
data collected before and after a policy shift by that are very different from those of experimental
government or industry. For example, the num- research. That makes it next to impossible for gov-
bers of car accidents before and after the lowering ernments to randomly assign people to different
of a speed limit can be compared. If the number conditions. Therefore, they usually have to make
goes down after the speed limit has been lowered, do with quasi-experimental evaluations of policy
the policy would appear to be a success, but one changes. A criminologist may want to randomly
cannot be sure because there is no control group assign criminals to jail or home custody and then
not experiencing the change. Perhaps the change compare the two forms of detention, but the po-
was caused by something other than the reduction tential for some of the at-homes to reoffend would
in the speed limit, such as increased media cover- be seen as too great a risk. Because minor crim-
age of car accidents. inals are more likely to get home custody than
Another example might involve the installa- are more serious criminals, a fair test is not pos-
tion of cameras to detect speeding. If the cameras sible without random assignment. Similarly, when
are placed only in randomly selected places and Canada abolished the death penalty, it was not for
not in others, the research changes from a quasi- the purpose of conducting an experiment: aboli-
to a real experiment, since the locations with- tion was a policy decision. Data could be examined
out cameras would constitute a control group. as if the change were part of a quasi-experiment,
Governments find it difficult, however, to subject for example, by comparing the murder rate before
some people to one condition and others to an- and after capital punishment was abolished; but
other. The prison system, schools, and other insti- the legislation was definitely not intended for that
tutions controlled by governments are part of the purpose.
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

support for people who take care of the elderly (Demers, comparison: at the very least, an experiment allows
1996) showed that the extra support made caregivers the researcher to compare the results obtained from
feel less depressed, but more burdened. Demers was an experimental group with those obtained from a
not sure why, although she speculated that perhaps control group. In the case of the Fisher and Ma (2014)
the help was seen as something else to be coordinated experiment in Box 2.1, the research compared the ef-
and managed. The situation that Demers found herself fects of different levels of a child’s physical attract-
in—not being sure how to interpret her findings—is iveness and distress on the level of empathy felt by an
actually quite common. It’s one thing to come up with observer. The advantage of such comparisons is that
findings like hers, but another to explain them. The they permit a better understanding of the phenom-
solution usually involves doing more research. enon in question than would be possible if it were
Quasi-experiments are also used to evaluate the examined under one condition alone. The argument
effectiveness of institutional policies. For example, that in certain situations greater physical attractive-
Hanson et al. (2004) wanted to know whether treat- ness in children evokes less empathy is much more
ing sex offenders after they are released from prison persuasive when the empathy felt for attractive chil-
has any effect on their chances of reoffending. A total dren can be compared with that elicited by those per-
of 724 convicted male sex offenders were divided into ceived to be less attractive. While the experimental
two groups: those who underwent mandatory treat- design is typically associated with a quantitative re-
ment after release to community supervision, and search strategy, the specific logic of comparison pro-
those who were released prior to the implementation vides lessons of broad applicability and relevance.
of the mandatory treatment program. (Some of the
latter group had received treatment during or prior Cross-sectional design
to incarceration, although the extent of that treat- Many people associate cross-sectional designs with
ment was not known.) This division was not perfect, questionnaires and structured interviewing. How-
since the men in the first group may have differed ever, other data-gathering techniques may also be
in some relevant way from those in the second; the used in cross-sectional research, including struc-
fact that some in the latter category received treat- tured observation and analysis of official statistics
ment is also an issue. In any case, the results showed or diaries. These will be covered in later chapters,
that, after an average 12-year follow-up period, there but we will outline the basic structure of the cross-
were equal rates of reoffending in both groups: about sectional design here. In cross-sectional studies,
20 per cent for sex crimes, which was not very en- observations are taken at one point in time—there
couraging for the professionals who treat offenders. are no before-and-after comparisons. Also, cross-
sectional designs do not involve any manipulation
Significance of experimental design of the independent variable: they are like snapshots
As we noted at the outset of this section, true experi- taken of a group or phenomenon at one point in time.
ments are rare in sociology, but they are worth dis- A cross-sectional design entails the collection of
cussing because, where they are practicable, they allow data (usually quantitative) on more than one case.
researchers to isolate the causal influence of a particu- Researchers are interested in variation between dif-
lar variable—a goal to which researchers using other ferent people, families, nation-states, and so on, and
designs also aspire. Cross-sectional designs of the kind variation can be established only when more than
associated with survey research (discussed below) one case is examined. Researchers employing this
offer another way of evaluating causal hypotheses. design usually select many cases, for at least two
reasons. For one thing, a larger number makes it
Logic of comparison more likely that variation will be encountered in all
It is important to draw attention to a significant the variables of interest. A second reason is that cer-
general lesson that an examination of experiments tain statistical techniques are likely to require large
reveals. A central feature of any experiment is a sample sizes (see Chapter 7).
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

BOX 2.3 Evaluation research
A key question asked in evaluation research is at its completion. The researchers also did a quali-
whether a new policy initiative or organizational tative analysis of the project by conducting five
change achieved its goals. Ideally, to answer that focus groups at the conclusion of the program. The
question the design would have one group that is quantitative results indicated that the participants
exposed to the treatment—the new initiative—and had higher levels of perceived overall health and
a control group that is not. Since it is often not feas- sense of community, and lower levels of physical
ible or ethical to randomly assign research partici- pain, when the program was over. The themes
pants to the two groups, such studies are usually that emerged from the focus groups included the
quasi-experimental. For instance, data gathered conclusions that the program provided the seniors
from people before a change may be compared with structure and discipline, facilitated coping, re-
with data acquired after; the “before” people quired hard work and effort, brought out their art-
become the control group, the “after” people the istic side, promoted social involvement, and made
experimental group. This approach has the added a positive contribution to the community.
advantage that the two groups are basically the Quantitative quasi-experimental designs in
same, making random assignment unnecessary. evaluation research go back a long way, but as the
Such a design was used to evaluate the effect Phinney et al. (2014) study indicates, evaluations
of a community arts program on the well-being of based on qualitative research have also emerged.
older adults in the Vancouver area (Phinney et al., Although there are differences of opinion about
2014). Over three years, four groups of participants how qualitative evaluation should be carried
took part in the collective creation of a physical out, there is consensus on the importance of,
work of art or a performance that was presented first, understanding the context in which an inter-
to the public. Baseline quantitative measures of vention occurs and, second, hearing the diverse
well-being were taken in the first year of the pro- viewpoints of the stakeholders (Greene, 2000).
gram, with the same measures administered again For example, Pawson and Tilley (1997) advocate
In quantitative studies, data are collected on to difficulties in holding down a job and thus
two or more variables, which are then examined poverty? Or is it a bit of both? To take another
to detect patterns of association. This approach example, a study of 1000 men found that those
sometimes makes it difficult to show cause and who had two or more orgasms a week exhibited
effect because the independent and dependent a 50 per cent lower mortality risk compared with
variables are measured simultaneously, making men who had on average less than one orgasm
any demonstration of time order (showing that per week. It may be tempting to conclude that
the cause actually precedes the effect in time) hard male orgasm leads to longer life expectancy, but
to prove. For example, there is a well-supported it is also possible that the causal arrow points in
negative relationship between social class and the other direction: men who are ill (and thus at
serious forms of mental illness: more poor people greater mortality risk) are less likely to be sexually
are mentally ill than rich people. But there active in the first place (Houghton, 1998, p. 14).
is also a debate about the nature of that relation- This finding and the preceding one are similar,
ship: Does being poor lead to stress and therefore showing what Blaxter (1990) called “an ambiguity
to mental illness? Or does being mentally ill lead about the direction of causal influence.” There is
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

CCTV deters car crime. For instance, it serves as

a direct deterrent to offenders (fewer people are
willing to commit crimes if they think they will
be filmed and caught), which encourages greater
Monkey Business Images/Shutterstock
usage of parking lots, which in turn increases

personal surveillance, which itself deters crime.
Examples of contexts relevant to the interven-
tion include time (such as when the parking lot
fills up and empties during rush-hour periods, or
slow times during the day), and the availability of
other nearby venues for offenders to commit car
As a researcher studying the effects of a music program, crimes. The kind of evaluation research advocated
what questions would you ask participants before and by Pawson and Tilley maps the different combina-
after the program in order to gauge its impact on their tions of cause and context in relation to different
health? Would you focus on their objective health outcomes.
indicators, or the subjective perception of their own Evaluation research is not limited to examining
health? What might be the benefit of doing both?
whether an initiative has achieved its stated goals.
In fact, some approaches are “goal free” (Scriven,
a pluralistic methodology that examines not only 1991) in that the people doing the evaluation are
the context but also the mechanisms that allow not told what the purpose of the program is. This
programs to work. Tilley (2000) provided an ex- allows evaluators to assess its overall impact, in-
ample of the approach in an evaluation of the use cluding unforeseen consequences, rather than
of closed-circuit television (CCTV) in parking lots. limit their attention to how well it achieved
He observed that there are several ways in which specific, preconceived goals.
only an association between the two variables—no administering research instruments (struc-
clear causal link. However, as will be shown below tured interview or self-completion question-
and in Chapter 8, there are a number of ways for naire, etc.), and the analysis of data.
researchers to draw cautious inferences about • Establishing causality can be problematic.
causality using cross-sectional designs. As was just suggested, it may be difficult to es-
tablish causal direction from the resulting data.
Replicability, causal inferences, Cross-sectional research designs can identify
and external validity associations that are indicative of causation,
How does cross-sectional research measure up in but other designs may have to be employed to
terms of replicability, the ability to establish causa- substantiate causal inferences.
tion, and external validity? • External validity is strong when the sample is
a random one. When non-random methods
• Replicability characterizes most cross-sectional of sampling are employed, external validity
research, so long as the researcher spells out becomes questionable, an issue addressed in
the procedures for selecting respondents, Chapter 7.
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Methods in Motion | Applications to Canadian Society
Why are some Canadians politically conservative and others liberal?

Bell and his colleagues (2009) did a behavioural “nurture” interact in the formation of political
genetic study of political attitudes and behaviours attitudes.
that addressed the issue of why some Canadians These sorts of studies are uncommon in soci-
tend to have conservative or right-leaning atti- ology, political science, or anthropology, although
tudes, while others are more liberal or left ori- they are conducted more frequently in psychology
ented in their political views. The methodology and the health sciences. They have far-reaching
used in the study (the details of which go beyond implications for the social sciences, in particular
the scope of this book) was a kind of natural ex- for disciplines in which biological influences are
periment comparing monozygotic (identical) seldom examined. The standard paradigm in the
twins, who are essentially genetic clones of each latter fields of study assumes that some combin-
other, with dizygotic (fraternal) twins, who share ation of socialization, culture, and situational fac-
only about 50 per cent of the genetic material that tors can explain the origins of most attitudes and
varies between human beings. The researchers behaviours. Researchers using behaviour genetic
examined how similar the identical twins were in methods, on the other hand, acknowledge that
their political attitudes, and compared that with those sorts of influences are crucial, but argue
how similar the fraternal twins were on that score. that a more complete understanding of the human
If identical twins are more similar on the character- condition can be achieved by considering both
istic under examination than the fraternal twins, biological and environmental factors.
that would indicate that genetic factors or other If behaviour genetics and other approaches that
biological influences have some impact on the consider both biological and environmental influ-
trait in question. The results of the study showed ences were to be more widely adopted in the social
that the identical twins were in fact much more sciences, that could have a major impact on the theor-
similar in their political attitudes than the frater- etical perspectives that are developed and rise to
nal twins, which suggests that genetic factors do prominence in academe. For example, a blend of Dar-
play a role in the formation of these attitudes. The winian evolutionary theory and sociological conflict
study also found that environmental factors (such theories is a possibility (e.g., Sanderson, 2001). Such
as educational influences and media effects) in- approaches have the potential to promote consili-
fluenced political orientations, although it did not ence or the unity of knowledge, (Bell & Kandler, 2017)
test for specific environmental influences. Taken which in this case would take the form of a closer re-
as a whole, the results indicate that “nature” and lationship between the social and biological sciences.
Variables that cannot be manipulated socioeconomic status are “givens” and not really
amenable to the kind of manipulation necessary
As noted in the section on experimental design, for a true experimental design. Fictitious manipu-
in much (if not most) social research it is not lations are possible, such as when an experimenter
possible to manipulate the variables of interest. digitally alters photographs to produce different
This is a key reason why most quantitative social ages or ethnicities, perhaps to see the effects on job
research employs a cross-sectional design rather offers, but the manipulation is limited to the exter-
than an experimental one. To more or less all in- nal signs of age and ethnicity, missing the more sub-
tents and purposes, things like ethnicity, age, and jective and experiential aspects.
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

However, the very fact that certain variables are accounts of factors that affected their past and
givens provides a clue as to how to make causal in- present behaviour.
ferences in cross-sectional research. Many of the
variables of interest can be assumed to be tem- Structure of cross-sectional designs
porally prior to other variables. For example, in a A cross-sectional design collects data on a series of
relationship between ethnic status and alcohol con- variables (Obs1, Obs2, Obs3, Obs4, Obs5, . . . Obsn) for
sumption, the latter cannot be the independent vari- different cases (people, households, cities, nations,
able because it occurs after ethnicity. Ethnicity still etc.) at a single point in time. The effect is to create
cannot be said with certainty to be the cause, how- a data set that comprises variables Obs1 to Obsn and
ever: it is just a possible cause. In other words, even cases case1 to casen, as in Figure 2.2. Each cell in the
though researchers are unable to manipulate things matrix has data in it.
like ethnic status or gender, causal inferences can
still be cautiously drawn from cross-sectional data. Longitudinal design(s)
The current discussion of the cross-sectional With a longitudinal design, cases are examined at a
design places it firmly in the context of quantitative particular time (T1) and again at a later time or times
research. But qualitative research can also use a form (T2, T3, and so on), but without the manipulation
of cross-sectional design. For example, Beardsworth of an independent variable that characterizes ex-
and Keil (1992) carried out a study of the dietary be- periments (see Box 2.4). When used in quantitative
liefs and practices of vegetarians. They administered research, it allows insight into the time order of vari-
“relatively unstructured interviews,” which were ables and is better able to deal with the problem of
“guided by an inventory of issues,” with 76 vegetarians “ambiguity about the direction of causal influence”
and vegans (1992, p. 261). The interviews were taped that plagues cross-sectional designs. Because poten-
and transcribed, yielding a large body of qualitative tial independent variables can be identified at T1, the
data. The research was not preoccupied with quanti- researcher is in a better, if not perfect, position to
tative criteria such as establishing causation, external infer that the effects identified at T2 or later occurred
validity, replicability, measurement validity, and so after changes occurred in the independent variables.
on. Nonetheless, the conversational interview style In all other respects, the points made above about
made the study more externally valid than research cross-sectional designs are the same as those for
using more formal instruments of data collection. longitudinal designs. In spite of its heightened abil-
The study was concerned with the factors that in- ity to show cause and effect, the longitudinal design
fluence food selection, like vegetarianism. The very is not frequently used in quantitative social research
notion of an “influence” carries a strong connotation because of the additional time and cost involved.
of causality, suggesting that qualitative researchers
can also be interested in the investigation of causes
and effects, although they do not use the language Obs1 Obs2 Obs3 ... Obsn
of quantitative research with its talk of independ-
Case1
ent and dependent variables. As well, the emphasis
was much more on understanding the experience of Case2
something like vegetarianism than is often the case Case3
with quantitative research.
Case4
This qualitative research bears many similarities
to the cross-sectional design in quantitative re- Case5
search. It entailed interviewing quite a large number ...
of people at a single point in time. And, as with Casen
many quantitative studies using a cross-sectional
design, the examination of people’s past and current FIGURE 2.2 The data rectangle in cross-sectional
eating habits was based on the subjects’ retrospective research
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

BOX 2.4 Longitudinal research and the case study
Case study research frequently includes a longi- A longitudinal element also occurs when a
tudinal element. The researcher can be a partici- case that has been studied is returned to at a
pant observer in an organization for an extended later time. A particularly interesting example of
length of time, or may do ethnographic research this occurred in “Middletown,” a pseudonym for
with a community for many months or years, or an American Midwest town first studied by Lynd
may conduct structured or qualitative interviews and Lynd (1929) in 1924–5 and restudied in 1935
with individuals over a prolonged period. More- during the Depression to see what changes had
over, the researcher may be able to inject an addi- occurred (Lynd & Lynd, 1937). In 1977, the com-
tional longitudinal element by analyzing archival munity was again restudied, this time in a post–
information and asking respondents to recall Vietnam War setting (Bahr et al., 1983), using
events that occurred before the study began, thus the same research instruments but with minor
discovering some history. changes.
Longitudinal studies can take different forms. There are two basic types of longitudinal design:
For example, Goyder et al. (2003) examined how the panel study and the cohort study. With the
gender-influenced evaluations of occupational pres- former, the same people, households, or other groups
tige had changed over 25 years. Some of the earlier are studied on at least two different occasions. One
male advantage had disappeared; indeed, some oc- example is the National Longitudinal Survey of
cupations showing a female incumbent were rated Children and Youth (NLSCY), described in Box 2.5,
more highly than the same occupation with a male in which the same children were studied in succes-
incumbent. Baer et al. (2001) did a 15-nation study sive years. Another is the three-wave study of family
examining whether membership in clubs and associ- structure and children’s socioeconomic attainment
ations had changed over time. Kerr (2004) and Kerr (Seabrook & Avison, 2015) discussed in the Research
and Michalski (2007) investigated hyperactivity in in the News box (p. 43). A panel study, especially
Canadian children as they grew older and its sources one at the household level, needs rules for handling
in poverty and family structure. new entrants (e.g., as a result of marriage or elderly
BOX 2.5 The National Longitudinal Survey of Children and Youth
This panel study is a long-term effort to monitor Resources Development Canada. The study hopes
Canadian children’s development and well-being to follow the subjects until they are 25 years old,
as they mature from infancy to adulthood. It began and to contribute to the development of policies
with a representative sample of children 11 years that help children live healthy, active, and reward-
of age or younger in 1994–5 being interviewed, ing lives (see Michaud, 2001). The data for Cycle 8,
with follow-ups every two years. Statistics Canada which began in September 2008, were released in
collects the data, with direction provided by Human November 2010 (Statistics Canada, 2010).
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

relatives moving in) and exits (e.g., as a result of some subjects may die, some may move away, and
marital break-up or children leaving home). some may simply choose to withdraw at later stages
In a cohort study people sharing the same experi- of the research. For instance, in the study by Din-
ence, such as being born in the same year or gradu- ovitzer et al. (2003) on the educational attainment of
ating from a particular school at the same time, are immigrant youth, researchers were able to talk to only
studied over time, but the same people may not be 65 per cent of those originally surveyed 19 years ear-
studied each time. For example, Walters (2004) used lier. Comparing those who participated in the second
the 1982, 1986, 1992, and 1995 National Graduates round with those who were lost from the study, the
Surveys to examine trends in the economic fortunes authors found no significant differences. That sort of
of Canadians graduating from post-secondary insti- comparison is a common practice; when no difference
tutions. The same people were not selected for the is found, the losses are treated as random and thus ac-
sample each year, but the information was still useful ceptable to ignore. The main problem with attrition
in understanding each graduating cohort, as well as is that those who leave the study may differ in some
the similarities and differences between cohorts. important respects from those who remain, so that
Panel and cohort studies share similar features. the latter do not form a representative group. How-
In social sciences such as sociology, social policy, and ever, there is some evidence from panel studies that
human geography, quantitative research using these attrition declines with time (Berthoud, 2000); in other
designs usually takes the form of repeated survey words, those who do not drop out after the first wave
research using a self-completion questionnaire or or two of data collection tend to stay on the panel.
structured interview. Used in this way, panel and Second, there are few guidelines for deter-
cohort studies can both illuminate social change and mining the best timing for further waves of data
improve the understanding of causal influences. collection. Finally, there is evidence of a panel con-
Panel and cohort studies also share similar prob- ditioning effect, whereby continued participation in
lems. First, there is the problem of sample attrition: a longitudinal study affects respondents’ behaviour.
Research in the News

Family structure and children’s socioeconomic attainment in Canada
Seabrook and Avison (2015) conducted a panel attainment than children from stable two-parent
study of single-parent and two-parent families families. Another key finding was that, regardless
residing in London, Ontario. Interviews were con- of family structure, where mothers had higher
ducted in 1993, in 1994, and between 2005 and levels of education, the children were more likely
2008. The goal of the study was to determine the to graduate from college or university.
effect of variables such as family structure and The study caught the attention of the Globe
mothers’ education levels on the socioeconomic and Mail, which discussed the findings in an arti-
ranking of their children’s occupations upon cle on the election promises of the major polit-
reaching adulthood and their children’s likelihood ical parties in Canada (Anderssen, 2015). Jamie
of graduating from college or university. Surpris- Seabrook, the lead author, was quoted as saying,
ingly, the results indicated that children who grew “More attention should be paid to addressing dis-
up in temporally stable single-mother families parities in education and family income than con-
and those who were raised in households that cerns about the kind of families kids grow up in.”
began as single-parent but transitioned to a two- “If I prioritized,” he added, “education would be
parent structure actually had higher occupational right at the top.”
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Menard (1991) cited a study of family caregiving in • a sector of the economy, such as the seafood
which 52 per cent of respondents attributed a change processing industry in rural New Brunswick
in the way they cared for relatives to their participa- examined in Knott’s (2016) research on differ-
tion in the research. ent types of mobile labour.
As suggested above, qualitative research may also
incorporate elements of a longitudinal design. For There is a tendency to associate case studies with
example, it is used in ethnographic research when qualitative research, such as Pratt and Valverde’s
the ethnographer is in a location for a lengthy period (2002) study of the Toronto Somali community.
of time or when interviews are carried out on more Exponents of the case study design often favour
than one occasion in order to examine change. In an methods like participant observation and unstruc-
example of the latter, Smith et al. (2004) described a tured interviewing, which are viewed as particularly
study of the experiences of citizenship for 110 young helpful in generating an intensive, detailed examin-
people. They were interviewed in depth in 1999 and ation of a case. However, case studies can also use
then re-interviewed at two-year intervals to exam- quantitative methods, or some combination of quan-
ine changes in their lifestyles, feelings, opinions, titative and qualitative.
and ambitions in relation to citizenship issues. Only With a case study, the case is an object of interest
64 people participated in all three waves of data col- in its own right and the researcher aims to provide
lection, illustrating the high rate of sample attrition an in-depth elucidation of it. Sometimes it is diffi-
in this style of research. cult to distinguish case studies from other research
designs because almost any kind of research can be
Case study design construed as a case study. Even research based on
The basic case study design entails a detailed and in- a national, random sample of Canadians could be
tensive analysis of a single case. A case may be: considered a case study of Canada.
What distinguishes a true case study is the goal
• a single community, as in Hughes’s (1943) clas- of finding and revealing the features of the case.
sic study of Drummondville, a textile town in Collecting in-depth, often qualitative data that may
Quebec, or in Pratt and Valverde’s (2002) re- be unique to time and place is characteristic of this
search on Somalis in Toronto; sort of research. Case studies are often idiographic in
• a single family, as in Lewis’s (1961) study of the nature, seeking to provide a rich description of the
Sánchez family in Mexico; subject matter, like Shalla’s (2002) study of how Air
• a single organization, such as the automobile Canada’s customer sales and service agents became
factory studied by Rinehart (1996), or a group victims of airline restructuring.
within an organization, like the nurses in a When the predominant research orientation is
Hamilton hospital researched by White (1990); qualitative, a case study tends to take an inductive
• a person, as in Nemni and Nemni’s (2006) approach to the relationship between theory and re-
study of Pierre Trudeau or Foran’s book on search. If a mainly quantitative strategy is adopted,
Mordecai Richler (2010); such research is the research is often deductive, guided by specific
characterized by use of the life history or bio- research questions derived from social theories.
graphical approach (see Chapter 11);
• a single event, like the fight against locating a Measurement validity, causal inference,
home for recovering addicts in a Richmond, external validity, and replicability of
British Columbia, neighbourhood (Huey, case studies
2003), or the Alberta election analyzed by Bell The question of how the case study fares on the re-
et al. (2007); search design criteria of measurement validity, estab-
• a state or province, as in Laplante’s (2006) lishing causation, external validity, and replicability
study of the rise of cohabitation in Quebec; or depends in large part on whether the researcher feels
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

these criteria are appropriate for their work. Writers up in Samoa seems to have been motivated
of qualitative case study research tend to play down by her belief that it represented a unique case
or ignore the salience of these factors (cf. Stake, and thus could challenge the then popular
1995). Those influenced by the quantitative research nature-over-nurture hypothesis. She reported
tradition see them as more significant, and usually that, unlike adolescents in most other societies,
try to develop case studies that meet the criteria. Samoan youth did not suffer a period of anxiety
One question that has generated a great deal of and stress in their teenage years. She explained
discussion concerns the external validity or gener- this by their culture’s strong, consistently en-
alizability of case study research. How can a single forced standards of conduct and morality. These
case possibly be representative of other cases? For factors were of interest because many readers
example, would the findings from a study of the To- thought they might contain lessons for the West.
ronto police department be generalizable to all large • The revelatory case. The basis for the revelatory
urban police departments in Canada? The answer, case exists “when an investigator has an oppor-
of course, is that the findings probably cannot be tunity to observe and analyse a phenomenon
applied to other police departments. Case study previously inaccessible to scientific investiga-
researchers do not usually delude themselves into tion” (Yin, 1984, p. 44). This can happen when
thinking that it is possible to identify typical cases previously unavailable evidence becomes ac-
that can represent a class of objects, whether factor- cessible, as was the case with certain KGB files
ies, mass media reporting, police services, or com- after the fall of the Soviet Union.
munities. In other words, they typically do not think
that a case study is a sample of one. Cases may also be chosen for mundane reasons,
such as convenience, and still provide an adequate
Types of case context for answering certain research questions or
Yin (1984) distinguished three types of case, each of examining key social processes. To take a concrete
which relates to the issue of external validity: example, Russell and Tyler (2002) studied a Girl
Heaven store in the UK (which caters to 3- to 13-year-
• The critical case. Here the researcher has a old girls) not because it was a critical or unique case,
clearly specified hypothesis, and a case is chosen or because it offered a context never before studied,
on the grounds that it will allow a better under- but because of its capacity to illuminate the links be-
standing of the circumstances under which the tween gender and consumption and the commodifi-
hypothesis does or does not hold. The classic cation of childhood in modern society. Indeed, often
study by Festinger et al. (1956) of a dooms- it is only at a very late stage in the research that the
day/UFO cult is an example. The fact that the singularity and significance of the case becomes ap-
world did not end allowed the researchers to parent (Radley & Chamberlain, 2001).
test propositions about how people respond to As we have mentioned, one of the standard criti-
thwarted expectations. What did cult members cisms of the case study is that its findings cannot be
do when, after quitting their jobs, leaving their generalized. Case study researchers argue strenu-
homes, and waiting on a mountaintop, nothing ously that this is not the purpose of their craft.
happened? Did they sneak down and move to A valid picture of one case is more valuable than a
another town? No: they decided that their faith potentially less valid picture of many. Their aim is to
had saved humankind and that their new role generate an intensive examination of a single case,
was to tell others of that miracle so more people which may or may not be used for theoretical analy-
could be converted to their religion. sis. Pratt and Valverde (2002) studied only Somalis
• The extreme, even unique, case; a common focus and expressed a hope that others would study other
in clinical studies. Margaret Mead’s (1928) well- immigrant groups in other places. Their central con-
known (albeit controversial) study of growing cern was the richness of the data and the quality of
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

the theoretical reasoning that the case allowed. As approach is associated with a qualitative research
we have noted, sometimes case studies are primarily strategy. As illustrated in Box 2.6, comparative case
inductive, used as information to generate theories. study work is often quantitative in orientation.
Other times they may be deductive in nature, pro- The strength of comparative designs is that
viding the data required to assess theories. they highlight the similarities and differences be-
Problems can arise when the research involves tween cases, which can be used to assess or gener-
comparison of two or more cases. Dyer and Wilkins ate theories. They exhibit certain features similar to
(1991), for example, argued that a multiple–case experiments and quasi-experiments, which also rely
study approach means that less attention is paid to on the capacity to establish comparisons.
the specific details of a particular case, and more to
the ways in which multiple cases can be contrasted.
Furthermore, the need for comparison often leads
Bringing research orientation
the researcher to choose an explicit focus at the and design together
outset, when it might be advantageous to adopt Finally, we can bring together the two general re-
a more open-ended approach. A preference for search orientations covered in Chapter 1 with the
contextual insight and a less structured research research designs outlined in this chapter. Table 2.1
BOX 2.6 Comparative research: Cross-cultural studies
Phenomena such as voting behaviour or crime cities may be very different in terms of ridership,
victimization in two or more countries can be safety, cleanliness, and so on.
compared using the same research instruments, A strength of cross-cultural research is that it
seeking similarities and differences and a deeper helps to illustrate how social scientific findings may
understanding of social reality in different na- be culturally specific. For example, Wilson’s (2002)
tional contexts. At the very least such research examination of Ontario raves made frequent com-
supplies a replication. parisons to the earlier rave scene in Britain. How-
Cross-cultural research is more expensive than ever, the UK scene was primarily an outgrowth of
other approaches. It also presents other problems. working-class struggles, whereas Canadian raves
When using existing data such as official statistics appealed more to middle-class, culturally alien-
or survey evidence, the researcher must ensure ated youths. Similarly, Baer et al. (2001) found
that the variable categories and data-collection that joining clubs and voluntary organizations in-
methods are comparable. When new data are creased toward the end of the last century in the
being collected, the researcher must ensure that US, West Germany, and the Netherlands, was stable
data-collection instruments (for example, ques- in Canada and 10 other countries, but decreased in
tionnaires and interview schedules) are translated Spain. Finally, Young and Dugas (2012) compared
properly. Even when translation is carried out Canadian print media coverage of climate change
competently, there may still be a problem with issues in English-language publications with those
insensitivity to specific national and cultural mi- in written in French, and found that the different
lieus. For example, the London “tube,” the Toronto environmental and media cultures in the two lan-
“subway,” and the Montreal “métro” differ in more guage communities contributed to important differ-
than name: public transit experiences in the three ences in how climate change topics were narrated.
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

shows the typical form associated with each com- studies, research that concentrates on a specific
bination of research orientation and research design, issue over time, and ethnography, in which the
along with a number of examples that either have researcher charts change in a single case, contain
been encountered so far or will be covered in later elements of both designs. Such studies are per-
chapters. Table 2.1 also refers to research methods to haps better conceptualized as longitudinal case
be introduced in later chapters, but not referred to studies rather than as belonging to one category
so far. The Glossary provides a quick reference for or another. A further point is that there is no typ-
unfamiliar terms. ical form in the qualitative research orientation/
The distinctions are not always perfect. In experimental research design cell. Qualitative re-
particular, in some qualitative research it is not search in the context of true experiments is very
obvious whether a study is an example of a longi- unusual; a quasi-experimental design is a more
tudinal design or a case study design. Life history realistic alternative.
TABLE 2.1 | Research strategy and research design

GENERAL RESEARCH ORIENTATION
Research design Quantitative Qualitative
Experimental Typical form: Most experiments involve Experiments are not used in qualitative
quantitative comparisons between research, although they may inspire
experimental and control groups on or be inspired by qualitative findings.
the dependent variable. Example: the For example, a qualitative study of the
Rosenthal and Jacobson (1968) study teachers who participated in the Rosenthal
discussed in this chapter. and Jacobson (1968) study discussed in
this chapter would be enlightening.
Cross-sectional Typical form: Survey research and Typical form: Qualitative interviews or
structured observation on a sample at a focus groups at a single point in time are
single point in time are two forms; content two forms; qualitative content analysis
analysis of a sample of documents is of a set of documents relating to a
another. Example: Box 1.3. single period is another. Example: the
Beardsworth and Keil (1992) study of
vegetarianism discussed in this chapter.
Longitudinal Typical form: Survey research on a Typical form: These include ethnographic
sample on more than one occasion, as research over a long period, qualitative
in panel and cohort studies, is one form; interviewing on more than one occasion,
content analysis of documents relating or qualitative content analysis of
to different time periods is another. documents relating to different time
Example: Box 2.5. periods. Such research is longitudinal
when the main purpose is to map change.
Example: the Lynd and Lynd (1929) study
of “Middletown” discussed in this chapter.
Case study Typical form: Survey research is conducted Typical form: The intensive study done
on a single case with a view to revealing by qualitative interviewing of a single
important features about its nature. case, which may be an organization,
Example: the Bell et al. (2007) study of person, family, or community. Example:
the 2004 Alberta provincial election. Box 1.4.
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Key Points
• There is an important distinction between a gen- • Replicability, validity (measurement and external),
eral research orientation (quantitative versus and the ability to establish causation are import-
qualitative) and a research design. ant criteria for evaluating the quality of quantita-
• The nomothetic approach to explanation involves tive social research.
discovering general laws and principles. • Four key research designs are experimental,
• Nomothetic explanations must satisfy three cri- cross-sectional, longitudinal, and case study.
teria of causation: correlation, time order, and • Threats to the establishment of causation are of
non-spuriousness. particular importance in non-experimental, quan-
• Qualitative researchers usually take the idio- titative research.
graphic approach to explanation, which entails • External validity is a concern with case studies
creating a rich description of a person or group (generalizability) and laboratory experiments
based on the perceptions and feelings of the (findings may not be applicable outside the re-
people studied. search environment).
Questions for Review (R) and Creative Application (A)

Criteria for the evaluation of social research among 15-year-olds at summer camp. You decide to
R Explain the time order criterion of causation. conduct an experiment, and have followed proper
A A survey researcher finds that people with high ethics protocols. How could you conduct the ex-
self-esteem make more money than people with low periment? Provide as much detail as possible.
self-esteem. You are tempted to conclude from this
that self-esteem influences earning power. But can R What is a quasi-experiment?
time order be established using this design? Explain. A You want to know whether the legalization of
marijuana in Canada will affect national crime
R What is a spurious correlation? rates. How could you use the quasi-experimental
A You are at a dance party where a lot of alcohol is method to research this issue? Explain.
being served. You abstain from drinking, but notice Cross-sectional design
that the people with the craziest dancing style are
R What is meant by a cross-sectional research design?
the most likely to go to the washroom and vomit.
A How could you use a cross-sectional design to de-
Should you conclude that dancing crazily induces
termine whether there is an association between
vomiting? Explain.
the amount of time spent studying and grades?
Assess the degree to which your method can es-
R What is a nomothetic explanation?
tablish causality.
A Come up with a nomothetic explanation for why
students sometimes drop out of university. Longitudinal design(s)
R Why might a longitudinal research design be su-
R What is an idiographic explanation? How do quali- perior to a cross-sectional one?
tative researchers produce them? A How could a qualitative researcher use a longi-
A Assume that your best friend just dropped out of tudinal design to study people active in a local en-
university. Come up with an idiographic explana- vironmental movement?
tion of how that happened. Case study design
R What is a case study?
Research designs
A Pick a particular case (it can be any person, group,
Experimental design or event) and explain how a qualitative researcher
R How are true experiments able to establish causal could study it. Then describe how a quantitative
connections between variables? Explain. researcher could gather information on the case
A You want to know whether the amount of time that is relevant to the findings of the qualitative
spent on social media affects loneliness levels investigator.
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Interactive Classroom Activities

1. This activity involves some friendly competition. together (e.g., it would be the severity of the fire for
Divide the class into small groups. Each group the fire trucks and damage example given above).
is given 15 minutes to come up with as many
examples of spurious correlations as they can 2. Divide the class into small groups. Each group is to
(e.g., the correlation between the number of fire
trucks at a fire and the amount of fire damage). a. pick a social phenomenon or experience of in-
Examples given in this chapter or in previous terest (e.g., sex work, homelessness);
classes are not allowed. After the time is up, a b. provide a nomothetic explanation for it; and
spokesperson from each group presents the c. provide an idiographic explanation for it.
group’s spurious correlations to the rest of the
class, and the instructor judges whether each The groups are to go into as much detail as pos-
one is in fact an example of a spurious correla- sible. When that is completed, each group is paired
tion. The instructor also keeps a record of all with another group (A and B, C and D, E and F, etc.)
groups’ accepted correlations. The group with such that Group A presents its results to the rest of
the most accepted correlations wins. the class, while Group B has the task of providing
Once that has been done, the class examines a critical evaluation (enumerating both strengths
each correlation individually. The instructor asks and weaknesses) of Group A’s arguments. Then the
for volunteers to name an antecedent third factor reverse: Group B presents and Group A critiques.
that might be causing the two variables to vary Repeat for Groups C and D, E and F, and so on.
Relevant Websites
The National Longitudinal Survey of Children and The Human Genome Project site offers some basic
Youth site provides information on an ongoing, longi- information on this 13-year project and its implica-
tudinal study in Canada. tions for understanding human behaviour.
www12.statcan.gc.ca/census-recensement/2011/ https://web.ornl.gov/sci/techresources/Human_
ref/92-135/surveys-enquetes/nationalchildren- Genome/index.shtml
nationaleenfants-eng.cfm
The Behavior Genetics Association, an international
The Online Dictionary of the Social Sciences site, organization, provides a forum for the dissemination
maintained by Athabasca University, “has 1000 entries of behavioural genetic research and offers methodo-
covering the disciplines of sociology, criminology, pol- logical workshops.
itical science and women’s study with a commitment www.bga.org
to Canadian examples and events and names.”
(Websites accessed 16 October 2018)
http://bitbucket.icaap.org/dict.pl
The Research Methods Knowledge Base provides a

detailed discussion of various types of experimental
design.
www.socialresearchmethods.net/kb/desexper.php
More resources are available on Dashboard.

Visit dashboard.oup.com for:
• Student Study Guide • Flash cards • Videos • Activities

• Student self-quiz • Audio clips • Web links
bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Alan Bryman - Social Research Methods

Uploaded by

Copyright:

Available Formats

Alan Bryman - Social Research Methods

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Alan Bryman - Social Research Methods

Uploaded by

Copyright:

Available Formats

2

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Introduction watched changes, the level of violence would

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/19/18 01:25 PM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

BOX 2.1 A laboratory experiment

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

BOX 2.2 Quasi-experiments

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

BOX 2.3 Evaluation research

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

CCTV deters car crime. For instance, it serves as

usage of parking lots, which in turn increases

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Methods in Motion | Applications to Canadian Society

Why are some Canadians politically conservative and others liberal?

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

BOX 2.4 Longitudinal research and the case study

BOX 2.5 The National Longitudinal Survey of Children and Youth

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Research in the News

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

BOX 2.6 Comparative research: Cross-cultural studies

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

TABLE 2.1 | Research strategy and research design

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Questions for Review (R) and Creative Application (A)

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

Interactive Classroom Activities

The Research Methods Knowledge Base provides a

More resources are available on Dashboard.

• Student Study Guide • Flash cards • Videos • Activities

bry29440_ch02_027-049.indd 12/10/18 11:05 AM

You might also like