Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis

Review of Educational Research
Fall 1994, Vol. 64, No. 3, pp. 363-423
Reinforcement, Reward, and Intrinsic Motivation:

A Meta-Analysis
Judy Cameron and W. David Pierce

University of Alberta
This article reviews research on the effects of reinforcement/reward on

intrinsic motivation. The main meta-analysis included 96 experimental stud-
ies that used between-groups designs to compare rewarded subjects to
nonrewarded controls on four measures of intrinsic motivation. Results
indicate that, overall, reward does not decrease intrinsic motivation. When
interaction effects are examined, findings show that verbal praise produces
an increase in intrinsic motivation. The only negative effect appears when
expected tangible rewards are given to individuals simply for doing a task.
Under this condition, there is a minimal negative effect on intrinsic motiva-
tion as measured by time spent on task following the removal of reward. A
second analysis was conducted on five studies that used within-subject
designs to evaluate the effects of reinforcement on intrinsic motivation;
results suggest that reinforcement does not harm an individual's intrinsic
motivation.
Reinforcement theory has had a significant impact on education. Education

professors routinely teach the basic elements of behavior theory. As a conse
quence, most classroom teachers have at least some rudimentary understanding
of the principles of reinforcement. These principles are often used to promote
learning and to motivate students. In recent years, however, there has been a
growing concern over the application of reward systems in educational settings.
Several researchers have presented evidence and argued that incentive systems
based on reinforcement may have detrimental effects. The contention is that
reinforcement may decrease an individual's intrinsic motivation to engage in a
particular activity. To illustrate, if a child who enjoys drawing pictures is externally
reinforced (e.g., with points or money) for drawing, the child may come to draw
less once the reward is discontinued. In other words, one alleged effect of
reinforcement is that it undermines intrinsic interest in a task.
The literature concerned with the effects of reinforcement on intrinsic motiva
tion draws mainly from experimental investigations. In an article published in
the American Psychologist, Schwartz (1990) cited the intrinsic motivation experi
ment of Lepper, Greene, and Nisbett (1973) and concluded that
This article is based on Judy Cameron's PhD dissertation. The research was sup
ported by the Social Sciences and Humanities Research Council (SSHRC) in Canada.
We thank Dr. W. Frank Epling and Dr. Steve Hunka for their helpful comments
and suggestions.
363
Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Reinforcement, Reward, and Intrinsic Motivation
reinforcement has two effects. First, predictably it gains control of [an]
activity, increasing its frequency. Second, . .. when reinforcement is later
withdrawn, people engage in the activity even less than they did before
reinforcement was introduced, (p. 10)
While several researchers agree with this conclusion (e.g., Kohn, 1993; Suther
land, 1993), others continue to favor the use of reinforcement principles in applied
settings (e.g., Hopkins & Mawhinney, 1992). This is, obviously, an important
issue. Incentive systems are often implemented (or not) in schools, industry,
hospitals, and so forth on the basis of research findings and conclusions. The
present article evaluates the literature concerned with the effects of reinforcement
and reward on intrinsic motivation by a meta-analysis of the relevant experimen
tal investigations.
Several researchers draw a distinction between intrinsic and extrinsic motiva
tion. Intrinsically motivated behaviors are ones for which there is no apparent
reward except the activity itself (Deci, 1975). Extrinsically motivated behaviors,
on the other hand, refer to behaviors in which an external controlling variable
can be readily identified. According to Deci (1975), intrinsic motivation is demon
strated when people engage in an activity for its own sake and not because of
any extrinsic reward. The result of such behavior is an experience of interest
and enjoyment; people feel competent and self-determining, and they perceive
the locus of causality for their behavior to be internal. Intrinsically motivated
behavior is seen to be innate and is said to result in creativity, flexibility, and
spontaneity (Deci & Ryan, 1985). In contrast, extrinsically motivated actions are
characterized by pressure and tension and result in low self-esteem and anxiety
(Deci & Ryan, 1985).
A great deal of debate has surrounded the intrinsic/extrinsic distinction. Several
critics (e.g., Guzzo, 1979; Scott, 1975) point out difficulties in identifying intrinsi
cally motivated behaviors. Although many human behaviors appear to occur in
the absence of any obvious or apparent extrinsic consequences, they may, in fact,
be due to anticipated future benefits (Bandura, 1977) or intermittent reinforcement
(Dickinson, 1989). From this perspective, intrinsically motivated behavior is
simply behavior for which appropriate controlling stimuli have yet to be specified.
In spite of these conceptual difficulties, other social scientists frequently accept
the intrinsic/extrinsic distinction. In fact, a large body of research is concerned
with the effects of extrinsic rewards and reinforcers on behavior that is thought
to have been previously maintained by intrinsic motivation. The next section of
this article presents a description of the early studies concerned with the effects
of reward and reinforcement on intrinsic motivation, the various research designs
used to further investigate the issue, the variables investigated, and major findings.
THE EFFECTS OF REWARD AND REINFORCEMENT ON
INTRINSIC MOTIVATION
The terms reward and reinforcement have frequently been used synonymously.
Although this is the case, behavioral psychologists make an important distinction
between the two terms. A reinforcer is an event that increases the frequency of
the behavior it follows. A reward, however, is not defined by its effects on
behavior. Rewards are stimuli that are assumed to be positive events, but they
have not been shown to strengthen behavior. Incentive systems (e.g., classroom
364

Cameron and Pierce
token economies) may be based on reward or reinforcement and are designed to
increase motivation. Because of these distinctions (between reward and reinforce
ment), this review separates those studies dealing with effects of reward from
those concerned with the effects of reinforcement on intrinsic motivation.
The Early Studies
The first laboratory investigations to test the effects of reward on intrinsic
motivation were conducted by Deci (1971, 1972a, 1972b). In the first experiment
(1971), 24 college students, fulfilling a course requirement, were presented with
a puzzle-solving task (Soma, a commercial puzzle, produced by Parker Brothers,
composed of seven different shapes that can be solved in a variety of ways). The
Soma puzzle was chosen because it was believed that college students would be
intrinsically interested in the task. The study was made up of three 1-hour sessions
over a 3-day period. Twelve subjects were assigned to an experimental group;
the other 12 to a control group. During each session, subjects were individually
taken to a room and asked to work on the Soma puzzles in order to reproduce
various configurations which were drawn on a piece of paper. Four puzzles were
presented in a session, and subjects were given 13 minutes to solve each one.
In the second session only, experimental subjects were told that they would
receive $1.00 for each puzzle solved. Control subjects were offered no money.
In the middle of each session, the experimenter made an excuse to leave the
room for 8 minutes. Subjects were told that they could do as they pleased. During
these 8-minute periods, the experimenter observed the subjects through one-way
glass and recorded the time that each subject spent engaged on the Soma task.
The amount of time spent on the task during the free periods was taken to be
the measure of intrinsic motivation, the dependent variable.
Deci hypothesized that reward (money) would interfere with subsequent intrin
sic motivation and that subjects in the experimental group would spend less time
on the task in the third session than they had in the first. He suggested that there
would be a significant difference between the experimental and control subjects
on this measure. Using a one-tailed t test, Deci found the difference between the
two groups to be significant at p < .10. The rewarded group spent less time on
the task than the control group. Although social scientists do not generally accept
results at/? > .05 as significant, and although Deci (1971) noted the marginal
nature of his result, the data have been taken as support for the hypothesis that
If a person is engaged in some activity for reasons of intrinsic motivation,
and if he begins to receive the external reward, money, for performing the
activity, the degree to which he is intrinsically motivated to perform the
activity decreases. (Deci, 1971, p. 108)
Deci's experiment is often cited as groundbreaking evidence for the negative
effects of reinforcement on intrinsic motivation (e.g., Kohn, 1993). Given the
distinction between reward and reinforcement, however, Deci's (1971) experi
ment, at most, demonstrates that rewards may have a negative impact on a
person's interest in a task. Nonetheless, his study was the first to investigate an
issue that was of prime concern to many psychologists. The experiment provided
researchers with a way to measure intrinsic motivation and with a paradigm to
investigate the negative effects of reward.
365

In another experiment, Deci (1971; Experiment 3) used the same experimental

paradigm to investigate the effects of verbal reward. The reward contingency
introduced in the second session was verbal praise, rather than money. During
the second phase, subjects in the experimental group were told after each trial
that their performance was very good or much better than average. Deci found
that the reinforced group spent significantly more time on the task (difference
scores between Session 3 and Session 1) than those who received no praise
(p < .05). These results suggest that social rewards may increase the motivation
to perform an activity.
One of the best known and most cited studies on the detrimental effects of
reward on behavior is the work of Lepper, Greene, and Nisbett (1973). In this
study, nursery school children were observed in a free-play period to determine
their initial interest on an activity (drawing). Two observers sat behind a one
way glass and recorded the amount of time each child was engaged in the activity.
Those children who spent the most time on the task were selected as subjects for
the experiment. Three experimental conditions were employed. In the "expected-
reward" condition, children were offered a "good-player" award, which they
received for drawing with magic markers. Children in the "unexpected-reward"
group received the award but were not promised it beforehand, and "no-reward"
subjects did not expect or receive an award.
In a subsequent free-play session, those children who were promised an award
(expected-reward subjects) spent significantly less time drawing than the other
two groups. Furthermore, the expected-reward group spent less time drawing in
the postexperimental session than they had in the initial session (preexperimental
free-play session). The unexpected-reward and no-reward subjects showed slight
increases in time on task from preexperimental to postexperimental sessions.
Lepper et al. (1973) concluded that their results provided "empirical evidence
of an undesirable consequence of the unnecessary use of extrinsic rewards,"
(p. 136).
However, those who received an unexpected reward spent more time on the
task during the postexperimental free-play period than either the expected-reward
or the control group. Because the unexpected- and expected-reward groups are
both reward conditions, the conclusion that these results demonstrate the negative
effects of reward may not be warranted. This is because reward was held constant
in the unexpected-reward and expected-reward groups; what differed was promise
or no promise. That is, the promises made or the instructions given could have
produced these results. Nonetheless, the findings of Lepper et al.'s (1973) study
are frequently cited in journal articles and introductory psychology textbooks as
evidence that extrinsic rewards and reinforcement undermine intrinsic interest in
a task.
The early studies by Deci (1971) and Lepper et al. (1973) have raised a number
of issues and controversies that have generated considerable research. Some
psychologists have claimed that the original findings provide evidence for the
view that reinforcement decreases intrinsic motivation (e.g., Schwartz, 1990).
Others recognize that not all types of reinforcement undermine intrinsic interest
(e.g., Deci & Ryan, 1985). Still others argue that one must demonstrate that
rewards are, in fact, reinforcers before any statements about the effects of rein
forcement can be made (Feingold & Mahoney, 1975; Mawhinney, 1990). Several
366

Cameron and Pierce
researchers are cautious about equating reward with reinforcement; their focus
has been to discover when and under what conditions reward is detrimental
(Bates, 1979; Morgan, 1984). In order to address these issues, researchers have
employed a variety of research paradigms.
Between-Group Designs
Studies designed to assess the effects of reward on intrinsic motivation have
been conducted using between-group designs. Typically, one of two methods is
employed. The first method, referred to as a before-after design (Deci & Ryan,
1985), involves a three-session paradigm. In these studies, a baseline measure
of intrinsic motivation on a particular task is taken. This entails measuring time
on task in the absence of extrinsic reward, usually from a session of short duration
(e.g., 10 minutes). Subjects are then assigned to a reward or no-reward (control)
condition, and an intervention with extrinsic rewards is carried out. Following
this, reward is withdrawn, and time on task is again measured. The procedure
is identical for both groups except that control subjects do not experience the
intervention in the second session. Mean differences in time on task between
pre- and postintervention are calculated for each group, and the scores for the
experimental and control subjects are then statistically compared. Any difference
between the two groups is considered evidence of the effects of withdrawal
of reward.
One advantage to the before-after procedure is that it allows the researcher to
examine differences within groups from pre- to postexperimental sessions as
well as differences between groups. In most studies of this type, however, only
differences between groups are investigated. This is because the before-after
procedure has generally been used to identify individuals who show an initial
interest in a specific task; those people are then selected as subjects for the study.
In such cases, differences between rewarded and nonrewarded subjects are usually
measured in the after-reward session only.
Most researchers have used an after-only between-groups experimental design
to assess the effects of rewards on intrinsic motivation. In this approach, no
pretreatment measure of intrinsic interest is collected. In the typical experiment,
subjects are presented with a task that is assumed to be intrinsically motivating—
solving and assembling puzzles, drawing with felt-tipped pens, word games, and
so on. Experimental subjects are rewarded with money or grades, candy, praise,
good-player awards, and so forth for performing the activity. In some studies,
the reward is delivered contingent on a certain level of performance on the task;
in others, subjects are simply rewarded for participating in the task. Control
subjects are not rewarded. The reward intervention is usually conducted over a
10-minute to 1-hour period. All groups are then observed during a nonreward
period. This usually occurs immediately after the experimental session, although
some researchers have observed subjects several weeks later. If experimental
subjects spend less time on the task (during the postreward observation) than the
controls, reinforcement/reward is said to undermine intrinsic motivation. The
amount of time subjects spend on the task during the nonreward period is one
of the major ways in which intrinsic motivation has been measured, and it is
usually referred to as free time on task.
367

Findings from the Group Design Studies

Generally, the results of the group design studies examining the main effects
of rewards are conflicting. While some researchers have found that rewards lead
to decreased time on the task relative to control groups (e.g., Deci, 1971; Fabes,
1987; Morgan, 1981), others report the opposite (e.g., Brennan & Glover, 1980;
Deci, 1972a; Harackiewicz, Manderlink & Sansone, 1984). Some studies report no
significant differences (e.g., Amabile, Hennessey, & Grossman, 1986; DeLoach,
Griffith & LaBarba, 1983).
Not all studies use the free-time measure of intrinsic motivation. Other depen
dent variables have included self-reports of task enjoyment, interest, and satisfac
tion; performance during the free time period (number of puzzles/problems solved,
number of drawings completed, etc.); and willingness to volunteer for future
projects without reward. Overall, the results from studies employing these mea
sures are conflicting and do not help to clarify the issue of whether reward leads
to decreased intrinsic motivation.
A number of reviewers (e.g., Bates, 1979; Deci & Ryan, 1985; Dickinson,
1989; Morgan, 1984) have noted the contradictory nature of the findings and
have attempted to identify the conditions under which extrinsic reward produces
decrements in intrinsic motivation. Some of the conditions thought to be critical
in determining the impact of rewards include the type of reward (tangible or
verbal), reward expectancy (whether reward is expected—i.e., offered beforehand
or received unexpectedly), and reward contingency (whether reward is delivered
simply for performing the task or is contingent on some specified level of
performance). Although this categorization system is useful, an examination of
the literature within each category reveals conflicting results.
Type of Reward
When verbally praised subjects are compared to a control group, some research
ers have found an increase in intrinsic motivation (e.g. Deci, 1971) while others
report no significant differences (e.g., Orlick & Mosher, 1978). The same holds
true when subjects receiving tangible rewards are compared to controls. While
some results provide evidence for a decrease in intrinsic motivation following
the receipt of a tangible reward (e.g., Danner & Lonkey, 1981), others indicate
an increase (e.g., Rosenfìeld, Folger, & Adelman, 1980).1
Reward Expectancy
Comparisons between subjects who receive an unexpected tangible reward
and subjects who receive no reward are also not clear cut. Some results indicate
that unexpected-reward subjects show a decrease in intrinsic motivation (e.g.,
Orlick & Mosher, 1978); others have found no significant differences (e.g.,
Greene & Lepper, 1974). Experiments designed to investigate the effects of
expected tangible rewards are also contradictory. Some studies, comparing sub
jects offered an expected reward to nonrewarded controls, show a negative effect
of reward on intrinsic motivation (e.g., Deci, 1971; Lepper, Greene, & Nisbett,
1973). Others, however, demonstrate that expected-reward subjects show an
increase in intrinsic motivation relative to controls (e.g., Brennan & Glover, 1980).
368

Cameron and Pierce
Reward Contingency
Morgan (1984) and Deci and Ryan (1985) suggest that reward contingency
may play a critical role in determining the negative effects on intrinsic motivation.
Again, however, results from such studies vary. When rewards are delivered
contingent on some level of performance, some researchers have found a positive
effect (e.g., Karniol & Ross, 1977); others report negative findings (e.g., Ryan,
Mims, & Koestner, 1983). When rewards are delivered contingent on engagement
in the task regardless of subjects' level of performance, an undermining effect
has been found in some studies (e.g., Lepper, Greene & Nisbett, 1973; Morgan,
1983, Experiment 1). Others report no decrease in intrinsic motivation (e.g.,
Pittman, Emery & Boggiano, 1982).
Within-Subject Designs
One of the criticisms of the group design research is that researchers employing
such a design often refer to their reward manipulation as a reinforcement proce
dure. By definition, a reinforcer is an event that increases the frequency of the
behavior it follows. In most studies on intrinsic motivation, researchers have not
demonstrated that the events used as rewards increased the frequency of the
behavior studied. In addition, critics (e.g., Feingold & Mahoney, 1975; Mawhin-
ney, 1990) suggest that the measurement phases in the group design research are
too brief to detect any temporal trends and transition states. In order to address
these issues, a few studies have been conducted using a repeated measures,
within-subject design.
In this paradigm, the amount of time subjects spend on a particular task is
measured over a number of sessions. Reinforcement procedures are then imple
mented over a number of sessions. In the final phase, reinforcement is withdrawn,
and time on task is again repeatedly measured. Intrinsic motivation is indexed
as a difference in time on task between pre- and postreinforcement phases where
differences are attributed to the external reinforcement.
In general, no substantial differences have been found when rate of performance
and time on task in postreinforcement sessions are compared to pre-reinforcement
phases (although, see Vasta & Stirpe, 1979).
The advantage of within-subjects designs is that the researcher can determine
whether the rewards used are actual reinforcers—that is, whether behavior
increases during the reinforcement phase. Statements can then be made about
the effects of reinforcement, rather than reward. However, only a handful of
studies have been conducted using this type of design.
Critics of within-subject research (e.g., Deci & Ryan, 1985) suggest that results
from these designs are not generalizable because so few subjects are studied in
any one experiment. A further criticism has to do with the lack of a control
group. The argument is that in the within-subject designs there is no group that
performs the activity without reinforcement; thus, one cannot know if there is
an undermining effect relative to a control group. Finally, for these studies,
the definition of a reinforcer is necessarily circular. That is, reward becomes
reinforcement only after its effects are shown to increase behavior.
369

THEORETICAL ACCOUNTS OF THE LITERATURE

Although the results of laboratory investigations into the effects of reward
and reinforcement on intrinsic motivation appear contradictory and confusing, a
general contention in many textbooks and journal articles is that reward and/or
reinforcement is detrimental to an individual's intrinsic motivation (e.g., Kohn,
1993; McCullers, 1978; Schwartz, 1990; Zimbardo, 1988). In an attempt to
account for the disparate outcomes, a few psychologists have offered theoretical
explanations. Three major accounts are outlined below.
The Overjustifìcation Hypothesis
One explanation that has been put forth to account for the detrimental effects
of reward is termed the overjustifìcation effect (Lepper, Greene, & Nisbett, 1973).
This hypothesis is largely based on attribution (Kelly, 1967) and self-perception
(Bern, 1972) theories. A person's perceptions about the causes of behavior are
hypothesized to influence future motivation and performance. In the presence of
external controls, people attribute their behavior to an external agent; when this
is removed, future motivation and performance decrease. Conversely, behavior
is attributed to internal causes in the absence of obvious external controls. In
this case, motivation and performance are not affected.
A decrease in intrinsic motivation following the withdrawal of a reward has
been termed the overjustifìcation effect because it is thought that an external
reward provides overjustifìcation for participating in an already attractive activity.
Put another way, when individuals are rewarded for engaging in an already
interesting activity, their perceptions shift from accounting for their behavior as
self-initiated to accounting for it in terms of external rewards. That is, they are
faced with too many reasons (justifications) for performing the activity, and the
role of intrinsic motivation is discounted resulting in a decline in intrinsic motiva
tion.
Lepper (1981) has suggested that extrinsic rewards lead to a decrease in
intrinsic motivation when they allow perceptual shifts of causality. According to
Lepper, this occurs when there is sufficient initial interest in an activity, when
the extrinsic rewards are salient, and when rewards do not increase perceived com
petence.
Cognitive Evaluation Theory
Deci and Ryan (1985) suggest that the overjustifìcation hypothesis should not
be considered a theory of motivation. They argue that self-attributions may affect
intrinsic motivation, but they do not see them as necessary mediators. Instead,
Deci and Ryan offer cognitive evaluation theory as an explanation for intrinsic
motivation.
Cognitive evaluation theory is based on the assumption that people have innate
needs for competence and self-determination. From this perspective, a person's
intrinsic motivation is affected by changes in feelings of competence and self-
determination. According to Deci and Ryan (1985), events facilitate or hinder
feelings of competence and self-determination depending on their perceived
informational, controlling, or amotivational significance. Events seen as informa
tional indicate skill in performing a task; hence, competence is facilitated, which
leads to increased intrinsic motivation. A controlling event is one perceived as
370

Cameron and Pierce
an attempt to determine behavior. This type of event diminishes an individual's
self-determination and intrinsic motivation. An amotivational event provides
negative feedback, indicating a lack of skill, which reduces one's competence
and intrinsic motivation.
Cognitive evaluation theory focuses on a person's experiences of an activity.
For this reason, Deci and Ryan (1985) emphasize the importance of self-report
measures of task interest, satisfaction, and enjoyment as more indicative of
intrinsic motivation than the free time-on-task measure.
According to cognitive evaluation theory, rewards are not always harmful.
Verbal rewards may be informational and lead to an increase in intrinsic motiva
tion. Tangible rewards, on the other hand, are seen as controlling when their
delivery is stated before the reward period (expected rewards). This is because
the cognitive evaluation process is believed to begin while the rewarded activity
is occurring. Further, rewards promised to persons for engaging in a task without
a performance criterion (referred to as expected task contingent rewards by Deci &
Ryan, 1985) are controlling and decrease intrinsic motivation. Deci and Ryan
suggest that rewards delivered to a person contingent on a specified level of
performance are more complicated. This type of reward can be informational or
controlling, but the difficulty is that its function can only be determined by how
well a person performs in relation to the specified standard. If the individual
performs well, the reward is informational, and, if performance is poor, it is con
trolling.
Rummel and Feinberg (1988) conducted a meta-analysis to assess cognitive
evaluation theory. Subjects who received rewards that were defined to convey
"controlling" information were compared to groups receiving other types of
rewards or no reward. The dependent measure of intrinsic motivation was a
combination of both free time-on-task measures and self-reports of satisfaction
and task interest. Results provided support for cognitive evaluation theory. Rum
mel and Feinberg concluded that controlling, extrinsic rewards have detrimental
effects on intrinsic motivation.
In Rummel and Feinberg's meta-analysis, rewards were defined as controlling
after the fact. That is, when a reward was found to produce a negative effect, it
was seen as controlling, and the study was selected for the analysis. This exempli
fies the major difficulty with cognitive evaluation theory. Rewards are defined as
controlling or informational after their effect on performance has been measured.
Although cognitive evaluation theory may account for the diverse findings of
the effects of reward on intrinsic motivation, there are difficulties with this
interpretation. One problem is that feelings of competence and self-determination
are seen as causes of changes in intrinsic motivation, but they are not measured.
They are assumed to be operating because behavior changes. In other words, the
existence of competence, self-determination, and intrinsic motivation is inferred
from the very behavior it supposedly causes. Rewards are defined as controlling
if measures of intrinsic motivation decrease and informational when the dependent
variable indexes an increase in motivation.
Behavioral Accounts
An operant analysis of behavior involves consideration of a prior learning
history and the three-term contingency, the SD: R SΓ relationship. The three
D
terms are: (a) discriminative stimulus (S ) or setting event, (b) the response (R)
371

or behavior, and (c) contingent reinforcement (SΓ). Flora (1990) has suggested that
all of the empirical results of the intrinsic motivation research can be accounted for
by considering the promised reward procedures (expected reward) as discrimina
tive stimuli. That is, telling a person that he or she will receive a reward is a
stimulus event that precedes the operant and, as such, is a discriminative stimulus
rather than a reinforcer. From this perspective, if behavior is regulated by its
consequences (i.e., reinforcement), no loss of intrinsic motivation is expected.
When individuals who are engaged in a task are reinforced for doing the task,
they will spend as much time on the activity as they originally did once the
reinforcer is withdrawn. A behavioral view suggests that it is only when rewards
function as discriminitive stimuli that one might expect to observe a decline in
intrinsic motivation.
Although discriminitive stimuli are part of the three-term contingency and
affect the probability of an operant, they can and do have very different effects
from reinforcers. Task performance evoked by instructions and promises of reward
(SDs) can be influenced by a number of factors such as the subject's history with
respect to whether promised rewards were actually received, the subject's verbal
repertoire, the nature of prior exposure to the object being offered as the reward,
and so on (Dickinson, 1989).
SUMMARY
The overjustifìcation effect, cognitive evaluation theory, and recent behavioral
explanations each attempt to account for the disparate effects of reward and
reinforcement on intrinsic motivation. Given the diverse findings reported in this
literature, however, it is not clear at this point what effect reward or reinforcement
has on intrinsic motivation. Reviewers on all sides of the issue tend to be highly
critical of research designed outside of their own paradigm, and, more often than
not, findings from studies in opposing camps are not considered relevant. For
these reasons, the literature and its interpretations are still contentious. Because
a substantial number of experimental studies have been carried out to assess the
effects of reward and reinforcement on intrinsic motivation, one way to evaluate
their effects is to conduct a meta-analysis.
THE PRESENT META-ANALYSIS

The primary purpose of the present meta-analysis is to make a causal statement
about the effects of extrinsic rewards and reinforcement on intrinsic motivation.
This analysis should be useful in addressing a number of concerns. Of major
importance is whether the bulk of evidence suggests that extrinsic rewards and/
or reinforcement produce decrements in intrinsic motivation. If so, what is the
size of the relationships being uncovered? Also, do different patterns emerge
with different reward types (e.g., tangible, verbal rewards), reward expectancies
(expected, unexpected), or reward contingencies (e.g., rewards delivered for
engaging in a task, competing or solving a task, or meeting a specified level of
performance)? In the following sections of this article, the research questions
addressed in the present meta-analysis are outlined, the steps involved in conduct
ing the meta-analysis are described, and the findings are presented and discussed.
372

Cameron and Pierce
Research Questions
The following questions have been addressed in this meta-analysis:
1. Overall, what is the effect of reward on intrinsic motivation? In order
to answer this question, a meta-analysis of the group design experiments was
conducted. Subjects who received a tangible reward and/or an extrinsic verbal
reward were compared to a nonrewarded control group. This analysis should
shed light on the overall effects of reward on intrinsic motivation.
2. What are the effects of specific features of reward on intrinsic motivation?
Several researchers note that reward interacts with other variables to produce
increments or decrements in intrinsic motivation. That is, intrinsic motivation is
affected differently by the type of reward implemented, the reward expectancy and
the reward contingency. Specifically, researchers have investigated the following:
(a) the effect of reward type on intrinsic motivation (i.e., whether rewards are
verbal or tangible),
(b) the effect of reward expectancy on intrinsic motivation (i.e., whether rewards
are expected—promised and delivered to subjects or unexpected—
delivered to subjects but not promised),
(c) the effect of reward contingency on intrinsic motivation (i.e., whether rewards
are delivered to subjects for participating in an experimental session regard
less of what they do, for engaging in a task, for completing or solving a
task, or for attaining a specific level of performance).
All analyses performed on these features were conducted with group design
studies in which a rewarded group was compared to a control group. These
analyses should lead to a greater understanding of the specific conditions under
which reward affects intrinsic motivation.
Although the present analyses present a breakdown of several features of
reward, there are other moderator variables mentioned in the literature (e.g.,
salience of reward, task type, reward attractiveness, goals of individuals, etc.).
These conditions may interact with reward to affect intrinsic motivation. Unfortu
nately, these variables appear in only one or two studies and are, thus, not
amenable to a meta-analysis. At this point in time, placing emphasis on interaction
effects that have few replications would not be beneficial to an understanding
of reward and intrinsic motivation.
3. Overall what is the effect of reinforcement on intrinsic motivation? One
of the criticisms of the group designs has been that reward is frequently cited
as synonymous with reinforcement, yet no evidence has been provided to indicate
that the rewards used in group designs are actual reinforcers. In the single-subject,
repeated measures designs, researchers have demonstrated that the rewards admin
istered increased behavior and can be considered as reinforcers. For this reason,
a separate analysis was conducted with the single-subject designs where subjects
served as their own controls. This analysis should allow a more definitive state
ment to be made about the effects of reinforcement on intrinsic motivation.
METHOD
Selection of Studies
A basic list of studies was assembled by conducting a computer search of the
psychological literature (PSYCH LIT) using intrinsic motivation as the search
373

term. The meta-analysis started with Deci (1971), and relevant articles published
up to September 1991 were identified. Studies not listed on the computer database
were identified through the bibliographies of review articles, chapters, books,
and papers located in the original search.
Two sets of studies were collected (between-group designs and within-subject
designs). The main analysis entailed assessing the overall effects of reward on
intrinsic motivation from studies involving group designs. Criteria for including
studies in the sample were:
(a) that the study involve an experimental manipulation of a reward condition
and include a nonrewarded control group;
(b) that any characteristics of rewarded subjects be either held constant or
varied but be represented identically for both rewarded and control groups; and
(c) that studies be published (no unpublished documents were collected) and
written in English.2
In addition, only studies that measured intrinsic motivation as a dependent variable
were included.
Intrinsic motivation has been measured as free time on task after withdrawal of
reward; self-reports of task interest, satisfaction, and/or enjoyment; performance
during the free time period (number of puzzles/problems solved, number of
drawings completed, etc.); and subjects' willingness to participate in future proj
ects without reward. One study which met the criteria was excluded (Boggiano &
Ruble, 1979) because the statistical contrasts used in the article were not logical
given the sample size of the study.3 Other studies were omitted from the sample
if some subjects in a reward condition were not actually given a reward (e.g.,
Pritchard, Campbell, & Campbell, 1977). The resulting sample consisted of 83
documents, reporting 96 independent studies.
A major criticism of the meta-analytic technique has been that researchers
often lump different measures together. This has been referred to as the apples-
and-oranges problem in that it is argued that logical conclusions cannot be drawn
from comparisons of studies using different measures of the dependent variable
(see Glass, McGaw, & Smith, 1981). In order to avoid this problem, separate
analyses were conducted on the overall effect of reward for each measure of
intrinsic motivation. Using this strategy, 61 studies compared a rewarded group
to a control group on the free-time measure; 64 studies investigated the attitude
(task interest, enjoyment, and satisfaction) measure; 11 studies assessed the
willingness to volunteer for future studies without reward measure, and 12 studies
measured performance during the free-time period.
In order to assess the impact of specific features of reward, further analyses
were conducted with data from the 96 group design studies. In these analyses,
subjects assigned to different types of rewards (tangible, verbal), reward expectan
cies (unexpected, expected), and reward contingencies were compared to nonre
warded control groups.
The second meta-analysis was conducted on studies that employed a within-
subject, multiple-trials design. In this type of design, subjects served as their
own controls. These experiments are conducted in three phases with a number
of sessions in each phase. Baseline measures of intrinsic motivation are taken
in the first phase; reinforcement procedures are then implemented over a number
of sessions, and in the third phase reinforcement is withdrawn. Changes in
374

Cameron and Pierce
intrinsic motivation are measured as differences between the pre- and postrein-
forcement phase.
Single-subject studies were included in this analysis when a reinforcement
effect was demonstrated (i.e., the rewards used showed an increase in behavior)
and when baseline, reinforcement, and postreinforcement phases involved
repeated measures. One study reporting a reinforcement effect was excluded
(Vasta, Andrews, McLaughlin, Stirpe, & Comfort, 1978, Experiment 1) because
the authors reported only one measure of behavior during the postreinforcement
phase. Two studies used a repeated measures group design to assess the effects
of reinforcement between and within groups (Greene, Sternberg & Lepper, 1976;
Mynatt, Oakley, Arkkelin, Piccione, Margolis, & Arkkelin, 1978). Although
subjects' performance in these studies was measured repeatedly as in the single-
subject designs, only group effects were reported. In addition, the rewards used
in these studies were not shown to be reinforcers for some of the rewarded
groups. Thus, these two studies were not included in the meta-analysis of single-
subject designs (Mynatt et al., 1978, are included in the meta-analysis of group
designs because their study included a nonrewarded control group). In all, five
studies were selected for the within-subject meta-analysis.
A list of studies included in the meta-analyses is presented in Appendix A.
Coding of Studies
Once all relevant articles had been collected, each study was read and coded.
The following general information was extracted from each report: (a) author(s),
(b) date of publication, (c) publication source, (d) population sampled (children
or adults), (e) sample size, (f) type of experimental design (before-after groups
design, after-only groups design, or single-subject multiple-trial design), and (g)
type of task used in the study.
The following aspects of the independent variable were also coded: (a) reward
type (tangible or verbal), (b) reward expectancy (expected or unexpected) and
(c) reward contingency. Reward contingency was coded according to Deci and
Ryan's (1985) taxonomy. Task noncontingent rewards referred to rewards deliv
ered to subjects for participating in an experimental session regardless of what
they did in the session. The term task contingent reward was used to mean that
a reward was given for actually doing the task and/or for completing or solving
the task. Performance contingent rewards were defined as rewards delivered for
achieving a specified level of performance. In addition to using Deci and Ryan's
classification, contingency was also coded in accord with a behavioral perspective.
Using operant definitions, rewards were defined as noncontingent or contingent.
Noncontingent rewards referred to rewards delivered for participating in the study
or engaging in the task regardless of any level of performance. Contingent rewards
were defined as rewards dependent on performance (i.e., rewards given for
completing a puzzle, solving a task, and/or attaining a specified level of perfor
mance).
Other characteristics of studies that were coded were: (a) type of dependent
measure (e.g., free time on task, task interest, etc.), (b) whether experimenter
was blind to conditions, and (c) whether experimenter was present or absent
during the post-reward phase. As well, statistical information was recorded, and
effect sizes were calculated from appropriate contrasts.
375

Descriptive characteristics and effect sizes of the reviewed studies are summa
rized in Appendix C.
Intercoder Reliability
From the pool of relevant studies, 10 were randomly selected and independently
coded by the second author. A standardized coding form4 was created that allowed
the second coder to extract information regarding independent variables (reward
type, reward expectancy, reward contingency), dependent variables (measures of
intrinsic motivation), sample sizes, type of task used in the study, and calculation
of effect sizes for available contrasts. Reliability calculated as percentage
agreement was 93.4%. For 6 of the 10 studies, agreement was 100%. Disagree
ments in the other four studies involved (a) miscommunication of formulas to
use for calculating effect size (for two studies), (b) mislabeling of reward expec
tancy (in one study), and (c) a misreading of the number of subjects in a group
(in one study). Disagreements were resolved through discussion and a more
careful reading of the studies and coding criteria.
Computation and Analysis of Effect Sizes
The procedures used in the meta-analysis of the group design studies followed
those of Hedges and Olkin (1985). Meta-analysis is a statistical technique for
aggregating the results of many experimental studies which compare two groups
on a common dependent measure. Once the studies and groups to be compared
are identified, the statistical result of each study is transformed into a measure
called an effect size. An effect size is found by converting the findings from
each study into a standard deviation unit. The effect size indicates the extent to
which experimental and control groups differ in the means of a dependent variable
at the end of a treatment phase. In its simplest form, the effect size calculated,
g, is the difference between the means of the rewarded group and a nonrewarded
control group divided by the pooled standard deviation of this difference. When
means or standard deviations were not available from reports, effect size was
calculated from t tests, F statistics, and p-level values (e.g., p < .05) by using
Hedges and Becker's (1986) formulas. Formulas for calculating effect size are
listed in Appendix B.
One problem that arises in conducting a meta-analysis is determining effect
sizes from studies with limited information. In a few studies, for example, con
trasts are simply reported as t or F < 1.00. In such cases, effect size estimates
were calculated by making t or F equal to a number between 0.01 and 1.00
chosen from a random numbers table. When results from a study were not reported
or were reported as nonsignificant and when t or F values were not available
but means and/or direction of means were known, a random number between
0.01 and the critical value of t or F alp = .05 was chosen to calculate an estimate
of effect size. When results for an outcome measure were not reported or were
reported as nonsignificant and when means and direction were unknown, the
effect size for that measure was set at 0.00 (indicating exactly no difference
between rewarded and nonrewarded groups). For each analysis, results were
calculated with 0.00 values included and with 0.00 values omitted.
For several studies, more than one effect size was calculated. For example, if
a single study contained two measures of intrinsic motivation (e.g., free time on
376

Cameron and Pierce
task, attitude) and two types of reward groups plus a control group (e.g., tangible
reward, verbal reward), a total of four effect sizes was calculated (e.g., free
time-tangible reward, free time-verbal reward, attitude-tangible reward, attitude-
verbal reward).
In order to satisfy the independence assumption of meta-analytic statistics
(Hedges & Olkin, 1985), only one effect size per study was entered into each
analysis. When two or more effect sizes from one study were appropriate for a
particular analysis, these effect sizes were averaged. To illustrate, for the estimate
of the overall effect of reward on the free-time measure of intrinsic motivation,
some studies assessed the effects of several types of rewards. If a single study, for
example, contained two or more reward groups (e.g., expected reward, unexpected
reward) and a control condition, the two effect sizes were averaged so that the
study contributed only one effect size to the overall analysis of reward. For an
analysis of the effects of expected reward on intrinsic motivation, only the one
appropriate effect size from the study would be used. This strategy retained as
much data as possible without violating the assumption of independence. Average
effect sizes were obtained by weighting each g index by the number of participants
on which it was based (see Cooper, 1989).
As was previously mentioned, in the single-subject, repeated measure designs,
there is no separate control group; subjects serve as their own controls. An
increase or decrease in intrinsic motivation is indexed by a difference in the
amount of time spent on the task between baseline and postreinforcement sessions.
Effect sizes for these studies were calculated by subtracting the average time
spent by all subjects in the baseline phase from the average time spent by all
subjects in the postreinforcement phase. This number was then divided by the
pooled standard deviation.
After all effect sizes were calculated, the analyses were run on the computer
program Meta (Schwarzer, 1991). Results reported in this article are based on
the weighted integration method (Hedges & Olkin, 1985). Using this technique,
effect sizes g are converted to ds by correcting them for bias (g is an overestimation
of the population effect size, particularly for small samples; see Hedges, 1981).
To obtain an overall effect size, each effect size is weighted by the reciprocal of
its variance, and the weighted ds are averaged. This procedure gives more weight
to effect sizes that are more reliably estimated. Once mean effect sizes are
calculated, 95% confidence intervals are constructed around the weighted mean.
In order to verify the accuracy of the computer program, one analysis (the
overall effect of reward on free time) was hand calculated. All obtained values
from the meta-analysis program and the hand calculations were identical within
rounding error.
To determine whether each set of effect sizes in a sample shared a common
effect size (i.e., was consistent across studies), a homogeneity statistic, Q, was
calculated. Q has an approximate chi-square distribution with k— 1 degrees of
freedom, where k is the number of effect sizes (Hedges & Olkin, 1985). The
null hypothesis is that the effect sizes are homogeneous (i.e., effect sizes in a
given analysis are viewed as values sampled from a single population; variation
in effect sizes among studies is merely due to sampling variation). For purposes
of the present analyses, samples were considered homogeneous at/? > .01.
377

When samples are not homogeneous, studies can be classified by characteris
tics, such that effect sizes within categories are homogeneous. This strategy
was undertaken by examining the effects of different types of rewards, reward
expectancies, and reward contingencies.
As a supplementary analysis, homogeneity was attained by removing outliers.
That is, studies were omitted when they provided estimates that were inconsistent
with those from other studies. Outliers in each data set were first identified using
Tukey's (1977) procedure. These outliers were then omitted from the analysis.
If homogeneity was still not attained, other studies that reduced the homogeneity
statistic by the largest amount were removed. Hedges (1987) has pointed out
that this is a common procedure in both the physical and social sciences. In one
area of physics, for example, Hedges (1987) found that data from 40% of the
available studies were omitted from calculations. For meta-analyses of psycholog
ical topics, Hedges (1987) notes that removal of up to 20% of the outliers in a
group of heterogeneous effect sizes usually results in a high degree of homoge
neity.
In an article in Psychological Bulletin, McGraw and Wong (1992) noted that
one of the problems with effect size statistics (e.g., d) is that many readers of
meta-analyses have difficulty interpreting the meaning and generalizability of
findings. McGraw and Wong have introduced another way to look at effect size,
by a statistic they call the common language effect size indicator (CL). CL refers
to the probability that a score sampled from one distribution will be greater than
a score sampled from some other distribution. McGraw and Wong suggest that
CL is a useful way to talk about effect size because it is easily interpretable.
They provide an example in which a sample of young adult men is compared
to a sample of young adult women on the variable height. A CL of .92 indicates
the probability of a male being taller than a female. Put another way, in any
random pairing of young adult men and women, the male will be taller than the
female 92 out of 100 times.
CL is calculated from means and standard deviations. Additionally, an effect
size, dy can be converted to CL by multiplying d by 1Λ/2 or 0.707 to obtain a Z
value (K.O. McGraw, personal communication, April 24, 1992). The upper tail
probability associated with this value corresponds to CL and can be calculated
using the unit normal curve.
To test the robustness of the CL statistic, McGraw and Wong (1992) conducted
a series of 118 tests (simulations) to determine the implications of violating the
assumption that sample data come from populations of values that are normally
distributed with equal variances. They found small discrepancies between the
estimate of CL under the normality assumption and the estimate of CL when the
normality assumption was violated in terms of skewness and kurtosis. The worst
case discrepancy was 0.1 which occurred with a large violation of the equal
variance assumption, considerable negative skewness, and a large violation of
kurtosis. Given the robustness of CL and the ease with which it can be interpreted,
results from the present analyses have also been expressed using the CL statistic.
The meta-analytic procedures used in the present review include: (a) the estima
tion of average effect sizes and 95% confidence intervals, (b) homogeneity
analyses to determine whether effect sizes are drawn from the same population,
(c) removal of outliers to attain homogeneity, and (d) conversion of average
378

Cameron and Pierce
effect sizes to the common language statistic (CL). Note that outliers are included
and excluded in each analysis.
RESULTS FROM GROUP DESIGNS
The Overall Effect of Reward on Intrinsic Motivation
To assess the overall effect of reward on intrinsic motivation, descriptive and
meta-analytic procedures were performed on each of the four different measures
of intrinsic motivation (free time on task, attitude, performance during the free-
time period; willingness to volunteer for future studies without reward). For each
measure, negative effects represent a decrement in intrinsic motivation; positive
effects indicate an increment.
Direction of Effects
The number of studies collected for each analysis of the overall effects of
rewards on intrinsic motivation and the direction of their effects is presented in
Table 1. On the free-time measure, the majority of studies showed that reward
decreased intrinsic motivation. However, when intrinsic motivation was measured
by attitude toward a task, performance during the free-time period, or willingness
to volunteer for future studies without reward, more studies showed positive
effects.
Distribution of Effect Sizes
Frequency distributions of the data are shown in Figure 1. Studies that found
no significant differences but did not provide sufficient information to calculate
effect sizes are not portrayed in the graphs.
When intrinsic motivation was measured as time on task following the removal
of a reward (free time), effect sizes ranged from -1.94 to 1.06. The bulk of
experiments found effects between -0.59 and 0.19. Using Tukey's (1977)
procedure, one negative outlier was identified in the free-time data. This effect
(g = -1.94) was calculated from a study conducted by Morgan (1983, Experiment
1). In this study, subjects who received an expected, task contingent (noncontin-
gent), tangible reward were compared to no-reward control subjects. The large
TABLE 1
Number of studies and direction of effects for reward versus control groups on
four measures of intrinsic motivation
Free Performance Willingness
Number of studies time Attitude in free time to volunteer
Showing a positive effect
of reward 22 31 6 6
Showing a negative effect
of reward 34 15 4 4
Showing no effect 1 1 — —
With lack of sufficient infor
mation to calculate effects 4 17 2 1
Total 61 64 12 11
379

ATTITUDE
N u m b e r of studies
FREE TIME
Number of studies
Effect sizes
Effect sizes
PERFORMANCE
Number of studies
Number of studies
WILLINGNESS TO VOLUNTEER
Effect sizes
Effect sizes
FIGURE 1. Frequency distributions of effect sizes for overall reward versus con-
trol groups on four measures of intrinsic motivation
negative effect could be due to the type of reward (tangible), the reward expec
tancy, and/or the reward contingency. All of these features are examined in further
analyses. In addition, this study was somewhat different from other studies in
that subjects who performed the activity for a reward were observed by other
subjects. That is, subjects were offered a reward for engaging in an activity while
their performance on the task was being watched. Thus, the large negative effect
could be a result of an interaction of reward type, expectancy, contingency,
and surveillance.
The attitude measure of intrinsic motivation refers to subjects' self-reports of
task interest, enjoyment, and/or satisfaction. Effect sizes ranged from —0.69 to
+1.98 with the majority of effects falling between — 0.19 and +0.59. Two positive
outliers in this data set come from studies conducted by Vallerand (1983) and
Butler (1987). In both of these studies, extrinsic verbal reward is compared to a no-
reward group. The effect of verbal reward on intrinsic motivation is investigated in
a subsequent analysis.
Effect sizes on the performance measure ranged from —3.72 to +0.96; the
median was +0.03. One large negative outlier (-3.72) comes from a study
conducted by Deci (1971, Experiment 2). This study differed from others in that
it was a field experiment where students working for a college newspaper were
380

Cameron and Pierce
paid to write headlines. Only eight subjects participated, and two subjects in the
control group dropped out and were not included in the analysis.
On the willingness-to-volunteer measure, effect sizes ranged from -0.63 to
+0.68. There were no outliers in this sample.
To establish whether the CL statistic (McGraw & Wong, 1992) could be used
confidently in the analyses, the extent to which the free-time distribution of effect
sizes deviated from normality was determined. Obtained values for skewness
and kurtosis were —0.21 and 0.55, respectively (where normal skewness and
kurtosis equal 0.00). McGraw and Wong tested the effect that violations from
normality would have on CL. Based on their findings and the skewness and
kurtosis values obtained here, in the meta-analysis of effect sizes for the free-
time measure, one could expect, at worst, an underestimate of 0.02 and an
overestimate of 0.04 for CL. Given this small discrepancy, the implication is that
the CL statistic can be used and interpreted without any serious concern about
violations of normality and homogeneity of variance.
Meta-Analysis of Effect Sizes

The overall meta-analysis of effect sizes presented in Table 2 allows one
to determine whether rewarded subjects showed less intrinsic motivation than
nonrewarded subjects as measured by time on task following the removal of
reward (free time); self-reports of task interest, satisfaction, and enjoyment (atti
tude); performance during the free-time period; and willingness to volunteer for
future studies without reward.
For each measure of intrinsic motivation, an analysis was conducted which
included all studies that provided sufficient information to calculate effect sizes
(see "All known effects" in Table 2). When samples were not homogeneous,
outliers were identified and removed using Tukey's (1977) procedure. If samples
were still significantly heterogeneous, additional outliers were removed. Homoge
neity was attained for the free-time and attitude measures by omitting approxi
mately 20% of the effect sizes, a typical meta-analytic procedure. An examination
of Table 2 indicates that the procedure of including and excluding outliers does
not drastically alter mean effect sizes.
On the free-time measure, rewarded subjects showed less intrinsic motivation
than nonrewarded controls (mean weighted d = -0.04), but this effect was not
significant (i.e., the confidence interval included 0.00). When the mean effect of
the homogeneous sample was converted to CL, results indicate that, given a
sample of studies designed to investigate the effects of reward on time on task,
51 out of 100 studies would show that overall, rewarded subjects spend less time
on the task than nonrewarded controls (assuming that all studies are of equal
importance and have the same characteristics).
Results from the attitude measure indicate greater intrinsic motivation for
rewarded subjects. This effect was small at 0.14 (from the homogeneous sample)
but differed significantly from the value of 0.00 (i.e., the confidence interval did
not include 0.00). The CL statistic was .54 and can be interpreted to mean that,
in comparisons of rewarded to nonrewarded subjects, rewarded subjects will
show a more positive attitude toward a task than nonrewarded subjects in 54 out
of 100 studies. Rewarded subjects also showed a tendency to score higher on
381

TABLE 2
Overall effect of reward versus control groups on four measures of intrinsic
motivation
Sample Mean 95% CI
Analysis k size weighted d for d CL
Free time on task
All known effects
(zeros excluded) 57 3539 -0.06 - 0 . 1 3 to 0.01 225.51* .48
Outliers removed
using Tukey's
procedure
(zeros excluded) 56 3459 -0.03 - 0 . 1 0 to 0.04 177.40* .49
Additional outliers
removed
(no zeros) 44 2634 -0.04 - 0 . 1 2 to 0.04 66.39 .49
All reports (zeros
and outliers
included) 61 3858 -0.06 - 0 . 1 2 to 0.01 225.80* .48
Attitude
All known effects
(zeros excluded) 47 3184 +0.21 0.14 to 0.29 167.50* .56
Outliers removed
using Tukey's
procedure
(zeros excluded) 45 3034 +0.17 0.09 to 0.24 110.70* .55
Additional outliers
removed
(no zeros) 39 2680 +0.14 0.06 to 0.22 58.03 .54
All reports (zeros
and outliers
included) 64 4431 +0.15 0.09 to 0.21 177.07* .54
Performance during free time period

All known effects
(zeros excluded) 10 575 +0.08 - 0 . 0 9 to 0.25 27.90* .52
Outliers removed
using Tukey's
procedure
(zeros excluded) 9 569 +0.09 - 0 . 0 8 to 0.26 21.63* .52
Additional outliers
removed
(no zeros) 8 509 -0.0004 - 0 . 1 8 to 0.18 11.73 .50
All reports (zeros
and outliers
included) 12 770 +0.06 - 0 . 0 9 to 0.21 28.07* .52

Cameron and Pierce
TABLE 2
Overall effect of reward versus control groups on four measures of intrinsic
motivation—continued
Sample Mean 95% CI
Analysis k size weighted d for d Q CL
Willingness to volunteer
All known effects
(zeros
excluded) 10 561 +0.05 -0.12 to 0.23 17.38 .52
All reports (zeros
and outliers
included) 11 609 +0.05 -0.12 to 0.22 17.42 .52
Note. Negative effect sizes indicate a decrease in intrinsic motivation for reward/
reinforcement groups; positive effect sizes indicate an increase, k = number of
effect sizes; sample size = sum of n in all studies; mean weighted d = mean of
weighted effect sizes (weighted by sample size); CI = confidence interval;
Q = homogeneity statistic for mean effect sizes; CL = common language effect
size statistic.
•Significance indicates rejection of the hypothesis of homogeneity.
*p < .01.
performance measures and to volunteer for the future projects more than nonre-
warded subjects, but these effects were not significant.
Studies that could not be represented with effect sizes were given a value of
0.00. When these studies were included in the overall analyses (see "All reports"
in Table 2), the mean effect size for each measure was little changed.
Overall, the results show that reward does not significantly affect intrinsic
motivation as measured by free time on task following removal of reward, by
performance during the free-time period, or by subjects' willingness to volunteer
for future projects without reward. When intrinsic motivation is measured by
attitude toward a task, rewarded subjects report higher intrinsic motivation than
nonrewarded subjects. It is important to point out that these main effect results
should be viewed with caution. This is because many studies show interaction
effects that are obscured when results are aggregated.
Previous reviewers (e.g., Deci & Ryan, 1985; Morgan, 1984) have suggested
that reward type, reward expectancy, and reward contingency may influence the
effect of reward on intrinsic motivation. In subsequent analyses, effect sizes have
been partitioned into groups based on these characteristics in an attempt to test
potential moderator variables and to establish homogeneity of variance.
Interactions: Effect Size as a Function of Reward Characteristics5

In the following section, type of reward and its impact on effect size are
presented. Studies are included that measured the effects of either verbal or
tangible reward (e.g., money) on intrinsic motivation. The second part of this
section involves an analysis of reward expectancy (i.e., expected and unexpected
rewards). Finally, reward contingency is assessed. Specifically, the question here
is whether effect size varies as a function of reward delivered for engaging in a
383

task, completing or solving a task, or achieving a certain level of performance.
Studies that could not be represented as effect sizes due to lack of sufficient
information are not included in further analyses presented in this article.6
Type of Reward
The purpose of the present analyses is to assess the effects of different types
of rewards (i.e., tangible and verbal) on intrinsic motivation. Because few studies
assessed intrinsic motivation as a function of "performance during the free-time
period" and "willingness to volunteer," no further analyses on these measures
have been conducted.
Effect sizes for both types of reward on the free-time and attitude measures
are presented in funnel distributions in Figure 2. Funnel graphs are used to plot
effect size against sample size of the study. The advantage of a funnel display
is that it capitalizes on a well-known statistical principle (Light & Pillemer,
1984). That is, the larger the sample, the closer the effect size will come to
represent the true underlying population value; variability due to sampling error
decreases. Conversely, smaller samples are more prone to sampling error and
are likely to deviate considerably about the true mean. For these reasons, the
distribution is expected to take the shape of an inverted funnel.
An inspection of the funnel distribution of effect sizes for the free-time measure
indicates that, overall, larger samples tend to concentrate around zero; greater
variation is evident with smaller samples. Verbal reward appears to produce a
positive effect. Results of tangible reward suggest a negative effect. These differ
ences suggest that, on the free-time measure, the effects of reward depend on
the type of reward. On the attitude measure, positive effects emerge from both
tangible and verbal reward studies; verbal reward appears to produce a slightly
more positive effect. There is no indication of a publication bias because studies
with small sample sizes and near zero effects are represented in the funnel
distribution (for a discussion of this issue, see Light & Pillemer, 1984). Although
it is not possible to rule out experimenter bias (Rosenthal, 1966), the funnel
graphs demonstrate that sampling variability may account for the fact that some
researchers find reward has a detrimental effect while others do not.
The results from the meta-analysis of the effects of reward type presented in
Table 3 indicate that, when studies compared subjects who received a verbal
reward (i.e., praise or positive feedback) to those who did not receive a reward,
rewarded subjects demonstrated significantly higher intrinsic motivation as mea
sured by both time on task and attitude. On the time measure, homogeneity was
attained by removing one outlier. This extreme positive value (+1.61) was
obtained from a study conducted in India (Tripathi & Agarwal, 1985). Because
all other studies in this analysis came from North America, the large effect size
may have been due to differences in the population studied.7 Three outliers
from studies measuring the effects of verbal reward were removed to achieve
homogeneity on the attitude measure. Inspection of these outliers suggested that
they did not differ in obvious ways from other studies in the sample except for
their tendency to generate extreme values of effect size. From these analyses,
one can estimate that the probability of a sample of verbally rewarded subjects'
being more highly intrinsically motivated than nonrewarded subjects is 0.61 (CL)
as measured by time on task and attitude toward task.
384

Cameron and Pierce
180 -
FREE TIME
O
160 -
o Tangible
140 - A A Verbal
120-
Sðmple size
100- 8 λ IDA
OO O A
80 - o o o°
o
o A OA
60 •] ° OO °¾ °¾S o O
40 A o ¾8σo A A
A
o o° o ° ġo m
A
20 A o° A o ° o O
nJ ¥ | “ ' » 1 |
u "1 ι T
2 -1 0 + +2
1
Effect size
160-
ATTITUDE o
140- o Tangible
A
A Verbal
120-
A
Sβmple size
100- CO A
A <¾B0
80- OQA O
θ
60- o
O
If*OA0CP
* <\\ A
o
OO
40- e>°oj»
° O
20 - A
fl - i • 1 •
u • ' 1 ' • 1
5 -2 1 0 1 +2 •3
Effect size
FIGURE 2. Funnel distributions of effect sizes for tangible and verbal reward
on two measures of intrinsic motivation
Studies assessing the effects of tangible reward on intrinsic motivation show

a decrease on the free-time measure as indicated by a negative mean effect size
that differed significantly from 0.00. The CL statistic of .44 implies that subjects
who receive a tangible reward will show a decrease in intrinsic motivation as
measured by time on task in 56 out of 100 studies. The mean effect size on
attitude for subjects given a tangible reward was positive, but once outliers were
removed, the mean did not differ significantly from 0.00.
385

TABLE 3
Effect size as a function of the type of reward delivered
Mean
Reward Sample weighted 95% CI
type Analysis size d ford CL
Free time on task
Verbal All known effects 15 958 4-0.42 0.29 to 0.56 29.37* .62
Verbal Outliers removed
using Tukey's
procedure 14 918 4-0.38 0.25 to 0.52 18.96 .61
Tangible All known effects 51 2983 -0.20 -0.28 t o - 0 . 1 2 181.01* .44
Tangible Outliers removed
using Tukey's
procedure 47 2761 -0.22 - 0 . 3 0 to - 0 . 1 4 97.55* .44
Tangible Additional
outliers
removed 43 2591 -0.21 - 0 . 2 9 to - 0 . 1 3 63.53 .44
Attitude
Verbal All known effects 15 1024 4-0.45 0.31 to 0.58 69.71* .63
Verbal Outliers removed
using Tukey's
procedure 13 874 4-0.30 0.15 to 0.43 26.75* .58
Verbal Additional
outliers
removed 12 785 4-0.39 0.24 to 0.53 8.73 .61
Tangible All known effects 37 2362 4-0.09 0.004 to 0.17 143.29* .52
Tangible Outliers removed
using Tukey's
procedure 33 2149 4-0.05 - 0 . 0 4 to 0.13 50.56 .52
size statistic.
*p < .01.
In summary, subjects rewarded with verbal praise or positive feedback show

significantly greater intrinsic motivation than nonrewarded subjects. Those who
receive a tangible reward evidence significantly less intrinsic motivation than
nonrewarded subjects, as measured by time on task, but they do not differ in
their reports of task interest or enjoyment.
The next step in the analysis involves a further breakdown of the effects of
tangible reward. The goal is to identify variables that may moderate the effects
of tangible reward on intrinsic motivation and to establish within-group homoge
neity. One factor that may impact effect size is whether the rewards implemented
in the studies were promised to subjects prior to the experimental sessions or
whether they were received unexpectedly.
386

Cameron and Pierce
Reward Expectancy
Within the intrinsic motivation literature, researchers draw a distinction
between expected and unexpected reward. Expected rewards refer to a procedure
whereby subjects are offered a reward prior to the experimental session and
delivered the reward following the session. Subjects who receive an unexpected
reward have not been promised the reward beforehand. These terms are generally
used to describe procedures involving the administration of tangible rewards.
In most studies on verbal reward, praise was delivered unexpectedly and was
not contingent on any specified level of performance. The few studies on verbal
reward that did employ expected and/or contingency procedures did not produce
effect sizes that deviated much from the mean effect size presented in Table 3.
For this reason, no further subdivision of effect sizes from verbal reward studies
was undertaken. The following analyses concern the effects of tangible reward.
Results are displayed in Table 4.
Only six studies assessed the effects of unexpected tangible reward on the
time measure of intrinsic motivation; five studies investigated attitude. The aver
age effect sizes for unexpected tangible reward versus control groups on free
time and attitude were slightly positive but did not differ from 0.00. These results
indicate that subjects receiving an unexpected reward do not differ significantly
from nonrewarded control subjects on measures of intrinsic motivation.
For the expected tangible reward versus control comparisons, expected reward
subjects demonstrated significantly less intrinsic motivation on the free-time
measure. On attitude, when homogeneity was attained, the two groups did not
differ.
In the following section of this article, studies comparing expected, tangible
reward groups to nonrewarded controls were further subdivided into groups based
on reward contingency.
Reward Contingency
In some studies, subjects were promised a tangible reward that was delivered
for participating in the study or for engaging in a specific task. In other studies,
a tangible reward was offered for solving a puzzle, completing a task, and/or
attaining a certain level of performance. Rewards administered in these various
ways have been labeled by Deci and Ryan (1985) as task noncontingent (rewards
offered for participating in the study regardless of what subjects do), task contin
gent (rewards offered for engaging in a task, and/or completing or solving a
task), and performance contingent (rewards offered for attaining a specified level
of performance). Table 5 presents results from the meta-analysis of these compari
sons.
Table 5 indicates that when subjects who are promised a tangible reward
regardless of what they do in the study (task noncontingent) are compared to
nonrewarded controls, no significant difference emerges on the free-time measure
of intrinsic motivation. No analyses were conducted with this type of reward
contingency on the attitude measure because only two studies of this type assessed
attitude. Subjects who receive an expected tangible reward for doing, completing,
or solving a task (task contingent) show significantly less intrinsic motivation
than controls, as measured by time on task, once reward is withdrawn. On attitude,
387

TABLE 4
Effect size as a function of reward expectancy for tangible reward versus
control comparisons
Mean
expectancy Analysis k size d for d Q CL
Free time on task: Tangible reward versus control
Unexpected All known
effects 6 275 +0.01 -0.24 to 0.25 7.38 .50
Expected All known
effects 50 2825 -0.23 -0.30 to -0.15 185.48* .44
Expected Outliers
removed
using
Tukev's
procedure 46 2603 -0.25 -0.33 to -0.17 101.36* .43
Expected Additional
outliers
removed 42 2408 -0.25 -0.33 to -0.16 64.78 .43
Attitude: Tangible reward versus control
Unexpected All known
effects 5 311 -f0.06 -0.16 to 0.28 12.42 .52
Expected All known
effects 35 2126 -f0.10 0.01 to 0.19 135.26* .53
Expected Outliers
removed
using
Tukey's
procedure 32 1961 +0.07 -0.02 to 0.16 50.48 .52
size statistic.
* Significance indicates rejection of the hypothesis of homogeneity.
*p < .01.
they show less intrinsic motivation, but this difference is not significant. When
rewards are delivered contingent on a certain level of performance, there is no
significant effect on the free-time measure; subjects in this condition do, however,
report a more positive attitude than controls.
Studies employing various reward contingencies were also categorized using
behavioral definitions. Rewards delivered for participating in a study or for
engaging in a task are referred to as noncontingent rewards. Rewards are called
contingent when they are offered for solving a puzzle, completing a task, or
reaching a specified level of performance. The results of this analysis are shown
in Table 6. The findings indicate that, when reward contingency is defined
behaviorally, subjects demonstrate a decrease in intrinsic motivation on the free-
388

TABLE 5
Effect size as a function of reward contingency (as defined by Deci & Ryan,
1985) for expected tangible reward versus control comparisons
Mean
contingency Analysis k size d ford Q CL
Free time on task: Expected tangible reward versus control
Task non-
contingent All known effects 6 225 + 0.55 + 0.27 to 0.83 20.02* .65
Task non-
contingent Outliers removed 4 124 -f0.10 -0.26 to 0.45 1.86 .53
Task
contingent All known effects 45 2257 -0.32 -0.41 to-0.24 130.90* .41
Task Outliers removed
contingent using Tukey's
procedure 44 2177 -0.28 -0.37 to -0.19 94.99* .42
Task Additional outliers
contingent removed 40 2015 -0.23 -0.32 to-0.14 62.08* .44
Performance
contingent All known effects 10 484 -0.12 -0.31 to 0.06 26.22* .47
Performance Outliers removed
procedure 8 439 -0.13 -0.34 to 0.06 17.83 .46
Attitude: Expected tangible reward versus Control
Task
Task Outliers removed
procedure 20 1157 -0.01 -0.13 to 0.10 36.24* .49
Task Additional outliers
contingent removed 19 1058 -0.08 -0.20 to 0.04 21.76 .48
Performance
contingent All known effects 14 819 +0.38 0.24 to 0.52 70.03* .61
Performance Outliers removed
procedure 13 762 + 0.29 0.14 to 0.43 27.35* .58
Performance Additional outliers
contingent removed 11 682 -f0.19 0.04 to 0.35 11.54 .55
effect sizes; sample size = sum of n in all studies; Mean weighted d = mean of
size statistic.
*p < .01.
No effect size was calculated for the attitude measure of task noncontingent rewards
because there were only two studies that fit in this category.

TABLE 6
Effect size as a function of reward contingency (as defined behaviorally) for
expected tangible reward versus control comparisons
Mean
contingency Analysis k size d ford Q CL
Free time on task: Expected tangible reward versus control
Contingent All known effects 18 906 -0.12 -0.26 to 0.01 37.44* .47
Contingent Outliers removed 16 861 -0.13 -0.26 to 0.01 29.06 .46
Non-
contingent All known effects 40 2017 -0.27 -0.35 to-0.18 167.05* .42
Non- Outliers removed
procedure 38 1894 -0.26 -0.35 to-0.16 100.86* .43
Non- Additional outliers
contingent removed 34 1728 -0.26 -0.36 to -0.16 54.66 .43
Attitude: Expected tangible reward versus control
Contingent All known effects 20 1224 -f0.24 0.12 to 0.36 88.64* .57
Contingent Outliers removed
using Tukey's
procedure 17 1087 -f0.11 -0.01 to 0.23 22.24 .53
Non-
Non- Outliers removed
procedure 16 853 +0.03 -0.10 to 0.17 31.52* .49
Non- Additional outliers
contingent removed 15 833 +0.05 -0.08 to 0.19 27.91 .48
weighted effect sizes (weighted by sample size); CI= confidence interval;
size statistic.
*Significance indicates rejection of the hypothesis of homogeneity.
*p < .01.
time measure when expected tangible rewards are not contingent on successful
performance. On the attitude measure, noncontingent rewards produce no signifi
cant effect. Rewards contingent on successful performance do not produce signifi
cant effects on either the free-time or attitude measures.
The major difference between a behavioral classification of contingency and
Deci and Ryan's categorization system concerns those studies where subjects are
given a reward for completing or solving a task. The first experiment conducted
by Deci (1971) is an example of a study coded as task contingent using Deci
and Ryan's categories and contingent using a behavioral framework. In this study,
subjects were paid money for each puzzle they solved. Deci and Ryan classified
such reward procedures as task contingent because the rewards were not contin
gent on how well subjects performed relative to some standard. From a behavioral
390

Cameron and Pierce
perspective, however, completion or solution of a task is seen as dependent on
successful performance; these studies were labeled contingent. Thus, performance
contingent rewards as defined by Deci and Ryan (1985) include only those studies
where subjects are offered a reward for attaining a certain level of performance;
using a behavioral definition, studies coded as contingent include both rewards
that are contingent on completing or solving a task and rewards that are contingent
on reaching a specified level of performance. Because these two types of reward
contingencies may have opposite effects on intrinsic motivation, a separate analy
sis was conducted on studies in which reward was delivered for completing or
solving a task. Results given in Table 7 show no significant differences between
rewarded and control groups on the free-time or attitude measures for this type
of reward contingency. These findings suggest that contingent rewards (which
include performance contingent rewards), as defined behaviorally, do not harm
Summary of Results From Group Designs
A summary of the various analyses conducted on the group design studies and
the major findings is given in Figure 3.8
When all types of reward are aggregated, overall, the results indicate that
reward does not negatively affect intrinsic motivation on any of the four measures
(free time on task once reward is withdrawn, self-reports of attitude, performance
during the free-time measure, willingness to volunteer for future studies without
reward). When rewards are subdivided into reward type (verbal, tangible), reward
expectancy (expected, unexpected), and reward contingency, the findings demon
strate that people who receive a verbal reward spend more time on a task once
the reward is withdrawn; they also show more interest and enjoyment than
nonrewarded persons.
Tangible reward produces no decrement in intrinsic motivation when it is
received unexpectedly. Expected tangible rewards produce differing effects
depending on the manner in which they are administered. Individuals who receive
an expected reward for solving or completing a task or for achieving a specific
TABLE 7
Effect size as a function of rewards contingent on task completion or solution
for expected tangible reward versus control comparisons
Mean
Sample weighted 95% CI
Measure k size d for d CL e
Free time 8 423 -0.12 -0.32 to 0.08 11.21 .47
Attitude 6 405 -0.05 -0.25 to 0.14 6.89 .48
size statistic.
*Significance indicates rejection of the hypothesis of homogeneity.
*/? < .01.
391

INTRINSIC MOTIVATION
Willingness to
Performance
Free T i m e
] [A t t i t u d e Volunteer
Revaid H£3S£Ĵ_ Revaid Revaid

its. (dv*0.14) n.3. n.s.
/ \ \
Verbal Tangible Verbal Tangible
(dv « 0.38) (ev*-0.21) (dvjLCL22) XLS.
Unexpected
n.3.
\
Expected Unexpected Expected
(tV--O.2S) O.S. IL9.
Contingent
n.3.
\
N«ncoαtingent
(dv - - 0 . 2 6 )
Contingent
XLS.
\ Noncontingent
\
Contingent on \
task completion Contingent on task
or sol ution completion or solution
Task contingent Task noncontingent Performance Task contingent

n.¾. c¢ntinoent n.s
<«V = - 0 . 2 3 )
JgV“P,i9).
FIGURE 3. A summary of the meta-analysis of the effects of reward versus control
groups on intrinsic motivation
Note, dw = mean weighted effect size (based on homogeneous samples); n.s. =
not significant; analyses in regular type indicate no effect; analyses in bold indicate
a negative effect; underlined analyses indicate a positive effect. When no dw is
reported, there was no significant effect. No analyses were conducted on the attitude
measure for task noncontingent reward because only two studies assessed this measure.
level of performance do not spend less time on a task than controls once the
reward is withdrawn. They do, however, report more interest, satisfaction, and
enjoyment of the task when the reward is given for a certain level of performance.
The detrimental effects of reward appear when rewards are offered to people
simply for engaging in a task, independent of successful performance. Under
these conditions, once the reward is removed, individuals spend less time on the
task than controls; they do not, however, report a less favorable attitude toward
the task.
RESULTS FROM SINGLE-SUBJECT DESIGNS
To determine the effects of reinforcement on intrinsic motivation, an analysis
was conducted on effect sizes from single-subject, repeated measures designs
where the rewards used were shown to be reinforcers for each subject in the
study. That is, rewards were shown to increase behavior during a reinforcement
phase. An increase or decrease in intrinsic motivation was measured as a differ
ence between behavior during the pre- and postreinforcement phases. Five studies
392

Cameron and Pierce
contributed an effect size to this analysis. Four studies showed that subjects spent
more time on the task during the postreinforcement phase than the baseline phase.
One study (Vasta & Stirpe, 1979) showed a decrease in time on task immediately
following the removal of reward but an increase in time when intrinsic motivation
was measured 2 weeks later. To make this analysis comparable to the analysis
of group design studies, however, only differences between the immediate postre
inforcement phase and baseline were analyzed.
The average effect size and confidence interval for this analysis was +0.34
(—0.28, 0.96) indicating no significant change in intrinsic motivation from base
line to postreinforcement phases. Effect sizes were homogeneous (Q = 2.96,
df — 4). These results suggest that reinforcement does not alter people's intrin
sic motivation.
As noted previously, two studies used a between- and within-group repeated
measures design to assess the effects of reinforcement on intrinsic motivation
(Greene, Sternberg, & Lepper, 1976; Mynatt, Oakley, Arkkelin, Piccione, Mar-
golis, & Arkkelin, 1978). Although these studies did not meet the criteria for
inclusion in the meta-analysis of within-subject designs, it is possible to assess
the within-group effects for reward conditions that were comparable in both
studies. Both Greene et al. (1976) and Mynatt et al. (1978) included a group of
subjects rewarded for playing with activities that they had spent the most time with
during baseline phases (high interest condition) and a group that was rewarded for
playing with activities they had spent the least time with during baseline (low
interest condition). In terms of the high interest conditions, Mynatt et al. did not
find a reinforcement effect but reported a decrease in intrinsic motivation from
baseline to postreward phases. Greene et al. reported a reinforcement effect for
the high interest group and a decrease in intrinsic motivation between baseline
and postreinforcement sessions. It is difficult to draw conclusions from only two
studies. Nonetheless, because a decline in intrinsic motivation occurred with or
without a reinforcement effect, it may be that reinforcement is not the critical
variable. Both studies reported a reinforcement effect for the low interest condi
tions, but there was no change in intrinsic motivation from baseline to postrein
forcement phases. Again, conclusions based on two studies are tenuous. One
interpretation, however, is that the time spent on low interest activities was so
low that a decline in intrinsic motivation could not be detected. Alternatively,
reinforcement does not interrupt intrinsic motivation for low interest activities.
DISCUSSION
A major contention in education and psychology is that rewards and reinforce
ment negatively impact a person's intrinsic motivation. The view is that, if people
are reinforced or rewarded for activities they already spend time on and enjoy,
they will be less motivated to engage in the activity than they were prior to the
introduction of reward, once the reward is no longer forthcoming. In other words,
rewards and reinforcement are said to decrease people's intrinsic motivation.
Over the past 20 years, dozens of studies have been conducted to investigate
this issue. The primary objective of this article was to assess the research findings
by conducting a meta-analysis of results from experiments on the effects of
reward and reinforcement on intrinsic motivation. What follows is a discussion
of the results obtained from the meta-analysis.
393

The vast majority of studies have assessed the effects of reward on intrinsic
motivation by using group designs. Rewarded subjects are compared to nonre-
warded controls. Intrinsic motivation is measured by differences between groups
on attitude, time spent on a task following the removal of reward (free time),
performance during the free-time period, and willingness to volunteer for future
studies without reward. The main meta-analysis reported in this article was
conducted on results from these studies. This analysis concerned assessing the
overall effects of reward on intrinsic motivation as well as the effects of a number
of reward characteristics. The results suggest that in the laboratory, overall, reward
does not negatively impact intrinsic motivation on any of the four measures
analyzed here.
A separate analysis was conducted using single-subject, repeated measures
designs. A few researchers employed this type of design to evaluate the effects
of reinforcement on intrinsic motivation. The rewards used in these studies were
shown to be reinforcers, and intrinsic motivation was indexed as differences in
subjects' behavior between pre- and postreinforcement sessions. Results from
the meta-analysis indicate no effect of reinforcement on intrinsic motivation.
That is, the evidence suggests that reinforcement does not decrease a person's
intrinsic motivation to engage in an activity.
In terms of rewards and extrinsic reinforcement, our overall findings suggest
that there is no detrimental effect on intrinsic motivation. These findings are based
on laboratory experiments, but a similar conclusion was reached by Workman and
Williams (1980) in their review of the effects of extrinsic rewards on intrinsic
motivation in the classroom. Generally, on task behavior, Workman and Williams
found that external reinforcement increased and maintained intrinsic motivation
for prolonged periods (up to 12 months). Thus, it no longer seems appropriate
to argue against the use of incentive systems in applied settings.
The findings from both experimental and applied research run contrary to the
views expressed by many psychologists and educators (e.g., Deci & Ryan, 1985;
Kohn, 1993; Levine & Fasnacht, 1974; Schwartz, 1990). For example, Deci and
Ryan (1987) state that:
In general [italics added], rewards have been found to undermine intrinsic
motivation. When people received rewards for working on an interesting
activity, they tended to display less interest in and willingness to work on
that activity after the termination of the rewards than did people who had
worked on the activity without receiving a reward, (p. 1026)
Results from the present meta-analysis suggest that this statement is erroneous.
The findings indicate that, in general, rewarded people are not less willing to
work on activities and they do not display a less favorable attitude toward tasks
than people who do not receive rewards.
When rewards are broken down into reward type, expectancy, and contingency,
results indicate that, on the free-time measure, verbal reward produces an increase
in intrinsic motivation; tangible rewards produce no effect when they are delivered
unexpectedly, and they are not detrimental when they are expected and contingent
on level of performance or completing or solving a task. Expected tangible
rewards produce a decrease in intrinsic motivation as measured by free time on
task when they are given to individuals simply for engaging in an activity. On
394

Cameron and Pierce
the attitude measure, verbal reward produces an increase in intrinsic motivation,

and tangible rewards do not lead to a decrease in intrinsic motivation under any
conditions. An increase in intrinsic motivation is shown on the attitude measure
when individuals are offered a reward for performing to a set of standards. Thus,
the present results suggest that rewards are detrimental only under a highly
specified set of circumstances. That is, when subjects are offered a tangible
reward (expected) that is delivered regardless of level of performance, they spend
less time on a task than control subjects once the reward is removed. The same
condition has no effect on attitude.
Given these results, why is it that one commonly finds general statements
condemning reinforcement and/or reward in journal articles and introductory
textbooks? The present meta-analysis makes it clear how circumscribed the
negative effect of reward really is. One possibility is that terms such as tangible,
expected, unexpected, contingent and noncontingent become very confusing to
a reader sorting through this literature. Consider, at its simplest, a study investigat
ing the effects of expected reward on intrinsic motivation. Suppose the results
showed a negative effect for expected reward. When discussing findings, do the
researchers talk about the negative effects of the promise of reward or about the
negative effects of reward, in general? There is no doubt that conclusions reached
from such studies are often made about reward or reinforcement in general, not
promise of reward. This has led to a great deal of misunderstanding about the
overall effects of reward and reinforcement on intrinsic motivation. Even an
informed reader can have difficulty keeping in mind what a particular study is
investigating. It may be for this reason that rewards are often equated with
reinforcers and, overall, have come to be seen as harmful. It is hoped that the
present meta-analysis has helped to clarify the issue.
Theoretical Implications
How do results from the present meta-analysis fit in with the various theories
that have been formulated to account for the negative effects of rewards on
intrinsic motivation?
Advocates of cognitive evaluation theory (e.g., Deci & Ryan, 1985) would
probably not have difficulty reconciling results from the free-time measure of
intrinsic motivation. According to cognitive evaluation theory, competence and
self-determination underlie intrinsic motivation. Rewards can facilitate or hinder
competence and self-determination depending on whether they are perceived as
informational, controlling, or amotivational. From this perspective, results from
the meta-analysis would suggest that verbal rewards increase a person's intrinsic
motivation because of their informational value. Verbal praise would be seen to
lead an individual to feel competent in performing a task; hence, intrinsic motiva
tion would increase. Because the cognitive evaluation process is said to take
place while the rewarded activity is occurring, unexpected rewards would not
alter a person's intrinsic motivation. On the other hand, rewards offered to people
for participating in a task, in spite of how well they perform, would be perceived
as controlling and would decrease intrinsic motivation.
The problem for cognitive evaluation theory arises when one considers results
from the attitude measure of intrinsic motivation. Deci and Ryan (1985) suggest
that interest, enjoyment, and satisfaction are central emotions that accompany
395

intrinsic motivation. A person's experience of an activity is a focal point of

cognitive evaluation theory. In other words, cognitive evaluation theory depends
on an internal attitude change that is later expressed behaviorally as time on task.
Results from the present meta-analysis indicate that reward does not negatively
affect attitude. Individuals who receive verbal praise report greater interest than
nonrewarded people. Tangible rewards produce no change in attitude when they
are given for doing, completing, or solving a task; a positive effect is evident
when rewards are contingent on a specified level of performance.
One way of mitigating the findings for cognitive evaluation theory is to question
the reliability of the attitude measure. In many studies, the attitude measure was
obtained from a single-item Likert scale. An additional problem is that the
questions designed to assess attitude toward the task may have been unable to
separate subjects' liking of the reward from their liking of the task. If the attitude
measures are unreliable, they will fail to reflect true differences between rewarded
and nonrewarded groups. This may be one way to handle the puzzling results;
however, it also suggests that there has been no test of the major mediator
proposed by the theory.
The problem of operationalizing the construct of intrinsic motivation was
recently addressed in a meta-analysis by Wiersma (1992).9 Results from Wiers-
ma's study depended on whether intrinsic motivation was operationalized as a
free-time measure or as a task performance during rewarded period measure.
Free-time measures showed a decline in intrinsic motivation; performance mea
sures showed an increase. As noted, in the present analyses, results from the
attitude measure do not coincide with the free-time measure. Additionally, mea
sures of intrinsic motivation as performance during free time or as willingness
to volunteer for future studies do not clarify the issue of operationalization of
Given the lack of covariation among the measures, it seems appropriate to
devote further research to clarifying the concept of intrinsic motivation and to
developing suitable measures. A different solution is offered by Rigby, Deci,
Patrick, and Ryan (1992) who suggest that attention be directed toward the
concept of self-determination rather than a pursuit of the intrinsic/extrinsic dichot
omy. Others concur but suggest that researchers should focus on goal definitions
(Sansone & Morgan, 1992). A final alternative would be to agree that constructs
such as self-determination, goal definition, and intrinsic motivation are scientifi
cally unclear and that it would be more appropriate to deal with the effects of
reward and reinforcement on behavior (e.g., Bandura, 1977, 1986; Dickinson,
1989). Such a course of action would mean abandoning cognitive evaluation
theory.
Another theoretical explanation that has been proposed to account for the
effects of rewards on intrinsic motivation is the overjustification effect (Lepper,
Greene, & Nisbett, 1973). The view is that people's perceptions about the causes
of their behavior influence future motivation. Rewards lead to a decrease in
intrinsic motivation when people's perceptions shift from accounting for their
behavior as self-initiated to accounting for it in terms of external reward. Because
the present analysis did not evaluate subjects' perceptions about the causes of
their behavior, it is impossible to determine whether overjustification explains
396

Cameron and Pierce
the results. Further research that measures subjects' attributions to internal and
external factors is warranted.
Finally, how would the findings of the meta-analysis be interpreted from a
behavioral perspective? The results from single-subject designs indicate that
reinforcement does not produce decrements in intrinsic motivation. This finding
is compatible with a behavioral view. That is, behaviorists maintain that behavior
returns to baseline after reinforcement is withdrawn. If the rewards used in the
groups' design studies are reinforcers, one would expect behavior to eventually
return to baseline. Research designed to investigate the effects of reward on
intrinsic motivation has typically measured time on task for a brief 8- to 10-
minute period, immediately following the removal of reward. Thus, if verbal
praise were a reinforcer, one might interpret the positive effect as a carryover of
the reinforcement procedure. Another interpretation is that the positive effect is
the result of an extinction burst. That is, when reinforcement is first withdrawn,
the immediate, short-term effect is that rate of response increases. After a period
of time, behavior would return to baseline. In terms of the negative effect of
expected, noncontingent, tangible reward, some writers (e.g., Dickinson, 1989;
Flora, 1990) have suggested that such a reward procedure does not represent a
reinforcement contingency. The promise of a reward is seen by behaviorists as
a discriminative stimulus (5°), and the negative effect is understood as the result
of a bribe. A difficulty with this interpretation is that it does not account for
findings from other conditions where promise of reward does not produce a
negative effect. Further research is necessary to determine when and under what
conditions promises of rewards function as bribes. Our data suggest that promises
linked to noncontingent reward may function as bribes rather than as positive
incentives.
Practical Implications
The present findings suggest that verbal praise and positive feedback enhance
people's intrinsic interest. This is an important finding. Most social interaction in
business, education, and clinical settings involves verbal feedback from managers,
teachers, and therapists. When praise and other forms of positive feedback are
given and later removed, people continue to show intrinsic interest in their work.
In contrast to recent claims made by Kohn (1993, p. 55), verbal praise is an
extrinsic motivator that positively alters attitudes and behavior.
Rewards can have a negative impact on intrinsic motivation when they are
offered to people for engaging in a task without consideration of any standard
of performance. In a classroom, this might occur if a teacher promised students
tangible rewards simply for doing an activity. For example, a teacher who prom
ises stars or other awards to students for spending time doing math problems
may undermine intrinsic motivation. In such a case, one could expect rewarded
individuals to enjoy the task as much as those who are not offered an incentive.
But, they may spend less time on the activity in a study period when the reward
is no longer forthcoming. According to our results, this would not occur if the
teacher used the same rewards but made them contingent on successful completion
of the problems.
Overall, the present review suggests that teachers have no reason to resist
implementing incentive systems in the classroom. This conclusion is based on
397

our findings, which show that verbal praise enhances intrinsic motivation and
that other rewards and reinforcement leave intrinsic motivation largely unaffected.
A small negative effect occurs when tangible rewards are promised without
regard to a standard of performance. Under this circumstance, the promise of
reward may act as a bribe. Importantly, on a practical level, the implication is that
reward offered in educational and other settings should be delivered contingent on
performance.
Notes
'Although there was an overall positive effect of tangible reward on intrinsic
motivation, Rosenfìeld et al.(l98O) also found that rewards that did not indicate
ability led to less intrinsic interest.
2
In addition to studies reported in English, five relevant Japanese experiments
were identified by the CD-ROM search. The information in the abstracts was not
adequate to code the findings. Therefore, these studies are not included in the
meta-analysis.
3
Boggiano and Ruble (1979) reported that 147 children participated in the study.
There were two reward conditions (task contingent, performance contingent) and a
nonrewarded control group. The contrast for the control versus task-contingent reward
groups on the free-time measure is reported as í(l3O) = 2.0, p < .05; the contrast
for the control versus performance-contingent reward groups is reported as /(130) =
1.16, n.s.
4
A copy of the coding form is available on request from the first author.
5
A list of the experiments included in each interaction is available on request from
the first author.
6
Further analyses which include studies that index effect size as 0.00 are available
in Cameron (1992).
7
The present review does not assess cultural differences in the impact of reward
on intrinsic motivation. However, it is interesting to note that, although the study
from India (Tripathi & Agarwal, 1985) shows an extreme positive value for the effect
of verbal praise on the free-time measure, the direction of the result is consistent
with the North American studies.
8
A few researchers have assessed the effects of expected tangible rewards on
intrinsic motivation relative to unexpected tangible rewards (e.g., Enzle & Ross,
1978; Fazio, 1981; Lepper & Greene, 1975). Other researchers have conducted studies
comparing expected noncontingent reward groups to expected contingent reward
groups (e.g., Fair, 1976; Phillips & Lord, 1980; Pinder, 1976). Such studies concern
direct comparisons between the two types of reward expectancies (expected versus
unexpected) and the two types of reward contingencies (noncontingent versus contin
gent) without reference to a nonrewarded control group. Results from meta-analyses
conducted on these comparisons and a list of studies included in such analyses can
be obtained in Cameron (1992). One significant effect emerged from these analyses;
subjects who received an expected tangible reward showed less intrinsic motivation
on the free-time measure than subjects who received an unexpected tangible reward.
The average effect size and confidence interval for this comparison was —0.26
(-0.45, -0.06).
9
Wiersma (1992) reported results of a meta-analysis of 23 experiments on reward
and intrinsic motivation. These studies make up a subset of those analyzed in the
present article. Effect sizes from Wiersma's study were not always based on a
comparison of a reward condition to a no-reward condition. This makes it impossible
to directly compare our findings with those of Wiersma.
398

Cameron and Pierce
APPENDIX A
Studies included in the meta-analysis of group designs
Amabile, T. M., Hennessey, B. A., & Grossman, B. S. (1986). Social influences on
creativity: The effects of contracted-for reward. Journal of Personality and Social
Psychology, 50, 14-23.
Anderson, R., Manoogian, S. T., & Reznick, J. S. (1976). The undermining and
enhancing of intrinsic motivation in preschool children. Journal of Personality
and Social Psychology, 34, 915-922.
Anderson, S., & Rodin, J. (1989). Is bad news always bad? Cue and feedback effects
on intrinsic motivation. Journal of Applied Social Psychology, 19, 449-467.
Arkes, H. R. (1979). Competence and the overjustification effect. Motivation and
Emotion, 3, 143-150.
Arnold, H. J. (1976). Effects of performance feedback and extrinsic reward upon
high intrinsic motivation. Organizational Behavior and Human Performance,
17, 275-288.
Arnold, H. J. (1985). Task performance, perceived competence, and attributed causes
of performance as determinants of intrinsic motivation. Academy of Management
Journal, 28, 876-888.
Blanck, P. D., Reis, H. T., & Jackson, L. (1984). The effects of verbal reinforcement
of intrinsic motivation for sex-linked tasks. Sex Roles, 10, 369-386.
Boal, K. B., & Cummings, L. L. (1981). Cognitive evaluation theory: an experimental
test of processes and outcomes. Organizational Behavior and Human Performance,
28, 289-310.
Boggiano, A. K., Harackiewicz, J. M , Besette, J. M , Main, D. S. (1985). Increasing
children's interest through performance contingent reward. Social Cognition, 3,
400-411.
Boggiano, A. K., & Hertel, P. T. (1983). Bonuses and bribes: mood effects in memory.
Social Cognition, 2, 49-61.
Boggiano, A. K., Ruble, D. N., & Pittman, T. S. (1982). The mastery hypothesis and
the overjustification effect. Social Cognition, 1, 38-49.
Brennan, T. P., & Glover, J. A. (1980). An examination of the effect of extrinsic
reinforcers on intrinsically motivated behavior: experimental and theoretical. Social
Behavior and Personality, 8, 27-32.
Broekner, J., & Vasta, R. (1981). Do causal attributions mediate the effects of extrinsic
rewards on intrinsic interest? Journal of Research in Personality, 15, 201-209.
Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects
of different feedback conditions on motivational perceptions, interest, and perfor
mance. Journal of Educational Psychology, 79, 474-482.
Calder, B. J., & Staw, B. M. (1975). Self-perception of intrinsic and extrinsic motiva
tion. Journal of Personality and Social Psychology, 31, 599-605.
Crino, M. D., & White, M. C. (1982). Feedback effects in intrinsic/extrinsic reward
paradigms. Journal of Management, 8, 95-108.
Daniel, T. L., & Esser, J. K. (1980). Intrinsic motivation as influenced by rewards,
task interest, and task structure. Journal of Applied Psychology, 65, 566-573.
Danner, F. W., & Lonkey, E. (1981). A cognitive developmental approach to the
effects of rewards on intrinsic motivation. Child Development, 52, 1043-1052.
Deci, E. L. (1971). Effects of externally mediated rewards on intrinsic motivation.
Journal of Personality and Social Psychology, 18, 105-115.
Deci, E. L. (1972a). Intrinsic motivation, extrinsic reinforcement, and inequity. Jour-
nal of Personality and Social Psychology, 22, 113-120.
399

Deci, E. L. (1972b). The effects of contingent and noncontingent rewards and controls
on intrinsic motivation. Organizational Behavior and Human Performance, 8,
217-229.
DeLoach, L. L., Griffith, K., & LaBarba, R. C. (1983). The relationship of group
context and intelligence to the overjustifícation effect. Bulletin of the Psychonomic
Society, 21, 291-293.
Dollinger, S. J., & Thelen, M. H. (1978). Overjustifícation and children's intrinsic
motivation: comparative effects of four rewards. Journal of Personality and Social
Psychology, 36, 1259-1269.
Earn, B. M. (1982). Intrinsic motivation as a function of extrinsic financial rewards
and subjects' locus of control. Journal of Personality, 50, 360-373.
Fabes, R. A. (1987). Effects of reward contexts on young children's task interest.
Journal of Psychology, 121, 5-19.
Fabes, R. A., Eisenberg, N., Fultz, J., & Miller, R (1988). Reward, affect and young
children's motivational orientation. Motivation and Emotion, 12, 155-169.
Freedman, S. M., & Phillips, J. S. (1985). The effects of situational performance
constraints on intrinsic motivation and satisfaction: the role of perceived compe
tence and self-determination. Organizational Behavior and Human Decision Pro-
cesses, 35, 397^416.
Greene, D., & Lepper, M. R. (1974). Effects of extrinsic rewards on children's
subsequent intrinsic interest. Child Development, 45, 1141-1145.
Griffith, K. M., DeLoach, L. L., & LaBarba, R. C. (1984). The effects of rewarder
familiarity and differential reward preference in intrinsic motivation. Bulletin of
the Psychonomic Society, 22, 313-316.
Hamner, W. C , & Foster, L. W. (1975). Are intrinsic and extrinsic rewards additive:
A test of Deci's cognitive evaluation theory of task motivation. Organizational
Behavior and Human Performance, 14, 398-415.
Harackiewicz, J. M. (1979). The effects of reward contingency and performance
feedback on intrinsic motivation. Journal of Personality and Social Psychology,
37, 1352-1363.
Harackiewicz, J. M., Abrahams, S., & Wageman, R. (1987). Performance evaluation
and intrinsic motivation: The effects of evaluative focus, rewards, and achievement
orientation. Journal of Personality and Social Psychology, 53, 1015-1023.
Harackiewicz, J. M., & Manderlink, G. (1984). A process analysis of the effects of
performance-contingent rewards on intrinsic motivation. Journal of Experimental
Social Psychology, 20, 531-551.
Harackiewicz, J. M., Manderlink, G., & Sansone, C. (1984). Rewarding pinball
wizardry: effects of evaluation and cue value on intrinsic interest. Journal of
Personality and Social Psychology, 47, 287-300.
Horn, H. L. (1987). A methodological note: time of participation effects on intrinsic
motivation. Personality and Social Psychology Bulletin, 13, 210-215.
Karniol, R., & Ross, M. (1977). The effect of performance relevant and performance
irrelevant rewards on children's intrinsic motivation. Child Development, 48,
482^‡87.
Koestner, R., Zuckerman, M., & Koestner, J. (1987). Praise, involvement, and intrinsic
motivation. Journal of Personality and Social Psychology, 53, 383-390.
Kruglanski, A. W., Alon, S., & Lewis, T. (1972). Retrospective misattribution and
task enjoyment. Journal of Experimental Social Psychology, 8, 493-501.
Kruglanski, A. W., Friedman, I., & Zeevi, G. (1971). The effects of extrinsic incentive
on some qualitative aspects of task performance. Journal of Personality, 39,
606-617.
400

Cameron and Pierce
Kruglanski, A. W., Riter, A., Amitai, A., Margolin, B. S., Shabatai, L., & Zaksh, D.
(1975). Can money enhance intrinsic motivation?: A test of the content-consequence
hypothesis. Journal of Personality and Social Psychology, 31, 744-750.
Lepper, M. R., Greene, D., & Nisbett, R. E. (1973). Undermining children's intrinsic
interest with extrinsic reward: A test of the "overjustification" hypothesis. Journal
of Personality and Social Psychology, 28, 129-137.
Loveland, K. K., & Olley, J. G. (1979). The effect of external reward on interest and
quality of task performance in children of high and low intrinsic motivation. Child
Development, 50, 1207-1210.
Luyten, H., & Lens, W. (1981). The effect of earlier experience and reward contingen
cies on intrinsic motivation. Motivation and Emotion, 5, 25-36.
McGraw, K. O., & McCullers, J. C. (1979). Evidence of a detrimental effect of
extrinsic incentives on breaking a mental set. Journal of Experimental Social
McLoyd, V. C. (1979). The effects of extrinsic rewards of differential value on high
and low intrinsic interest. Child Development, 50, 1010-1019.
Morgan, M. (1981). The overjustification effect: A developmental test of self-percep
tion interpretations. Journal of Personality and Social Psychology, 40, 809-821.
Morgan, M. (1983). Decrements in intrinsic interest among rewarded and observer
subjects. Child Development, 54, 636-644.
Mynatt, C., Oakley, D., Piccione, A., Margolis, R., & Arkkelin, J. (1978). An examina
tion of overjustification under conditions of extended observation and multiple
reinforcement: Overjustification or boredom? Cognitive Therapy and Research,
2, 171-177.
Ogilvie, L., & Prior, M. (1982). The overjustification effect in retarded children:
durability and generalizability. Australia and New Zealand Journal of Develop-
mental Disabilities, 8, 213-218.
Orlick, T. D., & Mosher, R. (1978). Extrinsic awards and participant motivation in
a sport related task. International Journal of Sport Psychology, 9, 27-39.
Palack, S. R., Costomotis, S., Sroka, S., & Pittman, T. S. (1982). School experience,
reward characteristics, and intrinsic motivation. Child Development, 53,
1382-1391.
Pittman, T S., Cooper, E. E., & Smith, T. W. (1977). Attribution of causality and
the overjustification effect. Personality and Social Psychology Bulletin, 3, 280-283.
Pittman, T. S., Davey, M. E., Alafat, K. A., Wetherill, K. V., & Kramer, N. A. (1980).
Informational versus controlling verbal rewards. Personality and Social Psychology
Bulletin, 6, 228-233.
Pittman, T. S., Emery, J., & Boggiano, A. K. (1982). Intrinsic and extrinsic motiva
tional orientations: reward-induced changes in preference for complexity. Journal
Porac, J. E, & Meindl, J. (1982). Undermining overjustification: Inducing intrinsic
and extrinsic task representations. Organizational Behavior and Human Perfor-
mance, 29, 208-226.
Pretty, G. H., & Seligman, C. (1984). Affect and the overjustification effect. Journal
Reiss, S., & Sushinsky, L. W. (1975). Overjustification, competing responses, and
the acquisition of intrinsic interest. Journal of Personality and Social Psychology,
31, 1116-1125.
Rosenfield, D., Folger, R., & Adelman, H. F. (1980). When rewards reflect compe
tence: A qualification of the overjustification effect. Journal of Personality and
401

Ross, M. (1975). Salience of reward and intrinsic motivation. Journal of Personality

and Social Psychology, 32, 245-254.
Ross, M., Karnio, R., & Rothstein, M. (1976). Reward contingency and intrinsic
motivation in children: a test of the delay of gratification hypothesis. Journal of
Ryan, R. M., Mims, B., & Koestner, R. (1983). Relation of reward contingency and
interpersonal context to intrinsic motivation: A review and test using cognitive
evaluation theory. Journal of Personality and Social Psychology, 45, 736-750.
Salinick, G. R. (1975). Interaction effects of performance and money on self-percep
tion of intrinsic motivation. Organizational Behavior and Human Performance,
13, 339-351.
Sansone, C. (1986). A question of competence: the effects of competence and task
feedback on intrinsic interest. Journal of Personality and Social Psychology, 51,
918-931.
Sansone, C. (1989). Competence feedback, task feedback, and intrinsic interest: An
examination of process and context. Journal of Experimental Social Psychology,
25, 343-361.
Sansone, C , Sachau, D. A., & Weir, C. (1989). Effects of instruction on intrinsic
interest: The importance of context. Journal of Personality and Social Psychology,
57, 819-829.
Sarafìno, E. R (1984). Intrinsic motivation and delay of gratification in preschoolers:
the variables of reward salience and length of expected delay. British Journal of
Developmental Psychology, 2, 149-156.
Shanab, M. E., Peterson, D., Dargahi, S., & Deroian, R (1981). The effects of positive
and negative verbal feedback on the intrinsic motivation of male and female
subjects. The Journal of Social Psychology, 115, 195-205.
Shapira, Z. (1976). Expectancy determinants of intrinsically motivated behavior.
Smith, T W, & Pittman, T S. (1978). Reward, distraction, and the overjustification
effect. Journal of Personality and Social Psychology, 36, 565-573.
Staw, B. M., Calder, B. J., Hess, R. K., & Samdelands, L. E. (1980). Intrinsic
motivation and norms about payment. Journal of Personality, 48, 1-14.
Swann, W. B., Jr., & Pittman, T S. (1977). Moderating influence of verbal cues on
intrinsic motivation. Child Development, 48, 1128-1132.
Taub, S. I., & Dollinger, S. J. (1975). Reward and purpose as incentives for children
differing in locus of control expectancies. Journal of Personality, 43, 179-195.
Tripathi, K. N., & Agarwal, A. (1985). Effects of verbal and tangible rewards on
intrinsic motivation in males and females. Psychological Studies, 30, 77-84.
Tripathi, K. N., & Agarwal, A. (1988). Effect of reward contingency on intrinsic
motivation. The Journal of General Psychology, 115 (3), 241-246.
Vallerand, R. J. (1983). The effect of differential amounts of positive verbal feedback
on the intrinsic motivation of male hockey players. Journal of Sport Psychology,
5, 100-107.
Vallerand, R. J., & Reid, G. (1984). On the causal effects of perceived competence
on intrinsic motivation: A test of cognitive evaluation theory. Journal of Sport
Weinberg, R. S., & Jackson, A. (1979). Competition and extrinsic rewards: Effect
on intrinsic motivation and attribution. Research Quarterly, 50, 494-502.
Weiner, M. J. (1980). The effect of incentive and control over outcomes upon intrinsic
motivation and performance. The Journal of Social Psychology, 112, 247-254.
Weiner, M. J., & Mander, A. M. (1978). The effects of reward and perception of
competency upon intrinsic motivation. Motivation and Emotion, 2, 67-73.
402

Cameron and Pierce
Wicker, F. W., Brown, G., Wiehe, J. A., & Shim, W. Y. (1990). Moods, goals, and
measures of intrinsic motivation. The Journal of Psychology, 124, 75-86.
Williams, B. W, (1980). Reinforcement, behavior constraint and the overjustification
effect. Journal of Personality and Social Psychology, 39, 599-614.
Wimperis, B. R., & Farr, J. L. (1979). The effects of task content and reward
contingency upon task performance and satisfaction. Journal of Applied Social
Psychology, 9 (3), 229-249.
Zinser, O., Young, J. G., & King, P. E. (1982). The influence of verbal reward on
intrinsic motivation in children. The Journal of General Psychology, 106, 85-91.
Studies Included in the Meta-Analysis of Single-Subject Designs

Davidson, P., & Bucher, B. (1978). Intrinsic interest and extrinsic reward: The effects
of a continuing token program on continuing nonconstrained preference. Behavior
Therapy, 9, 222-234.
Feingold, B. D., & Mahoney, M. J. (1975). Reinforcement effects on intrinsic interest:
Undermining the overjustification hypothesis. Behavior Therapy, 6, 357-377.
Mawhinney, T C , Dickinson, A. M., & Taylor, L. A. (1989). The use of concurrent
schedules to evaluate the effects of extrinsic rewards on "intrinsic motivation."
Journal of Organizational Behavior Management, 10, 109-129.
Vasta, R., Andrews, D. E., McLaughlin, A. M., Stirpe, L. A., & Comfort, C. (1978).
Reinforcement effects on intrinsic interest: A classroom analog. Journal of School
Vasta, R., & Stirpe, L. A. (1979). Reinforcement effects on three measures of children's
interest in math. Behavior Modification, 3, 223-244.
403

APPENDIX B
Formulas for calculating effect size, g
1.
where
XE mean of experimental group
Xc mean of control group
Sp = pooled standard deviation
_ (nE - \)Sļ + (nc - \)Sl

nE + nc — 2
where
Sļ = pooled variance
SE = variance of experimental group
Sc = variance of control group
nE = sample size of experimental group
nc = sample size of control group
b
g= t - for equal ns; n = sample size of each group
g=t I
Λl
1 for unequal ns
E nc
4.
-jff Λl
nEnc
404

APPENDIX C
Characteristics of studies included in the meta-analysis
Reward Expect- Contin- Dep. N N Effect
Author(s) Journal Design Subjects Task type ancy gency measure exp. control size (g)ab
Deci (1971) JPSP B/A Adults Soma T E Cont, TC Free time 12 12 -0.54
Exp. 1
Deci (1971) JPSP B/A Adults Soma T E Cont, TC Attitude 12 12 0.00a
Exp. 1
Deci (1971) JPSP Field study Adults Writing T E Cont. TC Performance 4 2 -3.72
Exp. 2 headlines
Deci (1971) JPSP B/A Adults Soma V U Free time 12 12 +0.82
Exp. 3
Deci (1971) JPSP B/A Adults Soma V U Attitude 12 12 0.00a
Exp. 3
Kruglanski et J of Pers. A/O 15-16 yrs Creativity T E Not, TC Attitude 16 16 -0.69
al. (1971) & recall
Kruglanski et J of Pers. A/O 15-16 yrs Creativity T E Not, TC Volunteer 16 16 -0.63
al. (1971) & recall
Deci (1972a) JPSP A/O Adults Soma V U Free time 48 48 +0.29
Deci (1972a) JPSP A/O Adults Soma T E Cont, TC Free time 32 32 +0.75
Deci (1972a) JPSP A/O Adults Soma T E Cont, TC Free time 32 32 -0.10
Deci (1972b) Org Beh & A/O Adults Soma T E Not, NC Free time 24 16 +0.08b
Hum Perf
Kruglanski et J. Exp. Soc A/O Children 5 games T U Attitude 36 33 -0.66
al. (1972) Psych
Lepper et al. JPSP B/A Children Drawing T E Not, TC Free time 18 15 -0.72
(1973)
Lepper et al. JPSP B/A Children Drawing T U Free time 18 15 +0.57b
(1973)
Greene, Child dev A/O Children Drawing T E Not, TC Free time 15 15 -0.70
Lepper(1974)
Greene, Child dev A/O Children Drawing T U Free time 13 15 +0.06
è Lepper(1974)

406
APPENDIX C—continued
Reward Expect Contin Dep. N N Effect
Author(s) Journal Design Subjects Task type ancy gency measure exp. control size (gf
Greene, Child dev A/O Children Drawing T U Free time 13 15 +0.22
Lepper(1974)
Ross (1975) JPSP A/O Children Playing T E Not, TC Free time 20 20 -0.54
Exp. 1 drum
Ross (1975) JPSP A/O Children Playing T E Not, TC Free time 20 20 +0.56
Exp. 1 drum
Ross (1975) JPSP A/O Children Drum T E Not, TC Free time 52 14 -0.81
Exp. 2
Taub, J of Pers A/O Children Coding T E Cont, PC Attitude 124 124 0.003
Dollinger
(1975)
Kruglanski et JPSP A/O 14-15-yr.- 2 tasks T E Cont, PC Attitude 24 24 + 1.15
al. (1975) olds
Exp. 1
Kruglanski et JPSP A/O 15-16-yr.- 2 tasks T E Cont, PC Attitude 40 40 +0.39
al. (1975) olds
Exp. 2
Reiss, JPSP A/O Children Listening T E Not, TC Free time 16 16 -0.84
Sushinski to songs
(1975)
Salanick (1975) Org Beh & A/O Adults Train T E Cont, PC Free time 38 39 -0.12 b
Hum Perf game
Salanick (1975) Org Beh & A/O Adults Train T E Cont, PC Attitude 38 39 -0.01 b
Hum Perf game
Hamner, Foster Org Beh & A/O Adults Scoring T E Not, NC Attitude 31 30 -0.23
(1975) Hum Perf questions
Hamner, Foster Org Beh & A/O Adults Scoring T E Cont, TC Attitude 37 30 +0.19
(1975) Hum Perf questions
Calder, Staw JPSP A/O Adults Puzzles T E Not, TC Attitude 20 20 +0.22b
(1975)

Calder, Staw JPSP A/O Adults Puzzles T E Not, TC Volunteer 20 20 +0.28
(1975)
Feingold, Behavior SS Children Dot-to- T E Cont #of 5 — +0.34
Mahoney Therapy Repeated dot connections
(1975) measures connections
Anderson et al. JPSP B/A Children Drawing V U Free time 18 46 + 1.07
(1976)
Anderson et al. JPSP B/A Children Drawing T E Not, TC Free time 36 46 +0.04
(1976)
Arnold (1976) Org Beh & Multiple Adults Computer T E Not, TC Attitude 17 36 0.003
Hum Perf trials game
Arnold (1976) Org Beh & Multiple Adults Computer T E Not, TC Volunteer 17 36 +0.02
Hum Perf trials game
Ross et al. JPSP A/O Children Drawing T E Not, TC Free time 12 12 -0.64
(1976)
Ross et al. JPSP A/O Children Drawing T E Not, NC Free time 12 12 +0.44
(1976)
Shapira (1976) JPSP A/O Adults Soma T E Cont, PC Attitude 30 30 +0.41bb
Swann, Pittman Child Dev A/O Children Drawings T E Not, NC Free time 20 20 -0.21
(1977) Exp. 1
Swann, Pittman Child Dev A/O Children Drawing T E Not, NC Free time 20 20 -0.78 b
(1977) Exp. 1
Swann, Pittman Child Dev A/O Children Drawing T E Not, TC Free time 39 26 -0.15 b
(1977) Exp. 2
Karniol, Ross Child Dev A/O Children Slide T E Not, TC Free time 17 20 -0.04
(1977) show
Karniol, Ross Child Dev A/O Children Slide T E Cont, PC Free time 20 20 +0.15
(1977) show
Pittmann et al. Per & Soc A/O Adults Gravitation T E Cont, PC Attitude 60 20 -0.20
(1977) Psy Bull
Mynatt et al. Cog Ther & B/A mult. Children Educ T E Not, TC Free time 10 10 + 1.01
(1978) Res trials games
Weiner, Mot & A/O Adults Decoding T E Not, TC Free time 30 30 -0.34
Mander (1978) Emotion cartoons
Weiner, Mot & A/O Adults Decoding T E Cont, PC Free time 30 30 -0.54
Mander (1978) Emotion cartoons
407

Ŝ APPENDIX C—continued
00
]Reward Expect- Contin- Dep. N N Effect
Weiner, Mot & A/0 Adults Decoding T E Not, TC Attitude 30 30 0.003
MandeΓ (1978) Emotion cartoons
Weiner, Mot & A/0 Adults Decoding T E Cont, PC Attitude 30 30 0.003
Weiner, Mot & A/0 Adults Decoding T E Not, TC Performance 30 30 -0.39 b
Weiner, Mot & A/0 Adults Decoding T E Cont, PC Performance 30 30 -0.39*
Orlick, Mosher Int J. of B/A Children Stabilometer T E Cont, TC Free time 14 12 -0.34
(1978) Sport Psy
Orlick, Mosher Int J. of B/A Children Stabilometer T U Free time 12 12 -0.82
(1978) Sport Psy
Orlick, Mosher Int J. of B/A Children Stabilometer V U Free time 11 12 -0.22
(1978) Sport Psy
Smith, Pittman JPSP A/0 Adults Labyrinth T E Cont, TC Attitude 66 33 -0.10 b
(1978)
Smith, Pittman JPSP A/0 Adults Labyrinth T E Cont, TC Performance 66 33 0.003
(1978)
Dollinger, JPSP A/0 Children Mazes T& V E Both Attitude 48 12 0.003
Thelan (1978)
Davidson, Behavior SS Children Playing T E Not # of responses 3 — + 1.83
BucheΓ(1978) Therapy Repeated with
measures clown
Vasta et al. J of School SS Children Coloring T& V U Time 6 — +0.74
(1978) Psych Repeated
measures
AΓkes (1979) Mot & A/0 Adults Soma T E Cont, TC Free time 32 32 -0.16
Emotion
AΓkes (1979) Mot & A/0 Adults Soma T E Cont, TC Attitude 32 32 +0.03
Emotion

Loveland, Child Dev A/O Children Drawing T E Not, TC Free time 12 12 0.003
Olley (1979)
Harackiewicz JPSP B/A 16-yr.-oIds Hidden V U Attitude 31 31 +0.59
(1979) puzzles
Harackiewicz JPSP B/A 16-yr.-olds Hidden T E Not, TC Attitude 31 31 -0.38
(1979) puzzles
McLoyd (1979) Child Dev A/O Children Reading T E Cont, TC Free time 36 18 -0.22
books
McLoyd (1979) Child Dev A/O Children Reading T E Cont, TC Performance 36 18 -0.40
books
Wimperis, Farr J. Applied A/O Adults Erector T E Not, TC Attitude 16 16 +0.56
(1979) Soc Psych sets
Wimperis, Farr J. Applied A/O Adults Erector T E Cont, PC Attitude 16 16 + 1.36
Wimperis, Farr J. Applied A/O Adults Erector T E Both Volunteer 32 16 +0.69
Weinberg, Research A/O Adults Stabilometer T E Cont, PC Attitude 40 40 0.003
Jackson Quarterly
(1979)
McGraw, J Exp Soc A/O Adults Water jar T E Cont, PC Attitude 20 20 -0.04
McCullers Psych problem
(1979)
McGraw, J Exp Soc A/O Adults Water jar T E Cont, PC Volunteer 18 17 -0.43 b
McCullers Psych problem
(1979)
Vasta, Stirpe Behavior ss Children Math T E Not Time 4 — -0.46
(1979) Mod Repeated problems
measures
Brennan, Soc Beh & B/A Adults Soma T E Not, NC Free time 19 39 + 1.06
Glover (1980) Pers
Weiner (1980) J of Soc A/O Adults Anagrams T E Cont, PC Attitude 24 24 0.002
Psych
Weiner (1980) J of Soc A/O Adults Anagrams T E Cont, PC Volunteer 24 24 0.003
Psych
Weiner (1980) J of Soc A/O Adults Anagrams T E Cont, PC Performance 24 24 +0.35
Psych

Rosenfield et JPSP A/O Adults AdLib V E Free time 30 59 +0.48
al. (1980)
Rosenfield et JPSP A/O Adults AdLib V E Attitude 30 59 -0.64
al. (1980)
Rosenfield et JPSP A/O Adults AdLib V E Volunteer 30 59 -0.76
al. (1980)
Rosenfield et JPSP A/O Adults AdLib T E Cont, PC Free time 30 27 +0.65
al. (1980)
Rosenfield et JPSP A/O Adults AdLib T E Cont, PC Attitude 30 27 + 2.80
al. (1980)
Rosenfield et JPSP A/O Adults AdLib T E Cont, PC Volunteer 30 27 +0.27
al. (1980)
Staw et al. J of PeΓs A/O Adults Puzzles T E Not, TC Attitude 47 46 +0.19
(1980)
Staw et al. J of PeΓs A/O Adults Puzzles T E Volunteer 47 46 +0.34
(1980)
Williams JPSP B/A Children 4 games T E Not, TC Free time 24 24 -0.32
(1980)
Williams JPSP B/A Children 4 games T E Not, TC Attitude 24 24 O.OO3
(1980)
Daniel, EsseΓ J Applied A/O Adults Puzzles T E Cont, TC Free time 32 32 -0.52
(1980) Psych
Daniel, EsseΓ J Applied A/O Adults Puzzles T E Cont, TC Attitude 32 32 -0.19 b
(1980) Psych
Daniel, EsseΓ J Applied A/O Adults Puzzles T E Cont, TC Volunteer 32 32 +0.08
(1980) Psych
Morgan (1981) JPSP A/O Children Puzzles T E Not, TC Free time 27 27 -0.98
Exp. 1
Morgan (1981) JPSP A/O Children Puzzles T E Not, TC Attitude 27 27 -0.31
Exp. 1

Morgan (1981) JPSP A/O Children Puzzles T E Not, TC Free time 20 20 -0.77
Exp. 2
Morgan (1981) JPSP A/O Children Puzzles T E Not, TC Attitude 20 20 +0.04
Exp. 2
Brockner, J of Res in A/O Adults Soma T E Cont, TC Free time 26 26 -0.37
Vasta (1981) Pers
Brockner, J of Res in A/O Adults Soma T E Cont, TC Attitude 25 26 -0.58
Vasta (1981) Pers
Pittman et al. Pers & Soc A/O Adults Soma V U Free time 24 12 +0.80
(1980) Psych Bull
Shanab et al. J of Soc A/O Adults Soma V U Free time 20 20 +0.64
(1981) Psych
Shanab et al. J of Soc A/O Adults Soma V U Attitude 20 20 +0.43
(1981) Psych
Danner, Child Dev A/O Children Class T E Not, TC Free time 30 30 -1.33
Lonkey (1981) inclusion
Danner, Child Dev A/O Children Class T E Not, TC Attitude 30 30 -1.23
Danner, Child Dev A/O Children Class V U Free time 30 30 -0.10
Danner, Child Dev A/O Children Class V U Attitude 30 30 -0.08
Boal, OrgBeh<fe Field study Adults Coding T E Not, NC Free time 21 22 +1.64
Cummings Hum Perf data
(1981)
Boal, Org Beh & Field study Adults Coding T E Cont, TC Free time 21 22 +0.38
Cummings Hum Perf data
(1981)
Luyten, Lens Mot & A/O Adults Wood T E Not, TC Free time 10 10 -0.96
(1981) Emotion models
Luyten, Lens Mot & A/O Adults Wood T E Not, TC Attitude 10 10 -0.88
Luyten, Lens Mot & A/O Adults Wood T E Not, TC Volunteer 10 10 -1.15
Luyten, Lens Mot & A/O Adults Wood T E Cont, PC Free time 10 10 -0.91
411

Luyten, Lens Mot & A/O Adults Wood T E Cont, PC Attitude 10 10 4-0.08
Luyten, Lens Mot & A/O Adults Wood T E Cont, PC Volunteer 10 10 4-1.08
Fabes et al. Am. J Psych A/O Adults Algorithms T E All Performance 57 19 -0.53
(1981) heuristic
tasks
Boggiano et al. Social A/O Children Hidden T E Not, TC Free time 81 84 4-0.28
(1982) Cognition pictures
ZinseΓ et al. J General A/O Children Hidden V U Free time 64 32 4-0.08
(1982) Psych pictures
Porac, Meindl Org Beh & A/O Adults Soma T E Not, TC Free time 40 20 -0.21
(1982) Hum Perf
Earn (1982) J of PeΓS A/O Adults Anagrams T E Not, TC Free time 40 20 -0.28
Earn (1982) J of PeΓS A/O Adults Anagrams T E Not, TC Attitude 40 20 4-0.18
Pittman et al. JPSP A/O Children Matching T E Not, NC Free time 10 10 4-0.37
(1982) Exp. 1 games
Pittman et al. JPSP A/O Children Matching T E Not, TC Free time 10 10 4-0.25
(1982) Exp. 1 games
Pittman et al. JPSP A/O Children Matching T E Not, TC Attitude 20 10 0.00a
(1982) Exp. 1 games
Pittman et al. JPSP A/O Children Drawing T E Not, TC Free time 28 28 -0.05
(1982) Exp. 2
Pallack et al. Child Dev A/O Children Drawing V U Free time 14 12 -0.48
(1982)
Pallack et al. Child Dev A/O Children Drawing V E Free time 14 12 4-0.32
(1982)
Pallack et al. Child Dev A/O Children Drawing T U Free time 15 12 -0.44
(1982)

Pallacketal. Child Dev A/O Children Drawing T E Free time 15 12 -0.16
(1982)
Crino, White J A/O Adults Puzzles V U Attitude 20 10 +0.01
(1982) Management
Crino, White J A/O Adults Puzzles V U Volunteer 20 10 +0.49
(1982) Management
Crino, White J A/O Adults Puzzles V U Attitude 20 10 +0.07
(1982) Management
Crino, White J A/O Adults Puzzles V U Volunteer 20 10 +0.64
(1982) Management
Ogilvie, Prior Aust & N.Z. B/A Children Drawing T E Not, TC Free time 26 26 -0.08
(1982) J Dev. Dis.
Boggiano, Social A/O Adults Memory T U Attitude 46 46 +.0.02
Hertel (1983) Cognition task
Ryan et al. JPSP A/O Adults Hidden T E Cont, PC Free time 32 32 -0.46
(1983) puzzles
Ryan et al. JPSP A/O Adults Hidden T E Cont, PC Attitude 32 32 0.003
(1983) puzzles
Ryan et al. JPSP A/O Adults Hidden T E Not, TC Free time 16 16 -0.35
(1983) puzzles
Ryan et al. JPSP A/O Adults Hidden T E Not, TC Attitude 16 16 0.003
(1983) puzzles
Ryan et al. JPSP A/O Adults Hidden V E Free time 64 32 +0.47
(1983) puzzles
Ryanetal. JPSP A/O Adults Hidden V E Attitude 64 32 0.003
(1983) puzzles
Morgan (1983) Child Dev A/O Children Puzzles T E Not, TC Free time 40 40 -1.94
Exp. 1
Morgan (1983) Child Dev A/O Children Puzzles T E Not, TC Attitude 40 20 -0.27 b
Exp. 1
Morgan (1983) Child Dev A/O Children Puzzles T E Not, TC Free time 40 40 -0.59
Exp. 2
Morgan (1983) Child Dev A/O Children Puzzles T E Not, TC Attitude 20 20 0.00
Exp. 2
Vallerand J Sport Psych A/O Children Slideshow V E Attitude 40 10 +1.98
413
(1983) game

Author(s) Journal Design Subjects Task type ancy gency measure exp. control size (g)00
DeLoach et al. Bull Psych B/A Children Connect T E Not, TC Free time 26 26 0.00*
(1983) Society dots
Blanck et al. Sex Roles A/O Adults Word V U Free time 70 69 +0.56
(1984) game
Blanck et al. Sex Roles A/O Adults Word V U Attitude 70 69 +0.46
(1984) game
Sarafino (1984) Br. J Dev A/O Children Riddles T E Not, TC Free time 85 15 -0.41
Psych
Sarafino (1984) Br. J Dev A/O Children Riddles T Not, TC Attitude 85 15 0.003
Psych
Harackiewicz J Exp. Psych A/O 16-yr.-olds Hidden T E Cont, PC Attitude 47 47 +0.33
et al. (1984) puzzles
Griffith et al. Bull Psych A/O Children Reading T E Not, TC Free time 64 32 0.00a
(1984) Society books
Griffith et al. Bull Psych A/O Children Reading T E Not, TC Performance 64 32 0.003
(1984) Society books
Pretty, JPSP B/A Adults Soma T E Not, TC Free time 30 30 -0.75
Seligman
(1984) Exp. 1
Pretty, JPSP B/A Adults Soma T E Not, TC Attitude 30 30 -0.05
Seligman
(1984) Exp. 1
Pretty, JPSP B/A Adults Soma T U Free time 30 30 +0.06
Seligman
(1984) Exp. 1
Pretty, JPSP B/A Adults Soma T U Attitude 30 30 +0.42
Seligman
(1984) Exp. 1

Pretty, JPSP B/A Adults Soma V U Free time 30 30 +0.35
Seligman
(1984) Exp. 1
Pretty, JPSP B/A Adults Soma V U Attitude 30 30 +0.46
Seligman
(1984) Exp. 1
Pretty, JPSP B/A Adults Soma T E Not, TC Free time 30 30 -0.13
Seligman
(1984) Exp. 2
Pretty, JPSP B/A Adults Soma T E Not, TC Attitude 30 30 -0.16
Seligman
(1984) Exp. 2
Pretty, JPSP B/A Adults Soma T U Free time 30 30 +0.06
Seligman
(1984) Exp. 2
Pretty, JPSP B/A Adults Soma T U Attitude 30 30 +0.38
Seligman
(1984) Exp. 2
Harackiewicz JPSP B/A Adults Pinball T E Cont, PC Free time 32 32 +0.07
et al. (1984)
Exp. 1
Harackiewicz JPSP B/A Adults Pinball T E Cont, PC Attitude 32 32 +0.03
et al. (1984)
Exp. 1
Harackiewicz JPSP B/A Adults Pinball T E Cont, PC Performance 32 32 +0.16
et al. (1984)
Exp. 1
et al. (1984)
Exp. 2
Harackiewicz JPSP B/A Adults Pinball T E Cont, PC Performance 15 15 -0.43
et al. (1984)
415

£r APPENDIX C—-continued
Harackiewicz JPSP B/A Adults Pinball T U Attitude 15 15 +0.15
et al. (1984)
Exp. 2
Harackiewicz JPSP B/A Adults Pinball T U Performance 15 15 +0.44
et al. (1984)
Exp. 2
et al. (1984)
Exp. 3
Harackiewicz JPSP B/A Adults Pinball T E Cont, PC Performance 26 26 +0.04
et al. (1984)
Exp. 3
Vallerand, J Sport Psych B/A Adults Stabilometer V E Attitude 28 28 +0.53 b
Reid (1984)
Arnold (1985) Acad. Man. B/A Adults Computer T E Both Attitude 26 16 -0.04
J. game
Boggiano et al. Social A/O Children Puzzles T E Not, TC Free time 26 13 -0.79
(1985) Cognition
Boggiano et al. Social A/O Children Puzzles T E Cont, PC Free time 26 13 -0.10
(1985) Cognition
Freedman, Org Beh & A/O Adults Proof T E Not, TC Attitude 52 47 +0.75
Phillips (1985) Hum Dec P reading
Freedman, Org Beh & A/O Adults Proof T E Cont, PC Attitude 47 47 +0.68
Phillips (1985) Hum Dec P reading
Tripathi, Psych A/O Adults Puzzles T E Not, TC Free time 20 20 +0.41
Agarwal Studies
(1985)
Tripathi, Psych A/O Adults Puzzles T E Not, TC Attitude 20 20 +0.54
Agarwal Studies
(1985)

Tripathi, Psych A/O Adults Puzzles T E Not, TC Performance 20 20 +0.54
Agarwal Studies
(1985)
Tripathi, Psych A/O Adults Puzzles V E Free time 20 20 +1.61
Agarwal Studies
(1985)
Tripathi, Psych A/O Adults Puzzles V E Attitude 20 20 +0.48
Agarwal Studies
(1985)
Tripathi, Psych A/O Adults Puzzles V E Performance 20 20 +0.54
Agarwal Studies
(1985)
Sansone (1986) JPSP A/O Adults Identify V U Attitude 44 11 +0.68
Exp. 1 names
Amabile et al. JPSP A/O Children 3 tasks T E Not, TC Free time 56 57 0.003
(1986) Exp.l
Amabile et al. JPSP A/O Children 3 tasks T E Not, TC Attitude 56 57 0.003
(1986) Exp. 1
Amabile et al. JPSP A/O Adults 3 tasks T E Not, TC Attitude 30 30 0.003
(1986) Exp.3
Harackiewicz JPSP A/O 16-yr.-olds Puzzles T E Cont, PC Attitude 24 27 -0.10
et al. (1987)
Horn (1987) Pers & Soc A/O Adults Pursuit T ? Not Free time 26 26 +0.1 lb
Exp. 1 Psych Bull rotor task
Horn (1987) Pers & Soc A/O Adults Pursuit T ? Not Attitude 26 26 0.00a
Exp. 1 Psych Bull rotor task
Horn (1987) Pers & Soc A/O Adults Solving V ? ? Performance 28 28 -0.37
Exp. 2 Psych Bull anagrams
Fabes (1987) J of Psych A/O Children Block T E Not, TC Free time 18 19 -0.82
Exp. 1 building
Fabes (1987) J of Psych A/O Children Block T E Cont, PC Free time 19 19 -0.87
Exp. 1 building
Fabes (1987) J of Psych A/O Children Block T E Not, TC Free time 14 14 -0.45
Exp. 2 building
417

418 APPENDIX C—continued
Koestner et al. JPSP A/O Adults Hidden V U Free time 35 18 +0.51
(1987) puzzles
Koestner et al. JPSP A/O Adults Hidden V U Attitude 35 18 0.003
(1987) puzzles
Butler (1987) J Ed Psych A/O Children Problem V U Attitude 50 50 + 1.59
solving
Butler (1987) J Ed Psych A/O Children Problem V U Performance 50 50 +0.39
solving
Tripathi, J Gen Psych A/O Adults Problem T E Not, TC Free time 20 10 +0.03
Agarwal solving
(1988)
Tripathi, J Gen Psych A/O Adults Problem T E Cont, PC Free time 20 10 + 1.18
Agarwal solving
(1988)
Tripathi, J Gen Psych A/O Adults Problem T E Both Attitude 40 10 +0.26b
Agarwal solving
(1988)
Fabes et al. Mot& A/O Children Beanbag T E Not, TC Free time 14 14 -1.34
(1988) Emotion game
Sansone (1989) J Exp Soc A/O Adults Identify V U Attitude 82 41 +0.46
Psych names
Sansone et al. JPSP A/O Adults Computer V U Attitude 40 40 +0.12
(1989) games
Anderson, J App Soc A/O Adults Brain V U Attitude 10 10 +0.90
Rodin (1989) Psych teasers
Mawhinney et J Org Beh SS Adults Video T E Not Time 3 — +0.15
al. (1989) Management Repeated game
measures
Wicker et al. J of Psych A/O Adults Think T E Not, TC Free time 29 29 0.003
(1990) TacTœ

Wicker et al. J of Psych A/O Adults Think T E Not, TC Attitude 29 29 0.00*
(1990) Tac Toe
Notes.
Design: B/A = before-after groups design, A/O = after-only groups design, SS = single-subject design
Reward type: T = tangible, V = verbal
Reward expectancy: E = expected, U = unexpected
Reward contingency: cont = contingent, not = not contingent; NC = nontask contingent, TC = task contingent, PC = performance contingent
indicates effect sizes given a value of zero (nonsignificant results with no report of means or direction of means)
indicates estimated effect sizes
JPSP = Journal of Personality and Social Psychology
J of Pers = Journal of Personality
Org Beh & Hum Perf = Organizational Behavior and Human Performance
J Exp Soc Psych = Journal of Experimental Social Psychology
Child Dev = Child Development
Per & Soc Psy Bull = Personality and Social Psychology Bulletin
Cog Ther & Res = Cognitive Therapy and Research
Mot & Emotion = Motivation and Emotion
Int J of Sport Psy = International Journal of Sport Psychology
J of School Psych = Journal of School Psychology
J Applied Soc Psych = Journal of Applied Social Psychology
Behavior Mod = Behavior Modification
Soc Beh & Pers = Social Behavior and Personality
J of Soc Psych = Journal of Social Psychology
J Applied Psych = Journal of Applied Psychology
J of Res Pers = Journal of Research in Personality
J General Psych = Journal of General Psychology
J Management = Journal of Management
Aust & N.Z. J Dev Dis = Australia and New Zealand Journal of Developmental Disabilities
J Sport Psych = Journal of Sport Psychology
Bull Psych Society = Bulletin of the Psychonomic Society
Br J Dev Psych = British Journal of Developmental Psychology
J Exp Psych = Journal of Experimental Psychology
Acad Man J = Academy of Management Journal
Org Beh & Hum Dec P = Organizational Behavior and Human Decision Processes
419
Psych Studies = Psychological Studies

J Org Beh Management = Journal of Organizational Behavior Management

References
Amabile, T. M., Hennessey, B. A., & Grossman, B. S. (1986). Social influences on
creativity: The effects of contracted-for reward. Journal of Personality and Social
Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive
theory. Englewood Cliffs, NJ: Prentice-Hall.
Bates, J. A. (1979). Extrinsic reward and intrinsic motivation: A review with implica
tions for the classroom. Review of Educational Research, 49, 557-576.
Bern, D. J. (1972). Self-perception theory. In L. Berkowitz (Ed.), Advances in Experi-
mental Social Psychology (Vol. 6, pp. 1-62). New York: Academic.
Boggiano, A. K., & Ruble, D. N. (1979). Competence and the overjustifìcation
effect: A developmental study. Journal of Personality and Social Psychology,
37, 1462-1468.
Brennan, T. P., & Glover, J. A. (1980). An examination of the effect of extrinsic
reinforcers on intrinsically motivated behavior: experimental and theoretical. Social
Behavior and Personality, 8, 27-32.
Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects
of different feedback conditions on motivational perceptions, interest, and perfor
mance. Journal of Educational Psychology, 79, 474^4-82.
Cameron, J. (1992). Intrinsic motivation revisited. Unpublished doctoral dissertation,
University of Alberta, Canada.
Cooper, H. M. (1989). Integrating research: A guide for literature reviews (2nd ed.).
Beverly Hills: Sage.
Danner, F. W., & Lonkey, E. (1981). A cognitive developmental approach to the
effects of rewards on intrinsic motivation. Child Development, 52, 1043-1052.
Deci, E. L. (1971). Effects of externally mediated rewards on intrinsic motivation.
Deci, E. L. (1972a). Intrinsic motivation, extrinsic reinforcement, and inequity. Jour-
nal of Personality and Social Psychology, 22, 113-120.
Deci, E. L. (1972b). The effects of contingent and noncontingent rewards and controls
on intrinsic motivation. Organizational Behavior and Human Performance, 8,
217-229.
Deci, E. L. (1975). Intrinsic Motivation. New York: Plenum.
Deci, E. L., & Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in
Human Behavior. New York: Plenum.
Deci, E. L., & Ryan, R. M. (1987). The support of autonomy and the control of
behavior. Journal of Personality and Social Psychology, 53, 1024-1037.
DeLoach, L. L., Griffith, K., & LaBarba, R. C. (1983). The relationship of group
context and intelligence to the overjustifìcation effect. Bulletin of the Psychonomic
Society, 21, 291-293.
Dickinson, A. M. (1989). The detrimental effects of extrinsic reinforcement on "intrin
sic motivation.,, The Behavior Analyst, 12, 1-15.
Enzle, M. E., & Ross, J. M. (1978). Increasing and decreasing intrinsic interest with
contingent rewards. Journal of Experimental and Social Psychology, 14, 588-597.
Fabes, R. A. (1987). Effects of reward contexts on young children's task interest.
Journal of Psychology, 121, 5-19.
Fair, J. L. (1976). Task characteristics, reward contingency and intrinsic motivation.
Organizational Behavior and Human Performance, 16, 294-307.
Fazio, R. H. (1981). On the self-perception explanation of the overjustifìcation effect:
the role of salience and initial attitude. Journal of Experimental Social Psychology,
17, 417-426.
420

Cameron and Pierce
Feingold, B. D., & Mahoney, M. J. (1975). Reinforcement effects on intrinsic interest:

Undermining the overjustification hypothesis. Behavior Therapy, 6, 357-377.
Flora, S. R. (1990). Undermining intrinsic interest from the standpoint of a behaviorist.
The Psychological Record, 40, 323-346.
Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-Analysis in Social Research.
Beverly Hills: Sage.
Greene, D., & Lepper, M. R. (1974). Effects of extrinsic rewards on children's
subsequent intrinsic interest. Child Development, 45, 1141-1145.
Greene, D., Sternberg, B., & Lepper, M. R. (1976). Overjustification in a token
economy. Journal of Personality and Social Psychology, 34, 1219-1234.
Guzzo, R. A. (1979). Types of rewards, cognitions and work motivation. Academy
of Management Journal, 22, 75-86.
Harackiewicz, J. K., Manderlink, G., & Sansone, C. (1984). Rewarding pinball
wizardry: effects of evaluation and cue value on intrinsic interest. Journal of
Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and
related estimators. Journal of Educational Statistics, 6, 107-128
Hedges, L. V. (1987). How hard is hard science, how soft is soft science? The
empirical cumulativeness of research. American Psychologist, 42, 4 4 3 ^ 5 5 .
Hedges, L., & Becker, B. J. (1986). Statistical methods in the meta-analysis of
research on gender differences. In J. S. Hyde & M. C. Linn (Eds.), The psychology
of gender: Advances through meta-analysis (pp. 14-50). Baltimore: John Hopkins
University Press.
Hedges, L., & Olkin, I. (1985). Statistical Methods for Meta-Analysis. Orlando: Aca
demic.
Hopkins, B. L., & Mawhinney, T. C. (1992). Pay for performance: History, contro-
versy, and evidence. New York: Haworth.
Karniol, R., & Ross, M. (1977). The effect of performance relevant and performance
irrelevant rewards on children's intrinsic motivation. Child Development, 48,
482-487.
Kelly, H. H. (1967). Attribution theory in social psychology. In D. Levine (Ed.),
Nebraska symposium on motivation (Vol. 15, pp. 192-238). Lincoln: University
of Nebraska Press.
Kohn, A. (1993). Why incentive plans cannot work. Harvard Business Review,
77(5), 54-63.
Lepper, M. R. (1981). Intrinsic and extrinsic motivation in children: Detrimental
effects of superfluous social controls. In W. A. Collins (Ed.), Aspects of the
development of competence: The Minnesota symposia on child psychology (Vol.
14, pp. 155-214). Hillsdale, NJ: Erlbaum.
Lepper, M. R., & Greene, D. (1975). Turning play into work: Effects of adult
surveillance and extrinsic rewards on children's intrinsic motivation. Journal of
Personality and Social Psychology, 31, 479^‡86.
Lepper, M. R., Greene, D., & Nisbett, R. E. (1973). Undermining children's intrinsic
interest with extrinsic reward: A test of the "overjustification" hypothesis. Journal
Levine, F. ML, & Fasnacht, G. (1974). Token rewards may lead to token learning.
American Psychologist, 29, 817-820
Light, R. J., & Pillemer, D. B. (1984). Summing Up: The Science of Reviewing
Research. Cambridge, MA: Harvard University Press.
Mawhinney, T. C. (1990). Decreasing intrinsic "motivation" with extrinsic rewards:
Easier said than done. Journal of Organizational Behavior Management, 11,
175-191.
421

McCullers, J. C. (1978). Issues in learning and motivation. In M. R. Lepper &

D. Greene (Eds.), The Hidden Costs of Reward: New Perspectives on the Psychology
of Human Motivation (pp. 5-18). Hillsdale, NJ: Erlbaum.
McGraw, K. O., & Wong, S. P. (1992). A common language effect size statistic.
Psychological Bulletin, 111, 361-365.
Morgan, M. (1981). The overjustification effect: A developmental test of self-percep
tion interpretations. Journal of Personality and Social Psychology, 40, 809-821.
Morgan, M. (1983). Decrements in intrinsic interest among rewarded and observer
subjects. Child Development, 54, 636-644.
Morgan, M. (1984). Reward-induced decrements and increments in intrinsic motiva
tion. Review of Educational Research, 54, 5-30.
Mynatt, C , Oakley, D., Arkkelin, D., Piccione, A., Margolis, R., & Arkkelin, J. (1978).
An examination of overjustification under conditions of extended observation
and multiple reinforcement: Overjustification or boredom? Cognitive Therapy and
Research, 2, 171-177.
Orlick, T. D., & Mosher, R. (1978). Extrinsic awards and participant motivation in
a sport related task. International Journal of Sport Psychology, 9, 27-39.
Phillips, J. S., & Lord, R. G. (1980). Determinants of intrinsic motivation: locus of
control and competence information as components of Deri's cognitive evaluation
theory. Journal of Applied Psychology, 65, 211-218.
Pinder, C. C. (1976). Additivity versus nonadditivity of intrinsic and extrinsic incen
tives: Implications for work, motivation, performance, and attitudes. Journal of
Applied Psychology, 61, 693-700.
Pittman, T. S., Emery, J., & Boggiano, A. K. (1982). Intrinsic and extrinsic motiva
tional orientations: reward-induced changes in preference for complexity. Journal
Pritchard, R. D., Campbell, K. M., & Campbell, D. J. (1977). Effects of extrinsic
financial rewards on intrinsic motivation. Journal of Applied Psychology, 62, 9-15.
Rigby, C. S., Deci, E. L., Patrick, B. C , & Ryan, R. M. (1992). Beyond the intrinsic-
extrinsic dichotomy: Self-determination in motivation and learning. Motivation
and Emotion, 16, 165-185.
Rosenfield, D., Folger, R., & Adelman, H. F. (1980). When rewards reflect compe
tence: A qualification of the overjustification effect. Journal of Personality and
Rosenthal, R. (1966). Experimenter effects in behavioral research. New York: Apple-
ton-Century-Crofts.
Rummel, A., & Feinberg, R. (1988). Cognitive evaluation theory: A meta-analytic
review of the literature. Social Behavior and Personality, 16, 147-164.
Ryan, R. M., Mims, B., & Koestner, R. (1983). Relation of reward contingency and
interpersonal context to intrinsic motivation: A review and test using cognitive
evaluation theory. Journal of Personality and Social Psychology, 45, 736-750.
Sansone, C , & Morgan, C. (1992). Intrinsic motivation and education: Competence
in context. Motivation and Emotion, 16, 249-270.
Schwartz, B. (1990). The creation and destruction of value. American Psychologist,
45, 7-15.
Schwarzer, R. (1991). Meta: Programs for secondary data analysis, MS-DOS Version
5.0 [Computer program]. Dubuque, IA: Wm. C. Brown.
Scott, W. E., Jr. (1975). The effects of extrinsic rewards on "intrinsic motivation.,,
Organizational Behavior and Human Performance, 25, 311-335.
Sutherland, S. (1993). Impoverished minds. Nature, 364, 767.
Tripathi, K. N., & Agarwal, A. (1985). Effects of verbal and tangible rewards on
intrinsic motivation in males and females. Psychological Studies, 30, 77-84.
422

Cameron and Pierce
Tukey, J. W. (1977). Exploratory data analysis. Reading, MA: Addison-Wesley.
Vallerand, R. J. (1983). The effect of differential amounts of postive verbal feedback
on the intrinsic motivation of male hockey players. Journal of Sport Psychology,
5, 100-107.
Vasta, R., Andrews, D. E., McLaughlin, A. M., Stirpe, L. A., & Comfort, C. (1978).
Reinforcement effects on intrinsic interest: A classroom analog. Journal of School
Vasta, R., & Stirpe, L. A. (1979). Reinforcement effects on three measures of children's
interest in math. Behavior Modification, 3, 223-244.
Wiersma, U. J. (1992). The effects of extrinsic rewards in intrinsic motivation: A meta-
analysis. Journal of Occupational and Organizational Psychology, 65, 101-114.
Workman, E. A., & Williams, R. L. (1980). Effects of extrinsic rewards on intrinsic
motivation in the classroom. Journal of School Psychology, 18, 141-147.
Zimbardo, P. G. (1988) Psychology and life (11th ed.). Glen view, IL: Scott, Foresman.
Authors
JUDY CAMERON is Assistant Professor, Department of Educational Psychology,
University of Alberta, Edmonton, Canada T6G 2G5. She specializes in learning
and motivation, social psychology, education, and second language acquisition.
W DAVID PIERCE is Professor, Centre for Experimental Sociology, 1^8 TORY,
University of Alberta, Edmonton, Canada TG6 2H4. He specializes in social
psychology and behavior analysis.
Received July 22, 1993

Revision received January 28, 1994
Accepted February 7, 1994
423

Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis

Uploaded by

Copyright:

Available Formats

Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis

Uploaded by

Copyright:

Available Formats

Review of Educational Research

Fall 1994, Vol. 64, No. 3, pp. 363-423

Reinforcement, Reward, and Intrinsic Motivation:

Judy Cameron and W. David Pierce

This article reviews research on the effects of reinforcement/reward on

Reinforcement theory has had a significant impact on education. Education

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

In another experiment, Deci (1971; Experiment 3) used the same experimental

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Findings from the Group Design Studies

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

THEORETICAL ACCOUNTS OF THE LITERATURE

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

THE PRESENT META-ANALYSIS

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Meta-Analysis of Effect Sizes

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Performance during free time period

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Interactions: Effect Size as a Function of Reward Characteristics5

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Studies assessing the effects of tangible reward on intrinsic motivation show

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

In summary, subjects rewarded with verbal praise or positive feedback show

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Revaid H£3S£Ĵ_ Revaid Revaid

Task contingent Task noncontingent Performance Task contingent

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

the attitude measure, verbal reward produces an increase in intrinsic motivation,

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

intrinsic motivation. A person's experience of an activity is a focal point of

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016

Downloaded from http://rer.aera.net at PENNSYLVANIA STATE UNIV on March 6, 2016