Journal of Educational Psychology © 2016 American Psychological Association

2017, Vol. 109, No. 5, 666 – 679 0022-0663/17/$12.00 http://dx.doi.org/10.1037/edu0000167

Can Collaborative Learning Improve the Effectiveness of Worked

Examples in Learning Mathematics?
Endah Retnowati, Paul Ayres, and John Sweller
University of New South Wales

Worked examples and collaborative learning have both been shown to facilitate learning. However, the
testing of both strategies almost exclusively has been conducted independently of each other. The main
aim of the current study was to examine interactions between these 2 strategies. Two experiments (N ⫽
182 and N ⫽ 122) were conducted with Grade-7 Indonesian students, comparing learning to solve algebra
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

problems, with higher and lower levels of complexity, collaboratively or individually. Results from both
This document is copyrighted by the American Psychological Association or one of its allied publishers.

experiments indicated that individual learning was superior to collaborative learning when using worked
examples. In contrast, in Experiment 2, when learning from problem solving using problem-solving
search, collaboration was more effective than individual learning. However, again in Experiment 2,
studying worked examples was overall superior to learning from solving problems, particularly for more
complex problems. It can be concluded that while collaboration could be beneficial when learning under
problem solving conditions, it may be counterproductive when studying worked examples.

Keywords: worked examples, cognitive load theory, collaboration, problem complexity

Across multiple domains ranging from mathematics to visual problem-solving search or other types of discovery methods, learn-
arts, researchers have demonstrated that when learning novel ma- ers are shown worked examples to study. Worked examples pro-
terial, guided instruction through worked examples is more effec- vide an expert’s problem-solving model, from which students can
tive for novice learners than conventional problem solving strate- study and learn (Atkinson et al., 2000). With worked examples,
gies (see Atkinson, Derry, Renkl, & Wortham, 2000; P. A. learners are able to focus on understanding a solution rather than
Kirschner, Sweller, & Clark, 2006; Renkl, 2014a, 2014b; Sweller, focus on solving the problem (Renkl, 2014a). The worked example
Ayres, & Kalyuga, 2011). However, most of the research into effect occurs when students who learn from studying worked
worked examples has focused exclusively on individual learning examples subsequently obtain superior test scores to students who
settings. Few attempts have investigated worked examples in col- learn from solving problems. From this perspective, we refer to
laborative settings (e.g., F. Kirschner, Paas, Kirschner, & Janssen, problem solving as solving problems with minimal teacher/instruc-
2011; Retnowati, Ayres, & Sweller, 2010). tor guidance on how to solve the problem.
A main aim of the current study (Experiment 1) was to inves- Using algebra problems, Cooper and Sweller (1987) and Sweller
tigate the effectiveness of collaborative learning compared with and Cooper (1985) provided the first demonstrations of the worked
individual learning within a worked examples environment. An- example effect (Sweller et al., 2011). They found that students who
other aim (Experiment 2) was to compare possible interactions were asked to study worked examples performed better on subsequent
between studying individually or collaboratively on the one hand
problem solving tests than students required to practice solving the
and studying worked examples or solving problems on the other
equivalent problems. The effect was explained by the suggestion that
hand. We will begin by outlining the worked example effect.
worked examples reduced extraneous working memory load com-
pared to solving the equivalent problems. A reduction in cognitive
The Worked Example Effect load facilitated the transfer of knowledge to long-term memory. These
A worked example provides a step-by-step solution to a problem findings led to further research in mathematics and scientific domains.
or task and is a form of explicit instruction (see P. A. Kirschner et For example, the worked example effect was replicated in algebra
al., 2006). Rather than trying to acquire new information through (Carroll, 1994), geometry (Paas & van Merriënboer, 1994; Tarmizi &
Sweller, 1988), statistics (Paas, 1992; Quilici & Mayer, 1996),
and physics (Ward & Sweller, 1990) using a range of age groups and
subject areas with the advantage appearing for both similar and
This article was published Online First December 19, 2016. transfer problems. Building on these initial findings, more contempo-
Endah Retnowati, Paul Ayres, and John Sweller, School of Education, rary research (for summaries, see Ayres & Sweller, 2013; Renkl,
University of New South Wales. 2014a) has found the effect in nonscience domains such as visual arts
Endah Retnowati is now at the Department of Mathematics, Education
(Rourke & Sweller, 2009) and English literature (Kyun, Kalyuga, &
Faculty of Mathematics and Sciences, Yogyakarta State University, Indo-
nesia. Sweller, 2013; Oksa, Kalyuga, & Chandler, 2010), as well as ongoing
Correspondence concerning this article should be addressed to Paul investigations in the science domain, such as problem solving in
Ayres, School of Education, University of New South Wales, Sydney, electrical circuits (van Gog & Kester, 2012), and geometry (Chen,
NSW 2052, Australia. E-mail: [email protected] Kalyuga, & Sweller, 2015, 2016a).

666
 COLLABORATIVE LEARNING AND WORKED EXAMPLES 667

Worked examples illustrate one of the main principles of cog- that collaborative learning requires active social interactions,
nitive load theory, the borrowing and reorganizing principle (see group goals, and individual accountability (see Slavin, 1995).
Sweller & Sweller, 2006). This principle suggests that the most The use of problem solving activities within collaborative learn-
effective way to obtain new information is by directly receiving it ing classrooms has been strongly advocated, especially by math-
from another person who already has this information. The major ematics educators (see, e.g., the National Council of Teachers of
mechanisms are listening to other people, reading what they write, Mathematics, 2000). According to De Corte (2004), one view of
and imitating what they do. In that sense, information is borrowed mathematics learning is that it is a social construction of knowl-
from another person’s long-term memory. In the case of worked edge through collaboration. An emphasis should be placed on
examples, information is borrowed from the long-term memory of problem solving, reasoning, and communication, forming commu-
the constructor of the worked examples. However, this information nities of mathematical inquiry (Goos, 2004; Staples, 2007). Shared
is reorganized by the learner by integrating the new information meanings of the main concepts emerge through the interactions
with old information stored in the learner’s long-term memory (see associated with group problem solving (Plass et al., 2013), as well
Sweller et al., 2011). Integrating new information with old infor- as learners constructing their own ideas and individual insights
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

mation that is already understood may assist in making sense of the

(Yackel, Cobb, & Wood, 1991).
This document is copyrighted by the American Psychological Association or one of its allied publishers.

new information. This conceptualization is consistent with other

theories of learning that emphasize reorganization and elaboration
(see Mayer, 2014). Because worked examples provide a low Collaborative Learning and Evolutionary Psychology
cognitive load environment compared with problem solving
search, learning is enhanced through the construction of new Evolutionary psychology is used as a base for cognitive load
schematic knowledge. theory, and this view of cognition can be used to provide a new
Virtually all published research into worked examples has been perspective on some of the fundamental underpinnings of collab-
conducted using individual rather than collaborative learning. A orative learning (Paas & Sweller, 2012; Sweller et al., 2011). A
notable exception was a study by Retnowati et al. (2010), who key aspect of this argument comes from the work of Geary (1995,
found in a single experiment using Grade 7 Indonesian students 2008, 2012), who distinguished between two types of knowledge:
that worked examples in geometry were superior to problem biologically primary and secondary knowledge. Biologically pri-
solving for both individual and collaborative learners on both mary knowledge is knowledge that we have evolved to acquire
retention and transfer tasks that required calculations as well as over many generations. It is easily and unconsciously acquired and
providing related explanations. Qualitative data also revealed that is modular with different skills likely to have evolved during
participants believed that they understood the material more easily different evolutionary epochs. Examples are learning to listen,
when using worked examples rather than problem solving. A goal speak, recognize faces. and use general problem solving strategies.
of the present study was to investigate whether worked examples Biologically secondary knowledge is knowledge that we need to
could be enhanced by using collaborative settings. acquire for cultural reasons. We have evolved to acquire secondary
knowledge as a general skill. We have not evolved to acquire
Collaborative Learning particular types of secondary knowledge in the same way that we
have evolved to acquire particular types of primary knowledge.
Collaborative learning occurs when students learn by collabo- Virtually every topic taught in education and training establish-
rating rather than by studying individually. It is widely used ments provides an example of biologically secondary knowledge.
(Gillies, 2003) and considered highly desirable in the community Geary argued that working in a collaborative environment may
and workplace (Barron, 2000). Considerable evidence suggests be natural and effortless, because it is a biologically primary
that collaborative learning has significant academic, social, and
activity that humans have evolved to engage in (Geary, 1995,
psychological benefits (Johnson, Johnson, & Smith, 1998). Mul-
2008). However, this advantage may come at a cost (Geary, 1995,
tiple studies and meta-analyses have found that the various forms
2008), as during collaborative learning, students may tend to
of collaborative or cooperative learning strategies where students
automatically develop their general communication and coordina-
work together have significant benefits over students who work
tion skills, rather than allocating more attention to the assigned
individually (see Johnson, Maruyama, Johnson, Nelson, & Skon,
biologically secondary knowledge. While Geary (2008) acknowl-
1981). Many of these studies have focused on learning mathemat-
ics, showing that small-group learning has led to greater mathe- edges that social context and interaction with teachers and peers
matical outcomes than traditional methods of teaching individuals contribute to a student’s learning, he also questions whether stu-
(see Davidson & Kroll, 1991). Explanations for this advantage are dents can learn better in social contexts, rather than through
usually grounded in social constructivist theory or social indepen- explicit instruction.
dence theory, which emphasize that learning should be facilitated As shown in many studies (see Johnson et al., 1998), social
through social and collaborative activities where students construct skills may automatically be improved through collaboration, which
knowledge by interactions with others and through collective goals is consistent with Geary’s argument outlined above. However,
(Johnson & Johnson, 1994; Schreiber & Valle, 2013). learning the content of a collaborative lesson is another matter
Studies have been conducted to identify the factors that improve because that content most likely requires the acquisition of bio-
collaborative learning (for reviews, see Cohen, 1994; Kreijns, logically secondary skills (e.g., mathematics) that require con-
Kirschner, & Jochems, 2003; Schreiber & Valle, 2013; Van den scious effort. As Geary suggested, collaboration may not neces-
Bossche, Gijselaers, Segers, & Kirschner, 2006; Webb, 2009; sarily produce advantages in academic outcomes if no more than
Weinberger, Stegmann, & Fischer, 2007). It is generally agreed an automatic improvement in collaborative skills occurs.
 668 RETNOWATI, AYRES, AND SWELLER

Collaborative Learning and Cognitive Load Theory This study aimed to extend the research into both strategies by
combining them in an authentic learning environment. More spe-
Similar to worked examples (as argued above), collaborative cifically, the main research question was to investigate if the
learning demonstrates another example of the borrowing and re- effectiveness of worked examples could be improved by using
organizing principle (see Paas & Sweller, 2012). Knowledge can collaborative learning. If, as indicated above, the borrowing and
be borrowed from other members of the group, and reorganized, reorganizing principle suggests that most learning is based on
linking new knowledge with old knowledge stored in long-term obtaining information from others, then the use of collaboration
memory. Group interactions can help individuals make sense of permits learners to not only obtain information from explicit
the information and steer the reorganization of the information instruction via worked examples, but also obtain information from
accordingly (see De Corte, 2004; Plass et al., 2013). Because co-learners. By studying worked examples collaboratively, learn-
humans have evolved to communicate, to share, and to obtain ers may obtain additional information from other learners that
information from each other as biologically primary skills, collab- would not be available if learning individually.
orative learning may have an advantage over individual learning in Notwithstanding the possible advantage of adding information
that it involves sharing information and learning from each other,
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

from collaborators to the information obtained from a worked

as occurs in everyday life (Sweller et al., 2011).
This document is copyrighted by the American Psychological Association or one of its allied publishers.

example, the expertise reversal effect suggests that for given levels
Another advantage of collaborative learning is that it may assist of expertise and complexity (Chen, Kalyuga, & Sweller, 2016b),
in learning complex materials. Complex materials are difficult to the provision of additional information can become redundant,
learn because they impose a heavy working memory load (Sweller resulting in an increase rather than a decrease in cognitive load
et al., 2011). However, if the learning material is shared among (Kalyuga, Ayres, Chandler, & Sweller, 2003). Evidence for such
several group members, an individual is required to process less an outcome was obtained by Nihalani, Mayrath, and Robinson
task-relevant information, potentially reducing working memory (2011). They found that for novices, feedback was more effective
load (F. Kirschner, Paas, & Kirschner, 2009a). Working memory than collaboration. For more expert learners, the addition of feed-
resources then can be allocated to learning about important aspects back reduced learning and reduced the advantages of expertise. For
of the materials by processing relevant information communicated these learners, feedback was redundant and redundancy has been
from other group members. Based on this view, collaboration shown repeatedly to interfere with learning due to an increased
should be effective by providing group members with information extraneous cognitive load. Hence, there may be conditions under
that they otherwise would need to search for themselves. This which the combination of collaboration and worked examples may
potential provision of information should reduce extraneous cog- be less advantageous.
nitive load. In this sense, a biologically primary activity, collabo- We also investigated how problem complexity has an impact on
ration, may provide an advantage in acquiring biologically sec- the effectiveness of collaborative learning and worked examples.
ondary knowledge such as mathematics. Combining the limited Furthermore, because of the design of the experiments it was
working memory resources of several individuals should increase possible under the conditions to examine if the collaborative
the resources available to all in a manner that does not occur when context was superior to individual learning, and whether worked
students are engaged in individual learning and have to deal with examples were superior to problem solving. Throughout the study,
all the working memory load themselves. Hence, through collab- authentic classroom environments were used rather than laboratory
oration, individuals may be better able to learn about complex conditions.
materials.
Initial experimental evidence in support of this hypothesis was
found by F. Kirschner, Paas, and Kirschner (2009b) using a high- Study Hypotheses
school biology topic, where an individual learning condition was Hypothesis 1: Worked examples will be enhanced by study-
compared to a collaborative learning condition consisting of three ing collaboratively compared to studying individually. This
group members. During the learning phase, students were given hypothesis was based on research that argues collaborative learn-
problem-solving tasks to complete individually or collaboratively. ing is superior to individual learning (see Johnson & Johnson,
For the collaborative learning condition, every member of a group 2002). In addition considerations of evolutionary psychology sug-
had information about one third of the whole task only, and hence gest that humans have evolved to collaborate naturally (Geary,
sharing was required to complete the task. In the individual con- 1995, 2008) and that collaborative environments provide an effec-
dition, individual students were given the whole task to solve. tive way to obtain new information by directly receiving it from
Following the learning phase, all students were tested individually another person who already has this information (Paas & Sweller,
using retention and transfer tasks. Significant interaction effects 2012; Sweller & S. Sweller, 2006).
were found. For the retention tasks individuals learned more effi- Hypothesis 2. The effectiveness of collaborative learning will
ciently, while for the transfer tasks collaboration led to more be increased by task complexity. This hypothesis flows from
efficient learning. In a follow up study also with biology content, the general research into collaborative learning and problem
F. Kirschner et al. (2011) found that collaborative learning was complexity (see Cohen, 1994). Evidence for the impact of
more effective than individual learning on high but not low com- complex tasks in collaborative learning compared to individual
plexity tasks. learning has been demonstrated by F. Kirschner, Paas, and
Kirschner (2011) using biology content, and by Zhang, Ayres,
The Current Study
and Chan (2011) using web design materials. As reported, F.
The evidence described so far suggests that both worked exam- Kirschner et al. (2011) also showed that effective collaborative
ples and collaborative learning are effective learning strategies. learning requires a high intrinsic cognitive load that cannot be
 COLLABORATIVE LEARNING AND WORKED EXAMPLES 669

tackled easily by individuals. Students who learned in groups creating the two grouping treatments. Each group was assigned at
gained a benefit by sharing the high working memory load random to either stay as a group or become uncoupled to study
created by the complex tasks with other group members. individually. This process produced 79 individual learners and 27
Hypothesis 3: Studying worked examples would be more collaborative groups (22 groups of 4, 5 groups of 3, n ⫽ 103). Both
advantageous than conventional problem solving. This hy- groups and individual learners were then randomly assigned to a
pothesis flows from cognitive load theory and the worked example specific task sequence of either low– high or high–low complexity.
effect. It was tested in Experiment 2. Due to absenteeism 168 students (88 girls, 80 boys) actually
participated, with an average age of 12.6 years (SD ⫽ 0.46). In the
low– high complexity sequence, 38 students completed the task
Experiment 1
individually and 45 students completed the task collaboratively (9
This experiment tested the hypotheses by investigating the in- groups of 4, 3 groups of 3). In the high–low complexity sequence,
fluence of problem complexity on individual and collaborative there were 33 students in an individual and 52 students in a
learning using a worked example strategy. Two types of algebraic collaborative context (10 groups of 4, 4 groups of 3).
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

problems were created with low or high levels of complexity. Materials. Two types of algebra problems were created based
This document is copyrighted by the American Psychological Association or one of its allied publishers.

Problem complexity was categorized by the number of steps on solving linear equations with differing levels of complexity.
required to complete the solution, and the level of conceptual Both task types required students to solve a linear equation. The
knowledge required. The topic, solving linear equations, was se- low complexity problem was presented in algebraic notation, but
lected from the National Curriculum of Indonesia as the experi- the high complexity problem required an equation to be derived, as
ment was conducted in Indonesian schools. it was presented as a word problem. The requirement to translate
With problems of differing complexity it was feasible that the the words into equations increased complexity.
order in which they were presented could influence subsequent An example of a low-complexity problem is “Solve 3n ⫹ 10 ⫽
learning. Hence, the problem sequence was counterbalanced in this 85, for n.” An example of an equivalent high-complexity problem
experiment to avoid sequential learning effects. All participants is “Three times the number of Dina’s marbles when added to 10
received a set of worked examples to test Hypotheses 1 and 2. A equals eighty-five. How many marbles does Dina have?” The
2 (learner grouping context: Collaborative vs. Individual) ⫻ 2 high-complexity problem required more solution steps, as not only
(level of complexity: Low vs. High) ⫻ 2 (task sequence: Low– does the equation have to be constructed, a conceptually demand-
High complexity vs. High–Low complexity) mixed experimental ing task, but it also has to be solved. Consequently, this word
design was used with level of complexity the repeated measure. problem was considered higher in element interactivity (Sweller,
2010; Sweller & Chandler, 1994), because several variables have
to be considered simultaneously to construct the equation, al-
Method
though the given problem context may describe operators (sym-
Participants. One hundred eighty-two students from six Year bols) in the constructed equation more meaningfully. In contrast,
7 mathematics classes in an Indonesian school in Magetan, East the low complexity problem does not have this additional task;
Java, participated in the study. The school followed the national hence, the algebra rules can be applied in a straightforward fash-
curriculum, and the topics used in the experiment were mandated ion. The students in this study had some previous experience with
by the curriculum. The Indonesian national curriculum requests linear equation solving and word problems, but mostly with fewer
teachers not to use teacher-centered learning methods such as variables, and not with a combination of constructing and solving
lectures but to use student-centered learning methods such as small equations. For each problem type, instructional and testing mate-
group discussions (BNSP, 2006; Depdiknas, 2004; National Min- rials were constructed.
istry of Education, 2006). The participating school indicated that An instructional materials booklet was designed using a worked
the students were used to studying in small groups in all subjects example approach. The worked example material used problem
with varied methods of instruction. The school also indicated that pairs, consisting of a worked example and a similar problem to be
they had allocated students to the six mathematics classes ran- solved (see Sweller & Cooper, 1985; Trafton & Reiser, 1993). The
domly at the beginning of the school year. A team of three worked example provided a problem statement and a step-by-step
mathematics teachers taught specific topics to all six classes, solution to the problem (i.e., algorithm, explanation, final answer)
indicating that all students received mathematics instruction from and was written on the left side of the page. The paired problem to
each teacher on set topic blocks throughout the school year. be solved was positioned on the right side of the page and con-
At the beginning of the school year students were assigned to sisted of the problem statement only. Final answers for these
small learning groups by the mathematics teachers based on hav- problems, but not step-by-step solutions, were provided on the
ing the same gender, and of mixed ability (heterogeneous group- same page of the booklet to allow students to know whether they
ings). Grouping students together according to gender was part of had correctly solved the problem, providing some support consis-
the school’s policy for students this age, as it was assumed that tent with previous research (see Cooper & Sweller, 1987). The
boys and girls interact minimally and form single-sex friendships. relevant instruction was provided directly above each problem. All
As friendship groupings can have positive effects on collaboration instructions were in the students’ native Indonesian. Appendixes A
(see Hanham & McCormick, 2009), it was thus assumed that the and B show examples, translated into English, of the format of the
group members had developed some level of cohesiveness and low-complexity and the high-complexity worked examples respec-
familiarity with each other, and could work collaboratively. tively.
These preexisting groups that had been created 3 months earlier The learning material of low-complexity problems consisted of
by the school, independent of this study, formed the basis for four worked example problem pairs. Hence. the worked example
 670 RETNOWATI, AYRES, AND SWELLER

condition required learners to study 4 worked examples and solve shown. The whole preparation period was repeated for each class
4 problems overall, whereas the problem solving condition re- (6 times) by the same researcher.
quired all 8 problems to be solved. The similar and transfer tests In the first stage of the experiment (Stage I), students in the
required 4 and 3 problems to be solved, respectively. The internal Low–High sequence were presented the low-complexity materials
consistency of the similar test using Cronbach’s alpha was .84, and first, whereas those in the High–Low sequence were presented the
.75 for the transfer test. The transfer test problems consisted of high-complexity materials first. This stage consisted of three phas-
modified equations requiring more solution steps than the similar es: acquisition, similar test, and transfer test, which were com-
test problems. The learning material of high-complexity problems pleted without pauses between them.
consisted of 3 worked example problem pairs. Hence, the worked Students in each class were separated into two classrooms
example condition required learners to study 3 worked examples according to their grouping classifications to begin the acquisition
and solve 3 problems overall, whereas the problem solving con- phase, with each classroom supervised by both a teacher from the
dition required all 6 problems to be solved. The similar and school and the researcher. First, each student received a worked
transfer tests consisted of 3 and 2 problems, respectively. Cron- example booklet specific to their learning condition. Twenty min
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

bach’s alpha was .86 for the similar test, and .71 for the transfer were allocated for all groups completing low-complexity prob-
This document is copyrighted by the American Psychological Association or one of its allied publishers.

test. The transfer test problems had the additional requirement of lems, and 30 min were allocated for high-complexity problems.
calculating a subgoal before the goal could be calculated. Before learning commenced, the supervising teacher explained the
To measure cognitive load during acquisition, a self-rating scale rules for studying individually or collaboratively, reading from a
of difficulty was used based on the scale developed by Paas (see common script for each strategy.
Paas, 1992; van Gog & Paas, 2008). Furthermore, consistent with For individual study, students were told to put an effort into
recent research, which suggested that multiple recordings produce understanding the learning material individually and were not
the most consistent results (see van Gog, Kirschner, Kester, & permitted to ask any questions of the other students or the teacher
Paas, 2012), every page of the instructional material had a subjec- during learning. For collaborative study, students were told by the
tive rating question, written on the bottom line of the page, that teacher to discuss the learning material together by reading the task
asked, “How easy or difficult was it to study and solve these together, eliciting understanding, helping each other, and making
sure every member understood the learning material. They were
problems? Circle your answer on a scale from 1 ⫽ Extremely easy
not permitted to ask any questions of other group members or the
to 9 ⫽ Extremely difficult.” The cognitive load measures collected
teacher during learning. For both groups it was also explained how
on each page were added and then averaged to describe the overall
students should complete the cognitive load measures that would
student’s cognitive load experience in this phase.
appear on each page of their booklet. No feedback was provided
Procedure. Before the experimental stage started, all students
during or after the acquisition phase.
underwent a preparation period. This initial session was conducted
Directly following the acquisition phase, the similar and transfer
by one of the researchers who was a native Indonesian mathemat-
tests were completed individually. All students were given the
ics teacher. First, students practiced translating a word problem
maximum time period and did not receive any feedback. Fifteen
containing one operator into an equation, based on the statement
min and 20 min were given to complete the low-complexity
“Bob has 3 more marbles than Wina.” The purpose was to activate
similar test and transfer tests, respectively. To complete the high-
students’ prior knowledge about translating a simple sentence complexity similar and transfer tests, 20 min were given for each
containing a variable into an equation along with the basic algebra test. After the transfer test, students were given a 15-min break.
rules. The researcher used explicit instruction to explain how to Stage II was completed directly after the break, and students
solve this problem. switched to the alternate complexity level materials. If students
Second, to familiarize students with instruction using worked had initially completed the low complexity problems, they then
examples, four pairs of worked examples, using the same format as completed those with high complexity next, and vice versa. Allo-
in the main experiment, were provided. Each pair consisted of an cated times depended on the material and activity as described in
example to study followed by a similar problem to solve and dealt Stage I.
with translating a simple sentence into an equation. This practice During the acquisition phase, group answers were allowed for
lasted 15 min and then the results of the constructed formula were some groups, and therefore these data were not analyzed, as
discussed with the teacher. Immediately afterward, three worked- individual responses were not available for all participants. Scor-
example pairs for solving simple linear equations by applying one ing for the similar and transfer tests used the following guidelines:
algebra rule were given (e.g., solve a ⫹ 20 ⫽ 65). The results of For a low-complexity problem, each successful answer had to
this 15-min practice were then also discussed with the teacher. complete two steps showing two algebraic manipulations. If the
This discussion was based on student questions with the researcher answer was entirely correct, a score of 2 was given. If only one
responding to the questions without further elaboration. step (one strategy) was correctly applied, a score of 1 was given.
To complete the preparation period, the teacher then provided an If the answer did not show any algorithmic validity, a score of 0
example of a word problem (complex problem), similar to the first was given. For a high-complexity problem, each correct answer
problem of the high complexity problem in the learning material. had to include three steps. The first was creating the linear equa-
This problem was written on the blackboard. and students were tion, while the second and third steps were solving the equation. If
shown how to translate the word problem into a linear equation. the answer was entirely correct, a score of 3 was given. If the
The teacher explained that two or more steps were required to equation was correctly created (the first step correct) but only
transform the linear equation in such a way that it could be solved. partially solved (1 correct step), a score of 2 was given. If the
However, the step-by-step solution and the final answer were not equation was correctly created (the first step correct) but incor-
 COLLABORATIVE LEARNING AND WORKED EXAMPLES 671

rectly solved (0 correct steps), a score of 1 was given. If the for the low-complexity transfer test (M ⫽ 0.44, SD ⫽ 0.37) were
equation was incorrectly created but was solved correctly, a score significantly greater than those for the high-complexity transfer
of 1 was given. If the answer did not demonstrate any correct steps, test (M ⫽ 0.34, SD ⫽ 0.35). A learner grouping effect was also
a score of 0 was given. To enable a comparison to be made found, F(1, 164) ⫽ 7.54, MSE ⫽ 0.19, p ⫽ .007, ␩2p ⫽ .044.
between the two types of problems, the total scores for each Learning individually (M ⫽ 0.46, SD ⫽ 0.38) resulted in better
measure were converted into a proportion. transfer results than learning collaboratively (M ⫽ 0.32, SD ⫽
0.34). A significant interaction between problem complexity and
Results and Discussion learner grouping context was also found, F(1, 164) ⫽ 19.51,
MSE ⫽ 0.07, p ⬍ .001, ␩2p ⫽ .106 (see Figure 1). Simple effects
A 2 (Collaborative vs. Individual) ⫻ 2 (Low–High vs. High–
tests indicated that learning individually resulted in better perfor-
Low complexity sequence) ⫻ 2 (Low- vs. High-complexity)
mance in high-complexity tasks than collaborative learning, F(1,
ANOVA with repeated measures on the last variable was used to
166) ⫽ 20.59, MSE ⫽ 0.13, p ⬍ .001, ␩2p ⫽ .11. However, no
analyze the data. The means (and standard deviations) of test
difference was found for low-complexity tasks (F ⬍ 1, ns.).
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

performance and cognitive load ratings are summarized in Table 1.

A nonsignificant difference between task sequences was found,
This document is copyrighted by the American Psychological Association or one of its allied publishers.

Cognitive load during acquisition. A significant complexity

F(1, 164) ⫽ 3.57, MSE ⫽ 0.19, p ⫽ .06, ␩2p ⫽ .021, although the
effect was found, F(1, 164) ⫽ 41.80, MSE ⫽ 1.40, p ⬍ .001, ␩2p ⫽
High–Low sequence generating higher scores (M ⫽ 0.41, SD ⫽
.203. The low-complexity materials in the acquisition phase were
0.31) than the Low-High sequence (M ⫽ 0.34, SD ⫽ 0.32). A
rated significantly easier (M ⫽ 3.05, SD ⫽ 1.63) than the high-
significant interaction effect between the learner grouping context
complexity materials (M ⫽ 3.89, SD ⫽ 1.89). However, no sig-
and task sequence was found, F(1, 164) ⫽ 3.89, MSE ⫽ 0.19, p ⫽
nificant main effect was found for learner grouping or task se-
.05, partial ␩2 ⫽ 0.02. The simple effects test results indicated that
quence (for both, F ⬍ 1, ns.). Nor was there a significant
interaction between learner grouping and task sequence, nor be- individual learning resulted in a better performance than group
tween learner grouping and problem complexity (for both, F ⬍ 1, learning, when the learning sequence was High–Low, F(1, 83) ⫽
ns.). 12.35, MSE ⫽ 0.17, p ⫽ .001, ␩2p ⫽ .13. When the task sequence
A significant interaction effect between problem complexity and was Low–High, no significant differences were found, (F ⬍ 1,
task sequence was found, F(1, 164) ⫽ 18.11, MSE ⫽ 1.40, p ⬍ ns.). Moreover, a significant interaction between the task complex-
.001, ␩2p ⫽ .099. Simple effect tests indicated that students reported ity and task sequence was found, F(1, 164) ⫽ 4.15, MSE ⫽ 0.07,
a higher increase in cognitive load for high-complexity problems p ⫽ .043, ␩2p ⫽ .025. The simple effects test results indicated a
compared to low-complexity problems when the task sequence significant difference of low and high complexity transfer perfor-
was Low–High, F(1, 82) ⫽ 61.80, MSE ⫽ 1.38, p ⬍ .001, ␩2p ⫽ mance when the learning sequence was Low–High, F(1, 82) ⫽
0.43, compared to when the sequence was High–Low, F(1, 84) ⫽ 16.20, MSE ⫽ 0.08, p ⫽ .001, ␩2p ⫽ .17. When the task sequence
1.958, MSE ⫽ 1.45, p ⫽ .165, ␩2p ⫽ .023. As inspection of the was High–Low, no significant differences were found, F(1, 84) ⫽
means indicates, the higher increase in cognitive load for the 3.48, MSE ⫽ 0.07, p ⫽ .07, ␩2p ⫽ .04).
higher-complexity problems under a Low-High sequence is pri- An important aim of this experiment was to create two types of
marily due to the relatively low load imposed by low complexity tasks based on simultaneous equations that had two levels of
problems when they are presented first. complexity. The results indicated that this aim was supported as
Similar test results. There was a main effect for task com- students scored significantly higher on the low-complexity prob-
plexity, F(1, 164) ⫽ 50.76, MSE ⫽ 0.06, p ⬍ .001 ␩2p ⫽ .236. lems compared to high-complexity problems on both tests. Addi-
Students scored significantly higher on the low-complexity prob- tionally, students also experienced a considerably lower cognitive
lems (M ⫽ 0.63, SD ⫽ 0.34) than the high-complexity problems load when learning using low-complexity problems compared to
(M ⫽ 0.44, SD ⫽ 0.35). There was no significant main effect for high-complexity problems. Therefore, it is likely that the higher-
learner grouping context (F ⬍ 1, ns.) or task sequence (F ⬍ 1, ns.). complexity task, with more steps for the solution, has a higher
There was no interaction between learner grouping and complex- level of element interactivity.
ity, F(1, 164) ⫽ 2.18, MSE ⫽ 0.06, p ⫽ .142, ␩2p ⫽ .013, and all The first hypothesis of this experiment (Hypothesis 1) predicted
other interaction measures were non-significant (all F ⬍ 1, ns.). that students would benefit from studying worked examples col-
Transfer test results. A main effect of complexity was found, laboratively rather than individually. No evidence was found dur-
F(1, 164) ⫽ 13.58, MSE ⫽ 0.07, p ⬍ .001, ␩2p ⫽ .076. The scores ing testing to support this prediction. In contrast, a number of

Table 1
Means (and Standard Deviations) for Test Results and Cognitive Load Ratings in Experiment 1

Cognitive load (1–9) Similar test (0 –1) Transfer test (0 –1)

Condition Low-complexity High-complexity Low-complexity High-complexity Low-complexity High-complexity

Low–High sequence
Collaborative 2.71 (1.44) 4.49 (2.26) .64 (.35) .42 (.32) .47 (.41) .17 (.33)
Individual 2.89 (1.56) 3.92 (1.87) .57 (.36) .44 (.35) .38 (.38) .34 (.38)
High–Low sequence
Collaborative 3.31 (1.67) 3.46 (1.60) .65 (.30) .41 (.34) .41 (.35) .23 (.28)
Individual 3.30 (1.85) 3.73 (1.63) .64 (.33) .47 (.38) .51 (.35) .59 (.41)
 672 RETNOWATI, AYRES, AND SWELLER

With the introduction of a problem solving treatment it was

possible to test for a worked example effect (see Atkinson et al.,
2000; P. Kirschner et al., 2006; Renkl, 2014a, 2014b). In other
words, it was predicted that worked examples would be advanta-
geous compared to conventional problem solving (Hypothesis 3).

Method
Participants. One hundred twenty-two students from four
Year 7 classrooms in an Indonesian school, in Kudus, Central Java,
participated in the study. Consistent with the sample in the previ-
ous experiment, the school had a similar organization and followed
the same national curriculum. Consistent with Experiment 1, a
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

team of three mathematics teachers taught all classes according to

This document is copyrighted by the American Psychological Association or one of its allied publishers.

set topics. Also similar to the participants in Experiment 1, col-

laborative learning was reported as a common learning strategy,
not only in mathematics classes but also in other subjects. As was
the case in Experiment 1, the small groups were created at the
beginning of the school year independently of this experiment, and
composed of mixed ability students with the same gender. Students
Figure 1. Interaction between task complexity and grouping context on were assumed to be familiar with each other since they had been
transfer test scores in Experiment 1. in the same groupings for more than five months.
First, students were randomly allocated into individual (n ⫽ 63)
or collaborative (n ⫽ 60, 9 groups of 4, and 8 groups of 3) learning
results indicated a reverse effect. On the transfer test, students who conditions, and then randomly assigned into worked example or
studied individually performed significantly higher than those who problem solving groups. Five students were excluded from the
studied collaboratively, and more specifically on the higher- analysis because they did not complete all experimental stages,
complexity tasks. Furthermore, for the High–Low order of study, leaving 118 students (46 girls, 72 boys) with an average age of
individual study resulted in a significant advantage. 12.50 years (SD ⫽ 0.55). Thirty students studied worked examples
Hypothesis 2 predicted that the effectiveness of collaborative individually, 29 solved problems individually, 31 (4 groups of 4,
learning would be increased by task complexity. No support for and 5 groups of 3) studied worked examples collaboratively, and
this hypothesis was found, as collaborative learning was not found 28 (5 groups of 4, 2 groups of 3, and 1 group of 2) solved problems
to be superior to individual learning on any specific task. The collaboratively.
significant interaction effect on transfer problems indicated that Learning materials and procedure. The materials used in
individual study was more advantageous than collaborative study, this experiment were identical to Experiment 1, except that a new
for high-complexity problems—the reverse of what was expected. conventional problem-solving group was introduced. Problem-
It was concluded, based on these results in the current context, that solving acquisition booklets for both levels of complexity were
when using worked examples, collaborative study was a disadvan- designed, based on the worked examples booklets. Where in
tage. Experiment 1 for each problem pair, the first problem had a fully
worked example given, this solution was no longer provided.
Instead, this problem now had to be solved by participants during
Experiment 2
the acquisition phase. Hence, for the worked example condition,
The results of Experiment 1 suggested that worked examples students studied a problem, and solved a similar problem; for the
may not be enhanced by collaborative learning, but it was notable problem solving condition, both problems had to be solved without
that worked examples were not compared with a problem solving solutions being shown. Each problem pair was placed on a single
control group as is usually the case in worked-examples research. page, positioned identically to the worked example material except
Because we did not test for the worked example effect in Exper- that no solutions were shown. Students were instructed to solve
iment 1, it is possible that the worked example approach was each problem. Equivalent to the worked example booklet, the final
unsuitable for this match of topic and learner, leading to no answer of every problem was provided on each page.
learning advantage for using worked examples. To rule out this The similar test and the transfer test materials were identical
possibility, Experiment 2 included a problem-solving treatment. to the low- and high-complexity problems used in Experiment 1,
This enabled the first two hypotheses to be tested again using a 2 and the allocated times remained the same. The internal consis-
(instructional strategy: Worked Example vs. Problem Solving) ⫻ tency of the tests was measured again using Cronbach’s alpha for
2 (grouping contexts: Collaborative vs. Individual) ⫻ 2 (level of this sample. For the low-complexity problems, the values were .88
complexity: Low vs. High) mixed experimental design. Because in for the similar test and .80 for the transfer test. For the high-
Experiment 1 few differences were found by balancing the se- complexity problems, the values were .82 for the similar test and
quencing of problem types, only the complex-simple sequence .67 for the transfer test.
(High–Low complexity task sequence) was used in Experiment 2, The procedures used in this experiment were identical to Ex-
as it produced the most significant interactions. periment 1. The only difference, apart from introducing additional
 COLLABORATIVE LEARNING AND WORKED EXAMPLES 673

problem-solving groups, was that only the complex-simple se- A significant effect of complexity was found, F(1, 114) ⫽
quence (High–Low complexity task sequence) was used for the 23.72, MSE ⫽ 0.04, p ⬍ .001, ␩2p ⫽ .172. Students performed
two types of problems, as this sequence previously produced significantly higher in low-complexity problems (M ⫽ 0.49, SD ⫽
significant interactions. 0.29) than in high-complexity problems (M ⫽ 0.36, SD ⫽ 0.25).
A 3-way interaction effect was also found, F(1, 114) ⫽ 8.83,
Results and Discussion MSE ⫽ 0.04, p ⫽ .004, ␩2p ⫽ .072, caused by the significant
differences found in low-complexity tests (see Figure 4). For
A 2 (Worked Example vs. Problem Solving) ⫻ 2 (Collaborative worked examples, individual study was superior to collaborative
vs. Individual) ⫻ 2 (Low- vs. High-complexity) ANOVA with study, F(1, 59) ⫽ 7.01, MSE ⫽ 0.08, p ⫽ .01, ␩2p ⫽ .106,
repeated measures on the last variable was used to analyze the
replicating the results of Experiment 1, but when problem
data. The means (and standard deviations) of test performance and
solving, collaborative study was superior to individual study,
cognitive load ratings are summarized in Table 2.
F(1, 55) ⫽ 4.67, MSE ⫽ 0.09, p ⫽ .035, ␩2p ⫽ .078. No
Cognitive load during acquisition results. A main effect of
significant differences were found for high-complexity prob-
instructional strategy was obtained, F(1, 114) ⫽ 90.96, MSE ⫽
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

lems (F ⬍ 1, ns., for both).

3.82, p ⬍ .001, ␩2p ⫽ .44, where the worked example conditions
This document is copyrighted by the American Psychological Association or one of its allied publishers.

Transfer test results. There was no worked example effect,

(M ⫽ 3.80, SD ⫽ 1.56) generated significantly lower cognitive
F(1, 114) ⫽ 2.06, MSE ⫽ 0.104, p ⫽ .154, ␩2p ⫽ .018, nor a learner
load scores (difficulty scale) than the problem solving conditions
grouping context effect (F ⬍ 1, ns.) However, there was a signif-
(M ⫽ 6.23, SD ⫽ 2.08). No significant effect for learner grouping
icant interaction between the instructional strategy and the learner
contexts was found, F(1, 114) ⫽ 1.97, MSE ⫽ 3.82, p ⫽ .164,
grouping context, F(1, 114) ⫽ 8.60, MSE ⫽ 0.104, p ⫽ .004, ␩2p ⫽
␩2p ⫽ .02. The low-complexity problems (M ⫽ 4.47, SD ⫽ 1.93)
.070 (see Figure 5). The simple effects test indicated a significant
generated significantly less cognitive load than high-complexity
problems (M ⫽ 5.56, SD ⫽ 1.71), F(1, 114) ⫽ 23.5, MSE ⫽ 2.99, difference for worked examples, F(1, 59) ⫽ 4.81, MSE ⫽ 0.08,
p ⬍ .001, ␩2p ⫽ .17. p ⫽ .032, ␩2p ⫽ .075, where individual study again was superior to
A significant 3-way interaction was found, F(1, 114) ⫽ 3.99, collaborative study. For problem solving, a significant effect again
MSE ⫽ 2.99, p ⫽ .048, ␩2p ⫽ .034. Simple effects tests showed that was found in favor of collaborative learning, F(1, 55) ⫽ 3.96,
individual learners experienced a significantly higher cognitive MSE ⫽ 0.13, p ⫽ .05, ␩2p ⫽ .067.
load than learners in the collaborative context when learning the A main effect of complexity was found, F(1, 114) ⫽ 33.64,
high-complexity problems using worked examples, F(1, 59) ⫽ MSE ⫽ 0.03, p ⬍ .001, ␩2p ⫽ .23. Students performed significantly
4.34, MSE ⫽ 2.45, p ⫽ .041, ␩2p ⫽ .069 (see Figure 2), but no other higher in the low-complexity transfer problems (M ⫽ 0.31, SD ⫽
significant effects were found. 0.27) than in the high-complexity problems (M ⫽ 0.18, SD ⫽
Similar test results. A significant worked example effect was 0.23). A significant interaction effect between the instructional
found, F(1, 114) ⫽ 24.93, MSE ⫽ 0.111, p ⬍ .001, ␩2p ⫽ .18, as strategy and problem complexity was also found, F(1, 114) ⫽
studying worked examples (M ⫽ 0.53, SD ⫽ 0.27) was found to 9.75, MSE ⫽ 0.03, p ⫽ .002, ␩2p ⫽ .08 (see Figure 6). The simple
be superior to problem solving (M ⫽ 0.32, SD ⫽ 0.28). No effects tests indicated that for the high-complexity transfer prob-
significant effect for the learner grouping context was found (F ⬍ lems, worked examples led to a significantly higher performance
1, ns.). However, there was an interaction effect between instruc- than problem solving, F(1, 116) ⫽ 7.96, MSE ⫽ 0.06, p ⫽ .006,
tional strategy and learner grouping context, F(1, 114) ⫽ 5.92, ␩2p ⫽ .064, but for low-complexity transfer problems, there were
MSE ⫽ 0.111, p ⫽ .017, ␩2p ⫽ .049 (see Figure 3). Simple effects no significant differences between the learning strategies (F ⬍ 1,
tests revealed that there were no significant differences between ns.).
the learner grouping contexts when students studied worked ex- No overall support was found for Hypothesis 1 that students
amples, F(1, 59) ⫽ 2.56, MSE ⫽ 0.10, p ⫽ .115, ␩2p ⫽ .042. There would benefit from studying collaboratively rather than individu-
was nonsignificant difference between means, with a small to ally when using worked examples. Instead, the reverse result was
medium effect size, F(1, 55) ⫽ 3.32, MSE ⫽ 0.12, p ⫽ .07, ␩2p ⫽ obtained, with individual study superior to collaborative study on
.057, in favor of collaborative learning when students studied both similar and transfer tests for high complexity problems.
through the problem-solving format. That difference can be as- Interestingly, this superiority was associated with a higher cogni-
sumed to have been the primary cause of the significant interac- tive load for individual study. Normally, a lower cognitive load is
tion. associated with improved performance. Future work will be re-

Table 2
Means (and Standard Deviations) for Test Results and Cognitive Load Ratings in Experiment 2

Cognitive load (1–9) Similar test (0 –1) Transfer test (0 –1)

Condition Low-complexity High-complexity Low-complexity High-complexity Low-complexity High-complexity

Worked examples
Collaborative 3.16 (1.32) 4.06 (1.34) .49 (.27) .48 (.24) .24 (.27) .19 (.20)
Individual 3.07 (1.80) 4.90 (1.77) .68 (.30) .47 (.27) .36 (.22) .30 (.23)
Problem solving
Collaborative 5.43(1.99) 6.68 (1.85) .55 (.31) .31 (.30) .40 (.30) .16 (.35)
Individual 6.21(2.60) 6.59 (1.86) .31 (.32) .16 (.23) .22 (.29) .07 (.13)
 674 RETNOWATI, AYRES, AND SWELLER
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
This document is copyrighted by the American Psychological Association or one of its allied publishers.

Figure 2. Interaction between instructional strategy and grouping context Figure 4. Interaction between instructional strategy and grouping context
on cognitive load during learning high complexity material in the acqui- for similar test scores on low-complexity task in Experiment 2.
sition phase in Experiment 2.

Hypothesis 3 predicted that students would benefit from study-

quired to establish whether this result is replicable. Nevertheless, ing worked examples rather than problem solving. An overall main
based on interaction effects, there was some advantage for collab- effect was found in support of this hypothesis for the similar test
oration. Collaborative learning was superior to individual learning problems. Although no main effect for worked examples was
on similar and transfer tests when students initially used the found on the transfer test phase, an interaction effect indicated that
problem solving strategy. on high-complexity transfer problems, those who studied worked
There was no support for Hypothesis 2 that the effectiveness of examples scored higher than those who initially solved problems.
collaborative learning would be increased by task complexity. It Furthermore, during the similar test phase, the cognitive load was
was found that for the problem solving strategy, collaborative found to be lower when studying worked examples rather than
learners performed better than individual learners on similar tests solving problems.
for the low-complexity problems. For the worked example strat- In summary, this experiment confirmed that the worked exam-
egy, however, individual learners performed better than collabor- ple strategy was superior to the problem solving strategy. When
ative learners for low-complexity similar tests. No effects were the material was higher in complexity, learning by worked exam-
obtained using high-complexity problems. ples in an individual setting was advantageous compared to col-

Figure 3. Interaction between instructional strategy and grouping context Figure 5. Interaction between instructional strategy and grouping context
on similar test scores in Experiment 2. on transfer test scores in Experiment 2.
 COLLABORATIVE LEARNING AND WORKED EXAMPLES 675

cated evidence in support of the effectiveness of a worked example

strategy compared to a problem solving strategy (the worked
example effect), as predicted. Worked examples were found to be
more effective during the similar test phase. Furthermore, students
using the worked example strategy experienced a lower cognitive
load for the similar test phase. A significant interaction was also
found. Students who originally studied worked examples had
higher scores on the high complexity transfer problems than those
who originally were asked to solve problems.
As summarized above, no evidence was found that when using
worked examples, collaborative learning was significantly superior
to individual learning. In contrast, some evidence emerged that
individual contexts were superior. A perspective from evolutionary
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

educational psychology (Geary, 1995, 2002) can be used to ex-

This document is copyrighted by the American Psychological Association or one of its allied publishers.

plain why collaborative learning was rarely superior to individual

learning. Geary argues that in social interactions, students develop
their biologically primary knowledge rather than the assigned
biologically secondary knowledge. In other words, they become
more adept at their social interaction, which is an evolutionary
primary skill, rather than the assigned mathematics task, which is
Figure 6. Interaction between instructional strategy and task complexity an evolutionary secondary skill, and requires considerable con-
on transfer test scores in Experiment 2. scious effort to learn. It was expected that group interactions would
have generated superior sense making and reorganization of the
information provided. However, there was no evidence for this
laborative learning. When the material was lower in complexity, suggestion. It is possible that worked examples provide sufficient
some benefits were found for problem solving in groups. information, rendering collaboration unnecessary.
The study also examined if the effectiveness of collaborative
General Discussion learning was important when dealing with complex tasks. High-
complexity problems were argued to increase active social inter-
Summary of Evidence in Support of the Hypotheses action during collaborative learning. As Hypothesis 2 was rejected,
it can be concluded that for the complexity levels used, neither
Three hypotheses were tested over the two experiments, and the low- nor high-complexity tasks improved collaborative learning
overall evidence is summarized below. compared with individual learning. In fact, it was found on several
The first hypothesis predicted that worked examples will be occasions that for high-complexity tasks, individual learning led to
enhanced by studying collaboratively compared to studying indi- higher performance than collaborative learning.
vidually. This hypothesis was examined in both experiments and Moreover, it was found that collaborative learning only had an
was not supported. When all students were tested individually on advantage over individual learning during problem solving. Col-
the similar and transfer tests, no superiority was found. In contrast, laborative learners scored higher than individual learners after
the results of Experiment 1 (transfer test performance in a worked having acquired their initial knowledge through problem solving.
example environment) showed that individual learners had a sig- It is also notable that the collaborative advantage occurred only on
nificant advantage over collaborative learners, while Experiment 2 low-complexity materials. It is possible that the low-complexity
indicated an advantage for individual study on both similar and problem solving imposed a lower cognitive load and thus could be
transfer tasks for high complexity problems. Consequently, Hy- managed in a collaborative learning setting.
pothesis 1 was rejected. The study also tested for a worked example effect. The evidence
The second hypothesis predicted that the effectiveness of col- obtained in this study is consistent with cognitive load theory
laborative learning is increased by task complexity. This hypoth- research, demonstrating that overall the worked example strategy
esis was examined in Experiments 1 and 2. As reported, these was superior to a problem solving strategy. Worked examples in
experiments used two tasks with different levels of complexity. It general can be used in individual or collaborative learning con-
was predicted that there would be interactions between task com- texts, replicating a previous finding (Retnowati et al., 2010). It is
plexity and the effectiveness of collaborative learning. Although important to note, however, that the various interactions identified
interactions were found, follow-up tests indicated that collabora- in this study indicated that the worked example strategy was best
tive learning was not superior to individual learning on the more used in individual rather than collaborative settings, particularly
complex tasks. In fact, in Experiment 1 (transfer test performance), for high-complexity problems.
evidence emerged that for the more complex tasks individual Collaborative learning creates conditions where students in a
learning was superior to collaborative learning. Thus, Hypothesis group are expected to discuss the learning material, which can be
2 was rejected. done by giving/receiving elaborated explanations (Cohen, 1994;
The final hypothesis predicted that studying worked examples Webb, 1991, 2009). However, worked examples contain step-by-
would be more advantageous than conventional problem solving. step explanations to reach a problem solution, so discussing
This hypothesis was examined in Experiment 2. The results indi- worked examples may have a redundant element (Chandler &
 676 RETNOWATI, AYRES, AND SWELLER

Sweller, 1991). Worked examples are unnecessary if members of cause they have already obtained the necessary information from an
the group can “borrow” (using the cognitive load theory borrowing instructor via the worked example. Regarding problem complexity,
and reorganizing principle) the information required to learn or individual study seemed to be most appropriate for the complex
solve the given problem from the other group members. Similarly, problems, although collaboration was helpful when problem solving
group interactions may well help enhance the reorganization of (the inferior strategy), presumably because collaboration permitted
new information. learners to obtain missing information from other learners. Hence,
While the current results were theoretically coherent and largely collaboration may be advantageous when problem solving because to
consistent, they will require replication in different contexts using some extent it is able to provide learners with missing guidance.
different populations and materials. We have established that at In conclusion, there appear to be limits to the conditions under
least under some circumstances, collaboration when studying which collaborative learning is effective. Those limits should be
worked examples has negative rather than positive effects, while considered when encouraging learners to study collaboratively.
collaboration using problem solving can have positive effects. As
far as we are aware, this finding is novel. We have interpreted
References
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

these findings in terms of redundancy (Nihalani et al., 2011).

This document is copyrighted by the American Psychological Association or one of its allied publishers.

Learners studying worked examples do not need additional infor- Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning
mation from collaborators to assist them when studying. Such from examples: Instructional principles from the worked examples re-
additional, redundant information may have negative rather than search. Review of Educational Research, 70, 181–214. http://dx.doi.org/
positive effects, leading to an expertise reversal effect (Kalyuga et 10.3102/00346543070002181
al., 2003). In contrast, when problem solving in the absence of Ayres, P., & Sweller, J. (2013). The worked example effect. In J. A. C.
worked examples, information from collaborators may be benefi- Hattie & E. M. Anderman (Eds.), International guide to student achieve-
cial. Whether collaboration when studying worked examples is ment (pp. 408 – 410). Oxford, England: Routledge.
advantageous under different circumstances requires additional Barron, B. (2000). Achieving coordination in collaborative problem-
solving groups. Journal of the Learning Sciences, 9, 403– 436. http://dx
data. For example, exceptionally complex worked examples may
.doi.org/10.1207/S15327809JLS0904_2
benefit from a collaborative approach. BNSP. (2006). Panduan penyusunan kurikulum tingkat satuan pendidikan
One potential limitation of the study occurred because we jenjang pendidikan dasar dan menengah [Guideline for developing
wanted to examine an authentic learning environment, and there- curriculum for elementary and middle education]. Jakarta, Indonesia:
fore some recommended steps, such as group processing training Badan Nasional Standar Pendidikan.
to prepare for effective collaboration, were not followed (see Carroll, W. M. (1994). Using worked examples as an instructional support
Johnson & Johnson, 1994). It is feasible that the group processes in the algebra classroom. Journal of Educational Psychology, 86, 360 –
conducted by this sample were not sufficient to optimize the 367. http://dx.doi.org/10.1037/0022-0663.86.3.360
impact of collaboration. Nevertheless, the groups had been work- Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of
ing together in mathematics classes for 3 (Experiment 1) and 5 instruction. Cognition and Instruction, 8, 293–332. http://dx.doi.org/10
.1207/s1532690xci0804_2
(Experiment 2) months and so had experience learning in their
Chen, O., Kalyuga, S., & Sweller, J. (2015). The worked example effect,
groups. Furthermore, the finding that collaboration was superior to the generation effect, and element interactivity. Journal of Educational
individual study for the problem-solving strategy suggests that Psychology, 107, 689 –704. http://dx.doi.org/10.1037/edu0000018
there were benefits, and therefore a certain amount of effective Chen, O., Kalyuga, S., & Sweller, J. (2016a). Relations between the
collaborative behavior can be assumed to have been present. To worked example and generation effects on immediate and delayed tests.
collect additional data on this issue was outside the scope of the Learning and Instruction, 45, 20 –30. http://dx.doi.org/10.1016/j
present study, but is a topic for further investigation. Furthermore, .learninstruc.2016.06.007
replicating this study with collaborative groups further prepared Chen, O., Kalyuga, S., & Sweller, J. (2016b). The expertise reversal effect
according to the steps often recommended for effective collabora- is a variant of the more general element interactivity effect. Educational
tion should be also be informative. Psychology Review. Advance online publication. http://dx.doi.org/10
.1007/s10648-016-9359-1
Many effective collaboration tasks consist of realistic ill-
Cohen, E. G. (1994). Restructuring the classroom: Conditions for produc-
structured problems (Hmelo-Silver, 2004), have high complexity
tive small groups. Review of Educational Research, 64, 1–35. http://dx
(F. Kirschner et al., 2009a), or cannot be completed by individuals .doi.org/10.3102/00346543064001001
(Cohen, 1994). In contrast, the tasks chosen for this study (middle Cooper, G., & Sweller, J. (1987). The effects of schema acquisition and
school equation solving) did not contain all these characteristics. rule automation on mathematical problem-solving transfer. Journal of
Nevertheless, we did test for the effect of complexity, and signif- Educational Psychology, 79, 347–362. http://dx.doi.org/10.1037/0022-
icant differences were found between the two complexity levels. 0663.79.4.347
Future studies could include richer problem solving tasks with Davidson, N., & Kroll, D. L. (1991). An overview of research on cooper-
more ill-defined goals as recommended. Also, delayed tests could ative learning related to mathematics. Journal for Research in Mathe-
be included in future studies to judge the permanence of learning, matics Education, 22, 362–365. http://dx.doi.org/10.2307/749185
De Corte, E. (2004). Mainstreams and perspectives in research on learning
although it should be noted that worked examples have been found
(mathematics) from instruction. Applied Psychology, 53, 279 –310.
to provide robust learning longevity (see Chen et al., 2016a).
http://dx.doi.org/10.1111/j.1464-0597.2004.00172.x
With respect to educational implications, our main research ques- Depdiknas. (2004). Kurikulum 2004 untuk sekolah menengah pertama dan
tion was Can collaborative learning improve the effectiveness of madrasah tsanawiyah [2004 Curriculum for junior high school and Islamic
worked examples? Under the given conditions, the answer is no. junior high school]. Jakarta, Indonesia: Departemen Pendidikan Nasional.
Worked examples seem to be most effective in individual settings. Geary, D. C. (1995). Reflections of evolution and culture in children’s
Asking learners to discuss worked examples may be redundant be- cognition. Implications for mathematical development and instruction.
 COLLABORATIVE LEARNING AND WORKED EXAMPLES 677

American Psychologist, 50, 24 –37. http://dx.doi.org/10.1037/0003- Human Behavior, 19, 335–353. http://dx.doi.org/10.1016/S0747-
066X.50.1.24 5632(02)00057-2
Geary, D. C. (2002). Principles of evolutionary educational psychology. Kyun, S., Kalyuga, S., & Sweller, J. (2013). The effect of worked examples
Learning and Individual Differences, 12, 317–345. http://dx.doi.org/10 when learning to write essays in English literature. Journal of Experi-
.1016/S1041-6080(02)00046-8 mental Education, 81, 385– 408. http://dx.doi.org/10.1080/00220973
Geary, D. C. (2008). An evolutionarily informed education science. Edu- .2012.727884
cational Psychologist, 43, 179 –195. http://dx.doi.org/10.1080/ Mayer, R. (2014). Cognitive theory of multimedia learning. In R. E. Mayer
00461520802392133 (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp.
Geary, D. (2012). Evolutionary educational psychology. In K. Harris, S. 43–71). New York, NY: Cambridge University Press. http://dx.doi.org/
Graham, & T. Urdan (Eds.), APA Educational Psychology Handbook 10.1017/CBO9781139547369.005
(Vol. 1, pp. 597– 621). Washington, DC: American Psychological As- National Council of Teachers of Mathematics. (2000). Principles and
sociation. standards for school mathematics. Reston, VA: Author.
Gillies, R. M. (2003). Structuring cooperative group work in classrooms. National Ministry of Education. (2006). Peraturan Menteri Pendidikan
International Journal of Educational Research, 39, 35– 49. http://dx.doi Nasional Republik Indonesia No. 22 Tahun 2006 (Regulation of the
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

.org/10.1016/S0883-0355(03)00072-7 Minister of National Education of the Republic of Indonesia No. 22 of

This document is copyrighted by the American Psychological Association or one of its allied publishers.

Goos, M. (2004). Learning mathematics in a classroom community of 2006).

inquiry. Journal for Research in Mathematics Education, 35, 258 –291. Nihalani, P., Mayrath, M., & Robinson, D. (2011). When feedback harms
http://dx.doi.org/10.2307/30034810 and collaboration helps in computer simulation environments: An ex-
Hanham, J., & McCormick, J. (2009). Group work in schools with close pertise reversal effect. Journal of Educational Psychology, 103, 776 –
friends and acquaintances: Linking self-processes with group processes. 785. http://dx.doi.org/10.1037/a0025276
Learning and Instruction, 19, 214 –227. http://dx.doi.org/10.1016/j Oksa, A., Kalyuga, S., & Chandler, P. (2010). Expertise reversal effect in
.learninstruc.2008.04.002 using explanatory notes for readers of Shakespearean text. Instructional
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do Science, 38, 217–236. http://dx.doi.org/10.1007/s11251-009-9109-6
students learn? Educational Psychology Review, 16, 235–266. Paas, F. (1992). Training strategies for attaining transfer of problem-
Johnson, D. W., & Johnson, R. T. (1994). Learning together and alone: solving skill in statistics: A cognitive load approach. Journal of Educa-
Cooperative, competitive and individualistic learning. Boston, MA: tional Psychology, 84, 429 – 434. http://dx.doi.org/10.1037/0022-0663
Allyn & Bacon. .84.4.429
Johnson, D. W., & Johnson, R. T. (2002). Learning together and alone: Paas, F., & Sweller, J. (2012). An evolutionary upgrade of cognitive load
Overview and meta-analysis. Asia Pacific Journal of Education, 22, theory: Using the human motor system and collaboration to support the
95–105. http://dx.doi.org/10.1080/0218879020220110 learning of complex cognitive tasks. Educational Psychology Review,
Johnson, D. W., Johnson, R. T., & Smith, K. A. (1998). Cooperative 24, 27– 45. http://dx.doi.org/10.1007/s10648-011-9179-2
learning returns to college: What evidence is there that it works? Paas, F., & van Merriënboer, J. J. G. (1994). Variability of worked
Change: The Magazine of Higher Learning, 30, 26 –35. http://dx.doi examples and transfer of geometrical problem-solving skills: A
.org/10.1080/00091389809602629 cognitive-load approach. Journal of Educational Psychology, 86, 122–
Johnson, D. W., Maruyama, G., Johnson, R., Nelson, D., & Skon, L. 133. http://dx.doi.org/10.1037/0022-0663.86.1.122
(1981). The effects of cooperative, competitive, and individualistic goal Plass, J. L., O’Keefe, P. A., Homer, B. D., Case, J., Hayward, E. O., Stein,
structures on achievement: A meta-analysis. Psychological Bulletin, 89, M., & Perlin, K. (2013). The impact of individual, competitive, and
47– 62. http://dx.doi.org/10.1037/0033-2909.89.1.47 collaborative mathematics game play on learning, performance, and
Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise motivation. Journal of Educational Psychology, 105, 1050 –1066. http://
reversal effect. Educational Psychologist, 38, 23–31. http://dx.doi.org/ dx.doi.org/10.1037/a0032688
10.1207/S15326985EP3801_4 Quilici, J. L., & Mayer, R. E. (1996). Role of examples in how students learn
Kirschner, F., Paas, F., & Kirschner, P. A. (2009a). A cognitive load to categorize statistics word problems. Journal of Educational Psychology,
approach to collaborative learning: United brains for complex tasks. 88, 144 –161. http://dx.doi.org/10.1037/0022-0663.88.1.144
Educational Psychology Review, 21, 31– 42. http://dx.doi.org/10.1007/ Renkl, A. (2014a). Toward an instructionally oriented theory of example-
s10648-008-9095-2 based learning. Cognitive Science, 38, 1–37. http://dx.doi.org/10.1111/
Kirschner, F., Paas, F., & Kirschner, P. A. (2009b). Individual and group- cogs.12086
based learning from complex cognitive tasks: Effects on retention and Renkl, A. (2014b). The worked examples principle in multimedia learning.
transfer efficiency. Computers in Human Behavior, 25, 306 –314. http:// In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning
dx.doi.org/10.1016/j.chb.2008.12.008 (2nd ed., pp. 391– 412). New York, NY: Cambridge University Press.
Kirschner, F., Paas, F., & Kirschner, P. A. (2011). Task complexity as a http://dx.doi.org/10.1017/CBO9781139547369.020
driver for collaborative learning efficiency: The collective working- Retnowati, E., Ayres, P., & Sweller, J. (2010). Worked example effects in
memory effect. Applied Cognitive Psychology, 25, 615– 624. http://dx individual and group work settings. Educational Psychology, 30, 349 –
.doi.org/10.1002/acp.1730 367. http://dx.doi.org/10.1080/01443411003659960
Kirschner, F., Paas, F., Kirschner, P. A., & Janssen, J. (2011). Differential Rourke, A., & Sweller, J. (2009). The worked-example effect using ill-defined
effects of problem-solving demands on individual and collaborative problems: Learning to recognise designers’ styles. Learning and Instruc-
learning outcomes. Learning and Instruction, 21, 587–599. tion, 19, 185–199. http://dx.doi.org/10.1016/j.learninstruc.2008.03.006
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance Schreiber, L. M., & Valle, B. E. (2013). Social constructivist teaching
during instruction does not work: An analysis of the failure of construc- strategies in the small group classroom. Small Group Research, 44,
tivist, discovery, problem-based, experiential, and inquiry-based teach- 395– 411. http://dx.doi.org/10.1177/1046496413488422
ing. Educational Psychologist, 41, 75– 86. http://dx.doi.org/10.1207/ Slavin, R. E. (1995). Cooperative learning: Theory, research, and practice
s15326985ep4102_1 (2nd ed.). Boston, MA: Allyn & Bacon.
Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the Staples, M. (2007). Supporting whole-class collaborative inquiry in a
pitfalls for social interaction in computer-supported collaborative secondary mathematics classroom. Cognition and Instruction, 25, 161–
learning environments: A review of the research. Computers in 217. http://dx.doi.org/10.1080/07370000701301125
 678 RETNOWATI, AYRES, AND SWELLER

Sweller, J. (2010). Element interactivity and intrinsic, extraneous and van Gog, T., Kirschner, F., Kester, L., & Paas, F. (2012). Timing and
germane cognitive load. Educational Psychology Review, 22, 123–138. frequency of mental effort measurement: Evidence in favour of repeated
http://dx.doi.org/10.1007/s10648-010-9128-5 measures. Applied Cognitive Psychology, 26, 833– 839. http://dx.doi
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New .org/10.1002/acp.2883
York, NY: Springer. http://dx.doi.org/10.1007/978-1-4419-8126-4 van Gog, T., & Paas, F. (2008). Instructional efficiency: Revisiting the
Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. original construct in educational research. Educational Psychologist, 43,
Cognition and Instruction, 12, 185–233. http://dx.doi.org/10.1207/ 16 –26. http://dx.doi.org/10.1080/00461520701756248
s1532690xci1203_1 Ward, M., & Sweller, J. (1990). Structuring effective worked examples.
Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a Cognition and Instruction, 7, 1–39. http://dx.doi.org/10.1207/
substitute for problem solving in learning algebra. Cognition and In- s1532690xci0701_1
struction, 2, 59 – 89. http://dx.doi.org/10.1207/s1532690xci0201_3 Webb, N. M. (1991). Task-related verbal interaction and mathematics
Sweller, J., & Sweller, S. (2006). Natural information processing systems.
learning in small groups. Journal for Research in Mathematics Educa-
Evolutionary Psychology, 4, 434 – 458. http://dx.doi.org/10.1177/
tion, 22, 366 –389. http://dx.doi.org/10.2307/749186
147470490600400135
Webb, N. M. (2009). The teacher’s role in promoting collaborative dia-
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Tarmizi, R. A., & Sweller, J. (1988). Guidance during mathematical

logue in the classroom. British Journal of Educational Psychology, 79,
problem solving. Journal of Educational Psychology, 80, 424 – 436.
This document is copyrighted by the American Psychological Association or one of its allied publishers.

1–28. http://dx.doi.org/10.1348/000709908X380772
http://dx.doi.org/10.1037/0022-0663.80.4.424
Trafton, J. G., & Reiser, B. J. (1993). The contributions of studying Weinberger, A., Stegmann, K., & Fischer, F. (2007). Knowledge convergence
examples and solving problems to skill acquisition. Paper presented at in collaborative learning: Concept and assessment. Learning and Instruc-
the Fifteenth Annual Conference of the Cognitive Science Society, tion, 17, 416 – 426. http://dx.doi.org/10.1016/j.learninstruc.2007.03.007
Hillsdale, NJ. Yackel, E., Cobb, P., & Wood, T. (1991). Small-group interactions as a
Van den Bossche, P., Gijselaers, W. H., Segers, M., & Kirschner, P. A. (2006). source of learning opportunities in second-grade mathematics. Journal
Social and cognitive factors driving teamwork in collaborative learning envi- for Research in Mathematics Education, 22, 390 – 408. http://dx.doi.org/
ronments: Team learning beliefs and behaviors. Small Group Research, 37, 10.2307/749187
490–521. http://dx.doi.org/10.1177/1046496406292938 Zhang, L., Ayres, P., & Chan, K. (2011). Examining different types of
van Gog, T., & Kester, L. (2012). A test of the testing effect: Acquiring collaborative learning in a complex computer-based environment: A
problem-solving skills from worked examples. Cognitive Science, 36, cognitive load approach. Computers in Human Behavior, 27, 94 –98.
1532–1541. http://dx.doi.org/10.1111/cogs.12002 http://dx.doi.org/10.1016/j.chb.2010.03.038
 COLLABORATIVE LEARNING AND WORKED EXAMPLES 679

Appendix A
Example of the Low-Complexity Learning Material Using the Worked Example Instruction

Study this example Solve this problem Final answer

Solve 4a ⫹ 13 ⫽ 65 a⫽13
3p ⫹ 10 ⫽ 85
3p ⫹ 10⫺10 ⫽ 85⫺10 [subtract 10 from both sides]
3p ⫹ 0 ⫽ 75
3p ⫽ 75
3p ⫽ 75 [divide both sides by 3]
3 3
p ⫽ 25
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Hence, the solution is p ⫽ 25.

This document is copyrighted by the American Psychological Association or one of its allied publishers.

Appendix B
Example of the High-Complexity Learning Material Using the Worked Example Instruction

Study this example Solve this problem Final answer

Twice the number of Dina’s marbles when added to five equals Four times the number of Bobi’s marbles when added 12
seventy five. How many are Dina’s marbles? to two equals fifty. How many are Bobi’s marbles?

Answer: Answer:
Step 1: Translate the sentence into an equation Step 1:
• Identify the keywords. These are underlined in the sentence
above.
• The variable is: the number of Dina’s marbles
• Give a symbol to the variable, say it is: p
• The equation is 2 ⫻ p ⫹ 5 ⫽ 75 or it can be written 2p ⫹
5 ⫽ 75
Step 2: Solve the equation Step 2:
2p ⫹ 5 ⫽ 75
2p ⫹ 5⫺5 ⫽ 75ⴚ5 [subtract 5 from both sides]
2p ⫽ 70
2p ⫽ 70 [divide both sides by 2]
2 2
p ⫽ 35

Step 3: Make a conclusion Step 3:

Hence, the number of Dina’s marbles is 35.

Received March 31, 2016

Revision received October 29, 2016
Accepted November 1, 2016 䡲