210

British Journal of Educational Psychology (2020), 90, 210–223

© 2019 The British Psychological Society

www.wileyonlinelibrary.com

Element interactivity as a factor influencing the

effectiveness of worked example–problem solving
and problem solving–worked example sequences
Ouhao Chen1 , Endah Retnowati2* and Slava Kalyuga3
1
National Institute of Education, Nanyang Technological University, Singapore
2
Faculty of Mathematics and Sciences, Yogyakarta State University, Indonesia
3
School of Education, University of New South Wales, Sydney, NSW, Australia

Background. The worked example effect in cognitive load theory suggests that
providing worked examples first followed by solving similar problems would facilitate
students’ learning. Using problem solving–worked example sequence is another way of
implementing example-based instruction. Although research has demonstrated the
superiority of worked example–problem solving sequence on learning materials that
presumably are high in element interactivity for novices, none of the previous studies have
compared the two sequences with levels of element interactivity experimentally
manipulated in a strictly controlled manner.
Aim. The reported study aimed to investigate the effects of levels of element
interactivity of the learning tasks and levels of learner prior knowledge on the
effectiveness of two alternative example-based sequences, worked example–problem
solving versus problem solving–worked example.
Sample. Fifty-two Year five students, around 10 to 11 years old, from a primary school
in Indonesia participated in Experiment 1, and 96 Year eight students, around 13 to
14 years old, from a secondary school in Indonesia participated in Experiment 2.
Methods. 2 (sequences: worked example–problem solving vs. problem solving–worked
example) 9 2 (levels of element interactivity: low vs. high) experimental design, with the
second factor repeatedly measured, was used in the two experiments conducted with
learners at different levels of prior knowledge.
Result. The results showed the advantage of using worked example–problem solving
sequence for learning materials high in element interactivity, especially for novice
learners, whereas there were no differences between the worked example–problem
solving and problem solving–worked example sequences for learning materials low in
element interactivity for more knowledgeable learners.
Conclusion. This study not only replicated the results of previous studies, but also
extended their findings by experimentally manipulating levels of element interactivity of
learning materials.

*Correspondence should be addressed to Endah Retnowati, Faculty of Mathematics and Sciences, Yogyakarta State University,
Colombo St. No. 1, Karang Malang, Sleman, Yogyakarta 55281, Indonesia (email: [email protected]).

DOI:10.1111/bjep.12317
 Worked example–problem solving and problem solving–worked example 211

Providing full-guidance instruction, such as a worked example, followed by solving a

similar problem (worked example–problem solving sequence) has been widely demon-
strated to be an effective and efficient approach within the framework of cognitive load
theory (worked example effect; see Sweller, Ayres, & Kalyuga, 2011, for a comprehensive
overview). Also, there are a few studies that have investigated the reversed sequence,
problem solving–worked example, as another form of example-based learning (e.g., Hsu,
Kalyuga, & Sweller, 2015; Reisslein, Atkinson, Seeling, & Reisslein, 2006; Van Gog, Kester,
& Paas, 2011). The two experiments reported in this paper conceptually replicated the
studies of Van Gog et al. (2011) and Reisslein et al. (2006) by comparing the alternative
example–problem and problem–example sequences, with added novelty of controlling
levels of element interactivity of learning materials and levels of learner expertise. The
paper describes the experiments and their results and concludes with educational
implications of the reported findings.

Cognitive load theory

Cognitive load theory is an instructional theory that is based on our knowledge of human
cognitive architecture. The most important aspect of this architecture is the relations
between working memory and long-term memory (Sweller & Sweller, 2006). Because of
its very limited capacity (Cowan, 2001; Miller, 1956) and limited duration (Peterson &
Peterson, 1959), working memory cannot process or hold more than a few novel elements
of information at one time. However, long-term memory has unlimited capacity and
duration in holding organized knowledge structures (schemas). When the relevant
schemas in long-term memory are activated and transferred to working memory, the
effective working memory capacity can be substantially increased by encapsulating many
elements of information into a small number of chunks based on those schemas (Ericsson
& Kintsch, 1995). Cognitive load theory aims to generate innovative instructional
methods to reduce learner working memory load.

Worked example effect and worked example–problem solving sequence

Working example–problem solving sequence is suggested within the framework of cognitive
load theory as a form of example-based learning. The original studies of worked example
effect demonstrated that providing a worked example followed by solving a similar problem
would facilitate students’ learning, compared to solving two problems continuously (Cooper
& Sweller, 1987; Sweller & Cooper, 1985). Due to the above characteristics of human
cognitive architecture, novice learners could more effectively acquire information structures
from studying worked examples than from generating solution themselves during problem
solving which would involve heavy working memory load. Also, worked example–problem
solving sequence is assumed to be more motivating for students as they would know that a
similar problem would follow the worked example (Sweller & Cooper, 1985).
The advantages of using worked examples and example–problem sequence have been
well-documented in many studies across different domains, starting from the very first
study with algebra tasks. Sweller and Cooper (1985) conducted five experiments in
solving algebra manipulation problems using Year 9, Year 11, and university mathematics
students. The results from Experiments 2 to 5 indicated that using worked examples not
only facilitated students’ learning but also reduced processing time and mathematics
errors. Similarly, in the domain of basic statistics (such as concepts of median and mode),
Paas (1992) compared conventional problem, worked example and completion of partial
212 Ouhao Chen et al.

worked example conditions. The results suggested that the completion of partial worked
example and worked example conditions were both superior to conventional problem-
solving condition with less time and effort. The similar findings were also demonstrated in
the areas of physics (Van Gog et al., 2011) and geometry (Paas & Van Merri€enboer, 1994;
Schwonke et al., 2009). Kirschner, Sweller, and Clark (2006) also strongly argued for the
effectiveness of using worked examples rather than problem-based approaches in
teaching, especially for novice learners.
However, the effectiveness of using worked example–problem solving sequence may
be moderated by the levels of element interactivity. The next section will discuss this issue
in details.

Element interactivity
Element interactivity is the core concept in cognitive load theory. Interactive elements are
the interconnected elements of information that must be processed simultaneously in
working memory in order to achieve understanding (Sweller et al., 2011).

Levels of element interactivity

The levels of element interactivity are determined by estimating the number of
interconnected elements that need to be processed at the same time to achieve
understanding (Sweller & Chandler, 1994; Tindall-Ford, Chandler, & Sweller, 1997).
When students are required to memorize chemical symbols, such as Fe for iron, they could
memorize it without referring to other symbols, for example Cu for cooper. They could
memorize each symbol individually. Similarly, when learning English vocabulary, Dog
could be memorized separately from Cat. This type of learning materials is low in element
interactivity as elements are not interconnected and could be processed individually. In
contrast, when solving the equation 3x + 5 = 8 for x, the elements of this equation could
not be processed individually; otherwise, we could not understand the question.
Therefore, the elements (3, x, +, 5, =, 8) are interconnected and must be processed
simultaneously in working memory to understand and solve this equation successfully.

Element interactivity and the role of expertise

The levels of element interactivity are also affected by levels of learner expertise or prior
domain-specific knowledge (Chen, Kalyuga, & Sweller, 2017; Chen, Retnowati, &
Kalyuga, 2019). For a given task, learners with low expertise in the task’s domain may
encounter more interactive elements, rendering the task being high in element
interactivity, whereas, at the same time, more knowledgeable learners could chunk
elements to reduce the number of interactive elements, thus lowering the level of element
interactivity. Therefore, for experts, the same task would become low in element
interactivity. With the increase in learners’ expertise, followed by the decrease in the level
of element interactivity, the instructional procedures that are effective for novices may
become ineffective for experts, indicating an expertise reversal effect (Kalyuga, 2007).

Element interactivity and worked example–problem solving sequence

Chen, Kalyuga, and Sweller (2015, 2016a, b) found that element interactivity might moderate
the effectiveness of worked example–problem solving sequences in the domain of
 Worked example–problem solving and problem solving–worked example 213

mathematics. In their experiments, two instructional effects, the worked example and
generation effects, were tested. When testing for the generation effect, a list of simple geometry
formulas (low in element interactivity) was used: students either generated or read formulas.
The results showed preference for the generation group in memorizing the list of formulas.
When testing for the worked example effect, students were required to apply some of
the geometry formulas in the list to calculate the area of compound shapes (high element
interactivity tasks). Some of the participants were presented with a worked example
followed by a similar problem to solve, while others were required to solve the equivalent
number of problems without any worked examples. The results showed advantages of the
worked example–problem solving sequence, indicating a worked example effect.
Therefore, for materials low in element interactivity, generation was superior to worked
examples, whereas for materials high in element interactivity, the result was reversed.
Since levels of learner expertise affect the levels of element interactivity, when testing
the same sets of materials with more knowledgeable learners, all materials became low in
element interactivity. Accordingly, the results indicated the generation effect for all sets of
materials with expertise reversal effects found across multiple experiments. It is very
likely that a similar kind of interaction exists between the effectiveness of worked
example–problem solving, problem solving–worked example sequences and levels of
element interactivity (which are affected by levels of learner expertise).

Studies of problem solving–worked example sequence

Most of studies within the framework of cognitive load theory have investigated and
established the effectiveness of worked example–problem solving sequences. However,
example-based learning may also be implemented in the form of a problem solving–
worked example sequence. The rationale for using the problem solving–worked example
design is that learners may face challenges and knowledge deficiencies when solving
problems, which may motivate them to focus better on the following worked example
that explains the associated procedures (Reisslein et al., 2006).
The worked example–problem solving and problem solving–worked example
sequences were compared in several studies (Hsu et al., 2015; Leppink, Paas, Van Gog,
van Der Vleuten, & Van Merrienboer, 2014; Van Gog et al., 2011). All these studies
demonstrated the advantages of the worked example–problem solving sequences. For
example, Van Gog et al. (2011) compared four example-based learning strategies – worked
examples only, worked example–problem solving, problem solving–worked example, and
problem solving only. The participants were secondary school students who were novices
in the electrical circuit troubleshooting tasks. Results indicated no differences between
worked examples only and worked example–problem solving or between problem
solving–worked example and problem solving only, but the worked examples only and
worked example–problem solving were more effective than problem solving only.
Similarly, Leppink et al. (2014) showed that the example–example/example–problem
condition was superior to the problem–example/problem–problem condition.

Role of expertise
According to the expertise reversal effect, the worked example effect is influenced by the
level of learner expertise. With increased levels of learner knowledge in the task domain,
the initial advantage of worked examples disappears, and with sufficiently high levels of
learner expertise, problem solving may become even more effective than studying
214 Ouhao Chen et al.

worked examples (Kalyuga, 2007). For example, Reisslein et al. (2006) investigated how
learner expertise affected the effectiveness of different instructional sequences (worked
example–problem solving vs. problem solving–worked example). The results supported
an expertise reversal effect: novices benefitted more from the worked example–problem
solving sequence, compared to more knowledgeable learners who benefitted more from
the problem solving–worked example sequence.

Present study
The experiments reported in this paper conceptually replicated the studies of Van Gog
et al. (2011) and Reisslein et al. (2006) on comparing the alternative example-based
instructional sequences: worked example–problem solving versus problem solving–
worked example. However, the current study added a new experimental factor, element
interactivity, into consideration. None of the previously conducted studies compared the
two sequences from this perspective, with levels of element interactivity experimentally
manipulated in a strictly controlled manner.
As mentioned above, levels of element interactivity are closely related to levels of
learner expertise; therefore, it could be assumed that the worked example–problem
solving sequence would be superior to the problem solving–worked example sequence
for materials high in element interactivity (when novice learners are recruited; Hypothesis
1), but this advantage would disappear or reverse for materials low in element interactivity
(when expert learners are recruited; Hypothesis 2).

Experiment 1
Participants
Fifty-two Year five students, around 10 to 11 years old, from one primary school in
Indonesia participated in this experiment. In the first phase of this study (low element
interactivity tasks), 28 students were randomly assigned to the worked example–problem
solving sequence, and the rest 24 students were assigned to the problem solving–worked
example sequence (See Figure 1). In the second phase of this study (high element
interactivity tasks), half of the students in the Phase 1 worked example–problem solving
sequence and half of the students in the problem solving–worked example sequence
were randomly assigned to a new worked example–problem solving sequence in Phase 2,
while the rest of students were assigned to a new problem solving–worked example
sequence in Phase 2 (See Figure 2). Four students from the problem solving–worked
example group in Phase 2 were excluded from the analysis, as they did not complete the
whole procedure. All students were novices in geometry tasks used in this study.

Materials
Learning materials
There were two sets of materials designed for each phase of this study separately. For the
first phase, materials were 11 geometry formulas for calculating area of simple geometric
shapes, such as the Area of a Circle. In the worked example–problem solving group, the
students studied the list of formulas without pen/pencil, and then, they generated the
formulas in writing, whereas in the problem solving–worked example group, students
generated the formulas in writing first, followed by studying the same list of formulas
 Worked example–problem solving and problem solving–worked example 215

Randomly assigned participants

Worked example–problem Problem solving–worked

solving group (Phase 1) example group (Phase 1)
N = 28 (Exp. 1)/N = 48 (Exp. 2) N = 24 (Exp. 1)/N = 48 (Exp. 2)

Studied 11 formulas

Re-studied then Generated then Re-

Generated formulas studied formulas

Cued-recall test

Figure 1. Phase 1: Procedure used for materials low in element interactivity.

without pen/pencil. In Phase 2, the materials included tasks about calculating the area of
composite shapes by applying formulas studied in Phase 1. Two pairs of worked example–
problem solving and problem solving–worked example tasks were designed. For each
pair of worked example–problem solving tasks, an example showing the full procedure
for calculating the area of a composite shape (see Figure 3 for an example) was followed
by a similar problem to solve; for each pair of problem solving–worked example tasks, the
only difference was the order of presenting the worked example and the similar problem.

Assessing levels of element interactivity

We assumed that the materials used in Phase 1 were low in element interactivity. For
example, when studying and memorizing the formula Area of a Circle = p 9 r 9 r,
students needed to know the meanings of p and r, and the multiplication relationship
between them, indicating a relatively low number of interactive elements (around four),
whereas it was assumed that the materials in Phase 2 were high in element interactivity.
For example, for novice learners facing the task showed in Figure 3, the first four elements
were associated with identifying the four equal length lines, including the missing line FC,
that form a rhombus. Then, another four elements included identifying the four lines that
constituted a trapezium, again including the missing line FC. To actually calculate the two
areas, the students needed to know the meanings of a and b in the rhombus and the
multiplication relationship between them, as well as the meanings of a, b, and h in the
216 Ouhao Chen et al.

Reviewed 11 formulas
Overview of the compound
shapes

Worked example–problem Problem solving–worked

solving group (Phase 1) example group (Phase 1)

50% 50% 50% 50%

Worked example–problem Problem solving–worked

solving group (Phase 2) example group (Phase 2)
N = 26 (Exp. 1)/N = 48 (Exp. 2) N = 22 (Exp. 1)/N = 44 (Exp. 2)

Studied one example, Solved one problem,

solved one similar studied one example
problem

Post-content test

Figure 2. Phase 2: Procedure used for materials high in element interactivity.

Figure 3. A sample of worked example used in Phase 2.

trapezium and addition, multiplication, and division by 2, that together added nine
elements. Finally, adding together those two separate area values involved another
element. Therefore, in total, around 18 interactive elements were involved in this task,
making it high in element interactivity, compared to the formulas in Phase 1.
 Worked example–problem solving and problem solving–worked example 217

Post-content tests
A cued-recall test (for materials low in element interactivity) was designed and
administered at the end of Phase 1. The order of the 11 geometry formulas in the cued-
recall test was randomized, based on the order used in the learning phase. A post-content
test (for materials high in element interactivity) with five questions on calculating the area
of composite shapes (similar to the problems in Phase 2 learning) was designed and
administered at the end of Phase 2.

Procedure
This joint study was approved by the University ethics committee of the Institute of
Research and Community Service. The approval numbers are 378/UN34.21/TU/2018 for
Experiment 1 and 817/UN34.21/TU/2018 for Experiment 2.
In Phase 1, students were randomly allocated to the alternative experimental
sequences (worked example–problem solving or problem solving–worked example; See
Figure 1) and sat in different rooms (10 min for moving class). All students studied 11
formulas first (10 min). Then, the students in the worked example–problem solving
group were presented the same 11 formulas again (10 min), followed by generating these
formulas (10 min), whereas students in the problem solving–worked example group
were required to generate the 11 formulas first (10 min), followed by the presentation of
these formulas (10 min). A cued-recall test of the 11 formulas was conducted at the end of
this phase (8 min), the Cronbach a = .80.
In Phase 2, all participants had been firstly randomly re-allocated to new experimental
groups, as described in the Participants section (See Figure 2). All the students were
again presented with the same 11 formulas as in Phase 1 as a review, and the researcher
provided an introduction to the compound shapes as combinations of common shapes.
The students were shown two examples of compound shapes (10 min). After that,
students in different groups sat in different rooms (10 min for moving class). The students
in the worked example–problem solving group were presented the first pair of worked
example–problem solving tasks, followed by the second pair, whereas the students in the
problem solving–worked example group were presented the first pair of problem
solving–worked example tasks, followed by the second pair (20 min). The only difference
between the two groups was the order of presenting the worked example and the
problem. Finally, a post-content test, including questions that were similar to those used
during the Phase 2 learning, on calculating the area of compound shapes was conducted
(10 min), the Cronbach a = .84.

Scoring
When scoring the cued-recall test, 1 mark was given to each correctly recalled formula;
therefore, the total of 11 marks could be awarded for the cued-recall test. For the second
phase test, three marks were given to each question: 1 mark for correctly calculating the
area of one separated shape; 1 mark for correctly calculating the area of another separated
shape; and 1 mark for adding the two separate areas together. Therefore, the total of 15
marks for five questions could be awarded for the test. The carry-over errors were
considered when marking the test. For example, if the area of the first shape was
calculated incorrectly (0 mark), but the area of the second shape was correct (1 mark) and
the addition of the two values was performed correctly (1 mark), even if the final answer
was incorrect, two marks were allocated for the task.
218 Ouhao Chen et al.

Results
A 2 (sequences: worked example–problem solving vs. problem solving–worked
example) 9 2 (levels of element interactivity: low vs. high) repeated measures ANOVA
was conducted with the second factor repeatedly measured. The raw test scores for both
post-tests were converted to the percentage correct scores (Table 1).
The main effect of sequence was not significant, F(1, 46) = .005, MSE = 729.56,
p = .944, g2p = 0. The main effect of element interactivity was also not significant, F(1,
46) = .144, MSE = 488.72, p = .706, g2p = .003. However, the interaction between
sequence and element interactivity was significant, F(1, 46) = 8.291, MSE = 488.72,
p = .006, g2p = .153. Following the significant interaction, simple effect analyses were
conducted. For materials that were low in element interactivity (the cued-recall test),
there was no significant difference between the sequences, t(47) = 1.45, SEdiff = 7.486,
p = .154, d = .41, whereas, for materials that were high in element interactivity (the post-
content test), the effect of sequence was significant, t(47) = 2.15, SEdiff = 6.913,
p = .037, d = .59, indicating that the worked example–problem solving sequence was
superior to the problem solving–worked example sequence.
Experiment 1 demonstrated the superiority of the worked example–problem solving
sequence with novice learners, but only for materials high in element interactivity (area of
composite shapes). For materials low in element interactivity (formulas for simple
common geometric shapes), the superiority of the problem solving–worked example
sequence was not found (although the effect size of 0.41 indicated a possibility of the
effect). Thus, for novice learners, the superiority of the worked example–problem solving
sequence for materials high in element interactivity disappeared with materials low in
element interactivity.
Since the level of element interactivity depended on levels of learner expertise,
materials high in element interactivity for novices could become low in element
interactivity for relatively more experienced learners. Therefore, in Experiment 2, more
knowledgeable learners were used, with the same sets of materials, to further compare the
effectiveness of worked example–problem solving and problem solving–worked exam-
ple sequences.

Experiment 2
Participants
Ninety-six Year eight students, around 13 to 14 years old, from one secondary school in
Indonesia participated in this experiment. In the first phase of this study, 48 students were
randomly assigned to the worked example–problem solving sequence, and the rest 48
students were assigned to the problem solving–worked example sequence (Figure 1). In
the second phase of this study, exactly the same procedure as in Experiment 1 was used
for randomly re-allocating participants to new experimental groups (Figure 2). Four

Table 1. Percentage correct scores for the post-tests in Experiment 1

Cued-recall test Post-content test

(for low element (for high element
Sequences interactivity) interactivity)

Worked example–problem solving (N = 26) 51.40 (27.45) 66.15 (25.15)

Problem solving–worked example (N = 22) 64.05 (22.70) 52.73 (22.44)
 Worked example–problem solving and problem solving–worked example 219

students from the problem solving–worked example group in Phase 2 were excluded
from the analysis, as they did not complete the whole procedure. All students had been
previously taught the type of geometry tasks that were used in this study and therefore
were considered to be relatively knowledgeable in the task domain.

Materials
All materials were identical to those used in Experiment 1. However, as students in this
experiment had been previously instructed on how to calculate the area of composite
shapes, they could retrieve the relevant schemas as single entities. Therefore, the
materials used in both phases of this experiment were low in element interactivity for the
participants.

Procedure and scoring

The procedure and scoring were the same as in Experiment 1.

Results
A 2 (sequences: worked example–problem solving vs. problem solving–worked
example) 9 2 (levels of element interactivity: low vs. high) repeated measures ANOVA
was conducted with the second factor repeatedly measured. The raw post-tests scores
were converted to the percentage correct scores (Table 2).
The effect of sequence was not significant, F(1, 90) = .233, MSE = 351.11, p = .631,
g2p = .003. The effect of element interactivity was not significant, F(1, 90) = 1.814,
MSE = 368.80, p = .181, g2p = .020. The interaction between sequence and element
interactivity was also not significant, F(1, 90) = 2.152, MSE = 368.80, p = .146,
g2p = .023. Therefore, the results confirmed that for materials low in element interactivity
(geometric formulas and area of composite shapes) in both phases of Experiment 2, there
were no significant differences between the worked example–problem solving and
problem solving–worked example sequences. The superiority of the worked example–
problem solving sequence for high element interactivity tasks found in Experiment 1 with
novices disappeared in Experiment 2 with more knowledgeable learners, which might
indicate an expertise reversal effect.

General discussion
This randomized, controlled experimental study was designed to investigate relations
between the effectiveness of worked example–problem solving and problem solving–

Table 2. Percentage correct scores for the post-test in Experiment 2

Cued-recall test Post-content test

(for low element (for high element
Sequences interactivity) interactivity)

Worked example–problem solving (N = 48) 74.24 (19.52) 74.58 (17.49)

Problem solving–worked example (N = 44) 77.07 (19.75) 69.09 (19.13)
220 Ouhao Chen et al.

worked example sequences by experimentally manipulating the levels of element

interactivity of the corresponding materials.
In Experiment 1, novices in the task domain learned how to calculate the area of
compound shapes after learning some relevant geometric formulas. The results showed
the advantage of using the worked example–problem solving sequence for learning
materials high in element interactivity, confirming Hypothesis 1, whereas there were no
differences between the worked example–problem solving and problem solving–worked
example sequences for learning materials low in element interactivity, confirming
Hypothesis 2.
In Experiment 2, more knowledgeable learners were recruited to learn the same sets of
materials. Since they had been previously taught how to solve the type of tasks used in the
experiment, all sets of materials were low in element interactivity for these participants.
Accordingly, the results showed no differences between the worked example–problem
solving and problem solving–worked example sequences for all sets of materials,
confirming again Hypothesis 2. The superiority of the worked example–problem solving
sequence found in Experiment 1 (with novice learners) disappeared in Experiment 2
(with more experienced learners), demonstrating an expertise reversal effect.
In summary, the study demonstrated that worked examples presented prior to
problem solving were more effective than the reversed sequence for high element
interactivity materials, with no difference between the sequences for low element
interactivity materials. This study replicated the results of Van Gog et al. (2011) and
Reisslein et al. (2006) indicating the superiority of worked example–problem solving
sequences for novices’ learning, but the study also extended their results by considering
levels of element interactivity of instructional materials. Namely, the results of Van Gog
et al. (2011) and Reisslein et al. (2006) might be also explained from the point of view of
element interactivity. The superiority of the worked example–problem solving sequence
was found with novices, because for materials high in element interactivity, using worked
examples first reduced the learners’ working memory load, which was in line with the
worked example effect in cognitive load theory (Chen et al., 2015, 2016a, b). The
superiority of the worked example–problem solving sequence disappeared or reversed,
because the materials were low in element interactivity for more knowledgeable learners,
making worked examples redundant for the learners (and unnecessarily increasing their
working memory load due to processing redundant information).
The results are also in accord with the previous research in cognitive load theory that
demonstrated that for novice learners, the example–problem instructional sequence
outperformed the problem–example sequence (Hsu et al., 2015; Leppink et al., 2014; Van
Gog et al., 2011), as well as with studies by Glogger-Frey, Fleischer, Gr€ uny, Kappich, and
Renkl (2015), and Glogger-Frey, Gaus, and Renkl (2017). Glogger-Frey et al. (2015) found
an advantage for guidance over invention activities, whereas the opposite result was
obtained by Glogger-Frey et al. (2017). The changed result was caused by the increased
learning time for the 2017 study, compared to the 2015 study, which was the major
difference between the two papers (Sweller & Paas, 2017). Increases in learning time
resulted in increases in expertise with the attendant decreases in element interactivity.
With low element interactivity, guidance was redundant and generation activities became
more important which is exactly the point being made in the current paper.
This study did not obtain any strong evidence in support of the problem solving–
worked example sequence, as there were no significant differences for low element
interactivity materials (even though a medium-size effect in Experiment 1 showed a
trend). Still, the concept of element interactivity in cognitive load theory may indicate the
 Worked example–problem solving and problem solving–worked example 221

conditions under which this approach becomes preferable to the worked example–
problem solving instructional sequence. In relation to this, one major limitation of this
study was that the materials were mostly associated with learning procedural knowledge.
There is a strong indication though, originating from research in productive failure and
invention learning approaches (DeCaro & Rittle-Johnson, 2012; Kapur, 2014; Loibl &
Rummel, 2014a,b), that problem solving–worked example sequences could be beneficial
for learning conceptual knowledge by novice learners. It is possible to speculate that in
case of conceptual knowledge, the initial problem solving attempts, even if unsuccessful,
could familiarize learners with some essential elements of the task, at least in isolated
rather than interactive form and without much meaning assigned at this stage. Such initial
rudimentary learning could allow reducing working memory load during the following
comprehensive explicit instruction phase. In cognitive load theory, this method has been
associated with the so-called isolated-interactive elements effect (Sweller et al., 2011).
Future empirical studies within a cognitive load framework may need to take a closer look
at this matter when investigating the effectiveness of alternative sequences of explicit
instruction and problem solving for learning conceptual knowledge.
Other limitations of this study are the absence of delayed post-tests and differential
retention and transfer tasks in the post-tests. Judging by the results of studies within
productive failure and invention learning frameworks, problem-first sequences might be
most effective when evaluated by delayed transfer tests. Such tests need to be included in
future studies, together with thorough analyses and comparisons of levels of element
interactivity of learning tasks.
In summary, the major theoretical implication of this study is that by considering the
factor of element interactivity, the results of the two reported experiments provide a
potential way of explaining the contradictory findings within different theoretical
frameworks in relation to the effectiveness of alternative sequences of worked examples
and problem solving tasks. For learning materials high in element interactivity, the worked
example–problem solving sequence is likely to be more effective, whereas, for materials
low in element interactivity, the problem solving–worked example sequence could be
superior.

Educational implications
The results of this study may inform educational practitioners that novice learners dealing
with complex information that is difficult to process in limited-capacity working memory
should initially be presented with sufficient instructional guidance, such as that provided
by worked examples. With increases in learner levels of expertise, a gradual shift to a
heavier emphasis on problem solving may be beneficial. Also, the teachers need to
carefully consider both the nature of learning materials and the levels of learner prior
knowledge when using worked example–problem solving and problem solving–worked
example instructional sequences.

Acknowledgement
Authors would like to thank the teachers and students, from SMP Negeri 5 Depok, Yogyakarta,
Indonesia, and SD Negeri Percobaan 2, Yogyakarta, Indonesia, participated in our experiments.
Also, authors would like to thank Marissa and Ibrohim, from Yogyakarta State University,
Yogyakarta, Indonesia, assisted in marking and data entry.
222 Ouhao Chen et al.

Ethics statement
This joint study is approved by the ethics committee at the institute of Research and
Community Service, Yogyakarta State University, Yogyakarta, Indonesia. The approval
numbers are 378/UN34.21/TU/2018 for Experiment 1 and 817/UN34.21/TU/2018 for
Experiment 2.

References
Chen, O., Kalyuga, S., & Sweller, J. (2015). The worked example effect, the generation effect, and
element interactivity. Journal of Educational Psychology, 107, 689–704. https://doi.org/10.
1037/edu0000018
Chen, O., Kalyuga, S., & Sweller, J. (2016a). Relations between the worked example and generation
effects on immediate and delayed tests. Learning and Instruction, 45, 20–30. https://doi.org/
10.1016/j.learninstruc.2016.06.007
Chen, O., Kalyuga, S., & Sweller, J. (2016b). When instructional guidance is needed. The
Educational and Developmental Psychologist, 33, 149–162. https://doi.org/10.1017/edp.
2016.16
Chen, O., Kalyuga, S., & Sweller, J. (2017). The expertise reversal effect is a variant of the more
general element interactivity effect. Educational Psychology Review, 29, 393–405. https://doi.
org/10.1007/s10648-016-9359-1
Chen, O., Retnowati, E., & Kalyuga, S. (2019). Effects of worked examples on step performance in
solving complex problems. Educational Psychology, 39, 188–202. https://doi.org/10.1080/
01443410.2018.1515891
Cooper, G., & Sweller, J. (1987). Effects of schema acquisition and rule automation on mathematical
problem-solving transfer. Journal of Educational Psychology, 79, 347–362. https://doi.org/10.
1037/0022-0663.79.4.347
Cowan, N. (2001). Metatheory of storage capacity limits. Behavioral and Brain Sciences, 24, 154–
176.
DeCaro, M. S., & Rittle-Johnson, B. (2012). Exploring mathematics problems prepares children to
learn from instruction. Journal of Experimental Child Psychology, 113, 552–568. https://doi.
org/10.1016/j.jecp.2012.06.009
Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102,
211–245. https://doi.org/10.1037/0033-295X.102.2.211
Glogger-Frey, I., Fleischer, C., Gr€ uny, L., Kappich, J., & Renkl, A. (2015). Inventing a solution and
studying a worked solution prepare differently for learning from direct instruction. Learning
and Instruction, 39, 72–87. https://doi.org/10.1016/j.learninstruc.2015.05.001
Glogger-Frey, I., Gaus, K., & Renkl, A. (2017). Learning from direct instruction: Best prepared by
several self-regulated or guided invention activities? Learning and Instruction, 51, 26–35.
https://doi.org/10.1016/j.learninstruc.2016.11.002
Hsu, C.-Y., Kalyuga, S., & Sweller, J. (2015). When should guidance be presented in physics
instruction? Archives of Scientific Psychology, 3, 37–53. https://doi.org/10.1037/arc0000012
Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction.
Educational Psychology Review, 19, 509–539. https://doi.org/10.1007/s10648-007-9054-3
Kapur, M. (2014). Productive failure in learning math. Cognitive Science, 38, 1008–1022. https://d
oi.org/10.1111/cogs.12107
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not
work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and
inquiry-based teaching. Educational psychologist, 41, 75–86. https://doi.org/10.1207/
s15326985ep4102_1
Leppink, J., Paas, F., Van Gog, T., van Der Vleuten, C. P., & Van Merrienboer, J. J. (2014). Effects of
pairs of problems and examples on task performance and different types of cognitive load.
Learning and Instruction, 30, 32–42. https://doi.org/10.1016/j.learninstruc.2013.12.001
 Worked example–problem solving and problem solving–worked example 223

Loibl, K., & Rummel, N. (2014a). The impact of guidance during problem-solving prior to instruction
on students’ inventions and learning outcomes. Instructional Science, 42, 305–326. https://doi.
org/10.1007/s11251-013-9282-5
Loibl, K., & Rummel, N. (2014b). Knowing what you don’t know makes failure productive.
Learning and Instruction, 34, 74–85. https://doi.org/10.1016/j.learninstruc.2014.08.004
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for
processing information. Psychological Review, 63, 81–97. https://doi.org/10.1037/h0043158
Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A
cognitive-load approach. Journal of Educational Psychology, 84, 429–434. https://doi.org/10.
1037/0022-0663.84.4.429
Paas, F., & Van Merri€enboer, J. J. (1994). Variability of worked examples and transfer of geometrical
problem-solving skills: A cognitive load approach. Journal of Educational Psychology, 86, 122–
133. https://doi.org/10.1037/0022-0663.86.1.122
Peterson, L., & Peterson, M. J. (1959). Short-term retention of individual verbal items. Journal of
Experimental Psychology, 58, 193–198. https://doi.org/10.1037/h0049234
Reisslein, J., Atkinson, R. K., Seeling, P., & Reisslein, M. (2006). Encountering the expertise reversal
effect with a computer-based environment on electrical circuit analysis. Learning and
Instruction, 16, 92–103. https://doi.org/10.1016/j.learninstruc.2006.02.008
Schwonke, R., Renkl, A., Krieg, C., Wittwer, J., Aleven, V., & Salden, R. (2009). The worked-example
effect: Not an artefact of lousy control conditions. Computers in Human Behavior, 25, 258–
266. https://doi.org/10.1016/j.chb.2008.12.011
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York, NY: Springer.
Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction,
12, 185–233. https://doi.org/10.1027/s1532690xci1203_1
Sweller, J., & Cooper, G. (1985). The use of worked examples as a substitute for problem solving in
learning algebra. Cognition and Instruction, 2, 59–89.
Sweller, J., & Paas, F. (2017). Should self-regulated learning be integrated with cognitive load theory?
A commentary. Learning and Instruction, 51, 85–89. https://doi.org/10.1016/j.learninstruc.
2017.05.005
Sweller, J., & Sweller, S. (2006). Natural information processing systems. Evolutionary Psychology,
4, 434–458. https://doi.org/10.1177/147470490600400135
Tindall-Ford, S., Chandler, P., & Sweller, J. (1997). When two sensory modes are better than one.
Journal of Experimental Psychology: Applied, 3, 257–287. https://doi.org/10.1037/1076-
898X.3.4.257
Van Gog, T., Kester, L., & Paas, F. (2011). Effects of worked examples, example-problem, and
problem-example pairs on novices’ learning. Contemporary Educational Psychology, 36, 212–
218. https://doi.org/10.1016/j.cedpsych.2010.10.004

Received 4 March 2019; revised version received 24 July 2019