Lin, H. (2011).

Facilitating Learning from Animated Instruction: Effectiveness of Questions and Feedback as Attention-directing
Strategies. Educational Technology & Society, 14 (2), 3142.

Facilitating Learning from Animated Instruction: Effectiveness of Questions

and Feedback as Attention-directing Strategies
Huifen Lin
National Kaohsiung Normal University, Kaohsiung City 802, Taiwan // [email protected]
ABSTRACT
The purpose of this study was to investigate the relative effectiveness of different types of visuals (static and
animated) and instructional strategies (no strategy, questions, and questions plus feedback) used to complement
visualized materials on students learning of different educational objectives in a computer-based instructional
(CBI) environment. Five hundred eighty-two (N = 582) undergraduate students enrolled in an eastern university
in the United States participated in the study. Students were randomly assigned to treatments and after
interacting with their respective treatments, they received four individual criterion posttests to measure
achievement of different educational objectives. Data analysis consisted of two phases. The first analyzed data
that included all items in the four criterion posttests (80 items) plus a composite score. The second phase
analyzed only the 34 enhanced items complemented by different instructional strategies and animation. Results
indicated that students who received the animated visual treatment scored significantly higher on all criterion
posttests than those who received the static visual treatment consistently for both phases of analysis. For the
instructional strategy, students who received questions plus feedback or questions in their treatment scored
significantly higher than those who received no strategy on selective criterion measures.

Keywords
Visualization, Animation, Questions, Feedback

Introduction
Recent technological advances have made possible individualized learning opportunities that integrate multiple ways
of combining such media devices as audio, varied types of visuals, graphics, and sounds. There has been a long
history of using visualization to complement textual material (Feaver, 1977; Slythe, 1970; Anglin, Vaez, &
Cunningham, 2004). Research findings have generally supported the proposition that human beings remember
pictures better than words (Anglin et al., 2004). Human memory is composed of two interdependent types of
memory mode to process and store information the verbal and nonverbal modes. Paivio (1990) has indicated that
the dual coding of pictures both in its verbal and nonverbal forms is more likely to occur than words, which are more
likely to be encoded verbally only. This hypothesis is presented to explain the superior effect of pictures to words
when used in instruction.
Animation has been used in various disciplines to deliver instructional material that is hard to present alone using
static visuals or that contains content that is highly abstract or invisible to human eyes. Animation, presented as
pictures in motion, is analogous to a subset of visual graphics (Weiss, Knowlton, & Morrison, 2002). In a computerbased instructional (CBI) environment, animation is typically used due to its inherent characteristics that facilitate
the instructional and learning processes. Animation also has the potential to provide feedback in various forms that
may be both entertaining and motivating to learners striving for the correct response.
Different types of questions or questioning strategies can be used to engage learners in deeper cognitive information
processing and therefore enhance their learning. King (1992) indicated that having students ask and answer highlevel questions facilitates their comprehension of the text material by engaging them in tasks such as . . . focusing
attention, organizing the new material, and integrating the new information with existing knowledge (p. 304).
The importance of feedback in the learning process has long been recognized, and feedback has been a variable of
interest in educational research. During a learning process, feedback generally plays a role as a motivator or
incentive to encourage accurate performance or as an information confirmer that learners can use to judge the
correctness of a previous response. In terms of its purpose, feedback has both reinforcing and informational
attributes. It is believed that letting learners know how well they are performing a task and that giving them the
opportunity to monitor or assess their learning progress can result in a better learning effect (Kulhavy & Wager,
1993).
ISSN 1436-4522 (online) and 1176-3647 (print). International Forum of Educational Technology & Society (IFETS). The authors and the forum jointly retain the
copyright of the articles. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies
are not made or distributed for profit or commercial advantage and that copies bear the full citation on the first page. Copyrights for components of this work owned by
others than IFETS must be honoured. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior
specific permission and/or a fee. Request permissions from the editors at [email protected].

31

Dynamic visualized materials created in an interactive learning environment always depend on learners actions
and . . . active learner engaged processing of learning materials . . . (Kalyga, 2007, p. 387). Cognitive load theory
(CLT) originated in the 1980s, heavily relies upon theories drawn from cognitive architecture and the memory
system of human beings. It provides instructional designers with theory-based guidelines for designing instructional
materials. Researchers conducting studies on effectiveness of animation or simulation-based instruction recognized
and discussed their findings mostly from a cognitive load perspective, especially when the cognitive load was
associated with the level of interactivity of learners engaged in the learning process (Paas, Van Gerven, & Wouters,
2007; de Koning, Tabbers, Rikers, & Paas., 2007; Moreno, 2007; Lusk & Atkinson, 2007). These studies have used
this framework to establish the conditions and methods for enhancing the effectiveness and efficiency of animated
instruction (Kalyga, 2007). Major findings of animated instruction design employing a cognitive load approach
included examinations of the learner differences and design principles to optimize the effect of animated instruction.
For example, Cohen and Hegartys study revealed that accuracy of mental representation of animated visuals greatly
depends on learners spatial abilities. Students with high-level spatial ability are more likely than their low-level
counterparts to interpret animated visuals more correctly and efficiently (Cohen and Hegarty, 2007).
Design principles of animated instruction have also focused on techniques to reduce cognitive load. Techniques that
have been discussed in previous studies include the employment of learner control of the pace of instruction rather
than a system-paced instruction (Hasler, Kersten, & Sweller, 2007). Research has also suggested that attention
cueing in the animation and embedded animated pedagogical agents, designed to direct students attention to relevant
visual information, also reduced extraneous cognitive load (Ayres & Paas, 2007; de Koning et al., 2007). Guideddiscovery principle is another design principle that has been utilized to develop animated instruction (De Jong, 2005;
Plass et al., 2009). Questions and feedback embedded in a computer-based animated instruction is one example that
follows guided-discovery principle. Moreno (2004) and Moreno and Mayer (2005) indicated that corrective feedback
is less effective than explanatory feedback in supporting retention and far transfer. The former simply informed users
whether they were right or wrong, and the latter provided relevant explanations (Plass, Homer, & Hayward, 2009).
In this paper, the author draws on cognitive load theories and guided-discovery principle to design both static and
animated visualized materials. By inserting questions and feedback into segments of the visualized materials, the
study further compared the relative effectiveness of such strategies in enhancing learning from both types of
visualized materials.

Statement of the problem

Although there is increasing interest in conducting animated visual research, there has been little work done to
precisely specify what educational objectives animated visuals are most effective in facilitating. There is a need to
specify the levels of learning outcomes that animated visuals are most effective in improving due to the high cost
associated with the development of animated instruction. Furthermore, a series of previous studies has shed light on
factors that may have undermined the effectiveness of animation (Owens & Dwyer, 2005; Wilson & Dwyer, 2001;
Rieber & Boyce, 1990; Lin, Kidwai, Munyofu, Swain, Ausman, & Dwyer, 2005). Researchers indicated that
learners, when presented with animated instruction, were not able to . . . effectively attend to the animation or were
. . . distracted by the combination of visual and verbal information presented to them (Rieber, 1990, p. 81). Owens
& Dwyer (2005) also found that learners failed to focus on critical aspects of the animation that depicted important
concepts.
Furthermore, unlike studies that employed animated visuals alone to examine their effects with different designs, this
study incorporated a comparison group using static visuals. As Ayres and Paas (2007) have argued, the effectiveness
of static visuals could be enhanced and therefore might be more effective than animation when additional techniques
are used. Based on previous research findings and suggestions, this study employed varied instructional strategies to
accompany animated and static visuals instruction to scaffold students. The primary purpose of this study was to
investigate the effect of varied types of visuals (static and animated) on students learning of different educational
objectives in a CBI environment. The study also investigated the relative effectiveness of using varied instructional
strategies (questions and feedback) used to complement static and animated visualization on students learning.

32

Methods
Participants
The participants of the study consisted of 582 undergraduate students enrolled in an eastern university in the United
States. They were recruited from a number of classes and majored in different disciplines, such as education,
engineering, physics, statistics, etc. Among them, 324 participants were female and 258 male. The components of the
participants were as follows: 13% freshmen (n = 77); 29% sophomores (n = 169); 35% juniors (n = 202); and 23%
seniors (n = 134). Participation was voluntary, and students received course credits for their participation.

Research design
This study employed a posttest only, a 2 x 3 factorial experimental design. The two independent variables were
visual type and instructional strategy. The dependent variables were four criterion posttests measuring differences in
subjects understanding of the materials after being exposed to the learning materials. The first independent variable,
that is, visual type, consisted of two levels: static visuals or animated visuals. The second independent variable, that
is, instructional strategy, was comprised of three levels: no strategy, questions, and questions plus feedback. Figure 1
describes the research design employed in the study.
No strategy
(n = 97)

Static visuals
(n = 291)

Questions
(n = 97)
Questions + feedback

(n = 97)

582
students

Four
criterion
posttests

No strategy
(n = 97)

Animated visuals
(n = 291)

Questions
(n = 97)
Questions + feedback

(n = 97)

Figure 1. 2 x 3 factorial-posttest-only research design

Computer-based instructional module

The instructional material used in this study, originally presented in print material, was modified into a computerbased format. The instructional module consisted of five units dealing with physiological knowledge of the human
heart. Content for each unit was presented in texts supported by either static or animated visuals. Enhancement
strategies, that is, questions or questions plus feedback, were integrated within each frame to facilitate learning. The
total number of instructional frames for the module was 20.

Pilot study
A pilot study employing the identical criterion measures and material was conducted using participants with similar
background as the main study. The purpose of the pilot study was to effectively develop animated visuals that would
facilitate the students learning and comprehension of the treatment instructional material. Based on the result of the
33

pilot study, difficult and complex concepts presented in the material were identified by item analyses and were later
on complemented by questions and feedback.

Respective treatment groups

Group One: Animation alone. Participants assigned to this treatment group received instructional material that
contained text and animated visuals in selective frames. In total, 14 animation sequences, developed to address 34
difficult items, were embedded in these frames. Students in this treatment group were instructed to first read the text
carefully and then interact with the animation. Each animated sequence was developed to complement a portion of
the text and to facilitate understanding of complex concepts that were found in the pilot studies to be difficult to
comprehend with static visuals only. Figure 2 provides a screenshot of the instructional frame containing an
animated visual.

Figure 2. Instructional frame containing an animation sequence

Group Two: Animation + questions. Participants assigned to this treatment group received instructional material that
contained text, animated visuals, and questions in selective frames. Students in this treatment group received exactly
the same instructional material as did the animation alone group; however, 32 questions addressing 34 difficult
items were embedded after the 14 frames to reinforce students comprehension and acquisition of the difficult
knowledge contained in previous frames.
Group Three: Animation + questions + feedback. The instructional material used for the treatment group contained
the text, animated visuals in 14 frames, questions following these 14 frames, and corresponding feedback. After
viewing the animation, students proceeded to a frame that contained a question. Participants needed to make an overt
response to each question. In addition, after a response was submitted, feedback on the correctness of the response
was displayed as either incorrect or correct. A short sentence elaborating upon the correct answer was provided
as well.
Group Four: Static visuals alone. The instructional module received by participants in this treatment group consisted
of text and static visuals used to complement the text. In total, there were 20 static visuals in this module, with one
static visual accompanying each instructional frame. The 20 visuals matched those in the previously described
treatments, with all being static in this treatment. Figure 3 provides a screenshot of the instructional frame that
contained an animated visual.
Group Five: Static visuals + questions. Participants in this treatment group received exactly the same instructional
material as that received by the static visuals alone treatment group; however, an additional instructional strategy
was embedded in the instructional module. Questions following the same 14 instructional frames as those in the
animated visual treatments were provided to students in this group. Students were required to read the question
34

carefully, recall what they had learned from previous frames, and choose a correct answer. There was no feedback in
regard to the correctness of the submitted response.
Group Six: Static visuals + questions + feedback. The instructional module received by this treatment group was
exactly the same as that received by the static visuals + questions group; however, the students received feedback
on their responses immediately after submission. The feedback, presented in the same format and with the same
amount of information as the animated visuals + questions +feedback treatment group, first assessed the students
submitted responses as either correct or incorrect, then provided a simple elaboration of the correct response.

Figure 3. Instructional frame containing a static visual

Criterion measures
Four criterion measures developed by Dwyer (1972) were used to assess students understanding and achievement of
the instructional material. These four criterion tests measured different learning outcomes in educational technology
area, such as facts, concepts, rules/procedures, comprehension, and problem solving. Each criterion test consisted of
20 items. All but the drawing test, terminology, identification, and comprehension tests consisted of 20 multiplechoice questions. Cronbachs alpha coefficients () were calculated to establish the reliability and internal
consistency of the dependent variables in this study. A detailed description of the criterion measures is summarized
below.
Drawing Test ( = .98).The purpose of the drawing test was to measure students overall understanding of the
instructional material as well as their ability to . . . reproduce the parts of the heart in their appropriate context . . .
(Dwyer, 1994, p. 391). This criterion test was developed to assess specifically the level of intellectual skills/concept
learning regarding the instructional module according to the types of learning outcomes developed by Gagne (1985).
Each student was provided with a blank piece of paper on which they were required to draw a simple diagram of the
human heart.
Identification Test ( = .87).The purpose of the identification test was to assess the students ability to identify parts
of the human heart. The level of knowledge measured in this test was verbal information based on Gagnes types of
learning outcomes (1985). In this test, a diagram of the human heart with 20 numbered arrows was provided to
students, who had to then choose the corresponding letter to the numbered arrow from four possible answer choices.
Terminology Test ( = .84).The terminology test measured several levels of learning including verbal information,
concepts, and rules/procedures. The students knowledge of specific terms of the human heart and their association
with various functions of the human heart were assessed.
35

Comprehension Test ( = .84).The test measured a higher-level learning outcome; the mastery of this learning
outcome would require the students competent acquisition of knowledge concerning facts, rule/procedures,
concepts, and problem solving pertaining to the instruction.
Total Composite Score ( = .96).Two composite scores were calculated. One composite score was calculated by
adding the separate scores of all items on the drawing, identification, terminology, and comprehension tests. Another
total composite score was calculated by adding the separate score of enhanced items only on the drawing,
terminology, and comprehension tests.

Data analysis
Two phases of analyses were conducted in the study to answer the research questions. The data analysis in the first
phase investigated the effectiveness of respective treatments by comparing the participants achievement scores on
all 80 items contained in the four criterion posttests and a composite score based on these 80 items. The second phase
of analysis focused on the 34 enhanced items, nine (9) items in the drawing test, none (0) in the identification test,
twelve (12) in the terminology test, thirteen (13) in the comprehension test was identified in the pilot study as
difficult. The latter analysis aimed to assess students achievement in those portions of the instructional module in
which animated visuals, questions, and feedback were included.

Analysis based on 80 enhanced items

The drawing test (number of items = 20).ANOVA source data indicated that the interaction between the visual type
and instructional strategy was not statistically significant: F = .352, df = 2/576, p = .704. The main effect of the
visual type was significant, F = 25.452, df = 1/576, p = .000. Participants receiving the animated visual treatment (M
= 12.66; SD = 5.80) scored significantly higher in the drawing test than did participants receiving the static visual
treatment (M =10.22; SD = 5.89). However, the main effect of the instructional strategy was not significant, F = .991,
df = 2/576, p = .372.
The identification test (number of items = 20). ANOVA conducted on the identification test indicated that the
interaction between the visual type and instructional strategy was not statistically significant: F = .655, df = 2/576, p
= .520. The main effect of the visual type was significant, F = 20.716, df = 1/576, p = .000. However, the main effect
of instructional strategy was not significant, F = .154, df = 2/576, p = .857. Participants receiving the animated visual
treatment (M = 14.51; SD = 4.71) scored significantly higher in the identification test than did participants receiving
the static visual treatment (M = 12.70; SD = 4.85).
The Terminology Test (number of items = 20). The interaction between the visual type and instructional strategy was
not statistically significant: F = 2.026, df = 2/576, p = .133. However, the main effects for both the visual type and
the instructional strategy were significant, for the visual type, F = 4.706, df = 1/576, p = .030, and for the
instructional strategy, F = 5.969, df = 2/576, p = .003. An inspection of the means for the static visual and the
animated visual treatment groups indicated that animated visual (M = 12.09; SD = 4.80) significantly outperformed
static visuals (M = 11.25; SD = 4.66). For the main effect of the instructional strategy, post hoc tests indicated that
the questions + feedback treatment (M = 12.30, SD = 4.88) significantly outperformed the no strategy treatment
(M = 10.74, SD = 4.49), and the difference is significant at the .003 level. In addition, the questions treatment (M =
11.97; SD = 4.74) also significantly outperformed the no strategy treatment (M = 10.74; SD = 4.49), and the
difference was significant at the .026 level. The observed differences between the questions + feedback and the
questions treatments were not significant at the .05 level.
The Comprehension Test (number of items = 20). The interaction between the visual type and the instructional
strategy was not statistically significant: F = 1.685, df = 2/576, p = .186. However, the main effects for both the
visual type and the instructional strategy were significant, for visual type, F(1,576) = 8.789; p = .003, and for the
instructional strategy, F = 4.154, df = 2/576, p = .016. An inspection of the means for the static visual and the
animated visual treatment groups indicated that animated visual (M = 11.63; SD = 4.64) significantly outperformed
static visuals (M = 10.47; SD = 4.81). For the main effect of the instructional strategy, post hoc tests indicated that
the questions + feedback treatment (M = 11.66; SD = 4.64) significantly outperformed the no strategy treatment
36

(M = 10.30, SD = 4.72), and the difference was significant at the .005 level. The observed differences between the
questions + feedback and the questions treatments and the questions and the no strategy treatments were not
significant at the .05 level.
The composite score (number of items = 80). The interaction between the visual type and the instructional strategy
was not statistically significant: F = 1.063, df = 2/576, p = .346. However, the main effect for the visual type was
significant, F = 17.235, df = 1/576, p = .000. The main effect for the instructional strategy was not significant, F =
1.388, df = 2/576, p = .250. An inspection of the means for the static visual and the animated visual treatment groups
indicated that animated visual (M = 50.89; SD = 18.33) significantly outperformed static visuals (M = 44.64; SD =
18.03).

Summary of results
Table 1 presents a summary of results for the data analysis of the learning achievement of all items based on visual
type. As indicated, differences on all criterion tests between static and animated visuals were significantly in favor of
animated visuals.
Table 1. Results of all items based on visual type
Measures
Static (S)
Animated (A)
Drawing
10.22a (5.89)b
12.66 (5.80)
Identification
12.70 (4.85)
14.51 (4.71)
Terminology
11.25 (4.66)
12.09 (4.80)
Comprehension
10.47 (4.81)
11.63 (4.64)
Composite
44.64 (18.03)
50.89 (18.33)
a
Mean score. b Value in the parentheses is the standard deviation.
*p < .05. **p < .01. ***p < .001.

Result
A>S
A>S
A>S
A>S
A>S

Sig.
.000***
.000***
.030*
.003**
.000***

Table 2 summarizes the results of the first phase analysis based on instructional strategies. As indicated, both
questions and questions + feedback were significantly more effective than no strategy in the terminology test,
and questions + feedback was significantly more effective than no strategy in the comprehension test.

Measures
Drawing
Identification
Terminology

Table 2. Results of all items based on instructional strategy

QS
QF
NOa
11.56b (5.91)c
10.97 (6.06)
11.78 (5.93)
13.72 (4.78)
13.46 (4.97)
13.64 (4.85)
10.74 (4.49)
11.97 (4.74)
12.30 (4.88)

Comprehension
10.30 (4.72)
11.18 (4.84)
Composite
46.33 (17.97)
47.59 (18.79)
a
NO = No Strategy. QS = Questions. QF = Questions + Feedback.
b
Mean score. c Value in the parentheses is the standard deviation.
d
p > =.05 * p < .05. **p < .01.

11.66 (4.64)
49.39 (18.48)

Result
nsd
ns
QF > NO
QS > NO
QF > NO
ns

Sig.
.372
.857
.003**
.026*
.005**
.250

Analysis based on 34 enhanced items

The second phase of the data analysis was focused on the items for which the instructional strategy and animation
were particularly designed to improve achievement. As with the first phase of data analysis, a two-way ANOVA was
conducted to compare the mean scores of the enhanced items in each dependent variable among the treatment
groups.
The drawing test (number of items = 9). ANOVA results indicated that the interaction between the visual type and
the instructional strategy was not statistically significant: F(2/576) = .042, p = .959. However, the main effect of the
visual type was significant, F = 38.328, df = 1/576, p = .000. The main effect of the instructional strategy was not
37

significant, F = 1.147, df = 2/576, p = .318. Participants receiving the animated visual treatment ( M = 5.10; SD =
3.04) scored significantly higher on the enhanced items in the drawing test than did the participants receiving the
static visual treatment ( M = 3.57; SD = 2.92).
The terminology test (number of items = 12). The interaction between the visual type and the instructional strategy
was not statistically significant: F = 1.392, df = 2/576, p = .249. However, the main effects for both the visual type
and the instructional strategy were significant: for the visual type, F = 4.140, df = 1/576, p = .042, and for the
instructional strategy, F = 7.603, df = 2/576, p = .001. An inspection of the means for the static visual and animated
visual treatment groups indicated that the animated visual (M = 6.67; SD = 3.13) significantly outperformed the static
visuals (M = 6.16; SD = 3.05). For the main effect of the instructional strategy, post hoc tests indicated that the
questions + feedback treatment (M = 6.87; SD = 3.26) significantly outperformed the no strategy treatment (M =
5.73; SD = 2.84), and the difference was significant at the .001 level. In addition, the questions treatment (M =
6.64; SD = 3.08) also significantly outperformed the no strategy treatment (M = 5.73; SD = 2.84), and the
difference was significant at the .01 level.
The comprehension test (number of items = 13). The interaction between the visual type and the instructional
strategy was not statistically significant: F = .863, df = 2/576, p = .423. However, the main effect for both the visual
type and the instructional strategy was significant: for the visual type, F = 6.215, df = 1/576, p = .013; and for the
instructional strategy, F = 3.397, df = 2/576, p = .034. An inspection of the means for the static visual and the
animated visual treatment groups indicated that the animated visual (M = 6.87; SD = 2.81) significantly
outperformed the static visuals (M = 6.29; SD = 2.87). For the main effect of the instructional strategy, post hoc tests
indicated that the questions + feedback treatment (M = 6.91; SD = 2.80) significantly outperformed the no
strategy treatment (M = 6.18; SD = 2.84), and the difference was significant at the .02 level.
The composite score (number of items = 34). The interaction between the visual type and the instructional strategy
was not statistically significant: F = .863, df = 2/576, p = .423. However, the main effects for both the visual type and
the instructional strategy were significant: for the visual type, F = 16.889, df = 1/576, p = .000; and for the
instructional strategy, F = 3.569, df = 2/576, p = .029. An inspection of the means for the static visual and the
animated visual treatment groups indicated that the animated visual (M = 18.64; SD = 7.88) significantly
outperformed the static visuals (M = 16.01; SD = 7.64). For the main effect of the instructional strategy, post hoc
tests indicated that the questions + feedback treatment (M = 18.34; SD = 7.91) significantly outperformed the no
strategy treatment (M = 16.25; SD = 7.61), and the difference was statistically significant at the .021 level.

Summary of results
Table 3 presents a summary of results for the data analysis of the learning achievement of enhanced items based on
visual type. As indicated, the differences on all criterion tests between the static and animated visuals were
significantly in favor of the animated visuals.
Table 3. Results of the enhanced items based on visual type
Static (S)
Animated (A)
Result
Sig.
Measures a
Drawing
3.57b (2.92)c
5.10 (3.04)
A>S
.000**
Terminology
6.16 (3.05)
6.67 (3.13)
A>S
.042*
Comprehension
6.29 (2.87)
6.87 (2.81)
A>S
.013*
Composite
16.01 (7.64)
18.64 (7.88)
A>S
.000***
a
Maximum score for the drawing test, 9; terminology test, 12; comprehension test, 13; composite score, 34.
b
Mean score. c Value in parentheses is the standard deviation.
*p < .05. **p < .01. ***p < .001.
With regard to learning achievement on the enhanced items, based on the instructional strategy, Table 4 indicates
that questions + feedback was a significantly more effective instructional strategy than no strategy in facilitating
achievement in the terminology, the comprehension test, and the composite test.

38

Table 4. Results of enhanced items based on instructional strategy

NO a
QS
QF
Result
Sig.
4.35b (3.15)c
4.10 (3.05)
4.56 (3.03)
nse
.318
5.73 (2.84)
6.64 (3.08)
6.87 (3.26)
QF > NO
.001**
QS > NO
.010*
Comprehension
6.18 (2.84)
6.66 (2.88)
6.91 (2.80)
QF > NO
.028*
Composite
16.25 (7.61)
17.39 (7.97)
18.34 (7.91)
QF > NO
.021*
a
NO = No Strategy. QS = Questions. QF = Questions + Feedback.
b
Mean score. c Value in the parentheses is the standard deviation.
d
The maximum score for the drawing test, 9; terminology test, 12; comprehension test, 13; composite score, 34.
e
p>=.05 * p<.05. **p<.01.
Measures d
Drawing
Terminology

Discussions
General findings
The major interest of this study concerned whether the provision of question and feedback would optimize students
learning in visualized instruction. The results of the study, however, indicate that questions with or without feedback
were equally effective in students learning from visualized instruction. This finding may suggest that the use of
feedback as simple as the one used in the study was not an effective instructional technique for increasing student
achievement of different types of learning objectives when accompanying visualized instruction.

Related work
Previous research has shown that embedded questions designed to focus student attention effectively reinforced
relevant learning cues. By responding to the embedded questions inserted before or after the text, students may
activate their prior knowledge, engage in deeper information processing, and therefore enhance their recall and
retention of the instructional material (Anderson & Pearson, 1984). The type of questions and feedback employed in
this study to enhance learning from visualized instruction are easy to construct in a CBI environment and, consequently, the instructional effect that can be expected is minimal. Other types of questions and more elaborate feedback may be used to produce more desirable learning effects. As Anderson and Biddle (1975) have indicated, . . .
higher order questions consistently increased both recall and application of information . . . (p. 122). In this study,
the provision of feedback to responses that students made to inserted questions seems not to have produced a
satisfactory instructional effect, which can be seen from both phases of the data analysis, which consistently showed
that the questions + feedback group did not significantly outperform the questions group in any posttests. This
finding was consistent with previous studies that found the no-feedback condition to be equally effective as
providing feedback of some type (Clariana, Ross, & Morrison, 1991; Clark & Dwyer, 1998; Pridemore & Klein,
1995; Kulhavy & Stock, 1989).
However, the finding also contradicted previous research that supported that providing some kind of feedback is
superior to no feedback at all (De Klerk & De Klerk, 1978). As suggested from research on feedback, many factors
contributed to the effectiveness of feedback. The type of feedback investigated in this study was analogous to the
combination of KCR and KOR. Learners were informed as to the correctness of their submitted responses as well as
a statement of the correct response. This kind of feedback is simple in nature and easy to construct in a CBI
environment. It is also associated with low cost in terms of instructional design and development. However, the
study found out that this type of feedback did not help students in the acquisition of even the simplest factual
knowledge, as reflected in their performance of the drawing and identification tests, which measured lower-level
learning outcomes. Other types of feedback, which are more dedicated and complex in nature and developed to
accompany visualization instruction to maximize students learning, may be more effective.
In regard to the effects of visual types, the findings of the study suggested the superior effectiveness of animated
visuals compared to static visuals. Students receiving the animated instructional treatment outperformed significantly
on all criterion posttests across both phases of the analysis than did students who received static visuals. The findings
were encouraging and consistent with previous studies that found significantly superior effects with animation than
39

with static visuals (Kaiser, Proffitt, & Anderson, 1985; Rieber, 1989; Rieber, Boyce, & Assad, 1990). Indeed, no
other studies conducted positioned treatments via item analysis. Animation, at its core nature, has attention-gaining
and entertaining features, which is believed to motivate student learning. Rieber (1990, 1991) indicated that the most
powerful and effective application of animation is in presenting instructional materials rather than focusing learners
attention or for its cosmetic function. The animation used in the study complemented textual material. The findings
of this study also contradicted previous studies that suggested that animation was no more effective than static
visuals (Caraballo, 1985; King, 1975; Moore, Nawrocki, & Simutis, 1979).

Conclusion
Mirzoeff (1998), as cited by Jeffers (2002), indicated that visual culture resides in everyday life and that to explore
visual culture, we need to understand how visuals have shaped our society, including how we visualize things and
how we learn visually. Anderson and McRorie (1997) further indicated that visual culture is determined by the
context in which visuals were made and used (Jeffers, 2002). Visuals have been well recognized in terms of their
function in facilitating knowledge construction and in shaping, reflecting, and representing our society. People in
post-modern society learn visually and most of the time, need to learn to visualize things that are not visual-based,
since visuals have occupied almost every aspect of our society and everyday life (Jeffers, 2002, p.157). Visuals have
dominated most instructional materials that we use and, therefore, instructional designers need to understand general
instructional principles that apply to all graphic materials. Rieber (2000) has suggested conditions that need to be
taken into consideration when designing graphics or visuals based instructional materials. First, the purposes that
graphics serve and the specific type of instructional objective each graphic is designed to achieve must be determined
prior to the design process and evaluated throughout the process. Second, the selection of the types of graphics
depends on the needs and the types of learner, the content to be delivered, and the characteristics of the learning task.
Thirdly, graphics should be designed to present a meaning context through which students can achieve the
predetermined learning objectives as well as feel highly motivated in the learning process. One important design
principle that Rieber (2000) pointed out that is especially relevant to this study is . . . the important relationship
between attention-gaining and presentation principles associated with graphics . . . (p. 222). As was revealed in
previous studies, students might not attend to relevant parts of a graphic presentation due to selective attention.
Therefore, . . . direct and overt directions to actively search for or use specific information in the visual . . . should
be provided to increase the chance that students would pay attention to a graphic. Such strategies also reduce
extraneous cognitive load resulting from irrelevant search and free working memory space for more intrinsic
learning.
Computers can be a powerful medium to design graphics for . . . attention-gaining and presentation purposes . . .
(Rieber, 2000, p. 224). However, much of the strengths of the computer in achieving above purposes have not been
explored. This study explored the effect of computer-based questions and feedback in complementing visuals (both
static and animated) to examine the relative effects. The significance of the study is two-fold. First, the results
provide instructional designers with empirical evidence that supports the use of varied types of visuals in a computerbased learning environment specifically to achieve designated instructional objectives. Second, although this study
did not intend to explore the effects of different types of feedback in learning, it examined a basic issue the
provision (or not) of feedback, specifically in a computer-based learning environment that used varied visuals. That
is, this study examined the notion that providing feedback to the responses would enhance learning from the
visualized instruction, especially with regard to animated visualized instruction.
The results of this study were affected by a particular system in which the author tested the hypothesis that questions
or questions plus feedback can be used to optimize learning from visualized instruction, in particular, animated
instruction. The learning outcomes of visualized materials integrated with questions or feedback are not superior to
those of visualized materials not integrated with questions or feedback. One plausible explanation might be that
simple textual feedback on the correctness of a submitted response was not effective in conveying information for
visualized materials. Another possible explanation might be that well-designed visualized materials, such as those in
this study, are self-contained and therefore no additional aids were necessary in conveying meanings.
Furthermore, this study found no effect of questions and feedback in facilitating student learning specifically from
animated visualization. Future research may investigate other types of cueing strategies such as advance organizers,
narrations, audios, or chunking. In addition, new and emerging technology has been becoming increasingly available
40

to offer more advanced types of instructional strategies that involve sounds, digital images, audio, and more.
Feedback certainly can be constructed combining these new advanced technology features. Future research on the
relative effectiveness of these multisensory instructional strategies is warranted.

References
Anderson, R. C., & Biddle, W. B. (1975). On asking people questions about what they are reading. In G. Bower (Ed.), Psychology
of Learning and Motivation, Vol. 9 (pp. 89132). New York: Academic Press.
Anderson, T., & McRorie, S. (1997). A role for aesthetics in centering the K2 art curriculum. Art Education, 50(23), 614.
Anderson, R. C. & Pearson, P. D. (1984). A schema-theoretic view of basic processes in reading comprehension. In D. Pearson
(Ed.), Handbook of reading research (pp. 255291). New York: Longman.
Anglin, G., Vaez, H., & Cunningham, K. (2004). Visual representations and learning: The role of static and animated graphics. In
D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp.865916). Mahwah, NJ:
Lawrence Erlbaum.
Ayres, P., & Paas, F. (2007). Making instructional animations more effective: A cognitive load approach. Applied Cognitive
Psychology, 21, 695700.
Caraballo, A. (1985). An experimental study to investigate the effects of computer animation on the understanding and retention
of selected levels of learning outcomes. Unpublished doctoral dissertation, The Pennsylvania State University.
Clariana, R. B., Ross, S. M., & Morrison, G. R. (1991). The effects of different feedback strategies using computer-administered
multiple-choice questions as instruction. Education Technology, Research and Development, 39 (2), 517.
Clark, K., & Dwyer, F. M. (1998). Effect of different types of computer-assisted feedback strategies on achievement and response
confidence. International Journal of Instructional Media, 25(1), 5563.
Cohen, C. A., & Hegarty, M. (2007). Individual differences in use of external visualizations to perform an internal visualization
task. Applied Cognitive Psychology, 21, 701711.
De Jong, T, (2005). The guided discovery principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of
multimedia learning (pp. 215228). New York: Cambridge.
De Klerk, L. F. W., & De Klerk, L. (1978). The effect of knowledge of correct results per item on verbal learning and retention,
Instructional Science, 7, 347358.
de Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2007). Attention cueing as a means to enhance learning from an
animation. Applied Cognitive Psychology, 21, 731746.
Dwyer, F. M. (1972). A guide for improving visualized instruction. State College, PA: Learning Services. Box 784. The
Pennsylvania State University.
Dwyer, F. M. (1994). One dimension of visual research: A paradigm and its implication. In D. Moore & F. M. Dwyer (Eds.),
Visual learning: A spectrum of visual learning (pp. 383401). Englewood Cliffs, NJ: Educational Technology Publications.
Feaver, W. (1977). When we were young: Two centuries of childrens book illustration. London: Thames & Hudson.
Gagne, R. M. (1985). The conditions of learning and theory of instruction. NewYork: Holt, Rinehart & Winston.
Hasler, B. S., Kersten, B., & Sweller, J. (2007). Learner control, cognitive load and instructional animation. Applied Cognitive
Psychology, 21, 713729.
Jeffers, C. (2002), Tools for exploring social issues and visual culture, In Y. Gaudelius and P. Speirs (Eds.), Contemporary issues
in art education (pp. 155168). Upper Saddle River, NJ: Prentice Hall.
Kaiser, M. K., Proffitt, D. R., & Anderson, K. (1985). Judgments of natural and anomalous trajectories in the presence and
absence of motion. Journal of Experimental Psychology: Learning, Memory and Cognition, 11, 795803.
Kalyga, S. (2007). Enhancing instructional efficiency of interactive e-learning environments: A cognitive load perspective.
Educational Psychology Review, 19, 387399.
King, W. A. (1975). A comparison of three combinations of text and graphics for concept learning. San Diego, CA: Navy
Personnel Research and Development Center.
King, A. (1992). Comparison of self-questioning, summarizing, and notetaking: Review as strategies for learning from lectures.
American Educational Research Journal, 29(2), 303323.
41

Kulhavy, R. W., & Stock, W. A. (1989). Feedback in written instruction: The place of response certitude. Educational Psychology
Review, 1, 279308.
Kulhavy, R. W., & Wager, W. (1993). Feedback in programmed instruction: Historical context and implications for practice. In
V. Dempsey & G. C. Sales (Eds.), Interactive instruction and feedback (pp. 320). Englewood Cliffs, NJ: Educational
Technology.
Lin, H., Kidwai, K., Munyofu, M., Swain, J., Ausman, B., & Dwyer, F. M. (2005). The effect of verbal advance organizers in
complementing animated instruction. Journal of Visual Literacy, 25(2), 237248.
Lusk, M. M., & Atkinson, R. K. (2007). Animated pedagogical agents: Does their degree of embodiment impact learning from
static or animated worked examples? Applied Cognitive Psychology, 21,747764.
Mirzoeff, N. (1998). Introduction to part two: Visual culture and everyday life. In N. Mirzoeff, (Ed.), The visual culture reader
(pp. 124129). New York: Routledge.
Moore, M., Naworcki, L., & Simutis, Z. (1979). The instructional effectiveness of three levels of graphics displays for computerassisted instruction. Arlington, VA: Army Research Institute for the Behavior and Social Sciences.
Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discoverybased multimedia. Instructional Science, 32(12), 99113.
Moreno, R. (2007). Optimizing learning from animations by minimizing cognitive load: Cognitive and affective consequences of
signaling and segmentation methods. Applied Cognitive Psychology, 21, 765781.
Moreno, R., & Mayer, R. (2005). Role of guidance, reflection, and interactivity in an agent-based multimedia game. Journal of
Educational Psychology, 97(1), 117128.
Owens, R., & Dwyer, F. M. (2005). The effect of varied cueing strategies in complementing animated visual imagery in
facilitating achievement of different educational objectives. International Journal of Instructional Media, 32(4), 373384.
Paas, F., Van Gerven, P. W. M., & Wouters, P. (2007). Instructional efficiency of animation: Effects of interactivity through
mental reconstruction of static key frames. Applied Cognitive Psychology, 21, 783793.
Paivio, A. (1990). Mental representations: A dual coding approach. New York: Oxford University Press.
Plass, J. L., Homer, B. D., & Hayward, E. O. (2009). Design factors for educationally effective animations and simulations.
Journal of Computing in Higher Education, 21, 3161.
Pridemore, D. R., & Klein, J. D. (1995). Control of practice and level of feedback in computer-based instruction. Contemporary
Educational Psychology 20, 444450.
Rieber, L. P. (1989). The effects of computer animated elaboration strategies and practice on factual and application learning in an
elementary science lesson. Journal of Educational Computing Research, 5(4), 431444.
Rieber, L. P. (1990). Animation in computer-based instruction. Educational Technology, Research and Development, 38 (1), 77
86.
Rieber, L. P. (1991). Effects of visual grouping strategies of computer-animated presentations on selective attention in science.
Educational Technology, Research and Development, 39(4), 515.
Rieber, L. P. (2000). Computers, graphics, & learning, The University of Georgia. Slythe, R. M. (1970). The art of illustration
17501900. London: The Library Association.
Rieber, L., & Boyce, M. (1990). The effects of computer animation on adult learning and retrieval tasks. Journal of Computerbased Instruction, 17, 4652.
Rieber, L. P., Boyce, M. J., & Assad, C. (1990). The effects of computer animation on adult learning and retrieval tasks. Journal
of Computer-based Instruction, 17(2), 4652.
Slythe, R. M. (1970). The art of illustration 17501900. London: The Library Association.
Weiss, R. E., Knowlton, D. S., & Morrison, G. R. (2002). Principles for using animation in computer-based instruction:
Theoretical heuristics for effective design. Computers in Human Behavior, 18, 465477.
Wilson, F., & Dwyer, F. M. (2001). Effect of time and level of visual enhancements in facilitating student achievement of
different educational objectives. International Journal of Instructional Media, 28, 159167.

42