Investigating Students Use and Evaluatio
Investigating Students Use and Evaluatio
Abstract. This study examined a group of English language learners’ use and
evaluation of digital videos as a form of computer assisted language learning
(CALL) material, as they were engaged to complete a video-based vocabulary
learning task using a CALL system. The results indicated that the students were
able to watch the assigned video clips for task completion with reasonable time
and effort, without any major learning difficulties. It was also found that the top
three most frequently used functionalities of the CALL system that enabled
learner-content interaction were video subtitles, followed by the built-in dic-
tionary, and adjustable playback speed. On the whole, the students considered
the way that the video material was integrated in the vocabulary learning task
was facilitative and capable of enhancing their English learning experiences,
indicating the overall appropriateness of such video use.
1 Introduction
The widespread use of computers and technology in the learning and teaching of
vocabulary has led to the emergence of Computer Assisted Vocabulary Learning
(CAVL) in the broader context of Computer Assisted Language Learning (CALL).
Historically, there are three developmental stages of CALL, including behavioristic
CALL (with an emphasis on drills and practices from 1970s to 1980s), communicative
CALL (with an emphasis on communicative tasks facilitated through CALL materials
from 1980s to 1990s), and integrative CALL (with an emphasis on language use in a
meaningful and authentic context in the first two decades of the 21st century) [1]. As a
subfield of CALL, the development of CAVL has arguably followed a similar path. At
present, the development of CALL and CAVL has progressed to the integrative stage.
According to a review article on CALL research [2], various forms of CALL tools or
activities have now been integrated in the teaching and learning of all four skills (i.e.,
reading, writing, listening, and speaking) and three language areas (i.e., pronunciation,
vocabulary, and grammar). As for CAVL, common tools or activities used to assist
© Springer International Publishing AG 2017
T.-T. Wu et al. (Eds.): SETE 2016, LNCS 10108, pp. 225–239, 2017.
DOI: 10.1007/978-3-319-52836-6_24
226 H.-T. Hung et al.
2 Theoretical Framework
This study draws upon the cognitive theory of multimedia learning (CTML), proposed
by Mayer and his colleagues [15, 16], to highlight the use of multimedia material in the
field of CALL. The CTML is probably the most influential theory for learning via
multimedia. This theory explains how the presentation of verbal information (e.g.,
words, narration, or printed text) and visual information (e.g., illustrations, images,
animations, or videos) interacts with learners’ sensory memory, working memory, and
long-term memory in learning. As applied to the learning of vocabulary in a multi-
media environment, to achieve effective learning, “the learner must engage in five
cognitive processes: (1) selecting relevant words for processing in verbal working
memory, (2) selecting relevant images for processing in visual working memory,
(3) organizing selected words into a verbal mental model, (4) organizing selected
images into a visual mental model, and (5) integrating the verbal and visual repre-
sentations” ([17], pp. 70–71). Briefly, the CTML argues that learning is more likely to
occur when learners can build meaningful connections between the verbal and visual
presentations of information [18].
The CTML has been used as a theoretical basis by many studies [5, 6, 9, 19]. Mayer
and his colleagues demonstrated in a series of works that the integration of words and
pictures can result in better learning [17, 20, 21], and thus it was concluded that
“people learn better from words and pictures than from words alone” ([22], p. 548).
Video, inherent with these two forms of multimedia, thus appears to be a promising
form of multimedia learning material for language learning [18, 23].
Investigating Students’ Use and Evaluation of Video 227
3 Method
3.1 Research Questions
To contribute to the CAVL literature in CALL contexts, the present study adopted the
theoretical framework of CTML to explore a group of English language learners’ use
and evaluation of online or digital videos as a form of CALL material, as they were
engaged to complete a video-based vocabulary learning task using a CALL system.
The following research questions were addressed in this study: (1) To what extent did
the students use the video material in the CALL system for the completion of the
video-based vocabulary learning task? (2) How did the students evaluate the overall
appropriateness of using video to learn English in the CALL system?
3.2 Participants
The present study was conducted at a university in Taiwan. Two intact classes of
sophomore students enrolled in an English language course were recruited, and 47 out
of the 48 enrolled students agreed to voluntarily take part in this study. There were 12
males and 35 females who participated in the study, and most of them had been
learning English as a foreign language (EFL) for more than 10 years before their
participation in this study. Nearly half of them had experience of learning English with
CALL systems featuring multimedia content, or something similar, although the
majority had never done so before.
• Video subtitles: The students can watch the videos along with synchronized sub-
titles either in English, Chinese, or both languages in parallel.
• Synchronized transcripts: The students can read transcripts of the videos to look up
certain vocabulary items. The transcripts can also be synchronized to play along
with the related videos.
• Built-in dictionary: The built-in dictionary can be used at all times with both the
video subtitles and synchronized transcripts. By clicking on any vocabulary item, a
pop-up window will then appear and the students can thus see its pronunciation,
English definition, example sentence, and synonyms.
• Word lists: All the vocabulary items and phrases that the students mark for each
video will be added automatically to the students’ personal word lists.
CALL System Logs. During the study period, the students had to watch a number of
assigned videos using the CALL system. The students’ online learning behaviors, such
as the time spent on the CALL task and the number of video clips watched, were
recorded in the system and retrieved for later analysis.
Student-Generated Word Lists. The CALL system allows the students to add
vocabulary items to their own word lists by clicking on any selected words. The
students could then review vocabulary items based on the alphabetical order of the
words, the date when they were entered into the system, or the number of times they
had been viewed, in whatever way best suited their needs.
Individual Student Interviews. To gain more insights into the students’ learning
experiences in this study, all 47 students were invited to participate in interviews. The
interviews were conducted individually in Chinese, the participants’ first language,
which allowed them to express their ideas fluently. Each interview lasted for an average
of 12 min. All the interviews were audio recorded and then transcribed and translated
into English for the content analysis. The interview protocol can be broadly divided
into two parts, involving seven generic questions with related prompts. The first part of
the interview contained six questions developed based on a framework for CALL
evaluation [24] to elicit student perspectives. This framework involves a set of six
evaluation criteria that can be used to determine the appropriateness of the mediating
resources of CALL (e.g., a given CALL task, material, or tool) for supporting second
language acquisition from an interactionist perspective. These criteria helped to focus
the students’ evaluation of English language learning with the use of CALL material as
they were engaged in completing the video-based vocabulary learning task. The second
part of the interview aimed to find out the students’ use of system functionalities for
learner-content interaction while learning vocabulary through watching videos. To get
more detailed information about the students’ learning processes in the CALL system,
the interviewees were asked one overarching question (i.e., What is your typical
approach to learning vocabulary through video watching?), with some follow-up
prompts (e.g., Do you usually watch videos with English or Chinese subtitles on? Do
you pause the videos to look up or highlight every word that you don’t know? Do you
re-watch the videos to gain a fuller understanding or to study the language?).
3.6 Procedure
The procedure used in this study started with the teacher introducing the learning
objectives as well as the CALL system to the students, followed by the students being
guided to experience a trial lesson and complete a sample task in the first two weeks.
Next, the students preceded to self-study the ten weekly lessons using the CALL
system out of class, and on completion of the CALL task they were asked to submit
their word lists produced on the basis of the weekly lessons to the teacher during this
study period. After finishing the CALL task as one of their course requirements, the
students took part in individual interviews during the last two weeks of this study.
230 H.-T. Hung et al.
4 Results
4.1 Students’ Use of Video for Task Completion
To better understand the students’ use of or interaction with the video material during
the completion of the video-based vocabulary learning task, the students’ learning
records, as kept by the CALL system, were analyzed to reveal information about the
time they devoted to completing the task and the number of vocabulary items that they
logged in their word lists. In addition, the interview data were analyzed to reveal the
students’ frequency of using the system functionalities for task completion.
Time Spent on Task. The findings for the time on task indicated that each student
spent approximately 198 (SD = 73.79) min creating word lists for the video clips they
reviewed over the course of ten weeks. Moreover, the descriptive statistics also
revealed that the most time a student spent per week on the task was 48 min, and the
least was 13. On average, each student spent around 20 (SD = 7.37) min per week on
the task.
As shown in Fig. 1, all of the students spent more than 11 min working on the task
weekly, with the largest group spending within the range of 11 to 15 min (N = 17), and
this was closely followed by 15 students who spent between 16 and 20 min. As for the
students who spent more than 21 min (N = 15), the interview data showed that they
viewed the video clips twice, both with and without subtitles, and thus spent much
more time on the online learning and completion of the task. In the interviews, the
students indicated that the length of the selected video clips (approximately three
minutes) was reasonable – long enough to allow them to engage with the material, but
not so long that they lost interest.
Fig. 1. Distribution of the time that each student spent on the task per week
Table 1. Descriptive statistics for the total vocabulary items logged per student and the number
of vocabulary items logged per video
Minimum Maximum Median Mean SD
The number of vocabulary items logged for 53 216 111 126.34 50.56
ten weeks
The number of vocabulary items logged per 2 9 5 5.34 2.09
video
deviation was 50.56. It also shows that the average number of vocabulary items logged
per student per video clip was five, with a standard deviation of 2.09.
Figure 2 shows the distribution of the average number of vocabulary items each
student logged per video. Among the 47 students, only seven logged between two and
three vocabulary items for each video, 27 logged between four and six, and 13 between
seven and nine. As seen in Table 1 and Fig. 2, the results indicate that the majority of the
students would intentionally study about five vocabulary items from each video clip.
Fig. 2. Distribution of the average number of vocabulary items each student logged per video
“Some video clips contain words which I don’t know, thus I can learn many new and useful
words and idioms from them. Many such words, for example, “discount” and “on sale,” are
relevant to our daily life, so that I can remember them easily. I often see them when the
department store is having an annual sale, which reinforces my vocabulary knowledge.” –
[Student 17]
Although listening is not the focus of this study, 55% of the students stated that
their listening comprehension improved after they studied with the video material, due
to their larger vocabularies. Students were allowed to rewind and repeat the video clips
as many times as necessary in order to enhance their listening comprehension. One
related comment from the students is presented below:
“I continue to increase my vocabulary from the video material, and I feel that my listening
comprehension became better after this learning experience. As I study with any video, I try to
listen carefully to the clip and repeat it again and again until I can understand it. I think
learning English with video is a good way to practice my listening skill.” – [Student 27]
The criterion of meaning focus looked at whether the students could understand and
thus be able to follow the video content. It was found in the interviews that 75% of the
students thought that the video material was at a good level of difficulty, and thus
neither too easy nor too difficult for self-study, as seen in the following comment.
“I thought I could understand most of the video content, say 70% to 80%, and I often guessed
the meaning from the context for the rest. Some video clips were hard to comprehend because
of technical terms and idioms, so I always watched English subtitles to learn those unknown
words and phrases.” – [Student 38]
The criterion of learner fit measured whether the difficulty level of the online videos
was appropriate. Nearly all the students (96%) felt that the teacher’s selection of video
clips was set at an appropriate level. They did not feel frustrated when they watched
these videos, and they also believed that any difficult vocabulary, idioms, and sentences
could motivate their learning. The following comment illustrates this point:
“I am at an intermediate level. I think most of the selected videos are appropriate for me. Some
videos contained a few difficult vocabulary items, but I can adjust somehow. I believe that if I
work hard, I will reach an advanced level.” – [Student 33]
This criterion of authenticity explored the usefulness and relevance of the video
material to the students’ real lives. The students stated that they liked to learn English
with online videos, because they could learn both new vocabulary and knowledge from
doing so. Almost all the students (96%) felt that they would benefit from the video
content, and that it would be helpful outside the classroom for various purposes, such
as preparing for a standardized English proficiency test or applying for a job. This can
be seen in the following comments:
“I liked to watch video clips about business, and I did learn many words from them. When I
took the TOEIC test, I saw some of the words on the test paper. I was glad that I had already
learned them in the video clips beforehand. I believe that I really learned many useful words
and phrases from the video material, and they helped me get good grades on the TOEIC test.” –
[Student 45]
“I thought learning from video was more practical than from textbooks. I think what I have
learned will serve me well in my job after I graduate.” – [Student 34]
Investigating Students’ Use and Evaluation of Video 235
The criterion of positive impact observed the impact of the video use on the
students’ English learning. The majority of students (91%) perceived digital video as a
beneficial form of CALL material and even expressed their desire to continue to learn
with it after the course was finished. One typical comment is shown below:
“I would like to continue to improve my English via watching videos, because I am exposed to a
variety of authentic contexts by doing so. The use of video makes learning more interesting.
I like to spend time on them rather than studying textbooks.” – [Student 28]
The criterion of practicality investigated whether the use of video in the adopted
CALL system was easy for the students to manipulate, and whether the incorporation
of the video material was capable of assisting in their English learning. All of the
students (100%) reported that they were satisfied with the video material and task
design of the study, stating that the CALL system offered them an user-friendly and
easy-to-use interface, with its functionalities for supporting their interaction with the
video material. The video material was presented in ways that made the content more
understandable, as the following comments made clear:
“I was pleased with the video material. The varied topics illustrated by the vivid images in the
clips made English learning more fun and interesting.” – [Student 16]
“I was satisfied with the use of video material in two aspects: convenience and system design.
In terms of convenience, the video clips were available for us to learn English anytime and
anywhere. We can use any mobile devices, such as smartphones and tablets for online learning
purposes. As for the system, the interface was very simple and easy for users to manipulate.
I was thus able to make good use and interact with the video material effectively and effi-
ciently.” – [Student 29]
Taken together, the results regarding how the students evaluated the overall
appropriateness of using video to learn English in the CALL system provide ample
evidence to suggest that the students considered the way that the video material was
integrated in the intentional vocabulary learning task was facilitative and capable of
enhancing their English learning experiences.
5 Discussion
This study was an effort to explore how the students used online videos to learn
vocabulary in a deliberate manner, and how they perceived the overall value of the
video material for vocabulary learning in specific and English learning in general. With
regard to the first research question, the analysis of the participants’ video use in this
investigation revealed that all the students were able to watch the weekly assigned
video clips and complete the required video-based vocabulary task with reasonable
time and effort, without any major learning difficulties. More specifically, the students’
average time spent on task was 20 min per week, which resulted in an average of 13
vocabulary items logged in each student’s weekly word list. It was also found that the
students exhibited sufficient learner-content interaction with the video material during
the 10-week task completion period, and that the top three most frequently used
functionalities that enabled such interaction were video subtitles, followed by the
built-in dictionary, and adjustable playback speed.
236 H.-T. Hung et al.
had missed in authentic videos that were somewhat challenging for them to understand.
This is in line with the results of the present study, which showed that the students
made good use of the adjustable playback speed to review or re-listen to parts of the
video content.
The results pertaining to the second research question revealed that the students
expressed overwhelmingly favorable perceptions of their learning experience in this
investigation in terms of the six CALL evaluation criteria: language learning potential,
meaning focus, learner fit, authenticity, positive impact, and practicality. These positive
findings reinforce the pedagogical benefits of video identified in recent studies [9, 14,
30]. Researchers have argued that videos can help students understand the meanings of
words they hear by providing them with various clues for meaning construction [31],
and thus incorporating more types of clues (e.g., verbal, visual, and contextual clues)
would help students learn better. The current study confirmed this claim, indicating that
the video material was able to provide certain supportive information that enabled the
students to better understand the vocabulary and content. Another well recognized
pedagogical benefit of video is the authenticity it affords. It is conducive to successful
language learning “when linguistic aspects are not singled out and when words and
phrases are used and learned in their syntactic, semantic and pragmatic contexts” ([32],
p. 315). One would agree that learning with video material creates a fostering envi-
ronment in a similar vein by exposing learners to the use of authentic language.
Likewise, in this study the students’ comments also indicated their preferences for and
appreciation of learning vocabulary from and with video. In other words, the students
found it relatively meaningful to understand and learn vocabulary from authentic
contexts, and apply it in authentic situations.
6 Conclusion
Although this study has demonstrated positive findings regarding a group of EFL
learners’ perceptions on the use of video to assist their vocabulary learning, the results
should be interpreted with consideration to the limitations of this work. First of all,
given that the current study recruited only a small number of students, the results
cannot be generated to all EFL students, and a larger sample is thus needed in future
research. Second, this study adopted a one-group design, measuring the students’
self-reported perceptions after they had taken part in the instructional intervention with
a focus on computer assisted vocabulary learning. The present study thus did not
include a control group to compare the students’ vocabulary learning performance. In
future research, a quasi-experimental research design is recommended, using both
experimental and control groups to determine the effects of engaging students to learn
vocabulary from digital video in an online learning system. It is also suggested that
future studies apply a pretest-posttest design in order to obtain more accurate assess-
ments regarding the effects of video use on students’ vocabulary gains and improve-
ment. Based on the results of such a study, it would be possible to be more confident in
attributing students’ vocabulary development to the use of video. Despite of these
limitations, the findings of the current study have two practical implications for EFL
teachers and system designers working with the design and implementation of CALL
238 H.-T. Hung et al.
material, particularly in the form of video within a CALL system. It is suggested that
EFL teachers incorporate online or digital videos into their classrooms to make
vocabulary learning more engaging and enjoyable. Furthermore, system designers are
recommended to take into consideration the frequently-used and positively-perceived
system functionalities identified in this study when developing EFL systems for
vocabulary learning.
References
1. Warschauer, M.: Computer-assisted language learning: An Introduction. In: Fotos, S. (ed.)
Multimedia Language Teaching, pp. 3–20. Logos International, Tokyo (1996)
2. Stockwell, G.: A review of technology choice for teaching language skills and areas in the
CALL literature. ReCALL 19(2), 105–120 (2007)
3. Kayaoglu, M.N., Dag Akbas, R., Ozturk, Z.: A small scale experimental study: Using
animations to learn vocabulary. Turk. Online J. Educ. Technol. 10(2), 24–30 (2011)
4. Lin, C.C.: Learning action verbs with animation. JALT CALL J. 5(3), 23–40 (2009)
5. Lin, C.C., Tseng, Y.F.: Videos and animations for vocabulary learning: A study on difficult
words. Turk. Online J. Educ. Technol. 11(4), 346–355 (2012)
6. Samur, Y.: Redundancy effect on retention of vocabulary words using multimedia
presentation. Br. J. Educ. Technol. 43(6), 166–170 (2012)
7. Farshi, N., Mohammadi, Z.: Use of podcasts in effective teaching of vocabulary: Learners’
attitudes, motivations, and limitations. Theory Pract. Lang. Stud. 3(8), 1381–1386 (2013)
8. Putman, S.M., Kingsley, T.: The atoms family: Using podcasts to enhance the development
of science vocabulary. Read. Teach. 63(2), 100–108 (2009)
9. Lin, L.F.: English learners’ incidental vocabulary acquisition in the video-based CALL
program. Asian EFL J. 12(4), 51–66 (2010)
10. Naraghizadeh, M., Barimani, S.: The effect of CALL on the vocabulary learning of
Iranian EFL learners. J. Acad. Appl. Stud. 3(8), 1–12 (2013)
11. Karakaş, A., Sariçoban, A.: The impact of watching subtitled animated cartoons on
incidental vocabulary learning of ELT students. Teach. Engl. Technol. 12(4), 3–15 (2012)
12. Yuksel, D., Tanriverdi, B.: Effects of watching captioned movie clip on vocabulary
development of EFL learners. Turk. Online J. Educ. Technol. 8(2), 48–54 (2009)
13. Giannakos, M.N.: Exploring the research on video learning: A review of the literature. Br.
J. Educ. Technol. 44(6), 191–195 (2013)
14. Goldstein, B., Driver, P.: Language Learning with Digital Video. Cambridge University
Press, Cambridge (2014)
15. Mayer, R.E.: Applying the science of learning: Evidence-based principles for the design of
multimedia instruction. Am. Psychol. 63(8), 760–769 (2008)
16. Mayer, R.E., Moreno, R.: Nine ways to reduce cognitive load in multimedia learning. Educ.
Psychol. 38(1), 43–52 (2003)
17. Mayer, R.E.: Multimedia Learning. Cambridge University Press, New York (2009)
18. Mayer, R.E., Moreno, R.: Aids to computer-based multimedia learning. Learn. Instr. 12(1),
107–119 (2002)
19. Jones, L.C.: Supporting listening comprehension and vocabulary acquisition with multime-
dia annotations: The students’ voice. CALICO J. 21(1), 41–65 (2003)
20. Mayer, R.E., Moreno, R.: Animation as an aid to multimedia learning. Educ. Psychol. Rev.
14(1), 87–99 (2002)
Investigating Students’ Use and Evaluation of Video 239
21. Clark, R.C., Mayer, R.E.: E-learning and the Science of Instruction: Proven Guidelines for
Consumers and Designers of Multimedia Learning. Pfeiffer, San Francisco (2011)
22. Mayer, R.E.: Applying the science of learning to medical education. Med. Educ. 44(6),
543–549 (2010)
23. Canning-Wilson, C.: Practical aspects of using video in the foreign language classroom.
Internet TESL J. 6(11) (2000). http://iteslj.org/Articles/Canning-Video.html
24. Chapelle, C.A.: Computer Applications in Second Language Acquisition. Cambridge
University Press, Cambridge (2001)
25. King, J.: Using DVD feature films in the EFL classroom. Comput. Assist. Lang. Learn. 15
(5), 509–523 (2002)
26. Frumuselu, A.D., De Maeyer, S., Donche, V., del Mar Gutiérrez Colon Plana, M.: Television
series inside the EFL classroom: Bridging the gap between teaching and learning informal
language through subtitles. Linguist. Educ. 32, 107–117 (2015)
27. Liu, T.C., Lin, P.H.: What comes with technological convenience? Exploring the behaviors
and performances of learning with computer-mediated dictionaries. Comput. Hum. Behav.
27(1), 373–383 (2011)
28. Levy, M., Steel, C.: Language learner perspectives on the functionality and use of electronic
language dictionaries. ReCALL 27(2), 177–196 (2015)
29. Chen, H.J.H.: Developing and evaluating SynctoLearn, a fully automatic video and
transcript synchronization tool for EFL learners. Comput. Assist. Lang. Learn. 24(2),
117–130 (2011)
30. Huang, P., Hwang, Y.: An exploration of EFL learners’ anxiety and E-learning
environments. J. Lang. Teach. Res. 4(1), 27–35 (2013)
31. Jung, R., Lee, C.H.: Using Internet video clips for university students’ vocabulary
development in blended learning. Multimedia Assist. Lang. Learn. 16(4), 67–96 (2013)
32. Tschirner, E.: Language acquisition in the classroom: The role of digital video. Comput.
Assist. Lang. Learn. 14(3–4), 305–319 (2001)