Computers & Education 94 (2016) 252e275

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

The effects of integrating mobile devices with teaching and

learning on students' learning performance: A meta-analysis
and research synthesis
Yao-Ting Sung a, Kuo-En Chang b, Tzu-Chien Liu a, *
a
b

Department of Educational Psychology and Counseling, National Taiwan Normal University, Taiwan, ROC
Grad. Institute of Information and Computer Education, National Taiwan Normal University, Taiwan, ROC

a r t i c l e i n f o

a b s t r a c t

Article history:
Received 12 August 2015
Received in revised form 17 November 2015
Accepted 19 November 2015
Available online 23 November 2015

Mobile devices such as laptops, personal digital assistants, and mobile phones have
become a learning tool with great potential in both classrooms and outdoor learning.
Although there have been qualitative analyses of the use of mobile devices in education,
systematic quantitative analyses of the effects of mobile-integrated education are lacking.
This study performed a meta-analysis and research synthesis of the effects of integrated
mobile devices in teaching and learning, in which 110 experimental and quasiexperimental
journal articles published during the period 1993e2013 were coded and analyzed. Overall,
there was a moderate mean effect size of 0.523 for the application of mobile devices to
education. The effect sizes of moderator variables were analyzed and the advantages and
disadvantages of mobile learning in different levels of moderator variables were synthesized based on content analyses of individual studies. The results of this study and their
implications for both research and practice are discussed.
2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Keywords:
Evaluation methodologies
Pedagogical issues
Teaching/learning strategies

1. Introduction
1.1. Integrating mobile devices with learning and instruction
Mobile computers have gradually been introduced into educational contexts over the past 2 decades. Mobile technology
has led to most people to carry their own individual small computers that contain exceptional computing power, such as
laptops, personal digital assistants (PDAs), tablet personal computers (PCs), cell phones, and e-book readers. This large
amount of computing power and portability, combined with the wireless communication and context sensitivity tools, makes
one-to-one computing a learning tool of great potential in both traditional classrooms and outdoor informal learning.
With regard to access to computers, large-scale one-to-one computing programs have been implemented in many
countries globally (Bebell & O'Dwyer, 2010; Fleischer, 2012; Zucker & Light, 2009), such that elementary- and middle-school
students and their teachers have their own mobile devices. In addition, in terms of promoting innovation in education via
information technology, not only does mobile computing support traditional lecture-style teaching, but through convenient

* Corresponding author. Department of Educational Psychology and Counseling, National Taiwan Normal University, 162, HePing East Road, Section 1,
Taipei, Taiwan, ROC.
E-mail addresses: [email protected] (Y.-T. Sung), [email protected] (K.-E. Chang), [email protected] (T.-C. Liu).
http://dx.doi.org/10.1016/j.compedu.2015.11.008
0360-1315/ 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/
licenses/by-nc-nd/4.0/).

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

253

information gathering and sharing it can also promote innovative teaching methods such as cooperative learning (Lan, Sung,
& Chang, 2007; Roschelle et al., 2010), exploratory learning outside the classroom (Liu, Lin, Tsai, & Paas, 2012), and gamebased learning (Klopfer, Sheldon, Perry, & Chen, 2012). Therefore, mobile technologies have great potential for facilitating
more innovative educational methods. Simultaneously, these patterns in educational methods will likely not only help
subject content learning, but may also facilitate the development of communication, problem-solving, creativity, and other
high-level skills among students (Warschauer, 2007).
However, despite the proposed advantages of using mobile computing devices for increasing computer accessibility,
diverse teaching styles, and academic performance, currently researchers found mixed results regarding the effects of mobiledevices (e.g., Warschauer, Zheng, Niiya, Cotten, & Farkas, 2014), and very few studies have addressed how best to use mobile
devices, and the effectiveness of doing so.

1.2. Review of the research into integrating mobile devices with teaching and learning
There are seven studies which reviewed the research into integrating mobile devices with teaching and learning and can
be divided into two types according to the devices they focused on: (1) those focused on how laptops are used in schools and
(2) those focused on the applications of various types of mobile device in education (see Appendix A).
Regarding the review of laptop-based programs, Zucker and Light (2009) believed that school programs integrating
laptops into schools have a positive impact on student learning. However, they also believed that laptop use did not achieve
the goals of increasing higher-level thinking and transformation of classroom teaching methods. Penuel (2006) reviewed
30 studies that examined the usage of laptops with wireless connectivity in one-to-one computer programs. Those studies
found that students most often used the laptops to do homework, take notes, and nish assignments. General-purpose
software such as word processors, web browsers, and presentation software were relatively common. Bebell and
O'Dwyer (2010) examined four different empirical studies of laptop programs in schools. They discovered that in most
schools participating in one-to-one programs there were signicant increases in grade-point averages or standardized tests
of student achievement, relative to schools that did not provide such programs. In addition, they found that most students
used their laptops to write, browse the Internet, make presentations, do homework, or take tests. Furthermore, teachers
made more changes to their teaching methods when they had increased opportunities to use laptops. Students participating in one-to-one programs also had a deeper engagement with what they were learning when compared to control
groups.
Fleischer (2012) conducted a narrative research review of 18 different empirical studies on the usage of laptops. These
studies found a large range in the number of hours that students used laptops, from a few days to as little as 1 h per week. The
most frequently used computer functions were searches, followed by expression and communication. In most studies it was
found that students had a positive attitude toward laptops, and felt that they were more motivated and engaged in their
learning, and it was further believed that teachers conducted more student-centered learning activities. Moreover, considerable differences in classroom educational practices arose from the diversity of teachers' beliefs about the usefulness of
laptops. Fleischer (2012) also found several challenges regarding the use of laptops in classrooms, such as encouraging
teachers to change their previous beliefs and teaching methods (e.g., teacher-centered lectures) in response to their students'
greater exibility and autonomy; how to reconcile the conict between the students' desire for independent study and the
need for teachers' guidance; and how to facilitate teachers' competence by designing an appropriate curriculum and teaching
models for laptop usage programs.
With respect to the research on the use of mobile technology in education, Hwang and Tsai (2011) provided a broad
discussion of studies on mobile and ubiquitous learning published in six journals between 2001 and 2010. In their review of
154 articles, they discovered that the use of mobile and ubiquitous learning accelerated markedly during 2008; researchers
mostly studied students of higher education, and the elds most often researched were language arts, engineering, and
computer technology. Frohberg, Goth, and Schwabe (2009) categorized 102 mobile-learning projects, and discovered that
most mobile-learning activities occurred across different settings, and took place within a physical context and an ofcial
environment, such as a classroom or workplace. Regarding the pedagogical roles that mobile devices play in education, most
research has used mobile devices primarily as a sort of reinforcement tool to stimulate motivation and strengthen engagement, and secondarily as a content-delivery tool. Few projects have used mobile devices to assist with constructive thinking
or reection. Furthermore, most learning activities using mobile devices have been controlled by the teacher, with there being
only a handful of learner-centered projects in existence. Concerning the communication functions, very few projects have
made any use of cooperative or team communication. Moreover, the vast majority of studies have made use of novice participants; little research has involved experienced participants. When sorted according to educational goals, it was found that
the vast majority of research has focused on lower-level knowledge and skills, and ignored higher-level tasks such as analysis
and evaluation. Wong and Looi (2011) investigated the inuence of mobile devices on seamless learning. Seamless learning
refers to a learning model that students can learn whenever they want to learn in a variety of scenarios and that they can
switch from one scenario or one context to another easily and quickly (Chan et al., 2006; Wong & Looi, 2011). Wong and Looi
(2011) selected and analyzed a sample of 54 articles on the use of mobile devices to facilitate seamless learning, and found
that all 54 articles contained 10 features, including formal and informal learning, personalized and social learning, and
learning across multiple durations and locations.

254

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

1.3. Purposes of this study

While analyzing the overall effectiveness of using mobile devices in education, the review research described above has
two major limitations. First, all of the reviews adopted a qualitative approach, which may be able to describe and summarize
how related studies were conducted and the problems encountered during their execution, but this makes it difcult to
evaluate the effects actually produced by the mobile devices in general and the specic moderator variables. Second, much of
the previous review research has focused on the usage of laptop computers as the subject of their investigation (e.g., Penuel,
2006), and most of the research participants in those reviewed articles were in primary and secondary schools. However, the
many new developments in mobile hardware have meant that diverse age groups now use different devices. Therefore, many
different moderators need to be accounted for when attempting to determine whether or not intervening variables have an
effect.
In the context of this background, the primary goal of this study was to perform a meta-analysis and research synthesis of
the research on the usage of mobile devices in education published in the last 2 decades. Specically, the purposes of this
study were as follows:
1. To provide an overview of the status of the use of mobile devices in educational experimental studies, including who is
using them, which domain subjects are being taught, what kinds of mobile device and software are being used, where such
programs take place, how the devices are used in teaching, and the duration of the interventions.
2. To quantify the overall effectiveness of integrating mobile technologies into education on student learning achievement.
3. To determine how the moderator variables inuence the effects of mobile devices on learning achievement.
4. To synthesize the advantages and disadvantages of mobile learning in levels of moderator variables based on the content
analysis of articles related with moderator variables.

2. Method
2.1. Data sources and search strategy
Journal articles published during the period 1993e2013 were searched electronically and manually, and via reference-list
checking to retrieve the relevant literature. For electronic searches, the main databases were the Education Resources Information Center (ERIC) and the Social Sciences Citation Index database of the Institute of Science Index (ISI). Two sets of
keywords were searched: (1) mobile-device related keywords, including mobile, wireless, ubiquitous, wearable, portable,
handheld, cell phone, personal digital assistant, PDA, palmtop, pad, web pad, tablet PC, tablet computer, laptop, e-book, digital
pen, pocket dictionary, and classroom response system; and (2) learning-related keywords, including teaching, learning,
training, and lectures. The two sets of keywords were combined when searching the electronic databases. Manual searches
included the major journals in educational technology and e-learning, such as the Australian Journal of Educational Technology, British Journal of Educational Technology, Computers & Education, Computer Assisted Language Learning, Educational Technology Research and Development, Journal of Computer Assisted Learning, Language Learning & Technology, and
ReCall.
After collating all of the related literature, another round of searches was conducted using the reference lists found in the
literature yielded by the electronic search to nd any omitted but relevant works.

2.2. Search results

2.2.1. Initial screening
The initial search yielded 4121 abstracts published between 1993 and 2013 (1718 in ERIC and 2403 in ISI) that were related
to mobile learning. Two authors read each abstract of the article and judged whether or not the article was related to teaching
and learning with a mobile device, which resulted in the selection of 925 articles.
2.2.2. Screening for experimental and quasiexperimental research
In the second stage, the studies were screened according to the research method. Experimental studies (including the
pretest-posttest equivalent-group, posttest-only equivalent-groups, and randomized matched subjects and posttest-only
control-group designs) and quasi-experimental studies (including the pretest-posttest nonequivalent-groups and counterbalanced designs; see Ary, Jacobs, & Razavieh, 2002; Best & Kahn, 1998, for a reference) were included. Conceptual analysis or
research reviews, case studies and qualitative research, survey research, and pre-experimental studies were all excluded at
this stage. At the completion of this stage there remained 182 articles.
2.2.3. Application of inclusion and exclusion criteria for the meta-analysis
Studies were eligible for inclusion in the meta-analysis if they conformed with the following three criteria:

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

255

1. The application of mobile devices was the key variable of the study. The experimental group had an intervention that
used mobile devices, and was compared with a control group that used traditional learning. If both the experimental
and control groups used mobile-device interventions, and only the teaching methods were compared, then the study
was excluded (e.g., Hsu, Hwang, Chang, & Chang, 2013; Jeong & Hong, 2013; Li, Chen, & Yang, 2013; Ryu & Parsons,
2012).
2. Sufcient information was presented to calculate effect sizes, such as means, standard deviations, t, F, or c2 values, or the
number of people in each group. Articles in which the sample sizes of each group were not cited, lacked any inferential
statistical results, or had inferential statistical results but were still inadequate for calculating an effect size according to
Lipsey and Wilson (2000) were excluded (e.g., Gleaves, Walker, & Grey, 2007; Langman & Fies, 2010; Purrazzella &
Mechling, 2013; Yang et al., 2013).
3. Experimental results were presented with learning achievement as a major dependent variable measured by standardized
or researcher-constructed tests. Studies for which the results were related to affective variables (e.g., learning attitude or
learning motivation) or interaction between peers but without learning achievement were excluded (e.g., Jian, Sandnes,
Law, Huang, & Huang, 2009; Lan et al., 2007; Mouza, 2008; Siozos, Palaigeorgiou, Triantafyllakos, & Despotakis, 2009).
Application of these criteria yielded 110 articles that were acceptable for inclusion in the meta-analysis. For a complete list
of these references, please see our online supplemental archive.

2.3. Selection and coding of the outcome variables

One of the most used framework for representing the research content and dimensions is the activity theory (AT), which
uses activity as a unit for analyzing human practices (Bakhurst, 2009). Recently, several researchers have used the AT as a
theoretical basis for analyzing mobile learning studies (e.g., Frohberg et al., 2009; Sharples, Taylor, & Vavoula, 2007) or for
designing mobile learning scenarios (e.g., Zurita & Nussbaum, 2007). This study used six major components of AT to select
moderator variables and analyze mobile learning: (a) Subjects: which involve all the people who may be involved in learning
curriculums through mobile devices, such as students of different age levels or teachers of different levels of teaching
expertise. (b) Objects (or objectives) of the mobile learning, which focus on the goal such as acquiring cognitive skills or
enhancing learning motivation through mobile devices. (c) Tools/instruments in the mobile learning, which may be artifacts
(e.g., hardware and software) or learning resources (e.g., tutors). (d) Rules/control for the activity, which are norms or regulations that circumscribe the mobile activities, such as the procedure in teaching scenarios designed for the learning pace or
styles designated. (e) Context of the activity, which refers to the physical (e.g., classroom or museum) or social (e.g., ambience
of learning in a group) environments for conducting mobile learning. (f) Communication/interaction, which refers to the
method of interaction between users and mobile technologies (such as the process teachers' adaption to mobile devices) or
the communications styles among learners.
2.3.1. Research name
This refers to the rst author's name, the year of publication, and the article title.
2.3.2. Research participants
In this review, for all the reviewed articles, the research participant corresponded to the subject of the AT framework,
and was coded by their learning stages, including kindergarten, elementary school, middle school, (senior) high school,
university, graduate school, teachers, adults, and mixed.
2.3.3. Treatments
The treatments of the reviewed articles corresponded to the tools component (e.g., the hardware and software), the
rules/control component (e.g., the teaching methods and domain subjects), and the context component (e.g., intervention
settings, intervention duration). The description for each of these treatment variables are as follows:
1. Hardware: Different types of mobile hardware, which comprised PDAs, laptops, tablet PCs, cell phones, iPods, MP3 players,
e-book readers, pads, digital pens, pocket dictionaries, and classroom response systems (CRSs), or any mixture of thereof.
2. Software: Different types of software, which encompassed general-purpose software and learning-oriented software
(Sung & Lesgold, 2007), the former referring to commercial software currently in circulation that was not designed
especially for teaching and learning (e.g., word processors or spreadsheets), and the latter having been designed specifically for educational programs or goals.
3. Teaching method: Different teaching methods, including lectures, cooperative learning (students were divided into groups
and completed learning tasks collaboratively, e.g., Chang, Lan, Chang, & Sung, 2010; Huang, Liang, Su, & Chen, 2012),
inquiry-oriented learning (using problem-, project-, or inquiry-based methods with mobile devices for learning, e.g., Chen,
2010; Lowther, Ross, & Marrison, 2003), self-directed study (teachers/researchers did not designate or implement specic
teaching scenarios for students to follow, students use mobile devices for self-paced learning, e.g., Chen & Li, 2010; Chen,

256

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Tan, & Lo, 2013), computer-assisted testing/assessment (using mobile devices for formative assessment or quizzes in
classroom or outdoors, e.g., Agbatogun, 2012), and mixed methods thereof.
4. Domain subject: Domain subjects were analyzed to establish the relative effectiveness of mobile devices for teaching
different subjects, including language arts, social studies, science, mathematics, multidisciplinary (if the mobile devices
were used in several subjects, but measurement of the achievement was presented as a whole instead of separately, this
was coded as multidisciplinary), specic abilities (e.g., spatial ability or creativity), health-care programs, education,
psychology, and computer and information technology.
5. Implementation setting: Implementation settings were included to establish whether the impact of mobile devices on
learning differed according to the environment in which they were used, which included classrooms, outdoors (e.g., zoo or
campus gardens), museum, laboratory, workplaces, and unrestricted settings (devices may be used anywhere).
6. Intervention duration: Different periods of time for the intervention, including periods no more than four hours (
4 h),
between ve and 24 h (>4 and
24 h), between one day and seven days (>1 day and
7 days), between one week and four
weeks (>1 week and
4 weeks), between one month and six months (>1 month and
6 months), and more than six
months (>6 months).

2.3.4. Dependent variables

The dependent variables corresponded to the Objective of the AT model, including two categories: the learning
achievement dependent variables refer to measurement of cognitive outcomes such as knowledge application, retention,
problem solvingetc. The affective variables refer to measurement of motivation, interest, participationetc.

2.4. Data analysis

2.4.1. Calculating the effect size
The following meta-analysis steps recommended by Borenstein, Hedges, Higgins, and Rothstein (2009) were employed in
this study: (a) determine the effect sizes of each article, (b) determine the weighted mean effect size across articles, (c)
calculate the condence interval for the average effect size, and (d) determine whether the effect size of any particular group
was inuenced by a moderator variable based on a heterogeneity analysis (QB).
Two formulae were used to calculate the effect sizes of the studies. Cohen's d formula (Cohen, 1988) was used to determine
the effect size for the experimental research with random assignment and without a pretest:

X1  X2
d r

(1)

n1 1s21 n2 1s22
n1 n2 2

where X 1 and X 2 represent the mean scores, n1 and n2 represent the sample sizes, and s21 and s22 represent the variances of the
experiment and control groups, respectively.
For experimental or quasiexperimental research with pretests, it was proposed that the pretest should be taken into
consideration instead of using the posttest in order to mitigate possible selection bias (Furtak, Seidel, Iverson, & Briggs, 2012;
Morris, 2008). Hence, the formula developed in Comprehensive Meta Analysis (version 2.0) was used to obtain effect sizes for
research with pre- and posttests:


ESPre=Post Test Two Groups

X1

Post

 X1


 X2
SDPost

Pre

Post

 X2


Pre

(2)

where X 1 Pre and X 1 Post represent the mean scores of the experimental group for the pretest and posttest, respectively, and
X 2 Pre and X 2 Post represent the mean scores of the control group for the pretest and posttest, respectively. SDPost can be
calculated as follows:

SDPost

s




n2 Post  1 s22 Post n1 Post  1 s21 Post



n2 Post n1 Post  2

(3)

where n1 Post and n2 Post represent the sample sizes of the experimental and control groups, respectively, for the posttest,
while s21 Post and s22 Post represent the variances of the experimental and control groups, respectively.
The two types of effect sizes were calibrated using the sample weights to calculate a Hedges' g according to


g

1


3
d
4df  1

(4)

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

257

2.4.2. Evaluating publication bias

The fail-safe N (i.e., classic fail-safe N) of Rosenthal (1979) was used to estimate how many insignicant effect sizes
(unpublished data) would be necessary to reduce the overall effect size to an insignicant level. The comparison criterion was
5n10, where n is the number of studies included in the meta-analysis. If the fail-safe N is larger than 5n10, it means that the
estimated effect size of unpublished research is unlikely to inuence the effect size of the meta-analysis. Moreover, the
present study also adopted Orwin's fail-safe N (Orwin, 1983) to estimate the number of missing null studies that would be
required to bring the mean effect size to a trivial level.
3. Results and discussion
3.1. Descriptive statistics information
Table 1 presents the distribution of moderator variables and their corresponding effect sizes (g). In total there were 110
articles, 419 effect sizes, and 18 749 participants. The largest proportion of studies involved the college-student-level learning
stage (38.4%); the next largest group was elementary-school students (33.9%). More studies used learning-oriented software
(62.7%) than general-purpose software (34.5%). Handheld devices (including PDA, cell phone, iPod, MP3 player, digital pen,
pocket dictionary, and classroom response system) were the most widely studied of the hardware (72.7%), followed by
laptops (21.8%, including laptop, pad, tablet PC, and e-book reader). The largest proportion of studies were set in the classroom
(50.0%), followed by outdoors (15.5%) and unrestricted settings (16.4%). For teaching methods, self-directed study (30.9%) was
the most frequently researched, and the most frequently studied intervention duration was >1 month and
6 months
(32.7%), followed by > 1 week and
4 weeks (25.5%) and
4 h (20.9%). Finally, language arts were the most often studied
domain subject (34.7%), followed by science (22.9%).
In addition, among those moderating variables, the evolution of hardware used, implementation setting, and domain
subjects may have seen the greatest amount of change during 1993e2013. The trends of those moderating variables during
the two decades are shown in Figs. 1e3. Fig. 1 shows the evolution of the use of different mobile devices. Compared with
laptop and mixed categories, handheld devices (e.g., cell phone) had been used more since 2009e2013 and showing an
obviously rising trend. Moreover, Fig. 2 shows the evolution of the use of different implementation settings. Compared with
informal settings (e.g., museums; outdoors) and unrestricted categories, formal settings (e.g., classroom; laboratories) had
been set more since 2004e2008 and showing an obviously rising trend. Finally, Fig. 3 shows the evolution of the domain
subject. Compared with other domain subjects, language arts had been studied more since 2009e2013 and showing an
obviously rising trend.
3.2. Overall effect size for learning achievement
The distribution of the effect sizes of the 110 articles is shown in Fig. 4. The forest plot of effect sizes and the 95%
condence interval of the 110 articles are shown in Appendix B. There were two unusually large effect sizes, g 4.045 (Hsu
& Lee, 2011) and g 3.050 (Wu, Sung, Huang, Yang, & Yang, 2011), which were larger than the average effect size for the
entire collection of 110 articles (g 0.628) more than three standard deviations, and so these were not included in further
analyses (Lipsey & Wilson, 2000). Using the procedure of Lipsey and Wilson (2000) with a random-effects model to
integrate the effect sizes of the 108 articles, there was an overall moderate mean effect size of 0.523, with a 95% condence
interval of 0.432e0.613. Researchers (e.g., McMillan, Venable, & Varier, 2013; Van der Kleij, Feskens, & Eggen, 2015) have
proposed that Hattie's (2009) criterion is appropriate for evaluating the effect sizes in educational contexts. Therefore, we
adopt Hattie's (2009) criterion to interpret the effect size of our research, in which an effect size of 0.60 is high, around
0.40 is medium, around 0.20 is low, and <0.20 is with little signicant meaning. In this study it was found that using mobile
devices in education had a medium effect size for learning achievement; in other words, 69.95% of learners using a mobile
device performed signicantly better in dependent variables related with cognitive achievement than those not using
mobile devices.
The Q statistics show that the effect sizes in the meta-analysis were heterogeneous (Qtotal 626.302, z 11.315, p < .001),
which indicates that there are differences among the effect sizes resulting from factors other than subject-level sampling
error, such as the diversity of the learning stage, the hardware used, and the teaching methods.
Furthermore, we also conducted an analysis for the studies related to the affective variables (such as motivation,
engagement, attitude, satisfaction, preference). The overall mean effect size of the 22 articles was 0.433 (z 6.148, p .001),
with a 95% condence interval of 0.295e0.570. According to Hattie's criterion, there is a medium effect size for affective
variables when using mobile devices in educational context.
The overall mean effect size for learning achievement in this meta-analysis was 0.523, meaning that learning with
mobiles is signicantly more effective than traditional teaching methods that only use pen-and-paper or desktop computers. Compared to past comparisons of effects between using computers and not using computers in education, the effect
size of using mobile devices reported herein seems larger than those found in meta-analysis into desktop-computer-based
instruction, such as in the studies of Kulik and Kulik (1991) and Tamim, Bernard, Borokhovski, Abrami, and Schmid (2011),
who found mean effect sizes for computer-based instruction of 0.30 and 0.35, respectively. One of the reasons for the
different effect sizes may be differences in the features of desktops and mobile devices; however, there are alternative

258

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Table 1
Categories and learning achievement effect sizes for 110 articles.
Variable

Category

Number of studies Number of effect

(k)
sizes

Proportion of
studies

Proportion of effect Effect size

size
(g)

Learning stage

1.
2.
3.
4.
5.
6.
7.

Kindergarten
Elementary school
Middle school
High school
College
Adults
Mixed

1
38
10
10
43
2
8

2
97
47
47
128
4
94

0.009
0.339
0.089
0.089
0.384
0.018
0.071

0.005
0.232
0.112
0.112
0.305
0.010
0.224

0.103
0.654
0.512
0.390
0.599
2.474
0.084

Intervention
duration

1.
2.
3.
4.
5.
6.
7.

Not mentioned

4 h
> 4,
24 h
>1,
7 days
>1 week,
4 weeks
>1 month,
6 months
>6 months

7
23
2
5
28
36
9

23
86
18
9
95
100
88

0.064
0.209
0.018
0.045
0.255
0.327
0.082

0.055
0.205
0.043
0.021
0.227
0.239
0.210

0.782
0.521
0.385
0.369
0.643
0.630
0.290

Hardware used

1. Not mentioned
2. Handhelds
3. Laptop
4. Tablet PC
5. Cell phone
6. iPod or MP3 player
7. E-book reader
8. Digital pen
9. Pocket dictionary
10. Classroom response systems
11. Mixed

2
40
14
8
24
5
2
1
2
8
4

8
87
109
19
84
16
41
1
11
31
12

0.018
0.364
0.127
0.073
0.218
0.045
0.018
0.009
0.018
0.073
0.036

0.019
0.208
0.260
0.045
0.200
0.038
0.098
0.002
0.026
0.074
0.029

1.421
0.743
0.276
0.615
0.676
0.524
0.693
0.217
0.160
0.369
0.273

Software used

1. Not mentioned
2. General purpose
3. Learning-oriented

3
38
69

29
223
167

0.027
0.345
0.627

0.069
0.532
0.399

0.355
0.494
0.626

Implementation
setting

0.
1.
2.
3.
4.
5.
6.
7.

2
55
4
3
17
18
3
8

3
242
13
12
27
94
14
14

0.018
0.500
0.036
0.027
0.155
0.164
0.027
0.073

0.007
0.578
0.031
0.029
0.064
0.224
0.033
0.033

0.700
0.487
0.833
0.329
0.760
0.480
0.247
1.032

Teaching method

1. Not mentioned
2. Lectures
3. Discovery and exploration
4. Cooperative learning
5. Problem-solving
6. Game-based learning
7. Self-directed study
8. Podcasting
9. Computer-assisted testing
10. Project-based learning
11. Mixed

9
13
13
9
10
4
34
1
6
1
10

84
45
25
60
32
7
122
6
8
7
23

0.082
0.118
0.118
0.082
0.091
0.036
0.309
0.009
0.055
0.009
0.091

0.200
0.107
0.060
0.143
0.076
0.017
0.291
0.014
0.019
0.017
0.055

0.186
0.556
0.920
0.261
0.572
0.404
0.521
0.153
0.660
2.551
0.847

Domain subject

1. Language arts
2. Social studies
3. Science
4. Mathematics
5. Multidisciplinary
6. Specic abilities
7. Health-care programs
8. Education
9. Psychology
10. Computer
and
information
technology

41
5
27
12
1
5
7
3
3
14

169
10
78
41
6
24
18
6
7
60

0.347
0.042
0.229
0.102
0.008
0.042
0.059
0.025
0.025
0.119

0.403
0.024
0.186
0.098
0.014
0.057
0.043
0.014
0.017
0.143

0.593
0.776
0.578
0.338
0.333
0.103
0.535
0.381
0.467
0.716

Not mentioned
Classroom
Museum
Laboratory
Outdoors
Unrestricted
Workplaces
Mixed

explanations, including differences in the meta-analysis methodology, dependent variable measurements, or software
employed. Whether computer-based instruction would be able to enhance students' learning motivation remained
equivocal (e.g., Jabbar & Felicia, 2015; Wouters, van Nimwegen, van Oostendorp, & van der Spek, 2013). Our study found
that mobile learning was able to facilitate students' affective learning outcomes, which provides more convergent evidence
for the effects of using computers in learning and teaching. Possible reasons may include that mobile learning integrated

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

259

Fig. 1. Histogram of the hardware used in mobile devices assisted learning across time.

Fig. 2. Histogram of the implementation setting in mobile devices assisted learning across time.

Fig. 3. Histogram of the domain subjects in mobile devices assisted learning across time.

more diverse type of teaching/learning strategies and involved more different learning scenarios in different situations (see
next section for more descriptions). However, because many of the articles included in our study used teaching programs
lasted for very short-term durations (see next section), the effect of novelty for technology should be taken into
consideration.

260

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Fig. 4. Histogram of the effect sizes of the 110 articles.

3.3. Effect sizes of learning achievement for moderator variables

To learn more about the effects of moderating variables on mobile devices with teaching and learning, this study conducted analyses for the effects of learning achievement with moderator variables. Because there were only 22 studies which
related to affective dependent variables can calculate effect size, which is not comprehensive enough to cover different levels
of moderating variables, the moderator analyses did not include the affective effects.
As indicated in Table 1, some levels of the moderator variables included small samples, and so a few of the levels were
merged within some moderator variables. For the learning stage, kindergarten and elementary school were combined into a
young-children category; middle schools and high schools were combined into secondary-schoolers; and college and
graduate students, teachers, and working adults were combined into adult users. With respect to the hardware, laptops,
tablet PCs, and e-book readers were combined into a laptops category, while PDAs, iPods, MP3 players, cell phones, digital
pens, dictionaries, and classroom response systems were bundled together to form one handheld category. In terms of
function, digital pen is different from other handheld devices, such as iPod, PDA, and smart phone. Also, there was only one
study on digital pen. Therefore, it was excluded in our moderator analysis. In terms of the settings, classrooms, laboratories,
and workplaces were combined into formal learning environments, while museums and outdoors were combined into
informal learning environments (Dierking, Falk, Rennie, Anderson, & Ellenbogen, 2003). Intervention durations were also
combined, with
4 h, > 4 and
24 h, and >1 day and
7 days becoming
1 week. For domain subjects, specic abilities and
multidisciplinary were combined into domain-general subjects. In addition, health-care programs, education, psychology,
and computer and information technology were combined into professional subjects. For teaching methods, discovery and
exploration, problem-solving, and project-based learning were combined into inquiry-oriented learning. Moreover, the
learning methods of self-directed study and podcasting were combined into self-directed study. Table 2 list the effect sizes
for the moderator variables.
3.3.1. Learning stage
Table 2 indicate that young children had a high effect size on learning achievement (g 0.636, z 8.000, p < .001), while
adults (g 0.552, z 7.360, p < .001) and secondary-schools (g 0.451, z 4.274, p < .001) had medium effect sizes. However,
Mixed (g 0.086, z 0.503, p .615) did not show signicant effect sizes. The QB achieved signicance (QB 9.226, p .026),
meaning that the mean effect size different signicantly between the categories.
The results indicated that mobile-assisted learning/instructions were not effective for groups with mixed-age students.
The possible reason may be that it is difcult to design appropriate teaching method or material for students with different
needs and competence in the same group.
3.3.2. Hardware used
Table 2 gives the effect sizes for the usage of different types of hardware in mobile learning. While ignoring the not
mentioned category, handheld devices (g 0.591, z 10.992, p < .001) were associated with a medium effect size, while
laptops (g 0.309, z 3.350, p .001) were associated with a low effect size. The QB was signicant (QB 18.426, p < .001),
indicating that the effect sizes differed signicantly among the various categories. The R2 was 7%, meaning that 7% of total
between-study variance in effects can be explained by hardware used.
The positive learning outcomes of implementing handhelds could be attributed to their features. For example, to make use
of the portability and communication functionality of cell phones, the short message service were used to help teach foreign
lu & Akdemir, 2010; Lu, 2008; Saran, Seferog
lu, & Cag
ltay, 2012), and because the messages
language vocabulary (e.g., Basog

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

261

Table 2
The learning-achievement effect sizes of categories and their related moderator variables.
Category

95% CI

Learning stage
1. Young children
2. Secondary-schoolers
3. Adults
4. Mixed

39
20
43
8

0.636
0.451
0.552
0.086

8.000***
4.274***
7.360***
0.503

[0.480e0.791]
[0.244e0.658]
[0.405e0.700]
[0.248 to 0.419]

Hardware used
1. Not mentioned
2. Handhelds
3. Laptops
4. Mixed

2
78
24
3

1.416
0.591
0.309
0.044

4.491***
10.992***
3.350**
0.173

[0.798e2.033]
[0.485e0.696]
[0.128e0.490]
[0.460 to 0.548]

Software used
1. Not mentioned
2. General purpose
3. Learning-oriented

3
37
68

0.347
0.429
0.590

1.262
5.407***
9.699***

[0.192 to 0.886]
[0.273e0.584]
[0.471e0.709]

Implementation setting
1. Not mentioned
2. Formal settings (classroom, laboratory, hospital)
3. Informal settings (museum, outside)
4. Unrestricted

2
60
21
25

0.701
0.430
0.768
0.550

2.069*
7.328***
7.096***
5.887***

[0.037e1.365]
[0.315e0.545]
[0.556e0.980]
[0.367e0.734]

Teaching method
1. Not mentioned
2. Lectures
3. Inquiry-oriented learning
4. Cooperative learning
5. Game-based learning
6. Self-directed learning
7. Computer-assisted testing
8. Mixed

9
12
24
9
4
34
6
10

0.186
0.394
0.844
0.261
0.407
0.440
0.656
0.839

1.369
3.120**
8.400***
1.673
1.922
5.492***
3.661***
5.702***

[0.080 to 0.452]
[0.146e0.641]
[0.647e1.041]
[0.045 to 0.566]
[0.008 to 0.822]
[0.283e0.597]
[0.305e1.006]
[0.550e1.127]

Intervention duration
1. Not mentioned
2.
1 week
3. >1,
4 weeks
4. >1 month,
6 months
5. >6 months

7
30
27
35
9

0.770
0.479
0.552
0.566
0.287

4.181***
5.175***
5.644***
6.870***
1.942

[0.409e1.130]
[0.298e0.661]
[0.360e0.743]
[0.405e0.728]
[0.003 to 0.577]

Domain subjects
1. Language arts
2. Social studies
3. Science
4. Mathematics
5. General
6. Professional subjects

39
5
27
12
6
27

0.473
0.768
0.565
0.337
0.151
0.592

6.352***
3.682***
6.397***
2.628**
0.868
6.808***

[0.327e0.619]
[0.359e1.177]
[0.392e0.738]
[0.086e0.588]
[0.190 to 0.491]
[0.422e0.763]

QB

R2

9.226*

0%

18.426***

7%

3.025

0%

7.993*

8%

26.744***

12%

4.924

0%

9.108

0%

Note. CI condence interval.

*p < .05; **p < .01; ***p < .001.

were short, students could efciently use their short periods of spare time to take small bites out of the material. Another
example is the use of cell phones to communicate, make records, and give and receive feedback. These functions can remind
students about their learning schedule, and promote self-awareness (Liu, Tao, & Nee, 2008; Runyan et al., 2013) and selfregulation (Kondo et al., 2012). The aforementioned advantages of the handhelds created the environment for seamless
learning, which should be able to prompt better learning outcomes.
According to the analysis result, the implementation of handhelds induced higher learning outcomes than the implementation of laptops. It is perhaps due to the fact that studies with handhelds tend to integrate innovative teaching methods
(Lu, 2012). Among the handheld research, there was 31.6% employing teaching methods, such as inquiry-oriented and
cooperative learning (Table C1 of Appendix C). In contrast, in a large portion of the laptop-related studies (50.0%, Table C1 of
Appendix C), the computers were placed into the classroom and used simply for lectures, self-directed study, or with no
specic teaching methods.
It is important to note here that most of the research on handhelds in education has involved only short-term interventions, with 29.1% (Table C2 of Appendix C) testing their effectiveness within 1 week. These users of handhelds also
probably experienced a transient effect because of their novelty (Kulik & Kulik, 1991). In contrast, most of the research on
laptops involved long-term use, with 25.0% (Table C2 of Appendix C) being used for > 6 months. Long-term laptop use without
appropriate supporting logistics may reduce both the students' level of commitment and the teachers' willingness to use
computers to integrate their teaching with the students' learning (Drayton, Falk, Stroud, Hobbs, & Hammerman, 2010; Inan &
Lowther, 2010; Penuel, 2006).

262

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

3.3.3. Software used

The data given in Table 2 indicated that the effect sizes for learning-oriented software (g 0.590, z 9.699, p < .001)
approached high effect size, and general-purpose software (g 0.429, z 5.407, p < .001) had medium effect size. The QB did
not achieve signicance at the p < .05 level (QB 3.025, p .220), which means that the average effect size did not differ
signicantly between the two categories.
According to the survey results, after 1990 most of the software that the teachers used was actually made for general
purposes (e.g., word processors, spreadsheets, and web browsers) (Becker, 1991, 2001; Drayton et al., 2010), instead of
learning-oriented software tailored for teaching and learning tasks. This made it difcult for most teachers to achieve the goal
of greater efciency and effectiveness in education using the technology-adapted instruction that they applied (Sung &
Lesgold, 2007; Weston & Bain, 2010). The present study indicates that the aforementioned shortage of learning-oriented
software has improved, with software specically designed for teaching and learning goals or activities being used in
62.7% of the research, and only 34.5% of the studies using general-purpose software.
Even though there was no signicant difference between learning oriented software and general-purpose software in our
research, learning oriented software showed interesting features in mobile based learning. First, the software and the curriculum were closely integrated. As an example, Looi and colleagues (Looi et al., 2011; Zhang et al., 2010) combined educational software with cell phones to make a mobilized curriculum for elementary-level natural science, which was able to
implement seamless learning in classrooms, outdoors, and in the home. Their designs were not only based on the pedagogy of
inquiry learning, but also promoted formative assessment, cooperative learning, and social interaction in teaching tasks. The
second feature of learning-oriented software is that it provides diverse educational activities. Within the studies included in
this research, those in which learning-oriented software was used implemented various educational methods, most of which
were related to inquiry, cooperation, game-based learning, problem-solving, and formative assessment. On the other hand,
for those studies using general-purpose software, lectures and self-directed study were implemented. Moreover, among the
37 studies with the general-purpose software, 6 of them did not mention the teaching methods (Table C1 of Appendix C). The
third feature of learning-oriented software is its ability to enable elaborate and efcient designs for teaching strategies and
learning scenarios. The steps and procedures of the aforementioned teaching strategies, such as inquiry, cooperation, gamebased learning, and problem-solving, were all fairly complex. Learning-oriented software allowed teachers with no programming skills to exibly and efciently implement mobile-assisted education. For example, Lan et al. (2007, Lan, Sung, &
Chang, 2009) designed an English foreign-language learning model based on cooperative learning and reciprocal teaching.
Procedures related to reciprocal teaching, such as reading text, questioning and probing, answering and feedback, were all
designed for specic modules that could be further arranged according to the needs of different teaching situations. Teachers
could substitute their own material, or even completely customize their program. In addition, the research of Roschelle et al.
(2010) on cooperative learning set out three stages of design and implementation for modules, modules for experiments and
classroom tryouts, and modules for classroom implementation. After 2 years of designs, tryouts, and revisions, their PDAbased cooperative learning modules were able to integrate the mathematics content, cooperative learning procedures, and
teacher-training programs for efcient use in the classroom.
3.3.4. Implementation settings
As indicated in Table 2, when the not-mentioned category is ignored, informal settings had a high effect size (g 0.768,
z 7.096, p < .001), while unrestricted settings (g 0.550, z 5.887, p < .001) and formal settings (g 0.430, z 7.328,
p < .001) had medium effect sizes. The effect size of informal setting was larger than that of the formal setting, as the 95% of
condence intervals of the two effect sizes did not overlap. The QB was signicant (QB 7.993, p .046), showing that the
average effect size differed signicantly with the category. The R2 was 8%, meaning that 8% of total between-study variance in
effects can be explained by implementation settings.
As found in the present study, the effect size was larger for using mobile devices in the outdoors and informal locations
than for using them in more formal places. Some observations on the use of mobile devices in informal places may be helpful
for explaining this phenomenon. First, this could be due to the motivation induced by the novelty of the technology and
activities. Students are keen to go outside or to museums to learn, and combining this with the use of novel learning tools can
facilitate learners' motivation (e.g., Zhang, Sung, Hou, & Chang, 2014). The second is that most of the informal educational
models, software functionality, and hardware characteristics were closely integrated in the included research, and this
probably improved the learning effects. In the present study, 77.9% of informal learning-oriented software was specially
designed for specic learning scenarios in specic settings (Table C3 of Appendix C). These more elaborately designed
teaching procedures allow educational effects to become more apparent. For instance, when learning in museums, one of the
important issues is how to guide learners' attention to exhibitions through an appropriate learning process, and informative
and interesting activities to promote interaction among visitors, computers, and the historical contexts (e.g., Hsi, 2003; Sung,
Hou, Liu, & Chang, 2010). Several of the studies included in our research combined the models of role-playing games and
problem-solving to immerse learners in the historical events, engaging them to observe and learn target exhibits more deeply
(e.g., Huizenga, Admiraal, Akkerman, & ten Dam, 2009; Sung, Chang, Hou, & Chen, 2010). Similarly, researchers are also
concerned with how to make the eldwork involved in the natural and social sciences structuralized, focused, and efcient,
rather than loose, absent-minded, and ineffective. In several studies (e.g., Hwang, Chu, Lin, & Tsai, 2011; Liu, Tan, & Chu, 2009),
the researchers tried to make observations, note-taking, problem-solving, information exchanges, and discussion more

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

263

structured, and to sharply focus the students' learning process by integrating mobile devices with other peripheral devices
such as camcorders, positioning functions, and measuring facilities.
3.3.5. Teaching methods
The data regarding the effect size for different teaching methods are given in Table 2. Three high effect sizes were found for
inquiry-oriented (g 0.844, z 8.400, p < .001), mixed methods (g 0.839, z 5.702, p < .001), and computer-assisted
testing (g 0.656, z 3.661, p < .001). Lectures (g 0.394, z 3.120, p .002) and self-directed study (g 0.440,
z 5.492, p < .001) were around medium effect sizes. However, cooperative learning (g 0.261, z 1.673, p .094) and
game-based learning (g 0.407, z 1.922, p .055) did not show signicant effect sizes. The QB achieved statistical signicance (QB 26.744, p < .001), indicating that the average effect sizes differed signicantly among the various categories.
The R2 was 12%, meaning that 12% of total between-study variance in effects can be explained by teaching methods.
The unique features of mobile devices can enhance the essential functionalities of certain specic teaching methods, and
thus promote educational outcomes. Because each student has his own mobile device, this individuality combined with
wireless communication enabled more accessible self-paced and self-directed study. Combining the features of individuality
and instant message delivery resolves the past difculties of putting instant formative assessment into the classroom (e.g.,
Chen & Chen, 2009), such that these assessments can even be performed outdoors with equal ease (e.g., Shih, Kuo, & Liu,
2012). Another feature that empowers the teaching and learning process is the portability and context awareness of mobile devices. These two features allow learners to exploit the information in the environments in which they are situated, and
to retrieve, record, and react to the data needed to resolve their learning issues by traversing multiple learning environments,
such as eldwork and museums (e.g., Tan, Liu, & Chang, 2007).
It is note-worthy that although researchers (Kukulska-Hulme & Shield, 2008; Roschelle & Pea, 2002) have proposed that
conveying information and giving feedback via mobile devices can help to keep learners in touch with their peers, promote
discussions, and to facilitate the effects of cooperative learning, our study found that in general theses features did not help
enhance cooperative learning outcomes. The researchers of cooperative learning used mobile devices' features of individuality and sharing coupled with mechanisms for enhancing social interaction, such as co-constructing concept maps (Lai
& Wu, 2006), peer evaluation (Lan et al., 2007; Roschelle et al., 2010), and building consensus (Zurita & Nussbaum, 2004).
Interestingly, perhaps these methods had facilitated the positive interactive relationships among team members (e.g., Lan
et al., 2007; Zurita & Nussbaum, 2004), however, these teaching methods did not enhance the learning outcomes
compared with the cooperative scenarios without using mobile devices. There are at least two possible reasons for the results.
Firstly, the cooperative learning tasks in those studies, when coupled with mobile devices, may be helpful for increasing the
interactive behaviors and social cohesions among team members. However, the increased social cohesion may not be
powerful enough to enhance learning achievement. As Slavin (2012) proposed, whether higher social cohesion is related with
higher learning achievement is not conclusive. Those methods used in the above-noted research may be insufcient to
empower the cognitive elaboration processes imperative for enhancing students' learning. In those studies students in both
the control and treatment groups received cooperative treatments: The only difference was mobile-device usage. Thus, the
inherent effects of mobile devices may not go much beyond sharing, communicating, and consensus building. Therefore,
elaborate design of learning scenarios, such as mechanisms for prompting questioning and explanatory strategies (Byun, Lee,
& Cerreto, 2014; Gillies & Haynes, 2011) specically related with the learning content, may be needed to be incorporated into
the mobile-device based activities in order to enhance students' cognitive elaboration processes and outcomes. The second
possible reason is that the intervention durations of the mobile-based cooperative learning programs were not long enough
to produce positive effects. Researchers have proposed that several weeks of duration is helpful for producing positive
learning outcomes in cooperative learning (Slavin, 1993), as sufcient time is important for learners to get familiar with team
members, tasks, and required procedure (Slavin, 1977). Time for familiarization may be even more important for mobiledevices based cooperative learning because learners need time to get familiar not only with members, tasks, and procedure, but also with the hardware and software. Most of the research included in our study lasted for less than one month,
which may be too short for the programs to produce sound effects.
Another note-worthy nding is that game-based learning did not achieve a signicant overall effect in mobile learning,
either. The major reason may be that most of the studies (e.g., Ketamo, 2003; Kim et al., 2011; Riconscente, 2013) focused on
using the mobile devices to provide learners with a handy and individualized game-based environment to enhance their
motivation and engagement. However, the relationships between the concepts to be learned and the content of the game may
not have been closely integrated, and therefore the effects of learning might not have been illustrated.
Researchers have pointed out that computer interventions in education have not yet led to practical implementations of
innovative educational methods (Ertmer & Ottenbreit-Leftwich, 2010; Gerard, Varma, Corliss, & Linn, 2011). Contrarily, it was
found in the present study that mobile devices seemed to elicit much more diverse and innovative educational methods from
researchers.
3.3.6. Intervention duration
When the not-mentioned category is ignored, interventions of >1 month and
6 months duration (g 0.566, z 6.870,
p < .001), those of >1 week and
4 weeks duration (g 0.552, z 5.644, p < .001), and those
1 week had medium effect
sizes (g 0.479, z 5.175, p < .001). Interestingly, interventions conducted for durations of >6 months had a non-signicant

264

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

effect size (g 0.287, z 1.942, p .052). The QB was not signicant (QB 4.924, p .295), which suggests that the effect size
did not differ signicantly between these categories.
The non-signicance of the effect size in long-term duration (>6 months) is counterintuitive, but consistent with those of
Kulik and Kulik (1991), who found that computer-based instruction had a greater effect when the duration was shorter. Kulik
and Kulik (1991) and Cheung and Slavin (2013) proposed three reasons for why short-term treatments have better effects:
high novelty value, stronger interventional supports, and different measurement tools for the dependent variables. These
explanations are also applicable to the present ndings. In most studies with intervention durations less than 6 months, the
use of mobile devices and the applied teaching methods were both novel, so the students were more easily engaged in the
activity. Cross-analysis of intervention duration with other moderator variables provides data that supports these arguments.
For example, most research that took place over a 6-month period used general-purpose software (66.7%; Table C2 of
Appendix C), which did not necessarily match the needs of the learning scenarios in specic learning topics. Furthermore,
around half of the studies (44.4%; Table C2 of Appendix C) with durations of >6 months placed the computers directly in the
classroom and did not specify the teaching methods to be used to achieve specic educational goals. Conversely, 57.1% of the
studies lasting for >1 month and
6 months used learning-oriented software for specic teaching and learning goals, and
94.3% specied a specic teaching strategy instead of simply using computers for some unspecied purpose in the classroom
(Table C2 of Appendix C).
In terms of the interventional supports, in most short-term studies, researchers could gather all of their resources for one
shot, so they chose the most appropriate hardware and software with more diverse functionality, prepared more elaborate
learning activities, and made every effort to control confounding factors. However, in studies lasting >6 months, the longer
duration made it more difcult to support the use of diverse resources, nding logistic assistance for technological problems,
and maintaining the enthusiasm associated with using new technologies. For example, Shapley, Sheehan, Maloney, and
Caranikas-Walker (2010) found that in laptop immersion schools, after four years of implementation, only 6 of 21 schools
reached a substantial level of immersion, and the level of student access and use of laptops in classrooms declined during the
period of implementation because of insufcient support.
Research in the eld of education mostly advocates that long-term teaching interventions are important for obtaining
reliable results (Hsieh et al., 2005; Pressley & Harris, 1994), but in the present study it was found that long-term interventions
with mobile devices in classrooms did not necessarily lead to better effects. Such ndings echo comments made by many
researchers about the use of laptops in the classroom: If computers are simply given to teachers and students to use for a long
time without any positive guidance, it will not necessarily produce satisfactory educational outcomes (Holcomb, 2009;
Zucker & Light, 2009), especially for higher levels learning skills such as reasoning and problem solving (Drayton et al.,
2010). In order for there to be abundant effects, long-term interventions need logistical support to integrate advanced
technologies with innovative and elaborate educational methods. Information technology applications in the classroom must
rst go through adoption and adaptation before they can proceed to innovation. These processes are also likely to take longer
than 1 year (Gerard et al., 2011), or even up to 3 years (Bebell & O'Dwyer, 2010). During such a long-term process, if the main
support provided to teachers and students is enthusiasm rather than appropriate support such as hardware, software, and
instructional designs, computer use in the classroom will ultimately be merely supercial.
3.3.7. Domain subjects
The data in Table 2 indicate the effect sizes for different domain subjects. Social studies (g 0.768, z 3.682, p < .001) had
a high effect size, while professional subjects (g 0.592, z 6.808, p < .001), science (g 0.565, z 6.397, p < .001), language
arts (g 0.473, z 6.352, p < .001) and mathematics (g 0.337, z 2.628, p .009) had medium effect sizes. No signicant
effect size was obtained for using mobile devices for domain-general abilities (g 0.151, z 0.868, p .386). The QB did not
achieve statistical signicance (QB 9.108, p .105), which shows that the average effect size did not differ signicantly
among these categories.

3.4. Evaluation of publication bias

The classic fail-safe N and Orwin's fail-safe N were adopted to demonstrate the publication bias for the 108 selected
studies. As suggested by the data in Table 3, the classic fail-safe N test determined that a total of 4144 studies with null results
would be needed in order to nullify the effect size. Moreover, the results of Orwin's fail-safe N test (see Table 4) show that the

Table 3
Results of the classic fail-safe N.
Z value for observed studies
p value for observed studies
Alpha
Tail
Z for alpha
Number of observed studies
Number of missing studies that would bring the p value to >alpha

22.51
0.00
0.05
2.00
1.96
108.00
4144.00

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

265

Table 4
Results of Orwin's fail-safe N.
Hedges' g in observed studies (xed effect)
Criterion for a trivial Hedges' g
Mean Hedges' g in missing studies
Number of missing studies needed to bring Hedge's g to under 0.01

0.33
0.01
0.00
3423.00

number of missing null studies required to bring the existing overall mean effect size to a trivial level (g 0.01) was 3423.
Both tests suggest that publication bias could not explain the signicant positive effects observed across all studies.
4. Conclusions and implications
Analysis of the empirical research on the use of mobile devices as tools in educational interventions that were published in
peer-reviewed journals has revealed that the overall effect of using mobile devices in education is better than when using
desktop computers or not using mobile devices as an intervention, with a moderate effect size of 0.523. Through the analysis
of moderator variables, we found that many different combinations of hardware, software, and intervention durations for
mobile devices have been applied to various ages of users, implementation settings, teaching methods, and domain subjects.
The effect of such usage was greater for handhelds than for laptops; usage in inquiry-oriented learning was more effective
than usage along with lectures, self-directed study, cooperative learning, and game-based learning; informal educational
environments were more effective than their formal counterparts, and medium- and short-duration interventions were
superior to long-term interventions. These ndings will contribute to a better understanding of where, for whom, and in
which way the use of mobile devices in the learning environment will best highlight the effects of particular educational
methods, and reveal the limitations of mobile devices in education.
Based on the ndings of this study, it is proposed that more elaborate instructional design developments are needed to
more thoroughly exploit the educational benets possible by utilizing mobile devices. We believe that the three implications
proposed below will be helpful for facilitating and achieving these goals.
4.1. Leveraging the pedagogical effects of mobile devices through elaborate designs of learning/teaching scenarios
Mobile devices have various distinctive features such as individualized interfaces, real-time access to information, context
sensitivity, instant communication, and feedback. These features may be able enhance the effects of certain pedagogies, such
as self-directed learning, inquiry learning, or formative assessment. However, it is note-worthy that the features of mobile
devices are not sufcient conditions for positive learning effects. The minor effects of mobile-device-based cooperative and
game-based learning in our study illustrated this fact. Instructional strategies are important for effective learning with information technology (Lan, 2014; Lan, Sung, Cheng, & Chang, 2015; Liu, Lin, & Paas, 2014). Researchers must nd the key to
integrating mobile devices with instructional strategies and ingeniously match the unique features of mobile devices to the
resolution of specic pedagogic challenges. Doing so will maximize the impact of those features on learning outcomes.
Some examples include using the instant-feedback functions to solve the difculty of efciently executing and managing
formative assessment in a class with many students (Penuel, Roschelle, & Shechtman, 2007) and, for cooperative groups,
using wireless communication to facilitate between-group scaffoldings and to avoid idling (Lan et al., 2007). As one of the
most used strategies in mobile learning/teaching, self-directed study is an example of a method that deserves more attention
paid to pairing specic features to specic challenges to yield improved results. In addition, most of the studies in our research
utilized mobile devices' features of individuality and wireless communication capacity for self-directed learning, such as
learning vocabularies through messaging services or using word processors for writing. However, few studies in our research
provided their mechanisms for using the instant feedback to facilitate the interaction between mobile devices and users (e.g.,
Oberg & Daniels, 2013; Ozcelik & Acarturk, 2011), which is an important element of effective self-directed learning with
computers. Therefore, more elaborate methods of implementation, such as a monitoring mechanisms for learning EFL vocabularies through the message services of cell phones, an annotation system for reading e-books (e.g., Hwang, Shadiev, &
Huang, 2011), speech recognition for providing feedback to students' oral practices (e.g., Tanner & Landon, 2009), etc.,
should be considered to enhance the interaction between learner and computers and the effects of self-directed learning.
4.2. Enhancing the quality of the experimental design for mobile intervention
While it was found in this study that mobile devices can enhance educational effects, the actual impact of mobile learning
programs needs to be enhanced by longer intervention durations, closer integration of technology and the curriculum, and
further assessment of higher-level skills.
The intervention duration will affect the reliability and ecological validity of mobile learning programs. Of all of the
included interventions in this study, those with durations of >6 months constituted only about 8.3% of the research, and more
than 27.2% took place within 1 week. With short programs, and especially those that last for only hours, it is difcult to prove
that any effects are produced by the features of mobile-integrated instruction rather than by the experience of technology

266

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

novelty. Moreover, short-term projects may adapt poorly to regular classroom practices that may last for several months.
Another issue related to teaching duration is the closeness of the integration between mobile devices and the curriculum.
Most of the short programs included in our study involved only one or two units of teaching materials in the curriculum of a
whole semester. Although it is not necessary for a teacher to use mobile devices in every class, different units or topics may
involve different instructional designs when such devices are being used, and hence an iterative trial process is likely to be
needed to determine the optimal procedure for the best effects. Therefore, an abundance of mobile learning units will help to
provide exemplar models for teachers and enhance the possibility of transferring practices to different lessons. Furthermore,
in terms of research, it can improve the reliability and ecological validity of mobile programs in education. Based on the above
considerations, researchers may consider appropriate intervention durations according to the skills or teaching methods to be
developed with mobile devices. For example, for vocabulary-learning, bite-size materials and short-term durations may be
appropriate for learners, but for more complex skills or methods such as inquiry or cooperative learning, longer interventions
may be needed to warrant the effect of mobile programs.
Another effect of mobile usage that could be strengthened is the expansion of measurements of dependent variables. Most
of the studies in our research currently still placed the interests on achievement in content knowledge (e.g., Liu, 2009; Wang
& Wu, 2011), and methods for measuring higher-level skills were scarce. Mobile devices were expected to encourage
innovation in education and increase high-level abilities (Frohberg et al., 2009; Sung, Chang, & Yang, 2015; Zucker & Light,
2009). Yet most of the research collected for this study focused on increasing content learning, and even though the
designed educational activities involve explorative, communication, and cooperative skills, the dependent variables had
almost no connection with these skills. For example, in the database of our research, only 5 of the 9 experimental/quasi
experimental studies explored the interactive behaviors of students during their mobile learning; furthermore, none of the 24
inquiry-oriented learning recorded and investigated process-related skills such as hypothesis-formation and hypothesistesting. Therefore, including dependent variables besides content knowledgedsuch as problem-solving, critical thinking,
interactive communication, or creative innovation skillsdin the measurements will make the persuasiveness of the
educational effects of mobile devices much more convincing.
4.3. Empowering educational practitioners through the orchestration of mobile devices, software, and pedagogical design
Scholars (e.g., Gao, Liu, & Paas, in press; Liu et al., 2012) have gradually reached a consensus that exerting the maximum
effect of information technology in the educational eld requires reconciliation of the connection among the components of
technology (hardware and software), educational context and missions (e.g., learning and teaching processes in different
settings), and users (teachers and students) in order to overcome many of the limitations present in the eld. Scholars
rez, 2013) came to agree that the
(Dillenbourg, Nussbaum, Dimitriadis, & Roschelle, 2013; Dimitriadis, Prieto, & Asensio-Pe
efforts of building harmonious relationships among those components to enable compatible, efcient, and effective
technology-enhanced teaching and learning environments may be called orchestration. To achieve orchestration in mobileintegrated education requires the pursuit of at least two directions for research and practices. The rst is strengthening the
functions and expanding the applicability and breadth of learning-oriented software. For example, the research analyzed in
this study paired many different learning-oriented software programs with educational activities (e.g., reciprocal teaching,
inquiry learning, and formative assessment) that have already proven effective. That software may be modied to provide the
functionality of authoring tools that allow teachers to exibly arrange their own teaching and learning ows in the classroom.
The second direction is strengthening professional teacher-development programs for mobile-enhanced instruction. Most
review research into the use of mobile devices for education has emphasized that one of the largest obstacles to implementing effective mobile learning programs is insufcient preparation of the teachers (Frohberg et al., 2009; Penuel, 2006).
The essence of effective professional development for technology-enhanced inquiry proposed by Gerard et al. (2011) is also
applicable to mobile learning programs. Teachers should be encouraged to modify already developed mobile-integrated
education programs, and to gradually customize them into their own personalized program rather than simply designing
their own program around the use of technology. The latter approach implicitly leads teachers to technology-adapted instruction, which means that the educational practices of the teachers may be restricted by the functions of technology, and
may make it difcult for teachers to change their existing beliefs and habits. In contrast, customizing existing research-based
mobile learning programs not only transfers researchers' visions and experiences for the use of technology to teachers, but
also minimizes the time teachers spend on formulating new ideas and performing trial-and-error iterative procedures
(Gerard et al., 2011; Penuel et al., 2007). To facilitate the transition of researchers' vision, experiences, and skills to school
teachers, it is also helpful to involve university-level researchers as mentors or collaborators. Diverse functions and types of
hardware and software are available for mobile devices, but conversely the complexity is also high, and hence designing and
using them can readily impose additional overhead on teachers. The plethora of technological knowledge and resources that
are available to researchers for educational technology means that their participation in a program can result in their
knowledge and experience greatly assisting the teachers' autonomy in implementation.
Another note-worthy fact is that, despite the importance of teachers' professional development during their adoption of
and adaptation to mobile-device based teaching (Newhouse, Williams, & Pearson, 2006; Penuel & Yarnall, 2005), the investigations into increasing the education of teachers regarding the use of mobile devices have been extremely limited.
Therefore, more in-depth experimental research is needed into how teachers reconcile mobile hardware and software, lesson
content, teaching methods, and educational goals.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

267

Acknowledgment
This research was supported by grants from the Ministry of Science and Technology, Taiwan (101-2511-S-003-047-MY3;
102-2911-I-003-301; 102-2511-S-003-001-MY3; 103-2911-I-003-301; 103-2511-S-003 -058 -MY3; 104-2511-S-003 -018
-MY3), and the Aim for the Top University Project of National Taiwan Normal University.
Appendix A. Related review of the research into Integrating Mobile Devices with Teaching and Learning.

Study

Devices are
focused on

Method

Number of studies Result

Penuel (2006) synthesized ndings from research and evaluation studies that analyzed
implementation and effects of one-to-one initiatives from a range of countries. Factors
related to successful implementation reported in the research include extensive
teacher professional development, access to technical support, and positive teacher
attitudes toward student technology use. Penuel (2006) found that outcome studies
with rigorous designs are few, but those studies that did measure outcomes
consistently reported positive effects on technology use, technology literacy, and
writing skills.
Frohberg et al. Laptops
Narrative 102 (mobile
Frohberg et al. (2009) used a mobile learning framework to evaluate and categorize
(2009)
review
learning projects) 102 mobile learning projects, and to briey introduce exemplary projects for each
category. Despite the fact that mobile phones initially started as a communication
device, communication and collaboration play a surprisingly small role in Mobile
Learning projects.
Zucker and
Laptops
Narrative 31(not provided
Zucker and Light (2009) found research in many nations suggests that laptop programs
Light (2009)
review
publication list)
will be most successful as part of balanced, comprehensive initiatives that address
changes in education goals, curricula, teacher training, and assessment.
Laptops
Narrative 5
Bebell and O'Dwyer (2010) summarized evidence that participation in the 1:1
Bebell and
review
computer programs was associated with increased student and teacher technology
O'Dwyer
use, increased student engagement and interest level, and modest increases in student
(2010)
achievement.
Hwang and
Various types of Content 154(not provided Hwang and Tsai (2011) examined the mobile or ubiquitous learning papers published
Tsai (2011) mobile device
analysis publication list)
in the Social Science Citation Index (SSCI) database from 2001 to 2010. Hwang and Tsai
(2011) found that the number of articles has signicantly increased during the past 10
years; moreover, researchers from the different countries have contributed to the
related eld in recent years.
Wong and Looi Laptops
Narrative 54
Wong and Looi (2011) aimed to further investigate the meaning of seamless learning
(2011)
review
and the potential ways to put it in practice. Through a thorough review of recent
academic papers on mobile-assisted seamless learning (MSL), Wong and Looi (2011)
identify ten dimensions that characterize MSL.
Fleischer
Narrative review Narrative 18
Fleischer (2012) reviewed cross-disciplinary accumulated empirical research on one(2012)
review
to-one computer projects in school settings as published in peer-reviewed journals
between 2005 and 2010, particularly the results of teacher- and pupil-oriented
studies. The results of Fleischer (2012) show that the research field has not developed
substantially since the previously published reviews. One the other hand, Fleischer
(2012) discussed the reasons for this lack of development, as well as the need for
political, scholarly and epistemological awareness when researching questions of oneto-one computer projects.

Penuel (2006) Laptops

Narrative 30(not provided

review
publication list)

Appendix B. Forest plot of the effect sizes and 95% CI of the 110 articles.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

269

Appendix C. Cross analyses of moderator variables.

Table C1
Cross-analysis of teaching methods, domain subjects, and hardware used.
Teaching method
Not
Lectures
mentioned

Inquiry-oriented Cooperative
learning
learning

Domain
Language
6 (15.4%) 1 (2.6%)
2 (5.1%)
subjects arts
Social studies 0 (0.0%)
0 (0.0%) 3 (60.0%)
Science
2 (7.4%)
5 (18.5%) 11 (40.7%)
Mathematics 2 (16.7%)
0 (0.0%) 2 (16.7%)
General
3 (50.0%)
0 (0.0%)
0 (0.0%)
Professional
1 (3.7%)
6 (22.2%) 6 (22.2%)
subjects
Total
14 (12.1%) 12 (10.3%) 24 (20.7%)
Hardware Not
used
mentioned
Handhelds
Laptops
Mixed
Total
Software
used

Not
mentioned
General
purpose
Learningoriented

Total

0 (0.0%)
2
6
1
9

0 (0.0%)

Game-based
learning

Selfdirected
study

5 (12.8%)

0 (0.0%)

20 (51.2%)

0 (0.0%)
1 (3.7%)
2 (16.7%)
0 (0.0%)
2 (7.4%)

1 (20%)
0 (0.0%)
3 (25.0%)
0 (0.0%)
0 (0.0%)

0 (0.0%)
2 (7.4%)
1 (8.3%)
3 (50.0%)
8 (29.6%)

4 (3.4%)

34 (29.3%)

10 (8.6%)

1 (50.0%)

0 (0.0%)

0 (0.0%)

(2.5%) 10 (12.7%) 18 (22.8%)

(25.0%) 2 (8.3%)
5 (20.8%)
(33.3%)
0 (0.0%)
0 (0.0%)
(8.3%) 12 (11.1%) 24 (22.2%)

7 (8.9%)
2 (8.3%)
0 (0.0%)
9 (8.3%)

3 (3.8%)
1 (4.2%)
0 (0.0%)
4 (3.7%)

1 (50.0%)
27
4
2
34

Mixed

Computerassisted testing

Total

4 (10.3%) 1 (2.6%)

39 (100%)

0 (0.0%)
5 (18.5%)
1 (8.3%)
0 (0.0%)
0 (0.0%)

5
27
12
6
27

10 (8.6%)

1 (20%)
1 (3.7%)
1 (8.3%)
0 (0.0%)
4 (14.8%)
8 (6.9%)

0 (0.0%)

0 (0.0%)

(34.2%)
(16.7%)
(66.7%)
(31.5%)

6 (7.6%)
4 (16.7%)
0 (0.0%)
10 (9.3%)

6 (7.6%)
0 (0.0%)
0 (0.0%)
6 (5.5%)

0 (0.0%)

0 (0.0%)

(100%)
(100%)
(100%)
(100%)
(100%)

116 (100%)
2 (100%)
79
24
3
108

(100%)
(100%)
(100%)
(100%)

1 (33.3%)

0 (0.0%)

0 (0.0%)

1 (33.3%)

0 (0.0%)

1 (33.3%)

6 (16.2%)

9 (24.3%)

5 (13.5%)

1 (2.7%)

0 (0.0%)

13 (35.1%)

2 (5.4%)

1 (2.7%)

37 (100%)

2 (2.9%)

3 (4.4%)

19 (27.9%)

7 (10.2%)

4 (5.9%)

20 (29.4%)

8 (11.8%) 5 (7.4%)

68 (100%)

9 (8.3%)

4 (3.7%)

34 (31.5%)

9 (8.3%)

12 (11.1%) 24 (22.2%)

10 (9.3%)

6 (5.6%)

3 (100%)

108 (100%)

Table C2
Cross-analysis for intervention durations, hardware used, software used, and teaching methods.
Intervention duration
Not mentioned

1 week

>1,
4 weeks

>1 month,
6 months

>6 months

Total

Hardware used

Not mentioned
Handhelds
Laptops
Mixed
Total

0 (0.0%)
4 (5.1%)
3 (12.5%)
0 (0.0%)
7 (6.5%)

0 (0.0%)
23 (29.1%)
5 (20.8%)
2 (66.7%)
30 (27.8%)

0 (0.0%)
20 (25.3%)
6 (25.0%)
1 (33.3%)
27 (25.0%)

2 (100.0%)
29 (36.7%)
4 (16.7%)
0 (0.0%)
35 (32.4%)

0 (0.0%)
3 (3.8%)
6 (25.0%)
0 (0.0%)
9 (8.3%)

2
79
24
3
108

(100.0%)
(100.0%)
(100.0%)
(100.0%)
(100.0%)

Software used

Not mentioned
General purpose
Learning-oriented
Total

1
2
4
7

(33.3%)
(5.4%)
(5.9%)
(6.5%)

0 (0.0%)
7 (18.9%)
23 (33.8%)
30 (27.8%)

2
7
18
27

(66.7%)
(18.9%)
(26.5%)
(25.0%)

0 (0.0%)
15 (40.5%)
20 (29.4%)
35 (32.4%)

0 (0.0%)
6 (16.2%)
3 (4.4%)
9 (8.3%)

3
37
68
108

(100.0%)
(100.0%)
(100.0%)
(100.0%)

Teaching method

Not mentioned
Lecture
Inquiry-oriented
Cooperative learning
Game-based learning
Self-directed study
Mixed
Computer-assisted testing
Total

1 (11.1%)
1 (8.3%)
3 (12.5%)
0 (0.0%)
0 (0.0%)
1 (2.9%)
0 (0.0%)
1 (16.7%)
7 (6.5%)

1 (11.1%)
5 (41.7%)
9 (37.5%)
2 (22.2%)
2 (50.0%)
9 (26.5%)
0 (0.0%)
2 (33.3%)
30 (27.8%)

1 (11.1%)
0 (0.0%)
5 (20.8%)
4 (44.4%)
2 (50.0%)
13 (38.2%)
1 (10.0%)
1 (16.7%)
27 (25.0%)

2 (22.2%)
5 (41.7%)
7 (29.2%)
2 (22.2%)
0 (0.0%)
11 (32.4%)
6 (60.0%)
2 (33.3%)
35 (32.4%)

4 (44.4%)
1 (8.3%)
0 (0.0%)
1 (11.1%)
0 (0.0%)
0 (0.0%)
3 (30.0%)
0 (0.0%)
9 (8.3%)

9
12
24
9
4
34
10
6
108

(100.0%)
(100.0%)
(100.0%)
(100.0%)
(100.0%)
(100.0%)
(100.0%)
(100.0%)
(100.0%)

270

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Table C3
Cross-analysis for implementation settings and software used.
Software used

Implementation setting

Total

Not mentioned
Formal settings
Informal settings
Unrestricted

Not mentioned

General purpose

Learning-oriented

Total

0 (0.0%)
2 (3.3%)
0 (0.0%)
1 (4.0%)
3 (2.8%)

1
22
4
10
37

1
36
17
14
68

2
60
21
25
108

(50.0%)
(36.7%)
(19.0%)
(40.0%)
(34.3%)

(50.0%)
(60.0%)
(81.0%)
(56.0%)
(63.0%)

(100.0%)
(100.0%)
(100.0%)
(100.0%)
(100.0%)

References*
Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education (6th ed.). Belmont, CA: Wadsworth.
Bakhurst, D. (2009). Reections on activity theory. Educational Review, 61(2), 197e210. http://dx.doi.org/10.1080/00131910902846916.
lu, E. B., & Akdemir, O. (2010). A comparison of undergraduate students' English vocabulary learning: using mobile phones and ash cards. The Turkish
Basog
Online Journal of Educational Technology, 9, 1e7. Retrieved from http://eric.ed.gov/?idEJ898010.
Bebell, D., & O'Dwyer, L. M. (2010). Educational outcomes and research from 1:1 computing settings. Journal of Technology, Learning, and Assessment, 9, 5e15.
Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1606.
Becker, H. J. (1991). How computers are used in United States schools: basic data from the 1989 I.E.A. Computers in Education survey. Journal of Educational
Computing Research, 7, 385e405. http://dx.doi.org/10.2190/P2UT-R3U3-FK1L-B89L.
Becker, H. J. (2001, April). How are teachers using computers in instruction?. Paper presented at the 2001 Annual Meeting of American Educational Research
Association, Seattle, WA.
Best, J. W., & Kahn, J. V. (1998). Research in education (8th ed.). Boston, MA: Allyn and Bacon.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.
Byun, H., Lee, J., & Cerreto, F. A. (2014). Relative effects of three questioning strategies in ill structured, small group problem solving. Instructional Science, 42,
229e250. http://dx.doi.org/10.1007/s11251-013-9278-1.
Chan, T. W., Roschelle, J., Hsi, S., Kinshuk, Sharples, M., Brown, T., et al. (2006). One-to-one technology-enhanced learning: an opportunity for global
research collaboration. Research and Practice in Technology Enhanced Learning, 1(01), 3e29. http://dx.doi.org/10.1142/S1793206806000032.
Chen, C. M., & Chen, M. C. (2009). Mobile formative assessment tool based on data mining techniques for supporting web-based learning. Computers &
Education, 52, 256e273. http://dx.doi.org/10.1016/j.compedu.2008.08.005.
Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms:
a meta-analysis. Educational Research Review, 9, 88e113. http://dx.doi.org/10.1016/j.edurev.2013.01.001.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Dierking, L. D., Falk, J. H., Rennie, L., Anderson, D., & Ellenbogen, K. (2003). Policy statement of the informal science education ad hoc committee. Journal of
Research in Science Teaching, 40, 108e111. http://dx.doi.org/10.1002/tea.10066.
Dillenbourg, P., Nussbaum, M., Dimitriadis, Y., & Roschelle, J. (2013). Design for classroom orchestration. Computers & Education, 69, 485e492. http://dx.doi.
org/10.1016/j.compedu.2012.10.026.
rez, J. I. (2013). The role of design and enactment patterns in orchestration: helping to integrate technology in
Dimitriadis, Y., Prieto, L. P., & Asensio-Pe
blended classroom ecosystems. Computers & Education, 69, 496e499. http://dx.doi.org/10.1016/j.compedu.2013.04.004.
Drayton, B., Falk, J. K., Stroud, R., Hobbs, K., & Hammerman, J. (2010). After installation: ubiquitous computing and high school science in three experienced,
high-technology schools. Journal of Technology, Learning, and Assessment, 9, 4e56. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/
view/1608.
Ertmer, P., & Ottenbreit-Leftwich, A. (2010). Teacher technology change: how knowledge, beliefs, and culture intersect. Journal of Research on Technology in
Education, 42, 255e284. http://dx.doi.org/10.1080/15391523.2010.10782551.
Fleischer, H. (2012). What is our current understanding of one-to-one computer projects: a systematic narrative research review. Educational Research
Review, 7, 107e122. http://dx.doi.org/10.1016/j.edurev.2011.11.004.
Frohberg, D., Goth, C., & Schwabe, G. (2009). Mobile learning projects e a critical analysis of the state of the art. Journal of Computer Assisted Learning, 25,
307e331. http://dx.doi.org/10.1111/j.1365-2729.2009.00315.x.
Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: a meta-analysis.
Review of Educational Research, 82, 300e329. http://dx.doi.org/10.3102/0034654312457206.
Gao, Y., Liu, T.C., & Paas, F. (in press). Effects of mode of target task selection on learning about plants in a mobile learning environment: effortful manual
selection versus effortless QR-code selection. Journal of Educational Psychology. doi: http://dx.doi.org/10.1037/edu0000080.
Gerard, L. F., Varma, K., Corliss, S. B., & Linn, M. C. (2011). Professional development for technology-enhanced inquiry science. Review of Educational Research,
81, 408e448. http://dx.doi.org/10.3102/0034654311415121.
Gillies, R. M., & Haynes, M. (2011). Increasing explanatory behavior, problem solving, and reasoning within classes using cooperative group work.
Instructional Science, 39, 349e366. http://dx.doi.org/10.1007/s11251-010-9130-9.
Gleaves, A., Walker, C., & Grey, J. (2007). Using digital and paper diaries for learning and assessment purposes in higher education: a comparative study of
feasibility and reliability. Assessment & Evaluation in Higher Education, 32, 631e643. http://dx.doi.org/10.1080/02602930601117035.
Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London, England: Routledge.
Holcomb, L. B. (2009). Results & lessons learned from 1:1 laptop initiatives: a collective review. TechTrends, 53, 49e55. http://dx.doi.org/10.1007/s11528009-0343-1.
Hsi, S. (2003). A study of user experiences mediated by nomadic web content in a museum. Journal of Computer Assisted Learning, 19, 308e319. http://dx.doi.
org/10.1046/j.0266-4909.2003.jca_023.x.
Hsieh, P., Acee, T., Chung, W., Hsieh, Y. P., Kim, H., Thomas, G. D., et al. (2005). Is educational intervention research on the decline? Journal of Educational
Psychology, 97, 523e529. http://dx.doi.org/10.1037/0022-0663.97.4.523.
Hsu, C. K., Hwang, G. J., Chang, Y. T., & Chang, C. K. (2013). Effects of video caption modes on English listening comprehension and vocabulary acquisition
using handheld devices. Educational Technology & Society, 16(1), 403e414. Retrieved from http://www.ifets.info/journals/16_1/35.pdf.
Hsu, L., & Lee, S. N. (2011). Learning tourism English on mobile phones: how does it work? Journal of Hospitality, Leisure, Sport and Tourism Education, 10,
85e94. http://dx.doi.org/10.3794/johlste.102.348.

Please see the Further readings for the studies included for meta-analysis.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

271

Huizenga, J., Admiraal, W., Akkerman, S., & ten Dam, G. (2009). Mobile game-based learning in secondary education: engagement, motivation and learning
in a mobile city game. Journal of Computer Assisted Learning, 25, 332e344. http://dx.doi.org/10.1111/j.1365-2729.2009.00316.x.
Hwang, G. J., Chu, H. C., Lin, Y. S., & Tsai, C. C. (2011). A knowledge acquisition approach to developing Mindtools for organizing and sharing differentiating
knowledge in a ubiquitous learning environment. Computers & Education, 57, 1368e1377. http://dx.doi.org/10.1016/j.compedu.2010.12.013.
Hwang, G. J., & Tsai, C. C. (2011). Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010. British
Journal of Educational Technology, 42, 65e70. http://dx.doi.org/10.1111/j.1467-8535.2011.01183.x.
Hwang, W. Y., Shadiev, R., & Huang, S. M. (2011). A study of a multimedia web annotation system and its effect on the EFL writing and speaking performance
of junior high school students. ReCALL, 23(2), 160e180. http://dx.doi.org/10.1017/S0958344011000061.
Inan, F. A., & Lowther, D. L. (2010). Laptops in the K-12 classrooms: exploring factors impacting instructional use. Computers & Education, 55, 937e944.
http://dx.doi.org/10.1016/j.compedu.2010.04.004.
Jabbar, A. I. A., & Felicia, P. (2015). Gameplay engagement and learning in game-based learning: a systematic review. Review of Educational Research, 85(4),
740e779. http://dx.doi.org/10.3102/0034654315577210.
Jeong, H. Y., & Hong, B. H. (2013). A practical use of learning system using user preference in ubiquitous computing environment. Multimedia Tools and
Applications, 64, 491e504. http://dx.doi.org/10.1007/s11042-012-1026-z.
Jian, H. L., Sandnes, F. E., Law, K. M. Y., Huang, Y. P., & Huang, Y. M. (2009). The role of electronic pocket dictionaries as an English learning tool among
Chinese students. Journal of Computer Assisted Learning, 25, 503e514. http://dx.doi.org/10.1111/j.1365-2729.2009.00325.x.
Ketamo, H. (2003). An adaptive geometry game for handheld devices. Educational Technology & Society, 6(1), 83e95. Retrieved from http://www.ifets.info/
journals/6_1/ketamo.html.
Kim, P., Buckner, E., Kim, H., Makanya, T., Taleja, N., & Parikh, V. (2011). A comparative analysis of a game-based mobile learning model in lowsocioeconomic communities of India. International Journal of Educational Development, 32, 329e340. http://dx.doi.org/10.1016/j.ijedudev.2011.05.008.
Klopfer, E., Sheldon, J., Perry, J., & Chen, V. H.-H. (2012). Ubiquitous games for learning (UbiqGames): Weatherlings, a worked example. Journal of Computer
Assisted Learning, 28, 465e476. http://dx.doi.org/10.1111/j.1365-2729.2011.00456.x.
Kondo, M., Ishikawa, Y., Smith, C., Sakamoto, K., Shimomura, H., & Wada, N. (2012). Mobile assisted language learning in university EFL courses in Japan:
developing attitudes and skills for self-regulated learning. ReCALL, 24, 169e187. http://dx.doi.org/10.1017/S0958344012000055.
Kukulska-Hulme, A., & Shield, L. (2008). An overview of mobile assisted language learning: from content delivery to supported collaboration and interaction. ReCALL, 20, 271e289. http://dx.doi.org/10.1017/S0958344008000335.
Kulik, C. L. C., & Kulik, J. A. (1991). Effectiveness of computer-based instruction: an updated analysis. Computers in Human Behavior, 7, 75e94. http://dx.doi.
org/10.1016/0747-5632(91)90030-5.
Lai, C. Y., & Wu, C. C. (2006). Using handhelds in a jigsaw cooperative learning environment. Journal of Computer Assisted Learning, 22, 284e297. http://dx.
doi.org/10.1111/j.1365-2729.2006.00176.x.
Lan, Y. J., Sung, Y. T., & Chang, K. E. (2007). A mobile-device-supported peer-assisted learning system for collaborative early EFL reading. Language Learning &
Technology, 11, 130e151. Retrieved from http://llt.msu.edu/vol11num3/pdf/lansungchang.pdf.
Lan, Y. J., Sung, Y. T., & Chang, K. E. (2009). Let us read together: development and evaluation of a computer-assisted reciprocal early English reading system.
Computers & Education, 53, 1188e1198. http://dx.doi.org/10.1016/j.compedu.2009.06.002.
Lan, Y. J. (2014). Does second life improve mandarin learning by overseas Chinese students? Language Learning & Technology, 18, 36e56. Retrieved from
http://llt.msu.edu/issues/june2014/action2.pdf.
Lan, Y. J., Sung, Y. T., Cheng, C. C., & Chang, K. E. (2015). Computer-supported cooperative prewriting for enhancing young EFL learners' writing performance.
Language Learning & Technology, 19, 134e155. Retrieved from http://llt.msu.edu/issues/june2015/lansungchengchang.pdf.
Langman, J., & Fies, C. (2010). Classroom response system-mediated science learning with English language learners. Language and Education, 24, 81e99.
http://dx.doi.org/10.1080/09500780903096553.
Li, L. Y., Chen, G. D., & Yang, S. J. (2013). Construction of cognitive maps to improve e-book reading and navigation. Computers & Education, 60, 32e39. http://
dx.doi.org/10.1016/j.compedu.2012.07.010.
Lipsey, M. W., & Wilson, D. B. (2000). Practical meta-analysis. Thousand Oaks, CA: Sage.
Liu, T. C., Lin, Y. C., & Paas, F. (2014). Effects of prior knowledge on learning from different compositions of representations in a mobile learning environment.
Computers & Education, 72, 328e338. http://dx.doi.org/10.1016/j.compedu.2013.10.019.
Liu, C. C., Tao, S. Y., & Nee, J. N. (2008). Bridging the gap between students and computers: supporting activity awareness for network collaborative learning
with GSM network. Behaviour & Information Technology, 27, 127e137. http://dx.doi.org/10.1080/01449290601054772.
Liu, T. C., Lin, Y. C., Tsai, M. J., & Paas, F. (2012). Split-attention and redundancy effects in mobile learning in physical environments. Computers & Education,
58, 172e180. http://dx.doi.org/10.1016/j.compedu.2011.08.007.
Liu, T. Y. (2009). A context-aware ubiquitous learning environment for language listening and speaking. Journal of Computer Assisted Learning, 25(6),
515e527. http://dx.doi.org/10.1111/j.1365-2729.2009.00329.x.
Liu, T. Y., Tan, T. H., & Chu, Y. L. (2009). Outdoor natural science learning with an RFID-supported immersive ubiquitous learning environment. Educational
Technology & Society, 12, 161e175. Retrieved from http://www.ifets.info/journals/12_4/15.pdf.
Looi, C. K., Zhang, B., Chen, W., Seow, P., Chia, G., Norris, C., et al. (2011). 1:1 mobile inquiry learning experience for primary science students: a study of
learning effectiveness. Journal of Computer Assisted Learning, 27, 269e287. http://dx.doi.org/10.1111/j.1365-2729.2010.00390.x.
Lu, M. (2008). Effectiveness of vocabulary learning via mobile phone. Journal of Computer Assisted Learning, 24, 515e525. http://dx.doi.org/10.1111/j.13652729.2008.00289.x.
Lu, Z. J. (2012). Learning with mobile technologies, handheld devices, and smart phones: Innovative methods (pp. 1e272). IGI-Global.
McMillan, J. H., Venable, J. J., & Varier, D. (2013). Studies of the effect of formative assessment on student achievement: so much more is needed. Practical
Assessment, Research & Evaluation, 18(2). Retrieved from http://pareonline.net/pdf/v18n2.pdf.
Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11, 364e386. http://dx.doi.org/10.
1177/1094428106291059.
Mouza, C. (2008). Learning with laptops: implementation and outcomes in an urban, under-privileged school. Journal of Research on Technology in Education,
40, 447e472. http://dx.doi.org/10.1080/15391523.2008.10782516.
Newhouse, C. P., Williams, P. J., & Pearson, J. (2006). Supporting mobile education for pre-service teachers. Australasian Journal of Educational Technology, 22,
289e311. Retrieved from http://www.ascilite.org.au/ajet/ajet22/newhouse.html.
Oberg, A., & Daniels, P. (2013). Analysis of the effect a student-centred mobile learning instructional method has on language acquisition. Computer Assisted
Language Learning, 26(2), 177e196. http://dx.doi.org/10.1080/09588221.2011.649484.
Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational and Behavioral Statistics, 8, 157e159. http://dx.doi.org/10.3102/
10769986008002157.
Ozcelik, E., & Acarturk, C. (2011). Reducing the spatial distance between printed and online information sources by means of mobile technology enhances
learning: using 2D barcodes. Computers & Education, 57(3), 2077e2085. http://dx.doi.org/10.1016/j.compedu.2011.05.019.
Penuel, W. R. (2006). Implementation and effects of 1:1 computing initiatives: a research synthesis. Journal of Research on Technology in Education, 38,
329e348. http://dx.doi.org/10.1080/15391523.2006.10782463.
Penuel, W. R., Roschelle, J., & Shechtman, N. (2007). Designing formative assessment software with teachers: an analysis of the co-design process. Research
and Practice in Technology Enhanced Learning, 2, 51e74. http://dx.doi.org/10.1142/S1793206807000300.
Penuel, W. R., & Yarnall, L. (2005). Designing handheld software to support classroom assessment: analysis of conditions for teacher adoption. Journal of
Technology, Learning, and Assessment, 3, 1e46. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1658.

272

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Pressley, M., & Harris, K. R. (1994). Increasing the quality of educational intervention research. Educational Psychology Review, 6, 191e208. http://dx.doi.org/
10.1007/BF02213181.
Purrazzella, K., & Mechling, L. C. (2013). Evaluation of manual spelling, observational and incidental learning using computer-based instruction with a tablet
PC, large screen projection, and a forward chaining procedure. Education and Training in Autism and Developmental Disabilities, 48, 218e235. Retrieved
from http://daddcec.org/Portals/0/CEC/Autism_Disabilities/Research/Publications/Education_Training_Development_Disabilities/Full_Journals/ETADD_
48_2_218-235.pdf.
Riconscente, M. M. (2013). Results from a controlled study of the iPad fractions game motion math. Games and Culture, 8, 186e214. http://dx.doi.org/10.1177/
1555412013496894.
Roschelle, J., & Pea, R. D. (2002). A walk on the WILD side: how wireless handhelds may change computer-supported collaborative learning. International
Journal of Cognition and Technology, 1, 145e168. Retrieved from https://hal.archives-ouvertes.fr/le/index/docid/190615/lename/A110_Roschelle_Pea_
02_WILD.pdf.
Roschelle, J., Rafanan, K., Bhanot, R., Estrella, G., Penuel, B., Nussbaum, M., et al. (2010). Scaffolding group explanation and feedback with handheld
technology: impact on students' mathematics learning. Educational Technology Research and Development, 58, 399e419. http://dx.doi.org/10.1007/
s11423-009-9142-9.
Rosenthal, R. (1979). The le drawer problem and tolerance for null results. Psychological Bulletin, 86, 638e641. http://dx.doi.org/10.1037/0033-2909.86.3.
638.
Runyan, J. D., Steenbergh, T. A., Bainbridge, C., Daugherty, D. A., Oke, L., & Fry, B. N. (2013). A smartphone ecological momentary assessment/intervention
App for collecting real-time data and promoting self-awareness. PLoS ONE, 8, e71325. http://dx.doi.org/10.1371/journal.pone.0071325.
Ryu, H., & Parsons, D. (2012). Risky business or sharing the load? Social ow in collaborative mobile learning. Computers & Education, 58, 707e720. http://
dx.doi.org/10.1016/j.compedu.2011.09.019.
lu, G., & Cag
ltay, K. (2012). Mobile language learning: contribution of multimedia messages via mobile phones in consolidating voSaran, M., Seferog
cabulary.
The
Asia-Pacic
Education
Researcher,
21,
181e190.
Retrieved
from
http://www.ejournals.ph/index.php?
journalTAPER&pagearticle&opviewArticle&path%5B%5D4826.
Shapley, K. S., Sheehan, D., Maloney, C., & Caranikas-Walker, F. (2010). Evaluating the implementation delity of technology immersion and its relationship
with student achievement. Journal of Technology, Learning, and Assessment, 9, 5e68. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/
view/1609.
Sharples, M., Taylor, J., & Vavoula, G. (2007). A theory of learning for the mobile age. In R. Andrews, & C. Haythornthwaite (Eds.), The Sage Handbook of
Elearning Research (pp. 221e247). London, UK: Sage Publications.
Shih, S. C., Kuo, B. C., & Liu, Y. L. (2012). Adaptively ubiquitous learning in campus Math path. Educational Technology & Society, 15, 298e308. Retrieved from
http://www.ifets.info/journals/15_2/25.pdf.
Siozos, P., Palaigeorgiou, G., Triantafyllakos, G., & Despotakis, T. (2009). Computer based testing using digital ink: participatory design of a tablet PC based
assessment application for secondary education. Computers & Education, 52, 811e819. http://dx.doi.org/10.1016/j.compedu.2008.12.006.
Slavin, R. E. (1977). Classroom reward structure: an analytical and practical review. Review of Educational Research, 47, 633e650. Retrieved from http://www.
jstor.org/stable/pdfplus/1170003.pdf?acceptTCtrue&jpdConrmtrue.
Slavin, R. E. (1993). When and why does cooperative learning increase achievement? Theoretical and empirical perspectives. In H. Daniels, & A. Edwards
(Eds.), The RoutledgeFalmer reader in psychology of education (pp. 271e293). New York, NY: RoutledgeFalmer.
Slavin, R. E. (2012). Instruction based on cooperative learning. In R. E. Mayer, & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp.
344e360). New York, NY: Routledge.
Sung, Y. T., Chang, K. E., Hou, H. T., & Chen, P. F. (2010). Designing an electronic guidebook for learning engagement in a museum of history. Computers in
Human Behavior, 26, 74e83. http://dx.doi.org/10.1016/j.chb.2009.08.004.
Sung, Y. T., Hou, H. T., Liu, C. K., & Chang, K. E. (2010c). Mobile guide system using problem-solving strategy for museum learning: a sequential learning
behavioural pattern analysis. Journal of Computer Assisted Learning, 26, 106e115. http://dx.doi.org/10.1111/j.1365-2729.2010.00345.x.
Sung, Y. T., & Lesgold, A. (2007). Software infrastructure for teachers: a missing link in integrating technology with instruction. Teachers College Record, 109,
2541e2575. Retrieved from http://www.tcrecord.org/Content.asp?contentid14536.
Sung, Y. T.,Chang, K. E., & Yang, J. M. (in press). How effective are mobile devices for language learning? A meta-analysis. Educational Research Review. doi:
10.1016/j.edurev.2015.09.001.
Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on
learning: a second-order meta-analysis and validation study. Review of Educational Research, 81, 4e28. http://dx.doi.org/10.3102/0034654310393361.
Tan, T. H., Liu, T. Y., & Chang, C. C. (2007). Development and evaluation of an RFID-based ubiquitous learning environment for outdoor learning. Interactive
Learning Environments, 15, 253e269. http://dx.doi.org/10.1080/10494820701281431.
Tanner, M. W., & Landon, M. M. (2009). The effects of computer-assisted pronunciation readings on ESL learners' use of pausing, stress, intonation, and
overall comprehensibility. Language Learning and Technology, 13, 51e65. Retrieved from: http://llt.msu.edu/vol13num3/tannerlandon.pdf.
Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer-based learning environment on students' learning outcomes: a
meta-analysis. Review of Educational Research, 85(4), 1e37. http://dx.doi.org/10.3102/0034654314564881.
Wang, S. L., & Wu, C. Y. (2011). Application of context-aware and personalized recommendation to implement an adaptive ubiquitous learning system.
Expert Systems with Applications, 38(9), 10831e10838. http://dx.doi.org/10.1016/j.eswa.2011.02.083.
Warschauer, M. (2007). A teacher's place in the digital divide. Yearbook of the National Society for the Study of Education, 106, 147e166. http://dx.doi.org/10.
1111/j.1744-7984.2007.00118.x.
Warschauer, M., Zheng, B., Niiya, M., Cotten, S., & Farkas, G. (2014). Balancing the one-to-one equation: equity and access in three laptop programs. Equity &
Excellence in Education, 47, 46e62. http://dx.doi.org/10.1080/10665684.2014.866871.
Weston, M., & Bain, A. (2010). The end of techno-critique: the naked truth about 1:1 laptop initiatives and educational change. Journal of Technology,
Learning, and Assessment, 9, 1e26. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1611.
Wong, L. H., & Looi, C. K. (2011). What seams do we remove in mobile assisted seamless learning? A critical review of the literature. Computers & Education,
57, 2364e2381. http://dx.doi.org/10.1016/j.compedu.2011.06.007.
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games.
Journal of Educational Psychology, 105(2), 249e265. http://dx.doi.org/10.1037/a0031311.
Wu, T. T., Sung, T. W., Huang, Y. M., Yang, C. S., & Yang, J. T. (2011). Ubiquitous English learning system with dynamic personalized guidance of learning
portfolio. Educational Technology & Society, 14, 164e180. Retrieved from http://www.ifets.info/journals/14_4/15.pdf.
Yang, Y., Zhang, L., Zeng, J., Pang, X., Lai, F., & Rozelle, S. (2013). Computers and the academic performance of elementary school-aged girls in China's poor
communities. Computers & Education, 60, 335e346. http://dx.doi.org/10.1016/j.compedu.2012.08.011.
Zhang, B. H., Looi, C. K., Seow, P., Chia, G., Wong, L. H., Chen, W., et al. (2010). Deconstructing and reconstructing: transforming primary science learning via a
mobilized curriculum. Computers & Education, 55, 1504e1523. http://dx.doi.org/10.1016/j.compedu.2010.06.016.
Zhang, J., Sung, Y. T., Hou, H. T., & Chang, K. E. (2014). The development and evaluation of an augmented reality-based armillary sphere for astronomical
observation instruction. Computers & Education, 73, 178e188. http://dx.doi.org/10.1016/j.compedu.2014.01.003.
Zucker, A. A., & Light, D. (2009). Laptop programs for students. Science, 323, 82e85. http://dx.doi.org/10.1126/science.1167705.
Zurita, G., & Nussbaum, M. (2004). Computer supported collaborative learning using wirelessly interconnected handheld computers. Computers & Education, 42, 289e314. http://dx.doi.org/10.1016/j.compedu.2003.08.005.
Zurita, G., & Nussbaum, M. (2007). A conceptual framework based on activity theory for mobile CSCL. British Journal of Educational Technology, 38(2),
211e235. http://dx.doi.org/10.1111/j.1467-8535.2006.00580.x.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

273

Further readings (The studies included for meta-analysis)

Ahmed, S., & Parsons, D. (2013). Abductive science inquiry using mobile devices in the classroom. Computers & Education, 63, 62e72. http://dx.doi.org/10.
1016/j.compedu.2012.11.017.
Agbatogun, A. O. (2012). Exploring the efcacy of student response system in a sub-Saharan African country: a sociocultural perspective. Journal of Information Technology Education, 11, 249e267. Retrieved from http://eric.ed.gov/?idEJ990469.
lu, E. B., & Akdemir, O. (2010). A comparison of undergraduate students' English vocabulary learning: using mobile phones and ash cards. The Turkish
Basog
Online Journal of Educational Technology, 9, 1e7. Retrieved from http://les.eric.ed.gov/fulltext/EJ898010.pdf.
Bebell, D., & Kay, R. (2010). One to one computing: a summary of the quantitative results from the Berkshire wireless learning initiative. The Journal of
Technology, Learning, and Assessment, 9, 1e60. Retrieved from http://napoleon.bc.edu/ojs/index.php/jtla/article/viewFile/1607/1462.
Billings, E. S., & Mathison, C. (2012). I get to use an iPod in school? Using technology-based advance organizers to support the academic success of English
learners. Journal of Science Education and Technology, 21, 494e503. http://dx.doi.org/10.1007/s10956-011-9341-0.
Brooks, G., Miles, J. N. V., Torgerson, C. J., & Torgerson, D. J. (2006). Is an intervention using computer software effective in literacy learning? A randomised
controlled trial. Educational Studies, 32, 133e143. Retrieved from http://www.jeremymiles.co.uk/mestuff/publications/p41.pdf.
Bruce-Low, S. S., Burnet, S., Arber, K., Price, D., Webster, L., & Stopforth, M. (2013). Interactive mobile learning: a pilot study of a new approach for sport
science and medical undergraduate students. Advances in Physiology Education, 37, 292e297. http://dx.doi.org/10.1152/advan.00004.2013.
Chaiprasurt, C., & Esichaikul, V. (2013). Enhancing motivation in online courses with mobile communication tool support: a comparative study. The International Review of Research in Open and Distance Learning, 14, 377e401. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1416/
2610.
Chang, K. E., Lan, Y. J., Chang, C. M., & Sung, Y. T. (2010). Mobile-device-supported strategy for Chinese reading comprehension. Innovations in Education and
Teaching International, 47, 69e84. http://dx.doi.org/10.1080/14703290903525853.
Chao, P. Y., & Chen, G. D. (2009). Augmenting paper-based learning with mobile phones. Interacting With Computers, 21, 173e185. http://dx.doi.org/10.1016/j.
intcom.2009.01.001.
Chen, C. M., & Chen, M. C. (2009). Mobile formative assessment tool based on data mining techniques for supporting web-based learning. Computers &
Education, 52, 256e273. http://dx.doi.org/10.1016/j.compedu.2008.08.005.
Chen, C. M., & Li, Y. L. (2010). Personalised context-aware ubiquitous learning system for supporting effective English vocabulary learning. Interactive
Learning Environments, 18, 341e364. http://dx.doi.org/10.1080/10494820802602329.
Chen, C. M., Tan, C. C., & Lo, B. J. (2013). Facilitating English-language learners' oral reading uency with digital pen technology. Interactive Learning Environments, 1e23. http://dx.doi.org/10.1080/10494820.2013.817442.
Chen, N. S., Teng, D. C. E., Lee, C. H., & Kinshuk. (2011). Augmenting paper-based reading activity with direct access to digital materials and scaffolded
questioning. Computers & Education, 57, 1705e1715. http://dx.doi.org/10.1016/j.compedu.2011.03.013.
Chen, Y. (2010). Dictionary use and EFL learning. A contrastive study of pocket electronic dictionaries and paper dictionaries. International Journal of
Lexicography, 23, 275e306. http://dx.doi.org/10.1093/ijl/ecq013.
Chen, Y. S., Kao, T. C., & Sheu, J. P. (2005). Realizing outdoor independent learning with a buttery-watching mobile learning system. Journal of Educational
Computing Research, 33, 395e417. Retrieved from http://www.csie.ntpu.edu.tw/~yschen/mypapers/JECR-2005.pdf.
Chiu, L. L., & Liu, G. Z. (2013). Effects of printed, pocket electronic, and online dictionaries on high school students English vocabulary retention. The AsiaPacic Education Researcher, 22, 619e634. http://dx.doi.org/10.1007/s40299-013-0065-1.
Chu, H. C., Hwang, G. J., & Tsai, C. C. (2010). A knowledge engineering approach to developing mindtools for context-aware ubiquitous learning. Computers &
Education, 54, 289e297. http://dx.doi.org/10.1016/j.compedu.2009.08.023.
De Joodea, E. A., Van Heugtena, C. M., Verheya, F. R. J., & Van Boxtela, M. P. J. (2013). Effectiveness of an electronic cognitive aid in patients with acquired
brain injury: a multicentre randomised parallel-group study. Neuropsychological Rehabilitation: An International Journal, 23, 133e156. http://dx.doi.org/
10.1080/09602011.2012.726632.
Du, H., Hao, J. X., Kwok, R., & Wagner, C. (2010). Can a lean medium enhance large-group communication? Examining the impact of interactive mobile
learning. Journal of the American Society for Information Science and Technology, 61, 2122e2137. http://dx.doi.org/10.1002/asi.21376.
Dunleavy, M., & Heinecke, W. F. (2007). The impact of 1:1 laptop use on middle school math and science standardized test scores. Computers in the Schools:
Interdisciplinary Journal of Practice, Theory, and Applied Research, 24, 7e22. http://dx.doi.org/10.1300/J025v24n03_02.
Eden, S., Shamir, A., & Fershtman, M. (2011). The effect of using laptops on the spelling skills of students with learning disabilities. Educational Media
International, 48, 249e259. http://dx.doi.org/10.1080/09523987.2011.632274.
Edwards, C. M., Rule, A. C., & Boody, R. M. (2013). Comparison of face-to-face and online mathematics learning of sixth graders. Computers in Mathematics
and Science Teaching, 32, 25e47. Retrieved from http://www.editlib.org/noaccess/39231/.
Gardner, J., Morrison, H., & Jarman, R. (1993). The impact of high access to computers on learning. Journal of Computer Assisted Learning, 9, 2e16. http://dx.
doi.org/10.1111/j.1365-2729.1993.tb00259.x.
Gebru, M. T., Phelps, A. J., & Wulfsberg, G. (2012). Effect of clickers versus online homework on students' long-term retention of general chemistry course
material. Chemistry Education Research and Practice, 13, 325e329. http://dx.doi.org/10.1039/C2RP20033C.
Gulek, J. C., & Demirtas, H. (2005). Learning with technology: the impact of laptop use on student achievement. The Journal of Technology, Learning, and
Assessment, 3, 1e38. Retrieved from http://napoleon.bc.edu/ojs/index.php/jtla/article/view/1655/1501.
Hassan, I. S., Ismail, M. A., & Mustapha, R. (2010). The effects of integrating mobile and CAD technology in teaching design process for Malaysian polytechnic
architecture student in producing creative product. Turkish Online Journal of Educational Technology, 9, 162e172. Retrieved from http://les.eric.ed.gov/
fulltext/EJ908082.pdf.
Hayati, A., Jalilifar, A., & Mashhadi, A. (2013). Using short message service (SMS) to teach English idioms to EFL students. British Journal of Educational
Technology, 44, 66e81. http://dx.doi.org/10.1111/j.1467-8535.2011.01260.x.
Ho, K., Lauscher, H. N., Broudo, M., Jarvis-Selinger, S., Fraser, J., Hewes, D., et al. (2009). The impact of a personal digital assistant (PDA) case log in a medical
student clerkship. Teaching and Learning in Medicine: An International Journal, 21, 318e326. http://dx.doi.org/10.1080/10401330903228554.
Hsu, L., & Lee, S. N. (2011). Learning tourism English on mobile phones: how does it work? Journal of Hospitality, Leisure, Sport and Tourism Education, 10,
85e94. http://dx.doi.org/10.3794/johlste.102.348.
Huang, Y. M., Liang, T. H., Su, Y. N., & Chen, N. S. (2012). Empowering personalized learning with an interactive e-book learning system for elementary school
students. Educational Technology Research and Development, 60, 703e722. http://dx.doi.org/10.1007/s11423-012-9237-6.
Huang, Y. M., Lin, Y. T., & Cheng, S. C. (2010). Effectiveness of a mobile plant learning system in a science curriculum in Taiwanese elementary education.
Computers & Education, 54, 47e58. http://dx.doi.org/10.1016/j.compedu.2009.07.006.
Huizenga, J., Admiraal, W., Akkerman, S., & ten Dam, G. (2009). Mobile game-based learning in secondary education: engagement, motivation and learning
in a mobile city game. Journal of Computer Assisted Learning, 25, 332e344. http://dx.doi.org/10.1111/j.1365-2729.2009.00316.x.
Hung, P. H., Hwang, G. J., Lin, Y. F., Wu, T. H., & Su, I. H. (2013). Seamless connection between learning and assessment applying progressive learning tasks in
mobile ecology inquiry. Educational Technology & Society, 16, 194e205. Retrieved from http://www.ifets.info/journals/16_1/17.pdf.
Hwang, G. J., & Chang, H. F. (2011). A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of
students. Computers & Education, 56, 1023e1031. http://dx.doi.org/10.1016/j.compedu.2010.12.002.
Hwang, G. J., Chu, H. C., Lin, Y. S., & Tsai, C. C. (2011). A knowledge acquisition approach to developing Mindtools for organizing and sharing differentiating
knowledge in a ubiquitous learning environment. Computers & Education, 57, 1368e1377. http://dx.doi.org/10.1016/j.compedu.2010.12.013.
Hwang, G. J., Kuo, F. R., Yin, P. Y., & Chuang, K. H. (2010). A Heuristic Algorithm for planning personalized learning paths for context-aware ubiquitous
learning. Computers & Education, 54, 404e415. http://dx.doi.org/10.1016/j.compedu.2009.08.024.

274

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Hwang, G. J., Shi, Y. R., & Chu, H. C. (2011). A concept map approach to developing collaborative Mindtools for context-aware ubiquitous learning. British
Journal of Educational Technology, 42, 778e789. http://dx.doi.org/10.1111/j.1467-8535.2010.01102.x.
Hwang, G. J., Wu, C. H., Tseng, J. C. R., & Huang, I. (2011). Development of a ubiquitous learning platform based on a real-time help-seeking mechanism.
British Journal of Educational Technology, 42, 992e1002. http://dx.doi.org/10.1111/j.1467-8535.2010.01123.x.
Hwang, G. J., Wu, P. H., Zhuang, Y. Y., & Huang, Y. M. (2013). Effects of the inquiry-based mobile learning model on the cognitive load and learning
achievement of students. Interactive Learning Environments, 21, 338e354. http://dx.doi.org/10.1080/10494820.2011.575789.
Hwang, W. Y., & Chen, H. S. L. (2013). Users' familiar situational contexts facilitate the practice of EFL in elementary schools with mobile devices. Computer
Assisted Language Learning, 26, 101e125. http://dx.doi.org/10.1080/09588221.2011.639783.
Hwang, W. Y., Chen, H. S. L., Shadiev, R., Huang, R. Y. M., & Chen, C. Y. (2012). Improving English as a foreign language writing in elementary schools using
mobile devices in familiar situational contexts. Computer Assisted Language Learning, 27, 1e20. http://dx.doi.org/10.1080/09588221.2012.733711.
Jones, S. J., Crandall, J., Vogler, J. S., & Robinson, D. H. (2013). Classroom response systems facilitate student accountability, readiness, and learning. Journal of
Educational Computing Research, 49, 155e171. http://dx.doi.org/10.2190/EC.49.2.b.
Kang, H., Lundeberga, M., Wolter, B., delMas, R., & Herreid, C. F. (2012). Gender differences in student performance in large lecture classrooms using personal
response systems (clickers) with narrative case studies. Learning, Media and Technology, 37, 53e76. http://dx.doi.org/10.1080/17439884.2011.556123.
Karaman, S. (2011). Effects of audience response systems on student achievement and long-term retention. Social Behavior and Personality: An International
Journal, 39, 1431e1439. http://dx.doi.org/10.2224/sbp.2011.39.10.1431.
Kert, S. B. (2011). The use of SMS support in programming education. Turkish Online Journal of Educational Technology, 10, 268e273. Retrieved from http://
les.eric.ed.gov/fulltext/EJ932245.pdf.
Kert, S. B. (2013). Using J-Query mobile technology to support a pedagogical prociency course. Journal of Educational Computing Research, 48, 431e445.
http://dx.doi.org/10.2190/EC.48.4.b.
Ketamo, H. (2003). An adaptive geometry game for handheld devices. Educational Technology & Society, 6, 83e95. Retrieved from http://www.ifets.info/
journals/6_1/ketamo.html.
Kim, P., Buckner, E., Kim, H., Makany, T., Taleja, N., & Parikh, V. (2012). A comparative analysis of a game-based mobile learning model in low-socioeconomic
communities of India. International Journal of Educational Development, 32, 329e340. http://dx.doi.org/10.1016/j.ijedudev.2011.05.008.
Kim, P., Hagashi, T., Carillo, L., Gonzales, I., Makany, T., Lee, B., et al. (2011). Socioeconomic strata, mobile technology, and education: a comparative analysis.
Educational Technology Research and Development, 59, 465e486. http://dx.doi.org/10.1007/s11423-010-9172-3.
Kondo, M., Ishikawa, Y., Smith, C., Sakamoto, K., Shimomura, H., & Wada, N. (2012). Mobile assisted language learning in university EFL courses in Japan:
developing attitudes and skills for self-regulated learning. ReCALL, 24, 169e187. http://dx.doi.org/10.1017/S0958344012000055.
de La Fuente, M. J. (2012). Learners' attention to input during focus on form listening tasks: the role of mobile technology in the second language classroom.
Computer Assisted Language Learning, 27, 261e276. http://dx.doi.org/10.1080/09588221.2012.733710.
Lai, C. H., Yang, J. C., Chen, F. C., Ho, C. W., & Chan, T. W. (2007). Affordances of mobile technologies for experiential learning: the interplay of technology and
pedagogical practices. Journal of Computer Assisted Learning, 23, 326e337. http://dx.doi.org/10.1111/j.1365-2729.2007.00237.x.
Lai, C. Y., & Wu, C. C. (2006). Using handhelds in a Jigsaw cooperative learning environment. Journal of Computer Assisted Learning, 22, 284e297. Retrieved
from http://140.115.126.240/mediawiki/images/6/6d/Jigsaw.pdf.
Lan, Y. F., Tsai, P. W., Yang, S. H., & Hung, C. L. (2012). Comparing the social knowledge construction behavioral patterns of problem-based online asynchronous discussion in e/m-learning environments. Computers & Education, 59, 1122e1135. http://dx.doi.org/10.1016/j.compedu.2012.05.004.
Lan, Y. J., Sung, Y. T., & Chang, K. E. (2009). Let us read together: development and evaluation of a computer-assisted reciprocal early English reading system.
Computers & Education, 53, 1188e1198. http://dx.doi.org/10.1016/j.compedu.2009.06.002.
Lan, Y. J., Sung, Y. T., Tan, N. C., Lin, C. P., & Chang, K. E. (2010). Mobile-device-supported problem-based computational estimation instruction for elementary
school students. Educational Technology & Society, 13, 55e69. Retrieved from http://www.thai-library.org/Resource/0000005765.pdf#page60.
Lawrence, L. E., & Victorina, W. (2004). Laptops, technology, and algebra 1: a case study of an experiment. Mathematics Teacher, 97, 136e142. Retrieved from
http://personal.stevens.edu/~llevine/mt_laptop_study.pdf.
Lin, Y. C., Liu, T. C., & Chu, C. C. (2011). Implementing clickers to assist learning in science lectures: the clicker-assisted conceptual change model. Australasian
Journal of Educational Technology, 27, 979e996. Retrieved from http://ascilite.org.au/ajet/ajet27/lin.html.
Liu, C. C., Tao, S. Y., & Nee, J. N. (2008). Bridging the gap between students and computers: supporting activity awareness for network collaborative learning
with GSM network. Behaviour & Information Technology, 27, 127e137. http://dx.doi.org/10.1080/01449290601054772.
Liu, T. Y. (2009). A context-aware ubiquitous learning environment for language listening and speaking. Journal of Computer Assisted Learning, 25, 515e527.
http://dx.doi.org/10.1111/j.1365-2729.2009.00329.x.
Liu, T. Y., & Chu, Y. L. (2010). Using ubiquitous games in an English listening and speaking course: impact on learning outcomes and motivation. Computers &
Education, 55, 630e643. http://dx.doi.org/10.1016/j.compedu.2010.02.023.
Liu, T. Y., Tan, T. H., & Chu, Y. L. (2009). Outdoor natural science learning with an RFID-supported immersive ubiquitous learning environment. Educational
Technology & Society, 12, 161e175. Retrieved from http://www.ifets.info/journals/12_4/15.pdf.
Looi, C. K., Zhang, B., Chen, W., Seow, P., Chia, G., Norris, C., et al. (2011). 1:1 mobile inquiry learning experience for primary science students: a study of
learning effectiveness. Journal of Computer Assisted Learning, 27, 269e287. http://dx.doi.org/10.1111/j.1365-2729.2010.00390.x.
Lowther, D. L., Ross, S. M., & Marrison, G. M. (2003). When each one has one: the inuences on teaching strategies and student achievement of using laptops
in the classroom. Educational Technology Research and Development, 51, 23e44. http://dx.doi.org/10.1007/BF02504551.
Lu, M. (2008). Effectiveness of vocabulary learning via mobile phone. Journal of Computer Assisted Learning, 24, 515e525. http://dx.doi.org/10.1111/j.13652729.2008.00289.x.
nez, L., Martnez, J. J., Gutie
rrez, J. A., et al. (2010). An experiment for improving students performance in
de-Marcos, L., Hilera, J. R., Barchino, R., Jime
secondary and tertiary education by means of m-learning auto-assessment. Computers & Education, 55, 1069e1079. http://dx.doi.org/10.1016/j.compedu.2010.05.003.
Martin-Dorta, N., Saorin, J. L., & Contero, M. (2011). Web-based spatial training using handheld touch screen devices. Educational Technology & Society, 14,
163e177. Retrieved from http://www.ifets.info/journals/14_3/14.pdf.
Martin, F., & Ertzberger, J. (2013). Here and now mobile learning: an experimental study on the use of mobile technology. Computers & Education, 68, 76e85.
http://dx.doi.org/10.1016/j.compedu.2013.04.021.
Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., et al. (2009). Clickers in college classrooms: fostering learning with questioning
methods in large lecture classes. Contemporary Educational Psychology, 34, 51e57. http://dx.doi.org/10.1016/j.cedpsych.2008.04.002.
McNaughton, D., Hughes, C., & Clark, K. (1997). The effect of ve proofreading conditions on the spelling performance of college students with learning
disabilities. Journal of Learning Disabilities, 30, 643e651. http://dx.doi.org/10.1177/002221949703000608.
Morris, J. P., Mahajan, N., Pasek, K. H., Golinkoff, R. M., & Collins, M. F. (2013). Once upon a time: parentechild dialogue and storybook reading in the
electronic era. Mind, Brain, and Education, 7, 200e211. http://dx.doi.org/10.1111/mbe.12028.
neza, J. J., & Osuna-Pe
rezb, M. C. (2013). Development and evaluation of a 3D mobile application for learning manual therapy in the
Noguera, J. M., Jime
physiotherapy laboratory. Computers & Education, 69, 96e108. http://dx.doi.org/10.1016/j.compedu.2013.07.007.
Oberg, A., & Daniels, P. (2013). Analysis of the effect a student-centred mobile learning instructional method has on language acquisition. Computer Assisted
Language Learning, 26, 177e196. http://dx.doi.org/10.1080/09588221.2011.649484.
lez, J. R. H., Herra
iz, J. J. M., Mesa, J. A. G. D., et al. (2011). Using M-learning on nursing courses to
Ortega, L. D. M., Plata, R. B., Rodrguez, M. L. J., Gonza
improve learning. Computers, Informatics, Nursing, 29, TC98eTC104. http://dx.doi.org/10.1097/NCN.0b013e3182285d2c.
Ozcelik, E., & Acarturk, C. (2011). Reducing the spatial distance between printed and online information sources by means of mobile technology enhances
learning: using 2D barcodes. Computers & Education, 57, 2077e2085. http://dx.doi.org/10.1016/j.compedu.2011.05.019.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

275

Powell, C. B., & Mason, D. S. (2013). Effectiveness of podcasts delivered on mobile devices as a support for student learning during general chemistry
laboratories. Journal of Science Education and Technology, 22, 148e170. http://dx.doi.org/10.1007/s10956-012-9383-y.
Radosevich, D. J., Salomon, R., Radosevich, D. M., & Kahn, P. (2008). Using student response systems to increase motivation, learning, and knowledge
retention. Innovate, 5, 7. Retrieved from http://eric.ed.gov/?idEJ840506.
Rau, P. L. P., Gao, Q., & Wu, L. M. (2008). Using mobile communication technology in high school education: motivation, pressure, and learning performance.
Computers & Education, 50, 1e22. http://dx.doi.org/10.1016/j.compedu.2006.03.008.
Riconscente, M. M. (2013 July). Results from a controlled study of the iPad fractions game motion math. Games and Culture, 8, 186e214. http://dx.doi.org/10.
1177/1555412013496894.
Rockinson-Szapkiw, A. J., Courduff, J., Carter, K., & Bennett, D. (2013). Electronic versus traditional print textbooks: a comparison study on the inuence of
university students' learning. Computers & Education, 63, 259e266. http://dx.doi.org/10.1016/j.compedu.2012.11.022.
 pez, X., & Seplveda, M. (2010). A monitoring and evaluation scheme for an ICT-supported education program in schools.
Rodrguez, P., Nussbaum, M., Lo
Educational Technology & Society, 13, 166e179. Retrieved from http://www.ceppe.cl/images/stories/articulos/tic/2.2-a-monitoring.-rodriguez1.pdf.
Roschelle, J., Rafanan, K., Bhanot, R., Estrella, G., Penuel, B., Nussbaum, M., et al. (2010). Scaffolding group explanation and feedback with handheld
technology: impact on students' mathematics learning. Educational Technology Research and Development, 58(4), 399e419. http://dx.doi.org/10.1007/
s11423-009-9142-9.
Sandberg, J., Maris, M., & De Geus, K. (2011). Mobile English learning: an evidence-based study with fth graders. Computers & Education, 57, 1334e1347.
http://dx.doi.org/10.1016/j.compedu.2011.01.015.
Saran, M., Seferoglu, G., & Cagiltay, K. (2009). Mobile assisted language learning: English pronunciation at learners' ngertips. Eurasian Journal of Educational
Research, 34, 97e114. Retrieved from http://94.73.145.182/0DOWNLOAD/pder/tr/902903948.pdf.
lu, G., & Cag
ltay, K. (2012). Mobile language learning: contribution of multimedia messages via mobile phones in consolidating voSaran, M., Seferog
cabulary.
The
Asia-Pacic
Education
Researcher,
21,
181e190.
Retrieved
from
http://www.ejournals.ph/index.php?
journalTAPER&pagearticle&opview&path%5B%5D4826&path%5B%5D5016.
Shih, W. C., & Tseng, S. S. (2009). A knowledge-based approach to retrieving teaching materials for context-aware learning. Educational Technology & Society,
12, 82e106. Retrieved from http://www.ifets.info/journals/12_1/8.pdf.
Siegle, D., & Foster, T. (2001). Laptop computers and multimedia and presentation software: their effects on student achievement in anatomy and physiology. Journal of Research on Technology in Education, 34, 29e37. http://dx.doi.org/10.1080/15391523.2001.10782331.
Skeate, R. C., Wahi, M. M., Jessurun, J., & Connelly, D. P. (2007). Personal digital assistant-enabled report content knowledgebase results in more complete
pathology reports and enhances resident learning. Human Pathology, 38, 1727e1735. http://dx.doi.org/10.1016/j.humpath.2007.05.019.
Solhaug, T. (2009). Two congurations for accessing classroom computers: differential impact on students' critical reections and their empowerment.
Journal of Computer Assisted Learning, 25, 411e422. http://dx.doi.org/10.1111/j.1365-2729.2009.00318.x.
Suhr, K. A., Hernandez, D. A., Grimes, D., & Warschauer, M. (2010). Laptops and fourth grade literacy: assisting the jump over the fourth-grade slump. The
Journal of Technology, Learning, and Assessment, 9, 1e45. Retrieved from http://136.167.2.46/ojs/index.php/jtla/article/view/1610/1459.
Sung, E., & Mayer, R. E. (2013). Online multimedia learning with mobile devices and desktop computers: an experimental test of Clark's methods-not-media
hypothesis. Computers in Human Behavior, 29, 639e647. http://dx.doi.org/10.1016/j.chb.2012.10.022.
Sung, Y. T., Chang, K. E., Hou, H. T., & Chen, P. F. (2010). Designing an electronic guidebook for learning engagement in a museum of history. Computers in
Human Behavior, 26, 74e83. http://dx.doi.org/10.1016/j.chb.2009.08.004.
Sung, Y. T., Chang, K. E., Lee, Y. H., & Yu, W. C. (2008). Effects of a mobile electronic guidebook on visitors' attention and visiting behaviors. Educational
Technology & Society, 11, 67e80. Retrieved from http://ifets.info/journals/11_2/7.pdf.
Tan, T. H., Liu, T. Y., & Chang, C. C. (2007). Development and evaluation of an RFID-based ubiquitous learning environment for outdoor learning. Interactive
Learning Environments, 15, 253e269. http://dx.doi.org/10.1080/10494820701281431.
Thornton, P., & Houser, C. (2005). Using mobile phones in English education in Japan. Journal of Computer Assisted Learning, 21, 217e228. http://dx.doi.org/
10.1111/j.1365-2729.2005.00129.x.
Uluyol, C., & Agca, R. K. (2012). Integrating mobile multimedia into textbooks: 2D barcodes. Computers & Education, 59, 1192e1198. http://dx.doi.org/10.
1016/j.compedu.2012.05.018.
Wang, S. L., & Wu, C. Y. (2011). Application of context-aware and personalized recommendation to implement an adaptive ubiquitous learning system.
Expert Systems with Applications, 38, 10831e10838. http://dx.doi.org/10.1016/j.eswa.2011.02.083.
Wei, C. W., Hung, I. C., Lee, L., & Chen, N. S. (2011). A joyful classroom learning system with robot learning companion for children to learn mathematics
multiplication. Turkish Online Journal of Educational Technology, 10, 11e23. Retrieved from http://les.eric.ed.gov/fulltext/EJ932221.pdf.
Wessels, A., Fries, S., Horz, H., Scheele, N., & Effelsberg, W. (2007). Interactive lectures: effective teaching and learning in lectures using wireless networks.
Computers in Human Behavior, 23, 2524e2537. http://dx.doi.org/10.1016/j.chb.2006.05.001.
Wood, C., Jackson, E., Hart, L., Plester, B., & Wilde, L. (2011). The effect of text messaging on 9- and 10-year-old children's reading, spelling and phonological
processing skills. Journal of Computer Assisted Learning, 27, 28e36. http://dx.doi.org/10.1111/j.1365-2729.2010.00398.x.
Wu, P. H., Hwang, G. J., Su, L. H., & Huang, Y. M. (2012). A context-aware mobile learning system for supporting cognitive apprenticeships in nursing skills
training. Educational Technology & Society, 15, 223e236. Retrieved from http://www.ifets.info/journals/15_1/20.pdf.
Wu, T. T., Sung, T. W., Huang, Y. M., Yang, C. S., & Yang, J. T. (2011). Ubiquitous English learning system with dynamic personalized guidance of learning
portfolio. Educational Technology & Society, 14, 164e180. Retrieved from http://www.ifets.info/journals/14_4/ets_14_4.pdf#page169.
Yang, C. C., Hwang, G. J., Hung, C. M., & Tseng, S. S. (2013). An evaluation of the learning effectiveness of concept map-based science book reading via mobile
devices. Educational Technology & Society, 16, 167e178. Retrieved from http://www.ifets.info/journals/16_3/13.pdf.
Yang, C. C., Tseng, S. S., Liao, A. Y. H., & Liang, T. (2013). Situated poetry learning using multimedia resource sharing approach. Educational Technology &
Society, 16, 282e295. Retrieved from http://www.ifets.info/journals/16_2/23.pdf.
Zhang, B., Looi, C. K., Seow, P., Chia, G., Wong, L. H., Chen, W., et al. (2010). Deconstructing and reconstructing: transforming primary science learning via a
mobilized curriculum. Computers & Education, 55, 1504e1523. http://dx.doi.org/10.1016/j.compedu.2010.06.016.
Zhang, H., Song, W., & Burston, J. (2011). Reexamining the effectiveness of vocabulary learning via mobile phones. The Turkish Online Journal of Educational
Technology, 10, 203e214. Retrieved from http://les.eric.ed.gov/fulltext/EJ944968.pdf.
Zheng, B., Warschauer, M., & Farkas, G. (2013). Digital writing and diversity: the effects of school laptop programs on literacy processes and outcomes.
Journal of Educational Computing Research, 48, 267e299. http://dx.doi.org/10.2190/EC.48.3.a.
Zurita, G., & Nussbaum, M. (2004a). Computer supported collaborative learning using wirelessly interconnected handheld computers. Computers & Education, 42, 289e314. http://dx.doi.org/10.1016/j.compedu.2003.08.005.
Zurita, G., & Nussbaum, M. (2004b). A constructivist mobile learning environment supported by a wireless handheld network. Journal of Computer Assisted
Learning, 20, 235e243. http://dx.doi.org/10.1111/j.1365-2729.2004.00089.x.