A Game-Based Learning Environment Using the

ARCS Model at a University Library

Kosuke Kaneko Yuriko Saito
Innovation Center for Educational Resource Kyushu University Library
Kyushu University Library Fukuoka, Japan
Fukuoka, Japan [email protected]
[email protected]

Yukari Nohara Eriko Kudo

Kyushu University Library Kyushu University Library
Fukuoka, Japan Fukuoka, Japan
[email protected] [email protected]

Masanori Yamada
Faculty of Arts and Science
Kyushu University
Fukuoka, Japan
[email protected]

AbstractThis study involves the development and evaluation our learning environment in Section III. Section IV describes
of a game-based learning environment for library instruction the system environment from both the client and server side.
that is based on the ARCS model and designed to run on mobile In Section V an analysis of our results is presented. Lastly, the
platforms such as iOS and Android. A questionnaire and conclusion examines the studys ramifications and explores
performance test was administered to compare game-based and
potential areas for future research.
e-learning performance; no significant difference in performance
was found between either groups, although three ARCS factors
were significantly higher for the experimental group when
II. LITERATURE REVIEW
compared to the control group. Furthermore, subjects in the
experimental group were more likely to recognize the effects of In their examination of game-based library education in a
the ARCS design when compared to those who used e-learning. university environment, Hori et al. [1] developed and
evaluated a document retrieval game. Donald [6] suggested
Keywordseducational technology; mobile learning; the use of alternate reality gaming, which transforms real
gamification; ARCS model; university library
world scenarios into a gaming platform, in order to familiarize
students with library services. Gallegos, Allgood, and Grondin
I. INTRODUCTION [4] also initiated a game development project designed to
A university library should be an environment in which familiarize students with library services.
students can learn and study effectively. Therefore, it is Gregory and Broussard [2] suggested using game-based
important for students to acquire the skills necessary to use learning materials, and highlighted the usefulness of
library services. More than 90% of the libraries at Japanese integrating game design into library orientations. Marcus and
universities offer an orientation program designed to Beck evaluated learning performance by comparing traditional
familiarize students with library services [3]. Nevertheless, and game-based approaches to library orientations [12].
approximately 20% of the students surveyed by Hyodo [8] Additionally, OHanlon, Diaz, and Roecker [11] proposed
reported that such seminars were either useless or questionable using multimedia such as games and video in library
benefit due to a lack of relevant information. Moreover, 80% orientations.
of the participants indicated that the orientation did not Hsu, Shu-Chen, and Yueh-Min [5] examined learning
motivate them to visit their institutions library. Following that performance in library instruction by developing a game-
investigation, the library integrated several instructional based learning system for PDAs. The adventure-styled game
experiences into the orientation in an attempt to improve upon included actual scenery from the library, and prompted users
it [9]. Based on these experiences, we decided to focus on to solve riddles related to library instruction. Although Hsu et
improving learning motivation, which led to the development al.s concept resembles the present study, they did not base
of a game-based learning environment for library instruction. their design on the ARCS model.
This paper begins with a short literature review in Section II, Smale [10] conducted a survey examining the adoption of
followed by an overview of the model that formed the basis of game-based learning in a library education context, which

403
 highlighted the limitations of traditional one or two session TABLE I. A PERSONA MODEL OF THE TYPICAL USER OF OUR
EDUCATIONAL MATERIALS
lectures, such as limited time, difficulty in attracting learner
interest, and a tendency to ignore the role of motivation in Target First-Year University Student
attracting learner interest. We also referred to Fujimoto and Believes it is sufficient to search for information on-line
in order to complete an assignment
Yamadas [13] research concerning game-based learning in
informal settings while developing the studys learning Background Rarely uses the university library and generally does not
environment. consider visiting it

Possesses a smartphone
III. INSTRUCTIONAL DESIGN Preoccupied with studies, club activities, and a part-
time job
The ARCS model functioned as a foundation for the game- Situation
Plays games or uses social networking services during
based learning materials developed during this study. The
their free time
following two sections describe how the materials were
designed, and provide an overview of their content.
B. A Game-based Learning Environment Using the ARCS
Model
A. The ARCS Model and Library Education
Based on our formula, a game-based learning environment
In a study examining the attitudes of university students
was created entitled Library Adventures: Unveil the Hidden
toward libraries, Hyodo [8] discovered three factors that
Mysteries! (LA). The game was delivered in the form of a
contributed to students avoidance of them: a notion that
mobile application, thus allowing students to play LA during
libraries are solely for reading books, not a place for study; a
their free time. As shown in the far left screenshot provided in
perception that there is no reason to visit them; and a belief
Fig. 1., LA includes 12 quizzes related to library services;
that libraries themselves serve no purpose. These opinions
students can obtain various items after successfully
indicate that many university students lack motivation to
completing a quiz. These quizzes contain questions related to
spend time at their campus libraries.
library services, and require students to physically explore the
Hyodo also surveyed participants concerning their
library premises or use its services in order to solve them.
librarys orientation program. Several students reported that
The center image in Fig. 1 shows a quiz in which users are
they were unable to recollect the contents of the orientation,
prompted to search for information by using a dictionary.
while others stated that they lacked opportunities to apply the
During the process, players must learn to use the librarys
knowledge that they acquired. Another series of interviews
Bibliotheca search engine to locate a specific document
prompted senior-year students to recollect their first-year
containing information needed to solve the quiz. Following
experiences in relation to their library orientation. Students
the search, players may go to the books location in order to
frequently reported that they were unable to understand its
retrieve it.
importance, or that they completed assignments primarily by
using the Internet.
In light of the aforementioned findings and the TABLE II. FORMULA FOR THE STUDYS EDUCATIONAL MATERIALS
observations of several librarians, a persona model (Table 1)
A game-based design that is attractive to students
was created to profile the typical user of our educational
materials. Based on concepts in the ARCS [7] and persona Cute characters that resonate with students
Attention
models, a formula was then devised for learning materials
with the aim of motivating students to partake in library A fieldwork-style educational design that concentrates
on student learning
education.
Useful content that assists students in tackling their
The ARCS model is a motivational design model Relevance
academic tasks
consisting of four components intended to evoke and sustain Varying difficulty levels that enable students to play
motivation throughout the learning process: attention, both easy and difficult games
relevance, confidence, and satisfaction. Attention involves
Several hints to assist students in reaching the correct
factors that users may consider attractive (e.g., role-playing or Confidence
answer
hands-on experiences); relevance concerns factors that a user
may want (e.g., immediate or future usefulness); confidence Practical experience that leads students to actually using
entails factors that should increase ones likelihood to succeed a technique in a library context
(e.g., self-growth); satisfaction involves feedback factors that Students feel a sense of accomplishment when a task is
completed
will likely secure success (e.g., rewards or benefits). The
contents of the formula categorized according to each ARCS Students experience slight difficulty in completing the
component are shown in Table 2. Satisfaction game in order to inspire learning

Students gain knowledge of the librarys services and

come to apreeciate its usefulness, even without
borrowing a book

404
 A. Client Application
During the conceptualization of LA consideration was
given not only to instructional design, but also the games
technical aspects. In order to reduce development costs, it was
decided that LA should be coded as a cross-platform
application capable of running on both iOS and Android
devices. Furthermore, we believed that the game should run as
a native application on both platforms, not merely as a
browser-based app. Accordingly, the cross-platform Unity 3D
game engine was selected as our development environment.
To facilitate data collection, it was deemed necessary for LA
to adopt a client-server configuration, wherein user accounts
are managed by a server application in order to log each users
Fig. 1. Screenshots from the library adventure game. respective activities. These logs, which are initially created by
the client application, are uploaded to the server whenever a
user saves his or her game data.
Other quizzes in the game are puzzle-styled, and include In conducting data analysis research, it is important to
crossword activities or games that require users to locate QR determine the kinds of information that should be collected.
codes found within the library. In the latter tasks, a game The librarians who participated in this study believed it would
character navigates the student through the building while be beneficial to obtain data concerning (1) how users
providing clues that will enable him or her to locate a given transitioned between activities, (2) the difficulty of quizzes
code, as shown in the far right image in Fig. 1. Upon finding that they selected, and (3) the times and places in which the
the appropriate QR code, the player scans it to successfully app was used.
complete the quiz. By completing each quiz, learners can The first items importance lies in its ability to reveal the
obtain the basic skills required to use the librarys services, types of questions that tended to appeal to users. As for the
which are outlined in Table 3. second item, this information can be obtained by examining
the logs uploaded by the client application, which reveal when
a user answered a question, in addition to his or her actual
IV. SYSTEM CONFIGURATION response. Moreover, the timestamps from these logs enable us
The client application runs on iOS and Android devices, to determine the amount of time taken by a user to complete a
and works in conjunction with the server application to collect given task. The IP address associated with a particular action
user data, which is then stored in a relational database. The is also recorded, which assists us in obtaining data pertinent to
sections that follow discuss the logged data in greater detail, the third item. A data format was created specifically to assist
and also describe the server and database configuration. us in obtaining the aforementioned information; the user log
shown in Fig. 2 is based on this format.
TABLE III. CONTENTS OF THE LIBRARYADVENTURE GAME

Content Details <?xml version="1.0" encoding="UTF-8"?>

<log>
Librarys hours of operation <enter scene="100" datetime="6/23/2014 6:16:41 PM" />
<enter scene="101" datetime="6/23/2014 6:16:43 PM" />
Facilties inside the library <enter scene="103" datetime="6/23/2014 6:16:45 PM" />
Basic <enter scene="102" datetime="6/23/2014 6:16:55 PM" />
Information <enter scene="104" datetime="6/23/2014 6:16:57 PM" />
Where books are located <save scene="104" datetime="6/23/2014 6:16:58 PM" />
<end scene="104" datetime="6/23/2014 6:16:58 PM" />
The arrangement of shelves .
How to borrow a book .
.
Library <save scene="102" datetime="2/24/2015 4:21:25 PM" />
Procedure for reserving group learning rooms <continue scene="102" datetime="2/24/2015 4:21:25 PM" />
Services
<resume scene="100" datetime="2/24/2015 4:29:01 PM" />
Using the automated archival system to locate a book <enter scene="100" datetime="2/24/2015 4:29:01 PM" />
<enter scene="102" datetime="2/24/2015 4:29:03 PM" />
Ways to search for a book <enter scene="0" datetime="2/24/2015 4:29:05 PM" />
<enter scene="102" datetime="2/24/2015 4:29:07 PM" />
Methods to locate a journal article <enter scene="104" datetime="2/24/2015 4:29:10 PM" />
Search <enter scene="102" datetime="2/24/2015 4:29:11 PM" />
Techniques <enter scene="104" datetime="2/24/2015 4:29:13 PM" />
How to locate a newspaper article
<save scene="104" datetime="2/24/2015 4:29:14 PM" />
<end scene="104" datetime="2/24/2015 4:29:14 PM" />
How to find a referenced book </log>
Other Spending fun time with staff members in the library.
Fig. 2. A sample log file showing one users activities.

405
 Due to its high parsability, XML was used to log Fig. 3 provides an overview of the server configuration.
application data. The following seven XML elements were For the purpose of receiving POST parameters from the client-
used to denote various user activities: side app, Nginx was configured to function as a reverse proxy,
which relays data to the web application framework. Gunicorn,
Enter. The user has reached a page. a web server gateway interface, was used to link the web
Pause. The game has switched to a background process, server and web application framework. Gunicorn runs several
such as when the user opens another application or workers to facilitate the web application process, which are
receives a phone call. managed as daemon processes by Supervisor, a Python-based
process management tool.
Resume. The application is no longer a background
The web application was built upon Bottle, a simple web
process.
framework featuring a model-view-controller (MVC)
Answer. The user has answered a question. architecture. The web application communicates with a
MySQL database, which depends on data-store and data-
Save. The user has saved his or her game data. getting processes. Peewee was adopted as an object-relational
Continue. After saving a game, the user is asked mapping library to connect the web application and database,
whether or not they would like to continue using the which contains the log data for each user. An entity-
application. This element indciates that the user decided relationship diagram for the database is shown in Fig. 4. The
to continue. database includes four tables:
End. The user has closed the application. Users. Contains information such as the users name
and affiliation.
Each of these elements possesses two types of common
attributes: scene, which is accompanied by a page ID, and Logs. Consists of data related to the users activities, in
datetime, which indicates the time when an action took place. which each schema represents one of the earlier
Consider the following example wherein the element is discussed actions.
enter, with a scene attribute of 100, and a datetime attribute of
Status. Includes game status data, such as whether a
6/23/2014 6:16:41 PM. This indicates that the user accessed a
user has successfully answered a specific question.
page with the ID 100 at 6:16:41 PM on June 23, 2014. By
comparing the datetime and scene attributes between various Comments. Contains user comments related to the
occurrences of the enter element, it is possible to establish the educational materials.
amount of time spent by a user on a given task. Furthermore,
The information contained in the users table facilitates
this information provides the data necessary to determine how
comparisons between users activities in relation to their
users transition between various activities.
affiliations and grades. The option schema in the logs table
The answer element is unique in that a string of text
enables the system to supply additional details, such as the
entered by the user follows it. Based on the users answer and
answers inputted by a user.
when it was entered, it is possible to estimate the questions
As mentioned above, the status table includes data
overall difficulty. When a user saves his or her game data, a
concerning a users game status, such as which quizzes he or
log is sent to the server containing the individuals IP address
she successfully completed. The aforementioned tables
and the time of submission, which can then be used to
include strictly Boolean parameters. However, upon
determine their location.
successfully completing all question users are prompted to
comment on the learning materials; these comments are
B. Server Application subsequently stored in the comments table. The users table has
Like with the client application, technical issues were a one-to-many relationship with other tables.
taken into consideration during the development of its server-
side counterpart. Since the server-side app needed to
accommodate at least 50 users simultaneously, it was not
feasible to use live streaming techniques for the transmission
of log data from the client. Instead, we opted to send data
using the HTTP protocol, but only during the save game
process; a reverse proxy was implemented to distribute the
server load across multiple machines. Log data was sent as
POST parameters from the client-side application to the server.
The server-side application was developed using the
Python programming language, which features many libraries
geared toward academic purposes (e.g., the numerical
operation and machine learning libraries), in addition to its Fig. 3. Configuration of the server application.
robust web framework libraries. Hence, the server is highly
capable of analyzing the supplied log data.

406
 model. Participants were asked to rate each item on a five-
point Likert-scale, wherein 1 = negative and 5 = positive.
To assess learning performance, a 20-question
performance test was compiled by university librarians, and
conducted both before and after the experiment. Table 4
contains a list of these questions.

C. Results
In evaluating the ARCS design, the data of 32 subjects
were analyzed (15 from the experimental group; 17 from the
control group); four participants responses were eliminated
Fig. 4. Entity-relationship diagram for the server-side application database. due to invalid data. The analysis was conducted using a
Wilcoxon sign-ranked test. The results are provided in Table 5,
along with mean and median scores.
The instructional design factors in the experimental group,
V. ANALYTICAL RESULTS with the exception of confidence, were significantly higher
The effectiveness of our game-based learning materials than for the control group. These results indicate that subjects
was determined by comparing them to traditional e-learning who used the game could more easily recognize the effects of
materials. E-learning materials are similar to game-based the ARCS design.
materials in that they are slide-based, and accompanied by
voice narrations; users are able to skip or return to slides
interactively as needed. Blackboard Learn, a learning TABLE IV. QUESTIONS FROM THE PRE AND POSTTEST
management system, was used in our experiment. In the No. Question
sections that follow, we discuss our approach to conducting 1 For how long can one borrow a journal from the library?
2 How is a book in the automatic archival system accessed
this experiment, our data collection methods, the experimental 3 What is the mascot name of the library staff?
results, and finally examine participants learning performance. Which of these choices does not reflect the role of a library staff
4
member?
5 Which of these choices correctly describes the librarys collection?
A. Procedure Which of these choices does not correctly reflect a feature of the
6
Thirty-six (11 male, 25 female) first-year university librarys collection?
What do you call the brochure gathering back-number of
students were equally divided into game (experimental) and e- 7
newspapers?
learning (control) groups. Following an explanation of the Which method is optimal for locating a newspaper article
8
experiment, subjects in both groups were asked to complete a containing a specific theme during a given decade?
performance test, after which they engaged in learning 9
How many books and journals can a first-year student borrow
activities using their respective game-based or e-learning simultaneously?
Which of these choices correctly depicts Vol. 3 No. 2
systems. 10
represntation?
Students in both groups were allotted up to three hours to 11 Where are monthly cooking magazines located in the library?
complete the activity. Members of the game group attempted 12
Which of these choices is best for locating a reference book in
to successfully answer the questions for each quiz. On average, corner 2F of the library?
13 Where would a general book be found in corner 2F of the library?
members of the game group took between two and three hours
Which of these choices does not correctly describe the usage of a
to complete the task, although some subjects were unable to 14
group learning room?
finish the activity in the amount of time given. Members of the 15 Where are the Blue Back series of books located in the library?
e-learning group were encouraged to review the provided 16 At what time does the library close?
materials until they reached a thorough understanding of the Which of these choices correctly describes something that can be
17
done in the librarys computer lab?
librarys services and how to use them. Subjects in this group
How are biographies classified according to the Nippon Decimal
took one hour on average to review their materials. 18
Classification
Participants from both groups later completed a performance How are titles involving language classified according to the
19
test and responded to a questionnaire. Nippon Decimal Classification?
Which of these choices incorrectly describes the process a first-
20
year student must follow in order to renew a borrowed item?
B. Data Collection
Data were collected by means of a questionnaire
concerning game design, and also by using a performance test.
The Instructional Material Motivational Survey (IMMS) [7]
was used to evaluate the game designs effects, and contained
36 items distributed across 4 factors related to the ARCS

407
 TABLE V. RESULTS OF THE IMMS SCORE ANALYSIS USING A Furthermore, the results indicated that individuals in the
WILCOXON SIGN-RANKED TEST
experimental group more easily recognized the effects of the
ARCS factor Mean Median ARCS design when compared to the control group. Future
Z
Game e-learning Game e-learning studies may investigate the relationship between user test
Attention scores and data logs, and also examine delayed test results.
(12 items; min 50.73
= 12; max = (6.80)
44.18 (5.28) 53.00 44.00 3.13b Additionally, it would be beneficial to investigate possible
60) correlations between the experimental groups duration of
Relevance
37.13 game use with each component of the ARCS motivation scale
(9 items; min 34.82 (4.07) 38.00 35.00 2.14 c and learning performance. The results in this study should lead
(4.76)
= 9; max = 45)
Confidence to new discoveries and, consequently, the development of
31.27
(8 items; min 29.59 (2.90) 32.00 29.00 1.40 improved game-based learning environments.
(3.81)
= 8; max = 40)
Satisfaction (5
22.80
items; min = 17.59 (3.20) 24.00 18.00 3.86a ACKNOWLEDGMENT
(2.54)
5; max = 25)
a. This study is supported by Foundation for the Fusion of
p < 0.001
b.
Science and Technology, and JSPS KAKENHI Grant Number
p < 0.01
c.
15K12415.
p < 0.05

D. Learning Performance REFERENCES

[1] A. Hori, K. Knoshita, E. Kobayashi, M. Murao, T. Watanabe, Y.
Thirty-six subjects completed the learning performance Kanematsu, K. Tsuji, and N. Uda, Evaluation of bibliographic search
test both before and after the experiment. The results of a two- game for library exploration study, Japan Society of Information and
way ANOVA (time x group) revealed a significant difference Knowledge, vol. 24(2), pp. 189196, 2014.
between the pre and posttests (F(3, 68) = 61.47, p < 0.001, R2 [2] A. S. Gregory and M. J. S. Broussard, Unraveling the mystery of the
= 0.73), although there was no significant difference between library: A big games approach to library orientation, Association of
College and Research Libraries Conference, 2011.
groups, and no interaction effect (see Fig. 5). These results
[3] A. Tateishi, http://www.mslis.jp/am2013yoko/10_tateishi.pdf, 2013.
indicate that learning performance between both groups
[4] B. Gallegos, T. Allgood, and K. Grondin, Quarantined: The Fletcher
improved. Library game project, LOEX Conference Proceedings, 2007, pp. 133
137.
[5] H. Sheng-Hui, C. Shu-Chen, and H. Yueh-Min, The experience of
VI. CONCLUSION AND FUTURE STUDIES adopting game-based learning in library instruction, Learning by
Playing: Game-based Education System Design and Development, 2009,
A game-based learning environment for library education pp. 571576.
was proposed based on the ARCS model, and its effectiveness [6] J. Donald, The blood on the stacks ARG: Immersive marketing meets
was compared to traditional e-learning materials. Thirty-six library new student orientation, in Gaming in Academic Libraries:
individuals were divided equally into two groups who utilized Collections, Marketing and Information Literacy, 2008, pp. 189211.
either game-based or e-learning materials. The results failed to [7] J. M. Keller, Motivational Design for Learning and Performance: The
reveal a significant difference in leaning performance between ARCS Model Approach, Springer Science and Business Media, 2009.
either groups, although instructional design factors in the [8] K. Hyodo, Library seminar worked with classes of the first year
experience program inquiry survey for improvement in quality, Kyushu
experimental group were significantly higher than for the Journal of Higher Education, vol. 16, pp. 167185, 2011.
control group. [9] K. Hyodo, E. Amano, and H. Nakazono, Redesigning the library
seminar in Kyushu University: Applying some principles of
instructional design to library guidance, Annual Report of Research
and Development Department of Kyushu University Library, 2012, pp.
2433.
[10] M. Smale, Learning through quests and contests: Games in information
literacy instruction, Journal of Library Innovation, vol. 2(2), pp. 3655,
2011.
[11] N. OHanlon, K. Diaz, and F. Roecker, A game-based multimedia
approach to library orientation, LOEX Conference Proceedings, 2007.
[12] S. Marcus and S. Beck, A library adventure: Comparing a treasure hunt
with a traditional freshman orientation tour, College and Research
Libraries, vol. 64, pp. 2344, 2003.
[13] T. Fujimoto and M. Yamada, Review of educational games research for
informal learning: Research frameworks and evaluation methods, Japan
Society for Educational Technology, vol. 37(2), pp. 343351, 2013

Fig. 5. ANOVA results for learning performance.

408