Computers & Education 95 (2016) 99e113

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

On the effectiveness of game-like and social approaches in

learning: Comparing educational gaming, gamification &
social networking
Luis de-Marcos*, Eva Garcia-Lopez, Antonio Garcia-Cabot

, Edificio Polit

Computer Science Department, Universidad de Alcala
de Henares, Madrid,
ecnico, Ctra Barcelona Km 33.1, 28871, Alcala
Spain

a r t i c l e i n f o a b s t r a c t

Article history: Game-like approaches are becoming increasingly popular in education, with educational
Received 1 June 2015 games and gamification drawing increasing levels of attention. While games specifically
Received in revised form 24 December 2015 designed for educational purposes have been used for decades, gamification is particularly
Accepted 26 December 2015
new and contrasting evidence was presented about its effectiveness. The potential of social
Available online 29 December 2015
networks has also been harnessed by educators and institutions either using popular social
networking sites or specific educational instances. This paper studies how well-established
Keywords:
approaches (educational game and social networking) compare with more novel ones
Gamification
Learning
(gamification and social gamification) in terms of learning performance in an under-
Social networking graduate course. Four experimental conditions were compared in an experiment
Educational game (N ¼ 379). Results suggest that all experimental conditions significantly impact on learning
performance, but social gamification returned better results in terms of immediacy and for
all types of assessments.
© 2015 Elsevier Ltd. All rights reserved.

1. Introduction

A game is a system that presents a set of meaningful motivating challenges to the player. Game designers can carefully
align game mechanics and dynamics with a compelling narrative and a feedback system to create a sense of seamless pro-
gression that captures player's attention and can keep her deeply immersed in the experience. Furthermore, game com-
munities provide a venue where players can communicate, share and build knowledge around the game. According to several
scholars (Gee, 2007; Squire, 2011) good videogames and their emergent cultures provide problem-solving spaces where
learning occurs, because challenge and learning are at the heart of motivation and entertainment.
The potential of videogames as educational tools has created a growing interest and expectations in the gameful world
drawing the attention of educators and institutions that want to harness the potential of videogames to create more engaging
and meaningful learning experiences that facilitate long-term learning. Such interest has been realized in different directions.
On one side of the spectrum, educational games are complete systems designed with the purpose of training their players.
Serious game mechanics assist in the translation of learning goals and practices into the mechanical element of gameplay
mapping design patterns and pedagogical practices (Arnab et al., 2015; Lim et al., 2014). On the other side, gamification takes

* Corresponding author.
E-mail addresses: [email protected] (L. de-Marcos), [email protected] (E. Garcia-Lopez), [email protected] (A. Garcia-Cabot).

http://dx.doi.org/10.1016/j.compedu.2015.12.008
0360-1315/© 2015 Elsevier Ltd. All rights reserved.
100 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113

game elements and uses them in non-game contexts to motivate action (Deterding, Dixon, Khaled, & Nacke, 2011; Werbach &
Hunter, 2012). One of the contexts in which gamification has drawn more attention is education (Kapp, 2012), to the point of
turning gamification into a buzzword situated at the peak of inflated expectations of the hype cycle in 2013.1 Early research on
the area outlined design and architectural models to incorporate game-like approaches in educational settings (Haksu &
Young Yim, 2012; Raymer, 2011; Simo ~ es, Redondo, & Vilas, 2013). First experimental studies reported mostly positive out-
comes in terms of learning and other educational outcomes (Bellotti et al., 2013; Denny, 2013; Fitz-Walter, Tjondronegoro, &
Wyeth, 2012; Li, Grossman, & Fitzmaurice, 2012; Santos et al., 2013; Sheldon, 2012), but also pointed to some preliminary
contradictions like the questionable quality of students' feedback (Halan, Rossen, Cendan, & Lok, 2010) or the convenience of
certain game elements to convey different kinds of learning (Domínguez et al., 2013). However, recent studies questioning the
lack of empirical evidence as well as the effectiveness of educational games and gamification in learning and instruction as
well as in broader contexts (Connolly, Boyle, MacArthur, Hainey, & Boyle, 2012; Dicheva, Dichev, Agre, & Angelova, 2015;
Hanus & Fox, 2015; Mekler, Brühlmann, Opwis, & Tuch, 2013), suggest that we are now on our way down “trough of disil-
lusionment” of the hype cycle.2
Social networks and social media are pervasive today. They offer almost unlimited possibilities for online sharing and
collaboration. User interactions are also stored and can be effectively used and mined to crowdsource contents and contri-
butions, offering endless opportunities for personalization. Such possibilities have also drawn the attention of the education
and research communities. There is an important and also growing body of literature about the uses and effects of different
social media in education. Popular social networks expedite studentestudent and studenteteacher interaction, communica-
tion and collaboration improving student's attitude (Despotovic-Zrakic, Labus, & Milic, 2011). Educational networking impacts
on students' motivation, retention, engagement, satisfaction, individual creativity and personal interaction, increasing the
efficiency of communication and facilitating differing viewpoints (Brady, Holcomb, & Smith, 2010). The individual position that
each student has on the network is also important, influencing social learning (Paredes & Chung, 2012), learning performance
(Cho, Gay, Davidson, & Ingraffea, 2007) and even creativity (Gaggioli, Mazzoni, Milani, & Riva, 2015). Still, critical accounts
question the claims about the purposeful integration of social media as an educational tool because most empirical evidence
about the utility and effectiveness of social media is based on self-reported data and content analysis (Tess, 2013).
Social gamification aims to bring together gamification and social networking to combine the potential of both approaches
to create compelling socially-driven user experiences. From an educational perspective, it can harness the motivational as-
pects of gamification to stimulate participation and engagement with learning contents and with other participants. Social
networks facilitate communication, explicit social ties and highlight relevant content elements for participants. Their po-
tential can also be harnessed to cooperate and create meaningful conversations in learning interactions. The combination of
both can create a kind of multiplication effect in which gamification can be used to promote social desirable learning be-
haviors, and actions in the social network can be used to design gamification props that produce motivational boosts in
educational settings. To our best knowledge, the only study that deals with social gamification in education is Simoes et al.
(Simo~es et al., 2013) which presents a framework for integrating and evaluating gamified elements in primary education. The
potential benefits of integration of both approaches have therefore only been cursorily studied.
Beyond the hype, critical voices call for serious empirical account of the motivational and learning potential of educational
games (Connolly et al., 2012), gamification (Dicheva et al., 2015) and social networking (Tess, 2013). Within this context, this
paper aims to study and compare the educational effectiveness (in terms of learning performance) of four instruments:
educational games, gamification, social networking and social gamification. Educational games can harness many of the
affordances of games to facilitate meaningful learning experiences. Gamification can be used to foster competition between
participants. Social networking promotes collaboration by providing tools for communication, sharing knowledge and by
opening spaces for contribution. As gamification and social networking can only be appealing for certain types of learners, we
think that the combination of both can successfully address the motivational needs of a wider audience of learners.
Educational games have been employed in a wide variety of contexts ranging from commercial off-the-shelf games that can
be used in the classroom to games specifically designed to meet learning goals. In terms of educational games, this study
focuses on the first type as specific games are difficult to find for each specific educational settings while off-the-shelf and
sometimes free games are already available. Although this also requires time to search and assess their suitability for
particular learning situations. Education is the area where gamification has been more extensively reported in research
(Hamari, Koivisto, & Sarsa, 2014). Therefore there is a wide spectrum of approaches, ranging from game-inspired instructional
design that aims to foster engagement and achievement (Fabricatore & Lo
pez, 2014), to reward-based strategies that focus on
extrinsic motivation promoting competition (Domínguez et al., 2013). Our focus is on reward-based gamification as it is more
common and easier to implement (Werbach & Hunter, 2012) although we are also aware that extrinsic rewards can ultimately
undermine motivation. The uses and results of educational social networking are widely reported also, ranging from models
of training and performance that integrate social media as a central part of learning experience (Paredes & Chung, 2012), to a
supportive role of social networking in the wider context of the learning experience (de-Jorge-Moreno, 2012). Our focus in
this case is driven by the communicative and cooperative affordances that social networks offer and by their capability to
highlight relevant contributions and key participants. This also contrasts with our approach to gamification and offers, in our

1
Gartner's 2013 Hype Cycle for Emerging Technologies: http://www.gartner.com/newsroom/id/2575515.
2
Gartner's 2014 Hype Cycle for Emerging Technologies: http://www.gartner.com/newsroom/id/2819918.
 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113 101

opinion, a better ground for comparative analysis. Our approach to social gamification will focus on the potential to combine
competition and cooperation around learning materials and learning activities to take advantage of external motivators
engaging students in social discussion and resulting in better learning outcomes.
In a recent taxonomy, Deterding (Deterding, 2014) suggests that different rhetorics to the gameful world reveal diverse
theoretical approaches. Following this we argue that a rhetoric of systems may account for the educational value of games as
self-contained learning systems. Meaning and learning are created by interacting with rules' systems through and around
games, constructing understanding and skill when engaging with the virtual world through experience. Educational games
try to harness such potentialities by creating compelling games that have an educational background resulting in positive
outcomes in terms of learning performance. A rhetoric of feedback motivates a model of gamification that perfectly aligns
quantified atomic activities and rewarding schemes within pre-specified paths of action in the system. Player-defined at-
tributes like levels and skills are explicit, transparent and reliable indicators of skills and status. Game elements and trans-
parent sets of rules can be implemented in educational systems that explicit learning goals and motivate action towards that
objectives resulting in quantifiable impacts on learning performance. And finally, a rhetoric of status addresses for the in-
dividual need of social presence and recognition of social networks. Social status is a basic human need. Social networks are
public environments that provide means for getting and publicizing status, fitness and trust. Designers can then harness
participants' natural drives for status to motivate them to pursue system's goals. In educational terms, needs for status,
participation and social recognition can similarly be addressed and aligned with learning objectives impacting in learning
outcomes. We argue that different approaches and rethorics can appeal to different learners impacting in learning when
gameful and social instruments are used in educational settings. So, in this paper a controlled experiment is designed and
performed in the same educational setting to get comparative results of the learning performance of different instruments
that represent each approach. Different approaches that are to some extent overlapping but still offering comparable results
with a careful design of the educational setting. The educational game focuses on narrative and a storyline that also offers
room for exploration. Gamification focuses primarily on competition. Social networking promotes cooperation. And social
gamification combines gamification and social networking to compare them with previous approaches. The rest of the paper
is organized as follows, Section 2 presents and justifies the experimental design in terms of the objectives, instruments and
method. Section 3 presents results. Section 4 discusses findings. And Section 5 presents conclusions outlining future research
opportunities.

2. Experimental design

Under the premise that educational games, gamification, social networking and social gamification of learning have the
potential to impact on learning, this section presents the research objectives, instruments and methodology to conduct a
comparative analysis in one educational setting in order to appraise and compare the learning performance that each in-
strument delivers.

2.1. Research questions

This experiment sets out to address the following research questions:

RQ1: Do experimental conditions (educational game, gamification, social networking and social gamification) impact on
learning performance?
RQ2: Is there any difference in terms of learning performance between experimental conditions?
RQ3: Does the kind of evaluation item influence results on learning performance for the different experimental
conditions?
Previous studies and literature appraisals suggest that educational games (Connolly et al., 2012), gamification (Hamari
et al., 2014) and social networking (Tess, 2013) impact on learning as well as other behavioral and affective outcomes, but
also point to the lack of empirical evidence to support several claims. Furthermore, as there are no comparative studies on the
same educational setting and social gamification has been only cursorily studied, we suggest that the first and second
research questions can provide additional empirical insights about the effects of different approaches on learning. Contra-
dicting evidence recently presented (Hanus & Fox, 2015) questions the effectiveness of gamification and raises issues about
the superficial learning that shallow gamification approaches may promote. Nevertheless, the nature of the evaluation items
is not considered in such appraisals. We conjecture that different experimental conditions may suit better with different
evaluation items such as tests, short questions, essays, practical assignment or oral examinations. So the third research
question addresses to what extent the type of evaluation may account for any influence in terms of measured learning
performance in the different experimental conditions. This study compares two kinds of evaluation items: assignments that
primarily assess practical skills and examinations that are designed to assess conceptual knowledge.

2.2. Setting

The experimental course was a 10-week undergraduate blended-learning course called ‘Qualification for ICT Users’ for
first-year students. The course was designed to introduce basic computing concepts and provide a working knowledge and
skills about standard office applications including word processing, spreadsheets, presentations and databases, which are the
102 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113

Table 1
Summary of instruments synthetizing the main features of the different means and targeted benefits.

Instrument Features Approach & targeted benefits

Educational Challenges, levels, points, narrative Not aligned with learning objectives
game Independent work & exploration of
students
Gamification Trophies, badges, challenges, leaderboard Competition-driven
plugin Motivate participation through
comparison with peers
Social Blogging, questions & answers, liking, friends, built-in twitter, dashboard Cooperation and communication among
networking participants.
website Boost participation, collaborative work &
community building
Promote student-driven discussion
Social Blogging, questions & answers, liking, friends, built-in twitter, dashboard, challenges, Competition and cooperation
gamification points, achievements, virtual currency, shop, external rewarding, personalization (status/ Boost participation, collaborative work &
website visibility), peer review community building
Motivate participation through
comparison with peers
Social interaction affords additional means
to motivate participation and engagement
Addressing needs of different students
(player types) and widen participation

learning modules of the course. Syllabus was based on the ECDL/ICDL certification which is becoming an international de
facto standard for digital skills.3 Students had one lecture every two weeks and they must work independently the rest of the
time to achieve the learning goals. Supporting learning materials, including videos, and communication tools were available
in the learning management platform.
The course had five evaluation items: four practical assignments and one final examination. Students submitted an
assignment at the end of each learning module as follows: word processing in week 3, spreadsheets in week 5, presentations
in week 7 and data bases in week 9. On week 10 students answered a final examination which was a comprehensive test of all
the course contents and modules. Practical assignments required that students completed a set of tasks with a given com-
puter application producing a set of artifacts that were submitted for evaluation. Therefore assignments were designed to
assess practical skills. The final examination was designed to assess conceptual knowledge. Assignments and final exami-
nation were the two different kinds of evaluation items considered in this study. Students were invited to practice with
several activities that in the end helped them to master and hone the skills required to complete each assignment. Students
had to use the different options and abstract constructs of the different applications to build complete versions of working
artifacts. In the word processing assignment, they were required to produce a professional document. In the spreadsheets
assignment, students were required to deliver a spreadsheet that combined data integrated from different sources, functions
and graphical elements. As for the learning module on presentations, students were required to create a professional visually
appealing presentation that combined text, multimedia elements, and visual effects. In the assignment on databases, students
had to present a database with the appropriate structure of tables and relations, and supporting elements like queries, forms
and reports. The final examination was a written test of all learning contents and modules that combined multiple choice, fill-
in-the-blank and short answer questions. The final examination had 15 questions and students had to complete it in 1 h.
Given the nature of this kind of evaluation item, it was designed to assess conceptual knowledge. Students were asked about
the meaning and practical application of different concepts and abstract constructs that were used in applications or that
were relevant to understand the underlying mechanisms of working artifacts, like “What is a range? Where is it useful? And
how is it used?”, or “What are the different types of relations that are possible in databases? How are they implemented?
Provide an example of each type”.

2.3. Instruments

In order to compare the effects of educational games, gamification, social networking and social gamification on learning
performance four instruments were used. Instruments were chosen based on their capabilities to address students' needs for
fun, competition and cooperation. The different instruments offered different features that may eventually impact on learning
performance. A summary is presented in Table 1. We chose an educational game that could be readily integrated into the
learning experience. It was related to the learning contents but lacking a clear alignment with learning objectives. We aimed
to explore the learning potential of educational games with which students can work and explore independently. All other
instruments were aligned with the learning objectives and used to support learning activities. The gamification plugin was
developed to explore and compare the effects of competition-driven gamification. The social network aimed at exploring and

3
European/International Computer Driving License e ECDL/ICDL (www.ecdl.com).
 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113 103

Fig. 1. Gamification plugin. Leaderboard and challenges.

comparing the impact of cooperation in learning performance. Finally, the social gamification instrument was chosen to bring
the potential advantages of competition and cooperation together.
The first instrument was the Ribbonhero4 educational game. Ribbonhero is an educational game sponsored by Microsoft
specifically designed to train skills of the Microsoft Office software suite. In the game, players embody a virtual character
called Clippy and are asked to help him overcome different challenges that require players to master skills and learn how to
use different options of word processing, spreadsheets and presentation applications. Challenges are arranged in six different
levels of increasing difficulty. Each challenge opens the corresponding application and asks players to complete a specific task.
Examples of tasks are: use text effects in word processing, use chart types & chart styles in a spreadsheet, or use built-in
themes in a presentation. Players get points for attempting and completing challenges. The amount of points depends on
how well they perform. For instance, hints are offered on demand, but in order to get the maximum possible amount of
points, players must not use hints and they have to find their own way to complete the task. Players have different challenges
in each level and they can choose which ones they prefer to complete to get enough points to get to the next level. Ribbonhero
uses the following game elements: narrative, challenges, levels and points. Ribbonhero is integrated with Microsoft Office
applications using an overly layer that also grants small amounts of points to players just for using the applications in their
daily activities (e.g. ‘italics þ2 points’). It features the Microsoft Office applications for word processing, spreadsheets and
presentations. The learning module on databases was therefore not supported by Ribbonhero, but this provided an inter-
esting ground to analyze the effects of lack of support after continued work with one of the experimental groups. If the
educational game acts as a facilitator of knowledge and skills, we could examine to what extend knowledge transfer ceases
when the instrument is no longer present. All other instruments facilitated knowledge transfer for all learning modules.
The second instrument was a gamification plugin integrated in the institutional learning management system (Fig. 1).
Learning activities were aligned with learning objectives and presented as challenges. Challenges were carefully divided in
tasks that provided short-term goals and students were awarded trophies upon completion of each task. Lecturers had to
review students' submissions. A set of badges was also designed to motivate participation by completing sets of challenges
(e.g. ‘Word expert’) or participating and collaborating with other students (e.g. ‘Rookie’ for enrolling). Finally, a leaderboard
was implemented using the number of trophies and badges to rank students. This instrument was designed mostly to foster
competition between students with the aim of motivating participation. For each game element (challenge or badge) students
could see how many students had completed it and so compare their performance with their peers.
The third instrument was a social networking platform (Fig. 2). The Elgg5 social networking engine was used as the basis to
provide a venue where students can interact and have conversation around learning materials. The platform included basic
functionalities to create blogs, ask and answer questions, link videos, and post files and other resources. But the critical el-
ements to harness social interactions were liking and friendship. Students could comment and like on any content thus
focusing on their own interests, but also providing important aggregate information that was then used to highlight hot topics
and relevant entries on the platform. A friendship functionality similar to Facebook's allowed students to connect with peers
with common interests and interact within their own community. A built-in twitter-like system was also included to boost

4
Ribbonhero 2: http://www.ribbonhero.com/.
5
Elgg Open Source Social Networking Engine: https://elgg.org/.
104 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113

Fig. 2. Social networking website.

communication and participation. Students used it mostly as a channel for free expression to share their thoughts and
concerns. Finally, a dashboard provided updated information about recent relevant actions in the platform. Lecturers acted as
community managers presenting the topics and guiding discussion. Initial contents for each topic were developed including
videos describing the most important concepts and skills, activities and supporting materials. At the end of each learning
module additional videos were uploaded providing step-by-step solutions to the activities. Students worked collaboratively
on the activities and posted their comments, solutions and additional resources. There were no reviews from lecturers in the
social networking platform because it was designed as a space for collaboration.
The fourth and final instrument was a gamified social platform (Fig. 3) that combined the potential benefits of gamification
and social networking using social interaction to offer additional rewards that motivate participation and engagement with

Fig. 3. Social gamification website: Gamification tools (left), dashboard (top-right) & leaderboard (bottom-right).
 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113 105

course materials and peers. The Elgg social networking engine provided the backbone offering support for the gamification
layer that was developed over it. Social features and tools were the same used in the previous instrument. Learning activities
were presented as challenges that students could complete and submit. All activities were peer reviewed based on a rubric.
Points were awarded for submitting, reviewing and getting positive reviews on their own submissions. Peer review generates
valid and reliable rating scores providing also an important room for reflection (Lu & Law, 2012; Xiao & Lucking, 2008) and
significantly reduces the burden of lecturers that only had to intervene in case of dispute. A set of achievements was carefully
designed to harness the potential of social actions providing additional points to students that participated significantly in the
social network (e.g. ‘Witty’ for getting 15 or more likes on a post, or ‘Ancient wisdom’ for answering 3 or more questions).
Activities and achievements provided points that were used to rank students on a learderboard. Points were also a virtual
currency that students could use in a virtual shop to buy several items like an additional 5% on their final mark (external
rewarding) or personalization features for their avatars and text messages that provided visibility and status in the platform.
Our design of the social gamification platform was based on previous research on learning styles in game contexts (Heeter,
Lee, Medler, & Magerko, 2011) that suggests that different students are driven by different interests. We mostly draw on the
literature on player types, and especially on Bartle's classical taxonomy of players (Bartle, 1996; Heeter, Magerko, Medler, &
Fitzgerald, 2009; Stewart, 2011), to try to accommodate different necessities around meaningful interactions and conver-
sations with peers and course materials. Activities, points, badges and external rewards were included to accommodate
students that may be looking to achieve as much as they can and to optimize their performance (‘achievers’ in Bartle's terms).
The leaderboard was included to accommodate competition. Bartle defines ‘killers’ as those players interested in ‘acting on
other players’ and in competing with them. Socializers are a type of players driven by their interest to cooperate, commu-
nicate and share. Social features, social achievements and personalization features were included to address their necessities
so that they could have meaningful interactions and impact on the system and on their peers. Finally, explorers are willing to
probe and scan (but not necessarily to complete) all courses of action and having a wide variety of possibilities is important for
them. A set of achievements was specially designed for them, providing a limited amount of points for having many small
interactions with the system but also hoping that ultimately they would find the way to participate regularly. Our underlying
aim for using Bartle's taxonomy was simple: although specific games usually target to specific player types or at least pri-
oritize the feedback for them, in education we usually have a wide variety of students with different learning styles that are
representative of all player types. Addressing the needs of as much types as possible could possibly widen the participation
resulting in better outcomes in educational terms. Bartle's taxonomy and its variations have already been used by gamifi-
cation practitioners to design and test gamified systems.
Finally we want to point to the possible overlapping that exists between the different instruments. We think that there is
no overlapping between the Ribbonhero educational game and other instruments. Although the game offers a gamification
module for Microsoft Office applications, we checked that the amount of points that can be obtained are really marginal
having no influence on the progression in the game story. In our opinion it is present mostly as a reminder that the game exist
providing a link between office tools and the game software. The gamification plugin was specifically designed to promote
competition while the social network was designed to address cooperation. No or minimal overlapping exists in terms of the
instruments. The social gamification instrument was designed specifically to combine the social and gamified instruments so
there is an obvious overlapping but the aim is to determine if a combined approach yields observable differences in terms of
learning performance.

2.4. Technical implementation

The Ribbonhero educational game is offered as free software by Microsoft. Students just had to download and install it. The
gamification tool was implemented as a plugin that was deployed on the Blackboard 9 learning management system.
Blackboard provides an API (Application Interface) that enables developers to extend its functionalities. The gamification
plugin used this API and was programmed using the JSP (Java Server Pages) web technology. A persistent storage external
service was used to store plugin's data and students' submissions. The social networking site and the social gamification
networking site used the Elgg open source social networking engine. Elgg is free to download and use. It is licensed under
GNU General Public License. Elgg runs on a Linux, Apache, MySQL, PHP web server. Since it also has many utilities and already
existing plugins, the technical implementation of the social networking instrument just required to install Elgg in a web
server and to configure the necessary utilities, plugins and widgets to support designed functionalities. Elgg's basic infra-
structure was also used as the backbone for the social gamification platform. In this case, a gamification plugin was developed
and deployed on the social networking engine. Elgg is easy to modify and extend providing extensive documentation for
developers. The plugin was developed using the same technologies that the Elgg platform uses. It was programmed in PHP
and MySQL was the database.

2.5. Participants

A group of 379 first-year undergraduate students participated in the study (212 women, 167 men, Mage ¼ 18.83 years,
range: 17e34 years). A quasi-experimental design was used (Fig. 4). Students decided in which group to enroll and experi-
mental conditions were randomly assigned to groups. Students' choice was based mostly on schedule and proximity issues
since different groups took place in different campuses and terms. Students did not know what experimental condition was
106 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113

Fig. 4. Experimental design with four experimental groups and a control group.

going to be used on each group and they were not aware of experimental conditions being used in other groups. The
educational game was administered to a group of 75 students (37 women, 38 men, Mage ¼ 18.68 years, range: 17e32 years).
The gamification plugin was administered to a group of 77 students (47 women, 30 men, Mage ¼ 18.57 years, range: 17e34
years). The social networking site was administered to a group of 75 students (43 women, 32 men, Mage ¼ 18.97 years, range:
17e24 years). The social gamification platform was administered to a group of 76 students (43 women, 33 men, Mage ¼ 18.99
years, range: 17e23 years). An additional group of 76 students was the control group (42 women, 34 men, Mage ¼ 18.95 years,
range: 17e25 years).

2.6. Procedure

Experimentation took place during Spring 2014 and Autumn 2014. The control group and the experimental groups using
gamification and social networking took place during Spring 2014 from February till May. Students of the experimental
groups using the educational game and social gamification took the course during Autumn 2014 from October till December.
All instances of the course run for 10 weeks from the start without breaks. Learning modules were delivered sequentially. On
the first week of the course, students took a multiple-choice computer pre-test. Experimental conditions were then deployed
(weeks 2e9). Students in the control group used a traditional e-learning approach in which all educational contents and
activities were available in the learning platform, as well as traditional communication tools including forum or chats. After
completing each learning unit (weeks 3, 5, 7 and 9), students had to submit an individual assignment that was assessed by
lecturers and that was used to measure their learning performance. Final examination was a 15-question written examination
comprising multiple choice, fill-in-the-blank and short answer questions. The final examination was designed to assess
conceptual learning and it was delivered in week 10.

2.7. Educational strategies

There was basically no educational strategy behind including Ribbonhero in the course. The aim was to test an off-the-
shelf software product and to analyze how it compares with other gameful and social approaches that are aligned with
 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113 107

learning objectives. A few Ribbonhero challenges were aligned with learning content and learning objectives, but others were
not aligned and not even included in the course. For example, students could also explore other tools of the Microsoft Office
suite. Ribbonhero was introduced in the second lecture and students were given supporting information and tutorials about
how to use it. Students had to work on Ribbonhero on their own. Activities were introduced in lectures and students worked
on them but had to complete them at home. Activities and progression on Ribbonhero were reported by students and
monitored by lecturers. Students were asked to gain one level each week.
In the gamification plugin, a set of trophies was designed taking learning activities and breaking them down into smaller
chunks. Therefore learning objectives were aligned with trophies. Between six and eight trophies were designed for each
learning module. For each trophy, students have to complete a part of the activity and submit it. Two activities were delivered
every two weeks including instructions that described the requirements to get the trophies. Activities were introduced during
lectures and students started working on them. Students then had to complete and submit them for evaluation using the
learning management system. Lecturers reviewed each submission and decided to award the trophy or return the activity to
the student. Badges were also included to motivate participation encouraging students to complete more activities and get
more trophies. The leaderboard was the first element displayed in the gamification plugin. It showed the total number of
trophies and badges earned by each student stressing the competitive nature of the instrument. The goal was to analyze the
impact of competitive approaches of gamification in learning performance.
The social networking platform was introduced in the course to foster participation and collaboration between students
around course materials and activities. Two learning activities and the corresponding learning materials were published
every two weeks for each learning module. Students were then asked to cooperate to complete the activities. Cooperative
activities included finding or creating supportive materials such as videos or descriptions of how to perform important
tasks. Questions could also be asked and collaboratively answered in the platform. For each activity, lecturers initiated the
discussion, for example suggesting what skills students have to master and encouraging them to practice and share their
approaches or problems. After that, discussion was mostly driven by students and lecturers only had to intervene to guide
learning if necessary. Activities and initial discussion were presented during the lecture. Then students had to work on their
own and continue the conversation in the social networking platform. Each student uploaded a final version of each activity
to her personal blog in the platform. Finally students were asked to review and provide feedback to the activities of other
two students. There were no particular instructions about the format of the review. Lecturers did not participate in the
reviewing process, except in particular cases where there were contrasting opinions, no agreement or wrong common
assumptions made by students. All contents in the platform were available to all students. Students could like or comment
any item, so most popular items representing most relevant pieces of information were given higher visibility. Lecturers
were also considered participants having the same weighting in liking or content creation than students. A dashboard
showed recent activity. And finally students could create their network of friends to communicate and collaborate with
them.
The social gamification instrument aimed to combine the benefits of gamification and networking in a single instrument.
Activities were gamified in a social networking environment addressing the specific needs for both competition and coop-
eration. Every two weeks two activities were published. Students could then cooperate to complete them, create and share
contents in the same way that they did with the social networking site. The main difference with the social networking site in
terms of educational strategy was in the reviewing process and feedback for activities. Peer-review was introduced to provide
room for meta-reflection. Students were given points for submitting activities and also for reviewing the submissions of their
classmates. For each learning module, students had to complete two activities and provide feedback to other two sub-
missions. Reviewers were automatically decided by the platform and reviews were initially blind. After initial feedback,
conversation followed between students around their solutions and how to improve them. A set of achievements was also
introduced. Achievements were aligned with learning objectives giving students recognition for completing sets of activities
and also for making significant contributions to the social network.
Supporting materials included learning modules, descriptions of the activities and videos were the same for all conditions
(experimental and control). Particularly in the educational game, students have access to learning materials online. As for
control group, a traditional blended-learning approach was used. Materials and further communication tools were available
on the BlackBoard e-learning platform. We want to stress that students were required to complete the same learning ac-
tivities in all cases and the experimental conditions determined the way in which contents and activities were delivered. In
the control and educational game group, learning modules and activities were delivered as documents, while in the rest of
experimental groups, activities were delivered as we have just described. Students in all groups had five 2-h lectures, one
lecture every two weeks, including one introductory lecture and one specific lecture for each module. Lectures introduced the
learning goals, main theoretical concepts and practical skills that students had to learn and practice. Activities were intro-
duced in the lectures and students had to work independently the rest of the time to achieve the learning goals and complete
the assignments.

2.8. Measures and data analysis

Pre-test results were returned from computers in a numeric scale. Assignments and the final examination were assessed
by lecturers using a numeric assessment scale. All marks were normalized to a 0e100 scale. The same conditions and
evaluation criteria were used in all groups.
108 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113

There was enough evidence that learning performance data gathered did not follow a normal distribution so non-
parametric tests were used. KruskaleWallis tests were used to compare pre-test and post-test results. ManneWhitney
tests with multiple comparisons were used to analyze post-test differences between pairs of groups. Bonferroni corrections
were used to adjust for the inflation of type I error in multiple comparisons.

3. Results

Experimentation took place during spring 2014 and autumn 2014. Data was gathered for appraisal by researchers as
described in the previous section. Pre-test results (Table 2) suggest that there was no significant difference between the
experimental and control groups. Table 3 presents post-test results. Significant differences were found for all evaluation
items. Results of the four practical assignments are summarized graphically in Fig. 5 showing that all experimental groups
outperformed the control group in three evaluation items (word processing, spreadsheets & presentations). For the evalu-
ation item on databases, analyses between groups (Table 4) return that there was no significant difference between the
control group and the educational game group (W ¼ 2739, p ¼ .346), and that all three other experimental groups (gami-
fication, social & social gamification) outperformed both the control and educational game group. As Ribbonhero did not
provide support for the learning module on databases, students of the educational game group completed this learning
module without instrumental support. This suggests that lack of support for the learning module on databases implies that
learning performance returns to a level that is comparable to the control group when the instrument does not offer support
for a given module. Thereby results suggest that all experimental conditions impacted on learning performance so the answer
to the first research question is positive (RQ1. Do experimental conditions impact on learning performance?) for the practical
assignments of the experiment.
As for RQ2 (Is there any difference in terms of learning performance between experimental conditions?) pairwise com-
parisons between groups are presented in Table 4. Significant although moderate differences were found in five cases. For the
word processing evaluation item, the group using the educational game performed worse than the social group (W ¼ 3446,
p ¼ .016, r ¼ .19) and the social gamified group (W ¼ 3415, p ¼ .034, r ¼ .17) suggesting that social approaches yield better
results in initial stages. As learning modules were delivered sequentially, results suggest that social approaches (social
network and social gamification) yield better results in terms of learning performance as early as week 3 when compared
with the educational game. No significant differences were found for the evaluation item on spreadsheets (week 5). As for the
evaluation item on presentations (week 7), the social group performed worse than all other experimental groups: social vs
educational game (W ¼ 2817, p ¼ .04, r ¼ .31), social vs gamification (W ¼ 1935, p < .001, r ¼ .28), social vs social gami-
fication (W ¼ 3910, p < .001, r ¼ .32). For this evaluation item, the majority of students in the experimental conditions
performed particularly well and a low level of variability can be observed resulting in moderate effect sizes that, in our
opinion, are not representative of the general trend. This general trend suggests that initially socially approaches (social and
social gamification instruments) produce better results but then such differences tend to disappear and all experimental
groups can be found in the higher end of the spectrum by week 5. Also, when treatment did not offer support for the learning
module in databases in the group with educational game, the effects disappeared and the group returned to a level similar to
the control group in terms of learning performance.

Table 2
Pre-test results of the control and the four experimental groups.

Evaluation item Group N Mean Std err. Std dev. Significance

Word Processing Control 76 47.92 1.79 15.59 H ¼ 2.14
Ed. Game 75 45.17 2.31 20.02 p ¼ .71
Gamification 77 44.75 2.05 17.96
Social 75 44.44 2.43 21.05
Social Gamif. 76 45.16 1.77 15.41
Spreadsheets Control 76 50.45 1.90 16.56 H ¼ 1.76
Ed. Game 75 51.77 1.79 15.53 p ¼ .78
Gamification 77 53.86 2.02 17.72
Social 75 52.07 1.89 16.41
Social Gamif. 76 51.39 2.10 18.30
Presentations Control 76 44.18 1.60 13.91 H ¼ .86
Ed. Game 75 44.44 1.41 12.19 p ¼ .93
Gamification 77 45.43 1.55 13.63
Social 75 45.76 1.63 14.13
Social Gamif. 76 44.58 1.37 11.90
Databases Control 75 51.80 2.09 18.06 H ¼ 3.88
Ed. Game 75 56.33 2.07 17.93 p ¼ .42
Gamification 77 55.01 2.00 17.52
Social 75 54.43 2.11 18.29
Social Gamif. 76 51.38 2.01 17.54

Significance is computed using KruskaleWallis tests.

 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113 109

Table 3
Final (post-test) results of the control and the four experimental groups.

Evaluation item Group N Mean Std err. Std dev. Significance

Word Processing Control 76 63.99 1.96 17.13 H ¼ 53.06
Ed. Game 75 78.11 1.99 17.21 p < .001
Gamification 77 75.02 2.73 23.96
Social 75 84.27 1.89 16.34
Social Gamif. 76 83.99 1.75 15.26
Spreadsheets Control 76 61.45 3.01 26.27 H ¼ 43.18
Ed. Game 75 80.93 2.78 24.04 p < .001
Gamification 77 80.06 2.71 23.79
Social 75 85.73 2.01 17.40
Social Gamif. 76 79.80 2.14 18.67
Presentations Control 76 75.75 1.44 12.52 H ¼ 82.41
Ed. Game 75 91.16 1.41 12.22 p < .001
Gamification 77 91.21 1.19 10.47
Social 75 86.32 1.30 11.23
Social Gamif. 76 89.87 1.83 15.96
Databases Control 76 52.28 2.41 21.04 H ¼ 37.96
Ed. Game 66 55.48 3.94 31.97 p < .001
Gamification 76 73.62 2.61 22.74
Social 71 66.85 3.15 26.57
Social Gamif. 74 70.86 2.41 20.75
Final Examination Control 76 74.77 1.57 13.64 H ¼ 55.97
Ed. Game 73 66.30 1.46 12.47 p < .001
Gamification 74 59.50 1.60 13.79
Social 74 61.24 1.83 15.79
Social Gamif. 76 72.00 1.76 15.37

Significance is computed using KruskaleWallis tests.

Fig. 5. Box-plots of post-test evaluation items of the control and four experimental groups.

Results of the final examination are presented graphically in Fig. 6 and unexpected differences were found. Results be-
tween groups (see also Table 4) show that the control group outperformed three experimental groups: educational game
(W ¼ 1778, p < .001), gamification (W ¼ 1236, p < .001) and social (W ¼ 1443, p < .001). No statistical difference is found
110 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113

Table 4
Comparison between groups of all evaluation items.

Groups compared Evaluation item (post-test)

Word processing Spreadsheets Presentations Databases Final examination

Control Ed. Gam W ¼ 4080 W ¼ 4144 W ¼ 4729 W ¼ 2739 W ¼ 1778
p < .001* p < .001* p < .001* p ¼ .346 p < .001*
Control Gamif W ¼ 3788 W ¼ 4207 W ¼ 4807 W ¼ 4360 W ¼ 1236
p ¼ .002* p < .001* p < .001* p < .001* p < .001*
Control Social W ¼ 4574 W ¼ 4467 W ¼ 4275 W ¼ 3662 W ¼ 1443
p < .001* p < .001* p < .001* p < .001* p < .001*
Control Social Gamif W ¼ 4715 W ¼ 4070 W ¼ 4588 W ¼ 4122 W ¼ 2593
p < .001* p < .001* p < .001* p < .001* p ¼ .278
Ed. Gam Gamif W ¼ 2881 W ¼ 2878 W ¼ 2806 W ¼ 3310 W ¼ 1879
p ¼ .981 p ¼ .973 p ¼ .755 p ¼ .001* p ¼ .002*
Ed. Gam Social W ¼ 3446 W ¼ 2944 W ¼ 2817 W ¼ 2817 W ¼ 2170
p ¼ .016* p ¼ .616 p ¼ .040* p ¼ .041* p ¼ .004*
Ed. Gam Social Gamif W ¼ 3415 W ¼ 2497 W ¼ 2869 W ¼ 3044 W ¼ 3446
p ¼ .034* p ¼ .182 p ¼ .940 p ¼ .012* p ¼ .001*
Gamif Social W ¼ 2406 W ¼ 3001 W ¼ 1935 W ¼ 2317 W ¼ 3059
p ¼ .051 p ¼ .671 p < .001* p ¼ .140 p ¼ .218
Gamif Social Gamif W ¼ 3377 W ¼ 2622 W ¼ 3140 W ¼ 2478 W ¼ 4122
p ¼ .100 p ¼ .260 p ¼ .406 p ¼ .211 p < .001*
Social Social Gamif W ¼ 2710 W ¼ 2972 W ¼ 3910 W ¼ 2730 W ¼ 3876
p ¼ .596 p ¼ .630 p < .001* p ¼ .685 p < .001*

Significance is computed using ManneWhitney tests with Bonferroni corrections to adjust for the inflation of type I error.

Fig. 6. Box-plot of the final score of the control and four experimental groups.

between the control group and social gamification group (W ¼ 2593, p ¼ .278). The gamification group and the social group
performed similarly (W ¼ 3059, p ¼ .218) but also poorly averaging 15 points less than the control group and the social
gamified group (W ¼ 4122, p < .001 for gamified vs social gamified; W ¼ 3876, p < .001 for social vs social gamified). The
educational game group can be found in-between performing better than the gamified (W ¼ 1879, p ¼ .002) and social groups
(W ¼ 2170, p ¼ .004) but worse than the control and social gamified (W ¼ 3446, p ¼ .001) groups. Thus in terms of final
examination, none of the experimental groups performed better than the control group, and the control group performed
better when compared with three experimental conditions. Therefore and as for RQ3 (Does the kind of evaluation item in-
fluences results on learning performance for the different experimental conditions?), results suggest that experimental tools
impact on learning results as measured by evaluation items that assess practical skills (assignments) but do not impact on
examinations that primarily assess knowledge acquisition. No relevant differences between experimental conditions are
found when assessing practical assignments. However when it comes to the final examination, the social gamification tool
returned better results, but similar to the control group. It was followed by educational gaming, whereas gamification and
social approaches yielded the worst results in terms of learning performance.

4. Discussion

Results of the present study in an undergraduate course on ICT qualification suggest that the Ribbonhero educational game
can be integrated into a learning experience that is related with the topics presented in the game boosting learning
 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113 111

performance if learning modules are supported by the game. Competition reward-based gamification also produces benefits
suggesting that gamified approaches based on extrinsic motivators also boost learning performance when learning activities
and learning objectives are carefully aligned with the instrument. Similarly, cooperative and collaborative student work
supported by an educational social network also produces comparative results, as does an instrument that combines
gamification and social networking. Results also show that social approaches (social networking and social gamification) yield
benefits in terms of learning performance sooner when compared with approaches that are only based on games suggesting
that the cooperative and collaborative nature of social networks can be better harnessed to produce short term results. In
terms of the type of evaluation item, a traditional blended-learning approach still produces better results than the educa-
tional game, competitive-driven gamification and social networking in a final examination. Results of blended-learning only
compare with social gamification for the final examination item. Therefore, combining gamification and social approaches
produces the better outcomes in terms of learning performance across the different evaluation items considered in this study.
So harnessing the motivational potential of both approaches (gamification and social networking) seems to be the most
promising way to address the needs of young undergraduate students producing significant results in gameful educational
settings. We conjecture that the combination of social and game-like approaches promotes interaction and discussion around
both conceptual constructs and practical artifacts resulting in better outcomes as measured by learning performance.
When the educational game did not offer support for the learning module on databases, the benefits in terms of learning
performance disappeared evidencing that learning effects are short-lived, as previous studies in non-educational settings
have also showed (Denny, 2013; Koivisto & Hamari, 2014; Thom, Millen, & DiMicco, 2012). This suggests that educational
games non-specifically aligned with learning objectives may produce a kind of shallow gamification (Dichev, Dicheva,
Angelova, & Agre, 2014) that yields short-term results but does not promote long-term motivational outcomes. As it has
been also pointed, gamification approaches can motivate participation by increasing the number of interactions but not their
quality (Halan et al., 2010). Nevertheless, our results also evidence the positive (although short-lived) effects that such
shallow gamification approaches convey. A recent study by Lieberoth (Lieberoth, 2014) argues that it is enough to “dress” an
activity as a game in order to facilitate motivational affordances. So, already available educational games can still produce
cost-efficient boosts on learning performance that can be harnessed by educators and easily implemented. Similar conclu-
sions cannot be drawn for the other experimental conditions (gamification, social networking and social gamification) as they
were used for all learning modules and duration of the learning action.
On the other hand, recent results of gamification in education (Hanus & Fox, 2015) suggests that as the novelty of the new
technology expires, excitement decreases resulting in decreasing motivation. In our study we did not find any decrease in
terms of learning performance when instruments provided support for the learning modules over a period of ten weeks.
Hanus & Fox concluded that gamification undermines motivation, effort and empowerment resulting in lower grades in a
final exam. Our study suggests that instruments and the kind of evaluation items are relevant for gamification in education
and that gameful approaches have the potential to impact on learning performance. Still, contrasting results found here for
different evaluation items raise concerns about the kind of learning that each instrument promotes, particularly when results
of the final examination are observed, as only the performance of the social gamification group compared with the control
group while students in all other experimental conditions performed particularly poorly. When relating the educational
means with the kind of competences and targeted competencies, we found that new media facilitated skill acquisitions as
measured by a set of assignments that assessed proficiency with computational tools for producing informational artifacts
and assets. Conversely, we also found that traditional media (blended-learning) improved knowledge acquisition as
measured by a written examination designed to assess the meaning and practical application of different concepts that were
used in applications and that were particularly relevant to understand how informational artifacts work and are produced
efficiently. Consequently, the effectiveness of new gameful approaches to convey traditional conceptual knowledge can also
be questioned. Our results point to the strengths and weaknesses of blended-learning and new (game-based and social)
media providing advice about how and for what different means and instruments can be used. Yet in another recent study Su
& Cheng (Su & Cheng, 2014) reported positive results with mobile gamification in children, suggesting that the age of par-
ticipants, the learning topic, or the nature of technology (context-awareness) can also be relevant for the success of
educational gamification approaches. As such results contrast with our findings, these factors require further enquiry. Also,
another possible limitation of this study has to do with the simplicity of the design of the gamification tool. Only challenges,
trophies, badges and a leaderboard were included resulting in a simple reward-based approach to gamification. Best practice
theories like meaningful gamification (Nicholson, 2012, 2015) may provide the theoretical background to design engaging
gamified experiences that produce long-term benefits.
As for social networking, our study supports previous literature on the positive effects of social networking in education
(Cho et al., 2007; de-Jorge-Moreno, 2012; Thoms, 2011) but it also provides additional insights when comparing such results
with other novel approaches like gamification, and also when both are combined. In terms of comparative results, this work
supports previous studies that compared gamification and social networking (de-Marcos, Dominguez, Saenz-de-Navarrete, &
Page
s, 2014) which also found that similar experimental conditions produced benefits in terms of learning performance. The
present study also includes new experimental conditions suggesting that benefits are similar in educational games, gami-
fication and social networking, but also that social gamification presents higher potential to yield benefits for both knowledge
and skill acquisition. Critical accounts on the effects of social networking in education (Tess, 2013) mostly point to meth-
odological concerns and experimental design issues. Although in technical terms our method was not a randomized control
trial, we think that such concerns are mitigated with a careful experimental design, but eventually this is a threat to validity.
112 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113

Other limitations of our study are concerned with generalization. Although sample sizes were reasonable (about 75
participants per experimental condition), the effects of lack of support of instruments for specific learning objectives were
only assessed incidentally for one experimental condition due to the nature of the instrument (the educational game did not
support the learning module on databases), so further research has to be conducted in order to determine the long-term
educational effects of the gamified and social networking approaches. Also, in terms of the analysis of the impact of in-
struments for different evaluation items, four practical assignments and only one final examination were included in the
present study. To make any claim for generalization, more cases should be included and pedagogical concerns about the kind
of learning that each evaluation item conveys should also be addressed. The possible biases of different assessments towards
instruments (e.g. skill evaluation towards social aspects) can also be accounted as possible limitations. Concerns about
generalizability also include the demographics and the context of use of the different instruments. Participants were solely
undergraduate students and the educational setting was a 10-week undergraduate course on ICT. Young adults interested in
ICT may feel particularly attracted by technology and game-like gadgets biasing the results of the study. Learning objectives
were aligned with instruments and content delivered for the gamification, social networking and gamified social networking
instruments. The educational game was not aligned with the learning objectives. Although it would be tempting to suggest
that the mere presence of technology suffices to boost learning performance, present results are circumscribed to this
particular instance and such claims are questionable without further research.

5. Conclusions and future research

This work studied the effect that an educational game, gamification, social networking and social gamification have on
learning performance in an undergraduate course. Our aim was to study the four experimental conditions on the same
educational setting to facilitate comparative analysis. Results suggest that all experimental conditions significantly impact on
learning performance. Moderate differences were also found when experimental conditions were compared suggesting that
social networking and social gamification produced better results even at early stages of the course (week 3). The effects on
the different kinds of evaluation items were also studied and we found that in a final examination designed to assess con-
ceptual knowledge, the new approaches did not yield any benefit when compared with a control group. Students that used
the educational game, the gamification plugin and social network performed poorly when compared with the social gami-
fication and control (blended-learning) groups. Social gamification returned better results in terms of immediacy across
different evaluation items. This study then stresses the difference between practical skills, where new media resulted
appropriate, and conceptual knowledge, where blended-learning resulted better, highlighting the necessity of comple-
mentarity for balanced teaching and learning, and providing insights about how and for what use the different tools. Results
are circumscribed to a very specific population of young adult undergraduate students in an undergraduate course on ICT
qualification. With these caveats about generalization in mind, we can also suggest that already available educational games
have the potential to be easily integrated in educational settings producing boosts in terms of learning performance of
practical skills. Reward-based competitive gamification or social networking can also be integrated with comparable results,
although social networking is effective sooner. Finally social gamification produces better results across all evaluation items
and also at earlier stages.
Incidentally, our work also studied the effects of the lack of support in one of the experimental conditions, namely the
educational game. Results showed that the positive effects disappear when learning module is not supported by the treat-
ment. This questions the effectiveness of gameful approaches to afford long-term intrinsically motivating and sustained
engagement. Contrasting opinions suggest that gamification fosters shallow learning. But ultimately the question of retention
and how gamification supports or undermines long-term learning still remains open and further enquiry is needed. Also,
combining gamification and social networks provides an interesting ground to study the structure of the underlying social
network, how gamification influences the social network and also the effects that the position in the social network can have
on learning performance of each participant.

References

Arnab, S., Lim, T., Carvalho, M. B., Bellotti, F., de Freitas, S., Louchart, S., et al. (2015). Mapping learning and game mechanics for serious games analysis.
British Journal of Educational Technology, 46(2), 391e411. http://dx.doi.org/10.1111/bjet.12113.
Bartle, R. (1996). Hearts, clubs, diamonds, spades: players who suit MUDs. Journal of MUD Research. Retrieved from http://www.mud.co.uk/richard/hcds.
htm.
Bellotti, F., Berta, R., De Gloria, A., Lavagnino, E., Dagnino, F. M., Antonaci, A., et al. (July 2013). A gamified short course for promoting entrepreneurship
among ICT engineering students. In , 2013. Paper presented at the advanced learning technologies (ICALT), 2013 IEEE 13th international conference on (pp.
15e18).
Brady, K. P., Holcomb, L. B., & Smith, B. V. (2010). The use of alternative social networking sites in higher educational settings: a case study of the e-learning
benefits of Ning in education. Journal of Interactive Online Learning, 9(2), 151e170.
Cho, H., Gay, G., Davidson, B., & Ingraffea, A. (2007). Social networks, communication styles, and learning performance in a CSCL community. Computers &
Education, 49(2), 309e329. http://dx.doi.org/10.1016/j.compedu.2005.07.003.
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and
serious games. Computers & Education, 59(2), 661e686. http://dx.doi.org/10.1016/j.compedu.2012.03.004.
Denny, P. (2013). The effect of virtual achievements on student engagement. In Paper presented at the proceedings of the SIGCHI conference on human factors
in computing systems, Paris, France. http://dx.doi.org/10.1145/2470654.2470763.
Despotovic-Zrakic, M. S., Labus, A. B., & Milic, A. R. (2011). Fostering engineering e-learning courses with social network services. In Paper presented at the
19th telecommunications forum (TELFOR). http://dx.doi.org/10.1109/TELFOR.2011.6143508.
 L. de-Marcos et al. / Computers & Education 95 (2016) 99e113 113

Deterding, S. (2014). The ambiguity of games: histories and discurses of a gameful world. In S. Deterding, & S. P. Walz (Eds.), The gameful world: Approaches,
issues, applications. Cambridge, Massachusetts: MIT Press. http://dx.doi.org/10.1080/00140139.2015.1067048.
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: defining “gamification”. In Paper presented at the
proceedings of the 15th international academic mindtrek conference: envisioning future Media environments, Tampere, Finland. http://dx.doi.org/10.1145/
2181037.2181040.
Dicheva, D., Dichev, C., Agre, G., & Angelova, G. (2015). Gamification in education: a systematic mapping study. Educational Technology & Society, 18(1), 1e14.
Dichev, C., Dicheva, D., Angelova, G., & Agre, G. (2014). From gamification to gameful design and gameful experience in learning. Cybernetics and Information
Technologies, 14(4), 80e100. http://dx.doi.org/10.1515/cait-2014-0007.
Domínguez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fern
andez-Sanz, L., Page
s, C., & Martínez-Herr
aiz, J.-J. (2013). Gamifying learning experiences:
practical implications and outcomes. Computers & Education, 63, 380e392. http://dx.doi.org/10.1016/j.compedu.2012.12.020.
Fabricatore, C., & Lo
pez, X. (2014). Using gameplay patterns to gamify learning experiences. In C. Busch (Ed.), Proceedings of the 8th European conference on
game based learning (pp. 110e117). Reading, UK: Academic Conferences and Publishing International Limited. http://doi.org/10.13140/2.1.2275.5844.
Fitz-Walter, Z., Tjondronegoro, D., & Wyeth, P. (2012). A gamified mobile application for engaging new students at university orientation. In Paper presented
at the proceedings of the 24th Australian computer-human interaction conference, Melbourne, Australia. http://dx.doi.org/10.1145/2414536.2414560.
Gaggioli, A., Mazzoni, E., Milani, L., & Riva, G. (2015). The creative link: investigating the relationship between social network indices, creative performance
and flow in blended teams. Computers in Human Behavior, 42(0), 157e166. http://dx.doi.org/10.1016/j.chb.2013.12.003.
Gee, J. P. (2007). What video games have to teach us about learning and literacy. New York: Palgrave Macmillan.
Haksu, L., & Young Yim, D. (Aug. 2012). A study on the relationship between educational achievement and emotional engagement in a gameful interface for
video lecture systems. In , 2012. Paper presented at the International Symposium on Ubiquitous Virtual Reality (ISUVR) (pp. 22e25). http://dx.doi.org/10.
1109/isuvr.2012.21.
Halan, S., Rossen, B., Cendan, J., & Lok, B. (2010). High score! e motivation strategies for user participation in virtual human development. In J. Allbeck, N.
Badler, T. Bickmore, C. Pelachaud, & A. Safonova (Eds.), Intelligent virtual agents (Vol. 6356, pp. 482e488). Springer Berlin Heidelberg. http://dx.doi.org/
10.1007/978-3-642-15892-6_52.
Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work? e a literature review of empirical studies on gamification. In Paper presented at the 47th
Hawaii international conference on system sciences, Hawaii, USA. http://dx.doi.org/10.1109/HICSS.2014.377.
Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison,
satisfaction, effort, and academic performance. Computers & Education, 80, 152e161. http://dx.doi.org/10.1016/j.compedu.2014.08.019.
Heeter, C., Lee, Y.-H., Medler, B., & Magerko, B. (2011). Beyond player types: gaming achievement goal. In Paper presented at the proceedings of the 2011 ACM
SIGGRAPH symposium on video games, Vancouver, British Columbia, Canada. http://dx.doi.org/10.1145/2037692.2037701.
Heeter, C., Magerko, B., Medler, B., & Fitzgerald, J. (2009). game design and the challenge-avoiding, self-validator player type. International Journal of Gaming
and Computer-Mediated Simulations (IJGCMS), 1(3), 53e67. http://dx.doi.org/10.4018/jgcms.2009070104.
de-Jorge-Moreno, J. (2012). Using social network and dropbox in blended learning: an application to university education business. Management and
Education, 10(1), 221e232. http://dx.doi.org/10.3846/bme.2012.16.
Kapp, K. M. (2012). The gamification of learning and instruction: Game-based methods and strategies for training and education. San Francisco: Pfeiffer.
Koivisto, J., & Hamari, J. (2014). Demographic differences in perceived benefits from gamification. Computers in Human Behavior, 35, 179e188. http://dx.doi.
org/10.1016/j.chb.2014.03.007.
Lieberoth, A. (2014). Shallow gamification: testing psychological effects of framing an activity as a game. Games and Culture. http://dx.doi.org/10.1007/
s11251-011-9177-2.
Li, W., Grossman, T., & Fitzmaurice, G. (2012). GamiCAD: a gamified tutorial system for first time autocad users. In Paper presented at the proceedings of the
25th annual ACM symposium on user interface software and technology, Cambridge, Massachusetts, USA. http://dx.doi.org/10.1145/2380116.2380131.
Lim, T., Louchart, S., Suttie, N., Hauge, J., Stanescu, I., Ortiz, I., et al. (2014). Narrative Serious Game Mechanics (NSGM) e insights into the narrative-
pedagogical mechanism. In S. Go €bel, & J. Wiemeyer (Eds.), Games for training, education, health and sports (Vol. 8395, pp. 23e34). Springer Interna-
tional Publishing. http://dx.doi.org/10.1007/978-3-319-05972-3_4.
Lu, J., & Law, N. (2012). Online peer assessment: effects of cognitive and affective feedback. Instructional Science, 40(2), 257e275. http://dx.doi.org/10.1007/
s11251-011-9177-2.
de-Marcos, L., Dominguez, A., Saenz-de-Navarrete, J., & Page
s, C. (2014). An empirical study comparing gamification and social networking on e-learning.
Computers and Education, 75, 82e91. http://dx.doi.org/10.1016/j.compedu.2014.01.012.
Mekler, E. D., Brühlmann, F., Opwis, K., & Tuch, A. N. (2013). Do points, levels and leaderboards harm intrinsic motivation?: an empirical analysis of common
gamification elements. In Paper presented at the proceedings of the first international conference on gameful design, research, and applications, Toronto,
Ontario, Canada. http://dx.doi.org/10.1145/2583008.2583017.
Nicholson, S. (2012). A user-centered theoretical framework for meaningful gamification. Gamesþ Learningþ Society, 8(1).
Nicholson, S. (2015). A recipe for meaningful gamification. In T. Reiners, & L. C. Wood (Eds.), Gamification in education and business (pp. 1e20). Springer
International Publishing. http://dx.doi.org/10.1007/978-3-319-10208-5_1.
Paredes, W. C., & Chung, K. S. K. (2012). Modelling learning & performance: a social networks perspective. In Paper presented at the 2nd international
conference on learning analytics and knowledge. http://dx.doi.org/10.1145/2330601.2330617.
Raymer, R. (2011). Gamification: using game mechanics to enhance e learning. eLearn, 2011(9). http://dx.doi.org/10.1145/2025356.2031772.
Santos, J., Charleer, S., Parra, G., Klerkx, J., Duval, E., & Verbert, K. (2013). Evaluating the use of open badges in an open learning environment. In D.
Herna
ndez-Leo, T. Ley, R. Klamma, & A. Harrer (Eds.), Scaling up learning for sustained impact (Vol. 8095, pp. 314e327). Springer Berlin Heidelberg.
http://dx.doi.org/10.1007/978-3-642-40814-4_25.
Sheldon, L. (2012). The multiplayer classroom: designing courseware as a game. Boston: Course Technology. Cengage Learning.
Simo ~es, J., Redondo, R. D., & Vilas, A. F. (2013). A social gamification framework for a K-6 learning platform. Computers in Human Behavior, 29(2), 345e353.
http://dx.doi.org/10.1016/j.chb.2012.06.007.
Squire, K. (2011). Video games and learning: Teaching and participatory culture in the digital age. New York: Teachers College Press.
Stewart, B. (2011). Personality and play styles: A unified model. Gamasutra.
Su, C. H., & Cheng, C. H. (2014). A mobile gamification learning system for improving the learning motivation and achievements. Journal of Computer Assisted
Learning. early view, n/a-n/a http://dx.doi.org/10.1111/jcal.12088.
Tess, P. A. (2013). The role of social media in higher education classes (real and virtual) e a literature review. Computers in Human Behavior, 29(5), A60eA68.
http://dx.doi.org/10.1016/j.chb.2012.12.032.
Thom, J., Millen, D. R., & DiMicco, J. (2012). Removing gamification from an enterprise SNS. In Paper presented at the ACM conference on Computer Supported
Cooperative Work (CSCW). http://dx.doi.org/10.1145/2145204.2145362.
Thoms, B. (2011). A dynamic social feedback system to support learning and social interaction in higher education. IEEE Transactions on Learning Tech-
nologies, 4(4), 340e352. http://dx.doi.org/10.1109/tlt.2011.9.
Werbach, K., & Hunter, D. (2012). For the win: How game thinking can revolutionize your business. Philadelphia: Wharton Digital Press.
Xiao, Y., & Lucking, R. (2008). The impact of two types of peer assessment on students' performance and satisfaction within a wiki environment. The Internet
and Higher Education, 11(3e4), 186e193. http://dx.doi.org/10.1016/j.iheduc.2008.06.005.