EETC 2022 JTP Paper
EETC 2022 JTP Paper
SINGAPORE POLYTECHNIC
EXCELLENCE IN EDUCATION AND TRAINING CONVENTION (EETC) 2022
JOURNAL OF TEACHING PRACTICE
School of Computing
ABSTRACT
Studies have shown that gamification in education, if applied correctly can enhance students’
engagement level and optimize their learning (Barata et al., 2013; Brewer et al., 2013; Todor and
Pitic, 2013). In this paper, we investigate the effects of gamification on student’s self-directed
learning using our own in-house developed web-based programming tool, Mimosa. Mimosa seeks
to test users on their offensive and defensive skills related to secure programming using
gamification elements composing of points, levels, badges, leaderboards, storyline and feedback.
We conducted a study with our second year Diploma in Cyber Security Management students
taking secure coding for four months. Our survey results collated from a total of 87 survey
respondents have indicated that the gamification elements in Mimosa are popular with our students
and have effectively enhanced their learning and motivation and encouraged self-directed learning.
1 Introduction
The gamification theory in education states that learners learn best when they are also having fun.
Many educational institutions have created their own gamified learning environments to improve
their students’ engagement level and improve learning outcomes with varying degrees of success.
Some researchers have introduced gamification into learning and obtained favorable results, such as
increased user engagement, knowledge retention, motivation and satisfaction in the learning process
(Barata et al., 2013; Brewer et al., 2013; Todor and Pitic, 2013). In this study, we investigate the
effects of gamification on our year 2 DISM students using our in-house developed web-based
programming tool, Mimosa. We introduce various gaming design elements such as points, levels,
badges, leaderboards, storyline and feedback in Mimosa. These gaming design elements have been
identified by researchers as effective design elements when incorporated into the teaching tools to
engage learners (Barata et al., 2013; Brewer et al., 2013; Kapp, 2013; Kumar and Khurana, 2012;
Todor and Pitic, 2013). Carrying out the study in a period of 4 months, we attempt to investigate
how gamification affects students’ learning, engagement and motivation. We also attempt to see if
the tool helps to promote self-directed learning among the students.
2 Background
While interacting with students who were not performing well in the course, we would hear
of feedback from them that they find programming and traditional lectures and practicals dry
and boring and hard to understand. The main reason for their poor performance and lack of
interest is usually attributed to the lack of motivation and disengagement in learning
(Broussard & Garrison, 2004).
This started a thinking process to re-evaluate the learning strategies in the classroom, to see
if there is a way to implement new learning strategies which will more effectively engage
the learner, increase their interest in the subject matter and their motivation to learn. It would
also be an added advantage if the tool could allow us to easily keep track of students’
progress and intervene in their learning if necessary.
We decided to look into gamification in education and co-developed the web programming
tool Mimosa with students through their final year project. As a project developed through
the collaboration of both students and lecturers, we have a better understanding of the needs
of our target users and successfully incorporated gaming elements comprising of points,
levels, badges, leaderboards, storyline and feedback and basic analytics capability into
Mimosa.
3 Research Questions
The aims of this project would be to evaluate the effectiveness of gamification in education
to improve students’ learning, engagement and motivation. With the enhanced learning
experience, we would also like to evaluate if the students have picked up attributes of self-
directed learning.
1) How do students feel about the gamification web tool Mimosa in their leaning?
- Are there any gamification features in Mimosa that the students generally like?
- Does Mimosa make learning secure coding more fun?
- Is Mimosa preferred over traditional teaching methods like lectures and practicals?
- Does Mimosa improve their motivation in learning?
4 Literature Review
A number of studies (Barata et al.,2013; Todor and Pitic, 2013) had introduced game-like
elements in the by introducing multiple game design elements into the course design,
including scoring, levels, leaderboards, challenges and badges and observed an improvement
in course participation and motivation among the students. Barata et al. (2013) found that
gamified approach was very promising, showing significant increases in online participation,
attendance and accessing of the course reference materials.
Brewer et al. (2013) compared the task completion rate of children with and without
gamification elements in an application to assess the effects of gamification on children,
who tend to be easily distracted and require higher motivation to complete the tasks given in
a lab environment. To increase motivation among children, they introduced point-based
scoring system and awarded small physical token prizes for children who achieve sufficient
points. The use of gamification increased the task completion rates from 73% to 97%. The
authors conclude that gamification is effective in engaging children and increased their
motivation.
While gamification has shown promising results in increasing motivation and engagement
within students, the strategy and way of implementation is important to determine success.
Todor and Pitic (2013) highlighted that badges should not only be awarded for obtaining
high scores but also for completing objectives, so as to encourage learning and not be overly
focused on being the best. While there should be aspects of competition, we should not
overly focus on it. Kumar and Khurana (2012) also indicated that gamification should
always include the objective of "fun” as part of the learning journey. It should not be just
about scores and grades.
Kapp (2012) proposed that game-based learning can engage disconnected learners if applied
correctly. Freedom to fail without penalty encourages learners to explore and trial and error.
Through storytelling, rather than presenting facts in a bullet list, exercises can be presented
in a fun and interesting way, and this helps to sustain learners’ interest and engagement.
Proper feedback presented in the right frequency, intensity and time would also help to
sustain engagement of the learner.
The profile of the students is also a consideration factor when introducing gamification.
Berkling and Thomas (2013) had introduced a web-based gamification learning platform to
teach a Software Engineering course at university level without success. Students expressed
they did not find the platform interesting and helpful. Berkling and Thomas (2013) analyzed
the results and attributed the negative results to a few reasons, including their students being
non-gamers in general, and they were unable to see the relevance of the system in preparing
for the exam. The aesthetic appeal of the system was also low. The survey highlights that for
gamification in learning to succeed, the audience matters and there is a need in customizing
the platform to the needs of the audience.
5 Methodology
Taking inspiration from the findings of the gamification related literature review findings,
we proceeded to incorporate the useful gaming elements into a web-based application to
teach secure coding. While there are open source tools, for example webgoat(
https://owasp.org/www-project-webgoat/) available online to learn and practice secure
coding, the applications are targeted at the general population to be used individually on
their local machine and not customized with gaming features or analytics.
We envisioned a web tool with fun exercises and gaming elements that teaches offensive
and defensive secure coding. Given that this tool is designed for student learning, we
decided that it would be best to co-develop the application with the students’ involvement
and participation following design thinking principles. Design thinking is defined as an
analytic and creative process that engages a person to experiment, create and prototype
models, gather feedback, and redesign iteratively for improvement (Rim Razzouk and
Valerie Shute, 2012).
Diploma in Cyber Security Management (DISM) students were recruited through their final
year project module to work on the project. While we set the overall vision of the project
and the gaming elements we want to be setup, students were given the autonomy to come up
with their own storyline for the challenges, overall user interface and case scenario. As the
target audience for the system, they form the users’ profile audience and their participation is
relied on for the successful creation of this system. Customizing the gamified application to
the profile and needs of the learners is important to the success of the experiment as
highlighted by Berkling and Thomas(2013).
The gamified experience was introduced during the start of the semester for a period of 4
months. During class, students were asked to sign up for Mimosa to practice their skills on
the system challenges during the weekly practical sessions. How Mimosa works is that
challenges with different storyline familiar to them from various categories are provided to
them and they will be awarded points upon completion of the challenges. Partial points may
also be awarded for progression towards solving the challenge as part of a feedback system.
Users can also earn badges when they complete challenges, a feature which we hope will
motivate them to work harder to solve more challenges. Students can compare their progress
with others in class through leaderboards. The key gamification elements that were
successfully implemented in Mimosa include points, levels, badges, leaderboards, storyline
and feedback (awarding of partial points for completion progress). Screenshots illustrating
the gamification elements are provided in Figures 1-3. Basic analytics features to monitor
the students’ progress, completion level of various challenges are shown in Figure 4.
Triangulation refers to the use of multiple data sources, comparing and cross-checking the
consistency of information derived at different times and means within qualitative methods
to develop a comprehensive understanding of the phenomena (Patton, 1999). The three main
sources of data collection for this study would be from the main stakeholders: students,
lecturers and the principal investigator.
For the students, an online survey was conducted to gauge their perception of the usefulness
of our gamified web application Mimosa towards their learning. Through a series of closed-
ended and open-ended questions, we collected both quantitative and qualitative feedback
from the students. A small focus group study was also conducted with a small group of
students to understand if the application has enforced self-directed learning attributes within
them.
For the lecturers, we conducted an interview to access whether they think Mimosa has
helped their students in their learning and if Mimosa has impacted their delivery of the
lesson in any way.
Finally, I will also be doing a self-reflection to evaluate how I feel gamification has affected
the delivery of the module and satisfaction in the course based on my observations of the
students I teach.
6 Results
An online survey consisting of 5 closed-ended questions and 2 open ended questions was
conducted with the students.
The 2 open-ended questions solicited feedback from students on what they like and dislike
about the system. The purpose of the 2 questions was to collect qualitative feedback on what
aspects of the web application students like and what improvements we can make for the
next iteration of Mimosa.
Table 1 shows the response rate of the survey. Table 2 shows the questions and rating for the
first 4 closed ended question. Figure 5 shows the results for the 5th question. Table 3 shows a
sample of the key responses for the open-ended questions.
Close-ended Questions:
Question Mean Median Standard
Deviation
Is the website easy to use? 4.72 5 0.474700767
Open-Ended Questions
Question 10 Sample Responses
What do The challenges are super fun and interesting to try out.
you like
about the Simple to use and scoring system good way of keeping track challenge completion
system? progress.
The badges at the top right that tells you that you are the first in class to solve x
problem. It motivates me to do better.
Rather than plainly learning without putting these knowledge to good use, I was
able to imply these knowledge first hand and better understand the usage of codes
through this experience.
The challenges are made in a fun way that makes me excited to find out what back
story the next challenege holds.
Some of the questions were relatable and funny, while the best part of how easy it
was to navigate and use. The leaderboard also motivated most, if not all, to work
hard towards finding
the right solution.
I like how the system simulates a real website. It makes learning more fun,
especially for practising offensive techniques, as we feel like we are actually
accomplishing something when we complete the tasks and the green success popup
appears. The UI is also simple and neat which I personally like. The challenges are
also quite funny to read with all the parody version of companies like fakebook,
dazala, etc and all the little descriptions and stuff.
I like that it’s interactive. One of the only modules where there’s actual feedback
from the application that allows us to know exactly if our solution is right
I like how the system gives out some marks even though the answer given is not the
final answer. This helped me make sure if I was in the right track when answering
questions.
Are there any Maybe to keep the solutions/show recommended solutions once the challenge have
improvements to been completed.
the website that
you would like to It is a bit buggy sometimes, some of the challenge hints aren't helpful...
share? [ rant all the
bad stuff about the It will be nice if you can provide a walkthrough for the difficult challenges.
website here ;) ]
More problems to solve
Make the hints more useful. "try burpsuite" isn't a very helpful hint and its worse if
we spend points during MST for a hint that doesn't help us.
Not really, maybe just provide more hints as sometimes im stuck and require more
help.
I feel like some of the challenges are definitely hard. But the thing is there are no
official solutions page. Which means that we have to wait for our lessons with our
lecturer in order to know how to solve the questions. As such, I do think a solutions
page would be great.
A small focus group study was conducted with 4 students who used Mimosa for secure
coding. The objective of the study was to find out how Mimosa impacted their learning and
if it helped develop attributes which contributed towards self-directed learning. An open
discussion was conducted with the study group to discuss the following questions:
Overall the students provided positive feedback on the system. Students feedback that the
categorization of the challenges enabled them to focus on the topic at hand and test their
skills category by category. The system also did not require any form of extra setup
compared to practical projects which sometimes require downloading and installation, and
possible debugging when the setup failed. This allowed them to focus on the exercise at
hand. Compared to conventional practicals which usually guided them step by step, Mimosa
stretched their critical thinking, making them trial and error to experiment with different
ways to solve the challenge and to look for different resources online focusing on the
relevant topic. This helped them review their learning process, and to evaluate how they can
apply the different skillsets taught into practice for different scenarios. The leaderboard and
the points system encouraged them to persevere to complete the challenge without activating
any hint as that would deduct points from them. To solve the challenges, they tended to
collaborate and discuss more, sharing which techniques did or did not work as they
attempted to solve the challenges. As the challenges can be solved in multiple ways, this
allowed them to extend their learning, showing them multiple perspectives in solving a
challenge during discussion sessions. One issue they did point out is that due to the less
guided format, some of the weaker students may give up after trying and in an attempt to
complete the challenge, they would just seek out the solution from the other classmates in
order to get the points from the system. So appropriate free hints or some form of guidance
may work well for this group of students who need more help. In conclusion, Mimosa has
helped increase their level of interest and motivation and encouraged them to become self-
directed learners who are able to tap on various resources to tackle different challenges
posed to them.
An informal interview was conducted with a follow member of the teaching team to discuss
the impact of Mimosa towards student learning in his classroom. This lecturer has had more
than 20 years of teaching experience in the education industry and familiar with using
different pedagogical tools in teaching. The key questions discussed include:
The lecturer commented positively on Mimosa, saying it generated excitement within his
class, with students looking forward to challenges after the lecture was conducted. The
gamification elements were something unique in secure coding and was well received
among his students. Students found it fun and it provided an element of competition with
students trying to compete with each other for badges and a top position in class. Besides
fun, given that the challenges were linked to formal assessments, it provided extra
motivation for students to complete the challenges and encouraged discussion among the
students as they shared resources and strategy to complete the challenges. As students
discussed and shared their methods, it helped in their critical thinking, understanding
different methods to tackle a challenge. This simulated real life problem solving and
learning, a key skillset we wish to develop in our students. The analytics feature has also
enabled him to identify students who have not been keeping pace with the rest as well as the
questions that most students are not scoring well. This allowed him to re-visit those
questions and address the obstacles in learning. Overall, the experience with Mimosa was
good, and it increased learning engagement and satisfaction among the students.
As a lecturer who teaches secure coding, I noticed the gamification tool has an impact on
intrinsic motivation of many students who continually challenge themselves to improve on
their results and scores to reach a perfect rating. Even for difficult challenges which are
beyond the scope of the topics taught and made optional, there were still students who
attempted to solve them in the hopes of raising their rank in the leaderboard and getting
more points. While there were hints that students can activate to solve the more challenging
exercises, students refused to activate them as their total attainable points would be reduced
and they persisted to solve it on their own. Such was the impact of the gamification
elements. There were also constant requests from students to release more challenges for
them to work on as they found a sense of accomplishment in completing the challenges.
The analytics feature of Mimosa also allowed me to monitor the scores of students and their
progress. Through this, I have managed to identify students who were facing difficulties or
falling behind in their work. This allowed me to give them a nudge in class at the
appropriate time. It also allowed me to see if they were facing difficulties in a particular
topic and recap parts of the topic in class.
Overall, I found Mimosa, with its gamification elements, have become an integral part of the
secure coding module. All the positive encouragement and observations in class have
motivated me to continue development for this project to address the weak points of the
system with a new group of students who used Mimosa and are ready to put forth their
creative ideas to the next enhanced version of Mimosa.
7 Discussion
Our study analyzed the potential impact of a gamification tool on learner engagement,
motivation and whether appropriately designed challenges can encourage the development
of self-directed attributes students.
Through our survey and focus group study, we seek to understand how students feel about
the gamification web tool Mimosa in their leaning.
The majority of students found Mimosa to be a fun learning tool and enjoyed the gamified
experience, preferring this new mode of learning compared to traditional teaching methods.
94.3% of respondents found the tool to have motivated them in learning secure coding.
Through the open-ended questions, we have received many positive feedback from our
students who complimented the application on the gamification elements we introduced:
points, levels, badges, leaderboards, storyline and feedback. Different students have found
their own preferred gamification element(s), and overall Mimosa engaged them and
encouraged them to explore more. Both the quantitative and qualitative feedback received
have ascertained that gamification elements in Mimosa have enriched the learning
experience of the students and improved learning engagement, knowledge application and
motivation of the majority of students.
We also investigated whether the new gamification approach instill aspects of self-directed
learning in students from the viewpoint of our lecturers and students.
Our study has confirmed that majority of students have indicated this new approach have
increased their motivation in learning secure coding, which is a core trait for self-directed
learning. Students in the focus group study have also shared how Mimosa had encouraged
them to explore and collaborate more in a bid to complete the challenges and score points.
They have exhibited higher motivation, independent learning and applied critical thinking,
all in a bid to solve the challenges and complete against their peers. Mimosa has also helped
them apply their skills in tackling the wide variety of challenges that simulated real
scenarios. This is all part of the journey towards being a self-directed learner.
Lecturers have also observed more activity and discussion within the classroom as students
discuss and share resources on solving challenges. Motivation and independent learning has
increased among the student and students and staff are happy with the new tool. Mimosa has
also allowed the teaching staff to monitor the progress of students in solving the challenges,
understand their difficulties in certain topics and questions, and subsequently address them
in class by recapping the topic or spending more time in going through the solution of the
questions. Appropriate intervention can be carried out to help students in areas they find
difficulty in.
8 Conclusion
With the success of the first iteration project, we will further develop Mimosa with another
new group student of students. This group of students had just completed the secure coding
module and will be inputting new ideas to improve this system based on their experience.
We will be embarking on the next phase to fix the bugs, implement more challenges and also
incorporate tutorial videos and better hints to guide the weaker students in completing the
challenges. It is hoped that additional support will help encourage the weaker students and
provide them a slight push in their learning journey. Mimosa has also been introduced to the
Continuing Education and Training part-time course for working adults in the Diploma in
Infocomma and Digital Media (Cyber Security). We would be interested to see how the
adult learners respond to gamification in learning when they learn secure coding in the
course.
REFERENCES:
Barata, G., Gama, S., Jorge, J., Goncalves, D.(2013). Engaging Engineering Students with
Gamification. 5th International Conference on Games and Virtual Worlds for Serious Applications,
pp. 1–8.
Brewer, R., Anthony, L., Brown, Q., Irwin, G., Nias, J., Tate, B.(2013) Using Gamification to
motivate Children to Complete Empirical Studies in Lab Environments. 12th International
Conference on Interaction Design and Children, pp. 388–391
Broussard, S. C., & Garrison, M. E. B. (2004). The relationship between classroom motivation and
academic achievement in elementary school-aged children. Family and Consumer Sciences
Research Journal, 33(2), pp. 106–120.
Kumar, B., Khurana, P.(2012). Gamification in Education. Learn Computer Programming with Fun.
International Journal of Computers and Distributed Systems 2(1), pp. 46–53.
Kapp, K.M. (2013). Games, Gamification, and the Quest for Learner Engagement. Training and
Development 66(6), pp. 64–68.
Leong, H., Yee, C., Kee, C. (2019). Flipped Learning to Nurture Self-Directed Learners at
Singapore Polytechnic. 15th International CDIO Conference at Aarhus University
Nah, Fiona & Zeng, Qing & Telaprolu, Venkata & Ayyappa, Abhishek & Eschenbrenner,
Brenda.(2014). Gamification of Education: A Review of Literature, pp. 401-409.
Patton MQ. (1999). Enhancing the quality and credibility of qualitative analysis. Health Serv Res.
34(5 Pt 2), pp. 1189-1208.
Rim Razzouk and Valerie Shute (2012). What Is Design Thinking and Why Is It Important? Review
of Educational Research, Volume 82, issue 3, pp. 330-348
Todor, V., Piticǎ, D.(2013). The Gamification of the Study of Electronics in Dedicated E-learning
Platforms. 36th International Spring Seminar on Electronics Technology, pp. 428–431 (2013)