2427-Article Text-12820-4-10-20221223
2427-Article Text-12820-4-10-20221223
2427-Article Text-12820-4-10-20221223
Copyright c 2022 MATRIK: Jurnal Manajemen, Teknik Informatika, dan Rekayasa Komputer.
This is an open access article under the CC BY-SA license.
Corresponding Author:
Siti Ummi Masruroh,
Department of Informatics,
Syarif Hidayatullah State Islamic University, Jakarta, Indonesia,
Email: [email protected]
How to Cite: S. Masruroh, V. Arifin, V. Handayani, L. Wardhani, and H. Suseno, User Interface and Experience Gamification-Based
E-Learning with Design Science Research Methodology, MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer,
vol. 22, no. 1, pp. 167-178, Nov. 2022.
This is an open access article under the CC BY-NC-SA license (https://creativecommons.org/licenses/by-nc-sa/4.0/)
1. INTRODUCTION
The expanding popularity of internet-based information technology makes educational institutions are investing in cutting-edge
teaching and learning technologies. Electronic learning, sometimes known as e-learning, is an educational internet-based, electronic-
based information technology program. It can be defined as the use of internet technology to deliver knowledge and resources to
individual learners [1]. E-learning has advantages of huge learning resources, enthusiastic learners, friendly interaction and flexible
spacetime [2]. Online e-learning provides students a flexible and efficient pathway to access course materials, communicate with the
instructor, interact with classmates, participate in different online activities and perform online assessments [3, 4]. The flexibility
that e-learning gives goes beyond the learners; it gives flexibility also for the instructors. In addition, educational institutions are
implementing E-learning technologies to improve the communication among learners and instructors for better knowledge exchange
as well as to strengthen the learning community to accomplish personal objectives [5].
Several problems arise in the learning process in schools, such as teachers only providing subject matter and rarely motivating
students in the classroom. Students feel unenthusiastic about learning and tend to be passive in receiving explanations from the
teacher. The increasing transfer of information makes it difficult to deliver education to individuals because rapid developments
in the field of technological innovation require people to adapt quickly [6]. The application of the concept of gamification in e-
learning is assessed as a technique that can help students understand the material provided so that students can be more motivated to
participate actively in the learning process. Gamification is thought to be able to help students increase their enthusiasm for learning.
Gamification helps in the development of critical thinking and multitasking skills in individuals [7].
The existence of e-learning is inextricably linked to the design of the e-learning service. User Interface (UI) and User Experi-
ence (UX) are becoming increasingly significant and altering user expectations and needs as a result of technological advancement
and new interaction patterns. Students’ attitudes toward adopting e-learning can be influenced by bad UI and UX in the context of
e-learning. This can lead to a misunderstanding or misuse of e-learning, affecting academic institutions’ acceptability, satisfaction,
and efficiency. Failure to adhere to high-quality UI UX components can lead to dissatisfaction, misunderstanding, or incorrect usage,
which can result in criticism and low acceptability of the e-learning platform [8].
Various educational institutions have begun to employ e-learning as a means of adapting to technology changes while also
improving the quality and effectiveness of education. Observations during an e-learning program for Islamic elementary school
teachers in DKI Jakarta Province in January 2020 corroborated this. The program seeks to increase teacher technological skills,
particularly in the application of e-learning in the classroom. Quizizz, a gamification-based learning tool (gamified learning platform),
is one of the things being taught. However, many Islamic elementary school teachers are still perplexed while utilizing the Quizizz
program, based on observations made during training. This is due to the application’s look and different functions, which might be
confusing to certain teachers who aren’t used to using it. That condition is aligning with [9] that states ”Different learners may have
different learning needs, and different users may have different requirements on how the program content should be displayed”.
Research by [10], proposes an interactive learning story framework and design a UI that follows the concept of flat design
and responsive mobile design, as well as to conduct testing and evaluation with interactive prototypes. However, it would be better
if teaching materials such as content, infographics, and animations need to be correlated with the learning experience and student
learning environment.
The study by [11], develops a web-based e-learning system Code Mania (CoMa) and applies the gamification concept to
increase student interactivity, motivation and student engagement using e-learning. The e-learning also has an automatic evaluation
feature to evaluate student work. The result from the testing shows that the e-learning system could run well as specified in the system
requirement. The usability of the system is not yet known so it is suggested that further study should examine the system usability
for a better user experience in using e-learning.
Research by [12], designed and built a training application for elementary science questions using gamification and the
Mersenne twister algorithm. But the design has a drawback, namely the need for the addition of other gamification elements so
that it can be more interesting and motivate students to use it.
Research by [13], shows that user-centered gamification is able to increase student interaction, engagement, and satisfaction
in e-learning systems. However, there are a few obstacles/ineffectiveness when analyzing the use of the system, including cheating
when one of the student’s posts an assignment through a discussion forum, so there is a possibility of partial data in the analysis
results.
Research by [14], develops e-learning applications by applying the gamification method to the implementation of its features
or content. No errors were found in the e-learning application by testing using Whitebox testing and BlackBox testing, but there is
still a lack of application of gamification elements, less in-depth calculations on tests using the User Engagement Scale (UES), as
well as test objects to be increased larger or more significant.
This article will review some of the UI and UX literature to determine the requirements and criteria for creating an e-learning
gamification system that meets the needs and experiences of students and teachers, as well as learning standards and user environments
in Islamic elementary schools in Jakarta. The evaluation is carried out using the User Experience Questionnaire (UEQ) which will
show that the e-learning design produced has been effective and efficient for users.
2. RESEARCH METHOD
There are five methods for solving the problems that have been mentioned above. The first process is Problem Identification,
which begins with observing the surrounding environment and then continues with a systematic literature review. Data Collection
with observation, interview and questionnaire are the methods use for collecting research data. The next step is Data Analysis with
Design Science Research Methodology (DSRM), which will producing a prototype as a design solution. The prototype will undergo
a design evaluation process using the User Experience Questionnaire (UEQ) and Usability Evaluation based on the simulation results
with the Cognitive Walkthrough obtained from the Demonstration stage.
The UEQ questionnaire which has been compiled in the form of a Google Form is then distributed online to respondents
with the criteria described previously. Then obtained as many as 164 respondents filled out the questionnaire. This amount is
sufficient for the author’s target, which refers to the research suggestion by Roscoe in [15] that the appropriate sample size in
the study is between 30 to 500.
4. Interview At this stage, the authors conducted semi-structured interviews which include several things, namely the identity of
the respondent, experience of using ICT and details of the respondent’s experience related to e-learning (strengths, goals, tasks
& needs, pain points, and user expectations) both gamification-based e-learning or not. The interview participants amounted
to 6 people consisting of 3 teachers and 3 students.
Based on the results of the UEQ questionnaire analysis related to the respondent’s experience in using e-learning, of the six
aspects of the questionnaire (Attractiveness, Clarity, Efficiency, Accuracy, Stimulation, and Novelty) the lowest impression value
is in the Novelty aspect. The novelty aspect includes components: creative / boring, inventive / conventional, ordinary / advanced,
conservative / innovative. This shows that the e-learning applications that respondents use do not yet have innovative and creative
product designs, do not attract the attention of users and seem monotonous.
Based on the information and analysis results from the previous stage, the authors conclude several criteria for e-learning
applications that become the needs, expectations, and solutions to problems faced by users. Departing from the data that has
been obtained in the previous stage, an analysis is carried out which then produces the user persona of the respondents who are
students and teachers.
The features developed are obtained from the needs and tasks of the user. In addition, the feature also refers to research by
[13] regarding user behavior references in e-learning systems based on three suggested categories to facilitate users when using
gamification-based e-learning systems. The three categories are Interaction, Communication, and Performance.
After analyzing the three categories and user needs, the authors determine several features or functions for the e-learning design
to be developed, namely Interaction which includes access to the material (either in the form of documents, presentations, or
videos), Communication in the form of class discussion forums and Performance in the form of practice questions, assignments
and exams. In addition, the author also adds several features obtained from user needs and tasks, namely grade reports, profiles,
classes, absences, and notifications.
2. Concrete Design
Concrete Design is the process of designing the interface concept of the application design. As the identity of the e-learning
application itself, the author adds a logo made based on the color and purpose of the application. The illustration of the book
was chosen as the meaning of ”learning”. There is also a molecular illustration which is defined as a group or a group of atoms
bonded to each other. In the logo, the molecules harness the science used together. This is expected to be in line with the
purpose of the application, which is to be a medium for learning various lessons & knowledge so that it becomes knowledge
for students.
There are two types of fonts, namely serif and sans serif. The sans serif typeface is used in this e-learning design, namely DM
Sans. Sans serif typefaces are characterized by fonts that do not have fins/legs DM Sans was chosen because of its simple,
clean, easy-to-read form, and creates a friendly, modern, and fun impression. This font is used in its entirety for both the
headline and body text of the application.
There are several color groups used in designing the e-learning design in this study. The green color (Dark Cyan to be precise)
with the color code HEX #008f 87 was chosen as the primary color for the e-learning application. while the color with the
HEX code #68D212 and the HEX code #F F 7171 is used as a color marker for the success or failure of the system. The
other color details can be seen in the following image.
3. Gamification Design
Based on the literature study that the author conducted on 12 literature that applies game elements to the e-learning system,
it can be concluded that the most widely used gamification elements are Level, Points, Leaderboard and Badges. Some
literature also applies Feedback, Challenge, Progression, Customization and Social. Then based on benchmark results from
two best practice gamification applications, namely Quizizz and Kahoot, it was found that both of them used the elements of
Leaderboard, Points, Customization, Progress Tracking & Feedback.
Based on these analyzes, several game elements were found that will be applied to the e-learning design of MIN 4 Jakarta,
namely, Points,Level, Mission, Badge, Progress Bar, Avatar, Leaderboard, Feedback
2. Mock Up After designing the wireframe, the next step is to design a gamification-based e-learning design mockup. The design
of the mockup is done by providing colors, icons and images that have been initiated in the previous stage. The placement of
elements in the mockup refers to those that were previously created in the wireframe. The following is a mockup of e-learning
for teacher and student accounts.
3.4. Demonstration
At this stage, the authors distributed the e-learning design prototype that had been made in the previous stage to 164 teachers
and students (questionnaire respondents at the data collection stage) to try to interact directly with the e-learning design. After that,
the respondents will be asked their willingness to give their assessment of the e-learning design by filling out the UEQ questionnaire.
Furthermore, the authors carry out a Usability Evaluation using the Cognitive Walkthrough method. This process involved 3
teachers and 3 students who were respondents at the interview stage.
3.5. Evaluation
The prototype of the e-learning design was given to 164 respondents/users who previously also became respondents at the data
collection stage using a questionnaire. Thus, the respondents can interact and try to use the prototype directly so that they can provide
an assessment and experience by answering the UEQ questionnaire given. After evaluating with the User Experience Questionnaire
(UEQ) on the gamification-based e-learning design that was designed, the following results were obtained.
Based on the pictures and graphs above, it can be seen that all aspects resulted in a positive evaluation value (> 0.8). The
aspect with the greatest value is Attractiveness (2,149) followed by Clarity (2,137), Efficiency (2,052), Stimulation (2,044), Accuracy
(1,986) and Novelty (1,739). When compared with the results of the evaluation of the e-learning that the respondents used previously,
there were significant differences in all aspects, especially the novelty aspect. This can be seen in the comparison chart below.
Figure 10. Result Comparison between Existing E-Learning & Proposed Gamified E-Learning
Based on the comparison above, it can be concluded that the gamification-based e-learning design designed in this study
has resulted in positive evaluation values for all aspects tested in the UEQ. In addition, e-learning gamification resulted in a positive
evaluation value (1.74) for the Novelty aspect which previously had a normal evaluation value (< 0.8). Therefore, it can be concluded
that the designed gamified e-learning has been considered innovative and attractive by users.
Based on the results of the simulation using the Cognitive Walkthrough method at the Demonstration stage, an analysis is
carried out based on the simulation results by measuring the simulation results using Performance Metrics. As for this study, the
author uses one type of Performance Metric to measure the effectiveness parameter, namely Task Success. Based on the results of
the simulation conducted by 6 people including 3 teachers with 10 Task Scenarios (ST) and 3 students with 8 Task Scenarios (ST),
the following data were obtained.
All teachers can complete 6 task scenarios smoothly and without any problems. However, there were some teachers who
experienced minor problems when working on several task scenarios (ST), including ST 3, ST 6, ST 7 and ST 8. As for ST 6, as
many as 2 out of 3 respondents experienced minor problems when doing these assignments. The problems encountered include the
teacher looking a little confused when faced with a series of steps that must be passed, looking for buttons and inputting student
scores. Based on the results of the simulation conducted by 6 people including 3 teachers with 10 Task Scenarios (ST) and 3 students
with 8 Task Scenarios (ST), the following data were obtained.
Refers to [16] that there is no need to worry about the results of the evaluation with a success rate of 1 (no problems) and 2
(there are minor problems), what needs to be considered is the evaluation results with success rates of 3 (there are major problems)
and 4 (failed/give up). Because of this level of success, it indicates the need for improvement and development of a better design.
Therefore, it can be concluded that teachers and students can interact with e-learning easily and understand the flow of use of the
e-learning design in this study.
Based on the comparison results that have been obtained previously, it shows that the gamification-based e-learning design
produced is innovative and attractive to users, namely students. When compared with research [12], the evaluation results prove that
the design can motivate students to use it.
3.6. Communication
At this stage, the authors document the problems, solutions and test results of the solution in the form of a gamification-based
e-learning design in the form of journals to be communicated to related audiences and researchers.
4. CONCLUSION
This study succeeded in designing a gamification-based e-learning application in the form of a High-Fidelity Prototype using
the Design Science Research Methodology method which consisted of 6 stages. Based on the results of the evaluation of the e-learning
design at the Evaluation stage, the User Experience Questionnaire (UEQ) obtained positive evaluation results for both students and
teachers which included Attractiveness, Clarity, Efficiency, Accuracy, Stimulation and Novelty. The aspect with the greatest value is
Attractiveness (2,149) followed by Clarity (2,137), Efficiency (2,052), Stimulation (2,044), Accuracy (1,986), and Novelty (1,739).
In addition, e-learning gamification resulted in a positive evaluation value (1.74) for the Novelty aspect which previously had a normal
evaluation value (¡ 0.8). Therefore, it can be concluded that the designed gamified e-learning has been considered innovative and
attractive by users. The design can meet the needs of students and teachers in carrying out learning in the user environment in Islamic
elementary schools.
In addition, based on the results of an evaluation with one of the performance metrics, namely task success to measure effec-
tiveness parameters, the results obtained for the success rate for students and teachers are levels 1 and 2, with the meaning 1 = users
do not get problems in working on tasks, and 2 = users get a minor problem. The minor problem experienced by students is the
lack of visibility of one of the buttons. While the minor problems experienced by the teacher were the lack of some information on
the application and the lack of visibility of one of the buttons and tabs. Therefore, based on the overall evaluation results, it can be
concluded that the e-learning design is considered attractive and users can interact with e-learning effectively and easily.
However, this research still has some shortcomings and weaknesses. Therefore, it is necessary to pay attention to several
suggestions that can be used as a reference for better development in the future, namely:
1. Improving the design based on several recommendations for improvement that have been described.
2. It should be considered for design evaluation using other performance metrics such as task on time and error.
3. It would be better if the design is also developed for mobile.
5. ACKNOWLEDGEMENTS
This research was funded by Puslitpen UIN Syarif Hidayatullah Jakarta.
6. DECLARATIONS
AUTHOR CONTIBUTION
All authors conceived and designed the study. Study concept and design Viva Arifin, Velia Handayani, and Luh Kesuma Wardhani.
Experiments, analyzed the data Velia Handayani, Hendra Bayu Suseno, Siti Ummi Masruroh.
FUNDING STATEMENT
This research was funded by Puslitpen UIN Syarif Hidayatullah Jakarta.
COMPETING INTEREST
The authors declare that there are no conflicts of interest regarding the publication of this paper.
REFERENCES
[1] M. M. The and T. Usagawa, “Effectiveness of E-Learning Experience Through Online Quizzes: A Case Study of Myanmar
Students,” International Journal of Emerging Technologies in Learning, vol. 13, no. 12, pp. 157–176, 2018.
[2] Y. Zou, L. Shen, and S. Dadparvar, “The Influence of E-Learning Behavior on Students’ Learning Performance of Disaster
Emergency Knowledge,” International Journal of Emerging Technologies in Learning (iJET), vol. 17, no. 01, pp. 49–59, jan
2022. [Online]. Available: https://online-journals.org/index.php/i-jet/article/view/28475
[3] A. Thoo, A. Thoo, S. Hang, Y. Lee, and L. Tan, “Students’ Satisfaction Using E-Learning as a Supplementary Tool,” Interna-
tional Journal of Emerging Technologies in Learning (iJET), vol. 16, no. 15, pp. 16–30, 2021.
[4] A. Apriani, H. Zakiyudin, and K. Marzuki, “Penerapan Algoritma Cosine Similarity dan Pembobotan TF-IDF System Peneri-
maan Mahasiswa Baru pada Kampus Swasta,” Jurnal Bumigora Information Technology (BITe), vol. 3, no. 1, pp. 19–27, 2021.
[5] A. Y. Alqahtani and A. A. Rajkhan, “E-Learning Critical Success Factors During The Covid-19 Pandemic: A Comprehensive
Analysis of E-Learning Managerial perspectives,” Education Sciences, vol. 10, no. 9, pp. 1–16, 2020.
[6] H. A. Yamani, “A Conceptual Framework for Integrating Gamification in E-Learning Systems Based on Instructional Design
Model,” International Journal of Emerging Technologies in Learning (iJET), vol. 16, no. 04, pp. 14–33, feb 2021. [Online].
Available: https://online-journals.org/index.php/i-jet/article/view/15693
[7] D. Orhan Göksün and G. Gürsoy, “Comparing Success and Engagement in Gamified Learning Experiences Via Kahoot and
Quizizz,” in Computers and Education, vol. 135, no. January. Elsevier, 2019, pp. 15–29.
[8] W. T. Nakamura, L. C. Marques, L. Rivero, E. H. T. De Oliveira, and T. Conte, “Are Scale-Based Techniques Enough for
Learners to Convey Their UX when Using a Learning Management System?” Revista Brasileira de Informática na Educação,
vol. 27, no. 01, pp. 104–131, jan 2019. [Online]. Available: http://www.br-ie.org/pub/index.php/rbie/article/view/7788
[9] Y. M. Lim, A. Ayesh, and K. N. Chee, “Socio-Demographic Differences in The Perceptions of Learning Management System
(Lms) Design,” International Journal of Software Engineering & Applications, vol. 4, no. 5, pp. 15–35, 2013.
[10] T. Phunkaew, C. Phandan, and C. Wongwatkit, “Design and Evaluation of Interactive Learning Story and User Interface Pro-
totyping for Mobile Responsive Learning Application,” in Proceedings of 2019 4th International Conference on Information
Technology: Encompassing Intelligent Technology and Innovation Towards the New Era of Human Life, InCIT 2019. IEEE,
2019, pp. 132–137.
[11] F. A. Bachtiar, F. Pradana, B. Priyambadha, and D. I. Bastari, “CoMa: Development of Gamification-Based E-Learning,” in
2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE). IEEE, jul 2018, pp.
1–6. [Online]. Available: https://ieeexplore.ieee.org/document/8534875/
[12] H. Naufal Irfan and S. Hansun, “Pembangunan Aplikasi Latihan Soal IPA SD dengan Gamifikasi dan Mersenne Twister,” Jurnal
Teknologi Informasi dan Ilmu Komputer (JTIIK), vol. 7, no. 1, pp. 87–98, 2020.
[13] A. C. T. Klock, I. Gasparini, and M. S. Pimenta, “User-Centered Gamification for E-Learning Systems: A Quantitative and
Qualitative Analysis of its Application,” Interacting with Computers, vol. 31, no. 5, pp. 425–445, 2020.
[14] I. Suprianto, F. Pradana, and F. Abdurrachman Bachtiar, “Pengembangan Aplikasi E-Learning dengan Menerapkan Metode
Gamification,” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer, vol. 3, no. 2, pp. 1716–1724, 2019.
[15] Sugiyono, Metode Penelitian Kuantitatif, Kualitatif dan R & D , 19th ed. Bandung: Alfabeta, 2014, 2014.
[16] T. Tullis and B. Albert, Measuring The User Experience. Elsevier, jul 2013. [Online]. Available: https:
//linkinghub.elsevier.com/retrieve/pii/C20110000169