Webinar Series

The 5th Cyber Education Forum

LEARNING ANALYTICS MODEL DEVELOPMENT

USING MACHINE LEARNING
August 26, 2021

Sri Suning Kusumawardani (Universitas Gadjah Mada)

Irwan Endrayanto (UGM), Dewi Juliah Ratnaningsih (UT), Wihendro (Binus
University), Uwes Anis Chaeruman (UNJ), Ary Setijadi Prihatmanto (ITB),
Syukron Abu Ishaq Alfarozi (UGM), Christian Pradana (UGM)
Webinar Series
The 5th Cyber Education Forum
August 26, 2021

Presentation Outlines

Introduction & Motivation Conceptual Design

The Progress Results &

Type of Machine Learning
Conclusion
 ABBREVIATION

LA Learning Analytics

LMS Learning Management System

MOOCs Massive Open Online Courses

ML Machine Learning
OULAD Open University Learning Analytics Dataset
DT Decision Tree

SVM Support Vector Machine

MLP Multi-Layer Perceptron

LSTM Long Short-Term Memory

VLE Virtual Learning Environment
 INTRODUCTION

01 COVID-19 PANDEMIC 02 LMS 03 LMS ACTIVITY DATA

MOOCs has grown fast & It is only used for delivering the It has not been used as
transformed the learning learning resources and has not meaningful data as they
process. utilized data insights generated just focused on the learning
by LMS. resources and exams.

04 LMS BIG DATA PROBLEM 05 LA DASHBOARD OF LMS

How to process and utilize LMS Big It provides information on leacher/student

Data such as the utility of learning behavior, performance, prediction,
resources and learning activities into personalized learning, learning retention
a meaningful information. incremental, online learning evaluation,
and cost-effectiveness compared to
manual.
Motivation in Learning Analytics:
Track student performance and activities

Human Technology

Personalization Tracking & Control

Motivation Blended Mobility & flexibility
Engagement Systematics
Relevance & Contextual (Humanizing Self-directed learning
Discipline Technology) Reduced Costs
Integrity Global Reach
 The existing reports in Moodle-based LMS

Insight?
The existing reports in Moodle-based LMS
 Track student performance and activities
The existing Learning Analytics Learning Analytics is an interactive tool that helps us better track performance and
activity within the course. Learn which students have viewed pages and resources a

in Canvas-based LMS participated in assignments—and which students may need a little more encouragem

How do I send a message to all students based on specific course criteria in New Analytics?

How do I send a message to an individual student in New Analytics?

How do I view analytics for an individual student?

 RESEARCH MOTIVATION &
TIMELINE
Learning Analytics Model Development Using Machine
Learning with the following objectives
FIRST YEAR (2021) SECOND YEAR (2022) THIRD YEAR (2023)

• Evaluating and providing Machine • Developing and implementing • System testing, deployment, and
Learning recommendations for LA prototype on LMS UT operation of developed Learning
Learning Analytics needs. (TUTON) and UGM (eLOK). Analytics.
• Developing a prototype of an • Testing the Learning Analytics
automatic data retrieval system prototype using Machine
on the Moodle system for Learning.
Learning Analytics needs.
 MACHINE LEARNING

Machine Learning: field of study that

gives computers the ability to learn
without being explicitly programmed
(Arthur Samuel, 1959)

This research will develop a Learning

Analytics model by utilizing various
Machine Learning such as Supervised
Learning, Unsupervised Learning, and
Reinforcement Learning.
TYPES OF MACHINE LEARNING
IDENTIFICATION OF MODELING VARIABLES
OF LEARNING ANALYTICS
Subject
No. Research Indicators
Student Instructor
1 Participation in tutorials V -
2 Number of participation/frequency of activity V V
3 Number of greetings at the beginning of the LMS - V

4 Duration of study/teaching V V
5 Posting of additional/enrichment materials - V
6 Number of instructor posts in discussion forums - V
7 Posting of discussion answers V -
8 Number of student responses in discussion per V -
session
9 Number of tutor responses in discussion - V
10 Number of assignments submitted by students V -
11 Tutor feedback on student assignments - V
12 Punctuality in submitting assignments V -
13 Value of participation frequency in the LMS - V
14 Value of discussion/discussion activities - V
15 LMS assignment grades - V
Conceptual Design of Learning Analytics Using
Machine Learning
Classification & Visualization:
Hidden Data: Clustering Modelling Learning Analytics
• Activities Log
Dashboard
• Assesment Log
• Scoring Log Data Basic
• etc Preprocessing Data Analytics Student: ISP
• Extraction • Classification Teacher: SP, ITP, ICP
Supporting Data: • Cleansing • Clustering Institution: CP, TP, SP
• Participants • Integration
• Demographics • Transform
• Quesioner (LS, SRL,
etc)
Advanced
Prediction
Data Analytics
• Prediction Modelling

Description:
SP: Student's Performance ISP: Individual SP
TP: Teacher's Performance ITP: Individual TP Visualization:
CP: Course’s Performance ICP: Individual CP Prediction Dashboard
 Scope of Research

LMS
01 Learning Management System
UT (TUTON) and UGM (eLOK)

SUBJECT
45 Courses 02
Academic Year of 2018 -
2020 PLATFORM
03 Utilizing at least
Moodle 3.9
 FIRST YEAR RESEARCH PHASE (2021)

Providing initial
Identifying reports recommendations for
log on Moodle Learning Analytics
models

01 02 03 04 05
Identifying various
Machine Learning Evaluating various Developed a prototype of
methods for Learning Machine Learning automatic data retrieval system
Analytics methods on Moodle for Learning
Analytics
 SECOND YEAR RESEARCH PHASE (2022)

Analyzing data Developing Learning

relation models for Analytics prototypes on
various learning TUTON (UT) and eLOK
analytics needs (UGM) LMS

01 02 03 04 05
Analyzing learning needs
and tools used Analyzing Learning Continuing testing of the LA
Analytics views and prototype using LMS
visualizations
 THIRD YEAR RESEARCH PHASE (2023)

Developing user Performing Learning

evaluation Analytics system testing
instruments and developed on Moodle
market analysis LMS

01 02 03 04 05
Identifying components
of user evaluation Conduct user evaluation System diffusion and
and market analysis dissemination
A Reference Model for Learning Analytics
M.A. Chatti, A.L. Dyckhoff, U. Schroeder, and H. Thüs (2012)
A Review: Usage of Student Log Data for Several Learning Analytics Problems (1)

studentAssessment assesments

*id_student (FK) *code_module (FK)

studentRegistration *id_assentment (FK) *code_presentation (FK)
+ date_submitted *id_assessment (PK)
*code_module (FK) + is_banked + assessment_type
*code_presentation (FK) + score + date
*id_student(FK)
+ date_registration
+ date_unregistration

courses

*code_module (FK)
studentInfo *code_presentation (FK)
+ length
*code_module (FK)
*code_presentation (FK)
*id_student (PK)
+ gender
+ region
vle
+ highest_education studentVle
+ imd_band
*id_site (PK)
+ age_band *id_site (FK) *code_module (FK)
+ num_of_previous_attempt *id_student (FK) *code_presentation (FK)
+ studied_credits *code_module (FK) + activity_type
+ disability *code_presentation (FK) + week_from
+ final_result + date + week_to
+ sumclick

Entity relationship diagram: OULAD [1]. *) keys, +) atribut.

A Review: Usage of Student Log Data for Several Learning Analytics Problems (2)

Drop-out/No
Classification
Engagement
Problems Final Result
Regression ( withdrawn, pass,
fail, distinction)
Score of Final Result
Demographics
Aggregation of click
VLE click stream
Learning Analysis Sequential
Both
OULAD Feature
or subset
LSTM/GRU
CNN+LSTM Sequential

TSF
MLP
SVM
Models or Methods
DT
PCA Table

RF
LR
GMM
Others

OULAD Hierarchical
 Learning Performance Prediction with Imbalanced Virtual Learning Environment
Students’ Interaction Data

Assignment x 1 Y Y Y Y

File x2 Pass Pass

Fail Pass
Fail
Fail
Forum x3
Learning Performance
Fail Pass

Label Distribution Undersampling Oversampling Hybrid

Quiz x4 (Imbalanced) Resampling Methods
Feature
Construction Course x5
Machine Learning Model
Label x6 NB, LR, RF

Page x7
Learning Performance Prediction
URL x8 (Pass/Fail)

Authors: Hsing-Chung Chen, Eko Prasetyo, Prayitno, Sri Suning Kusumawardani, Shian-Shyong Tseng*,
Tzu-Liang Kung*, and Kuei-Yuan Wang
 Learning Analytics
Performance Dashboard
User Target
• Teacher
•Student Performance (Asessment, Activities,
Student’s Response, Learning Duration)
•Course Performance
•Individual Teacher Performance (Activities,
Teacher’s Response, Teaching Duration)

• Student
•Individual Student Performance (Asessment,
Activities, Student’s Response, Learning Duration)
Dashboard Parameter

• Final, Mid, Quiz Exam: obtained from quiz, mid, final exam
score
• Activities: Get data from Moodle Log
Answering
Questioning
Only Participate
• Response
(Teacher/Student) engagement time in LMS
• Duration
Duration of Teacher/Student using LMS
Example: Student Comparison Chart
Webinar Series
The 5th Cyber Education Forum
A Large number of e-learning environment
data are available on several learning
platforms, especially the logs that track the
student activity and performance. These
data can be used for further processing with
learning analytics methods.
Student performance and student at-risk
Conclusion can be mitigated using learning analytics.
Thus, we can increase student performance
and decrease the student drop-out/failure
rate.
The learning dashboard monitors the
student activity and performance insights
based on the data logs provided by the
LMSs, either for students or lecturers.
Using machine learning as a complement of
learning analytics, we can further predict
student performance using a particular
predictive model or analyze the student
learning factors identifying the characteristic
of student performance.
Webinar Series
The 5th Cyber Education Forum August 26, 2021

Acknowledgement
Very great appreciation to:
Prof. Dr. Paulina Pannen. M.Ls.
Universitas Gadjah Mada
& ICE Institute Team

Universitas Terbuka ICE Researcher

& All Partners
THANK YOU!