See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/338104947

LEARNING ANALYTICS SOFTWARE IMPLEMENTATION FOR THE MOODLE

LEARNING MANAGEMENT SYSTEM

Conference Paper · November 2019

DOI: 10.21125/iceri.2019.1655

CITATIONS READS

5 1,253

5 authors, including:

Thomas Dondorf Christiane Pyka

RWTH Aachen University RWTH Aachen University
12 PUBLICATIONS 22 CITATIONS 7 PUBLICATIONS 34 CITATIONS

SEE PROFILE SEE PROFILE

Richard Gramlich
RWTH Aachen University
2 PUBLICATIONS 8 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

WEF Nexus Lab View project

University Chairs on Innovation “UNCHAIN” View project

All content following this page was uploaded by Thomas Dondorf on 07 January 2020.

The user has requested enhancement of the downloaded file.

 LEARNING ANALYTICS SOFTWARE IMPLEMENTATION FOR THE
MOODLE LEARNING MANAGEMENT SYSTEM
T. Dondorf, C. Pyka, R. Gramlich, H. Sewilam, H. Nacken
RWTH Aachen University (GERMANY)

Abstract
Learning Analytics provides methods for measuring, collecting and analyzing data about learners and
their contexts, for the purpose of understanding and optimizing the learning process and its
environments. In recent years, most research has been conducted regarding methodologies or
technological challenges of Learning Analytics. Little research has been done regarding software
engineering and the software implementation of Learning Analytics for the use in Learning
Management Systems. This makes it currently difficult or even impossible for researchers and
teachers to use current research results without writing software on their own.
In this paper, we first present existing Learning Analytics tools for the Learning Management System
Moodle. We discuss different implemented methodologies and how each software integrates into the
learning platform. After that, we present the possible approaches to develop Learning Analytics
software for Moodle. Based on the presented software, we explain the advantages and disadvantages
of the different development approaches. While some approaches allow a deep integration into the
Learning Management System but are focused on only the Moodle system, other approaches can
track the user over various different systems.
Lastly, we describe our own Learning Analytics software implementation for Moodle. It is developed as
plugin for the platform allowing it to be easily installable and well integrated into Moodle from a user
perspective. Furthermore, its functionality can be extended by installing additional plugins. This makes
it possible to extend the functionality by writing minimal and focused plugins. In the paper, we explain
how the software is integrated into Moodle as well as how plugins can be developed to extend its
functionality. We discuss our software design decisions and explain their benefits.
Keywords: learning analytics, moodle, e-learning, virtual learning environment

1 INTRODUCTION
Learning analytics focuses on the analysis of performance [1, 2], learning resources [3] and activities
inside and outside of virtual learning environments [4]. The results allow an improvement of quality of
the teaching and learning processes. Today, most learning management systems (LMS) include basic
learning analytics functionality. In this paper, we present existing Learning Analytics software and
present the implementation decision and the current development state of our own Learning Analytics
solution.
One of the most used learning management systems is Moodle. Moodle is free software distributed as
Open Source. It is the second most used LMS in the higher education market in the US after
Blackboard [5] and widely used in the higher education sector in Germany [6]. This paper first
discusses general Learning Analytics implementations before focusing on the Moodle LMS. We
present what kind of functionality Moodle has already built-in by default and afterwards discuss
possibilities on how to extend this.
As no Learning Analytics solution for the Moodle LMS fully supports our demands, we present our own
requirements for a Learning Analytics software grouped by functional and technical requirements. The
approach we present can be installed as a Moodle plugin and can be easily extended with more
reports.
The remainder of this paper is structured as follow. In Section 2 we explain related work by presenting
existing Learning Analytics software. Section 3 lists our requirements before the next section gives
examples of our implementation. Section 5 briefly mentions our current evaluation. After that we
present the future work and conclude the paper.
2 RELATED WORK
In this section, we first present different existing Learning Analytics systems. First, we go over
Learning Analytics systems in general before we take a closer look at the Moodle LMS. Regarding
Moodle, we first present built-in functionality, which is integrated into every Moodle installation.
Afterwards, we look at multiple existing plugins in the Moodle plugin store and explain how they
integrate into the platform and what functionality they offer.

2.1 Learning Analytics systems for other LMS

Multiple universities already developed custom Learning Analytics systems to support the teaching
and learning process. The following list makes no claim to completeness, but is intended to give an
overview of prominent examples.

2.1.1 Course Signals at Purdue University

The software Course Signals “was developed to allow instructors the opportunity to employ the power
of learner analytics to provide real-time feedback to a student” [1]. Based on student data like test
scores and also demographic characteristics like age, residency, Course Signals calculates the
performance for the courses of each student and displays a red, yellow or green traffic light to reflect
the personal learning progress.
The system was used for thousands of students across 100 courses. In general, it received a positive
feedback from instructors, the faculty and the students. The students tended to be “more proactive”
and reported the system as a “helpful and important tool” regarding the overall academic success. [1]

2.1.2 OU Analyse
OU Analyse [7], developed by the The Open University, offers a more general approach and presents
the lecturer with a dashboard view that combines different statistics into a single view. The main task
of the software is to predict “students at risk”, which are students that are likely to fail the course or
likely to not submit assignments to improve the retention of students. For its predictions, the system
combines the results of multiple machine learning algorithms: k Nearest Neighbors (k-NN),
Classification and Regression Tree (CART) and Bayes network [8].
The used data is based on the students’ activities in the virtual learning environment as well as their
demographic data. Evaluation showed that teachers endorsed the use of predictive data. In addition,
the direct comparison showed that the use of OU Analyse led to a higher module completion rate in
courses. [9]

2.1.3 Student Activity Meter

The Student Activity Meter of KU Leuven implements a visualization of user actions with the goal of
increasing “awareness for learners and teachers and to support self-reflection” [10]. In contrast to the
other two applications, the Student Activity gives a descriptive view of the past instead of predicting
learner’s outcome. Lecturers can select different views showing how much time was spend inside the
virtual learning environment. It offers a view that visualizes the accumulated time giving an overview
when users learned in addition to a view that show the spent time per learning resource.
Based on interviews with the users, the authors conclude that “there is a large potential for
visualizations of tracking information, not only to assist teachers and students, but also in other
domains like personal Informatics and Quantified Self” [10].
The presented approaches show the different methodologies used, some using machine learning
approaches to predict future actions and others focussing on descriptive analytics and self-reflection.

2.2 Built-in Learning Analytics functionality in Moodle

By default, Moodle has some limited Analytics functionality built-in.

2.2.1 Reports
Moodle supports reports [11] showing statistics based on the internal logging system. From our
experience, the most important ones are:
  Activity Report
 Participation Report
 Logs
The Activity Report shows the number of views for each activity of the course. The data is presented
as table showing how many people accessed each resource and how many clicks it received in total.
The report can be used by lecturers to optimize the course for used course materials and remove
potentially unused resources.
The Participation Report displays for a selected activity who participated in it and how many times.
The report can be used to identify under and well performing students regarding a single task.
The “Logs” report shows for an individual user what actions the user performed inside the learning
management system. This provides a detailed overview over when and how a user has learned.

2.2.2 Moodle Analytics

Previously called Project Inspire, but later renamed to Analytics, Moodle offers a built-in predictive
Analytics system. The system supports Machine Learning based on logistic regression and neural
networks (using Tensorflow) [12] and comes with three models to be used. Unfortunately, two of the
three models (“No Teaching”, “Upcoming activities due”) focus on the administration instead of the
learning and teaching process.
The report “Students at risk of dropping out” is similar to the feature provided by OU Analyse. It
predicts which students are likely to drop out of the course during the last quarter of the course. The
model has to be trained with data from the Moodle system itself and does not come preconfigured.
Our chair tried using the tool with our internal Moodle system, but we were unable to get results that
were better than chance when predicting for individual users whether they will drop out or not.

2.3 Learning Analytics Plugins for Moodle

In total, there are 1,615 plugins in the Moodle store [13]. Of these, 40 plugins are in the reports
category and provide custom reports for Moodle. Note, that most of the reports are targeted at
administrators of the platform. In the following, we focus on plugins for Analytics for teaching and
learning processes.

2.3.1 Reports plugins

The most interesting plugins are the following ones:
 Ad-hoc database queries
 Overview statistics
 Courses Usage Statistics
The plugin “Ad-hoc database queries” offers to create custom SQL queries for the database to
generate tabular data to be displayed or downloaded. This allows complete flexibility regarding what
kind of data can be displayed as long as it is available in the database. The plugin requires some
technical knowledge as the user has to write queries in the SQL language, which most users are not
able to do.
Overview statistics offers primarily side wide statistics regarding users and courses. The plugin makes
it possible to easily generate reports showing the number of logged in users per day as well as
statistics including the language or country of users. In general, this plugin is more suitable to get a
“full view” of the Moodle instance and cannot be used by individual lecturers.
The “Course Usage Statistics” plugin also targets side wide statistics. It classifies courses as one of
three categories (forum, repository or activity) by analyzing the activities used inside the course.

2.3.2 Other statistics

While most Analytics plugins are in the “Reports” category, there are also other plugins. The most
important ones are the following ones:
  Configurable Reports
 Completion Progress
 Level up!
 Course dedication
The plugin “Configurable Reports” is similar to the plugin “Ad-hoc database queries” mention above
but instead of displaying the data as a report on its own, the result of the database query is displayed
as a block inside the course.
The “Completion Progress” is a time-management tool for students. For each activity it displays the
progress as green, yellow or red colored visualization.
“Level up!” is different to the other plugins as it serves as a gamification and stimulation system for the
students. Every action of students inside Moodle makes a student earn points which make him level
up when he earned enough. While this is primarily used to motivate students, it can also serve as an
Analytics system when comparing the levels of students.
Lastly, the “Course dedication” plugin works similar to the previous one. It calculates the estimated
dedication time by looking at clicks and session duration to estimate the dedication.

2.4 External Data Stores for Moodle

The previous subsections focused on plugins and functionality that is directly integrated into Moodle.
Another possibility for Analytics systems is, to store the data outside of the LMS.

2.4.1 Web Analytics

There are many Web Analytics tools like Google Analytics or Piwik that are used to track the behavior
of users on “standard websites”. These tools can also be integrated into Moodle by the use of plugins.
Although, this allows to use standard tracking tools, these tools are not suited very well for Learning
Analytics purposes as they do not differentiate between different activities and learning resources.
In general, these tools can be used for general information about a Moodle instance like traffic
management or user retention. To improve the teaching and learning process, these tools do not fit.

2.4.2 Learning Record Stores

Alternatives to classic Web Analytics services are offered by external services, often labeled Learning
Record Stores. Learning Record Stores (LRS) store the logging data in an external database. The
data is often transferred via the xAPI [14] from Moodle to the external system. There are also hybrid
approaches that transfer only some data to the external system and also provided calculations inside
the LMS. Most of these external data stores are commercial products, but there are also a few Open
Source products that directly integrate into the Moodle LMS like IntelliBoard [15].

3 REQUIREMENTS
After reviewing existing Learning Analytics software and plugins, we defined the requirements for a
Learning Analytics system. These are divided into functional requirements, mainly proposed by
lecturers and teachers, and technical requirements.

3.1 Functional Requirements

The functional requirements were identified by executing qualitative interviews with lecturers. They
were asked what kind of functionality they think is currently missing with regards to Analytics in
Moodle and what kind of functionality they would like Moodle to have.
Although our chair is using Moodle for multiple years already, multiple lecturers named basic
functionality which is already present in Moodle. In practice, the report to show the amount of times a
file has been accessed was named multiple times. After showing to the lecturers that this functionality
is already present, one lecturer noted that it seems to be hidden as too many clicks are needed to
access it. In fact, when the user is on the main page of his course, two clicks are needed to access a
page with “More” information about the course. There the report is one of 24 links listed. We noted that
for a future Analytics solution important graphs need to be easily accessible.
Another point, mentioned multiple times was that lecturers are interested in correlations between the
(exam) grades and the behavior of users. In particular, the question “when did the students start to
learn?” and “did the students learn throughout the semester?” were named multiple times with regards
to their exam results.
Similarly, the lecturers were interested to see the changes of access patterns over the course of the
weeks to be able to motivate the students more early in the semester. In addition, the desire to
compare the statistics easily to a previous semester was mentioned.
Summarized, the functional requirements are:
 The system needs to be easy to use
 Important visualizations need to be easy to access
 Visualize how numbers change over the course of weeks (like OU Analyse does)
 Compare course statistics to previous courses
 Show correlations between the grades and the learners behavior

3.2 Technical Requirements

Based on our own development experience with the Moodle platform from previous projects, we now
give an overview of the technical requirements.
First, we decided that we do not want to implement the software as a “normal report” for Moodle. The
current problem with reports inside of Moodle is, that there are many different reports, but all reports
are independent from each other. For the lecturer, this means when an additional report is installed
the number of links in the report section will be increased by one and will make it harder to separate
the interesting links from the less interesting ones. In a future system we want to make sure that each
report can be categorized, so that each report can be shown in the right context instead of listing them
altogether. In addition, we want to present a dashboard-like starting screen to the lecturer with the
most important information or even a preview of each report or category.
Second, it must be very easy to develop a report inside our solution. During previous development for
the Moodle platform, we already experienced how hard the development of a full plugin for the Moodle
LMS can be. On one hand, the system needs to be extensible, on the other hand we want it to be as
easy as possible to write a visualization for the system. To give a concrete example: Right now to
create a chart the full HTML body of the page needs to be generated.
Another lacking feature of Moodle is the logging of meta information regarding technical details like the
operating system, the browser or whether the user access the page via a mobile device or not. This is
helpful feature for lecturers to decide which kind of format to choose when publishing files. Right now
this is not logged by Moodle and therefore, cannot be visualized by reports.
As we plan to use the software at a university with thousands of students, we expect a lot of data to be
collected via the Moodle log. The last technical requirement therefore includes to keep data
duplication as low as possible to avoid any data handling issues. If possible, the application should
fully rely on the Moodle database. This also allows the plugin to be easily installable as no external
application like a Learning Record Store needs to be setup.

4 IMPLEMENTATION
Our Learning Analytics software was implemented as a plugin for the Moodle LMS. Therefore, the
PHP programming language was used. The plugin uses no external data sources and relies on the
Moodle database to store information. In general, most functionality of the plugin is provided by relying
on the existing data sources of Moodle, its logging data and course information. Only three more
tables are added to the database on installation to log technical information like browser and operating
system and to summarize statistics of courses and learners.
To be directly integrated into the course room, a link was added to the navigation menu of the course.
Each lecturer can directly access the dashboard from there to check out key figures regarding the
course. By default, the plugin comes with multiple reports with the default report being the dashboard.
Below is an example of the default report lareport_coursedashboard. The dashboard shows key
figures of the course. Like OU Analyse, we present a visualization of the number of users and the
number of clicks over the course of the weeks. Below that we show how many users are currently
enrolled into the course room and how that number changed. Next to that we show how many users
were active during the last seven days. A user is counted as active if he visited the course room at
least once. Next to that, we show the total number of clicks that have happened during the last seven
days in the course room. Below each value we show how this value has changed to the previous week
to easily spot peaks and changes.

Figure 1 Screen capture of the course dashboard

In addition to the dashboard, the following reports are included:

 Activities: The report displays the most used activities featuring a plot with the number of
clicks at the top followed by two tables showing the number of clicks per activity type (e.g.
quiz, url, forum) and the most used activities.
 Grades: The report shows a scatter plot with each learner being represented as a point. The x
axis shows the number of sessions for each learners and the y axis the grade of the student
as determined by the course grades. In addition, a trend line is shown.
 Learners: The report shows what kind of courses the learners have heard before as well as
what courses are heard in parallel by the learners.
 Sections: The report features a plot showing the number of clicks for each section of the
course.
As the plugin is currently still in development, the functionality of the reports might change. Depending
on the feedback of the lecturers and the evaluation phase, there might also be more reports added.
A feature of the developed solution was to be easily extensible. To support this use case, a report can
be created by extending the report_base class. Figure 2 shows sample source code for a module that
lists the used modules and the number of clicks for each as a table.
To implement a report the created class implements the get_parameter function and the run function.
The first function returns a list of parameter objects, which contain information about the type of the
object and whether the parameter is optional or not to create the report. The run function returns an
array with multiple possible outputs. In the sample in Figure 2 a simple table is returned which is filled
by executing a query against the database and filling the table with the resulting data. Alternatively,
normal HTML code or plots can be returned. Plots are supported by using a special plot class which
creates the data of the plot in PHP and creates a plot on the client-side via Plotly.js [16]. We decides
to use Plotly.js although Moodle comes with an API to create charts to allow more customization and
more complex plots.
Additional reports can either be directly included into our source code by us or installed as an
additional Moodle plugin allowing the usage of the Moodle Plugin Management to administrate the
reports of our plugin. To support this, we added the custom plugin type lareport (Learning Analytics
Report). Creating a report therefore consists of creating two files. First, the version.php which contains
Meta information like the version number and the name of the (sub) plugin. Second, the actual
functionality extending the report_base class as explained above.
class lareport_topmodules extends report_base {

public function get_parameter() : array {

return [new parameter_course('course')];
}

public function run(array $params): array {

global $DB;
$output = new table();
$records = $DB->get_records_sql("
SELECT COUNT('id') as hits, eventname
FROM {logstore_standard_log}
WHERE courseid = {$params['course']}
GROUP BY eventname
ORDER BY hits DESC
");
$output->set_header_local(['hits', 'eventname'], 'lareport_topmodules');

foreach ($records as $row) {

$output->add_row([$row->hits, $row->eventname]);
}

return [$output];
}
}

Figure 2 Sample source code of "Top Modules" report

5 EVALUATION
Currently, the plugin is still in development. Nevertheless, a lecturer was asked for initial feedback by
using a think-aloud test. The lecturer was confronted with his course and it was asked to have a look
at the Analytics functionality. In terms of usability and performance, the lecturer liked the interface and
the representation of information. Overall, the plugin was received well.
A detailed evaluation follows after the next milestone release. The detailed evaluations will include
qualitative as well as quantitative feedback.

6 FUTURE WORK
The application is reaching feature-completeness. After the “1.0” release, we will conduct a detailed
evaluation as mentioned above.
One feature on our roadmap is the implementation of statistics for the students. At the moment, the
statistics are only displayed to the lecturer. In the future, we also want to ask students what kind of
statistics they are interested in and investigate how Learning Analytics statistics could be separated
from the statistics of the lecturers.
Currently our solution is also focused on descriptive statistics. Another planned feature is to implement
predictive statistics like the built-in Moodle Analytics system.

7 CONCLUSIONS
In this paper, we gave an overview of the existing possibilities to add Analytics to Learning
Management Systems. We first presented different systems for other LMS and explained the research
results. After that we focused on the Moodle LMS. Reports can be used to present statistics to
lecturers. There is also the Moodle Analytics functionality that uses a Machine Learning approach to
predict Learning outcomes. In addition to the built-in functionality, we presented existing Moodle
plugins as well as external data stores like Learning Records which can be linked to the LMS.
 After that, the requirements for our own Learning Analytics implementation were presented. The
implementation is done as a Moodle plugin allowing an easy installation and integration into the
Moodle LMS. By default multiple reports are included as well as a dashboard with an overview of
information regarding the course. The plugin is easily extensible by writing plugins of the type lareport
allowing to add custom reports with minimal effort.
At the moment, the plugin is still in development, we are hoping to release version 1.0 soon. Due to
the development process the evaluation process has been limited so far. As soon as the next
milestone is reached, we will execute a more detailed evaluation including qualitative and quantitative
feedback of lecturers. In the long term, we want to release the plugin as into the official Moodle store
to allow it to be used on any Moodle instance.

REFERENCES
[1] Arnold KE, Pistilli MD. Course signals at Purdue: Using learning analytics to increase student
success. In: Course signals at Purdue: Using learning analytics to increase student success;
2012; 267–70.
[2] Siemens G, Long P. Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE
review 2011; 46(5): 30.
[3] Niemann K, Schmitz H-C, Scheffel M, Wolpers M. Usage Contexts for Object Similarity:
Exploratory Investigations. In: Usage Contexts for Object Similarity: Exploratory Investigations;
2011. New York, NY, USA: ACM; 81–5.
[4] Pardo A, Kloos CD. Stepping out of the Box: Towards Analytics Outside the Learning
Management System. In: Stepping out of the Box: Towards Analytics Outside the Learning
Management System; 2011. New York, NY, USA: ACM; 163–7.
[5] Green KC. The National Survey of Computing and Information Technology. The Campus
Computing Project 2013.
[6] Klaus Steitz. Moodle-Instanzen an deutschsprachigen Hochschulen; 2019. Available from: URL:
https://blog.e-learning.tu-darmstadt.de/2010/04/09/moodle-instanzen-an-deutschsprachigen-
hochschulen/.
[7] Kuzilek J, Hlosta M, Herrmannova D, Zdrahal Z, Wolff A. OU Analyse: analysing at-risk students
at The Open University. Learning Analytics Review 2015: 1–16.
[8] Wolff A, Zdrahal Z, Herrmannova D, Kuzilek J, Hlosta M. Developing predictive models for early
detection of at-risk students on distance learning modules 2014.
[9] Herodotou C, Rienties B, Boroowa A, Zdrahal Z, Hlosta M, Naydenova G. Implementing predictive
learning analytics on a large scale: the teacher’s perspective. In: Implementing predictive learning
analytics on a large scale: the teacher’s perspective; 2017; 267–71.
[10] Govaerts S, Verbert K, Duval E, Pardo A. The student activity meter for awareness and self-
reflection. In: The student activity meter for awareness and self-reflection; 2012; 869–84.
[11] MoodleDocs. Course reports - MoodleDocs [cited 2019 September 19] Available from: URL:
https://docs.moodle.org/37/en/Course_reports.
[12] MoodleDocs. Machine learning backends - MoodleDocs [cited 2019 September 19] Available
from: URL: https://docs.moodle.org/dev/Machine_learning_backends.
[13] Moodle.org. Moodle Plugins [cited 2019 October 2] Available from: URL:
https://moodle.org/plugins/.
[14] Kevan JM, Ryan PR. Experience API: Flexible, decentralized and activity-centric data collection.
Technology, knowledge and learning 2016; 21(1): 143–9.
[15] IntelliBoard. IntelliBoard.net; 2019. Available from: URL: http://www.IntelliBoard.net.
[16] Plotly Technologies Inc. Collaborative data science; 2015. Available from: URL: https://plot.ly.

View publication stats