Finland Math Guide To Learning Analytics (Part 2)
Finland Math Guide To Learning Analytics (Part 2)
com
01 13 / PERSONALIZED LEARNING
12 / STRESS-FREE LEARNING
01
Introduction
Welcome to part two of our Finland Math Guide to Learning Analytics. If
you haven’t read part one yet, we strongly recommend that you start
there.
In this second part, we dig a little deeper into learning analytics and
discuss different levels of analytics and types of data sources in detail.
As Big Data and Artificial Intelligence are terms that are mentioned in the
context of learning more and more, we also cover the connections that
those concepts have with learning analytics. Therefore, after reading
this guide you will have a good understanding of the role of learning
analytics in the field of modern education and technology.
on top of the automatically generated analytics. opportunity! You can sign up for a free pilot at this link: www.eduten.
com/get-started.html?role=teacher. The platform is easy to learn how
to use.
02
Levels of Learning Analytics Since the levels of analytics are built upon the lower levels, let’s start from the
bottom. In the descriptive analytics level, the focus is on what has happened.
A commonly accepted taxonomy of learning analytics has four categories: An example of this could be a list of students’ scores gathered from three
descriptive, diagnostic, prescriptive and predictive. mathematical exercises. Information collected via descriptive analysis can be
useful for several goals, and it is commonly used in assessment. It does not,
however, tell us why the scores are what they are or the implications of how to
DIAGNOSTIC ANALYTICS
DESCRIPTIVE ANALYTICS
David 100.0% 53.3% 95.8%
Since this guide is part of the Finland Math series, let’s use data gathered in Example of descriptive analytics data: the scores from exercises
that domain (from mathematical exercises) as an example to illustrate what
the levels actually mean:
03
In the next diagnostic level, the focus is on why things have happened. If we a student’s problem might be different to what’s first suspected. Although the
extend our data set to contain the time used to complete the exercises, we scenario here is simplified, it illustrates the basic idea of diagnostic analysis:
can likely understand the whole process better. Hence, the core idea of the wanting to know why things happened the way they happened.
diagnostic level is to extend our understanding of the learning process in our
context. The third level is the predictive level. At this level, we try to utilize the data
we have already collected and analyzed to predict future happenings. A
common (although by no means simple) method would be to collect data
STUDENT E XE R C I S E 1 EXERCISE 2 EXERCISE 3
from several instances of the same course, analyze it, then try to come up
with a model that can be used to predict future performance. For example,
John 15 / 86 sec 11 / 36 sec 22 / 123 sec we might collect data from several 3rd grade math classes to come up with
the model that predicts results of future classes.
Example of diagnostic analytics data: the scores and times spent on exercises
Data Sources
05
Data Sources When the data of several submissions are connected, we have the
ingredients for successfully analyzing students’ learning experience from
Learning analytics, like all data analytics, is heavily dependent upon data. In multifaceted points of view. Moreover, analytics in the digital tools enable
learning analytics, the data sources are connected to learning and teaching. observing changes happening over (long) periods of time. This can be an
In this section, we present some typical examples. important tool for teachers to understand the effectiveness of changes made
in their teaching methodology or new teacher interventions’ usability.
Digital Learning Tools Other typical data points collected by digital learning tools include:
Digital learning tools, including for example Learning Management Systems LOG DATA: ATTENDANCE AND ACTIVITY
IN EXTERNAL TASKS:
(like Moodle or Blackboard) or exercise and assessment systems (such as Which views the students (or
Eduten Playground) are very typical data sources in learning analytics. Digital teachers) visit, which buttons keeping track of students’ lesson
systems can automatically gather a lot of data from students’ actions while and other controls they use, or lecture attendances or other
they interact with the systems. For example, for each submitted solution, how much time they spend in activities might be close to
Eduten Playground automatically logs: individual views and what kind impossible in larger courses
• The score achieved • Detailed steps taken to solve the inside the tool. This data can be courses as well). Technology
exercise (including e.g. answers to
used to analyze the learning and can assist data collection in this
• Time when the submission was
individual calculations and time
made teaching, but also to improve sense as well, as technology like
it took to solve them, the order of
solving subtasks) the general experience of using RFID tags and readers or Wi-
• Time used to solve the exercise
the tools. Fi access logs can be used to
• Possible other relevant information
about the exercise easily collect attendance data.
06
Sensor Data
SOCIAL INTERACTION AND CONNECTIONS:
Sensors enable monitoring of the environment or the actors in the
Another aspect that is easily networks formed with other
environment. The sensors can be roughly divided into two categories.
collectable when using digital students. With machine learning
tools is the social activity and and AI algorithms it has become
students’ social connections possible to also automatically PERSONAL SENSORS AMBIENT SENSORS
with other students. Typical analyze the content and quality monitor a single person, in this monitor a larger space, typically the
context typically a teacher or a whole learning environment. This
examples are discussion forums, of messages, which enables
Digital Learning
learning diaries and chats.Tools
With further chances for deeper and
student. Sensors like this can typically
measure things such as heartbeat
environment is usually a classroom or
other restricted space where learning
digital tools it is easy to track the more meaningful analysis of the and body temperature, and the happens. The sensors can measure
values can be used to analyze factors room temperature, air pressure or
number of messages and the whole learning experience.
like anxiety and stress levels. The quality, amount of lighting, or volume
students’ mutual connection sensors can also work together, as level. As the environment can have
we have devices that measure the a huge effect on students’ wellbeing,
distance to other similar devices, and with that, the whole learning
enabling analysis of social networks in experience, ambient sensors can
All in all, it can be said that digital learning tools are among the most classrooms or other learning spaces. provide data that can be easily used
important data sources for modern learning analytics. Still, other sources In a learning context, the data is to improve the experience. Again, the
often most useful when combined data can be even more useful when
are important as well, especially if we can combine the data from several
with data from other sources. For combined with other information, for
sources. example, we can find out if certain example by finding out how changes
kinds of exercises cause more stress in ambient sound level affect
for students. Personal sensors need to students’ focus to exercises.
be easy to carry, so typically they are
wristbands (or built-in watches), rings
or something you can easily carry in
your pocket.
07
Perception Data
In addition to automatically gathered data described in previous sections, • Students’ perceptions about their motivation, wellbeing and learning
data provided by students themselves can be extremely useful. Surveys performance: it is important to give students a chance to reflect their
are a great tool for collecting learning analytics data, especially when we own mindset in learning, i.e. their metacognition. In learning analytics,
are talking about supplementing data collected automatically from other this kind of data can be utilized with two different perspectives: first, it
sources. Although surveys have their own limitations and difficulties (for can be used to provide students themselves a chance for self-reflection
example they can be very difficult to conduct reliably with younger students), of the changes in their own attitudes and wellbeing. Second, the data
they still enable collecting data that is very difficult or impossible to collect by is useful for teachers and administrators to improve the learning
any other means. experience and to validate the effectiveness of teaching methods and
materials. Data collected via surveys can be even more useful when the
Surveys can be used to collect students’ and teachers’ perceptions and correlations between different data points are analyzed.
opinions. In learning analytics, they are useful in collecting for example:
• Student feedback about used materials, methods and teachers • Teachers’ perceptions about their work, including for example
actions: students’ opinions about all aspects of teaching are crucial to workload, working hours, the quality of materials, teaching facilities and
improve the quality of teaching. When analyzing this kind of data, it is curriculum. Teachers are the leading experts in their classroom, and
important to remember that the collection time and method may have their perceptions should be taken into account whenever possible to
an effect on the results. enhance the learning experience for all.
08
Manually Collected Data
The traditional method of collecting data is to manually assess and grade
students’ exams, assignments and exercises. Although automatic data
collection has some obvious benefits in learning analytics, there are still
various examples of tasks that cannot be automatically assessed- at least
not completely. Essays, for example, are difficult to fully assess automatically
but can be crucial for data collection and assessment in certain types of
courses. Moreover, although the development of AI is rapid, some topics (like
any forms of art) are still close to impossible to evaluate by machines. A clever
machine learning algorithm may recognize a poem within text but is hardly
capable of assessing how good it is.
can provide valuable insights that are hard to detect by any other means. sources which are considered too big to handle with “traditional” data
processing methods. The term was generalized after the modern data
revolution, involving companies and systems (such as social media
platforms) which collect or generate a LOT more data than what we
Other Types of Data had used to. The amount of other types of data often surpasses the
amount of learning data. There are natural reasons for this: learning
Potentially, any data related to students or teachers could be utilized in
setups (i.e. class sizes) are often quite small, and even if the data is
learning analytics (provided that good ethical guidelines are followed). collected meticulously from all student actions, the number of rows
Examples, not included in the previous sections, include for example national in the data set does not become huge enough to justify calling it Big
data registers and national assessments (although the organizations data. Even with 100,000 students using the same online tool, there are
collecting the data with national and international assessments are often usually several grade levels, all students are not completing the same
reluctant to share it with anyone). exercise and so on. This means that the actual subgroups may be
rather small (or at least too small to justify the term big data).
Often the data stored in national registers, such as data associated with
There are some exceptions, though. MOOCs (Massive Open Online
medical conditions, socio-economic status or criminal activities could not be
Courses) can have hundreds of thousands of participants, which
used based on the ethical and privacy issues. means that the amount of data they are generating is far bigger
than what we have used in traditional educational setups. Another
10
is data collected from several different classes (or schools or such simple mechanics, that they would never be classified as
countries): with digital education tools, it could be possible to artificial intelligence by anyone, who knows even a little bit of AI.
combine the learning data from all fifth grades using the system There are some likely reasons AI is not utilized more in an educational
around the world. This would enable comparisons between different context. Machine learning algorithms often rely heavily on the amount
countries and cultures, which could enable quite useful data-driven of data. The supervised machine learning algorithms are taught by
comparisons at the administration level. presenting various examples, and the system then learns to come up
with solutions to new problems. In education there is often not enough
data for this. Moreover, the stakeholders in learning analytics may feel
satisfied with the current implementation of “AI” as they do not know
any better.
Learning Analytics As we discussed in the Examples section, there are real applications
Terms artificial intelligence (AI) and machine learning are quite mindset and other learning preferences, we would need to utilize an
often mentioned with learning analytics and with digital learning in intelligent system capable of analyzing all the relevant information
general. The research in AI has progressed rapidly in recent years, and building its decisions on that information. Intelligent systems
and we are now seeing applications everywhere, from self-driving are important at the predictive level of learning analytics, but they
cars to algorithms that write the news articles. Learning analytics are essential if we really want to utilize analytics at a prescriptive
often involves complex scenarios with various variables which might level. Example of this would be a system, which would recommend
be difficult to analyze. As such, AI could be highly useful for extracting suitable actions to be taken for each student if certain types of
information. misconceptions are detected. Although we are not there yet, we can
get close by combining the real-time identification of misconceptions
AI is used as a keyword frequently when educational technology with teachers’ expertise and deep knowledge of their students. As we
isdiscussed. However, most often it is just for marketing purposes. have said before, AI and a teacher make a perfect 21st-century team
Many applications that claim to use “AI algorithms” are built around for teaching and learning.
11
Stress-Free Learning
The third principle of Finland Math is stress-free learning. The idea is that
the students can learn from their mistakes, and also to favor continuous
assessment instead of exams (which are infamous for causing students
a lot of stress). Learning analytics can substantially help to reach both
of these goals. First, the experience of learning from mistakes can be
improved when we can find the root cause for mistakes. This way the
13
teacher (or the tool) can provide meaningful exercises or other tasks
for trying again. Second, continuous assessment requires detailed
knowledge of learning from as many aspects as possible, and again,
learning analytics can provide exactly this kind of information.
Personalized Learning
Every student is unique with their own strengths and weaknesses. As all
students learn differently, it is essential to be able to provide suitable,
personalized content for all learners. This is the fourth principle of Finland
Math. To provide personalized content, it is essential to know the students
and their skill level as well as possible. Learning analytics can provide the
teacher with all the essential information to help them give their students
suitable learning content, improving the students’ learning experience
and results.
Growth Mindset
The sixth and the final principle feedback. Moreover, analytics
of Finland Math is growth can help students understand
mindset. The point is to help their mistakes by providing the
the students understand that necessary information about
mistakes are not a bad thing - issues that caused the mistakes.
instead, they are opportunities Finally, visualizing students’
to learn something new. Growth progress can help in self-
mindset enables trying new reflection, which can lead to a
challenges without fear of better understanding of their
failure. Learning analytics can own learning process and foster
foster the mindset by enabling mental growth.
good quality, personalized
15
Eduten Playground is a digital learning tool that is built around the principles
of Finland Math. In fact, utilizing Eduten Playground is the easiest way to adapt
Finland Math into your classroom. As we discussed in the previous section,
learning analytics is an important factor in Finland Math.
Let’s discuss the analytics in Eduten Playground by following the Screenshot from Eduten Playground
recommended digital path of a teacher in the system.
As discussed before, the graph displays all students in a single course based
on two variables: the number of points collected (x-axis) and the time used
(y-axis). In an ideal situation, all students would be located in the descending
START HERE: STUDENT DILIGENCE GRAPH...
diagonal line. This would indicate a direct positive correlation between the
The best starting point for our teacher is the Student Diligence Graph on the time and the score (i.e. the more time you use, the better score you have).
front page. An example of the graph is displayed below. Anomalies, i.e. the students who are not located on the line should be
interpreted like this:
16
...AND CONTINUE TO THIS: STUDENT ACTIVITY
• The dots located well above the • The dots located well below the
line indicate students who spend a line indicate students who get the
To fully understand the reasons behind students’ performance, the teacher
lot of time but do not receive that perfect score in a very little time.
many points. should dig deeper into the analytics. The next logical view is the Student
Activity.
What the values indicate, is that the exercises are either too easy or too The general lesson statistics give an overview of the lecture, but from an
difficult, respectively. In both cases, the effect will be decreased motivation analytical point of view is most useful when comparing the lesson to other
and a worse learning experience. Hence, the teacher should concentrate on lessons.
the following indicators (and probably in this order):
The general lesson statistics give an overview of the lecture, but from an
analytical point of view is most useful when comparing the lesson to other
lessons.
01 often indicates low engagement to the Moderate and Hard). Are the students
lesson in whole. Typically accuracy should selecting tasks suitable for their skill levels?
Check the trophies. Are there students who be between 70 and 90 %. Anything below Selecting too difficult or too easy tasks can
did not achieve a bronze trophy from this that indicates too difficult content, and lower motivation.
lesson? This could indicate that the lesson numbers above 90% indicate that the
has been overly difficult or problems with content may be too easy. If anomalies are found in any of the three steps, it
motivation. is important to try to isolate the difficult tasks and
the problems in skills. That is what the next view is
designed for.
18
SKILLS
The suggested steps teachers should take in this view are:
Now, to dig even deeper behind the students’ skills, possible problems and
the reasons behind them, the next logical step is to open the Skills View. An
01
example is provided in the figure below.
Check if there are topics that are difficult for all students. These
can usually be logically explained (for example, a lesson in Eduten
Playground was opened before the topic was covered in school).
02
Check to see if there are students with more than one red indicator in
Screenshot from Eduten Playground their row. These may indicate more serious problems in mathematics
learning, and if they cannot be explained with logical reasons (such as a
Eduten Playground uses an artificial intelligence algorithm to detect students’
student being away for being ill), some kind of intervention is needed.
skill levels based on all their activity in the system. The numbers can be
interpreted as
03
• 1 to 2: according to • 3: the student is • 4 to 5: the student is
Check for other anomalies. Are there students who usually score high
data, the student has displaying average performing well on
problems with this performance in the this topic but have sudden drops (or the other way around)? Sudden changes in
topic topic the performance also indicate a need for intervention.
04 19
Sometimes there are obvious reasons on some of the low skill levels. For
Also acknowledge the students with no anomalies at the moment.
example, the student may have missed the class where the topic was taught.
These are the students who likely do not require any additional support
Nevertheless, understanding the reasons causing the low skill levels and
at the moment. It may be wise to spend resources on the students
acting accordingly is always necessary.
who are struggling with some of the topics. Also notice the needs for
differentiating upwards: too easy exercises can lead to decreased
Next, the teacher should observe the Misconception Trends. The graph can
motivation and worse mindset in general.
reveal topics which are rehearsed less than the others, which may be a cause
for worse skill levels. Moreover, if the accuracy is below 70% in any of the topics,
When the teacher locates a student who likely requires assistance, it is easy the topic should be addressed more. If this is not a general trend, but rather
to gather more information by observing the misconception data of the only concerns a single student, it is maybe time for an intervention: easier
selected student. An example is provided in the figures below. tasks and personal guidance often help to get the student back on track.
100
Although the analysis and the visualizations from Eduten Playground are often
75
Exam Score
other data with the data received from Eduten Playground. Let’s consider 50
In the case we are talking about, the final exam of a course was conducted 3 3.5 4 4.5 5
Perceprtion
23
As seen from the figure, the students who got the most points out of the exam First, it is likely that we are learning analytics and in all digital
also valued the exam tool the highest. In fact, there seems to be a direct going to have new and more learning solutions. With AI it will be
correlation between the grade and the points received from the final exam. comprehensive data sources. possible to build comprehensive
And if we stop to think about this, it is quite natural. The survey was conducted Sensor technology is improving, models for predicting students’
immediately after the exam, and it is likely that the student’s success (or their which means that it will be possible performance and changes in their
gut feeling about their success) naturally had an effect on their perceptions to capture more and more reliable mindset and motivation based
about the tool. information from the environment on the existing data. Moreover,
and from students and teachers. we may see a boost in automatic
Hence, if we wanted truly objective opinions about the exam tool, it would It is likely that we are going to see prescriptive analytics with AI-
probably be wiser to survey the students a day or two after the exam. a growth in the role of the so- based recommendations on how
called soft skills in analytics. This to avoid and solve problems and
means that we can measure and misconceptions. However, it is likely
use variables such as motivation, that at least for some decades the
mindset, stress and problem AI will complement teachers’ work
solving skills to deepen the instead of doing it for them. As we
analytics. This way, we could have said before, teachers and AI make
The Future of exercises and learning materials a perfect team for teaching.
Learning Analytics
that can truly adapt to students’
learning preferences. In the future, we also anticipate
advances in learning analytics
Predicting the future is always difficult, but let’s try it anyway. What will likely The role of artificial intelligence centered around teachers:
happen in the field of learning analytics in the future? will likely increase significantly in potentially, we could collect and
24
and analyze a lot more data of analytics. This should be
about teachers’ actions and done among all stakeholders.
their whole teaching process. Teachers and administrators
As a result of this analysis, have likely heard about
we could aim to improve the analytics, but most of them
teaching experience in new lack knowledge about the
ways. Teachers’ motivation possibilities and practical use.
and mindset are factors which Moreover, most students are not
are not discussed too much familiar with the whole concept.
in the context of learning As we discussed in the section
analytics today, but which about privacy and ethics,
should be factors that the getting all stakeholders on board
school administrators are the analytics process is very
very interested in. The goal of important. Since Finland Math is
analytics should be to improve strongly connected to learning
the whole learning environment analytics, we will do our share
to all stakeholders sharing the to bring students, teachers and
experience. administrators together to learn
about learning analytics in the
One very important goal future as well.
related to learning analytics is
to improve the understanding