CogInfoCom 2013 • 4th IEEE International Conference on Cognitive Infocommunications • December 2–5, 2013 , Budapest, Hungary

A general framework for innovative mobile

biofeedback based educational games
Bertalan Forstner, Luca Szegletes, Róbert Angeli, András Fekete
Department of Automation and Applied Informatics
Budapest University of Technology and Economics

Abstract -- Educational games on mobile devices are getting states can be applied into the game, and the prediction of
more and more popular due to the widespread usage of tablets reward can be estimated. Some of the sensors are measuring
among children. The effectiveness of autonomous learning can brainwaves, muscle tone, skin conductance or heart rate. The
be increased with the use of properly processed biofeedback performance of players can be improved by applying
signals. The aim of our research is to create skill-developing feedback and responses into the underlying structure of the
and educational games with a framework based on estimating game design [2].
cognitive state and neurological feedback, which is general
enough to be extended with different biofeedback sensor In our research, we created an easy-to-use framework in
devices. The goal is to improve the learning efficiency by order to develop not only the interaction between games and
adapting to the users learning capabilities. Using the heart rate players, but also the improvement of learning ability. This
monitor and the EEG device we can examine the emotional approach achieves adaptive gaming by the alteration of
and cognitive states of the user, modify the difficulty level and different kinds of rewards. We focus on this reward and how
the attributes of the reward to fit the user’s needs, helping to we attempt to manipulate and modify it through the game.
get the best results.
The primary aim of the research is to enable development
Keywords: Human Computer Interaction; Software of adaptive mobile assisted educational games.
framework; Biofeedback; Mobile Games;
Another impact on the gaming society is the rapid
evolution of mobile devices. The number of games
I. INTRODUCTION developed for these devices increased rapidly recently.
Human Computer Interaction (HCI) research has a strong Tablets have been introduced in education as well, and the
influence on video game development and modern interfaces anytime-anywhere availability of these devices makes them a
were designed to enhance dynamic interaction between perfect tool in education. Our research focuses on these
games and players in order to achieve a better user devices.
experience.
Educational games represent a remarkable part of video II. RELATED WORK
games, where the primary purpose is not the pure Previous studies investigated different biofeedback
engagement but to educate players and to produce progress methods on mobile devices [3]. Their primary objective was
in learning. They are designed to teach players about certain to estimate the emotional state of players by monitoring heart
subjects and reinforce development. rate, blood flow or biological alternations. It was shown by
Hercegfi[4], that with applying spectral analysis on the HPV
In this paper, the aim is to achieve better performance in
(Heart Period Variance) it is possible to draw conclusions on
learning, therefore, we extend the concept of educational
the subject’s mental load, as it causes a decrease in the mid-
games with adaptability. By definition adaptive games
frequency (MF, 0.07 Hz – 0.15 Hz) power band.
recognize and comprehend the interaction of players and
intelligently alter themselves to adapt to their needs or goals Such observations can also be extended by
in order to improve the gameplay experience. neurofeedback technology. Brain Computer Interface (BCI)
research has seen progress beyond the medical domain [5]
The concept of our approach is based on reinforcement
[6]. Human Computer Interaction (HCI) researchers
learning. It is a well-established algorithm in computer
developed an interest towards Interactive Gaming and they
science, but it initially originates from behavioral science [1].
proposed, future applications and challenges in previous
The fundamental purpose of the theory is to explain how
papers.
reward-seeking behavior affects the decision-making process
of an agent. In a certain environment there are different One of the most important signals that can be used is the
actions to choose from and the primary goal of an agent is to so called P300 signal. It is an event related potential, that is
achieve the maximal reward. in connection with – among other things – rewarding, and it
is sensitive for the size of the reward. It is also important to
We attempt to estimate the reward expectation using
mention, that the power of brain waves in different ranges
biofeedback technology. Affective games are games, where
(alpha, beta, theta) can also provide some metrics about the
usually physiological sensors provide information about the
degree of concentration
hidden emotional state of players. With biofeedback these

978-1-4799-1546-0/13/$31.00 ©2013 IEEE 775

 B. Forstner et al. • A general framework for innovative mobile biofeedback based educational games

III. REWARDING
Our aim is to enhance the learning ability by
manipulating the rewards in the game, which is based on
reinforcement learning. Reinforcement learning is a well-
known learning algorithm established in machine learning.
Over the past decades, reinforcement learning algorithm was
adapted to cognitive neuroscience. The fundamental purpose
of the theory is to explain how reward-seeking behavior
affects the decision-making process of an agent. In a certain
environment there are different actions to choose from. The
Fig. 2. Simplified architecture. The architecture contains three main roles:
primary goal of an agent is to achieve the maximal global the game, the supervisor and the framework. The game and the framework
reward in this environment. The agent attempts to interact run on the same tablet in order to reduce the communication overhead. The
with the environment to receive this particular reward. supervisor works on a different tablet.

Therefore, the agent takes an action according to the

anticipated reward gained through this concrete action. V. FRAMEWORK
However, before learning, this anticipated reward differs
The framework contains well-defined components for
from the actual one, and this error indicates learning
different roles.
progression as a result [7] [8] [9] [10].
In a typical classroom environment, a supervisor is
always present, who is a qualified pedagogue, and is
responsible for the coordination of the measurement. A
supervisor is able to monitor the whole gaming process and
modify the outputs of the framework whenever necessary.
The next component of the process is the game itself. The
main aim of the framework is to enable easy portability of
Fig. 1. Reinforcement learning existing, traditional educational games to framework-aware,
adaptive, affective educational games. Therefore, an easy to
For adaptability we developed a reward system with use programming interface was designed between the games
three components. The first one is difficulty; it corresponds and the framework. The communication between them is
to the complexity level of the game section. For example, the adaptive because of the possibility of dynamic type
user gets more of the words that should be paired in an registration at runtime. To achieve this we constructed a
exercise. The next component of the reward system is the unique combination of dynamic proxy pattern and the active
value of the reward. This means, that the size or the number object pattern. As a result, developers can very easily
of the given reward can be influenced by the rewarding integrate the framework functionality. Establishment of
system as well. Finally, we added a reward type to the network communications, and the realization of inter process
system in order to enable the framework to differentiate communication remain hidden transparently from the
between categories of reward as well. For example, it may be developers, so they do not need to overcome these problems.
more motivating for a little girl to get a pink puppet as The core component is the framework, and it is
reward, rather than getting a blue toy car. responsible for the whole evaluation process and it enables
attaching different sensors and measurement processes to the
IV. USE CASE experiment. This component can be devided to multiple
The typical scenario of a session contains 5 parts. (1) modules, which can be registered in or out dynamically at
Subjects are playing on their mobile device. During run time, thus making the framework more adaptive. It is
gameplay the game communicates through predefined events necessary to process data from multiple sources, so we
with the framework. (2) The sensors are collecting data side decided to use event-based communication.
by side with the gaming process. Mental activity and Attaching a new sensor can be done easily, as the classes
engagement levels are typically measured by these devices. provided by the framework’s library can be used and
(3) These signals are processed considering the extended. This way the framework’s other components will
characteristics of the measurement. (4) A background not be affected, and the changes can be reduced to code that
process runs alongside the evaluation process and sends the is necessary to communicate with the device.
results to the server. (5) After the evaluation process has
ended, the framework provides the rewards. While subjects With the analysis of the requirements from educators,
are playing with educational games, an expert is we found that these software components should run on 2
simultaneously present to check their development. devices: the first part of the system takes place on the
student machine which runs the game and communicates
with the biofeedback sensors, and the second part should be
located on the supervisor device that is operated by the
appropriate educator.

776
 CogInfoCom 2013 • 4th IEEE International Conference on Cognitive Infocommunications • December 2–5, 2013 , Budapest, Hungary

The framework provides an easy to use interface for

There is also a fourth component, which is the server. It game developers. We intended to create a system that can be
is responsible for storing the data collected by the easily integrated with any kind of games. Therefore, we
framework. This can be the basis of data-mining researches created a library in order to handle the communication
in the future. between the two components: the service of the framework
and the activity of the game.
The main component on the student machine has 4
interfaces: (1) one for the biofeedback sensors, (2) one for The library provides base classes for rewards and events,
the different game software components, (3) another and the developer of the game has to override them. The goal
interface to the supervisor machine and (4) one for the is that the framework should not be modified for each game,
backend side. In this section we will analyze those interfaces but the reward and the event classes should be generated by
in detail. the game, and therefore, they are unknown by the framework
at compilation time.
VI. SENSORS One way to handle the above mentioned problem is to
The framework is flexible enough to enable the represent the type of events and rewards by descriptor
adaptation of different biofeedback sources. The most classes, but this solution would be very uncomfortable for
important sensor for our system is the EEG device. The game developers.
component processes and interprets the EEG signals, then Finally, we introduced annotations into the framework,
the brain activities mentioned as markers in the related which indicate the parameters of the event subclasses that
articles [11] (such as P3A signal, beta waves) can be should be sent to the framework, and the parameters of
identified. The other sensor that is the basis of the feedback rewards that should be set by the framework based on the
in our framework is a heart rate monitor device. Having data coming from the sensors, the supervisor and the game.
more sensors ensures the precise guessing of the user’s Using these annotations the library generates the descriptor
mental state and helps the calibration of these devices, as and class instances of general event and reward. They can be
every person brain waves and heart rate might have different deserialized and processed by the framework. This method
characteristics. The spectral analysis is currently based on also handles the registration of the descriptors.
Fast Fourier Transformation (FFT), but for more detailed
analysis we plan to change to wavelets or to Burg’s method, The supervisor can send some general commands to the
wich provides better frequency resolution, and can be framework in order to monitor and manage the gameplay.
computed efficiently as well. These commands are similar and common for each game (e.
g. start, pause, stop and restart the game). For these scenarios
VII. INTERFACES we introduced callback methods.
We can differentiate two classes of interfaces: the first is However, each game can have some specific commands,
between games and the framework and the other one is which can also be sent by the supervisor. To make the
between the supervisor unit and the framework. implementation of these commands easy, the game
developer creates a method with the appropriate annotation,
The framework assumes only general features of the and the library takes care of registering these commands by
game, for example, it might have different difficulty levels, the framework and sending them to the supervisor
or it can give score or other rewards for the different player application, if it is available. In addition, screenshot events
actions, or the game logic can choose the next challenge for are also included in these groups of events. These are sent at
the player. certain periods of time in order to enable the supervisor to
All these data sets are collected via the game interface of track the gameplay by watching the screen of the player.
the framework. The measurement component can send the
suggested actions to the game engine that way. As this
interface is simple enough, it is easy to attach any existing
game to our framework.
Through the supervisor interface, the framework sends its
inference on the state of the player (e. g. student is annoyed
by the current task, it is not challenging for him anymore and
so on).
The supervisor is able to indicate signals to the
measurement unit through the interface. Such indications
could be manual modifications of the measurement, like
overriding the next level of difficulty or invalidate certain
sensor measurements, if some external factors influenced the Fig. 3. Interface. The interface between the game and the framework
player. This ensures that they will not be able to alter the provides easy access to the biofeedback system.
final results.

777
 B. Forstner et al. • A general framework for innovative mobile biofeedback based educational games

It is necessary for us to provide an appropriate estimate (Washington, DC, USA), pp. 270 - 276, IEEE Computer Society,
for the difficulty of the next level. This is an important step 2010.
to reach the optimal level of the learning process. [3] M. K. Petersen, C. Stahlhut, A. Stopczynski, J. E. Larsen, and L.
K. Hansen, “Smartphones get emotional: mind reading images
To do this, the framework uses a neural network and reconstructing the neural sources”, in Proceedings of the 4th
implementation. This neural network can be taught by the international conference on Affective computing and intelligent
interaction - Volume Part II, ACII'11, (Berlin,Heidelberg), pp.
supervisor application, which means that at the end of the 578 - 587, Springer-Verlag, 2011.
levels, the supervisor can suggest the difficulty of the next [4] K. Hercegfi, “Heart Rate Variability Monitoring during Human-
level of the game. From these suggestions the neural network Computer Interaction”, Acta Polytechnica Hungarica, Vol. 8, No.
can learn the correlation between the behavior of the player 5, 2011
and difficulty of the level. The amount of feedback from the [5] A. Nijholt, D. P.-O. Bos, and B. Reuderink, “Turning
supervisor will enable framework to provide accurate shortcomings into challenges: Brain-computer interfaces for
suggestion to the next level of difficulty. games”, Entertainment Computing, vol. 1, no. 2, pp. 85 - 94,
2009. Intelligent Technologies for Interactive Entertainment.
The mentioned network is a multi-input neural network. [6] J. B. F. van Erp, F. Lotte, and M. Tangermann, “Brain-computer
The weights of the edges of this neural network are amended interfaces: Beyond medical applications”, IEEE Computer, vol.
based on difficulty level proposed by the supervisor. The 45, no. 4, pp. 26 - 34, 2012.
inputs include for example the average length of touch [7] W. Schultz, “Reward signaling by dopamine neurons,”
Neuroscientist, 7(4), pp. 293-302, Aug 2001.
events, the variance of the time among touch events etc. The
[8] W. Schultz, P. Dayan, P. R. Montague, “A neural substrate of
number of inputs can be continuously expanding (f. e theta prediction and reward,” Science, 275, pp. 1593–1599. 1997.
brain waves).
[9] M. Aggarwal, B. I. Hyland, J. R Wickens, “Neural control of
dopamine neurotransmission: implications for reinforcement
VIII. CONCLUSION learning,” Euro, J. Neurosci., 35, pp. 1115–1123, 2012.
[10] S. B. Flagel, J.J. Clark, T. E. Robinson, L. Mayo, A. Czuj, I.
In the past year, we developed a framework with the goal Willuhn, C. A. Akers, S. M. Clinton, P. E. Phillips, H. Akil, “A
of improving learning progression in educational games. The selective role for dopamine in stimulus-reward learning,” Nature,
concept of the framework is based on biofeedback gaming 469, pp 53-57, 2011.
and reinforcement learning. [11] H. W. Ray, William J.; Cole, “Eeg alpha activity reflects
attentional demands, and beta activity reflects emotional and
The architecture of the system is loosely coupled in order cognitive processes”, Science, vol. 228, pp. 750 - 752, 5 1985.
to enable developers and researchers to attach games and
further components (sensors, drivers, measurement methods)
to the framework.
Furthermore, we designed a group of educational games
for children with disabilities and we attached these games to
the framework. The design process was carefully discussed
with experts from interdisciplinary fields.
Future work will focus on new game development and
optimization of the framework, furthermore improving the
spectral analysis algorithm used with the brain waves and the
heart rate, to make it even more reliable and to be able to use
smaller windows from the signals.

ACKNOWLEDGMENTS
This work was partially supported by the European
Union and the European Social Fund through project
FuturICT.hu (grant no.: TAMOP-4.2.2.C-11/1/KONV-2012-
0013) organized by VIKING Zrt. Balatonfüred.
This work is connected to the scientific program of the
"Development of quality-oriented and harmonized R+D+I
strategy and functional model at BME" project. This project
is supported by the New Széchényi Plan (Project ID:
TÁMOP-4.2.1/B-09/1/KMR-2010-0002).

REFERENCES
[1] R. S. Sutton and A. G. Barto, Reinforcement Learning: An
Introduction. Cambridge, MA: The MIT Press, 1998.
[2] Q. Wang, O. Sourina, and M. K. Nguyen, “Eeg-based "serious"
games design for medical applications”, in Proceedings of the
2010 International Conference on Cyberworlds, CW '10,

778