http://dx.doi.org/10.5755/j01.eee.18.10.3064 ELEKTRONIKA IR ELEKTROTECHNIKA, ISSN 1392-1215, VOL. 18, NO.

10, 2012

Real-time Control System for Various

Applications using Sensor Fusion Algorithm
E. Artemciukas1,2, R. Plestys1, A. Andziulis1, K. Gerasimov1,3, E. Zulkas1, L. Pasviestis1, A. Krauze4
1
Department of Informatics Engineering, Faculty of Maritime Engineering, Klaipeda University,
Bijunu St. 17, LT-91225 Klaipeda, Lithuania, phone: +370 699 55203
2
Department of Electrical and Mechanical Engineering, Faculty of Technology, Klaipeda State College,
Bijunu St. 10, LT-91223, Klaipeda, Lithuania, phone: +370 610 44599
3
Department of System Analysis, Faculty of Informatics, Kaunas University of Technology,
Studentu St. 50, LT-51368, Kaunas, Lithuania; phone: +370 617 18603
4
Ventspils University College, Inzenieru St. 101a , LV – 3600, Ventspils, Latvia; phone: +371 63628303
[email protected]


Abstract—Real-time control and monitoring systems uses and tracking techniques are based on image recognition [3]–
sensors (accelerometers and gyroscopes), that can be applied to [7], to ensure effective real-time interface with robots.
design inertial mouse, object tracking, motion sensing, However, this method has lots of problems: firstly, because
stabilization and other systems. In order to acquire high system
of the lighting conditions, secondly, there is a huge demand
accuracy, most of the implementations demands huge real-time
computing. Therefore, the prototype of real-time control system of real time computing because of the chaotic and various
was designed using sensor fusion algorithm that ensures environment backgrounds and, lastly, the complexity of
accurate various systems controls with minimal computing hand, that has more than 20 degrees of freedom and the
resources. Accelerometer data read and accelerometer with more degrees of freedom we have the more computing
gyroscope fusion algorithm results were compared and simple resources we need.
3D visualization was presented to verify and visually
demonstrate the efficiency of proposed real-time control system
prototype. II. RELATED WORK
Inertial measurement units (IMU) also are gaining
Index Terms—Real time systems, accelerometers, popularity in body motion monitoring [1], robot navigation
gyroscopes, control system analysis. systems [3] and tracking techniques. Micro-
electromechanical accelerometers and gyroscopes are used
I. INTRODUCTION for these purposes, however, they still present several
In modern society gyroscopes, accelerometers and undesired characteristics in their output that should be
magnetometers are dominant sensors in consumer compensated through proper device calibration [4].
electronics. User interfaces, augmented reality, image Calibration of the separate sensors is not sufficient, because
stabilization, handsets, gaming and pointing devices, motion the influence of noise is not compensated enough and this is
sensing, medical and industrial instrumentation, orientation critical to the systems. In order to obtain optimum
estimation systems [1], [2], hard disk drive protection and sensitivities of all sensors were presented in [6] work, but
other applications are based on these sensors (Fig. 1). the solution demands huge computing resources, therefore,
the latency of the systems performance increases. Other
implementation use 12 accelerometers to acquire better
accuracy [5], but the systems reliability decreases, when we
use many devices, therefore, redundant accelerometers are
not as effective as using the array of different sensors, for
instance, gyroscope with accelerometer, magnetometer or
GPS for better position accuracy estimation. To sum it up, if
the sensors are used in separate way, it is complicated to
Fig. 1. Typical applications.
obtain high accuracy in the system. Only by combining data
Some of the implementations on hand detection (motion) of the sensors it is possible to develop high precision and
quick response devices. Therefore, the objective of the paper
is to design efficient prototype of real-time control system
Manuscript received March 11, 2012; accepted May 22, 2012.
using sensor fusion algorithm.
The authors thank the Project LLII-061 Development of Joint Research
and Training Centre in High Technology Area (Latvia-Lithuania Cross III. SELECTION OF HARDWARE: MICROCONTROLLER AND
Border Cooperation Programme Under European Territorial Cooperation MICRO-ELECTROMECHANICAL SENSORS
Objective 2007-2013. Subsidy Contract No: LV-LT/1.1./LLII-061/2010/)
for the possibility to complete a scientific research. For real-time control system prototype Arduino

61
 ELEKTRONIKA IR ELEKTROTECHNIKA, ISSN 1392-1215, VOL. 18, NO. 10, 2012

microcontroller was selected, which is an open-source IV. SENSOR FUSION ALGORITHM FOR RELIABLE AND
prototyping platform, a tool that is designed for developing ACCURATE ORIENTATION SENSING
interactive objects, taking inputs from the variety of switches Using an accelerometer for tilt sensing is simple, however,
or sensors, in order to control lights, servos, motors and calculating tilt using only an accelerometer has some
other actuators (Fig. 2). limitations. If the user moves the device rapidly or if the
device is being used in a vibrating environment, for instance,
Programming: LCD
Processing, displays
Servo LED’s
Output devices
in a car or a plane, the computed pitch and roll angle values
C++, etc. will be completely wrong and unreliable, because sensor
suffers from external accelerations and vibrations, which
Arduino
USB (serial)
ATMEGA328
cause errors. Averaging accelerometer readings could be a
microcontroller solution, but only with the assumption that the device is
almost steady. In section 2 we described the selected
Sensors Switches GPS’s Input devices hardware and communication between devices that were
used in real-time control system prototype. Therefore,
Fig. 2. Arduino prototyping platform application example. simple, yet efficient self-explanatory algorithm of sensor
data fusion (accelerometer and gyroscope) is presented in
Arduino can operate autonomously or communicate with
Fig. 3.
other software on computer, for instance, Flash, Blender,
MaxMSP or Processing. Arduino port specification: 14 Arduino
microcontroller Notation:
digital 5 V tolerant I/O pins with 40 mA DC current per I/O Starting up system

= Start
(6 of them PWM outputs), 6 analogue inputs. Arduino
microcontrollers
Nowadays various sensors are designed to enable input/output setup
= Process / Action

selectable sensitivity, for instance, 1.5 g, 2 g, 4 g, etc. The Accelerometer and

capability to choose multi-axis sensitivities is critical to gyroscope register

initialization

n
io
is
ec
detect single/double tap, activity/inactivity, free-fall, motion,

D
=
Data read from
tilt, shock and vibration for multiple portable applications accelerometer and
gyroscope
and functions. Therefore, in order to design real-time control = Stop

system prototype, the digital 3-axis ADXL345 accelerometer NO

io l
at fu
n?
YES Data sent over
liz ss
itia ce

serial port to
was selected that uses low power, high resolution (13 bit)
In uc
S

computer

measurement modes from ±2 g at up to ±16 g, 16 bit digital

output data of every axis is accessible through either a SPI
or I2C digital interfaces. The digital 3-axis ITG-3200
Processing
Data processing,
applying sensor
sketch
fusion algorithm
gyroscope angular rate sensor with a sensitivity of 14 LSBs
per degrees/sec and a full-scale range of ±2000 degrees/sec
3-axis estimation
was also selected. Low power sensor modes enable
intelligent motion-based power management.
All the selected sensors are digital, which means that NO
io l
at sfu
n?
tim es
es ucc

digital logic capable of converting the analogue signal

S

coming from the mechanical components into digital values YES

accessible using a digital communication protocols, in our Real-time control

of 3D object
case, I2C a 2-wire interface comprised of the signal serial
data (SDA) and serial clock (SCL). Moreover, sensors have
em e

NO YES
st th

different configurable parameters, which change how it Power

?
sy top

off
S

works internally, for example, configurable sampling rate as

well as configurable interrupt conditions. By connecting 2 Fig. 3. Sensor fusion algorithm.
breakout boards of the accelerometer ADXL345 and
gyroscope ITG-3200 on the same I2C bus in parallel to both After the power up of Arduino microcontroller, the
SDA and SCL lines, which are open drain and bidirectional, input/output setup, accelerometer and gyroscope register
it is possible to design the 6 degrees of measurement system, initialization processes takes place, which verifies
with the speed up to 400 KHz. It is important to note, that continuous sensor communication. As mentioned in previous
every sensor has its own unique address, for instance, section, sensors are connected to I2C interface, where SDA
accelerometer has 0x53h and gyroscope – 0x69h. It is also channel is responsible for data transmission and SCL for
essential to indicate the appropriate addresses sequence for addressing. Furthermore, because there are 2 connected
reading or writing data of single, multiple-byte from or to devices on the same I2C interface, every sensor must have its
registers, firstly, referring to device address, secondly, to own unique device address, also different register addresses
register address. In order to acquire 1-axis data it is are required to determine measurement modes, speed of the
necessary to read 2 bytes from 2 different device register data gathering, sensitivity, etc. Accelerometer initialization
addresses. modes starts from powering up sensor, then enabling auto-

62
 ELEKTRONIKA IR ELEKTROTECHNIKA, ISSN 1392-1215, VOL. 18, NO. 10, 2012

sleep functionality, that switches to sleep mode if the are estimated and if at least 1 axis is not a number (NaN),
inactivity function is enabled and measurement mode, where the process returns to data processing (according to Fig. 3),
data read is set to ±2 g, 10 bit resolution and gyroscopes because of the wrong data transmission over the serial port.
data read is set to 1 sample/read every 8 ms (125 Hz), full- Otherwise, the loop repeats and depending on feedback of
scale range +/-2000 degrees/second and low pass filter is the sensors acceleration and angular rate, virtual 3D object is
5Hz. After the initialization processes are verified, sensors controlled. Moving gyroscope and accelerometer sensors in
proceeds to two parallel raw data read processes of your hand and depending from the acceleration and
gyroscope and accelerometer. All the data read of x, y, z axis orientation, the virtual object moves smoothly in that
with Arduino microcontroller is then sent over serial port to direction. Such prototype of real-time control system using
Processing programming language (Processing sketch) on sensor fusion algorithm can be applied for various
computer for data processing, where sensor fusion algorithm applications.
is applied (combined data) for accurate control, otherwise, if
the data read of a single axis from the sensor is unsuccessful V. RESULTS
(all values does not change or equal to zero) the processes In order to verify the effectiveness of sensor fusion
starts from the beginning. The model of real-time control algorithm for prototype of real-time control system, the data
system prototype is presented in Fig. 4. were analyzed and the following experiments were
performed:
Real-time control 1) Sensor board in stable state;
of virtual 3D object
Accelerometer
Serial
2) Sensor board tilting without vibrations;
ADXL345 I2C
Arduino port 3) Sensor board tilting applying vibrations by tapping
interface
Gyroscope Microcontroller
(ATMEGA328) it.
ITG-3200
Sensors output of 1-axis data of the accelerometer and
IMU sensor
combined accelerometer with gyroscope via serial port are
Fig. 4. Model of real-time control system prototype. presented in Fig. 6.
The prototype of real-time control system was designed Angle, o Accelerometers Combined accelerometers and
data gyroscopes data
and visualization of virtual 3D object was implemented 90

using Processing programming language, in order to 45

Time, s
estimate effectiveness of proposed solution (Fig. 5). 5 10 15 20 25 30 35 40 45 50 55 60
-45

-90
Stable Tilting
Tilting with vibrations
Sensor readings and estimation state without vibrations

Fig. 6. Experiment results of 1-axis.

3D object control in real-time The sensors data output of other 2 axis would be
analogical. As the experiment results demonstrate, acquiring
data from only one accelerometer is not reliable because of
Arduino Microcontroller
these reasons: accelerometer measures inertial force and in
Accelerometer ADXL345 ideal case it is caused only by gravity and devices
acceleration. Therefore, even if accelerometer is in stable
Gyroscope ITG-3200
state, it is affected, in general, by mechanical noise (Fig. 6
a) black line). The gyroscope is also subject to noise and
suffers from drift, when not coming back to zero-rate (0 g)
value after rotation stops. However, since gyroscope
measures rotation it is less sensitive to linear mechanical
movements, as a result, gyroscope compensates
accelerometer errors. Signal of the combined accelerometer
and gyroscope data (Fig. 6 red line) is more immune to
noise, than the accelerometer readings alone of every axis.
Nevertheless, by averaging and combining accelerometers
and gyroscopes data it is possible to obtain accurate data
b) c) d) estimation for real-time control system, than we would
Fig. 5. Prototype (a) of real-time remote control system and visualization obtain using sensors separately.
of virtual 3D objects; real-time control of 3D object: (b) – horizontal state;
(c) – x-axis roll to the right; (d) – x-axis roll to the left.
VI. CONCLUSIONS
The left 3D object is controlled only with accelerometer The research indicates that the proposed system suggests
and the right one for smooth control with accelerometer data efficient real-time control system prototype. The biggest
combined with gyroscopes data (gyroscope compensates advantage of the designed system is its wide range of
accelerometers errors). Before the control (rotation) of the application capabilities in various fields, for instance, the use
3D object, the coordinates of accelerometer and gyroscope of IMU inertial sensor in real-time control system for new

63
 ELEKTRONIKA IR ELEKTROTECHNIKA, ISSN 1392-1215, VOL. 18, NO. 10, 2012

generation inertial mouse, using single/double tap

capabilities of the ADXL345 accelerometer it is possible to
design a mouse that would require no buttons, where buttons
are replaced by taps and menu scrolling by orientation; also,
it can be used for an unmanned aerial vehicle for
stabilization and tracking during flight, augmented reality
technology, to manipulate the elements of 3D model,
developing prototypes of human computer interaction
devices, tangible user interfaces, in general, for accurate
control of technical-systemic platforms. 3-axis data read
from the sensors (3-axis accelerometer and 3-axis
gyroscope) were implemented and sensor fusion algorithm
applied for the prototype of real-time control system. The
experiment results were compared and demonstrate, that the
data read from one sensor is not sufficient, therefore, the
array of sensors with 6 degrees of freedom were used.
Furthermore, the visualization of 3D virtual object was
implemented using Processing programming language, in
order to visually estimate the effectiveness, increased
sensitivity and reliability of the orientation sensing
capabilities of the proposed solution. Accordingly, the
measured data of linear acceleration, gravity and orientation
were used in sensor fusion algorithm to get minimal biases,
while controlling the orientation of virtual 3D object.
In further development process the magnetometer
HMC5883 and wireless ZigBee module will be added in
order to design more accurate real-time remote control
system over wireless long range communication or wireless
close range communication with computer via Bluetooth.
Depending on the task it is possible to use other sensor
capabilities, for instance, temperature measurements with
gyroscope or free-fall, activity and inactivity detection with
accelerometer, altogether designing tangible user interfaces.

REFERENCES
[1] Hongyin Lau, Kaiyu Tong, “The reliability of using accelerometer
and gyroscope for gait event identification on persons with dropped
foot”, Health Technology and Informatics, vol. 27, no 2, pp. 248–
257, 2008. [Online]. Available:
http://dx.doi.org/10.1016/j.gaitpost.2007.03.018
[2] C. F. Kao, T. L. Chen, “Design and analysis of an orientation
estimation system using coplanar gyro-free inertial measurement unit
and magnetic sensors”, Sensors and Actuators A: Physical, vol. 144,
no. 2, pp. 251–262, 2008. [Online]. Available:
http://dx.doi.org/10.1016/j.sna.2008.02.008
[3] Lu Lou, Xin Xu, “An Approach to Improving Attitude Estimation
Using Sensor Fusion for Robot Navigation”, Procedia Engineering,
vol. 15, pp. 5601–5605, 2011. [Online]. Available:
http://dx.doi.org/10.1016/j.proeng.2011.08.1040.
[4] A. Olivares, G. Olivares, J. M. Gorriz, J. Ramirez, “High-efficiency
low-cost accelerometer-aided gyroscope calibration”, Test and
Measurement, vol. 1, pp. 354–360, 2010. [Online]. Available:
http://dx.doi.org/10.1109/ICTM.2009.5412920
[5] K. Parsa, T. A. Lasky, B. Ravani, “Design and Implementation of a
Mechatronic, All-Accelerometer Inertial Measurement Unit”,
Mechatronics, IEEE/ASME Transactions, vol. 12, no. 6, pp. 640–
650, 2007. [Online]. Available: http://dx.doi.org/
10.1109/TMECH.2007.910080.
[6] S. A. Wahyudi, W. Widada, S. P. Hadi, “Inertial Measurement Unit
using multigain accelerometer sensor and gyroscope sensor”, in Proc.
Electrical Engineering and Informatics (ICEEI) International
Conference, 2011, pp. 1–5. [Online]. Available:
http://dx.doi.org/10.1109/ICEEI.2011.6021822
[7] K. Nickel, R. Stiefelhagen, “Visual recognition of pointing gestures
for human–robot interaction”, Image and Vision Computing, vol. 25,
no. 12, pp. 1875–1884, 2007. [Online]. Available:
http://dx.doi.org/10.1016/j.imavis.2005.12.020

64