2015 International Conference on Advances in Biomedical Engineering (ICABME)

M-health application for Neonatal Incubator signals monitoring

through a CoAP-based multi-agent system
L. Nachabe, M. Girod-Genet, B.ElHassan, and J.Jammas, IEEE members

Abstract— The number of premature babies is increasing in multi-agent monitoring system in a way to monitor vital
the world. With Internet connection accessibility and the signals of pre-mature babies in an incubator.
development of smart phones, m-health application is playing In 2012, the New Orleans hospitals in US have
an essential role for monitoring biomedical data. In that launched a mobile application that permits the babies’
context, we are proposing a distributed neonatal incubator relatives to monitor their new born babies due to the use of
monitoring system (NIMS) dedicated for premature babies’ web cameras [5]. As Dr. Curtis Lowery said, premature
dynamic control. Our system is made up of distributed software
babies are more to be ill-tempered or to have physical or
agents (within incubator’s data hubs, medical servers and
mobile terminals of medical staff and parents) based on emotional problems. That’s why they are more likely to be
Constraint Application Protocol (CoAP). The use of CoAP shaken or beaten [6]. But what is important to parents is
based agents, as well as URIs for data IDs, enables the not only to see their babies’ via a web cam but also to
integration of our proposed NIMS system and its collected data monitor the sensitive parameters that affect the life of the
into an Internet of Things (IoT) wide scenario. The proposed pre-mature babies. These newborn babies spent their first
system was successfully carried out and tested in a hospital weeks or months in a neonatal incubator, a controlled and
North Lebanon. isolated room where the ambient humidity, the baby’s
Keywords—Neonatal; incubator; monitoring system; heart rate, and the baby’s temperature are considered as
NIMS; premature babies; multi-agents; CoAP; IoT; URI.
fundamental parameters for the safety and the normal
growth of the baby. Therefore our distributed agent based
I. INTRODUCTION
neonatal incubator monitoring system (NIMS)
The wide usage of smart phones in addition to the rapid dynamically collects the aforementioned parameters from
growth of wireless technologies offered by the ISPs with incubator’s sensors and permits the medical staff as well
affordable prices led to the explosion of mobile as the babies’ parents to retrieve these values via a mobile
application developments. In fact, more than one quarter application. In addition, they will receive notifications in
of the global population are using smartphones in 2015 case of abnormal values and/or the need of urgent
[1]. Mobile applications range from social media interaction. This solution also enables the integration of
applications, gaming, sports, news, m-business to m-health these monitored values in the world of IoT (Internet of
applications. The evolution of biomedical sensors, the Things) due to the use of Constraint Application protocol
increasing numbers of elderly people in many societies (CoAP) mechanisms in its distributed software agents.
and the high cost of health care solutions push the This paper is divided as follow. Section II presents the
developer to create m-health applications. Many are hardware components, its characteristics and
designed at medical staff level for database handling, functionalities. In Section III, we verified our usage for
sophisticated monitoring, diagnostic data, clinical advices, CoAP to publish data and we explained the main features
etc. Others are designed at patient level to monitor vital of CoAP in our software agents and application. The
signals, to be healthy (fitness applications), or to benefit general process of the architecture, the data transmission
from online medical advices and drugs prescriptions. and publishing are detailed in Section IV. While in Section
Basically, these monitoring systems are classified based V we presented the testing results of our solution, we
on two architectures: Flat or Multi-tier [2]. In the flat finally concluded this paper in Section VI.
architecture, the mobile device gathers all the vital signals
and sends the information to the external servers. But this II. SYSTEM’S HARDWARE
architecture affects dramatically the battery’s lifetime due
Infant incubators are widely used in hospitals to protect
to permanent communication between the sensors and the
newborns, and to provide them with appropriate
mobile [3]. The multi-tier architecture encompasses, in
environmental conditions. This is essential for the proper
addition to the mobile device, a hub or data collector
growth of those newborns in the very early stage of their
between the sensor and the servers. This architecture
life, and to keep them away from airborne infectious
offers the opportunity to distribute the functionalities of
diseases. Incubators provide a closed and tightly
the system into different software or agents. Thus, it can
controlled environment that warms the infant by
be used to create multi-agent systems (MAS) capable of
circulating heated air over skin. The heat is then absorbed
integrating the necessary data capture systems, monitoring
into the body by tissue conduction and blood convection.
systems and interpretation function in an independent way
Ideally, both the skin and core temperatures should be
[4]. Therefore, in this paper, we are proposing a
maintained with only minor variations [7]. In our proposed
distributed agent based system consisting of a data
solution, we used a humidity sensor, a body temperature
collector agent, data publisher agent, and a notification
sensor and a heart rate sensor connected to a data hub
agent. Because very few applications provide remote real
device. This data hub is provided with a software agent
time monitoring for premature babies, we designed this
(the data hub agent) for sensed data preprocessing. To
minimize the effect of wireless signals around the
premature infant, as well as to benefit from the secure

978-1-4673-6516-1/15/$31.00 ©2015 IEEE 170

 2015 International Conference on Advances in Biomedical Engineering (ICABME)

infrastructure of the hospital network, the data hub is Ethernet to the local medical server. This will be in
connected via Ethernet to the local medical server at the particular tackled via the data hub (or data collector)
hospital. It is externally powered via the local router in the agent. The Arduino Uno microprocessor, which is based
room since it always listens for incoming data. The server on the ATmega328 chipset, was retained for our data hub.
will play the role of the publisher agent responsible for This is due to its availability in the market (with an
publishing the data into the hospital information system adequate Ethernet shield), its ease of programmability, its
and for making this data available through the Internet. It flexibility (6 Input signals, so our system can be extended
will also be responsible for notifying the monitoring and with additional sensors when needed), and its processing
control application (notification agent) of the caregivers speed (16 MHz Clock speed) [10].
and parents. Let us precise that this solution can be
E. The medical server
extended to encompass more sensors without any change
of its general architecture. Fig. 1 depicts the sensors used The medical server is a secure server inside the
in our solution. hospital provided with a distributed software component
(the medical server agent). The medical server agent (or
data publisher agent) will receive the sensed data from the
Arduino data hub, using UDP socket programming and
CoAP protocol. This agent will also publish the sensed
data into the hospital information system and will make
this data available over the Internet due to the use of CoAP
protocol. All these mechanisms constitute the first step
toward integrating our monitoring system within an IoT
wide solution. An overview about CoAP protocol is given
Figure 1 - Humidity, heart rate, and body temperature sensors and data in section III, and the detailed functionalities of the
hub device medical server and the mobile application are explained in
A. Heart Rate sensor section IV.
Heart rate is defined as the number of heartbeats per unit
of time, typically expressed as beats per minute (bpm). III. COAP USAGE FOR DATA PUBLISHING
Heart rate is one of the important parameters of In this section, the CoAP protocol is overviewed by
cardiovascular system. The heart rate used in NIMS verifying the reason behind using it in our proposed
system is an optical heart rate that uses the red and infra- system and by describing its reliability, freshness and
red frequencies to determine the percentage of hemoglobin security features. We will detail the overall process of our
in the blood. The newborns heart beat should be between neonatal incubator monitoring system in section IV.
100-160 bpm [8]. Lower heart rates are usually an
A. Reasons for choosing CoAP
indication knows as Bradycardia, while higher are known
as Tachycardia [9]. After being collected, the medical data should be
published over the Internet to be shared by other parties.
B. Temperature sensor These data can either be stored in a database server (within
The temperature sensor is used by taping it to the body the hospital information system) and the client can retrieve
skin. It measures the baby’s temperature rather than air the data using queries, or published using web service
temperature. The LM35 temperature sensor is more techniques. While our main aims are: to provide a flexible
advantageous for the baby since it directly senses 10 mv and extendable solution for any monitoring system, to
for every 1 degree; the maximum detectable temperature is integrate these collected data in the IoT in order to be
100 degrees which corresponds to a 1 volt at output (the shared by other applications and medical cloud servers for
output voltage is linear to the temperature by 10mV/0C). data aggregation and management, and to mask the
The normal temperature of the baby needs to be kept heterogeneity between different development platform and
between 36 and 37 degrees Celsius [8]. If the temperature software (regardless the programming language and the
goes too high or too low, then the organs can be damaged. operating system), we choose to publish our data using
web service techniques. In fact, web services are of two
C. Humidity sensor
types: SOAP and REST. SOAP defined a set of rules for
It measures the ratio of the amount of moisture in the data communication encapsulated in XML messages. The
air at a certain temperature to the maximum amount of SOAP server defines certain functions that can be invoked
moisture that the air can retain at the same temperature. by a client [11]. Dealing with real monitoring application
Most infant incubators have built in water reservoirs, pushes us to use the Rest architecture because it is a
humidity control modules that allow the medical staff to stateless protocol, with cashing capability, and therefore,
vary relative humidity as per need; this humidity is also faster than SOAP. Two application protocols use the Rest
responsible for providing warmth to the breaths and technique, the HTTP and the CoAP. Both of these
humidified air into the baby’s lungs. Relative humidity is protocols use a URI for accessing resources; the server is
expressed as a percentage (0 to 100). It should be about responsible of creating the resources while the client can
80% [8]. GET, PUT, POST or DELETE a resource. The basic
D. Data hub device with Ethernet shield difference between CoAP and HTTP resides in the
transport layer. While HTTP uses TCP for data
The sensed signals should be forwarded to a data hub transportation, COAP uses UDP. Thus, CoAP is faster and
device, with processing capabilities, in order to be requires less resource usage at network side as well as at
converted into preprocessed digital data and to be sent via

171
 2015 International Conference on Advances in Biomedical Engineering (ICABME)

the node (server/client) side (HTTP is more complicated

for constraint devices) [12]. CoAP also enables IP
multicasting, which complies with IoT group
communication needs. Furthermore, CoAP is provided
with additional retransmission mechanisms compensating
for UDP’ unreliability [13]. That is why CoAP is the most
adopted protocol for the IoT devices. Focusing on
reducing mobile’s battery consumption and providing real
time monitoring with the minimum delay, we used the
CoAP protocol for data publishing. CoAP permits the
Figure 2-General Process
transfer of any types of messages, starting from simple
value to embedded videos. That’s why it offers the A. Set up Phase
possibility to integrate any sensor in the network (web The first phase is the set up phase where we set a patient
camera, audio recording, localization, etc.). ID (unique identifier for each patient formed of 2 digits)
B. Reliability, Fresness & Security provided by CoAP for each Arduino data hub device and associated data hub
Biomedical data requires from the developer to ensure agent. When the Arduino data hub device is turned on for
reliability, freshness and security. In our solution we are the first time, its hub device agent sends its associated
providing reliability by the use of Confirmable CoAP patient ID to the medical server, i.e. the medical server
messages that are followed by an acknowledgment. In case agent. This medical server agent then creates a new
this message is lost, an automatic retransmission occurs resource specified by the patient ID and associated with
[11]. sub-resources for the temperature, humidity and heart rate.
CoAP offers the caching option which permits to reuse B. Data transmission
a prior fresh stored response without the need for a The data hub agent continuously receives sensed data
network request, thus reducing latency and network round from the sensors. It converts those collected raw data into
trips. The Max-Age option is set by the server indicating preprocessed digital data. Then, if those preprocessed data
the freshness of a stored value [12]. CoAP is secured differ from the previously collected ones, the data hub
using Datagram TLS (DTLS). It is designed with 4 level agent sends them to the medical server agent. A sent
of security: No security; Pre-shared Key: DTLS is enabled preprocessed data will be preceded with the letter T if it is
and there is a list of pre-shared keys [12] and each key a temperature, H if it is humidity data and HR if it reflects
includes a list of which nodes it can be used to the heart rate. As the heart beat is measured in bpm (beat
communicate with; Raw Public Key: DTLS is enabled and per minute), a one minute delay is performed before the
the device has an asymmetric key pair without a certificate generation of another corresponding value. Fig. 3 shows
(a raw public key) that is validated using an out-of-band an example of data sent for Patient number 01. The data
mechanism. The device also has an identity calculated server identifies the type of sent data (temperature, heart
from the public key and a list of identities of the nodes it rate or humidity) and sends a POST request to the CoAP
can communicate with; and Certificate: DTLS is enabled server. When an abnormal value is detected, a notification
and the device has an asymmetric key pair with certificate is sent to patient’s relative and medical staff application
that binds it to its Authority Name and is signed by some agents. The temperature should be between 36 and 37
common trust root [12]. We have integrated the DTLS in degree Celsius, the humidity should be 80 percent, and the
our proposed solution. We have also adopted the second heart rate should be between 100 and 160 bpms. Note that
level of security because the list of nodes is limited to the these values can change from one infant to another. They
patients’ relatives and medical staff in our neonatal are therefore configured on the Arduino data hub device.
monitoring system.
C. Mobile Application
IV. NIMS GENERAL PROCESS The application’s administrator will download for each
Fig. 2 depicts the general process of our NIMS system. user (infants’ relatives and the medical staff) the mobile
The medical server agent (instantiated in an hospital application where the right Patient ID has been set up. For
server) is implemented in Java. For the CoAP library, we example, a doctor can monitor many infants, thus the
used jcoap [14]. It will play the role of both: a data server administrator will set the Patient ID for all the infants
listening on port 2222 and receiving UDP packets from under his responsibility. This setting ensures the privacy of
Arduino data hub (its data hub agent), and generating the infant’s data where only authorized users can monitor
alarm events (notification agent), and a CoAP server the baby. Our mobile application relies on a CoAP based
listening on port 5684, for creating and managing application agent. Both application and CoAP agent have
resources (data publisher agent). been carried out on an Android mobile terminal.
Furthermore, we only developed this mobile
application for our proposed system testing purposes. That
is why we actually rely on the Google API for
authentication (this will obviously have to be changed for
a commercial version).
Thus, when the user opens the application for the first
time, he will be asked to enter his Gmail account
information. If he succeeded to login, he will be redirected

172
 2015 International Conference on Advances in Biomedical Engineering (ICABME)

to the main page of application where he can choose

which patient (i.e. infant) he wants to monitor. Then the
corresponding monitored value will be displayed on the
screen. In fact, the application agent is identifying the URI
(based on the Patient ID) and is sending 3 GET requests to
the CoAP server of the medical server agent for retrieving
the temperature, humidity and heart rate values.
Figure 4- Mobile Application for data monitoring
V. TESTING AND RESULT agents, as well as URIs for monitored data IDs, enables
our NIMS system to be integrated in an IoT wide strategy.
For testing purposes, we physically implemented this Fortunately, our system is not limited to neonatal
solution in a hospital, North Lebanon. Five Arduino’s monitoring application but can be extended to any other
were used in order to monitor 5 premature infants where application where a sensor network is collecting data and
two of them were twins. The Arduino chipsets were should published them over the Internet for further
connected to the hospital local server using an Ethernet analysis, or diagnostic. The tests conducted in an hospital
switch inside the room. The local server is connected to in Noth Lebanon showed that our system is fully
the Internet using a proxy server and a firewall to ensure functional and accurate, with an excellent acceptance level
security. The mobile’s application was downloaded on the from both involved medical staff and parents.
mobile’s phone of 4 parents, two doctors and two nurses. We are actually working on finalizing the required
The system was tested for one week. Fortunately, the data security mechanisms in CoAP, on enhancing the user
was accurately monitored (compared to the data acquired graphical interface, and on creating an Apple IOS
locally in the incubator) and the notification was received. application. We are also studying, with the medical staff
On one hand, we insured with the help of the nurses and the hospital administration, the extension of our
that the babies weren’t affected by the signals transmitted system with additional sensors and the voice message
by our system (mainly due to the use of cabled network sending to the babies. Our main objective is to help
and UDP small packets for data transmission). On the parents struggle the stress of having premature babies and
other hand, we collected the feedback of the parents and enjoy the experience of having new born coming to the
the medical staff. We deduced based on questionnaires world.
that parents are less stressed and felt more engaged and
involved in their infant’s care. Additional functionalities REFERNCES
were suggested by the parents like sending voice messages
[1] Billion Consumers worldwide, http://www.emarketer.com/Article/2-
to their babies and adding a camera into the incubator. Billion-Consumers-Worldwide-Smartphones-by-2016/1011694.
Regarding the doctors, they have praised the efforts done [2] G. V. Crosby, T. Ghosh, R. Murimi and C. A.Chin, “Wireless body
in this project and they have required more data to be area networks for healthcare: A survey”, International Journal of Ad hoc,
monitored like blood pressure and the oxygen. Moreover, Sensor & Ubiquitous Computing (IJASUC) Vol.3, No.3, June 2012.
[3] F. Bruno, J. A. Afonso, and R. Simões, "Vital signs monitoring and
the hospital’s director have encouraged the project and our management using mobile devices.”, In the proceeding of the
system, especially because it only costs about 60$ per Information IEEE 6th Iberian Conference, Systems and Technologies
patient, but he has insisted on finalizing the security part (CISTI), 2011.
of the CoAP server of the medical server agent. Fig. 4 [4] S. D. J McArthur, and E. M. Davidson, "Multi-agent systems for
diagnostic and condition monitoring applications." , In the proceeding of
presents a screen shot of the mobile application. the IEEE Power Engineering Society General Meeting, 2004.
[5] http://usatoday30.usatoday.com/news/health/wellness/story/2012-08-
23/newborn-icu-webcams/57254376/1.
[6] C. Lee. Ventola, "Mobile devices and apps for health care
professionals: uses and benefits." Pharmacy and Therapeutics 39 (5) pp.
356, 2014.
[7] Incubator, Infant, Core medical equipment information, World Health
Organization,
Figure 3- Data transmission from Arduino data hub agent to the http://www.who.int/medical_devices/innovation/incubator_infant.pdf
medical server agent [8] D. Ruscansky, D. Vecchione, R. Krishnan, and M. Zenouzi, “Design
a low cost neonatal incubator”, Electronics and Mechanical Department
VI. CONCLUSION & PERSPECTIVE of Wentworth Institute of Technology, Boston, 2015.
[9] L. Welborn, L. Rice, R. Hannallah, L. Broadman, U. Ruttimann, and
In this paper, we presented a distributed multi-agent R. Fink, "Postoperative apnea in former preterm infants: prospective
based neonatal incubator monitoring system (NIMS) comparison of spinal and general anesthesia", Anesthesiology 72, vol. 5,
dedicated for premature babies. Our system is in particular pp. 838-842, 1990.
[10] http://www.arduino.cc/
made up of distributed software agents (within incubator’s [11] M. Muehlen, J. Nickerson, and K. Swenson, "Developing web
data hubs, medical servers and mobile terminals of services choreography standards—the case of REST vs. SOAP."
medical staff and parents) based on Constraint Application Decision Support Systems 40, vol. 1, pp. 9-29, 2005.
Protocol (CoAP). The use of CoAP within our distributed [12] Z. Shelby , K. Hartke, and C. Bormann, Constrained Application
Protocol (CoAP), draft-ietf-core-coap-18, June 2013.
software [13]M. Kovatsch, S. Duquennoy, and A. Dunkels. "A low-power CoAP
for Contiki." In the proceeding of IEEE 8th International Conference “
Mobile Adhoc and Sensor Systems (MASS)”, 2011.
[14] https://github.com/dapaulid/JCoAP.

173