Big Data Model Simulation on a Graph Database for Surveillance in Wireless

Multimedia Sensor Networks

Cihan Küçükkeçecia , Adnan Yazıcıa

a Middle East Technical University, Department of Computer Engineering, Ankara, Turkey
arXiv:1708.03878v1 [cs.DB] 13 Aug 2017

Abstract
Sensors are present in various forms all around the world such as mobile phones, surveillance cameras, smart tele-
visions, intelligent refrigerators and blood pressure monitors. Usually, most of the sensors are a part of some other
system with similar sensors that compose a network. One of such networks is composed of millions of sensors connect
to the Internet which is called Internet of things (IoT). With the advances in wireless communication technologies,
multimedia sensors and their networks are expected to be major components in IoT. Many studies have already been
done on wireless multimedia sensor networks in diverse domains like fire detection, city surveillance, early warning
systems, etc. All those applications position sensor nodes and collect their data for a long time period with real-time
data flow, which is considered as big data. Big data may be structured or unstructured and needs to be stored for fur-
ther processing and analyzing. Analyzing multimedia big data is a challenging task requiring a high-level modeling to
efficiently extract valuable information/knowledge from data. In this study, we propose a big database model based on
graph database model for handling data generated by wireless multimedia sensor networks. We introduce a simulator
to generate synthetic data and store and query big data using graph model as a big database. For this purpose, we
evaluate the well-known graph-based NoSQL databases, Neo4j and OrientDB, and a relational database, MySQL. We
have run a number of query experiments on our implemented simulator to show that which database system(s) for
surveillance in wireless multimedia sensor networks is efficient and scalable.
Keywords: Internet of things (IoT), big graph databases, NoSQL databases, wireless multimedia sensor networks,
simulator

1. Introduction pected to be one of the major components in the Internet

of things (IoT).
A wireless multimedia sensor network (WMSN) is a
A typical application for a WMSN is a surveillance
distributed wireless network that consists of a set of mul-
system or a monitoring system. Smart city surveillance
timedia sensor nodes, which are connected to each other
cameras with 7/24 recording, or one million sensor nodes
or connected to leading gateways. Nowadays, smart de-
reporting meteorological data produce data in various for-
vices such as mobile phones, smart televisions, and smart
mats as video, audio, and text [1]. All that huge structured
watches are equipped with sensors and network connec-
or unstructured data is considered as big data, which is
tions. Hence, with the advances in wireless communi-
defined by a number of Vs; Volume, Velocity, Variety, Ve-
cation technologies, multimedia sensor networks are ex-
racity, and Value. Min et al. [2] present a comprehen-
sive survey of big data and they identify that “defining
Email addresses:
the structural model of big data” is a fundamental prob-
[email protected] (Cihan lem. Fusing and analyzing big data are challenging tasks
Küçükkeçeci), [email protected] (Adnan Yazıcı) and there are many research studies that are related to big

Preprint submitted to August 15, 2017

data from different points of view in recent years. As context, to the best of our knowledge, there has not been
pointed out by many researchers, relational database man- any applicable graph-based big data model for WMSNs
agement systems (RDBMS) are inadequate for efficiently based on a graph database yet.
handling big data; therefore NoSQL database systems are This article is organized as follows: next two sections
mostly utilized [3, 4, 5]. There are four main types of provide background information and related work. Then
NoSQL databases, which are key-value store (e.g. Ama- we introduce our real WSN system to give technical de-
zon’s Simple DB), big table (e.g. Apache Cassandra), tails of our deployment. Section 5 is the proposed graph-
document store (e.g. MongoDB) and graph-based model based big model explanation and Section 6 presents the
(e.g. Neo4j) [6]. prototype implementation of our model. Simulation in-
Graph databases consist of nodes and edges (rela- frastructure of wireless multimedia sensor networks is
tions between nodes) which store data as properties. given in Section 7. Section 8 illustrates a case study in
Graph databases are very efficient and convenient to han- the military surveillance domain and Section 9 presents
dle social networks, fraud detection, graph-based oper- the experimental results and evaluations. Finally, conclu-
ations, real-time recommendations and hierarchical re- sions are drawn in Section 10.
lations. While storage of the big data is an important
task, processing the streaming data and taking action for 2. Background
mission-critical applications are crucial. Arkady et al. [7]
state that analyzing and extracting the valuable data from 2.1. Internet of Things
dirty raw data is an important research topic. In order to The Internet of Things (IoT) is used by Kevin Ashton
process that kind of data, we need to identify the data flow. in 1999 [8] and it roughly means that the Things use the
In this paper, we propose to use a graph-based model Internet instead of Humans. The sensors, RFIDs, and nan-
for handling big data generated from surveillance appli- otechnology help this mission to be accomplished by tak-
cations in wireless multimedia sensor networks. For this ing away the need for human-entered data.
reason, we propose a graph-based model as a generic Pankesh et al.[9] propose a domain model for IoT to
model to be used for different surveillance applications. make a common understanding. To define the model, they
Big sensor data is stored in a graph database for the pur- reference to the real world applications and summarize
pose of advanced analytics, such as data mining, predic- under three headings; Intermittent Sensing, Regular Data
tion, and statistics. Our graph model represents both the Collection, and Sense-Compute-Actuate loops.
data flow among the nodes and wireless multimedia sen- As parallel to Sense-Compute-Actuate cycle, Sensor-
sor network topology. The applicability of our solution as-a-Service (Senaas) notion is defined by Sarfraz et al.
is illustrated with a prototype implementation including [10]. They virtualized the sensors as services by an ab-
simulation of synthetic data. A case study in the military straction on technical details of sensors. They trigger ser-
surveillance domain is simulated and several experiments vices with an event in the sensor and compute it to reply
are done to measure the efficiency of our solution. Simu- an action. Their IoT virtualization framework is validated
lation results show that our proposed multimedia wireless by a case study.
sensor network model is applicable in large-scale real-life Atzori et al. [11] prepare a comprehensive survey about
application scenarios. IoT. They identify the enabling technologies as sens-
The contribution of our study is to store the multime- ing, identification and communication systems like RFID,
dia sensor network data in a well-defined graph-based big WiFi, and sensors. In addition, middleware applications
database model. The big data stored in an open-source using Service Oriented Architecture (SOA) are important
graph database can be used for analyzing, filtering, aggre- for data distribution. In the end, they list open issues such
gating and correlating big data. A simulation infrastruc- as privacy, addressing of things and non-standardized ap-
ture is implemented for simulating multimedia wireless plications for the future.
sensor networks to run a number of complex experimen- IoT is fully connected to sensor technology and the re-
tal queries. Although there have been some related studies searches about sensor networks directly or indirectly im-
in literature about the surveillance systems in the big data prove the IoT. Hong et al. [12] propose an approach to

2
IoT using IP-based wireless sensor networks. They real- Volume is the quantity of stored data. Velocity is the
ize that IoT probably has the same problems that Internet speed of data generation or processing. Variety is the type
itself had in the past. So, they identify problems like IPv6 and structure of the data. Value is the importance of in-
adaptation, mobility, web enablement, global time syn- formation that data provides. Veracity is the variation in
chronization, and security. They also share evaluation re- quality of data and inconsistency and uncertainty of data.
sults of an implementation of their proposed SNAIL (Sen- The survey paper [2] points the relation between IoT
sor Networks for an All-IP World) platform. and big data. For example, jet aircraft engines pro-
duce one terabyte of data per flight using various sen-
2.2. Sensor Networks sors. Think about a huge number of flights in a day all
around the world and then you can have really big data.
A set of sensors called sensor nodes connected to each
HP prepared a business-value white paper related to big
other or connected to leading gateways is simply called
data. According to the paper, one trillion sensors, roughly
a sensor network (SN). If sensors have capability of col-
150 sensors for every person will be existed by 2030. The
lecting multimedia data and have a communication infras-
generated data will be mostly unstructured data and the
tructure among the sensors, it is called multimedia sensor
value of it depends on how the information is extracted
network (MSN). If sensors are connected to each other
from the data. As the number of sensors increases,much
using wireless technology then it is a wireless multimedia
more storage and processing will be required. And all of
sensor network (WMSN).
these create some new challenging issues.
Akyıldız et al. [13] discuss the state of the art of re-
Many papers [6, 4], state that the relational database
search on WMSNs as well as the challenges. The chal-
management systems (RDBMS) are inadequate for big
lenges related to WSN deployment configurations are
data and NoSQL database systems are a solution at least
summarized by Perera et al. [14] in their research. An-
for time being.
other survey paper [15] enlists the challenging issues to
design middleware systems for WSN. Some of the iden- 2.4. NoSQL Graph Databases
tified challenges are as follows: data fusion, resource
Moniruzzaman et al. [3] evaluate NoSQL databases in
management, scalability and network topology, security,
the aspect of big data analytics. In their survey, they en-
Quality of Service and limited power.
list different types of NoSQL databases according to char-
Peng et al. [16] propose a wireless sensor network in
acteristics (features and benefits of NoSQL databases),
which sink node is replaced by a cloud. They called sink
classification (key-value, document, column-based and
point instead of gateway. Their simulation results show
graph); and evaluation with a matrix on the basis of few
that cloud based architecture increases the WSN perfor-
attributes like design, integrity, indexing, distribution, and
mance.
system.
Arati et al. focus on the information retrieval from
There are four types of NoSQL databases which are
sensor networks and propose a hybrid protocol which is
key-value store (e.g. Amazon’s Simple DB), big table
called APTEEN [17]. They make experiments by exe-
databases (e.g. Apache Cassandra), document-oriented
cuting queries to show that proposed protocol performs
(e.g. MongoDB) and graph databases (e.g. Neo4j). Graph
better.
databases [19] consist of nodes and edges (relations be-
From the database point of view, Ramesh et al. [18]
tween nodes) which store data as properties. Most of
define a database layer on top of sensor network so that
graph databases provide the capability to label nodes and
a database query is mapped to traversing sensor nodes in
edges. NoSQL Graph databases provide many ways to
the WSN.
query data;

2.3. Big Data • User interface via SQL-like query language (Cypher
for Neo4j, SQL for OrientDB)
The buzz word of the recent years, Big Data, defined
by a number of Vs; Volume, Velocity, Variety, Value and • User interface via Graph visualization to interact
Veracity as shown in Figure 1. with the nodes and edges

3
 Figure 1: 5 Vs of Big Data

• Application program interface (API) to programmat- layers; application, query and analysis, and data storage.
ically connect to database In data storage layer, they used a NoSQL database as we
do. But they use HBase which is a column-oriented key-
Unfortunately, there is not any standardized way of query- value store, we use graph database which is better for re-
ing, so that you have to write database specific queries lational analytics.
every time. There is an open-source framework Apache Christine et al. [23] discuss big data with spatial data
TinkerPop to provide graph computing capabilities for received from wireless sensors using real life scenarios.
graph databases. Gremlin is a part of the TinkerPop to One of the scenarios is related to smart cities which is
traverse the graphs. And by the support of most of the similar to surveillance domain. They propose a scalable
graph databases, any gremlin query can be written once solution using Hadoop and HBase NoSQL database to
and works on every graph database. prototype a platform for storage and processing wireless
Graph databases are generally preferred to handle so- data. We also design and implement a simulation proto-
cial networks, fraud detection, graph-based operations, type from storage layer to analytics layer and more im-
real-time recommendations and hierarchical relations. portantly, we propose a grah based data model on top of
that architecture.
3. Related Work Renzo et al. [24] present a survey paper on graph
database models. They compare graph database models
Over the years, various methods have been used for with the other database models, i.e. a relational model. In
wireless sensor networks data representation and manage- this paper, we also compare well-known graph database
ment ([20, 21]). Our approach is different basically by models with the relational database model. Furthmore
the aspects of big data, graph database storage, and our we perform queries to benchmark the performance of
unique graph data model. databases.
Yang et al. [22] present a service platform which is Another survey paper is a written by Felemban [25]
called Wiki-Health. They have designed platform in three which is about border surveillance. His research en-

4
lists the literature for experimenting work done in border 4. Real WSN System
surveillance and intrusion detection using the technology
of WSN. Our research differs from the existing works by Our reference WMSN system is composed of wireless
employing a graph based approach for surveillance do- multimedia sensors and some scalar sensors. The system
main and focusing on the simulation of the big data. is designed as a multi-tier automated surveillance system.
The first layer is the sensing layer with scalar sensors in-
PipeNet [26] is a multi-layered wireless sensor net- cluding acoustic, seismic, and PIR. The second layer is
works application focused on pipeline monitoring. Sys- triggered by first layer. Multimedia sensors like camera
tem aims to detect the leaks and other anomalies in water and microphone are used capture video and audio. After
pipelines. They have used various types of sensors like applying the fusion, object type and location of the sen-
pressure, pH and ultrasonic sensors on top of Intel Moto sor is extracted to be provided to the next layer, which is
platform. Our sensor nodes are built on Rasberry Pi plat- called the sink layer. The sink layer provides the capabil-
form and have seismic, acoustic, and PIR sensors but also ity to do analytics on all collected and generated informa-
a multimedia camera. A camera needs further analysis tion in the network.
like image processing and feature extraction. Their multi- The overall architecture is given in Figure 2. Sen-
layer architecture is similar to our prototype but in another sor nodes are connected to the gateway nodes via Zig-
domain with different sensors and different analytical ap- Bee (IEEE 802.15.4) interfaces. A special thread for se-
proaches. They try to analyze the collected multi-modal rial messaging is developed to send the events from sen-
data for detection of the leaks. But we try to identify ob- sor nodes to the gateway. The gateway prepares XMPP
jects and track their movement. On the other hand, we messages using the gathered events coming from leading
approach our sensor data as big data. nodes via broadcast messages. Those XMPP messages
Suvendu Kumar et al. [27] propose an analytic archi- are transferred to the sink node. The XMPP messages and
tecture for big data to detect intruders using camera sen- multimedia data is transfered over IP based (IEEE 802.11)
sors. Our work differs by using additional scalar sensors connection.
like acoustic and seismic sensors but also proposed graph The hardware of our sensor node is based on a Rasp-
based data model. berry Pi (RPi) 512-MB Model B board and includes the

Figure 2: Real WMSN System Architecture

5
following hardware components: the base station and there are a number of clusters con-
nected to the sink. Each cluster consists of a gateway,
• ARM1176 700MHz processor, which can be called the cluster head, and a set of sensor
• Graphical processing unit (GPU), nodes.
• 512 MB SDRAM shared with GPU, The data flow occurs from the sensor nodes to gateways
and from gateways to the other gateways (multi hop) and
• SD card slot for on board storage, finally to the sink node. Each sensor node holds a set of
• On board 10/100 Mb Ethernet port, data sensed by the sensors and camera of the node. Sen-
• 2x USB 2.0 ports, sor nodes include embedded programs for handling cor-
relation, transformation, and aggregation on the raw data,
• 1 CSI input connector for the camera module, which is called first level fusion. The sensor fused data
• Video and audio outputs, are reported to the leading gateway by all of its connected
• GPIO ports, sensor nodes.
The gateway waits for all sensor nodes to report. When
• 5V 700-mA microUSB power requirement.
all reports are ready, the gateway applies an aggregation
The following list is the components installed on a node or filtering on the received data. The second-level fu-
in our WSN system to fulfill its functions: sion is done at this point and the output of the fusion is
a summary of that cluster. The gateway fused data are
• Motion sensor (PIR), forwarded to the sink node for a final decision.
• Acoustic sensor (AS), Similar to the second-level fusion, the sink waits for
all gateways’ fused data. By applying some patterns to
• Vibration sensor (VS),
detect anomalies or other kinds of analysis are done at the
• Raspicam camera module, third level fusion. The output of the last level fusion is an
• Xbee ZigBee (IEEE 802.15.4) adapter, action like triggering an alarm or a notification message
to another system.
• 4400-mAh 5V 1A power bank,
Figure 3 shows the first level fusion graph model from
• Microphone, raw data collection to fusion. The sensor node is respon-
• Wi-Fi (IEEE 802.11) dongle, sible for fusion at this level. The input is the raw data and
• XMPP client software

Sensor to sensor communication, as well as gateway

to sink communication, is completed via ZigBee inter-
faces. Nodes are equipped with low-bandwidth radio de-
vices using IEEE 802.15.4 (ZigBee) standard and ZigBee
provides a line of sight up to of 1500 m. at outdoor con-
ditions and 250 Kbps at most.
In our real WSN system, sensor node and gateway roles
are all predefined, there is not any dynamic gateway selec-
tion. Because different roles may need different kinds of
hardware components.

5. The Graph-Based Big Data Model

Our graph-based big data model is built on the multi- Figure 3: First Level Fusion model (SN:Sensor Node, SRD:Sensor Raw
media sensor networks topology. There is a sink node in Data, SFD:Sensor Fused Data)

6
Figure 4: Second Level Fusion model (GW:Gateway, SN:Sensor Node,
SRD:Sensor Raw Data, SFD: Sensor Fused Data, GFD:Gateway Fused Figure 5: Third Level Fusion model (GW:Gateway, GFD:Gateway
Data) Fused Data, SINKFD:Sink Fused Data)

the output is sensor fused data. Last collected raw data sen as default storage system. Other subsections show the
and last fused fusion data are explicitly pointed for the implementation of the data model with the detailed design
purpose of the direct link. of generic infrastructure and simulation infrastructure.
Figure 4 shows the second level fusion graph model to
identify the relations between sensor nodes and the gate- 6.1. Graph Database Selection
way. The gateway is responsible for the fusion at this
level. The input is all sensor fused data coming from sen- Our research includes applying the currently available
sor nodes and the output is gateway fused data which can databases to our graph-based big data model. Salem et
be filtered, aggregated or transformed data. al. [28] compare a set of databases like Cougar and
Figure 5 shows the third level fusion graph model TinyDB. Li-Yung Ho et al. [29] propose a distributed
which is executed at the sink to aggregate all data fused graph database based on an open-source graph database
from gateways. The input is the all fused data coming which is called Neo4j. The options are limited if you are
from gateways and the output is sink fused data which looking for a graph database. Neo4j, Titan, and OrientDB
cause actions like triggering an alarm or notifying the op- are featured open-source graph databases.
erators. Neo4j is a well-known graph database and used by
many researchers. In addition, Neo4j is relatively eas-
6. Prototype Implementation for Proposed Model ier to be used rapidly by developing some small pieces of
code. Spring Framework support is really helpful to put
We have already implemented the proposed graph things together very fast.
model using OrientDB Graph Database and developed a Titan is another open-source option for a graph
simulator to generate synthetic data. In the following sub- database but its development is stopped and discontinued
section, we describe why OrientDB graph database is cho- in early 2015. Therefore we did not prefer to utilize Titan.

7
 (GatewayFusedData) and SinkFusedData. Edge types
Table 1: Compare OrientDB and Neo4j Community Editions
are; Lead, Collect, LastCollected, Next, Fusion, FusedBy,
Feature OrientDB Neo4j
LastFusion, Reported and Forwarded. The edge types are
Graph Database Yes Yes defined for the usage between specific nodes. Table 2 lists
TinkerPop Standard Compliance Yes Yes the edge types in our graph database.
ACID Transaction Yes Yes Each sensor node has the capability to hold temporar-
Unique Constraints Yes Yes ily a set of data which are sensed by sensors like PIR,
Fulltext Support Yes Yes seismic, acoustic and camera. That capability is provided
Spatial Support Yes Yes by an in-memory database. For the fusion at the sensor
Java Hooks Yes Yes node level, this in-memory database is used as a cache to
Record Level Security Yes No analyze the changes in scalar sensors and provide some
User and Role Security Yes No additional data to the first level fusion. The algorithm of
the first level fusion is shown in Algorithm 1. The fu-
SQL Yes No
sion result is stored in the fusedData including the video,
Dynamic Triggers Yes No
silhouette, foreground and low level features.
Custom Data Types Yes No
Additional Constraint Types Yes No
Indexes on Multiple Properties Yes No Algorithm 1: Sample first level fusion algorithm exe-
cuted on sensor nodes
Different Schema Modes Yes No
1 Function firstLF(PIR, seismic, acoustic,threshold)
Multi-Master Replication Yes No Input : Boolean PIR identifies if there is a
Sharding Yes No movement or not, two integers seismic
Elastic Scalability Yes No and acoustic values are scalar data
Server-Side Functions Yes No sensed by the node, threshold is used to
Embeddable with No Restrictions Yes No identify that there is an object with
Sequences Yes No enough sound and vibration on sensor
node
Output: f usedData is the fusion data involving
OrientDB is another open-source graph database which the multimedia data
is not as popular as Neo4j for now but has many advan- 2 initialize f usedData
tages over it. Table 1 shows the comparison of OrientDB 3 if PIR = true and seismic ≥ T HRESHOLD and
and Neo4j Community Editions which is provided by the acoustic ≥ threshold then
official website of OrientDB. From all those compared 4 f usedData.video = startVideoRecording() ;
features, ”Multi-Master Replication”, ”SQL” and ”Elastic 5 f usedData. f rame =
Scalability with Zero Configuration” are the most impor- selectFrame( f usedData.video);
tant features for us to choose OrientDB. 6 f usedData. f rmFeatures =
findLowLevelFeatures( f usedData. f rame);
6.2. Data Model Implementation 7 f usedData. f oreground =
selectForeground( f usedData. f rame);
Nodes (vertices) and relations (edges) are defined in
8 f usedData. f gndFeatures =
graph databases to store data. Compared to the traditional
findLowLevelFeatures( f usedData. f oreground)
RDBMS approach, every row in a table is replaced with
9 f usedData.silhouette =
a node and its properties. There are edges to represent
extractSilhouette( f usedData. f oreground);
cross-table references.
At the first step, we define the node and edge types. 10 return f usedData;
Node types are; Sink, Gateway, SensorNode, SensorRaw-
Data, SnFusedData (SensorFusedData), GwFusedData

8
 Table 2: Node and Edge Types Algorithm 2: Sample second level fusion algorithm
Edge Type From Node To Node executed on gateways
Lead Gateway SensorNode 1 Function secondLF( f usedDataList,threshold)
Lead Gateway Gateway Input : The list of f usedData reported by
Lead Sink Gateway leading sensor nodes, threshold is used
Collect SensorNode SensorRawData to identify the difference between two
LastCollected SensorNode SensorRawData scalar value
LastFusion SensorNode SnFusedData Output: Filtered list of f usedData by
duplication removal
LastFusion Gateway GwFusedData
2 initialize an empty array f ilteredDataList
Next SensorRawData SensorRawData 3 for i = 0 to f usedDataList.size do
Next SnFusedData SnFusedData 4 current = f usedDataList[i];
Fusion SensorRawData SnFusedData 5 previous = previous element of current;
Fusion SnFusedData GwFusedData 6 di f f = current.acoustic - previous.acoustic;
Fusion GwFusedData SinkFusedData 7 di f f Rate = current.acoustic *
FusedBy SnFusedData SensorNode threshold/100;
FusedBy GwFusedData Gateway 8 if di f f < di f f Rate then
FusedBy SinkFusedData Sink 9 mark current as duplicate and drop
Reported SnFusedData Gateway 10 else
11 add current to f ilteredDataList
Forwarded GwFusedData Gateway
Forwarded GwFusedData Sink 12 return f ilteredDataList

Sensor nodes apply the first level fusion and reports the
output fusedData to leading gateway. The gateway ap- Algorithm 3: Sample third level fusion algorithm ex-
plies the second level fusion as in Algorithm 2. The pur- ecuted on the sink
pose of fusion at the gateway is a refinement of the sensor 1 Function thirdLF( f ilteredDataList,threshold)
fused data before sending to the sink node. As a sample Input : The list of f usedData reported by
refinement algorithm, removing the duplications and nor- reporting gateways, threshold is used to
malizing the data according to scalar data can be written. identify the difference between two
The output of the second level fusion is reported to the scalar value
sink node. As the final decision maker, the sink correlates Output: Actions generated by reported
the concepts forwarded by gateways and decides that if an f usedData list
action is necessary or not. If an action is required, notifi- 2 initialize an empty array actionList
cation of the operator is triggered as given in Algorithm 3. 3 for i = 0 to f ilteredDataList.size do
All those three fusion algorithms are sample algorithms to 4 current = f usedDataList[i];
provide the proof of concept execution of all phases of the 5 previous = previous element of current;
simulated environment. 6 if current.acoustic > threshold and
All those three fusion algorithms are sample algorithms previous.acoustic > threshold then
to provide the proof of concept execution of all phases of 7 action = createAction(current);
the simulated environment. 8 add action to actionList;
9 notify operator using action;
6.3. Generic Infrastructure
10 return actionList
We have developed the whole system by using Java
1.8 as maven projects to use Apache Maven as the de-

9
 cluster has 4 sensor nodes at one side and there are 3 clus-
Table 3: Defined Message Queues
ters in an edge of the total area. So, we have 120x120
Queue Name Process Queue Element
sized grid layout for our simulation.
Scalar Data Collected Raw Data SensorRawData
Fused Level 1 and 2 Fusion SnFusedData 7.2. Data Flow Simulation
Forward Forwarding to Sink GwFusedData Data flow simulation is started with Raw Data Gen-
Action Level 3 Fusion SinkFusedData erator which produces synthetic data with position, PIR,
seismic and acoustic information. These data are sensed
by sensor nodes which are close to the position. A sen-
pendency management framework. To develop the ap- sor raw data object is created for each sensor node with
plication of our research project beyond OrientDB, we the calculated sensor data according to the distance to the
have designed a generic infrastructure which can be eas- position of the raw data created by Raw Data Generator.
ily adopted to other database systems. To achieve that, we Then the generic infrastructure explained in the previous
have developed business logic without any dependency to section is used to simulate all processes of the data flow.
OrientDB.
There are two managers called DataManager and Net- 7.3. Simulator
workManager. DataManager defines the necessary inter-
Currently, we adopt our simulation infrastructure on
faces to populate data for the underlying database. Net-
OrientDB, Neo4j which is the main rival of OrientDB and
workManager has the business logic to construct network
MySQL for the relational to graph database comparison.
topology and defines the necessary interfaces to create
Every simulation is replicated to all three databases simul-
network entities for the underlying database.
taneously so that the same test environment is provided.
As there is a data flow between sensor nodes and gate-
Figure 8 is the screenshot of our data simulator to
ways and sink, in order to cope with the bottleneck of
generate the network topology in Figure 6. “(Re)Create
high throughput of streaming data, we position Apache
Database” button cleans the database and generates sink,
ActiveMQ message queues between each process. We
gateways and sensor nodes according to the given param-
define message queues as seen on Table 3 below. Fusion
eter related to node count and cluster count.
logic is developed on top of messaging queues and there
“Start simulation” button starts simulation by sending
are 3 business logic handlers; Level1and2Fusion, Gate-
an entity from one of the edges of the area. Then, the
wayForward and Level3Fusion.

7. Simulation

To develop and experiment the graph data model, we

have developed a simulator which is mainly focused on
data simulation but also supports network topology simu-
lation.

7.1. Network Topology Simulation

For our network topology, we assume that nodes are
distributed with a grid layout to the simulation area which
is assumed to be square. Figure 6 shows a visualization
of 16 sensor nodes in each cluster with a gateway in the
middle and 9 gateways in total with Sink in the center.
Between two sensor nodes, the distance is 10 units. Each Figure 6: Simulated Network Topology

10
 entity moves randomly according to its speed and simula-
tion ends when the entity moves out of the area. Possible
moves are going to north, south, east, or west and don’t
move.
It is possible to run parallel simulations by clicking the
button at any time. And if “Repeat Simulation” check-
box is checked, a new simulation is automatically started
when current simulation ends.
There are several Entity types to simulate. Each en-
tity has its own speed, acoustic and seismic values. Ta-
ble 4 show each type and its simulation values. “En-
tity Speed” selection combobox decreases or increases the
speed value of the simulating entity. Acoustic and seismic
values are defined by the selection of ”Entity Type” com-
bobox and according to entity’s distance from the sensor
node, those values are recalculated to degrade its effect on
the sensor node.
Assume that simulation put an “Animal” entity at
(37,120) position. The entity is between the sensor nodes
at (30,120) and (40,120). Table 5 and Table 6 show the
Figure 7: Data Flow Simulation sensed values between those two nodes for Animal Type.
Another important capability of the simulation tool is
Event generation. We can identify event types and gener-
ate it at any time while the simulation is running. At first
stage, we have 2 types of events.
• Attack: As seen in Figure 6, operational base is lo-
cated at north. If something comes from south and
directly moves toward the base, this movement is an
Attack event for us.
• Smuggling: If a group of human is moving together
with a group of animal and they are coming from
west and going in the direction of south-east, this
movement is smuggling event for us.
When “Start Event” button is pressed, selected event is

Table 4: Simulated Entity Types

Entity Type Speed Acoustic Seismic
Human 1 20 10
Animal 2 40 20
Figure 8: Data Simulator
Vehicle 4 70 80
GroupOfHuman 1 60 30
GroupOfAnimal 2 80 60

11
 Table 5: Sensed Values at Node(30,10)
ANIMAL 30,10 31,10 32,10 33,10 34,10 35,10 36,10 37,10 38,10 39,10 40,10
Acoustic 20 18 16 14 12 10 8 6 4 2 0
Seismic 40 36 32 28 24 20 16 12 8 4 0

Table 6: Sensed Values at Node(40,10)

ANIMAL 30,10 31,10 32,10 33,10 34,10 35,10 36,10 37,10 38,10 39,10 40,10
Acoustic 0 2 4 6 8 10 12 14 16 18 20
Seismic 0 4 8 12 16 20 24 28 32 36 40

started to be simulated. Additional event types can be a concept of the moving object and maybe even a silhou-
added for further analysis. ette. Fusion output is reported to the gateway which is the
leading node of that district.
8. A Case Study: Surveillance Application
A gateway is connected to a set of sensors and cam-
Surveillance systems need robust and scalable infras- eras. The described operations are done for all leaded
tructure. To achieve that, all data flow and data itself are sensors so that the gateway receives many concepts and
needed to be analyzed and modelled. silhouettes. By applying some algorithms like filtering the
A set of sensors and video cameras are needed to mon- duplicate concepts or aggregating them to normalize the
itor the whole city. Assume that, we cluster the sensors received data, the gateway accomplishes the second level
and cameras according to the districts and each cluster fusion. After fusion, we have more accurate concepts and
forwards sensed data to the HQ (Head Quarter) which is silhouettes provided by many sensors. Fusion output is
the operation center. At the HQ, an alarm is triggered, or forwarded to the HQ (Headquarter) for a final decision.
a notification is sent to the officers to early detection of
violence. As there are many districts in a city, there are many
Sensor types can be seismic, acoustic and PIR (Pas- gateways which are far away from the HQ and not di-
sive Infrared) which are types of scalar sensors. In ad- rectly connected to it. In that case, data is forwarded over
dition to them, video cameras or thermal cameras can be other gateways. At the HQ, the received data from all dis-
added to critical locations. As the default, cameras are tricts are analyzed, aggregated or filtered for the purpose
switched off. According to the sensed information from of detecting anomalies or finding some patterns, which is
scalar sensors, the predefined conditions can be extracted called the third level fusion. At the end, the fusion comes
using rule-based approaches. The motion or environmen- to the conclusion and an alarm is triggered to the officers
tal change may be detected and interpreted to activate the or the external system of the armed forces is notified.
camera by providing a rough prediction of the moving ob-
ject.
We categorize the moving object as; Animal, Human,
and Vehicle. The data collected from scalar sensors are
analyzed to guess the category of the objects according to Table 7: Sample Thresholds To Identify Objects
predefined thresholds (Table 7). Object Type PIR Seismic (Hz) Acoustic (dB)
After activation of the camera by analyzing scalar sen-
Animal True 5 - 20 5 - 30
sor data, video and audio streams from the camera are
Human True 21 - 55 31 - 50
started to be processed. That processing in the sensor
node is called the first level fusion. After fusion, we have Vehicle True >35 >50

12
9. Experimental Work

We setup a test environment to make some experiments

on our graph model. Test environment specifications are;

• Intel i7-4710HQ Quad Core CPU

• 16 GB DDR3 RAM

• 240 GB SSD Storage

• 4 GB NVIDIA 860GTX GPU

Test environment has three database systems which are;

• OrientDB v2.2.5 (Graph database)

• Neo4j v2.3.2 (Graph database)

• MySQL v5.7.1 (Relational database)

We have simulated a multimedia wireless sensor net-

work with synthetic raw data. Sensor nodes are placed in
a square shaped area. Gateways are located in the center
of each group of sensor nodes. The sink node is placed in
Figure 9: Relational Database Schema
the center of the whole area as shown in Figure 6.
There are 25 million sensor nodes which are leaded by
2,500 gateways. There is sink node to collect all data in MATCH ( b : f u s e d d a t a ) −[: f u s e d b y ]−>( s d : s e n s o r n o d e ) WHERE
our simulation environment. Each gateway leads 10,000 b . c o n c e p t = ”Human” AND b . w e i g h t >0.90 RETURN b .
c o n c e p t , b . w e i g h t , b . f u s i o n D a t e , s d . indexX , s d .
sensor nodes. indexY
Relational SQL Query:
9.1. Comparison to Relational Data Model SELECT b . c o n c e p t , b . w e i g h t , a . f u s i o n D a t e , s d . i n d e x x ,
s d . i n d e x y FROM s i n k f u s e d d a t a a , g a t e w a y f u s e d d a t a
This experiment is done on all three databases installed g , s e n s o r f u s e d d a t a b , s e n s o r n o d e s d WHERE a .
c o n c e p t = ’ V e h i c l e ’ AND a . w e i g h t >0.90 AND a . i d =
in our test environment. We have run many queries on g . f u s i o n AND g . i d = b . f u s i o n AND b . f u s e d b y = s d .
both our graph data model and relational data model. Fig- i d ORDER BY a . f u s i o n D a t e ASC
ure 9 shows the relational database representative of our
graph data model on MySQL.
9.1.2. Video Based Query
Below queries are randomly selected sample queries
and Table IV shows the test results of the experiment. This query finds the possible explosions by identifying
continued high volume around the surveillance area with
9.1.1. Concepts Based Query their recorded video paths and video duration. The value
bigger than 15 is assumed to be a high volume sound.
This query finds the specific type of objects with the
highest probability of its detection time and location. OrientDB Query:
SELECT i n ( ” c o l l e c t ” ) . name [ 0 ] , i n ( ” c o l l e c t ” ) . indexX [ 0 ] ,
OrientDB Query: i n ( ” c o l l e c t ” ) . indexY [ 0 ] , a c o u s t i c , o u t ( ” v i d e o ” ) .
SELECT c o n c e p t , w e i g h t , f u s i o n D a t e , o u t ( ” f u s e d b y ” ) . videoPath [0] , out (” video ”) . videoDurationSec [0]
indexX , o u t ( ” f u s e d b y ” ) . indexY FROM f u s e d d a t a FROM s e n s o r r a w d a t a WHERE a c o u s t i c >15 AND o u t ( ”
WHERE w e i g h t >0.90 AND c o n c e p t = ” V e h i c l e ” ORDER n e x t ” ) . a c o u s t i c [0] >15 AND o u t ( ” n e x t ” ) . o u t ( ” n e x t ” )
BY f u s i o n D a t e . a c o u s t i c [0] >15 ORDER BY name
Neo4j Query: Neo4j Query:

13
MATCH ( s n : s e n s o r n o d e ) −[: c o l l e c t ]−>( s a : s e n s o r r a w d a t a )
−[: n e x t ]−>( s b : s e n s o r r a w d a t a ) −[: n e x t ]−>( s c : Table 8: Test Results (Numbers are in milliseconds)
s e n s o r r a w d a t a ) −[: v i d e o ]−>( s v : s e n s o r r a w v i d e o d a t a ) Query OrientDB Neo4j MySQL
WHERE s a . a c o u s t i c >15 AND s b . a c o u s t i c >15 AND s c .
a c o u s t i c >15 RETURN s n . name , s a . a c o u s t i c , s n . Concept Based 209 618 938
indexX , s n . indexY , sv . videoPath , sv .
v i d e o D u r a t i o n S e c ORDER BY s n . name
Video Based 355 145 422
Relational SQL Query: Recursive 4,293 79,812 36,469
SELECT s . name , s . i n d e x x , s . i n d e x y , r a . c o l l e c t D a t e , r a .
a c o u s t i c , v . v i d e o p a t h , v . d u r a t i o n FROM
s e n s o r n o d e s , s e n s o r r a w d a t a r a , s e n s o r r a w d a t a rb ,
s e n s o r r a w d a t a r c , s e n s o r r a w v i d e o d a t a v WHERE r a . For the second query, Neo4j performs better than Ori-
a c o u s t i c >15 AND r b . a c o u s t i c >15 AND r c . a c o u s t i c >15 entDB and the graph model is again better than MySQL.
AND s . i d = r a . s e n s o r n o d e i d and r a . i d = r b .
n e x t i d and r b . i d = r c . n e x t i d and r c . v i d e o i d = This time again graph databases beat relational databases
v . i d ORDER BY s . name ASC because of their join-free query capability. To understand
why Neo4j is faster than OrientDB, we have to dive into
9.1.3. Recursive Query queries. The query is neighbors and neighbors of neigh-
This query finds the detected “Human” typed objects bors query, which is a typical graph matching problem
with high accuracy and calculates the distance of the sen- considering paths of length 1 or 2. In PostgreSQL we
sor node to the sink node. use a relational table with id from / id to backed by an
OrientDB Query: index. Neo4j performs better because of its “index-free
SELECT $ n o d e I d , o u t ( ’ f u s e d b y ’ ) . name [ 0 ] , f u s i o n D a t e , adjacency” for the edges.
$ d e e p . c o u n t FROM f u s e d d a t a LET $ n o d e I d = o u t ( ’
f u s e d b y ’ ) . @rid , $ d e e p = SELECT COUNT( ∗ ) FROM (
The last query is to test the recursive SQL type of a
TRAVERSE i n ( ’ l e a d ’ ) FROM $ n o d e I d ) WHERE c o n c e p t = query. The graph-based model is much faster than the
’Human ’ AND w e i g h t > 0 . 9
relational model. Neo4j fails for this recursive query.
Neo4j Query: There can be some optimizations possible while using the
MATCH p = ( a : s i n k ) −[: l e a d ∗]−>(b : g a t e w a y ) WITH b . name a s
gname , l e n g t h ( p ) AS d e p t h MATCH ( s f d : f u s e d d a t a ) Cypher language (the query language of Neo4j) but we
−[: f u s e d b y ]−>( s n : s e n s o r n o d e ) <−[: l e a d ] −( g : g a t e w a y )
WHERE s f d . c o n c e p t = ”Human” AND s f d . w e i g h t >0.9
couldn’t find them in the available Neo4j documentation.
AND g . name = gname RETURN gname , s n . name , s f d .
fusionDate , depth 9.2. Doubling Sensed Raw Data Size
Relational SQL Query:
WITH RECURSIVE s e a r c h g r a p h ( i d , name , l e a d , d e p t h ) AS An OrientDB graph database is selected for the execu-
SELECT g . i d , g . name , g . l e a d , 0 FROM g a t e w a y g tion of experiments. The previous experiments are ap-
WHERE g . l e a d i s n u l l UNION ALL
SELECT g . i d , g . name , g . l e a d , 0 FROM g a t e w a y g WHERE g . plied on generated synthetic data of one month where
l e a d i s n u l l UNION ALL each sensor node can sense data with 5 minutes of pe-
SELECT g . i d , g . name , g . l e a d , s g . d e p t h + 1 FROM g a t e w a y
g , s e a r c h g r a p h s g WHERE g . l e a d = s g . i d riod. Now we increase the simulation duration from 1
SELECT s . name , s 2 . name , s 2 . indexX , s 2 . indexY , s . d e p t h month to 5 months step by step and diagnosed the query
FROM s e a r c h g r a p h s i n n e r j o i n
s e l e c t d i s t i n c t s n . i d , s n . name , s n . indexX , s n . indexY , performance.
s n . l e a d FROM s i n k f u s e d d a t a s f d , g a t e w a y f u s e d d a t a Figure 10 shows the chart of query times affected by
gfd , s e n s o r f u s e d d a t a s r f d , s e n s o r n o d e s n WHERE
s r f d . f u s i o n = g f d . i d AND g f d . f u s i o n = s f d . i d AND the increased simulation duration. The query time is in-
s r f d . f u s e d b y = s n . i d AND creased and it is better than linear which is fairly well
s f d . c o n c e p t = ’Human ’ AND s f d . w e i g h t >0.9
s 2 ON s 2 . l e a d = s . i d compared to the doubled data size.
Table 8 shows the performance results of our example
queries. For the first query is a simple range query which 10. Conclusions
is focused on the basic query performance. OrientDB
performs better than Neo4j because of the under-hood In this paper, we propose a graph-based model for com-
architecture of OrientDB which is a multi-model graph plex multimedia and sensor big data. Our first aim is
database. And graph model performs better than the rela- to represent the wireless sensor networks with multime-
tional model since there is no join operation as promised dia data. Our implementation is composed of two main
by the NoSQL databases. modules. The first module is the implementation of the

14
 Neo4j.
We have simulated our WMSN prototype system with
millions of data to test the proposed graph model sim-
ulating the system. We have tested the query perfor-
mance with many complex scenarios. We have showed
that our generated millions of synthetic data can be effi-
ciently queried efficiently on our graph database.
As future research work, we plan to do graph analytics
on our big graph model. Object tracking, topology opti-
mization, and path extraction like analytics suit well for
our surveillance domain.

Acknowledgements

This work is supported in part by a research Grant from

Figure 10: OrientDB Query Time/Simulation Duration Chart TÜBİTAK with Grant No. 114R082. We thank to the
researchers of CEng Multimedia Database Lab. for their
very valuable comments.
proposed data model. The second module is the simula-
tion which includes a simulator to produce synthetic big References
sensor multimedia data and the simulation infrastructure
which represents the objects moving in our multimedia References
sensor networks.
[1] C. Perera, A. Zaslavsky, P. Christen, D. Geor-
We have focused on the surveillance systems to de-
gakopoulos, Sensing as a service model for smart
sign our graph data model. The network topology and the
cities supported by internet of things, Transac-
data flow between each sensor nodes, gateways and sink
tions on Emerging Telecommunications Technolo-
are modeled so that all static and kinetic data is stored
gies 25 (1) (2014) 81–93.
within the sensor network which must be assumed to be
big data. The database to store big data is the NoSQL [2] M. Chen, S. Mao, Y. Liu, Big data: a survey, Mobile
graph database. Because graph databases are good at rep- Networks and Applications 19 (2) (2014) 171–209.
resentation of complex relations and scalable to store big
multimedia sensor data. [3] A. Moniruzzaman, S. A. Hossain, Nosql database:
We have tested our model on both graph databases New era of databases for big data analytics-
and a relational database. Test results show that classification, characteristics and comparison, arXiv
graph database model performs better that the relational preprint arXiv:1307.0191.
database model. To decide which graph database is more [4] C. Vicknair, M. Macias, Z. Zhao, X. Nan, Y. Chen,
convenient and efficient, we chose two well-known graph D. Wilkins, A comparison of a graph database and
databases, OrientDB and Neo4j, for experiments. Neo4j a relational database: a data provenance perspec-
is the market leading graph database, but it does not fit tive, in: Proceedings of the 48th annual Southeast
into our needs, which is to store complex multimedia big regional conference, ACM, 2010, p. 42.
data. Because Neo4j supports high availability with the
master-slave approach, which can scale vertically. But, in [5] Z. Xu, Y. Liu, L. Mei, C. Hu, L. Chen, Semantic
order to survive in the big data world, we need a master- based representing and organizing surveillance big
master approach, which OrientDB has. In addition to that, data using video structural description technology,
our experiments show that OrientDB performs better than J. of Systems and Software 102 (2015) 217–225.

15
 [6] J. Han, E. Haihong, G. Le, J. Du, Survey on nosql Computer Science and Electronics Engineering. At-
database, in: Pervasive computing and applica- lantis Press, 2013, pp. 472–475.
tions (ICPCA), 2011 6th international conference
on, IEEE, 2011, pp. 363–366. [17] A. Manjeshwar, D. P. Agrawal, Apteen: A hybrid
protocol for efficient routing and comprehensive in-
[7] A. Zaslavsky, C. Perera, D. Georgakopoulos, Sens- formation retrieval in wireless sensor networks., in:
ing as a service and big data, arXiv preprint Ipdps, Vol. 2, 2002, p. 48.
arXiv:1301.0159.
[18] R. Govindan, J. Hellerstein, W. Hong, S. Madden,
[8] K. Ashton, That ‘internet of things’ thing, RFiD M. Franklin, S. Shenker, The sensor network as a
Journal 22 (7) (2009) 97–114. database, Tech. rep., Citeseer (2002).

[9] P. Patel, A. Pathak, T. Teixeira, V. Issarny, Towards [19] I. Robinson, J. Webber, E. Eifrem, Graph Databases:
application development for the internet of things, New Opportunities for Connected Data, ” O’Reilly
in: Proceedings of the 8th Middleware Doctoral Media, Inc.”, 2015.
Symposium, ACM, 2011, p. 5. [20] O. Diallo, J. J. Rodrigues, M. Sene, Real-time data
[10] S. Alam, M. M. Chowdhury, J. Noll, Senaas: An management on wireless sensor networks: a sur-
event-driven sensor virtualization approach for in- vey, Journal of Network and Computer Applications
ternet of things cloud, in: Networked Embed- 35 (3) (2012) 1013–1021.
ded Systems for Enterprise Applications (NESEA), [21] B. Bostan-Korpeoglu, A. Yazici, I. Korpeoglu,
IEEE, 2010, pp. 1–6. R. George, A new approach for information pro-
cessing inwireless sensor network, in: 22nd Interna-
[11] L. Atzori, A. Iera, G. Morabito, The internet of
tional Conference on Data Engineering Workshops
things: A survey, Computer Networks 54 (15)
(ICDEW’06), IEEE, 2006, pp. 34–34.
(2010) 2787 – 2805.
[22] Y. Li, C. Wu, L. Guo, C.-H. Lee, Y. Guo, Wiki-
[12] S. Hong, D. Kim, M. Ha, S. Bae, S. J. Park, W. Jung, health: A big data platform for health, Cloud Com-
J.-E. Kim, Snail: an ip-based wireless sensor net- puting Applications for Quality Health Care Deliv-
work approach to the internet of things, IEEE Wire- ery (2014) 59.
less Communications 17 (6) (2010) 34–42.
[23] C. Jardak, P. Mähönen, J. Riihijärvi, Spatial big
[13] I. F. Akyildiz, T. Melodia, K. R. Chowdhury, A sur- data and wireless networks: experiences, applica-
vey on wireless multimedia sensor networks, Com- tions, and research challenges, IEEE Network 28 (4)
puter networks 51 (4) (2007) 921–960. (2014) 26–31.
[14] C. Perera, P. P. Jayaraman, A. Zaslavsky, D. Geor- [24] R. Angles, C. Gutierrez, Survey of graph database
gakopoulos, P. Christen, Sensor discovery and con- models, ACM Comput. Surv. 40 (1) (2008) 1.
figuration framework for the internet of things
paradigm, in: Internet of Things (WF-IoT), 2014 [25] E. Felemban, Advanced border intrusion detection
IEEE World Forum on, IEEE, 2014, pp. 94–99. and surveillance using wireless sensor network tech-
nology, International Journal of Communications,
[15] X. Li, S. Moh, Middleware systems for wireless sen- Network and System Sciences 6 (5) (2013) 251.
sor networks: A comparative survey, Contemporary
Engineering Sciences 7 (13) (2014) 649–660. [26] I. Stoianov, L. Nachman, S. Madden, T. Tok-
mouline, Pipenet: A wireless sensor network for
[16] P. Zhang, Z. Yan, H. Sun, A novel architecture based pipeline monitoring, in: 2007 6th International Sym-
on cloud computing for wireless sensor network, in: posium on Information Processing in Sensor Net-
Proceedings of the 2nd International Conference on works, IEEE, 2007, pp. 264–273.

16
[27] S. K. Mohapatra, P. K. Sahoo, S.-L. Wu, Big data an-
alytic architecture for intruder detection in heteroge-
neous wireless sensor networks, Journal of Network
and Computer Applications 66 (2016) 236–249.
[28] S. Hadim, N. Mohamed, Middleware: Middleware
challenges and approaches for wireless sensor net-
works, IEEE Distributed Systems 7 (3) (2006) 1.
[29] L.-Y. Ho, J.-J. Wu, P. Liu, Distributed graph
database for large-scale social computing, in: Cloud
Computing (CLOUD), 2012 IEEE 5th International
Conference on, IEEE, 2012, pp. 455–462.

17