Real-Time Database: Firebase INFO-H-415: Advanced Database
Real-Time Database: Firebase INFO-H-415: Advanced Database
Abstract
This report will present our project concerning the model of real-
time database systems (RTDBS). We will define the concept of real-time
database, their features and their specificities. Then we will present some
existing softwares to manage a RTDBS and give examples of applications
of such systems. Finally, we will develop a Firebase real-time database
for one of these examples and explain its functionalities.
1 Introduction
A classic database is a collection of information organized and which can be
easily accessed, managed and updated. Generally, data are organized in tables.
This is not always the case as we will see in our example of real-time database
system. This kind of databases without predefined tables is called schema-free.
A database can be of different types. The most common are SQL, NoSQL and
cloud databases.
A real-time database system is a classic database system which is providing
real-time constraints and ensure reliability on system’s timing requirements.
Timing constraints are not required to be extremely short but the database need
to manage explicit time constraints in a predictable way using time-cognizant
methods. This kind of database combines multiple features facilitating :
— Description of data;
— Maintenance of correctness and integrity of data;
— Efficient acces to the data;
— Correct execution of query and transaction execution in spite of concur-
rency and failures.
We will define in a more formal way these notions and explain differences
between traditional database and real-time database.
1
2 Definitions
2.1 Real-time processing
Real-time processing or computing is a system which is subject to ”real-time
constraints”. It means that this system is able to ”control an environment by
receiving data, processing them, and returning the results sufficiently quickly to
affect the environment at that time”[1].
2
4 Is a RTDBS a temporal database ?
No. Even if the two systems support time-specific transactions, temporal
databases associate to data a span of validity and real-time databases have a
defined span of time in which a transaction has to be executed.
5 Model
A real-time database is composed of two main parts :
— Controlling system : Meeting point between the computer and its soft-
ware.
and
— Controlled system : System which is perceiving the state of the environ-
ment observed and saved in the database.
A high degree of accuracy must be maintained between the actual state of the
environment and the state of the environment in the database. Therefore the
environment’s monitoring at fixed interval is essential.
5.1.1 Example
Assume we want to know the electric efficiency of a train, we will need at
least two tables:
— Electric consumption
— Covered distance
The train consumption is registered at time 100msec and has consumed 0.6W
and the distance it traveled is 10meters. and is registered at time 110msec.
Both information have an absolute consistency of 20msec.
3
ddistance (10m, 20msec, 110msec)
With those data, we can observe that the two datas are contemporary for an
interval of 10msec.
4
Figure 1: Deadlines represented with value functions
There are three types of transaction read-only (read and transmit the data
from the database to the controlling system), write-only (obtain the state of
the environment from the controlled system and report it to the database) and
updates (derive a new data item and report it to the database).
Now that these notions are well defined, we can say that a transaction is
specified by:
— its implication of missing deadline, which can be hard, critical or soft;
— its arrival pattern which can be periodic or not;
— its data access pattern which can be read-only, write-only, update or
random.
— its runtime requirement which can be known or unknown
6 Processing
When using a real-time database, one of the main issue is predictability. We
need to be able to predict if a transaction will respect its deadline or not, thus
5
the only way to make sure of it, would be to know the worst-case execution
time. However there are many sources of unpredictability present:
— dependence of the transaction’s execution sequence
— data and resource conflicts
— dynamic paging and I/O
— transactions abort and the resulting rollbacks and restarts
— communication delays and site failures
which sometimes make it impossible to predict the worst-case scenario.
Yet we can appease the effect of those unpredictibility sources. We can use
a main memory database and give priority to the I/O controllers to solve the
problem linked to dynamic paging and I/O. We can also allow a transaction to
write-only in its memory area before writing the transaction’s changes to the
database which could cause a rollback to occur.
The queuing of the transactions are based on their criticalness and priority.
6.1 Priority
We can approach priority with different methods :
— EDF (Earliest Deadline First) : This algorithm consists in a priority
queue where the next task to accomplish is chosen based on the closest
deadline. The algorithm updates the task to handle every time a new
task is added to the priority queue;
— Highest value first : The task defined as the most important has the
priority on the other tasks;
— Highest computation time first : The task which is estimated to take
more time to execute than the others is executed first;
— complex function of deadline, value.
Those different methods will have a different impact on the database system
performance.
6
7 Comparing differents softwares
There are many softwares which can be used to manage a real-time database.
We chose to get a closer look at four of them, CouchDB, MariaDB, RethinkDB
and Firebase.
CouchDB is a document storage open-source database system using JSON
standard file format developped by Apache software foundation and written in
Erlang. It provides an intuitive API which is restfull HTTP to read and write
database document. The documents are stored using B-Trees 1 and hashed
according to the files’ name. It allows to reliably store data from different places
in sync over unreliable network. A transaction is either totally completed or
completely failed, thus a document will never be partially stored. This database
system is schema-free which means that a data can be stored without a previous
defined structure [9]. This feature facilitates the migration of the data.
MariaDB is a relational open-source database system using JSON standard
file format since the version 10.2 developped by MariaDB Corporation Ab and
written in C and C++ based on MySQL. It is why MariaDB includes core
functionality of MySQL. It has a high scalability. It can be accessed via APIs
like ADO.NET, JDBC and ODBC. The data is stored using B-Trees, Hash or R-
Trees 2 . Unlike most databases system, MariaDB provides a secure environment.
RethinkDB is a document storage open-source database system using JSON
standard file format developed by the Linux foundation and written in C++.
It does not provide a specific API but can be accessed through external APIs
handling JSON files. Alike CouchDB, it is schema-free. RethinkDB is using an
embedded domain-specific query language called ReQL.
Firebase is a commercial cloud-hosted document storage database system
developped by Google. It provides multiple platforms for the different usage of
the sofware (Android, IOS, Javascript, Restfull HTTP,...). It is also schema-
free. Alike MariaDB, Firebase hosting provides a secure environment. Finally,
this RTDBS is using the NoSQL storage mechanism.
Of the four presented options, we chose to continue with Firebase because it
is well-documented and furnish all the features we need for our implementation.
Even though Firebase is a commercial database system, a free license is available.
7
Storage system SQL JSON Open-Source Schema-free APIs
CouchDB Document x x x x Restfull HTTP
MariaDB Document x x x ADO.NET, JDBC, ODBC
8
RethinkDB Relational ReQL x x x external handling JSON files
Firebase Cloud NoSQL x Commercial x Android, IOS, Javascript, Restfull HTTP
8 Firebase
8.1 NoSQL
Firebase does not use traditional relational queries but NoSQL which stands
for ”Not only SQL”, it is more scalable and more performant. NoSQL allows
horizontal scaling between different server where relational databases only offer
vertical scaling which is not optimize when storing loads of information.
There are four types of NoSQL database:
— Document databases oriented, information is represented as documents
(JSON, XML,...). Those documents can hold lots of key-value pairs.
— Graph oriented, information is represented as node and relation. It allows
the user to recover complex data easily.
— key-value oriented, information is stored as a value (integer, string, ob-
ject,...) paired with a key.
— wide-column oriented, store columns of data instead of rows.
8.2 Cloud
As a result of firebase being developped by Google, it uses a few Google
platform such as the Cloud. It is a cloud computing platform designed by
Google enabling storage.
9
— perform realtime database sanitization and maintenance (backend)
— perform rigorous task in the cloud to not overload the app
Inform the user : Cloud functions can inform the app users with relavant
information about the app (updates, followers, confirmation email,...)
The system works as follow:
1 The function is triggered by a new element in the database
2 The function composes a message to send
3 FCM sends the notification to the user’s device
Perform rigorous task : Firebase client can use the Google cloud resources
(CPU or networking) to relieve the user’s device. It can be used to periodically
delete unused accounts, send email to users,...
It is this feature that will allow us to write events triggers to keep our
database consistent
8.3 Authentification
In order to securely store an user data and personalized the user page, we
need to know the identity of the user to meet this need, Firebase has its own
backend services for the allow authentication. The authentication can be based
on password, phone numbers or federated identity (Google, Facebook, Twit-
ter,...).
You can identify the user in multiple ways:
Email and password
Federated identity provider integration
Phone number
customize your own system integration
or Anonymous
There are several way to sign-in, to integrate the feature, we can either use
FirebaseUI or use the Firebase Authentication SDK.
10
Firebase SDK Authentication : This way is used when we want to man-
ually integrate one or several sign-in methods.
8.5 Storage
As explained before, Firebase Real-Time Database system is cloud-stored
and encoded as JSON objects. The database can be seen as a tree where each
information is stored as a node coded in JSON with an associated key. The
data itself is structured according regular expression rules language.
In a Firebase database, there is no tables or records. A node is added to the
existing structure when a new data is added into the database. This kind of tree
can accept up to 32 level of node. In order to keep the data safe, it is important
to know that giving access to a node is also giving access to all children of this
node. This is why Firebase documentation recommands to keep a relatively flat
JSON tree.
8.6 Hosting
A Firebase Database cannot be hosted locally and has to be hosted on Fire-
base servers. But it is possible to test it locally via the Firebase Database Man-
ager called Firebase Command Line Interface. This interface allows to manage,
test and deploy a database.
8.7 Performance
The Realtime Database Firebase API is design to only allow operations that
can be quickly achieved. And to control that, Firebase provides a system which
allows the administrator to monitor the performance of the database. This
monitoring is charaterizing performance by different metrics which are :
11
— Response time : Time between the client’s request and the end of the
full transmition of the answer to this request;
— Payload size : charge down- and uploaded by the app in byte;
— Success rate : Ratio between successfully transmited responses and total
amount of responses. Allows to know the percentage of failed transmi-
tions.
To every data of the performance monitoring is associated a set categories which
describes on which device, operating system, version of the application and in
which country, etc. the request has been made.
9 Applications
9.1 Stock Exchange Markets
One application of a real-time database is in the context of Stock Exchange
Markets transactions. In this particular example, a database has to save every:
— variation of price of each stock quote;
— trade of a quantity of stock;
— offer of a quantity of stock;
— bid on a quantity of stock;
— ...
Even if the number of entries is depending on the granularity 3 chosen, the state
of the current price of a stock quote seems to be very unstable and constantly
changing. Furthermore, this information can be needed by a buyer, seller or
owner in a short period of time. Thus, a stock exchange markets is an example
of environment which requires the kind of database we are currently studying.
12
account is accurate at every moment. This database has to be reacheable from
every interface where a user can manage his account. Thus, the bank can ensure
that a costumer cannot spend the same money twice for example.
Furthermore, a real-time database system can be even more useful in ”real-
time banking”[14]. The use of real-time databases can give to the banks the
opportunity to offer immediate money transfers.
We will implement this application in a Firebase real-time database.
13
Figure 3: Levels of nodes in our database
14
— A transaction must be commited within 5 seconds. This is a hard dead-
line. By doing so, we ensure that one client cannot transfer two times the
same money. A transaction consists in a transfer of money from one ac-
count to another. In a five seconds lap time, we must ensure the client’s
balance and deduct the amount. If the constraint is not respected, the
operation is aborted.
— The database will have to check that the beneficiary is existing in the
database. This means the receving end exists and is a client of the bank.
If the constraint is not respected, the operation is aborted.
The two triggers presented above were implented in the same following
script.
function checkBenef(s){
var accountNumber = s
ref.child("/{Id}/Accounts/{AcId}/AccountNumber")
15
.equalTo(accountNumber)
.once("value",snapshot => {
const userData = snapshot.val()
if (userData){
console.log("Beneficiary exists!")
return true
}
return false
})
}
— The date and time of a transaction must be prior to the date of the
account’s openning ; which must be prior to the date of the client’s
registering.
})
function checkTransDate(trDate, acDate){
var transDate = trdate
var openDate = acDate
ref.child("/{Id}/Accounts/{AcId}/Date")
.equalTo(transDate)
.once("value",snapshot => {
if (transDate < acDate){
return true
}
return false
16
});
}
})
17
}
})
18
return null
}else{
console.log("Removing client. Has no account anymore")
return event.data.adminRef.getRoot().ref("/{Id}").remove()
}
})
19
if (post.OpeningBalance() >= 500){
return null
}else{
console.log("Canceling account creation. Not enough money")
return event.data.adminRef.getRoot()
.ref("/{Id}/Accounts/{AcId}")
.cancel()
}
})
9.3.4 Configuration
In order to begin our project, we had to create a google account to get access
to firebase. Once the account created, we had to add a new project and enter
the project in the web interface to obtain the configuration access. From the
web, we have access to the database data, so we were able to import our own
data with a JSON file generated with JSON-Generator[16].
To implement the trigger functions, we first had to install the firebase com-
mand line interface (CLI) through the console. Then we needed to login on the
CLI and specify which project we wanted to alter.
Once we entered the said project, 5 options are available:
— Database
— Firestore
— Functions
— Hosting
— Storage
Only two were useful for our example, database and functions. These options
allowed us to set rules on the access to the database and to configure the triggers.
The functions feature created a javascript file in which we had to define our
triggers before deploying them into the project.
20
10 Conclusion
This project gave us the oportunity to understand the difference between
a temporal and a real-time database system. Their names suggest that those
systems are similar and indeed both are linked to time. But we have learned
that they are related to two different aspects of time.
The usage of RTDBS may touch a lot of different sectors and are more present
in our daily life than what we could think. For example, it is used in banking
system as we have seen but also in position tracking system, messaging system,
stock exchange market, learning websites as Duolingo, commercial aviation, etc.
Even if Firebase is popular and well documented, we encoutered a lot of
difficulties to configurate the database. This is due to big differences between
Firebase and database systems we have learned up to now and the fact that
Firebase requeries up-to-date version of modules. Furthermore to make our
application work, Firebase asked to add a credit card to use the Google App
Engine but we could not afford it, thus we were unable to try our system.
21
References
[1] James Martin. Programming real-time computer systems. 1965
[4] Jan Lindström. Real Time Database System [online]. 2008 [Retrieved 04
October 2017]
Available on : https://www.cs.helsinki.fi/u/jplindst/papers/
rtds.pdf
[10] Sang H. Son. Real-time database systems and Data Services : Issues and
challenges [online]. 2011 [Retrieved 4 December 2017]
22
Available on : https://www.slideserve.com/jaden/
real-time-database-systems-and-data-services-issues-and-challenges
[11] Firebase Legacy Documentation [online]. 2016 [Retrieved 15 December
2017]
Available on : https://www.firebase.com/docs/
[12] Google. Firebase Documentation [online]. 2017 [Retrieved 3 December
2017]
Available on : https://firebase.google.com/docs
[13] Nipuna Hewamaddumage Thilina, Dinushika Rathnayaka, Nil-
shani Jayakodi, Yana Yonitha. Real-time databases [online]. 2014 [Re-
trieved 4 December 2017]
Available on : https://fr.slideshare.net/nipunahewamadduma/
real-time-databases
[14] Accenture Bank. Real-time payments for real-time banking [online].
2015 [Retrieved 6 December 2017]
Available on : https://www.accenture.com/
t00010101T000000__w__/de-de/_acnmedia/Accenture/
Conversion-Assets/DotCom/Documents/Global/PDF/Dualpub_22/
Accenture-Banking-Realtime-Payments-Realtime-Bank.pdf
[15] Ambade Shruti Dinkar, S.A Shaikh. Design and Implementation Of
Vehicle Tracking System Using GPS [online]. 2011 [Retrieved 6 December
2017]
Available on : http://www.iiste.org/Journals/index.php/JIEA/
article/download/798/703
[16] Vazha Omanashvili. JSON Generator [online]. [Accessed 7 December
2017] Available on : https://www.json-generator.com/
[17] Frank van Puffelen. Have you met the real time database [online].
2016 [Retrieved 16 December 2017]
Available on : https://firebase.googleblog.com/2016/07/
have-you-met-realtime-database.html
23