2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International

Conference on Ubiquitous Computing and Communications (ISPA/IUCC)

Secure Cloud Computing: Communication Protocol

for Multithreaded Fully Homomorphic Encryption
for Remote Data Processing
Alexander Oppermann, Federico Grasso Toro, Jean-Pierre Seifert
Artem Yurchenko Security in Telecommunications
Department 8.5 Metrological IT Technische Universitat Berlin
Physikalisch-Technische Bundesanstalt (PTB) Berlin, Germany
Berlin, Germany Email: [email protected]
Email: [email protected]

Abstract—A significant disadvantage of fully homomorphic where multiplication and addition in the encrypted domain are
encryption is the long periods of time needed to process encrypted feasible.
data, due to its complex and CPU-intensive arithmetic techniques. The LSIM (LibScarab extended for Integer arithmetics and
In this paper, a communication protocol is developed to ensure
authenticity, integrity and privacy of measurement data across a Multithreading) is based on the LibScarab implementation [11]
distributed measuring system. The fully homomorphic encryption and extends it for integer arithmetics. It uses zero compar-
library LibScarab was extended by integer arithmetics, compar- isons, parallelization and massive multithreading of arithmetic
isons, decisions and multithreading to secure data processing. operations to specifically suit the needs of tariff applica-
Furthermore, it enhances 32 and 64-bit arithmetic operations, tions (TAF) for smart meter gateways (SMGW). Nevertheless,
improving them by a higher factor. This extension is integrated
into a cloud computing architecture establishing privacy by LSIM should be widely applicable in other areas as well where
design. The resulting parallelized algorithm solved the time fast integer and fixed-point arithmetics are required.
constraint issues for smart meter gateway tariffs. Several tests This paper is organized as follows: Section II describes the
were performed, fulfilling tariff specifications, where preserving related work and gives an overview of the available algorithms.
privacy of accumulated data was necessary. It was concluded
The use cases and communication protocol are presented in
that this extension of the fully homomorphic encryption library
meets the requirements of real world applications. Section III. Afterwards, Section IV details the used library and
the authors’ contributions. Section V comprises performance
I. I NTRODUCTION tests. Finally, the summary and planned future work are
In this paper, a communication protocol is developed to provided in Section VI.
ensure authenticity, integrity and privacy of measurement
data across a distributed measuring system within the cloud II. R ELATED W ORK
computing architecture. By creating a signature for fully In FHE schemes, all arithmetic operations, especially mul-
homomorphic encryption (FHE) all parts of the distributed tiplications, add noise to the encrypted data, a concept called
measuring system (sensor, processing, display) can be verified bootstrapping was introduced by Gentry [5] in order to reduce
throughout the process. Protecting measurement data and their the noise and allow unlimited number of successive arithmetic
distributed processing by a virtual measuring instrument in operations.
cloud computing [4] poses new challenges. This is achieved by “recrypting” the cipher-text in the
In this scenario, traditional security technologies cannot encrypted domain without giving up security. One receives
guarantee secure centralized processing. Therefore, the authors a new noise-reduced cipher-text for the same clear-text. The
propose an approach to secure the processing within a cloud libraries given in Table I are shortly summarized and describe
computing environment, via FHE [3], which allows processing the related work.
of encrypted data without decrypting them.
A handful of open source libraries has been developed TABLE I: Overview of fully homomorphic libraries
since the first realization of FHE in 2009 by Gentry [5].
Library Language Encryption Scheme Key handling
New algorithms, patterns and optimizations of the original
approach can be found in [6][7][8]. HElib [9] is a very HElib C++ BGV Asymmetric
SEAL C++ Fan & Vercauteren Asymmetric
versatile but complex to handle library. On the other hand, LibScarab C Smart-Vercauteren Asymmetric
there are very lightweight solutions like FHEW [10] and FHEW C LWE Symmetric
LibScarab (see Table I). Only those libraries and schemes that HE R Fan & Vercauteren Asymmetric
implement a fully homomorphic approach are considered, i.e.

0-7695-6329-5/17/31.00 ©2017 IEEE 503

DOI 10.1109/ISPA/IUCC.2017.00084
Authorized licensed use limited to: COMSATS INSTITUTE OF INFORMATION TECHNOLOGY. Downloaded on October 24,2024 at 09:10:36 UTC from IEEE Xplore. Restrictions apply.
 1) HELib: HELib [9] is a C++ implementation of III. U SE C ASES AND C OMMUNICATION P ROTOCOL
the homomorphic encryption scheme of Brakerski-Gentry- Classical security approaches use encryption to ensure pri-
Vaikuntanathan (BGV) [12]. It provides Smart-Vercauteren vacy and signatures for verifying the authenticity of messages.
[8] cipher text packing techniques and Gentry-Halevi-Smart These attempts aim for securing the communication between
optimizations. In 2015, the library was extended to include two endpoints. In the cloud computing context, the atten-
bootstrapping, which allows cipher text refreshes in the en- tion shifted to secure computation (see Figure 1). Classical
crypted domain. Very flexible and configurable but at the encryption is static, thus messages have to be decrypted
same time quite complex to handle for non-experts, it provides first before being able to be processed. Similar conditions
low-level routines like set, add, multiply, shift,. Furthermore, apply for classical signature schemes. They are designed for
it is highly optimized for single instruction multiple data static messages. In order to verify remote computation, a
(SIMD) operations, which are unrewarding for our application proof is needed for correct and trustworthy results. Verifiable
scenarios, since the library cannot compare numbers in the Computing demands that the evaluation of an interactive proof
encrypted domain. While testing it with small numbers for the [16] needs to be more compact than the actual computation.
security parameters, the memory usage for 6 multiplications FHE offers an elegant way to ensure privacy by design since
on 7-bit integers amounted to over 6 GBytes. The library is it prevents clear-text information leaks. While externalizing
still under active development. confidential processes to the cloud, industry places little trust
2) SEAL: The Simple Encrypted Arithmetic Library in external service providers. The authors chose FHE over a
(SEAL)[13] is being developed by the Microsoft Research homomorphic signature scheme not only to help solving this
Group and originally implemented the YASHE scheme. In conflict, but it also offers a more powerful, practical and flex-
the most recent version, the scheme was switched to Fan ible approach to secure remote computing. Furthermore, there
and Vercauteren [7]. In the tested version 2.1 of the library it are several approaches that combine homomorphic encryption
supported only leveled homomorphism, i.e. by using modulus with verifiable computation [17][18], so that it is still feasible
switching, one can only perform a predefined amount of at a later time to integrate verifiable computation with this line
multiplications. Since a flexible approach is needed for the of research.
targeted application scenarios, it would be counter-productive
to limit the amount of operations by the library beforehand. A. Communication Protocol
The library itself is well documented and relatively easy to An overview of the proposed communication protocol is
use, but it is restricted to a Microsoft Windows environment. described in detail below (see Figure 2). It has been developed
In addition, it does not support bootstrapping and comparisons to ensure a distributed, authentic and secure communication
in the encrypted domain. using FHE to secure the computation within a cloud comput-
3) LibScarab: LibScarab is a C implementation of the ing architecture. The parts of sensor, processing and display
Smart-Vercauteren scheme [6] created by Perl, Brenner and correlate in the protocol as the following three sections: Client
Smith [11]. It supports bootstrapping, XOR as well as AND (Steps 1-7), Server (Steps 8-14) and Receiver (Steps 15-22).
operations on single bits. The library is easy to use, along The client generates the Public Key Pk and Secret Key
with a clear and compact structure. It is clearly structured Sk . The Sk will be send via a secure channel directly to the
and easy to extend. The authors had to port the library to trustworthy receiver (Steps 4-6), in order to be able to decrypt
the latest FLINT 2.5.2, GMP 6.1.1 and MPFR 3.1.5 libraries, the encrypted measurements results (Step 18) produced by the
since LibScarab was last updated in 2013 and does not support cloud (server). With the Public Key the client encrypts the
multi-threading out of the box. A recrypt operation is executed
automatically after every arithmetic operation.
4) FHEW: The “Fastest Homomorphic Encryption in the
West” (FHEW) [10] is a C implementation of the Learning
With Errors (LWE) scheme. It uses a symmetric key, i.e.
the same key is used for encryption and decryption. It also
offers bitwise encryption and NAND-Gates. Because of the
symmetric key procedures this library is unsuitable for the
envisioned application scenarios (see Table V). This library
was last updated in 2015.
5) Homomorphic Encryption (HE): The Homomorphic En-
cryption (HE) [15] is a package for the statistic language R
and implements a scheme from Fan and Vercauteren [7]. The
R package itself is a wrapper for the arithmetic functions
written in C/C++. While easy and intuitive to use, it lacks
bootstrapping and therewith the possibility for a cipher-text- Fig. 1: Overview of different security techniques and their area
refresh, preventing running several multiplications consecu- of application
tively, making it not practical for the application scenarios.

504

Authorized licensed use limited to: COMSATS INSTITUTE OF INFORMATION TECHNOLOGY. Downloaded on October 24,2024 at 09:10:36 UTC from IEEE Xplore. Restrictions apply.
   
  LSIM client-server communication protocol (overview)

  

Input: Measurement data m1 . . . mn , Encryption Parameter

  eP , Public Key Pk , Secret Key Sk , upload/download usable
bandwith (U,D), index of elements i = 1 . . . n
Output: Measurment results mr1 . . . mrn
      1: Client:
2: creates measurement data mi
  
   
3: encrypts measurement data and index twice:
  

 

  ci ← mi and ci ← mi ,
   index vi ← i and vi ← i
  
note: ci = ci but dec(ci ) = dec(ci ), vi = vi but dec(vi ) =
dec(vi )



4: repeat
Fig. 2: Overview of encrypted communication protocol 5: send Sk to trustworthy receiver,
6: until key exchange is successful
7: sends two encrypted messages with ci , vi and ci , vi to the
same measurement twice and sends them to the Cloud. This server, respectively
procedure 8: Server:
creates the simplest form of a signature to ensure authen- 9: checks certificate and the origin of messages
ticity and integrity of the measurement results. 10: if check is successful then
Even though homomorphic encryption usually aims at en- 11: process data
suring data privacy [5], it may be used in other areas and → ci and ci will be treated the same
for other purposes as well. A potential attacker trying to 12: compute ri and ri , respectively
manipulate the measurement results from within the cloud 13: else
computing architecture will face no clear-text processing and 14: throw data away, → error message to logbook
thus can only tamper randomly a measurement or the attacker 15: Receiver:
consequently manipulates all measurement data in the same 16: authenticates against server
way. If the first approach is chosen the receiver will detect the 17: pulls measurement results from server
manipulation (Step 19) and discard the measurement result. If 18: decrypts measurement results and index with Sk
all measurements are tampered in the same way only test data 19: if dec(ri ) = dec(ri ) and dec(vi ) = dec(vi ) then
with a known outcome can detect this kind of manipulation. 20: no manipulation and mri is correct
Implementing aperiodic test runs with precalculated data will 21: else
decrease the possibility for an attacker to stay undetected for 22: throw measurement data away,
a long period. return error message to logbook
If homomorphic encryption is carefully combined with test- f lag measuring system failure.
ing a running algorithm via precalculated test data, even such
random effects may be detected. Subsequently, the scheme
detailed here [3] may be used to achieve a certain degree of an energy amount are combined to form a price to be paid.
robustness towards algorithm and data manipulation, too. Finally, comparisons of input values are essential to realize
B. Required Logical and Arithmetic Operations. input- and time-dependent switching statements.
The implemented algorithm described in Section IV aims
at realizing the SMGW’s tariffing functionality, with suitable C. Required time for processing.
computation capacity without realizing additional protective The SMGW performs two tasks: Accumulation of mea-
measures. This approach can be easily applied to many other surement data and monthly reading of the accumulated mea-
measuring systems, performing similar price calculations such surement data according to the chosen SMGW tariffs. The
as: encrypted data coming from the meters need to be processed
• addition, subtraction, multiplication, division, in the cloud. In order to guarantee privacy of the measurement
• comparison, data FHE is used.
• input-dependent source selection, A maximum of 32 meters can be connected to a gateway
• input-dependent destination selection. simultaneously and since every 15 minutes new measurement
The addition operation is needed to add new energy values data arrive, therefore the accumulation has to be processed
to an existing tariff register. Subtraction, division and negative under 15×60sec
32 = 28seconds. Due to the non feasibility of
numbers are required to calculate the current energy flow (the the parallelization of the accumulation process, since there
power) based on consecutive readings of an cumulative meter. are inter-dependencies between measurements that may have
The multiplication operation is required when a tariff and an effect on the chosen register for subsequent values, the

505

Authorized licensed use limited to: COMSATS INSTITUTE OF INFORMATION TECHNOLOGY. Downloaded on October 24,2024 at 09:10:36 UTC from IEEE Xplore. Restrictions apply.
 monthly retrieval of accumulated register values should be the encrypted number which corresponds to toggling bits in the
done at a later time. encrypted domain. With these two operations (see Equation 1
Therefore, parallelization can be implemented in the homo- & 2) it is possible to derive all the boolean functions and, as
morphic core operations, like multiplication and addition, as a consequence, to provide arithmetic and logical operations
well as in the monthly reading for the tariffs. on encrypted integers, which are represented as arrays of
encrypted bits. The addition, subtraction and multiplication
IV. A LGORITHM D ESCRIPTION AND E XTENSIONS operations for integers had to be reimplemented according to
In this section, the extensions of LibScarab are described, in the chosen binary word sizes (e.g. 32-bit and 64-bit length).
order to implement the needed algorithms for the application
scenarios TAF 1-4, explained in detail in Table V. In subsec- A ∨ B = ¬(¬A ∧ ¬B) (1)
tion IV-B, a brief description of addition, multiplication and
division operations is given, as well as simple decision and ((1 ⊕ A) = ¬A). (2)
comparisons in the FHE environment. 1) Zero-Test of encrypted Integers.: This binary approach
yields the opportunity to implement a zero-test as a simple
A. Choosing and extending the library
bitwise ∨ on all bits of the represented number. The result of
Fully homomorphic encryption schemes support two oper- this operation should be complemented to return an encrypted
ations: addition and multiplication. Depending on the imple- 1 in case the examined number was 0. This corresponds to
mented scheme, also a sign change is possible, which enables a logical NOR operation with 32-bit inputs. The encrypted
subtraction. A lot of simple algorithms are based on these three result can be directly used as an input for further encrypted
operations and thus can be implemented. But nevertheless, no arithmetic and logical operations. To verify the computed
solution exists for comparing two numbers in the encrypted result it has to be decrypted in the end.
domain, which makes it impossible to make a decision in the 2) Comparison of encrypted Integers.: A simple implemen-
encrypted domain and consequently to implement a division tation of a comparator consists of the subtraction of both input
algorithm for example. parameters (A − B) and of the sign-evaluation of the result.
One possible way to implement a division algorithm is This operation delivers two possible results A < B or A ≥ B.
to represent numbers as fractions by saving nominator and The latter needs to be checked with the help of the zero-test,
denominator separately in order to bypass division. But this so that it can be distinguished between A > B and A = B. If
approach does not solve the problem of comparisons in the this clarification is not needed, this approach can be reduced
encrypted domain. It also seems impossible to render an to the calculation of borrow-bits only, since the result of the
unencrypted result from an encrypted operation without giving subtraction itself is not important.
up security and privacy. This means that all algorithms should 3) Simple decisions on encrypted Integers.: It is significant
be either completely deterministic or there should be enough to highlight that the result of a decision should remain in the
computing power to calculate all possible results in parallel. encrypted domain. Therefore, it is not possible to externally
A final decision should always be made in the unencrypted influence the program flow. Nevertheless, simple algorithmic
domain for the targeted algorithms. Since they neither have a constructs are feasible based on the result of the comparison or
completely deterministic structure nor is the decryption of the zero-test operator. These are source and destination selections.
data in an insecure environment an option, a different approach The source selection, on the one hand, is represented by
was pursued replacing integers with binary numbers. the following C-construct, which is similar to an if-then-
The LibScarab library is a good choice for the prototype, else-construct with the constraint that only data flow can be
since it provides all necessary tools without adding too much controlled:
complexity. In addition, it is easily configurable yet simple to Y = (condition)?A : B (3)
modify and to extend. On top of that it is very fast. A “recrypt”
procedure is executed after each operation. Thus, there is To implement this decision a 2:1-multiplexer is required,
no further need for noise control to fulfill the requirement which can be described as:
of unlimited multiplications as mentioned in Section III. As Y = ¬C · A + C · B (4)
already pointed out in Section II-3, the library did not support
multi-threading out of the box, since it was built on old where C is the output of the zero-test, the comparison operator
versions of the libraries FLINT, GMP and MPFR. Therefore, or any other possible boolean equation.
it was ported to newer versions in order to comply with the The destination selection, on the other hand, consists of a
requirements. demultiplexer and binary adders. The selection is triggered by
a boolean equation (e.g. comparison or zero-test). After the
B. Implementation of arithmetic operators output selection operation, the selected output or destination
Using binary numbers only two operations are left; the contains the input number and all the other outputs are set to
modulo-2-addition which corresponds to an XOR-gate and an encrypted 0. All outputs will be added to the corresponding
a modulo-2-multiplication that corresponds to an AND-gate. registers. All registers are modified, but only one will be
Another operation is the (unencrypted) binary complement of increased by the input value of the demultiplexer. All the

506

Authorized licensed use limited to: COMSATS INSTITUTE OF INFORMATION TECHNOLOGY. Downloaded on October 24,2024 at 09:10:36 UTC from IEEE Xplore. Restrictions apply.
 Fig. 4: Multi-threaded combined adder and subtracter

conventional word adders provides insufficient performance.

Fig. 3: Tree structure of the optimized adder part
For this reason, a multi-level bit-wise tree adder is used here
(see Figure 3).
others will be increased by an encrypted 0. It is important to The main principle of a tree adder is that a full adder is used
mention that the addition of an encrypted zero always produces to add three bits. At the output, one sum bit and one carry bit
a different encrypted version of the same number. Hence it is are generated. The resulting number, consisting of carry bits,
not possible to say which of the registers are actually changed is shifted one place to the left. Due to the fact that there is
by a FHE-operation. no carry propagation in the case of the 32 partial products
4) Addition of encrypted Integers.: The implementation of of the multiplication of two 32 bit numbers, 30 (x0 to x29 )
arithmetic circuits is done with special regard to multithread- of them can be processed by bit-wise full adders in a bit-
ing. A particular challenge was to find a solution with a small parallel manner in the first stage. The resulting 20 numbers are
amount of gates and a small circuit depth at the same time. supplemented by x30 to deliver 21 numbers and are processed
An analogue problem is known in the field of digital design in the next stage. After several iterations, there are only two
as Area-Speed-Tradeoff, since a lower circuit depth usually numbers left which are added together by a conventional adder.
results in a higher number of gates used. Each additional gate The difficulty here is the bitwise addition of the shifted partial
needs computing time, thus fast digital circuits have often poor products with respect to the significant bit position, which
performance executing homomorphic operations. Especially requires manual optimization and proper planning of the multi-
for multiplication and division some advanced algorithms threaded operations.
exist, but due to the massive use of multiplexers or lookup- 6) Division of encrypted Integers.: The division operation
tables these solutions are not reasonable with respect to the is the most difficult one to parallelize because of inter-step
execution time. dependencies. One possible improvement leading to a reduced
Three different versions of an adder were implemented and number of iterations is the use of high-radix number systems
compared. A Carry Look Ahead Adder (CLA) [22] with block to perform the division. Unfortunately, the complexity of each
size of 4 and 8 bit, also known as “fast adder” proved to be the iteration is also increased in this case. Advanced higher radix
slowest adder within the constraints of this paper. The Carry algorithms like Sweeney-Robertson-Tocher (SRT) division
Select Adder (CSA) [22] rendered mid-range performance. uses lookup-tables and/or a lot of multiplexers [22][23]. These
The modified Ripple Carry Adder (RCA) [22] revealed the are very expensive in case of a FHE software implementation,
shortest execution time, since the operations of the first half so that a simpler solution was investigated.
adder was executed completely in parallel. The same adder The simplest way to implement the division is the well-
could be used to implement subtraction, but this would contain known paper and pencil method adapted to the binary number
an array of XOR-gates to form the complement. Considering system also known as restoring-division [22]. The divisor is
performance the subtraction was outsourced to a separate subtracted from the shifted dividend, in case of a negative
routine, in order to save time during addition operations. remainder the divisor is added back to the remainder and the
5) Multiplication of encrypted Integers.: Fast multiplica- result will be shifted. The restoring operation implicates one
tions using higher radix solutions [22][23] require a massive extra addition or use of a multiplexer in each iteration. In
use of multiplexers or/and operand recoding which are costly contrast to the restoring division the non-restoring-algorithm
in size as well as time consuming. requires only one addition or subtraction in each iteration. The
In case of conventional binary multiplications, two problems underlying concept can be easily derived from the restoring
can be identified: 1.) the generation of the partial products, algorithm. In case of a negative remainder after one subtraction
2.) their summation. While the generation of partial products the divisor (D) should be added back to the remainder (R)
can be carried out completely (bitwise) in parallel, the sum- and the result (S) shifted to the left, i.e. S = 2 · (R + D).
mation can only be partially parallelized. A tree structure of In the next iteration D will be subtracted again: 2 · R + 2 ·

507

Authorized licensed use limited to: COMSATS INSTITUTE OF INFORMATION TECHNOLOGY. Downloaded on October 24,2024 at 09:10:36 UTC from IEEE Xplore. Restrictions apply.
 TABLE II: Overview of key geometry and performance gain in comparison to [11].
key geometry {logν ,S1,S2} key size (kB) keygen (s) speedup recrypt (ms) speedup
384/16/05 1.8/30 1.1 15.5 34 7.7
384/32/16 1.8/60 1.17 11.9 90 3.4
384/64/16 1.8/120 1.24 12 181 3.7
2048/64/16 9.6/626 516 6.1 670 2
4096/64/16 18/1250 5204 2.7 1660 1.9

D − D = 2 · R + D, which is equal to the simple shift of A. Results of homomorphic operations

the negative remainder of the first iteration and replacing of 1) Key generation and recrypt operations.: While paral-
the subtraction by an addition in the next iteration. Addition lelizing the basic arithmetic operations, e.g. addition and
and subtraction are performed by the same boolean circuit multiplication, it was determined that the speedup factor is
(see Figure 4). The sign-bit of the result is used to select the not significant for creating a monic and irreducible polynomial
operation (addition/subtraction) for the next iteration. F (x) with a resultant p being prime (see KeyGen() [6]) in
V. E XPERIMENTAL C OMPARISON a multi-threaded environment compared to a single threaded
one, i.e. no relevant time gain for key generation is realized.
This section describes the experimental comparisons that
The authors assume that performance gain is lost due to the
were conducted on a Linux server with an Intel Xeon CPU
randomized generation of the polynomials and their evaluation,
E5-2620 v3 @2.40 GHz, 24 cores and 64GB of RAM. In the
because of the great spread of the results and the necessary
first subsection, key generation, recrypt operation and single
synchronization of the threads. Thus, the times stated in
arithmetic operations of FHE are compared, after improving
Table II for key generation and recrypt operation are single
and extending LibScarab as described in Section IV. The sec-
threaded. Nevertheless, in comparison to Brenner’s reference
ond subsections comprises results of the application scenarios
implementation [11] and the stated times for key generation
TAF 1-4 that were already outlined in Table V. The speedup
for 384-bit key length a speed up of 15.5 could be yielded,
and efficiency are calculated for both single operations as well
i.e. instead of 17 seconds it only took 1.1 second to generate a
as for the tariff applications.
key. The main factors are optimizations in the implementation,
The relative performance gain achieved by parallelization
modernized libraries as already explained in Section II-3 and
(speedup) can be measured and is defined in [24] as a metric.
faster hardware than in 2012. The recrypt operation achieved
This can be written as:
a speed up factor of 7.7 for the encryption parameter logν =
Ts (n)
S(n) = (5) 384, i.e. a recrypt costs only 34ms instead of 263ms. The disk
Tp (n) space for public and private keys does not distinguish from
where Ts is the execution time of the best sequential algorithm Brenner’s data.
for solving the problem and Tp marks the execution time of Brenner’s key geometry consists of three paramters: logν
the parallelized algorithm using n processing units. is the encryption parameter responsible for spanning the
The efficiency is also defined by [24] as a metric that cryptographic space and creating the public key; S1 determines
measures the fraction of time in which a processing unit is the elements for the size of the hint inside the public key; S2
usefully employed. This can be asserted as: determines the number of valid elements in the hint.
S(n) 2) Arithmetic operations.: By parallelizing the arithmetic
E(n) = (6) operations the greatest benefit was earned within multiplica-
n
where S is the speedup for the algorithm (see Equation (5)) 25.000

20.000
and n marks the number of processing units.
Memory [kB]

15.000

10.000

5.000

0 5 10 15 20 25 30 35 40 45 50
800 Addtion
Multiplication
Time [sec]

600 Division

400

200

0
0 5 10 15 20 25 30 35 40 45 50
Threads [n]

Fig. 6: Plot of single 64 Bit-FHE-operations (Add, Mult, Div)

Fig. 5: Plot of single 32 Bit-FHE-operations (Add, Mult, Div) on a Server with an Intel Xeon CPU E5-2620 v3 @ 2.40 GHz
on a Server with an Intel Xeon CPU E5-2620 v3 @ 2.40 GHz and 64GB RAM.
and 64GB RAM.

508

Authorized licensed use limited to: COMSATS INSTITUTE OF INFORMATION TECHNOLOGY. Downloaded on October 24,2024 at 09:10:36 UTC from IEEE Xplore. Restrictions apply.
 TABLE III: Overview of 32 bit operation results TABLE V: Implemented tariff application scennarios (TAF)
n t (sec) S(n) E(n) in % TAF 1 Tariffs with low data usage, where energy is always billed with
the same price and collected data are only sent at the end of the
Add 1 2 - - billing period requiring no external trigger.
2 2 1 50 TAF 2 Time-dependent tariffs, where energy is billed with different
48 1 2 4 tariffs according to the time at which a certain amount of energy
Mult 1 124 - - is consumed. The switching between tariffs is time-dependent
16 19 6.5 41 but the switching points are static.
48 17 7.3 15 TAF 3 Power-dependent tariffs, where the price does not depend on the
total energy consumed but on the current power consumption
Div 1 196 - - (energy per time interval) The switching into different price
16 126 1.6 10 categories is therefore done according to the value of the current
48 121 1.6 3 measurement result.
TAF 4 Consumption-dependent tariffs, where a new price is used when
tion by a factor of 7.29. In Figure 5 one can see the asymptotic a certain energy budget has been consumed. The budgets are
statically predefined and the condition for assigning new mea-
characteristic of the optimization for the time usage. While surement values to the next price category has to be checked
executing a single thread utilization took 124 seconds for regularly.
a single multiplication, a 48 threads utilization only took
17 seconds for a single multiplication. Considering memory with a speedup of 6.3 and a efficiency gain of 39 % needing
usage, the optimum is reached by utilizing 16 threads with 19 75 seconds.
seconds for a single multiplication with a calculated efficiency Additions in the 64 bit space are optimized by a factor of
gain of 41 % (see Equation 6) and a speedup of 6.53. The 1.6, i.e. needing 5 seconds for one addition using one thread
least significant benefit from parallelization was realized for compared to 48 threads lasting 3 seconds. The optimum is
the addition operation. It is already the fastest operation in the reached for 4 threads with a speedup of 1.3 and a 31 %
encrypted domain needing only two seconds for one addition efficiency. Again addition is the fastest operation.
for a single thread while it could be pushed down to one For the division the same factor of 1.6 could be reached as
second for a single addition utilizing 48 threads. The efficiency for the 32 bit operands. Speaking in absolute numbers it is still
optimum is reached here by using only two threads with a a huge difference from roughly 13 minutes (796 seconds) for
50% efficiency and a speedup of 1 needing two seconds. The a single threaded division compared to about 8 minutes (479
division is traditionally a very complex arithmetic operation seconds) using 48 threads. The optimum is reached using 16
which often has its own compartment on modern CPUs in threads with a speedup of 1.6 and 10 % efficiency needing 8.5
order to optimize its performance. Bearing this in mind for the minutes (497 seconds).
software implementation in the encrypted domain, the benefit
through parallelizing the division operation was achieved by a B. Results of application scenarios
factor of 1.6. This means that a single thread for one division In contrast to the tests performed in Subsection V-A where
took 196 seconds while this was reduced to 121 seconds only arithmetic operations are measured, these results cover
utilizing 48 threads. the combination of arithmetic operations and comparisons
The optimum for this operation is reached using 16 threads applied to the application scenarios described in detail in Table
with a speedup of 1.6 with a 10 % efficiency. Similar results V. A recrypt is included after each arithmetic operation to
could be yielded for 64 bit arithmetic operations as one can reduce the noise. The calculations for the application scenarios
be seen in Figure 6. Obviously, more memory was needed are split into accumulating the measurement data (see Figure
for the single arithmetic operations due to the nature of the 7) and summing them up on demand (see Figure 8), e.g. at the
bigger operands. Again, the multiplication benefited the most end of the month. The latter includes the more complex arith-
by a factor of 6.8, needing almost 8 minutes (470 seconds) metic operations and comparisons in the encrypted domain.
for a single threaded multiplication compared to 69 seconds
using 48 threads. The optimum is reached using 16 threads 35

30 TAF 1 accumulation
TAF 2 accumulation
Time [sec]

TABLE IV: Overview of 64 bit operation results 25 TAF 3 accumulation

TAF 4 accumulation
20
n t (sec) S(n) E(n) in % 15

Add 1 5 - - 0 5 10 15 20 25 30 35 40 45 50
10
4 4 1.3 31
Time [sec]

8
48 3 1.7 3
6
Mult 1 470 - -
4
16 75 6.3 39
48 69 6.8 14 0 5 10 15 20 25 30 35 40 45 50
Threads [n]
Div 1 796 - -
16 497 1.6 10 Fig. 7: Plot of accumulating measurement data on a Server
48 467 1.7 3 with an Intel Xeon CPU E5-2620 v3 @ 2.40 GHz and 64GB
RAM.

509

Authorized licensed use limited to: COMSATS INSTITUTE OF INFORMATION TECHNOLOGY. Downloaded on October 24,2024 at 09:10:36 UTC from IEEE Xplore. Restrictions apply.
 600
R EFERENCES
500 [1] European Parliament and Council. Directive 2014/32/EU of the Euro-
TAF 1 Read out pean Parliament and of the Council. Official Journal of the European
400 TAF 2 Read out
TAF 3 Read out
Union, 2014.
TAF 4 Read out [2] European Parliament and Council. Directive 2009/72/EC of the Euro-
300
pean Parliament and of the Council. Official Journal of the European
Union, 2009.
200
[3] Alexander Oppermann, Jean-Pierre Seifert, and Florian Thiel. Secure
cloud reference architectures for measuring instruments under legal
100
control. 2016.
0
[4] Fang Liu, Jin Tong, Jian Mao, Robert Bohn, John Messina, Lee Badger,
0 5 10 15 20 25 30 35 40 45 50 and Dawn Leaf. Nist cloud computing reference architecture. NIST
special publication, 500(2011):292, 2011.
Fig. 8: Plot of “read out” of the measurement data. Final [5] Craig Gentry et al. Fully homomorphic encryption using ideal lattices.
calculation for different application scenarios on a Server with In STOC, volume 9, pages 169–178, 2009.
[6] Nigel P Smart and Frederik Vercauteren. Fully homomorphic encryption
an Intel Xeon CPU E5-2620 v3 @ 2.40 GHz and 64GB RAM. with relatively small key and ciphertext sizes. In International Workshop
on Public Key Cryptography, pages 420–443. Springer, 2010.
While TAF 1 and 4 performed very similar in accumulating [7] Junfeng Fan and Frederik Vercauteren. Somewhat practical fully
measurement data (see Figure 7) in respect to time and homomorphic encryption. IACR Cryptology ePrint Archive, 2012:144,
2012.
utilizing threads, the gain is 1.3 needing about 5 seconds for [8] Nigel P Smart and Frederik Vercauteren. Fully homomorphic simd
a single thread to only 3.8 seconds utilizing 48 threads. The operations. Designs, codes and cryptography, pages 1–25, 2014.
optimum is reached using only 4 threads with a speedup of [9] Shai Halevi and Victor Shoup. Algorithms in helib. In International
Cryptology Conference, pages 554–571. Springer, 2014.
1.3 and about 30 % efficiency needing 4.2 and 4.3 seconds. [10] Léo Ducas and Daniele Micciancio. Fhew: bootstrapping homomorphic
For the monthly “read out” of the summed up measurement encryption in less than a second. In Annual International Conference on
data the application scenarios (see Figure 8) differ more than the Theory and Applications of Cryptographic Techniques, pages 617–
640. Springer, 2015.
for the accumulation process. TAF 1, low data usage, is the [11] Henning Perl, Michael Brenner, and Matthew Smith. Poster: an
fasted and simplest one gaining factor 7for a single thread implementation of the fully homomorphic smart-vercauteren crypto-
using 123.3 seconds compared to 48 threads consuming only system. In Proceedings of the 18th ACM conference on Computer and
communications security, pages 837–840. ACM, 2011.
17.6 seconds. The optimum is reached using 16 threads with [12] Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. (leveled)
a speedup of 6.3 and a 39 % efficiency needing 19.6 seconds fully homomorphic encryption without bootstrapping. ACM Transac-
to finish. tions on Computation Theory (TOCT), 6(3):13, 2014.
[13] Kim Laine and Rachel Player. Simple encrypted arithmetic library-seal
(v2. 0). Technical report, Technical report, September, 2016.
VI. CONCLUSION AND FUTURE WORKS [14] Matteo Frigo and Steven G Johnson. The fastest fourier transform in
the west. Technical report, DTIC Document, 1997.
[15] L. J. M. Aslett, P. M. Esperança, and C. C. Holmes. A review of
In this paper, an extension to the fully homomorphic en- homomorphic encryption and software tools for encrypted statistical
cryption library LibScarab has been presented. After a brief machine learning. Technical report, University of Oxford, 2015.
comparison of the existing encryption schemes, this extension [16] Joe Kilian. A note on efficient zero-knowledge proofs and arguments.
In Proceedings of the twenty-fourth annual ACM symposium on Theory
(LSIM) and its properties were described and evaluated. of computing, pages 723–732. ACM, 1992.
LSIM comprises fast integer arithmetic, logic operations and [17] Rosario Gennaro, Craig Gentry, and Bryan Parno. Non-interactive
multithreading. The resulting gain can be as high as a factor of verifiable computing: Outsourcing computation to untrusted workers. In
Annual Cryptology Conference, pages 465–482. Springer, 2010.
7.3 for multiplications of 32 bit numbers, with smaller gains [18] Kai-Min Chung, Yael Kalai, and Salil Vadhan. Improved delegation of
for addition, subtraction and division. The key generation and computation using fully homomorphic encryption. In Annual Cryptology
recryption were also optimized by a factor of 15.5 and 7.7, Conference, pages 483–501. Springer, 2010.
[19] Marko Esche and Florian Thiel. Software risk assessment for measuring
respectively. instruments in legal metrology. In Computer Science and Information
Furthermore, a communication protocol has been developed Systems (FedCSIS), 2015 Federated Conference on, pages 1113–1123.
IEEE, 2015.
to prove the functionality of fully homomorphic encryption [20] BSI. Technische Richtlinie BSI TR-03109-1 Anforderungen an die Inter-
within a cloud computing architecture and successfully applied operabilität der Kommunikationseinheit eines intelligenten Messsystems.
to real world applications. This protocol ensures authenticity, Bundesamt für Sicherheit in der Informationstechnik, Bonn, 2013.
[21] BSI. Schutzprofil für die Kommunikationseinheit eines intelligenten
integrity and privacy of the measurement results. Messsystems für Stoff- und Energiemengen (Smart Meter Gateway PP),
Future work will focus on porting LSIM to CUDA / Certification-ID: BSI-CC-PP-0073. Bundesamt für Sicherheit in der
OpenCL and carrying out tests on graphic cards for mas- Informationstechnik, Bonn, 2014.
[22] Israel Koren. Computer arithmetic algorithms. Universities Press, 2002.
sive multithreading and parallelization. Furthermore, since [23] Mi Lu. Modular structure of large multiplier. Arithmetic and Logic in
OpenMP was utilized for this prototype, it will be interesting Computer Systems, 1st ed, New Jersey: John Wiley & Sons, Inc, pages
for further investigations to find out if Posix Threads (pthreads) 120–122, 2004.
[24] Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar.
or a GMP-library optimization will improve parallelization Introduction to parallel computing. Pearson Education, second edition,
with respect to computing time. 2003.
Additionally, ongoing challenges should be studied, such as
the computing times in comparison with those realized by a
secure hardware solution for FHE.

510

Authorized licensed use limited to: COMSATS INSTITUTE OF INFORMATION TECHNOLOGY. Downloaded on October 24,2024 at 09:10:36 UTC from IEEE Xplore. Restrictions apply.