Integrated Computer-Aided Engineering 21 (2014) 3–18 3

DOI 10.3233/ICA-130444
IOS Press

Information dependability in distributed

systems: The dependable distributed storage
system
Salvatore Distefanoa,∗ and Antonio Puliafitob
a
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
b
Dipartimento di Ingegneria, Università di Messina, Messina, Italy

Abstract. In distributed infrastructures data are usually scattered among different nodes and thus accessible by several users. It is
therefore necessary to provide mechanisms and tools for managing the information, ensuring the access exclusively to authorized
users avoiding malicious ones. Furthermore, satisfactory performance and adequate fault tolerance have to be provided while
addressing information confidentiality, availability and integrity issues. In this paper we face the problem of data dependability
in distributed system, proposing a redundant lightweight cryptography algorithm combining symmetric and asymmetric cryptog-
raphy approaches. The proposed algorithm, named dependable distributed storage system (D2 S2 ), has been implemented on top
of the Grid gLite middleware file access and replica management libraries, as a file system service with cryptography capability,
redundancy management and POSIX interface. To demonstrate the effectiveness and the applicability of the D2 S2 gLite imple-
mentation, performance and reliability of its operations have been evaluated and compared to those of existing solutions. The tests
have been performed on real applications to provide a complete overview of the D2 S2 implementation. The results thus obtained
demonstrate the effectiveness of the D2 S2 implementation also compared to existing solutions and former implementations.

Keywords: Dependability, information security, performance, reliability, distributed system, grid

1. Introduction terfaces for aggregating, managing and interconnect-

ing (physical and virtual) resources according to a
Information and Communication Technology (ICT) service computing paradigm, as a service. All such
moves towards network-distributed computing paradi- mechanisms and tools allow to aggregate from thou-
gms. From clusters to data-centers built on multi- sands (Grid, Cloud) to millions (P2P, volunteer com-
core systems, such a trend aims at aggregating dis- puting) resources into large scale distributed systems.
Even though the final goal of these paradigms differs,
tributed resources accessed through the Internet. Peer
ranging from business-commercial applications (Grid,
to peer systems exploit overlay networks in order to
Cloud) to scientific-academic-free of charge contexts
connect peers, volunteer computing adopts a client-
(Grid, volunteer computing) and to file/resource shar-
server approach to aggregate volunteer contributors,
ing (P2P, volunteer computing), there are problems and
Grids adopt resource brokers (or workload manage-
issues common to all of them.
ment systems) for managing the resources and the in- Among them dependability often assumes strategic
coming job requests, Clouds implement specific in- importance. Indeed, in order to implement distributed
computing goals it is mandatory to consider depend-
∗ Corresponding author: Salvatore Distefano, Dipartimento di
ability aspects such as reliability and security. Espe-
Elettronica, Informazione e Bioingegneria, Politecnico di Milano,
cially in the information management: data have to be
Piazza L. Da Vinci 32, 20133 Milano, Italy. E-mail: salvatore. promptly available from different locations, ensuring
[email protected]. reliability and security in accesses. A possible solution

ISSN 1069-2509/14/$27.50 
c 2014 – IOS Press and the author(s). All rights reserved
4 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system

is to replicate and disseminate data among nodes. But The solution we propose to store data in a distributed
sharing information in distributed multi-user environ- environment taking into account dependability require-
ments triggers problems of security concerning data ments is to combine both the symmetric and the asym-
confidentiality and integrity. metric cryptography with a data redundancy algorithm
Some of the middleware implementing the above re- into the dependable distributed storage system (D2 S2 ).
ferred paradigms usually provide data and resources D2 S2 has been implemented on top of the gLite mid-
management’s capabilities such as indexing and replica dleware in order to demonstrate the feasibility of the
management, ensuring security on accessing services approach, exploiting libraries and API provided by the
and on communicating data, but they often lack of data specific middleware. Interesting and remarkable con-
protection from direct malicious accesses, at system tributions of the implementation are the organization
level. In other words, the fact that data are dissemi- of data in an encrypted file system, the protection of
nated and stored in remote nodes, directly accessible both data/files and file system structure, the introduc-
from their administrators, constitutes one of the main tion of the capability of file rewriting in the gLite stor-
risks for data security in such environments, known as age systems and the use of a cache to improve the D2 S2
insider abuse/attack. It is therefore mandatory to in- operation performance.
troduce adequate data protection mechanisms, which Moreover, the D2 S2 gLite implementation has been
deny data intelligibility to unauthorized users, also if evaluated considering both single I/O operations and
they are (local) system administrators. possible applications, investigating its performance
The focus this paper is therefore on information re- and reliability against existing solutions and former
liability and security of distributed systems identified implementations. In particular we demonstrated the ef-
and characterized hereinafter as information depend- fectiveness of our implementation through real appli-
ability. Specifically, the main contribution of this work cations, by which we evaluated the impact of cache on
is to provide a mechanism to store data in distributed the system performance and reliability.
systems, overcoming the above discussed problems This paper extends previous work [8,9] mainly
and issues concerning data dependability. In terms of broadening the scope to reliability issues, thus turn-
reliability, the design and implementation of a specific ing into the wider dependability context to provide
data redundancy management mechanism is required further investigation on the impact of dependability
to achieve such goal. In terms of security, it is neces- on the overall storage system. Furthermore, the refer-
sary to develop an algorithm that provides the highest ence architecture of D2 S2 is here specified identify-
data protection, ensuring confidentiality and integrity. ing functionalities and modules of both its Client and
To this extent, all the work proposed in literature is Server, also providing and discussing new evaluation
mainly based on symmetric cryptography. Most of the results obtained by testing the resulting implementa-
adopted solutions implement key splitting algorithms. tion through real applications and against the former
The underlying idea of the key splitting approach is implementations.
that at least a subset of the systems (key servers) over The reminder of the paper is organized as follows:
which the keys are distributed has to be trustworthy. Section 2 provides an overview of the state of the art of
However this approach is weak from three points of information dependability in distributed environments.
views: security, since often it is not possible to ad- In Section 3 we describe the main algorithms of the
equately secure the list of servers with key parts, in proposed technique, while Section 4 deals with its im-
fact the system administrators can always access the plementation into the gLite Grid middleware. Then, in
key parts and it is really hard to achieve trustworthy Section 5, the results obtained by evaluating our im-
on remote and distributed nodes for users; reliability/ plementation are presented. A discussion on the pro-
availability, since keys and consequently data can be posed solution, final remarks and possible future work
unavailable if one or more of the key servers, depend- are provided in Section 6.
ing on the fault tolerance policy implemented, cannot
be accessed; performance, since there is an initial over-
head to rebuild a key, depending on the number of parts 2. Related work
in which the key is split. A possible solution for im-
proving the reliability/availability of key splitting algo- Several work in literature addressed problems and
rithms is to replicate the key servers, but this contrasts issues related to reliable, available and secure storage
with security challenges. systems, but few of them consider both aspects to-
 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system 5

gether. Focusing on both reliability and security the and replica services specifically conceived for redun-
perspective slightly changes, since choices and algo- dant data management.
rithms can be affected by multi-objective, combined On the other hand, the problem of a secure storage
constraints. has been mainly faced in literature as definition of ac-
In this context, an interesting solution has been de- cess rights [14], in particular addressing problems of
veloped in [28]. It implements a distributed storage data sharing, whilst the coding of data is demanded
system (Oasis+) entirely deployed in-memory using to the user, since no automatic mechanism to access
the distributed shared memory approach, operating as a secure storage space in a transparent way has been
a backbone service in a computing cluster. The Oa- defined. In such context, several solutions propose to
sis+ storage system implements an interesting solu- encrypt data in order to achieve information secu-
tion from the reliability perspective in a small cluster rity [5,6,19,20,22]. Symmetric encryption techniques
context, but it does not take into account security as- are mainly adopted, moving the problem towards the
pects. Both dependability (in terms of reliability, avail- symmetric key hiding in order to protect data from ma-
ability and fault tolerance) and security are instead ad- licious users, external attacks and insider abuses.
dressed in [10], which proposes a solution based on The most effective way to achieve data security is
intrusion tolerance, i.e. the fragmentation-redundancy- encryption, based on the cryptography theory. Cryp-
scattering, that can be scalable to large pervasive sys- tography algorithms are grouped into two main classes:
tems. A drawback of the technique is the high over- symmetric (also known as conventional or secret
head in term of messages required for managing the in- key) [30] and asymmetric (public key) [18]. Asym-
formation security, since higher security standards im- metric ciphers are much slower, even if more secure,
ply higher fragmentation that, on the other hand, af- than symmetric ones. An interesting technique com-
fects the performance of data operations. Another in- bining the high security of asymmetric cryptography
teresting work on the topic is proposed in [26], based algorithms with the efficiency of the symmetric ap-
on a dependable distributed storage system named Ca- proach is PGP (Pretty Good Privacy) [11]. PGP en-
gra. It is optimized for 3D rendering and implements crypts data through symmetric cryptography and then
an interesting fault tolerance mechanism through re- applies an asymmetric cryptography algorithm to se-
dundancy, based on a consistent hashing algorithm to cure the symmetric key. GNU has developed a PGP-
manage and locate data that unfortunately do not en- like algorithm in the open source project GPG (GNU
sure key data coherency thus limiting its applicability Privacy Guard) [31].
to large datasets. Different solutions adopting cryptography to se-
The most important and effective way for achieving cure information in distributed computing environ-
data reliability and fault tolerance is redundancy, con- ment have been conceived [3,6,7,11,13,16,17,19–21,
sisting of replicating information and disseminating re- 31]. In [19], the authors propose a technique for se-
dundant data onto the system. In distributed system, re- curing data disseminated in a distributed environment
liability can be implemented both locally at the single based on symmetric cryptography. The key security is
node of the system and globally considering the sys- entrusted to a unique keystore server that stores it, to
tem as a whole. Several techniques have been devel- which all the data access requests must be notified in
oped and implemented at local level, among others the order to decrypt the data. This algorithm implements
redundant array of independent disks (RAID) technol- a spatial security policy: the security lies in physically
ogy is the most effective and widely used. The problem hiding and securing the keystore server, and the access
of local reliability can be thus considered as solved, but to the keystore is physically restricted and monitored in
it is still open at a global level. Several attempts have order to protect from malicious users, external attacks
been implemented in such direction. For example, [23, and insider abuses.
24] proposes a middleware system that features spe- Shamir in [20] studied in depth the problem of data
cific enhancements designed to support the develop- access in distributed environments, proposing a so-
ment and assessment of highly dependable service- lution based on symmetric keys. The secret sharing
oriented systems and applications named CROWN-C. scheme splits the (AES) symmetric key in n parts,
Fault tolerance issues have been faced at higher service which can be recomposed if and only if at least k  n
layer by implementing a specific fault tolerant Grid parts are available (k/n threshold scheme). Moreover,
service. Reliability and data redundancy also are dealt in order to prevent unauthorized accesses, the symmet-
with in [4,10,12,27], mainly adopting data catalogue ric key is stored in different key-servers. In this way,
6 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system

the system is both resistant to attacks and reliable since pursued avoiding both outsider and, in particular, in-
n − k + 1 key-servers have to be compromised or have sider attacks: no one except the user/owner can access
to fail in order to make the key unavailable and conse- data, including system administrators.
quently data inaccessible. Brunie et al. have specified a In such scenario two main actors can be identified:
similar technique in [6], decomposing the (AES) sym- the final user (or also customer) and the storage sys-
metric key in n trunks that can be rebuilt only if all the tem administrator (or provider). The former accesses
n parts are available. the storage system through a specific Client that sends
The solutions above discussed are based on symmet- the user request to the storage system through the net-
ric cryptography. Most of them implement key split- work in a client-server fashion. On the other hand, the
ting algorithms. The underlying idea of the key split- Server (that could be a Web-Grid service) processes
ting approach is that at least a subset of the nodes (key the incoming requests returning the corresponding
servers) over which the keys are distributed has to be results/data. As discussed above the system should en-
trustworthy. This approach well addresses data-sharing sure data reliability/availability and reliability preserv-
and reliability/availability issues but it is weak from the ing from both outsider and insider abuses.
security point of view, since the list of servers with key Data reliability can be achieved by introducing re-
parts has to be adequately secured, also from system dundancy. Storage systems are usually implemented
administrators that can always access the keys. This as redundant, starting from very robust and reliable
is a serious problem to adequately address in an alter- RAID configurations. This adequately covers internal-
native way, in fact, even though higher security could isolated faults. In order to face common failure modes
be achieved by increasing the fragmentation into key and causes it is necessary to adequately replicate data
parts and consequently of keystores (anyway vulnera- by splitting and distributing them among different stor-
ble to insider abuses), this should have a heavy impact age systems physically independent. The main prob-
on performance. lem to face is therefore related to the replica manage-
This fact has been also demonstrated in [17,29] ment since it is necessary to ensure data consistency. In
evaluating and comparing different dependable storage order to achieve data security, the best solution is the
system solutions in distributed environments. More cryptography. As discussed above, till now, the most
specifically, [29] proposes a model and evaluation successful approach adopted for solving the problem
metrics used in the analysis of 4 schemes: replica- is the symmetric cryptography due to its performance
tion with shared keys (through key server), replica- against the asymmetric one. A problem of symmetric
tion with lockboxes (mixing symmetric and asymmet- encryption is the symmetric key (DataKey) securing-
ric encryption), and two variants adopting the local hiding as in the Kerckhoffs’ principle [15].
key approach, threshold secret sharing with local en- On the other hand, a whole asymmetric cryptogra-
cryption and short secret sharing. Experimental mea- phy algorithm has a heavy impact on data access times,
sures demonstrate that the replication with lockboxes since this requires greater computational resources and
and private keys’ scheme ensures the greatest security, therefore is slower than the symmetric algorithms. This
while all the schemes provide similar level of reliabil- usually does not allow encrypting large amounts of
ity/availability. The best performance in retrieving data data in acceptable time for users, considerably slowing
is provided by the short secret sharing scheme. down the overall system’s performance. Thus, it be-
Thus, in order to achieve the highest security also comes necessary to implement a solution representing
preventing insider abuses the use of private keys and a trade-off between security and performance require-
asymmetric encryption is required, possibly taking into ments. The only condition to satisfy is that no other
account also performance requirements. than the owner of data can access the private key, that
is the only exclusive way to decrypt data encrypted by
the coupling public key. In other words we assume that
3. The proposed solution the private key is securely stored in a trust location (for
example a physical token such as a smartcard device).
The main goal of this work is to achieve data re- As above introduced, the most successful approach
liability and security in distributed systems specifi- for addressing the key-security problem is to split the
cally conceived for providing users with access to key (DataKey) among different KeyServers. The main
their storage accounts. To this purpose, data reliabil- drawback of such approach is the total failure of the
ity/availability, confidentiality and integrity must be protection mechanism in case a malicious user obtains
 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system 7

– Replica manager: Manages data replica imple-

menting specific data redundancy policies, index-
ing replicas and keeping track of where portions
of the data set can be found.
– Interface: Implements the storage system inter-
(a) (b) face to end-users. It mainly receives the user re-
quests and forwards them to the specific module
Fig. 1. Modular architecture of D2 S2 client (a) and server (b). for processing. The service has to be network pro-
vided, thus specific protocols have to be adopted.
the key. This technique does not protect the owner of
data from who has (legally or illegally) the privileges 3.1.2. Client
of administrator, since administrators can access to all Four main modules are identified for the D2 S2
the key components/splits. For this reason we retain it Client in Fig. 1(b): the loader, the encrypter/decrypter,
is necessary to use a more effective solution. the store and replica manager and the interface with the
An adequate one could be to implement a two-stage remote storage system. They implements the following
encryption algorithm: at the first stage data are en- functionalities:
crypted by exploiting the faster symmetric encryption,
while at the second stage the symmetric DataKey used – Loader: The module that loads data from the re-
for encrypting data is in its turn encrypted through an mote storage system. It initially loads the DataKey
asymmetric encryption technique, solution adopted in and the data index-map from the remote storage
D2 S2 as detailed in the following. system. Thus it manages each remote data load-
ing/read.
3.1. Static view: Architecture – Encrypter/decrypter: Encrypts and decrypts data
to/from the storage system. It uses the keys (both
As shown in Fig. 1 the D2 S2 system is com- the asymmetric and the symmetric ones) provided
posed of two main parts connected through the net- by the loader.
work, the storage system and the client, following – Store and replica manager (SRM): Manages the
the client/server architecture. In the following we pro- data storage operation. It therefore implements
vide modular-layered architectures of D2 S2 server and functionalities and provides primitives for writing
client, where modules at the same layer directly inter- data into the remote storage system. For perfor-
acting and cooperating, providing functionalities to the mance issues data loaded from the storage sys-
upper layers. tems can be also locally buffered (cached).
– Interface: It implements the interface to the stor-
3.1.1. Server age system. It forwards all the requests incoming
According to the considerations made above and from the lower level modules to the storage sys-
starting from the assumption of reliable storage nodes tem through the network, adopting adequate pro-
implementing some specific RAID technology, in or- tocols.
der to implement a high reliable storage system it is All such modules need to interact in order to ade-
necessary to use redundant storage nodes. Different quately perform the required functions. Moreover, ad-
possible options are available as redundant data man- equate tricks and countermeasures have to be adopted
agement policies: replication, advanced coding algo- in order to protect decrypted data (in clear) from mali-
rithms, parity, etc. The main drawback of redundant cious accesses.
storage nodes is the necessity of coordination through
a management system that distributes data according 3.2. Dynamic view: Algorithms
to a specific redundancy policy.
Figure 1(a) shows the architecture implementing the Once identified and described the D2 S2 main func-
D2 S2 server. Three main components are identified: tionalities and the corresponding modules, it is neces-
– Identity manager: Manages the user account (ac- sary to deal with their interactions, detailing how they
counting), authentication, authorization, roles, have to collaborate for performing their intended func-
and privileges/permissions within or across the tions. To this purpose we can start roughly assigning
storage system, with the goal of increasing secu- reliability/availability commitments to the D2 S2 server
rity and performance, ensuring privacy, while de- side, while security issues are mainly in charge of the
creasing cost and downtime. D2 S2 client.
8 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system

Fig. 3. Client initialization algorithm.

Fig. 2. D2 S2 Client-Server interaction workflow.

The server replica manager handles redundancy ap-

plying some specific policies. Such management sys-
tem has to be highly reliable, more than the underly-
ing storage subsystem usually based on RAID config-
urations. Thus, it should be as much decentralized as
possible, in order to provide adequate guarantees.
With regards to security, the proposed solution com-
bines both symmetric and asymmetric cryptography Fig. 4. Data sharing algorithm.
into a hierarchical approach, applied in the client
side. An authorized user, authenticated by his/her own all the other information concerning these latter have to
asymmetric keys through the client, contacts the stor- be secured and preserved from unauthorized accesses,
age system where his/her data are located. Data in the as discussed above. In this way the highest layer of se-
storage system are encrypted by a symmetric cryptog- curity is achieved and ensured: data and keys are al-
raphy algorithm whose DataKey K is also stored in the ways encrypted both in the remote storage system and
storage, in its turn encrypted by the user/owner pub- in transfers, while they are in clear only in the trust
lic key (KPUB), obtaining the encrypted KPUB (K). In user host, which implements mechanisms for securing
this way, only the user that has the matching private them.
key KPRIV can decrypt the DataKey and therefore the The algorithm related to the D2 S2 Client-Server in-
encrypted data. KPUB(K) is stored together the data to teractions can be better decomposed into the four sub-
allow the owner to access data from any node of the in- activities identified in Fig. 2: initialization, data shar-
frastructure. He/she only needs the smartcard contain- ing, data I/0 and finalize, detailed in the following sub-
ing the private key. sections through activity diagrams which swim-lines
In order to implement data sharing, K is replicated are associated to the D2 S2 modules identified in Fig. 1.
into as many copies as the users authorized to access Since all the interactions are managed by the inter-
data, then saved into the storage system. In this way a faces, for the sake of clarity in diagramming we omit
copy of DataKey encrypted by the ith authorized user to highlight specific activities on interfaces.
i i
public key KPUB (KPUB (K)) must be stored into the
storage system in order the user can access the data. 3.2.1. Initialization
It is important to remark that, in the proposed algo- The first phase of the system is devoted to the initial
rithm, the decryption is exclusively performed into the setting of the environment, as described by the activ-
authorized users’ node where the asymmetric keys are ity diagram of Fig. 3. Once a user logs into the sys-
hosted, and the decrypted symmetric key, the data and tem trough the Client, the algorithm requests to the
 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system 9

Client: Client: Server: Client: Client:SRM Server: Client:SRM Server:

Loader Decrypter Replica Man. Encrypter Replica Replica Man.
Encrypt(Data,K) Man.
Request(Data) Retrieve(K(Data))
Send(OPReq) Recv(OPReq)
Send(K(Data)) Recv(K(Data))
Recv(K(Data)) Send(K(Data)) R=Process(OPReq)
[NO] [OK]
R=Store(K(Data))
Decrypt(K(Data),K)
Recv(R) Send(R)
Recv(R) Send(R)
Error(Read)

(a) (b) (c)

Fig. 5. Read (a) write (b) and generic operation (c) algorithms.

Client:SRM Server: the DataKey does not exist the storage system has to
Replica Man. be initialized. Thus a new DataKey K is generated
by the Client, then encrypted and sent to the Server
Send(ENDREQ) Recv(ENDREQ)
replica manager, that replicates it and stores the copies
Recv(R)
as shown in Fig. 3.
R=CheckStatus

[NO]
Send(R)
3.2.2. Data sharing
[OK]
In order to access a specific data set, the generic
FlushMem(User) jth user needs the copy of the DataKey K used to en-
crypt such data. This implies that the storage system
Logout(User) has to store a copy of K encrypted by the user public
j j
key KPUB , and so KPUB (K). If the jth user has cre-
j
ated the data set, KPUB(K) is automatically stored into
Fig. 6. Termination algorithm. the storage system at the initialization phase, as dis-
cussed above. Otherwise it is necessary that another
remote storage system the symmetric DataKey K en- authorized user, the ith one, creates a copy of K, en-
crypted by the public key of the user KPUB. The Client crypts that by the jth user public key, thus obtaining
j
Loader therefore forwards the request to the storage KPUB (K), and stores this latter into the storage system
system that first verifies the identity of the user through as specified in the activity diagram of Fig. 4. After that,
the Server ID manager and then, in case it is autho- the ith user has access to the data set.
rized, forwards the requests to the replica manager. If
the storage system has been already initialized, a spe- 3.2.3. I/O
cific query returns the set of storage elements hosting The data stored in the storage system are organized
the encrypted DataKey KPUB(K). The replica manager according to a specific structure, i.e. a file system tree.
has to retrieve KPUB(K) from one of the storage el- Data are therefore split into blocks and files that are
ements: in order to achieve fault tolerance, the corre- logically distributed in a folder of the tree. They are
sponding algorithm starts by querying the first storage managed and accessed by using well-known I/O prim-
element and, in case of faults, iterates by querying the itives (open, close, read, write, unlink, access, chmod,
other storage elements until the data are retrieved or all closedir, create, lseek, lstat, mkdir, opendir, read-dir,
the storage elements have been queried without suc- rename, rmdir, stat and unlink). In Fig. 5, the algo-
cess. The data thus retrieved, or a Null value in case of rithms implementing read, write and the other generic
unsuccessful queries or an error code in case of unau- operations are represented. In particular the read algo-
thorized user, are then forwarded to the Client. In case rithm of Fig. 5(a) implies the decryption of data re-
of success, this latter has to decrypt KPUB (K) by using ceived from the remote storage system, while the write
the user private key KPRIV (K) and stores the decrypted algorithm of Fig. 5(b) requires the encryption of data
DataKey K in a safe memory location. before they are sent to the storage system Server. A
Otherwise, if the user is not recognized by the sys- generic operation is just a command or a signal sent by
tem the algorithm terminates with error, while in case the Client to the Server, as shown in Fig. 5(c).
10 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system

UI Storage FILE GUID SFN(K(BLK[1]))

gLite D2S2 (K(BLK[1]))
FILE INDEX
D 2S 2 System D2S2FI GUID SFN(K(BLK[n]))
SE
SFN(K(BLK[1])) SFN(K(BLK[1]))
Unswappable Mem (K(BLK[n])) SFN(K(GDS2FI))
GUID SFN(KPUB
LFC SE FILE K(D2S2FI)
SFN(K(BLK[n])) (K)) SFN(K(BLK[n]))

SE SE
BLOCKS GUID(KPUB(K)) SFN(K(D2S2FI)) SE SFN(K(D2S2FI)) SE
UI
CACHE
LFC
SFN(KPUB(K)) SFN(KPUB(K))
GFAL D2S2FBC

(a) (b)

Fig. 7. D2 S2 implementation schema (a) and file system (b).

3.2.4. Finalize tructure (PKI) [25]. This introduces the necessity, as

The termination algorithm is described in the activ- discussed above, of an initialization phase in which the
ity diagram of Fig. 6. Before a user leaves the system, environment is configured (DataKey and indexes load-
it is necessary to remove the symmetric DataKey and ing) and a termination phase to finalize it.
the other information from the Client node memory.
But, since a user could still have one or more data I/O 4.1. The grid storage system
pending operations, it is necessary to check the status
of such operations on the storage system. The Server The schema mapping the logical D2 S2 architecture
replica manager thus receives and processes such re- of Fig. 1 into the gLite middleware according to the
quest, asking to all the storage elements hosting the requirements and specifications above described is de-
data for their status. Then, according to the results of picted in Fig. 7(a). D2 S2 is implemented as a layer
such inquiry the user decides whether to close the cur- working on top of GFAL, providing a dependable
rent session or to wait for the completion of some op- file service with security/cryptography capability by
eration. means of POSIX interface.
The D2 S2 storage service creates a virtual file sys-
tem structuring data in files, directories and subdirec-
4. The grid implementation tories without any restrictions on levels and number
of files per directory. Since we build this architecture
The dependable storage system architecturally and on top of GFAL, in D2 S2 all data objects are seen as
logically described in Section 2, has been implemented files stored on the SE, accessible by users through the
as a service of the gLite Grid middleware [32]. More GFAL interface (LFN, . . . SRM, grid user ID GUID,
specifically, the D2 S2 implementation has been inte- etc.). One of the most interesting capability of such im-
grated in the gLite middleware, which provides li- plementation is the file modification and/or rewriting,
braries and services supporting storage (the Grid file operation not implemented by the GFAL library. GFAL
access libraries or GFAL [33]) and data management only allows to create/write new files, without any pos-
(LCG File Catalog or LFC). In order to ensure security sibilities of modifying those after creation.
it is necessary that the dependable storage service is A D2 S2 file can be entirely stored in the SE in one
available in interactive mode both on the D2 S2 Client chunk with variable length or it can be split into two
node (user interface or UI in Grid-gLite) and in the or more blocks with fixed, user defined length, spec-
D2 S2 Server side (implementing a distributed storage ified in the D2 S2 setup configuration, as reported in
system on top of storage elements or SE) in order to Fig. 7(b). To avoid conflicts among file names, we uni-
perform data management and encryption-decryption vocally identify each chunk or block of data file by
operations. a GUID identifier. The file index shown in Fig. 7(b)
The D2 S2 reliability/availability mechanisms stron- (D2S2FI), maps a file to the corresponding blocks.
gly base on the gLite LFC providing services that Such file index is encrypted through the symmetric
implement an enhanced file catalog, with adequate DataKey and is kept in UI unswappable memory lo-
replica management facilities. With regards to secu- cations. In this way the user operates on a virtual file
rity, we adopt well-known and effective standards, system whose logic structure usually does not corre-
the AES [30] symmetric encryption algorithm and the spond with its physical structure. The main goal of
RSA asymmetric cryptography [18] based on the X- file indexing is the optimization of the file I/O opera-
509 certification mechanism of the public key infras- tions, since it reduces the data access time. Moreover,
 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system 11

since the D2 S2 file rewriting and modification has to Table 1

D2 S2 primitive headers
be implemented through GFAL primitives, these oper-
D2 S2 Primitives
ations are performed by deleting the file and rewriting
its modified version; splitting a D2 S2 file into several int d2s2_access(const char *, int)
int d2s2_chmod(const char *, mode_t)
chunks/blocks files in the SE is the only feasible way int d2s2_close(int)
to achieve the rewriting goal. int d2s2_closedir(DIR *)
The file system is created at D2 S2 initialization. int d2s2_create(const char *, mode_t)
Each file and block is encrypted through the symmetric int d2s2_errmsg(char *, int, const char *)
int d2s2_finalize(char *)
DataKey, encrypted, in turn by the user public key, and int d2s2_flush(char *)
both are stored together. int d2s2_init(char *, char *)
The reliability of the D2 S2 storage system is del- off_t d2s2_lseek(int, off t, int)
egated to LFC. This latter implements a reliable int d2s2_lstat(const char *, struct stat *)
int d2s2_mkdir(const char *, mode_t)
replica management system that indexes and replicates int d2s2_open(const char *, int, mode t)
the blocks univocally identified by the corresponding DIR *d2s2_opendir(const char *)
GUIDs, storing the copies, characterized by site file ssize t d2s2_read(int, void *, size_t)
names (SFN), into the selected SE (specified in the struct dirent *d2s2_readdir(DIR *)
D2 S2 setting parameters). This also allows exploiting int d2s2_rename(const char*, const char*)
int d2s2_rmdir(const char *)
all the features provided by LFC, that itself implements int d2s2_share(const char *, struct stat *)
a whole file system with POSIX interface and access int d2s2_stat(const char *, struct stat *)
control list management. Moreover, in order to ensure int d2s2_unlink(const char *)
the highest level of information security, it is necessary ssize_t d2s2_write(int, const void *, size_t)
to also encrypt the file system structure.
Another strength point of D2 S2 is the possibility set, and from programming environments, due to the
of optimizing the performance of I/O operations. In- D2 S2 API. In such a way, the access to a file of the
deed D2 S2 implements a local cache of encrypted virtual encrypted file system is similar to the access to
blocks/chunks (D2S2FBC), locally managed in the a local file and the mechanisms, apart from the access
UI unswappable memory in order to avoid unautho- time latency, is totally hidden to the final user. D2 S2
rized or malicious accesses. All the operations involv- specifies the same library function set of GFAL, com-
ing blocks/chunks already loaded in the UI cache are posed of 22 primitives reported in Table 1. D2 S2 func-
performed locally, by directly varying such blocks/ tions are prefixed by “d2s2_*” while the gLite-LCG
chunks. When a file is closed, the blocks stored in one by “lcg_*” and the GFAL one by “gfal_*”.
cache are updated through LFC into the SEs. A specific But, as discussed in Section 2, D2 S2 requires the ini-
D2 S2 primitive (d2s2_flush) has been specified to tialization and the termination phases, therefore spe-
force the flushing of data from the UI cache to the stor- cific functions (d2s2_finalize, d2s2_flush,
age. This could have a significant impact on storage op- d2s2_init, d2s2_share) are defined to this pur-
eration performance reducing the number of accesses pose. In the following we specify the D2 S2 primitives
to the remote storage system, especially in read opera- starting from the same phases characterization adopted
tions. above.
Indeed, write operations trigger problems of cache
coherence and therefore require adequate mechanisms 4.2.1. Initialization.
for their management. In the current implementation a The initialization phase is mandatory in D2 S2 . In
relaxed consistency protocol is applied, allowing hav-
this phase the library context is initialized with the user
ing different copies of the same data on local caches.
preferences set on environment variables: D2S2DK
The relaxation model adopted is relaxing all program
(GUID of the DataKey block), D2S2FI (GUID of the
orders, more specifically implementing a weak order-
file index block), D2S2PUBKEY (user’s public key
ing [1] where the data synchronization is up to the pro-
used to encrypt), D2S2PRVKEY (user’s private key
grammer through explicit invocation of a specific syn-
used to encrypt).
chronization operations.
A user needing to access the D2 S2 has to invoke the
4.2. D2 S2 API and commands d2s2_init function in order to read the symmetric
DataKey K encrypted by the user public key KPUB
D2 S2 implements a POSIX.1 interface ensuring from the Grid storage. As shown in Fig. 8 first and
high usability both from shell, through the command successive accesses are distinguished into two cases.
12 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system

(a)

(b)

Fig. 8. d2s2_init implementation of the first (a) and following (b) D2 S2 initializations.

j
In the first initialization phase, d2s2_init generates encrypted DataKey KPUB (K) into the SE by invoking
the symmetric key K as a sequence of random num- a gfal_write operation. Thus, the jth user can have
bers obtained by invoking an OPENSSL ad-hoc func- access to the data through d2s2_init.
tion. In the following accesses d2s2_init loads K
and the file index D2S2FI from the storage elements. 4.2.3. I/O
The algorithms in both cases are similar: first the Client D2 S2 data I/O operations are implemented through
on the UI checks for the presence of the encrypted key I/O POSIX primitives such as: open, read/write and
KPUB (K) in the LFC by invoking a lcg_lr primitive close. Files are always encrypted in memory; the en-
specifying the GUID of the DataKey block; then, in cryption is performed at runtime. To improve the D2 S2
case it does not exist, a new key is created, encrypted performance and the usability of its library the ac-
and sent to the storage system (Fig. 8(a)), that creates cessed files’ chunks are locally buffered into a cache
and stores it in the SE closest to the UI (by implicitly in the UI until the corresponding files are closed. At
invoking a gfal_write from the UI to the selected file closing, the UI cache is synchronized with the
SE) through lcg_cr, and finally replicates the en- Grid storage systems (LFC and SE) by invoking a
crypted DataKey into the SE through lcg_rep; oth- d2s2_flush.
erwise, if the Datakey exists, this and the file index are More specifically, as pictorially described in Fig. 10,
loaded in the UI by two consecutive gfal_read op- the gds2_read reads the blocks set BLK1 corre-
erations (Fig. 8(b)). KPUB (K) is therefore decrypted sponding to the selected part of the file. Some of
by the user private key and placed into an unswappable such blocks could be already loaded and stored on the
memory location of the UI to avoid malicious accesses. cache D2S2FBC. The blocks not present in the cache,
identified by the set BLK2 ⊆ BLK1, are firstly lo-
4.2.2. Data sharing cated by inquiring the LCG through a lcg_lr in-
D2 S2 data sharing is implemented by the d2s2_sh vocation and therefore loaded from the SE by an ex-
are primitive. A user i that wants to share the data plicit gfal_read call. This data, with that loaded
stored into an SE with a generic jth user in the same from cache, are placed in the output buffer, and the file
virtual organization has to access the X509 PKI certifi- blocks cache is updated with the data just loaded from
cate of user j. The d2s2_share algorithm therefore the SE. The sets BLK1 and BLK2 correspond to the
j
extracts the user j public key of (KPUB ) and then cre- vectors BLK1[] and BLK2[] of Fig. 10.
j
ates a new copy of the DataKey K encrypted by KPUB . The d2s2_write is an operation entirely per-
As shown in Fig. 9, d2s2_share then stores the new formed by the Client locally, as shown in Fig. 11. The
 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system 13

Fig. 9. d2s2_share primitive implementation.

Fig. 10. d2s2_read implementation on gLite.

4.2.4. Finalize
The main goal of the termination operation is to
synchronize Client and Server and in particular the
data modified by the final user in the UI cache, to
be updated to the Grid storage system. This is per-
formed by the d2s2_finalize function by invok-
ing d2s2_flush. The gLite implementations of such
primitives are shown in Fig. 13. With specific regards
to the latter, d2s2_flush synchronizes a specific file
or part of it by firstly identifying all the corresponding
Fig. 11. d2s2_write implementation on gLite. dirty blocks (DBLK[]), and then writing them to the
storage system. Since gLite libraries do not allow any
data blocks to modify are temporarily saved into the modification of files, the write operation has to delete
the original stored blocks (by performing a lcg_del
file blocks cache. When the file is closed, renamed,
erasing all the copies stored in the SEs) and thus stores
moved, deleted, the flush of the cache is forced, or the
the new blocks (lcg_cr) then replicated by LFC
gLite D2 S2 session is terminated, the data in cache are
(lcg_rep) as in Fig. 13(a). d2s2_finalize has
synchronized with the corresponding one catalogued
to just synchronize the whole cache D2S2FBC and the
by LFC and stored into the SEs. index D2S2FI invoking two times d2s2_flush as
d2s2_<op> is a generic data I/O operation mapped shown in Fig. 13(b).
into the corresponding LFC/GFAL operation lcg/
gfal_<op>. When a d2s2_<op> modifies the file
system structure (delete, rename, move, mkdir, etc.) it 5. Evaluating the implementation
is necessary to update the file index and all its replica
stored in the SEs by rewriting the such file index, in- The D2 S2 gLite implementation has been evaluated
voking a LFC lcg_del followed by a lcg_cr and through several experiments and measurements. Ac-
consequently lcg_rep as depicted in Fig. 12. cording to the classification done in [29] the technique
14 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system

Fig. 12. d2s2_<op> implementation on gLite.

(a)

(b)

Fig. 13. implementation of d2s2_flush (a) and d2s2_finalize (b) D2 S2 primitives on gLite.

here implemented can be classified as replication with (220 B) have been performed. The results thus obtained
lockboxes, which ensures the highest level of secu- have been compared against the LOCAL, GFAL and
rity (1). Thus, we have addressed our investigation on encrypted GFAL file system primitive results. In the
evaluating the impact, in terms of performance, of en- experiments we have measured the invocation response
suring the highest security (Section 5.1). Then, in Sec- time Tr , i.e. the time elapsed from launching the oper-
tion 5.2 we have evaluated the performance and relia- ation till the reply is received.
bility of D2 S2 on real applications, also comparing it More specifically we have performed the same mea-
against the former Grid secure storage system (GS3 ) surements of [8], where we evaluated GS3 at the basis
implementation. of the D2 S2 gLite implementation. Thus, in the exper-
iments we have disabled the cache in order to effec-
5.1. Functional tests tively evaluate the actual operation response time, al-
ways operating on new files. In order to provide useful
In order to assess the performance of D2 S2 13 ex- measures, we have repeated each test 10000 times, cal-
periments consisting in invoking the main D2 S2 op- culating the average value of the results thus obtained.
erations (read, write, and delete) varying the file size One of the aims of the experiments on D2 S2 is the
from 256 Bytes (28 B) and doubling it up to 1 GBytes comparison against the GS3 implementation, in order
 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system 15

to understand the impact on performance of data re-

dundancy and corresponding LFC management. Fur-
thermore, in the D2 S2 we consider a wider range of
data size than in GS3 (1 GB vs 128 MB) in order to
provide effective information about the performance of
big data transfers on the system as, for example, those
related to virtual machine migration.
Figure 14 shows the results obtained by the exper-
iments on the read, write and delete operations. All
the curves in the graph follow a similar trend since
they mainly implement data transfers, which Tr can be
roughly approximated by the formula:
(a)
Size
Tr = Lat +
Bw
where Lat is the latency, the constant time to access
the disk/network, Bw is the bandwidth, the maximum
data transfer rate, and Size is the amount of data to be
transferred.
According to this formula, the resulting curves are
characterized by an initial zone where the response
time is almost constant since Lat  Size/Bw, followed
by a zone in which Size/Bw becomes predominant pro-
ducing a quick rise. Since a delete operation just sends
a command to the server, the amount of data exchanged
by client and server does not depend on the amount of (b)
data to delete, i.e. Size is constant, and accordingly the
Tr is almost constant as shown by the corresponding
diagram of Fig. 14(c).
With specific regards to the read operation, the re-
sults of the D2 S2 shown in Fig. 14(a) are compara-
ble to the GFAL and E-GFAL ones. This is due to the
fact that D2 S2 read operations do not have to update
the file index, and therefore they only have one GFAL
open/access operation, as in GFAL and E-GFAL. How-
ever, the gap between D2 S2 and GFAL performance
increases with the dataset size. This may involve prob-
lems in the management of huge datasets, especially in
government systems with very high and bursty work-
load. However, D2 S2 allows providing adequate guar- (c)
antees on information security, overperforming the
Fig. 14. D2 S2 read (a), write (b) and delete (c) performance.
simple encryption implementation E-GFAL. If no se-
curity requirements are specified the GFAL storage
system can be used providing better performance than two consecutive write operations have to be performed.
D2 S2 . However, the presence of the file index table allows im-
The results obtained by the write operation tests are plementing the file modification/rewriting capability.
shown in Fig. 14(b) and highlight that D2 S2 is consid- Analogously, in Fig. 14(c) we can observe a great
erably slower than GFAL, E-GFAL and LOCAL calls. gap between the D2 S2 performance and the others:
This is due to the fact that, each time a D2 S2 write op- a D2 S2 delete operation requires updating the remote
eration is performed, it is also necessary to update the storage system file index after removing a file. This
file index stored into the storage element, and therefore requires further GFAL file open and write operations
16 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system

Table 2
thus increasing the overall elaboration time. D2 S2 primitive headers
With regards to the GS3 implementation, it is possi-
Tr (sec) Reliability
ble to observe that D2 S2 provides better performance
File system Gcc SQL SQL SQL
than the former. This is probably due to the impact of
insert update delete
data redundancy that allows optimizing data transfer to Local 0.019 0.006 0.007 0.008 1
and from the SE through LFC. GS3 12.535 15.248 12.347 11.123 0.999525
Another interesting information that can be obtained D2 S2 no cache 47.528 21.423 19.357 18.947 0.999875
by the diagrams concerns big data transfers in D2 S2 : to D2 S 2 10.733 12.662 10.287 9.709 0.999975
read a file of 1 GB 100 sec. are required, while 180 sec.
for writing it, approximately. This means that, for ex- D2 S2 , demonstrating its effectiveness. With regards to
ample, a virtual machine (status or difference) migra- the reliability, a long-term test has been performed in
tion in D2 S2 can be actually performed in a reasonable order to adequately evaluate our implementation. More
time ensuring the highest level of security achievable. specifically we have performed 10000 invocations per
operation, in total 40000 tests on each implementation.
5.2. Application tests Among the N = 40000 operations performed we have
collected the number of successful invocations S, thus
To evaluate the D2 S2 behavior we then select two identifying the reliability R as:
I/O bound applications: Gcc (GNU C compiler) and
S
DB-SQLite (on-file data base). In the former case, the R=
compilation of a 10 KB source file located on the re- N
mote storage system through Gcc has been considered. In this way we have obtained the results showed in
The other analysis has instead measured the response the rightmost column of Table 2, by which we can ap-
time of insert, update and delete queries on a remote preciate the impact of the redundancy management on
SQLite DB. Both the performance and the reliability D2 S2 . Indeed, the reliability measure corresponding to
of such operations have been investigated, comparing GS3 is significantly lower than the D2 S2 ones. Fur-
the full D2 S2 implementation against a modified D2 S2 thermore, also in this case the impact of the cache on
version in which the cache has been disabled, against D2 S2 reliability is highlighted by the experiments. In
the former GS3 implementation and also against lo- fact, the system experienced just 1 failure in the cache-
cal file system operations. This allows evaluating the enabled D2 S2 implementation (N − S = 1), while
impact of redundancy management on the D2 S2 com- 5 failures affected the same implementation without
pared to the preliminary GS3 implementation. More- cache and 19 failures have been experienced for the
over, we have also evaluated the cache impact on D2 S2 , cache-enabled GS3 implementation. From the data col-
even if in a particular case characterized by specific lected it is not possible to understand the cause of such
conditions as detailed above. failures, but very likely the network is the first cause
The same testbed of the previous analysis has been of failure. Such effect in particular affects the GS3 im-
used for the tests, each repeated 10000 times. The re- plementation since it does not provide any redundant
sults thus obtained are shown in Table 2. source for retrieving the requested data. In fact, a 4-
In terms of performance we can observe that the GS3 nine reliability is achieved by D2 S2 against a 3-nine
implementation is slower than the D2 S2 one, thus con- reliability by GS3 .
firming the positive impact of redundancy/LFC man- This is also confirmed by the results shown in Ta-
agement on the implementation. Furthermore, with re- ble 2, where no failures are experienced by using the
gards to the cache, the results show that the request re- local file system.
sponse time of the D2 S2 operations without consider-
ing the cache is almost the double of that with cache,
in case of SQLite operations, while it is 4 times greater 6. Conclusions
than the cache one in the case of Gcc compilation. Al-
though, as already discussed, this strongly depends on In this work we proposed the dependable distributed
the application data locality in processing (Gcc com- storage system, a reliable and secure (encrypted) stor-
pilation has greater data locality than SQLite DB ap- age system.
plications), the results allow us to definitely state that Reliability in D2 S2 is achieved by data redundancy
the cache has a significant impact on performance in at different level: RAID policies are implemented in
 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system 17

order to improve internal reliability, while physical [6] L. Brunie, L. Seitz and J.-M. Pierson, Key management for
redundancy and adequate replica management is re- encrypted data storage in distributed systems, in: IEEE Secu-
rity in Storage Workshop, Washington DC, IEEE Computer
quired to prevent common causes of failures.
Society, USA, (October 2003), 20–30.
With regards to security, the algorithm is based [7] A. Chakrabarti, Grid computing security, Springer-Verlag,
on the idea of combining symmetric and asymmetric New York, Inc., Secaucus, NJ, USA, 2007.
cryptography following a two-step approach. The sym- [8] V.D. Cunsolo, S. Distefano, A. Puliafito and M. Scarpa, GS3:
metric cryptography is directly applied to data, while A grid storage system with security features, J Grid Comput
8(3) (2010), 391–418.
the DataKey is (asymmetric) encrypted by the user [9] S. Distefano and A. Puliafito, Achieving distributed system
public key. Decryption and encryption are performed information security, Seventh International Conference on
in the user node, and both the key and the data are Computational Intelligence and Security (2011), 526–530.
managed by avoiding malicious accesses. In this way [10] L. Guy, P. Kunszt, E. Laure, H. Stockinger and K. Stockinger,
Replica management in data grids, Global Grid Forum In-
owner(s) can exclusively access their data. In order to formational Document, GGF5, Tech Rep (2002).
share such data with other users it is necessary to store [11] S. Garfinkel, PGP: Pretty good privacy, O-Reilly Media
in the storage system copies of the DataKey encrypted (November 1994).
by the users private keys. [12] B. Han, S. Yang and X. Yu, Greplica: A web-based data grid
The D2 S2 algorithm has been implemented into the replica management system, in: Semantics, Knowledge and
Grid, SKG’05 First International Conference on (November
gLite middleware as a dependable file system on top of 2005), 114.
LFC and GFAL libraries. This choice allows protecting [13] K. Hwang, Y.K. Kwok, S. Song, M. Cai, Y. Chen, Y. Chen,
both data/files and also their structure, the whole file R. Zhou and Lou: Gridsec: Trusted grid computing with se-
system, achieving reliability through LFC capabilities. curity binding and self-defense against network worms and
ddos attacks, in: International Workshop on Grid Computing
This implementation has been evaluated in terms of Security and Resource Management (GSRM’05) (2005), 187–
both performance and reliability by mainly consider- 195.
ing read, write and delete operations, providing satis- [14] L. Junrang, W. Zhaohui, Y. Jianhua and X. Mingwang, A
factory results also in case of real applications (GCC, secure model for network-attached storage on the grid, in:
SQLite). SCC’04: Proceedings of the 2004 IEEE International Confer-
ence on Services Computing, Washington, DC, USA: IEEE
Further development and future work on D2 S2 will Computer Society (2004), 604–608.
mainly focus on its implementation and adaptation to [15] A. Kerckhoffs, La cryptographie militaire, Journal des Sci-
Cloud computing, dealing with optimization on perfor- ences Militaires IX (1883), 5–83.
mance and reliability in order to provide a valid sup- [16] Y. Lim, H.M. Kim, S. Kang and T. Kim, Vehicle-to-grid com-
munication system for electric vehicle charging, Integrated
port for big data management, a strategic challenge in Computer-Aided Engineering 19(1) (2012), 57–65.
the current ICT scenario. [17] N. Looker and J. Xu, Dependability assessment of grid
middleware, in: DSN’07: Proceedings of the 37th Annual
IEEE/IFIP International Conference on Dependable Systems
References and Networks, Washington, DC, USA: IEEE Computer Soci-
ety, (2007), 125–130.
[1] S.V. Adve and K. Gharachorloo, Shared memory consistency [18] R.L. Rivest, A. Shamir and L.M. Adelman, A method for ob-
models: A tutorial, Computer 29(12) (December 1996), 66– taining digital signatures and public-key cryptosystems, Com-
76. doi: 10.1109/2.546611. mun ACM 21(2) (1978), 120–126.
[2] R. Ball, J. Grant, J. So, V. Spurrett and R. De Lemos, De- [19] D. Scardaci and G. Scuderi, A secure storage service for the
pendable and secure distributed storage system for ad hoc glite middleware, in: International Symposium on Informa-
networks, in: Proceedings of the 6th International Confer- tion Assurance and Security, Los Alamitos, CA, USA: IEEE
ence on Ad-hoc, Mobile and Wireless Networks (ADHOC- Computer Society, (2007), 261–266.
NOW’07), E. Kranakis and J. Opatrny, eds, Springer-Verlag, [20] A. Shamir, How to share a secret, Commun ACM 22(11)
(2007), 142–152. (1979), 612–613.
[3] Z. Bankovic, J.M. Moya, A. Araujo, D. Fraga, J.C. Vallejo [21] Z. Sun, J. Shen and J. Yong, A novel approach to data dedu-
and J.M. de Goy, Distributed intrusion detection system for plication over the engineering-oriented cloud systems, Inte-
wireless sensor networks based on a reputation system cou- grated Computer-Aided Engineering 20(1) (2013), 45–57.
pled with kernel self-organizing maps, Integrated Computer- [22] D. Thain and M. Livny, Parrot: Transparent user-level mid-
Aided Engineering 17(2) (2010), 87–102. dleware for data-intensive computing, Scalable Computing:
[4] G. Belalem and Y. Slimani, A hybrid approach to replica man- Practice and Experience 6(3) (2005), 9–18.
agement in data grids, Int J Web Grid Serv 3(1) (2007), 2–18. [23] P. Townend, N. Looker, D. Zhang, J. Xu, J. Li, L. Zhong and J.
[5] C. Blanchet, R. Mollon and G. Deleage, Building an en- Huai, Crown-c: A high-assurance service-oriented grid mid-
crypted file system on the egee grid: Application to protein dleware system, in: HASE’07: Proceedings of the 10th IEEE
sequence analysis, in: ARES’06: Proceedings of the First In- High Assurance Systems Engineering Symposium, (2007).
ternational Conference on Availability, Reliability and Secu- [24] P. Townend and J. Xu, Dependability in grids, IEEE Dis-
rity, Washington, DC, USA: IEEE Computer Society, (2006), tributed Systems Online 6(12) (2005), 1–7.
965–973. [25] S. Tuecke, V. Welch, D. Engert, L. Pearlman and M. Thomp-
18 S. Distefano and A. Puliafito / Information dependability in distributed systems: The dependable distributed storage system

son, Internet x.509 public key infrastructure (pki) proxy cer- [29] L. Xiao, Y. Ye, I.L. Yen and F. Bastani, evaluation and com-
tificate profile, RFC 3820 (Proposed Standard), jun 2004. parisons of dependable distributed storage designs for clouds,
Available: http://www.ietf.org/rfc/rfc3820.txt Accessed May high-assurance systems engineering (HASE), IEEE 12th Int
2013. Symposium on (2010), 152–161. doi: 10.1109/HASE.2010.
[26] K. Ueno and S. Furuhashi, Cagra dependable distributed stor- 22.
age system for 3D computer graphics rendering, in: Pro- [30] Federal information processing standard publication 197.
ceedings of the 2009 Software Technologies for Future De- 2001.
pendable Distributed Systems (STFSSD ’09 IEEE Computer [31] GPG – GNU Privacy Guard – Documentation Sources.
Society, Washington, DC, USA, 179–183. DOI=10.1109/ GnuPG.org. [Online]. Available: http://www.gnupg.org/
STFSSD.2009.19 documentation/.
[27] S. Vazhkudai, S. Tuecke and I. Foster, Replica selection in [32] gLite middleware website, http://glite.cern.ch, Accessed May
the globus data grid, in: CCGRID’01: Proceedings of the 1st 2013.
International Symposium on Cluster Computing and the Grid, [33] gLite Middleware technical committee, the grid file access-
Washington, DC, USA: IEEE Computer Society (2001), 106. GFAL C API description. CERN, Geneve, http://
[28] D. Watson, Y. Luo and B.D. Fleisch, The Oasis+ dependable griddeployment.web.cern.ch/griddeployment/gis/GFAL/
distributed storage system, Proceedings of the 2000 Pacific GFALindex.html. Accessed May 2013.
Rim International Symposium on Dependable Computing Los
Angeles, CA, (Dec 2000), 18–19.
Copyright of Integrated Computer-Aided Engineering is the property of IOS Press and its
content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.