International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169

Volume: 4 Issue: 6 615 618

____________________________________________________________________________________________________________________
Trends in the Solution of Distributed Data Placement Problem

Rohini T V Ramakrishna M V
Asst. Prof, Dept. of ISE,SJBIT, Bengaluru, India Professor, Dept. of ISE
Research Scholar, VTU, Belagavi SJBIT, Bengaluru, India
e-mail: [email protected] e-mail: [email protected]

Abstract Data placement for optimal performance is an old problem. For example the problem dealt with the placement of relational data in
distributed databases, to achieve optimal query processing time. Heterogeneous distributed systems with commodity processors evolved in
response to requirement of storage and processing capacity of enormous scale. Reliability and availability are accomplished by appropriate level
of data replication, and efficiency is achieved by suitable placement and processing techniques. Where to place which data, how many copies to
keep, how to propagate updates so as to maximize the reliability, availability and performance are the issues addressed. In addition to processing
costs, the network parameters of bandwidth limitation, speed and reliability have to be considered. This paper surveys the state of the art of
published literature on these topics. We are confident that the placement problem will continue to be a research problem in the future also, with
the parameters changing. Such situations will arise for example with the advance of mobile smart phones both in terms of the capability and
applications.

Keywords- Data placement; Replication; Distributed system

__________________________________________________*****_________________________________________________

science. This paper deals with the research which has

I. INTRODUCTION addressed the problem for the newly emerged heterogeneous
In the modern computing environment processors, environment as discussed above. In the next section we
memory and internet have become ubiquitous. Companies describe the Google file system, the most significant
such as yahoo, ebay, facebook, google are dealing with the distributed file system today. This is followed by QFS
problem of processing enormous amount of data which is (Quantcast file system) in the section 3 which deals with wide
being produced continuously. For example Google processed area placement of data replicas, in which the main aim is to
20 petabytes of data per day as of 2009 [5, 16]. Similar maintain multiple copies at multiple locations. We follow this
volumes of data are being dealt with by other companies. The with a discussion of automatic and location aware data
main issue here is the amount of data and the speed with placement techniques in section 4.
which data need be processed for user requests. These
companies resorted to non-traditional approach of horizontal II. GOOGLE FILE SYSTEMS
scaling where a large number of commodity hardware is used Google File System(GFS) is designed to provide efficient,
to store and process the data. Google developed Google File reliable access to data using large clusters of commodity
System (GFS), to store their data on a large number of hardware [6]. GFS provides a familiar file system interface.
processors, each of them being a commodity machine [6]. To Files are organized hierarchically in directories and
deal with semi-structured data they developed Bigtable data identified by pathnames. GFS supports the usual operations
model and processing system [3]. Google file system is an to create, delete, open, close, read, and write files and GFS
example of a distributed file system which enables storing and has snapshot and record append operations. Snapshot creates
processing of large volumes of data on thousands of a copy of a file or a directory tree at low cost. Record
commodity processors. In such a system in general, all the append allows multiple clients to append data to the same
processors may not be identical in terms of the processing file concurrently while guaranteeing the atomicity of each
capacity as well as the storage capacity. When a query comes individual clients append. A GFS cluster consists of multiple
to a machine to be processed, it would access the data stored nodes. These nodes are divided into two types: one Master
on various other machines. The access delay will vary node and a large number of Chunkservers. Files are divided
depending on how data is placed or distributed on the into fixed-size chunks. Chunkservers store these chunks.
machines. Accessing data on the disk of the same machine is Each chunk is assigned a unique 64-bit label by the master
fastest. Next is accessing from the same rack followed by node at the time of creation, and logical mappings of files to
machines located in the same data center which have to go constituent chunks are maintained. Each chunk is replicated
through network switches, and lastly from remote location. As several times throughout the network, with the minimum
the time progresses the query pattern changes; in other words being three, but even more for files that have high end-in
the access pattern and nature of queries will vary and not stay demand or need more redundancy.
static. Hence the proper placement of data which enables high
availability with minimum delay is important. The Master server doesnt usually store the actual
chunks, but rather all the metadata associated with the chunks,
The goal of solution to data placement problem is to such as the namespace, access control information, the
achieve best throughput, minimize delay and maximize the mapping from files to chunks, and the current locations of
utilization of resources. The problem of data placement has chunks, what processes are reading or writing to a particular
manifested in various forms for a long time in computer chunk, or taking a snapshot of the chunk pursuant to
615
IJRITCC | June 2016, Available @ http://www.ijritcc.org
_______________________________________________________________________________________
International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169
Volume: 4 Issue: 6 615 618
____________________________________________________________________________________________________________________
replicate it (usually at the instigation of the Master server, The first algorithm returns highest utility value and
when, due to node failures, the number of copies of a chunk replica locations that lead to value. The algorithm chooses
has fallen beneath the set number). All this metadata is kept set of replica locations that lead to optimal solution from
current by the Master server periodically receiving updates the candidate locations. In this algorithm, let L be the
from each chunk server Heart-beat messages. GFS client candidate locations, T is the set of already chosen replica
code linked into each application implements the file system locations and main aim is to maximize the objective
API and communicates with the master and chunk servers to function (delay, availability). In this approach, data access
read or write data on behalf of the application. Clients or pattern is collected by forming cluster of user nodes. For
applications interact with the master for metadata operations, each replica placement plan R, the objective function needs
but all data-bearing communication goes directly to the to be evaluated based on the estimated average data access
chunkservers. GFS typically has hundreds of chunkservers delay and availability. It would be impractical to collect
spread across many machine racks. These chunkservers in information about all the users across data replicas and
turn may be accessed from hundreds of clients from the same store it at a central server. Thus, this approach maintains
or different racks Communication between two machines on for each replica, a summary of the coordinates of the users
different racks may cross one or more network switches. that have recently accessed the replica.
When the user population is scattered over the entire world,
communicating with every user machine over a short network Whenever r new locations of replicas need to be
path is impossible. Large-scale Internet services often solve determined, these summaries are collected at a central
this problem by means of replication. For reliability, each location and then used for estimating the overall data
chunk is replicated on multiple chunkservers The chunk access delay and availability for each replica placement
replica placement policy serves two purposes maximize data under consideration. The second algorithm describes the
reliability and availability, and maximize network bandwidth manner in which the coordinates of users are summarized
utilization. The master re-replicates a chunks soon as the at each replica location and the coordinate of such users are
number of available replicas falls below a user-specified goal. classified int m clusters. The distance between user nodes
This could happen for various reasons: a chunkserver and replica locations is obtained from network coordinate
becomes unavailable; it reports that its replica may be systems RNP [13]. By knowing coordinates of replica
corrupted, one of its disks is disabled because of errors, or the locations, it can predict the nearest replica to access data
replication goal is increased. Finally, the master re-balances with high accuracy.
replicas periodically. It examines the current replica
distribution and moves replicas for better disks pace and load Assaf and Danny deal with replica placement strategy in
balancing. which a replica must be synchronized with the original
content server in order to supply reliable and precise
III. WIDE-AREA PLACEMENT OF DATA REPLICAS service to the client requests [2]. Replica placement across
In a distributed system in general data is often replicated at data centers is a very common approach for improving
multiple locations for the sake of reliability, and kept performance and availability of content services. Content
consistent to serve user requests [18, 7]. The replica replication algorithms deploy a set of servers, distributed
placement serves two purposes: maximize data reliability throughout the data centers network, and replicate the
and availability and maximize network band width. Ping et. relevant data across these servers. Both the time required to
al. address the wide area placement of data replicas for fast access the data and the traffic in the network are reduced
and highly available data access[12]. They have addressed by Replica placement deals with the actual number and
the problem of how to place replicas in wide area networks, network location of the replicas. Clearly, would like to
where data is replicated at multiple locations to serve users minimize the network distance between an email
with lower latency and higher availability. Having replicas application and the closest replica containing the desired
in wide area applications leads to fast and highly available content (in this example the authentication server) and thus
data access [8]. Since it is in-feasible to know in advance having more replicas helps On the other hand, having more
the user access pattern of data, this research focuses on replicas is more expensive so it needs to model the cost and
determining replica locations based on the past data the benefit in a way that can allow to make the appropriate
accesses. A main challenge that arises in this case is to decisions regarding the number and the network locations
analyze data access patterns in an efficient, scalable of the replicas. This problem is strongly related to a family
manner so as to deal with a large number of users. Such of optimization problems generally referred to as facility
analysis of data accesses needs to be done efficiently even location problems [15, 1]. Most of the algorithms neglect
across geographic regions since each data replica can be the cost of keeping the replicas across the network up to
accessed by a different user population. The analysis must date. In this work, A replica must be synchronized with the
facilitate robust estimation of both data access delay and original content server in order to supply reliable and
availability for feasible replica placement scenarios [17]. precise service to the client requests. The amount of
Here optimal replica locations are chosen, further, synchronization traffic across the network depends on the
redirecting the applications requests to a nearby replica. In number of replicas deployed in the network, the topology
this work they proposed two algorithms, one for finding the of the distributed update and the rate of updates in the
optimal replica locations and another to create user cluster content of the server.
node .

616
IJRITCC | June 2016, Available @ http://www.ijritcc.org
_______________________________________________________________________________________
International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169
Volume: 4 Issue: 6 615 618
____________________________________________________________________________________________________________________
Chervenak and Deelman deal with Data Placement For with commodity hardware, that is distributed widely in
Scientific Applications In Distributed Environments [4]. The geographic terms. The parameters of the problem include the
aim of this work is to distribute data and to make it network bandwidth, limitations of maximum capability, as
advantages for application execution. They study the well as delays. In this paper we have provided a survey of the
relationship between data placement services and work flow research literature on the solution techniques and issues
management systems and placement activity data sets are addressed. We discussed the balanced data placement problem
largely asynchronous with respect to work flow execution. and its solution for distributed system. We see this problem
TevFik and Miron focus on the idea of placement of data in will continue to be significant research issue in foreseeable
distributed computing systems similar to the I/O subsystem future with new environments emerging. The smart phone
in operating system [9]. In this data placement strategy getting smarter and more powerful will be a source of more
different data transfer protocols may have different optimum information all over the world, and accordingly new
concurrency level for any two source and destination. They engineering details will emerge for the data placement
have not focused on optimal location with respect user problem. New research will have to address these issues.
access pattern..
REFERENCES
IV. AUTOMATIC DATA PLACEMENT [1] Vijay Arya, Naveen Garg, Rohit Khandekar, Adam Meyerson,
The problem of automatic data placement with replicated key Kamesh Munagala, and Vinayaka Pandit. Local search heuristics
value stores is address by Joao et. al. [11]. The main objective for k-median and facility location problems. SIAM Journal on
is to automatically optimize the replica placement in a way Computing, 33(3):544562, 2004.
[2] Danny Raz Assaf Rappaport. Update aware replica placement.
that locality patterns in data accesses, such that the In Proceedings of the 9th
communication between nodes is minimized. The issues [3] CNSM and Workshops, pages 9299. IFIP, 2013.
addressed are, the placement of the objects generating most [4] Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C Hsieh,
remote operation for each node and combining the usage of Deborah A Wallach, Mike Burrows, Tushar Chandra, Andrew
consistent hashing with al data structure. Fikes, and Robert E Gruber. Bigtable: A distributed
[5] Ann Chervenak, Ewa Deelman, Miron Livny, Mei-Hui Su, Rob
Yu and pan addressed the problem of placement in three Schuler, Shishir Bharathi, Gaurang Mehta, and Karan Vahi.
scenarios. The first case without replicas, the problem is Data placement for scientific applications in distributed
addresses with hyper-graph formulation. In the second environments. In Proceedings of the 8th IEEE/ACM
International Conference on Grid Computing, GRID 07, pages
scenario with replicas, the numbers of replicas allowed are 267274, Washington, DC, USA, 2007. IEEE Computer
considered. In the third scenario replica migration is Society.
considered In this paper [19] data placement problem solved [6] E. F. Codd. A relational model of data for large shared data
using three scenarios, first, for the scenario without replicas, in banks. Commun. ACM 13(6):377387, June 1970.
this scenario placement problem is solved by the hyper-graph [7] Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. The
formulation. In the second scenario, with replicas and the google file system. In ACM SIGOPS Operating Systems
number of replicas allowed are considered. In the third Review, pages 2943. ACM, 2003.
scenario replica migration is considered. Amol et. al. [14] [8] Hui Jin, Xi Yang, Xian-He Sun, and Ioan Raicu. Adapt:
deals with a technique to address the data placement problem Availability-aware mapreduce data placement for non-dedicated
distributed computing. In Distributed Computing Systems
called SWORD: a scalable workload-aware data partitioning (ICDCS), 2012 IEEE 32nd International Conference on, pages
and placement approach. In which the problem addressed in 516525. IEEE, 2012.
OLTP systems. The techniques introduced by them are to [9] Magnus Karlsson and Christos Karamanolis. Choosing replica
significantly reduce the overheads of initial placement of data placement heuristics for wide-area systems. In Distributed
and also during processing of the query. They have used Computing Systems, 2004. Proceedings. 24th International
hyper-graph compression technique over the data items. Conference on, pages 350359. IEEE, 2004.
Further they address data repartitioning technique which [10] Tevfik Kosar and Miron Livny. A framework for reliable and
modifies data placement without complete workload efficient data placement in distributed computing systems.
repartitioning. The technique enables availability and load Journal of Parallel and Distributed Computing 65(10):1146
1157, 2005.
balancing. The availability-aware data placement problem to [11] Michael Ovsiannikov, Silvius Rus, Damian Reeves, Paul Sutter,
improve the application performance without Extra storage Sriram Rao, and Jim Kelly. The quantcast file system.
cost has been addressed by Yang et. al. [7]. They proposed a Proceedings of the VLDB Endowment, 6(11):1092 1101, 2013.
technique of ADAPT which deals with Hadoop framework [12] Joao Paiva, Pedro Ruivo, Paolo Romano, and Lus Rodrigues.
and the performance of ADAPT is evaluated in non-dedicated A uto p lacer: Scalable selftuning data placement in distributed
distributed environment. ADAPT is to migrate data based on key-value stores. ACM Transactions on Autonomous and
the availability of each node, reduce network traffic, improve Adaptive Systems (TAAS), 9(4):19, 2015.
data locality, and optimize the application performance. [13] Fan Ping, Jeong-Hyon Hwang, XiaoHu Li, Chris McConnell,
and Rohini Vabbalareddy. Wide area placement of data replicas
for fast and highly available data access. In Proceedings of the
V . CONCLUSIONS: FUTURE OF PLACEMENT fourth international workshop on Data-intensive distributed
PROBLEM computing, page18. ACM, 2011
Although data placement problem is old (such as in [14] Fan Ping, Christopher McConnell, and Jeong-Hyon Hwang. A
relation to OLTP systems), the engineering details have retrospective approach for accurate network latency prediction.
changed considerably in the present computing environment. In Computer Communications and Networks (ICCCN), 2010
We have massively parallel processing and storage capability
617
IJRITCC | June 2016, Available @ http://www.ijritcc.org
_______________________________________________________________________________________
International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169
Volume: 4 Issue: 6 615 618
____________________________________________________________________________________________________________________
Proceedings of 19th International Conference on, pages 16. [18] Michal Szymaniak, Guillaume Pierre, and Maarten Van Steen.
IEEE, 2010. Latency-driven replica placement. In Applications and the
[15] Abdul Quamar, K Ashwin Kumar, and Amol Deshpande. Internet 2005.Proceedings. The 2005 Symposium on, pages
Sword: scalable workload-aware data placement for 399405. IEEE, 2005.
transactional workloads. In Proceedings of the 16th International [19] Radu Tudoran, Alexandru Costan, Rui Wang, Luc Bouge,
Conference on Extending Database Technology, pages 430441. Gabriel Antoniu, et al. Bridging data in the clouds: An
ACM, 2013. environment-aware system for geographically distributed
[16] David B Shmoys, Eva Tardos, and Karen Aardal. datatransfers. In 14th IEEE/ACM International Symposium on
Approximation algorithms for facility location problems. In Cluster, Cloud and Grid Computing,2014 .
Proceedings of the twenty-ninth annual ACM symposium on [20] Boyang Yu and Jianping Pan. Location-aware associated data
Theory of computing, pages 265274. ACM, 1997 placement for geo-distributed data-intensive applications. In
[17] Michael Stonebraker. Sql databases v. nosql databases. 2015 IEEE Conference on Computer Communications
Commun. ACM, 53(4):1011, April 2010. (INFOCOM), pages 603611. IEEE, 2015.

618
IJRITCC | June 2016, Available @ http://www.ijritcc.org
_______________________________________________________________________________________