Facebook Distributed System Case Study For Distributed System Inside Facebook Datacenters PDF
Facebook Distributed System Case Study For Distributed System Inside Facebook Datacenters PDF
Facebook Distributed System Case Study For Distributed System Inside Facebook Datacenters PDF
ISSN 2347-4289
Abstract: Facebook is recognized as the largest online social network system in the last few years, which is come up with billions numbers of users in
the last 2013. The system is recognized as distributed system in its design, infrastructure and architecture .The datacenters behind this network system
are huge, robust, keeping the system scalable, reliable, secure, and let the Facebook accessible from anywhere with highly availability
Keywords: facebook; distributed system; availabilty; scalability; Hadoop ; social cloud ;Hive ;HDFS ;
Notifications: keeping track of all the most recent actions Messages service which give a user an account
or updates. It is an indicator to inform the user that an under the Facebook.com ,This system is available to
action has been added to profile page, his or her wall or all of the users, providing text messages, instant
time line, any Comment or like, shared media that being messaging, emails and regular messages, every user
tagged in [9]. has a strong controls over his mail box ,it was the
foundation of a Social Inbox [2] .
Networks, groups, and pages: Facebook allow users to 4- Facebook Applications: the biggest motivation and
build their networks ,groups and creating pages which convenient integration between many application and
combine them around an idea or specific community .they the web site interface of the Facebook pages, which
can used for posting items or issuing messages for a group leads many such users to still connected with
of users who join these communities. Facebook home pages and being in touch with many
advertisers ,besides the applications are games , this
Messaging and outbox: a service allows users to send commercial OSN attracts users and advertiser to be
messages to each other. Users can send a message to any there ,the integration between the games
number of friends at a time. Managing messages also components and users, their profiles ,images ,lists of
provided .By the year of 2010, Facebook announced a new their friends and already joined groups ,increase the
Facebook Messages service which give a user an account functionality and integration levels with different
under the Facebook.com ,This system is available to all of components [10].
the users, providing text messages, instant messaging,
emails and regular messages [2][10] . Most Facebook Applications are more simplified than most
casual modern games, requiring an average of one or two-
All these features and more are being served on Facebook click actions and supplying a random outcome mostly
, adding the different applications that are : events ,market independent from skills, usually in a very short span of time
places ,notes ,places, questions ,photos ,videos ,and (seconds). Frequently, the actual gameplay is substituted
Facebook pages ,we are interested on the system features by a text offering a narration of the events and their
that will produce the traffic basically ,we will categorized outcome, as some sort of prize in exchange of the minimal
them in few later lines in major categories in which they will (one click) engagement required. Facebook Applications
help us in system investigations .this categories are based feature several elements of social play, Making the
on the type of data and the communication mechanisms [9] participation of the users Friends is a must, in order to
access the Application, or by proposing primers for
Confrontation with others [2][10] .
batch jobs as well as any ad hoc analysis that the A typical Hadoop environment consists of a master node,
user want to do on historical data sets . and worker nodes with specialized software components.
3- federated Mysql is the data base engine which hold Hadoop consists of multiple master nodes to avoid single
the data bases holding up the whole system [8] point of failure in any environment. The elements of master
.these tier parts are described in the figure (2) . node are:-
Job Tracker: Job tracker interacts with client applications.
3.2 Distributed systems components:- It is distributing Map and reducing tasks to particular nodes
Scalability and reliability are mandatory requirements within a cluster.
according to the globalization of the system, Facebook is Task tracker: it is process receives the tasks from a job
global OSN that serving billions of requests and being tracker in in the master node like Map, Reduce it to specific
responsible for replying back to their requests in just few cluster node and shuffle.
seconds, and not being too late, these requirements need Name node (NN): they are responsible for keeping track for
scalability ability in size, geographically scalability and save each file in Hadoop Distribute File System HDFS ,a client
the robustness of the system [9]. Systems design, big data application contact NN to locate file ,delete ,copy ,or add.
processing and analysis and huge Storage that are Data Node (DN): they are responsible for storing in HDFS ,
examples of these components that are Facebook relying they are keeping indexes for files stored in , they are
on, because of their ability to holding text, multimedia and interact between client applications and the NN .providing
many third party applications and advertisement and put the clients with name of NN that are hold the required data .
them on the surface to the users [8] . Facebook is relying Worker Nodes: they are the servers who are responsible
on Hadoop platform, which is well suited to deal with for processing tasks; each worker (slave) holds DN and a
unstructured text,logs,and events steams , and structured task tracker. See figure (4).
data, as well as when a data discovery process is needed.
it is built for the purpose of handling larger volumes of data,
so preparing data and processing it should be cost
prohibitive [2][3] .
Figure (5): Map /Reduce whole steps Figure (6): HDFS architecture DN and NN
3.2.2 Hadoop distributed file system (HDFS) 3.2.3 Hadoop and Hive
Distributed file system that serve the Facebook is mainly In Facebook Hive is a data warehouse infrastructure built
Hadoop distributed file system (HDFS) ,which is designed on top of Hadoop technology, that provides tools to enable
to run on low-cost hardware ,and being highly fault- easy data summarization, heavily reporting ,adhoc querying
tolerance (as it supports block replication) . HDFS is and analysis of large datasets data stored in Hadoop files
designed to store very large data sets reliably; it is able to HDFS . Providing a mechanism to put structure on this data
stream those data sets at high bandwidth to user and it also provides a simple query language called HiveQL
applications. It used In a large cluster, thousands of servers which is based on SQL and which enables users familiar
are directly attached storage and execute user application with SQL to query this data [1]. In System design of
tasks. By distributing storage and computation across many Facebook without Hive, the same job would take hours if
servers, which give the system ability to dynamically scale not days ,in order to move to the second phase and author
,the resource can grow on demand while remaining in map-reduce process . While Using Hive the task could be
economical at every size and retaining the system available expressed very easily in a matter of minutes. It has been
and reliable l. An HDFS instance may consist of hundreds possible with Hive to bring the immense scalability of map-
or thousands of server machines, each storing part of the reduce to the non-engineering users as well business
file system's data; HDFS is designed more for batch analysts, product managers and the like who, though
processing rather than 5 interactive use by users. The familiar with SQL would be in a very strange environment if
emphasis is on high throughput of data access rather than they were to write map-reduce programs for querying and
low latency of data access, a typical file in HDFS is analyzing data by themselves and without Hive-QL syntax
gigabytes to terabytes in size. HDFS applications need a [1]. Figure (7) show Hive system architecture.
write-once-read-many access model for files. This
assumption simplifies data consistency issues and enables 3.2.4 Apache HBase
high throughput data access [2]. HDFS exposes a file Facebook messaging system has recently added to the
system namespace and allows user data to be stored in application, by the support of Apache HBase which is a
files. Internally, a file is split into one or more blocks and database-like layer built on Hadoop designed to support
these blocks are stored in a set of Data Nodes The billions of messages per day. The applications
existence of a single Name Node in a cluster greatly requirements for consistency, availability, partition
simplifies the architecture of the system. The Name Node is tolerance, data model and scalability. Enhancements made
the arbitrator and repository for all HDFS metadata. The to Hadoop to make it a more effective real time system,
system is designed in such a way that user data never Facebook made many tradeoffs while configuring the
flows through the Name Node [3][4] ,see figure (6). system, to add significant advantages over the shared
MySQL database scheme used in applications at Facebook
[2]. HBase will add the following to Facebook as it moves to
real time rather than being offline ,this emerging
movements are support Facebook billion messages
capacity which will be increased with minimal overhead and
no down time , with Highly write throughput ,efficient and
low-latency that support the strong consistency semantics
within a data center, the efficient random reads from disks ,
and being highly available specially in disaster recovery , Although the CDN regional servers posed an attractive
and fault isolation ,and retaining the atomic read modify solutions for infrastructure expansion another solutions
write primitives .It added a zero downtime in case of mentioned here will serve a good support for the huge
individual data center failure, running on Active-Active growth and datacenters extensions ; TCP proxies and
serving capabilities across different data centers [2]. regional OSN caching servers would be attractive solutions
to enhance the network performance and reduce latency;
unfortunately these solutions are under tacking and are not
being applied yet, which cause slow performance and long
latency measurements in Facebook overall statistics [9]. In
figure(9) : we can see that a user will contact webservers in
U.S ,CDN should maintain connected in more than 4 steps
then CDN complete serving the user requests ,while figure
(10) which use TCP proxy or figure (11) that illustrated the
OSN cache solutions .
servers are served in term of RPC ,this mechanism of 3.4.1 Memcahed servers
communication improved for real time work load since Recent design of Facebook, let Hadoop performing a
Facebook have published Messaging service in later years random access workloads that provides low latency access
of working as online social network ,and being enhance a to HDFS, by using a combination of large clusters of
little bit in Hadoop to be limited with time constrains [2] . MySQL databases and caching tiers built using
Hadoop exploits tcp connections by sending RPCs. When a memcached ,that will be support a better in performance
RPC client detects a tcp-socket timeout limits , it sends a while all results from Hadoop are directed to MySQL or
ping to the RPC server instead of declaring RPC timeout memcached for consumption by the web tier side [2] , see
.now if the server is still alive and could communicate with figure (13) .
clients , client can continue waiting for a response. While in
case of a RPC server is experiencing a communication
burst, a temporary overhead or load, the client should wait
and direct its traffic to the server. And from opposite side in
case of throwing a timeout exception or retrying the RPC
request causes tasks to fail unnecessarily or add additional
load to a RPC server [2]. In another side of system,
choosing infinite wait will have an impact on any application
that has a real time requirement. For example An HDFS
client occasionally makes an RPC to some Data node DN ,
and it is not good when the Data Node fails to respond back
in time and the client is stuck in an RPC. A better scenario
is to fail fast and try a different Data Node DN for either
reading or writing. Hence, Hadoop has the ability for
specifying an RPC-timeout for each request depending on
the job which could be served from application servers or
want to call data base servers that had to call HDFS in
deed .when starting a RPC session with a server; Hadoop
is responsible for these tuning and configurations [2][3][4].
Facebook Messaging service combines existing old fashion
Facebook messages service with e-mail messaging , chat,
and SMS. Hadoop offer a persisting communication Figure (13) : memcached servers
between clients, it added a new threading model also
requires messages to be stored for each participating user Recently, a new generation of applications has been
this feature gives user ability to manage his social inbox applied at Facebook in which requires very high write
account with highly write /read throughput ,the idea of this throughput and cheap and elastic storage, while keeping
threading model As part of the application server low latency and disk efficient sequential and random read
requirements, letting each user be sticky to a single data performance [1][3][4].MySQL storage engines are proven
center at a time [2]. and have very good random read performance, but suffer
from low random write throughput. Scaling up Database
MySQL clusters rapidly is difficult to deal with, because of
the needs to maintain load balancing and have long and
high uptime. Administration of MySQL clusters requires
higher managing overhead and costly hardware [2] [3] [8].
We sum-up the whole system components in the figure (14)
below and listing the major parts that we have discussed in
this paper in table (1).
interrupting Facebook servers ,which is more attractive requests. Being geographically distributed by using
performance behavior, since once a user request an URL centralized data centers located on US and being replicated
for any application, and all communications later are served by distributed CDN, is providing the system the level of
from specific application server hold that application ,this acceptable scalability, with the CDN the system is still
scheme is adding a positive point in design considerations working in an acceptable levels, the TCP proxies and OSN
,Facebook JavaScript (FBJS) are used often to request cache servers will provide the system the up limits
Facebook servers asynchronously and in transparent scalability they are under studying and research and
manner without routing through applications servers [5][6]. unfortunately are not applied yet. Hadoop projects and
whole components are example of success story that
provide Facebook system with its requirements to be the
most popular social network by the year of 2013 ,while
rapidly added services and being occasionally updating
their services ; messaging and chat are examples of these
services that requires Hadoop to do a little bit
enhancements on their design to be real-time system rather
than to work offline processing and save the low latency
issues required to access the HDFS as fast as possible ,
adding RPC timeout as final enhancement . Memcahed
severs are also another example of these enhancements to
decrease the load of accessing the data base in each case
that require access to the data base. Cloud computing is
model example that Facebook used to integrate with its
features and services .this integration is done without any
infrastructure modifications or any architectural changes ,
this is because cloud computing is offering an acceptable
solution for integrating Facebook with such examples of
cloud applications .the most interesting examples of these
solutions the social cloud being built by the virtualization
organizations that provided ,these are being scaled
dynamically and on demand .