Wormhole: Reliable {Pub-Sub} to Support Geo-replicated Internet Services

Y Sharma, P Ajoux, P Ang, D Callies… - … USENIX Symposium on …, 2015 - usenix.org
Y Sharma, P Ajoux, P Ang, D Callies, A Choudhary, L Demailly, T Fersch, LA Guz, A Kotulski…
12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15), 2015usenix.org
Wormhole is a publish-subscribe (pub-sub) system developed for use within Facebook's
geographically replicated datacenters. It is used to reliably replicate changes among several
Facebook services including TAO, Graph Search and Memcache. This paper describes the
design and implementation of Wormhole as well as the operational challenges of scaling the
system to support the multiple data storage systems deployed at Facebook. Our production
deployment of Wormhole transfers over 35 GBytes/sec in steady state (50 millions …
Abstract
Wormhole is a publish-subscribe (pub-sub) system developed for use within Facebook’s geographically replicated datacenters. It is used to reliably replicate changes among several Facebook services including TAO, Graph Search and Memcache. This paper describes the design and implementation of Wormhole as well as the operational challenges of scaling the system to support the multiple data storage systems deployed at Facebook. Our production deployment of Wormhole transfers over 35 GBytes/sec in steady state (50 millions messages/sec or 5 trillion messages/day) across all deployments with bursts up to 200 GBytes/sec during failure recovery. We demonstrate that Wormhole publishes updates with low latency to subscribers that can fail or consume updates at varying rates, without compromising efficiency.
usenix.org
Showing the best result for this search. See all results