Application Note and Technical

Overview
Release Dell R720xd
April 09, 2014
Google Global Cache
[email protected]
Contents
1 Introduction 1
2 Overview 3
2.1 GGC Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 How GGC Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Determining where to serve users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3 Equipment Provided by Google 5
3.1 Typical Server Specications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4 Conguration Overview 7
4.1 The Peering Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.2 BGP Session Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5 Network Connectivity 9
5.1 Access Control Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5.2 Proxies and Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5.3 Conguration Options with Multiple Cache Nodes . . . . . . . . . . . . . . . . . . . . . . . . . 10
6 Maintenance and Support Overview 11
7 Frequently Asked Questions 13
8 Further Reading 15
8.1 Google Global Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
8.2 Server Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
i
ii
CHAPTER 1
Introduction
The explosion of broadband access and rich multimedia content continually increases the demand on service
provider networks. Google Global Cache (GGC) allows you to serve Google content, primarily video, from the
edge of your own network.
This eases congestion within your network, and reduces the amount of trafc on your peering and transit links.
In short, GGC saves you money while improving the experience of your users.
Note: The details in this document are subject to change. This agreement and associated activity is commercially
condential.
1
Application Note and Technical Overview, Release Dell R720xd
2 Chapter 1. Introduction
CHAPTER 2
Overview
2.1 GGC Features
Transparent to users
Google transparently serves your users requests from caches inside your network.
Reduced trafc through your network
Although cache hit rates can vary (based on the unique consumption patterns of end users on each operators
network), typically between 70-90% of Google cacheable trafc can be served from GGC.
Robust:
A GGC installation provides resilience through multiple levels of redundancy which are transparent to users.
Google takes care to ensure that your users are always served by the best available node, even in the event of
a temporary outage. If a GGC node is unavailable for any reason, user requests are automatically redirected
to other nodes in your network.
When no further nodes exist in your network, your users are automatically redirected to other Google serving
locations (same behavior as having no GGC node).
Easy to set up:
Google provides the server hardware, and takes care of the remote conguration of the cache once you have
completed the initial setup. You only need to provide rack space, a keyboard and monitor, and a connection
to your network.
2.2 How GGC Works
When a user requests cacheable Google content, such as video or image data:
GGC application will determine whether the user requesting the content is authorized to access the GGC
node. If not, the user is redirected to other Google serving locations.
The GGC application will then determine if the requested content should be served from a GGC node inside
your network. If not, the user is redirected to other Google serving locations.
If the GGC node already has the requested content in its local cache, it will serve the content directly to the end
user, improving the user experience and saving transit costs.
If the content is not stored on the GGC node, an algorithm will determine whether the content and the request
meet the criteria for caching. Cacheable content is then fetched from the Google network and served to the user.
Should the caching criteria not be met, the users request is redirected to other Google serving locations.
3
Application Note and Technical Overview, Release Dell R720xd
For more information on how the GGC application determines which users it should serve, please refer to (BGP
Session Overview) in the Conguration chapter.
2.3 Determining where to serve users
Users are directed towards specic GGC nodes based on their Source IP and Resolver IP addresses. The GGC
node can also determine if a user is permitted to request content from it, and redirect users that are not.
For this process to work, the GGC node requires a list of User and Resolver prexes that you would like served
by the node. You must provide this information by advertising the prex list to the node through a dedicated BGP
session. Note that the BGP session is not used for traditional routing, it is only used to collect the list of prexes
you authorize to be served by the node.
User mapping variations can occur in the following scenarios;
Another GGC location has a higher prex specicity (most specic prex wins)
Another GGC location has a Shorter AS path (shortest path wins)
Multiple GGC locations with equal criteria (location offering currently best user experience wins)
4 Chapter 2. Overview
CHAPTER 3
Equipment Provided by Google
A GGC node can comprise between 3 and 8 servers, which Google will provide. An installation can consist of
many GGC nodes working together (please refer to section 4.6 for further details on multi-node congurations).
Each GGC node will require space, power, and network connectivity (exactly how much will vary depending on
the size of the conguration).
Each server supports multiple (up to 4) bonded Gigabit Ethernet interfaces. Link Aggregation Control Protocol
(LACP) is used between each GGC machine and your network switch to achieve the bonding.
Google will work with you to determine the best conguration for your network.
Table 3.1: Power Requirements
# Servers Rack Space Power Peak Power # GE ifs IPv4 Subnet IPv6 Subnet
3 6RU 900W 1200W 12 /26 /64
4 8RU 1200W 1600W 16 /26 /64
6 12RU 1800W 2400W 24 /26 /64
8 16RU 2400W 3200W 32 /26 /64
Each machine has an A and B power feed. Each feed should ideally be plugged into an independent power source.
The GGC node requires a dedicated subnet and broadcast domain. Each server has a single physical address and
a number of virtual IP addresses (VIPs):
The physical address is used for monitoring and management of the machine itself.
All trafc is served from the VIPs
VIPs can move between machines in the event of equipment failure or during maintenance. The GGC system is
designed with multiple levels of redundancy. Content and user requests are spread across all VIPs, so if a server
failure occurs, another server in the GGC node can immediately take over its VIPs and respond to the users
requests. If a server is unable to respond to a request, it will redirect the users back to Google.
External monitoring systems are used to ensure that no trafc is sent to an unavailable node.
3.1 Typical Server Specications
2U Rack-mountable chassis (28.46/723mm D x 17.48/444mm W x 3.44/87.3mm H)
Server weight: 71.5lbs/32.5kg each
Input Power: 100/240 VAC (each server has redundant power with IEC 60320 C13 to C14 cords)
Heat dissipation: 2891 BTU/h
4 x 1000Base-T copper Gigabit Ethernet interfaces
5
Application Note and Technical Overview, Release Dell R720xd
6 Chapter 3. Equipment Provided by Google
CHAPTER 4
Conguration Overview
4.1 The Peering Portal
Google maintains a web portal at http://peering.google.com/. Use this site to provide node conguration and
shipping information.
An initial user is invited to access the online portal by the Google Global Cache team. After this user accepts the
GGC agreement, that person can invite additional users.
In order to properly congure the GGC node, Google requires the following information:
Maximum bit rate of trafc you want to be sent from the GGC node towards your users.
IP subnet addresses (IPv4/IPv6) for the GGC node as described in chapter 3
Peer ASN and IP address (IPv4/IPv6) for the BGP feed.
Technical point(s) of contact, in the event physical access is required.
Shipping instructions, contacts, and addresses.
4.2 BGP Session Overview
GGC node requires a list of User and Resolver prexes that you would like served by the node, and will establish
a BGP session with your network to collect this information. Through this session, you will advertise the list of
User and Resolver prexes that you authorize to be served by the node.
The BGP session is not used for traditional routing, it is only used to collect the list of prexes you would like
served by the node, so any transient BGP apping is not considered an issue.
Note: - The node does not learn routes via BGP, a default route is used for all outgoing trafc. - The node does
not advertise any prexes via BGP. - The BGP session is not used to determine if the node is available. - BGP
multihop is supported.
Detailed conguration information is provided in the Installation Guide
http://peering.google.com/static/downloads/GGCInstallation-R720.pdf
Note: To achieve maximum reliability, the GGC system does not interpret an interruption of the BGP session as
a loss of the node. The node will continue to serve based on the most recent valid feed received until the session
is restored.
7
Application Note and Technical Overview, Release Dell R720xd
8 Chapter 4. Conguration Overview
CHAPTER 5
Network Connectivity
The following is intended as an overview. Detailed conguration information can be found in the Installation
Guide.
Each server in the GGC node is connected to a single ISP-provided Ethernet switch via (up to) 4 x Gigabit Ethernet
interfaces with LACP link aggregation. In addition VRRP is used to move VIPs between servers in the event of
equipment failure
See the illustration Network Connectivity Diagram.
Figure 5.1: Network Connectivity Diagram
9
Application Note and Technical Overview, Release Dell R720xd
5.1 Access Control Lists
Access Control Lists (ACLs) are not recommended on network equipment serving the GGC node (an IP rewall
runs on each server in the GGC node).
If ACLs are used, the following ports must be allowed for the entire subnet:
Inbound and outbound HTTP and HTTPS (TCP/80, TCP/443)
Inbound and outbound ICMP
Inbound and outbound SSH (TCP/22)
Inbound and outbound insecure and secure QUIC (UDP/80, UDP/443)
Outbound DNS (UDP/53 and TCP/53)
Outbound NTP (UDP/123), with no restriction on source port
Outbound BGP (TCP/179)
The node must be reachable from any IP on the Internet.
Note: Future services may require additional ports.
5.2 Proxies and Filters
No transparent proxies or lters may be placed in the path of communications between the GGC Node and
Googles back-end servers.
5.3 Conguration Options with Multiple Cache Nodes
Some service provider networks require multiple cache nodes. There are two scenarios for multiple node installa-
tions, each of which is congured slightly differently.
Multiple caches nodes can be deployed in a single logical location or region for increased serving capacity:
* The cache catalog is automatically spread across the nodes in that location * In the event that a node (or
part of a node) becomes unreachable, requests can be served either from another logical location in your
network (if you have multiple logical locations) or from caches on Googles network
If your network is divided into multiple regions with limited inter-region bandwidth and/or high inter-region
latency, cache nodes can be deployed in each region: * Each regions nodes are congured as a separate
logical location * In the event that a node (or part of a node) becomes unreachable, requests can be served
from nodes in another region (or logical location) in your network or from caches on Googles network
Further information can be found in the MultiNode Concepts Deployment Guide Further Reading.
10 Chapter 5. Network Connectivity
CHAPTER 6
Maintenance and Support Overview
Googles GGC operations team remotely monitor the node for server, connectivity, and application health.
Server and Application issues are handled by the GGC Operations team (including ordering of replacement hard-
ware), with additional support from your (onsite) technical contacts in the event that physical server interaction is
required.
Connectivity issues within your network are reported by the GGC Operations team to your technical contacts for
resolution.
Specic details of Support and Maintenance responsibilities are provided in the Contract of Service.
11
Application Note and Technical Overview, Release Dell R720xd
12 Chapter 6. Maintenance and Support Overview
CHAPTER 7
Frequently Asked Questions
What is the expected hit rate of the cache?
Cache hit rate will vary by network based on the number of users served by the cache, their usage
patterns, the size and type of GGC node, and the number of nodes deployed in your network. We
have typically seen hit rates of 70% to 90%.
Will the users be required to make any changes to take advantage of GGCs?
No, users do not have to make any changes to take advantage of the GGC node. Requests are routed
to the node automatically by Googles servers. If the GGC node is unavailable for any reason, user
requests will be sent to another node or directly to Google.
What changes are required in our network or DNS infrastructure to send trafc to the GGC node?
You must ensure that the GGC node is reachable from the Internet, with no lters or proxies in the
path. A BGP session is also required between the node and your BGP peer, so that you can advertise
the list of prexes that you would like served by the node. Further information can be found in the
Installation Guide.
We provide transit services for other ISPs. Will their end users use the GGC node?
They certainly can, and if your network is their primary path to Google, then they probably should.
If you would like this to happen, you must ensure that their User and DNS resolver IP ranges are
included in the prexes you advertise to the node.
How will hardware and software installation be handled?
Google will deliver the rack-mountable servers to your specied shipping address. You are then
required to physically install these servers into your rack(s) and carry out the initial software instal-
lation. Google takes care of the remaining installation remotely. Further details are available in the
Install Guide.
Can I control the bandwidth used by the GGC node in my network?
Yes, Google works with operators during activation to set the maximum bandwidth level for each
GGC node deployed in an operators network. This process takes into account the nodes congura-
tion and network capacity available to the GGC node. This is generally ramped up over several days.
After activation, this maximum can be changed by contacting the GGC Operations team.
How will Google send content to the GGC Node? Will there be content pre-loaded on the machines?
The local cache is lled on a read-through basis when content is requested by the end user. If the GGC
node already has the requested content in its local cache, it will serve the content via your network
to the end user, improving the user experience and saving bandwidth. If the content is not stored on
the GGC node, and the content is cache-eligible, the node will retrieve it from Google, serve it to the
user, and store it for future requests. Otherwise, the request will be served from the nearest upstream
node which has the content. No content is pre-loaded.
13
Application Note and Technical Overview, Release Dell R720xd
Who will own the GGC node?
Google retains ownership of the hardware and software that makes up the node.
Will sensitive Personally Identiable Information (PII) be stored on the GGC node?
No, user privacy is of utmost importance to Google. No PII is ever stored on a GGC node.
Will other ISPs customers be sent to the GGC node on our network?
The node gracefully redirects away all requests from users outside of the prexes you are advertising.
For this reason, access control lists limiting IP ranges that can reach the cache are not permitted.
If you provide service to downstream ASNs, you must ensure that their prexes and resolvers are
provided via the BGP feed.node.
What Google services will be supported by the GGC node?
Typically, a majority of the trafc routed through the GGC node is static content such as YouTube
videos and Android Market downloads. Other Google web services, such as Google Search, may
also be proxied and/or cached based on a number of factors, including legal requirements, available
capacity and expected improvement in performance for end users. These services could include (but
are not limited to):
YouTube
Google Search
Google Plus
Google Maps (including map tiles and street view)
Google Earth
Google Docs
Google Scholar
Google News
Android Market
Picasa Web Albums
DoubleClick by Google
Note: Exact services cached and/or proxied by GGC will vary based on Googles legal and commer-
cial requirements, and Googles discretion on possible performance impact of serving trafc through
the node. Please understand that Googles intent is to serve users from the best possible location for
each service, which may or may not be the local GGC node.
Can I talk about GGC with my customers and peers? Can I include references to GGC in my marketing
collateral?
In short, no. We ask your company to consider this agreement and associated activity to be commer-
cially condential.
14 Chapter 7. Frequently Asked Questions
CHAPTER 8
Further Reading
8.1 Google Global Cache
GGC Multi-Node concepts:
http://peering.google.com/static/downloads/GGCMultinodeDeployments.pdf
GGC Install Guide:
http://peering.google.com/static/downloads/GGCInstallation-R720.pdf
Section on caching in Google Peering & Content Delivery:
https://peering.google.com/about/ggc.html
8.2 Server Hardware
Dell PowerEdge R720 and R720xd Technical Guide: http://i.dell.com/sites/content/shared-content/data-
sheets/en/Documents/dell-poweredge-r720-r720xd-technical-guide.pdf
Dell PowerEdge R720 and R720xd Owners Manual: ftp://ftp.dell.com/Manuals/Common/poweredge-
r720_Owners%20Manual_en-us.pdf
15