Cloud  

Computing
Lecture  2

Datacenter  internals
Amazon  Getting  Started

Dan  Amiga
[email protected]

Dan  Amiga  – IDC  Cloud  Computing  2012

Problems

• Time
• Cost
• Scale
• Storage Size
• Availability
• Connectivity
• Security
• How-to-do-it-right

Dan  Amiga  – IDC  Cloud  Computing  2012

Why Now?

• It’s possible
• Sometimes It’s mandatory
• “Web Space Race”
– Build extremely large datacenters (10,000/0’s)
– Driven by growth (more users, more data)
• Operations & Infrastructure expertise
• Broadband got better

Dan  Amiga  – IDC  Cloud  Computing  2012

 Cloud  Principals

Pay  only  for  what  you  use

Ability  to  scale  up  and  scale  down

Dan  Amiga  – IDC  Cloud  Computing  2012

Cloud Services

Packaged Infrastructure Platform Software

(as a Service)
Software (as a Service) (as a Service)

You manage
Applications Applications Applications Applications
You manage
Data Data Data Data

Runtime Runtime Runtime Runtime

Managed by vendor
Middleware Middleware Middleware Middleware
You manage

Managed by vendor
O/S O/S O/S O/S
Managed by vendor

Virtualization Virtualization Virtualization Virtualization

Servers Servers Servers Servers

Storage Storage Storage Storage

Networking Networking Networking Networking

State of Cloud Computing
• Perceptions > Types
– “The end of software” • Public
– On-demand infrastructure • Private
– Cheaper and better • Internal
• Reality • External
– Hybrid world; not “all-or-nothing” • Hybrid
– Leverage existing IT skills and
investments > Categories
– Seamless user experiences • SaaS
– Evolutionary; not revolutionary • PaaS
• Drivers • IaaS
– Ease-of-use, convenience
– Product effectiveness
– Simplify IT, reduce costs

Dan  Amiga  – IDC  Cloud  Computing  2012

 Amazon Prices March 2012

Questions:
1. Is  this  cheap  or  expensive?  Why?
2. Will  windows  be  cheaper?  Why?
3. Why  are  the  memory  numbers  are  floats?
4. How  can  I  make  this  cheaper?
Dan  Amiga  – IDC  Cloud  Computing  2012
Spot Instances – bid!

Dan  Amiga  – IDC  Cloud  Computing  2012

Dan  Amiga  – IDC  Cloud  Computing  2012
Inside  a  datacenter

Dan  Amiga  – IDC  Cloud  Computing  2012

Dan  Amiga  – IDC  Cloud  Computing  2012
Dan  Amiga  – IDC  Cloud  Computing  2012
Dan  Amiga  – IDC  Cloud  Computing  2012
Virtualization

• Many virtualized X on top of real X

• Types
– Full (VMWare, Intel VT)
– Hardware assist
– Partial
– Paravirtualization (Xen)

Dan  Amiga  – IDC  Cloud  Computing  2012

Virtual Machine Monitors

• VMM is just software

– Applications not modified (so we just deploy our code..)
– Do you see any issues with sharing?

VM0 App0 VM1 App1 VMn Appn

...
Guest  OS0 Guest  OS1 Guest  OSn

Virtual  Machine  Monitor  (VMM)

Platform    HW
Memory Processor/CS I/O  Devices

Dan  Amiga  – IDC  Cloud  Computing  2012

 Virtualization Capabilities
Workload  Isolation Workload  Consolidation

App1 App2 App1 App2 App1 App2 App1 App2

OS OS OS OS1 OS2 OS1 OS2

HW VMM HW1 HW2 VMM

HW HW

Workload  Migration Workload  Embedding

App App App App

OS OS OS1 OS2

VMM VMM VMM VMM VMM

HW1 HW2 HW1 HW2 HW

Virtualization  has  powerful  capabilities

(Slide  taken  from  “Understanding  Intel  VT”)
Dan  Amiga  – IDC  Cloud  Computing  2012
Virtualization Challenges
• Functionality
• Overhead
• Speed & Performance
• Security (Resource Isolation, e.g. UDP)

Dan  Amiga  – IDC  Cloud  Computing  2012

XEN Hypervisor (Amazon)
• Paravirtualization
– Requires modifying the underlying OS to
overcome and simplify virtualization challenges
– One of the reasons not all OS’s are applicable for
EC2
– BVT Scheduling
• Virtualization is for
– CPU
– Memory Increasing  Privilege  Level

Ring  0
– IO Ring  1
Ring  2

Dan  Amiga  – IDC  Cloud  Computing  2012 Ring  3

Amazon Xen benchmarks
• The Impact of Virtualization on Network Performance of Amazon EC2 Data Center (Guohui
Wang T. S. Eugene Ng Dept. of Computer Science, Rice University)

Dan  Amiga  – IDC  Cloud  Computing  2012

Amazon CPU Utilization

• Average 7.3% with possible CPU cap

allocation

Dan  Amiga  – IDC  Cloud  Computing  2012

Warehouse scale computers
• Massive Scale - ~10K-100K servers
• Homogenous hw/sw
• Lots of attention to energy savings and efficiency
• Design goals
– Needs to be highly available (or it hits the news)
– Doesn’t solve all problems
– Competitive (prices always go down)

Dan  Amiga  – IDC  Cloud  Computing  2012

No “welcome to Google” sign

• What is the unit of modularity?

• Sometimes no building..
• http://www.youtube.com/watch?v=zRwPSFpLX8I

Dan  Amiga  – IDC  Cloud  Computing  2012

Dan  Amiga  – IDC  Cloud  Computing  2012
Dan  Amiga  – IDC  Cloud  Computing  2012
Power Usage Effectiveness (PUE)

• Less conversion from of DC/AC is a must

• How efficient you deliver power and cooling
to your servers; 1 is optimal

Dan  Amiga  – IDC  Cloud  Computing  2012

Facebook data centers
• Is Facebook a cloud provider?
• Customized racks (3 columns)
• Next to a rack there is
– DC power strip (in case of utility failure)
– DC UPS
– AC power strip
• 30 Servers per column, 1.5 U per server (why?)
• https://www.facebook.com/note.php?note_id=1
0150144039563920
• http://opencompute.org/specs/
• PUE = 1.07

Dan  Amiga  – IDC  Cloud  Computing  2012

Server Drill Down
• Servers
• Racks (U)
– Rack format
– 40-80 servers
– Communication via
In-rack-switch
• Clusters of racks
– Communication through
Cluster switch (expensive)

Dan  Amiga  – IDC  Cloud  Computing  2012

Google Server

Dan  Amiga  – IDC  Cloud  Computing  2012

Amazon Data Center
• http://huanliu.wordpress.com/2012/03/13/amazon-data-center-size/

• Each EC2 instance is assigned an internal IP address in the form of 10.x.x.x.

• Each server rack is assigned a 10.x.x.x/22 IP address range, i.e., all virtual
machines running on that server rack will have the same 22 bits IP prefix.

• A 10.x.x.x/22 IP address range has 1024 IP addresses, but the first 256 are
reserved for DOM0 virtual machines (system management virtual machine in
XEN), and only the last 768 are used for customers’ instances.

• Within the first 256 addresses, two at address 10.x.x.2 and 10.x.x.3 are reserved
for routers on the rack. These two routers are arranged in a load balanced and
fault-tolerant configuration to route traffic in and out of the rack. I verified that
the uplink capacity from 10.x.x.2 and 10.x.x.3 are roughly 2 Gbps total, further
suggesting that they are routers each with a 1Gbps uplink.

Dan  Amiga  – IDC  Cloud  Computing  2012

Amazon Data Center Size (03/13/2012)
data  center\size #  of  server  racks #  of  blade  servers
US  East  (Virginia) 5,030 321,920

US  West  (Oregon) 41 2,624

US  West  (N.  
630 40,320
California)

EU  West  (Ireland) 814 52,096

AP  Northeast  
314 20,096
(Japan)

AP  Southeast  
246 15,744
(Singapore)

SA  East  (Sao  
25 1,600
Paulo)
Total 7,100 454,400

Dan  Amiga  – IDC  Cloud  Computing  2012

Memory/Storage

• Where is the Lie? What’s on the top?

Dan  Amiga  – IDC  Cloud  Computing  2012

Dan  Amiga  – IDC  Cloud  Computing  2012
Hard drives

Dan  Amiga  – IDC  Cloud  Computing  2012

SSD Hard drive

• SSD FLASH vs SSD DRAM

• Half to third of power
• Reliability and lifetime
• Software Controller
– ECC, Wear leveling, Caching, GC,
• Price per GB will fall below $1 for 1GB
• Dynamo DB

Dan  Amiga  – IDC  Cloud  Computing  2012

Principal of locality + WSC
Latency vs Throughput Architecture consideration

Local Rack Array

Rack 1 30
Servers 1 80 2400
Cores 8 640 19200
DRAM 16 1280 38400
DISK 4000 320000 9600000
DRAM  Latency   0.1 100 300
Disk  Latency 10,000 11,000 12,000
DRAM Bandwidth 20,000 100 10
Disk  Bandwidth 200 100 10
Dan  Amiga  – IDC  Cloud  Computing  2012
Dan  Amiga  – IDC  Cloud  Computing  2012