Scribd
Upload
78 views

Case Study: Amazon AWS: CSE 40822 - Cloud Computing Prof. Douglas Thain University of Notre Dame

Amazon Web Services (AWS) provides on-demand cloud computing platforms and APIs that allow users to access virtual computing resources like servers, storage, databases, and more. Some key services include: - Elastic Compute Cloud (EC2) allows users to launch virtual machine instances with configurable hardware. - Simple Storage Service (S3) provides storage through a global cloud storage service with buckets and objects. - Elastic Block Store (EBS) provides persistent block-level storage volumes for use with EC2 instances.

Uploaded by

awslab8
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
78 views

Case Study: Amazon AWS: CSE 40822 - Cloud Computing Prof. Douglas Thain University of Notre Dame

Amazon Web Services (AWS) provides on-demand cloud computing platforms and APIs that allow users to access virtual computing resources like servers, storage, databases, and more. Some key services include: - Elastic Compute Cloud (EC2) allows users to launch virtual machine instances with configurable hardware. - Simple Storage Service (S3) provides storage through a global cloud storage service with buckets and objects. - Elastic Block Store (EBS) provides persistent block-level storage volumes for use with EC2 instances.

Uploaded by

awslab8
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 33

Case Study: Amazon

AWS
CSE 40822 Cloud Computing
Prof. Douglas Thain
University of Notre Dame
Caution to the Reader:

Herein are examples of prices consulted in October 2014, to give a sense of the
magnitude of costs. Do your own research before spending your own money!
Several Historical Trends
Shared Utility Computing
1960s MULTICS Concept of a Shared Computing Utility
1970s IBM Mainframes rent by the CPU-hour. (Fast/slow switch.)
Data Center Co-location
1990s-2000s Rent machines for months/years, keep them close to the network access
point and pay a flat rate. Avoid running your own building with utilities!
Pay as You Go
Early 2000s - Submit jobs to a remote service provider where they run on the raw
hardware. Sun Cloud ($1/CPU-hour, Solaris +SGE) IBM Deep Capacity Computing on
Demand (50 cents/hour)
Virtualization
1960s OS-VM, VM-360 Used to split mainframes into logical partitions.
1998 VMWare First practical implementation on X86, but at significant performance
hit.
2003 Xen paravirtualization provides much perf, but kernel must assist.
Late 2000s Intel and AMD add hardware support for virtualization.
Virtual-* Allows for the Scale of
Abstraction to Increase Over Time
Run one process within certain resource limits.
Op Sys has virtual memory, virtual CPU, and virtual storage (file system).
Run multiple processes within certain resource limits.
Resource containers (Solaris), virtual servers (Linux), virtual images
(Docker)
Run an entire operating system within certain limits.
Virtual machine technology: VMWare, Xen, KVM, etc.
Run a set of virtual machines connected via a private network.
Virtual networks (SDNs) provision bandwidth between virtual machines.
Run a private virtual architecture for every customer.
Automated tools replicate virtual infrastructure as needed.
Amazon AWS
Grew out of Amazons need to rapidly provision and configure
machines of standard configurations for its own business.
Early 2000s Both private and shared data centers began using
virtualization to perform server consolidation
2003 Internal memo by Chris Pinkham describing an
infrastructure service for the world.
2006 S3 first deployed in the spring, EC2 in the fall
2008 Elastic Block Store available.
2009 Relational Database Service
2012 DynamoDB
Does it turn a profit?
Terminology
Instance = One running virtual machine.
Instance Type = hardware configuration: cores,
memory, disk.
Instance Store Volume = Temporary disk associated
with instance.
Image (AMI) = Stored bits which can be turned into
instances.
Key Pair = Credentials used to access VM from
command line.
Region = Geographic location, price, laws, network
locality.
EC2 Pricing Model
Free Usage Tier
On-Demand Instances
Start and stop instances whenever you like, costs are
rounded up to the nearest hour. (Worst price)
Reserved Instances
Pay up front for one/three years in advance. (Best price)
Unused instances can be sold on a secondary market.
Spot Instances
Specify the price you are willing to pay, and instances get
started and stopped without any warning as the marked
changes. (Kind of like Condor!)
http://aws.amazon.com/ec2/pricing/
Free Usage Tier
750 hours of EC2 running Linux, RHEL, or SLES t2.micro
instance usage
750 hours of EC2 running Microsoft Windows Server t2.micro
instance usage
750 hours of Elastic Load Balancing plus 15 GB data processing
30 GB of Amazon Elastic Block Storage in any combination of
General Purpose (SSD) or Magnetic, plus 2 million I/Os (with
Magnetic) and 1 GB of snapshot storage
15 GB of bandwidth out aggregated across all AWS services
1 GB of Regional Data Transfer
Surprisingly, you cant scale up that
large.
Simple Storage Service (S3)
A bucket is a container for objects and describes location, logging,
accounting, and access control. A bucket can hold any number of
objects, which are files of up to 5TB. A bucket has a name that
must be globally unique.
Fundamental operations corresponding to HTTP actions:
http://bucket.s3.amazonaws.com/object
POST a new object or update an existing object.
GET an existing object from a bucket.
DELETE an object from the bucket
LIST keys present in a bucket, with a filter.
A bucket has a flat directory structure (despite the appearance
given by the interactive web interface.)
Easily Integrated into Web
Applications
<form action="http://examplebucket.s3.amazonaws.com/" method="post"
enctype="multipart/form-data">

<input type="input" name="key" value="user/user1/" />

<input type="hidden" name="acl" value="public-read" />


<input type="hidden" name="success_action_redirect"
value="http://examplebucket.s3.amazonaws.com/successful_upload.html" />
...
<input type="text" name="X-Amz-Credential
value="AKIAIOSFODNN7EXAMPLE/20130806/us-east-1/s3/aws4_request" />
...
<input type="submit" name="submit" value="Upload to Amazon S3" /> </form>

http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-
example.html
Bucket Properties

Versioning If enabled, POST/DELETE result in the


creation of new versions without destroying the old.
Lifecycle Delete or archive objects in a bucket a
certain time after creation or last access or number of
versions.
Access Policy Control when and where objects can
be accessed.
Access Control Control who may access objects in this
bucket.
Logging Keep track of how objects are accessed.
S3 Weak Consistency Model
Direct quote from the Amazon developer API:
Updates to a single key are atomic.
Amazon S3 achieves high availability by replicating data across
multiple servers within Amazon's data centers. If a PUT request is
successful, your data is safely stored. However, information about
the changes must replicate across Amazon S3, which can take
some time, and so you might observe the following behaviors:
A process writes a new object to Amazon S3 and immediately attempts to
read it. Until the change is fully propagated, Amazon S3 might report "key
does not exist."
A process writes a new object to Amazon S3 and immediately lists keys
within its bucket. Until the change is fully propagated, the object might not
appear in the list.
A process replaces an existing object and immediately attempts to read it.
Elastic Block Store
An EBS volume is a virtual disk of a fixed size with a
block read/write interface. It can be mounted as a
filesystem on a running EC2 instance where it can be
updated incrementally. Unlike an instance store, an
EBS volume is persistent.
(Compare to an S3 object, which is essentially a file that
must be accessed in its entirety.)
Fundamental operations:
CREATE a new volume (1GB-1TB)
COPY a volume from an existing EBS volume or S3 object.
MOUNT on one instance at a time.
EBS is approx. 3x more expensive by volume and 10x more expensive by IOPS
than S3.
Use Glacier for Cold Data
Glacier is structured like S3: a vault is a container for an arbitrary
number of archives. Policies, accounting, and access control are
associated with vaults, while an archive is a single object.
However:
All operations are asynchronous and notified via SNS.
Vault listings are updated once per day.
Archive downloads may take up to four hours.
Only 5% of total data can be accessed in a given month.
Pricing:
Storage: $0.01 per GB-month
Operations: $0.05 per 1000 requests
Data Transfer: Like S3, free within AWS.
S3 Policies can be set up to automatically move data into Glacier.
Durability
Amazon claims about S3:
Amazon S3 is designed to sustain the concurrent loss of data in two facilities, e.g. 3+ copies
across multiple available domains.
99.999999999% durability of objects over a given year.
Amazon claims about EBS:
Amazon EBS volume data is replicated across multiple servers in an Availability Zone to
prevent the loss of data from the failure of any single component.
Volumes <20GB modified data since last snapshot have an annual failure rate of 0.1% - 0.5%,
resulting in complete loss of the volume.
Commodity hard disks have an AFR of about 4%.
Amazon claims about Glacier is the same as S3:
Amazon S3 is designed to sustain the concurrent loss of data in two facilities, e.g. 3+ copies
across multiple available domains PLUS periodic internal integrity checks.
99.999999999% durability of objects over a given year.

Beware of oversimplified arguments about low-probability events!


Architecture Center
Ideas for constructing large scale infrastructures using
AWS:
http://aws.amazon.com/architecture/
Command Line Setup
Go to your profile menu (your name) in the upper right hand
corner, select Security Credentials and Continue to
Security Credentials
Select Access Keys
Select New Access Key and save the generated keys
somewhere.
Edit ~/.aws/config and set it up
[default] like
Note the this:
syntax here is different from how
output = json
it was given in the web console!
region = us-west-2
AWSAccessKey=XXXXXX
aws_access_key = XXXXXX
AWSSecretAccessKey=YYYYYYYYY
aws_secret_access_key = YYYYYYYYYYYY

Now test it: aws ec2-describe-instances


S3 Command Line Examples
aws s3 mb s3://bucket
... cp localfile s3://bucket/key
mv s3://bucket/key s3://bucket/newname
ls s3://bucket
rm s3://bucket/key
rb s3://bucket

aws s3 help
aws s3 ls help
EC2 Command Line Examples
aws ec2 describe-instances
run-instances --image-id ami-xxxxx -- count 1
--instance-type t1.micro
--key-name keyfile
stop-instances --instance-id i-xxxxxx

aws ec2 help


aws ec2 start-instances help
Warmup: Get Started with Amazon
Skim through the AWS documentation.
Sign up for AWS at http://aws.amazon.com
(Skip the IAM management for now)
Apply the service credit you received by email.
Create and download a Key-Pair, save it in your home directory.
Create a VM via the AWS Console
Connect to your newly-created VM like this:
ssh -i my-aws-keypair.pem ec2-user@ip-address-of-vm
Create a bucket in S3 and upload/download some files.
Demo Time
http://aws.amazon.com

You might also like