Lecture: Developing Storage

Solutions with Amazon S3

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lecture overview

• Part 1: Introducing Amazon S3

• Part 2: Creating Amazon S3 buckets
• Part 3: Working with Amazon S3 objects
• Part 4: Protecting data and managing access to Amazon S3 resources

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 2
Lecture objectives

At the end of this topic, You should be able to

• Describe how Amazon S3 can be used as a storage solution
• Identify Amazon S3 bucket features and components
• Describe the function of the S3 object operations
• Explain how to manage access to Amazon S3 resources

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 3
 • Shared Responsibility
Model
• Authentication with IAM
• Authorization with IAM

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4
Part 1: Introduction to Amazon S3

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon S3

• Object storage service that offers industry-leading

scalability, data availability, security, and performance
• Offers 99.999999999% (11 nines) of durability
Amazon S3 • Provides easy-to-use management features

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 6
Amazon S3

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 7
Amazon S3 use cases

Content storage and Backup and restore; Data lakes and Disaster recovery Static website
distribution archive big data analytics hosting

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 8
 Amazon S3 components

https://s3-<aws-region>.amazonaws.com/<bucket name>/

Bucket Region-specific endpoint

Each bucket is
regional.

https://s3-<aws region>.amazonaws.com/<bucket name>/<object key>

Object Object key example:
preview.mp4

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 9
LO1: Describe how Amazon S3 can be
used as a storage solution
A company is developing a serverless website that consists of images, videos,
HTML pages and JavaScript files. There is also a requirement to serve the
files with lowest possible latency to its global users. Which combination of
services should be used in this scenario? (Select TWO.)
1. Amazon Glacier
2. Amazon CloudFront
3. AWS Elastic File System
4. AWS S3
5. AWS Storage Gateway

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 10
Part 2: Creating Amazon S3 buckets

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon S3 bucket names

• Bucket names must be globally unique.

• Additional rules to follow when choosing bucket names:
• Use 3 – 63 characters.
• Use only lowercase letters, numbers, and hyphens (-).
• Do not use uppercase characters or underscore ( _ )
character.
• Do not use a period (.) when using virtual hosted-style
buckets with Secure Sockets Layer (SSL).

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 12
 Amazon S3 bucket Regions
Deciding factors:
• Latency
• Cost
• Regulatory requirements

# Region & Number of Availability Zones

New Region (coming soon)

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 13
Accessing buckets: Bucket URLs
Path-style URL
• Bucket name is not part of the domain.
• Structure: http://s3-<aws-region>.amazonaws.com/<bucket name>/<object key>
• Example: http://s3-eu-west-1.amazonaws.com/mybucket/cat.jpg

Virtual-hosted-style URL
• Bucket name is part of the domain name in the URL.
• Structure: http://<bucket name>.s3-<aws-region>.amazonaws.com/<object key>
• Example: http://mybucket.s3.eu-west-1.amazonaws.com/cat.jpg

• Useful for hosting a static website (must be enabled)

• Structure: http://<bucket name>.s3-website-<aws-region>.amazonaws.com
• Example: http://catlostandfoundwebsite.s3-website-eu-west-1..amazonaws.com
© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 14
Creating folder structure in buckets: Using
prefixes
Use prefixes to imply a folder structure in an S3 bucket.

Bucket name: scores

Specify prefix: 2019/score/math

Returns the following object keys:

GET Object • 2019/score/math/john.txt
Bucket objects: • 2019/score/math/sam.txt
2019/score/english/john.txt
2019/score/english/sam.txt
2019/score/math/john.txt
2019/score/math/sam.txt
2019/score/summary.txt

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 15
LO2: Identify Amazon S3 bucket features
and components
Which of the following statements are false about the S3 buckets? (Select
all that apply)
1. S3 buckets can be provided with same names in different AWS regions
2. Each S3 bucket can only allow up to 5TB data inside the bucket
3. S3 bucket name can be changed after creation of the bucket
4. S3 buckets don’t provide any limitations regarding the number of objects stored inside
the buckets
5. Virtual-host style URL has been deprecated for the S3 bucket

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 16
Part 3: Working with Amazon S3
objects

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Object keys

• Unique identifier for a bucket object Object key examples

• Key encoding: UTF-8 • AWS-ACD/v1.0/Labs.html
• AWS_ACD/v1.0/Labs.html
• Maximum key length: 1,024 bytes • 2019/score/summary.txt
• Key names • overallsummary.txt
• Alphanumeric characters [0-9, a-z, A-Z]
• Special characters: !, -, _, ., *, ', (, and )
• Path prefixes and delimiters (/)

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 18
Object metadata
Set of key-value pairs that provides additional information about the object

System-defined User-defined
• Information that is controlled by Amazon S3 • Information that you assign to the object
• Object creation date • x-amz-meta key followed by a custom name
• Object size • For example:
• Object version x-amz-meta-alt-name

• Information that you can modify

• Storage class configuration
• Server-side encryption

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 19
 PUT Object
• Upload entire objects to a bucket
• Should use single upload for objects
Size < 5 TB
up to 5 GB in a single PUT operation Size < 5 GB

• Must use multipart upload for larger

objects up to 5 TB (max individual Single Multipart
object size) upload upload
• Multipart upload resumes where you
left off. Recommended if
size > 100 MB

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 20
GET Object

Get complete object Get range of bytes

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 21
SELECT Object

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 22
Versioning

• Way of keeping multiple variants

PUT
of an object in the same bucket. Key = photo.gif
• Use to recover from unintended
user actions and application failures.
• In versioning-enabled S3 buckets,
each object has a version ID. Key = photo.gif
ID = 121212
• Once enabled, you can only Key = photo.gif
suspend versioning (you cannot disable it) ID = 111111

• Object locking is supported on S3 bucket with

versioning enabled
versioned buckets.

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 23
DELETE Object: Versioning disabled

Key: jazz.mp3

Delete Key: jazz.mp3

DELETE

Key: jazz.mp3

S3 bucket with
versioning disabled

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 24
DELETE Object: Versioning enabled

Delete marker
Delete Key: jazz.mp3
Key: jazz.mp3
Version ID: 2222222
Key: jazz.mp3
Version ID: 1111111
Key: jazz.mp3
Version ID: 2222222
Key: jazz.mp3
Key: jazz.mp3 Version ID: 2222222
Version ID: 1111111

S3 bucket with
Delete Key: jazz.mp3
versioning enabled
Version ID:2222222
Key: jazz.mp3
Version ID: 1111111
© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 25
LO3: Describe the function of the S3
object operations
A financial institute is storing all its critical documents in the Amazon S3
bucket which has versioning enabled. The retention period is applied to
documents using object lock, but you are observing some objects are
getting overwritten. Which of the following could be a possible reason?

1. Object should have both retention period & legal hold to prevent from overwriting.
2. Object locks apply to a specific version of an object in a versioned bucket.
3. Object lock should be applied at bucket level to prevent all objects in a bucket from
overwriting.
4. Object locks need to apply all versions of the objects in a versioned bucket.

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 26
Part 4: Protecting data and
managing access to Amazon S3
resources

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data encryption

Securing data in transit Securing data at rest

• SSL-encrypted endpoints with HTTPS • Client-side encryption
• Client-side encryption • Server-side encryption
• Amazon S3-managed keys (SSE-S3)
• AWS KMS-managed keys (SSE-KMS)
• Customer-provided keys (SSE-C)

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 28
User policies and resource-based policies
Identity-Based Policies Resource-Based Policies

{} </> </>
JSON XML XML
</>
Admin User User policy XML
(with Administrator
S3 objects
privileges)
Object ACLs

{} {} </>
JSON JSON XML
{} {}
JSON S3 bucket JSON
Users Bucket
User policies Bucket ACL
policy
© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 24
Access control lists (ACLs)

Read

Full Control
S3 objects
</>
XML

Write

S3 bucket

• Resource-based access policy to manage access at the object or bucket level

• Use to grant basic read/write permissions to other AWS accounts

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 30
Bucket policies

An IAM policy language option {

"Version":"2012-10-17",
that grants granular permissions "Statement": [
{
to Amazon S3 resources. "Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject"],
"Resource":
["arn:aws:s3:::examplebucket/*"]
}
]
}

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 31
 Pre-signed URLs
• Provide access to PUT or GET objects AWS Cloud
without opening permissions to do Availability Zone
anything else.
Request Generates
• Use permissions of the user who creates pre-signed URL URLs
the URL. Instance

• Provide security credentials, a bucket

name, an object key, HTTP method, and Return
expiration date and time. pre-signed URL

• Are only valid until expiration time

(maximum 1 week). GET or PUT objects

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 32
Cross-origin resource sharing (CORS)

Cross-origin resource sharing Enable CORS to allow

(CORS) defines a way for client </> JavaScript to access

resources in S3 bucket
Website hosted in
web applications that are loaded S3 bucket at
in one domain to interact with website.s3.amazonaws.com
JavaScript in
resources in a different domain. webpage
http://www.example.com

<CORSConfiguration>
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
</CORSRule>
</CORSConfiguration>

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 33
LO4: Explain how to manage access to
Amazon S3 resources
A start-up has recently launched a high-quality photo sharing portal
using Amazon EC2 and S3. They noticed that there are other external
websites which are linking and using their photos without permission.
This has caused an increase on their data transfer cost and potential
revenue loss. Which of the following is the MOST effective method to
solve this issue?
1. Enable cross-origin resource sharing (CORS) which allows cross-origin GET requests
from all origins.
2. Block the IP addresses of the offending websites using Network Access Control List.
3. Configure the S3 bucket to remove public read access and use pre-signed URLs with
expiry dates.
4. Use a CloudFront web distribution to serve the photos.
© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 34
Lecture review

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lecture review

• Amazon S3 features, use cases, and basic concepts

• Data protection in Amazon S3 at rest and in transit
• Basic operations on Amazon S3 objects
• Access management of Amazon S3 resources
• Experience developing with Amazon S3 using the AWS SDKs.

• To finish this lecture, complete the knowledge check.

© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 36
Up next: Lecture 5 –
Developing Flexible NoSQL
Solutions with Amazon DynamoDB

© 2019 Amazon Web Services, Inc. or its affiliates. All rights reserved. This work may not be reproduced or redistributed, in whole or in part, without prior written permission from Amazon
Web Services, Inc. Commercial copying, lending, or selling is prohibited. Corrections or feedback on the course, please email us at: [email protected]. For all other
questions, contact us at: https://aws.amazon.com/contact-us/aws-training/. All trademarks are the property of their owners.