Getting Started with AWS Cloud

Reading 1.2: What is AWS? The Six Benefits of Cloud Computing

 Pay as you go. Instead of investing in data centers and hardware before you know how you
What is the Cloud? are going to use them, you pay only when you use computing resources, and pay only for how
As internet usage became more widespread, the demand for compute, storage, and networking much you use.
equipment increased. For some companies and organizations, the cost of maintaining a large physical  Benefit from massive economies of scale. By using cloud computing, you can achieve a
presence was unsustainable. To solve this problem, cloud computing was created. lower cost than you can get on your own. Because usage from hundreds of thousands of
customers is aggregated in the cloud, AWS can achieve higher economies of scale, which
Cloud computing is the on-demand delivery of IT resources over the internet with pay-as-you-go translates into lower pay as-you-go prices.
pricing. You no longer have to manage and maintain your own hardware in your own data centers.  Stop guessing capacity. Eliminate guessing on your infrastructure capacity needs. When you
Companies like AWS own and maintain these data centers and provide virtualized data center make a capacity decision prior to deploying an application, you often end up either sitting on
technologies and services to users over the internet. expensive idle resources or dealing with limited capacity. With cloud computing, these
problems go away. You can access as much or as little capacity as you need, and scale up and
If you ran your application in the cloud, you can replicate the entire environment as often as needed in down as required with only a few minutes notice.
a matter of minutes or even seconds. Instead of physically installing hardware and connecting cabling,  Increase speed and agility. IT resources are only a click away, which means that you reduce
you can logically manage your physical infrastructure over the internet. the time to make those resources available to your developers from weeks to just minutes.
This results in a dramatic increase in agility for the organization since the cost and time it
Using cloud computing not only saves you time from the set-up perspective, but it also removes the takes to experiment and develop is significantly lower.
undifferentiated heavy lifting. If you look at any application, you’ll see that some of the aspects of it  Stop spending money running and maintaining data centers. Focus on projects that
are very important to your business, like the code. However, there are other aspects that are no differentiate your business, not the infrastructure. Cloud computing lets you focus on your
different than any other application you might make: for instance the compute the code runs on. By customers, rather than on the heavy lifting of racking, stacking, and powering physical
removing repetitive common tasks that don’t differentiate your business, like installing virtual infrastructure. This is often referred to as undifferentiated heavy lifting.
machines, or storing backups, you can focus on what is strategically unique to your business and let  Go global in minutes. Easily deploy your application in multiple Regions around the world
AWS handle the tasks that are time consuming and don’t separate you from your competitors. with just a few clicks. This means you can provide lower latency and a better experience for
your customers at a minimal cost.
So where does AWS fit into all of this? Well AWS simply just provides cloud computing services.
Those IT resources mentioned in the cloud computing definition are AWS services in this case. We’ll Video AWS Global Infrastructure:
need to use these AWS services to architect a scalable, highly available, and cost effective AWS has clusters of data centers around the world.
infrastructure to host our corporate directory application. This way we can get our corporate directory
app out into the world quickly, without having to manage any heavy-duty physical hardware. There are An AZ consists of one or more data centers with redundant power, networking, and
the six main advantages to running your workloads on AWS. connectivity. Unfortunately, sometimes natural disasters like hurricanes or other disasters might also
extend to impacting an entire AZ, but AWS has planned for that, too, again, using redundancy. Like
data centers, AWS also clusters AZs together and also connects them with redundant high speed and
low latency links. A cluster of AZs is simply called a region. In AWS, you get to choose the location AWS continues to expand to meet the needs of its customers. Each AWS Region is associated with a
of your resources by not only picking an AZ, but also choosing a region. Regions are generally named geographical name and a Region code.
by location so you can easily tell where they are. For example, I could put our employee photos in a
region in Northern Virginia called the Northern Virginia Region. So, knowing there are many AWS
regions around the world, how do you choose an AWS region? As a basic rule, there are four aspects
you need to consider when deciding which AWS region to use, compliance, latency, price, and
service availability.

1. Before any other factors, you must first look at your compliance requirements. You might find that
your application, company, or country that you live in requires you to handle your data and IT
resources in a certain way.
2. Latency is all about how close your IT resources are to your user base.
3. The pricing can vary from region to region, so it may be that some regions, like the Sao Paulo
Region, are more expensive than others due to different tax structures. So even if I wanted to store my
employee photos in Brazil, it might not make sense from the latency perspective or the pricing
perspective.
4. Often when we create new services or features in AWS, we don't roll those services to every region
we have right away. Meaning, if you want to begin using a new service on day one after it launches,
then you'll want to make sure it operates in the region that you're looking at running your infrastructure
in.

To recap, regions, availability zones, and data centers exist in a redundant, nested sort of way. There
are data centers inside of availability zones and availability zones inside of regions. And how do you
choose a region? By looking at compliance, latency, pricing, and service availability. Those are the
basics, but it isn't the end of the story when it comes to AWS global infrastructure.

Edge locations and regional Edge caches are used to cache content closer to end users, thus reducing
latency

Reading 1.3: Global Infrastructure

Regions are geographic locations worldwide where AWS hosts its data centers. AWS Regions are
named after the location where they reside. For example, in the United States, there is a Region in
Northern Virginia called the Northern Virginia Region and a Region in Oregon called the Oregon
Region. There are Regions in Asia Pacific, Canada, Europe, the Middle East, and South America, and
Here are a few examples of region codes: AVAILABILITY ZONES

 us-east-1: This is the first Region created in the east of the US. The geographical name for this
Region is N. Virginia.
 ap-northeast-1: The first Region created in the northeast of Asia Pacific. The geographical
name for this Region is Tokyo.

AWS Regions are independent from one another. This means that your data is not replicated from
one Region to another, without your explicit consent and authorization.

CHOOSE THE RIGHT AWS REGION

Consider four main aspects when deciding which AWS Region to host your applications and
workloads: latency, price, service availability, and compliance.

Latency. If your application is sensitive to latency, choose a Region that is close to your user base.
This helps prevent long wait times for your customers. Synchronous applications such as gaming,
telephony, WebSockets, and IoT are significantly affected by higher latency, but even asynchronous
workloads, such as ecommerce applications, can suffer from an impact on user connectivity.

Price. Due to the local economy and the physical nature of operating data centers, prices may vary
from one Region to another. The pricing in a Region can be impacted by internet connectivity, prices of
imported pieces of equipment, customs, real estate, and more. Instead of charging a flat rate
worldwide, AWS charges based on the financial factors specific to the location.
Inside every Region is a cluster of Availability Zones (AZ). An AZ consists of one or more data centers
Service availability. Some services may not be available in some Regions. The AWS documentation with redundant power, networking, and connectivity. These data centers operate in discrete facilities
provides a table containing the Regions and the available services within each one. with undisclosed locations. They are connected using redundant high-speed and low-latency links. AZs
also have a code name. Since they’re located inside Regions, they can be addressed by appending a
Data compliance. Enterprise companies often need to comply with regulations that require customer letter to the end of the Region code name. For example:
data to be stored in a specific geographic territory. If applicable, you should choose a Region that
meets your compliance requirements.  us-east-1a: an AZ in us-east-1 (Northern Virginia Region)
 sa-east-1b: an AZ in sa-east-1 (São Paulo Region in South America)
If you see that a resource exists in us-east-1c, you know this resource is located in AZ c of the us-east-1
Region.
SCOPE AWS SERVICES

Depending on the AWS Service you use, your resources are either deployed at the AZ, Region, or
Global level. Each service is different, so you need to understand how the scope of a service may affect
your application architecture. When you operate a Region-scoped service, you only need to select the
Region you want to use. If you are not asked to specify an individual AZ to deploy the service in, this
is an indicator that the service operates on a Region-scope level. For region-scoped services, AWS
automatically performs actions to increase data durability and availability. On the other hand, some
services ask you to specify an AZ. With these services, you are often responsible for increasing the data
durability and high availability of these resources.

MAINTAIN RESILIENCY

To keep your application available, you need to maintain high availability and resiliency. A well-known
best practice for cloud architecture is to use Region-scoped, managed services. These services come
with availability and resiliency built in.When that is not possible, make sure the workload is replicated Video: Interactig with AWS
across multiple AZs. At a minimum, you should use two AZs. If one entire AZ fails, your application
will have infrastructure up and running in at least a second AZ to take over the traffic. how to interact with the infrastructure of a cloud computing service called AWS (Amazon
Web Services). When you own physical infrastructure, like a server in your closet, it's easy to
understand and work with because you can see and touch it. But with AWS, the infrastructure is virtual,
so you have to manage it differently. Instead of physically managing it, you use API calls to AWS to
create, delete, or change resources like virtual servers or storage systems.
three main ways we're going to talk about in AWS are the AWS Management Console, the AWS
Command Line Interface and the AWS Software Development Kits or SDKs.

 the AWS Management Console. This is a web-based method that you log into from your
browser. The great thing about the console is that you can point and click. By simply clicking
and following prompts, you can get started with some of these services without any previous
knowledge of the service. With the console, there's no need to worry about scripting or finding
the proper syntax.
When you log into the console, the landing page will show you services you've
recently worked with but you can also choose to view all of the possible services organized
into relevant categories such as compute, database storage, and more.
 directs the browser to make requests to a whole different AWS Region, represented by a different
subdomain.
Reading 1.4: Interacting with AWS
THE AWS COMMAND LINE INTERFACE (CLI)
Every action you make in AWS is an API call that is authenticated and authorized. In AWS, you can
make API calls to services and resources through the AWS Management Console, the AWS Command Consider the scenario where you run tens of servers on AWS for your application’s frontend. You want
Line Interface (CLI), or the AWS Software Development Kits (SDKs). to run a report to collect data from all of these servers. You need to do this programmatically every day
because the server details may change. Instead of manually logging into the AWS Management
THE AWS MANAGEMENT CONSOLE Console and copying/pasting information, you can schedule an AWS Command Line Interface (CLI)
script with an API call to pull this data for you.The AWS CLI is a unified tool to manage AWS
One way to manage cloud resources is through the web-based console, where you log in and click on services. With just one tool to download and configure, you control multiple AWS services from the
the desired service. This can be the easiest way to create and manage resources when you’re first begin command line and automate them with scripts. The AWS CLI is open-source, and there are installers
working with the cloud. Below is a screenshot that shows the landing page when you first log into the available for Windows, Linux, and Mac OS.Here is an example of running an API call against a service
AWS Management Console. using the AWS CLI:

You get this response:

"Reservations": [

{"Groups": [],

"Instances": [

{"AmiLaunchIndex": 0,

and so on.

AWS SOFTWARE DEVELOPMENT KITS (SDKS)

The services are placed in categories, such as compute, database, storage and security, identity and API calls to AWS can also be performed by executing code with programming languages. You can do
compliance. On the upper right corner is the Region selector. If you click it and change the Region, you this by using AWS Software Development Kits (SDKs). SDKs are open-source and maintained by
will make requests to the services in the chosen Region. The URL changes, too. Changing the Region AWS for the most popular programming languages, such as C++, Go, Java, JavaScript, .NET, Node.js,
PHP, Python, and Ruby.Developers commonly use AWS SDKs to integrate their application source
code with AWS services. Let’s say the frontend of the application runs in Python and every time it We don't view solutions built on AWS as one singular thing to be secured. We see it as a collection of
receives a cat photo, it uploads that photo to a storage service. This action can be achieved from within parts that build on each other. AWS is responsible for the security of some aspects. The others, you are
the source code by using the AWS SDK for Python. responsible for their security. Together with both you and AWS following best practices, you have an
environment that you can trust.
Here is an example of code you can implement to work with AWS resources using the Python
AWS SDK. AWS is responsible for securing these services from the host operating system up through the
virtualization layer. For example, let's say you want to host some virtual machines or VMs on the
import boto3 cloud. We primarily use the service Amazon EC2 for this use case. When you create a VM using EC2,
AWS manages the physical host the VM is placed on as well as everything through the hypervisor
ec2 = boto3.client('ec2') level. If the host operating system or the hypervisor needs to be patched or updated, that is the
responsibility of AWS. This is good news for you as the customer, as it greatly reduces the operational
response = ec2.describe_instances() overhead in running a scalable and elastic solution leveraging virtualization.

print(response) Reading 1.5: Security and the AWS Shared Responsibility Model

When you begin working with the AWS Cloud, managing security and compliance is a shared
responsibility between AWS and you. To depict this shared responsibility, AWS created the shared
responsibility model. This distinction of responsibility is commonly referred to as security of the cloud,
Security in the AWS Cloud versus security in the cloud.

Video: VideoSecurity and the AWS Shared Responsibility Model

You already know that by using AWS, you won't be managing every single aspect of hosting your
solutions. You'll rely on AWS to manage portions of your workloads for you taking care of that
undifferentiated heavy lifting, like running the day-to-day operations of the data centers and managing
the various virtualization techniques employed to keep your AWS account isolated from, say my AWS
account.

So, the question is who is ultimately responsible for security and AWS? Is it A, you the customer,
or B, AWS?

 The answer? Well, the correct answer is yes. Both you and AWS are responsible for securing
your AWS environment.
WHAT IS AWS RESPONSIBLE FOR?

AWS is responsible for security of the cloud. This means AWS is required to protect and secure the
infrastructure that runs all the services offered in the AWS Cloud. AWS is responsible for:

 Protecting and securing AWS Regions, Availability Zones, and data centers, down to the
physical security of the buildings.
 Managing the hardware, software, and networking components that run AWS services, such
as the physical server, host operating systems, virtualization layers, and AWS networking
components

The level of responsibility AWS has depends on the service. AWS classifies services into three
different categories. The following table provides information about each, as well as the AWS
responsibility.

Note
Container services refer to AWS abstracting application containers behind the scenes, not Docker
container services. This enables AWS to move the responsibility of managing that platform away from
customers.

WHAT IS THE CUSTOMER RESPONSIBLE FOR?

You’re responsible for security in the cloud. When using any AWS service, you’re responsible for
properly configuring the service and your applications, as well as ensuring your data is secure.The
level of responsibility you have depends on the AWS service. Some services require you to perform all laws and regulations. It’s important to note that you maintain complete control of your data and are
the necessary security configuration and management tasks, while other more abstracted services responsible for managing the security related to your content. Here are some examples of your
require you to only manage the data and control access to your resources. Using the three categories of responsibilities in context.
AWS services, you can determine your level of responsibility for each AWS service you use.
 Choosing a Region for AWS resources in accordance with data sovereignty regulations
 Implementing data protection mechanisms, such as encryption and managing backups
 Using access control to limit who has access to your data and AWS resources.

Video: VideoProtect the AWS Root User

I want you to understand that when you log into your AWS account using an email address and
password, it means you are logging in as the root user. This root user can do whatever they want in the
account. It has all of the powers that can be had. And with great power comes great responsibility, or
something like that

Create a hard-to-crack password, and that will give you some level of security. This, however, is an
example of single factor authentication where all someone needs to do is match the password with the
email address, and boom, they're in.

We recommend as a best practice that right after you create your AWS account, you enable multi-factor
authentication, or MFA, on the root user. MFA introduces an additional unique piece of information
that you need to enter to gain access to the account.

For example, I personally use a virtual MFA device that is an app on my phone. This app produces a
string of numbers for one time use that I type into the console after I log in using my email address and
password. Even if someone guessed the password, they cannot gain access to the account without the
numbers generated by the virtual MFA device. No matter what type of MFA device that you choose to
use, and I will include a link to the supported devices and the readings for you to look into, the most
important thing is that you are using MFA on the root user. That way, even if someone, the nefarious
actor, cracks your password, they still cannot gain access to your account. All thanks to MFA. On top
of enabling MFA for the root user, we strongly recommend that you do not use the root user for your
Due to the varying level of effort, it’s important to consider which AWS service you use and review the everyday tasks, even the administrative ones.
level of responsibility required to secure the service. It’s also important to review how the shared
security model aligns with the security standards in your IT environment, as well as any applicable
Reading 1.6: Protect the AWS Root User AWS Command Line Interface (AWS CLI) or AWS API. Access keys consist of two parts:

What’s the Big Deal About Auth?  An access key ID, for example, A2lAl5EXAMPLE
 When you’re configuring access to any account, two terms come up frequently: authentication  A secret access key, for example, wJalrFE/KbEKxE
and authorization. Though these terms may seem basic, you need to understand them to
properly configure access management on AWS. It’s important to keep this mind as you Similar to a username and password combination, you need both the access key ID and secret access
progress in this course. Let’s define both terms. key to authenticate your requests via the AWS CLI or AWS API. Access keys should be managed with
the same security as an email address and password.
Understand Authentication
 When you create your AWS account, you use a combination of an email address and a Follow Best Practices When Working with the AWS Root User
password to verify your identity. If the user types in the correct email and password, the
system assumes the user is allowed to enter and grants them access. This is the process of Keep in mind that the root user has complete access to all AWS services and resources in your account,
authentication. Authentication ensures that the user is who they say they are. Usernames and as well as your billing and personal information. Due to this, securely lock away the credentials
passwords are the most common types of authentication, but you may also work with other associated with the root user and do not use the root user for everyday tasks. To ensure the safety of
forms, such as token-based authentication or biometric data like a fingerprint. Authentication the root user:
simply answers the question, “Are you who you say you are?”
 Choose a strong password for the root user.
Understand Authorization  Never share your root user password or access keys with anyone.
 Once you’re inside your AWS account, you might be curious about what actions you can take.  Disable or delete the access keys associated with the root user.
This is where authorization comes in. Authorization is the process of giving users permission  Do not use the root user for administrative tasks or everyday tasks.
to access AWS resources and services. Authorization determines whether the user can perform
an action—whether it be to read, edit, delete, or create resources. Authorization answers the When is it OK to use the AWS root user? There are some tasks where it makes sense to use the AWS
question, “What actions can you perform?” root user. Check out the links at the end of this section to read about them.

What Is the AWS Root User? Delete Your Keys to Stay Safe
 When you first create an AWS account, you begin with a single sign-in identity that has
complete access to all AWS services and resources in the account. This identity is called the If you don't already have an access key for your AWS account root user, don't create one unless you
AWS root user and is accessed by signing in with the email address and password that you absolutely need to. If you do have an access key for your AWS account root user and want to delete the
used to create the account. keys:

Understand the AWS Root User Credentials 1. Go to the My Security Credentials page in the AWS Management Console and sign in with
 The AWS root user has two sets of credentials associated with it. One set of credentials is the the root user’s email address and password.
email address and password used to create the account. This allows you to access the AWS 2. Open the Access keys section.
Management Console. The second set of credentials is called access keys, which allow you to 3. Under Actions, click Delete.
make programmatic requests from the 4. Click Yes.
The Case for Multi-Factor Authentication information is a temporary numeric code provided by an MFA device.Enabling MFA adds an additional
layer of security because it requires users to use a supported MFA mechanism in addition to their
When you create an AWS account and first log in to that account, you use single-factor authentication. regular sign-in credentials. It’s best practice to enable MFA on the root user.
Single-factor authentication is the simplest and most common form of authentication. It only requires
one authentication method. In this case, you use a username and password to authenticate as the AWS Review Supported MFA Devices
root user. Other forms of single-factor authentication include a security pin or a security AWS supports a variety of MFA mechanisms, such as virtual MFA devices, hardware devices, and
token.However, sometimes a user’s password is easy to guess. Universal 2nd Factor (U2F) security keys. For instructions on how to set up each method, check out
the Resources section.
For example, your coworker Bob’s password, IloveCats222, might be easy for someone who knows
Bob personally to guess, because it’s a combination of information that is easy to remember and
describes certain things about Bob (1. Bob loves cats, and 2. Bob’s birthday is February 22).

If a bad actor guessed or cracked Bob’s password through social engineering, bots, or scripts, Bob
might lose control of his account. Unfortunately, this is a common scenario that users of any website
often face.

This is why using MFA has become so important in preventing unwanted account access. MFA
requires two or more authentication methods to verify an identity, pulling from three different
categories of information.

 Something you know, such as a username and password, or pin number

 Something you have, such as a one-time passcode from a hardware device or mobile app
 Something you are, such as fingerprint or face scanning technology

Using a combination of this information enables systems to provide a layered approach to account
access. Even though the first method of authentication, Bob’s password, was cracked by a malicious
user, it’s very unlikely that a second method of authentication, such as a fingerprint, would also be
cracked. This extra layer of security is needed when protecting your most sacred accounts, which is
why it’s important to enable MFA on your AWS root user.

Use MFA on AWS

If you enable MFA on your root user, you are required to present a piece of identifying information
from both the something you know category and the something you have category. The first piece of
identifying information the user enters is an email and password combination. The second piece of
 Another best practice to follow is that we recommend when you create your AWS account, you set up
MFA for the root user. Then create an IAM user with admin permissions. Log out of the root user and
AWS Identity and Access Management then log in with the IAM user that you just created. From there, you can use this user to create the rest
of the IAM groups users and policies. The reason we suggest you do this is because you cannot apply a
Video: VideoIntroduction to AWS Identity and Access Management
policy to the root user but you can to an IAM user.

Just because both Amazon EC2 and Amazon S3 have existing resources in this account, it doesn't mean
The EC2 instance needs credentials to be able to make the signed API call two S3 for reading and
that the API calls made from the code running on the EC2 instance to S3 are automatically allowed to
writing employee images. Am I suggesting that you make an IAM user with a username and password
be made. In fact, all API calls in AWS must be both signed and authenticated in order to be allowed, no
for the application running on EC2 to use? No. No, I am not. This is where role-based access comes
matter if the resources live in the same account or not. The application code running on the Amazon
into the picture. Coming up we will learn about the temporary access that IAM roles provide and how
EC2 instance needs access to credentials to make this signed API call to Amazon S3. So that's another
it can apply to this use case here.
place with a need for a credential and access management. Now let's take this a step further
Reading 1.7: Introduction to AWS Identity and Access Management
How are you going to build out this architecture? Well, you'll need access to an AWS account through
the use of a login. Your identity within this AWS account will need permissions to be able to do things
WHAT IS IAM?
like create this network, launch the EC2 instances and create the resources that will host and run the
 IAM is a web service that enables you to manage access to your AWS account and resources.
solution in AWS. Yet another place you need credentials.
It also provides a centralized view of who and what are allowed inside your AWS account
(authentication), and who and what have permissions to use and work with your AWS
AWS identity and access management or IAM can help take care of these two spots on the diagram.
resources (authorization).With IAM, you can share access to an AWS account and resources
AWS IAM manages the login credentials and permissions to the AWS account and it also can manage
without having to share your set of access keys or password. You can also provide granular
the credentials used to sign API calls made to AWS services. IAM would not, however, be responsible
access to those working in your account, so that people and services only have permissions to
for application level access management. The code running on this instance would use separate
the resources they need. For example, to provide a user of your AWS account with read-only
appropriate mechanisms for authenticating users into the application itself, not IAM.
access to a particular AWS service, you can granularly select which actions and which
resources in that service they can access.
IAM allows you to create users and each individual person who needs access to your AWS account
would have their own unique IAM user. Creating users for everyone who needs access to the account,
takes care of authentication. Authentication being verifying if someone is who they say they are
GET TO KNOW THE IAM FEATURES
because they had the proper credentials to log in. Now it's time to introduce authorization.
To help control access and manage identities within your AWS account, IAM offers many features to
Authorization is this.
ensure security.
IAM users take care of authentication and you can take care of authorization by attaching IAM policies
to users in order to grant or deny permission to specific actions within an AWS account. Keep in mind
 IAM is global and not specific to any one Region. This means you can see and use your IAM
when I say action here, I'm referring to an AWS API call. Everything in AWS is an API call. IAM
configurations from any Region in the AWS Management Console.
policies are JSON-based documents.
 IAM is integrated with many AWS services by default.
 You can establish password policies in IAM to specify complexity requirements and
mandatory rotation periods for users.
  IAM supports MFA. An IAM group is a collection of users. All users in the group inherit the permissions assigned to the
 IAM supports identity federation, which allows users who already have passwords elsewhere group. This makes it easy to give permissions to multiple users at once. It’s a more convenient and
—for example, in your corporate network or with an internet identity provider—to get scalable way of managing permissions for users in your AWS account. This is why using IAM groups
temporary access to your AWS account. is a best practice.If you have a an application that you’re trying to build and have multiple users in one
 Any AWS customer can use IAM; the service is offered at no additional charge account working on the application, you might decide to organize these users by job function. You
might want IAM groups organized by developers, security, and admins. You would then place all of
WHAT IS AN IAM USER? your IAM users in the respective group for their job function.This provides a better view to see who
 An IAM user represents a person or service that interacts with AWS. You define the user has what permissions within your organization and an easier way to scale as new people join, leave,
within your AWS account. And any activity done by that user is billed to your account. Once and change roles in your organization.Consider the following examples.
you create a user, that user can sign in to gain access to the AWS resources inside your
account.You can also add more users to your account as needed. For example, for your cat  A new developer joins your AWS account to help with your application. You simply create a
photo application, you could create individual users in your AWS account that correspond to new user and add them to the developer group, without having to think about which
the people who are working on your application. Each person should have their own login permissions they need.
credentials. Providing users with their own login credentials prevents sharing of credentials.  A developer changes jobs and becomes a security engineer. Instead of editing the user’s
permissions directly, you can instead remove them from the old group and add them to the
IAM USER CREDENTIALS new group that already has the correct level of access.
An IAM user consists of a name and a set of credentials. When creating a user, you can choose to
provide the user: Keep in mind the following features of groups.

 Access to the AWS Management Console  Groups can have many users.
 Programmatic access to the AWS Command Line Interface (AWS CLI) and AWS Application  Users can belong to many groups.
Programming Interface (AWS API)  Groups cannot belong to groups.

To access the AWS Management Console, provide the users with a user name and password. For The root user can perform all actions on all resources inside an AWS account by default. This is in
programmatic access, AWS generates a set of access keys that can be used with the AWS CLI and contrast to creating new IAM users, new groups, or new roles. New IAM identities can perform no
AWS API. IAM user credentials are considered permanent, in that they stay with the user until there’s a actions inside your AWS account by default until you explicitly grant them permission.The way you
forced rotation by admins. When you create an IAM user, you have the option to grant permissions grant permissions in IAM is by using IAM policies.
directly at the user level. This can seem like a good idea if you have only one or a few users. However,
as the number of users helping you build your solutions on AWS increases, it becomes more WHAT IS AN IAM POLICY?
complicated to keep up with permissions. For example, if you have 3,000 users in your AWS account,  To manage access and provide permissions to AWS services and resources, you create IAM
administering access becomes challenging, and it’s impossible to get a top-level view of who can policies and attach them to IAM users, groups, and roles. Whenever a user or role makes a
perform what actions on which resources. If only there were a way to group IAM users and attach request, AWS evaluates the policies associated with them. For example, if you have a
permissions at the group level instead. Guess what: There is! developer inside the developers group who makes a request to an AWS service, AWS
evaluates any policies attached to the developers group and any policies attached to the
WHAT IS AN IAM GROUP? developer user to determine if the request should be allowed or denied.
IAM POLICY EXAMPLES Putting all this information together, you have a policy that allows you to perform all actions on all
Most policies are stored in AWS as JSON documents with several policy elements. Take a look at the resources inside your AWS account. This is what we refer to as an administrator policy.
following example of what providing admin access through an IAM identity-based policy looks like.
Let’s look at another example of a more granular IAM policy.

In this policy, there are four major JSON elements: Version, Effect, Action, and Resource.

 The Version element defines the version of the policy language. It specifies the language
syntax rules that are needed by AWS to process a policy. To use all the available policy
features, include "Version": "2012-10-17" before the "Statement" element in all your policies.
 The Effect element specifies whether the statement will allow or deny access. In this policy,
the Effect is "Allow", which means you’re providing access to a particular resource.
 The Action element describes the type of action that should be allowed or denied. In the
above policy, the action is "*". This is called a wildcard, and it is used to symbolize every
action inside your AWS account.
 The Resource element specifies the object or objects that the policy statement covers. In the After looking at the JSON, you can see that this policy allows the IAM user to change their own IAM
policy example above, the resource is also the wildcard "*". This represents all resources password (iam:ChangePassword) and get information about their own user (iam:GetUser). It only
inside your AWS console. permits them to access their own credentials because the resource restricts access with the variable
substitution ${aws:username}.
 so how will the application gain access to the needed AWS access key ID and AWS secret access key
UNDERSTAND POLICY STRUCTURE to sign the API call? The answer is IAM roles. IAM roles are identities in AWS that like an IAM user
When creating a policy, it is required to have each of the following elements inside a policy statement. also have associated AWS credentials used to sign requests. However, IAM users have usernames and
passwords as well as static credentials whereas IAM roles do not have any login credentials like a
username and password and the credentials used to sign requests are programmatically acquired,
temporary in nature, and automatically rotated.

AWS assigns a role to a federated user when access is requested through an identity provider.

Reading 1.8: Role Based Access in AWS

Throughout these last few lessons, there have been sprinklings of IAM best practices. It’s helpful to
have a brief summary of some of the most important IAM best practices you need to be familiar with
before building out solutions on AWS.

LOCK DOWN THE AWS ROOT USER

Video: VideoRole Based Access in AWS

Policies can be applied to AWS identities like users and groups to assign permissions. They also,
however, can be applied to another AWS identity, IAM roles. An IAM role is an identity that can be Throughout these last few lessons, there have been sprinklings of IAM best practices. It’s helpful to
assumed by someone or something who needs temporary access to AWS credentials have a brief summary of some of the most important IAM best practices you need to be familiar with
before building out solutions on AWS.
IAM users have associated credentials like an access key ID and secret access key that are used to sign
requests. However, with regards to our architecture, the code running on the EC2 instance needs to sign The root user is an all-powerful and all-knowing identity within your AWS account. If a malicious user
the request sent to S3. were to gain control of root-user credentials, they would be able to access every resource within your
account, including personal and billing information. To lock down the root user:
  Don’t share the credentials associated with the root user.
 Consider deleting the root user access keys.
 Enable MFA on the root account.

FOLLOW THE PRINCIPLE OF LEAST PRIVILEGE

Maintaining roles is easier than maintaining users. When you assume a role, IAM dynamically
provides temporary credentials that expire after a defined period of time, between 15 minutes and 36
hours. Users, on the other hand, have long-term credentials in the form of user name and password
Least privilege is a standard security principle that advises you to grant only the necessary permissions
combinations or a set of access keys.User access keys only expire when you or the admin of your
to do a particular job and nothing more. To implement least privilege for access control, start with the
account rotates these keys. User login credentials expire if you have applied a password policy to your
minimum set of permissions in an IAM policy and then grant additional permissions as necessary for a
account that forces users to rotate their passwords.
user, group, or role.

CONSIDER USING AN IDENTITY PROVIDER

USE IAM APPROPRIATELY

If you decide to make your cat photo application into a business and begin to have more than a handful
IAM is used to secure access to your AWS account and resources. It simply provides a way to create
of people working on it, consider managing employee identity information through an identity provider
and manage users, groups, and roles to access resources within a single AWS account. IAM is not used
(IdP). Using an IdP, whether it be an AWS service such as AWS IAM Identity Center (Successor to
for website authentication and authorization, such as providing users of a website with sign-in and
AWS Single Sign-On) or a third-party identity provider, provides you a single source of truth for all
sign-up functionality. IAM also does not support security controls for protecting operating systems and
identities in your organization.You no longer have to create separate IAM users in AWS. You can
networks.
instead use IAM roles to provide permissions to identities that are federated from your IdP.For
example, you have an employee, Martha, that has access to multiple AWS accounts. Instead of creating
USE IAM ROLES WHEN POSSIBLE
and managing multiple IAM users named Martha in each of those AWS accounts, you can manage
Martha in your company’s IdP. If Martha moves within the company or leaves the company, Martha
can be updated in the IdP, rather than in every AWS account you have.

CONSIDER AWS IAM IDENTITY CENTER

 Week 1 Exercise & Assessment

Demo AWS IAM

You could have an AWS account, this would allow you to allow cross-account access to permissions
for resources in your account. You also could select a web identity, which would allow for federated
users to assume a role. You have a SAML 2.0 federation, so if you have a corporate directory that is on
premises that would be using SAML, you could use this as your trusted entity type or you could create
a Custom Trusts policy.
If you have an organization that spans many employees and multiple AWS accounts, you may want
your employees to sign in with a single credential. AWS IAM Identity Center is an IdP that lets your Access keys are going to allow your users to make programmatic calls to AWS using things like the
users sign in to a user portal with a single set of credentials. It then provides them access to all their AWS command line, the AWS software development kits, where maybe they're developing locally on
assigned accounts and applications in one central location. AWS IAM Identity Center is similar to their laptop, and they need their code to be able to reach out to AWS.
IAM, in that it offers a directory where you can create users, organize them in groups, and set
permissions across those groups, and grant access to AWS resources. However, AWS IAM Identity IAM Roles and the Employee Directory Application
Center has some advantages over IAM. For example, if you’re using a third-party IdP, you can sync
your users and groups to AWS IAM Identity Center. This removes the burden of having to re-create Throughout the demos, you’ll notice that the Amazon EC2 instance is using an IAM role to manage
users that already exist elsewhere, and it enables you to manage those users from your IdP. More access to AWS APIs for the employee directory application application. This is a common way to
importantly, AWS IAM Identity Center separates the duties between your IdP and AWS, ensuring that provide AWS credentials to applications that need to access AWS APIs. This IAM role is providing
your cloud access management is not inside or dependent on your IdP. temporary credentials that are rotated automatically via the instance profile for the EC2 instance. The
AWS SDK is automatically pulling these credentials and refreshing them as needed and using the
credentials to authenticate the AWS API calls being made to Amazon S3 and Amazon DynamoDB by
the code for the employee directory app. Without this role, the application would not be authenticated
and the API calls to those services would fail.

In the video Demo AWS IAM you saw Morgan create an IAM role called EmployeeWebApp which
used the managed policies AmazonS3FullAccess and AmazonDynamoDBFullAccess. Later in the
Hosting the Employee Directory Application on AWS video and subsequent demos you may see an
IAM role named S3DynamoDBFullAccessRole being used when configuring the EC2 instance. Both
of these roles use the same policies Morgan selected in the Demo AWS IAM role.

The next video in this course demonstrates how to host the employee directory application on AWS
using services like Amazon EC2 and Amazon VPC. While watching this video, you may be looking for
a copy of the scripts used so you can follow along. The exercises in next weeks content includes step To recap, we have a new default AMI for EC2 instances called the Amazon Linux 2023 AMI. The
by step instructions on how to launch the employee direction application, including the user data script. videos show us using Amazon Linux 2. Because of changes between these two AMIs the user data
For this next video, we recommend that you watch without following along yet so you can understand script shown in the videos will not run properly on Amazon Linux 2023 based instances. You can either
the AWS services at a high level, then next week you will have the opportunity to walk through this choose Amazon Linux 2 as the AMI when launching the instance, and use the original user data script
demonstration in your AWS account using the instructions included in the exercises. or you can use the Amazon Linux 2023 AMI and use the updated user data script.

If at any point, you are unsure of how roles work or how to properly set up an IAM role, please refer Amazon Linux 2 user data script:
back to the Demo AWS IAM video.

Hosting the Employee Directory Application on AWS Beta

AWS provides you with something called a default VPC which is a private network for your AWS
resources. Every EC2 instance you launch using AWS must live inside of a network. So in order to
complete this demo with the limited amount of information that we have shown you about AWS
services, we will be using the default network provided by AWS and we will accept most of the default
values for launching an EC2 instance

Amazon EC2 is a compute service that allows you to host virtual machines.

From here, we'll launch a new EC2 instance. An instance is just what we call one single virtual
machine. Now we have to select the configurations for this EC2 instance.

Default Amazon Machine Image (AMI) for Amazon EC2

Hello learners!

As of March 15, 2023 the default Amazon Machine Image (AMI) for Amazon EC2 has been updated
to use the Amazon Linux 2023 AMI. In the demonstrations for this course, we use the Amazon Linux 2
AMI. If you are following along with the videos please be aware that if you use the new Amazon
Linux 2023 AMI with the user data the way it appears in the videos the script will not run properly and
the application will not launch. We are in the process of updating the course to reflect this change.

In the meantime, there are a few ways to work around this issue. You can either use the Amazon Linux
2 AMI with the user data as shown in the demonstrations and this will resolve the issue, or you can use
an updated version of the user data script which I will include in this message.
Amazon Linux 2023 user data script:
AWS COMPUTE
Introduction to Week

An EC2 instance is simply just a form of compute, meaning it gives our application power in the form
of CPU, memory and networking capacity so that we can process our user's requests. This concept of
compute leads us into the next lesson where we discuss your compute options on AWS.

Video: Compute as a Service on AWS

Every business needs raw compute capacity to run applications. These applications could be web
servers, batch jobs, databases, HR software, machine learning, or just about whatever you can think of.
The time, money, and effort it takes to get up and running with on-premises computing resources are
fairly high.

When you own your own fleet of physical servers, you first have to do a bunch of research to see what
type of servers you want to buy and how many you need to buy. Then you need to purchase that
hardware upfront. You'll wait multiple weeks or months for servers to be delivered. You then take them
to a data center space that you own or rent to install them, rack and stack them and wire them all up.
Then you make sure the servers are secured, powered up and ready to be used. Only then can you
begin to host your applications on top of these servers.

When using the user data scripts, remember to replace the <INSERT REGION HERE> with whatever
Following a Compute as a Service model is much easier to get started and support operations over
AWS region you are operating in, and ensure you remove both brackets as well.
time. AWS took care of the hard part for you already as far as getting started goes. AWS already built
and secured the data centers. AWS has already bought the servers, racked them and stacked them, and
they are already online ready to be used.

Reading 2.1: Compute as a Service on AWS

Understanding Servers

The first building block you need to host an application is a server. Servers often times can handle
Hypertext Transfer Protocol (HTTP) requests and send responses to clients following the client-server
model, though any API based communication also falls under this model. A client being a person or
computer that sends a request, and a server handling the requests is a computer, or collection of
computers, connected to the internet serving websites to internet users. These servers power your
application by providing CPU, memory, and networking capacity to process users’ requests and
transform them into responses. For context, common HTTP servers include:

 Windows options, such as Internet Information Services (IIS).

 Linux options, such as Apache HTTP Web Server, Nginx, and Apache Tomcat.

To run an HTTP server on AWS, you need to find a service that provides compute power in the AWS
Management Console. You can log into the console and view the complete list of AWS compute
services.

Choose the Right Compute Option

If you’re responsible for setting up servers on AWS to run your infrastructure, you have many compute
options. You need to know which service to use for which use case. At a fundamental level, there are
three types of compute options: virtual machines, container services, and serverless. If you’re coming
to AWS with prior infrastructure knowledge, a virtual machine can often be the easiest compute option
in AWS to understand. This is because a virtual machine emulates a physical server and allows you to
install an HTTP server to run your applications. To run these virtual machines, you install a hypervisor
on a host machine. This hypervisor provisions the resources to create and run your virtual machines.In
AWS, these virtual machines are called Amazon Elastic Compute Cloud or Amazon EC2. Behind the
scenes, AWS operates and manages the host machines and the hypervisor layer. AWS also installs the
virtual machine operating system, called the guest operating system.Some AWS compute services use
Amazon EC2 or use virtualization concepts under the hood, therefore it is best to understand this
service
first before moving on to container services and serverless compute.
Video: Introduction to Amazon Elastic Compute Cloud

EC2 instances give you a lot of flexibility and control in the cloud, and you configure them to meet
your needs. You can provision one or many instances easily and at the end of the billing cycle, you
only pay for what you use, either per second or per hour depending on the type of the instance. When
you no longer need an instance, you can terminate or stop the instance and you will stop incurring
charges. Not all servers are the same, and you are probably looking to run a specific type of operating
system on your EC2 instance.

AWS supports a range of operating systems including Linux, MacOS, Ubuntu, Windows, and more. To
select the operating system for your server, you must choose an Amazon Machine Image or an AMI.

The AMI contains information about how you want your instance to be configured including the
operating system, possible applications to be pre-installed on that instance upon launch, and other
configurations. For example, Amazon Linux 2 is the AMI we selected when we launched our Employee Directory
Application. This AMI is provided by AWS and it's essentially a pre-built EC2-optimized Linux Image
that has long-term support provided by AWS. Beyond the properties determined by the AMI, you can
also configure the instance type in size which correspond to the amount of compute, memory, and
network capabilities available per instance.

The instance types you can choose from are grouped for use cases like compute-optimized, memory-
optimized, storage-optimized instances, and more. There's a list of instance types you can find in the
AWS documentation, and you can expect this page to be updated regularly as new instance types are
released.

For example, the G instance family are optimized for graphics intensive applications, which would
work best for use cases such as 3D visualizations or video encoding. Whereas the M5 General Purpose
EC2 instance family provides a balance of resources and are great for applications that use these
resources in equal proportions like web servers, our Employee Directory Application example, or co-
repositories. When you are launching an EC2 instance, you will see something like this when selecting
You can launch one or many instances from a single AMI, which would create multiple instances that an instance type and size. The T3 or A1 is the instance type that determines the blend of hardware
all have the same configurations. Some AIs are provided by AWS whereas others are provided by the capabilities, then the dot, then the size like small, medium, large. It goes down to nano and up to many,
community and could be found using the AWS Marketplace or you can build your own custom AMIs many extra large sizes. The great thing about this type of selection existing right at your fingertips is
as needed. that you are no longer locked into hardware decisions upfront. You can choose an initial EC2 instance
type, evaluate its performance for your specific use case, and then later change it to a different type that  Hardware specifications, like CPU, memory, network, and storage.
is better suited for the application. EC2 is also re-sizable with a few clicks in the console or can be  Logical configurations, like networking location, firewall rules, authentication, and the
done programmatically through an API call. operating system of your choice.

When launching an EC2 instance, the first setting you configure is which operating system you want
The flexible and low-cost nature of EC2 instances as well as the ease of provisioning servers allows for by selecting an Amazon Machine Image (AMI).
programmers and businesses to innovate more quickly by spinning up servers for a short amount of
time to run experiments and find optimal configurations for your applications. What Is an AMI?

In the traditional infrastructure world, the process of spinning up a server consists of installing an
EC2 offers a wide variety of hardware options to choose from so you can optimize your solutions by operating system from installation disks, installation drives, or installation wizards over the network. In
selecting the right EC2 instance type for your application, and then you can optimize even further by the AWS Cloud, this operating system installation is no longer your responsibility, and is instead built
right-sizing the resource or selecting an instance size that is appropriate for your application and not into the AMI that you choose.Not only does an AMI let you configure which operating system you
over provisioning like is often done on premises. This type of optimization is hard to achieve on your want, you can also select storage mappings, the architecture type (such as 32-bit, 64-bit, or 64-bit
own because with traditional on-premises deployments, you are working with hardware constraints that ARM), and additional software installed.
simply don't exist in the same way with the cloud. The ability to adapt to changes and choose really
specific configurations for your virtual machines all through a couple of API calls is very powerful and What Is the Relationship Between AMIs and EC2 Instances?
EC2 is really just the beginning of the story.
EC2 instances are live instantiations of what is defined in an AMI, much like a cake is a live
Reading 2.2: Introduction to Amazon Elastic Compute Cloud instantiation of a cake recipe. If you are familiar with software development, you can also see this kind
of relationship between a Class and an Object.
What Is Amazon EC2?
A Class is something you model and define, while an object is something you interact with. In this
case, the AMI is how you model and define your instance, while the EC2 instance is the entity you
interact with, where you can install your web server, and serve your content to users.When you launch
a new instance, AWS allocates a virtual machine that runs on a hypervisor. Then the AMI you selected
is copied to the root device volume, which contains the image used to boot the volume. In the end, you
get a server you can connect to and install packages and any additional software. In this case, you
install a web server along with the properly configured source code of your employee directory app.
Amazon EC2 is a web service that provides secure, resizable compute capacity in the cloud. It allows
you to provision virtual servers called EC2 instances. Although AWS uses the phrase “web service” to
describe it, it doesn’t mean that you are limited to running just web servers on your EC2 instances. You
can create and manage these instances through the AWS Management Console, the AWS Command
Line Interface (CLI), AWS Software Development Kits (SDKs), or through automation tools and
infrastructure orchestration services.In order to create an EC2 instance, you need to define:
 Where Can You Find AMIs?

You can select an AMI from the following categories.

 Quick Start AMIs that are premade by AWS and allow you to get started quickly.
 AWS Marketplace AMIs that provide popular open source and commercial software from
third-party vendors.
 My AMIs that are created from your EC2 instances.
 Community AMIs that are provided by the AWS user community.
 Build your own custom image with EC2 Image Builder.

Each AMI in the AWS Management Console has an AMI ID, which is prefixed by “ami-”, followed by
a random hash of numbers and letters. These IDs are unique to each AWS region.

One advantage of using AMIs is that they are reusable.

You might choose a Linux-based AMI and configure the HTTP server, application packages, and any
additional software you may need to run your application.

If you wanted to create a second EC2 instance with the same configurations, how can you easily do
that? One option is to go through the entire instance creation and configuration process and try to
match your settings to the first instance. However, this is time consuming and leaves room for human
error.

The second, better option, is to create an AMI from your running instance and use this AMI to start a
new instance. This way, your new instance will have all the same configurations as your current
instance, because the configurations set in the AMIs are the same.
 and as you learned in a previous lesson, once the EC2 instance is launched, it enters a pending state.
This state is essentially your VM booting up.

Once the instance is ready for use, it enters the running state. In the running state, you will be charged
for the EC2 instance. From running, you have a couple of different options. You can reboot the
instance, which is similar to rebooting, say, your laptop. It turns off, then it turns back on again. Pretty
straightforward. You can also choose to stop your instance. It will enter a stopping phase, then enter the
stopped phase. Stopping an instance is like powering down your laptop. You can always turn it back on
and it will go through its usual boot sequence, moving through the pending state and back to the
running state. The other option, which is similar to stop, is to stop-hibernate your instance.

This also enters the stopping phase and then the stopped phase. You can compare this to how you lock
your laptop and shut the lid, but when you open it back up, everything is still in place where you left it.
No boot sequences required. You are back up and running after a couple of seconds of the computer
Video: Amazon EC2 Instance Lifecycle waking up. Since the state of the machine was written to memory when you stopped it, the state of the
machine can be drawn from memory and put back into place. Then, you're back up and running.
EC2 allows you to stop and start instances at will, which enables you to treat your fleet of EC two
instances as elastic, scaling them in or out. Then at the end of the billing cycle, you only pay for what
you use. We are building up to the idea of scaling your fleet of EC2 instances in and out to serve
demand. But before we get there, it's important to understand some of the more basic things about
EC2.

let’s talk about the EC2 instance lifecycle. An Amazon EC2 instance transitions through different states
from the moment you launch it through to its termination. You launch an EC2 instance from an AMI,
Now, the last option depicted here is the terminate option. When you terminate an instance, it enters the You of course can do in place updates as well, but just know that you have the option to decide how to
shutting down phase then the terminated phase. Terminating an EC2 instance is like taking your laptop handle these sorts of tasks.
out for a long boat right off the coast and throwing it into the ocean, getting rid of it forever. It's now
lost in the great blue sea. There is no hope of finding your laptop on an island, shipwrecked one day, You can launch or terminate EC2 instances to meet demand. So as demand for your application
having been saved after spelling SOS in the sand. increases, you can launch more instances and as it decreases, you can terminate instances. This keeps
your EC2 instance fleet in line with demand. So again, terminating an instance isn't a bad thing and
here is a feature called termination protection that you can enable if you're worried about instances shouldn't be feared. Now, let's discuss the cost aspect of this. You only get charged for an EC2 instance
being terminated accidentally. You also will learn about persistent storage and EC2 in future lessons, so if you're in the running state or if you are in the stopping state when preparing to hibernate. This means
don't fret. that you can stop instances when they aren't in use, say if you have applications that are only used
during the work week. Run them when your employees are clocked in and then stop them when they
So that is the EC2 instance lifecycle. It's definitely a good idea to remember how all of this works if aren't. Remember, you can always start from a stop state, allowing them to resume working when
you plan on using EC2 to host your applications. And don't think of terminating instances as a bad necessary.
thing.
Reading 2.25: Amazon EC2 Instance Lifecycle
If your EC2 instances is having technical trouble for one reason or another, maybe it needs an update
or a patch, instead of logging into the instance to fix it or install software, you can launch a new one Now that you know how to select an operating system for your EC2 instance, it’s time to choose other
with the new changes in its place and then you could terminate the old instance, having a new instance configurations to create your EC2 instance, such as the instance type, network, and storage. For an
take that place. application like the employee directory application, you need instances with enough capacity to run
web servers and process incoming customer requests. Your instance sizing will depend on both the
demands of your application and the anticipated size of your user base. Forecasting server capacity for
an on-premises application requires difficult decisions involving significant up-front capital spending,
while changes to the allocation of your cloud-based services can be made with a simple API call.
Because of AWS’s pay-as-you-go model, you can match your infrastructure capacity to your
application’s demand, instead of the other way around.

What Makes Up an EC2 Instance?

EC2 instances are a combination of virtual processors (vCPUs), memory, network, and in some cases,
instance storage and graphics processing units (GPUs). When you create an EC2 instance, you need to
choose how much you need of each of these components.

AWS offers a variety of instances that differ based on performance. Some instances provide you with
more capacity and others provide less. To get an overview of the capacity details for a particular
instance, you should look at the instance type. Instance types consist of a prefix identifying the type of
workloads they’re optimized for, followed by a size. For example, the instance type c5.large can be
broken down into the following elements.

 c5 determines the instance family and generation number. Here, the instance belongs to the
fifth generation of instances in an instance family that’s optimized for generic computation.
 large, which determines the amount of instance capacity.

What Are Instance Families?

Where Does Your EC2 Instance Live?

By default, your EC2 instances are placed in a network called the default Amazon Virtual Private
Cloud (VPC). This network was created so that you can easily get started with Amazon EC2 without
having to learn how to create and configure a VPC. Any resource you put inside the default VPC will
be public and accessible by the internet, so you shouldn’t place any customer data or private
information inside of it. Once you get more comfortable with networking on AWS, you should change
this default setting to choose your own custom VPCs and restrict access with additional routing and
connectivity mechanisms.
 When you stop your instance, it enters the stopping state, and then the stopped state. AWS does not
Architect for High Availability charge usage or data transfer fees for your instance after you stop it, but storage for any Amazon EBS
volumes is still charged. While your instance is in the stopped state, you can modify some attributes,
Inside this network, your instance resides in an Availability Zone of your choice. AWS services that are like the instance type. When you stop your instance, the data stored in memory (RAM) is lost. When
scoped at the Availability Zone level must be architected with high availability in mind. While EC2 you stop-hibernate your instance, AWS signals the operating system to perform hibernation (suspend-
instances are typically reliable, two is better than one, and three is better than two. Specifying the to-disk), which saves the contents from the instance memory (RAM) to the Amazon EBS root volume.
instance size gives you an advantage when designing your architecture because you can use more Consider a scenario where you build a standard three tier application, where you have web servers,
smaller instances instead of a few larger ones. If your frontend only has a single instance and that application servers and database servers. Turns out, the application you built becomes extremely
instance fails, your application goes down. On the other hand, if your workload is distributed across 10 popular. To relieve some stress on the database that supports your application, you want to implement a
instances and one fails, you lose only 10 percent of your fleet and your application availability is custom backend layer that caches database information in memory (RAM). You decide to run this
hardly affected. When architecting any application for high availability, consider using at least two custom backend caching solution on Amazon EC2. In this scenario, the stop-hibernate feature would
EC2 instances in two separate Availability Zones. be instrumental in persisting storage. It would prevent you from having to manually create scripts to
save this RAM data before shutting down the server.
Explore the EC2 Instance Lifecycle
What Makes Up the Pricing?
An EC2 instance transitions between different states from the moment you create it all the way through
to its termination. To understand EC2 pricing, let’s decouple the instance price from other services attached to it, such as
storage and networking costs. In this unit we refer to the instance cost as the cost associated with the
When you launch an instance, it enters the pending state (1). When the instance is pending, billing has instance in terms of specifications and not the total blended cost of running an instance. Once an
not started. At this stage, the instance is preparing to enter the running state. Pending is where AWS instance is launched in your AWS account, the billing usually accrues on a per-second basis. For
performs all actions needed to set up an instance, such as copying the AMI content to the root device simplicity of calculation, prices are stated per-hour. For example, if you have an instance running for 5
and allocating the necessary networking components. When your instance is running (2), it's ready to minutes and 38 seconds during a given month, you only pay for 338 seconds of utilization at the end of
use. This is also the stage where billing begins. As soon as an instance is running, you are then able to the month. One exception to this pricing convention may be third-party AMIs purchased from the AWS
take other actions on the instance, such as reboot, terminate, stop, and stop-hibernate. When you reboot Marketplace, which may have a minimum billing of 1 hour. For more details, check out the resources
an instance (3), it’s different than performing a stop action and then a start action. Rebooting an section of this unit.
instance is equivalent to rebooting an operating system. The instance remains on the same host
computer and maintains its public and private IP address, and any data on its instance store. It typically What Are the EC2 Pricing Options?
takes a few minutes for the reboot to complete. When you stop and start an instance (4), your instance
may be placed on a new underlying physical server. Therefore, you lose any data on the instance store One of the ways to reduce costs with Amazon EC2 is to choose the right pricing option for the way
that were on the previous host computer. When you stop an instance, the instance gets a new public IP your applications run. There are three main purchasing options for EC2 instances: on-demand,
address but maintains the same private IP address. When you terminate an instance (5), the instance reserved, and spot instances.
store are erased, and you lose both the public IP address and private IP address of the machine.
Termination of an instance means you can no longer access the machine. Pay As You Go with On-Demand Instances

What Is the Difference Between Stop and Stop-Hibernate?

With On-Demand instances, you pay for compute capacity with no long-term commitments. Billing determines. If the amount you pay is more than the current Spot price and there is capacity, then you
begins whenever the instance is running, and billing stops when the instance is in a stopped or will receive an instance. While they are very promising from the billing perspective, there are some
terminated state. The price per second for a running On-Demand instance is fixed. For applications that architectural considerations you will need to consider in order to use them effectively. One
require servers to be running all the time, you are less likely to benefit from the On-Demand pricing consideration is that your spot instance may be interrupted. For example, if AWS determines that
model, simply because there is no situation where you will need to turn servers off. For example, you capacity is no longer available for a particular spot instance or if the Spot price exceeds how much you
might want the web server hosting the frontend of your corporate directory application to be running are willing to pay, AWS will give you a 2-minute warning before it interrupts your instance. That
24/7 so that users can access the website at any time. Even if there are no users connected to your means any application or workload that runs on a Spot instance must be able to be interrupted. Because
of this unique consideration, inherently fault-tolerant workloads are typically good candidates to use
website, you don’t want to shut down the servers supporting the site in case of potential user activity.
with Spot instances. These include big data, containerized workloads, continuous
In the case when servers cannot be stopped, consider using a Reserved Instance to save on costs.
integration/continuous delivery (CI/CD), web servers, high-performance computing (HPC), image and
media rendering, or other test and development workloads.
Reserve Capacity with Reserved Instances (RIs)

RIs provide you with a significant discount compared to On-Demand instance pricing. RIs provide a
Video: Demonstration: Launching the Employee Directory Application
discounted hourly rate and an optional capacity reservation for EC2 instances. You can choose between
three payment options: All Upfront, Partial Upfront, or No Upfront. You can select either a 1-year or 3- The AMI is the Amazon machine image, and this is a template that contains the software configuration
year term for each of these options. Depending on which option you choose, you are discounted for your instance on boot like the operating system, any sort of application server or any applications
differently. that you wanna have pre-installed when you launch your instance.

We're gonna be using the Amazon Linux 2023 AMI for this, though you could browse the AWS
 All Upfront offers a higher discount than Partial Upfront instances.
marketplace to get access to the different types of AMIs that vendors offer. So, if you're using some
 Partial Upfront instances offer a higher discount than No Upfront. sort of third party software that you want to launch on an EC2 instance, they might have an AMI out
 No Upfront offers a higher discount than On-Demand. there that you could use to make your life a little bit simpler.

On-Demand and No Upfront are similar since both do not require any upfront payment. However, You also can create and manage your own AMIs and share them internally within your organization to
there is a major difference. When you choose an On-Demand instance, you stop paying for the instance say you can only launch an EC2 instance using one of our approved AMIs that might have some
when you stop or terminate the instance. When you stop an RI, you still pay for it because you specific security or compliant sort of packages pre-installed.
committed to a 1-year or 3-year term. Reserved Instances are associated with an instance type and an So leaving the Amazon Linux 2023 AMI selected I'm going to scroll down and then we can select our
Availability Zone depending on how you reserve it. The discount applied by a Reserved Instance instance type. The instance type will determine how much CPU, how much memory, what types of
purchase is not directly associated with a specific instance ID, but with an instance type. hardware is available to you on this instance. Some instances come with a GPU, for example. In this
case, we are gonna use the free-tier eligible EC2 instance type, T2 micro which is the default that's
Save on Costs with Spot Instances selected. So we'll leave that the way it is. Then we can scroll down and select whether we want to use a
key pair. The key pair is going to allow you to have access to this instance doing something like SSH.

Another way of paying for EC2 instances is by using Spot Instances. Amazon EC2 Spot Instances Security groups are instance level firewalls.
allow you to take advantage of unused EC2 capacity in the AWS Cloud. They are available at up to a
90% discount compared to On-Demand prices. With Spot Instances, you set a limit on how much you
would like to pay for the instance hour. This is compared against the current Spot price that AWS
I'm going to select proceed without a key pair because if I need to connect to this instance, I'm gonna gonna be using S3 to store our photos. We don't have that value yet, so we'll just leave this place holder
use the connect button in the AWS console where I don't need to have a key pair to connect to this there. Then we have the default region we're operating out of Oregon, and then we have our Dynamo
instance to view things like logs. mode on and this means that we want our application to use Dynamo DB and then we're running this
application and hosting it on port 80. Now we can select launch instance, and this will begin to go
Then scrolling down some more we then can configure our network settings. This is using a default build out the different things that we need like our security group and we can see that our instance has
VPC and a default subnet. This default VPC is going to have subnets that have internet access, so launched.
meaning that there is an internet gateway attached to this VPC that allows internet traffic to flow into
this default VPC. You'll learn more about VPCs coming up, but just understand that this default VPC
does allow public internet access which is good for our use case, but might not be great for other use
cases where you would more than likely put your instance behind something like a load balancer and Now if I select this instance ID, that brings us to a page where we can then select our instance and
then you would use private subnets. But we wanna be able to access this instance directly. So with that view some information about it. We can see things like our public IP address our private IP address,
in mind, we're going to use the default VPC and we're also going to leave this auto-assigned public IP and our public DNS name. But we can't access this directly yet. We can see that our instant state is
to enable and this will allow our EC2 incidents to have a publicly reachable IP address once it's been running. But if I refresh this, you can see that we have some status checks that are still initializing. I
launched. wanna wait until both of these status checks have passed. So we'll give this a few minutes and then
we'll come back once it's ready.
Then scrolling down a little bit more we have to configure our firewall. This is going to be our security
group. Security groups are instance level firewalls. We want to disable traffic from SSH because we're
not going to need that. Then we're going to allow HTTPS and allow HTTP. In our application, we're All right, and we can see that our employee directory application has been opened in a new tab and we
gonna be using HTTP directly. Then scrolling down, we can configure some storage. We're gonna have an empty employee directory. This is exactly what we would expect at this point as we don't have
leave our root volume here, but we're not going to add any more EBS volumes. Then we can expand a place to store our photos or the employee information with our S3 bucket or Dynamo DB table.
the advanced details section and then we need to select an IM instance profile. In this case, I'll be
selecting the employee web app role and this role has permissions attached to it to allow whoever's
using the role to make calls to S3 or to Dynamo DB. We know that our application has some code
Video: Container Services on AWS
that's running that's going to be making API calls to S3 and Dynamo DB. We need to allow our AWS
SDK to actually get access to these credentials, and the way that we do that is through this IM instance For example, let's say you are looking for a way to run containers at scale on AWS. Containers can
profile. So we're associating this role with this instance profile that will allow our code to get access to provide you with efficiency and portability. In this context, I am specifically referring to containers like
these temporary credentials. Then scrolling down some more we will come to the user data section, and docker containers or containers that use the containered runtime. If you aren't sure what containers are,
this is where you can provide a bash script that will run on launch. So, this script here is what's gonna please review the readings on containers. Containers are portable because everything an application
be used to download and install our application. Our application was written in Python using Flask as needs to run, including the application code, dependencies, and configuration are all packaged up
the web server, and this is going to be for our employee directory application. So on the first line, we together as one executable, which is the container itself. Since everything is self-contained, it means
are just declaring that this is a Bash script. Then we are using Wget to download a zip file from S3. So that you can expect your containers to behave the same way across different environments, like
we hosted the zip file in an S3 bucket for this to be downloaded. Then we're unzipping that, and then development, QA and production.
we're changing directories into that new directory. Then we're using Yum to install Python three and
PIP. Then we're using PIP to install any dependencies for our Flask application, which will all be listed AWS offers container services like Amazon Elastic Container Service, otherwise known as ECS, or
in a file called requirements.txt, which got downloaded from this zip file. Then we're installing a Amazon Elastic Kubernetes Service, otherwise known as EKS. Both of these services are container
package called stress. This stress package is going to be used to simulate a CPU spike on this instance, orchestration tools. When you use containers on AWS, you need processes to start, stop, restart, and
which is going to allow us to simulate autoscaling whenever we use Amazon EC2 autoscaling in a monitor containers running across, not just one EC2 instance, but a number of them together, called a
future lesson. Then we have three different environment variables. One for our photos bucket, we're cluster. The process of doing these tasks is called container orchestration. And turns out, it's really hard
to do on your own. If you have a few containers, it's not so bad. But once you get to hundreds or that packages up your code and all of its dependencies. This package is designed to run
thousands of containers you are managing, it can be quite complex. reliably on any platform, because the container creates its own independent environment. This
makes it easy to carry workloads from one place to another, such as from development to
Orchestration tools were created to help you run and manage containers. ECS is designed to help you production or from on-premises to the cloud.
run your containerized applications at scale without the hassle of managing your own container
orchestration software. EKS does essentially the same thing, but uses different tooling with different WHAT IS DOCKER?
features.
 When you hear the word container, you may associate it with Docker. Docker is a popular
And then the orchestration tool carries out the management tasks. You can automate scaling of your container runtime that simplifies the management of the entire operating system stack needed
cluster hosting your containers, as well as automate scaling of the containers themselves. Containers for container isolation, including networking and storage. Docker makes it easy to create,
have a shorter boot-up time when compared to virtual machines. So if you need super fast response to package, deploy, and run containers.
increasing demand, then containers might be the service for you.

However, what if you were looking for an alternative to hosting your containers, because you either
don't need access to the underlying OS, or you don't want to manage those EC2 instances? Well, you WHAT IS THE DIFFERENCE BETWEEN CONTAINERS AND VMS?
can use a compute platform called AWS Fargate, which handles most of the underlying details for you.
Fargate is a serverless compute platform for ECS and EKS.

Reading 2.3: Container Services on AWS

AWS offers a broad spectrum of compute offerings that give you the flexibility to choose the right tool
for the right job. The three main categories of compute are virtual machines, containers, and serverless.
There is no one-size-fits-all service because it depends on your needs. The key is to understand what
each option has to offer in order to build a more appropriate cloud architecture for your use case. In
this unit, you learn about containers and how to run them on AWS. Containers can host a variety of
different workloads, including web applications, lift and shift migrations, distributed applications, and
streamlining of development, test, and production environments.

WHAT IS A CONTAINER?
Containers share the same operating system and kernel as the host they exist on, whereas virtual
 While containers are often referred to as a new technology, the idea started in the 1970s with machines contain their operating system. Since each virtual machine has to maintain a copy of an
certain Linux kernels having the ability to separate their processes through isolation. At the operating system, there’s a degree of wasted space. A container is more lightweight. They spin up
time, this was configured manually, making operations complex. With the evolution of the quicker, almost instantly. This difference in startup time becomes instrumental when designing
open source software community, containers evolved. Today, containers are used as a solution applications that need to scale quickly during input/output (I/O) bursts. While containers can provide
to problems of traditional compute, including the issue of getting software to run reliably speed, virtual machines offer you the full strength of an operating system and offer more resources,
when it moves from one compute environment to another. A container is a standardized unit like package installation, a dedicated kernel, and more.
ORCHESTRATE CONTAINERS

In AWS, containers run on EC2 instances. For example, you may have a large instance and run a few
containers on that instance.While running one instance is easy to manage, it lacks high availability and
scalability. Most companies and organizations run many containers on many EC2 instances across
several Availability Zones.If you’re trying to manage your compute at a large scale, you need to know:

 How to place your containers on your instances.

 What happens if your container fails.
 What happens if your instance fails.
 How to monitor deployments of your containers.

This coordination is handled by a container orchestration service. AWS offers two container
orchestration services: Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes
Service (EKS).

MANAGE CONTAINERS WITH AMAZON ELASTIC CONTAINER SERVICE (AMAZON

ECS)

Amazon ECS is an end-to-end container orchestration service that allows you to quickly spin up new To run and manage your containers, you need to install the Amazon ECS Container Agent on your EC2
containers and manage them across a cluster of EC2 instances. instances. This agent is open source and responsible for communicating back to the Amazon ECS
service about cluster management details. You can run this agent on both Linux and Windows AMIs.
An instance with the container agent installed is often called a container instance.
Once the Amazon ECS container instances are up and running, you can perform actions that include,
but are not limited to, launching and stopping containers, getting cluster state, scaling in and out,
scheduling the placement of containers across your cluster, assigning permissions, and meeting
availability requirements.

To prepare your application to run on Amazon ECS, you create a task definition. The task definition is
a text file, in JSON format, that describes one or more containers. A task definition is similar to a
blueprint that describes the resources you need to run that container, such as CPU, memory, ports,
images, storage, and networking information.

Here is a simple task definition that you can use for your corporate director application. In this
example, the runs on the Nginx web server. USE KUBERNETES WITH AMAZON ELASTIC KUBERNETES SERVICE (AMAZON EKS)

Kubernetes is a portable, extensible, open source platform for managing containerized workloads and
services. By bringing software development and operations together by design, Kubernetes created a
rapidly growing ecosystem that is very popular and well established in the market.If you already use
Kubernetes, you can use Amazon EKS to orchestrate these workloads in the AWS Cloud.Amazon EKS
is conceptually similar to Amazon ECS, but there are some differences.

 An EC2 instance with the ECS Agent installed and configured is called a container instance.
In Amazon EKS, it is called a worker node.
 An ECS Container is called a task. In the Amazon EKS ecosystem, it is called a pod.
  While Amazon ECS runs on AWS native technology, Amazon EKS runs on top of is responsible for moves up. You do not have access to the underlying operating system that is running
Kubernetes. the service. Therefore, you cannot be responsible for carrying out tasks like patching. You are,
however, still responsible for things like data encryption and access management.
If you have containers running on Kubernetes and want an advanced orchestration solution that can
provide simplicity, high availability, and fine-grained control over your infrastructure, Amazon EKS is I like to think of AWS services as existing on a spectrum. On one side of the spectrum,
the tool for you.

Video: Introduction to Serverless

When using Amazon EC2 or container services running on top of EC2 as a compute platform, you are
required to set up and manage your fleet of instances. This means that you are responsible for patching
your instances when new software packages or security updates come out, setting up the scaling of
those instances, as well as ensuring that you've architected your solutions to be hosted in a highly
available manner which means deploying instances across two AZs at a minimum as we discussed
earlier.
you have convenience and on the other side, you have control. Many services exist to give you control
This is less management and operational overhead than you would have if you hosted the same like Amazon EC2 whereas other services exist to give you convenience, like the serverless compute
solutions on premises but management processes will still need to be in place, and it's your AWS Lambda. We'll dive deeper into some of these serverless compute technologies coming up.
responsibility to create those processes to fit your use case. This is actually really great and in many
use cases is exactly what you want because it gives you a great deal of control over your solutions.
Video: Serverless with AWS Fargate
Whatever happens on your instances and with your instances is in your hands.
ECS or EKS is the container orchestrator. This is the tool that is managing the container's lifecycle.
You can create simple or complicated setups that run in scale on AWS exactly to your specification.
Then you need the compute platform. This is where the containers actually run.
That being said, not every single solution requires this level of control over the underlying
environment.

A lot of AWS services are serverless in nature. Serverless means that you cannot see or access the
underlying infrastructure or instances that are hosting your solution. Instead, all of the management of
the underlying environment from a provisioning, scaling, fault tolerance, and maintenance perspective
are taken care of for you. All you need to do is focus on your application. The rest is taken care of, or
more accurately the rest is abstracted from you. This means that serverless offerings are very
convenient to use. You get to focus on the value that the service delivers and not the underlying
implementation details thus reducing the number of operational support processes that you need to
have in place.

For example, with Amazon EC2 instances, you are required to patch the OS when new security updates
are released. With serverless offerings, the line between what you are responsible for versus what AWS
Previously, you learned that ECS and EKS run containers on clusters of EC2 instances. And in that
case, you were using EC2 as the compute platform for your containers, and you also have tight control
over those instances. Now, EC2 is certainly not serverless. So thinking about serverless compute for
containers, I want to introduce to you a service named AWS Fargate.

To break that down a bit, the way you work with Fargate is you first build your container images and
push them into a repository like Amazon Elastic Container Registry, for example. Which is a place
where you can store container images for them to be pulled and deployed from. So you have your
container images in your repo, and then you define memory and compute resources for your task if
you're using ECS or your pod if you're using EKS. Then you run your containers. And at the end, you
only pay for the amount of vCPU, memory and storage resources consumed by your containerized
applications.

AWS Fargate is a serverless compute platform for containers that you can use with either ECS or EKS. Fargate does support spot and compute savings plan pricing options just like with Amazon EC2
With AWS Fargate as the compute platform, you run your containers on a managed serverless instances. So there is some flexibility on how you plan to run containers on Fargate. So you can still
platform. The scaling and fault tolerance is built-in, and you don't need to worry about the underlying get a good amount of control for your container's deployment, but without needing to worry about the
operating system or environment. provisioning, patching and managing the underlying operating systems or instances. And no need to
scale the infrastructure in and out to meet demand like you do with EC2. As far as use cases go, AWS
Instead, you define your application content, networking, storage and scaling requirements. There is no Fargate can be used for all of the common container use cases including microservice architecture
provisioning, patching, cluster capacity management or infrastructure management required. applications, batch processing, machine learning applications and migrating on-premises applications
to the cloud. All right, hopefully, you now have a 10,000-foot view of AWS Fargate. You can see how
Fargate is an example of how serverless simplifies your operations and management for running scaled
container solutions.

Video: Introduction to AWS Lambda

 the language your Lambda function is coded in, the amount of memory and CPU your function is
allocated in the environment, permissions, dependencies, and many other aspects of how the function
runs.

AWS Lambda is one of the serverless compute options you have available to you on AWS. Lambda
allows you to package and upload your code to the Lambda service, creating what is called a Lambda
function. Once you create a Lambda function, it isn't always running all of the time. Instead, Lambda
functions run in response to triggers. You can configure a trigger for when you want your function to
run, and from there the Lambda service waits for the trigger or polls for an event to trigger the function
depending on what the trigger is that you choose.

A couple of common examples of triggers for Lambda functions are an HTTP request, an upload of a If you have 1 or 1,000 incoming triggers AWS Lambda will scale your function to meet demand, each
file to the storage service Amazon S3, events originating from other AWS services, or even in-app in its own isolated environment. Lambda is currently designed to run code that has a runtime of under
activity from mobile devices. 15 minutes.

An execution environment provides a secure and isolated runtime environment for your Lambda
Function.

So this isn't for long running processes like deep learning or batch jobs. You wouldn't host something
like a WordPress site on AWS Lambda. It's more suited for quick processing like a web backend
handling request, or a backend report processing service or microservice hosting.

AWS Lambda participates in Compute Savings Plans

One of the best things about Lambda is that you aren't billed for code that isn't running. You only get
billed for the resources that you use rounded up to the nearest one millisecond interval.

In this demo, I will create a Lambda function that resizes images uploaded into the employee directory
to be uniform thumbnail size. It makes sense to create this sort of logic with a Lambda function. You
When the trigger is detected, the code is automatically run in a managed environment, an environment
don't need an application to be running 24/7 on EC2 to resize photos uploaded to the application. You
that you do not need to worry too much about because it is automatically scalable, highly available,
really only need to run the resize logic when a new photo is uploaded.
and all of the maintenance of the environment itself is done by AWS. You do, however, get to choose
 All right, let's go ahead and build this out. You can see that I'm already in the AWS Lambda console
and I'm going to first click Create function. Next, we get to select what type of function we want to
build. Do we want to author a function from scratch? Do we want to use a blueprint which will give
you some sample code and configuration presets? Or you can use a container image with AWS Lambda
as Lambda does support running containers.

So, here's our diagram for the app. What I want to do here is when a new photo is uploaded to S3, it
triggers a Lambda function that then resizes the image and uploads it into that same S3 bucket, but to a
different location than where the original image is stored.

We are going to author a function from scratch. For the function name, we will call this ResizeImage,
and then we get to select the runtime. The runtime is the language that your Lambda function is written
in, because with Lambda, it runs in the Lambda environment, but it's still your code. So I'm gonna go
ahead and select Python 3.9 for this as we wrote this function in Python. And then we have to select the
permissions. And the permissions are going to be delegated to the function through IAM roles. So I'm
gonna click Use an existing role, and then if I expand this dropdown, we can select
LambdaS3FullAccess which is an IAM role that I've already created in this account that will give this
function permissions to read and write from S3.

Now we can scroll down and select Create function. I already have an S3 bucket created in this account called photo-upload-bucket-mw, we'll select that
one. And then if we scroll down, I'm going to select the event type. I want to select only a PUT, so I
Now we can add a trigger for this Lambda function, and I'll select Add trigger. And here we get to want PUT operations to trigger this AWS Lambda function. Then we can provide a prefix. With this
configure the trigger source, and if we expand this dropdown, you can see a list of AWS services that prefix, what we're essentially doing is we're going to say: I only want to trigger this Lambda function if
can act as triggers like API Gateway, an Application Load Balancer, DynamoDB. We're going to scroll a PUT occurs in a specific location inside of this bucket. We're going to upload photos to an input
down and select S3 for our trigger. And then we need to select which S3 bucket will be the event prefix and that will then trigger the Lambda function. We then need to acknowledge that if we're using
source. the same S3 bucket for both input and output that it could cause recursive invocations. So the way that
we're getting around that is by supplying this input prefix, the event will be triggered when an image is
uploaded to this prefix, but then the output will actually be uploaded to a different location. We'll have
an output prefix where the image will be uploaded to. So we're gonna go ahead and acknowledge by
checking this checkbox and then click Add.
 Video: Choose the Right Compute Service

Consider a scenario where you are a developer who is tasked with creating a new feature for a web
application being hosted on EC2. The web application is an online store. And right now, all the items
being sold in the store are loaded into a database manually behind the scenes.

By manually, I mean there is a person who adds a new row to a database for each new item to be sold
in the store.

This process takes a long time, isn't very scalable, and is prone to error. You are tasked with
automating the process of getting the new item information loaded into the inventory database. The
Now back on our Function overview page, what we need to do next is actually upload the code for this.
goal is to have a person upload an inventory spreadsheet into Amazon S3, the object storage service,
So if we scroll down, we can select the Code tab,
then have a process automatically load the data into the inventory database.
and then you can see that there's a stubbed out Lambda function here that essentially just says, "Hello
What compute would you use to host the processing logic to load the items from the file into the
world, hello from Lambda." And what we need to do is upload our code. So I'm going to click Upload
database?
from, and then select zip file. Click Upload, and then I will select this ResizeImage.zip, click Open,
and then click Save. - All right, let's discuss how you could have answered this question. You could have decided to
use Amazon EC2 here. You could spin up a new instance specifically for this process and
All right, our function has been successfully created. We have our trigger defined, and we have our
write some code that polls the location of the spreadsheet for a new upload every so often.
code uploaded. What I want to do now is test this out by actually uploading an image to S3 and then
Updating the database when it finds a new file, that would work, but before I make my final
viewing the output. So in the search bar, I'll type in S3 and then select S3.
answer here, I have a question.

How often does new inventory get added to the database?

Then from here we'll select the bucket,
- New inventory gets updated once a quarter. - Good to know. So, it's not very often, which
and then we'll go into the input folder and then select Upload, means this process would only run once a quarter and that does change my answer. Here's
why.
click Add files, and then I'll upload Seph's image, and select Open, and then Upload. - Amazon EC2 charges per second or per hour. So, if I have an instance running all the time to
serve requests that happens once per quarter, that seems like a lost opportunity to optimize for
So now if we go check, we should see the output with the thumbnail version of that image. So now
cost. I would be spending money on a resource I rarely use. It certainly would work, but it
going back into our bucket, if we go up one level, we can now see that the output prefix has been
may not be the best fit for this use case. I could automate the process of starting and stopping
created, so I'll select that, and we can see that the thumbnail image was created. And that's it, that's how
the instance when needed.
you create an AWS Lambda function. You could host the entire employee directory applications
backend on Lambda with some refactoring, but I'm gonna go ahead and save that for a later But instead, what about using AWS Lambda?
conversation.
- AWS Lambda is the correct answer for this one. There are a few reasons. First of all, to
address your concern on cost, AWS Lambda only charges you for the compute you consume
when the code is actually running. And code is only run in response to triggers or a direct
 invitation. So here's my suggestion. You know that the goal is to have someone upload an - That's correct. So if you answered EC2 at home/
inventory document to S3, which should kick off the process of updating the database. You
also learned that AWS Lambda has triggers that run your Lambda functions code. AWS Explain the thought process behind eliminating the other compute services we covered in the answers
Lambda integrates with many AWS services to act as triggers, and Amazon S3 is one of them. to this question?

- So, AWS Lambda could work, but you can't just upload the same code you would run on
Amazon EC2 into a Lambda function. There would have to be a decent amount of refactoring
in order to take advantage of that service. Same idea with any of the AWS container services,
like ECS or EKS. Again, you'd have some amount of rework required to migrate to
containers. Therefore, Amazon EC2 is the best option for this migration.

FINAL QUESTION:

Imagine a scenario where you are planning to write a brand-new application using a microservices or
service-oriented design. And you want to architect the application where it can scale up or down
quickly, and you want to lower the risk of deploying new changes to production.

Which AWS compute service would you use?

- The answer for this use case is... One of the AWS container services like Amazon ECS or
Amazon EKS.
- Correct. The answer is either ECS or EKS for this one because using containers makes it
So, my suggestion would be to create an AWS Lambda function; configure a PutEvent as the easier to support microservice or service-oriented designs. Containers boot up quickly, so
trigger from Amazon S3; then when the inventory is uploaded, Amazon S3 will trigger the scaling is quicker than EC2 instances, and the use of containers helps with code portability.
Lambda function to run and the code in the function will parse the inventory document and
add each item to the database.

Next question:

Let's say you have an application currently hosted in your on-premises data center, which needs to be
migrated to AWS. It's currently running on Linux servers in the data center, and you want to minimize
the amount of refactoring needed to migrate to AWS. It's important that this workload is elastic and
can support varying demand.

What compute option would you choose? Five seconds on the clock. (clock ticking)

- Considering the fact that minimizing refactoring is an important aspect of this workload, I
would architect a solution using Amazon EC2 as the compute service. EC2 instances can be
launched from Linux-based AMIs, and the application could be hosted on the EC2 instance
the same way it would be hosted on a Linux server on premises. Amazon EC2 also has the
ability to scale in or out based on demand, so I think EC2 is the best service for this one.
 - Meaning, if I write the code on my laptop and run it in a container, test it in QA in a container,
I can then expect the same container to behave the same way once deployed to production,
thus reducing the risk of deployments causing errors because of environmental issues

Reading 2.4: Serverless and AWS Lambda

REMOVE THE UNDIFFERENTIATED HEAVY LIFTING

If you run your code on Amazon EC2, AWS is responsible for the physical hardware and you are
responsible for the logical controls, such as guest operating system, security and patching, networking,
security, and scaling.If you run your code in containers on Amazon ECS and Amazon EKS, AWS is
responsible for more of the container management, such as deploying containers across EC2 instances
and managing the container cluster. However, when running ECS and EKS on EC2, you are still AWS Fargate is a purpose-built serverless compute engine for containers. Fargate scales and manages
responsible for maintaining the underlying EC2 instances.If you want to deploy your workloads and the infrastructure, allowing developers to work on what they do best: application development. It
applications without having to manage any EC2 instances, you can do that on AWS with serverless achieves this by allocating the right amount of compute, eliminating the need to choose and handle
compute. EC2 Instances and cluster capacity and scaling. Fargate supports both Amazon ECS and Amazon EKS
architecture and provides workload isolation and improved security by design.

AWS Fargate abstracts the EC2 instance so you’re not required to manage it. However, with AWS
GO SERVERLESS
Fargate, you can use all the same ECS primitives, APIs, and AWS integrations. It natively integrates
 Every definition of serverless mentions four aspects. with AWS Identity and Access Management (IAM) and Amazon Virtual Private Cloud (VPC). Having
 No servers to provision or manage. native integration with Amazon VPC allows you to launch Fargate containers inside your network and
 Scales with usage. control connectivity to your applications.
 You never pay for idle resources.
RUN YOUR CODE ON AWS LAMBDA
 Availability and fault tolerance are built-in.
If you want to deploy your workloads and applications without having to manage any EC2 instances or
With serverless, spend time on the things that differentiate your application, rather than spending time
containers, you can use AWS Lambda.AWS Lambda lets you run code without provisioning or
on ensuring availability, scaling, and managing servers. AWS has several serverless compute options,
managing servers or containers. You can run code for virtually any type of application or backend
including AWS Fargate and AWS Lambda.
service, including data processing, real-time stream processing, machine learning, WebSockets, IoT
EXPLORE SERVERLESS CONTAINERS WITH AWS FARGATE backends, mobile backends, and web apps, like your corporate directory app!

Amazon ECS and Amazon EKS enable you to run your containers in two modes. AWS Lambda requires zero administration from the user. You upload your source code and Lambda
takes care of everything required to run and scale your code with high availability. There are no servers
 Amazon EC2 mode to manage, bringing you continuous scaling with subsecond metering and consistent performance.
 AWS Fargate mode
HOW LAMBDA WORKS A trigger integrates your Lambda function with other AWS services, enabling you to run your Lambda
function in response to certain API calls that occur in your AWS account. This makes you quicker to
There are three primary components of a Lambda function: the trigger, code, and configuration.The respond to events in your console without having to perform manual actions.All you need is the what,
code is source code, that describes what the Lambda function should run. This code can be authored in how, and when of a Lambda function to have functional compute capacity that runs only when you
three ways. need it to.Amazon’s CTO, Werner Vogels, says, “No server is easier to manage than no server.” This
quote summarizes the convenience you can have when running serverless solutions, like AWS Fargate
and AWS Lambda.

In the next unit, you apply all the information you’ve learned about Amazon EC2, Amazon ECS and
Amazon EKS, and AWS Fargate and learn the use cases for each service.

AWS Lambda function handler

The AWS Lambda function handler is the method in your function code that processes events. When
your function is invoked, Lambda runs the handler method. When the handler exits or returns a
response, it becomes available to handle another event.You can use the following general syntax when
creating a function handler in Python:

def handler_name(event, context): ... return some_value

NAMING
 You create the code from scratch.
 You use a blueprint that AWS provides. The Lambda function handler name specified at the time you create a Lambda function is derived from
 You use same code from the AWS Serverless Application Repository, a resource that contains the following:the name of the file in which the Lambda handler function is locatedthe name of the
sample applications, such as “hello world” code, Amazon Alexa Skill sample code, image Python handler functionA function handler can be any name; however, the default on the Lambda
resizing code, video encoding, and more. console is lambda_function.lambda_handler. This name reflects the function name as lambda_handler,
and the file where the handler code is stored in lambda_function.py.If you choose a different name for
When you create your Lambda function, you specify the runtime you want your code to run in. There your function handler on the Lambda console, you must update the name on the Runtime settings
are built-in runtimes such as Python, Node.js, Ruby, Go, Java, .NET Core, or you can implement your pane.
Lambda functions to run on a custom runtime.The configuration of a Lambda function consists of
information that describes how the function should run. In the configuration, you specify network
placement, environment variables, memory, invocation type, permission sets, and other configurations. BILLING GRANULARITY
To dive deeper into these configurations, check out the resources section of this unit.Triggers describe
when the Lambda function should run. AWS Lambda lets you run code without provisioning or managing servers, and you pay only for what
you use. You are charged for the number of times your code is triggered (requests) and for the time
your code executes, rounded up to the nearest 1ms (duration). AWS rounds up duration to the nearest
millisecond with no minimum execution time. With this pricing, it can be very cost effective to run
functions whose execution time is very low, such as functions with durations under 100ms or low
latency APIs. For more information, see AWS News Blog

SOURCE CODE
AWS NETWORKING
This video used a small amount of sample code illustrating a pattern for lazily generating assets using
AWS Lambda and Amazon S3. If you’re looking to deploy a service to resize images to production, Video: Networking on AWS
consider using the new release Serverless Image Handler which is a robust solution to handle image
manipulation and can be deployed via an AWS CloudFormation template.

You can find a tutorial on creating the AWS Lambda function as well as the code used in the AWS
Lambda demo here: see AWS News Blog.

In a lesson you saw earlier, we launched an instance and connected to that instance over the internet
but to launch this instance, we needed to select a network. How you configure this network or VPC is
actually what helps enable internet traffic to flow into your application. You might be asking, Seph,
how did we select a VPC when we haven't even built a VPC yet? You're right, we haven't built a VPC,
but fortunately AWS creates VPCs for us in every region by default.
  The payload or letter inside the envelope.
 The address of the sender in the From section.
 The address of the recipient in the To section.

Let’s go further. Each address must contain information such as:

 Name of sender and recipient

 Street
 City
 State or province
 Zip, area, or postal code\
 Country

You need all parts of an address to ensure that your letter gets to its destination. Without the correct
When we launched our EC2 instances earlier in the course we simply chose the default VPC. These address, postal workers are not able to properly deliver the message. In the digital world, computers
default VPCs are meant to provide an easy way for you to get started with EC2 instances quickly and handle the delivery of messages in a similar way. This is called routing.
view your applications over the internet.

However, you have to be careful with what you put inside of these default VPCs, as those resources
will be subject to well, the internet. And with internet access comes potential dangers. If we didn't WHAT ARE IP ADDRESSES?
select this default VPC, we would've needed to select a VPC that we custom-built.
 In order to properly route your messages to a location, you need an address. Just like each
In this section of the course we'll focus on doing just that, building a custom VPC for our application home has a mail address, each computer has an IP address. However, instead of using the
that is more secure and provides more granular access to the internet than the default option we combination of street, city, state, zip code, and country, the IP address uses a combination of
originally chose. While viewing this section, keep in mind that this information is geared towards EC2- bits, 0s and 1s.
related services. If you are using a different kind of compute, such as a lambda function, you might not
Here is an example of a 32-bit address in binary format:
need a network at all. However, that doesn't mean this isn't important. Networking on AWS is the basis
of most architectures, so consider this information highly valuable.

Reading 2.5: Networking on AWS

WHAT IS NETWORKING?
It’s called 32-bit because you have 32 digits. Feel free to count!
 Networking is how you connect computers around the world and allow them to communicate
with one another. In this trail, you’ve already seen a few examples of networking. One is the
AWS global infrastructure. AWS has created a network of resources using data centers,
Availability Zones, and Regions. WHAT IS IPV4 NOTATION?

KNOW THE NETWORKING BASICS Typically, you don’t see an IP address in this binary format. Instead, it’s converted into decimal format
and noted as an Ipv4 address.
Think about sending a letter. When sending a letter, there are three pieces of information you need.
In the diagram below, the 32 bits are grouped into groups of 8 bits, also called octets. Each of these
groups is converted into decimal format separated by a period.

32 total bits subtracted by 24 fixed bits leaves 8 flexible bits. Each of these flexible bits can be either 0
or 1, because they are binary. That means you have two choices for each of the 8 bits, providing 256 IP
addresses in that IP range.

The higher the number after the /, the smaller the number of IP addresses in your network. For
example, a range of 192.168.1.0/24 is smaller than 192.168.1.0/16.
In the end, this is what is called an Ipv4 address. This is important to know when trying to When working with networks in the AWS Cloud, you choose your network size by using CIDR
communicate to a single computer. But remember, you’re working with a network. This is where CIDR notation. In AWS, the smallest IP range you can have is /28, which provides you 16 IP addresses. The
Notation comes in. largest IP range you can have is a /16, which provides you with 65,536 IP addresses.
USE CIDR NOTATION

192.168.1.30 is a single IP address. If you wanted to express IP addresses between the range of Video: Introduction to Amazon VPC
192.168.1.0 and 192.168.1.255, how can you do that?
you'll be learning how to create a VPC
One way is by using Classless Inter-Domain Routing (CIDR) notation. CIDR notation is a compressed
way of specifying a range of IP addresses. Specifying a range determines how many IP addresses are The idea of a VPC is similar to how you think of walls around a data center. In a data center, walls act
available to you. as a boundary between the outside world and all of your infrastructure. A VPC in AWS acts the same
way. It creates a boundary where your applications and resources are isolated from any outside
CIDR notation looks like this: movement, so nothing comes into the VPC and nothing comes out of the VPC without your explicit
permission.

It begins with a starting IP address and is separated by a forward slash (the “/” character) followed by a
number. The number at the end specifies how many of the bits of the IP address are fixed. In this
example, the first 24 bits of the IP address are fixed. The rest are flexible.
When you create a VPC, you have to declare two specific settings; the region you're selecting, and the
IP range for the VPC in the form of CIDR notation. (notification dings) For our VPC, we'll be locating
If you're looking at the console, you'll first check to make sure that you're in the correct region by
it in the region where the rest of our infrastructure will be, the Oregon region. and our IP range will say
clicking on the upper right hand corner. Seph mentioned we're going to run out of the Oregon region,
is 10.1.0.0/16.
so we'll go ahead and make sure that this says Oregon. Once we choose the right region, we can now
build our network. You'll type in VPC in the service search bar and that'll bring up your VPC
dashboard for this region. From there, you'll click your VPCs and then Create VPC. Now we'll have a
few settings to configure. We'll put in the CIDR block, which is 10.1.0.0/16, and the VPC name, which
we'll say is app-vpc. We'll leave the rest as default, and then we'll click Create VPC. Easy as that.
After you create your VPC, you then divide the space inside the VPC into smaller segments called
subnets. You put your resources, such as your EC2 instances, inside of these subnets. The goal of these
subnets is to provide more granular control over access to your resources. So if I have public resources,
like our Employee Directory app, that we want to be accessed over the internet, I could put these
resources inside a subnet with internet connectivity. If I have more private resources, like a database, I
could create another subnet and have different controls to keep those resources private. To create a
subnet, you need three main things; the VPC you want your subnet to live in, which is this one, the AZ
you want your subnet to live in. In this case, we'll choose AZ A, or in other words us-west-2a, and then
the CIDR range for your subnet, which must be a subset of the VPC CIDR range. For this, we'll choose
the range 10.1.1.0/24.

We'll call this our Public Subnet for public facing resources. Then we'll create another subnet for our
private resources. We'll place it in the same AZ, specify a different non-overlapping CIDR range, say
10.1.3.0/4, and then name it our Private Subnet. Alright, now that we've got two subnets added to our
VPC, it's time to put Morgan to the test again and have her build these out. And this time I'm timing
her. - All right, we're on a time limit here, so let's create these subnets.
Let's go ahead and start creating the public subnet. Back in the console, we'll click on Subnets in the
side panel, and then select Create subnet. Then we'll select the VPC we're working with, in this case is
app-vpc. Once you do that, you'll be prompted to provide a name. We'll say this one is Public Subnet 1.
Then we'll choose the AZ, which Seph mentioned was us-west-2a, and then the CIDR range will be
10.1.1.0/24. We'll leave the rest as defaults, scroll down to the bottom, and click Add new subnet.
We'll now repeat the same steps for our private subnet. Give it a name such as Private Subnet 1. Put it Create an internet gateway and attach it to our VPC. - Okay. Back in the VPC dashboard, you'll click
in the same availability zone, us-west-2a, and then type in the CIDR range, which is 10.1.3.0/24. Now on Internet gateways on the side panel, then Create internet gateway. You'll give the internet gateway a
we can click Create subnet, and both subnets will be created. name, and then click Create. On the details page, you'll then select the Actions dropdown box and
select Attach to VPC. Choose the app-vpc we've been working with, and then click Attach.
As we mentioned earlier, when you create a new VPC, all the resources you put in it are isolated, and
only have access to other resources in that VPC by default. For websites like our corporate directory
application, we want users to access our site over the internet.

So, we have an internet gateway that we can use to allow access from the internet, but what if we had a
VPC with all internal private resources that we want to reach only from our on-premises data center? If
we only want traffic to flow between AWS and our data center, and we don't want to allow direct
To enable internet connectivity, we need a component called an internet gateway. Think of this gateway access from the internet, what do we do? Luckily, there's another gateway designed for this very
as similar to a modem. Just as a modem connects your computer to the internet, the internet gateway purpose called a virtual private gateway or VGW. It allows you to create a VPN connection between a
connects your VPC to the internet. When you create an internet gateway, you then need to attach it to private network, like an on-premises data center or internal corporate network, to your VPC
your VPC. If you create an internet gateway and don't attach it, it won't do anything except sit there.
With the help of a VGW, you can establish an encrypted VPN connection to your private internal AWS
resources. We won't be using a VGW for our application but it's good to know. We'll talk more about
that in an upcoming lesson.
 To do this, we'd need to duplicate the resources in the first AZ to the second AZ, so that means we'd
Alright, so we have one VPC, two subnets, and an internet gateway. Now, every time you look at an need to create two additional subnets, each within another AZ, say AZ B. As a best practice, you
architecture, you should begin to think, how do I make this better? You're not going to have all of the should always be using at least two AZs, and hosting your resources redundantly.
answers right away, but I do want you to take 10 seconds and think about some solutions.
Morgan is going to build out some subnets in another AZ in the background for the rest of our demos.
She's also going to launch an EC2 instance hosting our application in one of the public subnets.

Reading 2.6: Introduction to Amazon VPC

A VPC is an isolated network you create in the AWS cloud, similar to a traditional network in a data
center. When you create a VPC, you need to choose three main things.

1. The name of your VPC.

2. A Region for your VPC to live in. Each VPC spans multiple Availability Zones within the
Region you choose.
3. A IP range for your VPC in CIDR notation. This determines the size of your network. Each
Okay, well, one option to make this better is the idea of having high availability. What that means is if
VPC can have up to four /16 IP ranges.
this AZ goes down for whatever reason, what happens to our resources in that AZ? They go down too.
So, ideally we would have resources in another AZ to take on the traffic coming to our application. Using this information, AWS will provision a network and IP addresses for that network.
Create a Subnet After you create your VPC, you need to create subnets inside of this network. Think
of subnets as smaller networks inside your base network—or virtual area networks (VLANs) in a High Availability with A VPC When you create your subnets, keep high availability in mind. In order
traditional, on-premises network. In an on-premises network, the typical use case for subnets is to to maintain redundancy and fault tolerance, create at least two subnets configured in two different
isolate or optimize network traffic. In AWS, subnets are used for high availability and providing Availability Zones.
different connectivity options for your resources. When you create a subnet, you need to choose three
settings. As you learned earlier in the trail, it’s important to consider that “everything fails all the time.” In this
case, if one of these AZs fail, you still have your resources in another AZ available as backup.
1. The VPC you want your subnet to live in, in this case VPC (10.0.0.0/16).
2. The Availability Zone you want your subnet to live in, in this case AZ1.
3. A CIDR block for your subnet, which must be a subset of the VPC CIDR block, in this case
10.0.0.0/24.

When you launch an EC2 instance, you launch it inside a subnet, which will be located inside the
Availability Zone you choose.
 Since AWS reserves these five IP addresses, it can impact how you design your network. A common
starting place for those who are new to the cloud is to create a VPC with a IP range of /16 and create
subnets with a IP range of /24. This provides a large amount of IP addresses to work with at both the
VPC and subnet level.

Gateways

Internet Gateway

To enable internet connectivity for your VPC, you need to create an internet gateway. Think of this
gateway as similar to a modem. Just as a modem connects your computer to the internet, the internet
gateway connects your VPC to the internet. Unlike your modem at home, which sometimes goes down
or offline, an internet gateway is highly available and scalable. After you create an internet gateway,
you then need to attach it to your VPC.

Virtual Private Gateway

Reserved IPs For AWS to configure your VPC appropriately, AWS reserves five IP addresses in each A virtual private gateway allows you to connect your AWS VPC to another private network. Once you
subnet. These IP addresses are used for routing, Domain Name System (DNS), and network create and attach a VGW to a VPC, the gateway acts as anchor on the AWS side of the connection. On
management. the other side of the connection, you’ll need to connect a customer gateway to the other private
network. A customer gateway device is a physical device or software application on your side of the
For example, consider a VPC with the IP range 10.0.0.0/22. The VPC includes 1,024 total IP addresses.
connection. Once you have both gateways, you can then establish an encrypted VPN connection
This is divided into four equal-sized subnets, each with a /24 IP range with 256 IP addresses. Out of
between the two sides.
each of those IP ranges, there are only 251 IP addresses that can be used because AWS reserves five.
Video: Amazon VPC Routing

So now we have two additional subnets, one public, one private, and a different AZ for a total of four
subnets. She also created an EC2 instance hosting our employee directory inside of the public subnet in
AZ A. But we are missing one large component here. Say we have a user and that user wants to access When you create a brand new VPC, AWS creates a route table called the main route table and applies it
our employee directory. to the entire VPC. AWS assumes that when you create a new VPC with subnets you want traffic to
flow between those subnets. The default configuration of the main route table is to allow traffic
between all subnets local to the VPC.

Let's get a quick look at the main route table of the VPC we just built out in the last video for that.

In the VPC console, we'll click on route tables on the side panel and it will bring up
all of the route tables that exist in this region. If we scroll to the side, we can see the
main column and the VPC column. We're going to look for the main route table for
the app VPC, which is this one. When we click on it, we can bring up the bottom
panel and then click on routes. Here we can see the local route that has the
destination of the VPC range. This means that all of the components inside of our
VPC can communicate with one another locally by default. This local route will be
Eventually that internet traffic would flow through the internet gateway, but then where would it go? present in every route table that you create. Alright, that's the main route table. -
Just because the traffic entered through the door of the VPC it doesn't mean it ever made it to the right
room. What we need to do is to provide a path for the internet traffic, not only to enter the door but also
make sure traffic reaches the right room or in other words, enter the internet gateway and then find the
right subnet. The way we provide that path is through route tables.

A route table contains a set of rules called routes that are used to determine where the network traffic
is directed. These route tables can be applied at either the subnet level or at the VPC level.
 Do you mind creating a custom route table and associating it to our public subnet? - Sure can.

While the main route table controls the routing for your entire VPC, you may want to be more granular
about how you route your traffic to specific subnets. Remember when we mentioned that subnets can
be used to group your resources together based on whether they are publicly or privately accessible?
Well, the subnet itself doesn't provide that access. Whether a subnet has access to the public internet or
not, depends on its associated route table.

Let's get started. Let's create the route table for our public subnet. Back in the VPC console once again,
we'll click on Route tables on the side panel and then Create route table. We'll give it a name such as
app-routetable-public, choose the app-vpc, and then click Create. Our route table has been created, but
we're not done yet. We still have to edit the routes to allow traffic to flow from the internet gateway. To
do this, we'll click on the ID of the route table and then go to the route section in the bottom of the
summary table. We'll click Edit routes, Add route, put 0.0.0.0/0 for the destination, meaning it can take
and deliver traffic from anywhere and then specify the target as an internet gateway. This will bring up
available internet gateways to attach it to and from here, we'll select the app IGW. We're done with the
routes, so we'll click Save.

But how do you know which subnet this route table applies to? Well, we have to configure that. If we
only want this route table to apply to our public subnets, we'll need to associate it with our two public
If a route from the internet gateway to the subnet exists, it has public access. If the route table doesn't subnets only. To do this, we'll click on the Subnet associations tab. Select Edit subnet associations and
have a route between the subnet and the internet gateway then it doesn't have public access. So, we call choose the public subnets we created earlier. Then click Save.
subnets public or private but it's really the route table that provides that access.
Alright, we've hooked up our public subnets to a route table that allows internet traffic from the IGW
to our employee directory application. If we wanted to create a route table for the private subnets, we
would follow the same steps. Create the route table, make sure there's no route to the internet gateway
this time and then associated to the private subnets. Okay, now we've configured a route to the internet We'll start with network ACLs. You can think of a network ACL as a firewall at the subnet level. With
gateway and we'll configure additional firewall rules later on. - Nice. network ACLs, you can control what kind of traffic is allowed to enter and leave your subnet. So if I
were to draw network ACLs in our diagram, they are placed around each of our subnets by default.

The default network ACL allows traffic in and out of your subnet. Using this default configuration is a
good starting place but if needed, you can change the configurations of your network ACLs to further
lock down your subnets.

So, this is our final state of the diagram. One VPC, four subnets, one public, one private in both
availability zones. Our two public subnets are associated with a route table that allows traffic from the
internet gateway. Our private subnets don't have a route table associated to them yet, so they follow the
rules of the main route table, local traffic only. For example, if I only wanted to allow HTTPS traffic into my subnet, I can do that. I can create a rule
in my ACL that allows HTTPS traffic from anywhere on port 443 and denies everything else.
Video: Secure Your Network with Amazon VPC Security

Now we have a complete network that enables internet traffic flow to our public subnet. But how do
we know it's secure? Well, at the base level, we know that any new VPC is isolated from internet
traffic, so that prevents risk. But when you start allowing internet traffic by opening up routes, you
need other options to keep your network secure. In AWS, you have two options to secure your VPC
resources, network access control lists, often referred to as network ACLs, and security groups.

However, just because I allow inbound HTTPS traffic does not mean my job is done. If I don't open up
the corresponding outbound port used for the protocol, then that means the traffic is allowed in but the
web server's response will never leave the subnet. This is because network ACLs are considered to be
stateless. I need to include both the inbound and the outbound ports used for the protocols. What that Security groups are considered to be stateful resources. They will remember if a connection is
means for our HTTPS example is that I need to allow outbound HTTPS traffic from the subnet to the originally initiated by the EC2 instance or from outside and temporarily allow traffic to respond
internet by creating an outbound rule. without having to modify the inbound rules. All right, is that everything? - That's it for this one, thanks.
Remember that security groups and network ACLs are powerful tools to filter network-wide traffic for
The other security feature is security groups. These are firewalls that exist at the EC2 instance level. a single instance or subnets traffic.
Security groups are not optional, so anytime you create an EC2 instance, you need to place that EC2
instance inside of a security group that allows the appropriate kinds of traffic to flow to your With security groups, everything is blocked by default so you can only use allow rules, whereas with
application. network ACLs, everything is allowed by default, but you can use both allow and deny rules. These
configurations are largely up to you.

For example, if you want ultimate convenience, you can leave the network ACLs in the default
configuration and use mainly security groups to restrict access. We'll be doing just that with our
employee directory application, and it will still be secure. But if you want an added layer of security,
you can configure your network ACLs further for more fine-grained control.

Video: Hybrid Connectivity with AWS

The solution we are building out with the employee directory application is an all in cloud meaning
that the components needed to run and operate the application will be in AWS.

However, many solutions require a hybrid model to be followed, where some components are in AWS
In our diagram, if I were to draw security groups, I would draw them around every EC2 instance in the and others are hosted in an on-premises data center. Let's take some time to talk about connectivity to
VPC. In our case, we simply just have one EC2 instance. So we would need a security group around AWS for hybrid deployments. For hosting your resources in AWS you would use a VPC which you
that instance. learned about in the previous lessons. For hosting resources on-premises you'd have your own
solutions in an on-premises data center. So for connectivity to AWS how will you connect the remote
data center to AWS? Let's talk about a few choices you can pick from where you're looking at
connectivity between AWS and a remote site like a data center.

First, let's talk about AWS Virtual Private Network or AWS VPN. VPNs are a really common and
popular way to securely connect your remote network to AWS.

Do you have some time to build out a security group for our employee directory application? I think AWS VPN consists of two different services, AWS Site-Site VPN and AWS Client VPN.
we need an example. - Sure, let's get started. The default configuration of a security group blocks all
AWS Site-Site VPN is used to connect a remote network like a data center to a VPC. This would allow
inbound traffic and allows all outbound traffic. If you want your EC2 instance to accept traffic from the
resources living in a customer managed data center to connect to AWS. Then there is AWS Client
internet, you'll need to open up inbound ports. If you have a web server, you may need to accept HTTP
VPN which is more for connecting your administrators to AWS or to your data center. So, this is more
and HTTPS requests to allow that type of traffic through your security group. You can create an
like when you need to log into a VPN on your laptop to access corporate resources. Using AWS VPN
inbound rule that will allow port 80, HTTP, and port 443, HTTPS, as shown here. The beauty of
you can connect to your VPC through the virtual private gateway which we covered in an earlier video.
security groups is that I don't have to open up an outbound port for traffic to be able to leave the
instance.
Just like the internet gateway is the doorway to the internet. The virtual private gateway is the doorway  The destination, which is a range of IP addresses where you want your traffic to go. In the
to your private data center through a VPN connection or through another service called AWS Direct example of sending a letter, you need a destination to route the letter to the appropriate place.
Connect. AWS Direct Connect is a service that provides a hosted private connection to AWS through a The same is true for routing traffic. In this case, the destination is the IP range of our VPC
Direct Connect delivery partner, or through AWS. network.
 The target, which is the connection through which to send the traffic. In this case, the traffic is
routed through the local VPC network.

Custom Route Tables

While the main route table controls the routing for your VPC, you may want to be more granular about
how you route your traffic for specific subnets. For example, your application may consist of a
frontend and a database. You can create separate subnets for these resources and provide different
routes for each of them.

If you associate a custom route table with a subnet, the subnet will use it instead of the main route
Direct Connect provides a private dedicated connection. This isn't using the regular open internet to table. By default, each custom route table you create will have the local route already inside it,
send data between point A and point B. While the data sent over Direct Connect is in transit the allowing communication to flow between all resources and subnets inside the VPC.
network traffic remains on the AWS global network and never touches the public internet. This reduces
the chance of hitting bottlenecks or unexpected increases in latency when compared to a VPN
connection. AWS Direct Connect supports a larger and more reliable throughput.

If you plan to send a high volume of traffic to AWS and you do need reliability in throughput for this
connection AWS Direct Connect would be a good choice. It really depends on your use case which one
you would use or in some cases you may use both. Where VPN is a failover for Direct Connect. Now,
let's get back to building the employee directory application.

Reading 2.7: Amazon VPC Routing and Security

The Main Route Table

When you create a VPC, AWS creates a route table called the main route table. A route table contains a
set of rules, called routes, that are used to determine where network traffic is directed. AWS assumes
that when you create a new VPC with subnets, you want traffic to flow between them. Therefore, the Secure Your Subnets with Network ACLs
default configuration of the main route table is to allow traffic between all subnets in the local network.
Below is an example of a main route table: Think of a network ACL as a firewall at the subnet level. A network ACL enables you to control what
kind of traffic is allowed to enter or leave your subnet. You can configure this by setting up rules that
define what you want to filter. Here’s an example.

There are two main parts to this route table.

 Since network ACLs are configured by default to allow incoming and outgoing traffic, you don’t need
to change their initial settings unless you need additional security layers.

Secure Your EC2 Instances with Security Groups

The next layer of security is for your EC2 Instances. Here, you can create a firewall called a security
group. The default configuration of a security group blocks all inbound traffic and allows all outbound
traffic.

The default network ACL, shown in the table above, allows all traffic in and out of your subnet. To
allow data to flow freely to your subnet, this is a good starting place. However, you may want to
restrict data at the subnet level. For example, if you have a web application, you might restrict your
network to allow HTTPS traffic and remote desktop protocol (RDP) traffic to your web servers.

You might be wondering: “Wouldn’t this block all EC2 instances from receiving the response of any
customer requests?” Well, security groups are stateful, meaning they will remember if a connection is
originally initiated by the EC2 instance or from the outside and temporarily allow traffic to respond
without having to modify the inbound rules.

If you want your EC2 instance to accept traffic from the internet, you’ll need to open up inbound ports.
If you have a web server, you may need to accept HTTP and HTTPS requests to allow that type of
traffic in through your security group. You can create an inbound rule that will allow port 80 (HTTP)
and port 443 (HTTPS) as shown below.

Notice that in the network ACL example above, you allow inbound 443 and outbound range 1025-
65535. That’s because HTTP uses port 443 to initiate a connection and will respond to an ephemeral
port. Network ACL’s are considered stateless, so you need to include both the inbound and outbound
ports used for the protocol. If you don’t include the outbound range, your server would respond but the You learned in a previous unit that subnets can be used to segregate traffic between computers in your
traffic would never leave the subnet. network. Security groups can be used to do the same thing. A common design pattern is organizing
your resources into different groups and creating security groups for each to control network Ensure that an Internet Gateway (IGW) is attached to your VPC. Without the internet gateway, no
communication between them. traffic will be allowed in or out of the VPC.

2. Route tables

Check the route table associated with the subnet of your EC2 instance. Ensure there is a route with a
destination of 0.0.0.0/0 that points to the Internet Gateway. This route allows outbound traffic to the
internet.

3. Security groups

Review the security group settings attached to your EC2 instance. Make sure there are inbound rules
allowing HTTP (port 80) and/or HTTPS (port 443) traffic from the internet (0.0.0.0/0). Also, verify
that outbound rules allow traffic to leave the instance.

4. Network Access Control Lists

Check the NACLs associated with your subnet. Ensure that they allow inbound and outbound traffic
for HTTP and HTTPS. Unlike security groups, NACLs are stateless, so you must explicitly allow both
inbound and outbound rules.

5. Public IP address

This example allows you to define three tiers and isolate each tier with the security group rules you Ensure your EC2 instance has a public IP address assigned. You can check this in the EC2 console
define. In this case, you only allow internet traffic to the web tier over HTTPS, Web Tier to Application under the instance details. If it does not have a public IP, relaunch the instance and ensure you have the
Tier over HTTP, and Application tier to Database tier over MySQL. This is different from traditional configuration for assigning a public IP address set correctly.
on-premises environments, in which you isolate groups of resources via VLAN configuration. In AWS, 6. HTTP vs HTTPS
security groups allow you to achieve the same isolation without tying it to your network.
Confirm that your application is accessible via the correct protocol. If your application is configured
Reading: Common network troubleshooting steps for Amazon VPC for HTTPS, ensure SSL/TLS certificates are correctly installed and configured. Also, check if the web
In the demos throughout this course, you will see the Employee Directory Application being launched browser is trying to connect via the wrong protocol (HTTP instead of HTTPS or vice versa). For this
onto a Amazon EC2 instance that is placed in a public subnet in an Amazon VPC. There are multiple course, the application is operating via HTTP, double check that your browser is not trying to connect
networking configurations that play into whether an instance is accessible to the internet or not. via HTTPS. You can do this by selecting the address bar in the browser and making sure the address
starts with http and not https.
Below we will run down a list of configurations you should check if you ever have a public EC2
instance with a web application that is not loading as expected. 7. User data script

1. Internet gateway If your instance uses a user data script to configure the application on launch, verify that the script has
run successfully. Check the instance logs (/var/log/cloud-init.log or /var/log/cloud-init-output.log) for
any errors that may have occurred during the execution of the user data script.
8. Permissions
MODULE 3:
Verify the permissions and roles attached to your EC2 instance. Ensure the instance has the necessary
IAM roles and policies to access any required AWS services, such as S3, DynamoDB, or RDS. STORAGE ON AWS
9. Personal network permissions

Ensure that your personal or corporate network does not have restrictions blocking access to the public
IP address of your EC2 instance. Some networks might have firewalls or proxy settings that could Video: Storage Types on AWS
block outbound traffic to certain IP ranges or ports.
The next thing we need to configure for our employee directory app is the storage. Our application
10. Application requires several types of storage for its data. For one, we need to store the operating system, software,
and system files of our app. We also need to store static assets, like photos for the employee headshots,
Ensure that your application code is correctly deployed and running. Check the application's logs to and then we have more structured data, such as the name, title, and location of each employee, as well.
diagnose any runtime errors. Also, make sure the web server (e.g., Apache, Nginx) is installed and All of that needs a home.
running.
The structured data usually requires a database, which we'll talk about later this week, so for now we'll
focus on storing the application files as well as the static content.

There are two main types of storage that we can use to store this data, block and object. Here's the
difference. Let's say that I have a one gigabyte file with text in it. If I'm storing this in block storage,
what happens is that this file is split into fixed size chunks of data and then stored. Object storage, on
the other hand, treats each file like a single unit of data. This might seem like a small difference, but it
can change how you access and work with your data. Let's say I want to change one character out of
that one gigabyte file. If my file is stored in block storage, changing that one character is simple,
mainly because we can change the block, or the piece of the file that the character is in, and leave the
rest of the file alone.
In object storage, if I want to change that one character, I instead have to update the entire file.
Each file has metadata such as file name, file size, and the date the file was created. The file also has a
Object storage often follows a WORM (write once, read many) model. path, for example, computer/Application_files/Cat_photos/cats-03.png. When you need to retrieve a
file, your system can use the path to find it in the file hierarchy.
Let's take these two types of storage and access patterns and try to apply them to the data we want to
store. For example, our static data, like the employee photos, will most likely be accessed often, but File storage is ideal when you require centralized access to files that need to be easily shared and
modified rarely. Therefore, storing in object storage is fine. For more frequently updated data or data managed by multiple host computers. Typically, this storage is mounted onto multiple hosts and
that has high transaction rates, like our application or system files, block storage will perform better. In requires file locking and integration with existing file system communication protocols.Common use
this section of the course, we'll discuss both block and object AWS storage services and how they'll cases for file storage include:
interact with our employee directory application. Before we do that, take a look at the notes to get a
refresher of the different types of storage. That way you can easily match the storage type to the AWS  Large content repositories
storage service that we talk about.  Development environments
 User home directories

Block Storage
Reading 3.1: Storage Types on AWS
While file storage treats files as a singular unit, block storage splits files into fixed-size chunks of data
AWS storage services are grouped into three different categories: block storage, file storage, and object called blocks that have their own addresses. Since each block is addressable, blocks can be retrieved
storage. efficiently.
File Storage When data is requested, these addresses are used by the storage system to organize the blocks in the
correct order to form a complete file to present back to the requestor. Outside of the address, there is no
You may be familiar with file storage if you’ve interacted with file storage systems like Windows File
additional metadata associated with each block. So, when you want to change a character in a file, you
Explorer or Finder on MacOS. You place your files in a tree-like hierarchy that consists of folders and
just change the block, or the piece of the file, that contains the character. This ease of access is why
subfolders. For example, if you have hundreds of cat photos on your laptop, you may want to create a
block storage solutions are fast and use less bandwidth.
folder called Cat photos, and place those images inside that folder to organize them. Since you know
these images will be used in an application, you may want to place the cat photos folder inside another
folder called Application files.
 Relate Back to Traditional Storage Systems

If you’ve worked with storage on-premises, you may already be familiar with block, file, and object
storage. Consider the following technologies and how they relate to systems you may have seen before.

 Block storage in the cloud is analogous to direct-attached storage (DAS) or a storage area
network (SAN).
 File storage systems are often supported with a network attached storage (NAS) server.

Since block storage is optimized for low-latency operations, it is a typical storage choice for high- Adding more storage in a traditional data center environment is a more rigid process, as you need to
performance enterprise workloads, such as databases or enterprise resource planning (ERP) systems, purchase, install, and configure these storage solutions. With cloud computing, the process is more
that require low-latency storage. flexible. You can create, delete, and modify storage solutions all within a matter of minutes.

Object Storage

Objects, much like files, are also treated as a single unit of data when stored. However, unlike file Video: Amazon EC2 Instance Storage and Amazon Elastic Block Store
storage, these objects are stored in a flat structure instead of a hierarchy. Each object is a file with a
When you launch an EC2 instance you're going to need some kind of block storage to go with it. This
unique identifier. This identifier, along with any additional metadata, is bundled with the data and
block storage can be used as a boot volume for your operating system or a separate data volume.
stored.

Changing just one character in an object is more difficult than with block storage. When you want to
change one character in a file, the entire file must be updated.

For example, think about your laptop. With a laptop you store your data in drives, and those drives are
either built-in internally to your laptop or connected externally. EC2 instances have the same options as
far as block storage goes. The internal storage is called Instance Store and the external connected
storage is called Amazon Elastic Block Store or Amazon EBS. Let's talk about Instance Store first.
With object storage, you can store almost any type of data, and there is no limit to the number of Instance Store is a form of directly attached storage which means the underlying physical server has at
objects stored, making it easy to scale. Object storage is generally useful when storing large data sets, least one storage unit directly attached to it.
unstructured files like media assets, and static assets, such as photos.
This direct attachment is also the main advantage of using this form of storage. Because it's so close to
the physical server it can be very fast and respond very quickly, but while it can be very fast, there is
also one big downside. With Instance Store being directly attached to an EC2 instance, its lifecycle is Reading 3.2: Amazon EC2 Instance Storage and Amazon Elastic Block Store
tied to that of the instance.
Amazon EC2 Instance Store
EC2 Instance Store is often referred to as ephemeral storage.
Amazon EC2 Instance Store provides temporary block-level storage for your instance. This storage is
That means if you stop or terminate an instance all data in the Instance Store is gone. It can no longer located on disks that are physically attached to the host computer. This ties the lifecycle of your data to
be used or accessed. the lifecycle of your EC2 instance. If you delete your instance, the instance store is deleted as well.
Due to this, instance store is considered ephemeral storage. Read more about it in the AWS
Naturally there are many use cases where you want the ability to keep data, even if you shut an EC2 documentation.
instance down. This is where EBS volumes come in. These volumes, as the name implies, are drives of
a user configured size that are separate from an EC2 instance. The drives are simply network attached Instance store is ideal if you are hosting applications that replicate data to other EC2 instances, such as
storage for your instances. Hadoop clusters. For these cluster-based workloads, having the speed of locally attached volumes and
the resiliency of replicated data helps you achieve data distribution at high performance. It’s also ideal
You can think of it as similar to how you might attach an external drive to your laptop. You can attach for temporary storage of information that changes frequently, such as buffers, caches, scratch data, and
multiple EBS volumes to one EC2 instance, and then you can configure how to use that storage on the other temporary content.
OS of the EC2 instance. When I connect that EBS volume to my instance, my instance now has a
direct communication line to the data in that volume. Nobody else can directly talk to that volume so Amazon Elastic Block Storage (Amazon EBS) As the name implies, Amazon EBS is a block-level
that it maintains secure communication. You need an EC2 instance to access data on an EBS volume. If storage device that you can attach to an Amazon EC2 instance. These storage devices are called
I decided I want to use that EBS volume with a different instance, that's no problem. We can stop the Amazon EBS volumes. EBS volumes are essentially drives of a user-configured size attached to an
instance, detach the volume, and then attach it to another instance in the same AZ. Much like you can EC2 instance, similar to how you might attach an external drive to your laptop.
unplug your drive from a laptop, and plug it into another one.

Or depending on the instance type and EBS volume we're using we may be able to attach it to multiple
instances at the same time, which is called EBS Multi-Attach. And perhaps the most important EBS volumes act similarly to external drives in more than one way.
similarity is that an EBS volume is separate from your instance.  Most Amazon EBS volumes can only be connected with one computer at a time. Most EBS
Just like an external drive is separate from your laptop. That means if an accident happens, and the volumes have a one-to-one relationship with EC2 instances, so they cannot be shared by or
instance goes down you still have your data on your EBS volume. This is what we refer to as persistent attached to multiple instances at one time. Note: Recently, AWS announced the Amazon EBS
storage. You can stop or terminate your instance, and your EBS volume can still exist with your data on multi-attach feature that enables volumes to be attached to multiple EC2 instances at one time.
it. EBS is often the right storage type for workloads that require persistence of data. However, the This feature is not available for all instance types and all instances must be in the same
question typically comes down to which EBS volume type do I use? That's right. There are many Availability Zone. Read more about this scenario in the EBS documentation.
different types of volumes, but they've divided into two main volume types. SSD backed volumes and  You can detach an EBS volume from one EC2 instance and attach it to another EC2 instance
HDD backed volumes. In the readings, you'll learn more about these two options. The last thing we'll in the same Availability Zone, to access the data on it.
need to talk about here is backing up data. Things fail, errors happen, so you need to backup your data,  The external drive is separate from the computer. That means, if an accident happens and the
even in AWS. The way you backup EBS volumes is by taking what we call snapshots. EBS snapshots computer goes down, you still have your data on your external drive. The same is true for
are incremental backups that are stored redundantly. The idea here is that if something goes wrong you EBS volumes.
can create new volumes from your snapshots and restore your data to a safe state.  You’re limited to the size of the external drive, since it has a fixed limit to how scalable it can
be. For example, you may have a 2 TB external drive and that means you can only have 2 TB
of content on there. This relates to EBS as well, since volumes also have a max limitation of
how much content you can store on the volume.
Scale Amazon EBS Volumes

You can scale Amazon EBS volumes in two ways.

1. Increase the volume size, as long as it doesn’t increase above the maximum size limit. For
EBS volumes, the maximum amount of storage you can have is 16 TB. That means if you
provision a 5 TB EBS volume, you can choose to increase the size of your volume until you
get to 16 TB.
2. Attach multiple volumes to a single Amazon EC2 instance. EC2 has a one-to-many
relationship with EBS volumes. You can add these additional volumes during or after EC2
instance creation to provide more storage capacity for your hosts.

Amazon EBS Use Cases

Amazon EBS is useful when you need to retrieve data quickly and have data persist long-term.
Volumes are commonly used in the following scenarios. There are two main categories of Amazon EBS volumes: solid-state drives (SSDs) and hard-disk drives
(HDDs). SSDs provide strong performance for random input/output (I/O), while HDDs provide strong
 Operating systems: Boot/root volumes to store an operating system. The root device for an performance for sequential I/O. AWS offers two types of each. The following chart can help you
instance launched from an Amazon Machine Image (AMI) is typically an Amazon EBS decide which EBS volume is the right option for your workload.
volume. These are commonly referred to as EBS-backed AMIs.
 Databases: A storage layer for databases running on Amazon EC2 that rely on transactional Benefits of Using Amazon EBS
reads and writes.
 Enterprise applications: Amazon EBS provides reliable block storage to run business-critical Here are the following benefits of using Amazon EBS (in case you need a quick cheat sheet).
applications.  High availability: When you create an EBS volume, it is automatically replicated within its
 Throughput-intensive applications: Applications that perform long, continuous reads and Availability Zone to prevent data loss from single points of failure.
writes.  Data persistence: The storage persists even when your instance doesn’t.
 Data encryption: All EBS volumes support encryption.
 Flexibility: EBS volumes support on-the-fly changes. You can modify volume type, volume
size, and input/output operations per second (IOPS) capacity without stopping your instance.
 Backups: Amazon EBS provides you the ability to create backups of any EBS volume.

EBS Snapshots

Errors happen. One of those errors is not backing up data, and then, inevitably losing that data. To
prevent this from happening to you, you should back up your data—even in AWS. Since your EBS
volumes consist of the data from your Amazon EC2 instance, you’ll want to take backups of these
volumes, called snapshots.

EBS snapshots are incremental backups that only save the blocks on the volume that have changed
after your most recent snapshot. For example, if you have 10 GB of data on a volume, and only 2 GB
of data have been modified since your last snapshot, only the 2 GB that have been changed are written This is what makes S3 designed for 99.99% availability and gives it 11 nines of durability.
to Amazon Simple Storage Service (Amazon S3).
Alright, let's learn about some S3 concepts. The first concept is a bucket. In S3, you store your objects
When you take a snapshot of any of your EBS volumes, these backups are stored redundantly in in a bucket. You can't upload any object, not even a single photo to S3 without creating a bucket first.
multiple Availability Zones using Amazon S3. This aspect of storing the backup in Amazon S3 will be You then place your objects inside of these buckets. And if you want to organize and arrange those
handled by AWS, so you won’t need to interact with Amazon S3 to work with your EBS snapshots. objects, you can also have folders inside of the buckets. Let's create a bucket in the console.
You simply manage them in the EBS console (which is part of the EC2 console).
S3 bucket policies are similar to IAM policies in that they're both defined using the same policy
EBS snapshots can be used to create multiple new volumes, whether they’re in the same Availability language in a JSON format. The difference is IAM policies are attached to users, groups and roles,
Zone or a different one. When you create a new volume from a snapshot, it’s an exact copy of the whereas S3 bucket policies are only attached to buckets. S3 bucket policies specify what actions you're
original volume at the time the snapshot was taken. allowed or denied on the bucket.

For example, you might want to attach an S3 bucket policy to it that allows another AWS account to
put objects in that bucket. Or you might want to create a bucket policy that allows read-only
permissions to anonymous viewers. S3 bucket policies can be placed on buckets and cannot be used for
Video: Object Storage with Amazon S3 folders, or objects.

A natural question is, why can't we just store these photos in Amazon EBS? S3 uses containers called buckets to store your objects and you have several options to control access
to those objects through the use of IAM policies and bucket policies.
Well, there's a few reasons. Number one, most EBS volumes are only connected to one EC2 instance at
a time. Multi-attach is not supported by all volume and instance types. Eventually, as my app scales, I'll
need to figure out how to access those photos from all of my instances, that's an issue. Reading 3.3: Object Storage with Amazon S3
The second consideration is that an EBS volume has size limitations. That means that eventually, there WHAT IS AMAZON S3?
will be a limit to how many HD 4K photos I store of my employees in one drive. Ideally, I'd store these
photos in a more scalable solution. So EBS probably isn't the right choice. Unlike Amazon EBS, Amazon S3 is a standalone storage solution that isn’t tied to compute. It enables
you to retrieve your data from anywhere on the web. If you’ve ever used an online storage service to
Fortunately, AWS has a service called Amazon Simple Storage Service or Amazon S3 that was back up the data from your local machine, then you most likely have used a service similar to Amazon
designed to be a standalone storage solution that isn't tied to compute, meaning you don't mount this S3. The big difference between those online storage services and Amazon S3 is the storage type.
type of storage onto your EC2 instances. Instead, you can access your data through URLs from
anywhere on the web, which gives this service its nickname, storage for the internet. S3 also allows Amazon S3 is an object storage service. Object storage stores data in a flat structure, using unique
you to store as many objects as you'd like with an individual object size limit of five terabytes. This identifiers to look up objects when requested. An object is simply a file combined with metadata and
makes it ideal for our employee photos. that you can store as many of these objects as you’d like. All of these characteristics of object storage
are also characteristics of Amazon S3.
Now, let's talk about how we store things in S3. The underlying storage type for S3 is object storage.
That means that all of the same characteristics of object storage are also characteristics of S3. So S3
uses a flat structure. It uses unique identifiers to look up objects when requested, you get the idea. S3 is
also considered distributed storage, meaning that we store your data across multiple different facilities UNDERSTAND AMAZON S3 CONCEPTS
within one AWS region.
In Amazon S3, you have to store your objects in containers called buckets. You can’t upload any  Backup and storage: S3 is a natural place to back up files because it is highly redundant. As
object, not even a single photo, to S3 without creating a bucket first. When you create a bucket, you mentioned in the last unit, AWS stores your EBS snapshots in S3 to take advantage of its high
choose, at the very minimum, two things: the bucket name and the AWS Region you want the bucket to availability.
reside in.  Media hosting: Because you can store unlimited objects, and each individual object can be up
to 5 TBs, S3 is an ideal location to host video, photo, or music uploads.
The first part is choosing the Region you want the bucket to reside in. Typically, this will be a Region  Software delivery: You can use S3 to host your software applications that customers can
that you’ve used for other resources, such as your compute. When you choose a Region for your download.
bucket, all objects you put inside that bucket are redundantly stored across multiple devices, across  Data lakes: S3 is an optimal foundation for a data lake because of its virtually unlimited
multiple Availability Zones. This level of redundancy is designed to provide Amazon S3 customers scalability. You can increase storage from gigabytes to petabytes of content, paying only for
with 99.999999999% durability and 99.99% availability for objects over a given year. what you use.
The second part is choosing a bucket name which must be unique across all AWS accounts. AWS  Static websites: You can configure your bucket to host a static website of HTML, CSS, and
stops you from choosing a bucket name that has already been chosen by someone else in another AWS client-side scripts.
account. Once you choose a name, that name is yours and cannot be claimed by anyone else unless you  Static content: Because of the limitless scaling, the support for large files, and the fact that
delete that bucket, which then releases the name for others to use. you access any object over the web at any time, S3 is the perfect place to store static content.

AWS uses this name as part of the object identifier. In S3, each object is identified using a URL, which CHOOSE THE RIGHT CONNECTIVITY OPTION FOR YOUR RESOURCES
looks like this: Everything in Amazon S3 is private by default. This means that all S3 resources, such as buckets,
folders, and objects can only be viewed by the user or AWS account that created that resource. Amazon
S3 resources are all private and protected to begin with.

If you decide that you want everyone on the internet to see your photos, you can choose to make your
buckets, folders, and objects public. Keep in mind that a public resource means that everyone on the
internet can see it. Most of the time, you don’t want your permissions to be all or nothing. Typically,
After the http://, you see the bucket name. In this example, the bucket is named doc. Then, the you want to be more granular about the way you provide access to your resources.
identifier uses the service name, s3 and specifies the service provider amazonaws. After that, you have
To be more specific about who can do what with your S3 resources, Amazon S3 provides two main
an implied folder inside the bucket called 2006-03-01 and the object inside the folder that is named
access management features: IAM policies and S3 bucket policies.
AmazonS3.html. The object name is often referred to as the key name.
UNDERSTAND IAM POLICIES
Note, you can have folders inside of buckets to help you organize objects. However, remember that
there’s no actual file hierarchy that supports this on the back end. It is instead a flat structure where all Previously, you learned about creating and using IAM policies, and now you get to apply this to
files and folders live at the same level. Using buckets and folders implies a hierarchy, which makes it Amazon S3. When IAM policies are attached to IAM users, groups, and roles, the policies define
easy to understand for the human eye. which actions they can perform. IAM policies are not tied to any one AWS service and can be used to
define access to nearly any AWS action. You should use IAM policies for private buckets when:
S3 USE CASES
 You have many buckets with different permission requirements. Instead of defining many
Amazon S3 is one of the most widely used storage services, with far more use cases than could fit on
different S3 bucket policies, you can use IAM policies instead.
one screen. The following list summarizes some of the most common ways you can use Amazon S3.
  You want all policies to be in a centralized location. Using IAM policies allows you to S3 Bucket policies can only be placed on buckets, and cannot be used for folders or objects. However,
manage all policy information in one location. the policy that is placed on the bucket applies to every object in that bucket. You should use S3 bucket
policies when:
UNDERSTAND S3 BUCKET POLICIES
 You need a simple way to do cross-account access to S3, without using IAM roles.
S3 bucket policies are similar to IAM policies, in that they are both defined using the same policy  Your IAM policies bump up against the defined size limit. S3 bucket policies have a larger
language in a JSON format. The difference is IAM policies are attached to users, groups, and roles, size limit.
whereas S3 bucket policies are only attached to buckets. S3 bucket policies specify what actions are
allowed or denied on the bucket. ENCRYPT S3

For example, if you have a bucket called employeebucket, you can attach an S3 bucket policy to it that Amazon S3 reinforces encryption in transit (as it travels to and from Amazon S3) and at rest. To
allows another AWS account to put objects in that bucket. protect data at rest, you can use:

Or if you wanted to allow anonymous viewers to read the objects in employeebucket, then you can  Server-side encryption: This allows Amazon S3 to encrypt your object before saving it on
apply a policy to that bucket that allows anyone to read objects in the bucket using "Effect":Allow on disks in its data centers and then decrypt it when you download the objects.
the "Action:["s3:GetObject"]".  Client-side encryption: Encrypt your data client-side and upload the encrypted data to
Amazon S3. In this case, you manage the encryption process, the encryption keys, and all
related tools.
Here’s an example of what that S3 bucket policy might look like. To encrypt in transit, you can use client-side encryption or Secure Sockets Layer (SSL).

USE VERSIONING TO PRESERVE OBJECTS

As you know, Amazon S3 identifies objects in part by using the object name. For example, when you
upload an employee photo to S3, you may name the object employee.jpg and store it in a folder called
employees. If you don’t use Amazon S3 versioning, anytime you upload an object called employee.jpg
to the employees folder, it overwrites the original file. This can be an issue for several reasons.

 employee.jpg is a common name for an employee photo object. You or someone else who has
access to that bucket might not have intended to overwrite it, and now that you have, you no
longer have access to the original file.
 You may want to preserve different versions of employee.jpg. Without versioning, if you
wanted to create a new version of employee.jpg, you would need to upload the object and
choose a different name for it. Having several objects all with slight differences in naming
variations may cause confusion and clutter in your bucket.

So, what do you do? You use S3 versioning! Versioning enables you to keep multiple versions of a
single object in the same bucket. This allows you to preserve old versions of an object without having
to use different naming constructs, in case you need to recover from accidental deletions, accidental 2. Amazon S3 Intelligent-Tiering: This tier is useful if your data has unknown or changing
overwrites, or even application failures. Let’s see how this works. access patterns. S3 Intelligent-Tiering stores objects in two tiers, a frequent access tier and an
infrequent access tier. Amazon S3 monitors access patterns of your data, and automatically
If you enable versioning for a bucket, Amazon S3 automatically generates a unique version ID for the moves your data to the most cost-effective storage tier based on frequency of access.
object being stored. In one bucket, for example, you can have two objects with the same key, but 3. Amazon S3 Standard-Infrequent Access (S3 Standard-IA): S3 Standard-IA is for data that
different version IDs, such as employeephoto.gif (version 111111) and employeephoto.gif (version is accessed less frequently, but requires rapid access when needed. S3 Standard-IA offers the
121212). Versioning-enabled buckets let you recover objects from accidental deletion or overwrite. high durability, high throughput, and low latency of S3 Standard, with a low per-GB storage
 Deleting an object does not remove the object permanently. Instead, Amazon S3 puts a marker price and per-GB retrieval fee. This storage tier is ideal if you want to store long-term
on the object that shows you tried to delete it. If you want to restore the object, you can backups, disaster recovery files, and so on.
remove this marker and it reinstates the object. 4. Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA): Unlike other S3 storage
 If you overwrite an object, it results in a new object version in the bucket. You still have classes which store data in a minimum of three Availability Zones (AZs), S3 One Zone-IA
access to previous versions of the object. stores data in a single AZ and costs 20% less than S3 Standard-IA. S3 One Zone-IA is ideal
for customers who want a lower-cost option for infrequently accessed data but do not require
UNDERSTAND VERSIONING STATES the availability and resilience of S3 Standard or S3 Standard-IA. It’s a good choice for storing
secondary backup copies of on-premises data or easily re-creatable data.
Buckets can be in one of three states.
5. Amazon S3 Glacier Instant Retrieval: Amazon S3 Glacier Instant Retrieval is an archive
 Unversioned (the default): No new or existing objects in the bucket have a version. storage class that delivers the lowest-cost storage for long-lived data that is rarely accessed
 Versioning-enabled: This enables versioning for all objects in the bucket. and requires retrieval in milliseconds.
 Versioning-suspended: This suspends versioning for new objects. All new objects in the 6. Amazon S3 Glacier Flexible Retrieval:S3 Glacier Flexible Retrieval delivers low-cost
bucket will not have a version. However, all existing objects keep their object versions. storage, up to 10% lower cost (than S3 Glacier Instant Retrieval), for archive data that is
accessed 1—2 times per year and is retrieved asynchronously.
The versioning state applies to all of the objects in that bucket. Keep in mind that storage costs are 7. Amazon S3 Glacier Deep Archive: S3 Glacier Deep Archive is Amazon S3’s lowest-cost
incurred for all objects in your bucket and all versions of those objects. To reduce your S3 bill, you storage class and supports long-term retention and digital preservation for data that may be
may want to delete previous versions of your objects that are no longer in use. accessed once or twice in a year. It is designed for customers—particularly those in highly
regulated industries, such as the Financial Services, Healthcare, and Public Sectors—that
WHAT ARE AMAZON S3 STORAGE CLASSES?
retain data sets for 7 to 10 years or longer to meet regulatory compliance requirements.
When you upload an object to Amazon S3 and you don’t specify the storage class, you’re uploading it 8. Amazon S3 Outposts: Amazon S3 on Outposts delivers object storage to your on-premises
to the default storage class—often referred to as standard storage. When you learned about Amazon S3 AWS Outposts environment.
in previous units, you were learning about the standard storage class without even knowing it! S3
storage classes let you change your storage tier as your data characteristics change. For example, if you
are now accessing your old photos infrequently, you may want to change the storage class those photos AUTOMATE TIER TRANSITIONS WITH OBJECT LIFECYCLE MANAGEMENT
are stored in to save on costs. There are six S3 storage classes.
If you keep manually changing your objects, such as your employee photos, from storage tier to
1. Amazon S3 Standard: This is considered general purpose storage for cloud applications, storage tier, you may want to look into automating this process using a lifecycle policy. When you
dynamic websites, content distribution, mobile and gaming applications, and big data define a lifecycle policy configuration for an object or group of objects, you can choose to automate
analytics. two actions: transition and expiration actions.
  Transition actions are used to define when you should transition your objects to another customer information. The database will frequently be accessed and updated so the storage layer
storage class. needs to respond quickly. It's important that the storage is fast and durable.
 Expiration actions define when objects expire and should be permanently deleted.
Which AWS storage service should you use?
For example, you might choose to transition objects to S3 Standard-IA storage class 30 days after you
created them, or archive objects to the S3 Glacier storage class one year after creating them.  And the answer is Amazon EBS.
 Add 30 points to your score if you got it.
The following use cases are good candidates for lifecycle management.  - It seems like we're looking for storage attached to the compute, so why not EC2 instance
store? - Right, that's also an option but it's not ideal. Since it's an e-commerce company, their
 Periodic logs: If you upload periodic logs to a bucket, your application might need them for a order and customer data is what drives the business which means the persistence and
week or a month. After that, you might want to delete them. durability of that data is really important. Using EC2 instance store would definitely give us
 Data that changes in access frequency: Some documents are frequently accessed for a the speed we're looking for but it wouldn't give us the durability needed to store this data long
limited period of time. After that, they are infrequently accessed. At some point, you might term. So EBS is the right option. - That makes sense. All right, moving on. Two more
not need real-time access to them, but your organization or regulations might require you to questions.
archive them for a specific period. After that, you can delete them.
The next one is you have a web application that needs to write to disk in order to perform certain
Video: Choose the Right Storage Service calculations. The application will store temporary data during the calculation. The most important
All right, this is the first question. aspects of this architecture are speed and cost.

Let's say you're a developer and you plan to build out an application to transcode large media files With five seconds on the clock, which storage solution would you choose?
like videos. You'll be using an AWS Lambda function to perform the transcoding, but you need a place  And the answer is EC2 instance store.
to store both the original media files and the transcoded media files. Due to regulations, you need to  - Seph, would you mind telling us how we chose instance store and not EBS? - Sure. Once
store these files for at least a year. again, we're looking for storage attached to compute in this case. The first thing I want to
Which of the storage services that we've talked about in this course should you use? point out is that this is temporary data we're talking about. We're not looking at a huge amount
of data and we also don't necessarily care about the durability of that data. If the instance fails
 And the answer is Amazon S3. mid calculation and you want to plan for failure, you can just restart the calculation from
 Why is S3 the best solution here, Morgan? - Well, first of all, the question says that they're scratch. So durability doesn't matter, but cost does. By not using EBS and instead using
using a Lambda function. Because of that, I'm already ruling EBS out as EBS volumes can instance store, you may save yourself some costs. That is because instance store is included in
only be attached to EC2 instances. Even if they were using EC2, video files are typically large the overall EC2 instance price. So instance store is the best option for this use case. - Okay, 30
in size, so you may have to use multiple EBS volumes to store that data which might not be more points on the board for those of you who got it. Now the final bonus question for an
cost effective in the long run. So EBS is out. Instance storage is out for the same reason. We're extra 10 points is next. This is a tricky one, and you might have to think outside of the storage
not using EC2 here but also because we want this data to persist for a year and instance options that we've talked about so far.
storage is considered ephemeral.
 - All right, S3 it is. Let's put some points on the board for those who got it right. Morgan, tell The question is, let's say you're creating a WordPress site on multiple instances. By default, WordPress
us the next question. stores user uploads on the local file system. Since you want to use multiple instances, you'll need to
move the WordPress installation and all of the user customizations into a shared storage platform.
The next question is, you're an architect for an e-commerce company that wants to run their
MySQL database on an EC2 instance. This database needs a storage layer to store their order and Which storage option would you use?
  And the answer is Amazon Elastic File System or Amazon EFS.  Performance depends on MB/s.
 This service was covered in an earlier reading so if you got points for this, great job. For those  Ideal for throughput-intensive workloads, such as big data, data warehouses, log processing,
of you who didn't, no worries but I would recommend that you go back and review the and sequential data I/O.
reading related to file storage on AWS.
 - Let's go ahead and talk about the options. Typically, when we talk about shared storage Here are a few important features of Amazon EBS that you need to know when comparing it to other
systems that multiple instances can access, we think Amazon S3. why wouldn't we use that in services.
this case? - Well, S3 isn't a file system. It's actually a flat structure for storing objects instead  It is block storage.
of a hierarchy. And you can't mount it onto multiple instances. Because S3 has a different  You pay for what you provision (you have to provision storage in advance).
underlying type of storage, it's not right for this use case.  EBS volumes are replicated across multiple servers in a single Availability Zone.
 So, by moving the entire WordPress installation directory onto an EFS file system and  Most EBS volumes can only be attached to a single EC2 instance at a time.
mounting it onto each of your EC2 instances when they boot, your WordPress site and all of
its data is automatically stored on a distributed file system that isn't dependent on any one Amazon S3
EC2 instance.
If your data doesn’t change that often, Amazon S3 might be a more cost-effective and scalable storage
Reading 3.4: Choose the Right Storage Service solution. S3 is ideal for storing static web content and media, backups and archiving, data for analytics,
and can even be used to host entire static websites with custom domain names. Here are a few
Here’s a recap of all the storage services mentioned so far. By the end of this reading, you should be important features of Amazon S3 to know about when comparing it to other services.
able to better answer the question “Which storage service should I use?” for some of the more common
scenarios.  It is object storage.
 You pay for what you use (you don’t have to provision storage in advance).
Amazon EC2 Instance Store  Amazon S3 replicates your objects across multiple Availability Zones in a Region.
Instance store is ephemeral block storage. This is preconfigured storage that exists on the same  Amazon S3 is not storage attached to compute.
physical server that hosts the EC2 instance and cannot be detached from Amazon EC2. You can think
of it as a built-in drive for your EC2 instance. Instance store is generally well-suited for temporary
storage of information that is constantly changing, such as buffers, caches, and scratch data. It is not
meant for data that is persistent or long-lasting. If you need persistent long-term block storage that can
Amazon Elastic File System (Amazon EFS) and Amazon FSx
be detached from Amazon EC2 and provide you more management flexibility, such as increasing
volume size or creating snapshots, then you should use Amazon EBS. In this module, you’ve already learned about Amazon S3 and Amazon EBS. You learned that S3 uses a
flat namespace and isn’t meant to serve as a standalone file system. You also learned most EBS
Amazon EBS
volumes can only be attached to one EC2 instance at a time. So, if you need file storage on AWS,
Amazon EBS is meant for data that changes frequently and needs to persist through instance stops, which service should you use?For file storage that can mount on to multiple EC2 instances, you can
terminations, or hardware failures. Amazon EBS has two different types of volumes: SSD-backed use Amazon Elastic File System (Amazon EFS) or Amazon FSx. Use the following table for more
volumes and HDD-backed volumes.SSD-backed volumes have the following characteristics. information about each of these services.

 Performance depends on IOPS (input/output operations per second).

 Ideal for transactional workloads such as databases and boot volumes.

HDD-backed volumes have the following characteristics:

 existing employees, as well as editing and deleting employees. All of this data will be stored in a
database, which we haven't created yet.

Here are a few important features of Amazon EFS and FSx to know about when comparing them to
other services.

 It is file storage.
 You pay for what you use (you don’t have to provision storage in advance).
According to the architecture diagram, we have chosen Amazon Relational Database, or Amazon RDS,
 Amazon EFS and Amazon FSx can be mounted onto multiple EC2 instances.
to store this data.

So, let's talk about databases for a moment. Relational databases are widely used across all industries
and it's likely your company has many databases supporting a variety of applications and solutions.
Relational database management systems, or RDBMS, let you create, manage, and use a relational
database. You can install and operate database applications on Amazon EC2 instances, and this is a
good option for migrating existing databases to AWS.

By running databases on EC2, you are already simplifying things from an operational perspective
when it comes to on-premises, and it's a common use case for EC2.

Databases on AWS

Video: Explore Databases on AWS

The employee directory application that we've been building out lets you keep track of employee data, When migrating a database from on-premises to EC2, you are no longer responsible for the physical
like their name, location, job title, and badges. The app supports adding new employees, viewing infrastructure or OS installation, but you are still responsible for the installation of the database engine,
setting up across multiple AZs with data replication in place, as well as taking on any database server UNDERSTANDING THE HISTORY BEHIND ENTERPRISE DATABASES
management tasks like installing security patches and updating database software when necessary. So
EC2 makes it easier, but there is a way to lift even more of the operational burden of running relational Choosing a database used to be a straightforward decision. There were only a few options to choose
databases on AWS. from. In the past, you likely considered a few vendors and then inevitably chose one for all of your
applications.
What if, instead of managing a database on EC2, you could use one of the managed AWS database
offerings like Amazon RDS? Businesses often selected the database technology they were going to use, even before they fully
understood their use case. Since the 1970s, the database most commonly selected by businesses was a
The big difference between these two options is instead of taking care of the instances, the patching, relational database.
the upgrades, and the install of the database,
WHAT IS A RELATIONAL DATABASE?

A relational database organizes data into tables. Data in one table can be linked to data in other tables
to create relationships—hence, the relational part of the name.

A table stores data in rows and columns. A row, often called a record, contains all information about a
specific entry. Columns describe attributes of that entry. Here’s an example of three tables in a
relational database.

AWS takes care of all of that undifferentiated heavy lifting for you. The task that you are then
responsible for is the creation, maintenance, and optimization of the database itself.
This shows a table for books, a table for sales, and a table for authors. In the books table, each row
includes the book ISBN, the title, the author, and the format. Each of these attributes is stored in its
So you are still in charge of creating the right schema, indexing the data, creating stored procedures, own column. The books table has something in common with the other two tables: the author attribute.
enabling encryption, managing access control, and more. But all the rest of the undifferentiated heavy That common column creates a relationship between the tables.
lifting that goes into operating a relational database AWS takes care of. To start off this section of
The tables, rows, columns, and relationships between them is referred to as a logical schema. With
lessons on databases, we will first cover RDS. The upcoming reading after the video will dive into the
relational databases, a schema is fixed. Once the database is operational, it becomes difficult to change
history of enterprise relational databases and explain what relational databases are and how they were
the schema. This requires most of the data modeling to be done upfront before the database is active.
used.
WHAT IS A RELATIONAL DATABASE MANAGEMENT SYSTEM?
Reading 3.5: Explore Databases on AWS
A relational database management system (RDBMS) lets you create, update, and administer a
Explore Databases on AWS
relational database. Here are some common examples of relational database management systems:
  MySQL  Customer Relationship Management (CRM) applications
 PostgresQL  Commerce and financial applications
 Oracle
 SQL server CHOOSE BETWEEN UNMANAGED AND MANAGED DATABASES
 Amazon Aurora If you want to run a relational database on AWS, you first need to select how you want to run it: the
You communicate with most RDBMS by using Structured Query Language (SQL) queries. Here’s an unmanaged way or the managed way.
example: SELECT * FROM table_name. The paradigm of managed versus unmanaged services is similar to the Shared Responsibility Model.
This query selects all of the data from a particular table. However, the real power of SQL queries is in The Shared Responsibility Model distinguishes between AWS’s and the customer’s security
creating more complex queries that let you pull data from several tables to piece together patterns and responsibility over a service. Similarly, managed versus unmanaged can be understood as a tradeoff
answers to business problems. For example, querying the sales table and the book table together to see between convenience and control.
sales in relation to an author’s books. This is made possible by a join, which we talk about next. On-Premises Database
THE BENEFITS OF USING A RELATIONAL DATABASE Let’s say you operate a relational database on-premises (in your own data center). In this scenario, you
There are many benefits to using a relational database. A few of them are listed here. are responsible for all aspects of operation, including the security and electricity of the data center, the
management of the host machine, the management of the database on that host, as well as optimizing
 Joins: You can join tables, enabling you to better understand relationships between your data. queries and managing customer data. You are responsible for absolutely everything, which means you
 Reduced redundancy: You can store data in one table and reference it from other tables instead have control over absolutely everything.
of saving the same data in different places.
 Familiarity: Relational databases have been a popular choice since the 1970s. Due to this
popularity, technical professionals often have familiarity and experience with this type of
database.
 Accuracy: Relational databases ensure that your data is persisted with high integrity and
adheres to the ACID (atomicity, consistency, isolation, durability) principle.

USE CASES FOR RELATIONAL DATABASES

Much of the world runs on relational databases. In fact, they’re at the core of many mission-critical
applications, some of which you may use in your day to day life. Here are some common use cases for
relational databases. Applications that have a solid schema that doesn’t change often, such as: Now, let’s say you want to shift some of this work to AWS by running your relational database on
Amazon EC2. If you host a database on Amazon EC2, AWS takes care of implementing and
 Lift and shift applications that lifts an app from on-premises and shifts it to the cloud, with
maintaining the physical infrastructure and hardware and installing the operating system of the EC2
little or no modifications.
instance. However, you’re still responsible for managing the EC2 instance, managing the database on
Applications that need persistent storage that follows the ACID principle, such as: that host, optimizing queries, and managing customer data.

 Enterprise Resource Planning (ERP) applications

This is what is often referred to as the unmanaged database option on AWS. AWS is responsible for Amazon Aurora is an AWS-specific database that was built to take advantage of the scalability and
and has control over the hardware and underlying infrastructure, and you are responsible and have durability of the AWS Cloud. Aurora is designed to be drop in incompatible with MySQL or
control over management of the host and database. Managed Database PostgreSql. It can be up to five times faster than the standard MySQL databases and three times faster
than standard PostgreSQL databases. So if you have some use cases that require large amounts of data
to be stored with high availability, durability, and low latency for data retrieval time, consider using
Amazon Aurora over a standard MySQL or PostgreSQL RDS instance.

As you learned already in a previous lesson, subnets are bound to one AZ, and as a best practice for
production workloads, we recommend that you always replicate your solutions across at least two AZs
for high availability. With RDS, one DB instance belongs to one subnet inside of one AZ, so that isn't
meeting the criteria for best practices. Now, before you get worried about managing this all on your
own, just know that you can easily configure RDS to launch a secondary DB instance in another subnet
and another AZ using RDS Multi-AZ deployment. RDS will manage all of the data replication between
the two instances so that they stay in sync. The other cool thing about RDS Multi-AZ deployments is
that RDS also manages the failover for the instances.

One instance is the primary and the other is the secondary database. Your app connects to one
endpoint. If the primary instance goes down, the secondary instance gets promoted. The endpoint
doesn't change, so no code change is needed. All of the failover happens behind the scenes and is
If you want to shift even more of the work to AWS, you can use a managed database service. These handled by RDS. All you do need to do is to make sure that your app can reconnect to the database if it
services provide the setup of both the EC2 instance and the database, and they provide systems for experiences a momentary outage by updating any cache DNS lookups and reconnecting to the endpoint
high availability, scalability, patching, and backups. However, you’re still responsible for database which now connects to the secondary instance.
tuning, query optimization, and of course, ensuring that your customer data is secure. This provides
you ultimate convenience, but you have the least amount of control compared to the two previous
options. Using services like RDS make operating databases significantly more accessible and lowers the
operational overhead that comes along with relational database management.

Video: Amazon Relational Database Service

Reading: 3.6: Amazon Relational Database Service
Amazon RDS is a service that makes it easier for you to set up, operate, and scale a relational database.
Instead of telling you about RDS, I am going to show you. What is Amazon RDS?

First, we will click Create database, and then we are going to select the Easy create option, which gives Amazon RDS enables you to create and manage relational databases in the cloud without the
us the ability to accept the standard best practices for backups and high availability. You could select operational burden of traditional database management. For example, if you sell healthcare equipment
Standard create if you wanted more granular control to pick and choose the different features of your and your goal is to be the number-one seller in the Pacific Northwest, building out a database doesn’t
database setup. directly help you achieve that goal though having a database is necessary to achieve the goal. Amazon
RDS helps you offload some of this unrelated work of creating and managing a database. You can
focus on the tasks that differentiate your application, instead of infrastructure-related tasks such as
provisioning, patching, scaling, and restoring. Amazon RDS supports most of the popular relational  Burstable Performance, which provides a baseline performance level, with the ability to burst
database management systems, ranging from commercial options, open source options, and even an to full CPU usage.
AWS-specific option. Here are the supported Amazon RDS engines.

 Commercial: Oracle, SQL Server

 Open Source: MySQL, PostgreSQL, MariaDB
 Cloud Native: Amazon Aurora

The DB instance you choose affects how much processing power and memory it has. Not all of the
options are available to you, depending on the engine that you choose. You can find more information
about the DB instance types in the resources section of this unit. Much like a regular EC2 instance, the
DB instance uses Amazon Elastic Block Store (EBS) volumes as its storage layer. You can choose
between three types of EBS volume storage.

 General purpose (SSD)

 Provisioned IOPS (SSD)
 Magnetic storage (not recommended)
Note: The cloud native option, Amazon Aurora, is a MySQL and PostgreSQL-compatible database
built for the cloud. It is more durable, more available, and provides faster performance than the
Amazon RDS version of MySQL and PostgreSQL. To learn more about Amazon Aurora, view the
Amazon Aurora FAQs.

Understand DB Instances

Just like the databases that you would build and manage yourself, Amazon RDS is built off of compute
and storage. The compute portion is called the DB (database) instance, which runs the database engine.
Depending on the engine of the DB instance you choose, the engine will have different supported
features and configurations. A DB instance can contain multiple databases with the same engine, and
Work with Amazon RDS in an Amazon Virtual Private Cloud
each database can contain multiple tables. Underneath the DB instance is an EC2 instance. However,
this instance is managed through the Amazon RDS console instead of the Amazon EC2 console. When When you create a DB instance, you select the Amazon Virtual Private Cloud (VPC) that your
you create your DB instance, you choose the instance type and size. Amazon RDS supports three databases will live in. Then, you select the subnets that you want the DB instances to be placed in. This
instance families. is referred to as a DB subnet group. To create a DB subnet group, you specify:
 Standard, which include general-purpose instances  The Availability Zones (AZs) that include the subnets you want to add
 Memory Optimized, which are optimized for memory-intensive applications  The subnets in that AZ where your DB instance are placed
The subnets you add should be private so they don’t have a route to the internet gateway. This ensures
your DB instance, and the cat data inside of it, can only be reached by the app backend. Access to the
DB instance can be further restricted by using network access control lists (ACLs) and security groups.
With these firewalls, you can control, at a granular level, what type of traffic you want to allow into
your database. Using these controls provide layers of security for your infrastructure. It reinforces that
only the backend instances have access to the database.

Use AWS Identity and Access Management (IAM) Policies to Secure Amazon RDS

Network ACLs and security groups allow you to dictate the flow of traffic. If you want to restrict what
actions and resources your employees can access, you can use IAM policies.

Back Up Your Data If you restore data from an automated backup, you have the ability to do point-in-time recovery. Point-
in-time recovery creates a new DB instance using data restored from a specific point in time. This
You don’t want to lose any of that precious cat information. To take regular backups of your RDS restoration method provides more granularity by restoring the full backup and rolling back transactions
instance, you can use: up to the specified time range.
 Automatic backups Manual Snapshots
 Manual snapshots
If you want to keep your automated backups longer than 35 days, use manual snapshots. Manual
snapshots are similar to taking EBS snapshots, except you manage them in the RDS console. These are
Automatic Backups backups that you can initiate at any time, that exist until you delete them.

Automated backups are turned on by default. This backs up your entire DB instance (not just For example, to meet a compliance requirement that mandates you to keep database backups for a year,
individual databases on the instance), and your transaction logs. When you create your DB instance, you would need to use manual snapshots to ensure those backups are retained for that period of time.
you set a backup window that is the period of time that automatic backups occur. Typically, you want
to set these windows during a time when your database experiences little activity because it can cause
increased latency and downtime. If you restore data from a manual snapshot, it creates a new DB instance using the data from the
snapshot.
You can retain your automated backups between 0 and 35 days. You might ask yourself, “Why set
automated backups for 0 days?” The 0 days setting actually disables automatic backups from
happening. Keep in mind that if you set it to 0, it will also delete all existing automated backups. This
is not ideal, as the benefit of having automated backups is having the ability to do point-in-time
recovery.
Which Backup Option Should I Use? We covered Amazon RDS and relational databases, and that was the default option for businesses for a
long time, but relational databases aren't the best choice for all business needs. AWS creates services to
The answer, almost always, is both. Automated backups are beneficial for the point-in-time recovery. support purpose-built databases, meaning that there are many database services that AWS offers, and
Manual snapshots allow you to retain backups for longer than 35 days. they each were built with a certain use case in mind, and therefore are optimized for those use cases.
Get Redundancy with Amazon RDS Multi-AZ Let's think about the Employee Directory app.

When you enable Amazon RDS Multi-AZ, Amazon RDS creates a redundant copy of your database in We had originally decided that we would use RDS for the database, but now after thinking about it
another AZ. You end up with two copies of your database: a primary copy in a subnet in one AZ and a some more, RDS might not be the best fit for our needs. All we are really doing is storing one record in
standby copy in a subnet in a second AZ. a table for each employee.

The primary copy of your database provides access to your data so that applications can query and There are no complex relationships that need to be managed, and it's essentially just a lookup table.
display that information. Relational databases offer all sorts of features that are great for complex schemas and relationships, but
those features add overhead that is unnecessarily complex for simple things like a lookup table. On top
The data in the primary copy is synchronously replicated to the standby copy. The standby copy is not of that, the RDS option we chose charges per hour of instance run time, so we will get charged for the
considered an active database, and does not get queried by applications. running instances regardless of whether we're using it or not.
To improve availability, Amazon RDS Multi-AZ ensures that you have two copies of your database Our employee directory application will have much higher usage during the week and no usage on the
running and that one of them is in the primary role. If there’s an availability issue, such as the primary weekends.
database losing connectivity, Amazon RDS triggers an automatic failover.
Is there an AWS database offering that better fits our needs? Introducing Amazon DynamoDB.
When you create a DB instance, a domain name system (DNS) name is provided. AWS uses that DNS Amazon DynamoDB is a NoSQL database that is great for storing key value pairs or document data.
name to failover to the standby database. In an automatic failover, the standby database is promoted to This service works great at a massive scale and provides millisecond latency.
the primary role and queries are redirected to the new primary database.
It also charges based on the usage of the table and the amount of data that you are reading from the
To ensure that you don’t lose Multi-AZ configuration, a new standby database is created by either: table, not by the hour or by the second. This is a better option for our simple employee lookup table.
 Demoting the previous primary to standby if it’s still up and running.
 Or standing up a new standby DB instance.
Now, besides the employee directory app, there are other use cases that require databases of varying
The reason you can select multiple subnets for an Amazon RDS database is because of the Multi-AZ types.
configuration. You’ll want to ensure that you have used subnets in different AZs for your primary and
standby copies. What if you are writing an application that needs a full content management system? Neither RDS nor
DynamoDB would be the best solution. Luckily, AWS has quite a number of other database offerings.
Video: Purpose Built Databases on AWS

Before we move on to learning about Amazon DynamoDB, I want to touch on an idea that's important
when you're making architecture decisions for your AWS solutions, choosing the right database to fit
your business requirements rather than forcing your data to fit a certain database choice. There is no
one size fits all database for all purposes. You should pick a database that fits your specific use case,
and with AWS, you have multiple choices for databases.
 What Is Amazon DynamoDB?

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable
performance with seamless scalability. DynamoDB lets you offload the administrative burdens of
operating and scaling a distributed database so that you don't have to worry about hardware
provisioning, setup and configuration, replication, software patching, or cluster scaling.

With DynamoDB, you can create database tables that can store and retrieve any amount of data and
serve any level of request traffic. You can scale up or scale down your tables' throughput capacity
without downtime or performance degradation. You can use the AWS Management Console to monitor
resource utilization and performance metrics.
For this use case, you might look into Amazon Document DB. It's great for content management,
catalogs, or user profiles. Let's think of another use case. What if you had a social network that you DynamoDB automatically spreads the data and traffic for your tables over a sufficient number of
wanted to track? Keeping track of those kind of social webs, figuring out who is connected to who can servers to handle your throughput and storage requirements, while maintaining consistent and fast
be difficult to manage in a traditional relational database. performance. All of your data is stored on solid-state disks (SSDs) and is automatically replicated
across multiple Availability Zones in an AWS Region, providing built-in high availability and data
So, you could use Amazon Neptune, a graph database engineered for social networking and durability.
recommendation engines, but it's also good for use cases like fraud detection, or perhaps you have a
supply chain that you have to track with assurances that nothing is lost, or you have a banking system Core Components of Amazon DynamoDB
or financial records that require 100% immutability.
In DynamoDB, tables, items, and attributes are the core components that you work with. A table is a
What you really need is an immutable ledger, so perhaps Amazon QLDB, or Quantum Ledger collection of items, and each item is a collection of attributes. DynamoDB uses primary keys to
Database, is a better fit for this use case. It's an immutable system of record where any entry can never uniquely identify each item in a table and secondary indexes to provide more querying flexibility.
be removed, and therefore is great for industries that need to be audited for regulatory and compliance
reasons.
The following are the basic DynamoDB components:
It can take a lot of experience and expertise to operate databases at scale, and that's why it's so
beneficial to utilize one of the AWS database offerings. You don't need to be an expert on running all of Tables – Similar to other database systems, DynamoDB stores data in tables. A table is a collection of
these different types of databases. Instead, you can just use the database service that is best for your use data. For example, see the example table called People that you could use to store personal contact
case and focus on your application and providing value to your end users. You don't need to build up a information about friends, family, or anyone else of interest. You could also have a Cars table to store
ton of in-house expertise to operate a highly scalable immutable ledger database. You can just use AWS information about vehicles that people drive.
QLDB instead. The key thing to understand is AWS wants to make sure that you are using the best tool
for the job. Items – Each table contains zero or more items. An item is a group of attributes that is uniquely
identifiable among all of the other items. In a People table, each item represents a person. For a Cars
table, each item represents one vehicle. Items in DynamoDB are similar in many ways to rows,
records, or tuples in other database systems. In DynamoDB, there is no limit to the number of items
Video: Introduction to Amazon DynamoDB
you can store in a table.

Attributes – Each item is composed of one or more attributes. An attribute is a fundamental data
Reading: Reading 3.8: Introduction to Amazon DynamoDB element, something that does not need to be broken down any further. For example, an item in a People
table contains attributes called PersonID, LastName, FirstName, and so on. For a Department table, an Breaking Up Applications and Databases
item might have attributes such as DepartmentID, Name, Manager, and so on. Attributes in
DynamoDB are similar in many ways to fields or columns in other database systems. As the industry changes, applications and databases change too. Today, with larger applications, you no
longer see just one database supporting it. Instead, these applications are being broken into smaller
Security with Amazon DynamoDB services, each with their own purpose-built database supporting it.

DynamoDB also offers encryption at rest, which eliminates the operational burden and complexity This shift removes the idea of a one-size-fits-all database and replaces it with a complimentary
involved in protecting sensitive data. For more information, see DynamoDB Encryption at Rest. database strategy. You can give each database the appropriate functionality, performance, and scale that
the workload requires.
Below you can find additional resources for learning about Amazon DynamoDB:

 External Resource: Introduction to Amazon DynamoDB

Reading 3.9: Choose the Right AWS Database Service

AWS Database Services

AWS has a variety of different database options for different use cases. Use the table below to get a
quick look at the AWS database portfolio.