Module 03
Module 03
By- Shyam
AWS Storage, Data Management, and
Testing
Elastic Block Storage
• Amazon Elastic Block Store (Amazon EBS) is a block-level storage
service provided by Amazon Web Services (AWS) designed for use
with Amazon Elastic Compute Cloud (Amazon EC2) instances.
• It provides persistent block-level storage volumes that can be attached
to EC2 instances to support various use cases such as database storage,
file system storage, and boot volumes.
• Amazon EBS provides scalable, high-performance block storage for
EC2 instances, offering features such as persistence, elasticity, backup
and recovery, encryption, and integration with other AWS services to
meet a wide range of storage requirements in the cloud.
Elastic Block Storage
key features and characteristics of Amazon EBS:
• Block-level Storage: Amazon EBS provides block-level storage volumes,
which are essentially virtual hard drives that can be attached to EC2 instances.
These volumes appear as raw block devices to the EC2 instances and can be
formatted and used as desired, similar to physical hard drives.
• Persistence: Amazon EBS volumes are persistent, which means that the data
stored on them remains intact even after the associated EC2 instance is
stopped or terminated. This makes EBS volumes suitable for storing important
data that needs to persist beyond the lifetime of an EC2 instance.
• Elasticity: Amazon EBS volumes can be easily resized to accommodate
changing storage requirements. You can increase the size of an EBS volume
on-the-fly without requiring any downtime for the associated EC2 instance.
Elastic Block Storage
key features and characteristics of Amazon EBS:
• Snapshot Backup: Amazon EBS supports creating snapshots of EBS volumes, which
are incremental backups of the volume's data. Snapshots are stored in Amazon Simple
Storage Service (Amazon S3) and can be used to create new EBS volumes or restore
volumes to a previous state.
• Encryption: Amazon EBS supports encryption of data at rest using AWS Key
Management Service (KMS). You can encrypt both the data stored on EBS volumes
and the snapshots created from those volumes to ensure data security and compliance
with regulatory requirements.
• Performance Options: Amazon EBS offers different volume types optimized for
various performance characteristics, including General Purpose SSD (gp2),
Provisioned IOPS SSD (io1), Throughput Optimized HDD (st1), and Cold HDD (sc1).
You can choose the appropriate volume type based on your performance and cost
requirements.
Elastic Block Storage
key features and characteristics of Amazon EBS:
• High Availability and Durability: Amazon EBS volumes are designed for
high availability and durability. They are replicated within an Availability
Zone (AZ) to protect against component failures, and snapshots are stored
durably in Amazon S3 across multiple AZs for added data durability.
• Integration with Other AWS Services: Amazon EBS integrates seamlessly
with other AWS services such as Amazon EC2, Amazon CloudWatch, AWS
Identity and Access Management (IAM), and AWS CloudFormation, allowing
you to manage and monitor your EBS volumes effectively within the AWS
ecosystem.
AWS S3 & Storage Gateway
• CLI Usage:
• Hands-on exercises with AWS Command Line Interface.
Glacier & Snowball
• Glacier and Snowball are both services offered by Amazon Web Services
(AWS) for data storage and transfer, particularly useful for large-scale or
offline data management.
• Glacier is primarily for long-term data storage with infrequent access
requirements, while Snowball is a physical data transfer solution for moving
large volumes of data in and out of AWS securely and efficiently.
• Snowball simplifies and accelerates large-scale data transfers to and from
AWS, making it an effective solution for scenarios where traditional
methods are impractical or inefficient.
• Both services offer solutions for different aspects of data management
within the AWS ecosystem.
Glacier & Snowball
Amazon Glacier:
• Glacier is a long-term storage service designed for data archiving and backup.
• It is optimized for infrequently accessed data that requires long-term retention,
such as regulatory compliance archives or backup data that you don't need to
access regularly.
• Glacier offers very low-cost storage compared to other AWS storage services,
but it's important to note that accessing data from Glacier can have higher
latency compared to more frequently accessed storage options like Amazon
S3.
Glacier & Snowball
Amazon Snowball:
• Snowball is a physical data transport solution offered by AWS for transferring
large amounts of data into and out of the AWS cloud.
• It addresses challenges associated with transferring large datasets over the
internet, such as limited bandwidth, security concerns, and high network costs.
• Snowball devices are rugged, tamper-resistant, and come in different sizes
(Snowball, Snowball Edge, and Snowmobile) to accommodate various data
transfer needs.
• Customers can request a Snowball device, transfer their data onto it, and then
ship it to an AWS data center where the data is uploaded into the customer's
AWS account.
Data Migration/Management Tools
Amazon Web Services (AWS) offers a variety of data migration and management tools to
facilitate the movement, transformation, storage, and analysis of data within the AWS
ecosystem.
Key tools include:
AWS Database Migration Service (DMS):
• DMS helps you migrate databases to AWS easily and securely. It supports homogeneous migrations
(e.g., Oracle to Oracle) as well as heterogeneous migrations (e.g., Oracle to Amazon Aurora).
• DMS can also be used for continuous data replication between source and target databases,
enabling near real-time data synchronization.
AWS DataSync:
• DataSync is a data transfer service designed to simplify and automate moving data between on-
premises storage and AWS storage services, such as Amazon S3, Amazon EFS, and Amazon FSx
for Windows File Server.
Data Migration/Management Tools
AWS Glue:
• Glue is a fully managed extract, transform, and load (ETL) service that makes it easy to
prepare and load data for analytics. It can automatically discover and catalog metadata
about your data, perform data transformation tasks, and generate ETL code.
Amazon Kinesis:
• Kinesis is a platform for streaming data on AWS, allowing you to collect, process, and
analyze real-time data streams such as website clickstreams, IoT device telemetry data, and
log data.
Amazon EMR (Elastic MapReduce):
• EMR is a cloud big data platform for processing large-scale data using frameworks such as
Apache Hadoop, Apache Spark, Apache Hive, Apache HBase, and more.
• It simplifies running big data frameworks on AWS by automating the provisioning and
scaling of resources.
Data Migration/Management Tools
Anycast IP Addresses:
• Global Accelerator assigns static Anycast IP addresses that act as a fixed entry point to your
application. These IP addresses are announced from multiple AWS edge locations globally, allowing
traffic to be routed to the nearest healthy endpoint.
Health Checks:
• Global Accelerator continuously monitors the health of your application endpoints by performing
health checks. It automatically reroutes traffic away from unhealthy endpoints to healthy ones to
ensure high availability and reliability.
Traffic Diversions:
• During periods of increased latency or packet loss, Global Accelerator can dynamically reroute traffic
to alternative healthy endpoints, improving application performance and reducing latency for users.
Accelerated DNS Resolution:
• Global Accelerator integrates with Amazon Route 53 to provide fast and reliable DNS resolution for
your application's Anycast IP addresses, reducing DNS lookup times and improving user experience.
AWS Global Accelerator (for global load testing)