Databases

Storing data in systematic way

 Traditional System
Earlier, many of the works were done on a traditional paper-based approach.

For example:

● Sending Messages (postal cards)

● Tickets Booking
● Government Application Forms

knowledge portal
 Newer Approaches
Instead of a paper-based approach, the data is now stored in computer systems.

A database is an organized collection of data, generally stored and accessed electronically from a
computer system.

Name Mobile Number

Mr A 9XXXXX

Mr B 022XXXX

knowledge portal
 Database Types
There are various types of databases; let’s explore few of them.

1. Flat File

A flat-file database is a database that stores data in a plain text file.

Example: Excel

knowledge portal
 Relational Databases
2. Relational Database

Data is organized into tables of columns and rows representing a specific entity type.

Generally uses SQL (Structured Query Language) to manage databases.

Example: MySQL

knowledge portal
 NoSQL Databases
3. NoSQL Databases

NoSQL databases stores and manages data that allows very high speeds and great operational
flexibility.

Example: MongoDB, DynamoDB

knowledge portal
 Installing & Managing Databases
Depending on the organizations, they can either decide to manage a database on their own
servers or use a managed database offerings like RDS.

1. Manage Database Own Servers:

● Provisioning Database.
● Host Security (Patching, Hardening and others)
● Configure Replicas, High-Availability, Upgrading, Monitoring and others

Install Database

knowledge portal
 Installing & Managing Databases
2. Managed Database Offering

● Provisioning Database via simple GUI steps.

● Host Security (Patching, Hardening and others) taken care by the provider.
● Configure Replicas, High-Availability within a single click.

knowledge portal
Introduction to RDS
Managed Database Service
 Challenges with EC2 DB

Maintaining database in EC2 instance leads to multiple challenges, this includes:

1. Provisioning Database.
2. Host Security (Patching, Hardening and others)
3. Configure Replicas, High-Availability, Upgrading and others

Most organizations simply offload these tasks to 3rd party vendors or hire database administrators.

knowledge portal
 Intro to RDS
Amazon RDS is a managed service that makes it easy to set up, operate, and scale a relational
database in the cloud.

AWS manages the underlying hardware, OS, security and software patching, automated failure
detection and recovery for you.

In click of few buttons we can :

● Provision / Resize hardware on-demand.
● Multi-AZ deployments.
● Create read replicas.

knowledge portal
RDS Read Replicas

12
 Use Case : Bank
In bank, for different kind of work purpose, there are different people you might
have to approach. For example :

- Cash Collector
- Cheque Counter
- Enquiry Counter
 Database Way
Using a single database for all kind of activity will increase the database load
and slow down the operations.

connect ()

Database
 Improved Architecture - Read Replica
Read Replica allows customers to offload read requests or analytics traffic from
the primary instance

All WRITE Operations

Primary DB

Replication

All READ Operations

Read Replica
 RDS Read Replica
RDS Read Replica feature allows customers to implement “Database Read
Replica” functionality for RDS databases.

Replication

Primary Database Read Replica

 Pointers to Note - 1
You can create one or more replicas of a given source DB Instance and serve
high-volume application read traffic.

Read Replica 1
Replication

Primary Database

Read Replica 2
 Pointers to Note - 2
With Amazon RDS, you can create a read replica in a different AWS Region
from the source DB instance.

Replication

ap-south-1
us-east-1
Amazon RDS Multi AZ Deployments

19
 Understanding the Challenge
If database is running in a specific availability zone and if the AZ is down or
unreachable then your entire application can be impacted.

connect ()

Clients
Availability Zone 1
 Multi-AZ Architecture
In this approach, Amazon creates a standby DB instance and synchronously
replicates data from the primary DB instance in a different availability zone.

Primary

Availability Zone 1 Replication

Clients

Standby

Availability Zone 2
 Failover Condition
If a planned or unplanned outage of your DB instance results from an
infrastructure defect, Amazon RDS automatically switches to a standby replica
in another Availability Zone if you have turned on Multi-AZ.

● Loss of availability in primary Availability Zone

● Loss of network connectivity to primary
● Compute unit failure on primary
● Storage failure on primary

Failover times are typically 60–120 seconds.

Amazon Aurora

Closed Source Database

 Overview of Database Offerings
Databases are generally divided into two types:

● Open Source Databases

● Commercial Databases

Commercial Offering does come with various aspects that are not found in the open
source databases.

Open Source Databases Commercial Databases

knowledge portal
 Introducing Aurora
Amazon Aurora is a MySQL and PostgreSQL-compatible relational database built for
the cloud, that combines the performance and availability of traditional enterprise
databases with the simplicity and cost-effectiveness of open source databases.

Amazon Aurora is up to five times faster than standard MySQL databases and three
times faster than standard PostgreSQL databases.

It provides the security, availability, and reliability of commercial databases at 1/10th the
cost

Performance / Simplicity and Cost

Availability of Effectiveness of Open
Enterprise Databases Source Databases

Amazon Aurora
 RDS - Multi-AZ & Read Replica Architecture
In a typical setup, primary, standby and read replicas are three different instances in
multiple availability zones. The underlying storage is EBS volume.

Synchronous
Replication Asynchronous
Replication

Standby Instance Primary Instance Read Replica

AZ 2 AZ 1 AZ 3

knowledge portal
 RDS with EBS Storage

RDS with MySQL

EBS Volume

knowledge portal
 Aurora Architecture
Two Primary Components: DB Instances + Storage Cluster Volume

Since Aurora and Storage Layer are independent, we can scale the storage easily.

Availability Zone
Availability Zone
Availability Zone

Write Write

knowledge portal
 Overview of Storage Volume

Availability Zone 1 Availability Zone 2 Availability Zone 3

Cluster Storage Volume

knowledge portal
 Scalability Aspect in Aurora
With this architecture, you can add a DB instance quickly because Aurora doesn't make
a new copy of the table data. Instead, the DB instance connects to the shared volume
that already contains all your data.

Availability Zone
Availability Zone
Availability Zone

Write Write

knowledge portal
 Scale at a Faster Pace

Availability Zone 1 Availability Zone 2 Availability Zone 3

Cluster Storage Volume

knowledge portal
 Aurora Architecture
Two Primary Components: DB Instances + Storage Cluster Volume

Since Aurora and Storage Layer are independent, we can scale the storage easily.

Write Write Write

Read Read Read

Aurora Read Replica Primary Instance Aurora Read Replica

knowledge portal
 Aurora Architecture

knowledge portal
 Aurora Endpoints
You can connect to Aurora Cluster through endpoints.

Endpoints is Aurora Specific URL consisting of host and port.

There are three primary types of endpoints available:

● Cluster Level Endpoints

● Reader Level Endpoints
● Instance Level Endpoints

knowledge portal
 Application

Cluster Endpoint Reader Endpoint

 Aurora Endpoints

Endpoint Types Description

Cluster Level Endpoints Connects to current primary DB instance in the cluster.

Used for performing write operations.

Reader Level Endpoints Built-In endpoints for Read Replicas.

For Multiple Read Replicas, this endpoint will balance load
among all read replicas.

Instance Endpoints Allows connection directly to the instance.

Custom Endpoints Ability to create custom endpoints for our own requirements.

knowledge portal
 Aurora Features
Aurora provides wide variety of interesting features. Some of these includes:

Global Databases Serverless

Cross Region Auto-Scaling

Replication

BackTrack IAM DB Authentication

Sharing DB Clusters RDS Proxy

knowledge portal
 Aurora Architecture
Two Primary Components: DB Instances + Cluster Volume

Since Aurora and Storage Layer are independent, we can scale the storage easily.

Primary Instance Aurora Read Replica Aurora Read Replica

knowledge portal
Amazon Neptune
Monitor Everything
 Different Types of Database Technologies
There are multiple different types of database technologies available and each has its own set of
benefits.

Some of the popular ones include:

1. Relational Database [MySQL]

2. NoSQL Databases [key/value] [DynamoDB]

3. Graph Databases [Neptune]

knowledge portal
 Revising Relational Databases
Data is organized into tables of columns and rows representing a specific entity type.

Generally uses SQL (Structured Query Language) to manage databases.

Example: MySQL

knowledge portal
 Graph Database
A graph database stores nodes and relationships instead of tables.

Whenever connections or relationships between entities are at the core of the data that you're
trying to model, a graph database is your natural choice.

You can easily find out who the "friends of friends" of a particular person are—for example, the
friends of Howard's friends.
 Amazon Neptune
Amazon Neptune is a fast, reliable, fully managed graph database service that makes it easy to
build and run applications that work with highly connected datasets.

knowledge portal
 Graph Visualization
In many cases the Neptune workbench can create a visual diagram of your query results.

knowledge portal
NoSQL Databases
Different Type of Database
 Importance of Schema Free Structure
In a traditional schema database like MySQL, before you start to add data in it, you must first
define the structure of those records.

Example: User Data Information

userID Name Age Interests AWS A/C Number

knowledge portal
 Schema Free Database
On Schema Free databases like MongoDB (NoSQL), you can simple add records without any
previous structure.

We can easily group records that do not have a same structure

knowledge portal
 Basics of NoSQL Database
NoSQL databases ("not only SQL") are non-tabular databases and store data differently than
relational tables.

NoSQL have gained huge popularity because they are simpler to use, flexible and can achieve
performance that are very difficult with traditional relational databases.

userID Name Age Interests

01 Harsh 26 Astronomy

02 zeal 26 Teaching

knowledge portal
 Advantages of NoSQL Database

There are lot of advantages of NoSQL database over standard relational databases :

● Schema Free

● Horizontal Scaling

● Easy Replication

● Can manage huge amount of data.

knowledge portal
Basics of DynamoDB
Storing data NoSQL Way
 Introduction
DynamoDB is a fully managed NoSQL database service by AWS.

Being managed service, it simplifies lots of operations like hardware provisioning, setup and
configurations, patching, replication, clustering etc for the users.

.
DynamoDB

knowledge portal
ElastiCache
Let’s Cache
 Simple Analogy

● There is a new vegetable shop in the locality which has become very popular.
● Every day 300-500 people visit and buy vegetables.
● Each visitor asks the price of at-least two-three veggies before making a purchase.

Imaging the condition of the employee inside that shop after a few days.

knowledge portal
 Simple Analogy - Smart Approach

Vegetable Shop Owner decided to create a dashboard that has a list of all the common vegetable
prices which are requested by the buyers.

Price List

Veggie A - $1

Veggie B - $2

Veggic C - $C

knowledge portal
 Simple Analogy - Learning

1. Since the price list of common items was listed, users no longer need to ask the employees
about it. This reduces the overall load on the employee.

2. Visitors can quickly get to go through the price list - Better Efficiency.

knowledge portal
 Challenges with Database Workloads

There can be certain common queries within the database that hundreds of users might
request.

This would increase the load on the database and can lead to performance degradation.

SELECT * from user_data;

knowledge portal
 Caching Solutions

With caching solutions, you can cache the response associated with frequent queries.

This allows better response time and decreases the load on the database servers.

knowledge portal
 Popular Caching Solutions

Two of the most popular caching solutions used for databases are:

1. Memcached
2. Redis

To use them, you will have to install, configure, optimize and secure the EC2 instances where
these engines would be running.

knowledge portal
 Introducing AWS ElastiCache

ElastiCache is a fully managed AWS service that makes it easier to deploy, operate and scale an
in-memory data-store or cache in the cloud.

It is like a managed service and within a few clicks, we can have an in-memory layer in our infra.

ElastiCache can also detect and replace failed nodes thus reducing the overhead.

knowledge portal
Local Zones
Let’s Extend Regions
 Understanding the Challenge
Each AWS Region has multiple, isolated locations known as Availability Zones.

It takes time to build AWS region.

Users can see certain amount of latency depending on how far they are from the region.
 Introducing Local Zones
A Local Zone is an extension of an AWS Region in geographic proximity to your users.

You should use AWS Local Zones to deploy workloads closer to your end-users for low-latency
requirements.

knowledge portal
 Important Pointers

Only Core AWS services are supported. Not ALL AWS services are available in Local Zone.

Limited Instance Type Support.

To get started, you first need to enable the AWS Local Zones for your AWS account before you
can deploy resources to them.

knowledge portal