How To Choose

The Right Database

Yaki Zakai
Manager, Solutions Architecture, AWS

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Agenda

• Database Considerations
• Common data models/use cases
• AWS database services

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Why did you choose this database?

“Because we heard X is the best new thing.”

“Because we have a site license for X.”

“Because X is what we know how to use.”

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Why should you choose this database?

“Because this database is purpose built to support

what my application is designed to do.”

© 2019, Amazon Web Services, Inc. or its Affiliates.

 A common question that I get is why do we offer so many database products? The answer for me is
simple: Developers want their applications to be well architected and scale effectively. To do this, they
need to be able to use multiple databases and data models within the same application.
https://www.allthingsdistributed.com/2018/06/purpose-built-databases-in-aws.html
© 2019, Amazon Web Services, Inc. or its Affiliates.
 Database Workloads

Data Considerations

Shape Size Compute

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Shape

Purpose-Built For Optimized for When you need to Example Workload

Row Store Operate on a record or group of records Payroll

Column Store Aggregations, scans and joins Analytics

Key-Value Store Query by key with high throughput & fast ingestion Tracking devices

Document Store Index & store documents for query on any property Patient data

Graph Store Persist and retrieve relationships Recommendations

Time-Series Store Store and process data sequence Process Engine telemetry

Unstructured Store Get and put of objects Store user reviews

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Size

Considerations Example Workload

Number of employees – bounded

Size at limit – bounded or unbounded
Number of sensors – unbounded

10-years of sales data but only the last 12-months is queried

Working set size & caching
Session data for users of a streaming service

Get one row

Retrieval size
Get one thousand rows

Storage and processing of car location data is partition-able

Partition-able or monolithic
Company payroll data has no natural partition boundary

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Compute

Considerations Example Workload

Sum of sales for the last 12-months

Compute functions
Get & Put data

Location telemetry from cars added to the database every minute

Rate of ingestion
New employees records being added to the database

Inventory counts are frequently updated

Change rate
Sales records are never updated

Million users browsing a product catalogue every second

Throughput
50 doctors looking at 300 patient records per day

Get the location of a car in 5 milliseconds

Latency
Get the min, max & average deal size for the last 12-months in 5 seconds

© 2019, Amazon Web Services, Inc. or its Affiliates.

 My [insert your favorite DB] works for everything

General purpose Special purpose

One size fits all Efficiency at scale

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Modern apps create new requirements

Users: 1 million+
Data volume: TB–PB–EB
Locality: Global
Performance: Milliseconds–microseconds
Request rate: Millions
Access: Web, mobile, IoT, devices
Scale: Up-down, Out-in
Economics: Pay for what you use
Ride hailing Media streaming Social media Dating Developer access: No assembly required

© 2019, Amazon Web Services, Inc. or its Affiliates.

 300M total users
7B exercises per month

CHALLENGE
Wanted to enable anyone to learn a
language for free.

SOLUTION
Purpose-built databases from AWS:
• DynamoDB: 31B items tracking
which language exercises completed
• Aurora: primary transactional
database for user data
• ElastiCache: instant access to
common words and phrases

Result:
More people learning a language on
© 2019, Amazon Web Services, Inc. or its Affiliates. Duolingo than entire US school system
 Common data categories and use cases

Relational Key-value Document In-memory Graph Search Time-series Ledger

Referential Low-latency, Indexing and Microseconds Creating and Indexing and Collect, store, Complete,
integrity, ACID key lookups storing latency, key- navigating searching and process immutable, and
transactions, with high documents based queries, data relations semistructured data sequenced verifiable
schema- throughput and with support and specialized easily and logs and data by time history of all
on-write fast ingestion for query on data structures quickly changes
of data any attribute

Lift and shift, Real-time Content Leaderboards, Fraud detection, Product IoT Systems
EMR, CRM, bidding, management, real-time social catalog, help applications, of record,
finance shopping cart, personalization, analytics, networking, and FAQs, event tracking supply chain,
social mobile caching recommendation full text health care,
engine registrations,
financial

© 2019, Amazon Web Services, Inc. or its Affiliates.

 AWS purpose-built databases

Relational Key-value Document In-memory Graph Search Time-series Ledger

Amazon RDS Amazon Amazon Amazon Amazon Amazon

DynamoDB Amazon Amazon
DocumentDB ElastiCache Neptune Elasticsearch
Timestream Quantum
Service
Ledger
Amazon Community Commercial Redis Memcached Database
Aurora

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Relational Key-value Document Graph

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Relational
Patient
* Patient ID
First Name
Last Name
Referential Gender Visit
integrity, ACID DOB * Visit ID
transactions, * Doctor ID * Patient ID
schema-on-write * Hospital ID
Date
Doctor
* Doctor ID * Treatment ID

First Name
Lift and shift, ERP, CRM,
finance Last Name
Medical Specialty Medical Treatment
* Hospital Affiliation * Treatment ID
Procedure
How Performed
Amazon Aurora Hospital Adverse Outcome
Amazon RDS * Hospital ID
Contraindication
Amazon Redshift Name
Address
Rating

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Relational
Patient
* Patient ID
First Name
Last Name
Referential Gender Visit
integrity, ACID DOB * Visit ID
transactions, * Doctor ID * Patient ID
schema-on-write * Hospital ID
Date
Doctor
//
// Doctors
Numberaffiliated
of patientwith Mercy
visits eachhospital
doctor completed last
* Doctor ID * Treatment ID
week First Name
SELECT Last Name
SELECT
d.first_name, d.last_name
Medical Specialty Medical Treatment
FROM d.first_name, d.last_name, count(*)
FROM
doctor as d, * Hospital Affiliation * Treatment ID
visit as v,
hospital Procedure
hospitalas
as hh,
WHERE doctor as d How Performed
WHEREd.hospital = h.hospital_id Hospital Adverse Outcome
v.hospital_id
AND = h.hospital_id
h.name = ‘Mercy'; * Hospital ID
Contraindication
AND h.hospital_id = d.hospital
AND v.t_date > date_trunc('week’, Name
CURRENT_TIMESTAMP - interval '1 week') Address
GROUP BY
d.first_name, d.last_name; Rating

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Graph

Creating and
navigating
relations
between data
easily
and quickly

Fraud detection, social

networking,
recommendation
engine

Amazon
Netptune

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Graph

Creating and
navigating
relations PRODUCT

between data PURCHASED PURCHASED

easily
and quickly KNOWS

PURCHASED

Fraud detection, social

networking,
recommendation
engine
FOLLOWS
Amazon SPORT

Netptune
FOLLOWS

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Graph

gremlin> V().has(‘name’,’sara’).as(‘customer’).out(‘follows’).in(‘follows’).out(‘purchased’)
( (‘customer’)).dedup() (‘name’) ('name')

PRODUCT
PURCHASED PURCHASED

// Identify a friend in common and KNOWS

make a recommendation PURCHASED
gremlin> g.V().has('name','mary').as(‘start’).
both('knows').both('knows’).
where(neq(‘start’)).
dedup().by('name').properties('name')
FOLLOWS FOLLOWS
SPORT
// Product recommendation to a user

FOLLOWS

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Key-value
Highly partitionable
data
Low-latency, Table 1 Table 2 Table N
key look-ups
with high
…
throughput and
fast ingestion … …
of data

…
…
…
Real-time bidding,
shopping cart, IoT
device tracking
Partitions Partitions Partitions

Amazon
DynamoDB … … …

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Key-value

Gamers

GET { Primary Key

Attributes
TableName:"Gamers", Gamer Tag Type
Key: { Level Points Tier
"GamerTag":"Hammer57", Rank
"Type":"Status” } } 87 4050 Elite
Health Progress
Hammer57 Status
90 30
Class Damage Range
// Return all Hammer57 Weapon
Taser 87% 50
Level Points Tier
Gamers Rank
5 1072 Trainee
GamerTag = :a FluffyDuffy
Health Progress
// Status of Hammer57 Status
:a Hammer57 37 8

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Document
!=
Indexing and
storing
documents with
support
for query on
any attribute JSON !JSON
Content management,
personalization,
mobile

(client) (app) (database)

Amazon
DocumentDB

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Document

User profiles
Indexing and
storing {
documents with id: 181276,
support { username: "sue1942",
id: 181276, name: {first: "Susan",
for query on
username: "sue1942", last: "Benoit"},
any attribute name: {first: "Susan", ExplodingAcorns: {
Self Contained last: "Benoit"} hi_score: 3185400,
Flexible Schema } global_rank: 5139,
bonus_levels: true
},
promotions: ["new user","5%",”acorn"]
}
Amazon
DocumentDB

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Amazon Quantum Ledger Database (QLDB) (Preview)

Fully managed ledger database

Track and verify history of all changes made to your application’s data
Cryptographically
Immutable verifiable Highly scalable Easy to use

Maintains a sequenced record of Uses cryptography to Executes 2–3X as many Easy to use, letting you
all changes to your data, which generate a secure output transactions than ledgers use familiar database
cannot be deleted or modified; file of your data’s history in common blockchain capabilities like SQL APIs
you have the ability to query and frameworks for querying the data
analyze the full history

© 2019, Amazon Web Services, Inc. or its Affiliates.

 But what about unstructured data?

© 2019, Amazon Web Services, Inc. or its Affiliates.

 Data lakes help you cost-effectively scale
Amazon
Amazon Athena Kinesis Amazon

Amazon
EMR Elasticsearch Store exabytes of data
Service
QuickSight

Amazon Stage from landing dock to transformed to curated–

AI Services
Redshift Make available in each
Amazon S3
Load, transform, and catalog once

Make data available to many tools

Open formats and interfaces support innovation

Snowball Kinesis
Data Streams
Snowmobile Kinesis
Kinesis Data Firehose
Video Streams
© 2019, Amazon Web Services, Inc. or its Affiliates.
 In Summary….

© 2019, Amazon Web Services, Inc. or its Affiliates.

 What have we learned?

Relational Key-value Document In-memory Graph Search Time-series Ledger

Amazon RDS Amazon Amazon Amazon Amazon Amazon

DynamoDB Amazon Amazon
DocumentDB ElastiCache Neptune Elasticsearch
Timestream Quantum
Service
Ledger
Amazon Community Commercial Redis Memcached Database
Aurora

Choose the right tool for the job Focus on shape, size and AWS purpose-built databases
compute
Start from the application and work
backwards

© 2019, Amazon Web Services, Inc. or its Affiliates.

Thank you!
Yaki Zakai
Manager, Solutions Architecture, AWS

© 2019, Amazon Web Services, Inc. or its Affiliates.