1

LinkedIn
Mar 27, 2018
Mike North https://github.com/mike-works/sql-fundamentals
© Mike.Works, Inc. 2017. All rights reserved
SQL FUNDAMENTALS 2

Why are you talking to me about a database?

▸ Lines between frontend and backend

continue to blur

▸ Web apps are becoming "thick clients"

▸ Mechanical sympathy is valuable for

improving performance

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 3

Two SQL Courses

▸ SQL Fundamentals is a great primer for developers who use databases.

▸ It mostly sticks to common SQL that’s implemented the same way across
SQLite, PostgreSQL, MySQL, etc…

▸ Professional SQL is a deeper course, intended for developers who wish to

design and maintain a database.

▸ It tackles several topics that are treated very differently depending on

your RDBMS. We’ll work with MySQL and PostgreSQL examples.

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS

SQL Fundamentals: Agenda

▸ Relational Algebra and SQL ▸ Transactions

foundations
▸ Creating/Deleting/Updating
▸ Basic SELECT Records

▸ Filtering results with WHERE ▸ Migrations

▸ Sorting and paginating ▸ Indices

▸ JOINs ▸ Types & Column Constraints

▸ Aggregate functions and GROUP BY

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 5

What makes something a database?

1. Organized collection of persisted data

2. Write something and read it out later

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 6

Databases: a diverse family Key-Value Stores

Column Stores

Relational Stores

Graph Databases
Voldemort

Document Databases

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 7

Our SQL databases

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 8

Codd's Relational Model

▸ A relation is a set of tuples (d1, d2, d3,!!... dn).

table rows

▸ Each element in a tuple dj is a member of Dj, a data domain.

set of
cell value
allowed values
▸ These elements are called attribute values, and are associated with an
attribute.

column name + type

© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 9

Schema
CUSTOMER
▸ The "shape" or structure of your data
id companyname contactname
(as opposed to the values themselves) INTEGER STRING STRING

▸ Tables, column names, types and

constraints

▸ Strongly connected to your "data

domain"

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS

Relational Algebra vs. Relational Calculus

Movies Jeff Goldblum is in

THE RELATIONAL ALGEBRA WAY

▸ Join Movie and Actor over actorId

▸ Filter the results to only include those that include Jeff Goldblum

▸ Select the movieTitle and year columns

THE RELATIONAL CALCULUS WAY

▸ Get movieTitle and year for Movie such that there exists an
Actor A who was in the movie and has the name Jeff Goldblum.

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS

Structured Query Language

▸ Used to manage data in a Relational Database Management System (RDBMS)

▸ Declarative, unlike its predecessors

▸ Can be procedural too! (PL/SQL)

▸ Inspired by Codd's Relational Model

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS

SQL Language Elements

SELECT clause SELECT * FROM Employee

Statement
WHERE clause WHERE id = 123 AND is_admin = 'true'

Expression
Predicate

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 13

Command Line SQL Database SQL

sqlite3 dev.sqlite "SELECT * FROM Employee"

psql northwind -c "SELECT * FROM Employee"

mysql -D northwind "SELECT * FROM Employee"

Database SQL

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 14

Creating a Database

# SQLite db is created automatically

sqlite3 northwind.sqlite

# PostgreSQL
createdb northwind

# MySQL
mysqladmin create 'northwind';

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 15

Deleting a Database

# SQLite db is created automatically

rm northwind.sqlite

# PostgreSQL
dropdb northwind

# MySQL
mysqladmin drop 'northwind';

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 16

SELECT (very simplified)

▸ Returns a result set SELECT * FROM Employee

▸ For now, assume results are rows

▸ FROM allows one or more Get Relation

elements to be returned from a Result Set (Table)
relation (Rows)

All Elements
(Columns)

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 17

SELECT - Choosing Elements

▸ Elements are returned in the SELECT id, firstname,

order they're asked for lastname
FROM Employee
▸ Good idea to be explicit because

▸ Future schema changes will not

affect results
Choose elements
▸ Developer intent is clear
(Columns) in order
▸ Less I/O

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 18

Quotes in SQL SELECT id FROM Employee

SELECT Id FROM Employee

▸ Usually table names, column
names and keywords are case SELECT 'id' FROM Employee

insensitive SELECT "Id" FROM Employee

▸ 'Single quotes' are used for string

literals > "SELECT id FROM Employee"
MySQL only 1 2 3 4 5 6 7 8 9

▸ "Double quotes" or `backticks` > "SELECT Id FROM Employee"

1 2 3 4 5 6 7 8 9
for words that conflict with SQL > "SELECT 'id' FROM Employee"
id id id id id id id id id
keywords, or when case > "SELECT \"Id\" FROM Employee"
sensitivity is desired ERROR: column "Id" does not exist

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 19

Aliases

▸ The AS keyword can be used to give a table or column a local name or "alias"

▸ Only impacts the current query

SELECT title
p.productname AS title Chai
FROM Chang
Product AS p Aniseed Syrup
Chef Anton's Cajun Seasoning
Chef Anton's Gumbo Mix

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 20

Our Project

▸ ./src/data - data layer code (SQL queries)

▸ ./src/routers - Express routers (HTTP handling)

▸ ./src/db - abstractions around JS database

drivers

▸ ./test - exercise tests

▸ ./views - handlebars templates

▸ ./public - static assets

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 21

Our Project

▸ The SQL tagged template literal can be used to syntax highlight

▸ Get a database client via getDb()

import { getDb } from '../db/utils';

let db = await getDb();

// Retrieve a collection of records
let allEmployees = await db.all('SELECT * FROM Employee');
// Retrieve a single record
let product71 = await db.get('SELECT * FROM Product WHERE id = $1', 71);
// Execute a statement, and return the last inserted ID (if applicable)
let { id } = await db.run('INSERT INTO Customer VALUES(...)');

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 22

Our Project

▸ To setup a database
npm run db:setup:pg npm run db:setup:mysql npm run db:setup:sqlite

▸ Run tests that match a filter

npm run test EX01 npm run test:watch EX01 👀
▸ Run an exercise's tests, and all tests from previous exercises

npm run test:ex 4 npm run test:ex:watch 4 👀

© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 23

Our Project

▸ To run the project on http://localhost:3000

npm run watch 👀

▸ Run tests with a database other than SQLite
DB_TYPE=pg npm run watch DB_TYPE=mysql npm run watch 👀

© Mike.Works, Inc. 2018. All rights reserved

 24

SELECTing columns 1
▸ There are several queries in our app that indiscriminately select all columns in a
table.

▸ Fix this so that we explicitly ask for particular columns we're interested in

▸ Consider cases where this part of the query may be used multiple times
Order Employee Supplier Customer
id id id id
customerid firstname contactname contactname
employeeid lastname companyname companyname
region
shipcity
hiredate
shipcountry npm run test:ex:watch 1
title
shippeddate reportsto
© Mike.Works, Inc. 2018. All rights reserved
 25

SELECTing columns 1
./src/data/customers.js

▸ Get Customers let customers = await getAllCustomers();

./src/data/employees.js

▸ Get Employees let employees = await getAllEmployees();

./src/data/suppliers.js

▸ Get Suppliers let suppliers = await getAllSuppliers();

./src/data/orders.js ./src/data/orders.js

▸ Get Orders await getAllOrders(); await getCustomerOrders('ALFKI');

npm run test:ex:watch 1

© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 26

WHERE

▸ Used to filter result set with a condition

▸ Mind your quotes!

SELECT firstname lastname

firstname, "Janet" "Leverling"
lastname
FROM
Employee
WHERE
lastname = 'Leverling'

Condition

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 27

Conditions

▸ >, <, >=, <=, = age >= 21

▸ Not equal: <> or !=. isDeleted != TRUE

▸ Within a range temperature BETWEEN 68 AND 75

▸ Member of a set companyname IN ('Microsoft', 'LinkedIn')

▸ String ends with email LIKE '%.gov'

▸ String includes summary LIKE '%spicy%'

▸ String length billing_state LIKE '__'

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 28

AND, OR, NOT

▸ Boolean (logic) operators

▸ Operate on conditions

▸ Parens for clarity

SELECT productname
productname
Mishi Kobe Niku
FROM
Product Carnarvon Tigers
WHERE ( Sir Rodney's Marmalade
unitprice > 60
AND unitsinstock > 20
)

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 29

Core Functions

▸ Each database's set of core functions is slightly different (SQLite, MySQL,

PostgreSQL)

▸ Some of these work across databases (lower, max, min, count, substr, etc...)

▸ Fine to use them in a comparison

SELECT productname
FROM Product
WHERE lower(productname) LIKE '%dried%'
▸ ...or as an additional column
SELECT lower(productname) AS label
FROM Product

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 30

Debugging conditions

▸ Conditions can be evaluated directly with a SELECT statement

SELECT '[email protected]' LIKE '%@example.com'; -- TRUE

SELECT '[email protected]' LIKE '%@example.com'; -- FALSE

© Mike.Works, Inc. 2018. All rights reserved

🛠 Tools Demo
© Mike.Works, Inc. 2017. All rights reserved
 32

Filtering with WHERE 2

▸ On the Products page, the "discontinued" and "needs reorder"
links don't work yet.

▸ In getAllProducts, use the options passed to the function to

filter the result set appropriately

▸ Product.discontinued is 1 when the product is

discontinued

▸ We need to reorder when

(Product.unitsinstock + Product.unitsonorder) <
Product.reorderlevel

./src/data/products.js

getAllProducts({ filter: { inventory: 'discontinued' } }); npm run test:ex:watch 2

getAllProducts({ filter: { inventory: 'needs-reorder' } });
© Mike.Works, Inc. 2018. All rights reserved
 33

Filtering with WHERE 2

▸ On the Customers page, filtering the list by
name doesn't work.

▸ In getAllCustomers, use the options

passed to the function filter on
Customer.companyName and
Customer.contactName columns for case-
insensitive values containing the "search filter"

./src/data/customers.js

getAllCustomers({ filter: 'Mike'}); npm run test:ex:watch 2

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 34

Sorting

▸ An ORDER BY clause declares the desired sorting of the result set

▸ Specify sort direction as ASC or DESC

SELECT productname, productname unitprice

unitprice
Côte de Blaye 263.5
FROM Product
ORDER BY unitprice DESC Thüringer 123.79
Rostbratwurst
Mishi Kobe Niku 97
Sir Rodney's 81
Marmalade
Carnarvon Tigers 62.5

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 35

Sorting

▸ An ORDER BY clause declares the desired sorting of the result set

▸ Specify sort direction as ASC or DESC

▸ Multiple sorts and directions may be provided, separated by commas

SELECT productname, unitprice productname unitprice

FROM Product
WHERE unitprice BETWEEN 9.6 AND 11 Jack's New England
9.65
ORDER BY unitprice, productname ASC Clam Chowder
Aniseed Syrup 10
Longlife Tofu 10
Sir Rodney's Scones 10

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 36

Sorting

▸ An ORDER BY clause declares the desired sorting of the result set

▸ Specify sort direction as ASC or DESC

▸ Multiple sorts and directions may be provided, separated by commas

SELECT productname, unitprice productname unitprice

FROM Product
WHERE unitprice BETWEEN 9.6 AND 11 Jack's New England
9.65
ORDER BY unitprice ASC, productname DESC Clam Chowder
Sir Rodney's Scones 10
Longlife Tofu 10
Aniseed Syrup 10

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 37

Limiting

▸ When dealing with large result sets, sometimes we just want the first N rows

▸ A LIMIT clause allows us to specify how many we want

▸ Performance can be much higher, if the database doesn't have to examine

each record for sorting purposes

SELECT productname, unitprice productname unitprice

FROM Product
ORDER BY unitprice DESC Côte de Blaye 263.5
LIMIT 3 Thüringer
123.79
Rostbratwurst
Mishi Kobe Niku 97

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 38

Offsetting

▸ Means "Start with the Nth result"

▸ We can paginate over a set of results using a LIMIT and OFFSET

LIMIT
OFFSET

SELECT productname, productname unitprice

unitprice
FROM Product Sir Rodney's Marmalade 81
ORDER BY unitprice DESC
LIMIT 3 Carnarvon Tigers 62.5
OFFSET 3
Raclette Courdavault 55

© Mike.Works, Inc. 2018. All rights reserved

 39

Sorting and Paging 3

▸ Clicking on a column header should sort
orders.

▸ Update getAllOrders /
getCustomerOrders to take the opts.sort
and opts.order options into account

▸ Fix getCustomerOrders while you're at it, so

that it returns only the appropriate orders

./src/data/orders.js

getAllOrders({ sort: 'shippeddate', order: 'desc'});

getAllOrders({ sort: 'customerid', order: 'asc' }); npm run test:ex:watch 3

© Mike.Works, Inc. 2018. All rights reserved

 40

Sorting and Paging 3

▸ Pagination buttons are already passing values
into getAllOrders / getCustomerOrders
that can be used to offset and limit the result
set (opts.page and opts.perPage).

▸ Default sorts:
getAllOrders by id, desc
getCustomerOrders by shippeddate asc

./src/data/orders.js

getAllOrders({ page: 3, perPage: 25 });

getCustomerOrders('ALFKI', { page: 5, perPage: 10 }); npm run test:ex:watch 3

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 41

Relationships

▸ Reminder: this is not where "relational" comes from

Order belongs to Customer
ORDER
Customer has many Orders
customerid
employeeid CUSTOMER

Order belongs to Employee

Employee has many Orders
EMPLOYEE

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 42

Relationships

▸ Reminder: this is not where "relational" comes from

ORDER

id customerid employeeid
CUSTOMER
10625 ANATR 3
10835 ALFKI 1 id companyname contactname
12617 ANTON 2 ALFKI Alfreds Maria Anders
ANATR Futterkiste
Ana Trujillo Ana Trujillo
EMPLOYEE ANTON Emparedados
Antonio y Antonio Moreno
Moreno
Taquería
id firstname lastname
1 Nancy Davolio
2 Andrew Fuller
3 Janet Leverling

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 43

Resolving Relationships

▸ Currently we have a result set of objects

that look like this
[{}, {}, {
id: 10257,
customerid: 'HILAA',
employeeid: 4,
shipcity: 'San Cristóbal',
shipcountry: 'Venezuela'
}, ...]

▸ We want to replace customerid with the

customer's name, but that lives in another
table
© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 44

JOIN: Assemble tables together

▸ The JOINing is done on a related column between the tables

▸ Four types of join, that differ in terms of how "incomplete" matches are handled

▸ Simplified example
ORDER CUSTOMER
No id customerid amount id name
Customer 1 1 99.00 1 Mike North
2 1 212.00 2 Mark Grabanski
3 3 14.50 3 Lisa Huang-North
4 12 1000.00

© Mike.Works, Inc. 2018. All rights reserved No Orders

 SQL FUNDAMENTALS 45

ORDER CUSTOMER

INNER JOIN id customerid Amount id name

1 1 99.00 1 Mike North
2 1 212.00 2 Mark Grabanski
3 3 14.50 3 Lisa Huang-North
4 ❌
12 1000.00

▸ Only rows that have "both ends" of the match will be selected

SELECT * id customerid amount id name

FROM CustomerOrder AS o 1 1 99.00 1 Mike North
INNER JOIN Customer AS c
ON o.customerid = c.id 2 1 212.00 1 Mike North
3 3 14.50 3 Lisa Huang-North

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 46

ORDER CUSTOMER

LEFT JOIN id customerid Amount id name

1 1 99.00 1 Mike North
2 1 212.00 2 Mark Grabanski
3 3 14.50 3 Lisa Huang-North
4 ❌
12 1000.00

▸ Rows from LEFT of join will be selected no matter what

SELECT * id customerid amount id name

FROM CustomerOrder AS o 1 1 99.00 1 Mike North
LEFT JOIN Customer AS c
ON c.id = o.customerid 2 1 212.00 1 Mike North
3 3 14.50 3 Lisa Huang-North
4 12 1000.00

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 47

ORDER CUSTOMER

RIGHT JOIN id customerid Amount id name

1 1 99.00 1 Mike North
2 1 212.00 2 Mark Grabanski
3 3 14.50 3 Lisa Huang-North
4 ❌
12 1000.00

▸ Rows from RIGHT of join will be selected no matter what

SELECT * id customerid amount id name

FROM CustomerOrder AS o 1 1 99.00 1 Mike North
RIGHT JOIN Customer AS c
ON o.customerid = c.id 2 1 212.00 1 Mike North
3 3 14.50 3 Lisa Huang-North
2 Mark Grabanski

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 48

ORDER CUSTOMER

FULL JOIN id customerid Amount id name

1 1 99.00 1 Mike North
2 1 212.00 2 Mark Grabanski
3 3 14.50 3 Lisa Huang-North
4 ❌
12 1000.00

▸ All rows in both tables are selected

SELECT * id customerid amount id name

FROM CustomerOrder AS o 1 1 99.00 1 Mike North
FULL JOIN Customer AS c
ON o.customerid = c.id 2 1 212.00 1 Mike North
3 3 14.50 3 Lisa Huang-North
2 Mark Grabanski
4 12 1000.00
Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 49

Outer Joins

▸ LEFT, RIGHT and FULL JOIN are sometimes referred to as OUTER joins

▸ In general, an OUTER join allows for the possibility that one or more rows may
be partially empty, due to not having a corresponding match in the other table

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 50

Which type of JOIN would we want here?

© Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 51

JOIN - Selecting Columns

▸ SELECT * is generally a bad idea, and it's particularly dangerous in the

context of a JOIN

▸ Choose columns with care, and alias any duplicates

SELECT o.id,
* id customerid amount
amount id name name
FROM CustomerOrder
o.customerid,
AS o 1 9.65
1 99.00
99.00 1 Mike Mike
NorthNorth
INNER JOIN
o.amount,
Customer AS c
ON o.customerid
c.name = c.id 2 10
1 212.00
212.00 1 Mike Mike
NorthNorth
FROM CustomerOrder AS o 3 10
3 14.50
14.50 Lisa
3 Lisa
Huang-North
Huang-North
INNER JOIN Customer AS c
ON o.customerid = c.id

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 52

Querying across tables 4

▸ There are a few places where we're showing Ids

▸ Product list: Suppliers, Categories

▸ Order list: Customers and Employees

▸ Order: Customer, Employee and Products

▸ Alter the relevant queries to include one or more JOINs so that these Ids are
replaced with the relevant human-readable name

npm run test:ex:watch 4

© Mike.Works, Inc. 2018. All rights reserved

 53

Querying across tables 4

▸ Product list: replace
Supplier and
Category Ids

▸ New join columns:

suppliername,
categoryname

./src/data/products.js

getAllProducts();
npm run test:ex:watch 4

© Mike.Works, Inc. 2018. All rights reserved

 54

Querying across tables 4

▸ Order list: replace
Customer and
Employee Ids

▸ New join columns:

customername,
employeename

./src/data/orders.js

getAllOrders(); npm run test:ex:watch 4

getCustomerOrders();
© Mike.Works, Inc. 2018. All rights reserved
 55

Querying across tables 4

▸ Order: replace Customer and
Employee Ids

▸ OrderDetail: replace Product Ids

▸ New join columns:

customername, employeename,
productname

./src/data/orders.js

getOrder(12149);
getOrderDetails(12149); npm run test:ex:watch 4

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 56

Aggregate Functions

▸ Perform a calculation on a set of values, to

arrive at a single value

▸ Each RDBMS has its own functions, but they

all generally have a foundational set

▸ SUM - Adds values together

▸ COUNT - Counts the number of values

SELECT sum((1 - discount) *
unitprice * quantity)
▸ MIN/MAX - Finds the min/max value AS revenue
FROM OrderDetail;
▸ AVG - Calculates the mean of the values
© Mike.Works, Inc. 2018. All rights reserved
 SQL FUNDAMENTALS 57

Aggregate Functions and GROUP BY

▸ Imagine we have this JOIN query, and we want total amount per customer

▸ This involves finding the relevant rows to consolidate and summing them up. A
relational DB is perfect for these tasks!

SELECT c.id, id name amount

c.name, o.amount
1 Mike North 99.00
FROM CustomerOrder AS o 1 Mike North 212.00
sum
INNER JOIN Customer AS c
ON o.customerid = c.id 3 Lisa Huang-North 14.50

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 58

Aggregate Functions and GROUP BY

▸ We can specify which rows to consolidate via a GROUP BY clause

▸ Then, we can SELECT an aggregate function, describing what to do on the

"groups" of rows

SELECT c.id, id name amount

sum
c.name,
sum(o.amount) 1 Mike North 311.00
99.00
FROM CustomerOrder AS o 1
3 Mike
Lisa North
Huang-North 212.00
14.50
INNER JOIN Customer AS c
ON o.customerid = c.id 3 Lisa Huang-North 14.50
GROUP BY c.id

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 59

Aggregate Functions and GROUP BY

CUSTOMER
id name
▸ Let's imagine what we could do with GROUP 1 Mike North
BY in this example 2 Mark Grabanski
3 Lisa Huang-North
▸ Total spent by customerid
ORDER
▸ Total spent by month
id customerid amount month

▸ Total spent by customerid, by month 1 1 99.00 5

2 1 212.00 3
3 3 14.50 3
4 12 1000.00 11
5 1 199.00 3

© Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 60

Aggregate Functions and GROUP BY

CUSTOMER
id name
▸ Total spent by customerid 1 Mike North
2 Mark Grabanski
SELECT customerid,
sum(amount) 3 Lisa Huang-North
FROM CustomerOrder
GROUP BY customerid ORDER
id customerid amount month
id sum
1 1 99.00 5
1 510.00
2 1 212.00 3
12 1000.00
3 3 14.50 3
3 14.50
4 12 1000.00 11
5 1 199.00 3

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 61

Aggregate Functions and GROUP BY

CUSTOMER
id name
▸ Total spent by month 1 Mike North
2 Mark Grabanski
SELECT month,
sum(amount) 3 Lisa Huang-North
FROM CustomerOrder
GROUP BY month ORDER
id customerid amount month
month sum
1 1 99.00 5
3 425.50
2 1 212.00 3
5 99.00
3 3 14.50 3
11 1000
4 12 1000.00 11
5 1 199.00 3

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 62

Aggregate Functions and GROUP BY

CUSTOMER
id name
▸ Total spent by customerid, by month 1 Mike North
2 Mark Grabanski
SELECT customerid,
month, 3 Lisa Huang-North
sum(amount)
FROM CustomerOrder
GROUP BY month, customerid ORDER
id customerid amount month
customerid month sum 1 1 99.00 5
1 3 411.00 2 1 212.00 3
1 5 99.00 3 3 14.50 3
3 3 14.50 4 12 1000.00 11
12 11 1000.00 5 1 199.00 3

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 63

Aggregate Functions and GROUP BY

CUSTOMER
id name
▸ What you SELECT, aggregate and GROUP BY 1 Mike North
must agree 2 Mark Grabanski
3 Lisa Huang-North
SELECT month,
customerid,
sum(amount) ORDER
FROM CustomerOrder id customerid amount month
GROUP BY month 1
? ?
1 ? 99.00 5
2 1 212.00 3
column
"CustomerOrder.customerid" 3 3 14.50 3
must appear in the GROUP BY 4 12 1000.00 11
clause or be used in an
aggregate function 5 1 199.00 3

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 64

Aggregate Functions

▸ Different for SQLite, PostgreSQL, MySQL, etc...

▸ A platform-specific one: string concatenation (PG: string_agg, SQLite:

group_concat)
pg

SELECT
c.id, c.name, id name orders
string_agg(CAST(o.id AS text), ', ') AS orders
FROM Customer AS c 1 Mike North "1, 2, 5"
LEFT JOIN CustomerOrder AS o 2 Mark Grabanski
ON o.customerid = c.id
GROUP BY c.id, c.name 3 Lisa Huang-North "3"
ORDER BY c.id

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 65

Concatenating In Aggregate

▸ Different for SQLite, PostgreSQL, MySQL, etc...

▸ Most common functions work everywhere: SUM, AVG, MIN, MAX, COUNT

▸ A platform-specific one: string concatenation (PG: string_agg, SQLite: group_concat)

pg

SELECT
c.id, c.name, id name orders
string_agg(CAST(o.id AS text), ', ') AS orders
FROM Customer AS c 1 Mike North "1, 2, 5"
LEFT JOIN CustomerOrder AS o 2 Mark Grabanski
ON o.customerid = c.id Convert integer
GROUP BY c.id to string 3 Lisa Huang-North "3"
ORDER BY c.id Concatenate strings

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 66

Concatenating In Aggregate

▸ Concatenating strings together can be done in all of the databases we're

studying

MySQL

SELECT group_concat(productname ORDER BY productname DESC SEPARATOR ', ')

FROM Product;

SQLite pg

SELECT group_concat(productname, ', ') SELECT string_agg(productname, ', ')

FROM Product; FROM Product;

© Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 67

Filtering grouped results

SELECT month,
▸ A WHERE clause is used to examine sum(amount) AS month_sum
individual rows. It cannot be used with an FROM CustomerOrder
aggregate function WHERE month_sum >= 300
GROUP BY month

column "month_sum" does not

exist

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 68

Filtering grouped results

SELECT month,
▸ A WHERE clause is used to examine sum(amount) AS month_sum
individual rows. It cannot be used with an FROM CustomerOrder
aggregate function WHERE month_sum >= 300
GROUP BY month
▸ A HAVING clause is used to examine groups, column "month_sum" does not
and include or exclude them from the result exist
set
SELECT month,
▸ It's totally fine to use both in the same query. sum(amount) AS month_sum
FROM CustomerOrder
WHERE to disregard irrelevant rows
GROUP BY month
HAVING to disregard irrelevant groups HAVING sum(amount) >= 300

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 69

Subquery

▸ A SELECT query can be nested in another query

▸ Useful when results from the database are needed in order to define "outer"
query

▸ Inner query cannot mutate data SELECT *

FROM Product
▸ Sometimes your RDBMS doesn't WHERE categoryid=(
support ORDER BY within the SELECT id
FROM Category
subquery WHERE categoryname='Beverages'
);

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS

If you want to solve exercises using more than one database

switch (process.env.DB_TYPE)
{
case 'mysql' /*...*/:
break;
case 'pg' /*...*/:
break;
case 'sqlite':
default:
/*...*/
break;
}

© Mike.Works, Inc. 2018. All rights reserved

 71

Aggregate Functions 5
▸ Use aggregate functions and GROUP BY to calculate

▸ Order: subtotal

▸ Supplier: productlist (concatenate Product.productname strings)

▸ Employee: ordercount

▸ Customer: ordercount

npm run test:ex:watch 5

© Mike.Works, Inc. 2018. All rights reserved

 72

Aggregate Functions 5
▸ Order: subtotal
./src/data/orders.js

getOrderDetails(12149);

npm run test:ex:watch 5

© Mike.Works, Inc. 2018. All rights reserved

 73

Aggregate Functions 5
▸ Supplier: productlist
(concatenate Product.productname
strings)

./src/data/suppliers.js

getAllSuppliers();

npm run test:ex:watch 5

© Mike.Works, Inc. 2018. All rights reserved

 74

Aggregate Functions 5
▸ Employee: ordercount
./src/data/employees.js

getAllEmployees();

npm run test:ex:watch 5

© Mike.Works, Inc. 2018. All rights reserved

 75

Aggregate Functions 5
▸ Customer: ordercount
./src/data/customers.js

getAllCustomers();

npm run test:ex:watch 5

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 76

Creating Records

▸ Add new records to a table via an INSERT INTO statement, indicating the
table to insert into

▸ This must be followed by a VALUES expression, with an array containing the

corresponding data, in the order aligned with the table's columns

INSERT INTO Customer

VALUES (8424, 'Tanner Thompson', 'Likes scotch');

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 77

Creating Records

▸ Sometimes, columns can have default (or other auto-generated) values and we
need only provide data for SOME columns.

▸ Specify the column names in your INSERT INTO statement, and ensure your
VALUES array is in the respective order.

INSERT INTO Customer (name, notes)

VALUES ('Tanner Thompson', 'Likes scotch');

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 78

SQL Injection

▸ Anytime new data is added to your database, it must be appropriately

sanitized

INSERT INTO Customer (name, notes)

VALUES ('Tanner ', 'Tanner notes')

Name '); DROP TABLE Customer; --

Tanner

▸ Most of the time your SQL driver (library) helps you out
Sanitized before
db.exec(`INSERT INTO Customer (name, notes) insertion
VALUES ($1, $2)`, ['Tanner', 'Tanner notes']);

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 79

Deleting Records

▸ Delete a table completely, including column definitions, indices, etc...

DROP TABLE Customer;

▸ Delete all records from a table, while leaving the structure and indices intact
DELETE FROM Customer;

▸ One or more rows can be deleted while leaving the rest intact, via a WHERE
clause
DELETE FROM Customer
WHERE name = 'Juicero';

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 80

Our Project

▸ The SQL tagged template literal can be used to syntax highlight

▸ Get a database client via getDb()

import { getDb } from '../db/utils';

let db = await getDb();

// Retrieve a collection of records
let allEmployees = await db.all('SELECT * FROM Employee');
// Retrieve a single record
let product71 = await db.get('SELECT * FROM Product WHERE id = $1', 71);
// Execute a statement, and return the last inserted ID (if applicable)
let { id } = await db.run('INSERT INTO Customer VALUES(...)');

© Mike.Works, Inc. 2018. All rights reserved

 81

Creating and Destroying 6

▸ The ❌ button on each row of the order list
ultimately calls deleteOrder.

▸ The "New" button on the order page ultimately

calls createOrder.

▸ Implement the queries in these two functions so

that orders are appropriately created and
removed from the database.
./src/data/orders.js

deleteOrder(12141); npm run test:ex:watch 6

© Mike.Works, Inc. 2018. All rights reserved

 82

Creating and Destroying 6

./src/data/orders.js

createOrder(
{
employeeid: 3,
customerid: 'ALFKI',
shipcity: 'Minneapolis, MN',
shipaddress: '60 South 6th St Suite 3625',
shipname: 'Frontend Masters',
shipvia: 1,
shipregion: 1,
shipcountry: 'USA',
shippostalcode: '455402',
requireddate: '2018-03-22T23:38:08.410Z',
freight: 2.17
},
[ { productid: 17, unitprice: 4.11, quantity: 4, discount: 0 },
{ productid: 11, unitprice: 3.37, quantity: 1, discount: 0.10 }]
);
© Mike.Works, Inc. 2018. All rights reserved npm run test:ex:watch 6
SQL FUNDAMENTALS 83

Transactions

▸ Sometimes we need to perform multi-statement operations

(i.e., create an Order and several OrderDetail items)

▸ What would happen if the Order was created, but something went wrong
with creating one of the OrderDetails?

INSERT INTO CustomerOrder ...; -- create new order

INSERT INTO OrderDetail ...; -- add order item to it

💥 INSERT INTO OrderDetail ...;
INSERT INTO OrderDetail ...;

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 84

Transactions

▸ Surround multiple statements with BEGIN and COMMIT to include them in a

transaction

✅
BEGIN;
INSERT INTO CustomerOrder ...; -- create new order

INSERT INTO OrderDetail ...; -- add order item to it

Transaction INSERT INTO OrderDetail ...;
INSERT INTO OrderDetail ...;

COMMIT; -- save!

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 85

Transactions

▸ Surround multiple statements with BEGIN and COMMIT to include them in a

transaction

BEGIN;
INSERT INTO CustomerOrder ...; -- create new order ❌
Transaction ROLLBACK; -- revert!

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 86

Transaction - ACID

▸ Principles that guarantee data consistency, even when things go wrong

✅▸ Atomic - All of the operations are treated as a single "all or nothing" unit

✅▸ Consistent - Changes from one valid state to another (never halfway)

▸ Isolated - How visible are particulars of a transaction before it's complete?

When? To whom?

✅▸ Durable - Once it completes, the transaction's results are stored

permanently in the system

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 87

Transaction Isolation Levels

▸ The degree to which other queries to the database can see, or be influenced by
an in-progress transaction

▸ There are levels of isolation, and we as developers have a choice

▸ Often this is a concurrency vs. safety and consistency trade-off

▸ First described in the ANSI SQL-92 standard

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 88

ORDER

Read Phenomena: Dirty Reads id customerid amount month

19 1 99.00 5
20 1 212.00 3
BEGIN; 21 3 14.50 3

SELECT month FROM CustomerOrder WHERE id = 21; 3

BEGIN;

UPDATE CustomerOrder SET month=4 WHERE id = 21;

SELECT month FROM CustomerOrder WHERE id = 21; 4

ROLLBACK;

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 89

ORDER

Read Phenomena: Dirty Reads id customerid amount month

19 1 99.00 5
20 1 212.00 3
BEGIN; 21 3 14.50 3

SELECT month FROM CustomerOrder WHERE id = 21; 3

BEGIN;

UPDATE CustomerOrder SET month=4 WHERE id = 21;

SELECT month FROM CustomerOrder WHERE id = 21; 4

ROLLBACK;

COMMIT;

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 90

Transaction Isolation Levels

▸ Read Uncommitted - Dirty reads are possible.

▸ Read Committed - Write locks are obtained across the whole transaction, read
locks are obtained only for the duration of a SELECT. Dirty reads are not
possible

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 91

ORDER

Read Phenomena: Non-Repeatable Reads id customerid amount month

19 1 99.00 5
20 1 212.00 3
BEGIN; 21 3 14.50 3

SELECT month FROM CustomerOrder WHERE id = 21; 3

BEGIN;

UPDATE CustomerOrder SET month=4 WHERE id = 21;

COMMIT;

SELECT month FROM CustomerOrder WHERE id = 21; 4

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 92

ORDER

Read Phenomena: Non-Repeatable Reads id customerid amount month

19 1 99.00 5
20 1 212.00 3
BEGIN; 21 3 14.50 3

SELECT month FROM CustomerOrder WHERE id = 21; 3

BEGIN;

UPDATE CustomerOrder SET month=4 WHERE id = 21;

COMMIT;

SELECT month FROM CustomerOrder WHERE id = 21; 4

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 93

Transaction Isolation Levels

▸ Read Uncommitted - Dirty reads are possible. Sequence of writes is guaranteed

▸ Read Committed - Write locks are obtained across the whole transaction, read
locks are obtained only for the duration of a SELECT. Dirty reads are not
possible

▸ Repeatable Read - Read and write locks are obtained across the whole
transaction.

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 94

ORDER

Read Phenomena: Phantom Reads id customerid amount month

19 1 99.00 5
20 1 212.00 3
BEGIN; 21 3 14.50 3

SELECT sum(amount) FROM CustomerOrder

WHERE month BETWEEN 19 AND 23; 226.50 BEGIN;

INSERT INTO CustomerOrder(customer_id, amount, month)

VALUES (22, 40000, 3);
COMMIT;

SELECT sum(amount) FROM CustomerOrder

WHERE month BETWEEN 19 AND 23; 40226.50
COMMIT;

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 95

ORDER

Read Phenomena: Phantom Reads id customerid amount month

19 1 99.00 5
20 1 212.00 3
BEGIN; 21 3 14.50 3

SELECT sum(amount) FROM CustomerOrder

WHERE month BETWEEN 19 AND 23; 226.50 BEGIN;

INSERT INTO CustomerOrder(customer_id, amount, month)

VALUES (22, 40000, 3);
COMMIT;

SELECT sum(amount) FROM CustomerOrder

WHERE month BETWEEN 19 AND 23; 40226.50
COMMIT;

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 96

Transaction Isolation Levels

▸ Read Uncommitted - Dirty reads are possible. Sequence of writes is guaranteed

▸ Read Committed - Write locks are obtained across the whole transaction, read
locks are obtained only for the duration of a SELECT. Dirty reads are not
possible

▸ Repeatable Read - Read and write locks are obtained across the whole
transaction.

▸ Serializable - Read and write locks are obtained on selected data, range locks
are obtained for WHERE clauses.

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 97

Transaction Isolation Levels

▸ Read Uncommitted - Dirty reads are possible. Sequence of writes is guaranteed

▸ Read Committed - Write locks are obtained across the whole transaction, read
locks are obtained only for the duration of a SELECT. Dirty reads are not possible

▸ Repeatable Read - Read and write locks are obtained across the whole
transaction.

▸ Serializable - Read and write locks are obtained on selected data, range locks are
obtained for WHERE clauses.

▸ Externally Consistent - See: Google Spanner

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 98

Databases & Storage Engines

▸ PostgreSQL - MVCC storage engine, and possible changes in v12.

▸ MySQL

▸ InnoDB - supports transactions, and is the best "default choice"

▸ MyISAM - no transactions, faster and smaller in some cases

▸ Others...

▸ SQLite

© Mike.Works, Inc. 2018. All rights reserved

 99

Transactions 7
▸ Creating an order involves multiple queries. If some succeed and some fail,
we'd be left in an inconsistent state!

▸ Use a transaction to make this multi-step operation "all or nothing", using best
practices to handle errors appropriately

▸ INSERT INTO must be a statement by its self

▸ Use try/catch

npm run test:ex:watch 7

© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 100

Updating Records

▸ One or more records may be updated at a time

SELECT id, unitprice, quantity, discount id unitprice quantity discount

FROM OrderDetail 10793/41 9.65 14 0
WHERE orderid = 10793 10793/52 7 8 0

UPDATE OrderDetail
SET discount = 0.15
WHERE orderid = 10793

SELECT id, unitprice, quantity, discount id unitprice quantity discount

FROM OrderDetail 10793/41 9.65 14 0.15
WHERE orderid = 10793 10793/52 7 8 0.15

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 101

Updating Records

▸ Sometimes we want to use current values to calculate new ones

UPDATE OrderDetail
SET discount = 0.15,

x4
x2
x3
unitprice = (1 - 0.15) * unitprice
WHERE orderid = 10793

SELECT id, unitprice, quantity, discount id unitprice quantity discount

FROM OrderDetail 10793/41 5.037360
6.972125
5.926306
8.2025 14 0.15
WHERE orderid = 10793 10793/52 3.654044
5.0575
4.298875
5.95 8 0.15

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 102

Updating Records Can be re-applied without further altering the value

▸ Often we want an update to be idempotent

UPDATE OrderDetail
SET discount = 0.15,
unitprice = (1 - 0.15) * ( SELECT unitprice
FROM Product

WHERE orderid = 10793

WHERE id = OrderDetail.productid )
x4
x2
x3
SELECT id, unitprice, quantity, discount id unitprice quantity discount
FROM OrderDetail 10793/41 6.972125
9.65 14 0.15
WHERE orderid = 10793 10793/52 5.0575
7 8 0.15

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 103

Insert or Update

▸ Many RDBMS’s allow for some sort of “Insert or Update” command

▸ Widely different implementations and usage — definitely not standard SQL

▸ PostgreSQL (10.x) is the most flexible

▸ SQLite (3.22.x) only allows “Insert or Replace”

▸ MySQL (4.1) “On conflict”

© Mike.Works, Inc. 2018. All rights reserved

 104

Updating Records 8
▸ Editing an order ultimately calls updateOrder.

▸ Implement the query, using a transaction, such

that changes to the order and all order items
are persisted appropriately

npm run test:ex:watch 8

© Mike.Works, Inc. 2018. All rights reserved

 105

Updating Records 8
./src/data/orders.js

updateOrder(12345, {
employeeid: 3,
customerid: 'ALFKI',
shipcity: 'Minneapolis, MN',
shipaddress: '60 South 6th St Suite 3625',
shipname: 'Frontend Masters',
shipvia: 1,
shipregion: 1,
shipcountry: 'USA',
shippostalcode: '455402',
requireddate: '2018-03-22T23:38:08.410Z',
freight: 2.17
},
[ { id: '12345/1', productid: 17, unitprice: 4.33, quantity: 4, discount: 0 },
{ id: '12345/2', productid: 11, unitprice: 2.18, quantity: 1, discount: 0.1 }
]
);

© Mike.Works, Inc. 2018. All rights reserved npm run test:ex:watch 8

SQL FUNDAMENTALS 106

Changing our schema

▸ We usually have at least one DB per

environment (dev, test, staging, prod)

▸ Apps that connect to a DB have a hard

dependency on a particular schema

▸ Having reproducible versions includes the

database!

▸ A happy state: a given git commit (or set of

commits) = reproducible version

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 107

DB Version Migrations
Migrations
Setup & Seed
▸ Incremental changes to a database
ADD NEW COLUMN TO
▸ Ideally, reversible
CUSTOMER TABLE
▸ May start with an initial “seed” script
CREATE A NEW
▸ “Applied” migrations are stored in a BILLINGINFO TABLE
special DB table
INCREASE THE WIDTH
▸ Ideally, backwards compatible DECREASE the width
OF THE “NOTES” of
the “Notes” Column in
COLUMN IN THE
the Account table
ACCOUNT TABLE

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 108

Migrations in Node

▸ A simple and popular solution: db-migrate/node-db-migrate

▸ Check out your project’s ./migrations folder

▸ Migrations are named, and prefixed by a timestamp (chronological sorting)

▸ Can be purely JS based, or JS that runs SQL scripts (recommended)

▸ Uses your ./database.json to connect to databases of various types

▸ Env var DATABASE_URL takes precedence

DATABASE_URL=postgres:!//mike:[email protected]:5432/customer_db

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 109

Migrations in Node - Creating

▸ Create a new migration using the CLI tool

./node_modules/.bin/db-migrate create MyMigration --sql-file

▸ We have a NPM script set up for our project (will always !--sql-file and
generates per RDBMS migration sql scripts)

npm run db:migrate:create MyMigration

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 110

Migrations in Node - Running Forward

▸ Attempt to run all migrations not yet applied to a database

./node_modules/.bin/db-migrate -e pg up

▸ We have a NPM script set up for our project

npm run db:migrate:pg up

npm run db:migrate:sqlite up

npm run db:migrate:mysql up

© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 111

Migrations in Node - Rolling back

▸ Roll back one migration on a given database

./node_modules/.bin/db-migrate -e pg down

▸ We have a NPM script set up for our project

npm run db:migrate:pg down

npm run db:migrate:sqlite down

npm run db:migrate:mysql down

© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 112

Indices - A memory-for-time trade-off

▸ Extra bookkeeping performed on records for quick retrieval later

Table: People Index: People(last_name)

id first_name last_name value records

A
345 Rick Sanchez Grabasnki 347
346 Mike North North 346
347 Mark Grabanski Sanchez 345
348 Morty Smith Smith 348, 349, 350, 351
349 Jerry Smith Z
350 Summer Smith
351 Beth Smith

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 113

Indices - A space-for-time trade-off

▸ Try grabbing OrderDetails by orderid

SELECT * FROM OrderDetail WHERE orderid = 10793 55ms

▸ We can ask our database to describe the work required to get these results
EXPLAIN SELECT * FROM OrderDetail WHERE orderid = 10793

Gather
(cost = 1000.00..9426.37 rows = 44 width = 33)
Workers Planned: 2
-> Parallel Seq Scan on orderdetail
(cost = 0.00..8421.97 rows = 18 width = 33)
Filter: (orderid = 10793)
© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 114

Indices - A space-for-time trade-off

▸ Let’s create an index on this column

CREATE INDEX orderdetail_oid ON OrderDetail(orderid)

SELECT * FROM OrderDetail WHERE orderid = 10793 28ms

▸ We can ask our database to describe the work required to get these results
EXPLAIN SELECT * FROM OrderDetail WHERE orderid = 10793

Index Scan using orderdetail_oid on orderdetail

(cost = 0.42..9.20 rows = 44 width = 33)
Index Cond: (orderid = 10793)

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 115

Indices - A space-for-time trade-off

▸ Indices can be created on multiple columns

CREATE INDEX orderdetail_order_customer
ON OrderDetail(orderid, customerid)

▸ When creating indices, the goal is to reduce (possibly eliminate) the exhaustive
searching done for common or perf-sensitive queries

© Mike.Works, Inc. 2018. All rights reserved

 116

Migrations and Indices 9

▸ In Exercise 4, we started combining tables together with JOIN
statements, but this has hurt performance

▸ Identify specific opportunities for adding indices to the database, with

the specific goal of reducing database time on

▸ The order list

▸ The employee list

▸ The product list

▸ The order page

▸ Create a new database migration to add these indices to the database

npm run test:ex:watch 9
© Mike.Works, Inc. 2018. All rights reserved
 117

Migrations and Indices 9

▸ Create a new migration
npm run db:migrate:create AddEx9Indices

▸ Run the migration forward

DB_TYPE=pg npm run db:migrate:up

▸ Roll the migration back

DB_TYPE=pg npm run db:migrate:down

npm run test:ex:watch 9

© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS 118

Column Constraints - Types

▸ One of the ways we can constrain values placed in particular columns is by

types

▸ Names and properties of these types may vary from system to system

▸ Choice of column type impacts performance, and options for indexing and
filtering

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 119

Column Constraints - Create a Table

▸ Creating a table gives us the opportunity to define columns as name/type pairs

CREATE TABLE UserPreferences (

id INT PRIMARY KEY, -- an integer
favorite_color CHAR(6), -- Exactly 6 characters
age SMALLINT, -- an "small" integer
birth_date DATE, -- date (may or may not include time)
notes VARCHAR(255), -- up to 255 characters
is_active BOOLEAN -- boolean
);

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 120

Column Constraints - NOT NULL

▸ We can forbid null values for one or more columns by adding NOT NULL

CREATE TABLE UserPreferences (

id INT PRIMARY KEY NOT NULL, -- required
favorite_color CHAR(6),
age SMALLINT,
birth_date DATE NOT NULL, -- required
notes VARCHAR(255),
is_active BOOLEAN NOT NULL -- required
);

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 121

Column Constraints - Updating a Table’s Definition

▸ Most RDBMS’s allow us to alter tables and column definitions

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 122

Column Constraints - UNIQUE

▸ A uniqueness constraint may be added to a column via UNIQUE

CREATE TABLE UserAccount (

email VARCHAR(255) UNIQUE NOT NULL,
name VARCHAR(255)
);

CREATE UNIQUE INDEX u_email ON UserAccount(email);

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS 123

Column Constraints - PRIMARY KEY

▸ A combination of a NOT NULL and UNIQUE constraint

CREATE TABLE UserAccount (

email VARCHAR(255) PRIMARY
UNIQUE NOT
KEY,
NULL,
name VARCHAR(255)
);

▸ Often you’ll have one identity column in each that serves as the “id” for each
addressable record

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS

Column Constraints - PRIMARY KEY

▸ Often we want these IDs to

auto-increment

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS

Column Constraints - PRIMARY KEY

MySQL
▸ Often we want these IDs to CREATE TABLE UserAccount (
auto-increment id INT PRIMARY KEY
KEY,AUTO_INCREMENT,
name VARCHAR(255)
▸ In MySQL we have to specify );
AUTO_INCREMENT or we’ll
get an error INSERT INTO UserAccount(name)
VALUES('foo')

✅
Field 'id' doesn't have a
default value

© Mike.Works, Inc. 2018. All rights reserved

SQL FUNDAMENTALS

Column Constraints - PRIMARY KEY

▸ In PostgreSQL we’ll create a SEQUENCE as our source of incrementing values

Postgres

CREATE SEQUENCE UserAccount_id_seq;

CREATE TABLE UserAccount (
id INTEGER PRIMARY KEY DEFAULT nextval('UserAccount_id_seq'),
name VARCHAR(255)
);
ALTER SEQUENCE UserAccount_id_seq OWNED BY UserAccount.id;

INSERT INTO UserAccount(name) VALUES('foo')

✅
© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS

Column Constraints - PRIMARY KEY

▸ In PostgreSQL we’ll create a SEQUENCE as our source of incrementing values

▸ The SERIAL type does this for us automatically!

Postgres

CREATE TABLE UserAccount (

id SERIAL PRIMARY KEY,
name VARCHAR(255)
);

INSERT INTO UserAccount(name) VALUES('foo')

✅
© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS

Column Constraints - PRIMARY KEY

▸ SQLite has an implicit rowid column, unless we explicitly opt out

▸ We run into id reuse problems when we delete rows and add new ones
SQLite

CREATE TABLE UserAccount (name TEXT); rowid name

INSERT INTO UserAccount(name) 1 foo
VALUES ("foo"), ("bar"), (“baz”); 2 bar
DELETE FROM UserAccount 3 baz
WHERE name="baz"; 3 new1
INSERT INTO UserAccount(name) 4 new2
VALUES ("new1"), ("new2")
SELECT rowid, name FROM UserAccount
© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS

Column Constraints - PRIMARY KEY

▸ Even if we create an id column and use PRIMARY KEY, we’ll still have the same
problem. It’s just an alias for rowid!

CREATE TABLE UserAccount ( SQLite

id INTEGER PRIMARY KEY,
KEY
name TEXT ); id name
INSERT INTO UserAccount(name) 1 foo
VALUES ("foo"), ("bar"), (“baz”); 2 bar
DELETE FROM UserAccount 3 baz
WHERE name="baz"; 3 new1
INSERT INTO UserAccount(name) 4 new2
VALUES ("new1"), ("new2")
SELECT * FROM UserAccount
© Mike.Works, Inc. 2018. All rights reserved
SQL FUNDAMENTALS

Column Constraints - PRIMARY KEY

▸ Only by using AUTOINCREMENT can we avoid reusing ids

CREATE TABLE UserAccount ( SQLite

id INTEGER PRIMARY KEY AUTOINCREMENT,
AUTOINCREMENT
name TEXT ); id name
INSERT INTO UserAccount(name) 1 foo
VALUES ("foo"), ("bar"), (“baz”); 2 bar
DELETE FROM UserAccount 3 baz
WHERE name="baz"; 4 new1
INSERT INTO UserAccount(name) 5 new2
VALUES ("new1"), ("new2")
SELECT * FROM UserAccount
© Mike.Works, Inc. 2018. All rights reserved
 SQL FUNDAMENTALS 131

ORDER

Column Constraints - Foreign Keys id customerid amount

1 1 99.00
2 1 212.00
▸ Used to enforce the “other end” of a relationship
3 3 14.50
▸ Validates against a column in the “foreign” table 4 12 1000.00

CUSTOMER
CREATE TABLE customer (
id SERIAL PRIMARY KEY, id name
name VARCHAR(255) 1 Mike North
);
2 Mark Grabanski
CREATE TABLE CustomerOrder ( 3 Lisa Huang-North
id SERIAL PRIMARY KEY,
customerid INTEGER,
amount REAL, INSERT INTO CustomerOrder
month INTEGER DEFAULT 5 NOT NULL (customerid, amount, month)
); VALUES (99, 300.50, 3);

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 SQL FUNDAMENTALS 132

ORDER

Column Constraints - Foreign Keys id customerid amount

1 1 99.00
2 1 212.00
▸ Used to enforce the “other end” of a relationship
3 3 14.50
▸ Validates against a column in the “foreign” table 4 12 1000.00

CUSTOMER
CREATE TABLE customer ( ERROR: insert or update on
id SERIAL PRIMARY KEY, table CustomerOrder
id violates
name
name VARCHAR(255) foreign key1 constraint
Mike North
); "order_customerid_fkey"
CREATE TABLE CustomerOrder ( Detail: Key2 (customerid)=(99)
Mark Grabanski
id SERIAL PRIMARY KEY, is not present in table
"customer".3 Lisa Huang-North
customerid INTEGER
NOT NULL REFERENCES customer(id),
amount REAL, INSERT INTO CustomerOrder
month INTEGER DEFAULT 5 NOT NULL (customerid, amount, month)
); VALUES (99, 300.50, 3);

Click for SQLfiddle © Mike.Works, Inc. 2018. All rights reserved

 133

Column Constraints 10
▸ To improve data integrity, add database constraints making it
impossible to have multiple OrderDetail records referring to
the same orderid and productid

▸ Create a new table called CustomerOrderTransaction with

columns

▸ id - an auto-incrementing primary key

▸ Auth - of type string, 255 characters or less, cannot be null

▸ orderid - of type Integer, cannot be null, must refer to an

actual id in the CustomerOrder table npm run test:ex:watch 10

© Mike.Works, Inc. 2018. All rights reserved