CMPT 354:

Database System I
Lecture 4. SQL Advanced

Sept 20 /18

1
Announcements!
• A1 is due today

• A2 is released (due in 2 weeks)

2
Outline
• Joins
• Inner Join
• Outer Join
• Aggregation Queries
• Simple Aggregations
• Group By
• Having
• Discussion

3
Joins: Recap
Student Enroll
name gpa stdName course
Mary 3.8 Mary 354
Tom 3.6 Tom 354
Jack 3.7 Tom 454
Bob 354

SELECT name, course

FROM Student, Enroll
WHERE name = stdName

name gpa
Mary 354
Tom 354
Tom 454 4
Two equivalent ways to write
joins
SELECT name, course
FROM Student, Enroll
WHERE name = stdName

SELECT name, course

FROM Student JOIN Enroll ON name =
stdName

5
Join Types
SELECT name, course
FROM Student INNER JOIN Enroll ON
name = stdName

SELECT name, course

FROM Student FULL OUTER JOIN Enroll ON
name = stdName

SELECT name
FROM Student LEFT OUTER JOIN Enroll ON
name = stdName

SELECT name
FROM Student RIGHT OUTER JOIN Enroll ON
name = stdName 6
Join Types
SELECT name, course
FROM Student INNER JOIN Enroll ON
name = stdName

SELECT name, course

FROM Student FULL OUTER JOIN Enroll ON
name = stdName

SELECT name
FROM Student LEFT OUTER JOIN Enroll ON
name = stdName

SELECT name
FROM Student RIGHT OUTER JOIN Enroll ON
name = stdName 7
Left Join
Student Enroll
name gpa stdName course
Mary 3.8 Mary 354
Tom 3.6 Tom 354
Jack 3.7 Tom 454
Bob 354

SELECT name, course

FROM Student JOIN
LEFTEnroll
JOIN ON
Enroll ON
name = stdName

We want to include all students no matter

whether they enroll a course or not. How?
8
SELECT name, course
FROM Student LEFT JOIN Enroll ON
name = stdName
Student Enroll
name gpa stdName course
Mary 3.8 Mary 354
Tom 3.6 Tom 354
Jack 3.7 Tom 454
Bob 354

Output
name course
Mary 354
Tom 354
Tom 454
Jack NULL

9
SELECT name, course
FROM Student RIGHT JOIN Enroll ON
name = stdName
Student Enroll
name gpa stdName course
Mary 3.8 Mary 354
Tom 3.6 Tom 354
Jack 3.7 Tom 454
Bob 354

Output
name course
Mary 354
Tom 354
Tom 454
NULL 354
10
SELECT name, course
FROM Enroll FULL JOIN Student ON
name = stdName
Enroll Student
stdName course name gpa
Mary 354 Mary 3.8
Tom 354 Tom 3.6
Tom 454 Jack 3.7
Bob 354

Output
name course
Mary 354
Tom 354
Tom 454
Jack NULL
NULL 354 11
Outer Join
TableA (LEFT/RIGHT/FULL) JOIN TableB

• Left outer join:

• Include tuples from tableA even if no match
• Right outer join:
• Include tuples from tableA even if no match
• Full outer join:
• Include tuples from both even if no match

12
Exercise - 1
SELECT name, course
FROM Student LEFT JOIN Enroll ON
name = stdName AND course = 354

Student Enroll
name gpa stdName course
Mary 3.8 Mary 354
Tom 3.6 Tom 354
Jack 3.7 Tom 454
Bob 354

name course
A Mary 354 name course
Tom 354 Mary 354
Jack NULL Tom 354
13
(A) (B)
Exercise - 2
SELECT name, course
FROM Student LEFT JOIN Enroll ON
name = stdName
WHERE course = 354
Student Enroll
name gpa stdName course
Mary 3.8 Mary 354
Tom 3.6 Tom 354
Jack 3.7 Tom 454
Bob 354

Select + From name course

Then add the
Where condition Mary 354 name course
Will remove the Tom 354 Mary 354
Jack row from A, Jack NULL Tom 354
so answer is B. 14
(A) (B)
Outline
• Joins
• Inner Join
• Outer Join
• Aggregation Queries
• Simple Aggregations
• Group By
• Having
• Discussion

15
Simple Aggregation

SELECT agg(column)
FROM <table name>
WHERE <conditions>

agg = COUNT, SUM, AVG, MAX, MIN, etc.

Except count, all aggregations apply to a single attribute

16
Examples
name gender gpa
Bob M 3
Mike M 3
Alice F 3
Mary F 4
Tom M 4

SELECT COUNT(*) FROM Student 5

SELECT SUM(gpa) FROM Student 17

SELECT AVG(gpa) FROM Student 3.4

SELECT MIN(gpa) FROM Student 3

SELECT MAX(gpa) FROM Student 4 17

Examples
name gender gpa
Bob M 3
Mike M 3
Alice F 3
Mary F 4
Tom M 4

SELECT COUNT(DISTINCT gpa) FROM Student 2

SELECT SUM(DISTINCT gpa) FROM Student 7

SELECT AVG(gpa) FROM Student

WHERE gender = ‘F’ 3.5
18
The need for Group By
• How to get AVG(gpa) for each gender?
SELECT AVG(gpa) FROM Student WHERE gender = ‘M’

SELECT AVG(gpa) FROM Student WHERE gender = ‘F’

• How to get AVG(gpa) for each age?

SELECT AVG(gpa) FROM Student WHERE age = 18

SELECT AVG(gpa) FROM Student WHERE age = 19

SELECT AVG(gpa) FROM Student WHERE age = 20

.
.
.
19
Grouping and Aggregation
SELECT agg(column)
FROM <table name>
WHERE <conditions>
GROUP BY <columns>

• How to get AVG(gpa) for each gender?

SELECT AVG(gpa) FROM Student GROUP BY gender

• How to get AVG(gpa) for each age?

SELECT AVG(gpa) FROM Student GROUP BY age

20
Grouping and Aggregation
• How is the following query processed?
SELECT gender, AVG(gpa)
FROM Student
WHERE gpa > 2.5
GROUP BY gender

• Semantics of the query

1. Compute the FROM and WHERE clauses
2. Group by the attributes in the GROUP BY
3. Compute the SELECT clause: grouped attributes and
aggregates
21
1. Compute the FROM and WHERE clauses

SELECT gender, AVG(gpa)

FROM Student
WHERE gpa > 2.5
GROUP BY gender

name gender gpa

Bob M 2 name gender gpa
Mike M 3 Mike M 3
Alice F 3 Alice F 3
Mary F 4 Mary F 4
Tom M 3 Tom M 3
22
2. Group by the attributes in the GROUP BY

SELECT gender, AVG(gpa)

FROM Student
WHERE gpa > 2.5
GROUP BY gender

name gender gpa gender name gpa

Mike M 3 Mike 3
Alice F 3 M
Tom 3
Mary F 4 Alice 3
Tom M 3 F
Mary 4
23
 3. Compute the SELECT clause: grouped
attributes and aggregates
SELECT gender, AVG(gpa)
FROM Student
WHERE gpa > 2.5
GROUP BY gender

gender name gpa

gender AVG(gpa)
Mike 3
M M 3
Tom 3
F 3.5
Alice 3
F
Mary 4
24
Exercise: Empty Group
name gender gpa
Bob M 3 SELECT gender, AVG(gpa)
Mike M 3 FROM Student
Alice F 4 WHERE gpa > 3.5
Mary F 4 GROUP BY gender
Tom M 3

gender AVG(gpa)
gender AVG(gpa)
VS F 4
F 4
M NULL

(A) (B) 25
 Exercise: Empty Group
name gender gpa
Bob M 3 SELECT gender, AVG(gpa)
Mike M 3 FROM Student
Alice F 4 WHERE gpa > 3.5
Mary F 4 GROUP BY gender
Tom M 3

name gender gpa

Alice F 4
Mary F 4

26
 Exercise: Empty Group
name gender gpa
Bob M 3 SELECT gender, AVG(gpa)
Mike M 3 FROM Student
Alice F 4 WHERE gpa > 3.5
Mary F 4
GROUP BY gender
Tom M 3

name gender gpa gender name gpa

Alice F 4 Alice 4
F
Mary F 4 Mary 4

27
 Exercise: Empty Group
name gender gpa
Bob M 3 SELECT gender, AVG(gpa)
Mike M 3 FROM Student
Alice F 4 WHERE gpa > 3.5
Mary F 4 GROUP BY gender
Tom M 3

name gender gpa gender name gpa

gender AVG(gpa)
Alice F 4 Alice 4
F F 4
Mary F 4 Mary 4

28
Exercise: Invalid Selection
name gender gpa
Bob M 3 SELECT gender, AVG(gpa), name
Mike M 3 FROM Student
Alice F 4 WHERE gpa > 3.5
Mary F 4 GROUP BY gender
Tom M 3

gender AVG(gpa） name gender AVG(gpa) name

VS
F 4 Alice F 4 Mary

(A) (B) 29
 Exercise: Invalid Selection
name gender gpa
Bob M 3 SELECT gender, AVG(gpa), name
Mike M 3 FROM Student
Alice F 4 WHERE gpa > 3.5
Mary F 4 GROUP BY gender
Tom M 3

name gender gpa

Alice F 4
Mary F 4

30
 Exercise: Invalid Selection
name gender gpa
Bob M 3 SELECT gender, AVG(gpa), name
Mike M 3 FROM Student
Alice F 4 WHERE gpa > 3.5
Mary F 4 GROUP BY gender
Tom M 3

name gender gpa gender name gpa

Alice F 4 Alice 4
F
Mary F 4 Mary 4

31
 Exercise: Invalid Selection
Everything in SELECT must be either a
GROUP-BY attribute, or an aggregate
name gender gpa
Bob M 3 SELECT gender, AVG(gpa), name
Mike M 3 FROM Student
Alice F 4 WHERE gpa > 3.5
Mary F 4 GROUP BY gender
Tom M 3

Some database will report invalid selection,

Some will pick randomly from Alice and Mary and display that one
name gender gpa gender name gpa
Alice F 4 gender AVG(gpa) name
Alice 4
F F 4 ???
Mary F 4 Mary 4

32
HAVING Clause
• Specify which groups you are interested in

SELECT agg(column)
FROM <table name>
WHERE <conditions>
GROUP BY <columns>
HAVING <columns>

33
HAVING Clause
• Same query as before, except that we require each
group has more than 10 students

SELECT AVG(gpa), gender

FROM Student
WHERE gpa > 2.5
GROUP BY gender
HAVING COUNT(*) > 10

HAVING clause contains conditions on aggregates.

34
 Order of Evaluation
SELECT S
FROM R1,…,Rn
WHERE C1
GROUP BY a1,…,ak
HAVING C2

• Create the cross product of the tables in the FROM clause

• Remove rows not meeting the WHERE condition
• Divide records into groups by the GROUP BY clause
• Remove groups not meeting the HAVING clause
• Create one row for each group and remove columns not in
the SELECT clause

35
Exercise
StudentInfo
name gender gpa
Bob M 3
Mike M 3
Alice F 4
Mary F 4
Tom M 3

SELECT gender, AVG(gpa) SELECT gender, AVG(gpa)

FROM StudentInfo FROM StudentInfo
WHERE gpa > 2.5 WHERE gpa > 2.5
GROUP BY gender GROUP BY gender
HAVING COUNT(*) > 2 A HAVING SUM(gpa) < 9 B

gender AVG(gpa) gender AVG(gpa) gender AVG(gpa)

M 3 F 4 M 3
F 4
36
(A) (B) (C)
Imagine you are a data scientist
at a Bank

37
 Computer Science vs. Data Science

What When Who Goal

Computer 1950- Software Engineer Write software to make computers work
Science

Plan  Design  Develop  Test  Deploy  Maintain

What When Who Goal

Data 2010- Data Scientist Extract insights from data to answer questions
Science

Collect  Clean  Integrate  Analyze  Visualize  Communicate

38
Discussion

Q1. Who is the richest customer?

Q2. Which customers have ONLY one account?

39
Discussion

Q3. How many customers does each branch have?

Q4. Which branch have a higher pay?

40
Outline
• Joins
• Inner Join
• Outer Join
• Self Join
• Aggregation Queries
• Simple Aggregations
• Group By
• Having
• Subqueries
• In the FROM clause
• In the WHERE clause
41
 Subqueries
• A subquery is a SQL query nested inside a larger query
• Such inner-outer queries are called nested queries

SELECT C.customerID, C.birthDate, C.income

Outer Query
FROM Customer C
WHERE C.customerID IN
(
SELECT O.customerID
FROM Account A, Owns A Inner Query
WHERE A.accNumber = O.accNumber
AND A.branchName = 'Lonsdale’
) 42
 Subqueries
• Subqueries may appear in
• A FROM clause,
• A WHERE clause, and
• A HAVING clause
SELECT <columns>
FROM <table name>
WHERE <conditions>
GROUP BY <columns>
HAVING <columns>

43
Subqueries in FROM
• Sometimes we need to compute an intermediate
table only to use it later in a SELECT-FROM-WHERE
• Who is the richest customer?

SELECT firstName, lastName, MAX(sumBalance)

FROM (SELECT firstName, lastName, sum(balance) AS sumBalance
FROM Customer C, Account A, Owns O
WHERE C.customerID = O.customerID
AND O.accNumber = A.accNumber
GROUP BY C.customerID )

44
Subqueries in FROM
• Sometimes we need to compute an intermediate
table only to use it later in a SELECT-FROM-WHERE
• Which customers have a total balance equal to 0?

SELECT firstName, lastName, sumBalance

FROM (SELECT firstName, lastName, sum(balance) AS sumBalance
FROM Customer C, Account A, Owns O
WHERE C.customerID = O.customerID
AND O.accNumber = A.accNumber
GROUP BY C.customerID) AS T
WHERE T. sumBalance = 0
45
Subqueries in FROM
• Sometimes we need to compute an intermediate
table only to use it later in a SELECT-FROM-WHERE
• Which customers have a total balance equal to 0?

SELECT firstName, lastName, sum(balance) AS sumBalance

FROM Customer C, Account A, Owns O
WHERE C.customerID = O.customerID AND O.accNumber = A.accNumber
GROUP BY C.customerID
HAVING sumBalance = 0

Rule of thumb: avoid nested queries when possible

46
 Subqueries in WHERE
• Subqueries return a single constant
• >, <, =, <>, >=, <=
• Find the customerIDs of customers whose income is
larger than avg(income)

SELECT C.customerID
FROM Customer C1
WHERE C1.income > (SELECT avg(C2.income)
FROM Customer C2)

47
 Subqueries in WHERE
• Subqueries return a relation
• IN
• NOT IN
• EXISTS
• NOT EXISTS
• ANY
• ALL

48
 Accounts IN Burnaby
• Find the customerIDs of customers with an account at
the Burnaby branch

SELECT C.customerID
FROM Customer C
WHERE C.customerID IN (SELECT O.customerID
FROM Account A, Owns O
WHERE A.accNumber = O.accNumber
AND A.branchName = ’Burnaby’)

49
 Accounts NOT IN Burnaby
• Find the customerIDs of customers who do not have an
account at the Burnaby branch

SELECT C.customerID
FROM Customer C
WHERE C.customerID NOT IN(
SELECT O.customerID
FROM Account A, Owns O
WHERE A.accNumber = O.accNumber AND
A.branchName = ’Burnaby’)

50
Uncorrelated Queries
• The query shown previously contains an uncorrelated,
or independent, sub-query
• The sub-query does not contain references to attributes of
the outer query

• An independent sub-query can be evaluated before

evaluation of the outer query
• And needs to be evaluated only once
• The sub-query result can be checked for each row of the outer query
• The cost is the cost for performing the sub-query (once) and
the cost of scanning the outer relation
51
EXISTing BurnabyAccounts
• Find the customerIDs of customers with an account at
the Burnaby branch
SELECT C.customerID
FROM Customer C
WHERE EXISTS ( SELECT *
FROM Account A, Owns O
WHERE C.customerID = O.customerID
AND A.accNumber = O.accNumber
AND A.branchName = ’Burnaby’)

EXISTS and NOT EXISTS test whether the associated

sub-query is non-empty or empty
52
Correlated Queries
• The previous query contained a correlated sub-
query
• With references to attributes of the outer query
• … WHERE C.customerID = O.customerID …
• It is evaluated once for each row in the outer query
• i.e. for each row in the Customer table
• Correlated queries are often inefficient

53
EXISTing BurnabyAccounts
• Find the customerIDs of customers with an account at
the Burnaby branch

SELECT DISTINCT C.customerID

FROM Customer C, Account A, Owns O
WHERE C.customerID = A.accNumber
AND A.accNumber = O.customerID
AND A.branchName = ’Burnaby’

54
 Have an account in all branches
• Find the customerIDs of customers who have an
account in all branches
SQ1 – A list of all SQ2 – A list of
branch names branch names that
a customer has an
account at
EXCEPT

If the customer has an account at every branch then

this result is empty
55
 Have an account in all branches
• Putting it all together we have

SELECT C.customerID
FROM Customer C
WHERE NOT EXISTS ( (SELECT B.branchName
FROM Branch B)
EXCEPT
(SELECT A.branchName
FROM Account A, Owns O
WHERE O.customerID = C.customerID
AND O.accNumber = A.accNumber))
56
 ANYone Richer Than Bruce
• Find the customerIDs of customers who earn more
than some customer called Bruce

SELECT C.customerID
FROM Customer C
WHERE C.income > ANY
(SELECT Bruce.income
FROM Customer Bruce
WHERE Bruce.firstName = 'Bruce')

Customers in the result table must have incomes greater

than at least one of the rows in the sub-query result 57
 Richer Than ALL the Bruces
• Find the customerIDs of customers who earn more
than all customer called Bruce
SELECT C.customerID
FROM Customer C
WHERE C.income > ALL
(SELECT Bruce.income
FROM Customer Bruce
WHERE Bruce.firstName = 'Bruce')

If there were no customers called Bruce this query would

return all customers 58
Summary
• Selection
• Projection
• Set Operators (UNION, INTERSECT, EXCEPT)
• Joins (INNER, OUTER)
• Aggregation
• Group By
• Having
• Order By
• Distinct
• Subqueries
59
SQL operators can be composed just like building LEGO buildings
Acknowledge
• Some lecture slides were copied from or inspired by the
following course materials
• “W4111: Introduction to databases” by Eugene Wu at
Columbia University
• “CSE344: Introduction to Data Management” by Dan Suciu at
University of Washington
• “CMPT354: Database System I” by John Edgar at Simon Fraser
University
• “CS186: Introduction to Database Systems” by Joe Hellerstein
at UC Berkeley
• “CS145: Introduction to Databases” by Peter Bailis at Stanford
• “CS 348: Introduction to Database Management” by Grant
Weddell at University of Waterloo
60