How Modern SQL Databases Come up with Algorithms that You Would Have Never Dreamed Of

Copyright (c) 2009-2017 by Data Geekery GmbH. Slides licensed under CC BY SA 3.0
NoSQL? No, SQL!
How Modern SQL Databases
Come up with Algorithms
that You Would Have Never
Dreamed Of

NoSQL? No, SQL!
TL;DR:
SQL = Awesome

Why are you here?
Because:
… This cannot be true
… This must not be true
… Is it true, though?
… OMG imma rewrite all code to SQL

Me – @lukaseder
SQL is a device whose mystery
is only exceeded by its power!
- Founder and CEO at Data Geekery
- Oracle Java Champion
- JUG.ch Board Member

When you leave the room

Why do I talk about SQL?
SQL is the only ever successful,
mainstream, and general-
purpose 4GL (Fourth-
Generation Programming
Language)
And it is awesome!

Why doesn’t anyone else talk about SQL?

¯_(シ)_/¯

Except Vlad

Vlad shows JPA’s only important feature
His blog: http://tiny.cc/jpa-only-useful-feature
Vlad: Hibernate
Developer Advocate

What is SQL?
Who has seen
my other talk?

What is SQL?
I mean this one

10 SQL tricks to convince you SQL is awesome

10 SQL tricks to convince you SQL is awesome
Not that
hard to
find

A SQL trick
-- Query from http://explainextended.com/2013/12/31/happy-new-year-5/
WITH RECURSIVE q(r, i, rx, ix, g) AS (
SELECT r::DOUBLE PRECISION * 0.02, i::DOUBLE PRECISION * 0.02,
.0::DOUBLE PRECISION , .0::DOUBLE PRECISION, 0
FROM generate_series(-60, 20) r, generate_series(-50, 50) i
UNION ALL
SELECT r, i, CASE WHEN abs(rx * rx + ix * ix) <= 2 THEN rx * rx - ix * ix END + r,
CASE WHEN abs(rx * rx + ix * ix) <= 2 THEN 2 * rx * ix END + i, g + 1
FROM q
WHERE rx IS NOT NULL AND g < 99
)
SELECT array_to_string(array_agg(s ORDER BY r), '')
FROM (
SELECT i, r, substring(' .:-=+*#%@', max(g) / 10 + 1, 1) s
FROM q
GROUP BY i, r
) q
GROUP BY i
ORDER BY i

A SQL trick: Generating the Mandelbrot Set
-- Query from http://explainextended.com/2013/12/31/happy-new-year-5/
WITH RECURSIVE q(r, i, rx, ix, g) AS (
SELECT r::DOUBLE PRECISION * 0.02, i::DOUBLE PRECISION * 0.02,
.0::DOUBLE PRECISION , .0::DOUBLE PRECISION, 0
FROM generate_series(-60, 20) r, generate_series(-50, 50) i
UNION ALL
SELECT r, i, CASE WHEN abs(rx * rx + ix * ix) <= 2 THEN rx * rx - ix * ix END + r,
CASE WHEN abs(rx * rx + ix * ix) <= 2 THEN 2 * rx * ix END + i, g + 1
FROM q
WHERE rx IS NOT NULL AND g < 99
)
SELECT array_to_string(array_agg(s ORDER BY r), '')
FROM (
SELECT i, r, substring(' .:-=+*#%@', max(g) / 10 + 1, 1) s
FROM q
GROUP BY i, r
) q
GROUP BY i
ORDER BY i

But today…
But today, we’ll talk
about more basic SQL

But today…
But today, we’ll talk
about more basic SQL
(still awesome)

But today…
Is this SQL?

Who can handle this SQL statement here?
SELECT *
FROM person
WHERE id = 42

Easy stuff
Perfect
This talk is for you.

What will I talk about?
• SQL is awesome
• SQL is productive
• SQL is fast

There are two ways of proving this
1. By example
2. By alternative (Java)

Disclaimer
This is going to be a high level,
conceptual story about SQL
We’re not going into deeper
levels.

Business is asking
What is the daily
revenue of each
store

Business is asking

Business is asking
What is the daily
revenue of each
store
Film
Money
Day
JOIN

No problemo

Easy, right?
SELECT
title, payment_date, SUM(amount)
FROM film
JOIN inventory USING (film_id)
JOIN rental USING (inventory_id)
JOIN payment USING (rental_id)
GROUP BY film_id, payment_date
ORDER BY title, payment_date

Easy, right?
SELECT
FROM film
Money per film

Easy, right?
SELECT
FROM film
Relationship film ↔ money

Easy, right?
SELECT
FROM film
Aggregation per film / date

Imperative style
Let’s write this in
«classic» Java

“Classic Java”

Let’s get rolling
class Film {
}

Let’s get rolling
You still with me?

Obviously, Film is a tuple
class Film {
int filmId;
String title;
}

Because, Java
class Film {
int filmId;
String title;
int getFilmId() {
return filmId;
}
...
}

Never forget these, of course
class Film {
int filmId;
String title;
int getFilmId() { ... }
void setFilmId(int filmId) { ... }
String getTitle() { ... }
void setTitle(String title) { ... }
int hashCode() { ... }
boolean equals(Object o) { ... }
String toString() { ... }
...
}

Abstractions, ladies and gentlemen!
class FilmFactory {
Film newFilm() { ... }
}
class FilmFactoryBuilder {
FilmFactory newFilmFactory() { ... }
}

Abstractions, ladies and gentlemen!
interface FilmFactory {
Film newFilm();
}
interface FilmFactoryBuilder {
FilmFactory newFilmFactory();
}

Just to be sure, super generic!
Film newFilm();
}
}
class FilmFactoryImpl {
@Override
public Film newFilm() { ... }
}
class FilmFactoryBuilderImpl {
@Override
public FilmFactory newFilmFactory() { ... }
}

Some Spring
Film newFilm();
}
}
@Bean
@Override
}
@Bean
@Override
@EnableCaching
}

Some Spring, and a little Lombok
@FunctionalInterface
Film newFilm();
}
}
@Bean
@EnableAspectJAutoProxy
@EnableAutoConfiguration
@Override
}
@Bean
@Configuration
@NoArgsConstructor
@Override
@EnableCaching
}

Oh, what the heck
Film newFilm();
}
}
@Bean
@Override
}
@Bean
@Configuration
@NoArgsConstructor @Cloneable @SneakyThrows
@ResultSetMapping @ConcurrentInitialiserProxyFactory
@SpringPrefetchAdapter @AdapterProxyBeanMethod @AdapterBeanProxyMethod
@Override
@EnableCaching
}

Oh, what the heck
Film newFilm();
}
}
@Bean
@Override
@ThisMethodDidntHaveAnnotationsYet @OKWeStillHaveSomeSpaceLeft @LetsSeeIfWeCanReachThePrintMargin
}
@Bean
@Configuration
@SpringPrefetchAdapter @AdapterProxyBeanMethod @AdapterBeanProxyMethod
@MoreAndMoreAnnotations @CanYouEvenReadThis
@IsThereStillAnyRealLogicLeft
@ThisJokeNeverGetsLame @Annotatiomania @WhoWillBeNextYearsAnnotatiomaniac
@Override
@EnableCaching
}

Average enterprise bean is visible from space
Film newFilm();
}
@FunctionalInterface @NoArgsConstructor @Cloneable @SneakyThrows
}
@Bean
@EnableAspectJAutoProxy @NoArgsConstructor @Cloneable @SneakyThrows @NoArgsConstructor @Cloneable @SneakyThrows
@EnableAutoConfiguration @NoArgsConstructor @Cloneable @SneakyThrows
@Override @NoArgsConstructor @Cloneable @SneakyThrows
}
@Bean
@Configuration @NoArgsConstructor @Cloneable @SneakyThrows
@SpringPrefetchAdapter @AdapterProxyBeanMethod @AdapterBeanProxyMethod @NoArgsConstructor @Cloneable @SneakyThrows
@MoreAndMoreAnnotations @CanYouEvenReadThis @NoArgsConstructor @Cloneable @SneakyThrows
@IsThereStillAnyRealLogicLeft @NoArgsConstructor @Cloneable @SneakyThrows @NoArgsConstructor @Cloneable
@SneakyThrows
@Override
@EnableCaching
}

Enough of this. More info here
www.annotatiomania.com

Hmm, relationships?
class Film {
int filmId;
String title;
// Hmm... Inventory? Store? Rental? Payment?
...
}

Let’s assume, we copy relational model 1:1
class Film {
int filmId;
String title;
List<Inventory> getInventories() { ... }
...
}

class Film {
int filmId;
String title;
@OneToMany(mappedBy = "film")
...
}

class Film {
int filmId;
String title;
@OneToMany(mappedBy = "film")
...
}
Anyone seen these lovely things?

class Film {
int filmId;
String title;
@OneToMany(mappedBy = "film", fetch = FetchType.EAGER)
...
}
How about this? Care discuss this?

class Film {
int filmId;
String title;
@OneToMany(mappedBy = "film", fetch = FetchType.LAZY)
...
}
How about this? Care discuss this?

If you must @OneToMany
Ask Vlad

Now, calculate. This be the result type
// This is the optimal result type
Map<Film,
Map<LocalDate, BigDecimal>>
result = new HashMap<>();

Now, calculate. This be the result type
// This is the optimal result type
Map<Film,
result = new HashMap<>();
Money
Per film
Per date

Money, eh?
JSR 354 FTW!
(Java Money API)

Now, calculate. Wait…
// Or this?
Map<Film,
result = new LinkedHashMap<>();

Now, calculate. Or better
// Or maybe this?
SortedMap<Film,
SortedMap<LocalDate, BigDecimal>>
result = new TreeMap<>();

Now, calculate. Aagh
// Kewl, wrote some library
MultiKeyMap<
Film, LocalDate, BigDecimal>
result = new MultiKeyMap<>();

Five hours later

Who in here has felt this way before?

Now let’s calculate, for real
Map<Film, Map<LocalDate, BigDecimal>> result = ...;

Easy, so far
List<Film> films = nPlusOneLoadAllFilms(); // TODO

Easy, so far
// FIXME 2002-04-01 Will optimize this later

Easy, so far
// FIXME 2003-04-01 Will optimize this “later”

Easy, so far
// FIXME 2003-04-01 Will optimize this “later”
// FIXME 2005-04-01 URGENT! This is really slow!

Still following?
for (Film film : films)
; // TODO

Still following?
for (Film film : films) {
Map<LocalDate, BigDecimal> daily = result.get(film);
// TODO
}

Aaah, lazy initialization!
if (daily == null) {
daily = new HashMap<>();
result.put(film, daily);
}
// TODO
}

Let’s have that Map discussion again
if (daily == null) {
daily = new LinkedHashMap<>(); // TODO: Better Map!
result.put(film, daily);
}
// TODO
}

OK, next
if (daily == null) { ... }
// TODO
}

OK, now looping
for (Inventory inventory : film.getInventories())
; // TODO
}

OK, now looping
for (Rental rental : inventory.getRentals())
; // TODO
}

OK, now looping
for (Payment payment : rental.getPayments())
; // TODO
}

OK, now looping
; // TODO
}
EAGER or LAZY?

OK, now looping
; // TODO
}
Or, entity graph?

Entity Graph – How JPA folks write SQL
@NamedEntityGraph(
name = "graph.film.inventories.rentals.payments",
attributeNodes = @NamedAttributeNode(
value = "inventories",
subgraph = "inventories"
),
subgraphs = {
@NamedSubgraph(
name = "inventories",
attributeNodes = @NamedAttributeNode(
value = "rentals",
subgraph = "rentals"
)
), ...
}
)

Remember this slide?
Film newFilm();
}
}
@Bean
}
@Bean
@SneakyThrows
@Override
@EnableCaching
}

Your code now visible from Andromeda Galaxy
Film newFilm();
}
}
@Bean
}
@Bean
@SneakyThrows
@Override
@EnableCaching
}

OK, now looping, done. Right?
daily.put(
payment.getPaymentDate(),
payment.getAmount());
}

OK, now looping, done. Wrong!
for (Payment payment : rental.getPayments()) {
LocalDate date = payment.getPaymentDate();
BigDecimal amount = payment.getAmount();
if (daily.containsKey(date))
daily.put(date, daily.get(date).add(amount));
else
daily.put(date, amount);
}
}

OK, now looping, done. OK, better with Java 8
Map<LocalDate, BigDecimal> daily =
result.computeIfAbsent(film, k -> new HashMap<>());
daily.compute(
payment.getPaymentDate(),
(k, v) -> v == null
? payment.getAmount()
: payment.getAmount().add(v));
}

Feel good about yourself?

Business is asking again

Programmer reactions
SQL Developer Java Developer

SQL dev: I knew this was coming!
SELECT
FROM film

SQL dev: Swoosh!
SELECT
FROM film

SQL dev: Joke’s on you!
SELECT
title, store_id, payment_date, SUM(amount)
FROM film
GROUP BY film_id, store_id, payment_date
ORDER BY title, store_id, payment_date

Java dev: Gaaah, OK
for (Payment p : rental.getPayments())
daily.compute(
p.getPaymentDate(),
(k, v) -> v == null ? p.getAmount() : p.getAmount().add(v));
}

Java dev: Let’s make some room
Map<Film, Map<LocalDate, BigDecimal> > result = ...;
daily.compute(
p.getPaymentDate(),
}

Java dev: Let’s see, if I tweak this and then…
Map<Film, Map<Integer, Map<LocalDate, BigDecimal>>> result = ...;
Map<Integer, Map<LocalDate, BigDecimal>> dailyPerStore =
for (Inventory inventory : film.getInventories()) {
dailyPerStore.computeIfAbsent(
inventory.getStoreId(), k -> new HashMap());
daily.compute(
p.getPaymentDate(),
}
}

daily.compute(
p.getPaymentDate(),
}
}Architect:
(that’s this guy)

daily.compute(
p.getPaymentDate(),
}
}Architect: We can’t have such types.
Factor out in a new class. And Factory

daily.compute(
p.getPaymentDate(),
}
}
And you’re like…

Meanwhile…
While you and your
architect engage in
nominal vs structural
typing philosophies...

Meanwhile… The inevitable

What on earth is a cumulative sum??

Java dev: 😵
// Thanks God it’s Friday!
throw new UnsupportedOperationException();

Meanwhile, the SQL developer…
SELECT
title, store_id, payment_date,
SUM(amount)
FROM film

Meanwhile, the SQL developer…
SELECT
SUM(SUM(amount)) OVER (
PARTITION BY title, store_id
ORDER BY payment_date
)
FROM film
ORDER BY title, store_id, payment_dateWindow function

What does it do?

3GL vs. 4GL
Do you
get the
message?

3GL vs. 4GL
SQL is a 4GL

3GL vs. 4GL
In SQL, we only
declare the result.

3GL vs. 4GL
The database will
figure out the
algorithm for us.

3GL vs. 4GL
Algorithms are
boring (mostly)

3GL vs. 4GL
We don’t care about
• Writing algorithms (using them is a different story)
• Premature Optimisation
• Data type details (HashMap<A, B> or List<Entry<A, B>>?)
• Local variables
• Caches
• Loops
• Initialisation (or NullPointerException)

3GL vs. 4GL
We do care about
• Business logic
• Just a bit of indexing
That’s it!

Is Java 8 Streams “declarative” ?
The Stream API
looks a lot like
SQL’s declarative
style

Is Java 8 Streams “declarative” ?
The Stream API
looks a lot like
SQL’s declarative
style
FROM
WHERE
FETCH NEXT
GROUP BY

In a way, it’s very similar
TABLE : Stream<Tuple<..>>
SELECT : map()
DISTINCT : distinct()
JOIN : flatMap()
WHERE / HAVING : filter()
GROUP BY : collect()
ORDER BY : sorted()
UNION ALL : concat()
See:
http://blog.jooq.org/2015/08/13/common-sql-clauses-and-their-equivalents-in-java-8-streams/

Let’s try that Java algorithm again
List<Film> films = loadAllFilms(); // TODO
daily.compute(
p.getPaymentDate(),
}
}

Map<Film, Map<Integer, Map<LocalDate, Double>>> result = loadAllFilms()
.stream()
.flatMap(f -> f.getInventories().stream())
.flatMap(i -> i.getRentals().stream())
.flatMap(r -> r.getPayments().stream())
.collect(... /* amount per film and store */ ...);

.stream() // Stream<Film>
.flatMap(f -> f.getInventories().stream()) // Stream<Inventory>
.flatMap(i -> i.getRentals().stream()) // Stream<Rental>
.flatMap(r -> r.getPayments().stream()) // Stream<Payment>

.stream() // Stream<Film>
.flatMap(f -> f.getInventories().stream()) // Stream<Inventory>
.flatMap(i -> i.getRentals().stream()) // Stream<Rental>
.flatMap(r -> r.getPayments().stream()) // Stream<Payment>
How to access «f» and «i» in
the Collector?

.stream()
.flatMap(f -> f.getInventories().stream().map(i -> entry(f, i)))
.flatMap(fi -> fi.v2.getRentals().stream().map(r -> entry(fi, r)))
.flatMap(fir -> fir.v3.getPayments().stream().map(p -> entry(fir, p)))
Use some wrapper types and pray
for stack allocation to happen

.stream()
.collect(
Collectors.groupingBy(firp -> firp.getKey().getKey().getKey(),
Collectors.groupingBy(firp -> firp.getKey().getKey().getValue()
.getStoreId(),
Collectors.groupingBy(firp -> firp.getValue().getPaymentDate(),
Collectors.summingDouble(
firp -> firp.getValue().getAmount().doubleValue()
)
)
)
)
);

.stream()
.collect(
.getStoreId(),
)
)
)
)
);
But what the hell is this?
(╯°□°)╯︵ ┻━┻

Well, it certainly
looks declarative

We can do a bit better, though
Better Streams:
https://github.com/jOOQ/jOOL

We can do a bit better, though
Seq.seq(persons)
.collect(
count(),
max(Person::getAge),
min(Person::getHeight),
avg(Person::getWeight)
);
// (3, Optional[35],
// Optional[1.69], Optional[70.0])

Map<Film, Map<Integer, Map<LocalDate, Double>>> result =
Seq.seq(loadAllFilms())
.collect(
.getStoreId(),
)
)
)
)
);
Use a Seq instead of a Stream

.crossApply(f -> f.getInventories())
.crossApply(fi -> fi.v2.getRentals())
.crossApply(fir -> fir.v2.getPayments())
.collect(
Collectors.groupingBy(firp -> firp.v1.v1.v1,
Collectors.groupingBy(firp -> firp.v1.v1.v2.getStoreId(),
Collectors.groupingBy(firp -> firp.v2.getPaymentDate(),
firp -> firp.v2.getAmount().doubleValue()
)
)
)
)
);
Use SQL style CROSS APPLY instead
of flatMap()

.crossApply(f -> f.getInventories()) // <Film, Inventory>
.crossApply(fi -> fi.v2.getRentals()) // <<F, I>, Rental>
.crossApply(fir -> fir.v2.getPayments()) // <<<F, I>, R>, Payment>
.collect(
)
)
)
)
);

.crossApply(fi -> fi.v2.getRentals())
.crossApply(fir -> fir.v2.getPayments())
.collect(
)
)
)
)
);
<<<Film, Inventory>, Rental>, Payment>

.crossApply(fi -> fi.v2.getRentals().stream()
.flatMap(r -> r.getPayments().stream()))
.collect(
Collectors.groupingBy(fip -> fip.v1.v1,
Collectors.groupingBy(fip -> fip.v1.v2.getStoreId(),
Collectors.groupingBy(fip -> fip.v2.getPaymentDate(),
fip -> fip.v2.getAmount().doubleValue()
)
)
)
)
);
Hm, never really needed the rentals

.collect(
Collectors.groupingBy(fip -> fip.v1.v1,
Collectors.groupingBy(fip -> fip.v1.v2.getStoreId(),
Collectors.groupingBy(fip -> fip.v2.getPaymentDate(),
fip -> fip.v2.getAmount().doubleValue()
)
)
)
)
);
A little bit better now...

Map<Tuple3<Film, Integer, LocalDate>, Double> result =
.groupBy(
fip -> tuple(
fip.v1.v1,
fip.v1.v2.getStoreId(),
fip.v2.getPaymentDate()
),
Agg.sum(fip -> fip.v2.getAmount())
);
Oh, not that bad, actually
┬─┬ ノ( ゜-゜ノ)

Map<Tuple3<Film, Integer, LocalDate>, Double> result =
.groupBy(
fip -> tuple(
fip.v1.v1,
fip.v1.v2.getStoreId(),
fip.v2.getPaymentDate()
),
Agg.sum(fip -> fip.v2.getAmount())
);
Result type a bit nicer, too

Bad news is:
How much time did
we spend on this
problem now?

Remember this slide? Was it all that hard?
SELECT
SUM(amount)
FROM film

Everyone understands now…
SQL is a 4GL
This means great
improvement of developer
productivity

Everyone understands now…
How much time did I spend talking
about SQL so far?
How much time on Java?
SQL is easier = SQL is more productive

Suspense!
Best chapter:
Performance

But first
How does SQL
work?

Business asking
How many films did each
actor whose last name
starts with «A» play in?

They want this
Starts with «A»
How many films

This is the query. But what really happens?
SELECT
first_name, last_name, count(*)
FROM actor
JOIN film_actor USING (actor_id)
WHERE last_name LIKE 'A%'
GROUP BY first_name, last_name
ORDER BY count(*) DESC

Logical Operations order
1. Load tables

1. Load tables
Observe: «LOGICAL». Not «ACTUAL»

1. Load tables
2. Apply predicates

1. Load tables
2. Apply predicates
3. Collect groups

1. Load tables
2. Apply predicates
3. Collect groups
4. Aggregate

1. Load tables
2. Apply predicates
3. Collect groups
4. Aggregate
5. Project

1. Load tables
2. Apply predicates
3. Collect groups
4. Aggregate
5. Project
6. Order

Complete order
Logical operations order
1. FROM, JOIN, APPLY
2. WHERE
3. CONNECT BY (Oracle)
4. GROUP BY
5. Aggregations
6. HAVING
7. WINDOW
8. MODEL (Oracle)
9. SELECT
10. DISTINCT
11. UNION, INTERSECT, EXCEPT
12. ORDER BY
13. OFFSET
14. LIMIT
15. FOR UPDATE

What’s the actual algorithm?
But that’s not what
really happens!
Any algorithm that produces
the same result is acceptable.

Can we see the
actual algorithm?

It’s called «execution plan»

Who has seen an execution plan?

SQL Developer

pgAdmin III

SQL Workbench

SQL Server Management Studio

How many rows from operation

How much time does it take

Imperative
Actual operations order
1. Access ROWIDs from Index given the predicate

Imperative
2. Access other columns from the same row in the table

Imperative
3. Load foreign key relationship index into memory

Imperative
4. Put both data sets in hashmaps and join them

Imperative
5. Create another hashmap to group rows

Imperative
5. Create another hashmap to group rows
6. Sort the result

Huh!
Declarative vs
Imperative
Would you have chosen the
same algorithm?

E.g. Apparently, it’s faster in this
case to load 5462 rows into
memory than seeking 6 counts
individually...
Huh!

Big O Notation
Hash join: O(N)
(N = film_actors (bigger table))
Nested loop join with indexes:
O(log M x log N)
(M = actors, N = film_actors)

Hash join: O(N)
(N = film_actors (bigger table))
Nested loop join with indexes:
O(log M x log N)
(M = actors, N = film_actors)
Big O Notation

Well...
It depends!

What you think you have

What you may really have

What you may really have
«Works on my machine» ಠ_ಠ

More info
bigocheatsheet.com

Statistics
Here’s an interesting
thing to think about:
Histograms

Statistics
Histogram

Statistics
Histogram
-- Predicate 1
WHERE amount BETWEEN -2000 AND 2000
-- Predicate 2
WHERE amount BETWEEN 5000 AND 9000

Statistics
Histogram
-- Predicate 1
-- Predicate 2
Do both
predicates profit
from indexing?

Statistics
Histogram
-- Not very selective predicate
-- Very selective predicate

Statistics
Accessing the index is only worth
the pain if it helps reduce the
result set
Otherwise, just read from the table

Statistics
The database knows all these things.
• How many rows are returned from a table?
• From a query?
• How much does it cost to access the disk?
• The cache?
• How often is this query run?
• How often is the table accessed?

Statistics
The database knows all these things.
• How much memory do we have?
• How many processes are running in parallel right now?
• Is the operating system doing anything?
• How does the query perform with this bind variable?
• Are we running on an HDD or SSD or Flash or RAM?
• How many locks do we have on our rows?

Expression trees
Everything has a cost

Expression trees
How does SQL work?
1. A parsed SQL string produces an expression tree

How does SQL work?
A + (A – B)
Expression trees
+
-A
A B

Expression trees
How does SQL work?
2. Multiple expression trees are equivalent: They can be
transformed into each other

How does SQL work?
A + (A – B)
Expression trees
+
-A
A B

How does SQL work?
A + (A – B) (A + A) - B
Expression trees
+
-A
A B
+
-
A A
B

How does SQL work?
A + (A – B) (2 * A) - B
Expression trees
+
-A
A B
*
-
2 A
B

How does SQL work?
A + (A – B) (2 * A) - B
Expression trees
+
-A
A B
*
-
2 A
B
E.g.: Accessing A is super expensive

Expression trees
How does SQL work?
3. Each expression tree has a corresponding execution
plan and an associated, estimated cost
4. The «best» plan (= lowest estimated cost) is chosen
5. The actual execution cost is compared with the
estimate to improve future estimates

Expression trees
How does SQL work?
3. Each expression tree has a corresponding execution
plan and an associated, estimated cost
4. The «best» plan (= lowest estimated cost) is chosen
5. The actual execution cost is compared with the
estimate to improve future estimates
When’s the last time you wrote Java
code that did this?

Expression trees
Special treat:
Oracle 12c can abort and switch
execution plans «in flight». This is
called «Adaptive Execution Plans».

Why do I talk about SQL?
Now that you know
all of this...
What do you prefer?

This?
daily.compute(
p.getPaymentDate(),
}
}

Or, this?
SELECT
title, store_id, payment_date, SUM(amount)
FROM film

Can I write SQL in Java?

Can I write SQL in Java? – Yes. With jOOQ

So what’s the key takeaway?
1. Can you do it in the database? Yes
2. Can you do it in the database? Yes
(... after visiting my 2 day SQL training)
3. Can you do it in your database? Yes
(... unless you're using MySQL)
4. Should you do it in the database? Yes
5. Do long talk titles attract attention? Yes
6. Will this talk ever end? Yes

If you haven’t had enough
Google «10 SQL Tricks»
and find the other talk’s
transcript
https://blog.jooq.org/2016/04/25/10-sql-
tricks-that-you-didnt-think-were-possible/

Thank you
Check out our trainings:
http://www.jooq.org/training
Coordinates
• Blog: http://blog.jooq.org (excellent Java SQL content)
• Twitter: @JavaOOQ / @lukaseder (more lame jokes)
• E-Mail: lukas.eder@datageekery.com
• Bank account: CH57 8148 7000 0SQL AWSM 7

How Modern SQL Databases Come up with Algorithms that You Would Have Never Dreamed Of

More Related Content

Viewers also liked (20)

Similar to How Modern SQL Databases Come up with Algorithms that You Would Have Never Dreamed Of (20)

Recently uploaded (20)

How Modern SQL Databases Come up with Algorithms that You Would Have Never Dreamed Of