SQL: Recursion

Introduction to Databases
CompSci 316 Fall 2014
 2

Announcements (Thu., Oct. 2)

• Homework #2 due next Tuesday
• Sample solution will be posted by Wednesday 8pm
• Midterm in class next Thursday (Oct. 9)
• Open-book, open-notes
• Same format as sample midterm (from last year)
• Sample solution also posted on Sakai
3
 4

A motivating example
Parent (parent, child)
Ape
parent child

Abe

Homer Marge

Bart Lisa

• Example: find Bart’s ancestors

• “Ancestor” has a recursive definition
• is ’s ancestor if
• is ’s parent, or
• is ’s ancestor and is ’s ancestor
 5

Recursion in SQL
• SQL2 had no recursion
• You can find Bart’s parents, grandparents, great
grandparents, etc.
! " ! ! " "
#$% !& " ! '!& " ! (
) $ ! * !+! ( "
,-! ( * !+!. ./
• But you cannot find all his ancestors with a single query
• SQL3 introduces recursion
• )0 clause
• Implemented in PostgreSQL (common table
expressions)
 6

Ancestor query in SQL3

)0 !$ 1$ 02
" 3 " '! 4! base case
33 ! " '! * !#$% !& " 4
1,0%,
3 ! " '! (
#$% ! " '! " ( Define a
) $ ! !+! ( " 44 a relation
! " recursion step recursively
#$% ! "
) $ ! +!. ./ Query using the relation
defined in )0 clause
 7

Fixed point of a function

• If : → is a function from a type to itself, a
fixed point of is a value such that =
• Example: What is the fixed point of = 2?
• 0, because 0 =0 2=0
• To compute a fixed point of
• Start with a “seed”: ←
• Compute
• If = , stop; is fixed point of
• Otherwise, ← ; repeat
• Example: compute the fixed point of = 2
• With seed 1: 1, 1/2, 1/4, 1/8, 1/16, … → 0
Doesn’t always work, but happens to work for us!
 8

Fixed point of a query

• A query is just a function that maps an input table
to an output table, so a fixed point of is a table
such that =
• To compute fixed point of
• Start with an empty table: ←∅
• Evaluate over
• If the result is identical to , stop; is a fixed point
• Otherwise, let be the new result; repeat
Starting from ∅ produces the unique minimal fixed
point (assuming is monotone)
 9

Finding ancestors
parent child
• )0 !$ 1$ 02
" 3 " '! 4!
33 ! " '! * !#$% !& " 4
1,0%,
3 ! " '! (
#$% ! " '! " (
) $ ! !+! ( " 44
• Think of the definition as Ancestor = (Ancestor) anc desc

anc desc
anc desc

anc desc
 10

Intuition behind fixed-point iteration

• Initially, we know nothing about ancestor-
descendent relationships
• In the first step, we deduce that parents and
children form ancestor-descendent relationships
• In each subsequent steps, we use the facts
deduced in previous steps to get more ancestor-
descendent relationships
• We stop when no new facts can be proven
 11

Linear recursion
• With linear recursion, a recursive definition can make
only one reference to itself
• Non-linear
• )0 !$ 1$ 02 ! " 3 " '! 4!
33 ! " '! * !#$% !& " 4
1,0%,
3 ! " '! (
#$% ! " '! " (
) $ ! !+! ( " 44
• Linear
• )0 !$ 1$ 02 ! " 3 " '! 4!
33 ! " '! * !#$% !& " 4
1,0%,
3 ! " '! *
#$% ! " '!& "
) $ ! +! " 44
 12

Linear vs. non-linear recursion

• Linear recursion is easier to implement
• For linear recursion, just keep joining newly generated
Ancestor rows with Parent
• For non-linear recursion, need to join newly generated
Ancestor rows with all existing Ancestor rows
• Non-linear recursion may take fewer steps to
converge, but perform more work
• Example: → → → →
• Linear recursion takes 4 steps
• Non-linear recursion takes 3 steps
• More work: e.g., → has two different derivations
 13

5 6
 14

Mutual recursion example

• Table Natural (n) contains 1, 2, …, 100
• Which numbers are even/odd?
• An odd number plus 1 is an even number
• An even number plus 1 is an odd number
• 1 is an odd number
)0 !$ 1$ 02 ! 7 "3"4!
3 !"!#$% !, 8 *
) $ !"!+! ,93 !": !#$% !% 44'
$ 1$ 02 !% 3"4!
33 !"!#$% !, 8 *!) $ !"!+! 4
1,0%,
3 !"!#$% !, 8 *
) $ !"!+! ,93 !": !#$% ! 7 "444
 15

Semantics of )0
• )0 !$ 1$ 02 ! ! '!;'
$ 1$ 02 ! !
/
• and ,…, may refer to ,…,
• Semantics
1. ← ∅, … , ←∅
2. Evaluate , … , using the current contents of ,…, :
← ,…, ←
3. If ≠ for some
3.1. ← ,…, ←
3.2. Go to 2.
4. Compute using the current contents of , …
and output the result
 16

Computing mutual recursion

)0 !$ 1$ 02 ! 7 "3"4!
3 !"!#$% !, 8 *
) $ !"!+! ,93 !": !#$% !% 44'
$ 1$ 02 !% 3"4!
33 !"!#$% !, 8 *!) $ !"!+! 4
1,0%,
3 !"!#$% !, 8 *
) $ !"!+! ,93 !": !#$% ! 7 "444
• Even = ∅, Odd = ∅
• Even = ∅, Odd = {1}
• Even = {2}, Odd = {1}
• Even = {2}, Odd = {1, 3}
• Even = {2, 4}, Odd = {1, 3}
• Even = {2, 4}, Odd = {1, 3, 5}
• …
 17

Fixed points are not unique anc desc

)0 !$ 1$ 02 parent child
" 3 " '! 4!
33 ! " '! * !#$% !
& " 4
1,0%,
3 ! " '! (
#$% ! " '! " (
) $ ! !+! ( " 44

Note how the bogus tuple 8 8

reinforces itself!
But if is monotone, then
all these fixed points must contain the fixed point we
computed from fixed-point iteration starting with ∅
Thus the unique minimal fixed point is the “natural” answer
 18

Mixing negation with recursion

• If is non-monotone
• The fixed-point iteration may flip-flop and never converge
• There could be multiple minimal fixed points—we
wouldn’t know which one to pick as answer!
• Example: popular users (pop ≥ 0.8) join either
Jessica’s Circle or Tommy’s
• Those not in Jessica’s Circle should be in Tom’s
• Those not in Tom’s Circle should be in Jessica’s
• )0 !$ 1$ 02 ! < * 38 4!
3 !8 #$% !1 !) $ ! !=+! 6
,-!8 ,% !0,!3 !8 #$% !> * 44'
$ 1$ 02 !> * 38 4!
3 !8 #$% !1 !) $ ! !=+! 6
,-!8 ,% !0,!3 !8 #$% ! < * 44
 19

Fixed-point iter may not converge

)0 !$ 1$ 02 ! < * 38 4!
3 !8 #$% !1 !) $ ! !=+! 6
,-!8 ,% !0,!3 !8 #$% !> * 44'
$ 1$ 02 !> * 38 4!
3 !8 #$% !1 !) $ ! !=+! 6
,-!8 ,% !0,!3 !8 #$% ! < * 44

uid name age pop

?( @
( ** " 6 6

TommyCircle JessicaCircle TommyCircle JessicaCircle

uid uid uid uid
?( ?(
( (
 20

Multiple minimal fixed points

)0 !$ 1$ 02 ! < * 38 4!
3 !8 #$% !1 !) $ ! !=+! 6
,-!8 ,% !0,!3 !8 #$% !> * 44'
$ 1$ 02 !> * 38 4!
3 !8 #$% !1 !) $ ! !=+! 6
,-!8 ,% !0,!3 !8 #$% ! < * 44

uid name age pop

?( @
( ** " 6 6

TommyCircle JessicaCircle TommyCircle JessicaCircle

uid uid uid uid
?( ( ( ?(
 21

Legal mix of negation and recursion

• Construct a dependency graph
• One node for each table defined in )0
• A directed edge → " if is defined in terms of "
• Label the directed edge “−” if the query defining is
not monotone with respect to "
• Legal SQL3 recursion: no cycle with a “−” edge
• Called stratified negation
• Bad mix: a cycle with at least one edge labeled “−”
−
Ancestor TommyCircle JessicaCircle
Legal! − Illegal!
 22

Stratified negation example

• Find pairs of persons with no common ancestors
)0 !$ 1$ 02 ! " 3 " '! 4!
33 ! " '! * !#$% !& " 4!1,0%,
3 ! " '! (
#$% ! " '! " (
) $ ! !+! ( " 44'
& "3 "4!
33 ! " !#$% !& " 4!1,0%, Ancestor
3 ! * !#$% !& " 44'
, " " 3 " '! "(4! − Person
33 ! "'! ( "
#$% !& " '!& " (
) $ ! "!A=! ( "4
B & NoCommonAnc
3 ! '! (
#$% ! " '! " (
) $ ! " !+! ( " 44
!C!#$% !, " " /
 23

Evaluating stratified negation

• The stratum of a node is the maximum number of
“−” edges on any path from
in the dependency graph Ancestor
• Ancestor: stratum 0
− Person
• Person: stratum 0
• NoCommonAnc: stratum 1
NoCommonAnc
• Evaluation strategy
• Compute tables lowest-stratum first
• For each stratum, use fixed-point iteration on all nodes
in that stratum
• Stratum 0: Ancestor and Person
• Stratum 1: NoCommonAnc
Intuitively, there is no negation within each stratum
 24

Summary
• SQL3 )0 recursive queries
• Solution to a recursive query (with no negation):
unique minimal fixed point
• Computing unique minimal fixed point: fixed-point
iteration starting from ∅
• Mixing negation and recursion is tricky
• Illegal mix: fixed-point iteration may not converge; there
may be multiple minimal fixed points
• Legal mix: stratified negation (compute by fixed-point
iteration stratum by stratum)