Introduction to Algorithms

6.046J/18.401J/SMA5503

Lecture 11
Prof. Erik Demaine
 Dynamic order statistics
OS-SELECT(i, S): returns the i th smallest element
in the dynamic set S.
OS-RANK(x, S): returns the rank of x ∈ S in the
sorted order of S’s elements.

IDEA: Use a red-black tree for the set S, but keep

subtree sizes in the nodes.

key
key
Notation for nodes:
size
size
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.2
 Example of an OS-tree
M
M
99
CC PP
55 33
AA FF NN QQ
11 33 11 11
DD HH
11 11

size[x] = size[left[x]] + size[right[x]] + 1

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.3

 Selection
Implementation trick: Use a sentinel
(dummy record) for NIL such that size[NIL] = 0.
OS-SELECT(x, i) ⊳ ith smallest element in the
subtree rooted at x
k ← size[left[x]] + 1 ⊳ k = rank(x)
if i = k then return x
if i < k
then return OS-SELECT(left[x], i )
else return OS-SELECT(right[x], i – k )
(OS-RANK is in the textbook.)
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.4
 Example
OS-SELECT(root, 5)
i=5 M
M
k=6 99
i=5 CC PP
k=2 55 33
AA FF i=3 NN QQ
11 33 k=2 11 11
DD HH i=1
11 11 k=1

Running time = O(h) = O(lg n) for red-black trees.

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.5
 Data structure maintenance
Q. Why not keep the ranks themselves
in the nodes instead of subtree sizes?
A. They are hard to maintain when the
red-black tree is modified.

Modifying operations: INSERT and DELETE.

Strategy: Update subtree sizes when
inserting or deleting.

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.6

 Example of insertion
INSERT(“K”)
M M
10
910
9
CC PP
6565 33
AA FF NN QQ
11 4343 11 11
DD HH
11 2121
KK
11

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.7

 Handling rebalancing
Don’t forget that RB-INSERT and RB-DELETE may
also need to modify the red-black tree in order to
maintain balance.
• Recolorings: no effect on subtree sizes.
• Rotations: fix up subtree sizes in O(1) time.
Example: EE CC
16
16 16
16
CC EE
4 7
11
11 88

7 3 3 4
∴RB-INSERT and RB-DELETE still run in O(lg n) time.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.8
 Data-structure augmentation
Methodology: (e.g., order-statistics trees)
1. Choose an underlying data structure (red-
black trees).
2. Determine additional information to be
stored in the data structure (subtree sizes).
3. Verify that this information can be
maintained for modifying operations (RB-
INSERT, RB-DELETE — don’t forget rotations).
4. Develop new dynamic-set operations that use
the information (OS-SELECT and OS-RANK).
These steps are guidelines, not rigid rules.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.9
 Interval trees
Goal: To maintain a dynamic set of intervals,
such as time intervals.
i = [7, 10]
low[i] = 7 10 = high[i]
5 11 17 19
4 8 15 18 22 23

Query: For a given query interval i, find an

interval in the set that overlaps i.

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.10

 Following the methodology
1. Choose an underlying data structure.
• Red-black tree keyed on low (left) endpoint.
2. Determine additional information to be
stored in the data structure.
• Store in each node x the largest value m[x]
in the subtree rooted at x, as well as the
interval int[x] corresponding to the key.

int
int
mm
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.11
 Example interval tree
17,19
17,19
23
23
5,11
5,11 22,23
22,23
18
18 23
23
4,8
4,8 15,18
15,18
88 18
18
high[int[x]]
7,10
7,10 m[x] = max m[left[x]]
10
10
m[right[x]]

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.12

 Modifying operations
3. Verify that this information can be maintained
for modifying operations.
• INSERT: Fix m’s on the way down.
• Rotations — Fixup = O(1) time per rotation:
11,15
11,15 6,20
6,20
30
30 30
30
6,20
6,20 11,15
11,15
30
30 19
19 30
30 19
19

30
30 14
14 14
14 19
19

Total INSERT time = O(lg n); DELETE similar.

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.13
 New operations
4. Develop new dynamic-set operations that use
the information.
INTERVAL-SEARCH(i)
x ← root
while x ≠ NIL and (low[i] > high[int[x]]
or low[int[x]] > high[i])
do ⊳ i and int[x] don’t overlap
if left[x] ≠ NIL and low[i] ≤ m[left[x]]
then x ← left[x]
else x ← right[x]
return x
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.14
 Example 1: INTERVAL-SEARCH([14,16])
x 17,19
17,19
23
23
5,11
5,11 22,23
22,23
18
18 23
23
4,8
4,8 15,18
15,18
88 18
18
7,10
7,10
10
10 x ← root
[14,16] and [17,19] don’t overlap
14 ≤ 18 ⇒ x ← left[x]
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.15
 Example 1: INTERVAL-SEARCH([14,16])
17,19
17,19
23
23
5,11 22,23
x 5,11 22,23
18
18 23
23
4,8
4,8 15,18
15,18
88 18
18
7,10
7,10
10
10
[14,16] and [5,11] don’t overlap
14 > 8 ⇒ x ← right[x]
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.16
 Example 1: INTERVAL-SEARCH([14,16])
17,19
17,19
23
23
5,11
5,11 22,23
22,23
18
18 23
23
4,8
4,8 15,18
15,18
88
x 18
18
7,10
7,10
10
10
[14,16] and [15,18] overlap
return [15,18]
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.17
 Example 2: INTERVAL-SEARCH([12,14])
x 17,19
17,19
23
23
5,11
5,11 22,23
22,23
18
18 23
23
4,8
4,8 15,18
15,18
88 18
18
7,10
7,10
10
10 x ← root
[12,14] and [17,19] don’t overlap
12 ≤ 18 ⇒ x ← left[x]
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.18
 Example 2: INTERVAL-SEARCH([12,14])
17,19
17,19
23
23
5,11 22,23
x 5,11 22,23
18
18 23
23
4,8
4,8 15,18
15,18
88 18
18
7,10
7,10
10
10
[12,14] and [5,11] don’t overlap
12 > 8 ⇒ x ← right[x]
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.19
 Example 2: INTERVAL-SEARCH([12,14])
17,19
17,19
23
23
5,11
5,11 22,23
22,23
18
18 23
23
4,8
4,8 15,18
15,18
88
x 18
18
7,10
7,10
10
10
[12,14] and [15,18] don’t overlap
12 > 10 ⇒ x ← right[x]
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.20
 Example 2: INTERVAL-SEARCH([12,14])
17,19
17,19
23
23
5,11
5,11 22,23
22,23
18
18 23
23
4,8
4,8 15,18
15,18
88 18
18
7,10
7,10 x
10
10
x = NIL ⇒ no interval that
overlaps [12,14] exists

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.21

 Analysis
Time = O(h) = O(lg n), since INTERVAL-SEARCH
does constant work at each level as it follows a
simple path down the tree.
List all overlapping intervals:
• Search, list, delete, repeat.
• Insert them all again at the end.
Time = O(k lg n), where k is the total number of
overlapping intervals.
This is an output-sensitive bound.
Best algorithm to date: O(k + lg n).
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.22
 Correctness
Theorem. Let L be the set of intervals in the
left subtree of node x, and let R be the set of
intervals in x’s right subtree.
• If the search goes right, then
{ i′ ∈ L : i′ overlaps i } = ∅.
• If the search goes left, then
{i′ ∈ L : i′ overlaps i } = ∅
⇒ {i′ ∈ R : i′ overlaps i } = ∅.
In other words, it’s always safe to take only 1
of the 2 children: we’ll either find something,
or nothing was to be found.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.23
 Correctness proof
Proof. Suppose first that the search goes right.
• If left[x] = NIL, then we’re done, since L = ∅.
• Otherwise, the code dictates that we must have
low[i] > m[left[x]]. The value m[left[x]]
corresponds to the right endpoint of some
interval j ∈ L, and no other interval in L can
have a larger right endpoint than high( j).
L i
high( j) = m[left[x]] low(i)
• Therefore, {i′ ∈ L : i′ overlaps i } = ∅.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.24
 Proof (continued)
Suppose that the search goes left, and assume that
{i′ ∈ L : i′ overlaps i } = ∅.
• Then, the code dictates that low[i] ≤ m[left[x]] =
high[ j] for some j ∈ L.
• Since j ∈ L, it does not overlap i, and hence
high[i] < low[ j].
• But, the binary-search-tree property implies that
for all i′ ∈ R, we have low[ j] ≤ low[i′].
• But then {i′ ∈ R : i′ overlaps i } = ∅.
i j
i′
L
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 20 L11.25