Balanced Binary Search Tree

Bùi Tiến Lên

2022
Contents

1. Balanced Tree

2. AVL Tree

3. Red-Black Tree

4. Optimal binary search trees

5. Workshop
 Balanced Tree
• Rotations
• Strategies in Balancing Tree
 Introduction
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
Balance may be defined by:
Red-Black Tree • Comparing the numbers of nodes of the two subtrees
Insertion
Deletion • Height balancing: comparing the heights of the twosub trees
Optimal binary
search trees • Null-path-length balancing: comparing the null-path-length of each of the
two sub-trees
Static
Dynamic
Splaying

Workshop
• Weight balancing: comparing the number of null sub-trees in each of the two
sub trees

4
 Introduction (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion Concept 1
Red-Black Tree
Insertion A binary tree is balanced if the difference in the numbers of nodes of both
Deletion
subtrees of any node in the tree either zero or one.
Optimal binary
search trees

Concept 2
Static
Dynamic
Splaying

Workshop
A binary tree is height-balanced if the difference in height of both subtrees of
any node in the tree either zero or one.

• A complete binary tree is is height-balanced

5
 Rotations
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• A rotation allows us to interchange the role of the root and one of the root’s
Insertion
Deletion

Red-Black Tree children in a tree while still preserving the BST ordering among the keys in
the nodes.
Insertion
Deletion

Optimal binary
search trees
• There are two kinds of rotations: right rotation and left rotation
Static
Dynamic
Splaying

Workshop

6
 Right rotation
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
A right rotation involves the root and the left child. The rotation puts the root on
Red-Black Tree the right, essentially reversing the direction of the left link of the root:
Insertion
Deletion
• Before the rotation, it points from the root to the left child
Optimal binary • After the rotation, it points from the old left child (the new root) to the old
search trees
Static
root (the right child of the new root)
Dynamic
Splaying
1 1
Workshop
S E

2 3

E S
greater than less than
S E
3 2

less than between between greater than

E E and S E and S S

7
 Example
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
15
Red-Black Tree 6 18
Insertion
Deletion
3 7 17 19
Optimal binary
search trees 2 4 13
Static
Dynamic
Splaying
9
Workshop

• Make right rotation at 15

6
3 15
2 4 7 18
13 17 19
9
8
 Implementation
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion void rightRotate (link& h) {
Red-Black Tree link x = h->left;
Insertion
Deletion h->left = x->right;
Optimal binary x->right = h;
search trees
Static
h = x;
Dynamic }
Splaying

Workshop

9
 Left rotation
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
A left rotation involves the root and the right child.
Red-Black Tree
Insertion
1 1
Deletion

Optimal binary E S
search trees
Static
Dynamic 2 3

Splaying

Workshop S E
less than greater than
E S
3 2

between greater than less than between

E and S S E E and S

10
 Implementation
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion void leftRotate (link& h) {
Red-Black Tree link x = h->right;
Insertion
Deletion h->right = x->left;
Optimal binary x->left = h;
search trees
Static
h = x;
Dynamic }
Splaying

Workshop

11
 Strategies in Balancing Tree
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Global rebalancing: an approach to producing better balance in BSTs is

Insertion
Deletion

Red-Black Tree periodically to rebalance them explicitly.

Insertion
Deletion • costs at least linear time in the size of the tree
Optimal binary
search trees • Local rebalancing: balancing BSTs after each operation (insert, delete)
Static
Dynamic
Splaying

Workshop

12
 DSW algorithm
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
The algorithm was proposed by Colin Day and later improved by Quentin F. Stout
Red-Black Tree and Bette L. Warren.
Insertion
Deletion
Idea of algorithm:
Optimal binary 1. Transform an arbitrary BST into a linked-list-like-tree called backbone or
search trees
Static vine by rotations
Dynamic
Splaying
2. Transform this tree into a perfectly balanced tree by rotations
Workshop

13
 DSW algorithm (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
createBackbone(root)
Red-Black Tree p := root
Insertion
Deletion
while p 6= null?
Optimal binary if p has a left chid?
search trees
Static
make right rotation at p
Dynamic
Splaying
else
Workshop p := p → right

14
 DSW algorithm (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
createCompleteTree(root)
Red-Black Tree n ← the number of nodes
Insertion
Deletion
m ← 2blog2 (n+1)c − 1
Optimal binary make n − m left rotations starting from the top of backbone
search trees
Static
while (m > 1)
Dynamic
Splaying
m ← m/2
Workshop make m left rotations starting from the top of backbone

15
 Illustration
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion

Red-Black Tree
Insertion Figure 1: BST Figure 2: Backbone Figure 3: Perfect
Deletion

Optimal binary 3 1 4
search trees
Static
Dynamic 2 4 2 2 6
Splaying

Workshop
1 5 3 1 3 5

6 4

16
 AVL Tree
• Insertion
• Deletion
 AVL Tree
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion Concept 3
Red-Black Tree
Insertion AVL tree
• proposed by two Soviet scientists G. M. Adelson-Velskii and E. M. Landis
Deletion

Optimal binary
search trees
Static
• is BST tree which is height-balanced
Dynamic

∀p : |height(LeftSubtree(p)) − height(RightSubtree(p))| ≤ 1 (1)

Splaying

Workshop

18
 The Height of an AVL Tree
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
Consider the worst case,
Red-Black Tree • To determine the maximum height that an AVL tree with N nodes can have,
Insertion
Deletion
we can instead ask what is the minimum number of nodes that an AVL tree
Optimal binary of height h can have (called AVL tree Fh ).
search trees
Static
• We have the recurrence relation
Dynamic
Splaying |Fh | = |Fh−1 | + |Fh−2 | + 1 (2)
Workshop
where |F0 | = 1 and |F1 | = 2
• Solve the equation, we have
" √ #h+2
1 1+ 5
|Fh | + 1 ≈ √ (3)
5 2
• The height of an AVL tree in the worst case
h ≈ 1.44 log2 |Fh | = 1.44 log2 N (4)
19
 Rebalancing technique
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
• After an insertion/deletion, we may find a node whose new balance violates
Red-Black Tree the AVL condition.
Insertion
Deletion
• Four cases: LL imbalance, LR imbalance, RR imbalance, RL imbalance
M M
Optimal binary
search trees E E

Static
G
Dynamic
Splaying

Workshop

M M

S S

20
 Rebalancing technique (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Case LL imbalance is corrected by executing a single right rotation at the

Insertion
Deletion

Red-Black Tree node with the imbalance.

Insertion
Deletion
M E
Optimal binary
search trees E M
Static
Dynamic
Splaying

Workshop

21
 Rebalancing technique (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Case LR imbalance is corrected by executing a double rotations

Insertion
Deletion

Red-Black Tree
M M G
Insertion
Deletion
E G E M
Optimal binary
search trees G E
Static
Dynamic
Splaying

Workshop

22
 Insertion
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
Insert(root,key)
Red-Black Tree if root = null
Insertion
Deletion
root := new Node(key)
Optimal binary return
search trees
Static
if root → key = key
Dynamic
Splaying
return
Workshop if root → key < key
Insert(root → right,key)
if root → key > key
Insert(root → left,key)
if unbalanced at root? rebalance at root

23
 Illustration
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• An AVL tree
Insertion
Deletion

Red-Black Tree
Insertion 44 h=4
Deletion

Optimal binary
search trees 17 h=2 78 h=3
Static
Dynamic
Splaying

Workshop
32 h=1 50 h=2 88 h=1

48 h=1 62 h=1

24
 Illustration (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Insert node 54 into the tree → node 78 become unbalanced

Insertion
Deletion

Red-Black Tree
Insertion
Deletion
44 h=?
Optimal binary
search trees
Static 17 h=2 78 h=?
Dynamic
Splaying

Workshop 32 h=1 50 h=? 88 h=1

48 h=1 62 h=?

54 h=1

25
 Illustration (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Case RL imbalance → rebalance by making double rotations

Insertion
Deletion

Red-Black Tree
Insertion
44 h=4
Deletion

Optimal binary
search trees
Static
17 h=2 62 h=3
Dynamic
Splaying

Workshop 32 h=1 50 h=2 78 h=2

48 h=1 54 h=1 88 h=1

26
 Illustration
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• An AVL tree
Insertion
Deletion

Red-Black Tree
Insertion 44 h=4
Deletion

Optimal binary
search trees 17 h=2 78 h=3
Static
Dynamic
Splaying

Workshop
32 h=1 50 h=2 88 h=1

48 h=1 62 h=1

27
 Illustration (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Delete node 32 from the tree → node 44 become unbalanced

Insertion
Deletion

Red-Black Tree
Insertion
Deletion
44 h=?
Optimal binary
search trees
Static 17 h=1 78 h=3
Dynamic
Splaying

Workshop 50 h=2 88 h=1

48 h=1 62 h=1

28
 Illustration (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Case RL imbalance → rebalance by making double rotations

Insertion
Deletion

Red-Black Tree
Insertion
50 h=3
Deletion

Optimal binary
search trees
Static
44 h=2 78 h=2
Dynamic
Splaying

Workshop 17 h=1 48 h=1 62 h=1 88 h=1

29
 Red-Black Tree
• Insertion
• Deletion
 Red-Black Tree
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion Concept 4
Red-Black Tree
Insertion A red–black (RB) tree is a special type of binary search tree that must statsify
1. Each node is either red or black.
Deletion

Optimal binary

2. The root is black. (sometimes omitted)

search trees
Static

3. If a node is red, then both its children are black.

Dynamic
Splaying

Workshop
4. Every path from a given node to any of its descendant null link has the same
number of black nodes (balance criteria).

Concept 5
A left-leaning red–black (LLRB) tree (leveraging Andersson’s idea AA tree) is a
variant of the red–black tree that has only left red children

31
 Example
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• A red–black tree
Insertion
Deletion

Red-Black Tree
Insertion 13
Deletion

Optimal binary
search trees 8 17
Static
Dynamic
Splaying
1 11 15 25
Workshop

6 22 27

32
 Example (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• A left-leaning red–black tree

Insertion
Deletion

Red-Black Tree
Insertion 13
Deletion

Optimal binary
search trees 8 17
Static
Dynamic
Splaying

Workshop
6 11

33
 Example (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• AA tree
Insertion
Deletion

Red-Black Tree
Insertion
Deletion

Optimal binary
search trees 30 70
Static
Dynamic
Splaying
15 50 60 85
Workshop

5 10 20 35 40 55 65 80 90

34
 The Height of a RB Tree
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion Theorem 1
The height of a red-black BST with N nodes is no more than 2 log2 N. It means
Red-Black Tree
Insertion
Deletion
that the height of an RB tree in the worst case
Optimal binary
search trees
Static
Dynamic
h ≤ 2 log2 N (5)
Splaying

Workshop

35
 Operations
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Case 1: Left rotation to orient a right red node to left red node.
Insertion
Deletion

Red-Black Tree
Insertion
1 1
Deletion

Optimal binary E S
search trees
Static
2 3
Dynamic
Splaying
S E
Workshop

3 2

36
 Operations (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Case 2: Right rotation.

Insertion
Deletion

Red-Black Tree
Insertion
1 1
Deletion

Optimal binary S E
search trees
2
Static
3
Dynamic
E
Splaying
3 A S
Workshop
A
2

37
 Operations (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Case 3: Color flip.

Insertion
Deletion

Red-Black Tree
Insertion
Deletion

Optimal binary E E
search trees
Static
Dynamic
Splaying
A S A S
Workshop

38
 Insertion
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
Insert(root,key)
Red-Black Tree if root = null
Insertion
Deletion
root := new Node(key) red node
Optimal binary return
search trees
Static
if root → key = key
Dynamic
Splaying
return
Workshop if root → key < key
Insert(root → right,key)
if root → key > key
Insert(root → left,key)
if isRed(root → right) and not isRed(root → left) rotateLeft(root)
if isRed(root → left) and isRed(root → left → left) rotateRight(root)
if isRed(root → left) and isRed(root → right) flipColors(root)

39
 Example
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Typical left-leaning red-black BST built from random keys

Insertion
Deletion

Red-Black Tree
Insertion
Deletion

Optimal binary
search trees
Static
Dynamic
Splaying

Workshop

40
 Illustration
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• A left-leaning red–black tree

Insertion
Deletion

Red-Black Tree
Insertion 13
Deletion

Optimal binary
search trees 8 17
Static
Dynamic
Splaying

Workshop
6 11

41
 Illustration (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Insert node 7 into the tree

Insertion
Deletion

Red-Black Tree
Insertion 13
Deletion

Optimal binary
search trees
8 17
Static
Dynamic
Splaying

Workshop 6 11

1 7

42
 Illustration (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Flip color at 6 into the tree

Insertion
Deletion

Red-Black Tree
Insertion 13
Deletion

Optimal binary
search trees 8 17
Static
Dynamic
Splaying

Workshop
6 11

1 7

43
 Illustration (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Right rotation at 13 into the tree

Insertion
Deletion

Red-Black Tree
Insertion 8
Deletion

Optimal binary
search trees 6 13
Static
Dynamic
Splaying

Workshop 1 7 11 17

44
 Illustration (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Flip color at root at change root to black color

Insertion
Deletion

Red-Black Tree
Insertion 8
Deletion

Optimal binary
search trees 6 13
Static
Dynamic
Splaying

Workshop 1 7 11 17

45
 Cost summary for symbol-table implementations
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
worst case average case
Red-Black Tree implementation key
Insertion
search insert remove search hit insert remove
Deletion
unordered list N 1 N N/2 1 N/2 equal
Optimal binary
search trees ordered list N N N N/2 N/2 N/2 compare
Static
Dynamic
ordered array log2 N N N log2 N N/2 N/2 compare
√
Splaying
BST N N N c log2 N c log2 N N compare
Workshop
AVL ca log2 N - - log2 N - - compare
RB cr log2 N - - log2 N - - compare
goal?

Note: c = 1.39, ca = 1.44, cr = 2.0

46
 Optimal binary search trees
• Static
• Dynamic
 Introduction
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion Concept 6
Red-Black Tree
Insertion An optimal binary search tree (optimal BST), sometimes called a
Deletion
weight-balanced binary tree, is a binary search tree which provides the smallest
Optimal binary
search trees possible search time (or expected search time) for a given sequence of accesses (or
access probabilities)
Static
Dynamic
Splaying

Workshop
• Optimal BSTs are generally divided into two types: static and dynamic
• In the static optimality problem, the tree cannot be modified after it has
been constructed.
• In the dynamic optimality problem, the tree can be modified at any
time, typically by permitting tree rotations.

48
 Introduction
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Suppose that we are designing a binary search tree for a program to translate
Insertion
Deletion

Red-Black Tree text from English to Vietnamese, we want words that occur frequently in the
text to be placed nearer the root.
Insertion
Deletion

Optimal binary
search trees
Static
Problem
Given a sequence of of n distinct keys in sorted order (k1 < k2 < · · · < kn ) and
Dynamic
Splaying

Workshop their frequencies

key k1 k2 ... kn
D=
frequency f1 f2 fn

What binary search tree has the lowest search cost?

49
 Search Cost
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Cost of search for key ki

Insertion
Deletion

Red-Black Tree
Insertion
Deletion
Cost(ki ) = Depth(ki ) (6)
Optimal binary
search trees
Static
where Depth(root) = 1
Dynamic
Splaying
• Denote ExpectCost(l, r ) be expected cost of search for a BST tree
Workshop containing {kl , ..., kr } given D
r
ExpectCost(l, r ) = Cost(ki )fi (7)
X

i=l

50
 Search Cost (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
• Compute the expected cost for the following binary search tree given
Red-Black Tree
Insertion key k1 k2 k3 k4 k5 k6
D=
frequency 0.1 0.2 0.1 0.3 0.2 0.1
Deletion

Optimal binary
search trees
Static
Dynamic
Splaying

Workshop

51
 Optimal Search Cost
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Denote OptimalCost(l, r ) be optimal cost of search for a BST tree T

Insertion
Deletion

Red-Black Tree containing {kl , ..., kr } given D

Insertion
Deletion • Denote OptimalCost(l, m, r ) be optimal cost of search for a BST tree T
Optimal binary
search trees containing {kl , ..., kr } and km be the root node given D
Static
Dynamic
• We have
Splaying

Workshop
r
OptimalCost(l, m, r ) = fi
X

i=l
+ OptimalCost(l, m − 1)
+ OptimalCost(m + 1, r ) (8)

OptimalCost(l, r ) = min {OptimalCost(l, m, r )} (9)

m∈{l,...r}

52
 Optimal Search Cost (cont.)
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion

Red-Black Tree
Insertion
Deletion

Optimal binary
search trees
Static
Dynamic
Splaying

Workshop

53
 Splay tree
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion Concept 7
Red-Black Tree
Insertion A splay tree is a binary search tree with the additional property that recently
Deletion
accessed elements are quick to access again.
Optimal binary
search trees
Static
Dynamic
• All normal operations (insert, look-up) on a binary search tree are combined
Splaying
with one basic operation, called splaying.
Workshop
• For many sequences of non-random operations, splay trees perform better
than other binary search trees.

54
 Splaying
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• When a node x is accessed, a splay operation is performed on x to move it to

Insertion
Deletion

Red-Black Tree the root.

Insertion
Deletion • To perform a splay operation we carry out a sequence of splay steps, each of
Optimal binary
search trees which moves x closer to the root.
Static
Dynamic
• There are three types of splay steps, each of which has two symmetric
Splaying
variants:
Workshop
• zig step
• zig-zig step
• zig-zag step

55
 Zig step
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
1 1
Red-Black Tree
Insertion
Deletion
S E

Optimal binary
search trees 2 3

Static
Dynamic
E S
Splaying

Workshop
3 2

56
 Zig-zig
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
1 1
Red-Black Tree
Insertion
Deletion
S A

Optimal binary 2 5

search trees
Static
E E
Dynamic
3 4 3 4
Splaying

Workshop A S
5 2

57
 Zig-zag
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree
Insertion
Deletion
1 1
Red-Black Tree
Insertion
Deletion
S E

Optimal binary 2

search trees 4 5

Static
A
Dynamic
3 A S
Splaying

Workshop E
3 2
4 5

58
Workshop
 � Quiz
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

1. What is an AVL tree?

Insertion
Deletion

Red-Black Tree .........................................................................

.........................................................................
Insertion
Deletion

Optimal binary .........................................................................

search trees
Static
Dynamic
2. What is a Red-black tree?
Splaying
.........................................................................
Workshop
.........................................................................
.........................................................................

60
 Ï Exercises
Balanced Tree
Rotations
Strategies in Balancing
Tree

AVL Tree

• Programming exercises in [Cormen, 2009, Sedgewick, 2002]

Insertion
Deletion

Red-Black Tree
Insertion
Deletion

Optimal binary
search trees
Static
Dynamic
Splaying

Workshop

61
References

Cormen, T. H. (2009).
Introduction to algorithms.
MIT press.
Sedgewick, R. (2002).
Algorithms in Java, Parts 1-4, volume 1.
Addison-Wesley Professional.
Walls and Mirrors (2014).
Data Abstraction And Problem Solving with C++.
Pearson.