Lectures 17-18

Trees, Binary Search Trees

EECS 281: Data Structures & Algorithms

1
 Context for Trees
Tree: a mathematical abstraction that
• Captures common properties of data
• Critical to the design and analysis of algorithms

(x + 3) * (y - 2)
2
 Context for Trees
Tree: a mathematical abstraction that
• Captures common properties of data
• Critical to the design and analysis of algorithms

Our goals
• Build and use explicit data structures
that are concrete realizations of trees
• Describe the dynamic properties of algorithms

3
 Two Kinds of Trees
1. (Simple) tree
– Acyclic connected graph
– Considered undirected
2. Rooted tree
– A simple tree with a selected node (root)
– All edges are directed away from root

• Any node could be selected as root

4
 Some Tree Terminology
• Root: “top-most” vertex in the tree
– The starting point for functions (root pointer)
• Parent/Child: direct links in tree
• Siblings: children of the same parent
• Descendent: successor in tree
– Further from root along path
• Ancestor: predecessor in tree
– Closer to root along path

5
 Some Tree Terminology
• External (Leaf) node: a node without
children
• Internal node: a node with children
• Ordered Tree: linear ordering for the
children of each node
• Binary Tree: ordered tree in which every
node has at most two children

6
 Tree Definitions
Depth:
depth(empty) = 0;
depth(node) = depth(parent) + 1;
Height:
height(empty) = 0;
height(node) = max(height(children)) + 1;
Max Height/Depth:
The maximum of the height and depth of a
tree’s nodes should be the same
7
 Tree Terminology
• Root:
• Leaf nodes:
• Internal nodes: A

• Max depth:
B C D
• Circle one subtree
• Is this a binary tree?
E F G H

I J K

8
Complete Binary Tree Property
Definition: complete binary tree
• A binary tree with depth d where
– Tree depths 1, 2, …, d-1 have the max
number of nodes possible
– All internal nodes are to the left of the external
nodes at depth d-1
– That is, all leaves are at d-1 or leftmost at
depth d

10
 Binary Tree Implementation
Array Binary Tree Implementation
– Root at index 1
– Left child of node i at 2 * i
– Right child of node i at 2 * i + 1
– Some indices may be skipped
– Can be space prohibitive for sparse trees

11
 Binary Tree Implementation
Complexity of array implementation
• Insert key (best case) O(1)
• Insert key (worst case) O(n)
• Remove key (worst case) O(n)
• Parent O(1)
• Child O(1)
• Space (best case) O(n)
• Space (worst case) O(2n)
12
 Binary Tree Implementation
• Pointer-based binary tree implementation
1 <typename KEY>
2 struct Node {
3 KEY datum;
4 Node *left, *right;
5 };
• A node contains some information, and points to
its left child node and right child node
• Efficient for moving down a tree from parent to
child
13
 Binary Tree Implementation
Complexity of pointer implementation
• Insert key (best case) O(1)
• Insert key (worst case) O(n)
• Remove key (worst case) O(n)
• Parent O(n)
• Child O(1)
• Space (best case) O(n)
• Space (worst case) O(n)
14
 Trees: Data Structures
• Another way to do it (not common)
1 <typename KEY>
2 struct Node {
3 KEY datum;
4 Node *parent, *left, *right;
5 };
• If node is root, then *parent is nullptr
• If node is external, then *left and *right are nullptr

15
 Translating General Trees
into Binary Trees
T: General tree
T’: Binary tree
Intuition:
– Take set of siblings {v1,v2,…,vk} in T
that are children of v
– v1 becomes left child of v in T’
– v2,…vk become chain of right children
of v1 in T’
– Recurse from v2
Left: new “generation”; Right: sibling
16
 Tree Traversal
Systematic method to process every node in a tree
• Preorder:
– Visit node,
– Recursively visit left subtree,
– Recursively visit right subtree
• Inorder:
– Recursively visit left subtree,
– Visit node,
– Recursively visit right subtree
17
 Tree Traversal
Systematic method to process every node in a tree
• Postorder:
– Recursively visit left subtree,
– Recursively visit right subtree,
– Visit node
• Level order:
– Visit nodes in order of increasing depth in tree

18
 Recursive Implementations
1 void preorder(Node *p) { 13 void inorder(Node *p) {
2 if (!p) return; 14 if (!p) return;
3 visit(p->datum); 15 inorder(p->left);
4 preorder(p->left); 16 visit(p->datum);
5 preorder(p->right); 17 inorder(p->right);
6 } // preorder() 18 } // inorder()
7 void postorder(Node *p) {
8 if (!p) return;
9 postorder(p->left);
10 postorder(p->right);
11 visit(p->datum);
12 } // postorder() 19
 Summary of Tree Algorithms
• Definitions of depth and height
• Methods of tree traversal
– Preorder
– Inorder All are depth-first search
– Postorder
– Level order (breadth-first search)

20
 Exercise
• In what order are nodes visited?
– Preorder
– Inorder
– Postorder
– Level order
0

1 3

2 4
21
 Exercise
• Draw the binary tree from traversals
• Preorder: 7 3 6 9 8 13 27
• Inorder: 3 6 7 8 9 13 27

23
 Exercise
• Write a function to do a level order
traversal, printing the datum at each node
void levelorder(Node *p) {

} // levelorder()
25
 Search
• Retrieval of a particular piece of information from
large volumes of previously stored data
• Purpose is typically to access information within
the item (not just the key)
• Recall that arrays, linked lists are worst-case
O(n) for either searching or inserting
• Even a hash table has worst-case O(n)
Need a data structure with optimal efficiency
for searching and inserting
What if order is important?
27
 Symbol Table: ADT
• insert a new item
• search for an item (items) with a given key
• remove an item with a specified key
• sort the symbol table
• select the item with the kth largest key
• join two symbol tables
Also may want construct, test if empty, destroy,
copy...

28
 Binary Search Tree
• The keys in a binary search tree satisfy
the binary-search-tree property
– The key of any node is:
> keys of all nodes in its left subtree and
≤ keys of all nodes in its right subtree
• Essential property of BST is that insert() is
as easy to implement as search()

29
 Binary Search Tree Property
• The key of any node is:
> keys of all nodes in its left subtree and
≤ keys of all nodes in its right subtree

30
 Concrete Implementation
• Node in a Binary Tree
1 struct Node {
2 TYPE key;
3 Node *left, *right;
4 Node(const TYPE &k) :
5 key{k}, left{nullptr}, right{nullptr} {}
6 };
• Note: template removed (compared to previous lecture)
to make code simpler in following slides

31
 1 void inorder(Node *x) {
Exercise 2

3
if (!x) return;
inorder(x->left);
4 print(x->key);
• Write the output for inorder, 5 inorder(x->right);
preorder and post-order 6 } // inorder()
traversals of this BST 7 void preorder(Node *x) {
8 if (!x) return;
9 print(x->key);
10 preorder(x->left);
11 preorder(x->right);
12 } // preorder()
13 void postorder(Node *x) {
14 if (!x) return;
15 postorder(x->left);
16 postorder(x->right);
17 print(x->key);
18 } // postorder()
32
 Sort: Binary Search Tree
Can you think of an easy method of
sorting using a binary search tree?

33
 Search
• How can we find a key in a binary search tree?

// return a pointer to node with key k if

// one exists; otherwise, return nullptr
Node *tree_search(Node *x, TYPE k);

• (remember) The key of any node is:

> keys of all nodes in its left subtree and
≤ keys of all nodes in its right subtree
• Bonus: what are the average and worst-case
complexities?
35
 Search
1 // return a pointer to node with key k if
2 // one exists; otherwise, return nullptr
3 Node *tree_search(Node *x, TYPE k) {
4 while (x != nullptr && k != x->key) {
5 if (k < x->key)
6 x = x->left;
7 else
8 x = x->right;
9 } // while
10 return x;
11 } // tree_search()
36
 Search
1 // return a pointer to node with key k if
2 // one exists; otherwise, return nullptr
3 Node *tree_search(Node *x, TYPE k) {
4 if (x == nullptr || x->key == k) return x;
5 if (k < x->key)
6 return tree_search(x->left, k);
7 return tree_search(x->right, k);
8 } // tree_search()
• Same as BST
• To search for a key k, we trace a downward path starting
at the root
• The next node visited depends on the outcome of the
comparison of k with the key of the current node
37
 Search Example
tree_search(root, 9);

38
 Search
• Complexity is O(h), where h is the
(maximum) height of the tree
• Average case complexity: O(log n)
– Balanced tree
• Worst case complexity: O(n)
– "Stick" tree

39
 Insert
• How do we insert a new key into the tree?
• Similar to search
• Start at the root, and trace a path
downwards, looking for a null pointer to
append the node

40
 Insert Example
tree_insert(root, 10);

10
41
 Insert with Duplicates
• For sets with no duplicates, use (<, >)
• For duplicates, we need a deterministic policy
– Choose (<= , >) or (<, >=)… slides use (<, >=)

42
 Insert
1 void tree_insert(Node *&x, TYPE k) {
2 if (x == nullptr)
3 x = new Node(k);
4 else if (k < x->key)
5 tree_insert(x->left, k);
6 else
7 tree_insert(x->right, k);
8 } // tree_insert()

• New node inserted at leaf

• Note the nifty(?) use of reference-to-pointer-to-Node
• Exercise: modify this code to set the parent pointer
43
 Exercise
• Start with an empty tree
• Insert these keys, in this order:
12, 5, 18, 2, 9, 15, 19, 17, 13
• Draw the tree
• Write a new order to insert the same keys
which generates a worst-case tree
• How many worst-case case trees are
possible for n unique values?
44
 Complexity
• The complexity of insert (and many other
tree functions) depends on the height of
the tree
• Average case (balanced): O(log n)
• Worst case (unbalanced "stick"): O(n)
• Average case:
– Random data
– Likely to be well-balanced

47
 Exercise
• Write a function to find the Node with the
smallest key
• What are the average and worst-case
complexities?
// Return a pointer to the Node with the min key
Node *tree_min(Node *x);

48
 Remove
• What if we want to remove a node?
• To remove node z:
1. z has no children (trivial)
2. z has no left child
3. z has no right child
4. z has two children

• Complete algorithm is in CLRS 12.3

– Very confusing, stick to slides 50
 Remove: Easy Case 1
• z has no left child: replace z by right child

51
 Remove: Easy Case 2
• z has no right child: replace z by left child

52
 Remove: Hard Case
• z has left and right children
• Replace with a “combined” tree of both
• Key observation
– All in LHS subtree < all in RHS subtree
– Transplant smallest RHS node to root
• Called the inorder successor
• Must be some such node, since RHS is not empty
• New root might have a right child, but no left child
– Make new root’s left child the LHS subtree
53
 Remove: Hard Case
To Remove V:

Replace V
Find smallest Remove V
with smallest
node in RHS from RHS
node’s value
54
 Single-Function Remove 1/3
1 template <typename T>
2 void BinaryTree<T>::remove(Node *&tree, const T &val) {
3 Node *nodeToDelete = tree;
4 Node *inorderSuccessor;
5
6 // Recursively find the node containing the value to remove
7 if (tree == nullptr)
8 return;
9 else if (val < tree->value)
10 remove(tree->left, val);
11 else if (tree->value < val)
12 remove(tree->right, val);
13 else {

55
 Single-Function Remove 2/3
14 // Check for simple cases where at least one subtree is empty
15 if (tree->left == nullptr) {
16 tree = tree->right;
17 delete nodeToDelete;
18 } // if
19 else if (tree->right == nullptr) {
20 tree = tree->left;
21 delete nodeToDelete;
22 } // else if

56
 Single-Function Remove 3/3
23 else {
24 // Node to delete has both left and right subtrees
25 inorderSuccessor = tree->right;
26
27 while (inorderSuccessor->left != nullptr)
28 inorderSuccessor = inorderSuccessor->left;
29
30 // Replace value with the inorder successor’s value
31 nodeToDelete->value = inorderSuccessor->value;
32 // Remove the inorder successor from right subtree
33 remove(tree->right, inorderSuccessor->value);
34 } // else
35 } // else
36 } // BinaryTree::remove()
57
Summary: Binary Search Trees
• Each node points to two children (left,
right), and possibly a parent
• All nodes are ordered: left < root ≤ right
• Modification of nodes
– External is easy
– Internal is more complicated
• In general, operations on BSTs are:
– O(log n) average
– O(n) worst case
58
 Tree Height Review

Height:
height(empty) = 0;
height(node) = max(height(children)) + 1;

59
 AVL Tree
Named for Adelson-Velskii, and Landis

• Start with a BST

• Add the Height Balance Property
– For every internal node v of T, the heights of
the children of v differ by at most 1
– Use rotations to correct imbalance
• Worst case search/insert is now:
O(log n)  Proof coming soon
60
 Is this an AVL tree?
• Height Balance Property
– For every internal node v of T, the heights of the
children of v differ by at most 1

Tree 0 Tree 1 Tree 2 Tree 3

5 5 5

2 2

61
 Is this an AVL tree?
• Height Balance Property
– For every internal node v of T, the heights of the
children of v differ by at most 1

Tree 4 Tree 5 Tree 6

5 5 5

2 7 2 7 2 7

3 3 6 3 9

8
62
 AVL Tree
Proof Setup
• h: height of tree
• n(h): minimum number of nodes in a tree of height h

• n(0) = 0 (Empty tree)

• n(1) = 1 (Root-only tree)

• For h > 1, an AVL tree of height h contains:

– Root Node 5
– AVL Subtree of height h-1 2 7
– AVL Subtree of height h-2
3
• Thus for h > 1, n(h) = 1 + n(h - 1) + n(h - 2) 63
 Proof of Height Balance
Property
• h: height of tree
• n(h): minimum number of nodes in a tree of height h
• n(h) = 1 + n(h - 1) + n(h - 2)

• Knowing n(h - 1) > n(h - 2) and n(h) > 2n(h - 2),

then by induction, n(h) > 2in(h - 2i)
• Closed form solution, n(h) > 2h/2 - 1
5
• Taking logarithms: h < 2 log n(h) + 2
• Thus the height of the tree is O(log n) 2 7

3
64
 AVL Tree Algorithms
• Search is the same as a BST
• Sort (same as BST)
– Insert nodes one at a time
• What is worst case complexity now?
– Perform an inorder traversal
• Still O(n) for this step

65
 AVL Tree Insert
• The basic idea:
1. Insert like a BST
2. Rearrange tree to balance height
• Each node records its height
• Can compute a node’s balance factor
– bal(n) = height(n->left) – height(n->right)
• A node that is AVL-balanced:
– bal(n) = 0: both subtrees equal
– bal(n) = +1: left taller by one
– bal(n) = –1: right taller by one
• |bal(n)| > 1: node is out of balance
66
 Balance Factor Example
• What is the height for each node?
height(node) = m ax(height(children)) + 1;

• What is the balance factor?

bal(n) = height(n->left) – height(n->right)

5 5

2 7 2 7

3 3

4 67
Balance Factor Example

-2 -2

+1
-1

0
0

bal(n) = height(n->left) – height(n->right)

68
 Balance Factor Exercise
Label the balance factor on each node

bal(n) = height(n->left) – height(n->right)

69
 Insert (begins like BST)
tree_insert(root,14);

Unbalanced!
70
 Rotations
• We use rotations to rebalance the binary tree
– Swap a parent and one of its children
– BUT preserve the BST ordering property

5 5

2 7 3 7

3
2 4
4

• Rotation is a local change involving only

three pointers and two nodes 71
 Rotations
• We use rotations to rebalance the binary tree
– Interchange the role of a parent and one of its
children in a tree…
– Preserve the BST ordering among the keys in the
nodes
• The second part is tricky
– Right rotation: copy the right pointer of the left child to
be the left pointer of the old parent
– Left rotation: copy the left pointer of the right child to
be the right pointer of the old parent
Rotation is a local change involving only three pointers and
two nodes
72
 Rotations
Rotate Right: RR(P) Rotate Left: RL(P)
P P

LC RC
P P
RST LST

C C C C
LST RST LST RST

LC RC

P P
C C
LST RST

C P
RST RST P C
LST LST
73
 Rotation Exercise
• Use rotations to balance these trees

A A

C
B

B
C

74
 Exercise
A B • First tree is easy:
B A C
rotate-left(A)
• single rotation
C

A A • Second is tricky:
C B
rotate-right(C)
rotate-left(A)
B C • double rotation
B

A C 75
 Insert
Four Cases
1. Single left rotation
• RL(a)
2. Single right rotation
• RR(a)
3. Double rotation
a. RR(c)
b. RL(a)
4. Double rotation
a. RL(a)
b. RR(c)

76
 Single Rotations

a b
b single rotation a c
c
RL(a)
T0 T3
T1 T3 T0 T1 T2
T2

c b
b single rotation a c
a
RR(c)
T3 T0
T0 T2 T1 T2 T3
T1 77
 Double Rotations
a
a
c b
b c
RR(c)
T0
T0
T2 T3 T1
T1 T T
2
3
double rotation RL(a)
is needed

b
a c

T2
T0 T1 T3
78
 Double Rotations
c c
a b
b
a
RL(a)
T3 T3
T0 T1 T2
T2 T1
T0
RR(c)
double rotation
is needed

b
a c

T1
T0 T2 T3
79
http://upload.wikimedia.org/wikipedia/commons/c/c4/Tree_Rebalancing.gif 80
 Checking and Balancing
• As insert finishes, after every recursive call,
update height of current node, then perform this
check on every node along the insertion path:
Algorithm checkAndBal(Node *n)
if bal(n) > +1
if bal(n->left) < 0
rotateL(n->left)
rotateR(n)
else if bal(n) < -1
if bal(n->right) > 0
rotateR(n->right)
rotateL(n)
• What is complexity of this?
• How many “fixes” are needed after insert? 81
 Checking and Balancing

• Outermost if:
Q: Is node out of
Algorithm checkAndBal(Node *n)
if bal(n) > +1 balance?
if bal(n->left) < 0 A: > +1: left too big
rotateL(n->left)
< -1: right too big
rotateR(n)
else if bal(n) < -1 • Inner ifs:
if bal(n->right) > 0 Q: Do we need a
rotateR(n->right)
rotateL(n)
double rotation?
A: Only if signs disagree
82
 Rotation Exercise
• Insert these keys into an AVL tree,
rebalancing when necessary
• 3, 2, 1, 4, 5, 6, 7, 16, 15, 14

83
 Show your work here
• Answer is on next slide

84
 Rotation Exercise
• Insert 3
• Insert 2
• Insert 1
• Rebalance  RR(3)

• Remaining: 4, 5, 6, 7, 16, 15, 14

85
Work

86
 Rotation Exercise
• Insert 4
• Insert 5
• Rebalance  RL(3)

• Remaining: 6, 7, 16, 15, 14

87
Work

88
 Rotation Exercise
• Insert 6
• Rebalance  RL(2)

• Remaining: 7, 16, 15, 14

89
Work

90
 Rotation Exercise
• Insert 7
• Rebalance  RL(5)

• Remaining: 16, 15, 14

91
Work

92
 Rotation Exercise
• Insert 16
• Insert 15
• Rebalance  RR(16), RL(7)

• Remaining: 14
93
Work

94
 Rotation Exercise
• Insert 14
• Rebalance  RR(15), RL(6)

95
 AVL Tree Remove
• The basic idea:
1. Remove like a BST
2. Rearrange tree to balance height
• Key observation
– All keys in LHS <= all in keys RHS
– Note that we wanted < on the left, but rebalancing may
rearrange duplicate values
– Rearrange RHS so that its smallest node is its root
• Must be some such node, since RHS is not empty
• New RHS root has a right child, but no left child
– Make the RHS root’s left child the LHS root

96
 Remove in an AVL Tree
• Remove as in a binary search tree
• Rebalance if an imbalance has been created
44 44

17 62 17 62
Remove 32
32 50 78 50 78

48 54 88 48 54 88

Before: balanced After: unbalanced

Now we rebalance
97
 Rebalancing after a Remove
• Travel up the tree from w, the parent of the removed node
• At the first unbalanced node encountered, rotate as needed
• This restructuring may unbalance one of its ancestors, so
we continue checking and rebalancing up to the root
44 62
w
17 62 44 78
RL(44)
32 50 78 17 50 88

48 54 88 48 54

• How many “fixes” are needed after remove?

98
 Rebalancing after a Remove
Want to remove 62 Copy 78
62 Find inorder successor over 62
44 78
78

44 78
17 50 88

17 50 88
48 54

48 54
78

44 88

17 50 Remove 78
from RHS
48 54 99
Rebalancing after a Remove

First:
78 RL(44) 78

44 88 50 88

17 50 44 54

48 54 17 48

78: +2 Bal
Need double rotation:
RL(44), RR(78)
44: -1 Bal
100
 Rebalancing after a Remove
Must finish double rotation;
do not recalculate balance

78 Second:
RR(78)
50 88
50

44 54
44 78

17 48
17 48 54 88

101
 Summary
• Binary Search Tree
– Worst case insert or search is O(n)
• AVL Tree
– Worst case insert or search is O(log n)
– Must guarantee height balance property
• Operations
– Search: O(log n) (same algorithm as BST, but faster)
– Insert: O(log n) (Starts like BST, then may rebalance)
– Remove: O(log n) (Starts like BST, then may rebalance)
– Sort: O(n log n) to build the tree, O(n) to do inorder traversal
• Rotation:
– O(1) cost for single rotation, O(1) cost for double rotation
102
 Useful Website
• https://visualgo.net/bn/bst
• Close the help
• Click on “AVL TREE” near the top
• You can insert several nodes at once, or
one at a time, slow down or speed up the
demo

103