CSE 326 Lecture 7: More on Search Trees

! Todays Topics: " Lazy Operations " Run Time Analysis of Binary Search Tree Operations " Balanced Search Trees # AVL Trees and Rotations ! Covered in Chapter 4 of the text

R. Rao, CSE 326

From Last Time: Remove (Delete) Operation

! Removing a node containing X:

94

1. Find the node containing X 2. Replace it with: If it has no children, with NULL If it has 1 child, with that child If it has 2 children, with the node with the smallest value in its right subtree, (or largest value in left subtree) 3. Recursively remove node used in 2 and 3
! Worst case: Recursion propagates all

10

97

24

the way to a leaf node time is O(depth of tree)

R. Rao, CSE 326

11

17

Laziness in Data Structures

! A lazy operation is one that puts off work as much as

possible in the hope that a future operation will make the current operation unnecessary

Da t a Stru -ure ct s

R. Rao, CSE 326

Lazy Deletion
! Idea: Mark node as deleted; no need to reorganize tree

" Skip marked nodes during Find or Insert " Reorganize tree only when number of marked nodes exceeds a percentage of real nodes (e.g. 50%) " Constant time penalty only due to marked nodes depth increases only by a constant amount if 50% are marked undeleted nodes (N nodes max N/2 marked)
! Modify Insert to make use of marked nodes whenever

possible e.g. when deleted value is re-inserted

! Can also use lazy deletion for Lists

R. Rao, CSE 326

Run Time Analysis of BST operations

! All BST operations (except MakeEmpty) are O(d), where d is

the depth of the accessed node in the tree

" MakeEmpty takes O(N) for a tree with N nodes frees all nodes
! We know: log N ! d ! N-1 for a binary tree with N nodes

" What is the best case tree? What is the worst case tree? ! Best Case Running Time of Insert/Remove/etc. = ? ! Worst Case Running Time = ? ! Average Case Running Time = ?

R. Rao, CSE 326

The best, the worst, and the average

! For a binary tree with N nodes, depth d of any node satisfies:

log N ! d ! N-1

! So, best case running time of BST operations is O(log N) ! Worst case running time is O(N) ! Average case running time = O(average value of d) = O(log N)

" Can prove that average depth over all nodes = O(log N) if all insertion sequences equally likely. " See Chap. 4 in textbook for proof

R. Rao, CSE 326

Can we do better?
! Worst case running time of BST operations is O(N)

! E.g. What happens when you Insert elements in ascending

(or descending) order?

" Insert 2, 4, 6, 8, 10, 12 into an empty BST
! Problem: Lack of balance Tree becomes highly asymmetric ! Idea: Can we restore balance by re-arranging tree according to

depths of left and right subtrees? " Goal: Get depth down from O(N) to O(log N)

R. Rao, CSE 326

Idea #1: Achieving the perfect balance

! First try at balancing trees: Perfect balance

" Re-arrange to get a complete tree after every operation

! Recall: A tree is complete if there are no

6 4 1 9 5 8
Insert 2 & make tree complete

holes when scanning from top to bottom, left to right

! Problem: Too expensive to re-arrange

" E.g. Insert 2 in the example shown

! Need a looser constraint

5 2 1

8 4 6 9
8

R. Rao, CSE 326

Idea #2: Leave it to the professionals

! Many efficient algorithms exist for balancing trees in

order to achieve faster running times for the BST operations " Adelson-Velskii and Landis (AVL) trees (1962) " Splay trees and other self-adjusting trees (1978) " B-trees and other multiway search trees (1972)

R. Rao, CSE 326

AVL Trees
! AVL trees are height-balanced binary

6 3 Heights 1 4 1 0 9 2 7 1 8 0 -1 6 1 4 0 1 9 Balance factors 1-(-1) = 2

search trees
! Balance factor of a node = height(left

subtree) - height(right subtree)

! An AVL tree can only have balance

factors of 1, 0, or 1 at every node

" For every node, heights of left and right subtree differ by no more than 1 " Height of an empty subtree = -1 ! Implementation: Store current

7 (-1)-0 = -1 8 0
10

heights in each node

R. Rao, CSE 326

Which of these are AVL trees?

6 4 1 9 7 8 6 4 1 5 9 1 4 6 7 9 3 4 5 1 4 6 9 5 8 6 7 8 9
11

R. Rao, CSE 326

AVL Trees: Examples and Non-Examples

Balance -1 6 Not AVL factors 9 1 4 1-(-1) = 2 0 1 7 (-1)-0 = -1 8 0 AVL 0 4 0 1 5 0 6 1 9 0 1 4 0 1 6 0 7 -1 0 9 AVL 6 0 AVL 0 4 0 1 9 1 5 8 0 0 6 0 2 5 1 4 0 3 Not AVL 7 -2 8 -1 9 0
12

R. Rao, CSE 326

The good news about AVL Trees

! Can prove: Height of an AVL tree of N

nodes is always O(log N)

! How? Can show:

6 0 0 4 0 1 9 1 5 8 0 0 6 0 1 4 7 -1 0 9
13

" Height h ! 1.44 log(N+2)-0.328 " Prove using recurrence relation for minimum number of nodes S(h) in an AVL tree of height h: S(h) = S(h-1) + S(h-2) + 1 " Use Fibonacci numbers to get bound on S(h) bound on height h Height = " See textbook for details

O(log N) 0 1

R. Rao, CSE 326

The really good news about AVL Trees

! Can prove: Height of an AVL tree of N

nodes is always O(log N)

! All operations (e.g. Find, Remove

using lazy deletion, etc.) on an AVL tree are O(log N)

! except Insert

Insert 3 6 0 1 5 0 4 7 -1 0 9
14

" Why is Insert different?

R. Rao, CSE 326

The bad news about AVL Trees

6 0 1 5 0 4 7 -1 0 9 Insert 3

6 1

2
1 4 0 3

7 -1 0 9

No longer an AVL tree (i.e. not balanced anymore)

R. Rao, CSE 326 15

Restoring Balance in (the life of) an AVL Tree

! Problem: Insert may cause balance factor to become 2 or 2

for some node on the path from insertion point to root node
! Idea: After Inserting the new node,

1. Back up to root updating heights along the access path 2. If Balance Factor = 2 or 2, adjust tree by rotation around deepest such node. 6 1

2
1 4
R. Rao, CSE 326

7 -1 0 9
16

0 3

Rotating to restore Balance: A Simple Example

6 0 1 5 0 4 7 -1 0 9

Insert 3

6 1

Rotate 0 4 0 3

6 0 7 -1 5 0 AVL 9 0

2
1 4 0 3

7 -1 0 9

AVL

Not AVL

R. Rao, CSE 326

17

Various Cases of Insertion

BF = 1 BF = 1 BF = 0 BF = 0 BF = 0 BF = 0 BF = 0 BF = 0 BF = 0

Tree before insertion (BF = Balance Factor)

R. Rao, CSE 326

18

Outside Case

BF = ? BF = ? BF = ? BF = ? BF = 0 BF = 0 BF = 0

Tree after insertion

R. Rao, CSE 326

19

Inside Case

BF = BF = BF = BF = BF = 0 BF = BF = BF = BF = BF =

Tree after insertion

R. Rao, CSE 326

20

Insertions in AVL Trees

Let the node that needs rebalancing be ". There are 4 cases: Outside Cases (require single rotation) : 1. Insertion into left subtree of left child of ". 2. Insertion into right subtree of right child of ". Inside Cases (require double rotation) : 3. Insertion into right subtree of left child of ". 4. Insertion into left subtree of right child of ". Rebalancing is performed through four separate rotation algorithms.
R. Rao, CSE 326 21

Insertions in AVL Trees: Outside Case

Consider a valid AVL subtree

j Z

k X
R. Rao, CSE 326

Y
22

Insertions in AVL Trees: Outside Case

Inserting into X destroys the AVL property

j Z Y

k X
R. Rao, CSE 326 23

Insertions in AVL Trees: Outside Case

j k

Do a right rotation

Z Y X
R. Rao, CSE 326 24

Insertions in AVL Trees: Outside Case

j k

Do a right rotation

Z Y X
R. Rao, CSE 326 25

Insertions in AVL Trees: Outside Case

k j X
R. Rao, CSE 326

Right rotation done!

Z
26

AVL property has been restored!

(Left rotation is mirror symmetric)

Insertions in AVL Trees: Inside Case

Consider a valid AVL subtree

j Z

k X
R. Rao, CSE 326

Y
27

Insertions in AVL Trees: Inside Case

Inserting into Y destroys the AVL property

j k

Does right rotation restore balance?

Z X
R. Rao, CSE 326

Y
28

Insertions in AVL Trees: Inside Case

k j X

Right rotation dont do nothin to restore balance

Z Y
R. Rao, CSE 326 29

Insertions in AVL Trees: Inside Case

Consider the structure of subtree Y

j Z

k X
R. Rao, CSE 326

Y
30

Insertions in AVL Trees: Inside Case

Y = node i and subtrees V and W

j i Z W
31

k X V
R. Rao, CSE 326

Insertions in AVL Trees: Inside Case

j k X V
R. Rao, CSE 326

Lets try a left-right double rotation . . .

i W

32

Insertions in AVL Trees: Inside Case

Steps for Left-Right Double Rotation

1. Left Rotation: Adjust relevant pointers

i k W X
R. Rao, CSE 326

Z V
33

Insertions in AVL Trees: Inside Case

Steps for Left-Right Double Rotation

i j W

k X
R. Rao, CSE 326

2. Right Rotation: Adjust relevant pointers 3. Make i the root

Z
34

Balance has been restored! (Right-left case is mirror-symmetric)

AVL Tree Exercise

! Insert 8, 1, 0, 2 in that order into following AVL tree:

4 3 6

R. Rao, CSE 326

35

Pros and Cons of AVL Trees

Arguments for AVL trees: 1. Search is O(log N) since AVL trees are always balanced. 2. The height balancing adds no more than a constant factor to the speed of insertion. Arguments against using AVL trees: 1. Difficult to program & debug; more space for height info. 2. Asymptotically faster but can be slow in practice. 3. Most large searches are done in database systems on disk and use other structures (e.g. B-trees). 4. May be OK to have O(N) for a single operation if total run time for many consecutive operations is fast (e.g. Splay trees).
R. Rao, CSE 326 36

Next Class: All about Splaying

To Do: Finish Reading Chapter 4

R. Rao, CSE 326

37