Lecture 19: Swinging from Up-Trees to Graphs

! Todays Agenda: " Smart Union and Find # Union-by-size/height and Path Compression " Run Time Analysis as tough as it gets! " Introduction to Graphs

! Covered in Chapters 8 and 9 in the textbook

R. Rao, CSE 326

Some of the material on these slides are courtesy of: S. Wolfman, CSE 326, 2000

Recall: Disjoint Set ADT

! Disjoint set ADT: Used to represent a collection of sets

containing objects that are related to each other

" Relations defined through Union operation " Union merges two sets their objects become related. " Find returns the name of the set an object belongs to

Example: Initial Classes = {1,4,8}, {2,3}, {6}, {7}, {5,9,10} Name of equiv. class underlined
R. Rao, CSE 326

find(4) {1,4,8} 8 {5,9,10} {2,3} union(2,6)

2

{6} {7} {2,3,6}

Recall: Up-Tree Data Structure

! Each equivalence class (or NULL NULL NULL

discrete set) is an up-tree with its root as its representative member

! All members of a given set

a b e {a,d,g,b,e}

c f

are nodes in that sets uptree

{c,f} {h}

R. Rao, CSE 326

Example of Find
Find: Just follow parent pointers to the root! find(f) = c find(e) = a Runtime depends on tree depth -(Tree size) Array up:
R. Rao, CSE 326

a d b e

c f

h i

0 (a) 1 (b) 2 (c) 3 (d) 4 (e) 5 (f) 6 (g) 7 (h) 8 (i)

-4

-2

-1

-2

7
4

Example of Union
Union: Just hang one root from the other! union(c,a) a d b e
0 (a) 1 (b) 2 (c) 3 (d) 4 (e) 5 (f) 6 (g) 7 (h) 8 (i)

c f

h i

Runtime = O(1)

Array up:
R. Rao, CSE 326

-6

-1

-2

7
5

Change a (from -4) to c (= 2), update size

Smart Union?
! For M Finds and N-1 Unions, worst case time is O(MN+N)

" Can we speed things up by being clever about growing our up-trees?

a b e d

c f

g h i b e a

c f d

g h i

Union(c,a)
R. Rao, CSE 326

Smart Union/Find: Union-by-Size

! Idea: In Union, always make root of larger tree the new root ! Why? Minimizes height of the new up-tree

a b e d

c f

g h i b e a

c f d

g h i b e

a d

g h c f i

Union(c,a)
R. Rao, CSE 326

Union-by-Size!
7

Union-by-Size: Run-Time Analysis

! Finds are O(max up-tree height) for a forest of

up-trees containing N nodes

! Number of nodes in an up-tree of height h using

union-by-size is ! 2h (prove by induction)

" Pick up-tree with max height " Then, N ! 2max height " max height # log N ! Find takes O(log N) when Union-by-Size is used " Same result with Union-by-Height (see text)

R. Rao, CSE 326

Smart Find?
Find(e) = a Runtime depends on tree depth d a b e If we do M Finds on the same element " O(M log N) time Can we make Find have side-effects so that next Find will be faster?
R. Rao, CSE 326 9

c f

h i

IntroducingPath Compression
! Path Compression: Point everything along path of a Find to root ! Reduces height of entire access path to 1: Finds get faster!

" Dj vu? " Idea similar to the one behind your old buddy the splay tree

a b e d

g
Find(e)

a b e d

Path compression
R. Rao, CSE 326 10

A P.C. example with more meat

Find(e) c a b i e d d i f h g a b c f e h g

R. Rao, CSE 326

11

How to P.C. Path Compression Code

public int Find(int X) { // Assumes X = Hash(X_Element) // X_Element could be str/char etc. if (up[X] < 0) // Root return X; //Return root = set name else //Find parent and update pointer to root return up[X] = Find(up[X]); }

Make all nodes along access path point to root

Trivial modification of original recursive Find New running time = ?

R. Rao, CSE 326 12

How to P.C. Path Compression Code

public int Find(int X) { // Assumes X = Hash(X_Element) // X_Element could be str/char etc. if (up[X] < 0) // Root return X; //Return root = set name else //Find parent and update pointer to root return up[X] = Find(up[X]); }

Collapsing the tree by pointing to root

Find still takes O(max up-tree height) worst case But what happens to the tree heights over time? What is the amortized run time of Find if we do M Finds?
R. Rao, CSE 326 13

What is the amortized run time per operation if we do a sequence of M Unions or Finds using Union-by-Size & P.C.?

10-second break to solve this problem

R. Rao, CSE 326

14

What is the amortized run time per operation if we do a sequence of M Unions or Finds using Union-by-Size & P.C.?

If you succeeded in solving thisyou shouldnt be in this class! One of the toughest run-time analysis problems ever!
(no, a similar problem wont be in the final)
Fine print: The final has not been written up yet, so we are not responsible for any statements about
the final made in this lecture or indeed any future lectures, but not including the final review lecture

R. Rao, CSE 326

15

Analysis of P.C. with Union-by-Size

! R. E. Tarjan (of the up-trees fame) showed that:

" When both P.C. and Union-by-Size are used, the worst case run time for a sequence of M operations (Unions or Finds) is $(M %(M,N))
! What is %(M,N)?

" %(M,N) is the inverse of Ackermanns function

! What is Ackermanns function?

R. Rao, CSE 326

16

Digression: Them slow-growing functions

! How fast does log N grow? log N = 4 for N = 16 = 2

2 2

" Grows quite slowly

! Let log(k) N = log (log (log (log N)))

(k logs)
2 2

! Let log* N = minimum k such that log(k) N # 1 ! How fast does

N grow? " Grows very slowly

log*

log*

N = 4 for N = 65536 = 22

R. Rao, CSE 326

17

Ackermann and his function

! Ackermann created a really explosive function

A(i, j) whose inverse %(M, N) grows very, very slowly (slower than log* N)
A(1, j ) ' 2 j for j ! 1 A(i ,1) ' A(i & 1,2) for i ! 2
Go aheadtry out some values for i, j

A(i , j ) ' A(i & 1, A(i , j & 1)) for i, j ! 2

% ( M , N ) ' min*i ! 1 | A(i, (M / N )) , log N +

R. Rao, CSE 326 18

How slowly does %(M, N) grow? %(M, N) # 4 for all practical choices of M and N (%(M, N) = 4 for M far larger than the number of atoms in
the universe (2300)!! (assuming M ! N))
Mighty profoundbut what in my dog Skips name does all this have to do with Unions and Finds?

R. Rao, CSE 326

19

Back to Smart Unions/Finds

! R. E. Tarjan showed that:

" When both P.C. and Union-by-Size are used, the worst case total run time for any sequence of M Unions and Finds is $(M%(M,N))
! Textbook proves weaker result of O(M log* N) time

" Requires 6 pages and 8 Lemmas! (Check it out!)

! Amortized run time per operation

= total time/no. of operations = $(M%(M,N))/M = $(%(M,N)) - $(1) for all practical purposes (%(M, N) # 4 for all practical M, N) - constant time!
20

R. Rao, CSE 326

Summary of Disjoint Set and Union/Find

! The Disjoint Set ADT allows us to represent objects that fall

into different equivalence classes or sets

! Two main operations: Union of two classes and Find class

name for a given element

! Up-Tree data structure allows efficient array implementation

" Unions take O(1) worst case time, Finds can take O(N) " Union-by-Size (or by-Height) reduces worst case time for Find to O(log N) " If we use both Union-by-Size/Height & Path Compression:
# Any sequence of M Union/Find operations results in O(1)

amortized time per operation (for all practical purposes)

R. Rao, CSE 326 21

Applications of Disjoint Sets

! Disjoint sets can be used to represent:

" " " " "

Cities on a map (disjoint sets of connected cities) Electrical components on chip Computers connected in a network Groups of people related to each other by blood Textbook example: Maze generation using Unions/Finds: # Start with walls everywhere and each cell in a set by itself # Knock down walls randomly and Union cells that become connected # Use Find to find out if two cells are already connected # Terminate when starting and ending cell are in same set i.e. connected (or when all cells are in same set)
22

R. Rao, CSE 326

We are now ready to tackle the grandmama of all data structures

The crme de la crme The most general, the all-encompassing Graphs and their algorithms!

R. Rao, CSE 326

23

What are graphs? (Take 1)

! Yes, this is a graph.

! But we are interested in a different kind of graph

R. Rao, CSE 326 24

Motivation for Graphs

! Consider the data structures we

have looked at so far

! Linked list: nodes with 1 incoming

node Value Next

node Value Next

edge + 1 outgoing edge

! Binary trees/heaps: nodes with 1

94

incoming edge + 2 outgoing edges

! Binomial trees/B-trees: nodes with

10 a g b

97

1 incoming edge + multiple outgoing edges

! Up-trees: nodes with multiple

96

99
25

incoming edges + 1 outgoing edge

R. Rao, CSE 326

Motivation for Graphs

! What is common among these data structures? ! How can you generalize them? ! Consider data structures for representing the following

problems

R. Rao, CSE 326

26

Course Prerequisites for CSE at UW

143 322 321 326 370 341

142

378 421

Nodes = courses Directed edge = prerequisite

R. Rao, CSE 326

401

27

Representing a Maze or Floor Plan of a House

F

Nodes = rooms Edge = door or passage

B
R. Rao, CSE 326 28

Representing Electrical Circuits

Battery Switch

Nodes = battery, switch, resistor, etc. Edges = connections

R. Rao, CSE 326

Resistor

29

Representing Expressions in Compilers

x1=q+y*z x2=y*z-q Naive: y*z calculated twice common subexpression eliminated: x1 + q y * z x1 + q * y z x2 q * x2 q

Nodes = symbols/operators Edges = relationships

R. Rao, CSE 326

30

Information Transmission in a Computer Network

56 Seoul 16 30 L.A. Sydney Nodes = computers Edges = transmission rates
31

Tokyo

Seattle 128 181 140 New York

R. Rao, CSE 326

Traffic Flow on Highways

UW

Nodes = cities Edges = # vehicles on connecting highway

R. Rao, CSE 326

32

Six Degrees of Separation from Kevin Bacon

Apollo 13 Gary Sinise Rosanna Arquette
After Hours

Cheech Marin

Tom Hanks
F or es

3 Apollo 1

sp De a er te ly ek Se

de Bri s s ce r in eP h T incess Bride Robin The Pr Wright

Wallace Shawn

in g n sa Su

t Gum

Cary Elwes

Toy Story

Laurie Metcalf
33

R. Rao, CSE 326

Soap Opera Relationships

Victor Ashley Wayne Brad Trisha Peter

R. Rao, CSE 326

34

Graphs: Definition
! A graph is simply a collection of nodes plus edges

" Linked lists, trees, and heaps are all special cases of graphs
! The nodes are known as vertices (node = vertex) ! Formal Definition: A graph G is a pair (V, E) where

" V is a set of vertices or nodes " E is a set of edges that connect vertices

R. Rao, CSE 326

35

Graphs: An Example
! Here is a graph G = (V, E)

" Each edge is a pair (v1, v2), where v1, v2 are vertices in V V = {A, B, C, D, E, F} E = {(A,B), (A,D), (B,C), (C,D), (C,E), (D,E)} B C A F D
R. Rao, CSE 326

E
36

Directed versus Undirected Graphs

! If the order of edge pairs (v1, v2) matters, the graph is

directed (also called a digraph): (v1, v2) . (v2, v1) v1 v2

! If the order of edge pairs (v1, v2) does not matter, the graph is

called an undirected graph: in this case, (v1, v2) = (v2, v1) v1

R. Rao, CSE 326

v2
37

Graph Representations
Space and time are measured in terms of both: Number of vertices = |V| and Number of edges = |E| There are two ways of representing graphs: The adjacency matrix representation The adjacency list representation

R. Rao, CSE 326

38

Graph Representation: Adjacency Matrix

The adjacency matrix representation:
1 M(v, w) = B A F D
R. Rao, CSE 326

Space = ? C 0 1 0 1 1 0 D 1 0 1 0 1 0 E 0 0 1 1 0 0 F 0 0 0 0 0 0
39

if (v, w) is in E otherwise A B C C D E E F

A 0 1 0 1 0 0

B 1 0 1 0 0 0

Adjacency Matrix for a Digraph

1 M(v, w) = B C A F D E 0 if (v, w) is in E otherwise A B C D E F
R. Rao, CSE 326

Space = |V|2 A 0 0 0 0 0 0 B 1 0 0 0 0 0 C 0 1 0 0 0 0 D 1 0 1 0 0 0 E 0 0 1 1 0 0 F 0 0 0 0 0 0
40

Graph Representation: Adjacency List

The adjacency list representation: For each v in V, L(v) = list of w such that (v, w) is in E
Space = ? A B A F D
R. Rao, CSE 326

B A B A C

D C D C D
41

B C D

E E

E F

Graph Representation: Adjacency List

Space = a |V| + 2 b |E| a A B A F D
R. Rao, CSE 326

b B A B A C D C D C D
42

B C D

E E

E F

Adjacency List for a Digraph

Space = ? a B C A F D Digraph
R. Rao, CSE 326

b B C D E Adjacency List
43

A B C D E F

Adjacency List for a Digraph

Space = a |V| + b |E| a B C A F D Digraph
R. Rao, CSE 326

b B C D E Adjacency List
44

A B C D E F

Graphs: Problem #1: Topological Sort

Graph of course prerequisites
142 143 322 321 326 370 341

Problem: Find an order in which all these courses can be taken. Example: 142, 143, 378, 370, 321, 341, 322, 326, 421, 401
R. Rao, CSE 326

378 421 401

To take a course, all its prerequisites must be taken first 45

Topological Sort Definition

Topological sorting problem: given digraph G = (V, E), find a linear ordering of its vertices such that: for any edge (v, w) in E, v precedes w in the ordering
B C A F D E
46

How would you Topo-Sort this digraph given an adjacency list representation of G = (V, E)?

R. Rao, CSE 326

Next Class: Getting intimate with Topo-sorts Finding shortest ways to get to your classrooms To Do: Homework #4 (to be posted on class web Monday 2/24) Read and enjoy chapter 9
R. Rao, CSE 326 47