Introduction to Algorithms

Class 9: Greedy Algorithm 1

Jianxi Gao

Department of Computer Science

Rensselaer Polytechnic Institute
www.gaojianxi.com
Exam 1: Wednesday , Feb 12, 8am to 9:50am.

Exam 1 will cover Chapters 0, 3, 4, and 5.1

Exam practice problems will be released on Friday, Feb 7

The HW4 due day is on Feb 13

You may use your printed notes and book(s)

No laptops, phones, or any electronic devices, etc.

UNDIRECTED NETWORKS

Notation. G = (V,E)
V = nodes (or vertices).
E = edges (or arcs) between pairs of nodes
Captures pairwise relationship between objects.
Network size parameters: n = |V |, m = |E|.

L
A
F M V = {A,B,C,D,E,F,G,H,I, L,M}
I
E {A-D,B-B,B-D,C-D,C-D,D-F,F-G,G-H,H-I,L-M}
D

B G

H
A-D is the same as D-A
C

n = 10, m = 10.
Tree

A tree is an undirected,
acyclic, connected graph.
Minimum Spanning Tree

For an undirected graph, can we pick

enough edges that the nodes are connected
and form a tree? If each edge has a weight
(maintenance cost), what is the cheapest
possible network?

A complex network A tree

Spanning tree from the complex networks: Brain

Human Brain
has between
10-100 billion
neurons.
Examples of Networks: BRAIN Brain

van Dellen, Edwin, et al. "Minimum spanning tree

analysis of the human connectome." Human brain
mapping 39.6 (2018): 2455-2471.
Examples of Brain Networks Brain
Example of Power Grid

https://www.nytimes.com/interactive/2021/02/16/us/winter-storm-texas-power-outage-map.html
Example of Power Grid
Example of Power Grid
Example of
Road network
• Minimum Spanning Tree of
North Seattle
Minimum Spanning Tree
on Surface of Sphere
5000 Vertices
Minimum Spanning Tree
on Surface of Torus
5000 Vertices
Recursive Tree
Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Q1: Is this a tree?

A1: No, it is not connected

remove

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Q1: Is this a tree?

A1: No, it is not connected

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Remove

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Q1: Is this a tree?

A1: No, it is not acyclic

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Q1: Is this a tree?

A1: No, it is not acyclic

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Q1: Is this a tree?

A1: No, it is not acyclic

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Q1: Is this a tree?

A1: No, it is not acyclic

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Q1: Is this a tree?

A1: No, it is not acyclic

Add a link

A tr ee is an undirected, acyclic, connected graph.

Examples of trees

Q1: Is this a tree?

A1: No, it is not acyclic

A tr ee is an undirected, acyclic, connected graph.

Minimum spanning tree

Which of the following properties are true for all spanning tree
H(V, T) of a graph G(V, E)?
A. Contains exactly |V|-1 edges
B. May contain |V| edges 10 mins
C. It is possible to contain less than |V| nodes
D. The removal of any edge disconnects it.
E. The addition of any edge creates a cycle.
F. There exist multiple shortest paths between a pair of nodes in H.
Greedy Algorithms

• Prim's algorithm. Start with some root node s and greedily grow a tree
T from s outward. At each step, add the cheapest edge e to T that has
exactly one endpoint in T.

• Kruskal's algorithm. Start with T = . Consider edges in ascending

order of cost. Insert edge e in T unless doing so would create a cycle.

• Boruvka's algorithm. Start with cheapest edges leaving every node,

add them unless they create a cycle….

• Remark. All three algorithms produce an MST.

Finding MST: Prim’s Algorithm

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

Start from node A.

Finding MST: Prim’s Algorithm
|T| = 0 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 1,2,5,10.

𝑙𝐴𝐶 = 1 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 1 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 2,5,7,10.

𝑙𝐴𝐺 = 2 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 2 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 5,7,8,10,12.

𝑙𝐴𝐵 = 5 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 3 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 6,7,8,10,11,12.

𝑙𝐵𝐸 = 6 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 4 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 3,7,8,10,11,12.

𝑙𝐸𝐷 = 3 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 5 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 7,8,10,11,12.

𝑙𝐶𝐷 = 7 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 5 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 8,10,11,12.

Addition of CD will make a cycle, so we can not add link CD.
Finding MST: Prim’s Algorithm
|T| = 5 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 8,10,11,12.

𝑙𝐺𝐼 = 8 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 6 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 9,10,11,12.

𝑙𝐻𝐼 = 9 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 7 < |V|-1 = 8 --- Continue

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8

The neighboring edge weights are 4,10,11,12.

𝑙𝐻𝐹 = 4 is the smallest.
Finding MST: Prim’s Algorithm
|T| = 8 = |V|-1 = 8 --- STOP

7 4
D C 1 10 F H
11
3 A 12 9

E B 5 2 G I
6 8
Finding MST: Prim’s Algorithm

4
D C 1 F H

3 A 9

E B 5 2 G I
6 8
Finding MST: Prim’s Algorithm

10 mins 7 7
5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Finding MST: Prim’s Algorithm

Interchangeable pairs
G
BE→EF
7 7
BD→DE
5
D H
GD→GH
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

Kruskal's algorithm. Start with T = .

Consider edges in ascending order of cost. G
Insert edge e in T unless doing so would
create a cycle. 7 7
5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

The lightest edge is

{AB}. G
7 7
5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edge is

{IJ}. G
7 7
5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edge is

{IJ}. G
7 7
5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{AC,BF}. G

We can pick one from them, for example AC. 7 7

5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edge is

{BF}. G
7 7
5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{BC,BE,EF,FJ}. G

We can pick one from them, for example BC. 7 7

5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

Form a cycle, so we remove BC.

G
7 7
5
D H
5 6
5 6
4 7
B E I
1
4 6 2
4 3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{BE,EF,FJ}. G

We can pick one from them, for example BE. 7 7

5
D H
5 6
5 6
4 7
B E I
1
4 6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{EF,FJ}. G

We can pick one from them, for example EF. 7 7

5
D H
5 6
5 6
4 7
B E I
1
4 6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

Form a cycle, so we remove EF.

G
7 7
5
D H
5 6
5 6
4 7
B E I
1
4 6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edge is

{FJ}. G
7 7
5
D H
5 6
5 6
4 7
B E I
1
6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{BD,DE,DH}. G

We can pick one from them, for example BD. 7 7

5
D H
5 6
5 6
4 7
B E I
1
6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{DE,DH}. G

We can pick one from them, for example DE. 7 7

5
D H
5 6
5 6
4 7
B E I
1
6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

Form a cycle, so we remove DE.

G
7 7
5
D H
5 6
5 6
4 7
B E I
1
6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edge is

{DH}. G
7 7
5
D H
5 6 6
4 7
B E I
1
6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edge is

{DH}. G
7 7
5
D H
5 6 6
4 7
B E I
1
6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{CF,DI,FI,HI}. G

But addition of any of them creates a cycle. 7 7

5
D H
5 6 6
4 7
B E I
1
6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{DI,FI,HI}. G

Addition of any of them creates a 7 7

cycle, so we remove them. 5
D H
5 6 6
4 7
B E I
1
6 2
3
A C F J
3 6 4
Kruskal’s Algorithm

The new lightest edges are

{DG,GH,EI}. G

We can pick one from them, for example DG. 7 7

5
D H
5
4 7
B E I
1
3 2

A C F J
3 4
Kruskal’s Algorithm

The new lightest edges are

{GH,EI}. G

Addition of any of them creates a 7 7

cycle, so we remove them. 5
D H
5
4 7
B E I
1
3 2

A C F J
3 4
Kruskal’s Algorithm

G
7 7
5
D H
5
4 7
B E I
1
3 2

A C F J
3 4
Kruskal’s Algorithm

G
7
5
D H
5
4
B E I
1
3 2

A C F J
3 4
Cuts

• A cut is a subset of nodes S: The

corresponding cutset D is the subset of edges
with exactly one endpoint in S.
2 3
1 3 4
6 4
2
5
6 5
7 8
1

7 8

Cut S = { 4, 5, 8 }
Cutset D = 5-6, 5-7, 3-4, 3-5, 7-8
Cut Property

Cut property: Suppose edges X are part of an MST. Let S be any

subset of nodes without X crossing between S and V-S, and let e
be a cheapest edge in this cutset. Then X+e is part of some MST.

S V-S

e
Minimum spanning tree

Consider the cut S = {1,4,7,8}, Select all the edges in the cutset of S.

5 mins A. 1-2
B. 2-3
2 3 C. 1-7
1
D. 5-6
6 4
E. 1-6
5
F. 5-7
7 8 G. 4-8
H. 6-7
I. 3-4
J. 5-8
K. 3-5
L. 4-5
M. 3-6
Dijkstra’s algorithm
Procedure Dijkstra(G,l,s)
Input: Graph G = (V,E), positive edge lengths 𝑙𝑒 : 𝑒 ∈ E; 𝑠 ∈ 𝑉.
Output: For all nodes reachable from 𝑠, 𝑑𝑖𝑠𝑡(𝑢) is set to the distance from 𝑠 to 𝑢.

For all u∈ 𝑉
𝑑𝑖𝑠𝑡 𝑢 = ∞,
Prev(u) = nil; (the node immediately before u on the shortest path.)
𝑑𝑖𝑠𝑡 𝑠 = 0,
𝐻 = 𝑚𝑎𝑘𝑒𝑞𝑢𝑒𝑢𝑒 𝑉 , (using distance values as keys, Priority queue)
While H is not empty
u = deletemin(H) (return the element with the smallest key, and remove it)
for all edges 𝑢, 𝑣 ∈ 𝐸:
if dist(u) +l(u,v) < dist(v)
dist(v) = dist(u) +l(u,v).
prev(v) = u;
Decreasekey(H,v); (notify that a certain key values has been decreased.)
Prim algorithm
Procedure Dijkstra(G,l,s)
Input: Graph G = (V,E), positive edge lengths 𝑙𝑒 : 𝑒 ∈ E; 𝑠 ∈ 𝑉.
Output: Minimum spanning tree

For all u∈ 𝑉
𝑑𝑖𝑠𝑡 𝑢 = ∞,
Prev(u) = nil; (the node immediately before u on the shortest path.)
𝑑𝑖𝑠𝑡 𝑠 = 0,
𝐻 = 𝑚𝑎𝑘𝑒𝑞𝑢𝑒𝑢𝑒 𝑉 , (using distance values as keys, Priority queue)
While H is not empty
u = deletemin(H) (return the element with the smallest key, and remove it)
for all edges 𝑢, 𝑣 ∈ 𝐸:
if dist(u) +l(u,v) < dist(v)
dist(v) = dist(u) +l(u,v).
prev(v) = u;
Decreasekey(H,v); (notify that a certain key values has been decreased.)
MST Algorithms: Theory

• Comparison based algorithms.

– O(m log n) [Jarník, Prim, Dijkstra, Kruskal,
Boruvka]
– O(m log log n). [Cheriton-Tarjan 1976, Yao 1975]
– O(m (m, n)). [Fredman-Tarjan 1987]
– O(m log (m, n)). [Gabow-Galil-Spencer-Tarjan
1986]
– O(m  (m, n)). [Chazelle 2000]
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

10 mins 1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1
2 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1 2
A B C
3 2 1 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1
A B C
3 2 1 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1
A B C
3 2 1 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1
A B C
3 2 1 1
1 4
D E F
3
2 4
2
G H
1
Kruskal’s Algorithm example

Using Kruskal’s algorithm to find the minimum spanning tree.

1
A B C
2 1 1
1
D E F

G H
1