Advanced Algorithms

Activity Selection Problems

Dr. Rubi Quiñones
Computer Science Department
Southern Illinois University Edwardsville
 Course Timeline
• Introduction to algorithms

ALGORITH

ANALYSIS
MIC
• Median findings and order statistics
• Time complexity
STRATEGIES • Activity selection problems
GREEDY

• Water connection problem, Egyptian fraction

• Huffman (de)coding
• shelves, mice, and policemen problem
•
CONQUER

Max subarray sum and Nearest neighbor

DIVIDE
AND

• Newton’s and bisection algorithm

• Matrix multiplication, skyline and hanoi
• Fibonacci and path counting
PROGRAMMING

• Coin row and collecting problem

DYNAMIC

• Matrix chain multiplication and longest common subsequence

• Knapsack and optimal binary trees
• Floyd Warshall Algorithm and A*
Concep

•
Advanc

Algorithm intractability
ed

ts

• Randomized algorithms 2
 Activity Selection Problems

• What is a greedy method?

• activity selection problem
• Kruskal’s Minimum Spanning Tree (MST)
• Prim’s Minimum Spanning Tree
• Dijkstra’s Shortest Path
• Water connection problem
• Egyptian fraction
• Policeman, shelves, and mice problem
• Huffman (de)coding

3
 Greedy Method
• Greedy methods: A technique for solving optimization problems
• Choose a solution to a problem that is best per an objective function
• Similar to dynamic programming in that we examine subproblems, exploiting optimal substructure
property
• Key difference: in dynamic programming we considered all possible subproblems,
• In contrast, a greedy algorithm at each step commits to just one subproblem, which results in its greedy
choice (locally optimal choice)

“At every step, we can make a choice that looks best at the moment, and we get the
optimal solution to the complete problem. ”

4
 Activity Selection
It is a combinatorial optimization problem concerning the selection of non-conflicting activities to perform
within a given time frame, given a set of activities each marked by a start time and finish time.

5
 Kruskal’s Minimum Spanning Tree Algorithm
Spanning Tree: tree-like subgraph of a connected, undirected graph that includes all the vertices of the graph
Minimum Spanning Tree: a spanning tree that has the minimum weight among all the possible spanning trees

6
 Kruskal’s Minimum Spanning Tree Algorithm
Kruskal’s Algorithm: sort all edges of the given graph in increasing order, then add new edges and nodes in the
MST if the newly added edge does form a cycle.

How to find MST using Kruskal’s algorithm?

1. Sort all the edges in increasing order of their weight.
2. Pick the smallest edge. Check if it forms a cycle with the spanning tree formed so far. If the cycle is not
formed, include this edge. Else, discard it.
3. Repeat step#2 until there are (V-1) edges in the spanning tree.

What is the greedy choice?

• Pick the smallest weight edge that does not cause a cycle in the MST constructed so far.

Let’s look at an example

7
Kruskal’s Minimum Spanning Tree Algorithm

Input graph (9 vertices, 14 edges)

The MST formed will be having (9-1) = 8 edges.

8
Go ahead and try to sort all the edges in increasing order of their weight
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
Select edges one-by-one right before a cycle is formed 10 5 4
11 1 7
9
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at first edge 7-6. No cycle is formed, include it 10
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 8-2. No cycle is formed, include it 11
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 6-5. No cycle is formed, include it 12
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 0-1. No cycle is formed, include it 13
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 2-5. No cycle is formed, include it 14
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 8-6. Including this edge results in a cycle discard 15
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 2-3. No cycle is formed, include it 16
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 7-8. Including this edge results in a cycle discard 17
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 0-7. No cycle is formed, include it 18
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 1-2. Including this edge results in a cycle discard 19
14 3 5
 Kruskal’s Minimum Spanning Tree Algorithm
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
Look at next edge 3-4. No cycle is formed, include it 20
Since the number of edges included in the MST equals (V-1) we are done 14 3 5
Kruskal’s Minimum Spanning Tree Algorithm
What is the time complexity?

21
Kruskal’s Minimum Spanning Tree Algorithm
What is the time complexity: O(E*logE + E*logV)
• Sorting of edges takes O(E*logE) time
• Finding operation takes O(logV)

22
Kruskal’s Minimum Spanning Tree Algorithm
Identify the MST using Kruskal’s Algorithm

23
Kruskal’s Minimum Spanning Tree Algorithm
Identify the MST using Kruskal’s Algorithm

Answer

24
Prim’s Minimum Spanning Tree Algorithm

• The idea of Prim’s is to maintain two sets of vertices

• First set: contains the vertices already included in the MST
• Second set: contains the vertices not yet included
• It considers all edges in every step that connect the two sets and picks the minimum weight edge
from these edges
• After picking the edge, it moves the other endpoint of the edge to the set containing MST

• NOTE: a group of edges that connects two sets of vertices in a graph is called cut

25
Prim’s Minimum Spanning Tree Algorithm

26
Prim’s Minimum Spanning Tree Algorithm

27
Prim’s Minimum Spanning Tree Algorithm

28
Prim’s Minimum Spanning Tree Algorithm

29
Prim’s Minimum Spanning Tree Algorithm

30
Prim’s Minimum Spanning Tree Algorithm

31
Prim’s Minimum Spanning Tree Algorithm

32
Prim’s Minimum Spanning Tree Algorithm

33
Prim’s Minimum Spanning Tree Algorithm

34
 Prim’s Minimum Spanning Tree Algorithm

We skipped steps to get the final answer. Practice getting from the last slide’s graph to this final answer
Weight of the edges of the MST is ____? 35
 Prim’s Minimum Spanning Tree Algorithm

We skipped steps to get the final answer. Practice getting from the last slide’s graph to this final answer
Weight of the edges of the MST is (4+8+1+2+4+2+7+9)= 37 36
Prim’s Minimum Spanning Tree Algorithm
Identify the MST using Prim’s Algorithm when we do select the edge {1,2} instead of {0,7} (slide 31)

37
Prim’s Minimum Spanning Tree Algorithm

Identify the MST using Prim’s Algorithm when we do select the edge {1,2} instead of {0,7} (slide 32)

38
 Prim’s Minimum Spanning Tree Algorithm
Time Complexity: O(V2) if we are using an adjacency matrix, and searching

39
 Dijkstra’s Shortest Path Algorithm
• Idea came to be in 1956
• Claimed he took 20 minutes to make the
algorithm while having coffee
• Published in 1959
• A Dutch pioneer in computer science
• Operating systems
• Graph algorithms
• Compiler construction
• Many more!

Edsger Wybe Dijkstra

1930 - 2002

40
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s Read it first

Node Shortest Previous

6
A B Distance Node
5
from A
C A 0
1 2 2
B 3 D
5 C 7 E
D E
1 D 1 A
E 2 D

41
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s Read it first
Path = C

Nodes Shortest Previous

6
A B Distance Node
5
from A
C A 0
1 2 2
B 3 D
5 C 7 E
D E
1 D 1 A
E 2 D

42
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s Read it first
Path = E → C

Nodes Shortest Previous

6
A B Distance Node
5
from A
C A 0
1 2 2
B 3 D
5 C 7 E
D E
1 D 1 A
E 2 D

43
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s Read it first
Path = D → E → C

Nodes Shortest Previous

6
A B Distance Node
5
from A
C A 0
1 2 2
B 3 D
5 C 7 E
D E
1 D 1 A
E 2 D

44
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s Read it first
Path = A → D → E → C

Nodes Shortest Previous

6
A B Distance Node
5
from A
C A 0
1 2 2
B 3 D
5 C 7 E
D E
1 D 1 A
E 2 D

45
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table

6 Nodes Shortest Previous

A B
Distance Node
from A
1 2 A
B
C C
46
Visited = [ ] Unvisited = [A, B, C]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
START at a node (A)
Distance from A to A is 0
Distance from A to OTHERS are unknown, ∴ ∞

6 Nodes Shortest Previous

A B
Distance Node
from A
1 2 A 0
B ∞
C C ∞
47
Visited = [ ] Unvisited = [A, B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the current node (A),
- Examine its unvisited neighbors (B and C)

6 Nodes Shortest Previous

A B
Distance Node
from A
1 2 A 0
B ∞
C C ∞
48
Visited = [ ] Unvisited = [A, B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the current node (A),
- Examine its unvisited neighbors (B and C)
- Calculate the distance of each neighbor from the start node

0+6=6
6 Nodes Shortest Previous
A B
Distance Node
from A
1 2 A 0

0+1=1 B ∞
C C ∞
49
Visited = [ ] Unvisited = [A, B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the current node (A),
- Examine its unvisited neighbors (B and C)
- Calculate the distance of each neighbor from the start node
- If calculated distance is < known distance, update shortest distance

0+6=6
6 Nodes Shortest Previous
A B
Distance Node
from A
1 2 A 0

0+1=1 B 6 ∵ (6 < ∞)
C C 1 ∵ (1 < ∞)
50
Visited = [ ] Unvisited = [A, B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the current node (A),
- Examine its unvisited neighbors (B and C)
- Calculate the distance of each neighbor from the start node
- If calculated distance is < known distance, update shortest distance
- Update the Previous Node for each of the updated distances

0+6=6
6 Nodes Shortest Previous
A B
Distance Node
from A
1 2 A 0

0+1=1 B 6 ∵ (6 < ∞) A
C C 1 ∵ (1 < ∞) A
51
Visited = [ ] Unvisited = [A, B, C]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the current node (A),
- Examine its unvisited neighbors (B and C)
- Calculate the distance of each neighbor from the start node
- If calculated distance is < known distance, update shortest distance
- Update the Previous Node for each of the updated distances
- Add current node (A) to the Visited list , remove from Unvisited list
0+6=6
6 Nodes Shortest Previous
A B
Distance Node
from A
1 2 A 0

0+1=1 B 6 ∵ (6 < ∞) A
C C 1 ∵ (1 < ∞) A
52
Visited = [A] Unvisited = [B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the unvisited node with the smallest known distance from the start node (C),
- Examine its unvisited neighbors (B)

6 Nodes Shortest Previous

A B
Distance Node
from A
1 2 A 0
B 6 A
C C 1 A
53
Visited = [A] Unvisited = [B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the unvisited node with the smallest known distance from the start node (C),
- Examine its unvisited neighbors (B)
- Calculate the distance of each neighbor from the current node

6 Nodes Shortest Previous

A B
1+2=3 Distance Node
from A
1 2 A 0
B 6 A
C C 1 A
54
Visited = [A] Unvisited = [B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the unvisited vertex with the smallest known distance from the start node (C),
- Examine its unvisited neighbors (B)
- Calculate the distance of each neighbor from the current node
- If calculated distance is < known distance, update shortest distance

6 Nodes Shortest Previous

A B
1+2=3 Distance Node
from A
1 2 A 0
B 3 ∵ (3 < 6) A
C C 1 A
55
Visited = [A] Unvisited = [B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the unvisited vertex with the smallest known distance from the start node (C),
- Examine its unvisited neighbors (B)
- Calculate the distance of each neighbor from the current node
- If calculated distance is < known distance, update shortest distance
- Update the Previous Node for each of the updated distances

6 Nodes Shortest Previous

A B
1+2=3 Distance Node
from A
1 2 A 0
B 3 ∵ (3 < 6) C
C C 1 A
56
Visited = [A] Unvisited = [B, C ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the unvisited vertex with the smallest known distance from the start node (C),
- Examine its unvisited neighbors (B)
- Calculate the distance of each neighbor from the current node
- If calculated distance is < known distance, update shortest distance
- Update the Previous Node for each of the updated distances
- Add current node (C) to the Visited list, remove from Unvisited list

6 Nodes Shortest Previous

A B
1+2=3 Distance Node
from A
1 2 A 0
B 3 ∵ (3 < 6) C
C C 1 A
57
Visited = [A, C] Unvisited = [B ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the unvisited node with the smallest known distance from the start node (B),
- Examine its unvisited neighbors. None?

6 Nodes Shortest Previous

A B
Distance Node
from A
1 2 A 0
B 3 C
C C 1 A
58
Visited = [A, C] Unvisited = [B ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Let’s generate the table
For the unvisited node with the smallest known distance from the start node (B),
- Examine its unvisited neighbors. None?
- Add current node (B) to the Visited list, remove from Unvisited list

6 Vertex Shortest Previous

A B
Distance Vertex
from A
1 2 A 0
B 3 C
C C 1 A
59
Visited = [A, C, B] Unvisited = [ ]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Now you try!!
START at a vertex (A)
Distance from A to A is 0; Distance from A to OTHERS are unknown, ∴ ∞

For the unvisited node with the smallest known distance from the start node,
- Examine its unvisited neighbors
- Calculate the distance of each neighbor from the current node
- If calculated distance is < known distance, update shortest distance
- Update the Previous Node for each of the updated distances
- Add current node to the Visited list, remove from Unvisited list Nodes Shortest Previous
Distance Node
2 A from A
B A
2
2 1 B
C C
3
D D
60
Visited = [] Unvisited = [ A, B, C, D]
 Dijkstra’s Shortest Path Algorithm
Find the shortest path between any two nodes in a graph.
Now you try!!
START at a vertex (A)
Distance from A to A is 0; Distance from A to OTHERS are unknown, ∴ ∞

For the unvisited node with the smallest known distance from the start node,
- Examine its unvisited neighbors
- Calculate the distance of each neighbor from the current node
- If calculated distance is < known distance, update shortest distance
- Update the Previous Node for each of the updated distances
- Add current node to the Visited list, remove from Unvisited list Nodes Shortest Previous
Distance Node
2 A from A
B A 0
2
2 1 B ∞→2 A
C C ∞→1 A
3
D D ∞→ 4 → 3 C→B
61
Visited = [A, C, B, D] Unvisited = [ ]
 Applications
Digital Mapping Services (Uses A* too) Social Networking Applications

62
 Course Timeline
• Introduction to algorithms

ALGORITH

ANALYSIS
MIC
• Median findings and order statistics
• Time complexity
STRATEGIES • Activity selection problems
GREEDY

• Water connection problem, Egyptian fraction

• Huffman (de)coding
• shelves, mice, and policemen problem
•
CONQUER

Max subarray sum and Nearest neighbor

DIVIDE
AND

• Newton’s and bisection algorithm

• Matrix multiplication, skyline and hanoi
• Fibonacci and path counting
PROGRAMMING

• Coin row and collecting problem

DYNAMIC

• Matrix chain multiplication and longest common subsequence

• Knapsack and optimal binary trees
• Floyd Warshall Algorithm and A*
Concep

•
Advanc

Algorithm intractability
ed

ts

• Randomized algorithms 63