Topological Sort
1
Lecture Flow
1) Pre-requisites
2) Understanding the Logic
3) Different Implementations
4) Recognizing in Questions
5) Applications of Topological sort
6) Common Pitfalls
7) Quote of the day
2
Pre-requisites
● Graph theory
● Cycles in graphs
● Graph traversal (BFS, DFS)
● Backtracking
3
Understanding the Logic
4
Real Life Example
5
Order of Clothes
You wear your clothes one by one
Boxer > Trouser > Belt > Undershirt > T-shirt > Cap > Coat > Shoes
Boxer > Trouser > Undershirt > T-shirt > Cap > Belt > Coat > Shoes
Boxer > Trouser > Undershirt > T-shirt > Cap > Coat > Shoes > Belt
6
Topological Sort
Topological Sort is a way to arrange a collection of tasks or events in such a
sequence that each task comes before the tasks that depend on it.
7
Topological Sorting
A topological sort is a linear ordering of nodes in a directed
acyclic graph (DAG)
8
What is a Directed Acyclic Graph (DAG) ?
9
Directed Acyclic Graph (DAG)
A directed graph with no cycles
Topological Sorting for a graph is not possible if the graph is not a DAG
Circular dependencies cannot be resolved in real life
10
Motivation Question
11
Course Schedule II
12
How is topological sorting done ?
13
Different Implementation
To find the topologically sorted order of DAG, we have 2 algorithms
that can be used
● Kahn's Algorithm
● Modified Depth-first search
14
BFS Implementation - Kahn’s
Algorithm
15
Kahn’s Algorithm
● It starts by finding all the nodes in the graph that have no
incoming edges - these are the source nodes.
● The algorithm adds these source nodes to the queue, since
they don't depend on any other nodes.
● The algorithm then removes the source nodes from the graph,
along with their outgoing edges.
● It then finds new source nodes in the reduced graph and the
process is repeated.
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Time Complexity Space Complexity
● Let V be number of Nodes (Courses) ● Let V be number of Nodes (Courses)
and E be number of edges . and E be number of edges .
● We are traversing through the entire ● We will store the nodes, and also the
graph and we will only visit a node edges.
once. Which would take a time Space Complexity = O (V + E)
complexity of O(V + E).
● What would the space complexity
be if we ignore the complexity of
building the graph?
31
DFS Implementation
32
DFS Algorithm
● It creates the topological ordering in reverse order.
● It starts from any node, and traverses depth-first until it reaches
a node with no outgoing edges.
● As it backtracks, it puts the nodes in to a stack. This stack
represents the reverse topological ordering.
● Maintains three colors to keep track of node state:
○ White - Unvisited node
○ Grey - Node visited in current path
○ Black - Node visited in a different path
33
DFS Algorithm
● During traversal:
○ Only traverse to white nodes.
○ If a black node is found, skip it - it has been processed.
○ If a grey node is found, this means there is a cycle. Why?
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
DFS Implementation
Can you think of another implementation where we don’t have
reverse the final answer?
54
Time Complexity Space Complexity
● Let V be number of Nodes (Courses) ● Let V be number of Nodes (Courses)
and E be number of edges . and E be number of edges .
● We are traversing through the entire ● We will store the nodes, and also the
graph and we will only visit a node edges.
once. Which would take a time Space Complexity = O (V + E)
complexity of O(V + E).
● What would the space complexity
be if we ignore the complexity of
building the graph?
55
Practice Problem
56
When do we use which implementation
What implementation would be better for a given DAG which is very sparse,
with n vertices and m edges, and you need to find a topological ordering of the
vertices?
57
When do we use which implementation
● One possible algorithm to use is Kahn's algorithm (BFS), which uses a
queue to keep track of vertices with no incoming edges.
● However, suppose that the DAG is very sparse, with only a few edges
relative to the number of vertices. In this case, using Kahn's algorithm may
not be the most efficient choice, as it requires maintaining a queue with a
large number of empty slots.
58
When do we use which implementation
● A better algorithm in this case might be a variation of depth-first search
(DFS) that uses a stack to store the vertices in reverse topological
order.
● This algorithm has a space complexity of O(V), which is much more
efficient than Kahn's algorithm when the graph is sparse.
59
Real Life Applications
Here are some notable applications of topological sort
● Task Scheduling
○ Topological sort ensures tasks are sequenced correctly by
analyzing task dependencies in a DAG.
● Software Dependency Resolution
○ Topological sort resolves dependencies between software
components, ensuring correct order during compilation or
deployment.
60
Real Life Applications Continued
● Compiler Optimizations
○ Topological sort optimizes code generation in compilers,
ensuring correct variable allocation and efficient loop
compilation techniques.
● DeadLock detection
● Course Scheduling
61
Recognizing in Questions
62
Directed Acyclic Graph (DAG)
63
Directed Acyclic Graph (DAG)
64
Courses / Prerequisites
65
Dependency
66
Scheduling
67
Common Pitfalls
68
Multiple Components
Graph may contain multiple components, don’t forget to
traverse all the components
69
Cycle Detection Common Pitfalls
Don’t try to detect cycle only by using incoming edge counts
70
Cycle Detection Common Pitfalls - 2
Don’t try to use vanilla BFS or DFS to detect cycles
Why Vanilla BFS with Storing Visited Nodes won’t Work?
71
Cycle Detection Common Pitfalls - 2
Why DFS with Storing Visited Nodes won’t Work?
72
Resources
● Topological Sort - TopCoder
● Kahn's Algorithm - TakeUForward
● Topological Sort using DFS - TakeUForward
● Topological Sort Visualization
73
Practice Problems
Course Schedule
All Ancestors of a Node in a Directed Acyclic Graph
Alien Dictionary
Loud and Rich
Course Schedule IV
Minimum Height Trees
Parallel Courses III
Sort Items by Groups Respecting Dependencies
Build a Matrix With Conditions
Longest Cycle in a Graph
Gardener and Tree
74
For the curious: Real-world implementation of Topological Sorting
(Not a comprehensive list)
● MakeFiles in Software Building Procedure
● Apache Airflow usually in MLOps
● Version Control Systems such as Git
● Compilers such as GCC
75
Quote of the day
"Order is the shape upon which beauty
depends."
By Pearl S. Buck
76
