Topological Sorting

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Programming, Data Structures and Algorithms using Python

Week 4
Directed Acyclic Graphs

G = (V , E ), a directed graph without

directed cycles 0 1
Topological sorting
Enumerate V = {0, 1, . . . , n − 1}
such that for any (i, j) ∈ E , i
2
4
appears before j

Represents a feasible schedule 3

5 6 7

Madhavan Mukund Topological Sorting PDSA using Python Week 4 2 / 10

Topological Sort

A graph with directed cycles cannot

be sorted topologically 0 1
Path i ; j means i must be listed
before j
2
Cycle ⇒ vertices i, j such that there 4
are paths i ; j and j ; i
3
i must appear before j, and j must
appear before i, impossible!
5 6 7

Madhavan Mukund Topological Sorting PDSA using Python Week 4 3 / 10

Topological Sort

A graph with directed cycles cannot

be sorted topologically 0 1
Path i ; j means i must be listed
before j
2
Cycle ⇒ vertices i, j such that there 4
are paths i ; j and j ; i
3
i must appear before j, and j must
appear before i, impossible!
5 6 7
Claim
Every DAG can be topologically sorted

Madhavan Mukund Topological Sorting PDSA using Python Week 4 3 / 10

How to topologically sort a DAG?

Strategy
First list vertices with no 0 1
dependencies
As we proceed, list vertices whose
2
dependencies have already been listed
4
...
3

5 6 7

Madhavan Mukund Topological Sorting PDSA using Python Week 4 4 / 10

How to topologically sort a DAG?

Strategy
First list vertices with no 0 1
dependencies
As we proceed, list vertices whose
2
dependencies have already been listed
4
...
3
Questions
Why will there be a starting vertex 5 6 7
with no dependencies?
How do we guarantee we can keep
progressing with the listing?

Madhavan Mukund Topological Sorting PDSA using Python Week 4 4 / 10

Algorithm for topological sort
A vertex with no dependencies has no
incoming edges, indegree(v ) = 0
0 1

2
4

5 6 7

Madhavan Mukund Topological Sorting PDSA using Python Week 4 5 / 10

Algorithm for topological sort
A vertex with no dependencies has no
incoming edges, indegree(v ) = 0
0 1
Claim
Every DAG has a vertex with indegree 0
2
4

5 6 7

Madhavan Mukund Topological Sorting PDSA using Python Week 4 5 / 10

Algorithm for topological sort
A vertex with no dependencies has no
incoming edges, indegree(v ) = 0
0 1
Claim
Every DAG has a vertex with indegree 0
2
Start with any vertex with 4
indegree > 0
3
Follow edge back to one of its
predecessors
5 6 7
Repeat so long as indegree > 0
If we repeat n times, we must have a
cycle, which is impossible in a DAG
Madhavan Mukund Topological Sorting PDSA using Python Week 4 5 / 10
Topological sort algorithm

Fact
Every DAG has a vertex with indegree 0 0 1

List out a vertex j with indegree = 0

Delete j and all edges from j 2
4
What remains is again a DAG!
3
Can find another vertex with
indegree = 0 to list and eliminate
5 6 7
Repeat till all vertices are listed

Madhavan Mukund Topological Sorting PDSA using Python Week 4 6 / 10

Topological sort algorithm
Compute indegree of each vertex

0 1

2
4
3

5 6 7

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency 0 0
matrix 0 1

2
2 1
4
1
3
2 1 4
5 6 7

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency 0
matrix 0
List a vertex with indegree 0 and
remove it from the DAG 2
2 1
4
1
3
2 1 4
5 6 7

Topologically sorted sequence

1,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency 0
matrix 0
List a vertex with indegree 0 and
remove it from the DAG 1
2 1
Update indegrees 4
1
3
2 1 3
5 6 7

Topologically sorted sequence

1,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG 1
2 1
Update indegrees 4
1
Can find another vertex with 3
indegree = 0 to list and eliminate 2 1 3
5 6 7

Topologically sorted sequence

1, 0,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG 0
2 0
Update indegrees 4
0
Can find another vertex with 3
indegree = 0 to list and eliminate 2 1 3
5 6 7

Topologically sorted sequence

1, 0,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG 0
2 0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 2 1 3
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG 0
2 0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 1 1 2
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 1 1 2
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 0 1 2
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 1 2
6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 0 2
6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 2
7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 1
7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
Update indegrees
Can find another vertex with
indegree = 0 to list and eliminate 1
7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6, 4,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
Update indegrees
Can find another vertex with
indegree = 0 to list and eliminate 0
7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6, 4,

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

Topological sort algorithm
Compute indegree of each vertex
Scan each column of the adjacency
matrix 0 1
List a vertex with indegree 0 and
remove it from the DAG
2
Update indegrees 4
Can find another vertex with 3
indegree = 0 to list and eliminate
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6, 4, 7

Madhavan Mukund Topological Sorting PDSA using Python Week 4 7 / 10

An implementation of topological sort
Compute indegrees by scanning columns def toposort(AMat):
(rows,cols) = AMat.shape
of adjacency matrix indegree = {}
toposortlist = []
List a vertex with indegree 0 and remove
it from the DAG for c in range(cols):
indegree[c] = 0
Update indegrees for r in range(rows):
if AMat[r,c] == 1:
Repeat till all vertices are listed indegree[c] = indegree[c] + 1

for i in range(rows):
j = min([k for k in range(cols)
if indegree[k] == 0])
toposortlist.append(j)
indegree[j] = indegree[j]-1
for k in range(cols):
if AMat[j,k] == 1:
indegree[k] = indegree[k] - 1

return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 8 / 10

An implementation of topological sort
Compute indegrees by scanning columns def toposort(AMat):
(rows,cols) = AMat.shape
of adjacency matrix indegree = {}
toposortlist = []
List a vertex with indegree 0 and remove
it from the DAG for c in range(cols):
indegree[c] = 0
Update indegrees for r in range(rows):
if AMat[r,c] == 1:
Repeat till all vertices are listed indegree[c] = indegree[c] + 1

Analysis for i in range(rows):

j = min([k for k in range(cols)
if indegree[k] == 0])
toposortlist.append(j)
indegree[j] = indegree[j]-1
for k in range(cols):
if AMat[j,k] == 1:
indegree[k] = indegree[k] - 1

return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 8 / 10

An implementation of topological sort
Compute indegrees by scanning columns def toposort(AMat):
(rows,cols) = AMat.shape
of adjacency matrix indegree = {}
toposortlist = []
List a vertex with indegree 0 and remove
it from the DAG for c in range(cols):
indegree[c] = 0
Update indegrees for r in range(rows):
if AMat[r,c] == 1:
Repeat till all vertices are listed indegree[c] = indegree[c] + 1

Analysis for i in range(rows):

j = min([k for k in range(cols)
Initializing indegrees is O(n2 ) if indegree[k] == 0])
toposortlist.append(j)
indegree[j] = indegree[j]-1
for k in range(cols):
if AMat[j,k] == 1:
indegree[k] = indegree[k] - 1

return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 8 / 10

An implementation of topological sort
Compute indegrees by scanning columns def toposort(AMat):
(rows,cols) = AMat.shape
of adjacency matrix indegree = {}
toposortlist = []
List a vertex with indegree 0 and remove
it from the DAG for c in range(cols):
indegree[c] = 0
Update indegrees for r in range(rows):
if AMat[r,c] == 1:
Repeat till all vertices are listed indegree[c] = indegree[c] + 1

Analysis for i in range(rows):

j = min([k for k in range(cols)
Initializing indegrees is O(n2 ) if indegree[k] == 0])
toposortlist.append(j)
Loop to enumerate vertices runs n times indegree[j] = indegree[j]-1
for k in range(cols):
Identify next vertex to enumerate: O(n) if AMat[j,k] == 1:
indegree[k] = indegree[k] - 1
Updating indegrees: O(n)
return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 8 / 10

An implementation of topological sort
Compute indegrees by scanning columns def toposort(AMat):
(rows,cols) = AMat.shape
of adjacency matrix indegree = {}
toposortlist = []
List a vertex with indegree 0 and remove
it from the DAG for c in range(cols):
indegree[c] = 0
Update indegrees for r in range(rows):
if AMat[r,c] == 1:
Repeat till all vertices are listed indegree[c] = indegree[c] + 1

Analysis for i in range(rows):

j = min([k for k in range(cols)
Initializing indegrees is O(n2 ) if indegree[k] == 0])
toposortlist.append(j)
Loop to enumerate vertices runs n times indegree[j] = indegree[j]-1
for k in range(cols):
Identify next vertex to enumerate: O(n) if AMat[j,k] == 1:
indegree[k] = indegree[k] - 1
Updating indegrees: O(n)
Overall, O(n2 ) return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 8 / 10

Using adjacency lists
def toposortlist(AList):
Compute indegrees by scanning adjacency (indegree,toposortlist) = ({},[])
lists for u in AList.keys():
indegree[u] = 0
Maintain queue of vertices with indegree 0 for u in AList.keys():
for v in AList[u]:
Enumerate head of queue, update indegree[v] = indegree[v] + 1
indegrees, add indegree 0 to queue zerodegreeq = Queue()
for u in AList.keys():
Repeat till queue is empty if indegree[u] == 0:
zerodegreeq.addq(u)

while (not zerodegreeq.isempty()):

j = zerodegreeq.delq()
toposortlist.append(j)
indegree[j] = indegree[j]-1
for k in AList[j]:
indegree[k] = indegree[k] - 1
if indegree[k] == 0:
zerodegreeq.addq(k)
return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 9 / 10

Using adjacency lists
def toposortlist(AList):
Compute indegrees by scanning adjacency (indegree,toposortlist) = ({},[])
lists for u in AList.keys():
indegree[u] = 0
Maintain queue of vertices with indegree 0 for u in AList.keys():
for v in AList[u]:
Enumerate head of queue, update indegree[v] = indegree[v] + 1
indegrees, add indegree 0 to queue zerodegreeq = Queue()
for u in AList.keys():
Repeat till queue is empty if indegree[u] == 0:
zerodegreeq.addq(u)
Analysis
while (not zerodegreeq.isempty()):
j = zerodegreeq.delq()
toposortlist.append(j)
indegree[j] = indegree[j]-1
for k in AList[j]:
indegree[k] = indegree[k] - 1
if indegree[k] == 0:
zerodegreeq.addq(k)
return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 9 / 10

Using adjacency lists
def toposortlist(AList):
Compute indegrees by scanning adjacency (indegree,toposortlist) = ({},[])
lists for u in AList.keys():
indegree[u] = 0
Maintain queue of vertices with indegree 0 for u in AList.keys():
for v in AList[u]:
Enumerate head of queue, update indegree[v] = indegree[v] + 1
indegrees, add indegree 0 to queue zerodegreeq = Queue()
for u in AList.keys():
Repeat till queue is empty if indegree[u] == 0:
zerodegreeq.addq(u)
Analysis
while (not zerodegreeq.isempty()):
Initializing indegrees is O(m + n) j = zerodegreeq.delq()
toposortlist.append(j)
indegree[j] = indegree[j]-1
for k in AList[j]:
indegree[k] = indegree[k] - 1
if indegree[k] == 0:
zerodegreeq.addq(k)
return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 9 / 10

Using adjacency lists
def toposortlist(AList):
Compute indegrees by scanning adjacency (indegree,toposortlist) = ({},[])
lists for u in AList.keys():
indegree[u] = 0
Maintain queue of vertices with indegree 0 for u in AList.keys():
for v in AList[u]:
Enumerate head of queue, update indegree[v] = indegree[v] + 1
indegrees, add indegree 0 to queue zerodegreeq = Queue()
for u in AList.keys():
Repeat till queue is empty if indegree[u] == 0:
zerodegreeq.addq(u)
Analysis
while (not zerodegreeq.isempty()):
Initializing indegrees is O(m + n) j = zerodegreeq.delq()
toposortlist.append(j)
Loop to enumerate vertices runs n times indegree[j] = indegree[j]-1
for k in AList[j]:
Updating indegrees: amortised O(m) indegree[k] = indegree[k] - 1
if indegree[k] == 0:
zerodegreeq.addq(k)
return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 9 / 10

Using adjacency lists
def toposortlist(AList):
Compute indegrees by scanning adjacency (indegree,toposortlist) = ({},[])
lists for u in AList.keys():
indegree[u] = 0
Maintain queue of vertices with indegree 0 for u in AList.keys():
for v in AList[u]:
Enumerate head of queue, update indegree[v] = indegree[v] + 1
indegrees, add indegree 0 to queue zerodegreeq = Queue()
for u in AList.keys():
Repeat till queue is empty if indegree[u] == 0:
zerodegreeq.addq(u)
Analysis
while (not zerodegreeq.isempty()):
Initializing indegrees is O(m + n) j = zerodegreeq.delq()
toposortlist.append(j)
Loop to enumerate vertices runs n times indegree[j] = indegree[j]-1
for k in AList[j]:
Updating indegrees: amortised O(m) indegree[k] = indegree[k] - 1
if indegree[k] == 0:
Overall, O(m + n) zerodegreeq.addq(k)
return(toposortlist)

Madhavan Mukund Topological Sorting PDSA using Python Week 4 9 / 10

Summary

Directed acyclic graphs are a natural way to represent dependencies

Topological sort gives a feasible schedule that represents dependencies
At least one vertex with no dependencies, indegree 0
Eliminating such a vertex retains DAG structure
Repeat the process till all vertices are listed

Complexity
Using adjacency matrix takes O(n2 )
Using adjacency list takes O(m + n)

More than one topological sort is possible

Choice of which vertex with indegree 0 to list next

Madhavan Mukund Topological Sorting PDSA using Python Week 4 10 / 10