Lecture 13 — Belmann-Ford and Dijkstra’s Algorithm

The Shortest Path Problem — recap

The problem of finding a shortest path from O to D in a directed graph G = (N , A) (with

arc weights wij associated with each arc (i, j) ∈ A) is the problem of finding the directed path
Pl−1
(n1 , . . . nl ) where n1 = O and nl = D which minimises the sum i=1 wni ni+1 . We will also find
it useful to use the convetion that wij = ∞ if (i, j) ∈ / A. Two methods in common use are
Bellman-Ford and Dijkstra’s Algorithm.

Dijkstra’s Algorithm

Dijkstra’s Algorithm was discovered by the pioneering mathematician and programmer E.W.Dijkstra
(1930 – 2002). The algorithm finds the shortest path to all nodes from an origin node, on a graph
G = (N , A). It requires that all arc weights are non-negative ∀(i, j) ∈ A : wij ≥ 0.
Dijkstra’s Algorithm involves the labelling of a set of permanent nodes P and node distances D j
to each node j ∈ N . Assume that we wish to find the shortest paths from node 1 to all nodes.
Then we begin with: P = {1} and D1 = 0 and Dj = w1 j where j 6= 1. Dijkstra’s algorithm then
consists of following the procedure:

1. Find the next closest node. Find i ∈

/ P such that:

Di = min Dj
j ∈P
/

2. Update our set of permanently labelled nodes and our nodes distances:
P := P ∪ {i}.
3. If all nodes in N are also in P then we have finished so stop here.
4. Update the temporary (distance) labels for the new node i. For all j ∈
/P

Dj := min[Dj , wij + Di ]

5. Go to the beginning of the algorithm.

Note that the version of this algorithm is subtly different from that in Bertsekas & Gallager
which finds the paths from all nodes to a single destination. (This is not an important detail
and it should be obvious how to reverse the algorithm). To understand the algorithm we make
the following claims:
Proposition 1. At the beginning of each iteration of Dijkstra’s algorithm then:

1. Di ≤ Dj for all i ∈ P and j ∈

/ P.
2. Dj is, for all j, the shortest distance from 1 to j using paths with all nodes (except, possibly
j) in P .

If this proposition can be proved then we can see that, when P contains every node in N then
all the Dj are shortest paths by the second part of this proposition. Therefore proving the above
proposition is equivalent to proving that Dijkstra’s algorithm finds shortest paths.

1
Proof. The proposition is trivially true at the first step since P consists only of the origin point
(node 1) and Dj is 0 for j = 1, is wij ≥ 0 for nodes reachable directly from node 1 and ∞
otherwise.
The first condition is simply shown to be satisfied since it is preserved by the formula:

Dj := min[Dj , wij + Di ]

which is applied to all j ∈

/ P when node i is added to the set P .
We show the second condition by induction. We have established already that it is true at the
very start of the algorithm. Let us assume it is true for the beginning of some iteration of the
algorithm and show that it must then be true at the beginning of the next iteration.
Let node i be the node we are adding to our set P and let Dk be the label of each node k at the
beginning of the step. The second condition must, therefore, hold for node j = i (the new node
we have added) by our induction hypothesis. It must also hold for all nodes j ∈ P by part one
of the proposition which is already proven. It remains to prove that the second condition of the
proposition is met for j ∈
/ P ∪ {i}.
Consider a path from 1 to j which is shortest amongst those with all nodes except j in P ∪ {i}
and let Dj0 be the corresponding shortest distance. Such a path must contain a path from 1 to
some node r ∈ P ∪ {i} and an arc(r, j). We have already established that the length of the path
from 1 to r must be Dr and therefore we have:

Dj0 = min [Dr + wrj ] = min[min[Dr + wrj ], Di + wij ]

r∈P ∪{i} r∈P

However, by our hypothesis Dj = minr∈P [Dr + wrj ] therefore, Dj0 = min[Dj , Di + wij ] which is
exactly what is set by the fourth step of the algorithm. Thus, after any iteration of the algorithm,
the second part of the propostion is true if it was true at the beginning of the iteration. Thus
the proof by induction is complete.

The observant will have noticed that this provides the shortest distance to each node from the
origin. To find the shortest path, simply work backwards from the destination, asking which
node in step four the new node was added from.

Bellman-Ford Algorithm

Notation. Dih is the distance of the shortest walk from node 1 to node i of h steps or less.

Set initially Di0 = ∞ for i 6= 1 and D1h = 0 for all h.

The Bellman-Ford Algorithm is then simply, for all i 6= 1,

Dih+1 = min[Djh + wji ]

j

The algorithm terminates after h iterations if

Dih = Dih−1 ∀i

2
Proposition 2. The scalars Dih generate by the algorithm from the starting values given for Di0
are equal to the shortest walk of length ≤ h from node 1 to node i.

Proof. The first iteration will clearly give us Di1 = w1i for all i apart from i = 1 — which is,
indeed, correct for the walk lengths of ≤ 1 from 1 to i. Let us suppose that Dik is the shortest
walk of length ≤ k (from 1 to i) then complete the proof by induction by showing that D ik+1 is
the shortest walk of length ≤ k + 1.
One possibility is that the shortest walk of length ≤ (k + 1) will be a walk of length k or less
— in this case, Dik+1 = Dik . Otherwise, the walk will be a walk of length k + 1 with the final
arc being (j, i) added on to some walk of length k to node j. Thus, we can conclude that our
shortest walk of length k + 1 is given by:

 
Shortest walk to i of length ≤ k + 1 = min Dik , min Djk + wji
j

However, since a walk of length ≤ k + 1 must always be shorter or equal to a walk of length ≤ k
(since the former contains the latter) then this reduces to

Shortest walk to i of length ≤ k + 1 = min Djk + wji

j

which is our original expression for Djk+1 and completes our proof by induction.

Proposition 3. The algorithm terminates after a finite number of iterations if, and only if, all
cycles not containing node i have non negative length. Furthermore, if the algorithm terminates,
it does so after at most h ≤ N iterations and, at termination, Dih is the shortest path length
from 1 to i.

Proof. If negative length cycles exist then such cycles could be repeated as many times as desired
to reduce the shortest walk length and thus the algorithm could never converge. Conversely, if no
negative length cycles exist then any walks containing cycles could be made shorter or kept the
same length by deleting such a cycle. Thus, our shortest walks contain no cycles. The maximum
length of a walk with no cycles is N −1 (since such a walk will have covered every node). Thus, it
trivially follows that DiN = DiN −1 and the algorithm terminates after, at most, N iterations.

8
2 4

1 2 1 2 4

2
3 5

Figure 1: Weighted graph for Dijkstra’s algorithm and Bellman-Ford