Stanford University CS161: Algorithms Luca Trevisan

Handout 7 April 22, 2013

Lecture 7
In which we implement Dijkstras algorithm eciently.

Shortest Paths

We want to describe an ecient implementation of the algorithm for nding a shortest path from Section 4.4 of the textbook. The algorithm takes in input a (possibly, directed) graph G = (V, E ), a start vertex s S , and a function (, ) that given an edge (u, v ) E returns a nonnegative length (u, v ) 0. For every vertex v reachable from s, we want to nd the length of the shortest path from s to v , where the length of a path s = v0 , v1 , . . . , vk = v is
k1

(vi , vi+1 ) ,
i=0

that is, the sum of the lengths of the edges in the path. The outline of the algorithm is given in Algorithm 1. The proof of correctness is given in the book: the algorithm terminates because the set S grows by one vertex at each step, and the algorithm maintains the invariant (at the beginning of each iteration of the while loop) that, for every vertex u in S , dist[u] is the correct length of the shortest path from s to u. This is true at the beginning, when S = {s}, and we have dist[s] = 0, and, if it is true at a certain iteration, then we need to show that it remains true after the addition of vertex v to S in Line 7 at the following iteration. Indeed, let v be the vertex added at the next iteration, and u be the vertex in S such that we set d[v ] = dist[u] + (u, v ); because the invariant is true at the previous step, dist[u] is the length of the shortest path from s to u, so there is a path from s to v of length d[v ]; the only way that the invariant could fail is if there is some other path in the graph from s to v of length shorter than d[v ]; if such a path existed, call it s = v0 , . . . , vk = v , and let vi be the rst vertex in the path that is not in S ; then the path v0 , . . . , vi would also be shorter than d[v ], but then we would have d[vi ] dist[vi1 ] + (vi1 , vi ), and so d[vi ] would be smaller than d[v ], contradicting the fact that v is the vertex with the smallest value of d[v ]. 1

Algorithm 1 Dijkstras algorithm, general outline. 1: function shortest-path(G = (V, E ), s, ) 2: S {s} 3: initialize all entries of a size-|V | array dist[] to 4: dist[s] 0 5: while there are edges with one endpoint in S and one endpoint in V S do 6: initialize all entries of a size-|V | array d[] to 7: for each v such that v S and u S such that (u, v ) E do 8: compute d[v ] = minuS :(u,v)E dist[u] + (v ) 9: end for 10: Let v be a vertex not in S whose value of d[v ] is minimum 11: Add v to S 12: dist[v ] d[v ] 13: end while 14: end function

Ecient Implementation

A rst idea to speed up an implementation of the algorithm is to keep the vertex d[] from the previous step, and adjusting it as needed for the next iteration, instead of recalculating it from scratch. The point is that the value of d[w] changes from an iteration to the next only if we add to S a vertex v such that the edge (v, w) exists. So we can update the vector d[] at the time in which we add a new element v to S , and we only need to update d[w] for neighbors v of w. In Algorithm 2, each iteration of the while cycle starts with the value of d[] already equal to the value computed in Lines 7-8 in Algorithm 1. To speed up the execution of Line 10 of Algorithm 2, we use a priority queue data structure. A priority queue keeps a set of items, and a numerical key value for each item. The allowed operations are to create an empty queue, to test if a queue is empty, to insert an element with a given key, to nd (and also return, and delete from the queue) the element with minimal queue, and to change the key of a given element by decreasing it. We maintain in a priority queue the vertices v that are not in S but such that there is some edge (u, v ) with u S ; the vertices v in the queue have key d[v ]. Initially, S = {s} and so the queue contains the neighbors of s. When the queue is empty, it means that there is no edge with an endpoint in S and one endpoint outside of S , which is the termination condition of the while loop. These ideas lead to the implementation in Algorithm 3. Algorithm 3 does O(|V |) work at the beginning (and, possibly, up to |V | insert operations), then the while loop is executed at most once for every vertex v (because

Algorithm 2 Dijkstras algorithm, some details lled in. 1: function shortest-path(G = (V, E ), s, ) 2: S {s} 3: initialize all entries of a size-|V | array dist[] to 0 4: initialize all entries of a size-|V | array d[] to + 5: for each v in s.neighbors do 6: d[v ] (s, v ) 7: end for 8: while there are edges with one endpoint in S and one endpoint in V S do 9: Let v be a vertex not in S whose value of d[v ] is minimum 10: Add v to S 11: dist[v ] d[v ] 12: for each w in v.neighbors such that w S do 13: if d[w] > dist[v ] + (v, w) then d[w] dist[v ] + (v, w) 14: end if 15: end for 16: end while 17: end function when a vertex v is removed from the queue it is put in S , and it is only vertices not in S that are ever put in the queue) and one iteration of the while loop requires O(outdegree(v )) operations such as comparisons and assignments that take O(1) time each, plus O(outdegree(v )) operations between decreasekey and insert. Since the total number of insert operation is at most |V | (because it is only vertices not in S that are inserted, and so no vertex can be inserted more than once), in total we have O(|V | + |E |) operations plus O(|V |) insert plus O(|E |) decreasekey. If the priority queue is implemented with a heap, then each queue operation takes O(log k ) times where k is the number of elements in the queue; at every step we have k |V |, and so the total running time is O((|V | + |E |) log |V |). It remains to clean up the (pseudo)code of Algorithm 3. A rst observation is that if we start the while loop with an empty S and with s being the only element of Q, then the rst iteration of the while loop will perform exactly the initialization that we currently do right before the loop. (That is, s will be put in S , and the neighbors of s will be put in the queue.) The second observation is that we only access dist[v ] for v S and d[v ] for v S , so may as well have a single vector dist[], with the understanding that the content of dist[v ] for v S is what we previously stored in d[]. The third observation is that the order in which vertices v are removed from the queue and added to S in Lines 11 and 12 of Algorithm 3 is an increasing (or, at least, nondecreasing) oder of dist[v ]. (This can be shown to be an invariant of the algorithm.) This means that even if do not check if w S in line 14, a vertex w S would never satisfy the if conditions in lines 15 and 18, and so may avoid checking if 3

Algorithm 3 Dijkstras algorithm, almost done. 1: function shortest-path(G = (V, E ), s, ) 2: S {s} 3: initialize all entries of a size-|V | array dist[] to 0 4: initialize all entries of a size-|V | array d[] to 1 5: initialize empty priority queue Q 6: for each v in s.neighbors do 7: d[v ] (s, v ) 8: insert(Q, v, d[v ]) 9: end for 10: while Q is not empty do 11: v removemin(Q) 12: Add v to S 13: dist[v ] d[v ] 14: for each w in v.neighbors such that w S do 15: if d[w] == 1 then 16: d[w] dist[v ] + (v, w) 17: insert(Q, w, d[w]) 18: else if d[w] > dist[v ] + (v, w) then 19: d[w] dist[v ] + (v, w) 20: decreasekey(Q, w, d[w]) 21: end if 22: end for 23: end while 24: end function

w is in S . But is the only use of the set S , so may avoid maintaining the set S at all. Algorithm 4 Dijkstras algorithm, cleaned up.
1: function Shortest-Path(G = (V,E), s) 2: vector<int> dist(n, -1); 3: vector<bool> opt(n, false); 4: priority_queue<node, weight> Q; 5: dist[s] = 0; 6: Q.insert(s, 0); 7: while (not Q.empty) 8: (v, _) = Q.remove_min(); 9: opt[v] = true; 10: for ((w, edgeCost) in v.neighbors) do 11: if (dist[w] == -1) then 12: dist[w] = dist[v] + edgeCost 13: Q.insert(w, d[w]); 14: else if dist[w] > dist[v] + edgeCost then 15: dist[w] = dist[v] + edgeCost 16: Q.decreaseKey(w, dist[w]) 17: end if 18: end for 19: end while 20: return dist 21: end function

In the ipbook animation, the bright red vertex is the current vertex v. The light red vertices are the vertices v for which opt[v] is true. The green edge/vertex is the current vertex whose distance were trying to decrease.