Vdocument - in - Advanced Algorithm Design and Analysis
Vdocument - in - Advanced Algorithm Design and Analysis
and Analysis
Jiaheng Lu
Renmin University of China
www.jiahenglu.net
Directed Acyclic Graphs
A directed acyclic graph or DAG is a
directed graph with no directed cycles:
DFS and DAGs
Argue that a directed graph G is acyclic iff a DFS of
G yields no back edges:
Forward: if G is acyclic, will be no back edges
Trivial: a back edge implies a cycle
Backward: if no back edges, G is acyclic
Argue contrapositive: G has a cycle a back edge
Let v be the vertex on the cycle first discovered, and u be the
predecessor of v on the cycle
When v discovered, whole cycle is white
Must visit everything reachable from v before returning from DFS-
Visit()
So path from uv is yellowyellow, thus (u, v) is a back edge
Topological Sort
Topological sort of a DAG:
Linear ordering of all vertices in graph G
such that vertex u comes before vertex v
if edge (u, v) G
Real-world example: getting dressed
Getting Dressed
Underwear Socks
Watch
Pants Shoes
Shirt
Belt
Tie
Jacket
Getting Dressed
Underwear Socks
Watch
Pants Shoes
Shirt
Belt
Tie
Jacket
14 2
10
15
3 8
Minimum Spanning Tree
Problem: given a connected,
undirected, weighted graph, find a
spanning tree using edges that
6 4
minimize the
5 total weight 9
14 2
10
15
3 8
Minimum Spanning Tree
Which edges form the minimum
spanning tree (MST) of the below
graph? A
6 4
5 9
H B C
14 2
10
15
G E D
3 8
F
Minimum Spanning Tree
Answer:
A
6 4
5 9
H B C
14 2
10
15
G E D
3 8
F
Minimum Spanning Tree
MSTs satisfy the optimal substructure property: an
optimal tree is composed of optimal subtrees
Let T be an MST of G with an edge (u,v) in the middle
Removing (u,v) partitions T into two trees T1 and T2
Claim: T1 is an MST of G1 = (V1,E1), and T2 is an MST of
G2 = (V2,E2) (Do V1 and V2 share vertices? Why?)
Proof: w(T) = w(u,v) + w(T1) + w(T2)
(There can’t be a better tree than T1 or T2, or T would be
suboptimal)
Minimum Spanning Tree
Thm:
Let T be MST of G, and let A T be
subtree of T
Let (u,v) be min-weight edge connecting
A to V-A
Then (u,v) T
Minimum Spanning Tree
Thm:
Let T be MST of G, and let A T be
subtree of T
Let (u,v) be min-weight edge connecting
A to V-A
Then (u,v) T
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r)
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
key[r] = 0;
p[r] = NULL; 3 8
while (Q not empty) Run on example graph
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r)
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
key[r] = 0;
p[r] = NULL; 3 8
while (Q not empty) Run on example graph
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r)
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
r 0
key[r] = 0;
p[r] = NULL; 3 8
while (Q not empty) Pick a start vertex r
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r)
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
u 0
key[r] = 0;
p[r] = NULL; 3 8
while (Q not empty) Red vertices have been removed from Q
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r)
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
u 0
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty) Red arrows indicate parent pointers
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r) 14
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
u 0
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r) 14
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0
key[r] = 0;
p[r] = NULL; 3 3 8
u
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r) 14
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8
key[r] = 0;
p[r] = NULL; 3 3 8
u
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r) 10
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8
key[r] = 0;
p[r] = NULL; 3 3 8
u
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r) 10
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8
key[r] = 0;
p[r] = NULL; 3 3 8
u
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r) 10 2
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8
key[r] = 0;
p[r] = NULL; 3 3 8
u
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
5 9
MST-Prim(G, w, r) 10 2
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
u
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
u
5 9
MST-Prim(G, w, r) 10 2
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4
u
5 9
MST-Prim(G, w, r) 10 2 9
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4 4
u
5 9
MST-Prim(G, w, r) 10 2 9
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4 4
u
5 9
MST-Prim(G, w, r) 5 2 9
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm u
6 4 4
5 9
MST-Prim(G, w, r) 5 2 9
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
u 6 4 4
5 9
MST-Prim(G, w, r) 5 2 9
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
u
6 4 4
5 9
MST-Prim(G, w, r) 5 2 9
Q = V[G];
for each u Q 14 2
10
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Prim’s Algorithm
6 4 4
5 9
MST-Prim(G, w, r) 5 2 9
Q = V[G];
for each u Q 14
10
2 u
key[u] = ; 15
0 8 15
key[r] = 0;
p[r] = NULL; 3 3 8
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
What is the hidden cost in this code?
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q How often is ExtractMin() called?
key[u] = ;
key[r] = 0;
How often is DecreaseKey() called?
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G]; What will be the running time?
for each u Q A: Depends on queue
key[u] = ; binary heap: O(E lg V)
key[r] = 0;
p[r] = NULL;
Fibonacci heap: O(V lg V + E)
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Single-Source Shortest Path
Problem: given a weighted directed
graph G, find the minimum-weight path
from a given source vertex s to
another vertex v
“Shortest-path” = minimum weight
Weight of path is sum of edges
E.g., a road map: what is the shortest
path from Chapel Hill to Charlottesville?
Binary Heap
A special kind of binary tree. It has two properties
that are not generally true for other trees:
Completeness
The tree is complete, which means that nodes are
added from top to bottom, left to right, without
leaving any spaces. A binary tree is completely full if
it is of height, h, and has 2h+1-1 nodes.
Heapness
The item in the tree with the highest priority is at the
top of the tree, and the same is true for every
subtree.
Binary Heap
Binary tree of height, h, is complete iff
it is empty
or
its left subtree is complete of height h-1
and its right subtree is completely full of
height h-2
or
its left subtree is completely full of height
h-1 and its right subtree is complete of
height h-1.
Binary Heap
In simple terms:
87 54 54 27
87 87 87
54 27 67 27 67 27
67 54 54 19
Binary Heap
54, 87, 27, 67, 19, 31, 29, 18, 32, 56, 7, 12, 31
87 87
67 27 67 27
54 19 19 31
54
Binary Heap
54, 87, 27, 67, 19, 31, 29, 18, 32, 56, 7, 12, 31
87
87
67 31 67 31
54 19 27 29
54 19 27 29
18 32
etc…
Binary Heap
To delete an element from the heap:
Algorithm:
If node is the last “logical node” in the tree,
simply delete it
Else:
● Replace the node with the last “logical
node” in the tree
● Delete the last logical node from the tree
● Re-heapify
Binary Heap
Example: Deleting the root node, T
Binary Heap
Binary Heap
Binary Heap
Min-Max Heap
Deaps
Binomial Heaps
Fibonacci Heaps
MIN-MAX Heaps (1/10)
Definition
A double-ended priority queue is a data structure that
supports the following operations:
Insert an element with arbitrary key
Delete an element with the largest key
Delete an element with the smallest key
Min heap or Max heap:
Only insertion and one of the two deletion operations are
supported
Min-Max heap:
Supports all of the operations just described.
Definition:
A mix-max heap is a complete binary tree such that if it is
not empty, each element has a field called key.
Alternating levels of this tree are min levels and max levels,
respectively.
Let x be any node in a min-max heap. If x is on a min (max)
level then the element in x has the minimum (maximum)
key from among
all elements in
the subtree with
root x. We call
this node a min
(max) node.
Insertion into a min-max heap (at a “max” level)
If it is smaller/greater than its father (a “min”), then it
must be smaller/greater than all “max”/“min” above. So
simply check the “min”/“max” ancestors
There exists a similar approach at a “min” level
Following the nodes the max node i to the root and insert
into its proper
place
item = 80
i=313
grandparent = 0
3
item.key = 80
5
*n = 14
12
13 complexity: O(log n)
parent = 7
6
[1 7
5 min
]
[2 70 [3 40
80 max
] ]
[4 [5 [6 [7
] 30 ] 9 ] 10 ]
7 15 min
45 50 30
20 12 10 40 max
[8 [9 [10 [11 [12 [13 [14
Deletion of min element
If we wish to delete the element with the smallest key,
then this element is in the root.
In general situation, we are to reinsert an element item
into a min-max-heap, heap, whose root is empty.
We consider the two cases:
1. The root has no children
Item is to be inserted into the root.
2. The root has at least one child.
The smallest key in the min-max-heap is in one of the children
or grandchildren of the root. We determine the node k has the
smallest key.
The following possibilities need to be considered:
a) item.key heap[k].key
No element in heap with key smaller than item.key
Item may be inserted into the root.
b) item.key heap[k].key, k is a child of the root
Since k is a max node, it has no descendants with key
larger than heap[k].key. Hence, node k has no
descendants with key larger than item.key.
heap[k] may be
moved to the
root and item
inserted into
node k.
c) item.key heap[k].key,
k is a grandchild of the root
In this case, heap[k] may be moved to the root, now
heap[k] is seen as presently empty.
Let parent be the parent of k.
If item.key heap[parent].key, then interchange them.
This ensures that the max node parent contains the
largest key in the sub-heap with root parent.
At this point, we are faced with the problem of inserting
item into the
sub-heap with
root k.
Therefore, we
repeat the above
process.
delete_min: complexity: O(log n)
Delete the minimum element from the min-max heap
*n = 12
11
i=5 1
last = 5
k = 11
5
parent = 2
temp.key =
x.key = 12
[1 79 [0 7
] ]
[2 70 [3 40
] ]
[4 [5 [6 [7
] 30 ] 912 ] 10 ] 15
45 50 30
20 12
[8 [9 [10 [11 [12
Deletion of max element
1. Determine the children of the root which are located on
max-level, and find the larger one (node) which is the
largest one on the min-max heap
2. We would consider the node as the root of a max-min
heap
3. There exist a max-min heap
similar
approach
(deletion of
max element)
as we
mentioned
above
Deaps(1/8)
Definition
The root contains no element
The left subtree is a min-heap
The right subtree is a max-heap
Constraint between the two trees:
let i be any node in left subtree, j be the
corresponding node in the right subtree.
if j not exists, let j corresponds to parent of i
i.key <= j.key
Deaps(2/8)
log 2 n 1
i = min_partner(n) = n2
log 2 n 1
j = max_partner(n)
n2
=
if j > heapsize j /= 2
Deaps Insert(3/8)
public void insert(int x) { } else {
int i; i = maxPartner(n);
if (++n == 2) { if (x > deap[i]) {
deap[2] = x; return; } deap[n] = deap[i];
if (inMaxHeap(n)) { maxInsert(i, x);
i = minPartner(n); } else minInsert(n, x);
if (x < deap[i]) { }
deap[n] = deap[i]; }
minInsert(i, x);
} else maxInsert(n, x);
Deaps Insert(3/8)
public void insert(int x) {
int i;
if (++n == 2) {
deap[2] = x; return; }
if (inMaxHeap(n)) {
i = minPartner(n);
if (x < deap[i]) {
deap[n] = deap[i];
minInsert(i, x);
} else maxInsert(n, x);
Deaps Insert(3/8)
http://www.csie.ntnu.edu.tw/~swanky/ds/ch
ap9.htm
S k Fk 2
That’s why the data structure is called
Fibonacci Heap
Fibonacci Heaps(8/8)
Application Of F-heaps