Greedy Algorithms

1
 Optimization problems

 An optimization problem is one in which you want

to find, not just a solution, but the best solution
 A “greedy algorithm” sometimes works well for
optimization problems
 A greedy algorithm works in phases. At each
phase:
 You take the best you can get right now, without regard
for future consequences
 You hope that by choosing a local optimum at each
step, you will end up at a global optimum

2
2
 Example: Counting money
 Suppose you want to count out a certain amount of
money, using the fewest possible bills and coins
 A greedy algorithm would do this would be:
At each step, take the largest possible bill or coin
that does not exceed
 Example: To make $6.39, you can choose:
 a $5 bill
 a $1 bill, to make $6
 a 25¢ coin, to make $6.25
 A 10¢ coin, to make $6.35
 four 1¢ coins, to make $6.39
 For US money, the greedy algorithm always gives
the optimum solution
3
3
 A scheduling problem
 You have to run nine jobs, with running times of 3, 5, 6, 10, 11,
14, 15, 18, and 20 minutes
 You have three processors on which you can run these jobs
 You decide to do the longest-running jobs first, on whatever
processor is available

P1 20 10 3

P2 18 11 6

P3 15 14 5

 Time to completion: 18 + 11 + 6 = 35 minutes

 This solution isn’t bad, but we might be able to do better
4
4
 Another approach
 What would be the result if you ran the shortest job first?
 Again, the running times are 3, 5, 6, 10, 11, 14, 15, 18, and 20
minutes

P1 3 10 15

P2 5 11 18

P3 6 14 20
 That wasn’t such a good idea; time to completion is now
6 + 14 + 20 = 40 minutes
 Note, however, that the greedy algorithm itself is fast
 All we had to do at each stage was pick the minimum or maximum
5
5
 An optimum solution
 Better solutions do exist:

P1 20 14

P2 18 11 5
P3 15 10 6 3
 One way: Try all possible assignments of jobs to processors

6
6
 Minimum spanning tree
 A minimum spanning tree is a least-cost subset of the edges of a
graph that connects all the nodes
 Start by picking any node and adding it to the tree
 Repeatedly: Pick any least-cost edge from a node in the tree to a
node not in the tree, and add the edge and new node to the tree
 Stop when all nodes have been added to the tree

4
6  The result is a least-cost
2 (3+3+2+2+2+2=14) spanning tree
4
1 5
3 2

3 3 2 3
3
4
2 4
7
7
 Traveling salesman
 A salesman must visit every city (starting from city A), and wants
to cover the least possible distance
 He can revisit a city (and reuse a road) if necessary
 He does this by using a greedy algorithm: He goes to the next
nearest city from wherever he is
 From A he goes to B
A B C  From B he goes to D
2 4
 This is not going to result in a
shortest path!
3 3
4 4  The best result he can get now
will be ABDBCE, at a cost of 16
 An actual least-cost path from A
D is ADBCE, at a cost of 14
E 8
8
 Analysis
 A greedy algorithm typically makes (approximately) n choices
for a problem of size n
 Hence the expected running time is:
O(n * O(choice(n))), where choice(n) is making a choice
among n objects
 Counting: Must find largest useable coin from among k sizes of coin (k is
a constant), an O(k)=O(1) operation;
 Therefore, coin counting is (n)
 Minimum spanning tree: At each new node, must include new edges and
keep them sorted, which is O(n log n) overall
 Therefore, MST is O(n log n) + O(n) = O(n log n)

9
9
 Other greedy algorithms
 Dijkstra’s algorithm for finding the shortest path in a
graph
 Always takes the shortest edge connecting a known node to an
unknown node
 Kruskal’s algorithm for finding a minimum-cost
spanning tree
 Always tries the lowest-cost remaining edge
 Prim’s algorithm for finding a minimum-cost spanning
tree
 Always takes the lowest-cost edge between nodes in the
spanning tree and nodes not yet in the spanning tree

10
10
 Connecting wires
 There are n white dots and n black dots, equally spaced, in a line
 You want to connect each white dot with some one black dot,
with a minimum total length of “wire”
 Example:

 Total wire length above is 1 + 1 + 1 + 5 = 8

11
11
 Collecting coins
 A checkerboard has a certain number of coins on it
 A robot starts in the upper-left corner, and walks to the
bottom left-hand corner
 The robot can only move in two directions: right and down
 The robot collects coins as it goes
 You want to collect all the coins using the minimum
number of robots
 Example:
 Do you see a greedy algorithm for
doing this?
 Does the algorithm guarantee an
optimal solution?

12
12