Greedy Algorithms:

Main Ideas
Michael Levin
Higher School of Economics

Algorithmic Toolbox
Data Structures and Algorithms

Outline
1 Largest Number
2 Car Fueling
3 Implementation and Analysis
4 Main Ingredients

Learning objectives
Come up with a greedy algorithm
yourself

Job Interview

Job Interview

Job Interview

Job Interview

Largest Number
Toy problem
What is the largest number that consists of
digits 3, 9, 5, 9, 7, 1? Use all the digits.

Largest Number
Toy problem
What is the largest number that consists of
digits 3, 9, 5, 9, 7, 1? Use all the digits.

Examples
359179, 537991, 913579, . . .

Correct answer
997531

Greedy Strategy
5 7 3 9 1 9

Greedy Strategy
Find max
5 7 3 9 1 9

Find max digit

Greedy Strategy
Find max
5 7 3 9 1 9

9
Append

Find max digit

Append it to the number

Greedy Strategy
Find max
5 7 3 9 1 9
Remove

9
Append

Find max digit

Append it to the number
Remove it from the list of digits

Greedy Strategy
Find max
5 7 3
Remove

1 9

9
Append

Find max digit

Append it to the number
Remove it from the list of digits

Greedy Strategy
Find max
5 7 3
Remove

1 9

9
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5 7 3
Remove

1 9

9
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5 7 3
Remove

1 9

9 9
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5 7 3
Remove

1 9

9 9
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5 7 3
Remove

9 9
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5 7 3
Remove

9 9
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5 7 3
Remove

9 9 7
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5 7 3
Remove

9 9 7
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5

Remove

9 9 7
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5

Remove

9 9 7
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5

Remove

9 9 7 5
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
5

Remove

9 9 7 5
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
3
Remove

9 9 7 5
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
3
Remove

9 9 7 5
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
3
Remove

9 9 7 5 3
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
3
Remove

9 9 7 5 3
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
1
Remove

9 9 7 5 3
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
1
Remove

9 9 7 5 3
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
1
Remove

9 9 7 5 3 1
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
1
Remove

9 9 7 5 3 1
Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
Find max
9 9 7 5 3 1
Remove

Append

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Greedy Strategy
5 7 3 9 1 9

9 9 7 5 3 1

Find max digit

Append it to the number
Remove it from the list of digits
Repeat while there are digits in the list

Outline
1 Largest Number
2 Car Fueling
3 Implementation and Analysis
4 Main Ingredients

Car Fueling
Distance with full tank = 400km

Car Fueling
Distance with full tank = 400km
0km

950km

Car Fueling
Distance with full tank = 400km
0km
A

200km 375km 550km 750km 950km

B

Car Fueling
Distance with full tank = 400km
0km
A

200km 375km 550km 750km 950km

B

Car Fueling
Distance with full tank = 400km
0km
A

200km 375km 550km 750km 950km

B

Car Fueling
Distance with full tank = 400km
0km

200km 375km 550km 750km 950km

A
Minimum number of refills = 2

Car Fueling
Input: A car which can travel at most L
kilometers with full tank, a source
point A, a destination point B and
n gas stations at distances
x1 x2 x3 xn in
kilometers from A along the path
from A to B.
Output: The minimum number of refills to
get from A to B, besides refill at A.

Greedy Strategy

Make some greedy choice

Reduce to a smaller problem
Iterate

Greedy Choice

Refill at the the closest gas station

Refill at the farthest reachable gas
station
Go until there is no fuel

Greedy Choice

Refill at the the closest gas station

Refill at the farthest reachable gas
station
Go until there is no fuel

Greedy Algorithm
Start at A

Greedy Algorithm
Start at A
Refill at the farthest reachable gas
station G

Greedy Algorithm
Start at A
Refill at the farthest reachable gas
station G
Make G the new A

Greedy Algorithm
Start at A
Refill at the farthest reachable gas
station G
Make G the new A
Get from new A to B with minimum
number of refills

Definition
Subproblem is a similar problem of smaller
size.

Subproblem
Examples
LargestNumber(3, 9, 5, 9, 7, 1) =

Subproblem
Examples
LargestNumber(3, 9, 5, 9, 7, 1) =
9 +

Subproblem
Examples
LargestNumber(3, 9, 5, 9, 7, 1) =
9 + LargestNumber(3, 5, 9, 7, 1)

Subproblem
Examples
LargestNumber(3, 9, 5, 9, 7, 1) =
9 + LargestNumber(3, 5, 9, 7, 1)
Min number of refills from A to B =

Subproblem
Examples
LargestNumber(3, 9, 5, 9, 7, 1) =
9 + LargestNumber(3, 5, 9, 7, 1)
Min number of refills from A to B =
first refill at G +

Subproblem
Examples
LargestNumber(3, 9, 5, 9, 7, 1) =
9 + LargestNumber(3, 5, 9, 7, 1)
Min number of refills from A to B =
first refill at G + min number of refills
from G to B

Safe Move
Definition
A greedy choice is called safe move if there is
an optimal solution consistent with this first
move.

Lemma
To refill at the farthest reachable gas station
is a safe move.

Proof

A
First case: G is closer than G2
Refill at G instead of G1

Proof
G1
A
First case: G is closer than G2
Refill at G instead of G1

Proof
G1

A
First case: G is closer than G2
Refill at G instead of G1

Proof
G1

A
First case: G is closer than G2
Refill at G instead of G1

Proof
G1

G2

A
First case: G is closer than G2
Refill at G instead of G1

Proof
G1

G2

A
First case: G is closer than G2
Refill at G instead of G1

Proof
G1

G2

A
First case: G is closer than G2
Refill at G instead of G1

Proof
G1

A
Second case: G2 is closer than G
Avoid refill at G1

Proof
G1 G2 G
A
Second case: G2 is closer than G
Avoid refill at G1

Proof
G1 G2 G
A
Second case: G2 is closer than G
Avoid refill at G1

Proof
G1 G2 G
A
Second case: G2 is closer than G
Avoid refill at G1

Proof
G1 G2 G
A
Second case: G2 is closer than G
Avoid refill at G1

Proof
Route R with the minimum number of
refills

Proof
Route R with the minimum number of
refills
G1 position of first refill in R

Proof
Route R with the minimum number of
refills
G1 position of first refill in R
G2 next stop in R (refill or B)

Proof
Route R with the minimum number of
refills
G1 position of first refill in R
G2 next stop in R (refill or B)
G farthest refill reachable from A

Proof
Route R with the minimum number of
refills
G1 position of first refill in R
G2 next stop in R (refill or B)
G farthest refill reachable from A
If G is closer than G2, refill at G instead
of G1

Proof
Route R with the minimum number of
refills
G1 position of first refill in R
G2 next stop in R (refill or B)
G farthest refill reachable from A
If G is closer than G2, refill at G instead
of G1
Otherwise, avoid refill at G1

Outline
1 Largest Number
2 Car Fueling
3 Implementation and Analysis
4 Main Ingredients

A = x0 x1 x2 xn xn+1 = B

MinRefills(x, n, L)
numRefills 0, currentRefill 0
while currentRefill n:
lastRefill currentRefill
while (currentRefill n and
x[currentRefill + 1] x[lastRefill] L):
currentRefill currentRefill + 1
if currentRefill == lastRefill:
return IMPOSSIBLE
if currentRefill n:
numRefills numRefills + 1
return numRefills

Lemma
The running time of MinRefills(x, n, L) is
O(n).

Lemma
The running time of MinRefills(x, n, L) is
O(n).

Proof
currentRefill changes from 0 to n + 1,
one-by-one

Lemma
The running time of MinRefills(x, n, L) is
O(n).

Proof
currentRefill changes from 0 to n + 1,
one-by-one
numRefills changes from 0 to at most n,
one-by-one

Lemma
The running time of MinRefills(x, n, L) is
O(n).

Proof
currentRefill changes from 0 to n + 1,
one-by-one
numRefills changes from 0 to at most n,
one-by-one
Thus, O(n) iterations

Outline
1 Largest Number
2 Car Fueling
3 Implementation and Analysis
4 Main Ingredients

Reduction to Subproblem

Make a first move

Then solve a problem of the same kind
Smaller: fewer digits, fewer fuel stations
This is called a subproblem

Safe move
A move is called safe if there is an
optimal solution consistent with this
first move

Safe move
A move is called safe if there is an
optimal solution consistent with this
first move
Not all first moves are safe

Safe move
A move is called safe if there is an
optimal solution consistent with this
first move
Not all first moves are safe
Often greedy moves are not safe

General Strategy
Problem

General Strategy
Problem

greedy choice

Make a greedy choice

General Strategy
Problem

greedy choice

Make a greedy choice

Prove that it is a safe move

Safe move

General Strategy
Problem

greedy choice

Subproblem
Make a greedy choice
Prove that it is a safe move
Reduce to a subproblem

Safe move

General Strategy
Problem

greedy choice

Subproblem
Make a greedy choice
Prove that it is a safe move
Reduce to a subproblem
Solve the subproblem

Safe move