Greedy Algorithms

Application: Optimal
Caching
Algorithms: Design
and Analysis, Part II
 The Caching Problem
Small fast memory (the cache).

Big slow memory.

Process sequence of “page requests”.

On a “fault” (that is, a cache miss), need to evict something from

cache to make room – but what?

Tim Roughgarden
 Example
Cache: a b c d
e f

Request sequence: c d e f a b

⇒ 4 page faults
- 2 were inevitable (e & f)
- 2 consequences of poor eviction choices (should have
evicted c & d instead of a & b)

Tim Roughgarden
 The Optimal Caching Algorithm
Theorem: [Bélády 1960s] The “furthest-in-future” algorithm is
optimal (i.e., minimizes the number of cache misses).

Why useful?
1. Serves as guideline for practical algorithms (e.g., Least
Recently Used (LRU) should do well provided data exhibits
locality of reference).
2. Serves as idealized benchmark for caching algorithms.

Proof: Tricky exchange argument. Open question: Find a simple

proof!

Tim Roughgarden
 Greedy Algorithms
Introduction

Algorithms: Design
and Analysis, Part II
 Algorithm Design Paradigms
Algorithm Design: No single “silver bullet” for solving problems.

Design Paradigms:
- Divide & conquer (see Part I)
- Randomized algorithms (touched in Part I)
- Greedy algorithms (next)
- Dynamic programming (later in Part II)

Tim Roughgarden
 Greedy Algorithms
“Definition”: Iteratively make “myopic” decisions, hope everything
works out at the end.

Example: Dijkstra’s shortest path algorithm (from Part I)

- Processed each destination once, irrevocably.

Tim Roughgarden
 Contrast with Divide & Conquer
1. Easy to propose multiple greedy algorithms for many problems.

2. Easy running time analysis.

(Contrast with Master method etc.)

3. Hard to establish correctness.

(Contrast with straightforward inductive correctness proofs.)

DANGER: Most greedy algorithms are NOT correct. (Even if your

intuition says otherwise!)

Tim Roughgarden
 In(correctness)
Example: Dijkstra’s algorithm with negative edge lengths. What
does the algorithm compute as the length of a shortest s-w path,
and what is the correct answer?

s 3 v
2 −2

A) 2 and 2 C) 1 and 2
B) 2 and 0 D) 2 and 1

Tim Roughgarden
 Proofs of Correctness
Method 1: Induction. (“greedy stays ahead”)

Example: Correctness proof for Dijkstra’s algorithm. (See Part I.)

Method 2: “Exchange argument”.

Example: Coming right up!

Method 3: Whatever works!

Tim Roughgarden
 Greedy Algorithms
A Scheduling Application:
Problem Definition
Algorithms: Design
and Analysis, Part II
 A Scheduling Problem
Setup:
- One shared resource (e.g., a processor).
- Many “jobs” to do (e.g., processes).

Question: In what order should we sequence the jobs?

Assume: Each job has a:

- weight wj (“priority”)
- length lJ

Tim Roughgarden
 Completion Times
Definition: The completion time Cj of job j = Sum of job lengths
up to and including j.

Example: 3 jobs, l1 = 1, l2 = 2, l3 = 3.

Schedule:
#1 #2 #3
0→
(time)

Question: What is C1 , C2 , C3 ?
A) 1, 2, 3 C) 1, 3, 6
B) 3, 5, 6 D) 1, 4, 6

Tim Roughgarden
 The Objective Function
Goal:PMinimize the weighted sum of completion times:
min nj=1 wj Cj .

Back to example: If w1 = 3, w2 = 2, w3 = 1, this sum is

3 · 1 + 2 · 3 + 1 · 6 = 15.

Tim Roughgarden
 Greedy Algorithms
A Scheduling Application:
The Algorithm
Algorithms: Design
and Analysis, Part II
 Intuition for Algorithm
Pn
Recall: Want to min j=1 wj .

Goal: Devise correct greedy algorithm.

Question:
1. With equal lengths, schedule larger or smaller-weight jobs
earlier?
2. With equal weights, schedule shorter or longer jobs earlier?

A) Larger/shorter C) Larger/longer
B) Smaller/shorter D) Smaller/longer

Tim Roughgarden
 Resolving Conflicting Advice
Question: What if wi > wj but li > lj ?

Idea: Assign “scores” to jobs that are:

- inscreasing in weight
- decreasing in length

Guess (1): Order jobs by decreasing value of wj − lj .

Guess (2): Order wj /lj .

Tim Roughgarden
 Breaking a Greedy Algorithm
To distinguish (1) & (2): Find example where the two algorithms
produce different outputs. (At least one will be incorrect.)
Example:
l1 = 5, w1 = 3 (longer ratio)
l1 = 2, w1 = 1 (larger difference)

Question: What is the sum of weighted completion times of

algorithms (1) & (2) respectively?
A) 22 and 23 C) 17 and 17
B) 23 and 22 D) 17 and 11

Alg#1: #2 #1 → 1 · 2 + 3 · 7 = 23
Alg#2: #1 #2 → 3 · 5 + 1 · 7 = 22
Tim Roughgarden
 The Story So Far
So: Alg#1 not (always) correct.

Claim: Alg#2 (order by decreasing ratio wj /lj ’s) is always correct.

[not obvious! - proof coming up next]

Running time: O(n log n). [just need to sort]

Tim Roughgarden