Algorithms Illuminated Part 3 Greedy Algorithms and Dynamic Programming Tim Roughgarden All Chapters Instant Download
Algorithms Illuminated Part 3 Greedy Algorithms and Dynamic Programming Tim Roughgarden All Chapters Instant Download
com
https://textbookfull.com/product/algorithms-illuminated-
part-3-greedy-algorithms-and-dynamic-programming-tim-
roughgarden/
OR CLICK BUTTON
DOWNLOAD NOW
https://textbookfull.com/product/algorithms-illuminated-part-3-greedy-
algorithms-and-dynamic-programming-1st-edition-tim-roughgarden/
textboxfull.com
https://textbookfull.com/product/beyond-the-worst-case-analysis-of-
algorithms-1st-edition-tim-roughgarden/
textboxfull.com
https://textbookfull.com/product/algorithms-for-competitive-
programming-1st-edition-david-esparza-alba/
textboxfull.com
https://textbookfull.com/product/twenty-lectures-on-algorithmic-game-
theory-1st-edition-tim-roughgarden/
textboxfull.com
Algorithms and Programs of Dynamic Mixture Estimation
Unified Approach to Different Types of Components Ivan
Nagy
https://textbookfull.com/product/algorithms-and-programs-of-dynamic-
mixture-estimation-unified-approach-to-different-types-of-components-
ivan-nagy/
textboxfull.com
https://textbookfull.com/product/programming-quantum-computers-
essential-algorithms-and-code-samples-1st-edition-eric-r-johnston/
textboxfull.com
https://textbookfull.com/product/art-of-computer-programming-the-
combinatorial-algorithms-volume-4b-1st-edition-donald-knuth/
textboxfull.com
Algorithms Illuminated
Part 3: Greedy Algorithms and Dynamic
Programming
Tim Roughgarden
c 2019 by Tim Roughgarden
All rights reserved. No portion of this book may be reproduced in any form
without permission from the publisher, except as permitted by U. S. copyright
law.
First Edition
Preface vii
14 Huffman Codes 23
14.1 Codes 23
14.2 Codes as Trees 28
14.3 Huffman’s Greedy Algorithm 32
*14.4 Proof of Correctness 41
Problems 49
v
vi Contents
Index 211
Preface
vii
viii Preface
This series of books has only one goal: to teach the basics of algorithms
in the most accessible way possible. Think of them as a transcript
of what an expert algorithms tutor would say to you over a series of
one-on-one lessons.
There are a number of excellent more traditional and encyclopedic
textbooks about algorithms, any of which usefully complement this
book series with additional details, problems, and topics. I encourage
you to explore and find your own favorites. There are also several
books that, unlike these books, cater to programmers looking for
ready-made algorithm implementations in a specific programming
language. Many such implementations are freely available on the Web
as well.
x Preface
Additional Resources
These books are based on online courses that are currently running
on the Coursera and Stanford Lagunita platforms. I’ve made several
resources available to help you replicate as much of the online course
experience as you like.
Videos. If you’re more in the mood to watch and listen than
to read, check out the YouTube video playlists available from
www.algorithmsilluminated.org. These videos cover all the topics
in this book series, as well as additional advanced topics. I hope they
exude a contagious enthusiasm for algorithms that, alas, is impossible
to replicate fully on the printed page.
Quizzes. How can you know if you’re truly absorbing the concepts
in this book? Quizzes with solutions and explanations are scattered
throughout the text; when you encounter one, I encourage you to
pause and think about the answer before reading on.
End-of-chapter problems. At the end of each chapter you’ll find
several relatively straightforward questions for testing your under-
Preface xi
Acknowledgments
These books would not exist without the passion and hunger supplied
by the hundreds of thousands of participants in my algorithms courses
over the years. I am particularly grateful to those who supplied
detailed feedback on an earlier draft of this book: Tonya Blust, Yuan
Cao, Carlos Guia, Jim Humelsine, Vladimir Kokshenev, Bayram
Kuliyev, and Daniel Zingaro.
I always appreciate suggestions and corrections from readers.
These are best communicated through the discussion forums men-
tioned above.
Tim Roughgarden
New York, NY
April 2019
Chapter 13
1
2 Introduction to Greedy Algorithms
The first half of this book is about the greedy algorithm design
paradigm. What is a greedy algorithm, exactly? Much blood and ink
have been spilled over this question, so we’ll content ourselves with
an informal definition.1
The best way to get a feel for greedy algorithms is through exam-
ples. We’ll see several over the next few chapters.2
Warning
Most greedy algorithms are not always correct.
Our first case study concerns scheduling, in which the goal is to sched-
ule tasks on one or more shared resources to optimize some objective.
For example, a resource could represent a computer processor (with
tasks corresponding to jobs), a classroom (with tasks corresponding
to lectures), or your calendar for the day (with tasks corresponding
to meetings).
Completion Times
Quiz 13.1
Consider a problem instance that has three jobs with `1 = 1,
`2 = 2, and `3 = 3, and suppose they are scheduled in this
order (with job 1 first). What are the completion times
of the three jobs in this schedule? (The job weights are
irrelevant for this question, so we have not specified them.)
a) 1, 2, and 3
b) 3, 5, and 6
c) 1, 3, and 6
d) 1, 4, and 6
3 · 1 + |{z}
|{z} 2 · 3 + |{z}
1 · 6 = 15.
job #1 job #2 job #3
6 Introduction to Greedy Algorithms
Greedy algorithms seem like a good fit for the problem of scheduling
jobs to minimize the weighted sum of completion times. The output
has an iterative structure, with jobs processed one by one. Why not
6
For example, n! is bigger than 3.6 million when n = 10, bigger than 2.4
quintillion when n = 20, and bigger than the estimated number of atoms in the
known universe when n 60. Thus no conceivable improvement in computer
technology would transmute exhaustive search into a useful algorithm.
13.3 Developing a Greedy Algorithm 7
job #3
time
3
job #2
1
job #1
0
Figure 13.1: The completion times of the three jobs are 1, 3, and 6.
Quiz 13.2
a) larger/shorter
b) smaller/shorter
c) larger/longer
d) smaller/longer
In the general case, jobs can have different weights and different
lengths. Whenever our two rules-of-thumb—to prefer shorter jobs
and higher-weight jobs—luckily coincide for a pair of jobs, we know
which one to schedule first (the shorter, higher-weight one). But what
if the two rules give conflicting advice? What should we do with one
short low-weight job and one long high-weight job?
What’s the simplest greedy algorithm that might work? Each
job has two parameters, and the algorithm must look at both. The
best-case scenario would be to come up with a formula that compiles
each job’s length and weight into a single score, so that scheduling
jobs from highest to lowest score is guaranteed to minimize the sum of
weighted completion times. If such a formula exists, our two special
cases imply that it must have two properties: (i) holding the length
fixed, it should be increasing in the job’s weight; and (ii) holding the
weight fixed, it should be decreasing in the job’s length. (Remember,
higher scores are better.) Take a minute to brainstorm some formulas
that have both of these properties.
* * * * * * * * * * *
There are plenty of other options. For example, the ratio of the
two parameters is another candidate:
wj
proposal #2 for score of job j: .
`j
These two scoring functions lead to two different greedy algo-
rithms.
GreedyDiff
Schedule the jobs in decreasing order of wj `j
(breaking ties arbitrarily).
GreedyRatio
wj
Schedule the jobs in decreasing order of `j
(breaking ties arbitrarily).
Thus, already, our first case study illustrates the first theme of the
greedy paradigm (Section 13.1.2): It is often easy to propose multiple
competing greedy algorithms for a problem.
Which of the two algorithms, if any, is correct? A quick way to
rule out one of them is to find an instance in which the two algorithms
output different schedules, with different objective function values.
For whichever algorithm fares worse in this example, we can conclude
that it is not always optimal.
Both algorithms do the right thing in our two special cases, with
equal-weight or equal-length jobs. The simplest possible example for
ruling out one of them would be a problem instance with two jobs,
having different weights and lengths, such that the two algorithms
schedule the jobs in opposite orders. That is, we seek two jobs whose
ordering by difference is the opposite of their ordering by ratio. One
simple example is:
Job #1 Job #2
Length `1 = 5 `2 = 2
Weight w1 = 3 w2 = 1.
The first job has the larger ratio ( 35 vs. 12 ) but the smaller (more
negative) difference ( 2 vs. 1). Thus the GreedyDiff algorithm
schedules the second job first, while GreedyRatio does the opposite.
10 Introduction to Greedy Algorithms
Quiz 13.3
What is the sum of weighted completion times in the sched-
ules output by the GreedyDiff and GreedyRatio algorithms,
respectively?
a) 22 and 23
b) 23 and 22
c) 17 and 17
d) 17 and 11
Correct answer: (a). First suppose that all n jobs have the same
length, say length 1. Then, every schedule has exactly the same
set of completion times—{1, 2, 3, . . . , n}—and the only question is
which job gets which completion time. Our semantics for job weights
certainly suggests that the higher-weight jobs should receive the
smaller completion times, and this is in fact the case. For example,
you wouldn’t want to schedule a job with weight 10 third (with
completion time 3) and one with weight 20 fifth (with completion
time 5); you’d be better off exchanging the positions of these two jobs,
which would decrease the sum of weighted completion times by 20 (as
you should check).
The second case, in which all jobs have equal weights, is a little
more subtle. Here, you want to favor shorter jobs. For example,
consider two unit-weight jobs with lengths 1 and 2. If you schedule
the shorter job first, the completion times are 1 and 3, for a total
of 4. In the opposite order, the completion times are 2 and 3, for
an inferior total of 5. In general, the job scheduled first contributes
to the completion times of all the jobs, as all jobs must wait for
the first one to finish. All else being equal, scheduling the shortest
job first minimizes this negative impact. The second job contributes
12 Introduction to Greedy Algorithms
to all the completion times other than that of the first job, so the
second-shortest job should be scheduled next, and so on.
w1 · C1 + w2 · C2 = 3 · 7 + 1 · 2 = 23.
3 · 5 + 1 · 7 = 22.
Two Assumptions
wi wj
(2) There are no ties between ratios: `i 6= `j whenever
i 6= j.
job #n
..........
job #3
time
job #2
job #1
σ
more more
stuff stuff
j i
exchange!
i j
time
time
stuff stuff
σ* σ’
(a) Before exchange (b) After exchange
Figure 13.3: Obtaining the new schedule 0 from the allegedly optimal
schedule ⇤ by exchanging the jobs in a consecutive inversion (with i > j).
Quiz 13.4
What effect does the exchange have on the completion time
of: (i) a job other than i or j; (ii) the job i; and (iii) the
job j?
Now is the time to use the fact that ⇤ scheduled i and j in the “wrong
order,” with i > j. Our standing assumptions (1) and (2) imply that
jobs are indexed in strictly decreasing order of weight-length ratio, so
wi wj
< .
`i `j
w i `j < wj `i .
|{z} |{z}
cost of exchange benefit of exchange
Because the benefit of the exchange exceeds the cost, equation (13.3)
tells us that
0 ⇤
objective function value of < objective function value of .
»Sanon näin paljon vain siksi», selitti hän, »että mr Merdle aina on
osoittanut suurta myötätuntoa Edmundia kohtaan ja aina sanonut
hartaasti harrastavansa hänen parastaan. Tehän tunnette Edmundin
julkisen aseman. Hänen yksityisasemansa on kokonaan mr Merdlen
varassa. Kun olen niin typerän kykenemätön käsittämään liikeasioita,
en todellakaan tiedä tämän enempää.»
»Mrs General.»
»Mr Dorrit», lisäsi hän ääneen, »on aina kovin kohtelias, ja pyydän
kiittää tästä huomaavaisuudesta, tahdonpa lisätä:
kunnianosoituksesta, joka on tullut osakseni sen kautta, että hän ja
miss Dorrit näin aikaisin ovat ilmoittaneet minulle tapahtumasta. Niin
hyvin kiitokseni kuin onnittelunikin osoitan sekä mr Dorritille että
miss Dorritille.»
»Älä luule, että olen kiivas ja paha, lemmikkini, sillä sitä en ole.
Mutta sinä olet niin merkillinen pieni olio! Sinä saatat toisen
puraisemaan sinulta pään poikki, vaikka oikeastaan tahtoisi olla
sinulle mahdollisimman hyvä. Enkös minä sanonut sinulle, rakkahin
vauvani, että Edmundia ei uskalla päästää yksin matkustamaan? Ja
etkö tiedä, ettei hän kykenisi siihen?»
»No niin, koska olet kuullut, kuinka asiat voitaisiin järjestää niin,
että tämä kävisi päinsä, niin onko tarkoituksesi neuvoa minua
menettelemään näin?»
»Kun jään tänne yksin mrs Generalin kanssa?» toisti Pikku Dorrit
hiljaa.
»Oi ei! Anna minun jäädä sinun luoksesi. Pyydän ja rukoilen, että
sallit minun jäädä luoksesi. En tahdo mitään muuta kuin olla sinun
kanssasi ja pitää huolta sinusta, rakas isä!» Hän sanoi tämän kuin
äkkihätään joutunut.
»Älä puhu mitään, Amy. Sanon varmasti, etten voi tehdä niin. En
— hm — saa tehdä niin. Hm — omatuntoni ei salli sitä. Sentähden,
ystäväni, käytän tämän vaikuttavan ja ilahduttavan tapahtuman
tarjoamaa tilaisuutta — hm — vakavasti ilmoittaakseni sinulle, että
harras haluni ja tarkoitukseni nyt on saada sinut — hm—sopivasti
(toistan: sopivasti) naitetuksi.»
Jos Pikku Dorritin mieleen tänä iltana juolahti ajatus, että isä perin
helposti luopui hänestä nyt, rikkaudessaan, ja aikoessaan asettaa
toisen hänen sijallensa, niin hän karkoitti sen ajatuksen kohta. Hän
pysyi isälle yhtä uskollisena nyt kuin vaikeinakin aikoina, jolloin oli
yksin ollut vanhuksen tukena ja apuna, eikä hän nytkään,
tuskaisessa levottomuudessaan, suonut mielessään sijaa
ankarammille ajatuksille kuin että isä nyt katseli kaikkea heidän
rikkautensa valossa ja lakkaamatta huolehtien Mitä, että he
pysyisivät rikkaina ja yhä rikastuisivat.
Edistystä
Siinä hän ajoi, kunnes Brook Street pysähdytti hänet. Siinä jalokivi
ilmestyi loistavasta lippaastaan, itse kaikkea muuta kuin loistavana.
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
textbookfull.com