This course material is now made available for public usage.

Special acknowledgement to School of Computing, National University of Singapore

CS3233
C
CompetitiveProgramming
ii P
i
Dr.StevenHalim
Week04 ProblemSolvingParadigms
(Dynamic Programming 1)
(DynamicProgramming1)

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

O tli
Outline
MiniContest#3+Break+Discussion+Admins
DynamicProgramming
Dynamic Programming Introduction
Treatthisasrevision forexCS2010/CS2020students
Listencarefully
Listen carefully fortheothergroupofstudents!
for the other group of students!
Iopenconsultationslots(Mon/Fri)forNUSstudents
whoneedhelpwiththistopic,especiallythose
p
p , p
y
whodidnotgothroughCS2010/CS2020before

DynamicProgramming
y
g
g
SomeClassicalExamples

PS:IwillusethetermDP inthislecture
OOT:DPisNOTDownPayment!
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

WeddingShopping
EXAMPLE1

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

M ti ti
Motivation
HowtosolveUVa11450 (WeddingShopping)?
Given1C
Gi
1 C 20classesofgarments
20 l
f
t
e.g.shirt,belt,shoe

Given1K
Gi
1 K 20differentmodelsforeachclassofgarment
20 diff
d l f
h l
f
e.g.threeshirts,twobelts,fourshoes,,eachwithitsownprice

Task:Buyjustone modelofeachclass ofgarment

Whatisourmaximumpossiblespending?
Outputnosolutionifthisisimpossible
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

Model
Garment

BudgetM=100
Answer:75

BudgetM=20

Alternativeanswers
arepossible

BudgetM=5
Budget M = 5
Answer:nosolution

10

C=3

50

14

23

Model
Garment

Answer:19

10

C=2

Model
M
d l
Garment

10

C=2

8
5
1

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

5
2

8
1

G d S l ti ?
GreedySolution?
Whatifwebuythemostexpensivemodelforeach
garment which still fits our budget?
garmentwhichstillfitsourbudget?
Counterexample:
Model
0
1
2
M=12
Greedywillproduce:
Greedy will produce:
nosolution

Wronganswer!
Wrong answer!

Garment
0
1
C=2

10

Thecorrectansweris12
(see the green dotted highlights)
(seethegreendottedhighlights)
Q:Canyouspotonemorepotentialoptimalsolution?

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

8
3

Di id
DivideandConquer?
dC
?
Anyidea?

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearch?(1)
l t S
h? (1)
Whatisthepotentialstate oftheproblem?
g(whichgarment?)
( hi h
t?)
id(whichmodel?)
money(moneyleft?)

Answer:
(money,g)or(g,money)

Recurrence(recursivebacktrackingfunction):
R
(
i b kt ki f ti )
shop(money, g)
if (money
y < 0) return -INF
if (g == C) return M money
return max(shop(money price[g][model], g + 1), model [1..K]

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearch?(2)
l t S
h? (2)
But,howtosolvethis?
M=200(maximum)
M 200 (
i
)

TimeComplexity:2020
Toomanyfor
3stimelimit

Model
Garment

19

32

12

55

53

50

14

22

11

99

19
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

O l
OverlappingSubProblemIssue
i S b P bl
I
Inthesimple2020 CompleteSearchsolution,
we observe many overlappingsubproblems!
weobservemany
overlapping sub problems!
Manywaystoreachstate(money,g),e.g.seebelow,M=12
Model
Garment

12, 0

6
6, 1

6
0 2
0,

8, 1

2 2
2,

2
0, 3

4 2
4,

3
1, 3

2
1
3, 3

2, 3

C 2
C=2

DP t th R
DPtotheRescue(1)
(1)
DP=DynamicProgramming
Programminghereisnotwritingcomputercode,
butatabularmethod!
a.k.a.table method

Aprogrammingparadigmthatyoumustknow!
A programming paradigm that you must know!
Andhopefully,master

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

DP t th R
DPtotheRescue(2)
(2)
UseDPwhentheproblemexhibits:
Optimalsubstructure
Optimalsolutiontotheoriginalproblemcontains
p
g
p
optimalsolutiontosubproblems
Thisissimilar astherequirementofGreedyalgorithm
Ifyoucanformulatecompletesearchrecurrences,youhavethis

Overlappingsubproblems
Overlapping sub problems
Numberofdistinctsubproblems areactuallysmall
Buttheyarerepeatedlycomputed
B t th
t dl
t d
Thisisdifferent fromDivideandConquer
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

DP S l ti
DPSolution
I l
Implementation(1)
t ti (1)
TherearetwowaystoimplementDP:
TopDown
BottomUp
Bottom Up

TopDown(Demo):
Recursionaspernormal+memoizationtable
It
Itisjustasimplechangefrombacktracking
is just a simple change from backtracking
(completesearch)solution!

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

T
TurnRecursionintoMemoization
R
i i t M
i ti
initializememotableinmainfunction(usememset)
return_valuerecursion(params/state) {
ifthisstateisalreadycalculated,
simplyreturntheresultfromthememotable
calculatetheresultusingrecursion(other_params/states)
save the result of this state in the memo table
savetheresultofthisstateinthememotable
returntheresult
}

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

D
DynamicProgramming(TopDown)
i P
i (T D
)
Forourexample:
shop(money, g)
if (money < 0) return -INF
if (g == C) return M money
if (memo[money][g] != -1) return memo[money][g];
return memo[money][g] = max(shop(money price[g][model], g + 1),
model [1..K]
[1 K]

Assimpleasthat
As simple as that

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

If O ti l S l ti ( )
IfOptimalSolution(s)areNeeded
N d d
CleversolutionforTopDownDP
(SeesolutionforBottomUpDPinExample2)

Forourexample:
For our example:
print_shop(money, g)
if (money < 0 || g == C) return
for each model [1..K]
if shop(money price[g][model], g + 1) == memo[money][g]
print "take model = " + model + " for garment g = " + g
print_shop(money price[g][model], g + 1)
break

Assimpleasthat
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

DP S l ti
DPSolution
I l
Implementation(2)
t ti (2)
Anotherway:BottomUp:
Prepareatablethathassizeequalstothenumber
ofdistinctstatesoftheproblem
Starttofillinthetablewithbasecasevalues
Getthetopologicalorder
G t th t
l i l d inwhichthetableisfilled
i hi h th t bl i fill d
Sometopologicalordersarenaturaland
canbewrittenwithjust(nested)loops!

DifferentwayofthinkingcomparedtoTopDownDP
y
g
p
p

NoticethatbothDPvariantsusetable!
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

D
DynamicProgramming(BottomUp)
i P
i (B tt
U )
Forourexample:
Startwithwith
St t ith ith tablecan_reach
t bl
h ofsize20(g)*201(money)
f i 20 ( ) * 201 (
)
Thestate(money,g)isreversedto(g,money)sothat
we can process bottomup DP loops in row major fashion
wecanprocessbottomupDPloopsinrowmajorfashion
Initializeallentriesto0(false)
Fillinthefirstrowwithmoneyleft(column)reachable
Fill in the first row with money left (column) reachable
afterbuyingmodelsfromthefirstgarment(g=0

Use
Usetheinformationofcurrentrowgtoupdatethevalues
the information of current row g to update the values
atthenextrowg+1

 BudgetM=20

Model
Garment

Answer:19

10

C=2

Alternative
Alternativeanswers
answers
arepossible

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

8
5

T D
TopDownorBottomUp?
B tt
U ?
TopDown

BottomUp

Pro:

Pro:

Naturaltransformation
fromnormalrecursion
Onlycomputesub
problems when necessary
problemswhennecessary

Cons:

Fasterifmanysub
problemsarevisited:
norecursivecalls!
Cansavememoryspace*
Can save memory space*

Cons:

Sloweriftherearemany
subproblemsdueto
recursive call overhead
recursivecalloverhead
UseexactlyO(states)
tablesize(MLE?)
(
)

Maybenotintuitivefor
thoseinclinedto
recursions?
IfthereareXstates,
bottomupvisits/fillsthe
p
/
valueofalltheseXstates
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

Flight Planner
FlightPlanner
(studythisonyourown)

EXAMPLE2
Click me to jump to the next section
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

M ti ti
Motivation
Howtosolvethis:10337 (FlightPlanner)?
Unit:1milealtitudeand
1(x100)milesdistance
Givenwindspeedmap
Fuelcost:{climb
F l
t { li b (+60),
( 60)
hold (+30),sink (+20)}
windspeedwsp[alt][dis]
Computemin
Compute min fuelcost
fuel cost
from(0,0)to(0,X=4)!

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearch?(1)
l t S
h? (1)
Firstguess:
Docompletesearch/bruteforce/backtracking
Findallpossible
Find all possible flightpathsand
flight paths and
picktheonethatyieldtheminimumfuelcost

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearch?(2)
l t S
h? (2)
RecurrenceoftheCompleteSearch
f
fuel(alt,
l( lt
min3(60
30
20

di
dis)
) =
- wsp[alt][dis] + fuel(alt + 1, dis + 1),
- wsp[alt][dis] + fuel(alt
, dis + 1),
- wsp[alt][dis] + fuel(alt - 1, dis + 1))

Stopwhenwereachfinalstate(basecase):
alt=0anddis=X,i.e.fuel(0,X)=0
Pruneinfeasiblestates(alsobasecases):
Prune infeasible states (also base cases):
alt<0oralt>9ordis>X!,i.e.returnINF*

Answeroftheproblemisfuel(0,0)
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearchSolutions(1)
l t S
h S l ti
(1)
Solution1

Solution2

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)
29+ 39+ 39+ 29=136

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)
29+ 39+ 69+ 19=156

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearchSolutions(2)
l t S
h S l ti
(2)
Solution3

Solution4

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)
29+ 69+ 11+ 29=138

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)
59+ 11+ 39+ 29=138

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearchSolutions(3)
l t S
h S l ti
(3)
Solution5

Solution6

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)
29+ 69+ 21+ 19=138

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)
59+ 21+ 11+ 29=120(OPT)

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearchSolutions(4)
l t S
h S l ti
(4)
Solution7

Solution8

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)
59+ 21+ 21+ 19=120(OPT)

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)
59+ 51+ 19+ 19=148

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

C
CompleteSearch?(3)
l t S
h? (3)
Howlargeisthesearchspace?
Maxdistanceis100,000miles
Eachdistancestepis100miles
Thatmeanswehave1,000 distancecolumns!
Note:thisisanexampleof
Note: this is an example of coordinate
coordinatecompression
compression

Branchingfactorperstepis3(climb,hold,sink)
Thatmeanscompletesearchcanendup
performing3
p
g 1,000 operations
p
Toomanyfor3stimelimit
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

O l
OverlappingSubProblemIssue
i S b P bl
I
Insimple31,000 CompleteSearchsolution,we
observemany
b
overlappingsubproblems!
l
i
b
bl
!
Manywaystoreachcoordinate(alt,dis)
y y
( , )
INF
2

INF

INF

INF

0
0

2
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

DP S l ti
DPSolution
Recurrence*oftheCompleteSearch
f
fuel(alt,
l( lt
min3(60
30
20

di
dis)
) =
- wsp[alt][dis] + fuel(alt + 1, dis + 1),
- wsp[alt][dis] + fuel(alt
, dis + 1),
- wsp[alt][dis] + fuel(alt - 1, dis + 1))

Subproblemfuel(alt,dis)canbeoverlapping!
p
( , )
pp g
Thereareonly10altand1,000dis=10,000 states
Alotoftimesavedifthesearenotrecomputed!
l
f
d f h
d!
Exponential31,000 topolynomial10*1,000!

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

alt > 2 not shown 2

40 19

29 0

DP S l ti (T D
DPSolution(TopDown)
)

Createa2Dtableofsize10*(X/100)

INF
INF

INF

INF

0
0

2
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

Save Space!

Set1forunexploredsubproblems(memset)
Storethecomputationvalueofsubproblem
Store the computation value of sub problem
Simplyreusewhen
it is needed again!
itisneededagain!

DP S l ti (B tt
DPSolution(BottomUp)
U )
fuel(alt,
min3(20
30
60

dis) =
- wsp[alt + 1][dis - 1] + fuel(alt + 1, dis - 1),
- wsp[alt
][dis - 1] + fuel(alt
, dis - 1),
- wsp[alt - 1][dis - 1] + fuel(alt - 1, dis - 1))

1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)

Tips:
(space-saving
trick)
We can reduce
one storage
dimension by
only keeping 2
recent columns
at a time
But the time
complexity is
unchanged:
( * X / 100))
O(10

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

59

29

110

59 80

29 68

110 131

59 80 101

29 68 91

110 131

59 80 101

29 68 91 120

If O ti l S l ti ( )
IfOptimalSolution(s)areNeeded
N d d
Althoughnotoften,sometimesthisisasked!
AswebuildtheDPtable,
record which option is taken in each cell!
recordwhichoptionistakenineachcell!
Usually,thisinformationisstoredindifferenttable
Then,dorecursivescan(s)tooutputsolution
Sometimes,therearemorethanonesolutions!
Sometimes, there are more than one solutions!

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

59 80 101

29 68 91 120

110 131

Sh t t P th P bl ? (1)
ShortestPathProblem?(1)
Hey,Ihavealternativesolution:
ModeltheproblemasaDAG
Vertexiseachpositionintheunitmap
Vertex is each position in the unit map
Edgesconnectverticesreachablefromvertex
( lt di ) i ( lt 1 di 1) ( lt di 1) ( lt 1 di )
(alt,dis),i.e.(alt+1,dis+1),(alt,dis+1),(alt1,dis)
Weightedaccordingtoflightactionandwindspeed!
Donotconnectinfeasiblevertices
alt<0oralt>9ordis>X

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

Vi li ti
VisualizationoftheDAG
f th DAG
1
1
1
1
| 9
1
1
1
1
| 8
1
1
1
1
| 7
1
1
1
1
| 6
1
1
1
1
| 5
1
1
1
1
| 4
1
1
1
1
| 3
1
1
1
1
| 2
1
9
9
1
| 1
1 -9 -9
1
| 0
========================
0
1
2
3
4 (x100)

21
59
29

S
Source

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

39

11
39

29

Whatt is
Wh
i the
th
shortest path
from source
to destination?

Sh t t P th P bl ? (2)
ShortestPathProblem?(2)
Theproblem:findtheshortestpath from
vertex(0,0)tovertex(0,X)onthisDAG
t (0 0) t
t (0 X)
thi DAG
O(V + E) solution exists!
O(V+E)solutionexists!
Visjust10*(X/100)
Eisjust3V
ThusthissolutionisasgoodastheDPsolution
Thus this solution is as good as the DP solution

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

B k
Break
Comingupnext,discussionofsomeClassicalDPs:

MaxSum(1Dfornow)
M
S
(1 D f
) Kadanes
K d
Algorithm
Al ith
LongestIncreasingSubsequence(LIS) O(nlogk)solution
01Knapsack/SubsetSum Knapsackstyleparameter!
CoinChange(theGeneralCase) skipped,seetextbook
TravelingSalesmanProblem(TSP) bitmaskagain:O

Iwilltrytocoverasmanyaspossible,butwillstop
at9pm;thedetailsareinChapter3ofCP2.9
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

LLetsdiscussseveralproblemsthataresolvableusingDP
t di
l
bl
th t
l bl
i DP
First,letsseesomeclassicalones

LEARNINGVIAEXAMPLES

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

M S
MaxSum(1D)
(1D)
Findacontiguous subarray in1DarrayAwiththemaxsum
i
0
[] 1
A[i]

1
2

2
6

3
3

4
2

5
6
12 6

7
7

Theansweris{6,3,2} withmaxsum6+3+2=11
Can we do this in O(n3)?
Can we do this in O(n2)?
Can we do this in O(n)?

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

8
1

L
LongestIncreasingSubsequence
tI
i S b
FindtheLongestIncreasingSubsequence (LIS)inarrayA
Subsequenceisnotnecessarilycontiguous
Subsequence is not necessarily contiguous

i
0
A[i] 7

1
10

2
9

3
2

4
3

5
8

Theansweris{7,2,3,8}withlength4
C we do
Can
d this
hi in
i O(n
O( 2)?
Can we do this in O(n log k)?

6
8

7
1

0 1K
01Knapsack/SubsetSum
k/S b tS
Red = 15 kg, $ 7
Blue = 8 kg
kg, $ 15

Can we do this in O(nS)?

n = # items
S = knapsack size

T
TravelingSalesmanProblem(TSP)
li S l
P bl
(TSP)

dist
0
1

n1

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

n1

T
TravelingSalesmanProblem(TSP)
li S l
P bl
(TSP)
State:tsp(pos,bitmask)
Transition:
Ifeverycitieshavebeenvisited
If every cities have been visited
tsp(pos,2N1)=dist[pos][0]

Else,tryvisitingunvisitedcitiesonebyone
tsp(pos,bitmask)=
p(p ,
)
min(dist[pos][nxt]+tsp(nxt,bitmask|(1<<nxt)))
nxt [0..N1],nxt !=pos,bitmask&(1<<nxt)==0

CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

Summary
Wehaveseen:
BasicDPconcepts
DPonsomeclassical
DP on some classical problems

WewillseemoreDPnextweek:
DPonnonclassical problems
DPanditsrelationshipwithDAG
DP and its relationship with DAG
DPonMath&StringProblems
SomeothercoolDP(optimization)techniques
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS

G dR f
GoodReferencesaboutDP
b t DP
CP2.9,obviously
Section3.5first
ThenSection4.7.1(DAG),5.4(Combinatorics),
(
),
(
),
6.5(String+DP),8.3(moreadvancedDP),partsofCh9

http://people.csail.mit.edu/bdean/6.046/dp/
http://people csail mit edu/bdean/6 046/dp/
CurrentUSACODirector

TopCoder AlgorithmTutorial
http://community.topcoder.com/tc?module=Static
h //
i
d
/ ? d l S i
&d1=tutorials&d2=dynProg
CS3233 CompetitiveProgramming,
StevenHalim,SoC,NUS