Creative Programming

Spring 2025
CUL1122 Lecture #12
Optimization Problems
Today

❖Computational Models
❖Optimization Models and their Solutions
▪ Brute Force Algorithm
▪ Greedy Algorithm
❖Knapsack Problem
▪ 0/1 Knapsack Problem vs. Fractional Knapsack Problem
▪ Solving the 0/1 Knapsack Problem: Using Brute Force and Greedy Algorithms
❖Exercise: Solving the 0/1 Knapsack Problem

3
Computational Models

❖Use computers to solve real-world problems,

▪ e.g., analyzing past events or predicting future outcomes (e.g., stock prices),
❖Can involve:
▪ 1) Optimization models
▪ 2) Statistical models
▪ 3) Simulation models

Images © source: https://twitter.com/MFHoz/status/1728850244404953457

4
Computational Models

❖Use computers to solve real-world problems,

▪ e.g., analyzing past events or predicting future outcomes (e.g., stock prices),
❖Can involve:
▪ 1) Optimization models
▪ 2) Statistical models
▪ 3) Simulation models

Images © source: https://twitter.com/MFHoz/status/1728850244404953457

5
Optimization Models

❖The goal is to find the optimal solution that satisfies given conditions.
❖You are given constraints and an objective function for optimal choices.
▪ Constraints must be strictly followed.
➢When going to Boston, arrive before 5 p.m. and spend $100 or less.
▪ The objective function should be maximized or minimized.
➢Minimizing travel costs from New York to Boston.
➢Minimizing travel time from New York to Boston.

Images © source: https://www.outofyourcomfortzone.net/the-best-travel-companies-and-resources/.

6
Strategy 1: Brute Force Algorithm

❖Determining the optimal solution by considering all possible cases.

❖Advantages:
▪ Simple and intuitive.
▪ Always guarantees the optimal solution.
❖Disadvantages:
▪ High time complexity, challenging for large-scale problems.
▪ Can be computationally expensive and inefficient.
❖Example: Finding the combination for a 5-digit lock.
▪ Try all numbers from 00000 to 99,999 until the lock is opened.
Images © source: https://studyalgorithms.com/theory/algorithmic-paradigms-brute-force/

7
Strategy 2: Greedy Algorithm
❖Selecting the ‘best’ option at each step to reach the optimal solution.
❖Advantages:
▪ Fast speed and simple implementation. ① ③ ②
▪ Can often find efficient solutions.
❖Disadvantages:
▪ No guarantee of global optimum;
may get stuck in local optima. ④
▪ Uncertain how good a local optimum is.
❖Example: Maximizing tutorials in 10 hours.
▪ Select tutorials with the shortest duration first.
Images © source: https://studyalgorithms.com/theory/algorithmic-paradigms-brute-force/

8
Brute Force Algorithm vs. Greedy Algorithm

❖Problem: Pack candies to maximize weight while staying within $25.

❖Brute Force Algorithm:
▪ Calculate the weight for every combination of candies
and select the combination with the smallest difference
from $25, without exceeding it.
❖Greedy Algorithm:
▪ At each step, pick the heaviest remaining candy and stop
once the weight exceeds $25.

9
Knapsack Problem: Motivation

❖Problem: A thief who has broken into someone’s house wants to pack
items, but the bag’s size is a concern.
❖Solution: Choose items with the highest value possible to pack.

10
Knapsack Problem: Definition

❖All necessary items must fit into one knapsack.

❖Each item has the following characteristics:
▪ 1) Weight (e.g., 12kg)
▪ 2) Value (e.g., $4)
❖The maximum weight the knapsack can hold is 𝑤.
▪ If the total weight is at most 𝑤, pack all items.
▪ If the total weight exceeds 𝑤, some items must be removed from the knapsack.
❖When the weight limit is exceeded, which items to pack and which to
remove?

11
Knapsack Problem: Objective

❖Determine the items to pack in the knapsack while satisfying the

following two conditions:
▪ 1) The total weight of the packed items must not exceed the knapsack capacity.
▪ 2) The total value of the packed items must be maximized.
❖Example: Given a choice, prefer an item <$2, 1kg> over an item <$2,
2kg>.

12
Knapsack Problem: Types

❖1) 0/1 Knapsack Problem

▪ Only two options exist for each item: either include the entire item or exclude it.
➢Example: Take the gold bar or leave it; no partial selections.
▪ Does not guarantee an optimal solution.
❖2) Fractional Knapsack Problem
▪ Partial items can be included.
➢Example: Fill the remaining bag space with gold dust.
▪ Since the maximum capacity can be precisely met,
an optimal solution is guaranteed.

13
0/1 Knapsack Problem: Design

❖A vector, I, of length n represents the list of all items.

▪ Each vector element represents an item as <value, weight>.
I 0 1 2 3 4
value ($) 4 2 2 1 10
weight (kg) 12 2 1 1 4

❖A vector, V, of length n indicates whether

each item is packed in the knapsack.
▪ If only <$4, 12kg> item I[0] is packed, then only V[0] is 1.
V 0 1 2 3 4
1 0 0 0 0
❖Items can be packed into the knapsack up to its capacity, 𝑤.

14
0/1 Knapsack Problem: Objective

❖The goal is to determine vector V that satisfies the following conditions:

Since I[i].value represents the value of the item and V[i]
indicates if the item is packed,
1) If V[i] = 1, i.e., if the item is packed, the knapsack value
increases by the item's value.
2) If V[i] = 0, i.e., if the item is not packed, there is no
change in the knapsack value.

However, constraints must always be obeyed, so adding

the weight of the item (I[i].weight) must not exceed the
knapsack capacity, 𝑤.

15
Strategy 1: Brute Force Algorithm

❖Step 1) For the given items, explore all combinations of items.

▪ That is, generate all subsets (Power Set) for n items.
▪ For example, if n = 3, there are 23 = 8 possible combinations:
➢{{0}, {1}, {2}, {0, 1}, {0, 2}, {1, 2}, {0, 1, 2}}

❖Step 2) For each combination, exclude it from consideration if the total

weight exceeds 𝑤.
❖Step 3) From the remaining candidates, select the combination with the
maximum value.

16
Strategy 1: An Often Unrealistic Approach

❖Each element of vector V indicates whether the corresponding item is

packed in the knapsack.
▪ If V[i] = 1, item I[i] is packed in the knapsack.
▪ If V[i] = 0, item I[i] is not packed in the knapsack.
❖All item combinations are represented by the power set P(V) of V.
▪ If n = 5, there are a total of 32 possible combinations of items.
❖How large will the size of P(V) be as n increases?
▪ The time complexity |P(V)| = O(2n) means it becomes impractical as n grows!
▪ For n = 100, |P(V)| = 2100 = 1,267,650,600,228,229,401,496,703,205,376.

17
Strategy 2: Greedy Algorithm
❖If there’s still room in the knapsack,
▪ Let’s pack the ‘best’ item at the moment!
❖What does ‘best’ mean?
▪ Among the unpacked items:
▪ 1) By value: Choose the item with the
highest value within capacity.
▪ 2) By cost: Choose the item with the
lowest weight within capacity.
▪ 3) By value-to-weight ratio: Choose the
item with the highest value-to-weight
ratio within capacity. An example of selecting based on
value-to-weight ratio.
18
Strategy 2: A Practical Approach

❖The greedy algorithm always selects the ‘best’ option at the current
moment.
❖This approach allows us to find a local optimum, and it’s hoped that the
local optimum will be the global optimum.
▪ A local optimum doesn’t necessarily match the global optimum.
▪ However, it can still be a sufficiently good solution even if it’s not the global
optimum.
global optimal
local optimal local optimal

Greedy: move towards max gradient and hope it is global optimal 19

Designing Greedy Algorithm
❖Given n items,
▪ Find one solution that satisfies the constraints.
▪ If the objective function becomes maximum or minimum, then the current solution
found is optimal.
❖Specifically,
▪ Step 1) According to the criterion of ‘best’, select one item from the remaining items.
➢ End if no items are left.
▪ Step 2) Check if including the selected item satisfies the constraints.
➢ If not, exclude the selected item from the candidates and go back to Step 1).
▪ Step 3) Include the selected item in the solution and update the objective function.
➢ Go back to Step 1).
20
Exercises: 0/1 Knapsack Problem
Exercise 1: Coin Counter at the Laundromat (Assignment 4)

❖At a laundromat, you need to collect the exact amount of coins needed
to pay for the washing and drying machines.
❖Create a Python script that calculates the minimum number of coins
needed to reach a specific amount.

22
Exercise 1: Implementation

❖1) Generate a list of coins and their names based on the amount.
▪ coins = [25, 10, 5, 1]
❖2) Also, generate a list of the corresponding coin names.
▪ names = [‘Quarters’, ‘Dimes’, ‘Nickels’, ‘Pennies’]
❖3) Traverse the sorted list and if there is still an amount to fill,
▪ Calculate how many coins of the current amount are needed.
▪ Subtract the amount that can be paid with the current coin from the remaining
amount.
▪ For example, if it’s 98 cents, you’ll need 3 quarters, and the remainder will be
23 cents.
23
Exercise 2: Choosing Survival Items

❖Create a script to select survival items within the bag’s capacity.

▪ The ‘best’ criteria could be ‘value-to-weight ratio’.
Item Water Food First Aid Kit Flashlight Knife Clothing Navigation Blanket Fire
Value 10 6 7 3 4.5 4 5 2.5 8
Weight(kg) 3 5 3 1 2 2 3 2 4

24
Exercise 2: 1) Writing the Item Class

❖Create an Item class to store information about survival items.

class Item(object):
def __init__(self, n, v, w):
self.name = n
self.value = v
self.weight = w
def getValue(self):
pass
def getWeight(self):
pass
def getRatio(self):
pass
25
Exercise 2: 2) Creating a List of Survival Items

❖Create an Item object for each survival item and add it to the list.
def build_list(names, values, weights):
l = []
for i in range(len(names)):
l.append(Item(names[i], values[i], weights[i]))
return l

26
Exercise 2: 3) Defining and Calling Greedy Function

❖Create a function that performs the greedy algorithm.

▪ Sort the item list in descending order of the ‘value-to-weight ratio’.
Item Water Flashlight First Aid Kit Knife Clothing Fire Navigation Blanket Food
Value 10 3 7 4.5 4 8 5 2.5 6
Weight(kg) 3 1 3 2 2 4 3 2 5
Ratio 3.3 3.0 2.3 2.2 2.0 2.0 1.7 1.2 1.2

27
Exercise 2: 3) Defining and Calling Greedy Function
▪ While traversing the sorted list, if there is remaining capacity,
➢Add current item to the list of packed items, increase total value and weight.

def greedy(items, capacity):

itemsCopy = sorted(items, key = Item.getRatio, reverse = True)
…
for i in range(len(itemsCopy)):
if (totalValue+itemsCopy[i].getWeight()) <= capacity:
packed_items.append(itemsCopy[i])
totalValue += itemsCopy[i].getValue()
totalWeight += itemsCopy[i].getWeight()
return (packed_items, totalValue, totalWeight)

28
수고하셨습니다!

29