Scribd
Upload
2 views

Week 10 Tutorial

This tutorial covers linear programming (LP) as a subclass of convex optimization, focusing on the Karush-Kuhn-Tucker conditions and the relationship between primal and dual problems. It explains the concepts of duality in LP, providing examples such as the 'diet problem' to illustrate the application of primal and dual formulations. Additionally, it discusses solution possibilities for LP, including finite optimal, unbounded, and feasible solutions.

Uploaded by

lalitkumawat1510k
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

Week 10 Tutorial

This tutorial covers linear programming (LP) as a subclass of convex optimization, focusing on the Karush-Kuhn-Tucker conditions and the relationship between primal and dual problems. It explains the concepts of duality in LP, providing examples such as the 'diet problem' to illustrate the application of primal and dual formulations. Additionally, it discusses solution possibilities for LP, including finite optimal, unbounded, and feasible solutions.

Uploaded by

lalitkumawat1510k
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Tutorial

on
Linear Programming,
Karush-Kuhn-Tucker Conditions,
Relationship between primal and dual problem
Course: MACHINE LEARNING FOUNDATIONS

M s . E. A m r u t ha
( Tu t orial In st ru ctor )
Linear Programming (LP)
❑ Linear programming is a subclass of convex optimization
problem.
❑ Both the constraints and the objective function are linear
functions.
❑ It is about solving systems of linear inequalities.

Tutorial - Week 9
Example
❑ Consider the following linear program.

𝑥1 − 𝑥2 + 4 ≤ 0
𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒 3𝑥1 + 𝑥2 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 ቐ−3𝑥1 + 2𝑥2 + 10 ≤ 0
𝑥1 , 𝑥2 ≥ 0

Tutorial - Week 9
Karush-Kuhn-Tucker Conditions
𝑛 𝑚
❑ Stationarity ∇𝑓(𝑥) + ෍ 𝑢𝑖 ∇𝑔(𝑥) + ෍ 𝑣𝑗 ∇ℎ(𝑥) = 0
𝑖=1 𝑗=1

❑ Complementary slackness 𝑢𝑖 𝑔𝑖 = 0 ∀𝑖

❑ Primal feasibility 𝑔𝑖 x ≤ 0 ∀𝑖

❑ Dual feasibility 𝑢𝑖 ≥ 0 ∀𝑖

Tutorial - Week 9
𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒 3𝑥1 + 𝑥2 From 1

𝑥1 − 𝑥2 + 4 ≤ 0 Substituting in 2
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 ቐ−3𝑥1 + 2𝑥2 + 10 ≤ 0
𝑥1 , 𝑥2 ≥ 0
Substituting in 1
Stationarity conditions 1
From 3
2

Complementary slackness conditions


5
3
Substituting in 4

Primal feasibility conditions 6

Substituting 6 in 5
Minimum value of
objective function is
Dual feasibility conditions

Tutorial - Week 9
Types of solution possibilities for LP
𝑚𝑖𝑛 𝑥1 + 𝑥2 𝑚𝑎𝑥 3𝑥1 + 4𝑥2 𝑚𝑎𝑥 𝑥1 + 6𝑥2
𝑥1 + 2𝑥2 ≤ 8 𝑥1 + 𝑥2 ≥ 5 𝑥1 ≤ 200
𝑠. 𝑡 ቐ3𝑥1 + 2𝑥2 ≤ 12 𝑠. 𝑡 ቐ3𝑥1 + 𝑥2 ≥ 8 𝑥2 ≤ 300
𝑥1 + 3𝑥2 ≥ 13 𝑥1 , 𝑥2 ≥ 0 𝑠. 𝑡
𝑥1 +𝑥2 ≤ 400
𝑥1 , 𝑥2 ≥ 0
𝒙𝟐 𝒙𝟐 𝒙𝟐
𝒙𝟐 =300
𝒙𝟏 + 𝒙𝟐 = 13
𝒙𝟏 + 𝒙𝟐 = 400
𝒙𝟏 + 𝟐𝒙𝟐 = 8

𝒙𝟏 + 𝟓𝒙𝟐 = 5
𝒙𝟏 𝒙𝟏 𝒙𝟏
𝟑𝒙𝟏 + 𝟐𝒙𝟐 = 12 𝒙𝟏 = 200
𝟑𝒙𝟏 + 𝒙𝟐 = 8
‘Infeasible’ ‘Unbounded’ ‘Feasible’
Tutorial - Week 9
Duality
❑ In linear programming, duality implies that each linear
programming problem can be analyzed in two different
ways but would have equivalent solutions.
❑ For any linear program (LP), there is a closely related LP
called the dual.
❑ Duality relates to the inversion of a maximization problem
into a minimization problem, or vice-versa, through a
change of variables based on Lagrange Multipliers and / or
Karush-Kuhn Tucker (KKT) multipliers.

Tutorial - Week 9
Interpretation of the dual: ‘Diet problem’
A student wants to purchase a snack from a bakery to meet
certain dietary requirements by choosing the best
combination of brownies and cheesecake. The student is
following some new diet trend which requires her to eat at
least 6 oz of chocolate, 8 oz of cream cheese, and 10 oz of
sugar. The cost of 1 piece of brownie and one piece of cake is
50 cts and 80 cts respectively. Her goal is to satisfy these
requirements at minimal cost.

Tutorial - Week 9
Ingredients
needed
3 oz 2 oz 2 oz

0 oz 4 oz 5 oz

Requirements 6 oz 10 oz 8 oz

‘Primal problem’ ‘Dual problem’


𝑚𝑖𝑛 50𝑥1 + 80𝑥2 𝑚𝑎𝑥 6𝑦1 + 10𝑦2 + 8𝑦3
3𝑥1 + 0𝑥2 ≥ 6 3𝑦1 + 2𝑦2 + 2𝑦3 ≤ 50
2𝑥1 + 4𝑥2 ≥ 10
𝑠. 𝑡 𝑠. 𝑡 ቐ0𝑦1 + 4𝑦2 + 5𝑦3 ≤ 80
2𝑥1 + 5𝑥2 ≥ 8
𝑦1 , 𝑦2 , 𝑦3 ≥ 0
𝑥1 , 𝑥2 ≥ 0

Tutorial - Week 9
Weak and strong duality

Tutorial - Week 9
Primal/dual solution possibilities

Primal
Finite optimal Unbounded Feasible
Finite optimal Possible Impossible Impossible
Dual Unbounded Impossible Impossible Possible
Feasible Impossible Possible Possible

Tutorial - Week 9
Thank you…!

Tutorial - Week 9

You might also like