IP Quality

IE 8030
Prof. Emily Tucker
Nov. 14, 2024
 What’d we cover last time?
• Branch-and-bound
 Today
• Branch-and-Bound
IP “Quality”
Spoiler Alert
• General idea: we want the LP relaxation to be close to
the IP
– “High quality”

• Why?
– We use LP relaxations to iteratively solve for IPs
– The closer the LP is, the fewer iterations you need
IP Quality
Motivating Example
• Matthew's Lego Workshop company will be opening a
production facility
– It has enough room to install exactly one of two production lines.

• Costs
– Production line 1: $10,000
– Production line 2: $8,000.

• Time to produce T or cake

Lego Production
• Decision variables: 3𝑥1 + 2𝑥2
– Ts (𝑥1 ) and cakes (𝑥2 ) produced max
−10000𝑦1 − 8000(1 − 𝑦1 )
– If you select Line 1 (𝑦1 )

• Constraints
– Production on line 1 max 3𝑥1 + 2𝑥2 − 2000𝑦1
– Production on line 2
– Inventory capacity 0.05𝑥1 + 0.033𝑥2 ≤ 40 + 𝑀(1 − 𝑦1 )
– Domain
0.025𝑥1 + 0.1𝑥2 ≤ 40 + 𝑀𝑦1
• Objective 40𝑥1 + 25𝑥2 ≤ 15000
– Maximize profit (revenues-costs)
𝑥1 , 𝑥2 ≥ 0
𝑦 ∈ 0,1
 How 𝑴 affects the LP relaxation
• Quality of LP relaxation
– % away it is from the IP value
LP Obj Value − IP Obj Value
IP Obj Value
• Optimal IP value:
– 1170.37

• LP relaxation with 1196 − 1170.37

= 2.1%
1170.37
– 𝑀 = 10000: 1196
– 𝑀 = 1000: 1170.37 vs. 0% with smaller M
Size of Feasible Region
 Tightness of LP Relaxation
• Say we have two LP relaxations - LP1 and LP2.
– We say LP1 is tighter than LP2 if we can show that:
• 𝑍 𝐿𝑃1 ≤ 𝑍 𝐿𝑃2
– The objective function value of LP1 is always less than or equal to the optimal objective function
value to LP2.
• 𝑍 𝐿𝑃1 < 𝑍 𝐿𝑃2
– AND there is at least one situation where it is strictly less than.

• We can typically do this in two steps:

– (1) We take any feasible solution to LP1 and convert it to any feasible
solution to LP2 with an objective greater than or equal to its objective in
LP1.
– (2) Find an example where the strict occurs.
 p-Median
• Consider a p-median facility
location problem

• Recall
– 𝑦𝑖 indicates if location 𝑖 ∈ 𝐼 is
selected
– 𝑥𝑖𝑗 indicates if population center
𝑗 ∈ 𝐽 is allocated to EMT location
𝑖∈𝐼
Alternate formulation
• Rather than:
– 𝑥𝑖𝑗 ≤ 𝑦𝑖 , ∀𝑖 ∈ 𝐼, 𝑗 ∈ 𝐽

• Could write:
– σ𝑗∈𝐽 𝑥𝑖𝑗 ≤ 𝑀𝑦𝑖 , ∀𝑖 ∈ 𝐼

Fewer constraints! Why wouldn’t we do this?

 Which is better?
Disaggregated Aggregated

𝑥𝑖𝑗 ≤ 𝑦𝑖 , ∀𝑖 ∈ 𝐼, 𝑗 ∈ 𝐽 ෍ 𝑥𝑖𝑗 ≤ 𝑀𝑦𝑖 , ∀𝑖 ∈ 𝐼

𝑗∈𝐽

It has to do with the LP relaxations

But the disaggregated
They have the same IP version is tighter than the
feasible region aggregated

This means there are LP solutions that

are feasible to aggregated that aren’t
feasible to the disaggregated
 Which is better?
• Example A 1

– Sets: 𝐿 = 1,2 ; 𝐷 = 𝐴, 𝐵
B 2
– Variables: 𝑥1𝐴 , 𝑥1𝐵 , 𝑥2𝐴 , 𝑥2𝐵 , 𝑦1 , 𝑦2
– Constraint options:
Disaggregated Aggregated

𝑥1𝐴 ≤ 𝑦1
𝑥1𝐵 ≤ 𝑦1 𝑥1𝐴 + 𝑥1𝐵 ≤ 2𝑦1 Zoom in
𝑥2𝐴 ≤ 𝑦2 𝑥2𝐴 + 𝑥2𝐵 ≤ 2𝑦2
𝑥2𝐵 ≤ 𝑦2
 Zoom in:
Disaggregated Aggregated

𝑥1𝐴 ≤ 𝑦1
𝑥1𝐴 + 𝑥1𝐵 ≤ 2𝑦1
𝑥1𝐵 ≤ 𝑦1

IP feasible region: same for both

LP feasible region: aggregated is larger

E.g., 𝑥1𝐴 , 𝑥1𝐵 , 𝑦1 = 0.6,0,0.4 Is feasible for aggregated

but not for disaggregated

That is, when 𝑥1𝐵 = 0, 𝑥1𝐴 ≤ 2𝑦1 if feasible!

Etc.
 In practice
• The disaggregated p-median
generally performs better
– Because the LP relaxations are
tighter
– Even though there are more
constraints
Valid Inequalities
 Valid Inequalities
• Definition
– Given an integer program, a valid
inequality is an inequality that is
one that does not remove any Feasible solutions to the LP

feasible integer solutions.

with valid inequalities

• Idea
– Cutting off feasible continuous
solutions
Example: Fixed Costs
• Decision variables
– 𝑦𝐵 = 1 if we install production line for product 𝐵
– 𝑥𝐵 : the amount of product 𝐵 we produce

• Constraint
– 𝑥𝐵 ≤ 𝑀𝑦𝐵
– Any valid upper bound on production of B can be used for 𝑀
Options for 𝑴 Anytime we tighten
𝑀, we are identifying
• Could say 𝑀 = 1,000,000 a valid inequality

• Looking at the other constraints

– Say another constraint is: 3𝑥𝐵 + 2𝑥𝐶 ≤ 20,
• Then 𝑀 = 6 is valid

• Why?
– Once 𝑦𝐵 = 1, then we haven’t cut out any feasible 𝑥𝐵 values
– But we have cut out a lot of feasible solutions in the LP relaxation
𝑥𝐵 𝑥
• Any solution with 𝑥𝐵 and 𝑦𝐵 ∈ , 𝐵
1,000,000 6
Example: Logical Constraints
• Decision variables
– 𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 binary
– 𝑥5 ≥ 0

• Constraint in the IP:

– 𝑥1 − 8𝑥2 − 10𝑥3 + 𝑥4 + 𝑥5 ≤ −2

• What happens if 𝑥2 = 𝑥3 = 0?
– Constraint can’t be satisfied
– Logically, at least one of 𝑥2 or 𝑥3 needs to be positive

• This ends up being a valid inequality

– 𝑥2 + 𝑥3 ≥ 1
 Integer + Integer = Integer
• Simple example
– Let 𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 ≥ 0, integer

– Initial inequality
• 𝑥1 + 𝑥2 + 𝑥3 + 𝑥4 ≤ 10.5

– Valid inequality
• 𝑥1 + 𝑥2 + 𝑥3 + 𝑥4 ≤ 10 = 10.5
 Extending the logic
• More complex example
– Let 𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 ≥ 0, integer

• Initial constraint
– 20𝑥1 + 13𝑥3 + 9𝑥3 + 10𝑥4 ≤ 42

• Find a valid inequality…

– Divide both sides by 10
• 2𝑥1 + 1.3𝑥2 + 0.9𝑥3 + 𝑥4 ≤ 4.2
– Take the floor of LHS coefficients
• 2𝑥1 + 𝑥2 + 0𝑥3 + 𝑥4 ≤ 4.2
– Since LHS will be integer, we can round down the RHS
• 2𝑥1 + 𝑥2 + 0𝑥3 + 𝑥4 ≤ 4

• This is a valid inequality and cuts out the solution (2.1, 0, 0, 0)

 Constraint-Based Valid Inequalities

• Let’s consider a previous example

– Used for the initial branch-and-bound
LP vs. IP Feasible Regions
LP corner point:
(3.5, 1.5)

We want to cut it out

while keeping (3,2)
and (4,0) feasible
LP vs. IP Feasible Regions
Consider 2𝑥1 + 𝑥2 ≤ 8

Note (3,2) and (4,0) are on

the line 2𝑥1 + 𝑥2 = 8

They become
extreme points of the
problem with this
constraint added!
 The IP with the VI added
Because (4,0) is an extreme point to the feasible region, when we solve the
linear relaxation – we’ll get the integer optimal solution

max 5𝑥1 + 2𝑥2

s.t. 3𝑥1 + 𝑥2 ≤ 12
𝑥1 + 𝑥2 ≤ 5
2𝑥1 + 𝑥2 ≤ 8
𝑥1 , 𝑥2 ≥ 0, integer

A remaining question though – how

do we come up with 2𝑥1 + 𝑥2 ≤ 8?
The IP we’re looking to solve

max 5𝑥1 + 2𝑥2

3𝑥1 + 𝑥2 ≤ 12
𝑥1 + 𝑥2 ≤ 5
𝑥1 , 𝑥2 ≥ 0, integer

How?
Linear relaxation
Solve LP relaxation via Simplex
 Solving the Relaxation

Linear Relaxation Convert to Standard Form

max 5𝑥1 + 2𝑥2 max 5𝑥1 + 2𝑥2

3𝑥1 + 𝑥2 ≤ 12 3𝑥1 + 𝑥2 + 𝑥3 = 12
𝑥1 + 𝑥2 ≤ 5 𝑥1 + 𝑥2 + 𝑥4 = 5
𝑥1 , 𝑥2 ≥ 0 𝑥1 , 𝑥2 𝑥3 , 𝑥4 ≥ 0
 If solved via Simplex
Final Tableau

The two constraint equations

𝑥1 + 0.5𝑥3 − 0.5𝑥4 = 3.5
𝑥2 + 0.5𝑥3 + 1.5𝑥4 = 1.5
 Generating a Valid Inequality
Consider the first equation 2 coefficients for each variable
(1) integer value (floor)
𝑥1 + 0.5𝑥3 − 0.5𝑥4 = 3.5 (2) the decimal value

“Pull” integers out of each coefficient

1𝑥1 + 0𝑥1 + 0𝑥3 + 0.5𝑥3 − 1𝑥4 + 0.5𝑥4 = 3 + 0.5

Integer pieces to the LHS; decimal pieces to the RHS

𝑥1 − 𝑥4 − 3 = 0.5 − 0.5𝑥3 + 0.5𝑥4

Valid inequality (in 4D):

Known: LHS must be integer
0.5 − 0.5𝑥3 + 0.5𝑥4 ≤ 0
Implies: RHS ≤ 0
 Valid Inequality in 2D
4D Valid inequality 0.5 − 0.5𝑥3 + 0.5𝑥4 ≤ 0

Original constraints 3𝑥1 + 𝑥2 + 𝑥3 = 12

𝑥1 + 𝑥2 + 𝑥4 = 5

Substitute 𝒙𝟑 and 𝒙𝟒 into 4D VI

0.5 − 0.5 12 − 3𝑥1 − 𝑥2 + 0.5 5 − 𝑥1 − 𝑥2 ≤ 0
Simplify

2𝑥1 + 𝑥2 ≤ 8 Valid inequality (in 2D)!

LP vs. IP Feasible Regions
LP corner point:
(3.5, 1.5)

We want to cut it out

while keeping (3,2)
and (4,0) feasible
LP vs. IP Feasible Regions
Add valid inequality:
2𝑥1 + 𝑥2 ≤ 8

Note (3,2) and (4,0) are on

the line 2𝑥1 + 𝑥2 = 8

They become
extreme points of the
problem with this
constraint added!
 Can we generate other VIs?
Consider the second equation
𝑥2 − 0.5𝑥3 + 1.5𝑥4 = 1.5

“Pull” integers out of each coefficient

𝑥2 + 0𝑥2 − 1𝑥3 + 0.5𝑥3 + 1𝑥4 + 0.5𝑥4 = 1 + 0.5

Integer pieces to the LHS; decimal pieces to the RHS

𝑥2 − 𝑥3 + 𝑥4 − 1 = 0.5 − 0.5𝑥3 − 0.5𝑥4
Same as last time
Valid inequality (in 4D):
Known: LHS must be integer
Implies: RHS ≤ 0 0.5 − 0.5𝑥3 + 0.5𝑥4 ≤ 0
 Standard Form IP
max 𝑐 𝑇 𝑥
s.t. 𝑛

෍ 𝑎𝑖𝑗 𝑥𝑗 ≤ 𝑏𝑖 , for 𝑖 = 1, … , 𝑚
𝑗=1
𝑥𝑗 ≥ 0, integer, for 𝑗 = 1, … , 𝑛

LP relaxation
If you solve the relaxation, you’ll end up with a final BFS
𝑎ത𝑖𝑗 is the coefficient in the 𝑖-th row of the BFS
𝑏ത𝑖 RHS
𝑛

෍ 𝑎ത𝑖𝑗 𝑥𝑗 = 𝑏ത𝑖
Write the rearranged equality:
𝑗=1
 Generic Approach to Find VI
𝑛

One of the constraints ෍ 𝑎ത𝑖𝑗 𝑥𝑗 = 𝑏ത𝑖

𝑗=1

Pull integers out of coefficients

𝑛 𝑛

෍ 𝑎ത𝑖𝑗 𝑥𝑗 + ෍ 𝑎ത𝑖𝑗 − 𝑎ത𝑖𝑗 𝑥𝑖𝑗 = 𝑏ത𝑖 + 𝑏ത𝑖 − 𝑏ത𝑖

𝑗=1 𝑗=1

Integer pieces to the LHS; decimal pieces to the RHS

𝑛 𝑛

෍ 𝑎ത𝑖𝑗 𝑥𝑗 − 𝑏ത𝑖 = 𝑏ത𝑖 − 𝑏ത𝑖 − ෍ 𝑎ത𝑖𝑗 − 𝑎ത𝑖𝑗 𝑥𝑖𝑗

𝑗=1 𝑗=1
 Generic Approach to Find VI
𝑛 𝑛

෍ 𝑎ത𝑖𝑗 𝑥𝑗 − 𝑏ത𝑖 = 𝑏ത𝑖 − 𝑏ത𝑖 − ෍ 𝑎ത𝑖𝑗 − 𝑎ത𝑖𝑗 𝑥𝑖𝑗

𝑗=1 𝑗=1

Known: LHS must be integer

Implies: RHS ≤ 0
𝑛

𝑏ത𝑖 − 𝑏ത𝑖 − ෍ 𝑎ത𝑖𝑗 − 𝑎ത𝑖𝑗 𝑥𝑖𝑗 ≤ 0

𝑗=1
 Example: Find a Valid Ineq.
One of the constraints from the optimal LP relaxation
𝑥1 + 4.25𝑥2 − 3.5𝑥3 + 5.3𝑥4 = 10.5

“Pull” integers out of each coefficient

𝑥1 + 4𝑥2 + 0.25𝑥2 − 4𝑥3 + 0.5𝑥3 + 5𝑥4 + 0.3𝑥4 = 10 + 0.5

Integer pieces to the LHS; decimal pieces to the RHS

𝑥1 + 4𝑥2 − 4𝑥3 + 5𝑥4 − 10 = 0.5 − 0.25𝑥2 − 0.5𝑥3 − 0.3𝑥4

Known: LHS must be integer

Implies: RHS ≤ 0 0.5 − 0.25𝑥2 − 0.5𝑥3 − 0.3𝑥4 ≤ 0
When is this used?
 When to use
• This is one method of generating cutting planes
– There are others!

• When to use?
– Embedded within IP algos
– Branch and bound is great but can be slow
• Adding cutting planes can speed it up
 Branch-and-Cut

Mitchell, 1999
 Within Gurobi,
E.g., Branch and Cut

https://www.gurobi.com/pdfs/user-events/2016-frankfurt/Die-Algorithmen.pdf
 Can display solver output

# Solve
solver = SolverFactory('gurobi')
Within your Pyomo code, solver.solve(model)
can add the bolded line: results = solver.solve(model, tree=True)
model.pprint()
print("Obj value: " + str(value(model.obj)))
 Solver Output

Background
on
license/your
machine

Coef. info

Pre-solve

LP relaxation
of the root
node
 Simple problem so only
Solver Output
uses root node
(doesn’t branch)
Nodes its
exploring

Cutting planes (what

type and how many)

Bounds on obj. value

and optimality gap

Everything here and

below we’ve seen
before
 Other Algorithms
• Branch-and-price
• Branch-price-and-cut
• Column generation

• Etc.
 Appendix
(Additional Detail on Tightness of LP Relaxations)
Fixed-Cost Example
• You run a company that is interested in launching a new
product: the exciting Product B.
– The cost of installing a production line to produce Product B is
$1000. You cannot produce Product B without installing this
production line.

– You have yet to finalize the projected demand (𝑑𝐵 ) for Product B.

– The company is interested in maximizing profits.

• Let's form a base IP model for this situation.

Base IP
• Decision variables
– Binary variable: (𝑦𝐵 ) producing
Product B.
– Continuous variable: 𝑥𝐵 (amount
of Product B to be produced)

• Constraint
– Can only produce if we install 𝑥𝐵 ≤ 𝑑𝐵 𝑦𝐵
production line

• Objective
– Max profits max [Revenues] – [costs] − 1000𝑦𝐵
Examine the LP Relaxation
• When you relax integrality
– Constraint: 𝑥𝐵 ≤ 𝑑𝐵 𝑦𝐵
• Still in the model, but the value of 𝑦𝐵 has a different domain
– 𝑦𝐵 ∈ 0,1 ⇒ 𝑦𝐵 ≥ 0, 𝑦𝐵 ≤ 1

• If 𝑥𝐵 = 100 in the LP relaxation, what will 𝑦𝐵 be equal to?

𝑥𝐵 100
– 𝑦𝐵 = =
𝑑𝐵 𝑑𝐵
– ⇒ 𝑦𝐵 always fractional (unless 𝑥𝐵 = 𝑑𝐵 )

• If you don’t know 𝑑𝐵 and use a Big-M approach,

– 𝑦𝐵 won’t be binary unless you branch on it
– If 𝑥𝐵 is small compared to M,
• The quality of the relaxation will be poor
 Tighten it up
• An approach to tighten M:
– Use other constraints to right-size M

20
– E.g., if 3𝑥𝐵 + 2𝑥𝐶 ≤ 20, then we won’t produce more than
3
units of product B
 Comparing LP Relaxations
• LP1: 𝑥𝐵 ≤ 6𝑦𝐵
• LP2: 𝑥𝐵 ≤ 100𝑦𝐵

• Suppose that (𝑥𝐵 , 𝑦𝐵 ) satisfy the first constraint

– 𝑥𝐵 ≤ 6𝑦𝐵 ≤ 100𝑦𝐵
– So they’re feasible to the 2nd constraint

• So we know the solution has the same objective in LP2

• Now suppose that we know in the LP1 relaxation that 𝑥𝐵∗ = 5, then
5
𝑦𝐵∗ =
6

5
• But in LP2, we can set 𝑦𝐵∗ =
100
– This means the objective increases (because costs decrease)
– So LP1 is “tighter” than LP2