0% found this document useful (0 votes)
27 views28 pages

Basics On Linear Programming: Combinatorial Problem Solving (CPS)

The document provides an overview of linear programming problems including: - Linear programs are optimization problems that minimize a cost function subject to constraints. - They can be expressed in canonical form with variables, objective function, and constraints. - Methods for solving linear programs include the simplex algorithm and interior-point algorithms. - Key definitions include feasible and optimal solutions, and whether a problem is bounded or unbounded. - The concept of bases is introduced, where a basis defines the basic and non-basic variables.

Uploaded by

erodricarb
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
27 views28 pages

Basics On Linear Programming: Combinatorial Problem Solving (CPS)

The document provides an overview of linear programming problems including: - Linear programs are optimization problems that minimize a cost function subject to constraints. - They can be expressed in canonical form with variables, objective function, and constraints. - Methods for solving linear programs include the simplex algorithm and interior-point algorithms. - Key definitions include feasible and optimal solutions, and whether a problem is bounded or unbounded. - The concept of bases is introduced, where a basis defines the basic and non-basic variables.

Uploaded by

erodricarb
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

Basics on Linear Programming

Combinatorial Problem Solving (CPS)

Javier Larrosa Albert Oliveras Enric Rodrı́guez-Carbonell

March 24, 2023


Linear Programs (LP’s)
■ A linear program is an optimization problem of the form

min cT x
A1 x ≤ b1
A2 x = b2
A3 x ≥ b3
x ∈ Rn

c ∈ Rn , bi ∈ Rmi , Ai ∈ Rmi ×n , i = 1, 2, 3
■ x is the vector of variables
■ cT x is the cost or objective function
■ A1 x ≤ b1 , A2 x = b2 and A3 x ≥ b3 are the constraints

2 / 22
Linear Programs (LP’s)
■ A linear program is an optimization problem of the form

min cT x
A1 x ≤ b1
A2 x = b2
A3 x ≥ b3
x ∈ Rn

c ∈ Rn , bi ∈ Rmi , Ai ∈ Rmi ×n , i = 1, 2, 3
■ x is the vector of variables
■ cT x is the cost or objective function
■ A1 x ≤ b1 , A2 x = b2 and A3 x ≥ b3 are the constraints
■ Example:
min x + y + z
x+y =3
0≤x≤2
0≤y≤2 2 / 22
Notes on the Definition of LP
■ Solving minimization or maximization is equivalent:

max{ f (x) | x ∈ S } = − min{ −f (x) | x ∈ S }

■ Satisfiability problems are a particular case:


take arbitrary cost function, e.g., c = 0

3 / 22
Equivalent Forms of LP’s
■ This form is not the most convenient for algorithms

■ We will assume problems to be in canonical form:


min cT x
Ax = b
x≥0

c ∈ Rn , b ∈ Rm , A ∈ Rm×n , n ≥ m, rank(A) = m

■ Often variables are identified with columns of the matrix,


and constraints are identified with rows

4 / 22
Methods for Solving LP’s
■ Simplex algorithms
■ Interior-point algorithms

5 / 22
Methods for Solving LP’s
■ Simplex algorithms
■ Interior-point algorithms

5 / 22
Basic Definitions (1)
min cT x
Ax = b
x≥0

■ Any vector x such that Ax = b is called a solution


■ A solution x satisfying x ≥ 0 is called a feasible solution
■ An LP with feasible solutions is called feasible;
otherwise it is called infeasible
■ A feasible solution x∗ is called optimal
if cT x∗ ≤ cT x for all feasible solution x
■ A feasible LP with no optimal solution is unbounded

6 / 22
Basic Definitions (2)
max x + 2y
x + y + s1 = 3
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 ≥ 0

■ (x, y, s1 , s2 , s3 ) = (−1, −1, 5, 3, 3) is a solution but not feasible


■ (x, y, s1 , s2 , s3 ) = (1, 1, 1, 1, 1) is a feasible solution

7 / 22
Basic Definitions (3)
max x + βy
x + y + s1 = α
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 ≥ 0

■ If α = −1 the LP is not feasible


■ If α = 3, β = 2 then
(x, y, s1 , s2 , s3 ) = (1, 2, 0, 1, 0) is the (only) optimal solution

8 / 22
Basic Definitions (3)
max x + βy
x + y + s1 = α
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 ≥ 0

■ If α = −1 the LP is not feasible


■ If α = 3, β = 2 then
(x, y, s1 , s2 , s3 ) = (1, 2, 0, 1, 0) is the (only) optimal solution
■ There may be more than one optimal solution:
If α = 3 and β = 1 then
{(1, 2, 0, 1, 0), (2, 1, 0, 0, 1), ( 32 , 32 , 0, 21 , 12 )} are optimal

8 / 22
Basic Definitions (4)
y

max x + 2y

y≤2 max x + y

(1, 2) x+y ≤3

(2, 1)
x≥0 x≤2

y≥0 x

9 / 22
Basic Definitions (5)
y

y≤2
min x + 2y x+y ≤3
x+y ≤3
0≤x≤2
y≤2 x≥0 x≤2

x
Unbounded LP
min x + 2y

10 / 22
Basic Definitions (6)
y

max x + 2y

max x + 2y
x+y ≤3 y≤2
0≤x≤2 x+y ≤3
(1, 2)
y≤2

LP is bounded, x≥0 x≤2


but set of feasible
x
solutions is not

11 / 22
Bases (1)
Let us denote by a1 , ..., an the columns of A
Recall that n ≥ m, rank(A) = m.

■ A matrix of m columns (ak1 , ..., akm ) is a basis


if the columns are linearly independent
■ Note that a basis is a square matrix!
■ If (ak1 , ..., akm ) is a basis,
then the variables (xk1 , ..., xkm ) are called basic
■ We usually denote
by B the list of indices (k1 , ..., km ), and
by R the list of indices (1, 2, ..., n) − B; and
by B the matrix (ai | i ∈ B), and
by R the matrix (ai | i ∈ R)
xB the basic variables, xR the non-basic ones

12 / 22
Bases (2)
max x + 2y
x + y + s1 = 3 x y s1 s2 s3
x + s2 = 2
 
1 1 1 0 0
y + s3 = 2 A= 1 0 0 1 0 
x, y, s1 , s2 , s3 ≥ 0 0 1 0 0 1

■ (x, s1 , s2 ) do not form a basis:


 
1 1 0
 1 0 1  does not have linearly independent columns
0 0 0

■ (s1 , s2 , s3 ) form a basis, where xB = (s1 , s2 , s3 ), xR = (x, y)


   
1 0 0 1 1
B= 0 1 0  R= 1 0 
0 0 1 0 1
13 / 22
Bases (3)
■ If B is a basis, then the following holds

BxB + RxR = b

Hence:

xB = B −1 b − B −1 RxR

Non-basic variables determine values of basic ones

■ If non-basic variables are set to 0, we get the solution

xR = 0, xB = B −1 b

Such a solution is called a basic solution

■ If a basic solution satisfies xB ≥ 0 then it is called a


basic feasible solution, and the basis is feasible

14 / 22
Bases (4)
Consider basis (s1 , s2 , s3 )

max x + 2y    
x + y + s1 = 3 1 0 0 1 1
x + s2 = 2 B= 0 1 0  R= 1 0 
y + s3 = 2 0 0 1 0 1
x, y, s1 , s2 , s3 ≥ 0

Equations xB = B −1 b − B −1 RxR are



 s1 = 3 − x − y
s2 = 2 − x
s3 = 2 − y

Basic solution is
 
3  
0
σB =  2  σR =
0
2
So basis (s1 , s2 , s3 ) is feasible 15 / 22
Bases (5)
Basis (x, y, s1 ) is not feasible

max x + 2y    
x + y + s1 = 3 1 1 1 0 0
x + s2 = 2 B= 1 0 0  R= 1 0 
y + s3 = 2 0 1 0 0 1
x, y, s1 , s2 , s3 ≥ 0
  
 x = 2 − s2 2  
0
y = 2 − s3 σB =  2  σR =
0
s1 = −1 + s2 + s3 −1

16 / 22
Bases (6)
A basis is called degenerate
when at least one component of its basic solution xB is null
For example:

max x + 2y    
x + y + s1 = 4 1 1 0 1 0
x + s2 = 2 B= 1 0 1  R= 0 0 
y + s3 = 2 0 1 0 0 1
x, y, s1 , s2 , s3 ≥ 0
  
 x = 2 + s3 − s1 2
y = 2 − s3 σB =  2 
s2 = s1 − s3 0

17 / 22
Geometry of LP’s (1)
■ Set of feasible solutions of an LP is a convex polyhedron

■ Basic feasible solutions are vertices of the convex polyhedron

18 / 22
Geometry of LP’s (2)
max x + 2y
x + y + s1 = 3
■ x + s2 = 2 y
y + s3 = 2 y≤2
x, y, s1 , s2 , s3 ≥ 0 (0, 2) 1 2 (1, 2)
x+y ≤3
■ xB1 = (y, s1 , s2 )
3 (2, 1)
■ xB2 = (x, y, s2 )
x≥0 x≤2
■ xB3 = (x, y, s3 ) 5 4
y≥0 x
■ xB4 = (x, s1 , s3 ) (0, 0) (2, 0)

■ xB5 = (s1 , s2 , s3 )

19 / 22
Geometry of LP’s (3)
■ Theorem (Minkowski-Weyl)
Let P be a feasible LP.
There exist basic feasible solutions v1 , ..., vr ∈ Rn and vectors
r1 , ..., rs ∈ Rn such that a point x is a feasible solution to P iff

r
X s
X
x= λi vi + µ j rj
i=1 j=1
Pr
for certain λi , µj such that i=1 λi = 1 and λi , µj ≥ 0.

20 / 22
Possible Outcomes of an LP (1)
■ Theorem (Fundamental Theorem of Linear Programming)
Let P be an LP.
Then exactly one of the following holds:

1. P is infeasible
2. P is unbounded
3. P has an optimal basic feasible solution

It is sufficient to investigate basic feasible solutions!

21 / 22
Possible Outcomes of an LP (2)
Proof: Assume P feasible and with optimal solution x∗ .
Let us see we can find a basic feasible solution as good as x∗ .
By Minkowski-Weyl theorem, we can write
Pr Ps
x∗ = λ ∗v
i=1 i i + µ ∗r
j=1 j j
Pr ∗
where i=1 λi = 1 and λ∗i , µ∗j ≥ 0. Then
Pr Ps
c T x∗ = ∗ T
i=1 λi c vi + ∗ T
j=1 µj c rj

22 / 22
Possible Outcomes of an LP (2)
Proof: Assume P feasible and with optimal solution x∗ .
Let us see we can find a basic feasible solution as good as x∗ .
By Minkowski-Weyl theorem, we can write
Pr Ps
x∗ = λ ∗v
i=1 i i + µ ∗r
j=1 j j
Pr ∗
where i=1 λi = 1 and λ∗i , µ∗j ≥ 0. Then
Pr Ps
c T x∗ = ∗ T
i=1 λi c vi + ∗ T
j=1 µj c rj

■ If there is j such that cT rj < 0 then objective value


can be decreased by taking µ∗j larger. Contradiction!

22 / 22
Possible Outcomes of an LP (2)
Proof: Assume P feasible and with optimal solution x∗ .
Let us see we can find a basic feasible solution as good as x∗ .
By Minkowski-Weyl theorem, we can write
Pr Ps
x∗ = λ ∗v
i=1 i i + µ ∗r
j=1 j j
Pr ∗
where i=1 λi = 1 and λ∗i , µ∗j ≥ 0. Then
Pr Ps
c T x∗ = ∗ T
i=1 λi c vi + ∗ T
j=1 µj c rj

■ If there is j such that cT rj < 0 then objective value


can be decreased by taking µ∗j larger. Contradiction!

■ Otherwise cT rj ≥ 0 for all j. Assume cT x∗ < cT vi for all i.


T ∗
Pr ∗ T
Pr ∗ T ∗ T ∗
Pr ∗ = c T x∗
c x ≥ i=1 λi c vi > i=1 λi c x = c x λ
i=1 i

22 / 22
Possible Outcomes of an LP (2)
Proof: Assume P feasible and with optimal solution x∗ .
Let us see we can find a basic feasible solution as good as x∗ .
By Minkowski-Weyl theorem, we can write
Pr Ps
x∗ = λ ∗v
i=1 i i + µ ∗r
j=1 j j
Pr ∗
where i=1 λi = 1 and λ∗i , µ∗j ≥ 0. Then
Pr Ps
c T x∗ = ∗ T
i=1 λi c vi + ∗ T
j=1 µj c rj

■ If there is j such that cT rj < 0 then objective value


can be decreased by taking µ∗j larger. Contradiction!

■ Otherwise cT rj ≥ 0 for all j. Assume cT x∗ < cT vi for all i.


T ∗
Pr ∗ T
Pr ∗ T ∗ T ∗
Pr ∗ = c T x∗
c x ≥ i=1 λi c vi > i=1 λi c x = c x λ
i=1 i

Contradiction! Thus there is i such that cT x∗ ≥ cT vi ;


in fact, cT x∗ = cT vi by the optimality of x∗ . 22 / 22

You might also like