0% found this document useful (0 votes)
38 views44 pages

3b.analysis of LP

The document discusses the steps of the simplex method for solving linear programming problems. It begins by recalling the standard form of a linear program. It then outlines the main steps of the primal simplex method, which are to 1) convert the problem to standard form, 2) obtain an initial basic feasible solution, 3) determine if the current solution is optimal, and if not 4) select entering and leaving variables to find a new solution, and 5) use elementary row operations to express the new solution. The document provides an example to demonstrate these steps. It also discusses how the simplex method can be applied to structured linear programs with non-negative right-hand sides.

Uploaded by

Esra Tanyıldız
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
38 views44 pages

3b.analysis of LP

The document discusses the steps of the simplex method for solving linear programming problems. It begins by recalling the standard form of a linear program. It then outlines the main steps of the primal simplex method, which are to 1) convert the problem to standard form, 2) obtain an initial basic feasible solution, 3) determine if the current solution is optimal, and if not 4) select entering and leaving variables to find a new solution, and 5) use elementary row operations to express the new solution. The document provides an example to demonstrate these steps. It also discusses how the simplex method can be applied to structured linear programs with non-negative right-hand sides.

Uploaded by

Esra Tanyıldız
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 44

IE 311 – Operations Research I

Part 3: Analysis of Linear Programming


3b. Simplex Method for ‘Structured’ LPs

Spring 2021
Recall: Linear Programming (Standard Form)

• Recall that any Linear Program can be written as below:

• Now, we will start understanding how LPs can be solved.

IE 311 2
How Does Gurobi Solve LPs?

IE 311 3
Main Steps of the (Primal) Simplex Method

The simplex algorithm proceeds as follows:


• Step 1 Convert the LP to standard form.
• Step 2 Obtain a BFS (if possible) from the standard form.
• Step 3 Determine whether the current BFS is optimal.
• Step 4 If the current BFS is not optimal, then determine which
nonbasic variable should become a basic variable and which basic
variable should become a nonbasic variable to find a new BFS with a
better objective function value (or determine if the problem is unbounded).
• Step 5 Use EROs to find a new BFS with a better objective function
value. Go back to step 3.
IE 311 4
Roadmap

• Although the Simplex Method can solve any LP in principle, we will first
look at well-structured LPs (those with an obvious initial BFS) and then
proceed with unstructured LPs (those without an obvious initial BFS).
• In particular, we will first learn how to solve maximization problems with
only ≤ constraints and a nonnegative right-hand-side (RHS) 𝒃 vector.
• In this case, an obvious BFS is available (WHY?)
• Then, we will discuss how to solve minimization problems with
minor modifications.
• Finally, we will learn how to incorporate ≥ and = constraints with an
arbitrary RHS vector.
• In these cases, some preliminary operations (Big-M or Two-Phase
methods) will be needed to obtain an initial BFS.
IE 311 5
A First Example

• We will first go over the following example LP and explain the steps of
𝑥2
the Simplex Method.

0
3 2
2

𝑥1
0 3
0 0
• For convenience, we defined a new variable 𝑥0 (column-0), which
represents the objective function value (row-0).

IE 311 6
A First Example: Step 1

• In Step 1, we convert the LP to standard form:

IE 311 7
A First Example: Step 2
𝑥2
• In Step 2, we obtain an initial BFS:

0
3 2
2

𝑥1
0 3
0 0
• Convention: Variable 𝑥0 will ALWAYS be basic.
• Note: Since the original LP has only ≤ constraints with nonnegative
RHS, the slack variables 𝑥3 and 𝑥4 are chosen as basic variables.
• RHS column in this tableau gives the current values of the basic variables.
IE 311 8
A First Example: Step 3

• In Step 3, we check whether the current BFS is optimal (look at row-0):

• Idea: If we move to an adjacent BFS, can we improve the objective


function variable 𝑥0 ?
• Moving to an adjacent BFS means that we make one of the nonbasic variables
basic while making one of the basic variables nonbasic.
• Note that increasing the value of 𝒙𝟏 or 𝒙𝟐 would make 𝒙𝟎 larger (WHY?)
• Therefore, the current BFS is NOT proven to be optimal.
IE 311 9
A First Example: Step 4

• In Step 4, we first decide which nonbasic variable to make basic (look at


row-0):

• Idea: Intuitively, choosing the most negative coefficient in row-0


might improve the objective largest (although any nonbasic variable
with negative coefficient can be chosen).
• In this case, we can choose the nonbasic variable 𝒙𝟐 and make it basic.
• Such a variable will be called the entering variable.

IE 311 10
A First Example: Step 4
• In Step 4, we next decide which basic variable to make nonbasic (look at
the column of the entering variable and RHS):

• Idea: What is the adjacent BFS obtained by making 𝑥2 basic?


• We need to perform the so-called minimum ratio test:
• Obtain the entrywise ratio of the RHS and the column of the entering
variable among the rows that have positive entry in the column of the
entering variable.
• Choose the basic variable that gives the minimum ratio as the leaving
variable (if there is no such variable, the LP is unbounded):
6 6
IE 311
The leaving variable is 𝑥4 since min , = 3. 11
1 2
A First Example: Step 5

• In Step 5, we use EROs to find a new BFS with a


better objective function value (pivoting):
𝑥2

0
3
2
• Idea: How can we make the column of the 2
entering variable a unit vector by pivoting?
𝑥1
0 3
0 0

IE 311 12
A First Example: Step 3 (Again)

• In Step 3, we check whether the current BFS is optimal (look at row-0):

• Idea: If we move to an adjacent BFS, can we improve the objective


function variable 𝑥0 ?
• Note that increasing the value of 𝒙𝟏 would make 𝒙𝟎 larger.
• Therefore, the current BFS is NOT proven to be optimal.

IE 311 13
A First Example: Step 4 (Again)

• In Step 4, we first decide which nonbasic variable to make basic (look at


row-0):

• Idea: Intuitively, choosing the most negative coefficient in row-0


might improve the objective largest (although any nonbasic variable
with negative coefficient can be chosen).
• In this case, there is only one candidate: the nonbasic variable 𝒙𝟏
enters the basis.
IE 311 14
A First Example: Step 4 (Again)

• In Step 4, we next decide which basic variable to make nonbasic (look


at the column of the entering variable and RHS):

• Idea: What is the adjacent BFS obtained by making 𝑥2 basic?


• We perform the minimum ratio test:
3 3
The leaving variable is 𝑥3 since min , = 2.
3/2 1/2

IE 311 15
A First Example: Step 5 (Again)

• In Step 5, we use EROs to find a new BFS with a better


objective function value (pivoting): 𝑥2

0
3
2
2
• Idea: How can we make the column of the
entering variable a unit vector by pivoting? 𝑥1
0 3
0 0

IE 311 16
A First Example: Step 3 (Again)

• In Step 3, we check whether the current BFS is optimal (look at row-0):

• Idea: If we move to an adjacent BFS, can we improve the objective


function variable 𝑥0 ?
• Note that increasing the value of 𝒙𝟑 or 𝒙𝟒 would make 𝒙𝟎 smaller.
• Therefore, the current BFS is OPTIMAL with objective value 14.
• Optimal solution: 𝑥1 = 𝑥2 = 2.
IE 311 17
Main Steps of the Simplex Method: Structured LPs

• Consider the following structured LP with 𝒃 ≥ 𝟎:

IE 311 18
Main Steps of the Simplex Method: Structured LPs

• Step 1 Convert the LP to standard form.

IE 311 19
Main Steps of the Simplex Method: Structured LPs

• Step 2 Obtain a BFS from the standard form.


• Note that the slack variables 𝑥𝑛+1 , … , 𝑥𝑛+𝑚 constitute an obvious basis
(since 𝑏 ≥ 0).
• We construct the following simplex tableau:

IE 311 20
Main Steps of the Simplex Method: Structured LPs

• Step 3 Determine whether the current BFS is optimal.


• Look at the coefficient in the row-0 of the current simplex
tableau (except 𝑥0 ).
• If all the entries are nonnegative, then STOP. The current BFS is
optimal.
• Otherwise, we cannot prove the optimality of the current BFS.
Proceed with Step 4.

IE 311 21
Main Steps of the Simplex Method: Structured LPs

• Step 4 Determine which nonbasic variable should become a basic


variable.
• Any nonbasic variable with negative coefficient in row-0 can be
selected as the entering variable (this will guarantee the objective value to
improve-or at least not to deteriorate).
• As a common practice, we will use the following rule: choose the
variable with the most negative coefficient.
• If there is a tie, we can choose the variable with the smaller index.

IE 311 22
Main Steps of the Simplex Method: Structured LPs

• Step 4 Determine which basic variable should become a nonbasic


variable.
• We need to employ the minimum ratio test (this will guarantee the next
BFS to remain feasible) to find the leaving variable:
• Let 𝐶𝑂𝐿 be the column of the entering variable and 𝑅𝐻𝑆 be the right-hand-
side in the current tableau (ignore row-0 elements).
𝑅𝐻𝑆𝑖
• Find a row 𝑖∗ such that 𝑖∗ ∈ arg min .
𝑖:𝐶𝑂𝐿𝑖 >0 𝐶𝑂𝐿 𝑖
• If there is no row with 𝐶𝑂𝐿𝑖 > 0, then STOP. The LP is unbounded!
• Otherwise, choose the basic variable in row 𝑖 ∗ as the leaving variable.
• If there is a tie, we can choose the variable with the smaller index.

IE 311 23
Main Steps of the Simplex Method: Structured LPs

• Step 5 Use EROs to find a new BFS with a better objective function
value.
• Use the entry in the intersection of the column of the entering
variable and the row of the leaving variable as pivot.
• Apply EROs to obtain the new BFS.
• Go to Step 3.

IE 311 24
Some Useful Observations

• The value of the entering variable in the next BFS is equal to the
minimum ratio.
• The change in the objective value is equal to the minimum ratio
times (minus) the objective coefficient of the entering variable.
• If there is a tie in the minimum ratio, the next BFS is degenerate.
• If the objective coefficient of a nonbasic variable is 0 in the optimal
tableau, then there exist multiple (or alternative) optima.
• If the LP is unbounded, the column of the entering variable can be
used to determine a direction of unboundedness which makes
positive inner product with the objective vector.

IE 311 25
Example 2

• Solve the following LP using the simplex method:

• Before pivoting, state the value of the entering variable in the


next BFS and the change in the objective function value.
IE 311 26
Example 2: Iteration 1

• Entering Variable: 𝑥1 (most negative coefficient = −6)


• Leaving Variable: 𝑥6 (minimum ratio = 4)

IE 311 27
Example 2: Iteration 2

• Entering Variable: 𝑥3 (most negative coefficient = −1/2)


• Leaving Variable: 𝑥5 (minimum ratio = 8)

IE 311 28
Example 2: Iteration 3

• STOP: This is the optimal tableau (there is a unique optimal solution


since objective coefficients of all nonbasic variable are positive).

IE 311 29
Degenerate BFS: Example 3

• If there is a tie in the minimum ratio, the next BFS is degenerate.


• Solve the following LP using the simplex method:
𝑥2

0
4 2
2

𝑥1
0 5/2
0 0

IE 311 30
Example 3: Iteration 1

𝑥2

0
4 2
• Entering Variable: 𝑥1 (most negative coefficient = −5) 2
• Leaving Variable: 𝑥4 (minimum ratio = 5/2)
𝑥1
0 5/2
0 0

IE 311 31
Example 3: Iteration 2

𝑥2

0
4 2
• Entering Variable: 𝑥2 (most negative coefficient = −7/4) 2
• Leaving Variable: 𝑥3 (minimum ratio = 2 – there is a tie!)
𝑥1
0 5/2
0 0

IE 311 32
Example 3: Iteration 3

𝑥2

0
4 2
• Entering Variable: 𝑥4 (most negative coefficient = −1/2) 2
• Leaving Variable: 𝑥5 (minimum ratio = 0 – degenerate pivot)
𝑥1
0 5/2
0 0

IE 311 33
Example 3: Iteration 4

• STOP: This is the optimal tableau (there is a unique optimal solution


since objective coefficients of all nonbasic variable are positive).

IE 311 34
Multiple (or Alternative) Optima: Example 4

• If the objective coefficient of a nonbasic variable is 0 in the optimal


tableau, then there exist multiple (or alternative) optima.
• Solve the following LP using the simplex method:
𝑥 2

56/17
45/17

0
1
𝑥1
0 7
0 0

IE 311 35
Example 4: Iteration 1

𝑥2

56/17
45/17

• Entering Variable: 𝑥2 (most negative coefficient = −6) 0


• Leaving Variable: 𝑥4 (minimum ratio = 1) 1
𝑥1
0 7
0 0

IE 311 36
Example 4: Iteration 2

𝑥2

56/17
45/17

• STOP: This is an optimal tableau (multiple optima exist). 0


• What happens if we make 𝒙𝟏 enter the basis? 1
𝑥1
0 7
0 0

IE 311 37
Example 4: Follow-up

• The set of all optimal solutions lie in the 𝑥2

following line segment: 56/17


45/17

0
1
𝑥1
0 7
0 0

IE 311 38
Unbounded LPs: Example 5

• If the LP is unbounded, the column of the entering variable


can be used to determine a direction of unboundedness which
makes positive inner product with the objective vector.
• Solve the following LP using the simplex method:
𝑥2

0
3

𝑥1
0 4
0 0

IE 311 39
Example 5: Iteration 1

𝑥2

0
• Entering Variable: 𝑥2 (most negative coefficient = −3) 3
• Leaving Variable: 𝑥4 (minimum ratio = 3)
𝑥1
0 4
0 0

IE 311 40
Example 5: Iteration 2

𝑥2

0
3

𝑥1
0 4
• Entering Variable: 𝑥1 (most negative coefficient = −7) 0 0
• Leaving Variable: NONE! LP is unbounded.

• How to find the direction of unboundedness from the tableau?

IE 311 41
Example 5: Follow-up

• Suppose 𝑥1 ‘enters’ the basis and its value is increased by 1 unit. Then,
• due to row-1, 𝑥3 is increased by 3 units,
• due to row-2, 𝑥2 is increased by 2 units,
• 𝑥4 is unaffected since it is nonbasic.
𝑑1 1
𝑑2 2
• ⇒ Direction of unboundedness (standard form): =
𝑑3 3
𝑑4 0
𝑑1 1
• ⇒ Direction of unboundedness (original variables): =
IE 311 𝑑2 2 42
Minimization Problems

• If we have a minimization LP, there are two options:


1. We can maximize the negative of the original objective
(no modification is needed in the simplex algorithm).
2. We can modify the simplex algorithm to solve a
minimization problem as follows:
• Step 3: If all the coefficients in row-0 are nonpositive, then we
STOP. This is an optimal tableau.
• Step 4: If this is not the case, we select the variable with the
most positive coefficient in row-0 as the entering variable.

IE 311 43
Tableau Implementation of the Simplex Method
with Initial BFS: Python Code

•Use the Python file


SimplexTableauWithBV.py
uploaded to Sucourse to solve
Examples 1-5.

IE 311 44

You might also like