02-24-01410 - Non-Linear

and Combinatorial
Optimization
DR. AHMED TAYEL
De p a rtment o f E n gi neering M at h em ati cs a n d P hysi cs, Fa c ul ty o f
E n gi neering , Al exandria Un i ve rsi ty
a h m e d .taye l @ a l ex u . e d u . e g
Linear Programming
1. Introduction.
2. Graphical method.
3. Simplex method.
Linear Programming: The Simplex
Method
 Introduction
▪ Graphical methods are fine for 2 variables.
▪ But most LP problems are too complex for simple graphical procedures.

▪ The Simplex Method:

o examines corner points, like in graphing;
o systematically examines corner points, using algebra, until an optimal solution is
found;
o does its searching iteratively.

▪ Can be programmed to solve large LP problems.

▪ We will study the Basic Simplex algorithm

▪ Restrictions: Max. problem
Functional constraint ≤ RHS(non negative)

This is to ensure that the starting point of the algorithm 0,0 is always feasible.
Setting UP the Simplex Method
• The algebraic procedure is based on solving systems of equations.

Step 1 is setting up the simplex method is to

• Convert the minimization problem to maximization. Min = − Max
• Convert the functional inequality constraints to equivalent equality
constraints. This conversion is accomplished by introducing slack variables.
• To illustrate, consider this constraint: X1 4
The slack variable for this constraint is defined to be
X1+S12=4 Problem: non-linear
Solution: X1+S1=4 and S1>0
• Consider this constraint: −X1 ≥ −4
 𝑥2
Example
• Objective Function
Min. W=−3X1−5X2
→ Max. Z=3X1+5X2
• Subject to:
Point 𝒛
X1 4
(0,0) 0
2X2 12
(4,0) 12
3X1+2X2 18 (4,3) 27 𝑥1
X1,X2 0 (2,6) 36
(0,6) 30
Example
• Objective Function
The Standard Form of the Model
Min. W=−3X1−5X2 → Max. Z=3X1+5X2 Z-3X1-5X2=0 ..………(0)

• Subject to:
X1 4 X1+S1=4 ...……...(1)
2X2 12 2X2+S2=12 ..………(2)
3X1+2X2 18 3X1+2X2+S3=18 ………...(3)
X1,X2 0
 The Simplex Method in Tabular Form
Tabular form
Eq. Basic Coefficient of: Right
variable Z X1 X2 S1 S2 S3 side

Obj. fn. (0) Z 1 -3 -5 0 0 0 0 Z-3X1-5X2=0

(1) S1 0 1 0 1 0 0 4 X1+S1=4

(2) S2 0 0 2 0 1 0 12 2X2+S2=12

(3) S3 0 3 2 0 0 1 18 3X1+2X2+S3=18

Basic variables Starting solution

≠𝟎 (0,0)
Flow chart Stop
The optimal solution has End
been found.

No

Step 1
Write the maximization problem Step 2
Yes Step 3
in standard form. Are there any negative
Start Select the
- Min to max problem. coefficients in the first row? pivot column.
- Introduce the slack variables.
- Write the initial tableau.
 The Simplex Method in Tabular Form
Tabular form
Eq. Basic Coefficient of: Right
variable Z X1 X2 S1 S2 S3 side

Z=3X1+5X2 (0) Z 1 -3 -5 0 0 0 0
(1) S1 0 1 0 1 0 0 4
(2) S2 0 0 2 0 1 0 12
(3) S3 0 3 2 0 0 1 18

Pivot column
Summary of the Simplex Method
• Step 2 (Optimality Test) The current basic feasible solution is
optimal if and only if every coefficient in row (0) is nonegative
(0). If it is, stop; otherwise, go to an iteration to obtain the next
basic feasible solution.
• Iteration:
Step3
Determine the entering basic variable by selecting the variable
with negative coefficient having the largest absolute value (i,e.,
the “most negative” coefficient) in Eq. (0).
Reason: Greedy algorithm large coefficient → 𝑧 increases faster.
Put a box around the column below this coefficient, and call this
the pivot column.
Flow chart Stop
The optimal solution has End
been found.

No

Step 1
Write the maximization problem Step 2
Yes Step 3
in standard form. Are there any negative
Start Select the
- Min to max problem. coefficients in the first row? pivot column.
- Introduce the slack variables.
- Write the initial tableau.

Step 5 Yes Step 4

Select the pivot element and Are there any positive elements
perform the pivot operation. in pivot column?

No

Stop
The decision variables
can be increased
without bounds
 The Simplex Method in Tabular Form
Tabular form
Eq. Basic Coefficient of: Right
variable Z X1 X2 S1 S2 S3 side

(0) Z 1 -3 -5 0 0 0 0
(1) S1 0 1 0 1 0 0 4 X1+S1=4

(2) S2 0 0 2 0 1 0 12 2X2+S2=12

(3) S3 0 3 2 0 0 1 18 3X1+2X2+S3=18

Pivot element
Pivot column
Summary of the Simplex Method/ continued
Steps 4 and 5
Determine the leaving basic variable by applying the minimum ratio test.
Minimum Ratio Test (based on the constraint to get how much the entering
variable should be increases)
1. Pick out each coefficient in the pivot column that is strictly positive (>0).
2. Divide each of these coefficients into the right side entry for the same row.
3. Identify the row that has the smallest of these ratios.
4. The basic variable for that row is the leaving basic variable, so replace that
variable by the entering basic variable in the basic variable column of the next
simplex tableau.
Put a box around this row and call it the pivot row. Also call the number that is in
both boxes the pivot number.
Flow chart Stop
The optimal solution has End
been found.

No

Step 1
Write the maximization problem Step 2
Yes Step 3
in standard form. Are there any negative
Start Select the
- Min to max problem. coefficients in the first row? pivot column.
- Introduce the slack variables.
- Write the initial tableau.

Step 5 Yes Step 4

Select the pivot element and Are there any positive elements
perform the pivot operation. in pivot column?

No

Stop
The decision variables
can be increased
without bounds
 # (0) Iteration
Tabular form
Eq. Basic Coefficient of: Right
variable side
Z X1 X2 S1 S2 S3

(0) Z 1 -3 -5 0 0 0 0
(1) S1 0 1 0 1 0 0 4
(2) S2 0 0 2 0 1 0 12
(3) S3 0 3 2 0 0 1 18
 # (1) Iteration
Tabular form
Eq. Basic Coefficient of: Right
variable side
Z X1 X2 S1 S2 S3

(0) Z 1 -3 0 0 5\2 0 30
(1) S1 0 1 0 1 0 0 4
(2) X2 0 0 1 0 1\2 0 6
(3) S3 0 3 0 0 -1 1 6
 # (2) Iteration
Tabular form
Eq. Basic Coefficient of: Right
variable side
Z X1 X2 S1 S2 S3

(0) Z 1 0 0 0 3\2 1 36
(1) S1 0 0 0 1 1\3 -1\3 2
(2) X2 0 0 1 0 1\2 0 6
(3) X1 0 1 0 0 -1\3 1\3 2
Summary of the Simplex Method/ continued
Continue Step 5
Solve for the new basic feasible solution by using elementary row
operations (by applying Gauss-jordan method), The method effects a
change in basis by using two types of computations:
1. Type 1 (pivot equation):
new pivot Eq.=old pivot Eq. ÷ pivot number
2. Type 2 (all other equations, including Z):
new Eq. = old Eq. – (its entering column coefficient) X (new pivot Eq.)
•Going to the optimality test, we find that this solution is
optimal because none of the coefficients in row (0) is
negative, so the algorithm is finished.
•Consequently, the optimal solution for this problem is
X1=2, X2=6.
Flow chart Stop
The optimal solution has End
been found.

No

Step 1
Write the maximization problem Step 2
Yes Step 3
in standard form. Are there any negative
Start Select the
- Min to max problem. coefficients in the first row? pivot column.
- Introduce the slack variables.
- Write the initial tableau.

Step 5 Yes Step 4

Select the pivot element and Are there any positive elements
perform the pivot operation. in pivot column?

No

Stop
The decision variables
can be increased
without bounds
Special Cases in Simplex Method Application
• We will consider special cases that can arise in the
application of the simplex method, which include:

1. Alternative optima (more than one optimum solution).

2. Unbounded solutions.

3. Non-existing (or infeasible) solutions.

Alternative Optima
• When the objective function is parallel to a binding constraint, the objective
function will assume the same optimal value at more than one solution point. For
this reason they are called alternative optima.
• Algebraically, after the simplex method finds one optimal basic feasible (BF)
solution, you can detect if there any others and, if so, find them as follows:
• Whenever a problem has more than one optimal BF solution, at least one of the
nonbasic variables has a coefficient of zero in the final row (0), so increasing any
such variable will not change the value of Z.
• Therefore, these other optimal BF solutions can be identified (if desired) by
performing additional iterations of the simplex method, each time choosing a
nonbasic variable with a zero coefficient as the entering basic variable.
Alternative Optima/example

Max. Z=2X1+4X2

ST
X1+2X2  5
X1+ X2  4
X1,X2  0
Alternative Optima/example
Basic X1 X2 S1 S2 R.S.

Z -2 -4 0 0 0
S1 1 2 1 0 5

S2 1 1 0 1 4
 Alternative Optima/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -2 -4 0 0 0
(starting) S1 1 2 1 0 5
X2 enters
S1 leaves S2 1 1 0 1 4
 Alternative Optima/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -2 -4 0 0 0
(starting) S1 1 2 1 0 5
X2 enters
S1 leaves S2 1 1 0 1 4

X2 1/2 1 1/2 0 5/2

 Alternative Optima/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -2 -4 0 0 0
(starting) S1 1 2 1 0 5
X2 enters
S1 leaves S2 1 1 0 1 4

Z 0 0 2 0 10
X2 1/2 1 1/2 0 5/2

S2 1/2 0 -1/2 1 3/2

 Alternative Optima/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -2 -4 0 0 0
(starting) S1 1 2 1 0 5
X2 enters
S1 leaves S2 1 1 0 1 4

1 Z 0 0 2 0 10
(optimum)
X2 1/2 1 1/2 0 5/2
X1 enters
S2 leaves S2 1/2 0 -1/2 1 3/2
 Alternative Optima/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -2 -4 0 0 0
(starting) S1 1 2 1 0 5
X2 enters
S1 leaves S2 1 1 0 1 4

1 Z 0 0 2 0 10
(optimum)
X2 1/2 1 1/2 0 5/2
X1 enters
S2 leaves S2 1/2 0 -1/2 1 3/2

2
( alternate
optimum)
X1 1 0 -1 2 3
 Alternative Optima/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -2 -4 0 0 0
(starting) S1 1 2 1 0 5
X2 enters
S1 leaves S2 1 1 0 1 4

1 Z 0 0 2 0 10
(optimum)
X2 1/2 1 1/2 0 5/2
X1 enters
S2 leaves S2 1/2 0 -1/2 1 3/2
Z 0 0 2 0 10
2
( alternate
optimum)
X1 1 0 -1 2 3
 Alternative Optima/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -2 -4 0 0 0
(starting) S1 1 2 1 0 5
X2 enters
S1 leaves S2 1 1 0 1 4

1 Z 0 0 2 0 10
(optimum)
X2 1/2 1 1/2 0 5/2
X1 enters
S2 leaves S2 1/2 0 -1/2 1 3/2
Z 0 0 2 0 10
2
( alternate X2 0 1 1 -1 1
optimum)
X1 1 0 -1 2 3
 𝑦

Every point on this darker line segment

is optimal, each with z = 10.

As in this case, any problem having multiple optimal solutions will have an infinite number of
them, each with the same optimal value of the objective function.
Flow chart Stop
The optimal solution has End
been found.

No

Step 1
Write the maximization problem Step 2
Yes Step 3
in standard form. Are there any negative
Start Select the
- Min to max problem. coefficients in the first row? pivot column.
- Introduce the slack variables.
- Write the initial tableau.

Step 5 Yes Step 4

Select the pivot element and Are there any positive elements
perform the pivot operation. in pivot column?

No

Stop
The decision variables
can be increased
without bounds
Flow chart Are there a non-basic variable
with 0 coefficient in the obj.
Yes Perform one more
iteration, then stop.
End
fn. row?

No Stop
No The optimal solution has
been found.
Step 1
Write the maximization problem Step 2
Yes Step 3
in standard form. Are there any negative
Start Select the
- Min to max problem. coefficients in the first row? pivot column.
- Introduce the slack variables.
- Write the initial tableau.

Step 5 Yes Step 4

Select the pivot element and Are there any positive elements
perform the pivot operation. in pivot column?

No

Stop
The decision variables
can be increased
without bounds
 Unbounded Solution
• In some LP models, the values of the variables may be increased
indefinitely without violating any of the constraints, meaning that the
solution space is unbounded in at least one direction.
• Unboundedness in a model can point to one thing only. The model is
poor constructed.
• The general rule for recognizing unboundedness is as follows:
If at any iteration the constraint coefficients of a nonbasic variable are
nonpositive, then the solution space is unbounded in that direction.
Unbounded Solution/example

Max 𝑧 = 3 𝑥1 + 2 𝑥2
Subject to 𝑥1 − 2 𝑥2 ≤ 10
2 𝑥1 − 𝑥2 ≤ 10
𝑥1 , 𝑥2 ≥ 0
Unbounded Solution/example
Basic X1 X2 S1 S2 R.S.

Z -3 -2 0 0 0
S1 1 -2 1 0 10

S2 2 -1 0 1 10
 Unbounded Solution/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -3 -2 0 0 0
(starting) S1 1 -2 1 0 10
X1 enters
S2 leaves S2 2 -1 0 1 10
 Unbounded Solution/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -3 -2 0 0 0
(starting) S1 1 -2 1 0 10
X1 enters
S2 leaves S2 2 -1 0 1 10

1
(No pivot)
X2 enters
X1 1 -1/2 0 -1/2 5
 Unbounded Solution/example
Iteration Basic X1 X2 S1 S2 R.S.

0 Z -3 -2 0 0 0
(starting) S1 1 -2 1 0 10
X1 enters
S2 leaves S2 2 -1 0 1 10

Z 0 -7/2 0 3/2 15
1
(No pivot) S1 0 -3/2 1 -1/2 5
X2 enters
X1 1 -1/2 0 -1/2 5
𝑦

Unbounded
feasible region

𝑥
Flow chart Are there a non-basic variable
with 0 coefficient in the obj.
Yes Perform one more
iteration, then stop.
End
fn. row?

No Stop
No The optimal solution has
been found.
Step 1
Write the maximization problem Step 2
Yes Step 3
in standard form. Are there any negative
Start Select the
- Min to max problem. coefficients in the first row? pivot column.
- Introduce the slack variables.
- Write the initial tableau.

Step 5 Yes Step 4

Select the pivot element and Are there any positive elements
perform the pivot operation. in pivot column?

No

Stop
The decision variables
can be increased
without bounds
 Infeasible Solution
• If the constraints cannot be satisfied, the model is said to have
no feasible solution.
• This situation can never occur if all the constraints are of the
type  (assuming nonegative right-side constants).
 Assignment
• Build a code for the simplex algorithm studied in
this lecture.
• Try solving the lecture examples/sheet/tutorial
examples using this code.
• Submission to Eng. Ali in the tutorial (week
starting 02-04-2022).