FUNDAMENTALS OF

OPTIMIZATION
Linear Programming

3
CONTENT

• Linear programs

• Geometric approach

• Simplex method

• Two-phase simplex method

• OR-TOOLS for linear programming

• Programming exercises

4
LINEAR PROGRAMS

• Standard form
f(x) = c1x1 + c2x2 + . . . + cnxn → max
a1,1x1 + a1,2x2 + . . . + a1,nxn ≤ b1
a2,1x1 + a2,2x2 + . . . + a2,nxn ≤ b2
...
am,1x1 + am,2x2 + . . . + am,nxn ≤ bm
x1, x2, . . . xn R, x1, x2, . . . xn  0

5
LINEAR PROGRAMS

• Standardize general linear programs

• f(x) → min  -f(x) → max
• g(x)  b  -g(x) ≤ -b
• A = B  (A ≤ B) and (-A ≤ -B)
• A variable xj  R can be represented by xj = 𝑥𝑗+ - 𝑥𝑗− where 𝑥𝑗+ , 𝑥𝑗−  0

6
LINEAR PROGRAMS

• Example: Convert a general linear program forms into standard form

f(x1, x2) = 3x1 + 2x2 → min
2x1 + x2 ≤ 7
x1 + 2x2 = 8
x1 - x2  2
x1, x2  R, x2  0

7
LINEAR PROGRAMS

• Example: Convert a general linear program forms into standard form

• Substitution: x1 = 𝑥1+ - 𝑥1−
f(𝑥1+ , 𝑥1− , x2) = -3 𝑥1+ + 3𝑥1− - 2x2 → max
2 𝑥1+ - 2𝑥1− + x2 ≤ 7
𝑥1+ - 𝑥1− + 2x2 ≤ 8
- 𝑥1+ + 𝑥1− - 2x2 ≤ -8
−𝑥1+ + 𝑥1− + x2 ≤ -2
𝑥1+ , 𝑥1− , x2  R, 𝑥1+ , 𝑥1− , x2  0

8
CONTENT

• Linear programs

• Geometric approach

• Simplex method

• Two-phase simplex method

• OR-TOOLS for linear programming

• Programming exercises

9
GEOMETRIC APPROACH

• Constraints (inequalities) form a feasible region

• Optimal points will be one of the corners of the feasible region

10
GEOMETRIC APPROACH

• Example
f(x1, x2) = 3x1 + 2x2 → max
2x1 + x2 ≤ 7
x1 + 2x2 ≤ 8
x1 - x2 ≤ 2
x1, x2  0

11
GEOMETRIC APPROACH

• Example
f(x1, x2) = 3x1 + 2x2 → max
2x1 + x2 ≤ 7
x1 + 2x2 ≤ 8
x1 - x2 ≤ 2
x1, x2  0

12
GEOMETRIC APPROACH

• Example
f(x1, x2) = 3x1 + 2x2 → max
2x1 + x2 ≤ 7
x1 + 2x2 ≤ 8
x1 - x2 ≤ 2
x1, x2  0

13
GEOMETRIC APPROACH

• Special cases
• Problem is unbounded
• Problem does not have feasible solutions

f(x1, x2) = 3x1 + 2x2 → max f(x1, x2) = 3x1 + 2x2 → max
-2x1 - x2 ≤ -7 2x1 + x2 ≤ 7
x1 - x2 ≤ 2 -4x1 - 2x2 ≤ -16
x1, x2  R, x1, x2  0 x1, x2  R, x1, x2  0

14
GEOMETRIC APPROACH

• Exercise
• A company must decide to make a plan to produce 2 products P1, P2.
• The revenue received when selling 1 unit of P1 and P2 are respectively 5$ and 7$
• The manufacturing cost when producing P1 and P2 are respectively 2$ and 3$
• The storage cost in warehouses for 1 unit of P1 and P2 are respectively 1$ and 3$
• Compute the production plan so that
• Total manufacturing cost is less than or equal to 200$
• Total storage cost is less than or equal to 150$
• Total revenue is maximal

15
CONTENT

• Linear programs

• Geometric approach

• Simplex method

• Two-phase simplex method

• OR-TOOLS for linear programming

• Programming exercises

16
STANDARD FORM

• Standard form to standard equational form by adding slack variables y1, y2, . . ., ym

Standard form Standard equality form

f(x) = c1x1 + c2x2 + . . . + cnxn → max f(x) = c1x1 + c2x2 + . . . + cnxn → max
a1,1x1 + a1,2x2 + . . . + a1,nxn ≤ b1 a1,1x1 + a1,2x2 + . . . + a1,nxn + y1 = b1
a2,1x1 + a2,2x2 + . . . + a2,nxn ≤ b2 a2,1x1 + a2,2x2 + . . . + a2,nxn + y2 = b2
... ...
am,1x1 + am,2x2 + . . . + am,nxn ≤ bm am,1x1 + am,2x2 + . . . + am,nxn + ym = bm
x1, x2, . . . xn R, x1, x2, . . . xn  0 x1, x2, . . . xn R, x1, x2, . . . xn , y1, y2, . . ., ym  0

17
SHORT FORM

• Consider a Linear Program (LP) under a standard equational form

Standard equational form Standard equality form

f(x) = c1x1 + c2x2 + . . . + cnxn → f(x) = cTx → max
max Ax = b
a1,1x1 + a1,2x2 + . . . + a1,nxn = b1 x 0
a2,1x1 + a2,2x2 + . . . + a2,nxn = b2
...
am,1x1 + am,2x2 + . . . + am,nxn = bm
x1, x2, . . . xn R, x1, x2, . . . xn  0

18
IDEA

z= 3x1 + 2x2 → max z= 3x1 + 2x2 → max

2x1 + x2 ≤ 7 2x1 + x2 + x3 =7
x1 + 2x2 ≤ 8 x1 + 2x2 + x4 =8
x1 - x2 ≤ 2 x1 - x2 + x5 =2
x1, x2  R, x1, x2  0 x1, x2 , x3, x4, x5  R, x1, x2, x3, x4, x5  0

z= - 4/3 x3 – 1/3 x4 + 12 → max

x2 -1/3x3 + 2/3x4 =3
– x3 + x4 + x5 =3
x1 + 2/3x3 -1/3 x4 =2
x1, x2 , x3, x4, x5  R, x1, x2, x3, x4, x5  0

19
BASIC FEASIBLE SOLUTION

• Consider a Linear Program (LP) under a standard equational f(x) = c1x1 + c2x2 + . . . + cnxn → max
form
a1,1x1 + a1,2x2 + . . . + a1,nxn = b1
• Suppose rank(A) = m
a2,1x1 + a2,2x2 + . . . + a2,nxn = b2
• Let B be the matrix of m linearly independent columns (indexed
...
j1, j2, . . ., jm) of A: B = (A(j1), A(j2), . . ., A(jm))
• Solution x is called a basic solution if : am,1x1 + am,2x2 + . . . + am,nxn = bm
• xj = 0 for j  {1, 2, . . ., n} \ {j1, j2, . . ., jm} x1, x2, . . . xn R, x1, x2, . . . xn  0
• Remain variables are found by solving this equation:

𝑎1,𝑗1 𝑎1,𝑗2 . . . 𝑎1,𝑗𝑚 𝑥𝑗1 𝑏1

𝑎2,𝑗1 𝑎2,𝑗2 . . . 𝑎2 𝑗𝑚 𝑥𝑗2 𝑏2
... ... ... ... ... = ...
𝑎𝑚,𝑗1 𝑎𝑚,𝑗2 . . . 𝑎𝑚,𝑗𝑚 𝑥𝑗𝑚 𝑏𝑚

20
BASIC FEASIBLE SOLUTION

• Let B be the matrix of m linearly independent columns (indexed j1, j2, . . ., jm) of A: B = (A(j1), A(j2), . .
., A(jm))
• Solution x is called a basic solution if :
• xj = 0 for j  {1, 2, . . ., n} \ {j1, j2, . . ., jm}
• Remain variables are found by solving this equation:

𝑎1,𝑗1 𝑎1,𝑗2 . . . 𝑎1,𝑗𝑚 𝑥𝑗1 𝑏1

𝑎2,𝑗1 𝑎2,𝑗2 . . . 𝑎2 𝑗𝑚 𝑥𝑗2 𝑏2
... ... ... ... ... = ...
𝑎𝑚,𝑗1 𝑎𝑚,𝑗2 . . . 𝑎𝑚,𝑗𝑚 𝑥𝑗𝑚 𝑏𝑚

• B is called a basis
• j1, j2, . . ., jm : basic indices, j  {1, 2, . . ., n} \ {j1, j2, . . ., jm} is called non-basic index
• 𝑥𝑗1 , 𝑥𝑗2 , . . ., 𝑥𝑗𝑚 : basic variables and xj (j  {1, 2, . . ., n} \ {j1, j2, . . ., jm}) is called non-basic variable
• A basic solution x with x ≥ 0 is called a basic feasible solution

21
Example
Example

f(x) = cx → max x = x1 c = (3, 2, 0, 0, 0)

Ax = b x2
x3 A = 2 1 1 0 0 b= 7
x0
x4 1 2 0 1 0 8
x5 2
1 -1 0 0 1

J = (1,2,3,4,5) – set of variable indices, I = (1,2,3) - set of constraint

indices
JB = (3,4,5), JN = (1,2),
B = 1 0 0 N = 2 1
xB = x3 = 7 xN = x1 = 0
x2 0 0 1 0 1 2
x4 8
x 2 0 0 1 1 -1
5

22
SIMPLEX METHOD: Tabular form
• Consider a linear program under a standard form
z = c1x1 + c2x2 + . . . + cnxn → max z = c1x1 + c2x2 + . . . + cnxn → max
a1,1x1 + a1,2x2 + . . . + a1,nxn ≤ b1 a1,1x1 + a1,2x2 + . . . + a1,nxn + y1 = b1
a2,1x1 + a2,2x2 + . . . + a2,nxn ≤ b2 a2,1x1 + a2,2x2 + . . . + a2,nxn + y2 = b2
... ...
am,1x1 + am,2x2 + . . . + am,nxn ≤ bm am,1x1 + am,2x2 + . . . + am,nxn + +ym = bm
b1, b2, . . . bm  0 b1, b2, . . . bm  0
x1, x2, . . . xn R, x1, x2, . . . xn  0 x1, x2, . . . xn R, x1, x2, . . . xn, y1, y2, …, ym  0

We need this additional

1 2 … n n+1 n+2 … n+m assumption to ensure we
have an initial feasible
0 x1 x2 … xn y1 y2 … ym z RHS
solution.
1 a1,1 a1,2 … a1,n 1 0 … 0 0 b1
2 a2,1 a2,2 … a2,n 0 1 … 0 0 b2
… … … … … … … … .. …
m am,1 am,2 … am,n 0 0 … 1 0 bm
m+1 -c1 -c2 … -cn 0 0 .. 0 1 0

23
 SIMPLEX METHOD: Tabular form
1 2 … n n+1 n+2 … n+m n+m+1
0 x1 x2 … xn xn+1 xn+2 … xn+m z RHS
1 1,1 1,2 … 1,n 1,n+1 1,n+2 … 1,n+m 1,n+m+1 1
2 2,1 2,2 … 2,n 2,n+1 2,n+2 … 2,n+m 2,n+m+1 2
… … … … … … … … .. …
m m,1 m,2 … m,n m,n+1 m,n+2 … m,n+m m,n+m+1 m
m+1 m+1,1 m+1,2 … m+1,n m+1,n+1 m+1,n+2 … m+1,n+m m+1,n+m+1 m+1

 J = {1, 2, . . ., n, n+1, . . ., n+m}

 Maintain linear constraints on each row k (k = 1, 2, …, m+1):
k,1 x1 + k,2 x2 + . . . + k,n xn + k,n+1xn+1 + . . . + k,n+mxn+m+ k,n+m+1z = k (*)
 Let Rk be a vector containing elements on row k of the table (k = 1, 2, …, m+1)
 Perform linear transformation below, constraint (*) is still satisfied:
 Replace Rk = Rk + *Ri (k, i = 1, 2, …, m+1), with some constant 

24
 SIMPLEX METHOD: Tabular form
 Optimality 1 2 … m m+1 m+2 … n+m n+m+1
0 x1 x2 … xm xm+1 xm+2 … xn+m z RHS
1 1 0 … 0 1,m+1 1,m+2 … 1,n+m 0 1
2 0 1 … 0 2,m+1 2,m+2 … 2,n+m 0 2
… … … … … … … … .. …
m 0 0 … 1 m,m+1 m,m+2 … m,n+m 0 m
m+1 0 0 … 0 m+1,m+1 m+1,m+2 … m+1,n+m 1 m+1

 With 1, 2, . . ., m  0,  JB = {j1, j2, . . ., jm } such that m+1,j = 0, j JB, m+1,j ≥ 0 j J\JB, columns j1, j2,
. . ., jm forms a unit matrix
 Without loss of generality, suppose that JB = {1, 2, . . ., n}, coefficients m+1,m+1, m+1,m+2, . . ., m+1,n+m 
0, columns 1, …, m forms a unit matrix: 1,1, 2,2, . . ., m,m = 1
 Constraint (*) is still satisfied. We have m+1,m+1xm+1 + m+1,m+2xm+2 + . . ., m+1,n+mxn+m + z = m+1
 z = m+1 – (m+1,m+1xm+1 + m+1,m+2xm+2 + . . ., m+1,n+mxn+m)  m+1 (because m+1,m+1, m+1,m+2, . . .,
m+1,n+m  0 and xm+1, …, xn+m  0).

25
 SIMPLEX METHOD: Tabular form
 Optimality 1 2 … m m+1 m+2 … n+m n+m+1
0 x1 x2 … xm xm+1 xm+2 … xn+m z RHS
1 1 0 … 0 1,m+1 1,m+2 … 1,n+m 0 1
2 0 1 … 0 2,m+1 2,m+2 … 2,n+m 0 2
… … … … … … … … .. …
m 0 0 … 1 m,m+1 m,m+2 … m,n+m 0 m
m+1 0 0 … 0 m+1,m+1 m+1,m+2 … m+1,n+m 1 m+1

 Moreover, there exists a solution (nonnegative values for variables x1, x2, …, xn+m) described below:
 x1 = 1 , x2 = 2 , . . ., xm = m
 xm+1 = xm+2 = . . . = xn+m = 0
Satisfying given constraints. Also, the objective value at this solution is equal to the upper bound
m+1. It means that this solution is an optimal solution to the given problem.

26
 SIMPLEX METHOD: Tabular form
1 2 … m m+1 i … n+m
 Simplex step
0 x1 x2 … xm xm+1 … xi … xn+m z RHS E
1 1 0 … 0 1,m+1 … 1,i … 1,n+m 0 1 E1
2 … … … … … … … … … … … …
… 0 1 … 0 k,m+1 … k,i … k,n+m 0 k Ek
m 0 0 … 1 m,m+1 … m,i … m,n+m 0 m Em
m+1 0 0 … 0 m+1,m+1 … m+1,i … m+1,n+m 1 m+1

 Select column i such that the element on row m+1 (which is m+1,i) is negative minimal
j
 Compute evaluations (column E): Ej = +, if j,i  0, and Ej =
j,i, if j,i > 0, j = 1, 2, …, m
 Select the row k such that Ek is minimal: if Ek = +, then the problem is unbounded, otherwise
 Update:
 Row Rk = Rk /k,i
 Row Rj = Rj - j,i * Rk, j = {1, 2, …, m+1} \ {k}

27
 SIMPLEX METHOD: Tabular form
 Example

z= 3x1 + 2x2 → max z= 3x1 + 2x2 → max

2x1 + x2 ≤ 7 2x1 + x2 + x3 =7
x1 + 2x2 ≤ 8 x1 + 2x2 + x4 =8
x1 - x2 ≤ 2 x1 - x2 + x5 =2
x1, x2  R, x1, x2  0 x1, x2 , x3, x4, x5  R, x1, x2, x3, x4, x5  0

x1 x2 x3 x4 x5 z RHS E
2 1 1 0 0 0 7
1 2 0 1 0 0 8
1 -1 0 0 1 0 2
-3 -2 0 0 0 1 0

28
 SIMPLEX METHOD: Tabular form
 Example

x1 x2 x3 x4 x5 z RHS E
1 2 1 1 0 0 0 7
2 1 2 0 1 0 0 8
3 1 -1 0 0 1 0 2
4 -3 -2 0 0 0 1 0

29
 SIMPLEX METHOD: Tabular form
 Example

x1 x2 x3 x4 x5 z RHS E
1 2 1 1 0 0 0 7
2 1 2 0 1 0 0 8
3 1 -1 0 0 1 0 2
4 -3 -2 0 0 0 1 0
• Select column 1: the corresponding element at the last row is minimal

x1 x2 x3 x4 x5 z RHS E
1 2 1 1 0 0 0 7
2 1 2 0 1 0 0 8
3 1 -1 0 0 1 0 2
4 -3 -2 0 0 0 1 0

30
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 2 1 1 0 0 0 7
2 1 2 0 1 0 0 8
3 1 -1 0 0 1 0 2
4 -3 -2 0 0 0 1 0
• Select column 1: the corresponding element at the last row is minimal
• Compute evaluation (column E)

x1 x2 x3 x4 x5 z RHS E
1 2 1 1 0 0 0 7 7/2
2 1 2 0 1 0 0 8 8/1
3 1 -1 0 0 1 0 2 2/1
4 -3 -2 0 0 0 1 0

31
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 2 1 1 0 0 0 7
2 1 2 0 1 0 0 8
3 1 -1 0 0 1 0 2
4 -3 -2 0 0 0 1 0
• Select column 1: the corresponding element at the last row is minimal
• Compute evaluation (column E)
• Select row R3: evaluation is minimal
• Update R3 = R3/1

x1 x2 x3 x4 x5 z RHS E
1 2 1 1 0 0 0 7 7/2
2 1 2 0 1 0 0 8 8/1
3 1 -1 0 0 1 0 2 2/1
4 -3 -2 0 0 0 1 0

32
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 2 1 1 0 0 0 7
2 1 2 0 1 0 0 8
3 1 -1 0 0 1 0 2
4 -3 -2 0 0 0 1 0
• Select column 1: the corresponding element at the last row is minimal
• Compute evaluation (column E)
• Select row R3: evaluation is minimal
• Update R3 = R3/1
• R1 = R1 – 2R3; R2 = R2 – R3; R4 = R4 + 3R3;
x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6

33
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6

34
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6
• Select column 2: the corresponding element at the last row is minimal

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6

35
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6
• Select column 2: the corresponding element at the last row is minimal
• Compute evaluations: column E

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3 3/3
2 0 3 0 1 -1 0 6 6/3
3 1 -1 0 0 1 0 2 +
4 0 -5 0 0 3 1 6

36
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6
• Select column 2: the corresponding element at the last row is minimal
• Compute evaluations: column E
• Select row R1: minimum evaluation

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3 3/3
2 0 3 0 1 -1 0 6 6/3
3 1 -1 0 0 1 0 2 +
4 0 -5 0 0 3 1 6

37
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6
• Select column 2: the corresponding element at the last row is minimal
• Compute evaluations: column E
• Select row R1: minimum evaluation
• Update R1 = R1/3

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6

38
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 3 1 0 -2 0 3
2 0 3 0 1 -1 0 6
3 1 -1 0 0 1 0 2
4 0 -5 0 0 3 1 6
• Select column 2: the corresponding element at the last row is minimal
• Compute evaluations: column E
• Select row R1: minimum evaluation
• Update R1 = R1/3
• R2 = R2 – 3R1; R3 = R3 + R1; R4 = R4 + 5R1

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11

39
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11

40
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11
• Select column 5: the corresponding element at the last row is minimal

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11

41
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11
• Select column 5: the corresponding element at the last row is minimal
• Compute evaluations: column E

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1 +
2 0 0 -1 1 1 0 3 3
3 1 0 1/3 0 1/3 0 3 9
4 0 0 5/3 0 -1/3 1 11

42
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11
• Select column 5: the corresponding element at the last row is minimal
• Compute evaluations: column E
• Select row 2: minimum evaluation

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1 +
2 0 0 -1 1 1 0 3 3
3 1 0 1/3 0 1/3 0 3 9
4 0 0 5/3 0 -1/3 1 11

43
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11
• Select column 5: the corresponding element at the last row is minimal
• Compute evaluations: column E
• Select row 2: minimum evaluation
• Update: R2 = R2/1

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11

44
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11
• Select column 5: the corresponding element at the last row is minimal
• Compute evaluations: column E
• Select row 2: minimum evaluation
• Update: R2 = R2/1
• R1 = R1 + (2/3)R2; R3 = R3 – (1/3)R2; R4 = R4+(1/3)R2

x1 x2 x3 x4 x5 z RHS E
1 0 1 -1/3 2/3 0 0 3
2 0 0 -1 1 1 0 3
3 1 0 2/3 -1/3 0 0 2
4 0 0 4/3 1/3 0 1 12

45
SIMPLEX METHOD: Tabular form

x1 x2 x3 x4 x5 z RHS E
1 0 1 1/3 0 -2/3 0 1
2 0 0 -1 1 1 0 3
3 1 0 1/3 0 1/3 0 3
4 0 0 5/3 0 -1/3 1 11
• Select column 5: the corresponding element at the last row is minimal
• Compute evaluations: column E
• Select row 2: minimum evaluation
• Update: R2 = R2/1
• R1 = R1 + (2/3)R2; R3 = R3 – (1/3)R2; R4 = R4+(1/3)R2

x1 x2 x3 x4 x5 z RHS E
1 0 1 -1/3 2/3 0 0 3 • Optimal solution: x1 = 2, x2 = 3, x3 = 0, x4 = 0, x5 = 3.
• Value of the objective function: 12
2 0 0 -1 1 1 0 3
3 1 0 2/3 -1/3 0 0 2
4 0 0 4/3 1/3 0 1 12

46
Exercises

• Example
f(x1, x2) = 3x1 + 2x2 → max
2x1 + x2 ≤ 7
x1 + 2x2 ≤ 8
x1 - x2 ≤ 2
x1, x2  0

47
Exercise

48
Exercise

49
CONTENT

• Linear programs

• Geometric approach

• Simplex method

• Two-phase simplex method

• OR-TOOLS for linear programming

• Programming exercises

50
 TWO-PHASE SIMPLEX METHOD
• Consider a linear program under a standard equational form
(LP) z = c1x1 + c2x2 + . . . + cnxn → max
a1,1x1 + a1,2x2 + . . . + a1,nxn = b1
a2,1x1 + a2,2x2 + . . . + a2,nxn = b2
...
am,1x1 + am,2x2 + . . . + am,nxn = bm
Coefficients b1, b2, . . . bm  0
x1, x2, . . . xn R, x1, x2, . . . xn  0

51
 TWO-PHASE SIMPLEX METHOD

• The two-phase Simplex Method is used when the initial feasible

solution is not obvious or difficult to find — particularly in cases where:
• The problem has "≥" (greater than or equal to) constraints.

• The problem has "=" (equality) constraints.

• The initial simplex table has no obvious feasible starting point.

52
 TWO-PHASE SIMPLEX METHOD
• Consider a linear program under a standard equational form
(LP) z = c1x1 + c2x2 + . . . + cnxn → max
Introduce an auxiliary linear program (ALP) with m
a1,1x1 + a1,2x2 + . . . + a1,nxn = b1
artificial variables y1, y2, . . ., ym
a2,1x1 + a2,2x2 + . . . + a2,nxn = b2
...
am,1x1 + am,2x2 + . . . + am,nxn = bm (ALP) g = -y1 - y2 - . . . - ym → max
Coefficients b1, b2, . . . bm  0 a1,1x1 + a1,2x2 + . . . + a1,nxn + y1 = b1
x1, x2, . . . xn R, x1, x2, . . . xn  0 a2,1x1 + a2,2x2 + . . . + a2,nxn + y2 = b2
... ...
am,1x1 + am,2x2 + . . . + am,nxn + +ym = bm
b1, b2, . . . bm  0
x1, x2, . . . xn R, x1, x2, . . . xn, y1, y2, …, ym  0

53
TWO-PHASE SIMPLEX METHOD

(ALP) g = -y1 - y2 - . . . - ym → max

a1,1x1 + a1,2x2 + . . . + a1,nxn + y1 = b1
a2,1x1 + a2,2x2 + . . . + a2,nxn + y2 = b2
...
am,1x1 + am,2x2 + . . . + am,nxn + +ym = bm
b1, b2, . . . bm  0
x1, x2, . . . xn R, x1, x2, . . . xn, y1, y2, …, ym  0

• Solve the (ALP) by the Simplex Method under the Tabular form: Basis is the column vectors
corresponding to artificial variables → obtain an optimal solution (x*, y*) and basic indices set is JB*
• Proposition: The original (LP) problem has feasible solutions iff the corresponding (ALP) has an
optimal solution (x*, y*) in which y* = 0 (proof ?)
• If y*  0, then the original (LP) problem does not have feasible solutions
• We consider the case that y* = 0

54
TWO-PHASE SIMPLEX METHOD
1 2 … n n+1 n+2 … n+m n+m+1
0 x1 x2 … xn y1 y2 … Ym z RHS
1 1,1 1,2 … 1,n 1,n+1 1,n+2 … 1,n+m 0 1
2 2,1 2,2 … 2,n 2,n+1 2,n+2 … 2,n+m 0 2
… … … … … … ... … … .. …
m m,1 m,2 … m,n m,n+1 m,n+2 … m,n+m 0 m
m+1 m+1,1 m+1,2 … m+1,n m+1,n+1 m+1,n+2 … m+1,n+m 1 m+1

• Case 1: JB* (set of indices in the basic) does not contain indices of artificial variables
• Move to the second phase, solve the original (LP) problem
• Remove columns corresponding to artificial variables: n+1, . . ., n+m
• Recompute elements on row m+1 (based on the original objective function)

55
TWO-PHASE SIMPLEX METHOD
1 2 … n n+1 n+2 … n+m n+m+1
0 x1 x2 … xn Y1 y2 … Ym z RHS
1 1,1 1,2 … 1,n 1,n+1 1,n+2 … 1,n+m 0 1
2 2,1 2,2 … 2,n 2,n+1 2,n+2 … 2,n+m 0 2
… … … … … … ... … … .. …
m m,1 m,2 … m,n m,n+1 m,n+2 … m,n+m 0 m
m+1 m+1,1 m+1,2 … m+1,n m+1,n+1 m+1,n+2 … m+1,n+m 1 m+1

• Case 2: JB* contains some indices of artificial variables

• Suppose JB* contains index n+j of the artificial variable (yj), perform the linear transformation
to remove index n+j from JB* as follows:
• Consider row k such that the element in row k and column n+j is 1 (column vector
corresponding to column n+j is a unit vector)
• Case 2.1: If all elements on row k, from column 1 to column n are equal to 0 (k,1 = . . . =
k,n = 0): it means, the constraint of row k is linear dependent on other constraints → we
can remove this row k and column n+j from the table

56
TWO-PHASE SIMPLEX METHOD
1 2 … n n+1 n+2 … n+m n+m+1
0 x1 x2 … xn y1 y2 … Ym z RHS
1 1,1 1,2 … 1,n 1,n+1 1,n+2 … 1,n+m 0 1
2 2,1 2,2 … 2,n 2,n+1 2,n+2 … 2,n+m 0 2
… … … … … … ... … … .. …
m m,1 m,2 … m,n m,n+1 m,n+2 … m,n+m 0 m
m+1 m+1,1 m+1,2 … m+1,n m+1,n+1 m+1,n+2 … m+1,n+m 1 m+1
• Case 2: JB* contains some indices of artificial variables
• Case 2.2: There exists a column i such that k,i  0
• In this optimal table, all artificial variables are equal to 0, so k is equal to 0
• Perform the rotation with the pivot k,i . With this rotation, column RHS is unchanged due
to the fact that k is equal to 0. Hence m+1 is always 0. It means that the new table
corresponds to another optimal solution in which one artificial variable is replaced by an
original variable xi
• The above procedure is repeated until all artificial variables are removed from the basic
• We now process the computation as the case 1 (above)
57
EXAMPLE

(LP) z = 40x1 + 10x2 + 7x5 + 14x6 → max (ALP) z = -y1 - y2 - y3 – y4→ max
x1 - x2 + 2x5 =0 x1 - x2 + 2x5 + y1 =0
-2x1 +x2 - 2x5 =0 -2x1 +x2 - 2x5 +y2 =0
x1 +x3 + x5 – x6 = 3 x1 +x3 + x5 – x6 +y3 =3
x2 + x3 + x4 + 2x5 + x6 = 4 x2 + x3 + x4 + 2x5 + x6 +y4 =4
x1, x2, x3, x4, x5, x6  R, x1, x2, x3 , x4, x5, x6  0 x1, x2, x3, x4, x5, x6,y1, y2, y3, y4  R, x1, x2, x3 , x4, x5, x6 ,y1, y2, y3, y4  0

1 2 3 4 5 6 7
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS
1 1 -1 0 0 2 0 1 0 0 0 0 0
2 -2 1 0 0 -2 0 0 1 0 0 0 0
3 1 0 1 0 1 -1 0 0 1 0 0 3
0 1 1 1 2 1 0 0 0 1 0 4
4 0 -1 -2 -1 -3 0 0 0 0 0 1 -7

58
EXAMPLE
1 2 3 4 5 6 7
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS E
1 1 -1 0 0 2 0 1 0 0 0 0 0 0
2 -2 1 0 0 -2 0 0 1 0 0 0 0 +
3 1 0 1 0 1 -1 0 0 1 0 0 3 3/1
4 0 1 1 1 2 1 0 0 0 1 0 4 4/2
5 0 -1 -2 -1 -3 0 0 0 0 0 1 -7
R1 = R1/2; R2 = R2 + 2R1; R3 = R3 – R1; R4 = R4 – 2R1; R5 = R5 + 3R1
1 2 3 4 5 6 7
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS
1 1/2 -1/2 0 0 1 0 1/2 0 0 0 0 0
2 -1 0 0 0 0 0 1 1 0 0 0 0
3 1/2 1/2 1 0 0 -1 -1/2 0 1 0 0 3
4 -1 2 1 1 0 1 -1 0 0 1 0 4
5 3/2 -5/2 -2 -1 0 0 3/2 0 0 0 1 -7
59
EXAMPLE
1 2 3 4 5 6 7
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS E
1 1/2 -1/2 0 0 1 0 1/2 0 0 0 0 0 +
2 -1 0 0 0 0 0 1 1 0 0 0 0 +
3 1/2 1/2 1 0 0 -1 -1/2 0 1 0 0 3 6
4 -1 2 1 1 0 1 -1 0 0 1 0 4 4/2
5 3/2 -5/2 -2 -1 0 0 3/2 0 0 0 1 -7
R4 = R4/2; R1 = R1+(1/2)R4; R2 = R2; R3 = R3 – (1/2)R4; R5 = R5 + (5/2)R4
1 2 3 4 5 6 7
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS
1 1/4 0 1/4 1/4 1 1/4 1/4 0 0 1/4 0 1
2 -1 0 0 0 0 0 1 1 0 0 0 0
3 3/4 0 3/4 -1/4 0 -5/4 -1/4 0 1 -1/4 0 2
4 -1/2 1 1/2 ½ 0 1/2 -1/2 0 0 1/2 0 2
5 1/4 0 -3/4 ¼ 0 5/4 1/4 0 0 5/4 1 -2
60
EXAMPLE
1 2 3 4 5 6 7
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS E
1 1/4 0 1/4 1/4 1 1/4 1/4 0 0 1/4 0 1 4
2 -1 0 0 0 0 0 1 1 0 0 0 0 +
3 3/4 0 3/4 -1/4 0 -5/4 -1/4 0 1 -1/4 0 2 8/3
4 -1/2 1 1/2 ½ 0 1/2 -1/2 0 0 1/2 0 2 4
5 1/4 0 -3/4 ¼ 0 5/4 1/4 0 0 5/4 1 -2
R3 = R3/(3/4); R1 = R1-(1/4)R3; R2 = R2; R4 = R4 – (1/2)R3; R5 = R5 + (3/4)R3
1 2 3 4 5 6 7
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS
1 0 0 0 1/3 1 2/3 1/3 0 -1/3 1/3 0 1/3
2 -1 0 0 0 0 0 1 1 0 0 0 0
3 1 0 1 -1/3 0 -5/3 -1/3 0 4/3 -1/3 0 8/3
4 -1 1 0 2/3 0 4/3 -1/3 0 -2/3 2/3 0 2/3
5 1 0 0 0 0 0 0 0 1 1 1 0
61
EXAMPLE

1 2 3 4 5 6 7 8 9 10 11
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS E
1 0 0 0 1/3 1 2/3 1/3 0 -1/3 1/3 0 1/3
2 -1 0 0 0 0 0 1 1 0 0 0 0
3 1 0 1 -1/3 0 -5/3 -1/3 0 4/3 -1/3 0 8/3
4 -1 1 0 2/3 0 4/3 -1/3 0 -2/3 2/3 0 2/3
5 1 0 0 0 0 0 0 0 1 1 1 0
The basic index set JB* = {2, 3, 5, 8}, where column 8 corresponds to the artificial
variable y2. In this column, the element corresponding to row R2 is 1. This row has an
RHS of 0 (because in the optimal solution, this RHS value equals y2, which is 0).
Additionally, there is an element in column 1 (the column corresponding to the original
variable) equal to -1 (which is nonzero), so we perform a pivot operation on this
element (row 2, column 1), specifically:
• R2 = R2/ (-1)
• R1 = R1; R3 = R3 – R2; R4 = R4 + R2; R = R5 – R2

62
EXAMPLE
1 2 3 4 5 6 7 8 9 10 11
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS E
1 0 0 0 1/3 1 2/3 1/3 0 -1/3 1/3 0 1/3
2 -1 0 0 0 0 0 1 1 0 0 0 0
3 1 0 1 -1/3 0 -5/3 -1/3 0 4/3 -1/3 0 8/3
4 -1 1 0 2/3 0 4/3 -1/3 0 -2/3 2/3 0 2/3
5 1 0 0 0 0 0 0 0 1 1 1 0
• Pivot row 2, column 1: R2 = R2/ (-1)
• R1 = R1; R3 = R3 – R2; R4 = R4 + R2; R5 = R5 – R2
1 2 3 4 5 6 7 8 9 10 11
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS
1 0 0 0 1/3 1 2/3 1/3 0 -1/3 1/3 0 1/3
2 1 0 0 0 0 0 -1 -1 0 0 0 0
3 0 0 1 -1/3 0 -5/3 2/3 1 4/3 -1/3 0 8/3
4 0 1 0 2/3 0 4/3 -4/3 -1 -2/3 2/3 0 2/3
5 0 0 0 0 0 0 1 1 1 1 1 0 63
EXAMPLE

1 2 3 4 5 6 7 8 9 10 11
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS E
1 0 0 0 1/3 1 2/3 1/3 0 -1/3 1/3 0 1/3
2 1 0 0 0 0 0 -1 -1 0 0 0 0
3 0 0 1 -1/3 0 -5/3 2/3 1 4/3 -1/3 0 8/3
4 0 1 0 2/3 0 4/3 -4/3 -1 -2/3 2/3 0 2/3
5 0 0 0 0 0 0 1 1 1 1 1 0

• We obtain the optimal solution for the first phase: no artificial variables are basic
variables. Therefore, we eliminate the columns corresponding to artificial variables
and proceed to the second phase. We retain the coefficients in the table (rows 1–4
and columns 1–6) and use the objective function of the original problem to
recalculate row 5.

64
EXAMPLE
1 2 3 4 5 6 7 8 9 10 11
0 x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 z RHS E
1 0 0 0 1/3 1 2/3 1/3 0 -1/3 1/3 0 1/3
2 1 0 0 0 0 0 -1 -1 0 0 0 0
3 0 0 1 -1/3 0 -5/3 2/3 1 4/3 -1/3 0 8/3
4 0 1 0 2/3 0 4/3 -4/3 -1 -2/3 2/3 0 2/3
5 0 0 0 0 0 0 1 1 1 1 1 0

1 2 3 4 5 6 7
0 x1 x2 x3 x4 x5 x6 z RHS
1 0 0 0 1/3 1 2/3 0 1/3
2 1 0 0 0 0 0 0 0
3 0 0 1 -1/3 0 -5/3 0 8/3
4 0 1 0 2/3 0 4/3 0 2/3
5 0 0 0 9 0 4 1 9
65
TWO-PHASE SIMPLEX METHOD

• Example 1 (exercise in class) (LP) z = x1 + 2x2 – x3 + x4 → max

x1 + x2 - x3 - x4 = 4
x1 + x3 + x4 = 7
2x1 + x2 =2
x1, x2, x3, x4  R, x1, x2, x3 , x4  0

66
TWO-PHASE SIMPLEX METHOD

• Example 2 (exercise in class) (LP) z = x1 + 2x2 – x3 + x4 → max

x1 + x2 - x3 - x4 = 4
x1 + x3 + x4 = 7
x1 - x2 - x3 =2
x1, x2, x3, x4  R, x1, x2, x3 , x4  0

67
TWO-PHASE SIMPLEX METHOD

• Example 3 (exercise in class) (LP) z = 40x1 + 10x2 + 7x5 + 14x6 → max

x1 - x2 + 2x5 =0
-2x1 +x2 - 2x5 =0
x1 +x3 + x5 – x6 = 3
x2 + x3 + x4 + 2x5 + x6 = 4
x1, x2, x3, x4, x5, x6  R, x1, x2, x3 , x4, x5, x6  0

68
CONTENT

• Linear programs

• Geometric approach

• Simplex method

• Two-phase simplex method

• OR-TOOLS for linear programming

• Programming exercises

69
INSTALL

• https://developers.google.com/optimization/install

70
 Example

Maximize 3x + 4y subject to the following constraints:

x + 2y ≤ 14
3x - y ≥ 0
x-y≤2
To solve a LP problem, your program should include the
following steps:

1.Import the linear solver wrapper,

2.declare the LP solver,

3.define the variables,

4.define the constraints,

5.define the objective,

6.call the LP solver; and

7.display the solution

71
Example

72
CONTENT

• Linear programs

• Geometric approach

• Simplex method

• Two-phase simplex method

• OR-TOOLS for linear programming

• Programming exercises

73
THANK YOU !

74