ECE 580 Optimization Methods for Systems and Control November 22, 2011 1

Slack, Surplus, and Free Variables

To convert an equation of the form

ai1 x1 + ai2 x2 + · · · + a1n xn ≤ bi

to standard form we introduce a slack variable yi to obtain

ai1 x1 + ai2 x2 + · · · + a1n xn + yi = bi .

To convert an equation of the form

ai1 x1 + ai2 x2 + · · · + a1n xn ≥ bi

to standard form we introduce a surplus variable yi to obtain

ai1 x1 + ai2 x2 + · · · + a1n xn − yi = bi .

To deal with a variable that is not constrained to be nonnegative we can use one of two
approaches. Suppose that xi is not constrained to be nonegative. In this case xi is called
a free variable.

1. Let
x i = ui − v i , where ui ≥ 0 and vi ≥ 0.
Then replace xi by ui − vi everywhere that it occurs in the objective function and
the rows of Ax = b. Note that this method introduces redundancy.

2. Select a row in which xi has a nonzero coefficient and solve it for xi . Then substitute
the expression obtained for xi everywhere it occurs in the objective function and
the rows of Ax = b. Note that the ith equation becomes trivial, namely, bi = bi .

Example
Put the LP problem

min x1 + 3x2 + 4x3

subject to x1 + 2x2 + x3 = 5
2x1 + 3x2 + x3 = 6
x2 ≥ 0
x3 ≥ 0
ECE 580 Optimization Methods for Systems and Control November 22, 2011 2

in standard form. Since x1 is a free variable we can use either of the two free variable
methods.
Method 1:
Introduce the two new nonegative variables u1 and v1 to obtain
x 1 = u1 − v 1 . (1)
Substitute for x1 everywhere. The objective function becomes f (x) = cT x where
 T
c = 3 4 1 −1 (2)
 T
x = x2 x3 u v (3)
Substituting into the constraints we obtain
u1 − v1 + 2x2 + x3 = 52(u1 − v1 ) + 3x + 2 + x + 3 = 6. (4)
Thus we have the standard form LP
min cT x
subject to Ax = b
x≥0

where
 
1 −1 2 1
A = (5)
2 −2 3 1
 T
b = 5 6 (6)
 T
c = 3 4 1 −1 (7)
 T
x = x2 x3 u v . (8)

Method 2:
Solve one of the constraint equations for x1 and substitute it into the objective function
and the remaining constraint equations. We’ll solve the first for x1 to obtain
x1 = 5 − 2x2 − x3 , (9)
substitute it into the objective function to obtain
5 − 2x2 − x3 + 3x2 + 4x3 = x2 + 3x3 + 5, (10)
and substitute into the second to obtain from
2(5 − 2x2 − x3 ) + 3x2 + x3 = 6, (11)
−x2 − x3 = −4. (12)
Note that minimizing x2 + 3x3 + c for any scalar c is equivalent to minimizing x2 + 3x3
so we can state the objective as minimizing x2 + 3x3 . Note also that we are supposed to
have b ≥ 0, so we need to multiply (??) by −1. Thus we have the LP in standard form
ECE 580 Optimization Methods for Systems and Control November 22, 2011 3

min x2 + 3x3
subject to x2 + x3 = 4
x2 ≥ 0
x3 ≥ 0

or in matrix form

min cT x
subject to Ax = b
x≥0

where
 
A = 1 1 (13)
b = 4 (14)
 T
c = 1 3 (15)
 T
x = x2 x3 . (16)

We note that the second method gives us a more efficient statement of the LP.

Two Ways to solve Example 15.11

Problem Statement: Convert the optimization problem

maximize x2 − x1
subject to 3x1 = x2 − 5
|x2 | ≤ 2
x1 ≤ 0

to a linear program in standard form.

Solution 1 (presented in class):

1. In order to state the problem as a minimization problem we must replace f (x) by

−f (x).
2. In order to be able to write the final constraint as x ≥ 0 we must replace x1 by
x̃1 = −x1 . Thus we have f˜(x) = −x̃1 − x2 .
3. After substituting −x̃1 for x1 in the first constraint, we have −x̃1 − x2 = −5. Note
that this equation is already an equality constraint so we do not need to uintroduce
any new variables. However, it is customary to require b ≥ 0 so we will multiply
this equation by −1 to obtain x̃1 + x2 = 5.
ECE 580 Optimization Methods for Systems and Control November 22, 2011 4

4. Replace |x2 | ≤ 2 by two inequalities x2 ≤ 2 and −2 ≤ x2 . We insert a slack variable

in the first inequality to obtain x2 + y1 = 2 where y1 ≥ 0. We insert a surplus
variable in the second to obtain x2 − y2 = −2, where y2 ≥ 0. Again, to obtain b ≥ 0
we multiply this second equation by −1. So far we have the matrices
 
1 1 0 0
A =  0 1 1 0  (17)
0 −1 0 1
 
b = 5 2 2 (18)
 T
c = −1 −10 0 (19)
 T
x = x˜1 x2 y1 y2 ; (20)

however, we do not have x2 ≥ 0. So we need one more step.

5. We are left with free variable x2 . To eliminate it we solve one of our equations for
x2 and substitute for x2 in the others.
From the last equation we have x2 = y2 − 2. Then the first equation becomes
x̃1 + y2 = 7. The second equation becomes y2 + y1 = 4 and the third becomes
trivial. The objective function becomes f (x) = −x̃1 −(y2 −2). However, minimizing
f (x) + α for a scalar α is equivalent to minimizing f (x) so we can write simply
f (x) = −x̃1 − y2 .

We now have

min cT x
subject to Ax = b
x≥0

where
 
1 0 1
A = (21)
0 1 1
 
b = 7 4 (22)
 T
c = −1 0 −1 (23)
 T
x = x˜1 y1 y2 . (24)

Solution 2 (details following the outline in the tex):

Note that steps 1 through 4 are identical to the first method shown above.
In step 5, instead of eliminating the free variable x2 , the authors introduce two new
nonnegative variables u and v and take x2 = u − v. Substituting for x2 everywhere then
yields
ECE 580 Optimization Methods for Systems and Control November 22, 2011 5

min cT x
subject to Ax = b
x≥0

where
 
3 0 0 1 −1
A =  0 1 0 1 −1  (25)
0 0 1 −1 1
 
b = 5 2 2 (26)
 T
c = −1 0 0 −1 1 (27)
 T
x = x˜1 y1 y2 u v , (28)

where our y1 is the textbook’s x3 and our y2 is the textbook’s x4 .