Advanced Mathematics (CIVL7001)

Lecture 9. Constrained Optimization

1. Introduction

2. Linear Constrained Optimization

2.1 Standard Form of Linear Programming

2.2 Graphical Solution

2.3 The Simplex Method

1
 1. Introduction

The procedure for finding the values of the variables

in the objective function that lead to maximum value
of the objective function is called constrained op-
timization if there are constrains on the variables or
conditions to be satisfied by the variables.

2. Linear Constrained Optimization

If the objective function is a linear function of the vari-

ables and the constrains are linear inequalities in terms
of the variables, the constrained optimization problem
is called linear constrained optimization or lin-
ear programming (LP).

The LP problem can be found in the optimizations of

linear systems.

2
 2.1 Standard Form of Linear Programming

The LP problem consists of two major parts, i.e., the

objective function and a set of constrains.

For maximization problem, the objective function is

generally expressed as
n
max Z = cixi (1)
X

i=1

where Z is the objective function, xi is the ithe variable

in the function and ci is the coefficient of xi.

The constrains can be generally expressed as

n
aij xj ≤ bi (i = 1, 2, ..., m) (2)
X

j=1

where aij is the coefficient of xj and bi is a constant in

the ith constrain inequality.

In many cases, the variables are constrained to be

positive or

xi ≥ 0 (i = 1, 2, ..., n) (3)

3
 The LP problem can be generally expressed as
Pn







max Z = i=1 ci xi


Pn

≤ bi



 j=1 aij xj (i = 1, 2, ..., m) (4)


xi ≥ 0

(i = 1, 2, ..., n)





2.1.1 Formulation of LP problem

Example 1. Suppose that a gas processing plant re-

ceived a fixed amount of raw gas each week. The raw
gas is processed into two grades of heating gas, i.e., reg-
ular and premium quality. These grades of gas are in
high demand (that is, they are guaranteed to sell) and
yield different profits to the company. However, their
production involves both time and on-site storage con-
strains. For example, only one of the grades can be
produced at a time, and the facility is open for only 80
hr/week. Further, there is limited on-site storage for
each of the products. All these factors are listed in the
following table. (Note that a metric ton, or tonne, is
equal to 1000kg).

Develop a LP formulation to maximize the profit of

this operation.

Solution: Denote the weight amount of regular and

4
Table 1.
Resource Regular Premium Available Resource
Raw gas 7m3 /ton 11m3 /ton 77m3 /week
Production time 10hr/ton 8hr/ton 80hr/week
Storage 9 ton 6 ton
Profit 150/ton 175/ton

premium produced weekly as x1 (ton/week) and x2

(ton/week), respectively, and the total profit as Z. Then
the total weekly profit can be calculated as

Z = 150x1 + 175x2

The total weekly volume amount of raw gas is 7x1 +11x2

(m3/week) which is limited by

7x1 + 11x2 ≤ 77
The total weekly production time is 10x1 + 8x2 which
is limited by

10x1 + 8x2 ≤ 80
The weekly storage limits are
x1 ≤ 9
and
x2 ≤ 6
The values of x1 and x2 must be nonnegative and there-
fore they are limited by x1 ≥ 0 and x2 ≥ 0.
5
Then the LP problem is formulated as








max Z = 150x1 + 175x2


7x1 + 11x2 ≤ 77 [1]










10x1 + 8x2 ≤ 80 [2]










x1 ≤ 9




[3] (5)

x2 ≤ 6

[4]










x1 ≥ 0 [5]










 x2 ≥ 0 [6]





Figure 1: The constraints define a feasible solution space.

6
 2.2 Graphical Solution

2.2.1 Unique solution

The maximum objective function intersects a single

point with the solution domain enclosed by the con-
strains.

Figure 2: The objective function can be increased until it reaches the highest value that obeys
all constraints.

2.2.2 Alternative solution

Corresponding to the maximum objective function,

there may be infinite number of solutions in some case.
In this case, the objective function is parallel to one of

7
the constrains.

Figure 3: The case of alternative optima or more than one optimum solutions

2.2.3 No feasible solution

In this case, it is over-constrained and there can be no

solution. This can happen when the problem is not well
formulated. The constrains have no common region in
this case.

2.2.4 Unbounded problem

In this case, it is under-constrained and the maximum

solution is unbounded. This can happen when the prob-
lem is not well formulated.

8
Figure 4: The case of no feasible solution

Figure 5: The case of an unbounded result

9
 • Consider the case when there is a unique solution.

Figure 6: Extreme points, feasible extreme points and non-feasible extreme points

A corner point where two (2 variables in the objective

function or 2D problem) or more (more variables in the
objective function or multi-dimensional problem) con-
strains meet is known as extreme point.

The corner point that satisfies all the constrains is

known as feasible extreme point. Otherwise, it
known as non-feasible extreme point.

For example, in Figure 6, the points A, B, C, D, and E

are feasible extreme points. The point F is non-feasible
extreme point.

The optimum solution always occurs at one of the

feasible extreme points.
10
 When all the feasible extreme points are identified,
the one yielding the maximum value of the objective
function is the optimum solution.

For high-dimensional problem, there can be huge num-

ber of feasible extreme points and the computational
effort needed by evaluating the function values at the
feasible extreme points can also be huge.

11
 2.3 The Simplex Method

The simplex method is a efficient method for finding

the optimum solution with minimum times of solving
linear equations.

2.3.1 Slack variables

For the following LP problem

max Z = Pni=1 cixi










Pn

≤ bi



 j=1 aij xj (i = 1, 2, ..., m) (6)


xj ≥ 0

(j = 1, 2, ..., n)





introduce the variables Si ≥ 0 (i = 1, 2, ..., m) such

that
n
aij xj + Si = bi (i = 1, 2, ..., m)
X

j=1

The variable Si is known as the ith slack variable

which makes the ith inequality become an equality.

After introducing the slack variables, the LP problem

becomes

12
 max Z = Pni=1 cixi









n

 P


 j=1 aij xj + Si = bi (i = 1, 2, ..., m)


xj ≥ 0 (j = 1, 2, ..., n); Si ≥ 0 (i = 1, 2, ..., m)






(7)
The variable xi is known as the ith structural vari-
able.

There are n structural variables and m slack vari-

ables, totally n+m variables, in the m linear equations.

2.3.2 Feasible points and feasible solutions

A point that satisfies the equations and the inequali-

ties in Eq. (7) is known as a feasible point and the
solution is known as a feasible solution.

A solution for the m linear equations with n + m un-

knowns can be obtained by setting n variables among
the n+m variables to zero, and solving the m equations
for the m remaining unknowns.

13
 Observations in Example 1
Feasible extreme point Zero variable
A x1 , x2
B x2 , S2
C S1 , S2
D S1 , S4
E x1 , S4

Figure 7: Feasible extreme points

It can be proved that the values of n variables among

the total n + m variables must equal zero for a feasible
solution.

The n zero variables are known as nonbasic vari-

ables and the remaining m variables are known as ba-
sic variables.

If all the basic variables are nonnegative, the result

is called a basic feasible solution because the so-
14
lution is constrained by xi ≥ 0, (i = 1, 2, ..., n), and
Si ≥ 0, (i = 1, 2, ..., m). The optimum solution is one
of them.

If n of the n+m variables are set to zero for solutions,

the total number of solutions is
n (n + m)!
Cn+m = (8)
n!m!
If n + m is great, the computational effort can be huge.

15
 2.3.3 Simplex method

The strategy of the simplex method is stated as fol-

lows. Start from a basic feasible solution. Then it moves
through a sequence of other basic feasible solutions that
improves the value of the objective function. Eventu-
ally, the optimal value is reached.

For example, consider the former gas-processing prob-

lem. 







max Z = 150x1 + 175x2
7x1 + 11x2 ≤ 77








 10x1 + 8x2 ≤ 80












x1 ≤ 9
x2 ≤ 6








 x1 ≥ 0, x2 ≥ 0





(1) Introducing the slack variables to the inequalities,

it gives








max Z = 150x1 + 175x2
7x1 + 11x2 + S1 = 77








10x1 + 8x2 + S2 = 80












x 1 + S3 = 9
x 2 + S4 = 6








 xi ≥ 0, (i = 1, 2); Si ≥ 0, (i = 1, 2, 3, 4)





16
 (2) Selection of start point

A start point can be selected as (x1, x2) = (0, 0) which

are the values of nonbasic variables. Then the basic
variables are Si ≥ 0 (i = 1, 2, 3, 4) which values can be
determined by the 4 equations to be (S1, S2, S3, S4) =
(77, 80, 9, 6). The equations and other information at
the first step are listed in the following Table 1.

Table 1.
Intercept with x1 -axis
Basic Z x1 x2 S1 S2 S3 S4 Solution when Si = 0
Z 1 -150 -175 0 0 0 0 0
S1 0 7 11 1 0 0 0 77 11
S2 0 10 8 0 1 0 0 80 8
S3 0 1 0 0 0 1 0 9 9
S4 0 0 1 0 0 0 1 6 ∞

Each row in the above table gives the coefficients in a

equation.

(3) Selection of entering variable

Increase a current nonbasic variable (x1 or x2) above

zero so that Z increases. It means that one of the cur-
rent nonbasic variables (x1 or x2) will become a basic
variable which is called entering variable. Mean-
17
Figure 8: Graphical depiction of how the simplex method successively moves through feasible
basic solutions to arrive at the optimum in an effi cient manner.

while one of the current basic variables (S1, S2, S3 or

S4) must be set to zero and will then becomes a nonba-
sic variable which is called leaving variable.

Anyone of the current nonbasic variables that can

increase the value of Z can be selected as the enter-
ing variable. It is seen that the current nonbasic vari-
ables with negative coefficient value can increase the Z
value. Then one of the current nonbasic vari-
ables with negative coefficient can be selected
as entering variable.

It is better to change the current nonbasic variable

that can increase the value of Z most to be the entering
variable. It is seen that the current nonbasic variable
with largest negative coefficient value can increase the Z
value most. Then the current nonbasic variable
18
with largest negative coefficient is better se-
lected as entering variable.

In the following analysis the x1 is selected as entering

variable.

(4) Selection of leaving variable

After the entering variable is selected and as it moves,

the line that has smallest positive (because xi ≥
0) intercept with the axis of entering variable can be
reached first as the value of entering variable increases.
Since the value of current slack variable cor-
responding to this line equals zero at the in-
terception point, this slack variable is selected
as the leaving variable.

As shown in the above table, the line with minimum

positive interception with x1 axis is the line correspond-
ing to S2. Then S2 is selected as the leaving variable
in the following analysis. The updated information are
listed in the following Table 2.

(5) Pivoting to remove the influence of entering vari-

able

19
Table 2.
Basic Z x1 x2 S1 S2 S3 S4 Solution
Z 1 -150 -175 0 0 0 0 0
S1 0 7 11 1 0 0 0 77
x1 0 10 8 0 1 0 0 80
S3 0 1 0 0 0 1 0 9
S4 0 0 1 0 0 0 1 6

Row operations can be used to remove the influence

of entering variable in all other equations except the
equation corresponding to the entering variable. Here
the entering variable is x1. The results are listed in the
following Table 3.

Table 3.
Intercept with x2 -axis
Basic Z x1 x2 S1 S2 S3 S4 Solution when Si = 0
Z 1 0 -55 0 15 0 0 1200
S1 0 0 5.4 1 -0.7 0 0 21 3.889 (min positive interception)
x1 0 1 0.8 0 0.1 0 0 8 10
S3 0 0 -0.8 0 -0.1 1 0 1 -1.25
S4 0 0 1 0 0 0 1 6 6

(6) The above procedure can be repeated until there

are no variables with negative coefficients in the objec-
tive function, which means the value of objective func-
tion can’t be further increased.

From Table 3, it is seen that the x2 can be selected

as the next entering variable and S1 can be selected as

20
the next leaving variable. The final results are listed in
the following Table 4. The solution is (x1, x2)=(4.889,
3.889) which gives Zmax = 1413.889.

Table 4.
Basic Z x1 x2 S1 S2 S3 S4 Solution
Z 1 0 0 10.1852 7.8704 0 0 1413.889
x2 0 0 1 0.1852 -0.1296 0 0 3.889
x1 0 1 0 -0.1481 0.2037 0 0 4.889
S3 0 0 0 0.1481 -0.2037 1 0 4.111
S4 0 0 0 -0.1852 0.1296 0 1 2.111

Note: The values of leaving variables (S1, S2 here) equal

zero because they are non-basic variables now.

Assignments 9: 15.3(a) and (b).

21