QUADRATIC

PROGRAMMING
 Quadratic Programming
A quadratic programming problem is a non-linear
programming problem of the form

Maximize z  c X  X DX T

 
Subject to A X  b, X  0
  x1   b1 
Here x  b 
 2   2  
X   .  , b   .  , c  c1 c2 . . . cn 
   
  . .
 xn  bm 

 a11 a12 . . a1n   d11 d12 . . d1n 

a a . . a2 n  d d . . d 2 n 
 21 22  21 22
A .  D . 
   
 .   . 
am1 am 2 . . amn  d n1 d n 2 . . d nn 
We also assume that the nn matrix D is
symmetric and negative-definite.
This means that the objective function is
strictly concave.
Since the constraints are linear, the feasible
region is a convex set.
In scalar notation, the quadratic programming
problem reads:
n n
Maximize z  c x  d
j 1
j j
j 1
x 2
2
jj j  d x x
1i j n
ij i j

a11 x1  a12 x2  . . .  a1n xn  b1

Subject to
a21 x1  a22 x2  . . .  a2 n xn  b2
.
.
am1 x1  am 2 x2  . . .  amn xn  bm
x1 , x2 ,. . . , xn  0
The solution to this problem is based on the
KKT conditions. Since the objective function is
strictly concave and the solution space is
convex, the KKT conditions are also sufficient
for optimum.
Since there are m+n constraints, we have m+n
Lagrange multipliers; the first m of them are
denoted by 1, 2, …, m; and the last n of them
are denoted by 1, 2, …, n.
The KKT (necessary) conditions are:

1. 1 , 2 , . . . , m , 1 , 2 , . . . , n  0
n n
2. c j  2 d ij xi   i aij   j  0
i 1 i 1

j  1, 2, . . . , n

(continued …)
  n 
3. i   aij x j  bi   0, i  1, 2,. . ., m
 j 1 
 j x j  0, j  1, 2,. . ., n
n
4. a x
j 1
ij j  bi , i  1, 2,. . ., m

x j  0, j  1, 2,. . ., n
Denoting the (non-ve) slack variable for the
ith constraint n

a
j 1
ij x j  bi

by Si, the 3rd condition(s) can be written in

an equivalent form as:
3. i Si  0, i  1, 2, . . . , m
 j x j  0 , j  1, 2, . . . , n
Referred to as " Restricted Basis " conditions.
Also condition(s) (2) can be rewritten as:
n n
2.  2 dij xi   i aij   j  c j
i 1 i 1

j  1, 2,. . ., n
And condition(s) (4) can be rewritten as:
n
4. a x
j 1
ij j  Si  bi , i  1, 2,. . ., m

x j  0, j  1, 2,. . ., n
 * * *
Thus we find the optimal solution x , x ,. . ., x
1 1 n

is a solution of the system of m+n linear

equations in the n+m+n unknowns x j , i ,  j
n n
 2 d ij xi   i aij   j  c j , j  1, 2, . . . , n
i 1 i 1

a
j 1
ij x j  Si  bi , i  1, 2, . . . , m

along with the " Restricted Basis " conditions

i Si  0, i  1, 2, . . . , m,
 j x j  0 , j  1, 2, . . . , n
Since we are interested only in a "
feasible solution " of the above system
of linear equations, we use Phase-I
method to find such a feasible solution.
By the sufficiency of the KKT
conditions, it will be automatically the
optimum solution of the given
quadratic programming problem.
Problem
Maximize z  8 x1  x  2 x2  x3
2
1

subject to x1  3x2  2 x3  12
x1 , x2 , x3  0
Denoting the Lagrange multipliers by
1,1, 2, and 3, the Lagrangean is:

L  8 x1  x  2 x2  x3
2
1

1 ( x1  3x2  2 x3  12)

 1 ( x1 )  2 ( x2 )  3 ( x3 )
i.e. L  8 x1  x  2 x2  x3
2
1

1 ( x1  3x2  2 x3  12)

 1 x1  2 x2  3 x3
Hence the KKT conditions are:
1. 1 , 1 , 2 , 3  0
L
2.  8  2 x1  1  1  0
x1
L
2  31   2  0
x2
L
1  21  3  0
x3
i.e. 2 x1  1  1 8
31   2  2
(Continued)
21  3  1
3. x1  3x2  2 x3  S1  12,
1S1  0
1 x1  2 x2  3 x3  0
All variables  0.
Solving this by " Modified Simplex Algorithm ",
the optimal solution is:
11 25
x1  , x2  , x3  0
3 9
193
and the optimal z =
9
Basic r x1 x2 x3 1 1 2 3 R1 R2 R3 S1 Sol
2 0 0 6 -1 -1 -1 0 0 0 0 11
r 1 0 0 0 0 0 0 0 -1 -1 -1 0 0
R1 0 2 0 0 1 -1 0 0 1 0 0 0 8
R2 0 0 0 0 3 0 -1 0 0 1 0 0 2
R3 0 0 0 0 2 0 L 0

-1 0 0 1 0 1
x2
S1 0 1 3 2 0 0 0 0 0 0 0 1 12

Now if 1 enters, R3 will only leave.

Since 1 S1 = 0 and S1 is in the basis, 1 cannot enter.

So we allow x1 to enter the basis and of course by
minimum ratio test R1 leaves the basis.
Basic r x1 x2 x3 1 1 2 3 R1 R2 R3 S1 Sol

r 1 0 0 0 5 0 -1 -1 -1 0 0 0 3

x1 0 1 0 0 1/2 -1/2 0 0 1/2 0 0 0 4

R2 0 0 0 0 3 0 -1 0 0 1 0 0 2
R3 0 0 0 0 2 0 0 -1 0 0 1 0 1
S1 0 0 3 2 -1/2 1/2 0 0 -1/2 0 0 1 8

Since 1 S1 = 0 and S1 is in the basis, 1 cannot enter.

So we allow x2 to enter the basis and of course by
minimum ratio test S1 leaves the basis.
Basic r x1 x2 x3 1 1 2 3 R1 R2 R3 S1 Sol

r 1 0 0 0 5 0 -1 -1 -1 0 0 0 3

x1 0 1 0 0 1/2 -1/2 0 0 1/2 0 0 0 4

R2 0 0 0 0 3 0 -1 0 0 1 0 0 2
R3 0 0 0 0 2 0 0 -1 0 0 1 0 1
x2 0 0 1 2/3 -1/6 1/6 0 0 -1/6 0 0 1/3 8/3

As S1 is not in the basis, now 1 enters the basis .

And by minimum ratio test R3 leaves the basis.
Basic r x1 x2 x3 1 1 2 3 R1 R2 R3 S1 Sol

r 1 0 0 0 0 0 -1 3/2 -1 0 -5/2 0 1/2

x1 0 1 0 0 0 -1/2 0 1/4 1/2 0 -1/4 0 15/4

R2 0 0 0 0 0 0 -1 3/2 0 1 -3/2 0 1/2
1 0 0 0 0 1 0 0 -1/2 0 0 1/2 0 1/2
x2 0 0 1 2/3 0 1/6 0 -1/12 -1/6 0 1/12 1/3 11/4

Now 3 enters the basis .

And by minimum ratio test R2 leaves the basis.
Basic r x1 x2 x3 1 1 2 3 R1 R2 R3 S1 Sol

r 1 0 0 0 0 0 0 0 -1 -1 -1 0 0

x1 0 1 0 0 0 -1/2 1/6 0 1/2 -1/6 0 0 11/3

3 0 0 0 0 0 0 -2/3 1 0 2/3 -1 0 1/3
1 0 0 0 0 1 0 -1/3 0 0 1/3 0 0 2/3
x2 0 0 1 2/3 0 1/6 -1/18 0 -1/6 1/18 0 1/3 25/9

This is the end of Phase I.

Thus the optimal solution is: Thus the optimal
11 25 value z is: 193
x1  , x2  , x3  0
3 9 9
Problem

Maximize z  8x1  4 x2  x  x
2
1
2
2

Subject to x1  x2  2
x1 , x2  0
Denoting the Lagrange multipliers by 1, 1,
and 2, the KKT conditions are:
1. 1, 1, 2  0
2. 8  2 x1  1  1  0
4  2 x2  1  2  0
i.e. 2 x1  1  1 8
2 x2  1  2  4
3. x1  x2  S1  2, 1S1  1x1  2 x2  0
All variables  0.
Introducing artificial variables R1, R2, we thus
have to
Minimize r  R1  R2
subject to the constraints
2 x1  1  1  R1 8
2 x2  1  2  R2 4
x1  x2  S1  2
1S1  1x1  2 x2  0 All variables  0.
(We solve by " Modified Simplex " Algorithm)
Basic r x1 x2  1 1 2 R1 R2 S1 Sol
2 2 2 -1 -1 0 0 0 12
r 1 0 0 0 0 0 -1 -1 0 0
R1 0 2 0 1 -1 0 1 0 0 8
R2 0 0 2 1 0 -1 0 1 0 4
S1 0 1 1 0 0 0 0 0 1 2
r 1 0 0 2 -1 -1 0 0 -2 8
R1 0 0 -2 1 -1 0 1 0 -2 4
R2 0 0 2 1 0 -1 0 1 0 4
x1 0 1 1 0 0 0 0 0 1 2
r 1 0 4 0 1 -1 -2 0 2 0
1 0 0 -2 1 -1 0 1 0 -2 4
R2 0 0 4 0 1 -1 -1 1 2 0
x 0 1 1 0 0 0 0 0 1 2
Thus we have got the feasible solution
x1 = 2, x2 = 0, 1 = 4, 1 = 0, 2 = 0
And the optimal value is: z = 12
x2 Level curves of z= f(x1, x2)
z=20

z=19
x1+x2=2 z=16
z=14
z=12
z=8
x1
(2,0) Feasible region
 Problem 1 Problem Set 21.2B
Pages 751-752
Maximize
z  6 x1  3x2  2 x  4 x1x2  3x
2
1
2
2

x1  x2  1
subject to

2 x1  3 x2  4
x1 , x2  0
Denoting the Lagrange multipliers by 1, 2,
1, and 2, the KKT conditions are:

1. 1, 2 , 1, 2  0
2. 6  4 x1  4 x2  1  22  1  0
3  4 x1  6 x2  1  32  2  0
i.e. 4 x1  4 x2  1  22  1  6
4 x1  6 x2  1  32  2  3
(Continued)
 3. x1  x2  S1 1
2 x1  3x2  S2  4
1S1  2 S2  0
1 x1  2 x2  0
All variables  0.
Solving this by " Modified Simplex Algorithm ",
the optimal solution is:
x1 = 1, x2 = 0 and the optimal z = 4.
Basic r x1 x2 1 2 1 2 R1 R2 S1 S2 Sol
8 10 2 5 -1 -1 0 0 0 0 9
r 1 0 0 0 0 0 0 -1 -1 0 0 0
R1 0 4 4 1 2 -1 0 1 0 0 0 6
R2 0 4 6 1 3 0 -1 0 1 0 0 3
S1 0 1 1 0 0 0 0 0 0 1 0 1
S2 0 2 3 0 0 0 0 0 0 0 1 4
r 1 4/3 0 1/3 0 -1 2/3 0 -5/3 0 0 4
R1 0 4/3 0 1/3 0 -1 2/3 1 -2/3 0 0 4
x2 0 2/3 1 1/6 1/2 0 -1/6 0 1/6 0 0 1/2
S1 0 1/3 0 -1/6 –1/2 0 1/6 0 -1/6 1 0 1/2
S2 0 0 0 -1/2 -3/2 0 1/2 0 -1/2 0 1 5/2
Basic r x1 x2 1 2 1 2 R1 R2 S1 S2 Sol
r 1 0 -2 0 -1 -1 1 0 -2 0 0 3
R1 0 4 4 1 2 -1 0 1 0 0 0 6
x1 0 1 3/2 1/4 3/4 0 -1/4 0 1/4 0 0 3/4
S1 0 0 -1/2 -1/4–3/4 0 1/4 0 -1/4 1 0 1/4
S2 0 0 0 -1/2 -3/2 0 1/2 0 -1/2 0 1 5/2
r 1 0 0 1 2 -1 0 0 -1 -4 0 2
R1 0 0 0 1 2 -1 0 1 0 -4 0 2
x1 0 1 1 0 0 0 0 0 0 1 0 1
2 0 0 -2 -1 –3 0 1 0 -1 4 0 1
S2 0 0 1 0 0 0 0 0 0 -2 1 2
Basic r x1 x2 1 2 1 2 R1 R2 S1 S2 Sol
r 1 0 0 0 0 0 0 -1 -1 0 0 0

1 0 0 0 1 2 -1 0 1 0 -4 0 2
x1 0 1 1 0 0 0 0 0 0 1 0 1
2 0 0 -2 0 –1 -1 1 1 -1 0 0 3
S2 0 0 1 0 0 0 0 0 0 -2 1 2

Thus the optimum solution is:

x1 = 1, x2 = 0, 1 = 2, 2 = 0, 1 = 0, 2 = 3
And the optimal value is: z = 4
 Problem 13.7-3 Hillier & Lieberman
Maximize

z  20 x1  50 x2  20 x  18x1 x2  5x
2
1
2
2
subject to
x1  x2  6
x1  4 x2  18 Max z
= 224
x1 , x2  0
Using Excel solver, the opt solution is: x1=2, x2=4
If the problem is a minimization problem,
say, Minimize z
we convert it into a maximization problem,
Maximize - z.
Problem 2 Problem Set 21.2B Page 752

Solve the following quadratic

programming problem
Minimize z  2 x  2 x  3x 
2
1
2
2
2
3

2 x1 x2  2 x2 x3  x1  3x2  5x3
subject to
x1  x2  x3  1
3x1  2 x2  x3  6, x1 , x2 , x3  0
Problem (from Hillier & Lieberman)
Solve the following quadratic
programming problem
Maximize
z  20 x1  50 x2  20 x  18 x1 x2  5 x
2
1
2
2

subject to x1  x2  6
x1  4 x2  18,
x1 , x2  0
Problem (from Hillier & Lieberman)
Solve the following quadratic
programming problem
Maximize
z  2 x1  3x2  x1  x
2 2
2

subject to x1  x2  2
x1 , x2  0