An application of separable programming

By A. J. Akeroyd'

A method of solving a general nonlinear function of several variables within a constrained region
using an interpolation technique with separable programming is described, with reference to a
problem formulated by Box (1965).

Box (1965) describes a nonlinear programming problem

Downloaded from https://academic.oup.com/comjnl/article/8/4/344/400788 by guest on 10 September 2020

that arose in his work, for which he described two new
computational procedures. The problem is in fact
linear except for the presence of four product terms. It
may be of interest to point out that problems of this
type can be solved by an extension of linear pro-
gramming known as separable programming, and Box's
problem has been used to illustrate this. The paper
outlines the technique of separable programming and
the interpolation procedure associated with it in the
C-E-I-R mathematical programming code LP/90/94
for the IBM 7094 computer. The application of the
technique to Box's problem is then described.
Fig. 1
Separable programming and interpolation
The method of separable programming was first
formulated by Miller (1963). It provides a simple Separable programming is a modification to the
technique for handling arbitrary nonlinear functions of normal simplex method that restricts the choice of
single arguments in otherwise linear programming variable to enter the basis, and thereby avoids having
problems—and can readily be adapted to handle product any illegal combinations of the special variables.
terms. In order to handle product terms by separable pro-
Assume we have a variable z and a function f(z) gramming we use the relationship
whose graph appears as in Fig. 1.
This curve can be replaced by a set of straight lines + ?
(2.4)
(
Z
zxz2 l\
defined by a finite number of points (in this example 8
points have been used). which converts a product of two linear variables to the
Allowing the coordinates of P, to be (a,-, £>,) we can difference of two nonlinear functions of linear variables.
introduce 8 new non-negative variables M, . . . w8 such If we replace the right-hand side of equation (2.4)
that by (M2 — v2) we have
w, + . . . + w8 = l (2.1)
axux asus = z (2.2) and v =
bxux bsus = f t (2.3) If we can restrict each z, to lie between 0 and 1, it
The variables «, . . . w8 are called a group of special follows that 0 < u < 1 and — ± < v < + }.
variables, and in a particular problem each nonlinear Thus we can handle a product term by introducing
constraint has a group associated with it. two groups of special variables (M and v), each group
If I/J = 1 then u2 • • • us = 0 a n d z = a{, f(z) = b\, being defined by two equations, the first corresponding
the point Px. If u{ = i, u2 = i, then u3. . . «8 = 0 to equation (2.1) and the second to equation (2.2).
and z = i(a, + a2), j[z) = K*i + b2), a point halfway For example, taking five equally spaced variables in
along the line P\P2. In this way, allowing two neigh- each group, we have
bouring special variables to take non-zero values, we (2.5)
can move along the set of lines PtP2, PjPy. • • / W u2 u5 =
However, if two non-neighbouring special variables (2-6)
take non-zero values we have an invalid point (e.g. if
"I = i, "3 = i we have a point midway along the line v i + v2 + v3 + «4 + v5 = 1 (2.7)
K»s) = ** (2-8)
C-E-I-R Ltd., 30-31 Newman Street, London, W.\.
344
 Separable programming
Interpolation and the following constraints.must be satisfied:
In order to get an accurate solution from a separable 0 < x,
programming problem it is necessary to specify a large
number of special variables, and this can considerably l - 2 < x2< 2-4
slow down the speed with which the problem is solved. 20 < x3 < 60
This difficulty can be overcome, as suggested by Miller
(Miller, 1963), by starting with a few variables and, 9 < x4 < 9 • 3
when a solution is reached, generating further variables 6-5 < * 5 < 7-0
in the region of this solution.
The interpolation agendum written by Mrs. P. Griffiths 0 < x6 < 294000
for the C-E-I-R LP/90/94 system interpolates between 0 < x1 < 294000

Downloaded from https://academic.oup.com/comjnl/article/8/4/344/400788 by guest on 10 September 2020

points P, and Pi+, by fitting a cubic to the points
P,_i, Pj, Pi+i and Pj+2 or by fitting a quadratic to 0 < xs < 277200.
three of these points if one of the end points does not
The numerical values of the constants are specified in
exist. Rows corresponding to equations of the type (2.6)
are given special names and the program looks at each the appendix to Box (1965). It is convenient to start by
of these rows. If one variable is in the basis (say the z'th), reducing all product terms to products of variables lying
it interpolates n equally spaced new variables between between 0 and 1. So we write;
the (i — l)th and the rth and another n new variables &, + k2x2 + k3x3 + k4x4 + ksxs = c, + dxzx (3.10)
between the ith and the (i + l)th, where n is a numerical
parameter specified on the agendum call card. If there Xl=d2z2 (3.11)
are two adjacent variables in the basis, it interpolates n y\ + yi + V3 = c3 + d3z3 (3.12)
equally spaced new variables between them. If there
are two non-adjacent variables in the basis, it inter- k21x2 k2Sx3 + k29x4 + k3Ox5 = c4 + d4z4 (3.13)
polates n variables in the interval spanned by them,
04^3 + 05J4 + 7840a6
although this situation rarely occurs in practice. If
there are no variables in the basis, it ignores this group. — 50800a7x2 — 508a7x3 + k3l
The entries in other rows are calculated as cubic functions + k32x2 + fc33x3 + £34X4 + &35X5 = c5 + d5z5 (3.14)
of the entries in the reference rows.
where the c,- and d, are chosen so that all z, lie between
0 and 1. We then have;
The use of interpolation to solve a problem
x6 = cxd2z2 + d^z^z-i, (3.15)
The example the author used is that described by Box
(Box, 1965), except that the function —/was minimized x 7 = c3d2z2 + d3d2z3z2 (3.16)
rather than / maximized. xs = cAd2z2 + d4d2z4z2 + x6 + x7 (3.17)
The problem was to minimize the function / subject
to the 8 constraints given below: / = - c5d2z2 - d5d2z5z2 + 24345 - o,x 6 . (3.18)
b = x2 +001x 3 (3.1) For each of the product terms (say ztz2) we have four
rows corresponding to equations (2.5), (2.6), (2.7) and
-Vg ==
yKj ~\~ f^2^2 ~i ^3"^3 ~T~ K4X4 ~\~ t sxs)x\ (3.2) (2.8):
yi =kb+ knx2 + Jfc8x3 + A:9x4 + Ar (3.3) «t u2 « 3 u4 u5 v{ v2 v3 v4 v5 z, z 2
r- k,5x$ (3.4) 1 1 1 1 1 = 1
y-i = kl6 + fr17x2 + ^, 8 x 3 + £, 9 x 4 + k2Ox5 (3.5) o +i +i +i i -i -i = o
y4 = k2l + «22^2 "T" «23^3 "1" k24X4 -I" ^25^5 (3.6) 1 1 1 1 1 --= 1

*1 = (>•• + ^2 + ^ ) * l (3.7) -i-io +i+i-i+i = o.

Equation (3.15) then becomes:
- x6+ x7 (3.8)
f— ~ ifliy\ + ^3^2 + a4^3 + a5)U + 7840a6 — (l/16)f 2 — (1/16)^4—(1/4)t>j) + ctd2z2 — x6 = 0.
- lOOOOOflo - 508006a 7 + k3l + k32x2
Similarly a further six sets of special variables and
+ ^33X3 + £34X4 + k^x^Xi + 24345 — O{X6 their related equations are used to represent the three
remaining product terms.
(3.9)
Since the z, are confined to the range 0 < z, < 1 the
where x,, x 2 , x 3 , x 4 and x 5 are the independent variables values of the c, and dt can be determined by calculating
345
 Separable programming
Table 1
NUMBER OF
TIME
/ X2 Xi X6 Xl ITERATIONS

5,007,517 7-30902 1-2 600 9-3 6-52841 38,444 240,848 277,200 0-37 50
1st Interp. 5,326,545 5-70859 1-2 60 0 9-3 6-94147 64,999 209,110 277,200 0-72 64
2nd Interp. 5,226,302 5-30990 1-2 600 9-3 7-0 69,887 204,103 277,200 1-07 69
3rd Interp. 5,251,280 5-03160 1-60338 60 0 9-3 70 71,783 201,993 277,200 1-46 85
4th Interp. 5,293,116 4-57073 2-4 60 0 9-3 70 75,118 198,612 277,200 1-89 109
5th Interp. 5,284,811 4-53681 2-4 600 9-3 7-0 75,831 197,896 277,200 2-26 115
6th Interp. 5,280,189 4-53710 2-4 600 9-3 70 75,580 198,148 277,200 2-59 115

Downloaded from https://academic.oup.com/comjnl/article/8/4/344/400788 by guest on 10 September 2020

7th Interp. 5,280,342 4-53743 2-4 600 9-3 70 75,570 198,157 277,200 2-92 115
8th Interp. 5,280,335 4-53743 2-4 60 0 9-3 70 75,570 198,157 277,200 3-25 115
9th Interp. 5,280,336 4-53743 2-4 60 0 9-3 70 75,570 198,157 277,200 3-85 115

the maximum and minimum of equations (3.10) to (3.14). method described by Fiacco and McCormick (1964).
Because many of the coefficients are so large the rows However, it should be pointed out that separable pro-
(3.3) to (3.6) and (3.10) to (3.14) were scaled by a factor gramming is a convenient general method of dealing
of 1000, and rows (3.15) to (3.18) by 10,000. with nonlinearities in an otherwise linear programming
The problem has 40 rows and 68 vectors, the rows problem, and has the important advantage over other
being the 11 constraints for xt . . . xs, the 16 defining apparently more general methods of constrained opti-
the product terms, the 4 (including the functional) con- mization that there is no real limit to the number of
taining the product terms, the 5 defining the z, and the linear variables that can be represented: typical real
4 defining yt . . . j>4. Many of the variables can be separable programming problems have a few hundred
eliminated by substitution but it is not worth the effort such variables. Also the LP/90/94 system is most effi-
in a problem as small as this. cient on larger problems—with small problems a large
Table 1 show the results obtained. In all cases the proportion of the total time is spent in noniterative
parameter n (the number of new variables interpolated) routines.
was 3. The two end columns show the time in minutes
and the number of simplex iterations from the start of
the run. Acknowledgements
The work took four minutes on an IBM 7094 Mark II The author wishes to acknowledge his sincere thanks
computer, and it cannot be claimed that this is parti- to Mr. E. M. L. Beale and Dr. P. F. Windley of
cularly fast. Dr. McCormick reports in a private com- C-E-I-R Ltd. for their valuable assistance in solving
munication that he has solved the same problem in the problem, and to Mr. Beale for his constant help and
about 30 seconds on an IBM 7040 computer using the advice in preparing this paper.

References
Box, M. J. (1965). "A new method of constrained optimization and a comparison with other methods," The Computer Journal,
Vol. 8, p. 42.
FIACCO, A. V., and MCCORMICK, G. P. (1964). "Computational Algorithm for the Sequential Unconstrained Minimization
Technique for Nonlinear Programming," Management Science, Vol. 10, p. 601.
MILLER, C. E. (1963). "The Simplex Method for Local Separable Programming" from Recent Advances in Mathematical
Programming, p. 89. Edited by R. L. Groves and P. Wolfe. McGraw-Hill.

346