Matrices and Simplex Algorithms

A. R. G. Heestennan
Department of Economics, University of Birmingham , U.K.

Matrices and
Simplex Algorithms
A Textbook in Mathematical Programming
and Its Associated Mathematical Topics

D. Reidel Publishing Company

Dordrecht : Holland / Boston : U.S.A . / London: England
library of Congress Cataloging in Publication Data

Heesterman, A. R. G.
Matrices and simplex algorithms.

Includes index.
1. Programming (Mathematics) 2. Ma trices. I. Title.
QA402.5.H43 1982 519.7 82-18540

ISBN-13:978-94-009-7943-7 e-ISBN -13: 978-94-009-7941-3

DOl: 10.1 007/978-94-009-7941-3

Published by D. Reidel Publishing Company

P.O. Box 17,3300 AA Dordrecht, Holland

Sold and distributed in the U.S.A. and Canada

by Kluwer Boston Inc.,
190 Old Derby Street, Hingham, MA 02043, U.S.A.

In all other countries, sold and distribu ted

by Kluwer Academic Publishers Group,
P.O. Box 322,3300 AH Dordrecht, Holland

D. Reidel Publishing Company is a member of the Kluwer Group

All Rights Reserved

Copyright (91983 by D. Reidel Publishing Company, Dordrecht, Holland
Softcover reprint ofthe hardcover 1st edition 1983
No part of the material protected by this copyright notice may be reproduced or utilized
in any form or by any means, electronic or mechanical, including photocopying,
recordipg or by any informational storage and retrieval system,
without written permission from the copyright owner
Table of Contents

Introduction vii

PART I

MATRICES, BLOCK-EQUATIONS, AND DETERMINANTS

Chapter I / EQUATIONS-SYSTEMS AND TABLEAUX 3

Chapter II/MATRIX NOTATION 5
Chapter III/BLOCK-EQUATIONS AND MATRIX-INVERSION 33
Chapter IV / SOME OPERATORS AND THEIR USE 62
Chapter V / DETERMINANTS AND RANK 68

PART II

GRAPHS AND LINEAR PROGRAMMING

Chapter VI/VECTORS AND COORDINATE-SPACES 115

Chapter VII/SOME BASIC LINEAR PROGRAMMING CONCEPTS 144
Chapter VIII/OUTLINE OF THE SIMPLEX ALGORITHM 149
Chapter IX / THE SEARCH FOR A FEASIBLE SOLUTION 181
Chapter X / MIXED SYSTEMS, UPPER AND LOWER BOUNDS 205
Chapter XI/DUALITY 223
Chapter XII/LINEAR PROGRAMMING ON THE COMPUTER 242
Chapter XIII/PARAMETRIC VARIATION OF THE L.P. PROBLEM 273

N.B. Answers to exercises are not included in this list of

contents, but may be found in the places indicated with the
exercises, usually at the end of the chapters.
vi T ABLE OF CONTENTS

PART III

SOME GENERAL MATHEMATICAL PROGRAMMING NOTIONS AND

RELATED MATRIX ALGEBRA

Chapter XIV / TOPOLOGY OF FEASIBLE SPACE AREAS AND ITS

RELATION TO DEFINITENESS 319
Chapter XV / OPTIMALITY CONDITIONS 363

PART IV

gUADRATIC PROGRAMMING

Chapter XVI/QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 402

Chapter XVII/PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 516
Chapter XVIII/GENERAL QUADRATIC PROGRAMMING 556

PART V

INTEGER PROGRAMMING

Chapter XIX / INTEGER PROGRAMMING AND SOME OF ITS

APPLICATIONS 637
Chapter XX / BRANCHING METHODS 656
Chapter XXI/THE USE OF CUTS 702

REFERENCES 773

INDEX 779
Introduction

This is a textbook devoted to mathematical programming

algorithms and the mathematics needed to understand such
algorithms. It was mainly written for economists, but the
mathematics itself obviously has relevance for other disciplines.

It is a textbook as well a~ in parts, a contribution to new

knowledge. There is, accordingly, a broad ordering of climbing
sophistication, the earlier chapters being purely for the
student, the later chapters being more specialist and containing
some element of novelty on certain points. The book is edited
in five parts.

Part I deals with elementary matrix operations, matrix

inversion, determinants, etc.

Part II is mainly devoted to linear programming.

As far as students' readability is concerned, these two parts

are elementary undergraduate material.

However, I would claim, in particular with respect to linear

programming, that I do things more efficiently than the
standard textbook approach has it. This refers mainly to the
search for a feasible solution i.e. Chapter 9, and to upper
and lower limits, i.e. Chapter 10. I have also argued that the
standard textbook treatment of degeneracy misses a relevant
problem, namely that of accuracy.

In short, I would invite anyone who has the task of writing

or designing an LP-code, to first acquaint himself with my
ideas.
viii INTRODUCTION

Parts III and IV are concerned with nonlinear programming.

Part III gives the bulk of the theory in general terms including
additional matrix algebra. It was obviously necessary to
introduce definiteness at this point, but a full discussion of
latent roots is refrained from. Proofs are therefore given as
far as possible, without reference to eigenvalues. However,
certain results will have to be taken on trust by those readers
who have no prior knowledge on this point.

The main contribution to the literature made in part III,

probably is Chapter 15, i.e. to explain both the first-order
conditions and the second order conditions for a constrained
maximum, in terms where one may expect the student to actually
understand this admittedly difficult problem.

The unconventional concept of subspace convexity is not, and

cannot be a true novelty; it is equivalent to the more usual
way of formulating the second order condition for a constrained
maximum in terms of determinants.

Part IV is concerned with quadratic programming. It does not

give a comprehensive survey of algorithms. It gives those
algorithms which I considered the most efficient, and the
easiest to explain and to be understood.

With respect to novelty, Chapters 16 and 17 do not contain any

original ideas or novel approaches, but some of the ideas
developed in Chapters 9 and 10 for the LP case are carried over
into quadratic programming. Chapter 19 does however, offer an
algorithm developed by myself, concerning quadratic programming
with quadratic side-conditions.

Part V deals with integer programming.

As in the QP case, the basic ingredients are taken from the

existing literature, but the branching algorithm of section 20.2,
although based on a well-established approach, was developed by
myself. Also, the use of upper and lower limits on the lines
of Chapter 10 proved particularly useful in the integer
programming context.

Nothing in this book is out of the reach of undergraduate

students, but if it is to be read in its entirety by people
without prior k~owledge beyond "0" level mathematics, the
consecutive ordering of the material becomes essential and a
two-year period of assimilation with a break between Part II
and Part III would be preferable. However Parts IV and V
will generally be considered to be too specialist on grounds
of relevance and curriculum load, and accordingly be considered
more suitable for postgraduate students specializing in O.R.
or mathematical programming.
INTRODUCTION ix

I have, however, myself used some sections of Chapter 16, not

so much because undergraduates need to know quadratic
programming for its own sake, but as reinforcement of teaching
optimality conditions.

Concerning presentation, it may be observed that more than

half of the number of pages is taken up by numerical examples,
graphical illustrations and text-listings of programme-code.

The numerical examples and graphical illustrations are obviously

there for purely educational purposes as are the student
exercises.

The code-listings serve, however a dual purpose to back up

the descriptions of algorithms, and to be available for
computational use. Some tension between these two purposes
is obviously unavoidable. I have made an effort to make the
programme-texts readable, not only for the machine but also
for the human reader, but if illustration for the benefit of
the human reader were the only consideration I would have to
cut down on the number of pages of code-listing much more than
I have in fact done.

It appears to be appropriate to comment here on the use of the

computer-language i.e. Algol 60, rather than the more widely
used Fortran. While it is true that I simply know Algol very
much better than Fortran, the choice appears also to be
justified on the following intrinsic grounds:

The use of alphanumerical labels which are meaningful to the

human reader, e.g.

PHASE I:, MAKE THE STEP:, etc., and

corresponding goto statements e.g.

'COTO' PHASE I; helps to bridge the gap between programme

description and programme-text in a way which is difficult
to achieve by comment (or its Fortran equivalent) only.

Many procedure and programme texts also contain alphanumerical

labels which are there purely for the human reader, as there
are no corresponding goto statements.

While such labels are also possible in an "Algol-like"

language as, for example Pascal, they. cannot be used in Fortran.
Furthermore, Fortran is simply more primitive than Algol.
x INTRODUCTION

I have made extensive use of block-structure and dynamic arrays

and as a result my programmes don:t require more core-space
than is strictly necessary. Also, I gather that not all versions
of Fortran permit recursive calls in the way I have used them
in section 5.6 for the calculation of determinants and in
section 20.4 for branching in integer programming.

The value of the text-listings as a direct source of ready-made

programme-text is further compromised by the presence of
warning-messages, not only in the main programmes but also
inside the procedures.

The presence of these warning messages enhances the readability

for the human reader but, as they are system-specific, they will
in general require adaptation if the algorithms are to be applied
in a different machine-environment.

Acknowledgement
I gladly acknowledge the use of the facilities of the University
of Birmingham Computer Center, as well as the help of
Dr. P. Soldutos, my student at the time, in setting up the
private graph-plotting package used to make the graphs in this
book.

University of Birmingham
2ls t May, 1982

A.R.C. Heesterman
Part I
MATRICES, BLOCK-EQUATIONS, AND DETERMINANTS

CHAPTER I

EQUATIONS-SYSTEMS AND TABLEAUX 3

1.1 Equations-systems 3
1.2 The use of a tableau 4

CHAPTER II

MATRIX NOTATION 5

2.1 The purpose of matrix notation 5

2.2 Some definitions and conventions 5
2.3 The transpose of a matrix 8
2.4 Addition and subtraction 9
2.5 Matrix multiplication 10
2.6 The product of a matrix and a vector II
2.7 Vector--vector multiplication 13
2.8 Multiplication by a scalar 14
2.9 Matrix-matrix mUltiplication by columns or by rows 14
2.10 Substitution 15
2.11 Forms 16
2.12 Recursive products 17
2.13 The addition of several matrices 18
2.14 Transposition of compound expressions 19
2.15 Some special mat~ices and vectors 19
2.16 Matrix partitioning 21
2.17 Multiplication by partitionins 22
2.18 Differentiation of matrix expressions 25
2.19 Reading and printing of large matrices by
electronic computers 29
2

CHAPTER III

BLOCK-EQUATIONS AND MATRIX-INVERSION 33

3.1 Notation of linear systems 33

3.2 Singularity 33
3.3 The elimination process 34
3.4 Computational arrangements 37
3.5 The sum-count column 39
3.6 The system A y = B x 40
3.7 The inverse - - 41
3.8 Inverse and reduced form 47
3.9 Multiplication instead of division: the
all-integer elimination method 48
3.10 Row permutation during inversion 51
3.11 Block elimination 53
3.12 Inversion of recursive products and transposes 56
3.13 The differentiation of an inverse 58
3.14 Text of an inversion procedure 59

CHAPTER IV

SOME OPERATORS AND THEIR USE 62

4.1 The summation vector 62

4.2 The aggregation matrix 63
4.3 Vector permutation 64

CHAPTER V

DETERMINANTS AND RANK 68

5.1 Determinants and minors 68

5.2 Permutations of determinants 75
5.3 Proportionality of vectors 75
5.4 Decomposability of a determinant 32
5.5 Determinant and inversion by row-operations 84
5.6 The calculation of determinants 87
5.7 Rank and the determinants of some
structured mat-rices 91
5.8 The adjoint and its relation to the all-
integer elimination method 101
5.9 Commented text of the adjoint all-integer
elimination method 105
5.10 The determinant of the product of two
square matrices 108
CHAPTER I

EQUATIONS-SYSTEMS AND TABLEAUX

1.1 Equations-systems

The following is an ordinary system of simultaneous linear

equations:

0.867 xl -0.066 x 2 240 (11)

-0.150 xl +0.850 x 2 210 (12)

-0.167 xl -0.100 x 2 + x3 0 (13)

This system can be solved by performing certain operations:

Divide (11) by 0.867 (multiply by 1/0.867), and obtain:

= 277 (21)

Multiply (21) by 0.150 and 0.167, respectively, and obtain:

0.150 xl -0.011 x 2 42 (22)

0.167 xl -0.013 x 2 46 (23)

Re-name (21) as (31), add (22) to (12) to become (32) ,

add (23) to (13) to become (33) , and obtain a new system:

xl -0.076 x 2 277 (31)

0.839 x 2 252 (32)

-0.113 x 2 + x3 46 (33)

Divide (32) by 0.839 (multiply by 1/0.839) to obtain:

= 300 (42)

Multiply (42) by 0.076 and 0.113, respectively, to obtain:

0.076 x 2 23 (41)

0.113 x 2 34 (43)

Add (41) to (31), to become (51); re-name (42) as (52);

add (43) to (33), to become (53), and obtain a new system:

3
4 CHAPTER I

300 (51)

300 (52)

80 (53)

1.2 The use of a tableau

In section 1.1 three equations were written 5 times. Each time

the variable-names xl' x 2 ' and x3 were written again. In total
we did this 15 times.

We can economize on our writing effort, by writing the names of

the variables only once. And if the procedure for obtaining a
system of equations from its predecessor is a standardized one,
we can di spense wi th exp 1aining it every time. tole could have
done the job by writing 5 tableaux, as listed below:

Xl x2

0.867 -0.066 240

-0.150 0.850 210
-0.167 -0.100 1,000

1.000 -0.076 277

0.150 -0.011 42
0.167 -0.0l3 46

1.000 -0.076 277

0.839 252
-0.113 1,000 46

1.000 300
0.076 23
0.113 34

1,000 300
1.000 300
1.000 80
CHAPTER II

MATRIX NOTATION

2.1 The purpose of matrix notation

Matrix notation provides a very compact way of describing

certain well-defined numerical operations. As such it saves
writing and reading effort in written communication about
numerical operations. This applies to communication between one
human being and another. It also applies to machine-programming.
For most computers, there is by now a certain body of established
programmes, routines, carrying out specific matrix- and vector
operations. Reference to such routines saves programming effort.
The use of matrix notation has also facilitated the analysis of
numerical problems. This refers in particular to the properties
of linear equation-systems. Such facilitation is really a
corollary of the reduction in effort. Problems, which were
formerly too complicated to grasp, now become manageable.

2.2 Some definitions and conventions

A matrix is a rectangular grouping of numbers, its elements.

The elements of a matrix are grouped into a number of rows;
each row containing the same number of elements, reading from
left to right. Alternatively, we can say that the elements of a
matrix are grouped into a number of columns, each column
containing the same number of elements, reading from top to
bottom.

Matrices often occur as tableaux, containing statistical

information. For example:

British consumer's expenditure, Central Statistical Office:

"National Income and Expenditure" (1967),

1964 1965 1966

Coal and coke 273 269 257

Electricity 381 ~17 435
Gas 167 194 223
Other 58 60 58

Another application of tableaux (or matrices), was met in section

1.2: the arrangement of computations.

5
6 CHAPTER II

The order parameters of a matrix are two non-negative integer

numbers. They are always listed in the same order. The first
number indicates the number of rows of the matrix; the second
order-parameter indicates the number of columns in the matrix.
Normally, order-parameters will be positive numbers. But a more
general validity of certain statements in matrix algebra may be
obtained, if we admit the value of zero as a borderline case.

The number of elements in each row is the number of columns in

the matrix. And the number of elements in each column is the
number of rows in the matrix. It follows that the number of
elements in the matrix is the product of its order parameters.
A matrix of which the two order-parameters are equal is called
a square matrix.

To indicate a matrix, we can use a letter. A capital letter is

always used for that purpose. In printed text, a capital letter
indicating a matrix, is generally given in heavy print.

We might for instance have:

273 269
[ 381 417 257]
435
c 167 194 223
58 60 58

A corresponding lower case letter, with 2 indices will indicate

an individual element of a matrix.
For example, c 3 2 = 194.
The indices are'always given in the same order: first the index
indicating the row, then the index indicating the column.

Not all numerical information is suitably presented in a

rectangular array. Suppose for instance we were interested only
~n total expenditure on fuel and light:

1964 1965 1966

Expenditure on fuel and light 879 940 973

This is a vector. A vector is a matrix of which one of the order

parameter is known to be unity. We distinguish between rows
(matrices with only one row) and columns (matrices with only one
column). The (total) fuel and light expenditure, was presented
as a row. We could have presented it as a column as well.

To indicate a vector, we can use a letter. For that purpose, one

always uses a non-capital letter. Vectors are normally indicated
with italic print or heavy print, or in typescript - including
photographically reproduced typescript, as in this book, with
MATRIX NOTATION 7

underlining, to avoid confusion with indices. One should of

course not use capital letters, this would create confusion with
matrices.

Vectors will normally be assumed to be columns. When we want to

indicate a row, this will be done by adding a prime to the letter.

The order of a vector is determined by listing the value of only

one order-parameter. Elements of a vector are indicated with a
small (non-capital) letter, without heavy print or underlining,
with one index.

For instance, we might write: let t be a colunm-vector of order

3. We can also indicate the fact that a vector is a colunm (row)
by stating its order as m by 1 (1 by n).

We may then define a (colunm) vector of consumer's expenditure

on fuel and light, of order 3 (by 1).

t
~879~
940
973

The corresponding row-vector will be of order 1 by 3:

t' [879 940

The statement t' is a row-vector of order 1 by 3 is legitimate,

but gives more-information than is strictly needed. The 1966
figure can be indicated either as t3 = 973, or as t'3 = 973.
The prime is quite superfluous here; hence we normally write
t = 973. Elements of a vector should always be indicated with
t~eir index. The use of ordinary small letters without index or
heavy print (underlining), is conventionally reserved for
variables or coefficients of an integer nature, such as indices
and order-parameters.

Occasionally, one may also meet a single coefficient, which is

not an element of a matrix or vector. Such a scalar is then
indicated with a Greek letter. If required, a scalar can be
interpreted as a matrix of order 1 by 1, as a column of order 1,
or as a row of order 1.

A matrix which satisfies the property aij = a .. (and therfore

.
obv10us 1 y m = n ) .1S ca 11 e d a symmetr1c
.mdtr1x,
. J e.g.
1

A
8 CHAPTER II

2.3 The transpose of a matrix

In paragraph 2 of this chapter, we met the following tableau:

British consumer's expenditure, at 1958 constant prices, on fuel

and light:

1964 1965 1966

Coal and coke 273 269 257

Electricity 381 417 435
Gas 167 194 223
Other 58 60 58

Now consider the tableau:

British consumer's expenditure, at 1958 constant prices, on fuel

and light:

Coal & Elect- Gas Other

coke ricity

1964
1965
1
273
269
381
417
167
194
58
60
1966 257 435 223 58

It will be observed, that this tableau gives exactly the same

numerical information, as the previous one. But the presentation
is different.

r
The corresponding matrices are said to be each other's transpose

269 381 167

C = [273
381
167
417
194
257J
435
223
and C' 269
73
257
417
435
194
223
5~
60
58
58 60 58

The transpose of a matrix is another matrix, with the rows of

the first matrix as columns, and the columns of the first matrix
as rows. It is conventional to indicate a transposition by a
prime.

It follows that if a matrix A is of order m by n, then A' is of

order n by m. The transpose of the transpose will have the rows
of the transpose (= the columns of the matrix itself) as columns,
and the columns of the transpose (= the rows of the matrix
itself) as rows. The transpose of the transpose is the matrix
itself, i.e.

(A') , A
MATRIX NOTATION 9

2.4 Addition and subtraction

The sum of two matrices of the same order is a matrix of the

same order again. The elements of the result are the sums of the
corresponding elements of the operands. The relation

A + B C

pre-supposes, that A, B, and C are all of the same order.

Furthermore, the statement is equivalent to:

a. . +
~,J

for all admissable values of i and j.

Since,

a. . + b .. b. . + a ..
~,J ~,J ~,J ~,J

we also have:

A + B B + A

Similarly, the statement

A B D

pre-supposes, that A, Band D are all of the same order, while

a .. b .. d ..
~,J ~,J ~,J

The same logic applies to vectors as well.

The statement:

a + b b + a

pre-supposes that ~, ~, and c are all of the same order and eives
the information:

a. + b. c.
~ ~ ~

And the subtraction of one vector from another:

a b d

means a. - b. = d., for all positive integer values of i,

smaller~than~or eijual to the (identical) orders of ~, ~ and d.

One does not always use a separate letter for the result of a
matrix (or vector) operation. Instead one can use compound
10 CHAPTER II

expressions within brackets, like (A + B); (~+ ~), etc.

Matrices and vectors can b~ added and subtracted like ordinary

numbers; by performing these operations on their corresponding
elements.

This also applies to rows. It follows that if we have

+ b

we will also have

a' + b' c'

Note that expressions like

~ + ~', or A + b

(in general additions of operands of different order) are just

plain nonsense.

Operations must be carried out on corresponding elements.

If the operands are not of the same order, there just are no
corresponding elements.

Example of the addition of two matrices:

In section 1.2 several of such additions were performed, such as:

[-0.150
-0.167
0.850
-0.100 1.000 :~~J + [0.150
0.167
-0.011
-0.013
4lJ
46

[---
---
0.839
0.113 1.000 25~J
46

Exercise: Find some more matrix additions, in the same paragraph.

(This may involve permutation of the rows of the tableaux).

2.5 Matrix multiplication

The product A . B is defined only if the number of columns of A

is equal to the number of rows of B. The result is a matrix with
the same number of rows as A, and the same number of columns as B.

Let A be a matrix of order m by n, and B a matrix of order n by

k. And let C be a matrix of order m by k. Then the numerical
content of A, B, and C satisfies

A.B C
 11
MATRIX NOTATION

if we have,
n
c.
~,J
.
r
L1 a.
~,r
b
r, J
.

(i = 1, 2, ... m, j = 1, 2, ... k)
From this definition it follows that we will not ~n general
have

A • B B . A
We cannot even say that the normal rule is A. B # B. A
The fact that A . B is defined does not imply that B . A is
defined. And something that ~s not defined cannot even be
unequal to something else.

Only if A, B, and C are all symmetric and of the same order we

have the identity

n n n
L a. b
rj L a
r~
b.
Jr L b. a .
Jr r~
c ..
~J
c ..
'J~
r=l
~r
r=l r=l

or, using matrix notation

AB = A'B' = B'A' = C C'

Examples:

A B C D E

[-i iJ [i 1
2 -~J = [; 5
1 ;l [~ ~J [-~ -~J
detail-example for c l ,2:

c l ,2 al,l bl ,2 + a l ,2 b2 ,2

1 1 + 2 2
1 + 4 5

Exercise: Analyse ~n a similar way, the computation of the other

elements of C.

2.6 The product of a matrix and a vector

Since a vector is a matrix with one of the order parameters being

unity, vector-operations have been defined implicitly in the
previous paragraph. The product of a matrix and a vector, if it
is defined (if the operands are of consistent orders), is a vector.
12 CHAPTER II

Let A be matrix of order m by n. And let b be a vector of order

n by 1. Then the product c A b exists~ and is of order m by
1. Its numberical value satisfies
n
c.
1 L a ..
1,J
b.
J
j 1

(i = 1, 2 ... m)

And for a row: Let A be of order m by n, and d' of order 1 by m.

Then the product f' = d'A exists, and 1S of order 1 by n.
Its numerical value satisfies
TIl
f.
J
L d.
1
a.
1,j
i=l

(j 1, 2 ... n)

The expression A b is known as post-multiplication of A by the

column b. The expression d'A is known as pre-multiplication of
A by the row d'. Note that-post-multiplication of a matrix by a
row is nonsense. The same applies to pre-multiplication of a
matrix by a column. But the expression b' A' is legitimate.
This is the pre-multiplication of the transpose of A, by the row
b' .

The statement c A b implies c' b' A'. The two expressions

are in fact identical,-both giving the numerical information
n
c.
1 L a.
1, j
b.
J
j 1

1xample:

The computation of a sum-total.

s' C t'

r" 257]
269
[1 1 1 lJ 381 417 435 [879 940 973J
167 194 223
58 60 58

Here a vector of 4 unity-elements serves as an operator, to add

the rows of C.
MATRIX NOTATION 13

2.7 Vector-vector multiplication

Again, the definition of the product of two vectors is implicitly

given in the definition of the product of two matrices. There
are two cases:

The outer product is always defined. A row is pre-multiplied by

a column, or, what is the same thing, a column is postmultiplied
by a row. The product is a matrix. Let k be a column of order
m, and let VI be a row of order n, v being the 'corresponding
column. The-product

U =k . -VI , then is of order m by n; u. .

1,J
k. v.
1 J
(i 1, 2 ... m, j = 1, 2 ... n)

The inner product of two vectors only exists, if the two vec'tors
are of the same order. The result is of order 1 by 1, i.e. a
scalar number. If v and k are (column) vectors of the same order
n, the inner product
n
~ = VI k exists, and has the value I V.
1
k.
1
i 1
Example of an outer product:

The substitution, or rather the numerical operation representing

a substitution carried out in section 1.2:

0.167J
l
[ 0.150 [1.000 -0.076 ---- 276J [
0.150 -O.Oll
0.167 -0.013
4lJ
46

Example of an inner product:

The computation of a sum of squares.

Let VI be a row-vector, of order 4, e.g. VI [2 5 -1
The 'sum of squares of the elements of v (or '!...I) will be

VI v [2 5 -1 7J

HJ
2 x 2 + 5 x 5 + (-1 x -1) + 7 x 7
4 + 25 + 1 + 49
79.

Exercise:

Find and evaluate 5 different legitimate product-expressions of

14 CHAPTER II

two different vectors, both operands being always two of the

following vectors

v
-1
= _v' = [3 " 21
J' v'
~
[4, -1, 2, lJ.
3

2.8 Multiplication by a scalar

The multiplication of a matrix by a scalar, and the multiplication

of a vector by a scalar, is defined as a multiplication of the
individual elements of the matrix, or of the vector, by that
scalar. The order in which the operands are written is arbitrary
here.

c aB means c.
~,J
. abo
~,J
.

The expressions aB and Ba are equivalent ways of writing the

product of a matrix and a scalar. For a column the two
expressions ab and ba are again both correct and equivalent. The
same applies for a row ab' b'a. But it will be prudent to
write ba and ab', instead of ab-and b'a. The reason for this is
the possibility to interpret a-scalar as a matrix of order 1 by 1.

Example of the multiplication of a row by a scalar

1.153 [0.867 -0.066 --- 240J = [1.000 -0.076 --- 277J

(See para 1.1 and 1.2, division of the (11) row by 0.867).

2.9 Matrix-matrix multiplication by columns or by rows

The product A. B = C can be evaluated by its columns:

A b. c.
-J -J
Here b. is the jth column of B, and c. is the J.th co 1umn 0 f C•
-J -J
Example:

and
A
[i -;]
B
= [-i 5
-2 ~J
We then have, for the first column of the result:

[i
MATRIX NOTATION 15

and for the second column

[i -;] [-~J [:iJ

and for the third column

[i -lJ
3 [~J [~iJ
This gives for the wt.ole matrix:

[i -;] [-i 5
-2 ~J [-~ 12
-1
-4J
12

The above procedure is not something different from the

definition of matrix multiplication in section 2.5. It is only
an ordered way of carrying out the computations of the elements
of c.

The product A . B C can also be evaluated, by its rows:

a! B
_.1.
= -c!1
Here a! is the ith row of A, and c! is the ith row of C. We can
illust}ate this with the same matrices A, Band C.

[2 [-3
~J
5 12
-2

(for the first row)

and (for the second row)

[1-' 5
-2 -1 12J
Obviously, the result C 12
-1
-4J
12

should be the same, irrespective of the order in which the

elements were computed.

2.10 Substitution

When two symbolic expressions are identical, one is allowed to

interchange, or substitute, the one for the other. This postulate
is vital to all algebra. It applies to matrix algebra as well.
16 CHAPTER II

Example:

Let A, B, ~, Z, and b be of orders m by n, n by k; n, k, and m.

And let us have:

Ax b (2.10.1)
and
(2.10.2)

Now substitute BZ for ~ by (2.10.2) into (2.10.1), to obtain:

A (BZ) b ............ (2.10.3)

2.11 Forms

Let a' be a row, of order 1 by m',

let B be a matrix, of order m by n;
let c be a column, of order n by 1.
-

Then

(~' B).£ (2.11.1)

holds.

The relation (2.11.1) is an identity, as may be shown as follows:

Evaluate each ith element of the column within the brackets on

the left-hand side of (2.11.1) as

n
I b .. c.
j =1 1.J J

and each jth element of the row within the brackets on the right-
hand side of (2.11.1) as

m
I a. boo
i=l 1. 1.J

The expressions on both sides of (2.11.1) are of order 1 by 1,

a scalar, and we may independently evaluate both sides of
(2.11.1) .

We obtain

m
E a.
i=l 1.
(I
j =1
b ..
1.J
c)
~
n
I
j=l
(r
i=l
a.
1.
b.)j)
1.
c.
J
(2.11.2)
MATRIX NOTATION 17

This is a true statement, an identity, and hence the same is

true for (2.11.1).

In both relations, the brackets are superfluous, and we may

write

m n n m
L a. Lb .. c·-L'c·La.b
j=l 1J J
..
j=l J i=l 1 1J
i=l 1

m n
- I I
a. b .. c. (2.11.3)
i=l j =1 1 1J J

and its matrix-equivalent

a' a'Bc (2.11. 4)

An expression of this type, 1.e. a matrix pre-multiplied by a row

and post-multiplied by a column, to yield a single number as
result, is known as a form (or bilinear form).

Example:

sl r-~l
l iJ
The form

9J
may be evaluated either as

or as

[2 o 7J
s

2.12 Recursive products

The identity a' (~' B) C (2.11.1)

1S a 1 so true 1. f "
a 1S t h e 1. th row of the matr1x
. A.

If A has h rows, we may then write

(2.12.1)

(i 1, 2 ......••.. h)
18 CHAPTER II

The same information, may be presented more compactly, as:

(A B) .£ (2.12.2)

But also, .£ may be a column of a matrix C, which has k columns,

A (B c.) (A B) c. (2.12.3)
-J -J
(j = 1, 2 .•....... k)

The information given by (2.12.3) is written more compactly as

A (B C) (A B) C (2.12.4)

A direct corrollary of (2.12.4) is

A (B C D) ::: (A B) (C D) (A B C) D (2.12.5)

Clearly there is no point in writing brackets inside product

expressions of this type, and (2.12.5) defines the notion of a
recursive product ABCD.

Exercise

Find the numerical content of

~J
ABC = [1 2 3J [ 1
-1
2

Check by independently calculating (AB)C and A(BC), finding the

result the same.

Note

AB = BA is not generally true. This applies to recursive products

just as well. Thus

[1
21 DJ = 5 , but
nJ [1 2] =
D ~J
Even if the order is the same, the numerical content may not be

2J4 = [1119 l6J.

28

2.13 The addition of several matrices

Since addition of matrices consists of addition of the elements

the normal rules of addition and subtraction of numbers also
MATRIX NOTATION 19

apply to matrices. The meaning of expressions like A + B - C.

-A + B + aC, etc. will need no further elucidation. We obviously
have

A + (B + C) (A + B) + C, etc.

In each case these expressions pre-suppose that all the operands

are of the required (same) orders.

2.14 Transposition of compound expressions

The transpose of a sum-expression is the sum of the transposes

of its terms.

(A + B + C + D)' A' + B' + C' + D'

The transpose of a product 1S somewhat more complicated:

( ABC D)' D' C' B' A'

In both cases, the transposition rule is a direct result from

the definition of a sum, or a product, respectively. For a sum,
the relation S = A + B and the relation S' = A' + B' both mean

s. . a .. + b ..
1,J 1,] 1,]

If the number of columns of A and the number of rows of Bare

equal and indicated as r,

p A • B 1S equivalent to p' B ' . A'

Both expressions give the numerical information

r r
P ..
1,] k
L1 a.
1,k L1 bk,j
k

2.15 Some special matrices and vectors

A matrix with two equal order-parameters is named a square

matrix. (The number of columns is the same as the number of rows.)
The vector, consisting of the elements of the matrix with equal
row- and column-indices, is then the (main) diagonal.

Example

-~
[-2
5
6 is a square matrix;
8
20 CHAPTER II

is the corresponding diagonal.

A square matrix, with non-zero elements on the main diagonal

only, is a diagonal matrix.

A special kind of diagonal matrix is the unit matrix. This is a

diagonal matrix, with all the elements on the diagonal being
unity. Unit matrices are normally indicated with a capital
letter I. When the order of the unit matrix is self-evident, it
is not strictly necessary to indicate that order. But one may
write an index beside the capital I, to indicate the order. A
unit matrix of order n, can be indicated as I, or as I .
Multiplication by a unit matrix amounts to copying, i.~.

I v v w' I w' and

I A A B I B.

For this reason the unit matrix is also known as the "identity
matrix".

A unit vector is a vector of zero's, and one single unity-

element. The unit-vector is usually indicated with the letter e.
An index with the vector will indicate the place of the unity-
element, not the order of the vector. If the order is not self-
evident, it must be stated separately .
. th
I f a. denotes the J column of A, we will have
-J
a. A e.
-J -J
.th
I f a! denote the ~ row of A, we will have:
-~

a! e! A
-~ -:~

Example

[1121 12
l~J D~J
22 23
[!]
[0 lJ [1121 12
22 13J
23
[21 22 23J

Unit vectors are often met as operator, to isolate vectors out of

matrices. If we restrict ourselves .to matrix notation (there
are other types of operators) an operator is a matrix, or a
vector, not giving numerical information, but only serving a
MATRIX NOTATION 21

purpose.

Another specially structured (square) matrix is a triangular

matrix. There are two kinds of triangular matrices, upper-
triangular matrices and lower-triangular matrices. As the name
suggests, the non-zero part of a triangular matrix is
triangularly shaped. An upper-triangular matrix has the non-
zero elements only on and above the diagonal; a lower-
triangular matrix has the non-zero part on or below the diagonal,

a. . 0 for j < i, or i < j for a lower-triangular matrix.

~J

The "non-zero" part may of course contain zeros as well.

[~ ~l
2
3 is an upper triangular matrix.
o
What matters is that there are ?nl~ zeros on the opposite side.

2.16 Matrix partitioning

A number of rows of a matrix may be grouped into a block-row.

Again, a number of columns of a matrix may be grouped into a
block-column.

Example

Let us have the matrix

A 2 o
4 o
o 5
o 7

We may then group the first two columns of A into a block-

column, to be denoted as AI:

Regretfully, there is no standard notation for the partitioning

of matrices. For block-rows, this may lead to ambiguity. The·
symbol A~ could either be the transpose of AI; or it could be
the firsE block-row. Also, a series of matrices is sometimes
indicated with indices, while they are not block-columns.
22 CHAPTER II

The position is somewhat better with blocks. A block is the

intersection of a block-column and a block-row. Blocks can be
indicated with two indices. The first index will indicate the
block-row, the second index, the block-column. In our above
example. A may then be partitioned as:

and
~] ~J
While A12 and A21 are zero blocks.

The fact that two of the 4 blocks were zero is not quite
accidental. Matrix partitioning is a useful device, particularly
when some of the blocks are zero.

Vectors may also be partitioned. Again, once the partitioning is

defined, an index will be useful in indicating a sub-vector.

Example

r' [5 6 8 0 0 OJ
r'
-1
[5 6 8]

r'
-2
[0 0 OJ
Another name for a partitioned matrix (vector) is a composite
matrix (vector).

2.17 Multiplication by partitioning

Let the matrix A be partitioned into two block-rows

Let A be of order m by n, and Ai and A' of order m1 by nand m2

by n. Similarly, let B be of order n b~ k, and be partitioned
into two block-columns.

B1 and B2 being of order n by k1 and n by k 2 .

MATRIX NOTATION 23

The product A B may then be evaluated as:

(2.17.1)

Proof

Take any element of A B to be indicated as the (i,j) element

(i th row, jth column). This is the inner product of the ith row
of A and the jth column of B, by both the left-hand and the
right-hand side of (2.17.1). This is so, irrespective of the
additional information, given by the right hand side of (2.17.1),
that a row of A is a row of the first/second block-row of A, and
a column of B is a column of the first/second block-column of B.
q.e.d.

The statement is hardly a theorem at all, but merely a corrollary

of the definition of matrix multiplication.

Example

A B
[ 1.1 1.2 1.3
1.2.44] fl
-1] -S'OJ
II
2.1 2.2 2.3 -1 [ 5.0
9.0 -9.0
-3.1 -3.2 -3.3 -3.4 -1 -13.0 13.0
-4.1 -4.2 -4.3 -4.4 -1 -17.0 17.0

The 2 by 1 positive block of the product A B

[ 5.01
9.0J

may also be obtained as AiBl

1.2 1.3 [ 5.01

P·l 1. ;]
2.1 2.2 2.3 2.4 9.0J

m
The case discussed above, could be compared with the outer
product of two vectors.

A corresponding analogy with the inner product of two vectors

exists as well.

Let A (of order m by n) be partitioned as

A
24 CHAPTER II

where the two block-columns Al and A2 are of order m by n l and

m by n 2 . Let B (of order n by k) be partitioned as

where the block-rows and Bi Bi

will be of order n l by k and n 2
by k. The product A B may now be evaluated as

A B Al -Hi + A2 Bi (2.17.2)

Again the statement hardly is a theorem at all.

Example

[;
2
6
-3
-7 -~J
P j]-1 -1

U ~J [~ ~J + [-3
-7 -~J
-8
[-1
-1 -~J
-1

[Ii 113J + [ 157 l~J =

[1026 l~J
26

The two types of partitioning may be combined, and A and B may

be partitioned as follows:

A and

The product A B may then be evaluated as

A B = All Bll + A12 B2l : All B12 + A12 B

22J
[
---------------------~--------------------
A2l Bll + A22 B2l ~ A2l B12 + A22 B22
(2.17.3)
The above relation assumes that all the expressions are legitimate,
if they are, (2.17.3) is true, the two sides are equivalent.
MATRIX NOTATION 25

It appears that we can treat blocks more or less like elements,

and block-rows like rows and block-columns like columns.

Generally, we may obtain the (i, j) block of the matrix A B as

the product of the ith block-row of A and the jth block-column
of B.

[A BJ 1,J
.. A~B. (2.17.4)
1 J h

Above it is assumed that A is partitioned in a number of block-

rows, and B into an equal number of corresponding block-columns.
It is also assumed that A is partitioned into p block-columns
and B into p corresponding block-rows.

Exercise

Evaluate the products A B and B A where

~] lHtHl
A 2 1 B
rl 4

l~ I
2
4
231
4 5 -
-
I

Do not forget to take advantage of the triviality of multipli-

cation by a unit-matrix (block)!

6 £L¥'-yZ61]
91 Z1

ZZ 6
~
[-----
O£ L
£1 11 ~ £ V H

HV

2.18 Differentiation of matrix expressions

Rules for differentiating a number of not too complicated matrix
expressions follow readily from the definition of the various
matrix operations.

We shall need to come back to this point after introducing more

complicated matrix expressions, but differentiation of the sums
26 CHAPTER II

and products of matrices and vectors is discussed here as follows:

The differentiation of a sum or difference expressions is

obvious, and is given here only for the sake of completeness.

If a matrix A is defined as being the sum of two matrices Band

e
e A + B (2.18.1)

and the elements of A and B are not constants, but variables,

we obviously have

de dA + dB (2.18.2)

Product expressions become more complicated.

From

e AB (2.18.3)

we obtain, for finite differences

e + t.e (A + M)(B + t.B) A(B + t.B) + M(B + t.B)

A B + A t.B + t.A B + t.A t.B (2.18.4)

From which we obtain (for finite differences)

t.e At.B + t.A B + t.A t.B (2.18.5)

As usual in differentiation, we assume infinitely small changes,

and we are allowed to suppress products of infinitely small
numbers. Therefore, from (2.18.5) we infer that (2.18.3) implies

de A dB + dA B (2.18.6)
n
The usual differentiation of a rational function, i.e. y = x
implies dy = nxn - l dx applies to matrices, only for a square
and symmetric matrix, i.e. for a symmetrjc matrix A, the matrix
of first difference of AP is
p-l
pA dA (2.18.7)

where AP indicates the pth power of A

'A A A ........ A

We first illustrate this rule for n 2.

MATRIX NOTATION 27

For n = 2, we obtain from (2.18.6)

d(A A) A dA + dA A (2.18.8)

Provided we also impose the requirement of symmetry on the

matrix of differences, the two terms on the righthand side of
(2.18.8) are identical, and the result is

dA 2 2A dA (2.18.9)

The more general result (2.18.7) may now be shown by recursive

induction, i.e. if we assume it to be valid for some p, we
obtain for q = p + 1, from (2.18.8)
p p-1
A dA + P A dA A

AP dA + P AP- 1 A dA
q-1
(p+1)AP d A q A dA (2.18.10)
th
The more complicated problem of differentiating the p power of
a square but non-symmetric matrix is not dealt with here.

Differentiation of compound matrix expressions is not in fact

very widely practiced. Much more common is the differentiation
of vectorial expressions in which matrices occur as known
coefficients matrices. In particular, for a symmetric quadratic
form

y x' A x

where A is a symmetric matrix of fixed coefficients, the

differential expression is

x' Adx + d x' Ax 2 x' Adx (2.18.11)

Example

A x

d Y 2J dx = 10dx 1 +

Matrix notation may also be used to obtain compact notation

of partial differentiation.
28 CHAPTER II

The significance of the operation is, of course, the same as in

"ordinary" calculus i.e. the partial derivative is the ratio
between the change in the value of a function and a small change
in one of its arguments, the other arguments of the function
maintaining a constant value.

Matrix notation comes in because we may use matrix notation to

indicate and to express as formulae, vectors partial derivatives,
and sometimes matrices of partial derivatives. The quadratic
form is again the most widely used example.

From (2.18.11), we obtain, by assuming dx.

d x. " 0, ~
°
for i " j, but
J
d y 2 x' a· d x. (2.18.12)
-J J
Hence, by the definition of a partial derivative

3y 2 x' a. (2.18.13)
-J
3x'
J
(all j)

The expression (2.18.13) is not very efficient use of matrix

notation, it was formulated purely to illustrate the relationship
with ordinary calculus. A more usual and more compact hence
more efficient - notation is

3x' Ax
2 A dx (2.18.14)
3x

It is however, quite possible to do the same for the matrix.

If a is a known coefficients vector, and X a matrix of variables,

y + ~y = ~'(X+ ~X) a a' X a + a' ~X a (2.18.15)

obviously implies

d y = a' d X a (2.18.16)

From which for d x ij " 0, d rk ° (at r " i, all k " j)

~ = ai aj (2.18.17)
3x ..
~J

or using matrix notation (an outer product)

a a' (2.18.18)
MATRIX NOTATION 29

Note, that no symmetry needs to be assumed in this case, and

moreover,

a' X b (2.18.19)
a b'
ax
is also true for a non-square matrix.

Exercise

Derive a formula for the matrix of second-order derivatives

of a quadratic form.

a2x' Ax
?

2.19 Reading and printing of large matrices by electronic computers

There is, in practice a considerable discrepancy between the

software provided as standard facilities on most computers and
what is practical for handling large matrices and tableaux.

The two procedures listed in this section are offered as a

possible way to fill this gap. (There are in fact other solutions
to this problem.)

The precise action of these procedures will be clear from their

text, and from the extensive use of comment in the ALGOL text
itself, and from the names of the labels used. But some remarks
about the processing of large matrices in general seem appropriate
at this point.

Firstly, many large matrices and tableaux e.g. linear programming

tableaux contain a very large percentage of zero elements. It is
obviously a waste of time and cards, to punch all these zeros.

Secondly, simple errors, numbers in the wrong column, decimal

errors do occur during card-punching. It seem~ wise to write
programmes, bearing that fact of life in mind. Hence the warning
messages for repeat requests at inappropriate points, and the
automatic back-print of a matrix which has just been read.

Matrix-output is by itself a problem with most standard facilities,

as these often make a ridiculously wasteful use of line-printer
paper and of the users hands and eyes, forcing him to skim through
a pile of paper in order to verify one element. Hence the print
per block-column and the splitting of large block-columns into two
square blocks per page.

The two procedures are now listed.

30 CHAPTER II

TEXT-LISTING Of THE MATRIX PRINTING PPOCEDCPE:

'PROCEDVRE' MATO(MATR.SCALE.H.N.SR.SC);
'ARRAY' MATR; 'REAL' SCALE; 'INTEGER' M.N.SR.SC;

'BEGIN' 'INTEGEF.' NP.DO.NRTD.NCOLDO.NCOLTD.

PAGEC.RPRINP.CPRINP.I.J;

'COMMENT'
MATO STANDS fOR MATRIX OUT.
IT ts A MATRIX PRINTING PROCEDURE. PRINTING THE MATRIX
IN BLOCKS.
FOR AN 180 CHARACTER LINEPRINTER. THE ORDER Of THE ELOCKS
IS PUT AT 15 EY 15.
SR STANDS FOR SKIP RO~S.
SC STANDS FOR SKIF COLL'MNS.
SCALE IS THE SCALE-FACTOR. I.E. THE MATRIX \.IILL BE PP.INTED.
MULTIFLI ED EY SCALE.
THE AUXILIARY VARIABLES NRDO.NPTD.NCOLDO.NCOLTD.PAGEC.
PFRINP. AND CPRINP STAND FOR:
NUMBER OF P.O\.lS DONE.
NUMBEF OF RO ... S TO DO.
NL'MEEP OF COLUMNS DONE.
Nt:MBEF. OF COLL'MNS TO DO.
PAGE CONTROL.
ROW PRINT FARAMETER (DETERMINATING
THE NUMBER OF RO\,'S IN A ELOCK)
COLL'MN PP.INT PARAMETEP.
(DETERMINING THE NL'MEER OF COLL'MNS IN A BLOCK);

NRDO:=NCOLDO:=O; NRTD:=M; NCOLTDp'N;

FRINT FOR COLUMN IF APFROFIUATE:

'IF' M > IS 'AND' 1'1=1 'THEN' 'BEGIN'
~.·RI TETEXT(' ('LINn;! SElFF.ES EtJTATI mao FlA%COLL'MN ') ');
Nn:LINE(1); 'END';

START:
NEI.'L INn 2);
FlRRI NP: =CPRI NF': = I 5;

ADJVST NL'MBER OF ~'LUMNS:

'IF' NCOLTD < CFRINF 'THEN' CFRINP:=NCOLTD;
NCOLTD:-NCDLTD-CPRINF;

CHECK 1'01'. PAGE:

'II" M>15 'ArID' 1'1>15 'THEN' PAGEC:-NRDO/30-ENTIEP.(NFDO/30)
'ELSE' PAGEC:=1;
'II" PAGEC=O 'THEN' 'EEGIN'
'COMMENT'
BLOCKS ALL TO START AT THE TOP OF A FAGE.
THIS ALLOWS THE USER TO STICK THEM TOGETHER.
WHILE KEEPING ROWS AND COLL'MNS IN LINE.;
RtJNOUT; , END';
MATRIX NOTATION 31

HEADINGI
'IF' M>15 'AND' N>l 'THEN' 'BEGIN'
'COMMENT'
IN THE INTEREST OF BEING ABLE TO FIND INDIVIDUAL ENTRIES
OF THE MATRIX, A LARGISH MATRIX IS EDITED.
THIS IS DONE BY PRINTING THE INDICES OF THE LEADING
ELEMENT AT THE HEAD OF EACH BLOCK';
NEWLINE(2)J
WRITE(30,FORMAT('('SNDD')'),NRDO+l);
WRI TE( 30, FORMAT( '( 'SNDDDDD') '), NCOLDO+ 1)1 'END' I

ADJUST NUMBER OF ROWSI

'IF' NRTD c RPR'INP 'THEN' RPRINPIO'NRTD;
NRTDI-NRTD-RPRINP;
PRINT THE LINESI
'FOR' 11-1 'STEP' 1 'UNTIL' RFRINP 'DO' 'BEGIN'

NEWLINE EXCEPT FOR COLUMN:

'IF' M>15 'AND' N-l 'THEN' 'GOTO' NO NEWLINE NEEDED;
NEWLINE( 1)J
NO NE\.'LINE NEEDEDI

'FOR' J:-l 'STEP' 1 'VNTIL' CPRINF 'DO' 'BEGIN'

'IF' MATRCSR+NRDO+I.SC+NCOLDO+Jl=O 'THEN'
WRI TE( 3 O. FORMAT ( '( 'SNDDDDSS') '>' 0)
'ELSE' WRITE(30.FORMAT( '('S-NDD.DD')').
SCALE*MATRCSR+NRDO+I,SC+NCOLDO+Jl);
'END'I 'END';

ADJUST INDICES AND RETURN t~LESS R~ADYI

NRDOI-NRDO+RPRINP;
'IF' NRDO-M 'THEN' 'BEGIN'
NRTDI-MJ NRDOI-O; NCOLDO:-NCOLDO+CFRINP;
'IF' NCOLDO-N 'THEN' 'GOTO' END OF MATO;
'GOTO' START; 'END' 'ELSE' 'GOTO' CHECK FOR PAGE;
END OF MATOI 'END';

TEXT-LISTING OF THE MATRIX READING PROCEDUREI

'PROCEDURE' MATI(MATR.M.N.SR.SC);
'ARRAY' MATR; 'INTEGER' M.N,SR.SC;
'BEGIN' 'INTEGER' I.J; 'REAL' Nt'M;
'PROCEDURE' MATO(MATR.SCALE.M.N.SR,SC);
'ARRAY' MATRI 'REAL' SCALE; 'INTEGER' M.N.SR.SCI 'ALGOL ,;

'COMMENT' MATRIX READING PROCEDURE.

MATI STANDS FOR MATRIX IN.
SR STANDS FOR SKIP ROWS.
I.E. A LEADING BLOCK-ROW OF SR ROWS IS NOT FILLED.
SC STANDS FOR SKIP COLUMNS.
I,E. A LEADING BLOCK-COLUMN OF SC COLUMNS IS NOT FILLED.
M AND N ARE THE ORDER PARAMETERS OF THE BLOCK TO BE READ.
32 CHAPTER II

1011 THIN A E01J OF A MATEIX. MATI INTERFRESTS NL'MEERS

IN EXCESS OF 1000000 AS REPEAT-REQUESTS.
THE EXCESS OVER A MILL.ION IS THE Mt..'MBEP OF REPEATS. I.E.
36.52 1000100 MEANS 36.52. FOLLO\'ED BY 100 EEFEATS OF
THAT SAME Nt..'M8ER.
HENCE IN TOTAL 101 TIMES 36.52 IS FL'T IN THE MATPIX';

NL'M:=O;
'FOP.' ..1:=1 'STEP' , L"N TIL' N '00'
, FO P. ' I: = I '5 TEE' 'UNTIL' M 'DO' MATP.[SP+I.SC+JJI=O;

P.EAD THE p.m.'s:

, FO R' I: = I 'S T EF ' , UN TIL' M 'DO'
'FOP' ..1:=1 'STEP' I 'UNTIL' N '00' 'BEGIN'
'IF' Nt.:M < 1000000.5 'THEN' NUM:=P.EAD;

'I F' (..1= I 'II.NO' NUM > 1000000.5)

'OP.' (J=N 'AND' NUM > 1000001.5) 'THEN' 'BEGIN'
'COMMENT'
THERE SHOt..'LON T BE A REPEAT REQUEST AT THIS POINT;
NE1JLINE( I);
\'RITETEXT< '( 'UNEXPECTEO%REFEATREQt..'EST%OF') ')J
1JPI TE< 3D. FOEMAT<' ( 'SNDDDDD. DS')') .NUM-I 000000);
'IF' ..1=1 'THEN' 1JPITETEXT<'('AT%THE%
EEGINNING%OF%ROW') ');
'IF' J=N 'THEN' 1Jf'lTETEXT( '( 'PEMAIlJING%
AT%THE%ENO%OF%RO\l') ');
\IF'ITE<3Q.FORMAT< '( 'SNDDDD') ' ) . 1 ) ;
Nt.:M:=O; 'END';

'IF' NUM < 1000000.5 'THEN' 'BEGIN'

'IF' Nt..'M=O 'THEN' 'GOTO' CELL FILLEr;
MATP( SR+I, SC+J J: =NUM 'END'
'ELSE' 'BE'GIN'
MATR( SR+I, SC+J J : =MATR( SR+I. SC+J - I J; NL'M: = Nt..'M- );
'END ';

CELL FILLFD: 'ENC';

MATO(MATR. I. O.M.N. S~, SC);

END OF MATI: 'END';
CHAPTER III

BLOCK-EQUATIONS AND MATRIX-INVERSION

3.1 Notation of linear systems

An "ordinary" system of linear equations can be formulated in

matrix notation, as follows:

A . x b (3.1.1)

Here, A is a known matrix, b is a known vector, while x is an

as yet unknown vector. If A is of order n by n, then X-must be
of order n by 1, and b of order n by 1. The solution of such a
system consists in evaluating x in such a way that the equations
are satisfied. The evaluated v;ctor x is then indicated as the
solution vector.

3.2 Singularity

Consider the systems:

+ 3 and + 3
+ 6 + 15

The first system is satisfied by xl 0, with x 2 = 3. And also

by xl = 1, with x2 = 2; and also by xl = -500, with x2 = 503.
Any vector ~, satisfying xl + x 2 = 3, will also satisfy
2x l + 2x 2 = 6.

On the other hand, the second system has no solution at all.

The two systems have in common, a special condition of the matrix

[~ ~J
The second row of this matrix is proportional to the first.
This condition is known as singularity. Singularity need not be
so evident at a glance, as in this example. It may also be that
a linear combination of rows is equal to another row.

The matrix 5 ~s singular as well.

-3
4

A square matrix A is said to be singular, if there exists a non-

zero vector y', such that y' A = O. (There exists then also a
vector x, such that Ax = O~ but no proof of that statement is
offered-at this stage~)
33
34 CHAPTER III

In our above two examples, we have

[0 ol
[2 -lJ
D ~J
with D ~J [-i] [~l
-l3J
I~
and [14 6 5
-3
4 -Ii [0 0 01

with

[~
5
-3
4 -I] HJ r:]I.~
Some of the more obvious cases of singularity are: proportionality
of two rows (see above); proportionality of two columns; a zero
row; or a zero column.

n~ -~l
Example of singularity through a zero row:

[0 o [0 o 0]

with n 3
6
o
-~] [-Ii] [~]
It may sometimes happen, that a row of a matrix is almost
proportional to some combination of the other rows. Such a
system is known as ill-conditioned.

If A is singular, the system Ax = b cannot have a unique

solution for x, irrespective of the numerical content of b. With
an ill-conditIoned system, we will have one of the rows being
almost proportional to some combination of the other rows. In
that case a unique solution of x does exist; but it is highly
dependent on small variations in some elements of A and b.

3.3 The elimination process

A non-singular system can be solved, by means of a series of

successive elimination steps. (See also sections 1.1 and 1.2)

To this purpose, A and b are written in a tableau. The tableau

BLOCK-EQUATIONS AND MATRIX-INVERSION 35

will be a composite matrix

T
~J
The elimination step considers each tableau as representing a
new system. We can then indicate each previous tableau as T, and
each next tableau as U.

A crUC1a- I ro 1 e 1n
- the p t h . _.
e11m1nat10n . p 1 aye d b y t h e
step 1S
element t . At this stage, p-l columns of A will be unit vectors
~ (j = 1~P2, .... , p-l). The element t is then the pivotal
.
e 1 ement or p1vot. Th e f·1n d 1ng
- P PP1vot
0 f a non-zero . may requ1re . a
permutation~ows. If A is non-singular, there should be at
least one non-zero element trp (p < r). We will then, if
necessary, interchange the rth and-the pth row.

Denoting the new tableau as U, we will then have:

-1
u' t t'
-p pp -p
(divide the pivot-row through the pivot; or, what is the same:
express the equation with a unity coefficient for x p .)

And for the other rows of the tableau:

t. u' (i f p).
1,p -p
(Subtract the pivotal row, from the remaining rows, with such
coefficients, as to cause vanishment of the remainder elements
in the pivotal column; or, what is the same, eliminate x from
the remainder equations, with help of the pth equation.)P

This procedure is repeated, until all columns of A ( = the first

n columns of T), have been transformed into unit vectors.

Example:

equations system:

.517 .055 x 2 .293 x3 .804

-.077 + .686 x 2 .010 x3 .450
-.056 .022 x 2 + ~63 x3 .750

T
[ .517
-.077
-.055
.686
-.293
-.010 .80:]
.450
-.056 -.022 .863 .750

u'
-1
[1. 000 -.106 -.567 1.555] (l/.5l7)..!:.i
·077 -'!.} [ .077
[ .056
-.008
-.006
-.044 .120J
-.032 .081J
.056 ~l
36 CHAPTER III

u' [1. 000 -.106 -.567 1. 555J (copied)

-1
~2 [ .678 -.054 .570J = ~2 + .077 ~i
u' [ -.028 .831 .837J = -3
t' + .056 u'
-1
-3
equations system:

.106 x 2 .567 x3 1.555

.678 x2 .054 x3 .570
.028 x 2 + .831 x3 .837

T 1. 000 -.106 -.567

[ 1. 555J
.678 -.054 .570
-.028 .831 .837

u' [ 1.000 -.080 .841]

-2
.106 ~ [ .106 -.008 .0891
.028 ~i [ .028 -.002 .0241

~l [ 1.000 -.575 1. 644} = ~i + • 106 ~i

[ 1.000 -.080 .841] (copied)
~2
~3 l .829 .861] = ~3 + .028 ~i

equations system:

1.644
.841
.861

T -.575
1. 644J
1.000 -.080 .841
.829 .861

u'
-3
.575 ~~
.080 ~3
t
[
1.000
.575
.080
1. 039]
.597]
.083]

u'
-1 [1. 000 2.2411 = ~l + .575 ~~
u' [ 1.000 .924) = t, + .080 u
-? [ 1.000 1. 039] (c"Opied) -3
~3

equations system:

:: 2.241
::: .924
1.039
BLOCK-EQUATIONS AND MATRIX-INVERSION 37

Verification:

-.055
. 804~
.517 -.293] 2.241]
[ -.077 .686 -.010 [ .924 [ .450
-.056 -.022 .863 1.039 .750
A x b

3.4 Computational arrangements

With an electronic computer, there is no need to store any other

matrix, but T. With most computer codes, multiplication and
subtraction can be done in one operation, and the new tableau is
written over the old one, on the same place. For a human computor,
such a procedure would quickly lead to errors. At the very least,
a human computor will have to write out n successive tableaux
T. More prudent (= less prone to errors), would be the writing
of 2n -1 tableaux, each second tableau consisting of the
intermediate vectors t. u ' , together with the corresponding
vector u ' . This was doAePyg paragraph 3.3. In that example, the
first ro~ of every second tableau was the new pivot-row, ~~. In
that way, computation of the successive vectors runs from
the top of the page, to the bottom.

But this arrangement of the tableau is not the most orderly one.
It would seem more logical, that u ' should also be the pth row
of the intermediate tableau. -p

The intermediate tableau is of course a matrix in its own

right, to be denoted as S.

s e
-p
(t
-p
- e t )t-lt '
-p pp pp-p
(3.4.1)

The two terms of (3.4.1) represent two separate expressions

from the previous paragraph. The first term represents the new
pivot-row, up = tp~ t p ' together with a unit column, serving as
an operator, in oraer to complete the row, with a set of zero
rows, to a matrix of the required order. The second term
represents the set of vectors -ti,p ~p' Without the restriction
i f p; this set of vectors would be -!p tpb !p (there would be
n rows). Then!p - ~p tpp is the pivotal column, with the
pivotal element itself replaced by a zero.

Example: (previous paragraph, first elimination step)

s 1. 934 [.517 -.055 -.293 .804J

38 CHAPTER III

([-.077
.517]
-.056
[~] .517) 1.934 [.517 -.055 -.293 .804J

[~l
[1. 000 -.106 -.567 1.555]

[-. ~77]
-.056
[1. 000 -.106 -.567 1. 555J

r:~~~O
~=~~51
-.106 -.567

L----
+ [-~O77 -.008 -.044 -~120]
.056 -.006 -.032 .087

[1.000
.077
-.106
-.008
-.567
-.044
1. 555]
.120
.056 -.006 - .032 .087

It is of course not suggested, that this elaborate calculation,

should be written out in full at every iteration. One writes
~ as the pth row of S, and then writes out the remainder rows
o S, in the reserved blank space.

Mathematically, it is also possible to simplify (3.4.1), quite

a bit in fact:

S ~ (1 + t pp ) - .!.0 t~~ ~ (3.4.2)

Same examp 1 e :

~l
rl.---517]
S 1. 934 [.517 -.055 -.293 .80~
- [.517]
-.077
--- -.056

The transformation of T and S, into the new tableau U T, will

then be:

u (I - e e')T + S (3.4.3)
-p t>
The expression (I - e p e'), ~s again an operator; it is a unit
matrix, from which the -gth diagonal unity element has been
replaced by a zero. The whole operator serves to delete
(replace by a zero row), the pth row of T, the old pivot row.
BLOCK·EQUATIONS AND MATRIX·INVERSION 39

3.5 The sum-count column

A special aspect of the arrangement of the tableau, for the

purpose of manual computation, is the use of a sum-count of all
the elements of a row. This will become an additional column of
the tableau. The elements of the "check" column are the sum of
all other elements in the same row. One can treat this column
as any other column in the tableau. It is of course never
transformed into a unit vector. All operations on the tableau
involve: multiplication of a row with a certain number, and
adding one row to another. These operations do not disturb the
sum-count of the row. The "check" column remains the sum of
the other columns.

Then, each time one has just computed a new row, one will
verify the row-count. Is the element in the "check" column
still the sum of the other elements in the row? If the
difference is within the tolerance of a rounding-off error, one
will adjust the elements in the "check" column. If a larger
difference is found, a mistake has been made. That mistake can
be:

a) in the sum-count itself- the row is correct after all.

b) in the newly computed row (do not forget the element in the
"check" column).
c) in an earlier part of the computation; such a mistake
remained undetected, either because the row-count was not
verified at all, or because the answer was misread for the
expected one.

The equation-system of section 3.3, could then be written and

solved as follows:
Xl x2 x3 check
[ -.077
.517
-.055
.686
-.293
-.010
.804
.450 .973J
1.099
T

-.056 -.022 .863 .750 1.535

[1.000
.077
-.106
-.008
-.567
-.044
1.555
.120
1..145
882] s
.056 -.006 -.032 .087 .105

[1.000 -.106
.678
-.':..67
-.054
1.555
.570
1.882]
1.194
U T
(next iteration)
-.028 .831 .837 1.640

[ .187J s
.106 -.008 .089
1.000 -.080 .841 1. 761
.028 -.002 .024 .050
40 CHAPTER III

[1.000 1.000
-.575
-.080
1.644
.841 1.2.069J
761
U T
(next iteration)
.829 .861 1. 690

[ 1..163
172J
.575 .597 s
.080 .083
1.000 1.039 2.039

[1.000 1.000
1.000
2.241
.924
1.039
3.24
1. 924
2.039
J U
(end result)
T

3.6 The system A Z B x

Consider the equations-system:

.867 Yl - .066 Y2 .244 xl + .150 x 2 + .722 x3 + .556 x 4

-.150 Yl + .850 Y2 .561 xl + .173 x 2 + .278 x3 + .222 x 4

When both y and x are unknown, it will not be possible to

"solve" the system, in the sense of finding a unique numerical
value. But when A is square and non-singular, it will be
possible to express y into x. (Expression of x into y will be
possible, only if B is square and non-singular.) The expression
of y into x can be obtained, with the help of the elimination
procedure,-developed in paragraphs 3.3 -3.4.

Only, the tableau T will not just consist of a matrix A, and a

vector, but of the matrix A, and another matrix B. The tableau
will be a composite matrix, consisting of two block-columns;
the matrices A and B, and of course there could be a "check"
column.

For our numerical example, we will then have:

Yl Y2 xl X2 x3 x4
check
[ .867 -.066 .244 .150 .722 .556 2.473] T
-.150 .850 .561 .173 .278 .222 1.934

[ 1.000 -.076 .281 .173 .833 .641 2.852] s

.150 -.011 .042 .026 .125 .096 .428

[1. 000 -.076

.839
.281
.603
.173
.199
.833
.403
.641
.318
2.852J
2.362
U T

[ .076
1.000
.055
.719
.018
.237
.036
.480
.029
.379
.214]
2.815
s
BLOCK-EQUATIONS AND MATRIX-INVERSION 41

[1.000 .336 .191 .869 .670 3.066] u T

1.000 .719 .237 .480 .379 2.815

The outcome, Z [.336 .191 .869 .670] x

.719 .237 .480 .379

is known as the reduced form of the system Ay = Bx. This assumes

that A is square and non-singular. If the order of A is n by n,
there will be n elements in y. The number of variables in x can
be any number, say m. Then B-must be of order n by m. The order
of the reduced form matrix will be the same.

Exercise

Check (by performing the calculation illustrated above), that

the reduced form of the block-equation

[ 2 -lJ
-1 1
Z 2J
4
~ , is correctly calculated as
Z = [47 106J
x

3.7 The inverse

Consider the following equations-system:

xl + 2x2 + 3x 3 Yl

xl + x2 + 2x3 Y2

xl + x2 + x3 Y3
Or, more generally,

A x (3.7.1)

where A is a square and non-singular matrix; x and Z unknown

vectors of the same order as the matrix.

Like the system Ay = Bx discussed in the previous paragraph,

this system cannot be "solved", in the sense of evaluating x.
Both x and yare unknown v~ctors. But it is possible, to infer
from (3.7.1), a matrix expression for ~ in Z' of the type

x BZ (3.7.2)

This is done by performing the elimination process, described

in the previous paragraphs:
42 CHAPTER III

check

[: 2
1
1
3
2
1
1
1
1 n T

-~ ]
s
[-i
2 3 1
-2 -3 -1
-1 -2 -3 -1 -7

[ 1
2
-1
-1
3
-1
-2
1
-1
-1
1
1
-i]
-3
u T

[
-2
1
1
-2
1
1
-2
1
1
2
-1
-1 -n s

-n
1 -1 2 u T
[ 1 1 1 1 -1
-1 -1 1

[ -1
-1
-1
-1
1
1
-1 ]
-1
s

n
1 1 -1 1

-1 1 1 u T
[ 1 1 1 -2 1
1 1 -1

And we have found:

1
-2
1

In this form, we have a new system

By'" (3.7.3)

We may perform the same operation again, to obtain an

expression of x in y. But we already know that expression, since
A x = y was given in the first place, by (3.7.1). The relation-
ship of A and B, by (3.7.1) and (3.7.3), is named an inverse
one. The matrix B is the inverse of A. This relationship is
written in symbolic matrix notation as B A-I, but we still
need to give a formal definition.
BLOCK-EQUATIONS AND MATRIX-INVERSION 43

The following questions need an answer:

Can the operation as indicated earlier in this section (known

as inversion by row operations) always be performed? And if not,
what can be said about the class of matrices on which it can be
applied? Are there any matrices for which inversion by row
operations is not possible, but for which the relation indicated
in this example may nevertheless exist?

To answer these questions, we begin by stating a somewhat more

formal framework.

Definition

Two square matrices A and B (of equal order n by n), are said to
satisfy the inverse relations for x and Z, if they satisfy

Ax z (3. 7. 1) and BZ x (3.7.3).

The relations Ax y and By x may then themselves be

indicated as the (vectoral) Inverse relations.

This definition does not actually require that x and yare non-
zero vectors, but we would normally think in terms of-the non-
trivial case x I 0, Z I 0, as otherwise no requirement is stated.

Definition

The square matrix B is said to be the inverse of another square

matrix A (of the same order), if

Ax Y implies BZ ~, for all vectors x

(the inverse-relations to hold for all x).

Note that two matrices may satisfy the inverse relations for one
vector x and corresponding Z but not for another vector x.

Example

A
[l ~]
2
1 B
n 2
-1
-;] [~] . m
x Z

n
1 2

[l ~l [~l m· -~ m [~J
2 2
1 -1
1 2
(the inverse relations) ,
44 CHAPTER III

but for x

m they do not apply

~] m m·n
2 2

[1 1
1
-1
2 -;] m m f

Some properties of inverses and non-singular matrices~

Firstly, the definition of the inverse matrix says that the

inverse relations are to hold for all vectors x.

That includes unit vectors x = -J

e. (j = 1, ... n).

Hence, if B is the inverse of A

A I A implies BA I

A matrix premultiplied by itS' inverse, yields the unit matrix

as product

I (3.7.4)

This property is an obvious practical test in checking whether

a calculated inverse has been correctly calculated.

Example

A
[1 ~l
2
1
-1
A
n 1
-2 ~ (calculated above)

n
1 1 -1

-1] [; ~1 [~ ~
1 2
-2 1 1
1 1 l~

But if by error, the minus sign is omitted from the top

lefthand element of the "inverse" we have

-; i] [i
1 -1 1
i ~lIJ [:- i ~J
1 1
f= [:
-
~ =] 1

The following property is stated here without proof at this

stage.

Every non-singular matrix has an inverse, (a proof of this

statement follows in section 5.5).
BLOCK-EQUATIONS AND MATRIX-INVERSION 45

Furthermore, from (3.7.4) we obtain, upon premultiplication

by a row-vector y', of appropriate order:

z' (3.7.5)

The lef!ihand side expression of (3.7.5) can be non-zero, only

if y' A is non-zero, hence y' f 0 implies y' A-I f 0, i.e.
all-inverses are non singular-

Therefore, if (as will be shown ~n section 5.5), all non-

singular matrices have an inverse, all inverses have an inverse.
-1
Pre-multiplication of (3.7.4) by the inverse of A and
re-application of (3.7.4) for the inverse results in:

A (A-I) -1
(3.7.6)
The inverse of a matrix, is the matrix itself

Since all inverses are non-singular we have the following

corollary:

A singular matrix does not have an ~nverse

Furthermore, direct application of (3.7.4), for the inverse and

substitution of A for (A-l)-l yields

A A- l I (3.7.7)

The product of a matrix and its inverse, is a unit matric

irrespective of the use of pre- or postmultiplication.

We have so far spoken of "the" inverse, without actually proving

that such a matrix is unique. In fact inverses are unique
functions of the inverted matrices, and we have~e following:

Theorem

If Band C are both square and of the same order as A, and both
the inverse of A, they are equal to each other

Proof

If both Band C are inverses of A, this implies, according to

(3.7.7),

A B A C I - I [0] (3.7.8)
46 CHAPTER III

-1
Since A has an inverse we may require, for any z' , y' = z' A ,
regardless of whether that inverse is B, C or possibly a-third
inverse of A.

Therefore

I.' [A B - A C] z' [B - C] (all ~')

This condition can only be satisfied for all z', if B - C is a

zero matrix, the left-hand side being zero by-(3.7.B).
q.e.d.

Theorem

If, for some square matrices A, B and X of equal order, X being

non-singular

A B X X 0.7.9)

is true

Then

Irrespective of what is initially assumed about the (non)

singularity of A and B, A and B are each other's inverses

Proof:
-1
Denote, for any ~, X ~ as z
-1
Postmultiply (3.7.9) by ~ = X c to obtain

AB~ c (3.7.10)

Re-name cans x and Bc as I., and we see that

B c Bx I. ~mplies A B ~ AI. c x

From which

B x I. implies A I. x (all~)

Showing that A is the inverse of B

q.e.d.

In terms of recogn1z1ng inverses, the most widely used

application of this theorem is that of the unit matrix product
property.

A B I 0.7.11)
BLOCK·EQUATIONS AND MATRIX·INVERSION 47

not only follows from the inverse relationship - 1.e. (3.7.4)

and (3.7.7) but also proves it.

We now come back to the question of invertability by row

operations. If the algorithm of inversion by row operations is
understood as "pivoting" along the main diagonal, then there
are non-singular matrices, which cannot be inverted in that way.

Example

This is a non-singular matrix. Its inverse exists, it is in

fact A itself as may be shown by verifying the unit-matrix
product property

[~ ~J [~ ~J [~ ~]
But plainly this matrix cannot be inverted along the main
diagonal. However in section 3.10 we will develop an adaptation
of the algorithm of inversion by row-operations, and that
adaptation is capable of inverting all non-singular matrices.

Exercise

Verify (by inversion by row operations, as well as by pre- and

post-multiplication) that the inverse of

A
[~
1
1
2
i
l
] is correctly
evaluated as
[~3
1
1
2
i]-l
I I I
= [-~ _~
NB This matrix may be inverted by row-operations, pivots being
taken on the main diagonal.

3.8 Inverse and reduced form

Consider again the equation-system:

AZ B x (3.8.1)

It is assumed, that A is a square and non-singular matrix; no

similar assumption need to be made about B. The first order
parameter, the number of rows of B, must be equal to the order
of A; otherwise the expression A y B x would not be
legitimate. -
48 CHAPTER III

Now pre-multiply (3.8.1) with A-I, the inverse of A:

A-I A Z A
-1
• B~, or

A-I B x ••...••••.•..•.••....... (3.8.2)

The matrix A-I B, and the corresponding block-equation (3.8.2),

is the reduced form of (3.8.1). Despite the apparent symmetry
between left- and right-hand side of (3.8.1), it does not
follow, that the expression B-1 A is meaningful. No assumption
about non-singularity of B was made; the number of columns in B
may even be different from the number of rows.

Example

(See also section 3.6) Compute the inverse of A.

Yl Y2 check
[ .867 -.066 1. 80lJ T
-.150 .850 1.000 1. 700

[ 1.000 -.076 1.153 2.077J S

.150 -.010 .173 .313

-.076 1.153 2.0771 u T

.840 .173 1.000 2.013J

[ .076
1.000
.015
.208
.091
1.192
.1821
2.400J
S

1.168 .091 2.259J

1.000 .208 1.192 2.400

[ 1.168 .091J B [ .244 .150 .722 .556J

.208 1.192 .561 .173 .278 .222

[ .336 .190 .868 .6701

.719 .237 .480 .379J

the same result as obtained in paragraph 6 by direct operation

on the rows of the full tableau.

3.9 Multiplication instead of division:

the all-integer elimination method

This method is useful, in case the initial coefficients matrix,

which defines a block-equation, consists of integer numbers
only.
BLOCK-EQUATIONS AND MATRIX-INVERSION 49

The "normal" elimination procedure. is related to algebraic

substitution; it starts with expressing a particular equation
and the corresponding pivotal row, in a particular variable,
with a unity coefficient.

Instead of dividing the pivotal row by the pivotal element, one

may also multiply the rest of the tableau by that same element,
avoiding the use of fractional numbers.

The usual procedure of subtracting an outer product is then

performed with the old pivot-row and the old pivot-column, the
outer product of which subtracted from the rest of the new
tableau.

One may avoid the unnecessary generation of too large figures,

by a later division of most of the tableau, by the pivotal
element, after the next elimination step.

Example

Let us assume, we wish to invert the matrix

A = [1;-s s
-7
3 ~J
or equivalent, solve the system

2 Yl + S Y2 + Y3 zl
11 Yl 7 Y2 + 8 Y3 z2
-s Yl + 3 Y2 + Y3 z3
with respect to l.

We now proceed as follows:

Initial tableau

Yl Y2 Y3 zl z2 Z3 I Ratio
2 5 I I 9 1
11 -7 8 I 13 1
--s 3 I 1 1
50 CHAPTER III

Multiply rows 2 and 3 both by t11 2, to obtain

Yl Y2 Y3 zl z2 Z3 I Ratio
2 5 1 1 9 2 (not divided)
22 -14 16 2 26 2 )
-10 6 2 ·2 2 ) multiplied

Subtract 11 times row 1 from row 2 and add 5 times row 1 to

row 3:

Yl Y2 Y3 zl z2 z3 Ratio
L..
2 5 1 1 9 2
- -69 5 -11 2 -73 2
- 31 7 5 2 45 2

Multiply rows 1 and 3 by -69:

Yl Y2 Y3 zl z2 Z3 I Ratio
-138 -345 -69 -69 -621 -138
-69 5 -11 2 -73 -69 (diagonal cell)
-2139 -483 -345 -138 -3105 -138

Subtract 5 times row 2 from row 1 and 31 times row 2 from row 3:
Y
1 Y2 Y3 zl Z
2 3
Z
Ratio
-138 -94 -14 -10 -256 -138
-69 5 -11 2 -73 -69
-638 -4 -62 -138 -842 -138

At this point, we note that both rows 1 and 3 admit for division
by 2, without losin3 their all-integer nature. This is systematic,
and related to the "ratio" column. This column gives the ratios
between the rows of the tableaux developed the all-integer method
and the corresponding "fractional" tableaux as used so far. For
reasons to be explained in section 5.8, it is always possible
to "scale down" this column to the entry in the pivotal row.

Accordingly, we now write a new tableau, we divide row 1 and

row 3 by 2

Yl Y2 Y3 zl z2 z3 y Ratio
-69 -47 -7 -5 -128 -69
-69 5 -11 2 -73 -69
-319 -2 -31 -69 -421 -69
BLOCK-EQUATIONS AND MATRIX-INVERSION 51

We now again start the third step with multiplication: we

multiply rows 1 and 2 with -319

Y2 Ratio
22011 14993 2233 1595 40832 22011
22011 -1595 3509 -638 23287 22011
-319 -2 -31 -69 -421 -319

Add 47 times row 3 to row 1 and subtract 5 yimes row 3 from row
2 to obtain

Ratio
22011 2139 138 -3243 21045 22011
22011 3519 -483 345 25392 22011
-319 -2 -31 -69 -421 -319

Divide rows 1 and 2 by -69 and obtain:

y y y z z z . I
1 2 3 1 2 3 Ratio

-319 -31 -2 47 -305 -319

-319
-319
-51
-2
7
-31
-5
-69
-368
-421 I -319
-319

The lnverse may now be obtained as:

A
-1 1
= -319
r-31
-51
L -2
-2
7
-31
471
-5
-69J
r0.160
O. 097
LO.006
0.006
-0.022
0.097
-0.14~l
0.016
0.216J

3.10 Row-permutation during inversion

In section 3.3 we mentioned the possibility that a system of the

specification Ax = ~ may need re~arrangement of the rows of A,
in order to find a non-zero pivot.

This is true for inversion by row-operations in general. We will

illustrate this problem in connection with the calculation of an
inverse

Example

~l
1
A -1
1 -2J
52 CHAPTER III

One way to find out that an inverse exists, is to calculate it.

We take the equations-system

x2 + 2x 3 Yl

xl x2 Y2

2xl + x2 2x3 Y3
We interchange the first two equations

xl x2 Y2
x2 + 2x3 Yl

2xl + x2 2x 3 Y3
And proceed as before.

Eliminate xl:

xl x2

x2 + 2x3

3x2 2x3 +

Eliminate x 2 :

xl + 2x3 + Y2

x2 + 2x3

8x3

Eliminate x3:
1
xl 4Y l + 2Y2
1
+ i;Y
1
3

x2 I
i;Y l - 1
2Y 2 + i;Y 3
I

x3
3
iiY l + i;I Y 2 - iiY
1
3
and we have established the inverse relationship

-1
1
~]-l
1 -2 [! -ll
BLOCK-EQUAnONS AND MATRIX-INVERSION S3

3.11 Block-Elimination

Consider the following partitioned system

+ (3.11.1)

wher: All is a square and non-singular block of the full

matr1x A.
-1
We pre-multiply the first block-equation in (3.11.1) by All '
to obtain

-1 B x.. A (3.11.2)
11 1
Re-ordering of the terms in (3.11.2) yields a reduced form
expression for ~l in terms of ~2 and y
-1 -1
~l All Bl x.. All A12 ~2 (3.11. 3)

Substitution of this reduced form expression for ~l into the

second block-equation of (3.11.1) yields:

We now combine (3.11.4) with (3.11.2), to obtain:

-1 -1
~ 1 + All A12 ~2 All Bl x..
-1
[A22 - A2l All A121 ~2 = [B2 - A2l
If we now compare (3.11.1) with (3.11.5) we note a remarkable
similarity between the elimination process which we found
applicable to a system of equations in numbers and a system of
block-equations in blocks and partitioned vectors.

We might as well write the operation down in a tabular

arrangement, as follows:

~l ~2 ,.

All A12 Bl

A2l A22 B2
54 CHAPTER III

~1 ~2

-1
I -1
I

I
AU A12 AU B1

-1 -1
A22-A21AU A12 B2-A 21 AU B1

The rules for block-elimination are very similar to the ones

which we found applicable for the solution of a system of
ordinary equations by elementary row-operations.

Recall the rules for ordinary elimination

a) Find, in the pivot-column a non-zero element in a row

corresponding to a not yet used equation

b) If necessary, permute rows, so as to have this pivotal

element on the diagonal

c) Multiply the pivotal row by the reciprocal of the diagonal

pivotal element

d) Add a multiple of the updated pivotal row to the other rows,

with the negative of the relevant element in the pivotal
column as multiplier, thereby causing the element in the
pivotal column to become zero

The similar rules for block-inversion are:

a) Find in the block-pivot column a square and non-singular

block, in a block-row corresponding to a not yet used
block-equation

b) If necessary permute block-rows, so as to have this pivotal

block on the diagonal

c) Pre-multiply the pivotal block-row by the inverse of the

pivotal diagonal block

d) Add the updated pivotal block-row, pre-multiplied by minus

the relevant block in the pivotal block-column, to the
other rows, thus causing blocks in the pivotal columns to
vanish.

Block-elimination, or block-pivoting is useful, in particular if

a partitioned matrix has zero or unit matrix blocks.
BLOCK-EQUATIONS AND MATRIX-INVERSION ss
Example

"Calculate" the inverse of:

where both A2 ,1 and Al ,2 are square and non-singular.

The partitioned form of the block-equation

M x (3.11.7)

is tabulated and we perform the elimination as follows:

~l ~2 Yl Y...2

[ I

I
I

~l ~2 Y...l Y...2

I A12 I

~2lAE> -A2l I

~l ~2 Yl Y..2
I A2l -l

I -A -1 -1
12 A2l

It is now readily verified that

S6 CHAPTER III

3.12 Inversion of recursive products and transposes

The inversion of a recursive product of several matrices gives

rise to a remarkable change in their ordering.

If A and B are square and non-singular matrices of the same

order, we have the following formula

(A . B)-l -1 -1
B • A (3.12.1)

i.e. we write the inverses of the individual matrices, but in

inverse order.

To show that (3.12.1) is indeed the correct expression for the

inverse of A B, we only need to mUltiply B-1 A-I by AB.

This proof can be given either by pre-multiplication, or by

post-multiplication. We give both:
"1"1cat10n
By pre-mu 1 t1P " 0 f B- 1 A-I byAB
:
-1 -1
A • B • B • A I (3.12.2)

The obtainment of the unit matrix as the product is (one of the)

definitions of the inverse, therefore (3.12.2) shows that A . B
and B-1 A-I are each others inverses. Similarly, post-
multiplication of B-1 A-I by AB gives:

-1 -1 B- 1 . B
B . A . A . B I (3.12.3)

Example

Calculate the inverse of

A . B
[~ [~
2
;
~ ~l 1~1
0
1
4
1
0 10 2~J
Manual calculation of the inverse of a 3 by 3 matrix which is
full of non-zero elements would take some time. The triangular
matrices are rather quicker to invert, especially because
computation is simplified by the diagonal unity-elements

o o
[-~5. ~]
-1
1 1
4 -4
BLOCK-EQUATIONS AND MATRIX-INVERSION 57

-2
-~
2 -1

[~ 1
0 ~l [~ 1
0

and we establish the inverse we were looking for as:

[[~ ~] [~ m~ =[~ -~
0
1
4
2
1
0

-22
-1
1
0 H~ 0
1
-4

[ 30
-2; 17
-4 -~
The correctness of this result is shown by verification of the
inverse relationship

[l l~J -~ [~ ~
2 -22 0
[ 30
5 -2; 17 1
10 26 -4 0

The example also suggests another relationship

[AI] -1 [A-I] , (3.12.4)

r'
i.e. the inverse of the transpose of a matrix, is the transpose
of its inverse. Once we establish

[~ [-~ ~
0 0
1 1
4 -4

the inverse relationship

[~ -~
-2

~
-1

[~
2
1 1
0 0

can be written down without further calculation.

That this is indeed so, is shown by writing the normal block-

equation system

Ax (3.7.1)

in transpose form, i.e.

x' A' z' (3.12.5)

58 CHAPTER III

Post-multiplication of (3.12.5) by the inverse of Al yields

lil Al [Alr l = Xl Zl [AI] -1 (3.12.6)

Transposition of (3.12.6) will give us:

X [ ']-1 )
(A I
(3.12.7)

where ([AIJ-l)' is the transpose of the inverse of AI.

However (3.7.1) inverted directly, gives us:

-1
X A Z (3.12.8)

Since A is square and non singular, there is one and only one
inverse of A. But (3.12.7) shows ( [AI]-l) I to be the inverse
of A. Therefore we conclude
-1
A (3.12.9)

3.13 The differentiation of an inverse

From the inverse relationship A A-I = I, we obtain (see

section 2.18 for the rules for differentiating the product of
two matrices)

+ [0] (3.13.1)

where the symbol [0 ] indicates a null matrix. From which we

obtain after re-ordering and pre-multiplication by A-I.

(3.13.2)

Just as in the case of differentiating the power of a matrix

a more simple formula which is akin to a scalar expression,
arises in the case of a symmetric matrix i.e.

(3.13.3)

may be written as

(3.13.4)
-1
by analogy to dx = -x-2 dx.

The most obvious application of (3.13.2) arises in case of

uncertainty concerning the precise magnitude of the elements of
A.
BLOCK-EQUATIONS AND MATRIX-INVERSION 59

For example;

some of the elements of A may be obtained as a result of some

kind of statistical estimation procedure, which also gives
confidence intervals (uncertainty margins). Thus, A might be,

A [ 1
0.41
0. 50
1
1 ± [-
0.10
o~ 101
J
If A has the estimated content, we find

1. 25 -0.63]
[ , therefore, a first order
-0.63 1. 25
-1
approximation of ~A is

[
1. 25
-0.63] ~A [
1. 25 -0.63J
-0.63 1. 25 -0.63 -1.25

where ~A may be any.- matrix of which the diagonal elements are

zero, and the off diagonal elements up to 0.10 in absolute
value.

Exercise

Derive a differential expression for the reduced form matrix

A-I B.

3.14 Text of an Inversion Procedure

The programmed procedure offered in this section computes a

reduced form, as defined in Section 3.6.

The process of computation is substantially the elimination

process outlined in sections 3.3 and 3.4.

The following points may now be elucidated here. Firstly, as

indicated by the 'COMMENT' in the relevant loop, we may get the
inverse as a reduced form if we put a unit-matrix as right-hand
side. (See also Section 3.7).

In Section 3.10 we discussed the issue of zero elements on the

main diagonal. When performing manual computations, it is
practical to restrict the permutation of rows to the condition
that a zero is found on the main diagonal.
60 CHAPTER III

The programmed procedure always interchanges rows, even if this

may turn out to be a trivial operation. The loop opening with
'COMMENT' FIRST FIND A NON ZERO PIVOT; amounts to a search for
the absolute largest element in the pivotal column, on or below
the main diagonal.

If that search comes up with something which doesn't

significantly differ from zero, the matrix is assumed to be
singular and the inversion is abandoned. Otherwide the absolute
largest element is found in the row with index r, and the
diagonal row has the index k, the index of the pivotal column.

The operation just below the label PERMUTE:

amounts to interchanging the kth and rth row. That operation is

trivial, if the element which is largest in absolute value is
found on the main diagonal.

The actual elimination step is performed just below the label

UPDATE: At this point there is a difference of substance with
the procedure outlined in the earlier sections of this chapter.
It so happens that no further computational use is made of the
pivotal column, nor of any column with lower index, to the
left of the pivotal column, once the elimination step has been
made.

Since the end-result is purely in terms of the right-hand side,

the updating of the pivotal-column and the rest of the tableau
further to the left, is suppressed. There is no point in
calculating numbers which are not required as results. The
interpretation of the calculation-tableau as a system of
equations on the lines of sections 1.1 and 3.3 does not hold
for such a partially updated tableau. Nor does the sum count
property (see Section 3.5), hold for a partially updated
tablea~. For these reasons, the use of this short-cut is not
recommended for manual calculation.

'PROCEDURE' INVECA.M.N)J
'VALUE' NJ 'ARRAY' AJ 'INTEGER' M.NJ
'BEGIN' 'INTEGER' I.J.R.K; 'REAL' P.NUM;
'COMMENT'
INVERSION BY ELEMENTARY ROV-OPERATIONS;

' I f ' N=O 'THEN' 'BEGIN'

'COMMENT'
N EQUAL ZERO MEANS THAT CALCl'LATION OF THE INVERSE
RATHER THAN THE REDUCED fOP.M I S ASKED.
THEREfORE VE EQUATE N TO M AND THE RIGHTHANDSIDE MATRIX
TO THE UNIT MATRIX;
BLOCK-EQUA nONS AND MATRIX-INVERSION 61

N:t:::MJ
'FOR' 11 .. 1 'STEP' 'UNTIL' M '00' 'BEGIN'
'FOR' JI .. I 'STEP' I 'UNTIL' M '00' ArI.M+Jll-O;
ArI.M+Ill=U 'ENO'; 'END';

'FOR' R:"I 'STEP' I 'L'NTIL' 1'1 'DO' 'BEGIN'

'COMMENT' FIRST FINO A NON-ZERO PI VOT;
1':" 0;
, FO it ' I:" R 'S T EP , , UN TiL' 1'1 'DO'
'I F' ABS(AC I. Rl) > ABS(P)
'THEN' 'BEGIN' KI=I; PI"'ACI.Rl 'ENO';
'IF' ABS(P) < 0.0001 'THEN' 'BEGIN'
'COMMENT'
IF THIS LOOP IS GONE THROUGH THE MATRIX HAS BEEN
FOUNO SINGULAR; 'GOTO' SINGULAIH 'ENO';

n:RMUTE:
'FOR' J:=R 'STEP' 'UNTIL' M+N 'DO' 'BEGIN'
NUM:=ACR.JH ACR.Jl:=ACK.JH ACK.Jl:=NUM; 'END';

VPDATEI
'FOR' J:=R+I 'STEP' 'UNTIL' M+N '00' ACR.Jl:=ACR.Jl/P;
'FOR' 1:=1 'STEP' I 'L'NTIL' R-I.R+I 'STEF' I 'UNTIL' 1'1 '00'
'IF' 'NOT' ACI.Rl = 0 'THEN'
'FOR' J:=R+I 'STEF' I 'UNTIL' M+N '00'
ACl.Jl:=ACI.Jl - ACR.Jl*ACI.RH 'END';
'GOTO' END OF INVE;

SINGULAR:
NE1.'LINE( I)J
VRITETEXT('('SINGt'LAR%~ITH%ORDER')');
VRITE(30.FORMAT( '( 'SNDDDD') ').1'1);
END OF INVEI 'END';
CHAPTER IV

SOME OPERATORS AND THEIR USE

4.1 The summation vector

We met the term operator already in Section 2.15 (and by way

of casual use before definition already in Section 2.6). An
operator is a matrix, or a vector, which is not defined for
the sake of the numerical information given by its coefficients,
but in order to perform some kind of transformation on another
matrix or vector. For example, we may define a .summation vector
~, of n unity element,

1
1

s
-n

If A is of the order m by n, A s is the column-vector of the

row-totals of A, and s' A is thenrow-vector of the column-totals.
-m
>~
For example, we may have an input-output table matrix

where the first block-row and block-column refer to the

producing sectors. The input-output convention that column
total equals row-total is then expressed as

s T' s
-n 1 -n
where [Tl] , is the transpose of the first block column Tl
and Ti is the first block-row,

*See my book Forecasting Models for National Economic Planning

[19], Chapter III.

62
SOME OPERATORS AND THEIR USE 63

e.g.

T 40 20 50 20 130 50 -10
45 45 115 23 50 20 2
50 30 40 10 20 -10
30 30
100 120 80
35 55

and

[40
45
20
45
50
115
20
23
130
50
50
20
-l~J 1
1 [30~J
300
1
1
1
1
1

as well as

[40 45 50 30 100 1
35J [300J
20 45 30 30 120 55 1 300
1
1
1
1

4.2 The aggregation matrix

Let us have the following table of statistical data:

Years Industries

1 2 3 4 5 6

1960 15 201 5 19 73 49
61 16 200 9 18 74 50
62 18 201 20 18 73 51
63 17 199 25 16 81 53
1964 19 200 29 16 83 52

As it happens, the interest of a research worker may be only

on the totals of industries 1, 2 and 3 together, 4 and 5
together, and 6 alone.
64 CHAPTER IV

The relevant transformation of the data is defined by the

aggregation matrix

(a) (b) (c)

(1) 1
(2) 1
(3) 1
(4) 1
(5) 1
(6) 1

'r
and we have

[15
16
18
201
200
201
5
9
20
19
18
18
73
74
73
49]
50
51
1
1
1
225
239
92
92
91
4]
50
51
17 199 25 16 81 53 1 241 97 53
19 200 29 16 83 52 1 248 99 52
1

The aggregation matrix has a quite typical structure. For each

pth column vector of the original data, to be aggregated into
the qth column vector in the aggregated matrix of data, there
is a unit-vector as the pth row in the aggregation matrix, with
the unity-element in the qth place.

Obviously, one may also aggregate rows. In that case one has to
pre-multiply the data-matrix with an aggregation matrix of
corresponding structure. Such a row-aggregation matrix will
have unit-vectors as columns, the aggregation of the pth row of
the original data-matrix in the qth row of the transformed data
matrix is then represented by the pth unity column of the
aggregation matrix having the unity-element in the qth position.

4.3 Vector-permutation

The term "permutation" applies to the relation between two

vectors or two matrices which contain the same numerical
information, In a different arrangement, e.g.

and

With matrices, we may distinguish between a column-permutation,

the interchanging of two columns, and row-permutation, the
 SOME OPERATORS AND THEIR USE 65

interchanging of two rows, e.g.

,~] ~
2 3 2 4 3l
[l 6
10
7
11
versus 6
10
8
12 liJ

is a column-permutation, and

[l
2
6
10
3
7
11
j] versus
~
6
2
10
7
3
11 ,~]
is a row-permutation.

Because of its use in the next chapter on determinants we must

also define the term single permutation.

Two matrices of the same order and containing the same numerical
information, differ by one (single) permutation, if the one may
be transformed into the other, by interchanging two adjoining
vectors (rows, e.g. columns of the matrix) ..

This definition includes the permutation of two adjoining

elements of a vector, which is a matrix, of which one of the
order parameters is known to be unity.

One may obtain more complicated re-arrangements of the elements

of a matrix, by means of a succession of single permutations.

The number of single permutations needed to transform a matrix

A into another matrix B, is the same as needed to transform B
into A; the same permutations may be performed in the reverse
order.

We may then speak of the number of permutations by which B

differs from A, without having to specify the transformation of
A into B or vice versa.

Two matrices (of the same order and numerical content), are
different by q permutations, if one needs q interchangings of
adjoining vectors (e.g. q single permutations), in order to
transform the one into the other.
66 CHAPTER IV

Permutations may be defined as matrix-multiplication, by means

of operators.
.
To ~nterc h ange t h e ~.th an d r th row 0 f a matr~x,
. we may pre-
multiply that matrix by a permutation-operator, which is a
matrix of unit vectors, which differs from the unit matrix
in that the ith and the rth row have been interchanged.

Example

[~ I
I

(The first and the fourth row have been interchanged).

The reader will note, that the operator can also be obtained h
by the interchangement of the first and the fourth (i th and rt )
column of a unit-matrix of appropriate order.

Permutation operators never consist of anything else but unit

vectors; one single unity-element and all zeros in the rest of
the column (row).

Column permutation may be defined in an analogeous way, as post-

multiplication by a permutation operator.

The use of operators will now enable us to state and proof some
theorems on vector-permutation.

Definition

A permutation operator ~s a square matrix, of which each row as

well as each column consists of n - I zeros and a single unity
element, n being the order of the operator.

Theorem

All rearrangements of the rows of a matrix may be obtained by

means of pre-multiplication of the matrix by a permutation-
operator.

Proof

Consider the transformation of the matrix A into the matrix B,

by re-arrangement of the rows.
SOME OPERA TORS AND THEIR USE 67

We postulate the existence of a transformation-operator T,

(of some description), satisfying

B T • A (4.3. 1 )

By the nature of the row-rearrangement operator, A and B must be

of the same order. Therefore T must be square, its order is the
number of rows of both A and B. The assumption that such an
operator-matrix T actually exists, is at this point arbitrary.
But if we show that an operator T, of certain characteristics,
actually satisfies (4.3.1) for all A and B, its existence is
shown in the process. We therefore investigate the properties
which T must satisfy, in order to comply with (4.3.1).

For each a~, for each ith row of A, there must be a new
row index-i, to become b ' , the rth row of B.
-r

This operation is defined by pre-multiplication of A, by an

operator T, which has a unit vector as the rth row, with the
unity-element in the ith position.

If there are m rows in A, there will also be m rows in B, and

we need m unit-vectors as rows of the operator, to define the
m rows of B.

Also, the ith row of A must be transformed into one and only one
single row of B. To this purpose we need one and only one single
unity-element in the i th column of the operator. This makes the
operator into a permutation-operator as defined.
q.e.d.

Corollary

Any rearrangement of the columns of a matrix may be obtained by

post-multiplication of the matrix, with a permutation-operator.

In that case the transformation of the jth column of a matrix A,

into the kth column of a matrix B, is defined by the kth column
of the operator, which has the unity-element in the jth row.

Exercise

Find the permutation-operator P, which transforms

11
[ 21
12
22
13
23
l4J into P. [1121
24
12
22
13
23 l4J
24 = [31
11
32
12
33
13 34J
14
31 32 33 34 31 32 33 34 21 22 23 24
CHAPTER V

DETERMINANTS AND RANK

5.1 Determinants and Minors

The determinant is a scalar function of a square matrix. It is

conventional to indicate the determinant by the symbol for the
matrix, placed between two vertical lines e.g. if A is a square
matrix, we write IAI for the determinant of A. The determinant
is best defined recursively.*

For a matrix of order 1 by 1, the determinant is the one single

element, e.g.

I[7] I 7

For matrices of order 2 and upwards, we have the recursive

relation
n

j
L1 (-1) l+j a lj I Alj I (5.1.1)

where n is the order** of A, and AI' is a matrix of order n-l,

namely A less its first row and itsJjth column. The determinant
of such a matrix of smaller order is called a minor.
Example

A 1 2 o
3 -4 5
o 6 o

* As a result many proofs concerning determinants are

obtained by means of induction or recursive application,
e.g. the theorem is true, as a restatement of a definition
for matrices of order 1 or 2, but can be shown to be true
for matrices of order k+l, if it is assumed to be true and
valid for all matrices of order k. The proof is then
carried k = 1, 2, 3, ..... etc.

** All matrices in this chapter'will be square and of order n,

unless otherwise stated.
68
DETERMINANTS AND RANK 69

We now develop the minors corresponding to the non-zero

elements of the first row of A.

-4101 - 30

3101 o
Hence we find

1 " - 30 + 0 - 30

The concept of a minor may be generalized.

If A is a square matrix of order n, the minor IAi' I is the

determinant of a square matrix of order n-l, whic~ is A, less
the ith row and the jth column.

The minor IAijl may also be indicated as the minor of aij where
a"1J is an element of A, on the intersection of the ith row and
the jth column.

The corresponding expression IAij I (_l)i+j which often occurs in

the development of determinants 1S known as the co-factor
of the element ai,j'

Hence for A 1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16

we have 1 3 4 c 3 ,2 1 3 4
5 7 8 5 7 8
13 15 16 13 15 16

The square matrix, from which the determinant is taken will be

indicated by its two indices*, 1.e.
. A less t h e 1. th row Rnd t h e J. th co 1 umn.
A.. 1S
1J
* There obviously is a potential possibility of confusion
between minors and blocks (see Sections 2.16 and 2.17). It
should, however be clear from the context, whether Aij is the
intersection of the i th block-row and the jth block-column
(a block) of A, with deletion of the ith row and the jth column
(a minor matrix). In practice, block-notation and determinant-
formulae do not occur together, at least not at this elementary
leve 1.
70 CHAPTER V

The matrix Aij will be called a minor-matrix. If no confusion

between the minor matrix and the minor, which is its
determinant, is possible, we may also use the term "minor"
for both the minor-matrix and its determinant.

It will be useful, to have a symbol for a minor of a minor.

I !Aij Irkl will be the minor of Aij' arising from deletion of

the rth row and the kth column from the original A.

e.g. A 11 12 13 14
21 22 23 24
31 32 33 34
41 42 43 44

11 12 14
31 32 34
41 42 44

31 32
1 41 42 1

If A is of order n, then IIAi j Ir k is the determinant of a I

matrix of order n-2, which i~ A, 'less its i th and its rth row,
and less the jth and kth column. Note that both sets of indices
refer to the original matrix A. It follows that

I IA.1,J·1 r, kl I IA r, kl·1, J·1

(5.1.2)

Theorem

For any square matrix A, of order n,

n ..

.
I a .. (_l)1+ J IA··I
1 1J 1J
(5.1. 3)
J =

holds for i = 1,2, ....... n.

i.e. we may develop the determinant of A, by any row, not

just the first one.

Proof

For i = 1 the statement coincides with the definition of a

determinant.
DETERMINANTS AND RANK 71

For n > i > 1 the proof ~s as follows:

Suppose the theorem is true for matrices up to the order n q,

e.g. n = 1, n = 2 ........ n = q.

Then for n = q + 1, we may evaluate IAI, by its definitional

expression (5.1.1), but evaluate each of the minors Alj
by means of (5.1.3).

e.g. ~ I : aik(-l)i+k-l/ I Aljl rk / +

n
+ L aik(-ni+k-21IAljlrkl (5.1.4)
k=j+l

The difference in the exponent of -1 arises because aik belongs

to the (i-l)tb row of Alj, the first row of A not being part
of Alj.

For k > j a similar effect arises for the columns. The element
ai k belongs to the kth column of A, but to the (k -l)th
column of Alj. Hence the exponent of -1 in the second group
of terms is ~ + k - 2.

For IAI we then obtain, by (5.1.1) and (5.1.4)

jAj jElalj(~~ 1 aik(-l) i +k-l/ IA I

lj ik
I
1

+ k.I+l a ik <-1)i+k-ZljA1j ikll) I (5.1.5)

(i = 2, 3, ...... n)

Now note, that in (5.1.5) each term contains a cornmon factor

alj aik· Therefore we change nothing if we place aik outside
the brackets and alj inside.

The summation is then over k 1 to n

and j 1, 2 ........ k-l
and j k+l ......... n.

This is equivalent to the original, as this expresses ~n both

cases the requirement j f k.

( For J. = k, we cannot exc 1 u d e t h e k th column of the or~g~na

. . 1
matrix from A .. since this column is not part of A .. ).
~J ~J
72 CHAPTER V

We obtain

(5.1.6)

(i = 2, 3 ......... n)

However, by (5.1.2) we have

II A1j I ik I II Aik 11j I

Hence we obtain

k
n
L1 a ik j
(k -~ 1
1 a 1j (-1)
i+ j -11 IAikl1j I
n
+ L alj(-'Ji+j-ZIIAikl,jl) (5.1. 7)
j=k+1

The expression within brackets is nothing else but IA.kl,

developed by the minors of its first row by the origiAa1
definition of a determinant.

Then, on the assumption that the theorem is true for n q,

it is shown to be true for n q + 1 as well.

However, for n q = 2 we have

-a 21 \2 + a 22 all

-a211A211 + a 22 1A221
which shows the theorem to be true for n q 2.

Hence it is true for n = 3, n = 4, etc.

q.e.d.
DETERMINANTS AND RANK 73

Example

A 1 2 3
4 5 6
7 8 9

Develop IAI by the minors of the first row.

IAI
1 I~ ~I - 2 I~ ~I
+ 3
I~ ~I
1 (5 x 9 - 8 x 6) - 2 ( 4 x 9 - 7 x 6)
+ 3 (4 IC 8 - 7 " 5)

-4 (2 x 9 - 3 x 8) + 5 (1 :lC 9 - 3 x 7)
- 6( 1 '18-2",7)

-4 12 31 + 5 11 3 - 6 1 2
I
!8 91 17 9 7 8

which is IAI. developed by the minors of the second row

Theorem
n i+l
i
L1 a·l(-l)
~
IA·11
~
IAI (5.1.8)

e.g. we can develop IAI by the minors of the first column, in

the same way as we may do for the first row (any row).

Proof

Develop IAI by the minors of the first row

~L a l · (-1) l+jl A l · I
j 1 J J

Suppose the theorem to be true for n = q.

Then for n = q + 1 we may develop all the minors, except All

by the first column

We may now re-write IAI as

+ ~L al·(-l) l+j IAl · I

j 2 J J
74 CHAPTER V

and hence
n n
IAI all IAlli + L a 1j (-1) l+j L ail (_l)il IAi111jl
j=2 i=2

all IAll I + I
. 2 d 1j
n
a. (-1) i+1 L a (-l)j II Ai l l 1j I
~= j=2

Now note that, by the original definition of a determinant, we

find:

developed by its first row.

Hence
n
+ '\'
L a' l
i 2 ~

j
L1
n
a .. (-l)
~J
i+1 1
A' l
~
I

Hence, on the assumption that the theorem is true for n q,

we have shown that it is true for n = q + 1.

However, for n = 2, we have

which shows the theorem to be true for n 2.

Hence it is true for

n = 2 + 1 = 3, n = 3 + 1 = 4 .........•.. etc.
q.e.d.
 DETERMINANTS AND RANK 75

By analogy to our proof for the development of a determinant

by the minors of an arbitrary row, we also have:

n
I a .. (-1) i+j I A .. I
~J ~J
(5.1.9)
i 1

Conclusion

The determinant may be developed by means of any row or any

column.

Exercise

Develop 1 2 3 , by the first row, and also by the third

456 column. The answers should obviously be
789 the same (zero).

5.2 Permutations of determinants

Under this heading we discuss the determinant of any square

matrix B, which may be obtained from another matrix A, by means
of vector-permutation (see section (4.3)) as well as one case
of elentswise reordering, the transpose.

Theorem

The interchanging of two adjoining rows causes a change in the

sign of the determinant, but leaves the absolute value
unchanged.

Proof
.
Let B be related to A, by permutat~on of the ~. th an d t h e
(i+l)th rows. e.g. the i th row of B is the i+l th row of A and
the (i+l)th row of B is the ith row of A, and for

r = 1, 2, ..... i - I , i + 2, ....... n

(all other rows)

th th
the r row of B equals the r row of A.
.th
Now develop I BI by its ~ row

IBI
j
I= 1
b .. (-l)i+jIB ..
~J ~J
1
76 CHAPTER V

However, by assumption, we have

b ..
1J
a.
J + 1 ,J
. and lB.1J·1 I A.1 + 1 , J·1
since B less the ith row and A less the i + lth row contains
the same numbers.

Therefore
n

j
L1 a· l .(_l)i+j
1+, J
IA.1+ 1 , J·1.

since (_l)i+j (-1) i+l+j

We have obtained:
n
\'
L a. 1 . (-1)
i+l+j
IA.1+ 1 , .
I
1 1+, J J
j

which is -I AI
q.e.d.

Corollary

If two adjoining rows of a square matrix are equal to each other,

the determinant 1S zero.

Theorem

Permutation of two rows of a matrix, whether adjoining or not,

causes the sign of the determinant to change, its absolute value
remaining unchanged.

Proof

If the ith and the (i + q)th rows are to be interchanged, this

may be effected by 2. q - 1 simple permutations. There are q - 1
rows between the i th and the (i + q)th row;

we need to interchange the ith row with the (i + l)th, the

(i + 2) th, ... and the (i+ q - 1) th in order to obtain a matrix
in which the (original) ith and the (i + q)th are adjoinging rows.

These are q-l simple permutations. It then takes one simple

permutation to interchange the (i + q)th row with~e (now
adjoining) old i th row, and again q - 1 simple permutations to
interchange the old (i + q)th row with the (old) (i + q -l)th
row, the (old) (i + q -l)th row etc., in order to obtain a
matrix with the (old) (i + q)th row in the ith position.
DETERMINANTS AND RANK 77

The total number of simple permutations is always odd.

The determinant is then multiplied by

<_1)[2 (q-l) + lJ - 1

q.e.d.

Corollary

The determinant of a square matrix with two equal rows is zero.

The corresponding theorem for columns is now stated without

further proof:

Permutation of two columns causes the sign of the determinant

to change, the absolute value of the determinant remaining
unchanged.

Corollary

A square matrix containing two equal columns has a zero

determinant.

Theorem

Proof

Suppose this theorem to be true for n = q, the theorem may

then be applied for the minors of a matrix of order n = q + 1.

Develop IA'I by the first row of A' which is the first column
of A
n
IA'I I a. l (-l)l+iIA'lol
111 1

As just observed, we assume the theorem to be true for the

minors, hence we may write IAil l for IA'li l

IA'I ~L ail (-1)

l+i I
Ail
I
i 1

wh~ch is A, developed by the first column of A.

Which shows that the theorem is true for n q + 1, if it is

true for n = q = 1.
78 CHAPTER V

However, for n = q 1, we have

IA'I
Hence the theorem is true for n = 1; hence it is also true for
n = 1 + 1 = 2, for n = 2 + 1 = 3, ....... etc.
q.e.d.

5.3 Proportionality of vectors

Theorem

If two matrices A and B differ only in one row, or one column

n, the difference being that one row of B is the corresponding
row of A multiplied by a scalar a, e.g. b~ = aa! the
determinant of B is a times the determinafit of-A.

Proof

Develop both determinants by the ith row.

n
IBI b .. (_l)i+j I IB .. I
~J ~J
j 1

n
I aa .. (-1) i+j
~J
lB .. I
~J
j 1

n
a I a ..
~J
I
(-1) i+j A .. 1
~J
j 1

q.e.d.

Theorem

If a square matrix C (of order n) has two proportional rows

c .. = ac . (5.3.1)
~J r~

(i " r, j = 1, 2 ..... n)

then o
DETERMINANTS AND RANK 79

Proof

Suppose two matrices A and B were the same except for two rows,
for which

b .. aa ..
lJ lJ was true for j 1, 2 ..... n
-1
b . a a ..
rJ lJ
By our previous theorem, we would obtain
-1
a·a IAI (5.3.2)

However, we may obtain B from A, by row-permutation; hence

is true. (5.3.3)

From (5.3.2) and (5.3.3) we infer A B = 0

However, the described relation is the one between C and C

itself. Hence

o (5.3.4)

is true.
q.e.d

The two proceeding theorems have an obvious

Corollary by analogy:

If a column of a matrix is multiplied by a scalar number, the

determinant is multiplied by that same number.

If two columns of a matrix are proportional the determinant

is zero.

Theorem

If two square matrices A and B, of the same order, are

different in one row only, and the difference consists in
the addition of a proportionality of one row to another row,
their determinants are equal.

e.g. if b ..
lJ
a ..
lJ
for j = 1, 2, . ..... n

and i 1, 2 ....... r - 1, r + 1, . ..... n

and b a + ya hj
ri rj
80 CHAPTER V

for j 1, 2 ..... n, and a specific r and a specific h, then

Proof

Assume the theory to be true for a particular q > 2

Then for n =q + 1

we may develop IBI by the m~nors of a row with row-index

i';' r and i "I h.

We may then apply the theorem to the minors which are of order
q and write IAI .. for IBI ... Also for such a row we have
b .. = a. . ~J ~J
~J ~J

Hence
n
I a .. <_l)i+j IAI ..
~J ~J
j 1

The theorem is therefore true for n = q + 1 if it is true for

n = 1. For n = 2 the theorem holds, since

Hence the theorem is true for n = 2 + 1 = 3, for n 3 + 1 4,

etc.
q.e.d.

Corollary by analogy

Adding a proportionality of a column to another column does

not influence the determinant.
DETERMINANTS AND RANK 81

With the help of this theorem, we can prove the following

Theorem*

A singular matrix has a zero determinant

Proof

Let A be a square matrix of order n, and let A be singular.

Then, by the definition of singularity

Z'A 0 is true for some Z' + o.

Partition A into a single and a remaining block-row; and Z'
into single element and a remaining vector.

A a'
-1
A'
-2
+ = 0 (5.3.5)

Since row permutation does not change the absolute value of

the determinant, we may without loss of generality interchange
the rows of A and ~orresponding elements of Z' .

We will then require

we further assume a'

-1
+ o.
(In the case a'
-1
o the theorem is trivial but true.)

Hence by (5.3.5) we obtain

y' A'
-2 2
-y a'
1 -1
+ 0 (5.3.6)

Accordingly, we may subtract each ith row ( i = 2, 3 .... n)

weighted with the finite proportionality factor Y.Yl-l ,
from the first row, without changing the deterfuinant.

The result is a matrix with a zero row, showing

q.e.d

* The reverse of this theorem (a matrix with a zero determinant

is singular) is also true, but its proof has to be deferred
(to Section 5.5).
82 CHAPTER V

The alternative singulari ty definition Ax = 0, 0 (not so

x "
far used in this book) also implies IAI = 0, since IA'I = o.

Example

A -1 1 0 2
2 3 0 1
4 -6 2 5 0
5 -2 2 8

This can, of course, be shown in two ways, the hard way, by

calculation of the determinant, or more easily, by finding the
fourth (bottom) row to be the sum of the three others, or

r-1 -1 -1 1 o
-6
-2
3 o
2
2
i] . [0 o o oj

Exercise

Find a suitable combination of the rows, to show the

!]
singularity of A = [1 2
4 5
7 8
5.4 Decomposability of a determinant

How many different permutation operators of a given order can

be written? At this point we recall the definition of a
permutation operator. This is a square matrix, which may be
obtained by re-ordering the rows of a unit matrix. Alternatively,
a permutation operator may be defined as a permutation of the
columns of the unit matrix; the result is the same.

Then if the order is n, there are n rows in which to place the

unity element in the first column, n - 1 rows in which one may
place the unity element in the second column, e.g. all
excluding the one in which the first element is unity.

The total number of choices is n(n - 1), ..... 2 . 1

i.e. n!

By precisely the same logic there are also n! recursive products

of n numbers in a determinant.

Developing the determinant by the first column, we have n

choices for the first factor in such a recursive product.
DETERMINANTS AND RANK 83

Then, developing each minor, we have n - I choices for the

relevant element in the first column in the minor, etc.

Example

3 choices in first column. For an we have the term alII AlII and
the group of permutation operators

1 I
I and I
I I

For a2l we have the term - a 21 IAI 21 and the group of permutation
operators

I I
1 and I
I I

etc.
The full list of permutation operators or recursive products
may be enumerated as
84 CHAPTER V

I
I

The reader will note, that the signs of these recursive products
are the signs of the determinants of the corresponding
permutation operators.

The sign of such a recursive product is dependent only on the

indices of the corresponding elements and these are the same
as the indices of the non-zero elements of the permutation
matrix.

We now come to the following result:

The determinant of a square matrix equals the sum of the

determinants of all possible elementwise products of the
matrix with a permutation operator, e.g.

+ +

I-

+ +

5.5 Determinant and inversion by row-operations*

Recall the process for calculating an inverse matrix, which we

discussed in Chapter III.

Each elimination step consists of:

(a) If necessary a row permutation, to ensure a non-zero

diagonal pivotal element. This part of the operation
changes the sign of the determinant of both the left hand
side and the right-hand side matrix.
*For further reference on this topic see also Maurer [281
DETERMINANTS AND RANK 85

(b) Multiplication of the pivotal row by the reciprocal of the

pivotal element. This part of the operation mUltiplies the
determinants of both the left-hand side matrix and the
right hand side matrix by the reciprocal of the pivotal
element.

(c) Addition of a multiple of the pivotal row to the other

rows. This part of the operation leaves both determinants
unchanged.

We start the inversion of a matrix A with a system A x Z

tabulated as

x Z
A I

and end the operation (if the inverse exists), with

Since ~he determinant of a unit matrix is always one, it

follows that, except for the sign we obtain the determinant
of A as the recursive product of the pivots.

Example

A
[~ ~J
Z
-4 (from Section 5.1) IAI -30
6

L R
xl X z x3 Yl yZ Y3 L
CD3
I~ IT]
z 0
-4 5 1
0 6 0 :1
ILl = -30 IRI 1

Xl Xz x3 Yl YZ Y3 L

I~ ~I I ~ IT]
z 1
EJ]) 3 1
6 0
ILI= I-l~
;1 IRj 1

= -30,
86 CHAPTER V

unchanged because the first pivot was one.

xl x2 x3 Yl Y2 Y3 L

il
1 2/5 1/5 2 3/5
1 3/10 -1/10 7/10
1 4/5 3/5 1 2 4/5

L 3
-30: -10
R 12/5
3/10
1/5
-1/10
I *1 -l/IO
1 : -10

xl x2 x3 Y3
L

EIJ
Yl Y2

1 0 -1/3 1 2/3
- 1 - 0 0 1/6 1 1/6
- - 1 -3/5 1/5 1/3 14/15

ILl IRI I-3/5

1~51
1 = 11 0 1/61 + (-1/3) 0
1/5 1/3

=-1/30

Exercise

Find the determinants of

II Ii il
A = 1 and of A 1

I~ 4
7
4
7

independently by two methods, i.e. by calculation according to

its definition, and by inversion or attempt to invert. (Do not
forget to swap sign when interchanging rows see section 3.10).

An obvious corrollary of the result obtained in this section is

the following:

Any non-singular matrix may be inverted by row - operations, if

necessary involving the interchanging of two rows of the matrix
as indicated in section 3.10.

If an attempt to invert by row operations has to be abandoned,

because no non-zero pivot may be found at all in the next column,
neither on or below the diagonal, this proves the singularity
of the matrix, the determinant being zero.
DETERMINANTS AND RANK 87

Furthermore, since all non-singular matrices can be inverted,

it follows that all matrics with zero determinants are singular.

We have so far defined singularity as

z' A = 0, Z' ~ o.
Since IAI
= lA' I,
it is now clear that y' = 0, y' ~ 0; A
A~ = 0, ~ ~ 0; and IAI
= 0, are all equivalent definitions of
singularity.

Exercise

Find the determinants of

1 1 2 1 -2 3
A 3 -4 , A 3 4 5 A -4 5 -6
-6 7 6 7 8 7 -8 9

independently by two methods

a) By calculating the determinants according to the definition

of a determinant
b) By inverting or attempting to invert the matrices

Do not forget to swap the sign if row-permutation is needed

(see section 3.10), and verify the correctness of any
successfully calculated inverses by premultiplication i.e.
application of (3.7.4) and (3.7.7).

5.6 The Calculation of Determinants

Two programmed procedures for the calculation of determinants

are offered in this section. To calculate a determinant
directly according to its definition is not a practical
proposition. The two major complications are the following:

a) The rules for the search for and the signs of the various
minors are somewhat complicated and would therefore lead
to a somewhat complicated programme.
b) The matrices from which minors would have to be extracted
do not as such exist, the elements of the larger matrix
would first have to be copied into a minor-matrix of the
appropriate order without the gaps for a missing row and
column.

This would require additional space-reservation in the

computer's memory. To calculate the determinant of a matrix
 88 CHAPTER V

of let us say, 65 by 65, by that method one would need space-

reservation for:

the 65 by 65 matrix itself

a 64 by 64 minor
a 63 by 63 minor of a principal minor etc ...

The second of these complications is side-stepped by following

Section 5.4 i.e. the determinant is evaluated as the sum of a
series of determinants of elementwise products of the matrix
with a permutation-operator. The first-mentioned complication
is, in a sense, also present in this approach. It is not all
that simple to generate "all possible" permutation operators
and (the signs of) their determinants. Yet that is exactly
what the following procedure does:

TEXT-LISTING OF THE FERMUTATION PROCEDURE:

'PROCEDURE' FERM(PO.N.X.SIGN.NEWSIGN.OFERATION)J
'VALUE' X.SIOtH 'ARRAY' FO; 'INTEGEP' N.K.SIGN.NE\,SIGN;
, PROC EOt'R E' OP ERAT ION;
'EEGIN' 'INTEGER' I.J;

'COMMENT'
PO IS THE PERMUTATION OPERATOR.
AN!: I S ASSUME!: TO BE 0 F OPDER N BY N

I F PERM I S CALLE!: 10/1 TH THE PARAMETER X BEING ONE.

THE wHOLE OFERATOR
MATRIX IS FIRST EQUATED TO ZERO.
EACH CALL OF PEPM WILL PERMUTE ONE COLt~N OF PO. THAT IS.
INVESTI GP.TE I F A l'NI T -ENTRY IN ANY I TH ROW OF A COLUMN
OF FO IS CONSISTENT WITH THE ENTRIES IN THAT I TH RD\,.
WHICH ARE ALREA!:Y PLACED TO THE LEFTHAND-SIDE OF THE COLUMN
WHICH IS Cl'RRENTLY BEING FERMUTATED.
AT THE EN!: OF EACH 'TREE' OF F'ERMUTATIONS. THERE IS A CALL
TO THE FROCEDl'RE OPERATION. I.E. THE OPERATION
TO BE PERFORME!: WITH THAT PARTICULAR PERMUTATION-OPERATOR.
THE FULL SET OF PERMt'TATION-OPERATOP.S I S GENERATED. BY
THE DEVICE OF PERM CALLING ITSELF.
IF SIGN "'AS ASSIGNED THE INITIAL VALUE OF ONE. THEN.
AT EACH EXI T-CP.LL TO OFERATION. SI GN WILL HAVE THE VALUE
PLUS OR MINUS ONE.
CEPENDING ON "'HETHER THE NUMBER OF FERMUTATIONS WAS
EVEN OP. DOC;

'IF' K > N 'THEN' 'BEGIN'

OFERATION;' 'GOTO' END OF PERM; 'END';

'IF' K"I 'THEN'

'FOR' II-I 'STEP' 'l'NTIL' N 'DO'
'FOR' J:"I 'STEF' , UN TIL' N 'DO' PO C I • J ] : " 0;
DETERMINANTS AND RANK 89

'rOR' 1:"1 'STEP' I 'UNTIL' N 'CO' 'SEGIN'

'ro~' J:=I 'STEP' I 'UNTIL' X-I 'DO'
'Ir' POtl.Jl I 0 'THEN' 'GOTO' ENtJLOOP;
PO ( I • Xl: " I ;
NEIOSIGNI=SIGN;
F'EP.M CrD. N. X+ I. tlEIiS I GN. NEilS 1 GN. OPERAT ION);

CHANGE SIGN SECAUSE or PERMUTATION: SIGN:= -SIGNJ

rOCl.Xl:=O;
ENDLOOPI 'END';

END or PERM.: 'ENO'J

As indicated by the 'COMMENT' this procedure caters for calling

another procedure called operation, every time it has called
back on itself with the index k - which indicates the column
to be permuted - in excess of its order. It has then generated
a particular permutation-operator, with its correct sign.

This sign should be flicked from plus to minus or vice versa

at a particular permutation in the kth column without
interference to its value to be transmitted to other minors
when permuting columns further to the left again. For this
reason this variable is not transmitted directly as a variable
but "per value". The eventual result is stored in the variable
NEWSIGN.

In the case at hand, this is the procedure TERM which is an

internal procedure, subordinate to the procedure PERD which
calculates the determinant by means of permutation-operators.

'PROCEDURE' PERDCA.N.D)J
'ARRAY' AJ 'INTEGER' NJ 'REAL' CJ
'BEGIN'
'INTEGER' I.J.SIGN.NEIISIGNJ

'FROCEDURE' FERMCPO.N.X.SIGN.NEIISIGN.OPERATION)J
'VALUE' SIGNJ 'ARRAY' POJ 'INTEGER' N.X.SIGN.NEWSIGNJ
'FROCEDURE' OFERATIONJ 'ALGOL' J

'PROCEDURE' TERMJ
'EEGIN' 'INTEGEP.' I.JJ 'REAL' TEl
'COMMENT'
THE TERM TE. VHICH TERM ADDS TO A DETERMINANT IN THE PROCESS
or CALCULATION. IS THE RECURSI VE PRODU.cT or THOSE ELEMENTS
or THE MATRIX A. FOR \IHICH THE FEF1MUTARION-OFERATOR HAS A
t'NITY-ELEMENT RATHER THAN A ZERO IN THE CORRESPONI)ING CELL.
MULTIPLIED BY THE CNE\DSIGN INDICATED.;
90 CHAPTER V

TE: =NEWSI GN;

'FOR' 1:=1 'STEP' 'l'NTIL' N 'DO'
'FOR' J:=I 'STEF' 'UNTIL' N 'DO'
'IF' POCI.JJ=I 'THEN' 'EEGIN'
'IF' ACI.JJIO 'THEN' TE:=TE*ACI.JJ
'ELSE' 'GOTO' END OF TEEM; 'END';
=
0: D+TE;
EN D 0 F TERM: 'EN D' ;

'ARRAY' Porl:N.I:NJ;

START OF THE CONTROLLING MAIN BODY OF PERD:

= 0 ; SIGN: = Ii
!):
PERMCPO.N.I.SIGN.NEWSIGN.TERM);

END OF PERD: 'END';

This is a fairly complicated way of calculating a determinant.

Calculation of the determinant as a by-product of inversion on

the lines of Section 5.5 is much simpler to programme. The
following procedure is an adaptation of the inversion procedure
which was offered in Section 3.13.
'PROCEDURE' INVDCA,M.N.DH
'VALUE' N; 'ARRAY' A; 'INTEGER' M,N; 'REAL' D;
'BEGIN' 'INTEGER' I,J,R,K; 'REAL' P,NUM;
'COMMENT' DETERMINANT-CALCULATION WITH INVERSION;
'IF' N=O 'THEN' 'BEGIN'
NI=M;
'FOR' 11=1 'STEP' 1 'UNTIL' M '~O' 'BEGIN'
'FOR' JI=I 'STEP' 1 'UNTIL' M '~O' A(I.M+J1I=O;
A(I.M+I1I=1i 'END'; 'END';

INITIATE 0: 01=1;

'FOR' R:=I 'STEP' 1 'UNTIL' M '~O' 'BEGIN'

'COMMENT' FIRST FIND A NON-ZERO PIVOT;
PI=O;
'FOR' I:=R 'STEP' I 'UNTIL' M 'DO' 'IF' ABSCA(I.Rl) > ABSCP)
'THEN' 'BEGIN' KI=Il PI"ACI.Rl 'END';

0:" O*P;
'IF' ABSCP) < 0,000001 'THEN' 'BEGIN'
SINGULAR: 'GOTO' END OF INVO; 'END';

'IF' 'NOT' R=K 'THEN' 'BEGIN'

INTERCHANGE ROWS AND CHANGE SIGN OF DETERMINANT:
01=-0;
PERMUTE:
'FOR' JI=R 'STEP' 1 'UNTIL' M+N ,~O' 'BEGIN'
NUM:=ACR.J); ACR.Jl:=ACK.JH ACK,JlI=NUM; 'END'; 'END';

UPDATE:
'FOR' J:=R+I 'STEP' 'UNTIL' M+N 'DO' ACR.Jl:=ACR.JJ/P;
"FOR' 1:=1 'STEP' 1 'UNTIL' R-I,R+I 'STEP' 1 'UNTIL' M ,~O'
'IF' 'NOT' ACI.Rl=O
'THEN' 'FOR' J:=R+I 'STEP' 'UNTIL' M+N 'DO'
ACI.Jl:=ACI.Jl - ACR.JhAtl.RH 'END';

END OF INVOI 'END' ;

DETERMINANTS AND RANK 91

The textual similarity between the INVD and INVE (See Section
3.13) is enhanced by the fact that the one was actually
obtained by reproducing and partially amending the other.

The subs tantial differences, besides "COMMENT" and the labels

are the following:

Firstly, there has to be a variable to indicate the determinant.

Since the calculation of the determinant is one of the tasks
of the procedure, this variable must be accessible outside the
procedure. Therefore it has to be a procedure-parameter. Hence
the additional parameter D.

Secondly there is the actual calculation of the determinant.

Only three instructions relate to this, i.e. the setting of an
initial value, D:=l, the evaluation of the recursive product
of the pivots, i.e. D:=D*P, and D:=-D, which relates to the
change in sign, in case two rows are interchanged or, as the
relevant label has it, permuted.

The one other difference is that no alarm-message is printed

when a matrix is found singular. This is because if someone
uses INVD rather than INVE, one assumes he will verify the non-
singularity himself, by investigating the value of the
determinant. If he has no intention of doing that, he should
use INVE rather than INVD.

If one desires the calculation of the determinant as such

rather than as a by-product of inversion one could gain a slight
gain in computational efficiency by taking the N=O loop for
putting a unit matrix as righthand side out of the procedure.
In that case the calculation would be performed without any
right-hand side reduced form calculation at all, if N=O was
supplied. However, a common feature of INVE and INVD is that
the original matrix is replaced by some intermediate result.
One is not likely to want calculation ~f a determinant and then
do nothing more with the matrix.

The other procedure for calculating the determinant by

permutation operators leaves the matrix itself unchanged.

5.7 Rank and the determinants of some structured matrices

The rank of a matrix is the order of the biggest square and

non-singular block that can be found for any ordering of the
rows and columns of the matrix.
92 CHAPTER V

Thus,

A I I

l~ ~]
2 2
3 3
4 4
5 5

~s of order 5 by 4, but its rank is only 2.

The blocks
[~ ~ , [; ~J' [~ ZJ' [; ~' [~ ~ and [~
~J
are square and non-singular. But all square blocks of order 3
or 4, e.g.

contain proportional rows and identical columns.

Obviously the rank of a matrix is at most equal to its smallest

order parameter. A matrix (or a block) of which the rank is
equal to the smallest order parameter is said to be of full
rank.

Thus, -2
~J
A = [ I
-2 I

~s of full rank, i.e. its rank is 2, and it cannot be more than

2, because there are only two rows.

Since we cannot have a negative number of non-zero elements,

the lowest possible rank is zero, in which case the matrix (or
the block) contains nothing but zeros.

The concept of rank is useful, ~n particular with proofs

concerning the non-singularity or singularity of partitioned
square matrices.

Theorem

Let

'A

be a square matrix with the diagonal blocks All and A22 being
square, and IAIII f O.
DETERMINANTS AND RANK 93

Then

IAI
Proof

Let us, for the moment assume IAI I 0

Consider the inversion of A by blocks, following the rules of

section 3.11. The associated partitioned system is

All ~l + AZI ~Z Zl

AZI ~l + AZZ ~Z Zz
We make the first block-inversion-step, as follows

~l ~Z Zl Zz

cg Al2 I

AZI AZZ I

~l ~Z Zl Zz

-1 -1
I All AZI All
-1 -1
AZZ -A2l Al! AlZ -AZlA ll I

Now recall section 5.5.

Assuming that the inversion is along the diagonal (i.e. the

matrix is ordered if needed, beforehand) we obtain the
determinant of A, as the recursive product of its pivots.

These are the determinants of All (i.e. the recursive product

of the pivots u.sed in inverting All), multiplied by the recursive
product of the pivots met in the second block-inversion step,
Le. IAzz - AZI All-l Al2I. q.e.d. for IAI 10. .
Fu.rthermore, if IAzz - AZI All-l AI?I is singular, the inversion
fails, showing IAI = o. q.e.d. for IAI = o.
Which completes the proof.
q.e.d.

Similarly, the inversion fails if no non-singular block-pivot

All can be found in the first place.
94 CHAPTER V

Therefore we have the following

Corollary

A square matrix of which any block-column is not of full-rank,

is singular

Since A = A' , we have the further corollary

A square matrix of which any block-row is not of full rank,

is singular.

Example

~l
2 3
A o o
o o
o o
the block-column

is of order 2 by 3, but its rank is only 1.

Therefore the whole matrix is singular.

For this small matrix, the fact is readily verified by

developing JAJ by the minors of the first row

0 0 0 2 0 0 2 0 0
A 1 0 0 0 -2 3 0 0 +3 3 0 0
0 0 0 4 0 0 4 0 0

2 0 0
-4 3 0 0 o.
4 0 0

The following applications are also worthwhi~e to note

If A is a block-triangular matrix with square diagonal blocks,

its determinant is the recursive product of the determinants
of its diagonal blocks.
DETERMINANTS AND RANK 95

Example

[-~ ~]
A -2 0 0
1 0 0
3 1 -2
5 -2 1
7 8 9

IAI
= I -il
1
-2 I-~ -il III (- ~ x (- 3) ~ 1 9.

The following application to what we might call a (square)

"bordered block diagonal matrix"

e.g.

is typical for a class of applications of the "full rank"

corollary. If the number of columns of (any of)the top left-
hand block exceeds the number of rows in the same block plus the
number of rows in the bottom block-row the whole matrix is
singular.

Thus if All has more columns than All and A4l together contain
rows, the first block-column of the whole matrix is of less than
full rank, showing the matrix to be singular.

Example

A 2 3 0

U J]
0 0 6
0 0 8
0 0 10
13 14 15 16

All [1 2 3J contains 3 columns, but

All [1 2 3J and A3l = [12 13 l4J

together only contains 2 rows
96 CHAPTER V

Therefore

although of order 5 by 3, is of rank 2 only, and the whole

matrix is singular.

Theorem

Let an m by n matrix A be of less than full rank.

Let A be partitionable as

(5.7.2)

where the top lefthand block All is square of order ml by ml, and
nono-singular, where ml is less than the rank of the full matrix A,
and cannot be chosen any bigger.

Then
there exists a matrix B, order ml by n-ml' for which the
following relation holds:

[A' .' ] B
[A I.2] (5.7.3)
A2 ,1 A2 ,2

Proof

Consider the equations system

AI,1 ~l + AI ,2 ~2 0

+ A2 , 2 ~2 0 (5.7.4)
A2 ,1 ~l
Because A is not of rull rank, a solution to (5.7.4) with x -I 0
must be assumed to exist.
 DETERMINANTS AND RANK 97

From (5.7.4) we obtain

0)
~l
) (5.7.5)
0)

-1
We must assume [A2 , 2 - A2 , 1 Al , 1 Al , 2J is a zero matrix.

The contra-assumption if a non-zero element in this block would

imply that we could have allocated the corresponding equation
to the top block row and the corresponding element of ~2 to ~l'
identifying a further non-zero pivot by which a larger block
Al,l could be inverted.

Therefore
-1
AZ, I Al , 1 Al , 2 A2 , 2 (5.7.6)

We now readily identify

-1
B Al,l Al ,2 (5.7.7)

and verify
-1 )
Al , I Al,l Al , 2 Al,l B Al ,2
)
) (5.7.8)
-1 )
A2 , 1 Al , 1 Al , 2 A2 , 1 B A
2,2 )
confirming (5.7.3)

q.e.d.

The following is simply a different formulation of the same

theorem:

If A is not of full rank and contains a block-column which is

of full rank and also of the same rank as A, all other columns
of A, not belonging to that block-column, can be expressed as
linear combinations of the full-rank block-column. (The columns
of B describe the combinations.)

Corollary

All rows of A, not themselves belonging to a largest full-rank

block-row, can be expressed as combinations of the full-rank
block-row.
98 CHAPTER V

Furthermore, if Allis just any square and non-singular block,

rather than specifically the biggest possible square and non-
singular block, we have the following generalization.

A residual non-inverted block [A2,2 - AZ,l Al,l-l A~,ZJ is

of rank r-ml where r is the rank of A. Note that thIS
formulation of the theorem also applies to matrices which
are of full rank, i.e. if A is of full rank, then
J
[A2Z - AZI All-l A12 is also of full rank.

Theorem

Let a square and non-singular matrix A be partitioned as

follows

A
[A1,l
A,J
Al , 2

A2 , 1 AZ , 2 (5.7.9)

A3 ,1 A3 ,Z A3 ,3
(i.e. A contains at least one zero element which has been put
in the top righthand corner.)

Then we may require the partitioning to be such that

a) Al,l is square and non-singular,

b) Ei ther

AZ Z is of order zero by zero, the partitioning being

,

[2tJ
in fact A

or

, ,J
IS square and [A2 l' AZ 2 is of full rank.

Proof

The non-singularity of A implies that the top block-row is of

full rank, hence a).

Concerning b), we note that b l ) is automatically implied if the

non-zero part of the top block-row is square.
 DETERMINANTS AND RANK 99

(We may not assume that there are more rows than columns in
the non-zero part of the top block-row). Otherwise, the
invertability of A implies the existence of a non-singular
block-pivot

The existence of such a block-pivot also implies the

invertability of

hence the full rank of [A2 1 A2 2]·

, ,
q.e.d.

Corollary

If a square matrix A is partitioned as

[~J
A

non~ingularity of A, if present, implies that the rank of A2 1

is at least equal to the excess of the number of columns of '
Al,l over the number of rows of Al,l

Exercise 5.7a

[l~ l~~
For A 2 3 4
7 8 9
12 13 14 15
16 17 18 19 20

(which is not of full rank).

* Establish the rank

* Find a block-column of full rank.
* Express the rest of the matrix by (5.7.3), having first
calculated B.
100 CHAPTER V

Exercise 5.7b

[~ ~J
For B 2 3

6 (which is non-singular)
8 9

* Find suitable orderings to present the matrix in block-

triangular form (several possible orderings)

* Check the rank of the off-diagonal blocks against the last

theorem in this section (for more than one ordering)

* Calculate IBI, using square diagonal blocks.

(The re-orderings may, or may not, lead to a change in

the sign of the determinant.)

Exercise 5.7c

Which of the following matrices C is the singular one?

Calculate the absolute value of the determinant of the non-
singular one. (Explain which theorem you used to prove
singulari ty.)

C 1 2 3 4 C 1 2 3
5 6 4 5
10 12 6 7
13 14 15 16 8 9 10 11

01 6 11 B

1 1. I~
Z 1 £
91 == 01
(SI-01).(0£-BZ) Z 16 ~
~I
~
=
L
~
9
1] = II::> II
·OBOI - = IHI a~oJa~aq~ 'mo~~oq aq~ ~g ~o~
do~ aq~ ~a2 o~ s~o~ 2u1U10rpg 2U12ugq~~a~u1 sam1~ £ papaau ~I

£ Z 1
OBOI 9£ lC O£ 6 B L
9
DETERMINANTS AND RANK 101

5.8 The adjoint and its relation to the all-integer

elimination method

Let a square matrix be partitioned, as follows

r'"
a'
-n,l
We assume I All I
:'." J
n,n
0, I AI 0.
(5.8.1)

The partial
'"
inversion
'"
of A was already performed in the previous
section fol' the general case of 2 square diagonal blocks.

The application of the result from section 5.7 for Az2 of order
1, gives us a formula for the nth pivot
-1
Pn an,n - ~n,l All ~l,n (5.8.2)
Again assuming IAlll '" °
and IAI '" 0, we readily develop a
formula for the bottom righthand element of an inverse. If the
inverse is indicated by the letter B, this formula is:

b (a (5.8.3)
n,n n,n
(The element of the inverse is the reciprocal of the last
pivot).

Comparison of (5.7.1) and(5.8.3) allows us to express the

element of the inverse in IAI and IAlll

IAI , 11
b (5.8.4)
n,n

Provided IAI '" °

is true, formula (5.8.4) is in fact valid for
IAlll = °
as well as may be shown by finding a solution to the
following system:

Al,l ~l +
~l,n
x
n ~l [OJ )
)
a' + a x b 1 ) (5.8.5)
-n,l ~l n,n n n
For IAI '" 0, IAlII = 0, we must assume ~l ,n 0, as
otherwise the top block row of A would not be of full rank.
'"
102 CHAPTER V

However, if A1 ,1 1S singular, we may require

A1 , 1 ~1 = A1 , 1 ~1 A = 0, ~1 '" 0, A '" 0, ~1 '" ° (5.8.6)

Furthermore a' 1 u
---n, -1
° would imp 1y

, showing the singularity of A.

We may therefore assume a' u '" 0.

- n, 1 -1
We now assign the value °
to x , and obtain a consistent
solution for A from the last e~uation in (5.8.5)

~'n, 1 ~1 A = 1
is resolved as

A l/(~'n 1 ~1) (5.8.7)

,.
We find that

x
[~ll A (5.8.8)

is the solution vector corresponding to (5.8.5). However,

this is the last column of the inverse, showing that

° implies b
n,n
0.

Hence (5.8.4) although initially developed on the assumption

that A1 ,1 1S invertab1e, is generally valid.

Note that in (5.8.4) IA1 11 is the determinant of [All]' a

top left-hand block. It {s also the minor associated ~ith a
While we developed (5.8.4) by reference to block inversion n,n
as practiced in the previous section, it is now more practical
to revert back to the notation which was used for minors in the
rest of this chapter i.e.

b IAn , nl
n,n (5.8.9)
TAl
In fact, (5.8.9) is valid for all the diagonal elements of the
inverse: Interchanging the nth~ast) element of x with the
kth (any) element of x while performinz the same operation on
the columns of A is compensated by a similar reordering of the
rows of A, thereby bringing the kth element of both A and the
DETERMINANTS AND RANK 103

inverse back on the diagonal, now is the n,n cell. This

operation does not affect the signs of either IAI or
IAn,n I·

For off-diagonal elements, we have the usual complication of

the sign, and the generalization of (5.8.9) is

b .. (-1) i+j (5.8.10)

1,]

Note also that the interchanging of the indices i and j in

(5.8.10) implies transposition. This is because the function of
rows and columns in terms of tableau-interpretation interchange
as a result of the inversion operation. In the structural
system Ax = y, the rows are associated with the elements of
y (the equations), the columns with the elements of x. In the
inverse form y = A-I x this is the other way round.

Example

[l~
5
A -7 (from section 3.9)
-5 3

We wish (for example) to obtain the b 2 1 element of the inverse.

To be able to apply (5.8.4) directly r~ther than (5.8.10), we
put the second variable at the end and similarly the first
equation.

Initial System: Transformed System:

2xl + 5x 2 + x3 Yl 11xl + 8x3 -7x Y2
2
llxl + -7x + 8x3 Y2 -5x l + x3 +3x 2 Y3
2
-5x l + +3x 2 + x3 Y3 2x l + x3 +5x 2 Yl
The element of the inverse is obviously the same for both
systems, but we can now apply (5.8.4) to

A
11
[ -5
2
8
1
1
-i]
104 CHAPTER V

Le.
III
15 ~I 51
b 2 ,1
1"
-5
2
8
1
1 -i I 319
0.160

11 8
In the above expression the numerator is the same
-5 1
as the corresponding minor from the original matrix A: the
v~ctors to be reordered are not included in the minor.

The denominator, however, has changed sign because the inter-

changing of columns 2 and 3 causes the sign to change once, and
the reordering of the rows requires two single permutations.
We could have left the second equation in place, and the sign
of IAI would then remain the same. If that device were adopted,
the numerator would change sign, the minor now becoming

The result of this section as developed so far may now be

summarized in one sentence:

The inverse is the transpose of the matrix of co-factors,

divided by the determinant

The transposed matrix of co-factors is (i.e. the inverse

multiplied by the determinant), is known as the adjoint.

We are now in a position to explain some of the finer points

behind the all-integer method of calculating the inverse of an
all-integer matrix, which was discussed in section 3.9.
Basically, this algorithm revolves around the calculation of
the adjoint of a block-pivot. If we assume inversion along the
main diagonal, the typical calculation tableau of this algorithm
is as follows

xl x2 =
X-l X-2

-1 -1
15 k I 15 k All A12 15 k All

-1 -1
15 k [A22 - A2l All Al2] -15 k A2l AU 15 k I
DETERMINANTS AND RANK 105

where ok is the determinant of the block-pivot All. At the

beginning of each kth step, the non-pivotal rows are multiplied
by IAk,kl. From the second step onwards, the ratio of the
entries in the non-pivotal rows, to the entries in the
corresponding fractional tableau, immediately after elimination,
is IAk-l k-ll . IAk kl. Hence the subsequent division by the
previous'multiplier: (N.B.: Here IAkkl is the determinant of a
block-pivot of order k, not a minor.)

Exercise
-1
For A
-5

which is a non-singular matrix), perform the following

calculations

1) Calculate all the co-factors (-1) i+j, see

section 5.1)

D ' and of A itself

2) Write the adjoint of All =lJ

3) Calculate the determinant and the inverse, using (5.8.10).

4) Invert the matrix using the all-integer elimination method.

Check the correctness of the inversion by verifying the
unit-matrix product properly.

5.9 Commented text of the adjoint all-integer elimination

procedure

The prosrammed procedure offered in this section is an adaption

of the procedure INVD from section 5.6. The differences between
the procedure IADJ and INVD may be summarised as follows

a) The calculation tableau is specified as an integer array.

(INVD expects a real array)

b) The end result is the determinant, and either the adjoint

(for N=O) or the righthand side pre-multiplied by the
adjoint. These results are integer and to obtain the inverse
or the reduced form, a division by the determinant is
requi red.

c) No search for the absolute largeslpivot in any column is

performed, only for a non-zero entry. Calculations with
integer numbers are exact and no problem of containing
rounding errors arise.
106 CHAPTER V

On most machines this procedure would, compared to an inversion

procedure in real numbers, save a factor two in storage space,
and be somewhat quicker. And it is exact. Against these
comparative advantages stands one obvious drawback:

Integer numbers are in practice limited to a fixed number of

digits, and overflow may arise, if the matrix is large.

The text of the procedure is now listed, as follows:

, FRO C EDVR E' I A lAJ ( A, M, N, l:') ;

'VALVE' -N; 'INTEGER' 'ARHAY' A; 'INTEGER' M,N,D;
'BEGIN' 'INTEGEH' I,J,E,K,NVM,CVERSC;
'COMMENT'
INVERSION BY ELEMEtlTARY Pol,;-OFERATIONS,
101 TH CALCULATION Of THE CETEP.MINJI.NT AS BY-FPoDVCT.
VSING THE ALL-INTEGER METHOD.
N.B. THE END RESULT NEECS SCALING BY THE DETEFMINANT, AS
IN FACT THE ADJOINT HATHEP. THAN THE INVEP.SE IS CALCVLP.TED.;

'IF' 11=0 'THEN' 'BEGIN'

• COMMENT'
fOR N EQVAL ZERO THE INVEP.SE I S ASKED.
THEP.EfoRE "E EOCATE !II TO M AND THE RI GHTHANDSI DE MATRIX
TO THE UNIT MATRIX;
N: =M;
'FOE' 1:=1 'STEF' 1 'l'NTIL' M 'DO' 'BEGIN'
'FOP' J:=I 'STEP' 1 'VNTIL' M 'CO' ACI,M+JJ:=O;
ACI.M+I]:=I; 'ENC'; 'END';

INITIATE OVERSCALE AND D:

OVEF.SC := D := I;

'fOP' P:=I 'STEP' 1 'UNTIL' M 'DO' 'BEGIN'

'COMMENT' fIRST fIND A NoN-ZEP.O PIVOT;
, FO P , I: = R 'S T EF ' 1 'l'N TIL' M 'DO' 'I f' A C I , P. J I 0
'THEN' 'BEGIrl' K:=Il 'GOTo' END Of PIVOTSEAP.CH; 'ENC';

'EEGIN'
'COMMENT' If THI S LoOF I S GONE THFOVGH THE MATRIX
HAS BEEN fO"VNl:' SINGl'LAR;
'GOTo' END Of lAC';; 'END';

UJD Of PIVOTSEARCH:
, IF' R I K ' TH EN' , B EG ~ N '
'COMMENT'
I F THE FI VOT .. AS fOurlD BEL01rl RATHEP. THAN ON THE MAIN
DIAGONAL INTEP.CHANGE Rol.'S AND CHANGE SI GN OF DETERMINANT;
DI=-D;
PEP.MVTE:
'fOP.' J:=R 'STEP' 1 'UNTIL' M+N 'CO' 'BEGIN'
NUM:=ACP',JJl ACP..JJ:=ACK.JJl ACK.JJ:"NUM; 'END'; 'END';
DETERMINANTS AND RANK 107

t:'PDATE:
'FOP' J:=P+I 'STEP' 't.'NTIL.' M+N 'DO'
'FOP' 1:=1 'STEP' I 'UNTIL.' R-I.R+I 'STEP' , LIN TIL.' M 'DO'
ACI.JJ := ACI.JJ*ACR.RJ;

'FOR' 11=1 'STEP' 't:'NTIL.' R-I.R+I 'STEP' 'UNTIL.' M 'DO'

'IF' ACI.RJIO 'THEN'
'FOR' JI=P+I 'STEP' I 'l'NTIL.' M+N 'DO'
ACI.JJ:=ACI.JJ - ACR.JJ*ACI.R];

'FOR' 1:=1 'STEP' 'l'NTIL.' P-I.R+I 'STEP' 'UNTI L.' M 'DO'

'FOP' J:=R+I 'STEP' I 'UNTIL.' M+N 'DO'
ArI.JJ := ArI.JJ/OVERSe;
OVERse := ACR.RJ;

'END';

, 1 F' D < 0 'TH EN '

'FOR' J:=M+I 'STEP' 'l'NTIL.' M+N 'DO'
'FOP' 1:=1 'STEP' I 't.'NTIL.' M 'DO' ACI.JJ := -ArI.JH

END OF IAN: 'END';

108 CHAPTER V

5.10 The determinant of the product of two square matrices

Theorem

If A and B are square matrices, of equal order, JABJ may be

evaluated as

JABJ (5.10.1)

Proof

We first consider some special cases

Case la

JAJ = 0; therefore r'A = 0, for some r # °

r'AB = 0, proves JABJ = 0, q.e.d. for--case la

Case lb

JBJ
ABk
0; therefore Bk = 0, for some k #
° proves
°
JABJ ~ 0, q.e.d. for case lb

Case 2a

A is partitionable as

[+.]
A

In that case A is an operator serving to add the combination

~i2B2 of the rows of B2, the block-row of B consisting of the
non-leading rows of B, to the leading row, therefore we find
(see section 5.3)

1, JABJ JBJ, q.e.d. for case 2a.

Case 2b

B is partitionable as

.[~]
B
DETERMINANTS AND RANK 109

Now B is an operator, serving to add a combination of the non-

leading columns of A to the leading column of A. The proof
follows ana10geous to case 2a. q.e.d. for case 2b.

Case 3a

A is partitionab1e as

[~
A

In that case, A is an operator, serving to add multiples of the

leading row of B, to the other rows

~
~bll ~;1
E.21 B22

IBI
q.e.d. for case 3a.

Case 3b

B is partitionab1e

[~J
B

will now be obvious.

We now consider the non-trivial case that A and B are both non-
singular and have no special structure.

In that case we may, if necessary after re-ordering the rows

of A (and C correspondingly) and/or the columns of B (and C
correspondingly), assume without loss of generality that the
leading elements of both A and B are non-zero.

(If A had a zero column, or B a zero row no further proof was

required).
110 CHAPTER V

[~2'a~j [bll 1~i2]

I A22 !>21-1 B2~_
is then evaluated as IABI

AhB~2
(5.10.2)

where (5.10.3)

and (5.10.4)

In (5.10.2), the first second and third members are equal to

each other (=IABI) on account of the application of special
cases 3a and 3b to the second member.

For IA~21 = 0, we find IAI 0, (see section 5.7) and special

case la is applicable, therefore no further proof is required
for IA~21 = 0.

For IA~21 f 0, the fourth member of (5.10.2) is now evaluated

(using special case 2a), as

[allbll
r'
IABI
- ;;i2At~'] + ..<::i2~2l -"i2B~21
I Ah~2 AhB~2 A~2B~2

tall bll (5.10.5)

Ah~2 AhBh]
DETERMINANTS AND RANK 111

The most righthand member of (5.10.5) 1S now evaluated by

development by the top row as

(5.10.6)

Now assume that if A and B are of order n, the theorem is

valid for matrices of orders up to n-l, (i.e. of orders
1,2, ... n).

If so, IA BI may be evaulated by the righthand-side of

(5.10.6) as

IAI· IBI (5.10.7)

Since for square matrices of order 1, the theorem merely

states the identity ab =
a.b, its general proof now follows
by recursive induction.
q.e.d.

Example

A B
[~ zJ ' [~ -z]
IAI 4 - 6 = -2 IBI 4 + 6 = 10, IAI·IBI -20
IABI
I[1 ~ l~] I'
= 70 - 90 = -20.

Exercise

Verify that IA B cl IAI IBI Icl

-363

by two separate calculations, i.e. by evaluating IAI, IBI,

and Icl,
and by calculating ABC
and extracting its
determinant. IABCI
Part II
GRAPHS AND LINEAR PROGRAMMING

CHAPTER VI

VECTORS AND CO-ORDINATE SPACES 115

6.1 The co-ordinate plane 115

6.2 Functions and relations 116
6.3 How to draw the graph of a linear relation 117
6.4 Vectors, points and linear subspaces 118
6.5 Restrictions and convex sets 124
6.6 Graphical mapping of some non-linear
functions and restrictions in two-
dimensional space 128
6.7 Interior points, boundary points,
outward points and extreme points 139

CHAPTER VII

SOME BASIC LINEAR PROGRAMMING CONCEPTS 144

7.1 The linear programming problem 144

7.2 The L.P. problem in matrix notation 145
7.3 A numerical example 145
7.4 Graphical solution nf a problem with two
variables 146

CHAPTER VI II

OUTLINE OF THE SIMPLEX ALGORITHM 149

8.1 The concept of a basic solution 149
8.2 The Simplex Tableau 151
112
 113

8.3 Choosing the pivot column 152

8.4 The chojce of the pivot row 153
8.5 The Simplex Step 156
8.6 The attainment of the optimum 158
8.7 The Simplex tableau in matrix notation 160
8.8 The shortened Simplex Tableau 162
8.9 The Rule of the highest step 163
8.10 Degeneracy 164
8.11 Simplex Tableaux and vector spaces 171

CHAPTER IX

THE SEARCH FOR A FEASIBLE SOLUTION 181

9.1 The case of one unfulfilled restriction 181

9.2 Again: The choice of the pivot row 186
9.3 The choice between a number of
violated restrictions 194
9.4 Pivot-selection in Phase I: the general
case 197
9.5 The method of artificial variables 199
9.6 Non-updating of the substitute
objective function 202

CHAPTER X

MIXED SYSTEMS, UPPER AND LOWER BOUNDS 205

10.1 Variables without sign restriction 205

10.2 Equations 207
10.3 Upper bounds and the two value columns 209
10.4 Lower bounds: a question of problem-formulation 217

CHAPTER XI

DUALITY 223

11.1 Block-pivoting with inequalities 223

11. 2 The Duality Theorem 228
11. 3 Applications of the Duality Theorem 233
11. 4 The dual ratio and Phase I column selection
in the presence of several violated
restrictions 237
11.5 Dual degeneracy 240
114

CHAPTER XII

LINEAR PROGRAMMING ON THE COMPUTER 242

12.1 Name-codes and name1ists 242

12.2 Ordering of the tableau 244
12.3 Commented text of a linear programming
procedure 249
12.4 Printing a Simplex Tableau 258
12.5 Text of a complete L.P. programme 264
12.6 Revised Simplex Algorithms 267

CHAPTER XIII

PARAMETRIC VARIATION OF THE L.P. PROBLEM 273

13.1 The parametric variation problem 273

13.2 Parametric variation of the right-hand
side of an L.P. problem 274
13.3 Parametric variation of the objective
function in an L.P. problem 289
13.4 Parametric adjustment of mixed systems 293
13.5 Treating the parameter as. a variable 298
13.6 Computational implementation of
parametric L.P. 306
CHAPTER VI

VECTORS AND COORDINATE-SPACES

6.1 The coordinate-plane

The coordinate-plane is a useful visual aid when for one

reason or another, numbers are ordered in pairs, i.e. for every
number x, there is a corresponding number y. Many properties of
two-dimensional coordinate-mappings (two-dimensional vectors),
can be generalized to vectors of any order. Pairs of numbers
are represented as points in the coordinate plane. It is
convention to indicate one variable by the "right" direction
and one variable by the "upward" direction. Thus x=2, y=3 is
presented as a point which is 2 units of distance to the right
and 3 units upwards from the origin.

3.5
y
graph 6.1 a
1.0 Ty
graph 6 I b
3.0 •••••••••••••••••• 0 ;~~ 0.5

X
2.5
-'- - Lo 5
°10
~I.C; I 0 -0 <

2.0 .,,""""""" """""""""

-0. '5

1.5

• • • • • • • -I. 0

1.0 """"""""" """""""""

-I.S

00 0.5 1.0 1.5 2.0 2.5 3.0 3.5

X=-IO
Y =- 2
• . . • • . -2.0

In the above graphical mapping of x=2, y=3 the horizontal line

is the x-axis, the vertical line is the y-axis as indicated
by the arrows. All points on the x-axis have y=O in common,
and all points on the y-axis have x=o in common. The point
(0,0) (the intersection of the two axes) is called the origin.
Since x is indicated by the right-hand direction, all points to
the left of the y-axis have negative x-coordinates. Similarly,
all points below the x-axis have negative y-coordinates, because
y is indicated by the upward direction. Hence x=-l, y=-2
is in the bottom left-hand part of the coordinate plane. A

115
116 CHAPTER VI

common notation for a two-dimensional vector is to use brackets,

e.g. (2,3) means x=2, y=2 and (-1,-2) means x=-l, y=-2.

Exercise
(and preparation for later sections)

Make two series of mappings of pairs of points (x,y) in the x,y

coordinate plane (best use proper graph paper).

Series 1:
(-3,6), (0,4), (3,2), (6,0).

Series 2:
(-6,14), (-5,7), (-4,2), (-3.42,0), (-3,-1), (-2,-2), (-1,-1),
(-0.58,0), (0,2), (1,7), (2,14).

6.2 Functions and relations

A function is a relation, a "law" which tells us the value of

one variable if we know the value of some other variable or
variables. We would normally think in terms of an algebraically
describable function e.g. y = 7x 2 + 5, z = 2pq + 4 etc. This
is not strictly implied by the definition: "the temperature in
Trafalgar Square" is a proper function of time. If we know
the time we can establish the temperature in Trafalgar Square,
provided someone takes the trouble of looking at a thermometer.

The one variable is the dependent variable or function value,

the other variable(s) are the arguments of the function. Two
or more variables may also be linked by a relation. A relation is
a condition imposed on the values of two or more variables e.g.
x + y + z = 20.

It is often, but not always, possible to formulate a relation-

ship between some variables as one variable being a function
of the others, (also referred to as explanatory variables).

x + y + z = 20 can be written as x 20 - Y - z, but x 2 + y2 4

yields 2 values for x if we know the value of y.
x=±/4-y2 is not a proper function.

We therefore say that x + y + z = 20 can be written as x being

an explicit function of y and z. For x 2 + y2 = 4 it is not
possible to write x as an explicit function of y.

Relationships between only two variables can be mapped in the

two-dimensional coordinate plane. This is so, irrespective of
whether or not they can be written explicitly. The combination
VECTORS AND COORDINATE SPACES 117

of the points which satisfy the relation will normally

constitute a line, and the shape of that line is characteristic
for the relation.

6.3 How to draw the graph of a linear relation

The relat ion

a.x + b.y c
(a;&O, b;&o)
can be written explicitly, either as x being a function of y,
or as y being function of x.

The two explicit functions are:

x = cia - b/a y (6.3.2)

and
y c/b - alb x (6.3.3)

This establishes 2 points of the graph without further

calculation. For x=O (6.3.3) gives y=c/b, and for y=O (6.3.2)
gives x=c/a.

Example

2x + 3y = 12
This can be written explicitly as follows:

x = 6 - 1 h; y=O -+ x=6
y 4 - 2/3x; x=O -+y=4

:~
I
9 rap h 6. 3 a
118 CHAPTER VI

The graph makes use of the fact that (as the name suggests) a
linear relationship is represented in the coordinate plane by
a straight line. This point will be discussed in more detail
~n the next section.

If a relation is already given ~n the explicit form, i.e.

y = a.x + b (6.3.4)

the graphical mapping ~s even simpler. The point x=o with y=b
is immediately obvious. The second point can be obtained by
writing the relation in the form of x being a function of y,
i. e.

x = l/a y - b/a (6.3.5)

But even simpler is the point x=l with y=a+b, or, if a small
scale is used, (for example)

x 2 with y 2a +b
x =2
Example Y = 9

y = 2x + 5

y =5

graph 6.3 b

If a function is presented in the explicit form with the

function-value on the vertical axis, the slope of the line is
related to the coefficient for the explanatory variable. This
coefficient is the tangent of the angle between the line and a
horizontal line, e.g. the x-axis. Hence in the example, this
angle is 60 0 , since tan 60 0 =2.

6.4 Vectors, points and linear subspaces

Those who are willing to take the equivalence between two-

dimensional graphical illustration, - which will be frequently
VECTORS AND COORDINATE SPACES 119

used in the rest of this book, and the general n-dim~nRi0na]

generalization of the illustrated propert1es for granted, may
wish to skip the formal proofs in this section.

In the previous section we made use of the fact that if two

points on a straight line satisfy the same linear relation,
then all the points on the line satisfy that same relation.
The algebraic equivalent of this property is the following:

Theorem

Let two vectors of order n,

x = x * and x x **

both satisfy the linear relation

a'x = b (6.4.1)

where a' is a vector of known coefficients and b is a constant.

Then, any combination of these two vectors,

x *** p ~ * + (l-p)~ ** (6.4.2)

(p any real scalar)

will also satisfy (6.4.1).

Proof

By assumption, (6.4.1) may be written for

x = x * as
a' x * b (6.4.3)

and for x x ** as
a' x ** = b (6.4.4)

From (6.4.3), we obtain

p~' ~ * p b (6.4.5)

and from (6.4.4)

(l-p) a' x ** (l-p)b (6.4.6)

120 CHAPTER VI

Together (6.4.5) and (6.4.6) yield, considering (6.4.2),

a' x *** = p a' x * + (l-p) a' x ** = p.b. + (l-p).b = b

(6.4.7)
q.e.d.

The main reason why the above proof (which is hardly more than
a re-statement of the definition of a combination) cannot
simply be applied to the line is that we do not normally define
a line as the combination of points satisfying (6.4.1). Indeed,
when applying geometry we rarely refer to any definition of a
straight line. By Euclid's postulate, the shortest route between
two points is the straight line. This defines a straight
connecting line between two points.

The concept of distance can be defined algebraically and

measured geometrically. The distance between two vectors is the
square root of the sum of the squares of the differences between
their corresponding elements.

Hence we can write the distance between two vectors x * and x **

as
n
bt - ./1 L
j=l
(x.*
J
- x.**
J
)2 (6.4.8)

Example ...
Y
The distance between '.1
x=l, y=2, and ...
x=2, y=4
1.1

,..
'.1

... graph 6.4 a

'.1 illuacration of the
I.' distance between

., two points
..x
I'

The rectangular sides of the marked triangle are the differences

between the coordinates of the two points, i.e. 2-1=1 in the
horizontal x-coordinate direction and 4-2=2 in the vertical
y-coordinate direction. Therefore, by Pythagoras' Theorem, the
distance between the two points is

'12 2 + 12 IS
VECTORS AND COORDINATE SPACES 121

Theorem
* * ** ** *** ***
Let (x , y ); (x , y ) and (x ,y ) be three points in
the two-dimensional x,y coordinate plane. And let (x***, y***)
be related to the other two points:
*** * + **
x p x (l-p) x )
) (6.4.9)
*** * + (l-p) **
y p y y )

(0 < p < 1)

Then

I (x *** , y *** ) - (x * , *
y) I + I (x *** ,y *** ) - (x ** y**) I =

* * ** **
I(x, y) - (x ,y )1 (6.4.10)
* * ** **
i.e. the shortest route from x , y to x • y as indicated
by the righthand side of (6.4.10) is via x***, y*** and one
obtains the total distance by addition of the two separate
distances.

Proof

Evaluate the first term in the left-hand side expression of

(6.4.10) by the definition of a distance (6.4.8)
* I
I (x *** , y *** )-(x * ,y) / ***-x)
* 2 *** * 2
= Vex + (y -y) (6.4.11)
***
Substitution of the right-hand sides of (6.4.9) for x and
y*** into (6.4.11) gives us

I (x *** ,y *** ) -
*
(x ,y )
*
I =
./ * ** 2 * ** * 2
V(p,x + (l-p)x ) + (p.y + (l-p)y - y )

.J * ** 2 * ** 2
±(l-p)V(x -x ) + (y -y )

(l-p) I (x * ,y * ) - (x
**
,y
**
) I (6.4.12)

For (0 < p < 1), the positive sign is applicable, otherwise the
distance wo~ld become negative. The same method of evaluation,
applied to the distance between (x***, y***) yields

I (x *** , y *** ) - (x
**
,y
**
) = p I (x , y ) -
* *
(x
**
,y
**
)
I
(6.4.13)
122 CHAPTER VI

Together (6.4.12) and (6.4.13) give the desired result, i.e.

I(x *** ,y *** )-(x,y)I+I(x

* * *** ,y*** )-(x** ,y ** )1=

(l-p) I(x * , y)-(x

* ** ,y** )I+pl (x * , y)-(x
* ** ,y ** )1=
* *
I(x,y)-(x
. ** ,y ** )1 (6.4.14)

which is equivalent to (6.4.10.)

q.e.d.

This proof cannot be generalized directly to an n-dimensional

coordinate space. This is because we took our definition of a
straight line from Euclid's postulate.

The algebraic equivalent of a linear relation in 3-dimensional

space is a flat plane, Euclid's postulate is applicable to a
line. It is more practical to turn the proposition round.

In the n-dimensional x coordinate-space, the set of point-

vectors x (of order nY, which satisfy

a' x b (6.4.1)

constitutes a Euclidean (or linear) subspace, of order n-l.

The elements of the n-dimensional row-vector a' and the

constant b determine the position of the subspace within the
whole of the n-dimensional space.

In the three-dimensional coordinate-space, a 2-dimensional

linear subspace is more commonly referred to by its geometrical
name: a flat plane. Similarly, in the two-dimensional coordinate
space, a one-dimensional linear subspace is usually referred to
by its geometrical name: a straight line.

With these definitions, we don't any more need to prove that

3 points in the two-dimensional coordinate plane, which satisfy
a common linear relation are on a straight line. That's how
we defined the straight line in the first place. Similarly, any
set of three-dimensional vector-points satisfying the same
linear equation, fit in the same flat plane.

The following result although slightly less intuitively obvious,

can now be picked up as well:

All points which satisfy two independent equations in 3-

dimensional space, are on the same straight line.
 VECTORS AND COORDINATE SPACES 123

That line is the intersection between the planes associated

with the two equations. The line is a one-dimensional subspace
in both of the two-dimensional subspaces defined by the two
equations.

Similarly, the solution of a system of 2 equations and 2

unknowns, is the point of intersection of the two corresponding
lines.

Example

x + 2y 5
2x + y 4

5.0

'.5 Ty
graph 6.4 b
'.0

3.5
graphical solution
of two equations
3.0

2.5

2.0

1.5

1 0

\\
0.5

_o.S 00 0.5 1.0 loS 2.0 2.5 3.0 3.5 10.0 10.15 S.O 5.S

_0. s

We find the intersection of the two lines at the point x 1,

y = 2. This is the solution of the system.

In 3-dimensional space, the solution of a system of 3 equations

is found as the cornerpoint where the 3 planes intersect. If
the system is singular, no unique intersection will exist. In
the two-dimensional case, contradictory equations correspond to
parallel lines, dependent equations to overlapping lines. By
analogy to the linear subspace of order n-l, we may ~all the
set of n-dimensional vectors, which satisfy k independent
equations, an n-k dimensional Euclidean subspace. 'Clearly an
n-dimensional vector which belongs to an n-k dimensional
subspace i.e. is required to satisfy k independent linear
equations, is fully determined, once we enumerate n-k elements
124 CHAPTER VI

The remaining k elements may then be found by back-substitution.

6.5 Restrictionsand convex sets

A restriction is a statement or property concerning vectors,

which is defined in some coordinate-space. We would normally
assume a restriction to be stated in the form of some function
of the vectors in that coordinate-space to attain certain
prescribed values.

Restrictions can be in the form of equations i.e. some function

to attain a certain value and no other, or they can be in the
form of inequalities, ~.e. some function to be for example,
greater than or equal to a certain number.
2
Thus x + 2y + Z = 20 ~s a restriction of equation-type and
2
x + 2y + z ~ 20 is a restriction of inequality-type in the
x,y,z coordinate space.

We will normally assume a restriction to be non-trivial, that

is, to define a characteristic of the vector not satisfied by
all vectors in the relevant coordinate-space.
2
An example of a trivial restriction would be (x - y) ~ O.
There are no pairs of numbers x and y, which don't satisfy this
restriction.

Triviality may also ar~se relative to some other restrictions,

~.e.

x . y > 0

is not by itself trivial, but it is a trivial restriction if

the restrictions x > 0, y > 0 are-already present when
x . y > 0 is added to the list of restrictions.

A set of vectors is the collection of the vectors which satisfy

some particular characteristic or characteristics. Normally
those characteristics will be stated restrictions, equations
and/or inequalities. A vector may be a (non-negative)
combination of two or more other vectors.

x is a non-negative combination of ~l' ~2 ...

if

(6.5.1)

~s true for some set of non-negative numbers

VECTORS AND COORDINATE SPACES 125

PI' P z ... Pk ' satisfying

PI + Pz + ...•. Pk = 1 (6.5.Z)

We made use of the concept of a combination in the previous

section, for the special case of a two-dimensional vector.
If for some set of vectors it is true that every non-negative
combination of some vectors in the set is also part of the
set, then such a set of vectors is named a convex set. (For
this reason non-negative combinations are sometimes also
indicated as convex combinations.)

An example of a non-convex set of vectors is the set of vectors

whose elements are integer numbers.

This is shown by finding

x as well as x
DJ
to belong to the set,

but then combination x 1

Z [ZlJ + 1.Z [ZJ1 = [HJ
I!
does not belong to the set of vectors with integer elements

Theorem

The set of vectors satisfying

at x .s;.. b (6.5.3)

is a convex set

Proof

Take some set of k vectors satisfying (6.5.3) 1.e.

at x < b (6.5.4)
-q
for q = 1, Z .... k

A non-negative combination of the ~q is defined by stating the

values of the proportions of the combination to be derived from
any ~q, i.e. PI, PZ ... Pk
126 CHAPTER VI

From (6.5.4) we derive

(6.5.5)
which proves that the combination (the left-hand side of
(6.5.5» satisfies (6.5.3)
q.e.d.

Theorem

I f x satisfies

a'x < b (6.5.3)

and for some other vector ~ the alternative relation

~' ~ ~ b (6.5.6)

is true,
then, for some non-negative combination of x and ~, to be
indicated as z

a' z = b (6.5.7)

is true.

Proof

State the actual values of

a' x = 'Il (6.5.8)

and

(6.5.9)

By assumption 'Il - b is a non-positive scalar, $ - b is a non-

negative. Therefore a non-negative combination of them, a
number in the interval 'Il-b,$-b, is zero

o (6.5.10)

or equivalently

(6.5.11)
VECTORS AND COORDINATE SPACES 127

From (6.5.8) (6.5.9) and (6.5.11) we derive

PI ~' ~ + P2 ~' X. a' (PI ~ + P2 x.) = b (6.5.12)

Which identifies ~ as

(6.5.13)

showing the existence of such a vector q.e.d.

Obviously, if the supposed relationships (6.5.3) and (6.5.6) are

put in the stronger < and > form, and th'e word "non-negative"
is replaced by "positive non-zero" throughout the corresponding
stronger theorem is also true.

The above theorems appear somewhat trivial, but they provide a

strict proof for what is more or less intuitively obvious.

If we draw a line representing an equation, then this line will

separate the coordinate space in two parts. On the one side of
the equation the inequality of the 2 type is satisfied, on the
other the inequality of the < type is

Example

x + 2y > 6

t
5.S

5.0
graph 6.5 a
y
4.5 ILLustratIon of
an mequaLlty
4.0

3. S
128 CHAPTER VI

We first draw the graph of x + 2y = 6 (see section 6.3), and

we then have to indicate at which side of the equation
x + 2y > 6 is true, and at which side x + 2y < 6 is true.
For x =-0, y = 0, the origin we find ° °
+ > 6 untrue.

Therefore the admissable side is the top right-hand side.

We will indicate the non-admissable side by shading the line
as a "wall" which inhibits access from the admissable to the
inadmissable side.

Since there always is an "equal to" point between two points

where the opposite signs i.e. > and < are true the relation
x + 2y > 6 is untrue for all points below and to the left of
the line. And, at least as important, we can generalise this
property to the n-dimensional case.

We will therefore call the set of n-dimensional vectors

satisfying a particular linear inequality e.g. (6.5.3) a
half-space.

Any linear subspace of order n-l splits the n-dimensional

coordinate space, in which it is defined, in two half-spaces.

The set of vectors which satisfies a set of restrictions, either

equations of inequalities, consist solely of vectors which
satisfy each of them. Therefore any combination also satisfies
each of them. This gives rise to the following

Corollary

The set of vectors which satisfy a particular combination of

linear restrictions (equations and/or inequalities), is a
convex set.

If we combine "convex" and "non-convex" restrictions we cannot,

in strict logic say very much about the convexity or the non-
convexity of the set of vectorpoints which satisfies them all.

Cherefore, it appears prudent to assume that a set of vectors

which is defined by a requirement that a number of restrictions
are to be satisfied, is a convex set, only if each separate
restriction also defines a convex set.

6.6 Graphical mapping of some non-linear functions and

restrictions in two-dimensional space

Firstly, we consider the polynomial function

(6.6.1)
 VECTORS AND COORDINATE SPACES 129

There will be intersections with the horizontal x-coordinate

axis at up to p points, these points being the real roots of
the polynomial equation:
2
y = a + a x + x + .•. a (6.6.2)
o 1 p
If the equation has no real roots, the graph will not meet the
horizontal axis.

There will be up to p-l points where the graph is horizontal.

These points of horizontal slope are the real roots of the
polynomial equation:
p-l
dy = a l + 2a 2 x + ... + pa x o (6.6.3)
dx p

Similarly, there will be up to p-2 points where the curvature

of the graph changes direction.

For a polynomial of second order the graph is a parabola, and

the mapping is made somewhat easier by its symmetry property.

The quadratic function is

(6.6.4)

First we note that for p=2 (6.6.3) gives us a linear equation

dy (6.6.5)
dx

i.e. for a 2 # 0 we find a horizontal point at

x -- - '2I a l / a2 (6.6.6)

The qualification "for a 2 # 0" is trivial, Since a 2 = 0

implies a linear function.

The corresponding value for y is found by evaluating (6.6.4)

for x ~ al/a2

y ao - I
'2 a 2/ I 2/
l a 2 + 4al a 2 = a o -
I 2/ a
4al 2 (6.6.7)

The graph is symmetrical to a vertical axis going through the

point of horizontal slope. This is shown by rewriting the.
function. From (6.6.4) we may obtain

y = a 2 (x + ~al/a2)2 + ao - !a~/a2 (6.6.8)

130 CHAPTER VI

Example
2
y = 2 + 4x + x

We find the point at which the graph is horizontal, by

requiring, according to (6.6.3) and (6.6.5),

dy 4 + 2x =0
dx

i.e. x = -2
For this va lue of x, we evaluate y as

y = 2 - 4 x 2 + 4 -2

We now have found the horizontal point x = -2, y = -2. The

graph will then be symmetric relative to the vertical line
x = -2. For x = 0, we find y = 2. The symmetry property around
the vertical line x = -2 then tells us that x = -4, y = 2
is also a point of the graph.

We find two more points by assigning a predetermined value to

y. In this case y 0 gives rise to real roots i.e.
2
x + 4x + 2 0

has the roots

x = -2 ± ! 116-8 -2 ± 12
should the zero value for y not supply real roots, a different
pre-assigned value of y should be taken.

It is in any case desirable to establish a third pair of

points on the graph. For y = 14 we find
2
14 x + 4x + 2

16 x 2 + 4x + 4 = (x+2)2

x + 2 ± 4 -+ x .. -2 ± 4

It is now possible to draw the graph with reasonable accuracy

and smoothness through these points.
VECTORS AND COORDINATE SPACES 131

16

Q) y
- 6 )( I: 2
X 14
Y I: I If
I "
(J)

12
0
-n
(J) 10

I~
I~ B

graph 6 • 6 a
rt
"}
6
1'< i l l u S t rat 10 n
o f
1 y = 2 + " x + X 2
4
X
1
x =- II
Y =2

~
-6 2 4 6

horizonta poi n t
x - 2 , y = - 2
132 CHAPTER VI

An immediate generalization of the quadratic function is the

square root function.

y = a + i"X"=l) (6.6.9)

(x > b)

The graph of this function is (the upper half of) a parabola

with a horizontal axis of symmetry, y = a, and a vertical
point at x = a, x = b. For a < 0, b > 0, this relationship is
often useful to represent mildly curvelinear relations, which
become flatter for higher values of the function-argument.

The funct ion y = - ~ + ~ is mapped in graph 6. 6b

graph 6.6 b
2.5 ~Y
2.0 Illustration o f
y = - 0 • 5 + s q r t 1 + )( )

loS

1.0

0.'0
".

-1.~.1 -0.'0 00 0.5 1.0 1.'0 2.0 2.5 3.0 3.5

~
I
-- -~.~
y
-----------------------
= 0.5

The "normal" presentation of the quadratic function may be

obtained from (6.6.9) by bringing the constant term a over to
the left-hand side, taking the square of both sides, and
reordering thereafter
222
(y-a) = y - 2ay + a x-b,

or equivalently,
2 2
x = Y - 2ay + a + b (6.6.10)
VECTORS AND COORDINATE SPACES 133

The one essential difference between (6.6.4) and (6.6.9) is

that x, instead of y has become the variable which is a unique
function of the other variable.

A polynomial function of the third degree is something like

S-shaped, except that, if the function-value is indicated by
the vertical direction, the S is turned on its side. The graph
below gives a mapping of y = 1/3x 3 - x 2 - 2x + 2.

graph 6.6 c

ILLustration of the
cubic function
y = 1/3 x 3 - x2 - 2x + 2

X
I + sqrt (3) :---1
--"+-+---'--2 --9---~ ~--~-~-.--

-I

-2

-3

For sufficiently large values of x (large positive or very

negative figures, the direction of the slope of the line is
dominated by the term ~n x 3 .

Thus, if the coefficient of x 3 is pos~t~ve, the graph goes from

bottom left to top right. In the middle, there is generally a
dent and this mayor may not go as far as a change in the
direction of the slope.

If the cubic polynomial function itself is

y =a x 3 + b x 2 + c x + d,

the condition for the temporary vanishment of the slope is

dy 3 a x2 + 2 b x + C =0
dx
134 CHAPTER VI

In the example at hand, this equation is

dy x 2 - 2 x -2 =0
dx
or x2 - 2 x + 1 = (x - 1)2 = 3

This equation has two real roots,

x = 1 - 13 = -0.41 and x = 1 + 13 = 2.41

In the interval between the roots, the slope has the opposite
direction from the one indicated by the sign of the cubic term.
For certain values of the coefficients the "dent in the middle"
may not be enough to cause a reversal of the slope.

For example:
3 2
y = 1/3 x - x + 2x + 1

would have a zero slope for

2
dy x - 2x + 2 = 0
dx
2
or x - 2x +1 = (x + 1)2 - 1

i.e. for no real value of x.

In such a case, the direction of the slope will be monotonously

upwards (or downwards if the coefficient of x 3 is negative.)

We now discuss the mapping of

(x - a)2 + (y - b)2 = c (6.6.11)

(c > 0)

This is a circle with x a, y b, as origin, and a radius of

c.

Example

(x - 3)2 + ( y - 4)2 4,

or equivalently
2 2
x + y - 6x - 8y + 21 0

That ~he graph of (x - a)2 + (y - b 2 ) = c is indeed a circle

is shown by applying Pythagoras' Theorem.
VECTORS AND COORDINATE SPACES 135

The horizontal and vertical sides of the triangle drawn in the

graph are x - a and y - b.

Therefore the sloping side is V(X -

a)2 + (y - b)2. As this is
the same at each point of the graph, i.e. IC by (~.6.11), this
distance is the same for all similar triangles. Constant length
of the radius is the geometric definition of a circle.

graph 6.6 d
illustration of the
constant radius of the circLe
by pythagoras theorem.
6 -

5 -

2 5
 CHAPTER VI
136

graph 6.6 e

Illustration of a
rectangular hyperbola

I 6

,
<
C'[)

rt

~. I "
0>1
,....
horizontal las mptotic aXIs
----- ~

x
Q)
(I)
'< ~
2 4 6
rt
o
rt

_2
o
Q)
X
(I) -4

_6
VECTORS AND COORDINATE SPACES 137

Finally we discuss the mapping of an implicit relationship

(x - a)(y - b) = c

The graph of this relationship is known as the rectangular

hyperbola. It has two asymptotic axes, as may be shown by
writing the ~elationship explicitly in terms of either
variable.

x = a + c/(y - b)

For c ~ 0, x will not, for any finite value of y, attain the

value x = a; but x will approach the value x = a asymptotically
if y moves towards +00, or towards _00, showing that x = a is an
asymptotic axis. The similar statement concerning y = b will
be obvious.

Typical for the hyperbola is that the graph comes in two

branches. We will discuss this feature while assuming c > 0,
(for c = 0, the graph reduces to the cross formed by the two
asymptotic axes. The case c < °
exists and is meaningful, but
readily deduced from the case c > °
by re-defining one of the
two variables, reversing the sign of either x or y).

For x > a, y > b we consider the positive branch ~n the top

right-hand part of the coordinate space, for x < a, y < b the
factors x - a and y - b are both negative and their product is
positive. The general shape of the graph may be seen in
graph 6.6e, which contains the graphical illustration of
(x + l)(y - 2) = 9.

A considerable wider group of functions and relationships

become manageable in terms of graphical mapping without undue
effort if we introduce a secbndary coordinate system.

This is normally related to a linear transformation, i.e. we

introduce secondary variables p and q, which are linearly
related to our originally specified variables x and y by the
linear equations.

p a x + b y + C
(6.6.12)
q d x + e y + f

The use of this device ~s considerably simplified if the

secondary coordinates are also associated with a rectangular
set of axes and equality of units in both directions i.e. we
simply rotate the graph paper, - but we may still need to
change the unit of measurement - .
138 CHAPTER VI

graph 6.6 f

Illustrat Ion 0 f a secon-

i.?
dary coordinate system

,
, I>
, , ~

, ,+
, ""
,, ""
,~

',I( \) ""
, , "0 "
, .. " "
, ""
)C

,// ',+
" ,,
"
-\ "
,,
0

+ "" ,,~
"
.. "
'\.
,,
q" " , x
"
,
,
,0

1~ 12

Example

(x-y) (lO-x-y) 16

We put

p x - y
q -x - y + 10

The secondary coordinate system now has its origin p = 0, q o

at x = 5, Y 5 and the p = 0 axis (q-direction) is the x = y
line, the q = o axis (p-direction) is the x + y = 10 line,

We have no difficulty in drawing the rectangular hyperbola

p.q = 16 in this new coordinate system, see graph 6.6f.
VECTORS AND COORDINATE SPACES 139

Exercise

Make graphical mappings of:

2 2
x + y 2 x + Y = 16
2
y x + 2 y =x + x + 1

x + 2y = 6 (x-l)2 + (y_2)2 9

x - y = 3 x • y = 16

(x-y+2)(x+y-S) = 36

Use proper graph paper, one sheet for each graph. Then read an
arbitrary point from the graph - not already used in drawing
it -, and check that it approximately fits the equation which
the graph represents.

6.7 Interior points, boundary points, outward points and

extreme points

An interior point may be defined loosely as a point inside the

space occupied by a set of vectors.

Take for example, the set of two-dimensional vectors

satisfying
2
(x - 3) 2 + (y- 4) < 4.

This is the inside, i.e. the surface of the circle in the

previous section. Interior points are points which are
actually inside rather than on the ring of the circle itself.

Definition

The vector x is an interior point of some set of vectors, if

for every vector z in the same coordinate space, whether inside
or outside the set, a combination of x and z

v = p z + (1 - p) -x (6.7.1)

for some p in the interval

o < p < 1 (6.7.2)

belongs to the same set as x.

140 CHAPTER VI
I
?4 graph 6.7 a
?2 ~X
~-
I
interior pOints, boundary pOints,
2.0 ,- outward pOints and extreme pOints.
1.8
I-
I
1.6 !- boundary pOints.
1.4
the shaded lines.
interior pOints.
1.2 the inside,
1.0
excluding the boundary.
outward pOints.
0.8 - the sloping top and
the outward bulge.
0.6
extreme pOints.
0.4 the corners and
the outward bulge.
0.2 r
I
1h-l.1-+-------'------'~')L--~-------
OJ() 1.5 2.() 2 ..
~-"3.5

The requirement that p should be non-zero is the crucial point

in this definition. The vector z - x denotes an arbitrary
direction i.e. towards "any" point in the coordinate space.

Some non-zero part of the line-segment from x to z must still

be in the set. A vector which belong to a set, but is not an
interior point of the set is a boundary point.

If a set of vectors is defined by a set of non-trivial

restrictions,

f.(x) > 0 (6.7.3)

1 - -

(i = 1, 2 ... m),

points for which one or more of these restrictions are binding

will normally be boundary points. There are, however, some
rather strange restrictions which provide counter-examples.
This is because a restriction can be ineffective at certain
points, without being trivial throughout the coordinate space.

For example, x(y - z)2 > 0 is not a trivial restriction of the

kind which we already excluded.

For x = -1, y = 1, z = 2, the restriction x(y-z)2 > 0 is not

satisfied. But in the x > 0 half-space this restriction is to
all practical purposes trivial. For x = 1, y = 1, z = 1, the
inequality x(y - z)2 ~ 0 is exactly satisfied, yet this is an
interior point.
VECTORS AND COORDINATE SPACES 141

To exclude complications of this kind, we must first define

their salient characteristic.

A particular restriction

f. (x) > 0 (6.7.3)

1. - -
1.S indifferent at the point x = x*
if all partial derivatives vanish at that point
H.
1.
ax. o
(6.7.4)
J
for j = 1, 2 .... n.

For linear restrictions these partial derivatives are simply

the coefficients a .. which occur in
1.J
n
f. L a .. x. + b. > 0
1. 1.J J 1.
j=l

It therefore follows that for a linear restriction which is

non-trivial the issue of indifference at a particular point
does not ar1.se.

Theorem

Let

f. (x) > 0 (6.7.3)

1. - -
i 1, 2, .... m

define a set of vectors (points) in some n-dimensional coordinate

space.

And let one restriction in (6.7.3), to be indicated as the h-

restrict1.on be exactly fulfilled at some point x ~*, fh(x)
not being indifferent at that point.

(6.7.5)

df h
ax. f 0 for x x* (some j) .( 6.7.6)
J
142 CHAFfER VI

Then

x = x* is a boundary point of the set of vectors satisfying

(6..i-:-3)

Proof

Consider a first-order linear approximation of

fh(~) at the point ~ x*

n (lf h
fh(~* + d~) = fh(~*) + L
(lx.
dx.
J
(6.7.7)
j=l J

We may take minus the vector of first-order derivatives as the

vector z in 6.7.1 and a very small increment indicated as dp
as the proportionality factor p in (6.7.1).

Denoting the vector of partial differentials as~, (6.7.7) is

now evaluated for that particular direction as

fh(~* + ~ dp) = fh(~*) - dp~' ~ (6.7.8)

The first term on the left-hand side of (6.7.8) is by

assumption i.e. (6.7.5) zero, the second is negative. this
contradicts (6.7.3) for the h-restriction.

Since an interior point is defined as staying inside the set

when starting to move away from it. x = x* cannot be an
interior point if a restriction is binding without indifference.
q.e.d.

The reverse is also true and even more obvious:

If a set of vectors is defined by some inequalities, and at

some point ~. ~* none of these inequalities is binding. then
that point is an interior point.

The terms "boundary point" and "one or more restrictions

bindin~' are therefore in practice interchangeable.

We will call x = x* an outward point of some set of vectors,

if there is a non-zero direction-vector a' ~ O. for which

a' x < a' x* (6.7.9)

is true for all x in the set.

VECTORS AND COORDINATE SPACES 143

The requirement that a' must not be a zero vector serves

merely to exclude triviality. For a' = 0 all vectors x would
satisfy (6.7.9) for all sets of vectors and for all points x*
in them.

Outward points are also boundary points. This is because the

direction-vector z - x in (6.7.1) can be taken in any direction.
Taking the column=prsentation of a' as the direction z - x in
(6.7.1), we may re-write (6.7.1) as

v = x* + p a (6.7.10)

where the point towards which one is moving is found by

comparing (6.7.10) and (6.7.1)

z = x* + a (6.7.11)

Evaluation of (6.7.9) for the direction indicated by (6.7.10)

yields

a' ~ = ~' ~* + p ~' ~ ~ ~' x* (6.7.12)

thus contradicting (6.7.9)

The definition of an interior point is that we can move away

from it over some distance indicated by the parameter p, and
that includes moving in a particular direction indicated by
the vector a' itself.

The reverse is however, not always true: boundary points are

not always outward points (This is so, only in convex sets)

Since outward points are boundary points they are, by previous

theorem, also points at which at least one restriction is
binding.
CHAPTER VII

SOME BASIC LINEAR PROGRAMMING CONCEPTS

7.1 The linear programming problem

Linear programming consists of:

(a) Finding a solution to a system of linear

inequalities, and
(b) Selecting from all solutions to such a system, a
solution for which a certain linear function of
the variables attains a maximum value.

A solution, satisfying (a), but not necessarily (b), is

indicated as a feasible solution. An arbitrary vector of the
required order which fails to satisfy the inequalities is named,
sOlnewhat euphemistically, a non-feasible solution. A solution,
satisfying both (a) and (b), is named a feasible and optimal
solution. The function to be maximised is the objective function
or preference function.

There is of course a mirror problem to the one formulated

above. One could search for a solution minimizing a certain
linear function of the variables. It would lead to a second
algorithm, differing only in sign from the one I want to
discuss. Any minimization problem can be formulated in terms
of maximizing, by inverting the sign of the objective function.
We will only deal with maximization problems.

I will in fact impose one further restriction. * All

inequalities should be in the form of a linear function of the
variables being smaller than, or equal to a constant. Again
inversion of the sign will turn the "larger than" type of
inequality into the "smaller than" type.

A more general convention is that variables shall be restricted

to non-negative values only. Historically, linear programming
dealt with physical activities, and this convention was self-
evident. Adherence to this convention also helps to simplify
certain features of the Simplex Algorithm. In a later stage
we will also discuss unconstrained activities. A variable,

* Where it concerns the signs of the coefficients, I follow the

conventions adhered to by Charnes, Cooper and Henderson [5J

144
 SOME BASIC LINEAR PROGRAMMING CONCEPTS 145

an activity is free, or unconstrained, or (as it will be

. . . to in code-listings in this book) of type absolute ,
referred
if it is exempted from the convention of non-negativity.

7.2 The L.P - problem in matrix notation

Let A be a m by n matrix of known coefficients. Let x be an n

by 1 column vector of unknown variables. Let b be a rn
by 1
column vector of known constants. Let w' be a-I by n row vector
of known coefficients. Let T be a scalar variabl~. The L.P.-
problem can then be formulated in matrix notation, as follows:

maximise T = w' ~, (7.2.1)

subject to A x < b (7.2.2)

x. > 0 (j 1, 2 n)
J
We now define an m by 1 column vector of slack-variables,
named ~. The problem can then be formulated as an undetermined
equations system:

maximise T in:

A x + s b
-
-w' x + T 0

x. > 0 j = 1, ... n)
J
s. > 0 (i = 1, ... m)
1

The non-negativity restriction applies to the elements of the

vectors, x and s,. It does not apply to the value of the
objective-function. We will meet examples where the optimal
value of T is negative.

7.3 A numerical example

Consider a factory with 2 different machines. The capacities

of these machines are known as 200 and 100. There are 3
production processes (products). The operation of 1 unit of
these processes gives the manufacturer a value added of 2, 5
and 1 unit of money respectively. There is a competitive
market, in which any output that is produced can be sold at
the current price. The prices cannot be significantly
influenced by the firm's small share in the market.

The objective function will then be:

146 CHAPTER VII

where xl' x2 and x3 are the 3 production processes. The

standard times for the different processes of both machines
are known. They give rise to 2 inequalities plus the non-
negativity requirements.

+ + < 200

+ < 100

> 0)

This little problem will serve us in doing some exercises. It

also is an example of an important class of practical
applications of linear programming. It has one important
simplifying feature. A feasible solution is already known.
Just doing nothing, xl = x 2 = x3 =0, will satisfy the
restrictions. It is of course not an optimal solution.

7.4 Graphical solution of a problem with 2 variables

When there are only 2 variables, an L.P.-problem can be

solved by graphical methods. The practical relevance of this
fact is not very great. Any realistic problem will involve
more variables. And the Simplex Algorithm can be applied
generally, with reasonable efficiency. But the exercise will
serve us, by illustrating some basic concepts of linear
programming.

Consider the problem:

minimise: 3 xl - x2

subject to: xl + x 2 < 7

3 xl + x 2 > 6

- xl + x 2 < 2

(Xl' x 2 > 0)

This problem, in its present form, does not satisfy our

conventions of uniform notation. Therefore we re-state the
problem, in a form which satisfies the conventions:

maximise: , = - 3 x 1 + x2 '
SOME BASIC LINEAR PROGRAMMING CONCEPTS 147

subject to:

Alternatively, the problem can be formulated as an under-

determined equations-system:

maximize Tin:

Xl + x2 + sl 7 )
)
-3x x2 s2 -6 )
1
)
- xl + x2 + s3 2 )
)
3x l x2 + T 0 )

(xl' x 2 ' sl' s2' s3 -> 0)

Because there are only two variables, we are able to map the
restrictions in the two-dimensional xl' x 2 co-ordinate plane,
and solve the problem by visual inspection of the mapping.
(See graph 7.4a below.)

graph 7.t.. a
graphical so lut ion
of an l.p. problem.

8
148 CHAPTER VII

The restrictions have been shaded on the non-admissible side,

as far as they appeared to be relevant. A restriction which is
satisfied by any solution satisfying the other restrictions in
the system, is named a redundant restriction. In the present
example, one of the tacit restrictions (xl ~ 0), is redundant.
The 4 points (2,0), (7,0), (21,41) and (1,3) are the corners
of a quadrangle. The area inside this quadrangle is the
feasible area or feasible region. Note that the origin of the
co-ordinate plane (the point xI=O, x2=0) is outside the
feasible area. This could be seen from the tabular presentation
of the inequalities. Once the restrictions are written in the
standardized form, all the variables on the left-hand side of
the < sign and a constant on the right-hand ~ide, the negative
value of the constant in the second restriction shows that
this inequality is not satisfied by xl = x2 = O. Apart from
the restrictions, the graph shows 2 dashed lines for constant
values of the objective function:

T = - 3x + x =-6
1 2
and

T = - 3x I + x 2 0

These two lines are parallel, as they should be. Any other
line for another constant value of the objective function
would run parallel to the ones in the graph. Clearly, no line
for a positive non-zero value of T goes through the feasible
area. This pins T = 0 as the maximum, and xl = 1, x2 = 3 as
the optimal solution.

In the optimum, 2 restrictions are exactly fulfilled. Apart

from the tacit restrictions, there is one restriction
(xl + x 2 ~ 7) amply fulfilled.

The corresponding slack should then be non-zero:

CHAPTER VIII

OUTLINE OF THE SIMPLEX ALGORITHM

8.1 The concept of a basic solution

The Simplex Method solves the Linear Programming problem by

solving a series of linear equations systems. The initial
system of inequalities is written as an equations-system,
by adding slack variables. (See Section 7.2). The numerical
example in 1.3 is then written as follows:

+ + + 200

+ 100

+ T o

One equation should always be read as expressing one variable,

always with a unity coefficient. That one variable is then
expressed as a constant, plus a linear combination of the
other variables. In the example, just re-written above, the
first equation expresses sl as being 200 minus Xl + x2 + x3.
The understanding is, that in this initial solution Xl, x2 and
x3 are in fact zero. That interpretation of a system of
equations written in this particular form,* allows us to read
a current solution-value from a tabulation of the system of
equations, without further calculation. In the example, sl has
the value 200. The coefficients of the other variable indicate
how sl would change if any of the other variables would be
given a non-zero value. This convention allows in each equation,
a non-zero coefficient for only one non-zero variable.
Conversely, a non-zero variable should have a unity coefficient
in one equation, and zero coefficients in all other equations.

Or, more strictly speaking, a variable that is solved for,

by the equations-system in question, is represented ~ith a
unity coefficient in one equation, and zero coefficients in
the other equations. In the above example, sl, s2 and T
are the variables that are solved for. We are of course not
assuming that xl, x2 and x3 should necessarily be zero. Only
our first current solution, considers non-zero values for
sl' s2 and T only.

*That form of presentation of a system of equations with a

single unity-coefficient in each equation, ~s often called the
canonical form. See W. Garvin [llJ • p.28.
149
150 CHAPTER VIII

The variables which are solved in the current solution are

named the basic variables of that solution.

One variable is always solved for, but not normally listed as

a basic variable: the value of the objective function.

The inference from this way of reading the equations-system is

the following: The number of basic variables is always equal
to the number of restrictions. During later iterations, other
variables will be solved for, will become basic variables. But
always when one variable enters the basis, another one is
leaving the basis. Here the word "basis" is understood as
meaning the list of names of the basic variables. This list of
names determines the corresponding solution completely; it
allows for solving their values by way of an ordinary linear
equations-system. Obviously, a list of m non-zero variables,
can be a list of basic variables, if and only if, the
corresponding equations-system is non-singular.

Basic solutions are only a restricted class among all possible

solutions. In a two-dimensional example (Section 7.4), this
can be seen at a glance. The basic solutions of the problem,
as far as they are feasible, correspond to the corners of the
feasible area. Yet the Simplex Algorithm analyses no other but
basic solutions. When the algorithm has been surveyed, it will
become apparent that this restriction is justified in the
general n-dimensional case as well.

Consider the system (7.Z.l). We now partition this system as

follows:

All ~l + AIZ ~Z ~l ~l)

)
AZI ~l + AZZ ~Z + ~Z ~Z)
) (8.1.1)
-WI
~l
- WI
Z ~Z + T 0 )
-1 -
The equations and the variables are supposed to be re-arranged
in such a way that the block-row Ai = [All' AlZ1 now corresponds
to the exactly fulfilled restrictions. The block-column
consisting of All and AZI then refers to those of the significant
variables that are basic in the current solution.

The composite vector consisting of ~l and ~Z is the (column)

vector of the basic variables.
OUTLINE OF THE SIMPLEX ALGORITHM 151

The values of the basic variables are found by solving the

linear equations-system:

All ~l
(8.2.2)
A2l ~l + ~2

This system is determinate, if and only if the composite

ma-rix

is non-singular. This matrix is then often named the basis-

matrix. If both groups 0f variables, elements of x, and
remaining slack-variables are present, the basis-~atrix is a
block-triangular matrix. One of its diagonal blocks is a unit
matrix, and as such always square and non-singular. We saw in
Section 5.7 that the determinant of a block-triangular matrix
is the product of the determinants of its diagonal blocks.
Then if the basis is to be non-singular, it is necessary and
sufficient, that All is square and non-singular.

All is the pivot-matrix or block-pivot *

8.2 The Simplex Tableau

A Simplex Tableau represents an equation system, written in

the standard "canonical" form, discussed in the preceding
paragraph. Only, we do not write the names of the variable
every time again.

The equations-system of Section 8.1 is represented by the

following Simplex Tableau

TABL EAU 8.2 (EXAMFLE 7.2)

A SIMF'LEX Ti>.BLEAU. (SET-UP TABLEAU, COMPLETE FORM)

NAME! I X I X 2 X 3 5 I S 2 T !! VALUE

5 I !! I !! 200
52! ! 3 ! ! 100

T !! -2 -5 - I ! !

*See
also: A.R.G. Heesserman, Special Simplex Algorithm for
Multi-Sector Problems [20]~as well as Section 3.11 of this book.
152 CHAPTER VIII

On account of computer-checking of all Simplex tableaux in

the manuscript the value of the objective function is referred
to in tableaux as t, even where otherwise the use of a Greek
letter would be more conventional.

The figures in the tableau are identical to those in

Section 8.1. The "T" column will remain a unit vector during
all iterations.

8.3 Choosing the pivot column

If we write the objective function explicitly, it becomes

again: T 0 + 3 xl + 5 x2 + X3. We are looking for a new
variable, with a view to increasing the value of the objective
function. We should then look for a positive coefficient in
the explicitly written objective function. This corresponds to
a negative figure in the implicitly written function

o
We will want to increase the value of the objective function
as much as possible. Hence we normally take the most negative
element in the objective function.

This rule is known as the principle of steepest ascent. It may

happen that two or more figures are equally qualified as the
most negative, because they are equal. We will assume the name
with the lowest index to be taken in such cases. This is a
device to make the choice unambiguous.

If we only want to reach the optimum, no matter after how many

steps, the principle of steepest ascent can be dispensed with.
Any negative element in the current preference function will
do. The principle of steepest ascent is a guide to an efficient
choice of a new basic variable. As such it is a somewhat
arbitrary criterion, and there are other rules for selecting
the incoming variable.

Once we have chosen a new basic variable, we have chosen a

pivot column. This is the column in the tableau, corresponding
to the new basic variable.

In our example, the rule of the steepest ascent gives us x2

as the new basic variable. The pivot column will be marked in
the tableau with an asterisk.

Also, following the suggestion ~n Section 3.5 we carry the

sum-count of all the entries in each row as a "check" column,
and perform the same operations on them.
OUTLINE OF THE SIMPLEX ALGORITHM 153
TABLEAU 8.3 (EXAMPLE 7.2)
SIMPLEX TABLEAU. "'t TH MARKING OF THE PI VOTAI. COLUMN.

NAI':E !! X X 2
* X 3 S 1 S 2 T !! VALUE 11 SUM CH

S II! 1 11 200 I! 204

S 2 !! 3 1 1 100 ! 1 105
------------------------------------------------------ -----------~
'!' !!-2 -5 - 1 ! ! ! ! -7

8.4 The choice of a pivot row

The pivot column will tell us how the (old) basic variables
will be reduced, by introducing a non-zero value for the new
basic variable, at a level of 1 unit. For instance, the s2 -
row should be read as 100 = xl + 3 x2 + s2' We now write S2
explicitly, and suppress the coefficients of the other non-
basic variables, which will remain zero-valued.

s2 = 100 - 3 x2

If s2 is to be the variable leaving the basis, it should be

reduced to zero. That solves x 2 as

x2 = 100 : 3 33 1/3

Considering sl as a candidate for leaving the basis, we would

find x 2 :

x2 = 200 : 1 = 200.

If we were to increase x2 to more than 33 1/3, s2 would become

negative. In particular for x2 = 200 (or sl leaving the basis),
we would have s2 = 100 - 3 x2 = 100 - 600 = - 500. This is no
longer an admissable solution. We should not increase X2 to
more than 33 1/3.

This gives us the rule of the smallest quotient: The name of

the variable to be eliminated out of the basis is found by
dividing all (positive) values of the basic variables through
their corresponding positive elements in the pivot column, and
by selecting the smallest from these quotients.

In its present form, this rule is valid only when the current
basic solution is feasible already. The question of how to
attain a feasible solution, if the trivial solution is not
feasible, will be discussed ln Chapter IX.

We will indicate our choice in the tableau. The tableau will

154 CHAPTER VIII

now become:

TABLEAU 8.1I A (EXAMPLE 7.2)

SIMFLEX TABLEAU. WITH RING-MARKING OF THE PIVOT.

NAME !! X X 2
* X 3 S I S 2 T ! I VALUE I! SUM eM
------------------------------------------------------------------
S I ! I I I! 200 !! 204
S 2 !! @ !! 100 II 105
------------------------------------------------------------------
T !!-2 -5 -I II II -7

The row of the tableau corresponding to the variable that is to

be eliminated from the basis is named the pivot-row. The circle
marks the intersection between the pivot-row and the pivot-
column. The encircled element is named the pivot. Actually,
we could have dispensed with a separate indication of the
pivot-column. The indication of the pivot (with a circle),
designates both pivot-row and pivot-column.

The question arises, if a search for the smallest quotient is

always successful. Could one not meet cases where there are
no positive entries in the pivotal column at all?

We could.

Example

Maximise

subject to

We write the Simplex Tableau, and make one step.

OUTLINE OF THE SIMPLEX ALGORITHM 155

TABLEAU 8.4 B (EXAMPLE 8.4)

AN UNBOLTNDED PF,OBLEM DEVELOPS AFTER ONE STEP.

(nm SIMFLEX TABLEAUX IN ONE REFERENCE TAELEAU)
NAME! ! X I X 2 S I T !! VALUE

5 \ ! ! -\ ! !

T !! -\ o o ! !

THE SECOND TAELEAU IS LTNBOLTNDED <IN X 2)

NAME! ! X \ X 2
* 5 I T !! VALUE

X 1 ! ! -1 ! !

T !! -1 ! !

The rule of the steepest ascent indicates x2 as the next

pivot-column, but the x2 column does not provide a pivot row.

The reason is that this is an unbounded problem and the x2

column is the unbounded* column.

Variable Xl' and as a consequence the objective function can

attain any value above 1, and a matching value for x2 can be
found. Xl = 1000000000000 with X2 = 1000000000001 is a feasible
solution, and there is no finite maximum. The general case is
that unbounded columns increase the value of the objective
function and the value of all basic variables (as far as they
change at all). Hence they~ not reduce any variable to zero
and no finte upper limit for such variables exists. Since they
increase the value of the objective function, no finite upper
limit for the objective function exists either.

*Sometimes one will meet the term "unbounded in the preference

direction". In. the next chapter, we will consider LP problems
where it is sometimes necessary to enter variables which will
reduce the value of basic variables in order to find a feasible
solution. However, a well-designed algorithm should ensure that
such columns are entered, only if one can be sure that they will
provide a pivot row. No such guarantee is possible with respect
to columns which are unbounded in the preference direction
because the LP problem itself may be unbounded.
156 CHAPTER VIII

In the case at hand, this is confirmed by a graphical mapping

of the one restriction, in graph 8.4a below.

,
5.

4.

4.

3.

3.

graph 8.4 a
illustration of an
unbounded lp problem.
I.

I.

o.

The one step from the or1g1n to the point xl = 1, x2 = 0

(marked 0) has been made, but further increase in Xl can be
obtained at infinitum, by moving along the binding restriction,
in the direction of the arrow. Linear programming algorithms
need a special "abnormal" exit-loop, to signal unbounded
problems.

The LP problem is held to have no feasible and optimal i.e.

finite maximum solution if the search for a pivot row fails.

8.5 The Simplex Step

Once a new list of (names of) basic variables is known, we will

want to rewrite the equations-system, in order to have it again
in its standard "canonical" form, with respect to the new
basis. This is achieved by a procedure which is very similar
to the one discussed in Section 3.3 for a block-equation.

First, the pivot-row is divided by the pivot. In our example,

the s2-row should then be divided by 3. Instead of

xl + 3 x 2 + s2 = 100
we will now write:

1/3 xl + x 2 + 1/3 s2 33 1/3

OUTLINE OF THE SIMPLEX ALGORITHM 157

In fact we only write the coefficients, the numbers

1/3, 1, 1/3, and 33 1/3

instead of

1, 3, 1, and 100.

The new row will now express x 2 as a constant (33 1/3), plus
a linear function of the non-basic variables xl and s2.

x2 - 1/3 xl - 1/3 s2 + 33 1/3

In all other equations, this last expression is now substituted

for x2. Or, more generally, the equation that is obtained after
dividing the pivot-row by the pivot, is used to eliminate the
new basic variable from all other equations. Arithmetically,
this is accomplished by subtracting the pivot row so many times
from all the other rows, as to cause vanishment of the
coefficients in the pivot column.

The operation can also be expressed in terms of matrices and

vectors. The "updating formula" is given here only for the
"rest" of the tableau, i.e. excluding the pivot-row and the
pivot-column itself.

T = T - k n- l r' (8.5.1)
n+l n
Here, T is the old tableau, T 1 is the new one. Both matrices
excludenthe pivot-row. ~ is th~+pivot column (as found in the
old tableau), r' is the pivot-row (as found in the old tableau),
and n is the pivot. Obviously then n-lr' is the new row, is
obtained by dividing the pivot row by the pivot. The outward
product of the old pivot-column and the new pivot-row, is
subtracted from the remainder of the tableau. The pivot row
itself is treated separately, it is divided by the pivot.

After the updating operation, our second Simplex Tableau wjll be:

TAELEAU 8.5 (EXAMPLE 7.2)

SIMPLEX TASLEAU. AFTER n RST COMPLETE STEP.

NAME!! X I X 2 X 3 S 1 S 2 T ! I VALUE I I SUM CH

--------------------------------------------------
S 1 !! 0.67 -0.33 !
.. --------------
I! 166.67 I I 169
X 2 !! 0.33 0.33 ! !! 33.33 II 35

T !! -0.33 -I 1.67 ! !I 166.67 II 168

 158 CHAPTER VIII

The sZ-row has been divided by 3. The new transformed sZ-row

(now named xz-row), has been subtracted with a factor 1, from
the sl-row, and with a factor -5 from the objective function.
(Or, alternatively, the transformed sZ-row) has been added with
a factor 5 to the ob;ective function.

Note the close analogy between the' pivoting operation in linear

programming and the elimination process in a block-equation,
discussed in Chapters I and III.

Following section 1.1, the updating operation for the Trow

may be made explicit as follows:

Multiply the new xz-equation by 5, to obtain:

+ 1 Z/3 Sz 166 Z/3

The old T-equation, to be added to it, is:

-z x
1
- 5 x
2
+ T o
to obtain the new T-equation
-1/3 xl -x 3 + 1 Z/3 Sz + T 166 Z/3

8.6 The attainment of the optimum

The "normal" end of the Simplex Algorithm occurs when a search

for a new pivot column fails, because no negative elements
occur in the current objective function. In our example, this
will be thp. case after the second step.
After our first step, we will find -1 for x3 to be the most
negative element in the current preference function. In the
x3-column, there is only one non-zero positive element. Again,
we will indicate our pivot-choice in the tableau with a circle.
TAB!.. EAt' 8.6 ( EX AM Fl. E 7.2)
SIMPLEX TABLEAU. AFTER FIRST COM-
PLETE STEP. ,,"'I TH NEW PI VOT MARKED.

NAME !! X I X 2 x 3 S I S 2 T !! VALl'E !! Sl"M CH

S I !! O. t:7 -0.33 ! !! I t:6. 67

!! 169
X 2 !! 0.33 0.33 ! I! 33.33!! 35

T !! -0.33 - I 1.67 ! !! 1 t: 6.67 !! 166

CONTINUATION TAELEAt: 8.6

THE NEXT TAB1.EAC IS THE OPTIMt:M.

NAME !! X I X 2 X 3 S 1 S 2 T !! VALt:E !I SCM CH

x 3 !! 0 • 67 -0.33 ! !! 166. t:7 !! 169

X2!! 0.33 0.33 I !! 33.33!! 35

T !! 0.33 1.33 ! !! 333.33!! 337

OUTLINE OF THE SIMPLEX ALGORITHM 159

We now write the objective function. From the tableau we read:

+ + 1 1/3 s2 + T 333 1/3

The equivalent explicit form of the objective function is:

T = 333 1/3 - 1/3 xl - sl - 1 1/3 s2

In the tableau, there were only positive coefficients in the

transformed objective function. As a result of this, all the
coefficients of the variables in the explicit form of the
objective function, are negative. All the non-basic variables
would, if introduced into the basis, reduce the value of the
objective function.

We conclude that we have reached the optimum.

The figures 1/3 for x~ 1 for sl, and 1 1/3 for s2, occurring
in the row "T" in the tableau, are known as the shadowprices of
these variables. In this case, they are the optimal shadowprices.
The row may also be named the transformed objective function.
The shadowprices are the elements of the transformed objective
function. The non-negativity of the transformed objective
function (of all the shadowprices), certifies the fact that an
optimum has been reached. If all the shadowprices are positive
non-zero, that is also the unique optimum, if there are zeros,
several vectors may be co-equally optimal.

Exercise 8.6a

Write the implicit and the explicit forms of the x3 and the x2
equations, as corresponding to the optimum tableau 8.6.

Exercise 8.6b

The following LP problem is given

Maximise T 2 xl - x 2

Subject to 2 xl + x 2 < 15

2 x 2 > xl - 3

(xl' x 2 2.. 0)

Make a graphical mapping of this problem and identify the

optimal and feasible solution by means of graphical analysis.
Then solve the problem by the Simplex Method, while also marking
each intermediate solution-point in the graph. (Answer-sheet
at the end of the chapter).
160 CHAPTER VIII

8.7 The Simplex Tableau in matrix notation

Consider the system (8.1.1) in Section 1 of this chapter. All

is assumed to be square and non-singular. We can then pre-
multiply the first block-row of (8.1.1) by All-I. The result
is:
-1 -1 -1
~l + All A12~2 + All ~l All E.l (8.7.1)

Or, solving for ~l:

-1 -1 -1
A12~2 - All + All (8.7.2)
~l =-A ll ~l E.l
With the help of this last expression we can eliminate ~l out
of the other block-rows of (8.1.1). Together with the
transformed first block-row, we obtain

)
)
)
)
)
)
)

(8.7.3)

The system (8.7.3) is the Simplex Tableau in matrix notation.

This emerges when we write the names of the (vectors of)
variables on top of the block-columns, instead of every time
after the matrices. We did the same with the equations-system
in figures.

name ~l ~2 ~l ~2 T value
r------ --- -------------- --------- ------ ----
-1 -1
--------------
-1
I AllA12 All A11£1
~l
-1 -1 -1
~2 A22-A2lAllA12 -A2lAll I £2 -A21 AllE.l
----- --- r--------------
--------- ------ ---- --------------
, 'A-lA -1 -1
T
-~2+~1 11 12 ~iAll 1 ~iAll£l
OUTLINE OF THE SIMPLEX ALGORITHM 161

The corresponding initial tableau can be expressed in matrix

notation as well:

name ~l ~2 ~l ~2
T value
------------- --------- --------
~l All A12 I E.l

~2 A2l A22 I E.2

------------- --------- --------
T -w' -w' 1 0
-1 -2

This tableau gives the initial numerical information, and no

other. Yet is refers to the current and in general non-trivial
solution. This is so because rows and columns have been
re-arranged to the partitioning by (8.1.1).

The block-columns "x " and "s " can be put together. They form
the basis-matrix. Welwill norilially indicate this matrix as B.

,J
B
(8.7.4)

The basis-matrix consists of the columns corresponding to the

basic variables, and of the rows corresponding to all the
restrictions. It is not normal practice to consider the
objective function as one of the restrictions. However, the
arithmetics of the Simplex method are the same for the
objective function and for a non-binding restriction. The one
difference is that the value of the objective function is always
solved for. Even if it should become zero or nega~ive, the
objective function row never becomes pivot row. Otherwise one
can consider the preference function as part of the basis, as
the m+l th restriction. The value of the objective function
should then be treated as one of the basic variables, except
for search operations.
162 CHAPTER VIII

It will be observed that the inverse of the basis is to be

found in any current Simplex Tableau, since

[~]-l
8.8 The shortened Simplex Tableau

To write a set of unit vectors for the basic variables at each

step, in each Simplex Tableau, is a bit tiresome. One could
dispense with that. This is done in the shortened Simplex
Tableau. When a column becomes pivot-column, we do not write a
unit vector in its place. The empty place in the tableau is
taken by the column associated with the just-eliminated variable.

The 3 tableaux of our numerical example can now be written as

follows:

TABLEAU 8.8 (EXAMl'LE 7.2)

TA8L~ATION Of ALL THREE SHoR~ENED TA8LEAUX.

SET-UP TABLEAU. ~ITH PIVOT MARKED.

NAME I I X I X 2 X 3 I I VALUE I I SUM + 1

S 1 ! ! I ! ! 200 ! ! 204
S 2 ! ! ® II 100 !I 105
------------------------------------------------
T ! ! -2 -5 -I ! ! ! ! -7

SECOND TABLEAU. Io.'ITH NEW PIVOT MARKED.

NAME· I I X I 5 2 X 3 I I VII.LUE ! ! SUM + 1
------------------------------------------------
S I
X 2
II 0.67 -0.33
! ! 0.33 0.33
<D
! ! 166.67 I I 169
! ! 33.33 II 35
------------------------------------------------
T ! ! -0.33 1.67 -I ! ! 166.67 I I 168

OPTIMAL TAELEAC.
NAME I! X I 5 2 5 I ! ! VALUE ! ! SUM +1

x 3 II 0.67 -0.33 I I 166.67 I I 169

X 2 ! I 0.33 0.33 II 33.33 II 35

'!' !! 0.33 1.33 I! 333.33 II 337

OUTLINE OF THE SIMPLEX ALGORITHM 163

The figures in these shortened tableaux are the same as in

the full tableaux on the same example in Sections 8.2-8.5.
Only the unit vectors have been suppressed. The corresponding
equations-systems are of course the ones according to the full
tableau.

The shortened tableau gives rise to a slightly different

procedure for updating. There now is a special treatment of
the pivot column. The new column can be obtained as the old
pivot-column, divided by minus the pivot, and the pivotal
element itself is replace~~its reciprocal. With the full
tableau, the same result would have been obtained without this
special rule; the unit vector would have served as an operator
to obtain the result.

Note that this new rule for updating the pivot column is almost
identical to the rule for updating the pivot row. The inversion
of the sign i.e. division by minus the pivot is the only
difference. The interpretatio~a shortened tableau the same
as for the corresponding explicit one, the tableau corresponds
to the equations-system given by the corresponding "full"
tableau. Note, that the "check" column now contains the
sumcount of the corresponding row, increased by one.

The suppressed unit-vector has to be taken into account when one

computes or verifies the "check" column.

The sumcount-property, as discussed in Sections 3.5 and 8.3 holds

because a multiple of the pivot-row is added to other rows.
This is only so in the full tableau, not in the shortened
tableau.

8.9 The Rule of the highest step

We now consider a rule for choosing the pivot column, which 1S

an alternative to the rule of the steepest ascent.

In section 8.3 it was assumed that we choose as a new basic

variable, the one which gives the maximum increase in
preference value, per unit of the new basic variable.

In general, the value of the new basic variable will not be

just one unit. We may anticipate the choice of the pivot row
and tentatively calculate for each column, the increase in the
value of the objective function to be obtained from entering
that particular column as new basic variable.

We may illustrate this new rule, for which I propose the name
given in the title of this section, with the example from
sections 8.1 and 8.2.
164 CHAPTER VIII

The shortened tableau for this problem is

TABLEAU 8.9 (EXAMPLE 7.2)

THE RULE OF THE HI GHEST STEP

FAVOURS LONG AS WELL AS STEEP STEPS.

NAME II X 1 X 2 X 3 II VALUE
-------------------------------------
S 1 l! 1 1 I I 200
S 2 l! CD 3 !! 100
-------------------------------------
T !I -2 -s -I !!

If xl is chosen as pivot column, then Sz

will be the pivot-row,
with an increase in the value of the objective function of 200.

If x 2 is chosen as pivot column, then s2 will be the pivot row,

with an increase in the value of the objective function of
166 2 /3.

If x3 is chosen as pivot column, then sl will be the pivot row,

and the value of the objective function will increase by 200.

The total increases for xl and x3 are co-equally 200. We assume

that the variable with the lowest index is then chosen. Hence xl
is chosen ~n preference to x2' on account of the greater length
of the step, despite the steeper ascent for the x2-variable.

In this small 2 by 3 example the number of steps required to

reach the optimum, is the same as when the rule of the steepest
ascent was used earlier on in this chapter. (Three in both
cases).

But it seems reasonable to assume that, in general, the rule of

the highest step will solve any given problem in, on the average
fewer steps, than the rule of the steepest ascent would require.
The computat~onal effort per step is obviously greater for the
rule of the highest step, on account of the more elaborate
search operation.

8.10 Degeneracy

A Linear programming problem is degenerate (at a particular

vertex), if several quotients between an element of the value
column and a corresponding element in the pivotal column are
non-negative, equal and co-equally the smallest.
OUTLINE OF THE SIMPLEX ALGORITHM 165

TABL EAt' 8010 A

AN EXAMPLE OF 1 Nl Tl AL DEGENERACY.

NAME" X 1 X 2 X 3 "VALUE

1 !! 10
S 1 " , , 0
S 2 " -I
S 3 'I -2 , , 0

T !! -I o o ! !

The s2-row (xl ~ x2) and the s3~row (xl ~ 2x3) both provide an
eligible pivot. Both quotients 0 : 1 = 0 for S2 and 0 : 1 for
s3 are equal.

Degeneracy normally occurs with a largish number of zero

entries in the trivial value column, but can also arise during
later iterations. If several non-zero quotients are co-equally
the smallest, the next tableau will contain at least one zero
in the value column, as two variables are reduced to zero ana
only one is eliminated.

Example

TABLEAU 8.10 B

EQUAL ELEMENTS LEADING TO DEGENERACY.

NAME ! ! X 1 X 2 X 3 I' VALUE

-------------------------------------
S 1 ! ! 1 1 1 ! ! 10
S 2
S 3
! !
! !
<D1 -1
-2
-1
1
! !
! !
1
1

T !I -1 o o ! !

In this case it is assumed that the restriction with the lowest

index is chosen and a zero will occur on the right-hand side in
the next tableau

TAEL EAt" 8010 C

NON-INITIAL DEGENRACY.

NAME '"' S 3 X 2 x 3 !! VALUE

S 1 " -1 2 2 , , 10
XI!! 1 - 1 -I , I
S 3 !! -1 - 1 2 ! !

-I -1 I'
T "
166 CHAPTER VIII

The form in which degeneracy usually occurs is the one of the

first example. Multiple degeneracy of the trivial basis due to
a large number of zeros in the value column is quite common.

Contrary to what is stated by some authors* degeneracy is a

problem which gives - in my experience - rise to computational
problems.

There are in fact two problems: convergency and accuracy. Of

these only the first is usually covered in the literature. If
there is no degeneracy the simplex Algorithm increases the value
of the objective function by a positive amount at each step.
Therefore a vertex which has once been a basic solution cannot
be met again as this would imply a reduction in the value of
the objective function.

Since the number of vertices is finite the algorithm must

sooner or later end. This proof of convergence does not hold
in the case of degeneracy. Well, I have met cases where
degenerate problems were not solved within a reasonable time
limit, whereas small changes in the value column, i.e. putting
0.0000001 instead of the exact zero did result in a solution
within a normal jobtime limit.

The other problem concerns accuracy. Consider the following

problem

TABLEAL' 8.10 D
A DEGENERATE PROBLEM. SEE ALSO 8.10 E.

NAME !! X 1 X 2 X 3 F 1 F 2 F 3 !! VALt'E
-------------------------------------------------------
5 1 ! ! -0.99 0.02 0.04
-0.92 0.05
<D ! ! 0
II 0
5 2 ! ! O. 00
5 3 ! ! 0.07 0.03 -0.97 ! ! 0
5 4 ! ! 0.42 0.37 0.49 II 6.34
-------------------------------------------------------
T ! I 0 o o -1 -1 -1 ! I -2

* Thus R.C. Geary and J.E. Spencer in their "Elements of Linear

programming" [14] p. 55, refer to degeneracy as "a special case
of trivial practical importance which usually receives a
disproportionate attention in the textbooks."
OUTLINE OF THE SIMPLEX ALGORITHM 167

The system represents an input-output allocation model* and it

is desired to maximise total final output, of 3 national sectors
of production subject to the appropriate interindustry relations,
and a macro-economic factor ~upply limit. Typical of this class
of systems is the square block in the top left-hand side
dominated by a negative diagonal. Some of the off-diagonal
elements in this block can be quite-sIDall.

Let us see what happens if the rules and procedures discussed

so far are applied to this system. The first pivot has been
marked already, but we reproduce below tableau number 4, where
things become interesting.

TAELEAt: 8.10 E
DEGENERACY. LEACING TO CHOICE OF A SMALL PIVOT.

NAME !! X 1 X 2 X 3 S 1 S 2 S 3 !! VALt:E
------------------------------------------------------
I ! ! -0.99 0.02 0.04 ! ! 0
F
F 2 ! ! <[;]» - 0 .92 0.05 I! 0
F 3 ! ! 0.07 0.03 - 0.97 ! ! 0
S /I ! ! 0.42 0.37 0.49 ! ! 6.34
------------------------------------------------------
T !! -0.91 -0.86 -0.87 I I 0

At this stage, Xl is the pivot column. The diagonal element

of -0.986 is not eligible on account of its negative sign.
Let us assume that the search for the smallest quotient is
done (going down along the Xl-column), by looking at candidate-
pivots, only if they give rise to a smaller quotient than the
previous one. Then f2 is the next pivot-row variable, as the
quotient for f3 is not smaller. But obviously f3 should have
been taken, and in general the largest of equally eligible
candidate-pivots should be selected.

A further point arises in connection with the use of floating

point zeros. Some machines substitute the absolute smallest
number which can be represented as a floating point number, for
the number zero. In that case zero plus zero is greater than
zero, and a number which is in any meaningful sense zero could
become the pivot, despite the fact that the programmer excluded
zero pivots.

I have seen the wrong "outcome" for a multi-degenerate input-

output allocation model in two separate instances. The first
case involved academic research supervision and a programme

*See also my book "Allocation Models and their use in

economic planning" [18]
168 CHAPTER VIII

written by myself. The second case involved consulting work,

and the programme was written by an employee of the client
organization. In that second case the distortion of the
specified linear programming problem went as far as finding a
problem with a feasible trivial basis empty, i.e. not to have
any feasible solution at all.

In both cases an amendment in the text of the programme, i.e.

attending to degeneracy, produced the correct results. The
most simple solution to this problem (used in both cases), is
to replace a zero in the value column, by a very small positive
number, e.g. 0.000000001. Thus, in our example, the search for
the smallest quotient would consider 0.000000001 : 0.001 as the
relevant quotient for the f2-row, and 0.000000001/0.74 as the
relevant quotient for the f3-row. This means, in effect, that
the largest of the available pivots (i.e. 0.074 in the above
example), is chosen. (This type of change is known as
perturbation of the originally degenerate problem).

The "time expired" degenerate linear programming problems which

I have met, may, or may not have been cases of true cycling
in degeneracy.

Loss of accuracy for the reasons indicated above, can also

cause spurious steps. This is so, in particular if spuriously
negative entries in the value column arise. The reason is that
the methods for finding a feasible solution in the first
place (to be discussed in the next chapter), are on the whole
less efficient than the elementary optimizing algorithm. And
spurious negative entries in the value column would cause a
spurious activation of this so-called "Phase I" part of the
programme.

One further aspect of the problem of degeneracy is computing

efficiency, i.e. avoiding unnecessary steps, irrespective of
the issue of cycling.

On balance, my recommendation is to modify the problem at the

outset, and make it into a non-degenerate problem. Zeros in the
value column are all replaced by small positive numbers.
These small numbers are (for example)

j
e.
~
0.000 000 000 001 (1 + L (-a .. )) (8.10.1)
~J
i f a .. <0
~J

i.e; one takes the number one, adds to it the absolute value
of all the negative entries in the relevant row, and scales the
whole lot down.
 OUTLINE OF THE SIMPLEX ALGORITHM 169

There are two main reasons for not simply taking the same
number for each row:

a) In some systems there are columns with the same entry in

a whole series of rows. If such a column becomes pivot-
column against a pivot-row with a pseudo-zero in the value
column, when all pseudo-zeros are equal one is back at
multiple degeneracy.

b) The suggested variation tends to prefer pivot-rows with few

or with only small negative entries to rows with many large
negative entries. The effect of this is to avoid these
negative entries to be added to the corresponding entries
in the target row, thus reducing the chances of having to
enter the corresponding columns as pivot columns at the next
step.

It is appropriate at this point, to comment on the discussion

degeneracy in the literature, and the methods recommended for
its resolution. First consider the "naive" rule of taking the
top one of any zero quotients, i.e. the first zero in the
value column for which a corresponding entry in the pivotal
column is positive. This is not a recommended method, simple
and practicable in the majority of cases as it may be. It has
been shown to lead to cycling, independently, by at least two
people, using examples constructed to that purpose. These two
examples are: Tucker's as referred to by Zoutendijk*, and
Beale's as referred to by Var. de Panne**. Both examples involve
initial degeneracy with two zeros in the value column and both
become readily solvable if the zeros are replaced by equal
pseudo-zeros. The fact that pertubation with equal pseudo-zeros
is effective in these examples obviously does not prove that
such a method is always effective - nor is that method
recommended here.

*Zoutendijk, G., Mathematical Programming Methods [42].

section 3.5. No bibliographical reference to Tucker given,
private correspondence assumed.

**Van de Panne, C., Methods of Linear and quadratic Programming,

[j6] , section 3.5. No bibliographical reference to Beale given.
 170 CHAPTER VIII

The following is the method recommended by Charnes (4), [5J

Ch VI, and methods which are in essence the same are still the
ones recommended by the dominant tradition. Replace each constant
b. by
~

b.* = b. + Ei (8.10.2)
~ ~

The dominant tradition has it that the actual figures in the

tableau are left as they are, including exact zeros, and one
then devises additional rules for selecting the pivot row
assuming that the trivial basis has been perturbed according
to (8.10.2).

What this effectively me~ns may be illustrated by reference to

tableau 8.l0e. The choice of the leaving variable is in first
instance restricted to f2 and f3 only. (f l is not eligible
because the quotient is classif~ed as -0 i.e. negative and s4
is not eligible because the quotient is in any case not the
smallest, being non-zero).

To choose between f2 and f3 as leaving variable, we now refer

to the sl column as substitute value column. The sl-column makes
the biggest "pseudo-zero" contribution to the value column - if
any non-zero entries in the sl column are available. In those
rows in which the sl-column contains non-zero entries, pseudo-
zero entries in the value column are dominated by the E entry
in the original sl-restriction.

*
Those "bl dominated" entries could either be just the original
E: entry in the sl-row - if the restriction were still in the
non-binding form with the slack-variable in the list of basic
variables - or the updated sl-column, multiplied by El.
(Compare 8.7.2 and 8.7.3).

In fact in the example at hand, no non-zero entries in the s2

and s3-row are available in the sl-column and we are still left
with an undecided choice between s2 and s3 as pivotal row.
Therefore, not having resolved the choice of a pivotal row, we
refer to the s2-column as substitute value column and find
as relevant quotients:

in the f 2 -row

(0 *
E2) / /0.07 in the f 3 -row, the latter quotient being
rather smaller.
OUTLINE OF THE SIMPLEX ALGORITHM 171

Charnes has proved that this method cannot lead to cycling.

However, it leaves the door wide open for what has been
indicated above as the main danger of degeneracy in practice,
loss of numerical accuracy. The method comes in practice very
near to choosing the last rather than the first of any available
zeros. It differs sufficiently from that rule to guarantee non-
cycling but not to guarantee meaningfully non-zero pivots~
If the entries of 0.00 in the S2/xl cell and 0.07 in the s3/xl
cell in tableau B.10d are interchanged, the method of exponential
perturbation would opt for the 0.00 entry (which is not actually
a zero) as pivot in tableau B.10e.

The choice of the pivot depends in no way on the actual figure

of the candidate-pivot, it is enough that it is a positive
figure. It would appear that while the relatively simple device
of actually perturbing the trivial basis according to (B.10.l)
is at the cost of a small loss of accuracy in the outcome, the
exponential perturbation, whether actually performed or
hypothetically assumed, is in practice prone to much greater
inaccuracies, and the more so if actual perturbation is replaced
by a rule which assumes hypothetical perturbation with a number
E approximating zero. (The actual ~erturbation would exclude
pure rounding errors as pivots.) In principle, that gain in
accuracy has to be balanced against the fact that the method
recommended above is not supported by a proof that it will
never lead to cycling.

B.ll Simplex Tableaux and Vector Spaces

Suppose we know for some LP problem the names of (some of) the
binding optimal restrictions. We could in that case interpret
the partitioning introduced earlier in this chapter (B.l.l) in
a "non simplex" way. This partitioning was also used in
section B.7. We now write an equivalent of (B.7.3) while
suppressing reference to ~l which vector is assumed to be zero.
-1 -1
+ All A12 ~2 All ~l )
~l
-1 -1 )
[A22 - A2l All A12] ~2 + ~2 ~2 - A2l All ~l) (B.l1.l)
)
+ ' -1 -1
+T )
L-w'
-2 ~l All A121~2 ~i All ~l
Note that ~l is effectively the vector of slack-variables
associated with the first block-row of (B.ll.l), if the system
is written in equality form. In the terminology of Chapter VI,
we might say that a simplex tableau from which a (binding) ~l
block-column has been deleted, describes the remaining
restrictions in the sl = 0 subspace. If the final block-row
172 CHAPTER VIII

is binding, the optimal solution is equivalent to the optimal

solution of the following residual LP problem:

Maximise

A =
r,
l!'l:; - !'I:J: -1
All A12
1 ~2 (8.11.2)

Subject to
-1 -1
All A12 ~2 ~ All ~l (8.11. 3)
-1 -1
[A22-A21All Al2] ~2 2.~2 - A21 All ~l (8.11.4)

Comparison of (8.11.2) to (8.11.4) with (8.7.3) shows that the

new (reduced) problem is related to the old one as follows:

Firstly, A is the old objective function less the current

solution value in (8.7.3).
-1
A= T - !'I:i All ~l (8.11. 5)

Comparison of (8.11.3) with the first block-row of (8.11.1)

confirms that ~l is the vector of slack-variables in (8.11.3).
If we prefer the more usual ordering with the slacks at the end,
the first block-row of (8.11.3) could be written as
-1 -1
All A12 ~2 + ~l = All ~l (8.11.6)

The vector of slack-variables in (8.11.4) is the old ~2 vector.

The "non-simplex" interpretation arises because (8.11. 2)

to (8.11.4) is treated as an LP problem in its own right and we
do not necessarily assume that all elements of Xl and none of
~ are in the collection of basic variables. We-don't know
whether the partitioning between ~l and ~2 is the optimal one.
We may have to assume that the partitioning of the tableau in
two block-columns is done so as to find a square and non-
singular block-pivot All.

Restrictions (8.11.3) apd (8.11.4) list the restrictions of

the LP problem in an n-ml dimensional co-ordinate space, the
x2-space. The set of points satisfied by (8.11.3) and (8.11.4)
is equivalent to the intersection of the ~l = 0 subspace with
the set of points satisfying the originally stated restrictions
in (8.1.1).
OUTLINE OF THE SIMPLEX ALGORITHM 173

The choice of any particular sub-vector ~l as well as the

non-negativity of the corresponding trivial solution,
~l = Ali~' is irrelevant provided the associated block-pivot
All is non-singular. Alternatively, we can say that the original
problem is stated in the m+n dimensional x, s coordinate space,
and the reduced problem is stated in the m+n~ml dimensional
~l' ~, ~2 co-ordinate space. We are now in a position to
formulate and prove a theorem which we have - in a sense -
already used. The simplex Algorithm investigates basic
solutions where the number of binding restrictions equals the
number of solved variables. We will show that if there is an
optimal and feasible solution there is an optimal and feasible
solution among the collection of simplex (basic) solutions.

Theorem

Let x = x* be an optimal and feasible solution to the LP

problem

Maximise

T = w' X (7.2.1)

Subject to

A x < 0 (7.2.2)

x ~ 0,

Then we may require the number of binding restrictions in that

optimal solution or else in some co-equally optimal solution
x = x**, to be not less than the number of non-zero valued
variables. We may also require the existence of a non-singular
block-pivot as defined in (8.1.1).

Proof

First we consider the case of a trivial objective function,

i.e. we assume w' = O. In that case we are free to declare any
vector x which satisfies the stated restrictions (7.2.2), as
being an optimal solution. That includes the optimal solution
to a secondary LP problem which we obtain by.substituting a
non-trivial objective function for the trivial one. One could
for example take (minus) the sum of all variables and maximand,
i.e. replace the zero vector w' by a summati'on vector, which
is not a zero vector. Therefore, if we obtain a proof of the
theorem for LP problems with non-trivial objective functions,
this includes the case of a trivial objective function. We
may thus assume that the objective-function is not trivial.
174 CHAPTER VIII

For LP problems with non-trivial objective functions, the

optimum solution satisfies the definition of an outward point
as given in Section 6.7.

It was shown in Section 6.7 that, irrespective of the linearity

or non-linearity of the side-conditions at least one side-
condition is binding at an outward point. (The proof in
section 6.7 depended only on the linearity of the maximand).
In Chapter VI trivial restrictions were excluded by assumption.
We do not now make that restrictive assumption, at least not
as a restrictive assumption.

In the case of a linear restriction, triviality (defined in

Chapter VI as not excluding any vector in the co-ordinate-
space), is possible only if for that particular i-restriction.

a!
-1
o (B.n.7)

is true.

For b. > 0 such a restriction is trivial. We assume that we

never 1 consider such a restriction as binding and we will in
fact still be able to prove the theorem. No difference in the
set of feasible points arises if we discard it. For bi < 0 no
meaningful problem has been stated. A restriction which states
that zero is less than a negative constant cannot be satisfied
at all. bi < 0 will not be true for ~i = o. We may therefore
assume, without loss of generality that all restrictions are
non-trivial.

Therefore the proof which we carried in Section 6.7 for an

outward point is applicable and at least one non-trivial
restriction may be required to be binding in the optimum. In
general, we cannot identify binding restrictions in isolation
from the list of basic variables. But let us for the sake of
argument assume that we could identify at least one binding
restriction of the optimum solution of an L.P. problem. (We
know there is at least one). We can make use of this result in
two cases in two different ways.

In the special case of only one variable, the proof is complete,

as we have shown that at least one restriction is binding. In
the more general case of n variables (n > 1), the one binding
restriction defines a linear (Euclidean) subspace and the
optimal solution is part of that subspace of n-l dimensions.
(Or m+n-l dimensions if we could the slacks as co-ordinate-
directions).
OUTLINE OF THE SIMPLEX ALGORITHM 175

Since the one binding restriction is non-trivial we can always

find a variable which provides a 1 by 1 non-singular block-
pivot All' Finding a non-trivial binding restriction is
enough to show that a partitioning by (8.1.1) exists. We
therefore have a well-specified residual problem, if we delete
reference to the slack of the one binding restriction from the
originally specified problem.

Example (of a residual problem, proof itself continued below)

Take the numerical example in section 7.4

Subject to 7

-xl + x 2 < 2

Once we know that the first restriction Xl + x 2 ~ 7 is binding

in the optimal solution we may pivot (for example) x2 against
the slack of the first restriction.

T ABL EAU 8. I I A
A SIMPLEX STEP MAY LEAD TO A ~E­
SIDUAL PROBLEM. (TABLEAU 8.11 Bl

NAME! ! X I X 2 !! VALUE
s----~~---~------d)---~~---~--
S 2 !! -3 -I !!-6
S 3 !! -I I ! ! 2
------------------------------
! ~
T !! -3

and the up-dated tableau is (see section 8.8 for the up-dating
rules in a shortened tableau)
176 CHAPTER VIII

TABLEAU 8.11 B

THE RESIDUAL FROBLEM. IN- DITTO.

DICATED IN TABLEAU 8.11 A. RE- INTERPRETED.

NAME ! ! X 1 S 1 ! ! VALl'E NAME ! ! X 1 !! VAL.

------------------------------ ----------------------
X 2 ! ! 1 7
1 ! ! S ! ! 1 !! 7
S 2 ! ! -2 1 ! ! 1 S 2 ! ! -2 ! ! 1
S 3 ! ! -2 -1 ! ! -s S 3 ! ! -2 ! ! -5

T !! -4 -1 ! ! -7 T !! -4 !! -7

The x2, sl pivot is not a very well-advised pivot. Without the

hindsight that xl + x2 ~ 7 is binding in the optimal solution
it is in fact a oarticularly ill-advised pivot because it
replaces on violated restriction (the second) by another
(the third). But it serves to state the residual problem. If
xl + x2 ~ 7 is binding in the optimum then we find the optimal
Xl from the residual problem.

Maximise

A 4 xl

Subject to

xl ~ 7

-2x ~ 1
1
-2x < 5
1
(xl 2:.. 0)

Continuation of the proof

The extremum property that at least one restriction is binding

is applicable to this residual problem as well as to the
original problem. In the particular example, this completes
the proof, as only one variable is left and we only need to
find the most binding restriction. In the general case, finding
another non-trivial restriction binding in the reduced problem
allows us to find a pivot in that row and hence a non-singular
block-pivot of which the order is one greater than at the
previous partitioning. (The determinant of a matrix has the
same absolute value as the recursive product of the pivots
which may be used to invert it, see section 5.5). Therefore
if there is an optimal solution there is another non-trivial
binding restriction, as long as there is an unused co-ordinate-
direction.
OUTLINE OF THE SIMPLEX ALGORITHM 177

The number of explicit co-ordinate directions in each residual

problem is one less than in the corresponding full problem. If
we identify an explicit restriction, i.e. the non-negativity of
some si becomes binding, we eliminate the explicit reference
to a variable by means of an elimination step, some Xj becomes
a slack in (8.11.3). Since the statement of an L.P. problem
does not require a positive righthand-side, this is true, even
if a negative pivot were used to eliminate the slack-variable
si. If we identify a binding "tacit" restriction we merely
drop further reference to the zero-valued variable.

The end of this hypothetical process of identifying binding

restrictions, is no variables to change being left.

Therefore the number of ~estrictions identified in the course

of the process, and the order of the block-pivot is the
initially available number of variables. The actual number of
exactly fulfilled restrictions cannot be less.
q.e.d.

The number of binding restrictions is not less than (rather

than always equal to) the number of co-ordinate directions
because of two footnotes to the proof.

Firstly, we assumed that if we met a trivial restriction in

some subspace, we would discard it. This may be an additional
binding optimal restriction. It may even be an additional
non-trivial restriction. It does not follow that a restriction
which is trivial in some subspace is also trivial in the
fully specified problem.

Example

Maximise

subject to

(i.e. xl + x 2 > 5)
178 CHAPTER VIII

The optimal solution to this problem is given in the following

tableau

TABLEAU 8.11 C
ILLt!STRA'!'lON OF
St!BSPACE-TRIVIALITY.
tJAME!! SI X2 !! VALt:E
XI! ! I! 5
S 2 !! ! !

T ! ! 1 I! 5

The second restriction is trivial in the sl = 0 subspace. It

makes no difference to the solution (the first restriction is
binding on the optimal solution anyway) but the second
restriction is not a trivial restriction. It excludes, for
example, the origin from the set of feasible vectors xl, x2.
Having discarded the s2-restriction as trivial in the sl=O
subspace, we identify the non-negativity of x2 as the second
binding restriction. Secondly there may be several equally
binding restrictions on the last variable, e.g.

x < 2, 2x < 4, 3x < 6, etc.

Again the restrictions may be dependent in some subspace but

they might nevertheless be independent restrictions in the
fully specified problem.

The assumed possibility of identifying successive binding

restrictions proves that, (because there is at least one non-
trivial binding restriction in any L.P. problem) the optimal
solution is a vertex, a basic solution. As an algorithm this
approach has one fatal flow: we cannot in fact identify an
individual binding optimal restriction without first solving
the problem as a whole. But the simplex method which
investigates only basic solutions, will get us there.

There are some further carefully formulated passages in the

theorem, i.e. "we may require" and "or in one of equal value".
There are indeed L.P. problems for which "non simplex"
solutions (with fewer binding restrictions) are co-equally
optimal with Simplex solutions. This possibility arises when
the objective function is trivial in some sub-space.
OUTLINE OF THE SIMPLEX ALGORITHM 179

Example

Maximise
graph 8.11 a
T = xl + X2 Illustration of trIvIality
3.0 In the optImal subspace,
Subject to due to dual degeneracy.
2.~

xl -< 2 2
x 2 -< 2 2.0

Xl + x 2 < 3
\.S

1.0

o.~

00 O.S \.0 \.S

Two vertices (Simplex solutions) are co-equally optimal

at T = 3. Any point on the facet between them is also co-
equally optimal. This situation arose because in the
(s3 = xl + x2 = 3) subspace, the objective function is trivial.

The fact that xl = 1.1, x2 = 1.9 is optimal, yet only one

restriction is binding does not invalidate the theorem. It is
the case of a trivial objective function referred to in the
proof. That objective function is trivial in the s3 = 0
subspace, not elsewhere in the xl, x2 co-ordinate space.

N.B.

A summary of this proof may be found in section 14.8, where

the same theorem is stated and proved for the more general
non-linear (and non-convex) case.
180 CHAPTER VIII

ANSIIER-SHEET 8.6 EX

graph B 6 ex

graphlca mappIng
of exercIse B.6 b

12 11,

THE SET-UP TABLEAUI

NAME!! X I X 2 S I S 2 T I! VALUE
THE RL'l.E OF THE
S I !! 2 I ! I
STEEFEST ASCENT15
S 2 ! I <D -2 ! I
INDICATES XI AS 3
(ONLY ELI GEAELE)
T II -2 ! I INCOMING VARIA-
SLE. THE RATIOS
BETIIEEN THE VALUE COLUMN AND THE XI COLUMN ARE 15/2 • 7.5 FOR SI
AND 3/1 • 3.00 FOR S2. THE RULE OF THE SMALLEST QUOTI nJT THEP.EFORE
INDICATES S2 AS THE LEAVING VARIAELE.

THE VERTEX FOLLOIiING STEP II

NAME I I X I X 2 I SIS 2 T II VALUE
THE Rt'\. E or THE
S I
X I
II
!!
®
-2 I
-2
3
II
STEEPEST ASCENT
NOW INDICATES X2
II
9

AS THE (ONLY ELI-

T 'I -3 2 ! I 6 GEAEL E) INCOMI NG
VARI AELE.
THE RATIO 3/-2 • -1.50 FOR XI AS LEAVING VARIABLE IS NOT ELI GEAELE.
liE THEREFOP.E CHOOSE SI AS LEAVING VARIABLE.

THE VERTEX FOLLOIiING STEP 21

NAME I I X I X 2 I SIS 2 T II VALUE NOli WE DON'T FIND
A tlEGATI VE ENTRY
X 2 II 0.20 -0.40 I II 1.80 IN TH E OBJEC TI VE
X I II 0.40 0.20 I II 6.60 FUNCTION ROW,
VE HAVE rOL'ND
T II 0.60 0.80 I II 11.40 THE OPTIMU1,
CHAPTER IX

THE SEARCH FOR A FEASIBLE SOLUTION

9.1 The case of one unfulfilled restriction

In Section 7.1 we defined the L.P.-problem as:

(a) finding a feasible solution,

(b) finding, among all feasible solutions, the optimum.

This definition is slightly shorter than the one formally

given in Section 7.1. This is so, because we now use some
terms, which we had not yet defined in Section /.1. The substance
is the same.

It is standard practice to tackle - if needed - (a) first. The

operation is known as "Phase I", leaving optimization as
"Phase II".

The Simplex Algorithm gave us as yet only a solution to (b),

on the assumption that we had one to (a). The reason for this
curious order is this:

The search for a feasible solution is done with help of the

maximizing algorithm.

The method of solving the "Phase I" problem, offered in this

book, is probably best described as the use of a substitute
objective function. It should, however, be stressed that the
method of artificial variables (summarized briefly in
Section 9.5), is in fact more commonly used, or at least
reported in the literature. In the case of just one violated
restriction, the substitute objective function is, in effect
the slack of that one violated restriction.

Consider the partitioned system:

Cx + t .£
d' x + f.I 0

-w' x + 0 0

We have split off one row from A, partitioning A into a

remainder matrix C, and the row-vector d'. The vectors sand b
are correspondingly split into scalars ~ and 0, and the-
remainder t and c.
181
182 CHAPTER IX

We assume ci ~ 0 (i = 1, 2, ... m-l), but 0 < O. The trivial

solution satisfies the first m-l restrictions, but not the
last one. The problem of finding a feasible solution will then
be solved by finding the maximum value of ~ = -d' x+o. We do
not always really need the maximum value of ~. But-if a non-
negative solution is at all possible, the maximum will be a
non-negative solution. We therefore start with maximizing
-d' x, subject to C ~ 2~.

For example, consider the problem used in Section 7.4 to

illustrate the possibility of graphical solution.

The initial Simplex tableau for that problem (now suppressing

the T and "check" columns) is:

TABL EAU 9. 1 A
SET-CP TABLEAU OF A LINEAR PROGRAMMING PROB-
ELEM. WITH AN INFEASI8LE STARTING SOLUTION.
NAME! ! X 1 X 2 'S 1 S 2 S 3 !! VALl'E
----------------------------------------------------
S ! ! 1 1 ! ! 7
S 2 !! -3 - 1 !! -6
S 3 !! -1 1 ! ! 2
----------------------------------------------------
! !! 3 - 1 ! !

We treat the s2-row as a substitute for the objection function.

For the time being we try to increase s2 instead of T. The
most negative element in the s2-row is provided by the xl-
column. Hence xl will be the first pi~ot-column. For the moment
we will assume, that the s2-row (now being the objective
function), is excluded as pivot-row. The real objective function
is of course not eligible as a pivot-row either. This leaves
us with the choice of sl and s2 as candidates for becoming
pivot-row. Here sl produces the only pair of positive numbers,
and will be our pivot row. We will indicate this in the
tableau, which now is:
THE SEARCH FOR A FEASIBLE SOLUTION 183

TAE'L FAU 9. I E
SE:T-l'P TAEL EAU 1rIi TH AN 1 NI'EASI EL E STAF.TI NG
SOLUTION, AND PIVOT MAF~ED.

NAME! ! X I X 2 S I S 2 S 3 I I VALUE
----------------------------------------------------
S I !! CD I II 7
S 2 !! -3 -I ! I -6
S 3 I! -I I ! I 2

T !! -I I!

We now carry out the indicated step. In graph 7.4a this step
corresponds to a movement along.the Xl-axis, until the top
left-hand corner of the quadrangle is reached. The new Simplex
tableau will be:

TAELEAU 9. I C
UPDATED TAELEAl' OF AN LF PROELEM, 1rIITH INFEASIBLE
STARTING SOLUTION, AND A NO'" AMFLY FULFILLED
RESTRICTION.

NAME! ! X I X 2 S I S 2 S 3 ! I VALUE

X !! I I ! ! 7
S 2 !! 2 3 !! 15
S 3 !! 2 I I,! 9

"!' !! -4 ! -3 !! -21

This tableau represents a feasible solution, witness the non-

negativity of the "value" column. The further search for the
optimum can then be carried out with the methods described in
Chapter VIII. There are, however, two remarks to be made in this
stage.

Firstly, the choice of sl as the pivot-row is in fact not quite

correct. It does lead to the desired result. But cases may arise,
where the extension of the rule for choosing a pivot row,
obtained in Chapter VIII to non-feasible problems, could lead to
trouble. The problem will be discussed in the next paragraph.

Secondly, one should realize, that the search for a feasible

solution, does not always end with finding one.
184 CHAPTER IX

If we have the system:

maximise Xl' subject to:

< 3)
)
< -4)

We will start with maximizing x 2 . But clearly, the maximum of

x2 will be obtained with x2 = 3. Yet no value of x2' smaller
than 4, will satisfy the second restriction. The system has no
feasible solution.

An L.P.-problem, to which no feasible solution can be found, is

known as an empty problem. Since the present problem again
involves only two variables, we can illustrate it graphically.
Again the shaded side of the restrictions indicates the non-
admissable side (See graph 9.1a).

5.0..• c'
........ graph 9.1 a
u I X1 iLLustration of
an empty probLem.
THE SEARCH FOR A FEASIBLE SOLUTION 185

There is no place in the graph, that is at the non-shaded

side of all the restrictions. The restrictions contradict each
other. This reveals the emptyness of the problem graphically.

The fact that the problem is an empty one, can also be

demonstrated by the Simplex Algorithm. This takes only one
step; two Simplex tableaux.

These Simplex tableaux will be:

TABLEAUX 9.1 D AND 9.1 E

RECOGNITION OF AN EMPTY PROBLEM

BY TH E 51 MPL EX M ETHO D.

NAME! ! X 1 X 2 5 52!! VALUE

5 1
5 2
I!
!I
CD
-I
I!
! I
3
-4

T !! -I o ! !

NAME! ! x X 2 s S 2 !! VALUE

X 2 !! ! ! 3
52!! I! -I

T !! -I ! I

The s2-row should now be read:

-1

This is an entirely non-negative combination of the non-

negative variables xl' sl and s2' Such a linear combination can
never be equal to something negative. It is an "impossible"
restriction. Whenever we find in any row, a negative element
in the "value" column, and positive elements (or zeros) in all
other columns, this certifies the problem to be 'empty. This
situation should then be listed as an indication of the end of
the algorithm.

When we are searching for a negativ~ element in a row, with a

view to finding a new pivot column, and do not find one, this
will always certify the end of the algorithm. If the row is the
objective function, the inference is that we have reached the
optimum. If the row is not the target-row, it must be a row
describing one of the basic variables or a combination of them
in some versions of the Simplex algorithm. The value of the
186 CHAPTER IX

corresponding variable must be negative. Then the problem has

been found empty. The situation will never arise with a row
describing a positive variable. We do not perform such search
operations in rows describing positive variables.

9.2 Again: The choice of the pivot row

In Section 8.3, we discussed the choice of the pivot row. We

then assumed the existence of a feasible solution. The rule
then said that we should divide the (positive) values of the
basic variables by the positive elements in the pivot column,
and select the smallest among these quotients. In this form,
the rule of the smallest quotient excludes the choices of
negative pivots. This rule will no longer do. We must assume
the normal result of the feasibilization-algorithm to be:
unfulfilled restrictions becoming exactly fulfilled, by
becoming pivot row.

Consider the L.P.-problem, that was first discussed in

Section 7.4, as an example of graphical solution. We solved a
feasible solution to this problem, by extending the old rule
of the smallest quotient, to this non-feasible problem. (See
the preceding Section I of this chapter). But from graph 7.4a
it will be clear that xl = 2 would have been sufficient, while
we increased xl in fact until xl = 7. The choice of s2 itself
would have been more efficient. The step we should have made,
is given below in the corresponding Simplex tableaux.
(Tableaux 9.2a and 9.2b). Tableaux 9.2b and 9.lc both indicate
feasible solutions, but 9.2b indicates the higher solution
value.

TABLEAUX 9.2 A AND 9.2 B

SELECT THE SMALLEST QUOTIENT. WITH A NEGATIVE PIVOT.
NAME! ! X 1 X 2 S 1 S 2 S 3 !! VAL LIE
S 1 ! ! 1 1 ! ! 7
S 2 !! @ -1 !! -6
S 3 !! -1 1 ! ! 2

T !! 3 -1 ! !

NAME! ! x X 2 S S 2 S 3 !! VALliE
----------~-----------------------------------------
S 1 ! ! 0.67 0.33 ! ! 5
XI! ! 0.33 ! -0.33 ! ! 2
S 3 !! 1.33 ! -0.33 ! ! 4

T !! -2 !! -6
THE SEARCH FOR A FEASIBLE SOLUTION 187

This is not just a problem of efficiency, of finding the optimum

in the shortest possible number of steps.

For consider the following L.P.-prob1em

TAELEAU 9.2 C

TO EltTER XI INTO THE BASIS AT ALL.

WE NEED A NEGATIVE PIVOT.

NAME! ! X I X 2 S I S 2 !! VALUE

S I !! ~ -I ! ! -6
S 2 !! -I I ! ! 2

'!' !! -I ! !

This is basically the same problem as the preceding one. The

first restriction, which was not binding on the optimum, has been
omitted. As a consequence of this the second and third
restrictions have been renamed first and second. Like the
preceding one, this problem has a normal optimal solution, again
xl = 1 and x 2 = 3. However, once we take xl as our pivot column,
we are forcea to take a negative pivot, as there are no positive
numbers in the xl-column.

We now formulate the new rule for choosing the pivot row, as
follows:

For each non-zero element of the pivot-column, establish whether

the corresponding element in the "value" column has the same sign
(zeros to be counted as positive). For all pairs of negative
values of basic variables and negative elements of the pivot
column, find the largest quotient of the variable divided by the
candidate-pivot. For all pairs of positive numbers, and this one
pair of negative numbers, find the smallest quotient of the value
of the variable divided by the candidate-pivot. It will be
observed, that all the quotients are positive.

The following examples may illustrate this new rule. We first

consider the example in Section 8.3. This is a feasible problem,
hence no pairs of negative numbers are available. In that case
the new rule is identical to the old one. Then there is the
example of Section 7.4, discussed earlier in this section. There
is one pair of positive numbers (sl), and one pair of negative
numbers (s2). The s3-row is not eligible as a pivot-row, as this
is a pair with unequal signs. If we took s3 as the new pivot-row,
xl would be pivoted into the basis with the negative value of -2.

The largest of the quotients of negative numbers is -6: -3 = 2

for s2' and it is the only quotient in this group. The
188 CHAPTER IX

smallest of the quotients of positive numbers is 7 : 1 = 7

for sl' again in fact the only one. From these two quotients
the smallest must be chosen, and this is -6: -3 = 2 for s2.

Then there is the example of the empty problem, discussed in

this chapter (Section 9.1):

TAELEAU 9.2 D

A SMALLER ~UOTIENT OF TWO POSITIVE

tJt'MEEP.S HAS PRIORITY OVER A EIGGER
QUOTIENT OF TWO NEGATIVE NUMEERS.
NAME!! X 1 X 2 S 1 S 2 !! VALUE
S 1
S 2
! !
!!
CD
-I
!!
!!
3
-I,)

T !! -1 o o o ! !

We search for the most negative element in the s2-row, and

find x2 as our pivot-column. We then consider the quotient
3 : 1 = 3 for sl' which is the smallest (the only one) among
quotients of positive numbers. And we consider -4 : -1 = 4
for s2' being the largest (the only one) among quotients of
negative numbers. From these two quotients, 3 for sl is the
smaller one, and is chosen.

And consider the example in this section (see tableau 9.2c)

In this example, the "search" for the smallest/biggest quotient

is again trivial on account of the presence of only one
quotient of the appropriate sign.
THE SEARCH FOR A FEASIBLE SOLUTION 189

We look for the most negative element in t~e 5 1 -row, f~n~ Xl

as the pivot column, and there are no quot1ents of posit1ve
numbers available, while there is one quotient of negative
numbers. This situation is typical for feasioilization-problems.
We have a row with a negative element in the "value" column, and
we look for the most negative element in that row. Therefore, if
we have found a pivot column, there must be at least one pair of
negative numbers, namely in the unfulfilled restriction that gave
us the pivot column. In the example at hand, we find sl as our
pivot row.

The next example will help us to see the essential efficiency-

advantage of the new rule. Consider the L.P.-problem as given
in~

TAB1.EAUX 9.2 E AND 9.2 r

CHOICE Or THE BIGGER Or TIJO Ql'OTIENTS Or

NEGATI VE NL'MEEPS: rEA!:I E1.E IN ONE STEF.

NAME II X 1 X 2 5 I S 2 5 3 !I VALUE

S 1 II 1 1 I! 6
5 2 11 -1 -3 I! -3
S 3 II 8) ! I -2

T II -1 II

NA'1E II x X 2 S S 2 5 3 I I VALUE

Sill 1 I II 1I
S 2 II -1 -3 II 3
X 2 II -1 II 2

T II -1 II -2

Suppose we start with maximising s2' We find -3 for x 2 as the

most negative elements in the s2- row .

There are now two pairs of negative numbers; -3: -3 = 1 for s2

and -2 : -1 = 2 for s3' of which s3 provides the largest quotient
of negative numbers. There is one quotient cf positive numbers,
6 : 1 = 6 for sl' Of the two quotients, 6 for sl and 2 for s3'
the latter one 1S the smaller. Therefore we choose s as our
pivot row. The next solution will then be x2 = 2, wi~h 5 =4
and s2 = + 3. We have satisfied two violated restriction~.in
one step.

As this again is a problem of only two variables, it can be

illustrated graphically, which has been done in graph 9.2a.
190 CHAPTER IX

7
graph 9.2 a
two restrictions have
broadLy the same sLope
and one may fly through one.

From the graph, we observe that the proposed step will in fact
bypass the second restriction, turning it from a violated into
an amply fulfilled restriction.

In fact, if it were not because of this efficiency-aspect, we

should have taken a more simple rule.

The "conservative" rule of taking always the smallest of all

available positive quotients (including quotients of negative
numbers), would always solve the problem. And it would have the
advantage of being simple.
THE SEARCH FOR A FEASIBLE SOLUTION 191

Under this "conservative" rule, a feasible solution would be

reached in two steps, instead of in one. The corresponding
Simplex tableaux would then be:

TAEL FAUX 9.2 G. H. AND I

CHOI CE OF THE SMALLER OF T1.'O QUOTI ENTS or

NEGATIVE NL'MEERS: FEASIBLE IN T1o.'Q STEPS.

NAME I! X I X 2 S I S 2 S 3 II VALVE

S I ! ! I I ! I 6
S 2 ! I -I E]) II -3
S 3 !! -I !I -2

T !! -I ! I

NAME !! x X 2 S S 2 S 3 II VALUE

S I ! ! 0.67 0.33 I! 5
X ~ ! ! 0.33 -0.33 ! ! I
S 3 ! ! 0.33 ~ ! ! -I
-----------------------------------------------------
T ! ! -1.33 0.33 ! I -I

NA!'1E ! ! X I X 2 S S 2 S 3 II VALL'E

S I ! I I !l 4
X 2 !! -I I! 2
S 2 ! I -I -3 I! 3

T !! -I II -2

It will be observed that, apart from the interchanging of the

position of the rows "X2" and "s2", this tableau is exactly
the same, as the one that would have been obtained in one step,
with the help of the more complicated rule.

There is, apart from its simplicity, yet another argument ~n

favour of the "conservative" rule. Like the ordinary rule of
the smallest quotients, the "conservative" rule tends to avoid
small pivots. The more complicated "efficient" rule on the
other hand, will to a certain extent favour the choice of small
pivots. For consider the following problem:
192 CHAPTER IX

TAELEAUX 9.2 J AND 9.2 K

CHOI CE Of THE BI GGER Of T\lO QUOTI nns Of NE-

GATI VE NL'MBERSI A VERY SMALL PI"VOT RESULTS.

NAME! ! X I X 2 5 I 5 2 ! I VALUE
;-~--~~--~~:~j)-:~-------------------~~--:~--
5 2 I! -I I II -I
----------------------------------------------
! I
T II

NAME! ! X I X 2 5 I 5 2 !I VALUE
----------------------------------------------
100 1-100 ! I 100
X I !!
S 2 I! 101 1-100 II 99
----------------------------------------------
T I I -99 I 100 !!-IOO

If we start this problem with trying to get sl non-negative, no

particular problem occurs. In that case x2 becomes the pivot-
column. And then only sl can become the pivot-row. But if we
start with trying to get S2 non-negative, some strange figures
may occur. Then Xl becomes the pivot column. We will then have
to choose between two pairs of negative numbers:

-1.00 : -0.01 = 100 for sl and -1.00 : -1.00 = 1

for s2' Choosing the larger of the two quotients then will give
us the number 0.01 as pivot. Then any rounding errors in the
sl-row are multiplied by 100. And it is not at all certain that
such a pivoc was really needed. In fact, with the "conservative"
rule, a feasible solution would be attained in two steps,
without the occurring of such small pivots. The next two
tableaux would then be:

TABLEAUX 9.2 L AND 9.2 M

CHOICE Of THE SMALLER Of T\lO QUOn ENTS OF

NEGATIVE NL'MEERS: A BIGGER PIVOT RESULTS.

NAME! I X I x e 5 I 5 2 I! VALUE

5 I !I -0.01 -I ! I -I
52 !IED I I I -I

T !! !I

NAME! I x X 2 5 5 2 I! VALUE

S I ! I - 1.0 I -0.01 !I -0.99

X I !! -I -I !! I

T !! 2 !! -I
THE SEARCH FOR A FEASIBLE SOLUTION 193

The problem can be overcome by modifying the "efficient" rule.

I t will now be:

For each non-zero positive element in the pivot-column, and a

corresponding positive (or zero) value of the basic variable,
find the smallest quotient of the value of the basic variable,
divided by the corresponding element of the pivot column. Compute
quotients of negative values of basic variables, divided by the
corresponding elements of the pivot-column (candidate pivots)
as far as the latter are negative and in absolute value greater
than a specific small non-zero number. If one of these quotients
should be larger than the smallest quotient of positive numbers,
the row which originally gave the smallest quotient of positive
numbers, is the pivot row. If all eligible quotients of negative
numbers have been computed, while none of them is larger than the
smallest quotient of positive numbers, the row in which the
largest quotient of negative numbers was found, becomes the
pivot row and the problem is feasible in one step. If no quotient
at all has been found in this way, whether of negative numbers, or
from positive numbers, find among the negative and non-zero
elements of the pivot-column, corresponding to negative values of
basic variables, the absolutely largest (= most negative) element
in the pivot column; the corresponding row will be the pivot row.
Note that we would not have entered such a column, unless a pair
of negative numbers were available.

This seems quite a mouthful, because it should once be formulated

with some precision. It can also be said a lot shorter:

Find the smallest quotient of positive numbers. Find out if any

quotients of negative numbers are larger, but reject too small
negative pivots. If a quotient of negative numbers is found to be
larger, the quotient of positive numbers provides the pivot row.
Otherwise choose the largest of the quotients of negative numbers.
If no pivot at all is found in this way, we will be forced to
accept a small negative pivot; take the largest (=most negative
one) avai::'able.

The L.P. code which is offered in Section 12.3 applies this rule
with some modifications; its bias for small pivots being one of
the main reasons for these modifications.

When the current solution is not feasible, preference is given

to columns which indicate an increase in the value of the
specified objective function as well as in the value of a
substitute objective function. In such "preferred" columns,
negative pivots are not accepted at all. In other columns,
representing variables which increase a substitute objective
194 CHAPTER IX

function (the sum of the slacks of all violated restrictions),

at the cost of a reduction in the value of the specified
objective function, the criterion of the dual ratio is
superimposed on that of row-selection. A discussion of that
criterion is, however delayed to section 11.3.

Exercise

Starting with tableau 8.lla, solve the example-problem from

section 7.4 several times, once for each pivot-selection rule
discussed in this section. Use shortened tableaux with a
sum+l check column throughout. Check the correctness of your
calculations by

a) Charting xl and x 2 in graph 7.4a, finding them to

match vertices

b) Comparing with the full tableaux, as far as available

in print

9.3 The choice between a number of violated restrictions

Suppose ~n the initial solution a whole series of restrictions

is not satisfied. For instance, let s3' s12' s13' s24' s26 and
s32 have negative slacks. In the preceding paragraph, we
assumed that we chose our pivot column by finding the most
negative number in the row, corresponding to a violated
restriction, or the objective function. In what row should we
look, if there are many negative values? One obvious approach
would of course be: start with s3' then take s12, etc. We
tackle the different restrictions in the order of their index
number. If we start with s3' we may find after one step, that
we came to a solution satisfying s13' by incident, but still
have a negative element in the s3-row. The collection is then
s3' s12' s24' s26 and s32. We then again take the most negative
element in the s3-row, to find a pivot-column. After the second
step, we may have satisfied s3, but also incidentally S12 and
s26. Since we never take a quotient, larger than the smallest
quotient of positive numbers, there are never new names added
to the list of violated restrictions. Our collection is then
s24 and s32' we now take s24 as our next substitute for the
target-row; etcetera.

At each step, we look for the negative variable with the lowest
index-number. In that row, we look for the most negative number.
If we do not find such a negative number, we know that the
problem has no feasible solution. Otherwise we will always end
up with finding a feasible solution.
THE SEARCH FOR A FEASIBLE SOLUTION 195

This procedure is not necessarily an efficient one. Consider

the following example:

TABLEAU 9.3 A

THE EFFI CI ENT STEP DOES NOT CORRESPOND TO THE

STEEPEST ASCENT IN ANY SEPERATE RESTRICTION.

NAME ! I X I X 2 X 3 X 5 II VALUE

S I ! I -2 - I ! I -2
S 2 ! ! -2 -I ! ! -2
S 3 ! ! -2 -I II -2
S II ! I -2 - I II -2
S 5 I! I c=::D I! -3

T ! I I!

In this problem, x5 = 3, with s5 as the corresponding exactly

fulfilled restriction, is a feasible solution. This can be seen
from the corresponding shortened Simplex Tableau:

TABLEAU 9.3 B

OPTIMAL AND FEASI BLE IN ONE STEF. AFTER

STARTING WITH FIVE VIOLATED RESTRICTIONS.

NAME ! ! X X 2 X 3 X II S 5 ! ! VALUE
----------------------------------------------
S I ! ! -3 - I - I - I - I ! !
S 2 ! ! -I -3 - I -I -I ! !
S 3 ! ! -I - I -3 - I - I ! !
S 4 ! ! -I - I -I -3 -\ ! ! \
X ~ ! ! -\ -\
-\ -
3 \ -I ! !
----------------------------------------------
T ! ! 2 2 2 2 !! -3

Now if the problem is started with maximizing s5' this feasible

solution is reached in one step; it is the optimum as well. But
if we start with sl' s2' etc., we will have to run through six
tableaux, before we reach a feasible solution. To some extent,
this is just the result of the fact that the Simplex method is
an iterative method; the properties of the optimum are unknown,
before the optimum is actually solved. But there is one
systematic aspect. If we start with maximizing sl', we will take
xl as pivot column. In doing so, we make s5 more negative. This
is so, because in the xl-column, the element in the s5-row is
positive. Apparently "good" columns are columns with negative,
196 CHAPTER IX

or at least non-pos1t1ve elements in all rows representing

violated restrictions. This cannot be made into a general rule
for finding the pivot column. The normal case will be that all
columns have both negative and positive signs. Could we perhaps
make it the rule, that we should take the column with the
largest number of negative elements in the rows that represent
the violated restrictions? We could not. For consider the
following example, where one step has been made according to
this rule (shortened tableaux).

TABLEAUX 9.3 C AND 9.3 D

INCONSISTENT SVBSTITUTE OBJECTIVE FUNCTION.

CHANGING AT EACH STEP: THE RESVLT IS CYCLING

NAME ! I X 1 X 2 X 3 X 4 X 5 !! VALVE

S
S
1
2
!!
! !
CD
-I -I
!
!
!
!
1
-2
S 3 ! ! -I -I ! ! -2
S 4 ! ! 1 -I ! ! -I
S 5 I! 1 -I ! I -I

T II ! I

NAME !! S 1 X 2 X 3 X 4 X 5 II VALUE

X
S 2
1 I!
! !
CD1 -I
! I
! !
1
-I
S 3 ! ! 1 -I !1 -I
S 4 ! ! -I -I ! ! -2
S 5 !! -I -I II -2

T ! I II

Again taking the column with the largest number of negative

elements in rows 2-5, reproduces the initial solution, and
leads to cycling. We really need the approach discussed in
Section 1 of this chapter. As long as the list of the violated
restrictions is the same, a substitute objective function
should be increased at each step. And as long as the list of
the violated restrictions is unchanged, this should be the same
function at each following step. But there is a compromise.

We can maximize the sum of all negative slacks. Our substitute

objective function will then simply be the sum of all
unfulfilled restrictions. On the average, this rule has a
higher probability of choosing "good" columns, relative to
attacking the violated restrictions in the order of their
ir.dices.
THE SEARCH FOR A FEASIBLE SOLUTION 197

Another reasonably efficient procedure is to apply the

principle of steepest ascent to the selection of the
restriction to be eliminated, but only intermittently. We
choose a "badname", indicating the most negative valued slack-
variable. Then we go on increasing ("maximizing") this negative
slack-variable, until it is eliminated. This is more or less
the procedure we first suggested in Section 9.1. The rule for
selecting the pivot-row is then as follows:

First, consider the "badname" row itself as pivot-row. In the

"badname" row, there is always a positive and non-zero quotient,
between the negative entry in the value column (indicating the
violated "badname" restriction) and the negative entry in the
pivot column (selected as the most negative entry in that
row) .

Refuse all other negative pivots. Apply the rule of the

smallest quotient between the one quotient of negative numbers
already selected, and such quotients of positive numbers as may
be found.

In our example (above) these rules would result in the same

step as for the "efficient" rule. We would take s5 as our
(first) "badname" and provisionally select the s5' x5 element
as pivot. Since there are no positive entries in the x5 column
at all, the question of selection of the smallest quotient does
not arise.

This rule is, on the whole likely to require more steps than
the use of all violated restrictions as substitute objective
function, but it has the advantage of being simpler. (It also
provides a better safeguard against loss of accuracy due to
the choice of a very small pivot. A negative entry in the
"badname" row, selected as the most negative in that row, is
not likely to be very small in absolute value.

9.4 pivot-selection in Phase I: the general case

By way of example, we will now tackle a "general" linear

programming problem. With the word "general" we mean that all
the problems discussed so far, are present in this example.
The initial tableau is neither optimal, nor feasible.

We will use shortened tableaux and apply the "efficient" rules

for selecting pivots. The problem is given in tableau 9.4a
on the next page.
198 CHAPTER IX

TABLEAl' 9 • .11 A
TABLEAU VITH SUBSTITUTE OBJECTIVE FUNCTIONI
THE GENERAL CASE.

NAME I! X I X 2 X 3 X.ll II VALUE

5 I ! I I -2 @ -I ! I 3
5 2 ! ! -2 3 ! ! 2
5 3 ! ! -3 I -2 3 !! -I
5 .II ! ! 3 -2 1 -3 II -I
5 5 ! ! -2 1 -3 -2 !1 -6

T II 3 -I -2 II

51 ! I -2 -2 II -8

The row S1 1S the substitute objective function row, si

standing for sum of infeasibi1ities. This row is obtained by an
addition of the violated restrictions, in this case s3, s4 and
s5. The rule of the steepest ascent, applied on the substitute
objective function, will indicate x3 as the pivot column;
the number -q being the most negative element in the substitute
function. The x3-co1umn is also the only "preferred" column,
indicated by both the specified and the substitute objective
function.

Among the pairs of numbers between the value-column and the

(x3) pivot column, three pairs have equal sign. The
corresponding quotients are 3 : 3 = 1 for sl, -1 : -2 = ! for
s3, and -6 : -3 = 2 for s5. The other pairs have unequal
signs, hence the corresponding rows are not to be considered
as pivot row in this stage. The largest quotient of negative
numbers is found to be -6 : -3 = 2 for s5. The smallest
quotient of positive numbers is 3 : 3 = 1 for sl. Of these
two quotients, the one for sl is the smaller one. The sl-row
will be the pivot-row.

The tableau is updated in the usual way, except for the

substitute objection function. This function is re-specified
itself, every time when one or more violated restrictions cease
to be violated. It is therefore more convenient to simply add
the relevant rows.
THE SEARCH FOR A FEASIBLE SOLUTION 199

Our second tableau will now be:

TABLEAU 9.4 B

WE HIT THE OTHER SIDE OF THE FEASIBLE

SPACE AREA. NO MORE FLYING THROUGH.

NAME I! X 1 X 2 S 1
" X 4
---------------------------------------------
VALUE
0.33
X 3
S 2
II
II -2
-0.67
3
0.33 -0.33 1
"" 2
S
S
3
4 "
II
-2.33
2.67
-0.33
-1.33
0.67
-0.33
2.33
"
-2.67 II
1
-2
S 5 II -1 -1 1 -3 II -3
---------------------------------------------
"
T II 3.67 -2.33 0.67 0.33 2

From the tableau, it will be observed, that s3 has dropped from

the list of the violated restrictions. The slack-variable s3 is
still in the basis, but now with a positive value. We could of
course have followed the conservative rule, and have taken s3
as the pivot-row, instead of sl as we did. The corresponding
value for x3' the new basic variable, would have been !, if s3
had been our pivot row. What have we gained by increasing x3
still more, until x3 = I? This can be seen from the previous
tableau. On inspection of the x3 column in that tableau we see
that such further increase of x3' has increased ss with
! x 3 units, and has reduced s4 with ! ~ 1 unit. Then the sum
of the negative slacks s4 and ss has increased 1 units, by
using the "efficient" rule for choosing the pivot row instead
of the "conservative" one. This "extra" is of course a per
chance possibility. But it is not quite incidentally. The
pivot column x3 was chosen because -4 was the most negative
element in the substitute objective function. This was the sum
of such elements in the x3 column, as corresponded to the
negative slacks. Because it was an in absolute value relatively
large number, this sum did not change sign after one of the
variables dropped from the list of negative slacks. The
remainder list still provided a negative sum. The other "extra"
possibility of the "efficient" rule is the dropping out of more
than one name from the list of negative variables. But this
did not materialize.

9.S The method of artificial variables

This is the method most commonly advocated in literature on

the subject. For each restriction, not satisfied by the trivial
solution, one defines an artificial activity. This artificial
variable in fact is minus the slack of the violated restriction.
This variable declares the restriction to be satisfied, but at
the price of a huge penalty-element in the objective function.
200 CHAPTER IX

Alternatively, one first solves the problem of minimizing the

sum of the artificial* variables.

This defines a related problem. The related problem is a

problem, of ~hich we may for some reason assume that it will
have a solution similar to the solution of the original problem.
This assumption is in the present case based on the "penalty"
elements in the objective function. Any solution to the related
problem, with one of the artificial variables being one of the
basic variables, will have a huge negative value for the
objective function.

When the original problem has no feasible solution (is empty),

the artificial variables cannot be eliminated out of the basis
of the related problem. Otherwise, the optimum of the related
problem will be the optimum of the original problem.

Let us now see, how an L.P.-problem is solved with the method

of artificial variables. Consider again the example given in
Section 2 of this chapter. The initial tableau for this problem
in "ordinary" extended form is already given as tableau 9.2e.

The related problem then is the one given below in tableau 9.5a

TAELEAU 9.5 A

TABLEAU WITH ARTIFICIAL VAPIABLESI TO

BE EXCHANGED AGAINST NEGATIVE SLACKS.

NAME! I X I X 2 A I A 2 S I S 2 S 3 ! I VALUE
-----------------------------------------------------------
S I ! ! I I !I 6
S 2 I! -I -3 II -3
S 3 !I -I II -2
-----------------------------------------------------------
T II -I 100 100 II

The artificial variables are now pivoted against the

corresponding slacks. To all practical purposes, this amounts
to changing the signs of the rows, representing the violated
restrictions. The artificial variables are the negative
counterparts of the slacks; they become basic variables, with

*cf W. W. Garvin "An Introduction to Linear Programming" nIl

pp.39-46 or G.B. Dantzig "Linear Programming and Extensions"
[8J pp.94-103.
THE SEARCH FOR A FEASIBLE SOLUTION 201

positive values, instead of the slacks with negative values.

After two steps, this automatically results in a feasible
solution to the related problem:

TABLEAU 9.5 E

ARTIFICIALLY FEASIELE STARTING SOLUTION.

NAME!! X 1 X 2 A 1 A 2 S 1 S 2 S 3 !! VALUE
-----------------------------------------------------------
I! 6
S \ ! ! 1
AI! ! @ -I I! 3
A 2 !! I -1 !!
-----------------------------------------------------------
!! - 50 0
T !1-101 -399 100 100

If the original problem has a feasible solution at all (if

it is not empty), the optimum of the related problem will be
such, that the artificial variables are not in the basis. Their
elimination is achieved by the normal process of simplex
operations. In the present example there will be two more
tableaux, before a 2 and a 3 will have been eliminated.

TABL EAUX 9. S C P.ND D

n:o FURTHER TAELEAL'X OF THE SAME PROBLD'!.

NAME I! X I A 1 A 2 S I S 2 S 3 II VALUE
-----------------------------------------------------------
S \ ! ! 0.67 -0.33 II 5 0.33
X 2 ! I 0.33 0.33 -0.33 II I
A 2 ! ! -0.33 -0.33 ~ -I ! I I
-----------------------------------------------------------
T ! ! 32 133 -33 100 ! ! - I0\

NAME ! ! X X 2 A A 2 S S 2 S 3 ! ! VALUE
-----------------------------------------------------------
1.00 !! 4
S \ ! ! - \ • 00
X 2 !! I -I ! I 2
S 2 !!-\ -\ 3 -3 II 3

T ! I - I 100 99 ! ! -2

Tableau 9.Sd corresponds to a feasible solution of the original

problem.

If we have tried to obtain a feasible solution to the original

problem, by maximising the sum of all negative slacks, while
using the "conservative" rule of pivot-row selection we would
have obtained the same tableau except for the absence of the
202 CHAPTER IX

artificial columns. Once the right-hand side is positive the

columns which represent artificial variables can be suppressed
anyway. If we had used the "efficient" rule for choosing the
pivot row, the same tableau would have been obtained in one
step. (x2 as pivot column, against s3 as pivot row). If we had
followed the "conservative" rule of taking the smallest among
all positive quotients, we would, to all practical purposes,
also have made the same steps. The difference would be that we
would not have described them as eliminating artificial variables,
but as eliminating negative-valued slacks out of the basis. The
pivoting operations would have been numerically the same, except
that we would have met positive quotients of negative numbers.
The artificial variables have in fact served as operators, for
adding the violated restrictions, blown up by a factor 1000, to
the objective function. However, it is not too difficult to find
an example (e.g. the one in section 9.4), for which several
versions of the substitute objective function method will take
advantage of the possibility to "fly through" several violated
restrictions in one step. With artificial variables this is not
possible.

9.6 Non-updating of the substitute objective function

It is not in practice necessary to computationally treat a

substitute objective function as a function at all. In particular
if the substitute preference function is the sum of the
infeasibilities, the substitute preference coefficients may
be computed ad hoc by adding the coefficients of the violated
restrictions.

Example

Minimise 3x l + 4x2 (= maximise - 3x l - 4x 2 )

xl + x 2 > 10 (- xl - x 2 < -10)

xl > 2x2 + 1 (- xl + 2x2 < -1)

x2 > 2 (- x 2 < -2)

xl < 8

(<Xl' x 2 .:.. 0)
THE SEARCH FOR A FEASIBLE SOLUTION 203

The set-up tableau with the first pivot and the second tableau
are given here with a substitute objective function row.

TI'.BLEAUX 9.6 A ANI) 9.6 B

THE St!M-Of-H:fEA!;I BILl TI ES-T:01..' 1 S OBTAINED BY ADCltlG

I'.LL THE RO\!S REFERRING TO VIOLATED RESTRICTIONS.

IN 9.6 1'.: It! 9.6 E: AND 3.

NI'.ME ! x X 2 VALUE NAME! X 2 VALUE

S 1 -1 - 1 - 10 S 1 -1 -2
!; 2 -1 2 -1 S 2 2 7
S 3 -1 -2 S 3 -1 -2
S lj CD
----------------------------
B X 1
----------------------------
8

T 3 T -3
51 -13 51 I -£

In the initial tableau the substitute objective function is

si = 51 + 52 + 53' and we computationally obtain it by adding
the corresponding rows of the tableau. If we now look at the
second tableau we note that the substitute objective function
now is si = 51 + 53' 51 and 53 being the negative-valued slacks,
the 52-restriction being now satisfied. From the 51 and 53 rows
of the tableau we read the expressions for 51 and 53' i.e.

- x 2 + 53

si 51 + 53 -54 + 2x2 4
Putting this again in implicit form, we obtain (the substitute
objective function row).

5 - = -4
4
It is obvious that this row is computationally most easily
obtained as the sum of the 51 and the 53 rows as they are
already in the tableau. Furthermore, if one were to update the
old substitute objective function, the result would not be
valid, insofar as 52 is concerned. The reason is
that the collection of violated slacks has changed, si = 51 +
52 + 53 is no longer the correct substitute objective function.
We ~lave no reason to increase the value of 52' now that this
slack has ceased to be negative.
204 CHAPTER IX

Exercise 9.6a

The following L.P. problem is given:

Minimise Xl + x2

subject to 3x l + 2x2 > 2

xl ~ x2 + 1

x2 ~ 2 (xl' x 2 > 0)

The following should be done:

1. Make a graphical mapping and find the optimal solution

by means of graphical analysis.

2. Solve the problem by means of a version of the Simplex

Method, using

(a) full explicit tableaux, including unit vectors.

Also write the corresponding systems of equations,
with explicitly written variables (carry a sum-count
column);

(b) condensed (shortened) tableaux.

Check that algebraically calculated solution vectors match

vertices in the drawn graph.
CHAPTER X

MIXED SYSTEMS, UPPER AND LOWER BOUNDS

10.1 Variables without sign restriction

So far, we have assumed that the so-called "tacit" restrictions

xl ~ 0 (all j) apply.

There is no good reason, other than convenience why this

restriction should always apply. Problems in which ~ome
variables are "free" variables, or variables "of type absolute",

,
occur and can be solved.

Some textbooks* recommend to represent "free" variables by

defining plus or minus the same variable as technically two
variables. For example, a problem in two variables where xl ~s
not restricted in sign might be written as:

Maximise x - x ln + x 2
lp
Subject to 2x - 2x 2n + x 2 < 10
lp
x 2 < 20

< 2

< 1

(x lp , x n ' x 2 ~ 0)

We have "split" Xtinto two variables, xl p being the positive

side of xl' and xln the negative side of xl.

There is however, no need for this cumbersome approach. It is

enough to require that xl is a basic variable irrespective of
its sign. As xl does not have to be positive it can be pivoted
into the current solution, if so desired with a negative pivot.
And, once xl is represented in the Simplex tableau by a row,
it is except from the search operations.

*cf. W, W. Garvin [1~ p.4

205
206 CHAPTER X

The following tableaux may now be developed.

TABL.EAU 10.1 A
SET-UP TABL.EAU WITH A fREE
VARIABL.E. NO SPECIAL. fEATURES.

NAME I X 1 X 2 II ITAL.UE

S 1 2 II 10
S 2 II 20
S 3 -I II 2
S 4 CD II 1

T -I -I II

Both s3 and s4 are acceptable as pivot-rows. Since x 2 = 1

gives rise to a higher solution value than x2 = -2, we might
take the s4-row. (The code offered on section 12.3 chooses
the lowest absolute critical ratio).

TABL.EAU 10.1 B
THE fREE VARIABL.E X I HAS
NOW BECOME A BASIC VARIABL.E.
~AME X 2 S 4 II VAL.UE
S
S
I
2
<DI -2 II
II
8
20
S 3 II 3
,. X I II 1 *NOT TO BECOME PIVOT ROW
T -I II
MIXED SYSTEMS, UPPER AND LOWER BOUNDS 207

We now enter x2 into the current solution, by the normal

rules of steepest ascent and smallest quotient, and write the
third tableau:

TABLEAU 10. I C

X2 IS ~OV ALSO IN THE BASIS.

NAME 5 5 4 II VALUE

X 2 I -2 'I 8
5 2 -I 2 I I 12
5 3 <DI II 3
* X I II I *NOT TO BECOME PI VOT ROV.

T -I II 9

This time the rule of the steepest ascent gives us s4 as

pivot-column. Excluding xl from the search-operation, we find
s3 as the pivot-row and make the step

TABLEAU 10.1 D

54 HAS DRIVEN OUT 531 NOT XI.

NAME I 5 I 5 3 II VALUE

X 2 1 2 II 111
S 2 -I -2 II 6
5 4 1 II 3
* X 1 -1 II -2 *ALLOVED TO BECOME NEGATIVE

T II 12

And we find Xz = 14, xl = -2 as the optimal solution.

Note that it is in this context meaningful to have a negative

lower limit on a free variable e.g. xl ~ -2, written as
-xl ~ 2. There is, however, a more efficient way of handling
restrictions of that type (see section 10.4).

10.2 Equations

The most practical way to integrate equations into the Simplex

Algorithm is first to make sure that they become binding
inequalities, and then to exclude their slack-variables/shadow
prices from the list of eligible pivot columns. In terms of
208 CHAPTER X

interpretation such "slack variables" are then in fact

artificial variables but the difference in computational
procedure is not all that much.

One method of ensuring the elimination of the "slack-variables"

of equations is to present them as violated restrictions, and
then to apply the "conservative" rule in the search operation
for a pivot row, i.e. to accept negative pivots in rows
representing equations.

Example

Maximise T xl

Subject to xl + ZxZ + x3 10

Xz > x3
I
Xz < iX3 + 3

The initial Simplex tableau is now written as follows:

TABl.EAU 10.2 A

PRESENT AN EQUATION IN THE

FORM OF A VIOl.ATED INEQUAl.ITY.

NAME! x X 2 X 3 VALUE

S 1 -2 -I I. -12
S 2 -1 1 1 ~ I o
S 3 1 - O. SO ! ~ I 3

T -I o o

The = sign in the sl-row is so far an intention, not a met

requirement: The trivial basis is Xl = Xz = x3 = 0 and this is
not equal to -lZ.

Quite apart from the equation-status of the sl-row, the efficient

way to solve this problem is by activating the "preferred"
Xl-column.
MIXED SYSTEMS, UPPER AND LOWER BOUNDS 209

The next tableau therefore is (after re-ordering):

TABLEAU 10.2 B

OPTIMAL rABLEAU. YITH A NEGATIVE-

VALUED 5HADOYPRICE OF AN EQUATION •

NAME I X 2 X. 3 5 1
• II VALUE

X 1 2 1 -I II 12
5 2 ·1 1 II o
S 3 1 -0.50 II 3

T 2 -I II 12

51 NOT TO BECOME
•
INCOMING VARIABLE.

This tableau is in optimal form, despite the fact that the

updated form of the objective function still conta;n~ ~
negative element in the sl-column.

The objective function row, represents the equation:

or, written in explicit form:

T =
Thus, we could increase the value-of the objective function
i.e. Xl by one unit for each unit of sl' But since Xl already
has the value of 12, Xl + 2x2 + x3 would become greater
than 12 if we did so, contrary to the first equation-requirement.
Shadow prices of equations are allowed to be negative-valued in
the optimal solution.

10.3 Upper bounds and the two value columns

It is obviously possible to list an upper bound on a variable

as a restriction.

In the example in section 10.1 there was an upper bound of 20 on

Xl' i.e. we listed the requirement Xl ~ 20 as a restriction.

Restrictions of this type occur fairly often, and some problems

could be substantially reduced in size, if all variables were
considered to have an upper bound. One coul~hen carry two
value columns, one giving the distance of the variables from
210 CHAPTER X

zero, i.e. their value, and the other from a specified upper
bound.

Example

Maximise T 2x l + x 2 + x3

xl < 2x2 + x3

We Will indicate the slack-variables of the upper limit

restrictions with the letter b, bj being the distance between
Xj and its upper limit. We need to store explicitly only the Xj
or the bj-column, not both at the same time. (While referring
to upper limits we shall refrain from using matrix notation,
as confusion with the elements of the vector b might otherwise
arise.)

A suitable place to write the upper limits in the tableau,

is below the columns which relate to the variables in question,
in the (m+2)nd row. The set-up tableau now becomes:

TABL EAU 10.3 A

SET-UP TABLEAU WITH UPPER LIMITS.

NAME X I X 2 X 3 II VALUE OIST.

S I CD -2 -I II o 1000
S 2 I I -I II 20 1000

T -2 -I -I II x
BOL'NO I 10 20 30 II x x

The column entitled "Dist" gives the distances between the values
of the basic variables and their upper limits - As no upper
limits for slack-variables have been specified, this column so
far consists solely of dummy-entries. In the interest of
uniform tableau-manipulation, a "fancyhigh" number of 1000
has been entered, but the algorithm does not include upper
limits on slack-variables in the search operations.

The rule for the choice of the pivot row is modified. Eligible
quotients are ratios between the entries in the value column
divided by the corresponding element in the pivot-column as
MIXED SYSTEMS, UPPER AND LOWER BOUNDS 211

well as the entries in the distance column and minus any

negative elements in the pivot column, (as far as they refer
to elements of ~). If the selection of the incoming variable
is by the steepest ascent, we first make an ordinary step,
developing the following tableau:

TABLEAU 10.3 B

THE SAME PROBLEM AFTER ONE ORDINARY STEP.

NAME S I X 2 X 3 II VALUE DIST.

X I
S 2
I
-I
rn
3
-I I I
II 20
0 10
1000

T I 2 -5 -3 II o x
BOL'ND I 1000 20 30 II X x

The entry of 10 in the "distance" column for xl is the

originally specified upper bound of 10, less the value at which
xl is brought into the basis - zero in this case -. In the
upper bounds row we copy in the new pivot column the sum of the
entries in the value column and the distance column.
(For the Sl slack-variable, this calculation is not really
relevant, as we do not recognize an upper limit on a slack-
variable. )

We now find x2 as incoming variable.

The normal rule of the smallest quotient amended by the

inclusion of the absolute value of negative quotients with the
upper bounds column, indicates that the upper bound on xl is
the pivot row. (The pivot has been marked with a square rather
than a circle, to avoid confusion with "normal" pivots).

We first write the bl-row in the tableau. The row which

describes the upper bound on xl differs from the xl-row, in the
sign, and in the entry in the value column. Anything which
increases xl decreases the distance from the upper bound on xl'
and vice versa. The signs in the xl-row have therefore been
reversed, except for the "value" and "distance" entries, which
have been interchanged. We thereby obtain tableau 10.3c:
212 CHAPTER X

TABL EAt: 10.3 C

FINDING AN UPPER LIMIT BINDINGI WE

VRITE THE CORRESPONDING ROV EXPLICITLY.
NAME S 1 X 2 X 3 !I VAL~E DIST.

------------------------------------------
T I 2 -5 -3 II o X
BOl'ND ! 1000 20 30 II X X

The pivot can now be marked in the "normal" way and we make the
step, to obtain Tableau 10.3d

TAELEAU 10.3 0

TAELEAU VITH A EINDING UPPER LIMIT

NAME S B X 3 II VALUE DIST.

X 2 -~ 0.50 0.50 II 5 15
S 2 ~ -1.50 -1.50 II 5 1015
---------------------------------------------
T I -0.50 2.50 -0.50 II 25 X
BOI..'ND ! 1000 10 30 II X X

The entries in the "distance'" column are, in terms of

interpretation, the slacks of the upper limit restrictions on
the corresponding variables . •-unply fulfilled upper limit
restrictions are not stored, except when they were specified
as "ordinary" restrictions.

We need a somewhat different rule for updating the distances

column. The sum of the value of a variable itself, plus the
distance from the upper bound should not change. Therefore,
when an entry in the value column is increased the corresponding
distance from the upper bound is reduced, and vice versa. Hence
we reduced, by normal updating rules the 52-entry from 20 to 5;
we therefore increase the corresponding "distance" entry by 15.
Thus their sum stays 1020 as it originally was.

The value- and distance column entries in the pivotal row

(bl/x2 in the example), are dealt slightly different. lhe 5 for
x2 itself is the critical ratio (as usual). The 15 in the
distance column is the difference between the original upper
bound of 20, and this figure of 5.
MIXED SYSTEMS, UPPER AND LOWER BOUNDS 213

Generally the upper limit on a new incoming variable is obtained

by subtracting the value of the incoming variable from its upper
limit, as stored in non-updated form.

We may now develop. a number of successive tableaux, in which

there is no direct reference to a variable which is at its upper
limit. If we bring in sl as the next incoming variable the rule
of the smallest quotient is applied between l5/! = 30 for the
upper limit on x2 (where the denominator is ! instead of -! as
the variable which might be driven out is not x2 but the slack
of the upper limit on x2, and

51! = 10 for s2.

This gives S2 as the leaving variable, and the next tableau is

tableau 10.3e.

TAELEAU 10.3 E
XZ MID ITS LIMIT-DISTANCE STILL ACD TO 20.

NAME S 2 B 1 X 3 II VALt:E DIST.

----------------------------------------------
X2 1 -1 EI II 10 10
S 1 2 -3 -3 II 10 990
----------------------------------------------
T I I 1 -2 II 30 x
8Cl'ND I 1020 10 30 II x x

The 1020 for s2 in the upper bounds row is the sum of the entries
in the two value columns in the previous tableau 10.3d. For a
slack-variable, this is not actually a relevant calculation, but
it illustrates the method of calculating the upper limit on a
leaving variable.

Now x3 is the incoming variable. There is in fact only one

quotient to which the rule of the smallest quotient may be
applied, viz. lOll for the upper limit on x2. (990/3 for the
"upper limit" of sl does not qualify, because we do not recognise
upper limits on slack-variables.)

At this point, we will dispense with further repetition of

illustration of this particular type of step, and leave the
completion of this example on the lines indicated, to the reader.

However, one point which warrants further discussion concerns

the updating by vector-operations only. If a variable drives out
its own upper limit distance, the coefficients for the non-basic
variables are unaffected. This is on account of the block-
214 CHAPTER X

diagonal structure of the block-pivot

(10.3.1)

where A*ll is the old block-pivot, before the upper limit became
binding, All being the new block-pivot; ~lj then is the column
of the old A12 block which describes the relation between the
incoming variable Xj and the old basic variables ~*j'

A current tableau extract, relating to these variables is

T* = x*
1
(10.3.2)
b.
J.
whereas the inverse of the new block-pivot is:

(10.3.3)

(see also section 3.11 for the rules of block-inversion).

Obviously, the reverse is true as well, if an upper limit

distance bj becomes incoming variable, arid the leaving variable
is the corresponding variable itself which moves from its upper
limit to zero, the coefficients of the non-basic variables are
not affected either.

We illustrate this possibility by solving the same problem again,

this time emplying the rule of the highest step as pivot
selection rule.

It so happens that this leads to a choice of pivots where all

three variables move right to their upper limits. Reference to
the similar problem with the explicitly written restrictions
illustrates the fact that the upper limit itself is an eligible
quotient. This is illustrated in tableau lO.3f.
MIXED SYSTEMS, UPPER AND LOWER BOUNDS 215

TAEL £AU 10.3 F'

THE SAME PROBLEM. \11TH

EXPLICITLY \lRITTEN LIMITS.

NAME I X 1 X 2 X 3 r VALUE
----------------------------------
S I -2 -1 o
S 2 1 -1 20
S 3 10
S 4 20
S 5 30
----------------------------------
T -2 -1 -1

In the set-up tableau itself, there are four eligible pivots,

viz:

sl/xi with ~T=O (actually selected when the rule of the steepest
ascent is applied), in the xl-column,

s2/x2 and s4/x2 with co-equal ratios of x2 20 for each, with

~T=20 in the x 2 column, and

ss/x3 with ~T=30 as the only elgible quotient in the x3 column.

When working with a tableau with a separate upper limits row,

we therefore have a further modification of the rule of the
smallest quotient.

If no quotient is found, or if the smallest quotient as found

exceeds the value of the upper limit on the variable itself, the
value of the incoming variable at the next vertex is the upper
limit on the incoming variable itself.

This is the case with the pivot which is actually selected:

b3/x~. As the unity coeff-icient ,is not stored, we put the usual
mark1ng on the value of the upper limit itself. We now summarize
the solution of the problem in the tableau series 10.2g to 10.2j.
216 CHAPTER X

TABLEAUX 10.3 G. H AND

SOLUTION OF THE SAME PROBLEM BY THE

RULE OF THE HIGHEST 5TEPI
VECTOR-UPDATING ONLY' 15 REQUIRED.

NAME X 1 X 2 X 3 11 VALUE DIST.

5 1 -2 -I I I o 1000
5 2 1 -I II 20 1000

T -2 -1 -1 II x
BOUND 10 20 (ID I1 x X

NAME X 1 X 2 B 3 II VALUE DIST.

5 I -2 , I 30 970
5 2 1 II 50 970

T -2 -1 I II 30 X
BOUND @ 20 30 I I X X

NAME B 1 X 2 B 3 'I VAL UE 01 ST.

5 I -1 -2 I I 20 980
5 2 -1 1 II 40 980

T 2 1 , I 50 x
BOL'ND 10 30 II X X

NAME B 1 B 2 B3 "VALUE DIST.

5 1 -1 2 II 60 9110
5 2 -1 -1 II 20 1000
T I 2 1 I I I 70 X
BOL'ND I 10 20 30 II X X
MIXED SYSTEMS, UPPER AND LOWER BOUNDS 217

To derive from the set-up tableau (10.3g = 10.3a), the

successor tableau 10.3h, we adjust the signs of all the entries
in the x3 column (but n.ot of the value of the upper limit!),
adjust the two value columns, and leave the rest of the tableau
unchanged.

The same applies for the xl/bl column at the next succession,
and for the x2/b2 column at the transition to the optimal
tableau.

Exercise 10.3a

Solve the example problem of this section three times, twice

using the "normal" set-up tableau 10.3f and similar successor
tableaux, and once using the rule of the steepest ascent,
once using the rule of the highest step, the third time by
completing the tableau-series started with tableau 10.3A to
10.3e, selecting the incoming variable on the basis of the
rule of the steepest ascent.

10.4 Lower bounds: a question of problem formulation

A lower bound on a variable x. is a minimum value which x. ~s

required to attain J J

x. > m. (10.4.1)
J J
Here mj is the minimum value of Xj which is an exogenous
number. We assume that, if such a restriction is specified for
mj < 0, it overrides the non-negativity requirement on Xj:
there is no point in listing it otherwise.

The most efficient way to handle restrictions of this type, ~s by

re-formulating the problem. We denote the slack-variable of
(10.4.1) as

y. = x. - m. (10.4.2)
J J J
The re-formulation of the problem then consists in substituting
Yj + mj for Xj at the outset, and Xj - mj for Yj after solving
the re-formulated problem. The shadowprice of Yj obviously
is that of the lower bound on Xj. We give two examples, one
for mj > 0, one for mj < 0:

Maximise

Subject to
218 CHAPTER X

Hence Y2 = x2 - 2, and the LP algorithm is applied to the

re-formulated problem

Maximise T = xl + Y2 - 2

Subject to xl + 2Y 2 < 6

That problem is then solved (we f~nd xl = 6, Y2 = 0), and the

solution of the original. problem 1S xl = 6, x 2 = 2.

Example for m. < 0

J
Maximise

Subject to 2xl + x2 < 6 (xl> - 5, x 2 > 0)

- xl + 2x2 < 2

Yl = xl + 5, the re-form~lated problem is

(substituting Yl - 5 for xl):

Maximise T = Yl + x2 + 5

Subject to 2Y l + x 2 < 16

-Y l + 2x 2 ..::. - 3

The solution of this problem is Yl = 7, x 2 2, therefore

xl = 2, x 2 = 2.

To implement the lowerbound facility computationally we need to

impose some conventions with respect to storage. It is here
assumed that the actual figures mj are initially stored as an
additionally (m + 3)d row of the tableau, i.e. the last example
is written as

TABLEAU 10.4 A TABLEAU 10.4 B

SPECIFIED PROBLEM WITH LIMITS RE-INTERPRETED PROBLEM
NAME! X 1 X 2 I VALUE DIS NAME I Y 1 X 2 ! VALUE DIS
S 1 2 1 6 x S 1 2 1 16 x
S 2 -I 2 2 x S 2 -1 2 -3 x
T 1-1 x T I -I 5 x
UB 100 100 x x UB IDS 100 X x
LB! I -S 0 x X LB -5 0 X x
MIXED SYSTEMS, UPPER AND LOWER BOUNDS 219

where the number 100 simply represents a "very high" non-

meaningful upperbound.

The operation of re-formulating the problem is sufficiently

simple to make it uneconomic to programme it as a separate
procedure.

The following lines may be incorporated in any programme instead

'FOR' J:= 1 'STEP' 1 'UNTIL N 'DO'

'FOR' 1:= 1 'STEP' 1 'UNTIL M+l 'DO'

:= T [1, N+l] - T [I,JJ * T [M+3, J1 ;

'FOR' J := 1 'STEP' 1 'UNTIL' N 'DO' T[M+2, J] := T[M+2 J]

- T[M+3,J];

A re-conversion at the end is not really needed, it is a

question of interpreting the outcome.

To obtain a result which relates to the originally specified

problem we do not need to change the optimal tableau at all.
It is sufficient to interpret the result in terms of the
originally specified problem. (See also tableau 10.4c).

TABl. £AU 10.4 C

INTERFRETATION OF AN OPTIML~ WITH A l.OWER l.IMIT

NAME I X 2 S 2 I! VALUE DIST

Y 1 -2 -1 ! I 3 102 (MEANS X 1 -2)

S I 5 2 I! 10 X

T 1 I! -3 X
UB 100 X ! I X X
l.B -5 o I I X X

The procedure REPO listed below (a result reporting procedure)

does just that, and its text needs little discussion here. One
point which is however, worth a mention at this point is the
possibility to use large negative lower limits to create, in
effect free variables. This is possible, but not advisable.
Non-zero lower bounds, negative or positive should only be
specified if they are meaningfully intended and do not involve
non-meaningful outsize numbers. The transformed problem with
220 CHAPTER X

the y-variables starts with the trivial basis and if wide-out

lower limits are supplied, this solution corresponds to wide-
out negative values of the corresponding elements of x in the
initially specified problem. The trivial solution of the
transformed problem will then normally contain large positive
as well as large negative entries in the value column i.e.
setting these variables at their far-away negative values
results in a solution which is not anywhere near the feasible
space area.

To start that far away from the feasible space area has two
main disadvantages, viz:

1) It requires the activation of the relatively inefficient

Phase I part of the algorithm.

2) Manipulation with large numbers may result in loss of

numerical precision e.g. if only 6 significant digits are
stored 1000 000 + 0.25 is not distinguishable from 1000 000.

The text of the reporting procedure contains a number of

references to the machine implementation of the L.P. algorithm.
In particular the coding conventions i.e. the recognition of
slack-variables and upper limit distances, as distinct from
ordinary variables may be unclear to the reader until he has
read chapter XII.

The main reason for listing it here rather than in chapter XII
is to further emphasize the fact that calculation of the actual
values of Xj by means of (10.4.2) is actually part of what is
basically a result-output procedure rather than a part of the
algorithm.

The text is now listed, as follows:

'PROCEDURE' REPOCT,M,N,NEQ,NAV,ROWL,COLL);
'ARRAY' r; 'INTEGER' M,N,NEQ,NAV;
'INTEG ER' 'ARRAY' R01o.'L, COLL;
'BEGIN' 'INTEGER' I ,J, R,K,NS;

NEWL INEC 1)J

WRITETEXTC 'C'SOLUTION REPORT')');
NE'Jl.INEC 3);
WRI TETEXT C ' C 'REPORT%ON% TH E% EL EMENTS %0 F%X') , );
NE'Jl.I N EC 2);
WRITETEXTC'C'FREE%VARIABLES') ');
N E1o'L I N E Cl ) J
MIXED SYSTEMS, UPPER AND LOWER BOUNDS 221

'TOR' .1 ..-1 'STEP' I 'UNTIL' NAV 'QO' 'BEGIN'

NE101LINECI)I 1o.'RITETEXTC'C'X')')I
PRINTCK. 6.0);
SPACECIO)I
'Ir' RO"'l.C.11=.1 'THEN' PRINTCTC.1.N+I1.5.2)
'ELSE' IoIIUTETEXTC 'C 'NOTITOl'NO') ')J 'END';

'IT' NAV "' 0 'THEN' Io'RITETEXTC 'C 'IINONE') ');

NE1o.'LINE(2);
IoIRITETEXTC'C'BOUNOEOIVARIABLES')');
NEIo'LINEC I)J

'TOR' KI=NAV+I 'STEP' 'UNTIL' N 'DO' 'BEGIN'

NEIoILINE<1) j
10TRI TETEXTC 'C 'X') ');
PRINTCK.3.0);
'TOR' .1:"'1 'STEP' I 'UNTIL' N '~O' 'BEGIN'
'Ir' COLLC.11=K 'THEN' 'GOTO' NON BA5ICI
'IT' COLLrJ1aI0000+K 'THEN' 'GOTO' UPPER LIM; 'END';
'rOR' 1:=1 'STEP' I 'UNTIL' M '~O'
'IT' RO"'LCI1'"'K 'THEN' 'GOTO' BASIC;

1oTRITETEXTC'C'ABSENTlIo'ITHOUTIEXFLANATION')');
'GOTO' END or XK REPORT;

NON BASICI
Io'RITETEXTC'C'IIATIL01o.'ERILIMIT%OTI')');
PRINTCTCM+3.K1. 5. 2);
'GOTO' END OT XK REPORT;

UPPER LIM:
1o.'RITETEXTC 'c 'IIATIUPPER%LIMITIOTI') ');
PRINTC TC M+ 2 • .1 l+TCM+ 3. K 1.5.2);
'GOTD' END OT XK REPORT;

BASIC:
SPACEC 20); PRINTC CTC I. N+ 11 +TCM+3.K1). 5.2);

END OF' XK REPORT: 'ENC';

N EWL IN EC 3);
10TRI TETEXTC 'C 'SLACKIVARI ABL ES') ');
NE1oTLINE(1); NS pc 0;
'TOR' R:=I 'STEP' I 'UNTIL' M 'DO'
'TOR' 1:=1 'STEP' 1 'UNTIL' M 'DO'
'IT' RO\'LCI1=1000+R 'THEN' 'BEGIN'

NE\'LINECI); NS:=N$+J;
1oTRITETEXT< 'C '5') '); PRIoNTCR.3.0);
SPACEClO); PRINTCTCI.N+I1.5.2); 'ENC';

'IT' tiS .. 0 'THEN' Io'RITETEXTC'C'IINONE')');

222 CHAPTER X

NE101LINE(3)J
101P.ITETEXT(·(·OUALISOLUTION·)·)J
NE ....LINE( 2)1
101RITETEXT('('SHADOWPRICESIOFIEXPLICITIRESTRICTIONS')')J
NEWLINE(l); NS:cO;

'FOR' R: .. I 'STEP' I 'UNTIL' M '00'

'YOR' Jlcl 'STEP' I 'UNTIL' N '00'
'IY' COLLCJ)-lOOO+R 'THEN' 'BEGIN'
NEWLINE( I); NSI =NS+ lJ
WRITETEXT( '( 'P')')1 PRINT<R.3.0);
SPACE( 10); PRINT(TCM+ I.J). 5.2>J 'ENO'J

'IY' NS = 0 'THEN' WRITETEXT('('IINONE")'H

NE1.'L I N E( 3)1 NS I" OJ

WRITETEXT(·(·SHAOO .... IPRICESIOFILOWEP.ILIMITS·)·)J
'YOR' KI=NAV+I 'STEP' I 'UNTIL' N ,~O'
'YOR' Jlcl 'STEP' 1 'l'NTIL' N '00'
'IY' COLLCJ)=K 'THEN' 'BEGIN'
NE'o'LINE(l); NSI=NS+lJ
WRITETEXT('('OFIX')')J
PRINT(K.3.0); .... RITETEXT(·(·IIATI')')J
PRINT(TCM+3.K).5.2);
SPACE( lO)J PRINT(T[M+ l.Jl. 5.2)1 'END'.;

'IF' NS .. 0 'THEN' .... RITETEXT(·( 'IINONE') 'H

NE1o.'LINE(3)J NS I"' OJ
WRITETEXT('('5HAOOWSPRICESIOFIUPFEP.ILIMITS') ');
'FOR' KI=NAV+l 'STEP' I 'UNTIL' N 'DO'
'YOR' J: .. l 'STEP' I 'l!NTIL' N 'DO'
'IF' COLLCJl"lOOOO+K 'THEN' 'BEGIN'
NEWLINE( I ) ; .tISI"NS+ I;
101RITETEXT('('OFIX')');
PRINTCK.3.0)J WRITETEXT('('SIATI')')J
PRINT(TCM+2.J)+TCM+3.K).5.2)J
SPACE( lO)1 PRINT(TCM+ l.J). 5. 2); 'END';

'IY' NS .. 0 'THEN' .... RITETEXT( '( 'IINONE') I);

END OF REFORT: 'END';

CHAPTER XI

Duality

11.1 Block-pivoting with inequalities

Consider a partitioned system of inequalities:

All ~l + A12 ~2 ~ ~l
(11.1.1)
A2l ~l + A22 ~2 ~ ~2

where All is the block-pivot.

We know from Section 5.5 that the recursive product of a

series of pivots equals, with possibly an adjustment of the
sign, the determinant of a matrix.

Therefore, we may assume that a Simplex solution of an L.P.

problem can be expressed (after re-ordering if necessary), by
the partitioned system (11.1.1), because non-zero pivots have
been found, with All being square and non-singular.

The first block of restrictions in (11.1.1) is exactly binding,

~ is zero-valued, and ~l is of the same order as ~l. Also,
~l is related to ~l by the inverse of All.
-1
~l = All ~l (11.1.2)

We may write (11.1.1) as a system of equations by adding

slack-variables, ~l and ~2 are the two sub-vectors of slack-
variables. We also include the target-row, and we obtain

All xl + A12 ~2 + ~l ~l )
)
+ ~2 ~l )
~2~ + A12 ~2
(8.1.1)
)
-w'
-1 ~l
- w'
-2 ~2 + T 0 )
We now apply the block-pivoting technique from section 3.11.

223
224 CHAPTER XI

Tableau ll.la

Symbolic set-up tableau In extensive form

x s s T Value
-1 -2 -1 -2 I

@ A12 I
E.l

A2l A22 I E.2

-WI -WI 1
-1 -2
I
Pre-multiply the pivotal block-row by the inverse of the block-
pivot.

The pivotal block-row now becomes:

~l ~2 Value
~l

-1 -1
All All E.l
I I
Add the pivotal block-row, pre-multiplied by minus the rest
of the pivotal block-column, to the remaining block-rows, and
the transformed tableau is written as:

Tableau 11.lb

Sybolic updated tableau in extensive form

~l T Value
- - -

-1 -1 -1
I All A12 All All E.l

-1 -1 -1
A22-A2lAll A12 -A 21 A11 I E.2- A2l All E.l

I IA -1 -1 -1
-~2 +~l 11 A12 ~iA11 1 ~iA11 E.l
DUALITY 225

We may re-order the tableau, according to the "shortened"

version of the tableau, and suppress the unit matrices, writing
the names of the (block-) rows at the side of the tableau
instead.

Therefore, the initial tableau is written as:

Tableau ll.lc

Symbolic set~up tableau in shortened form

Name ~l x < Value

-2 -

~l Q A12 E.l

~2 A2l A22 E.2

T -w' -w' 0
-1 -2

The updated tableau associated with the block-pivot All then

becomes:

fableau ll.ld
Symbolic updated cableau ~n shortened form

Name ~l ~2 -< Value

-1 -1 -1
~l All All Al2 All E.l

-1 -1 -1
~2 -A 2l A ll An -A2l All Al2 E.2- A2l All E.l

-1 I , -1 -1
T ~iAll -~l +~2All Al2 ~lAll E.l
I

Apparently, the updatingrules for a shortened tableau, can also

be extended to block-pivoting.

Recall the rules for updating a simplex tableau, when the

shortened version of the tableau is used:
226 CHAPTER XI

Replace the pivotal element by its reciprocal and interchange

the names of the pivot-column and the pivot-row.

Multiply the rest of the pivot-row by the reciprocal of the

pivot.

Add the updated pivot-row, multiplied by minus the corresponding

element in the pivot-column, to the other rows.

Multiply the remainder of the pivot-column by minus the

reciprocal of the pivot.

The corresponding rules for block-pivoting are:

Replace the block-pivot by its inverse and interchange the names

of the pivotal block-column and pivotal block-row.

This part of the (block) pivoting operation, starting again with

the original tableau:

Name ~l I ~2 -
< Value

~l All Al2 E.l

~2 A2l A22 E.2

-w' -w' o
-1 -2

is completed for the block-pivot itself, by writing its inverse

and replacing the names

Name ~l

~l

The equivalent of this part of the block-pivoting operation

in ordinary elementwise pivoting is the replacement of the
pivot by its reciprocal. Pre-multiply the rest of the pivotal
row by the pivot-inverse. The updated pivotal block-row
becomes:
DUALITY 227

Name ~l ~2 I Value I
-1 -1 -1
~l An An A12 An E.l
<

The equivalent part-operation of this part of the block-

pivoting operation in ordinary simplex-operations is the division
of all the other elements of the pivot-row by the pivot-element.

Add to the non-pivotal blocks of the tableau, the updated

pivotal block-row, ..E,!!:.-multiplied by minus the corresponding
block in the pivotal block-column, i.e. -A2l and ~i

Name I ~2
Value

-1 -1
~2 A22-A2lAn A12 E.2- A2l An E.l

'+ 'A -lA -1

T
-~2 ~l n 12 ~iAn E.l

Post-multiply the remainder of the pivotal block-column by the

pivot-inverse, i.e. post-multiply A2l by -All-I. We now write
the updated tableau, which is now fully up-dated:

Name I ~l ~2 Value
<-
-I -1 -1
~l An An Al2 An E.l

-1 -1 -1
~2 -A2lAn A22 -A2lAn A12 E.2- A21 An E.l

-1 , 'A -lA -1
T ~iAn -~2+~1 n 12 ~i An E.l

That these rules do indeed give us the correct updated tableau

is verified by recalling section 8.7 where the equivalent
explicitly written tableau (with unit matrices) was obtained by
means of substitution.
228 CHAPTER XI

This includes the special rule for updating the pivot-column,

which we first formulated in Section 8.8. This rule is
generalized to block-pivoting as well, in the form of post-
multiplication of the pivotal block-column, by the inverse of
the block-pivot. The reason why this rule is applicable is the
same as in the case of an individual pivot-column. The unit-
matrix which links the ~l block-row to the ~l block-column in
the explicitly written tableau serves as an operator.

11.2 The Duality Theorem

The analogy between the updating of (bloc~ pivot-rows and

(block) pivot-columns in a simplex tableau (shortened version),
suggests a close analogy between the structure of a simplex
tableau and its transpose. There is indeed a "transpose" of a
linear programming problem. That "transpose" problem is not
1n terms of x and s.

To give significance to the transpose of any current simplex

tableau, we introduce a new set of variables. We write u for
the vector of specified variables, and d for the vector of
slacks, and ~ for the objective function. The transpose of a
current simplex tableau may then be written and interpreted
as:

Transposed updated tableau

Name ~l ~2
< Value
-

~l [A' rl
11
r l A , 21
- [A' 11 [Ailrl~l

i2 A' l2[A' 11J-l A' 22- A ' 12 [A' 11rlA' 21 -~2+Ai2[AilJ-l~1

-1
~\[A'11] b' -b' [A'
- 2 - 1 11
1- l A' 21 ~ , 1 [A'r
11 ~l
l

I
~

The corresponding "original" tableau is retraced by applying the

same block-pivoting rules to [A'lJ -1 as block-pivot. That
corresponding "original tableau 1S found to be:
DUALITY 229

Backward traced transposed tableau

Name <
~l ~2 --
Value
, -A 2l
,
~l A11 ~l

-A 12
, ,
~2 A22 -~2

-b
, , 0
jJ
-1 ~2

This is a cur~ous tableau, in particular because the partitioning

changes with each step.

More meaningful than the simple transpose of the current solution

~s a very much similar problem, where the signs have been adapted.
This adaptation of the signs ensures that the "original" problem
is independent from the partitioning. A further requirement to
be satisfied b) this sign-adaptation is preservation of the signs
of the value column and objective function row. That way, both
the original L.P. problem and the related "transposed" problem
are optimal and feasible for the same vertex. These desiderata
are met by the dual problem, which is written in a simplex
tableau, as follows:

Dual set-up tableau (shortened form)

Name ~l ~2 -< Value

, ,
~l I -A 11 -A 2l -w
-1

-A 12
, -A 22
, -w
~2 -2

, , 0
jJ ~l ~l
230 CHAPTER XI

The corresponding updated tableau for this dual problem is:

Updated dual tableau 1n symbolic form

Name ~l 2:2 <

-
Value

2:1 -[A11 . l'r fA 11 'r l A2l , [All'rl~l

~2 -A 12 ,[All 'r l -A 22 '+A12 ' [All'flA2l -~2+A12' [All '1-l~

1

]J b '[A 11
-1 'J- l .!:2 'b l ' [All 'rlA2l' -.!:l' [All 'rl~l

This is the transpose of the current updated tableau with all

the signs in the tableau itself changed round, while the two
vectors have the same sign. The sign of the solution value is
again inverted. Accordingly the linear programming problems

Maximise and Maximise

T = w'x ]J = -b'u

Subject to

A x < b (~~ 0) -A'u < -w (2: ~ 0)

are named the primal (or original) and the dual problem, and are
optimal and feasible for the same vertex.

This formulation of the Duality Theorern is somewhat 'different

from whAt is conventional in the literature. See, for example
Garvin [11J, p.248, or Baumol L2J , p.l04. The more conventional
form of the Duality Theorem states the equivalence of the L.P.
problems:
DUALITY 231

Maximise and Minimise

T = W'X A = b'u
Subject to Subject to

A x < b (~~ 0) A' u > w (~~ 0)

This difference in formulation arises, because minimization

problems and restrictions of the> type are not directly dealt
with in this book. The two formul;tions are equivalent, except
for the value of the objective function. In the conventional
formulation of the Duality Theorem the original "primal"
problem and the dual problem have the same solution value. In
the form in which the Duality Theorem is presented here, primal
and dual problems (both being maximization problems), have
solution values of the same absolute value, but of opposite
sign.

Example

Consider the following "primal" problem

Maximise

Subject to

< 15

(xl ~ 0, X z~ 0)

The Duality Theorem tells us that we may also find the optimal
values of xl and xz as the shadow prices of the binding
restrictions in the following (dual) problem:

Minimise 5 u l + 15 Uz - u3

Subject to ul + Uz - u3 > 3

2 ul + u 3 > -1

(u l > 0, u 2 ~ 0, u 3 -> 0)
232 CHAPTER XI

or, equivalently

Maximis.e A = - 5 ul - 15 u 2 + u 3

Subject to - ul - u2 + u3 < - 3

- 2 ul - u3 < 1

(u l ~ 0, u 2 > 0, u 3 ~ 0)

Not only can we "dualize" the initial and the final optimal
and feasible tableau, we can do that with each simplex tableau.

The Simplex tableaux contained in summary tableau 11.2a,

illustrate an example where the Simplex Algorithm, applied to
the primal problem, leads to the optimum in one step. The
summary gives the optima of both problems, ordered on the
assumption that the corresponding step is also made in the
dual problem.

TABL EAU 11.2 A

SUMMARY OF A PRIMAL AND A DUAL PROBLEM.

PRIMAL SET-UP TABLEAU DUAL SET-UP TABLEAU

NAME I X I X 2 II VALUE NAME I U I U 2 ll' 3 I I VALUE

S I
S 2
<D
2
2 I I
I I
S
IS
DI
D 2
Ie::!)
I -2
-2 I
-I
II
II
-3
I
S 3 -I II -I
L 5 15 -I II
T -3 II

PRIMAL OPTIMUM DUAL OPTIMUM

NAME I S I X 2 II VALUE NAME I DIU 2 U 3 II VALUE

X I I 2 ! ! 5 U I -I 2 -I II 3
S 2 -2 -4 II 5 D 2 -2 4 -3 II 7
S 3 I 3 II 4
L 5 5 II -15
T 3 7 I I 15

The dual problem was solved in this case, by choosing a pivot

indicated by the primal tableau. In this example that was also
a "reasonable" pivot in the context of the dual problem when
seen as an L.P. problem in its own right.
DUALITY 233

This is not always so, as may be seen when solving both problems
by the steps which arise from applying the rule of the steepest
ascent to the dual - That rule, applied to the dl-row of the
dual problem activates the u2 column. The resulting steps are
shown in the summary tableau 11.2b.

SL'MMAPY 11.2 B

THE SAME PFfOBLFM. THE OPTIMA NOW REACHEO BY APPLYING

THt RULE OF THE STEEPEST ASCENT TO THE r:UAL, MAKING
CDRRESPDrlDING STEPS IN THE PRIMAL PROBLEM.

PRIMAL SET-UP TABLEAU DUAL SET-UP TABLEAU

NAME! X 1 X 2 !! VALl'E NAM E U 1 L' 2 L' 3 !! VALVE

S 1 2 !! 5 o 1 !-1 1 !! -3
S 2
S 3
®
-I
!!
!!
15
- 1
D 2 !-2 -I !! I

L 5 IS -I !I
T !-3 ! !

PRIMAL EQtJI VALENT TABLEAU DUAL INTERMEDIATE TABLEAU

NAME! S 2 x 2 !! VALVE NAt1 E! U 1 0 I V 3 !! VAL liE

~-;--~~:~--;---~~-:;:~~
---------------------------------
L' 2 ! ~-0.50 -0.50 !I 1.50
XI! .50 - !! 7.50 o 2 ! -2 -I! ! I
S 3 0.50 1 !! 6.50
L ! -2.50 7.50 6.50! !-22.50
T 1.50 !! 22.50

?RIMAL OPTIMVM DVAL OPTIMUM

flAME! S i X 2 !! VALUE NAM E! U 2 0 I V 3 !! VAL VE
--------------------------
S 2 -2 4 ! ! 5 V I 2 -I -I !! 3
X 1 1 2 ! I 5 D 2 4 -2 -3 !1 7
S 3 1 3 ! ! 4
T 5 5 4 ! ! -I 5
T 3 7 !! 15

The second of these two steps is logical in terms of both the

primal and the dual problem, but the first does not make much
sense as a step in the primal problem at all. In particular,
the rule of the smallest quotient is violated and the sl -
restriction in the first updated tableau in primal form.

This corresponds to a loss of optimal form of the dual problem,

i.e. the u l entry in the A-row was originally positive and
became negative.

11.3 Application oP the Duality Theorem

Some authors (e.g. Kim [23], p.27l) mention the possibility

to substitute a dual problem with more variables
 234 CHAPTER XI

than restrictions for a primal problem with more restrictions

than variables as a cumputational advantage on account of the
size of the tableau. The argument is that the use of an
explicit simplex tableau requires a unit matrix, of which the
order is given by the number of restrictions. However, if the
shortened version of the tableau is used, this problem
dnes not arise. In other words this claim arises, because
an inefficient algorithm is used in the first place. The
Duality Theorem has played an important role in development*
of the theory of linear programming. However, many of the
currently useful applications of the concept of duality are
valid and indeed more useful in the wider realm of convex
programming generally.
This includes the important subject of the interpretation of
dual variables as imputed prices.

Computational efficiency is, however, relevant with respect

to the number of steps. This is so, in particular when one
deals with problems which are effectively minimization problems
(See also Dantzig rBJ , Section 11.2). The term minimization
problem here means that a non-negative combination of (non-
negative) specified variables is to be minimized. Obviously
that minimum can never be less than zero. That solution is
attained already in the trivial solution, but this is not a
feasible solution. In economic applications this problem
usually arises as one of cost minimization, and costs will have
to be incurred because certain requirements (restrictions) have
to be satisfied. Solution of the problem by its dual maintains
optimality, while application of the normal sequence of Phase I
and Phase II to the original problem does not maintain the
initial optimality.

"Phase I" methods are also generally less efficient than the
elementary optimizing algorithm.

It is not even necessary to actually write dual tableaux. The

dual Simplex method performs the dual search operations on
tableaux written in their "normal" form.

The following problem illustrates the point

Minimise xl + x 2 + x3

Subject to 2x l + x2 < 3)
) (xl' x 2 ' x~ ~ 0)
2x2 + x3 < 8 )

*See for example Tucker [34)

DUALITY 235

Formulated as a primal maximization problem, this problem

becomes:

Maximise

Subject to

The dual problem

MaximisE' A = 3 ul + 8 U z
Subject to Z ul < 1 )
) (u l ' U z > 0)
ul + Z U z< 1 )
)
U z < 1 )

is optimal ~n one step (see tableau 11.3 a ).

TABLEAU 11.3 A

A SMALL PRIMAL FEASIBLE PROBLEM, IT IS OPTIMAL IN ONE STEP.

NAME I lC 1 U 2 !! VALUE NAME U 1 D 2 II VALUE

1> 1 2 I! D 1 2 I! 1
02
D3
1 ®1 I!
II
U2
1) 3
0.50 0.50
-0.50 -0.50
II
II
0.50
0.50

T 1-3 -8 I! T 4 II 4

In this example, Xl = 0, x 2 = 4 and x3 = 0 is the optimal and

feasible primal solution. But to atta~n this solution, by
applying the rules of the steepest ascent and the smallest
quotient to the initially infeasible primal problem requires
two steps rather than one. The two steps are summarized in
tab leau 11. 3b.
236 CHAPTER XI

TAELEAU 11.3 B

ILLUSTRATION Of LOSS
Of OPTIMAL fOP.M.

NAME X I X 2 X 3 II VALUE

S I I-~ 8) II -3
S 2 -2 -I II -8

T II

NAME x S X 3 II VALUE

X 2 2 -I II 3
S 2 II @) - I II -2

T -·1 I I -3

NAME x S 2 X 3 II VALUE

X 2 I -0.50 0.50 II II
S I 1-2 -0.50 0.50 II I

T 0.50 0.50 II -4

The two systematic factors in the relatively inefficient

performance of the Simplex Algorithm on an initially infeasible
but optimal (dual feasible) problem are:

a) Loss of optimal form. The shadowprices of xl was initially

positive, but became negative during the simplex operations.

b) The use of the rule of the smallest quotient in a context

where it is not really appropriate. If the "efficient" rule
for Lhe choice of the pivot-row had been taken, we would have
"flown" through the sl restriction and reached the solution
in one step. But, as discussed in section 9.2, the
"efficient" rule for the choice of the pivot-row has certain
definite drawbacks. The possibility to interchange the
concepts of "optimal" and "dual feasible" has its
computational advantages after all.

The following suggested modification of the rule for choosing

the pivot column avoids unnecessary reductions in the value of
DUALITY 237

the specified objective function. When there is no feasible

solution, we may first of all give preference to columns which
promise an increase in a substitute objection function, e.g.
the sum of all slacks of violated restrictions, as well as
in the value of the specified objective function. When no such
columns are available the rule of the steepest ascent may be
replaced by the criterion of the dual ratio. We will take the
column which promises the greatest increase in the value of
the substitute objective function per unit of reduction in the
value of the specified objection function. If the substitute
objection function is the slack of just one violated restriction,
this rule can amount to applying the rule of the smallest
quotient to the dual. That quotient then is the reduction in
the value of the specified objective function, per unit of
increase of the substitute objective function. This full
analogy with the rule of the smallest quotient applied to
the dual problem arises whenever the restriction which provides
the substitute obj£ctive function becomes binding in one step.
This is the case in the illustration-example contained in
tableau l1.3c.

TABLEAU 11.3 C

ILLUSTRATION OF THE DUAL RATIO.

NAMEI X I X 2 X 3 I VALI:E NAME! U I U 2 IVALUE RATIO

~_~_~_:~ ____~____ :~ ____ l:~~ __ o I

D. 2
D 3 ,
I -I
I ~
5

3
, 8
I
, 2
I
1.60
0.50
0.67
T I B I 2
RATIO -1.60 -O·.SO -0.67 T , -10 ,-
NAMEI X I S 2 X 3 I VALUE NAME' U D 2 I VALUE

S I! I , D I I -I -2.50 5.50
X 2! 2.50 -0.50 1.50 I 5 U 2 I 0.50 I 0.50
031 -1.50 I 0.50
T! 5.50 0.50 0.50'-5
T I 5 , 5

11.4 The Dual Ratio and Phase I Column Selection in the

presence of several violated restrictions

If there are no zeros in the updated form of the objective

function row, and no pseudo-zeros either, the dual ratio is
also a useful additional criterion for selecting columns during
238 CHAPTER XI

Phase I of the primal problem. It is here assumed that this is

in addition to the "efficient" rule of row-selection, and the
rule of the highest step as column-selection criterion. This
additional criterion is useful fot two reasons. Firstly the
dual ratio is the ratio between the loss in specified objective
function value and the corresponding gain in (a facet of) the
substitute objective function i.e. the value of the slack
variable in question. Secondly, simply because the denominator
is the tentative pivot, a ratio criterion has a bias against
small pivots; as is illustrated in tableau l1.4a

TAELEAU 11.4 A

AVO I DANCE OF SMALL PI VOTS

BY THE USE OF THE DUAL RATIO.

NAME! X 1 X 2 ! I VALUE

S 1 I -0.01@ !! -1
S 2 I 0.90 !! -I

T ! !

We may think of this tableau as being an extract or,

alternatively, it represents an empty problem, which should
not be mishandled either. The rule of the highest step, applied
to the sum of the infeasibilities as substitute objective
function, indicates the sl/xl cell as pivot, with a figure of
-0.01.

Having tentatively indicated this as a possible pivot, we now

investigate other columns, while imposing the following
modifications on both the row and column-selection rules

a. A violated restriction which is already tentatively

indicated as pivotal row, is "badname" and flying through
it is allowed only with an incoming variable which also
increases the specified objective function. (If a "preferred"
column is actually selected in this way there is no longer a
"badname".) In potential incoming variables, columns which
increase the substitute objective function at the cost of a
reduction in the value of the specified objective function,
eligible pivotal rows are:

equations
already non-negative-valued variables
the "badname" negative slack-variable
DUALITY 239

other negative slack-variables if the restriction cannot

be reached by that incoming variable - the "efficient"
rule of row-selection being again applicable -. (In
preferred columns which indicate an increase in the
specified and the substitute objective function, negative
pivots areonly eligible in rows representing equatiuns.)

b. Once we establish the fact that a pivot in such a "badname"

row is permissable, the dual ratio criterion has priority
over the rule of the highest step. Hence the SI/x2 entry
becomes pivot instead of the SI/xI cell irrespective of the
fact that the rule of the highest step would favour the
sl/xI cell.

That situation may change if the "badname" status disappears

for one of the following reasons:

hl. A "preferred" column is activated.

b2. The "efficient" rule of row-selection, applied to a

column in which no negative entry for the "badname"
restriction occurs, indicates a different violated
restriction as the pivotal row and the rule of the
highest step (applied to the substitute objective
function) indicates that the new pivot is preferable
to the old one.

The following implication of rule a) is stated here separately,

mainly in order to discuss its significance.

c. Columns which have been identified as contradicting the

dual ratio between the objective function and the badname-
row become ineligible as incoming variable columns,
notwithstanding the presence of other possible pivots in
such columns.

The rationale of this last point may be illustrated by the

example given In tableau 11.4b below

T A8L EAU I I .4 8

X2 IS NOT A SUITABLE COLUMN.

NAME I X X 2 II VALUE

S I I-I -0.20 I! -I
5 2 ! -2 -0.01 1\ -I

T II
240 CHAPTER XI

We assume that the sl/xl cell has been tentatively selected as

pivot, at the stage of scanning the xl column.

The entry of -0.20 in the sl/xZ-cell contradicts the criterion

of the dual ratio between the updated form of the objective
function row and the sl-row. However, throwing only th~t one
pivot in the x2-column out, does not put this problem right;
the -0.01 entry in the s2/x2 cell is no better pivot. Yet if
we use the sum of the two negative slack-variables as substitute
objective function and then apply the rule of the highest step,
we find that the s2/x2 cell comes out with sl+s2 = 99 + 0 and
is the highest. The only reasonable practical way of avoiding
the kind of step is to disqualify (as far as this vertex is
concerned) x2 as incoming variable altogether.

Exercise

Complete the problem tackled in section 9.4.

11.5 Dual Degeneracy

The significance of the term dual degeneracy will be obvious:

zeros in the objective function row. If there are a whole
series of exact zeros in the objective function row, the
criterion of the dual ratio ceases to be an effective criterion
of choice between one column and another. Yet dual degeneracy
occurs quite often, if anything it is more rather than less
common than primal degeneracy.

The criterion of the dual ratio may, however happen to hit a

zero in the objective function row and indicate a small pivot.

Example

Tableau 11. Sa

Dual degeneracy: it may lead to a pseudo-zero pivot

Name xl x2 Value

sl -1 -0.00 -1

s2 1 2 2

1 0

Assuming that the -0.00 entry is not actually a zero, the rules
indicated in the previous section would lead to the selection
of the sl/x 2 entry as pivot. Obviously something should be done
DUALITY 241

to avoid that.

As far as selection between several columns with zero entries

in the objective function row is concerned, perturbation of
the objective function row on similar lines as indicated in
Section 8.10 for primal degeneracy is likely to be an effective
remedy.

The above example indicates that this needs to be supplemented

by a direct restriction on the selection of a small pivot.

The way in which we introduced the criterion of the dual ratio

only as a supplementary criterion which is considered when a
pivot has already been tentatively selected makes this possible.
We refuse to change from one negative Phase I pivot to another
similar one in the same row, if the latter is an order of
magnitude smaller.

The code offered in section 12.3 puts this requirement as the

ratio between the two pivots not being less than 0.1.
CHAPTER XII

LINEAR PROGRAMMING ON THE COMPUTER

12.1 Name-codes and name1ists

We need some means to relate rows and columns in Simplex

tableaux to their significance in terms of equations, variables
etc.

There are two main reasons for this requirement. The user needs
to know the significance of the end result, and the machine
must be programmed to distinguish equations from inequalities,
non-negative variables from variables without sign restriction,
distances from upper bounds from "ordinary" variables etc.

Reference to alphanumeric names like x2' s4' u etc is not very

practical in a programmed computer procedure. ~he use of an
integer number as a namecode is much more practical, at least
for purposes of internal machine-use~output of results is a
different matter.

There clearly are two lists of names in any L.P. problem. They
are the list of the basic variables which are associated with
the rows of the tableau, and the list of non-negativity
restrictions which are associated with the columns of the
tableau. In an explicit tableau the columns include unit vectors
for basic variables, in a shortened tableau the list of co1umn-
names is the list of binding non-negativity restrictions.
Tableau 12.1a below gives the usual presentation of the optimum
of an example from Chapter 8, tableau 12.1b gives a completely
numerical presentation of the same optimal tableau.

TABLEAU 12.1 A
THE USUAL ALPHANUI1EP.I CAL PRES EN-
TATION OF A SHORTENED TABLEEAU.

NAME I X I S I S 2 VALUE
------------------------------------~
X 2 0.33 0.33! 33.33
X 3 0.67 -0.33! 166.67
-------------------------------------
T 0.33 1.33 333.33

TABLEAU 12.1 B'

A NUMERICALLY COOED TABLEAU.

1001 1002
---------------------------------~---
2 I 0.33 0.33 I 33.33
3 I 0.67 -0.33 I 166.67
-------------------------------------
1.33 333.33

242
LINEAR PROGRAMMING ON THE COMPUTER 243

Besides the obvious method of distinguishing individual

elements of vectors by their indices, we need some other
device to distinguish between classes of variables, i.e. the
vectors to which they belong. The two most suitable devices
appear to be the plus or minus sign, and the enlargement. In
the above example an enlargement of 1000 has been used to
indicate row-names i.e. slack-variables. Problems with more
than 1000 variables will normally have a partitional structure
and a special algorithm will be used in any case. Special
algorithms for partitioned problems will obviously have their
own system of name-codes where the name-codes refer to the
partitioning. An alternative method of coding, might be the
one used in tableau l2.lc, using positive and negative numbers.

TABLEAU 12.1 C

USE OF THE -SIGN AS QUALIFIER.

-I -2
2 ! 0.33 0.33 I 33 • .33
3 I 0.67 -0.33 I 166.67

0.33 1.33 I 333.33

Above, the namelists have been written as an extra row and

column of the tableau. There are, however, some fairly
convincing reasons for declaring namelists as separate arrays
in their own right. (Or dimensions, the Fortran Programmer
would say.) First of all, these separate arrays can be
indicated with names which convey their significance to the
human reader of the programme text. Thus, an instruction which
is preceded by the condition.

'IF' ROWLST [IJ >NAV 'THEN'

clearly refers to elements of the list of rownames.

There are also more strictly computational arguments. Name-

codes are integer variables and some machines use only one
address for an integer variable and two for a "real!' i. e.
possibly fractional valued variable. Also, some computer
languages e.g. Fortran cannot accommodate rows wtth a zero or
negative row-index or columns with a zero or negative column-
index. Several vectors, e.g. value column, upper bounds vector
etc. also have to be fitted in with the tableau, and if they
all have to be allocated an index in excess of the normal size
of the tableau, it becomes somewhat confusing. No useful
244 CHAPTER XII

purpose is served with reserving space for the namelists as

well in the tableau itself.

12.2 Ordering of the tableau

The choice of pivots on the indication of the numerical content

of the tableau usually results in a random ordering of the
tableau. If the "original" simplex method is used only the
rows lose ordering, the columns stay in place. If the
shortened version of the tableau is used, the columns also get
into a random non-ordering.

Re-ordering can be done on the indication of the numerical

values of namecodes, i.e. the inappropriately ordered tableau
l2.2a (below) may be re-ordered on the indication of the name-
codes, to become tableau l2.lb.

TABLEAU 12.2 A

A 'DIS-ORDERED' TABLEAU
1002 100 I
3 I -0.33 0.67 166.67
2! 0.33 0.33 33.33
1.33 0.33 333.33

In order to be able to (re) order in a meaningful way it is

necessary to impose certain conventions for ordered tableaux
and namelists. I suggest the following rules:

(a) Positive names to be placed before negative names

(b) Names with low absolute value to be placed before

names with higher absolute value.

This ensures ordering according to ·index, and places names with

enlargements at the end of the list. In fact, no use of
negative name-codes is made in this chapter, and although
negative name-codes are used in Chapter 16, with quadratic
programming, a different solution for the ordering problem
is used there. The following programmed procedure which
incorporates these rules is however offered here.
LINEAR PROGRAMMING ON THE COMPUTER 245

'PROCEDVP.E' ORDRCT.M.N.ER.RH.ROVLST.COLLST);
'ARRAy'r; 'I.NTEGER' M.N.ER.P.H; 'INTEGER' 'ARRAY' R01Jl..ST.COLLST;
'EEGIN' 'INTEGER' F"lRST.LAST.I.J.NAME; 'REAL' NUM1
'BOOLEAN' FINI SHED;
'COMMENT'
ORDERING PROCEDURE FOR A TABLEAU AS USED FOR MOST SIMPLEX
ALGORITHMS FOR MATHEMATICAL PROGRAMMING.
THE ROVS ARE ORDERED ACCORDING TO THE NAMES IN THE RO~~IST.
AND THE COLULMNS ACCORDING TO THE NAMES IN THE COLUMN-LIST.
POSITIVE NAMES ARE PLACED BEFORE NEGATIVE NAMES.
VITHIN EACH GROUP OF NAMES VITH HOMOGENEOUS SIGNS.
ROWS. AS WELL AS COLt~NS WITH NAMES OF LOV ABSOLUTE VALUE
ARE PLACED BEFORE THOSE WITH NAMES OF HIGHER ABSOLUTE VALUE.

THE TABLEAU IS ASSt~ED TO CONTAIN M PROPER ROVS. I.E. RES-

TRICTIONS. N PROPER COLUMNS. I.E. VARIABLES. AND EXTRA
COLUMNS ON THE RIGHT OP LEFT-HANDSIDE. E.G. VALUE COLUMN.
AND EXTRA ROVS ON THE TOP OF. BOTTOM. 1. E. TARGET-ROW.
SUBSTITUTE oBJECTI VE FUNCTI ON. UPPER BOUNDS ROW. ETC.
THE NUMBER OF EXTRA CoLt~NS IS INDICATED BY THE NlmBER RH.
I F THE EXTRA COLUMNS ARE ON THE RI GHTHAND SI DE. RH IS
POSITIVE. A NEGATIVE VALUE OF RH INDICATES EXTRA COLUMNS
ON THE LEFTHAND-SI DE.
THEIR Nt~EER IS THE ABSOLUTE VALUE OF THE PARAMETER RH.
THE PARI'.METER ER INDICATES THE Nl~BER OF EXTRA RO~·S.
IF ER IS FOSITIVE THEY ARE AT THE END. IF ER IS NEGATIVE.
THEY ARE BEFORE THE TAELEAU ITSELF. I.E. AT THE TOP;

START: F"lNISHED := 'TRUE';

FIRST:=); LAST:=N;
'IF' RH > 0 'THEN' LAST:=LAST+F.H 'ELSE' FIRST:=F"lRST+P.H;

'FOR' 1:=2 'STEP' ) 'UNTIL' M 'DO' 'BEGIN'

'IF' RO~~ST[lJ < RO~'LSTCI-)J 'AND' RoVLSTrIJ > 0
'THEN' 'GoTO' REARRANGE RO~'S;
'IF' ROVLSTCIJ > RO~'l.STCI-)J 'AND' Rot.'l.STCI-)J < 0
THEN' 'GO TO , REARRANGE ROVS;
'GOTO' END OF ROW ORDERING LOOP;
REARP.ANGE ROVS: F"lNI SHED := 'FALSE';
'FOR' J:=FlRST 'STEP' ) 'UNTIL' LAST 'DO' 'EEGIN'
NUM:=TCI-J.JH TCI-J.JJ:=TCI.J]; T[I.JJ:=Nl~; 'END'l
NAME: =Ro~'l.STC 1- J]; R01o:LST[ I -) J: = R01ol1. ST[ I H ROVLST[ I J: "NAME;
END OF ROW ORDERING LOOP: 'END';
FIRST:"); LASTI=Ml
'IF' ER > 0 'THEN' LAST:"'LAST+ER 'ELSE' FlP.STI=FlRST+ER;
246 CHAPTER XII

'FOR' J:=2 'STEP' I 'UNTIL' N 'DO' 'BEGIN'

'IF' COLLST[JJ < COLLST[J-IJ 'AND' COLLST[JJ > 0
'THEN' 'GOTO' REARRANGE COLUMNS;
'IF' COLLST[JJ > COLLST[J-IJ 'AND' COLLST[J-IJ < 0
'THEN' 'GOTO' REARRANGE COLUMNS;
'GOTO' END OF COLUMN ORDERING LOOPl
REARRANGE COLUMNS: FINISHED:= 'FALSE';
'FOR' I:=FIRST 'STEP' I 'UNTIL' LAST 'DO' 'BEGIN'
NUM:=T[l.J-IJJ T[1.J-J):=T[I.JJJ T[I.J)lcNUM; 'END';
NAM EI c COLLST[ J- J JJ COLLST[ J - J ) I = COLLST[ J)J COLLSTC J) I "'NAMEl
END OF COLUMN ORDERING LOOP: 'END';

'IF' 'NOT' FINISHED 'THEN' 'GOTO' STARn

END OF ORDR: 'END';

If the tableau is at all large, the ordering method which is

used by the procedure ORDR is not very efficient.

The procedure interchanges only adjoining names and their

associated vectors. Whether the new positions are correctly
ordered is then tested in the next ordering loop. The obvious
implication of this approach is that large numbers of
permutations may be made which do not so far brin8 any name
in its correct eventual position.

Considering the following 4-element list:

1002, 1001, 4, 3.

The eventual ordering of this list is 3, 4, 1001, 1002, in

the following stages:

1002, 1001, 4, 3 interchange 1002 and 1001

1001, 1002, 4, 3 interchange 1002 and 4
1001, 4, 1002, 3 interchange 1002* and 3

1001, 4, 3, 1002 interchange 1001 and 4

4, 1001, 3, 1002 interchange 1001* and 3
4, 3, 1001, 1002 interchange 4* and 3*

3, 4, 1001, 1002, final ordering

Permutations which bring a name in its actually correct

position have been marked with an asterisk after the namecode
in question.

The ordering of a large list may involve many permutations

which don't bring any name in the position of its final
orderin8'
LINEAR PROGRAMMING ON THE COMPUTER 247

For large lists this problem tends to be aggravated, the

"ineffective" permutations becoming much more numerous, even
relative to the "effective" ones.

As far as the logic of the ordering problem is concerned, this

is more or less unavoidable, because the final "correct"
ordering is not initially known.

What is, however, possible, is to delay re-ordering of the

tableau-matrix until the ordering of the namelists is known.
It is after all quite a bit cheaper to interchange 2 name-codes
than to interchange 2 rows of a large tableau:

ORDR can in fact be used to order only a namelist. One just

lists one of the two-order parameters as b2ing zero, i.e.

ORDR (M, 0, 0, 0, ROWLIST, COLLST)

only accesses the first zero columns of the tableau-matrix,

and the first zero elements of the list of column-names. This
effectively means re-ordering the list of row-names. The
tableau matrix itself, and the list of column names is not
accessed.

The following procedure makes use of this device to order the

tableau in two stages:

'PROCEDURE' ORDL(T,M,N,ER,P~,RO~~,COLL);
'ARRAY' T; 'INTEGER' M,N,ER,RH; 'INTEGER' 'ARP.AY' ROVI..,COLL:

'BEGIN' 'INTEGER' I,J,R,K; 'REAL' NUMJ

'PEOCEDURE' ORDR(T,M.N,ER,RH,RO~~.COLL)J
'ARRAY' T; 'INTEGEP.' M,N.ER.P~;
'INTEGER' 'ARRAY' RO"'~' COLLI 'ALGOL ':

'INTEGER' 'ARP.AY' DUP RO~'L[l:M], DUP COLL[l:Nl1

'COMMENT'
ORDERING PROCEDURE FUR A LARGE TABLEAU.
THE TABLEAU IS ORDERED IN TWO STAGES.
FIRST THE NAMELISTS ARE COPIED INTO THE DUPLICATE LISTS,
\/HICH ARE THEN ORDERED BY CALLING ORDR, WITH ONE OF THE
ORDEP.-PARAMETERS SET AT ZERO.
THEP.EAFTER, THE TABLEAU ITSELF IS ORDERED, ON INDICATION
OF THE DIFFERENCES BETWEEN THE ORIGINAL AND THE REORDERED
NAMELISTS.
FINALLY, THE REORDERED NAMELISTS ARE COPIED TO THE ORI-
GINAL ONES.
:
248 CHAPTER XII

ORDER R01t.'Ll3Tr
'FOR'· Rr-I 'STEF' I 'UNTIL' M '00' oUP ROI".'LtRll=R01r.'l..tRH
ORoRCT,M,O.O,O,oUP R01r.'I..,COLL)1

ORDER COLl~N LISTr

'FOR' Kr-I 'STEP' I 'UNTIL' N 'DO' OUP COLLtKl:"COLLtKlJ
ORoRCT,O,N,O,O,ROVL,oUP COLL)I

ORDER ROVSr
'FOR' R: .. I 'STEP' I 'l.'NTIL' M-I '00' 'BEGIN'
'FOR' Ir=F.+1 'STEP' I 'UNTIL' M '00'
'IF' OUF R01t.'LtRl=ROloJLtll 'AND' 'NOT' ROWLt!U"DUP R01o'LtRl
'THEN' '8EGIN'
ROVLtIlr=ROYLtRll R01r.'l..tRll-oUP R01r.'l..tR)1
'FOR' J:-I 'STEP' I 'UNTIL' N+PJi ,~O' '8EGIN'
NUMrcTtR,JlJ TtP.,J)I=TtI,JH TtI,Jl:=NUMl 'ENo'l
'GOTO' NEXT RD1';; 'ENo'l
NEXT R01r.'r 'END';

ORDER COLl~NSr
'FOR' KI-I 'STEP' I 'UNTIL' N-I '00' 'BEGIN'
'FOR' J:=K+I 'STEP' 1 'UNTIL' N '00'
'IF' OUF COLLtKl"COLLtJl 'AND' 'NOT' COLLtKl"'oUP COLLtK]
'THEN' '8EGIN'
COLLtJl:cCOLLtKl; COLLtK)r"DUP COLLtKl;
'FOR' 1:=1 'STEP' 1 'UNTIL' M+ER '~O' 'BEGIN'
Nt..~:=TtI,KH TtI.Kl:=TtI,JlJ TtI,Jl:=Nl~1 'END'I
'GOTO' NEXT COLUMNI 'ENo'l
NEXT COLUMN I 'ENo'l

END OF ORoL: 'END';

UNEAR PROGRAMMING ON THE COMPUTER 249

12.3 Commented text of a linear programming procedure

The programmed procedure offered in this section assumes a

mixed system as discussed in Chapter X. It contains two calls
to the ordering procedure discussed in the previous section. Of
these, the call just below the label OUT is strictly for the
purpose of orderly presentation of the end result. The first
call to the ordering procedure, following the label ORDER, is
part of the linear programming algorithm. The reason is that
random ordering may already arise during the preliminary
inversion phase. As may be seen from its listing below there
are two fairly distinct phases in the algorithm. One phase
consists in making the variables without sign-restrictions into
basic variables irrespective of their sign, the other phase
is the linear programming algorithm proper.

The search operation for a pivot row following the label

RETURN IN INVERSION is comparable with the similar search
operation which occurs in ordinary matrix inversion. The phase
of an LP algorithm which deals with the systematic search for
a feasible solution is conventionally indicated as "Phase I".
In this algorithm, "Phase I" is preceded by a phase which
deals with entering "free" variables into the basis. That
phase "enter variables without sign restriction" could
therefore be called "Phase 0".

The search operations which control this phase are sufficiently

different from the ones in the proper LP algorithm to justify
keeping them completely separate. The actual updating of the
tableau, including the administration of the name lists is
however common for all phases. The return to "Phase 0" in
order to find a pivot row to match the next "free" variable
is controlled by the logical variable INVERTED.

Just as in ordinary matrix inversion (compare Sections 3.10

and 3.14), the pivot may be found elsewhere than on the main
diagonal. Or no non-zero pivot may be found at all, if the
rank of the left-hand block column is lower than the number of
free variables. In that case the "unbounded" loop of Phase 0
will be activated. At t,e end of Phase 0, the tableau is
re-ordered. The implication of this arrangement is obviously
that free variables must be placed before bounded variables,
i.e. they are recognisable as the ones with the lowest indices.

The re-ordering will therefore put the rows which refer to

variables without sign restriction at the top of the tableau.
The search operations for a pivot row during the,LP algorithm
proper then run from the row with index NAV+l onwards.
250 CHAPTER XII

The two phases of the normal simplex operations, i.e. Phase I

for finding a feasible solution and Phase II for finding the
optimum are almost completely integrated.

The distinct "Phase I" part of the programme consists of:

* Finding whether a feasible solution exists, and setting

the logical variable FEASIBLE accordingly. Technically,
Phase I is re-entered at every step, but this becomes
trivial, once a feasible solution exists.

* Setting the objective function coefficient for each column.

During Phase I, this is done as follows: In the first
instance, the substitute objective function is the sum of
all infeasibilities (negative stacks). If the current
basis is not feasible, no column qualifies as incoming
variable, unless it promises a positive ascent of this
substitute objective function.

Columns which pass this test are then loaded as "preferred"

or "non-preferred" columns. Preferred columns are those which
promise an incredse in both the specified and the substitute
objective function, and the actually used preference
coefficient is the sum of the two objective functions.

Non-preferred columns are those which promise an increase in

the substitute objective function, but only at the cost of a
loss in specified objection function value. Preference for the
preferred columns is made effective by scaling the preference
coefficient of non-preferred columns down by a factor 1000.

If the current basis is feasible, the objective function

obviously is specified objective function. Control then passes
to the search operations of the optimizing algorithm.

In principle all columns - other than those referring to

equations - are scanned for potential pivots, going through
the tableau from left to right. The following rules apply:

* The rule of the smallest quotient, applied to positive

pivots and negative ones in connection with equations.
This row-selection rule applies to columns which indicate
a positive non-zero ascent in the specified preference
direction (= a negative non-zero entry in the m+lth row
of the tableau).
 LINEAR PROGRAMMING ON THE COMPUTER 251

* The "efficient" rule of row-selection (see section 9.2).

When this rule applies we are dealing with a non-preferred
Phase I column. The acceptable minimum value of a negative
pivot in an inequality-row is 1/3 of the average of all the
negative entries in the column in question which are
associated with violated restrictions (including equations).
(There is also a minimum absolute value of 0.000001 for all
pivots.)

* The rule of the highest step with, for violated restrictions,

modifications as discussed in sections 11.4 and 11.5. When
the current basis is feasible the rule of the highest step
works as a column-selection rule, but in Phase I this is only
true for the choice between different preferred columns.

Once it becomes necessary to find the pivot in a non-preferred

column, the rule of the highest step effectively becomes a row-
selection rule.

Note, however, that the dual ratio condition is enforced only

partially, i.e. going from left to right. If ~ shift to a
different violated restriction actually takes place, this may
well be one for which the dual ratio is in fact violated in
columns further to the left than the one which is currently
being scanned.

The other complicating feature which arises concerns the

upper limits on specified variables and the fact that negative
entries in the tableau "stand in" for positive potential pivots
in relation to upper limits. The procedure follows section 10.3
fairly precisely, but it does in practice complicate the search
operations quite noticeably.

The procedure-text is now listed, as follows:

'PROCEDVRE' LINP(T.M.N.NEQ.NAV.RO~LST.COLLST.REENTRY);
'ARRAY' TJ 'INTEGER' M.N.NEQ.NAV.REENTRY;
'INTEGER' 'ARRAY' PD\lLST. COLLST;

'BEGIN' 'INTEGER' I .J. R. TRYR.K. COLN. RO~N. TRYN.N OF NC;

'REAL' ASC.HIG.QUO.TQUO.PIV.NL~.COF.VNEV.DUAL RATIO.
rs ASC;
'BOOL EAN' INVERTED. FEASI EL E. UPPEREOUND;

'PROCEDURE' ORDL(T.M.N.ER.RH.RO~LST.COLLST);
'ARRAY'T1 'INTEGER' M.N.ER.RH;
'INTEGER' 'ARRAY' RO\ILST.CIJLLST; 'ALGOL';
252 CHAPTER XII

'COMMENT' LINEAR PROGRAMMING PROCEDURE.

M RESTRICTIONS AND N VARIABLES.

THE PROCEDURE ACCOMODATES A MIXED SYSTEM,

WITH UPPER EOUNDS ON ALL NON-NEGATIVE VARIABLES.

THE FIRST NEQ RO~S REFER TO EQUATIONS,

iHE REST TO INEQUALITIES.
EQUATIONS SHOL~D BE PRESENTED ~ITH A NON-POSITIVE CONSTANT,
I F NECESSARY BY CHANGING THE SI GN OF ALL ENTRI ES IN A R01.:.
IF THI SIS NOT DONE, I T ~ILL EE DONE BY THE PROGRAMME,
WITHOUT FURTHER WARNING.

THE FI RST NAV COLLUMNS REFER TO VARI AEL ES FOR lo.'HI eH

THE NON-NEGATIVITY RESTRICTION DOES NOT APPLY, WHILE IT DOES
APPLY TO THE REMAINING N-NAV COLUMNS.

ONE SHOULD PREVIOUSLY DECLARE THE ARRAYS, T, THE TABLEAU,

AS \iELL AS THE INTEGER ARRAYS RO\iLST, THE LI ST OF ROYNAMES,
AND COLLST, THE LIST OF COLUMN-NAMES.
ONE SHOULD SUPPLY THE TABLEAU, EUT NAMES ARE GENERATED
BY LINP.
THE TABLEAU SHOULD BE SUPPLIED IN LESS THAN OF. EQUAL TO FORM,
~I TH THE VALUE COLUMN AS THE LAST COLUMN, INDEX N+ 1,
AND THE TARGET-ROW AS THE LAST RO\', INDEX M+ 1.

THE TABLEAU-DECLARATION SHOL~D RESERVE AN ADDITIONAL N+2 TH

COLUMN FOR THE UPDATED FORM OF UPPER BOUNDS, AND AN M+2 TH
RO\i FOR THE NON-UPDATED FORM OF THE UPPER BOUNDS-VECTOR.
THE USER SHOL~D SUPPLY THE NUMERI CAL CONTENT OF THE UPPER
BOUNDS RO"'-VECTOR, THE UPPER BOUNDS TO BE PUT ON THE
SPECIFIED NON-NEGATIVE VAEIABL.ES.
TO ACCOMODATE LARGE UPPER BOUNDS FOR VARIABLES "'HERE NO
'1EANINGFUL UPPER BOUND I S INTENDED, P. ZERO MAY EE SUPPLI ED
INSTEAD, AND THE FROGRAMME "'ILL SUBSTITUTE A MILLION FOR IT.
VARIAELES 1r:ITHOUT SIGN F.ESTRICTION DO NOT HAVE
UPPER EoUNDS, BUT DUMMY-NUMBERS HAVE TO BE Pl'T FOR IfIiEM.

THERE ARE T"'o NAMELI STS, "'HI CH AF.E FILLED BY THE PROCEDURE.
THE SPECIFIED ECONOMIC VARIABLES HAVE NAME-CODES EQUAL TO
THEIR INDICES, I.E. 1 TO N,
AND THE SLACKS OF THE INEQUALI TI ES HAVE NAME-CODES, EQUAL TO
THEIR INDICES PLUS 1000, I.E. FROM 1001 TO 1000 + M.

ON EXIT, THE LIST OF CoLL'MN NP.MES MAY ALSO CONTAIN ENTRIES

1.:ITH ENLARGEMENTS OF 10000.
A NAME-CODE OF 10000 + J INDICATES THE UPPER BOUND ON THE
J TH VARIABLE';

'IF' REENTRY I 0 'THEN' 'BEGIN'

'COMMENT'
 LINEAR PROGRAMMING ON THE COMPUTER 253

FOR REENTRY"O THE NORMAL LP ALGORITHM IS FOLLOWED.

INCLUDING THE FILLING OF THE NAMELISTS.
oTHER ... ISE THE PROCEDURE EXPECTS AN ALREADY
CPDATED TABLEAU. \.'1 TH NAMELI STS FILLED. "''HI CH NEEEDS
RE-oPTIMIZING AND/OR FINDING A NEW FEASIBLE SOLUTION.
THE REENTRY PARAMETER ALLO""S RE-ENTRY OF THE ALGORI THM
AfTEP. CHANGING THE RI GHTHAND-SI DE OR THE VALUE COLUMN.
IN WHICH CASE REENTP.Y=I SHOULD BE SUPPLIED.

THE SAME PARAMETER ALSO SERVES AS ABNOPJ1AL EXI T PARAMETER.

FOR NOP.MAL. I.E. OPTIMAL AND FEASIBLE EXIT.

THE REENTRY PARAMETER IS ASSIGNED THE VALUE ZERO. EVEN IF
ITS VALUE ON ENTRY ... AS 01 FfERENT.
AN UNBOUNDED PROBLEM IS INDICATED·BY REENTRY = I.
AN EMPTY PP.08LEM BY REENTRY = - 1;

INVERTED := 'TRUE';
'GOTO' PHASE I; 'END';

FILL NAMELISTS:
'FOR' J:=I 'STEP' 'CNTIL' N 'DO' COLLSTrJJ := J;
'fOR' 1:=1 'STEP' , UN TIL' M 'DO' RO",'L S T rI J : = I 00 0 + X;

SET UPPER BOUNDS AND FILL DL'MMY ENTRI ES:

'FOR' J:=NAV+I 'STEP' I 'CNTIL' N 'DO' ' I f ' TrM+2.JJ=O
'THEN' TrM+2.JJ:=IOOOOOO;
T(M+I.N+IJ:=TrM+2.N+I]:=TrM+I.N+2J:=TrM+2.N+2J:=O;
, FO R' I: = I 'S T EP , I ' UN TIL' M+ 2 ' DO' T r I • N + 2 J : = 0;
'FOR' J:"I 'STEP' I 'UNTIL' NAV 'DO' TrM+2.Jl:=O;

ATTEND EQUATIONS ""1 TH THE ",'RONG 01 RECTIoN:

'FOR' 1:=1 'STEP' I 'UNTIL' NEQ 'DCl' 'BEGIN'
, IF' T ( I • N + I 1 > 0 'TH EN '
'FOR' J:=I 'STEP' I 'UNTIL' N+I 'DO'
, If' , NO T' T r I • J 1" 0 ' TH EN' T r I • J 1 : = - T r I • J ]; • EN D' ;

ATTEND PRII'!P.L DEGENERACY:

, Fo R' I: = I 'S T EP , I ' UN TIL' M ' DO' '1 f' T r I • N + I J = 0
'THEN' 'BEGIN'
TrI.N+IJ:=I;
, Fo R' J: = I 'S T EP , , t:N TIL' N ' DO' 'I f' T r I. J J < 0 'TH EN '
T(I.N+IJ:=TrI.N+Il-TrI.JJ;
T(I.N+IJ:=O.OOOOOOOOOOOOOOI*TrI.N+Il;
' I f ' I < NEC+I 'THEN' TrI.N+Il:=-TrI.N+I]; 'END';

ATTEND DUAL DEGENERACY:

'FOR' J:=I 'STEP' I 'UNTIL' N 'DO' ' I f ' TrM+I.JJ=O
'THEN' 'BEGIN'
TrM+I.JJ:=1;
'FOR' 1:=1 'STEP' 'UNTIL'M 'DO' 'IF' TrI.JJ > 0 'THEN'
TrM+I.JJ:=T(M+I.Jl+TrI.JJ;
T(M+I.Jl:=O.OOOOOOOOOOOOI*TrM+I.Jl;
' I f ' J < NAV+I 'THEN' TrM+I.Jl:=-TrM+I.J]; 'END';
254 CHAPTER XII

PHASE 0:
ENTER VARIABLES VITHOUT SIGN RESTRICTION:
K: =0; INVERTED: =' FALSE';
RETURN IN INVERSION:
K:=K+I;
'IF' K > NAV 'THEN' 'BEGIN'
INVERTED:='TRUE'; 'GOTO' ORDER; 'END';

COLN:=K; QUO:"IOOOOOOOOOOOOOOO; Ro\lN:=O;

'FOR' 1:-1 'STEP' I 'UNTIL' M 'DO'
'IF' ROWLSTCIl > NAV 'AND' TCI~Kl , 0 'THEN' 'BEGIN'
TQUO:"TCI~N+11/TCI~Kl;
'].1'" TQUO < 0 'THEN' TQUO:=-TQUO;
'1,' TQUO < QUO 'THEN' 'BEGIN'
QUO:'"TQUO; R:=l1 ROWN:=ROWLSTCIH 'END'; 'END';
'IF' ROWN=O 'THEN' 'GOTO' UNBOL'NDED;
QUO:=VNBV:"TC R~N+ Il/TCR~KlJ 'GOTO' MAKE THE STEP;

ORDER:
ORDL(T~M~N~2~2~RO'JLST~COLLST);

PHASE I~ FEASIBLE := 'TRUE';

FIND WETHER A FEASIBLE SOLUTION EXISTS:
'FOR' l:zNAV+1 'STEp· I 'UNTIL' M 'DO' 'IF' TCI.N+ll<O 'THEN'
FEASIBLE P' 'FALSE';

MAXIMIZE: HIG:=O; COLN:=O; VNBV:=O; R:=K:=O;

DUAL RAT 1 0 :.. I 0 0 0 0 0 0 0 0 0 0 0 0 ;
'FOR' J:=I 'STEP' I 'UNTIL' N 'DO'
'IF' COLLSTCJl < 1000 'OR' COLLSTCJ] > NEQ + 1000
'THEN' 'BEGIN'

INITIALIZE SUBSTITUTE PREFERENCE DIRECTION:

N ASC :'" ASC:- 0; N OF NC p. 0;

'IF' FEASIBLE 'THEN' ASCI=-TCM+I.J] 'ELSE'

'FOR' I:-NAV+I 'STEP' I 'UNTIL' M '00'
'IF' TCI~N+ll < 0 'THEN' 'BEGIN'
ASC :- ASC-TCI.Jl;
'IF' TCI~Jl<O 'THEN' 'BEGIN'
N OFNC:- N OFNC +IJ
N ASC :- N ASC - TCI.JlJ 'END'; 'END';

REFUSE L'NDERSIZE PHASE 1 PIVOTS:

'IF' 'NOT' FEASIBLE 'AND' ASC < 0.0000001
'THEN' 'GOTO' FINISHED WITH THIS COLUMN;

'IF' 'NOT' ASC > 0 'THEN' 'GOTO' FINISHED WITH THIS COLL'MNJ

PUT PREFERENCE FOR PREFERRED COLUMNS:

'IF' 'NOT' FEASIBLE 'A.rD'
TCM+ I~Jl > 0 'AND' ASC > 0 'THEN'
ASC : .. 0.00 UASCJ
'IF' 'NOT' FEASIBLE 'AND' TCM+I~Jl < 0 'THEN'
ASC :- ASC-TCM+I~J]J
LINEAR PROGRAMMING ON THE COMPUTER 255

SEARCH FOR SMALLEST ~VO ~ITH JTH COLUMN:

QUO:=IOOOOOOOOOOOOOOO;
TRYN := TRYR := 0;

'FOR' II=NAV+I 'STEP' 'UNTIL'M 'DO'

'IF' ABS(TCI.JJ> > 0.00000001 'THEN' 'BEGIN'
'IF' TCI.J] > 0 'AND' TCI.N+I] > 0
'THEN' 'GOTO' CHECK QUOTIENT;
'IF' TCI.J] < 0 'AND' TCM+I.J]<O 'THEN' 'BEGIN'
'IF' R01.,'LSTCl]<IOOO 'THEN' 'GOTO' TRY UPPERBOllND;
'IF'ROIJLSTCI]>IOOO+NEQ
'THEN' 'GOTO' FINISHED ~ITH THIS PIVOT; 'END';

, IF' TC I. J J < 0 'AN D' TC I. N+ I J < 0

'THEN' 'GOTO' CHECK ~l'OTIENr;
, IF' T C I • J J < 0 'AN 0' R01•.'L S T C I J < I 0 0 0 ' TH EN '
'GOTO' TRY UPPER EOl'ND;
'IF' TCI.JJ <0 'OR' TCI.N+1J < 0 'THEN'
'GOTO' FINISHED ~ITH THIS PIVOT;

CHECK G:UOTI ENT:

VPPERBOL'ND:= 'FALSE';
TQUO:=TCI.N+IJ/TCI.JJ;

, IF' TC I. J] < 0 'AN 0' TC I. N+ I J < 0 'TH Erl' 'BEGI N '

,I" RO IJL S T CI J ,. I 0 0 0 ' AN D ' RO 1,.'L S T CI] < NEG: + I 00 I
'THEN' 'GOTO' CHECK DUAL RATIO;
' I , ' I=R 'AND' 'NOT' K=O 'THErl' 'BEGIN'
'IF' TCM+I.JJ > 0 'AND' TCM+l.KJ>O
'THEN' 'GOTO' CHECK DUAL RATIO; 'END'; 'END';

'IF' TCI.JJ < 0 'AND' -TCI.JJ*N OF NC*3 < N ASC

'THEN' 'GOTO' TRY l'PPERBOl'NO;

'IF' (TCI.N+I]<O 'AND' TCI.J]<O) 'THEN' 'BEGIN'

'1,' 'NOT' TRYR=O 'THEN' 'BEGIN'
'IF' (RO ....LSTCTRYRJ=TP.YN 'AND' TCTRYP..JJ<O
'AND' TCTRYP..N+I]<O)
'THEN' 'BEGIN'
'IF' TQl'O < TCTRYR.N+IJ/TCTRYR.J]
'THEN' 'GOTO ~ TRY l'PPERBOllNO; 'END'; 'END' J 'END' J

CHECK Dl'AL RATIO:

'IF' (TCI.N+I]<O 'AND' TCI.J]<O 'AND' TCM+l.JJ>O)
'AND' 'NOT' K=O 'THEN' 'BEGIN'
' I , ' (I"'R 'AND' -TCM+l.JJ/TCI.J] > DUAL RATIO)
'AND'TCM+I.K]>O
'THEN' 'GOTO' FniISHED \11TH THIS COLL'MNJ 'END'J
 256 CHAPTER XII

MEASURE:
'IF' TeUO' Qt:O 'THEN' 'BEGIN'
'IF' TCI.Jl , 0 'OR' UPPEREOlJND
'THEN' 'GOTO' FINISHED WITH THIS PIvor;
'IF' TRYR' 0 'THEN' 'BEGIN'
'IF' TRYN < 10000 'AND' TCTRYR.Jl , 0
'THEN' 'GOTO' FINISHED \11TH THIS PIvor;
'IF' TRYN' 10000 'AND' TCTRYR.JJ < 0
'THEN' 'GOTO' FINISHED WITH THIS PIvor; 'END'; 'END'L

QUO:=TQUO;
'IF' QUO*ASC < HIG 'AND' CTCI.Jl , 0 'OR' UPPERBOlJND>
'THEN' 'GOTO' FINISHED WITH THIS COLUMN;
TRYR:=I; TRYN:=ROWLSTCI1;
'IF' UPPERBotJND 'THEN' TRYN:'"TRYN+IOOOO;
'GOTO' FINISHED WITH THIS Pivor;

TRY UPPER BOl'ND:

'I F' TC I.Jl , 0 'THEN' 'GOTO' FINI SHED WI TH THI S PI vor;
'IF' RO\lLSTCll,IOOO 'AND' ROWLSTCl]<IOOI+M
'THEN' 'GOTO' FINI SHED \II TH THI S PI vor;
UPPERBOUND:='TRUE'; TQt:O:=-TCI.N+21/TCI.Jl;
'GOTO' MEASURE;
FINISHED WITH THIS PIVOT: 'END';

'IF' QUO' 999000000000000 'AND' COLLSTCJl , 1000

'AND' COLLSTCJJ < 10000 'THEN' 'GOTO' lJNBOUNDED;
'I F' QUO ,. TCM+2.J.l 'THEN' 'BEGIN'
'IF' COLLSTCJl , 1000 'AND' COLLSTCJl < 10000
'THEN' 'GOTO' END OF DIRECT HIT LOOP;
QUO: '"TC M+2.J]I
'IF' COLLSTCJJ < 10000 'THEN' TRYN := COLLSTCJ1+IOOOO
'ELSE' TRYN := COLLSTCJ1-IOOOO;
END OF DIRECT HIT LOOP: 'END';

CHECK HEI GHT OF STEP:

'IF' TRYR, 0 'AND' KIO 'THEN' 'BEGIN'
'I F' CTRYR=R 'AND' ROWLSTC TRYRl =TRYN
'AND' TCTRYR.Jl<O 'AND' TCTRYR.N+IJ<O 'AND' TCM+I.Jl>O
'AND' TCM+I.Kl'O 'AND' -TCM+I.JJ/TCTRYR.Jl < DUAL RATIO)
'THEN' HIG:=O; 'END';

'IF' QUO*ASC , HIG 'THEN' 'BEGIN'

HIG:-CUO*ASC; VNBV:=QUO;
'IF' 'NOT' ABSCTRYN-COLLSTCJ1) = 10000 'THEN' R:"TRYR
'ELSE' R:-O;
'IF' 'NOT' R-O 'THEN~ 'BEGIN'
'IF' ROWLSTCRJ=TRYN 'AND' TCR.Jl<O 'AND' TCR.N+IJ<O
'THEN' DUAL RATIO:z-TCM+I.J1/TCR.Jl;
, END';
RO\lN:=TRYN; K:=J; COLN:=COLLSTCJlI 'END';
LINEAR PROGRAMMING ON THE COMPUTER 257

FINISHED .... ITH THIS C01.UMN: 'END';

'IF' C01.N = 0 'THEN' 'BEGIN'

'IF' FEASIB1.E 'THEN' 'GOTO' OUT 'ELSE' 'GOTO' EMP'rO
, END';

MAKE THE STEP:

C01.1.STCK): = R01t.'NJ QUO: =VNBV;

ADJUST RI GHTHANO SI DE AND UPPER BOUNDS C01.L'M~l:

'FOP.' 11"'1 'STEP' I 'L'NTI1.' R-I~ R+I 'STEP' I 'L'NTIL' M+l
'~O' 'IF' T[l~Kl I 0 'THEN' 'BEGIN'
TCI~N+I)I=TCI.N+Il-T[l~K)*QUO; 'IF' T[l~N+Il"'O
'THEN' TCI.N+I):=O.OOOOOOOI;
TCI~N+2l:=TCI~N+2)+TCI.Kl*~UO;
'IF' TCI.tl+2)=0 'THEN' TCI~N+2l:=0.0000000U 'END';

CONSIDER ONE COLUMN UPDATE:

'IF' ABSCR01.!N-COLN)=IOOOO 'THEN' 'BEGIN'
'COMMENT'
01 RECT HI T OF THE CPPER BOUND ON THE COLL'MN-VARIABLE.
NO FU1.1. UPOATl NG 0 F THE TAE1. EAU 1 S REI:UI RED;
'FOR' 1:=1 'STEP' 1 'UNTI1.' M+I ,~O'
'IF' TCI~KJIO 'THEN' TCI~Kll=-TCI.K);
'GOTO' CHECK FOR STATUS; 'END';

ATTEND UPPERBOUNDS OF PIVOTAl. PAIR:

COP:=TCM+2~KJ; NUM:=TCM+2.K):=TCR~N+ll+TCP..N+2);
TCR.N+)) := QCO; TCR.N+2) := COP-~UO;
'IF' TCR~N+2)"'0 'THEN' TCR.N+2l:-0.00000001;
'IF' G:L'O=O 'THEN' TCR.N+I):=O.OOOOOOOU

REFORMU1.ATE RO,.. \11 TH t'PPER BOUND NAME:

, 1 F' R01t.'N > I 0000 'TH EN '
'FeR' J:=I 'STEP' I 'UNTI1.' 1'1 '~O' TCP'~J):=-TCR~JH

UPDATE:
PIV:=TCR~KH
'FOR' J:=I 'STEP' 'UNTIL'N 'DO' 'IF' TCR • ..11IO 'THEN'
TCR.J):=TCP. • ..1)/PIV;

'FOR' ..1:=1 'STEP' 'UNTI1.' K-I.K+I 'STEP' 1 'UNTI1.' 1'1 'DO'

'IF' 'NOT' TCR • ..1) o 'THEN'

'FOR' 1:=1 'STEP' 't:NTI1.' R-I.R+I 'STEP' I 'UNTIL' M+I '~O'

'IF' 'NOT' TCI.K) =0 'THEN' TCI~J):=T[l.J)-TCR.JJ*TCI.K]J
'FOR' 1:=1 'STEF' I 'UNTI1.' M+I 'DO' 'BEGIN'
'IF' 1 < M+I 'AND' ABSCTCI.Kl) < 0.0000001
'THEN' TC 1 ~ K): = 0;
'IF' 'NOT' TCI~K)=O 'THEN' TCI~K):=-T[l.K)/PIV; 'END';
TCR.Kl:=I/PIV;
258 CHAPTER XII

ROVLST[R):&COLN;

I F NECESSARY REFORMULATE NEV ROV "'I TH UFF'ERBOUND NAMEI

'IF' COLN > 10000 'THEN' 'EEGIN' RO"'l.STrR):=COLN-IOOOO;
'FOR' J:=I 'STEP' I 'UNTIL' N 'DO' 'IF' T[R.J)IO 'THEN'
TCR.J):&-T[R.JlJ
COP:&T[R.N+I)+T[R.N+2)J T[R.N+I):=COP-QUO; T[P.N+2)I&QUO;
• END';

CHECK FOR STATUS 1

'IF' 'NOT' INVERTED 'THEN' 'GOTO' RETURN IN INVERSION;
'iF' FEASIBLE 'THEN' 'GOTO' MAXIMIZE 'ELSE' 'GOTO' PHASE lJ

EMFTY:
REENTRY 1& -I;
NEIo.'LINE( I);
VRITETEXT('C'EMPTY%PPOBLEM')');
NEVLINE( I);
'GOTO' ORDER FOR EXI Tl

UNBOUNDEDI
REENTRY : '" IJ
NEVLINE( I);
~RITETEXT('('lWBOUNDED%COLUMN')');
VRI TE( 30. FOP.MAT( '( 'S-NDDDDDD') '). COLLST[J]);
NE\.'LI NEC I);
'GOTO' END OF LINP;

OUT:
'IF' FEASIBLE 'THEN' REENTRY :=0 'ELSE' REENTRY:" -IJ
ORDER FOR EXI T:
ORDLCT.M.N.2.2.RO\.'LST.COLLST);

END OF LINP: , END';

12.4 Printing a Simplex Tableau

The tableau printing procedure offered in this section is an

adaptation of the matrix printing procedure mentioned in
Section 2.18. The tableau is printed with its two namelists.
The following different features are, however, noticeable.

In connection with the printing of a rownameslist on the left-

hand side of each block, the number of columns per block-column
is reduced (from 15 to 13). The facility to "skip" leading rows
and columns has been integrated with the printing of names.
Name-codes are printed only for "proper" rows and columns, not
for supplementary vectors, e.g. target-row and value column.
If all proper rows are skipped, i.e. only one or more "extra"
rows-are printed, no list of rownames is required and 15 rather
than 13 columns per block-column are printed.

The printing of the indices of the leading element of each

block is replaced by the printing of the list of columnnames.
This is done every thirty lines, and a blank line is inserted
LINEAR PROGRAMMING ON THE COMPUTER 259

half-way. The procedure can also be used for printing certain

blocks of the tableau itself, in the same way as with the
matrix printing procedure. The facility for scaling has not
been maintained. The text of the procedure is as follows:

TEXT-LISTING OF THE TABLFAl'-FRINTItJG FRoCEDl'F.E.

'PRoC FDl'RE' TABo (HATP.. H. N. Sr.. SCI F.H. EP.. Ro1o.'L ST. CoLL STH
'AF-P.AY' MATR; "INTEGEP' M.N.SP..SC.RH.EP.;
'INTEGER' 'I'.F.F:AY' RoIJLST. CoLLST;

'E EG IN' 'I NTEG ER' NRDO. NP. TD. NCoL DO. NCoL TD.
PAGEC. P.PRI NP. CPRI tiF. I. J. NAME. IN DEX;
'REAL' NUM;

'COMMENT'
TABo STI'.NDS FOR TI'.ELEAU oVT.
I TIS A TAELEAl' FRINTING FRoCEDURE. PRINTING THE TABLEAU
IN BLOCKS.
THESE ELoCY.S ARE EoRDERED AT THE TOP ANt; THE LEFTHAI'JDSI DE.
EY THE APFP.OFEIATE FAP.TS OF THE NAMELISTS.

THE TABLEAl'-MI'.TRIX 1 S OF ORDER M+ER EY N+PJ.!.

EP. ~TANDS FOR EXTRA Roll.'S. E. G. TARGET-POlo:. ETC.
RH STI'.tJDS FOF rI GHT-HArJD 51 DE CoLL"MNS. E. G. VAl.UE CoLl'MN.
DI STANCES CoLl"MN. ETC.
SR ~TAND5 FOR SKIF P.01o:S.
SC STANDS FOF: SKIF COLUMNS.

THE l'.l'XILIARY VAP.II'.BLES NF:Do.NFTD.NCoLI:O.NCoLTD.FAGEC.

EFF:INF. AND CFrINF HI'.VE THE SAME SI GNI n CANCE AS IN MATO.

11 EDD: "NCoL DO: = 0; NET D: "11+ ER; ,NCoL TD: =N+RH;

NEI-'LINE< 1>;

STAr.T:
NE\.'LINE( 3);
!'.FRINP:=30;
'IF' SC < N 'THEN' CFRltJF:=13 'ELSE' CFP.INF:=IS;

ADJl'ST NUMEER OF COLVMNS:

'IF' NCoLTD < CPP.INF 'THEN' CFRINF:=NCoLTD;
NCOLTD:=NCoLTD-CPRINF;

CHECK fOP. FAGE:

, I 1'" NP.TD > lS 'J'.~JI:" N > 1 S 'TH EN '
FAGEC:=:,JEDO/30-ENTI ER(NEDo/30) 'ELSE'
,AGEC:=l;
'IF' PAGEC=O 'THEN' 'EEGIN'
'COMMENT'
BLOCKS I'LL TO STAF.T AT THE TDF OF A FAGE.
THIS Al.LOll.'S THE usn: TO STICY. THEM TOGETHER.
1o:HIL'E y'EEFING PO\:S ArH: CoLVt':NS II': LINE.;
F.t:Not:T; , EN D';
ND'LI:IE( 2);
260 CHAPTER XII

HEADING:

ALFHANCMERI CAL COLUMrlNP.MES LIST:

WRITETEXTC'C'N~~E%I%%%%%%I!%')');
'FOP.' J:-I 'STEP' I 'CNTIL' CFIHNF 'DO' 'BEGIN'

'I F' SC+NCOLDO+J > N 'THEN' 'GOTO' FRINT FOP. VALUE COLt!MIU

SPACEC 2);
NAME :" COLLSTC SC+NCOLDO+J H

X ISTAP. IN CL:
'IF' 'NOT' crJ~'1E>N 'ANI:' NANE<2*N+1 'AND' N<300)
'THEN' 'GOTO' X 2STAR IN CL;
IoHUTETEXTC 'C '%X*') '); INDEX:=NAME-N;
'GOTO' "'JUTE THE INDEX IN CL;

X 2STAR IrJ CL:

'IF' 'NOT' CtJA.'1E>2*N 'AND' tIAME<3*N+1 'AtH) , N<300)
'THEN' 'GOiO '. X It: CL;
"~F.I TETEXTC 'C 'X**')'); INDEX: =rIA!'1E-~*N;
'GOTO' ~F.ITE THE INDEX IN CL;

X IN CL:
'IF' 'NOT' CNAME>O 'AND' NAME<IOOO)
'THEN' 'GOTO' SLACK IN CL;
Ir.'RI TETEXTC 'C ' %%X') '); INDEX: =NA!'1E;
'GOTO' \lRITE THE INDEX IN CL;

SLACK IN CL:
'IF' 'rIOT' CNA.'1E>IOOO 'AND' NAME<2000)
'THEN' 'GOTO' UFLIM IN CL;
1r:RITETEXTC 'C '%:1:5') '); INDEX := NAME-IOOO;
'GOTO' "'RITE THE INDEX IN CL;

UPL I MIN CL:

'IF' 'NOT' CCNAME>2000 'AND' NAME<3000)
'OP.' NP.ME >10000) 'THEN' 'GOTO' ARTFV IN CL;
WP.ITETEXTC' C '%%E')')J
'IF' NAME < 10000 'THEN' INDEX:=NAME-2000
'ELSE'INtEX:=NAME-IOOOO;
'GOTC' "'EITE THE ltlDEX IN CL;

ARTFV IN CLI
'IF' 'NOT' CNAME>300C 'AND' NAME<1I000)
'THEN' 'GO TO , DUAL V 1 N CL;
VRITETEXTC' C' %%A')'); INDEX: =NAME-300C;
'GOTO' ~P.ITE THE INDEX IN CL;

DUALV HI CL:
'IF' 'NOT' (NP.ME<-IOCO 'AND' NAME>-2COC)
'THEN' 'GOTO' DCALSL IN CL;
"'F.ITETEXT(' C ':%P')')J INDEX: =-tJAME-1 000;
'GOTO' WRITE THE INDEX IN CL;
 LINEAR PROGRAMMING ON THE COMPUTER 261

Dl'AL SL IN CL I
'IF' 'NOT' (NA."'IE<O 'AND' NAME>-lOOO)
'THEN' 'GOTO' DUAL UL IN CLI
lo.' RI T I:T EX T ( , ( , %It D' ) , ) I INDEX: = -N AM EI
'GaTe' ",'RI T E TH E ltIDEX IN CLI

DUAL UL 1 N CL I
'IF' 'NOT' (-NAME>3000 'AND' -NAME<4000)
'THEN' 'GOTO' SHADO~P~ICE UPPER LIMIT IN CLJ
\.iPoI TETEXT< '( '% %Z ') '); INDEX: =-NAME-300 0;
'GOTO' 101P.ITE THE INDEX IN CL;

SHADO\oiPRICF ~FPEP. LIMIT IN CLI

'IF' 'NOT' (-NAME>2000 'AND' -NAME<3000)
'THElJ' 'GOTO' VF.ITE THE INDEX IN CL;
"'RI TETEXT(' (' 1%l")'); INDEX: =-NAME-2000;

\:F.ITE THE INDEX IN CL:

'IF' INDEX<99 'THEN'
101fUTE(30.FOF.MAT( '( 'NDS') ').INDEX)
'ELSE'
\:RITE(30.FOF.MAT( '( 'NDD') ').INDEX); 'D~I:';

'GOTO' Pt:T UPFEF. EORI:EP.LINE OF TABLEAt: 1 TSELFI

FRINT FO~ VALUE COLl~N:

'IF' SC+NCOLDO+J=N+l 'THEN' lo.'!UTETEXT('('%!!~%%VALUE')')J

PUT urFER EORDEP.LINE IUSI DE HEADI tJG:

NE\:LINE( 1)J
'FOR' J:=l 'STEP' 1 'l'NTIL' CFF.INF 'DO'
~F.ITETEXT('('--------')');
'IF' SF<M 'THEN' \.'F.ITETEXT('('--------------,)');
, 1 F' SC+NCOLDO+CFRI NF N+ 1 =
'THEN' 101RI TETEXT( '( '----,) ' ) J

CODING COLUMNLIST:
NE",'LINF( 1)J
'I F' SF.<M 'THEN' lo.'RITETEXT(' (' %%%%% 1 :CODEI! U')')J

'FOR' J:cl 'STEF' 1 'UNTIL' CPRINF 'DO' 'SEGIN'

'IF' SC+J+NCOLDO > N+l 'THEN' 'GOTO' ELP.NK;
'I F' SC+J+NCOLDO = N+ 1
'THEN' 'GOTO' SEPARP.TION HEJ'.DING;
'IF' AES(COLLSTCSC+NCOLDO+J]) > 9999 'THEN'
101RITE( 3 D. FOP.MP.T ( , ( '-NDDDDDD' ) '). COLLSTC SC+NCOLDO+J])
'ELSE' \.'RITE(30.FOP.MAT( '( '-NDI:DDDS')').COLLSTCSC+NCOLDO+J]);
, GOTO' ELANY.;
SEFAF.ATION HEADltJG:
IOF:ITETEXT( '( '11 !%1%%%') ' ) J
ELI'JJK: ' EtJ I:' ;
Nn.'LlNE< 1)J
262 CHAPTER XII

Pl'T UPPEF. BOP.DEFLINE Of TAELEAl' ITSELF:

'fOR' J:-I 'STEP' I 'l'NTIL' Cf'r,lNF '00'
"'RI TETEXT( '( ,--------') ');
, I 1" SF. < M 'TH EN' 1.'1': I T ET EX T C ' ( '- - - - - - - - - - - - - - ') , )J
' I f ' SC+NCOLDD+Cf'FINP - N+I
'THEN' V~ITETEXTC'C'----')');

AruU,ST Nl1l':BEF. Of R01r.''s:

' I f ' NRTD < RPflINF 'THEN' RF'fINF:"'NF:TP;
NfiTD:=N~TD-~F'FINP;

PFINT THE LINES:

'fOP.' 1:=1 'STEP' I 'UNTIL' RH'ilNF '00' 'BEGIN'
NH'LINEC I);

' I f ' S1'+NF,rO+I=M+I 'THEN' 'BEGIN'

Ft'T LO,.'ER SOT,[,E'RLHIEI
'fOR' J:=I 'STEP' I 't'NTIL' CFFINF 'DO'
"'T!I TETEXTC 'C ,--------,) ' ) J
' I f ' SP<M 'THEN' ,.'f,ITETEXTC '( ,-------------- ') ');
'I F' SC+NCOLDO+CffINF - N+ I
'THEN' ,.TI TETEXTC' ('---- ') ');
NE"'Llt:E( I)J 'El~P';

'11"1=16 'TH01' NEl.'LINECI)J

'IF' SF. > 1'1-1 'TH01' 'GOTO' NO RO"'LIST NEEDED;

' I f ' SR+I+NF.DO > 0 'AND' SR+I+NF.DO < 1'1+1

'THEN' 'EE'GI N'

ALPHANt'MERI CAL F.01.'NJ!.MES LISTI

NAME : - F.01t.'LSTr SF.+tJF.DO+I JJ

X I STAR IN RL:
' I f ' 'NOT' CNAME>N 'AND' NAME<2*N+I 'AND' N<300)
'THEN' 'GOTO' X 2STAF IN RL;
1011'.1 TETEXTC' C 'XX*')')J INDEXp=NAME-N;
'GOTO' "'f. I TE THE INDEX IN RL;

X 2STAR IN P.l.:
'IF' 'NOT' CNAME>2*tJ 'AND' NA!'1E<3*N+I 'A."JD' N<30:0)
'THEN' 'GoTO' X IN P.l.;
IJFITETEXTC'C'X**')'); INDEX:=NAME-2*N;
'GOTo' ,.'F,ITE THE H'OEX IN PoL;

X IN RL:
' I f ' 'UOT' CNAME>O 'ANP' NAME<IOOO)
'THEN' 'GOTO' SLACY. IN PJ..;
"'1'.1 TETO:T( 'C 'X') ')J INDEX: -tJAMEJ
'GOTO' "PI TE THE INDEX INRI.;

SLACY. 1 tJ RL:
' I f ' 'NOT' OIAME>IOOO 'AND' NAME<2000)
'TH :!'I' ',GO TO , t'FL I MIN PJ..;
1.iF.ITETEY.TC'C'S')')J INDEX :- NAME-IOOO;
'GOTO' "'RITETHE INDEX IN PJ..;
 LINEAR PROGRAMMING ON THE COMPUTER 263

UP1.IM IN Fl.:
'IF' 'NOT' «NAME>2000 'A'JD' NAMEc3DDD)
'01'\' NI-.ME >10000) 'THEN' 'GOTO' AP.TFV.IN R1..1
\lP.ITETEXT( '( 'B') ')J
'IF' NAME c 10000 'THEN' INDEX:=NAME-2DOO
'ELSE' INDEX:=NAME-IOOCO;
'GoTo' \lRITE THE INDEX IN R1..;

ARTFV 1 tI Pl.:
'IF' 'NOT' (NAME>3000 'P.ND' NAMEc4000)
, TH EN' 'GO TO' DUAl. V 1 N Rl;J
1011'.1 TETEXT( '( 'A') '); INDEX: =NAME-3000;
'GoTo' 1o.'P.ITE THE INDEX IN Fl.;

DUAl. V I N P.L:
'IF' 'NOT' (NAMEc-IOCO 'AND' NAME>-2000)
'THEN' 'GOTO' DUAl.S1. IN P.l.J
1o.'FtITETEXT( '( 'P') '); INDEX:=-NAME-ICOO;
'GOTo' VRITE THE INDEX IN P1.J

DUAl. S1. 1 N P.L:

'IF' 'NOT' (NAMEcC 'ANI:' NAME>-IOOO)
','fH EN' , GO TO' DUAl. Ul. I tJ F.L;
1o.'RITETEXT( '( '0') '); INDEX:=-NAME;
'GoTO' ,.RI TE THE INI:EX IN P.L;

DUAl. Ul. I N P.L :

'IF' 'NOT' (-NAME>3000 'AND' -NAMEc4o.00)
'THEN' 'GOTO' SHADo\lPRICE UPFER LIMIT IN R1..;
1o.'RITETEXT( '( 'Z') ')J INDEX:=-NAME-3000J
'GoTO' IrJF.ITE THE INDEX IN R1.;

SHAI:OIrJPRICE UFFER LIMIT IN P.L:

'IF' 'NOT' (-NAME>2000 'A'JD' -NAMEc3000)
'THEN' 'GoTO' 1o.'P.ITE THE INDEX IN F.L;
.... F.ITETEXT( '( 'U') '); INDEX:=-NAME-2000;

.... P.ITE THE INDEX IN P.L:

'IF' NAME> N 'AND' NAME c 2*N+I 'THD~'
1o.'RITE(30.FORMAT('('NI:')').ItJDEX) 'ELSE' 'P-EGIN'
'I F' INDEX < 100
'THDJ' ,.·RITE(3D.FoP.MAT( 'C 'UDS') '>'INPEX)
'ELSE' \.:RITEC3D.FORMATC '( 'NOD') ').INDEX); 'END';
'IF' AESCNP.ME)c 10CDD 'THEN' \..'RI 1 ETEXT( '( '%! ') ')J

CODED P.01o.'NAMES LIST:

'IF' I.ESOJAME»IOOOO
'THEN' 1o.'F.I TEe 30. FOF.MATC' C 'S-NDDDDD') ') .NAME)
'ELSE' VEl TEC 30. FoRMATC ' C '-NI:OD') ') .NAME) J
'END'

'ELSE' 'EEGIrJ'
'IF' SF.+I+r:F.OO = M+I 'THDJ' 1o.'RITETEXTC'('%%T%%!%%%%%')')
'ELSE' Io.'F.ITETEXTC'('%%%%%!%%%%%')'); 'END';
264 CHAPTER XII

NO ROl.'LI ST NEEDED:

'FOR' .1:=1 'STEF' 1 'UNTIL' C.FRINF 'DO' 'EEGIN'

'IF' SC+NCOLDO+J-I = (T 'THEN' 101F.ITETEXTC'C'%I!%')');
'IF' SC+lJCOLDO+.1-'1 = N 'THEN' "'F.ITETExTC'C'%I!')');
'IF' NCOLDO+.1=1 'AND' SC>O 'THEN' I..'RITETEXTC'C'%lll')');
NUM : = MATRr SR+NRDO+ 1. SC+NCOL DO+J H.
'IF' tJUM=O 'THEN' .1tJRITETEXTC 'C '%%ll-%%l') ')
'ELSE' 'BEGIN'
'IF' NL'l1=ENTlERCNtJM)
'THEN' Io:P.ITEC 30. FOEMATC 'C 'S-NDDSSS') ').NUM)
'ELSE' VP.ITEC30.FORMP.TC 'e 'S-NDD.DD') ').Nl'M); 'END';
'END'; 'OH";

AlJ.1UST INDI CES AND RETl'P.N UNLESS READY I

NRDO:=NEDO+FFFItJF;
'I F' NEDO=M+EF. 'THEN' 'BEGIN'
NRTD:=M+EP.; NFDO:=O; NCOLDO:=NCOLDO+CFF.INF;
'I F' NCOLDO=N+F.H 'THEN' 'GOTO' END OF TABO;
'GOTO' STAP.T; 'END' 'ELSE' 'GOTO' CHECK FOP. FAGE;

EN D 0 F T ABO: N H.'L 1 tJ EC 2); 'END';

When reading the text of the tableau-printing procedure listed

above, one will have noticed, that it contains references to
other name-codes than only the ones discussed in section 12.1.

This is because the same procedure is also used for printing

quadratic programming tableaux, in which negative name-codes
and enlargements of 2000 occur, and integer programming tableaux,
in which enlargements by nand 2n occur. The precise significance
of these codes will be discussed later on in this book.

12.5 Text of a complete L.P. programme

In this section we offer a complete "main" programme, despite

the fact that this contains unavoidably, relatively many
system-specific features. The shortness of this programme is
the main justification. The main programme controls the input
of the data, i.e. the tableau-matrix, and the printing of the
results. The Simplex Algorithm itself is performed by the LINP
procedure presented in Section 12.3.
LINEAR PROGRAMMING ON THE COMPUTER 265

Even the input and output of data and results is largely

delegated to subordinate procedures. These are the matrix-
reading procedure MATI listed in Section 2.18 and the tableau
printing procedure listed in the previous section 12.4 of this
Chapter, and the results reporting procedure from section 10.4.

The one e:ement of evaluation by the main programme is whether

or not to print the full tableau. With a big system it is
obviously undesirable to have a pile of paper full of numbers.
But the particular limit of 13 columns and 40 rows and columns
together arises from the economical use of line-printer
paper. If there are no more than 13 columns or less than 15 rows
and only two block-columns to print the printing of the full
tableau is the most pronomical way to present the output. For
larger systems, output is restricted to a call to the result-
reporting procedure only.

Note that whereas the separate call to the reporting procedure

provides re-interpretation of the solution vector, the tableau-
printing procedure does not do that, i.e. the figure printed
after the symbol "x4" is the distance between x4 and its lower
limit, which may, or may not be zero. The programme text is now
listed, as follows:

'BEGIN' 'INTEGER' M,N,NAV,NEQ,REENTRY,I,J;

'PROCEDURE' LINP(T,M,N,NEQ,NAV,ROWLST,COLLST,REENTRY);
'ARRAY' T; 'INTEGER' M,N,NEQ,NAV,REENTRY;
'INTEGER' 'ARRAY' ROWLST,COLLST;
'ALGOL' ;
'PROCEDURE' MATI(MATR,MB,NB,FR,FC);
'ARRAY' MATR; 'INTEGER' MB,NB,FR,FC; 'ALGOL' ;

'PROCEDURE' REPOCT,M,N,NEQ,NAV,ROWL,COLL);
'ARRAY' T; 'INTEGER' M,N,NEQ,NAV;
'INTEGER' 'ARRAY' ROWL,COLL; 'ALGOL';
'PROCEDURE' TABOCMATR,M,N,SR,SC,RH,ER,ROWLST,COLLST);
'ARRAY' MATR; 'INTEGER' M,N,SR,SC,RH,ER;
'INTEGER' 'ARRAY' ROWLST,COLLST; 'ALGOL';

'COMMENT'
LINEAR PROGRAMMING BY THE SIMPLEX ALGORITHM.
FOR DETAILS OF THE ALGORITHM, SEE THE TEXT OF THE LINP
PROCEDURE.
266 CHAPTER XII

PRESENTATION OF DATA:
FIRST THE NUMBER OF RESTRICTIONS I.E. M.
THEN THE NUMBER OF VARIABLES. I.E. N.
FOLLOWED BY THE NUMBER OF EQUATIONS. NEQ.
AND AS LAST INTEGER ORDER PARAMETER. NAV.
THE NUMBER OF 'FREE' VARIABLES TO WHICH THE
TACIT (NON-NEGATIVITY) RESTRICTION DOES NOT APPLY.

THEREAFTER PUNCH EACH ROW OF THE COMPOSITE MATRIX

A B
W' 0
U' 0
L' 0

TO REPRESENT A.X < OR s B.

MAXIM -W'X
AND X < OR = U
AND X > OR = L
THE PROGRAMME READS ALL THE ELEMENTS OF THE UPPER AND LOWER
BOUNDS VECTORS U AND L. DESPITE THE FACT THAT
THE MAIN LINP PROCEDURE USES THESE NUMBERS ONLY FOR
BOUNDED VARIABLES
M:=READ; N:=READ; NEQ:=READ; NAV:=READ;
REENTRY:=O;

'BEGIN'
'ARRAY' TACI:M+3.I:N+2l;
'INTEGER' 'ARRAY' ROWLC 1 :Ml. COLLC 1 :N];
READ MAIN TABLEAU MATRIX:
MATI(TA.M+3.N+I.0.0);

REINTERPRET:
'FOR' J:=NAV+I 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'
'FOR' 1:=1 'STEP' 1 'UNTIL' M 'DO'
TACI.N+Il:=TACI.N+ll-TACI.Jl*TACM+3.Jl;
'IF' TACM+2.Jl=0 'THEN' TA(M+2.Jl:=1000000;
TA(M+2.Jl:=TA(M+2.Jl-TA(M+3.Jl;
'IF' TA(M+2.Jl < 0 'THEN' 'BEGIN'
NEWLINE<l );
WRITETEXT('('YOU%HAVEISPECIFIED%AILOWERILIMIT%
%INIEXCESS%OF%THE%CORRESPONDING1UPPER%LIMITI
%THEREBYICAUS ING%EMPTYNESS.')'); 'END';
, END';

LINP(TA.M.N.NEQ.NAV.ROWL.COLL.REENTRY);

REPO(TA.M.N.NEQ.NAV.ROWL.COLL);
'IF' N < 14 'OR' M+N < 40 'THEN'
TABO(TA.M.N.O.O.l.I.ROWL.COLL);
'END'; 'END'
LINEAR PROGRAMMING ON THE COMPUTER 261

12.6 Revised Simplex Algorithms

The algorithms which we discuss under this heading have the

following features in common:

a) Packed storage of the set-up tableau

The tableau matrix which defines the typical large L.P.

problem tends to a very predominant degree to consist of nothing
but zeros, quite often in excess of 99% zeros. Obviously,
substantial economies in storage-space are obtained if this
matrix is stored in some other way than storing each element.

For example, if the matrix-reading procedure from section 2.10

were used to read the tableau-matrix, one would simply begin
by storing the input-information as such, and at the same time
compile a list of addresses of the heads ot the cl1"'cessive rows.

(There are in fact other input-conventions and other ways of

"packed" storage.)

Even if,-as may be desirable in some versions of the revised

simplex - the set-up tableau is stored twice, once per column,
once per row, that still is a fraction of the storage
requirement of the unpacked tableau-matrix.

b) No updating of the tableau

The current tableau is normally more or less full of non-zero

elements, even if the set-up tableau consists largely of zeros.

Storage and updating of the current tableau is therefore

suppressed. Instead, some other representation of the current
solution is used. The value column and the list of name-codes
are kept currently updated all the time, and in some versions
also the objective function row. Other rows and columns are
generated as needed.

c) They are utterly impracticable on paper, but specifically

geared towards making an efficient use of computer-resources.
We now review the separate algorithms.

Explicit inverse with row-updating

This is the oldest one of this group of algorithms and one will
find it referred to in some older references as "the" reversed
simplex method.

In this method, one stores besides the packed form of the

original tableau, the current value column, the current
268 CHAPTER XII

objective function row and the inverse of the basis-matrix, the

objective function being counted as a basic variable.

The current updated tableau (explicit form) is then:

(12.6.1)

where T is the original tableau matrix in shortened form.

B- 1 itself is kept updated at each step in the usual way.

B- 1 is the inverse of the "extended basis-matrix"

I (12.6.2)
,1
If a slack-variable is indicated as incoming variable, the
pivotal column does not need calculation as it is already
present in the inverse itself.

If an element of x indicated as incoming variable, the pivotal

column is obtained by post-multiplication of the inverse by the
corresponding column of the original tableau.

Example (from Chapter VIII)

xl x2 x3 Value

T sl 1 1 1 200

s2 1 3 100

T -2 -5 -1

sl s2 T

B- 1 1 -1. 33
sl
x2 0.33

T 1. 67 1
LINEAR PROGRAMMING ON THE COMPUTER 269

The xl column may now be calculated as follows:

sl s2 T. xl xl

sl 1 -0.33 sl 1 sl 0.67

x2 0.33 s2 1 x2 0.33

T 1.66 1 T -2 T -0.33

(The actual storage of the unit vector fOr the objective

function is normally dispensed with.) The pivotal row is
selected in the normal way and the missing part of the pivotal
row is now calculated by means of a matrix-vector multiplication,
i.e. the original tablpRll is premultiplied by the relevant row
of the inverse. Ref~rp.nce to the objective function ~ay be
suppressed at this stage.

Example

xl x2 x3 xl x2 x3

[: ~]
sl [0.67
-0.33] 1 lJ

The inverse, the value column and the obje~tive function row
can then be updated in the usual way.

If the initial tableau-matrix T is entirely full of non-zero

elements, this method requires the same amount of storage as a
full explicit tableau, and more computational effort.

Calculation of the remainder of the pivotal row, plus updating

the inverse requires as much elementary computation as updating
the full tableau, and calculating the pivotal column comes to
it, unless the incoming variable is a slack-variable. Obviously,
under those assumpt ions, the shortened tableau, updated at each
step, is more efficient. The method comes into its own, only if
T consist'S largely of zero elements. That cuts the memory-space
requirement on account of the "packed" representation of T, and
it cuts the calculation-time needed for the matrix-vector
multiplications, on account of the suppression of multiplication
by zero.

Explicit inverse, without row-updatin8

In this version of the explicit inverse method, the calculation

of the remainder part of the pivotal row is suppressed, and it
270 CHAPTER XII

becomes therefore impossible to update the objective function

row in the usual way. One then first of all gives priority to
slack-variables as incoming variables: their shadowprices are
always available and no need to calculate a pivotal column
arises from selecting a slack-variable as incoming variable.
When a pivotal column is found in the inverse itself, the step
is made, otherwise the objective function row is calculated by
means of a matrix vector multiplication (the same could also be
done with a substitute objective function).

We have here a trade-off between the calculation effort per

step, and the number of steps needed. One is forced to
compromise the rule of the steepest ascent (the rule of the
highest step is out anyhow for all versions of the revised
simplex).

The method is likely to be more efficient than the version with

row-updating, if the number of restrictions is substantially
less than the number of variables.

If there are say 50 restrictions and 1000 variables, steps

which are performed solely on the 50 x 50 inverse are
comparatively very cheap indeed and the fact that some of them
will not lead to much gain in the solution value does not
terribly matter.

The product form inverse

In this algorithm only vectors are updated. The name refers

to the fact that one obtains the explicit inverse, as the
product of a series of matrices. Each of these matrices
consists of a unit matrix of which one column has been replaced
by the column which describes the variable which has just left
the basis, i.e. the updated pivotal column, divided by minus
the pivot, with the pivot itself replaced by the reciprocal.

Example

(The same, for the actual columns, see tableau 8.8, for the
explicit inverse see tableau 8.6.)

sl s2 T sl s2 T sl s2 T-

] [~ ]
x3 sl -0.33 x3 1 -0.33
x2
l' [:
1 x2

T
0.33

1. 67
x2

T
[1
0.33

1.33 :J
"s -matrix" "s 2-matrix" -1
1 B
 LINEAR PROGRAMMING ON THE COMPUTER 271

In application one obviously only stores columns (and pivots

and their name-codes and the row-indices of successive pivotal
rows), not matrices.

The product-form representation of the inverse is effective in

updating columns as they are needed. In effect one simply
reconstructs the updating of the relevant column of the tableau
as it would have taken place in the standard form of the simplex
method.

Example

Update the value column

Value Value Value

sl
1200
-0.33 sl 1€66.6~ 1 x3 166.67

S3 9 0.33 -+ x2 33.33 -+ x2 33.33

']; 1. 67 T 166.67 1 T 333.33

(x 2 incoming) (x 3 incoming)

The ring here indicates the leaving variable, the actual pivot
is the reciprocal of the corresponding entry in the already
updated pivotal column.

Unfortunately, if only columns are stored, there is no

practicable way of updating rows, no consistent information
for their updating is available, short of still belatedly
updating the whole tableau.

Two possible alternatives ar~ now open:

a. Update the column for which the log indicates that it is the
one which was the longest inactive, without even having been
able to consult its shadowprice, and accept it as incoming
variable if its shadowprice turns out to have the appropriate
sign.

b. Keep a set of pivotal rows as well. At first sight, neither

procedure appears to be particularly efficient. Procedure
(a) is clearly inefficient, and there are other indications,
i.e. the machine requirements of the commercially available
LP packages, that (b) is used (two channels of backing
store tend to be asked for). At-first sight, that appears
not very efficient either, the total memory storage
272 CHAPTER XII

requirement of the two product-form presentations of the

(primal and dual) inverses easily exceeds th~ size of the
updated tableau-matrix in shortened form.

There is, however, a machine-technical point which greatly

enhances the relative advantage of the product-form inverse.
Typically one has available a limited amount of high-speed
memory ("core") and a virtually unlimited amount of slower
back-up memory (disc or magnetic tape). If the problem is of
small or medium size, i.e. if the size of the shortened
tableau fits the capacity of the core alone, efficient choice
of pivots and updating of the tableau at each step is not
easily beaten. But once backing-store is involved we get a
quite different story:

The back-up memory has to be accessed linearly and asking for

only one element at the end of the tableau takes as much time -
most of which is spent in waiting for t~e mer~aniral part of
the back-up memory to move - as performing a routine calculation
on each column, provided they are accessed in the order in
which they are stored. The product-form inverse is ideally
suited to that situation in that it is always accessed in the
order in which it was generated. The one obvious disadvantage
of the product-form algorithm is that the size of the product-
form inverse increases with the number of steps and that
number may be several times the number of variables. For this
reason a product-form algorithm usually contains a provision
for re-inversion. After a set number of steps the current
vertex is re-created by a shorter route. Starting from the
trivial basis the elements of x, as far as they are in the
(new) current list of basic variables are brought back in,
in exchange for the slacks of binding restrictions, without
any regard to sign of either the pivot or of any entry in the
value column or the objective function.

This re-inversion-feature makes the gain in pure computational

effort compared with the explicit inverse, or for that matter
the ordinary simplex method (shortened tableau) largely
illusory. The product-form wins because for problems with a
large number of variables and a large number of restrictions,
other methods are not nearly--as well suited to accommodate the
specific features of a linearly accessible backing-store device.

With a change in computer technology (e.g. silicon chips), we

may well see a comeback of other versions of the simplex
algorithm. .
CHAPTER XIII

PARAMETRIC VARIATT OF THE LP PROBLEM

13.1 The parametric variation problem

The term "parametric programming" is usually referred to as

being due to Gass and Saati (12), [13J. The term "sensitivity
analysis" appears to be originally due to G. B. Dantzig (8J,
section 12.4, who discusses the use of the optimal tableau for
purposes of analysis, within the range of validity of the
optimal vertex.
If there are minor changes in the originally specified problem,
the optimal tableau tells us how this will affect the solution.
Dantzig's treatment of the problem includes the analysis of
changes in individual coefficients aij. This is a fairly
co~plicated mathematical problem and it is not easily possible
to tackle more than one coefficient at a time, except by
computational means, i.e. solving the whole problem again.
(See however, also section 2.18 for approximate solutions of
this problem).

In this book we follow Garvin Ill] and investigate changes in

the right-I-land side vector b and the preference-vector w', and
include the analysis of adjoining vertices. That definition of-
sensitivity analysis turns it into parametric programming.

As far as the mathematical and computational logic of the problem

is concerned, sensitivity analysis is parametric programming.
The term "sensitivity analysis" simply refers to an important
field of application. It might also be called parametric
postoptimality analysis. One has found an optimal and feasible
solution to an LP program, and wishes to know how sensitive that
solution is to variations in the coefficients.

As far as this relates to changes in the right-hand side or in

the (linear component of the) objective function, parametric
methods are a particularly efficient way of investigating this
problem, comparpi with independently solving a series of closely
related programmlng problems, each differing from the original
one by a variation in a few coefficients.

The other main application is in solving mathematical programming

problems for the first time. In that case, one first specifies
a related problem for which an optimal and/or feasible solution
is known or easy to obtain. One then solves this related problem,
and then converts the solution of that problem by means of
parametric adjustment, into the problem which one really wants
to solve.
273
274 CHAPTER XIII

In the computational set-up of this book parametric variation

is technically a re-entry of the Simplex Algorithm.

In that context it is obviously necessary that one can solve,

with reasonable efficiency, the general case of a problem
which is initially neither feasible nor optimal. One can then
re-enter the Simplex Algorithm for additional parametric
exercises, i.e. post-optimality analysis.

Parametric programming as a method of solving problems for the

first time, has, however been recommended by Dantzig. (Linear
programming and extensions [8] sections 11.3 and 11.4).

13.2 Parametric variation of the right-hand side of an

L.P. problem

This parametric variation problem is defined as finding a range

of solutions to the L.P. problem.

Maximise T = w'x (7.2.1)

Subject to Ax < b + AV (13.2.1)

This problem differs from the "ordinary" linear programming

problem as discussed in section 7.2, by the presence of the
parametric variation vector. The vector v gives a particular
direction of variation. Thus the constants of restrictions 3, 5,
and 6 to be increased in the proportions 2, 1, and 5 would be
indicated as v3 = 2, v5 = 1, v6 = 5, other elements of ~ being
zero. The parameter A gives the extent of variation. For A = °
we have the originally 0pecified problem. The actual change
relative to tnis original problem is then AV. Obviously, A is the
parameter which gives the method its name.

We will now investigate how gradually increasing values of A,

starting with A = 0, affect the solution of a previously optimal
and feasible solution.

Recall sections (8.1) and (8.7) concerning the partitioned

solution of a Simplex vertex, and again the summary of these
results at the beginning of section 8.11.

Once a particular partitioning (vertex) has been established,

we find that the optimality of a particular vertex is independent
of the right-hand side vector b. Reference to this vector does
not occur in the updated form of the objective function, nor in
the main body of the tableau, only in the value column.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 275

Therefore, as long as the solution remains feasible, we find

the solution-vector by substitution of b + AV, for v into the
appropriate expressions from section 8.7. To-do this, we first
express (13.2.1) in a partitioned form, the partitioning being
understood to refer to All as block-pivot.

This partitioned expression is:

(13.2.2)

Substitution of ~l + A~l for ~l

and ~2 + A~2 for ~2 in the "value column"

expressions in section 8.7 (or 11.1), yields the following
results:

x = (13.2.3)
-1
and

(13.2.4)

These expressions are equivalent to

x = (13.2.5)
-1
and
-1 -1
~2 = ~2 - A21 A ll ~l + A(~2 - A2lAll ~l) (13.2.6)

The last terms on the right-hand sides of (13.2.5) and (13.2.6)

are similar to the expressions which we obtain for the updated
form of a non-basic variable.

We shall make use of this fact to obtain the parametric column

l.e.
276 CHAPTER XIII

as an ordinary updated column, representing a non-basic variable.

Example

Consider the following* linear programming problem

Maximise T = 3x I + 7x Z + 3x3 + 7x4 -ZOx

S
Subject to xl + Xz Xs < I

xl + ZxZ - 3xS < 0

x3 + x
4
- Xs < I

x3 + Zx 4 - 3xS < 0

xl + 4x Z + x3 + 4x4 -IZx S < -Z

We first give the initial and the optimum tableau,

(tableaux 13.Za and 13.Zb, respectively).

TAELEAU 13.2 A

THE ORIGINAL PROBLEM OF A

PARAMETRIC VARIATION EXAMPLE.

NAME!! X I X 2 x 3 X 4 x 5 I! VALUE
-----------------------------------------------
S I ! ! I -I II I
S 2 ! t 2 -3 II 0
S 3 ! ! I -I II I
S 4 ! ! 2 -3 II 0
S 5 ! ! 4 4 -12 II -2
-----------------------------------------------
T ! I -3 -7 -3 -7 20 I!

TAELEAU 13.2 B

THE INI TIAL OPTIMUM OF A

PARAMETRI C VARI ATt ON EXAMPLE.

NAME I! S I S 2 S 3 S 4 5 5 II VALUE
----------------------------------------------
X I II 1.00 0.50 -1.00 1.50 -0.50 II " 00
X 2 II I • 00 -2 2 -3 1.00 II 1.00
X 3 II -1.00 I. SO I 0.50 -0.50 II 1.00
X 4 II 2 -3.00 1.00 -2.00 I II I • 00
X 5 II I • 00 -I. SO 1.00 -1.50 0.50 II 1.00
----------------------------------------------
T II I • 00 I • 00 1.00 1.00 1.00 II 0

*This problem was taken from: A.R.G. H~esterman, Special Simplex

Algorithm for Multi-Sector Problems [201
PARAMETRIC VARIATION OF THE L.P. PROBLEM 277

We now wish to know, what solutions would arise, when the

constant of the s4 - restriction was not the zero which was
actually specified, but some non-zero (or more generally, a
different) number instead. Clearly, for outward adjustment of
the sl-restriction, the non-updated variation-vector in (13.2.1)
is

[!l
and for inward adjustment we have to change the sign of the unit
vector. And we already know the updated form of that vector,
i.e. the s4 column of the tableau.

We write the equation equivalent of (13.2.1) as follows:

Ax + S = b + AV (13.2.7)

For the particular example, with v being the unit-vector

corresponding to s4' we write (13~2.7) as

xl + x2 Xs + sl 1

xl + 2x2 - 3xS + s2 0

x3 + x - Xs + s3 1
4
x3 + 2x 4 - 3x S + s4 o + A

xl + 4x2 + x3 + 4x4 -12x S + Ss -2

The corresponding updated tableau is written explicitly, as

follows:

+ Is - I 1 + l! A
xl sl + 2 2 s3 + 1 !s4 - 2SS
x2 + sl - 2s2 + 2s
3
- 3s 4 + Ss 1 - 3 A

I
IS 4 - 2SS 1 + A
x3 sl + 1!s2 + s3 + 2

x4 + 2s l - 3s 2 + s
3
- 28 4 + Ss 1 - 2 A

Xs + sl - 1!s2 + s3 - 1!s4 + !sS 1 - I! A

(13.2.8)
with a corresponding objective function equation, i.e.
sl + s2 + s3 + s4 +sS +, = 0 + A
278 CHAPTER XIII

There appears to be an extension of the rule of the smallest

quotient, which allows us to calculate a suitable value of A.

We find the greatest positive value, for which all the basic
variables remain positive, by taking the smallest quotient from
the ratios of the positive entries in the value column, divided
by minus the corresponding negative element in the variation
vector presented in updated form.

We compare 1 : 3 for x 2 ' 1 : Z for x 4 ' and 1 : I! for x 5 ' and

find A = 1/3 for Xz to be the smallest quotient.

Apparently, the vertex for which Xl, x 2 ' x3' x 4 and x5 are the
basic variables is feasible for A < 1/3.

At the upper end of this inverval, A = 1/3, the vertex ceases

to be feasible, but generally there will be a different feasible
vertex which may be obtained in one step from the parametrically
amended current one.

Thus, for A = 1/3, the system becomes

Xl + sl +
I
2SZ s3 + 1!s4
I
2s5 n
Xz + sl - 2s Z + ZS3 - 3s 4 + s5 -0

x3 sl + l!sZ + s3 +
I
2s4
I
2s5 1 1/6

x4 + ZSl - 3s Z + s3 ZS4 + s5 1/3

with a corresponding updated objective function equation, l.e.

1/3

We consider the -0 for x3 to be a negative number. When

implementing parametric variation of the right-hand side in a
practical computational context, one will put the corresponding
tableau-entry at a small non-zero negative value, e.g. -0.0000001,
and the normal "Phase I" part of the L.P. algorithm will be
activated.

Typically, we need one normal Simplex step to eliminate such a

small negative figure. At least, that is the case if an
efficient version of the Simplex Algorithm is used.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 279

Both the programmed procedure offered in section 12.3, LINP and

the Simplex procedure offered later in this chapter, apply the
dual ratio as criterion for selecting the pivotal column, when
applied to a tableau in which there is only one violated
restriction, while that one violated restriction can be made
binding with help of several alternative pivot column-variables,
all of them being "non-preferred", i. e. leading to loss of
preference-value.

We now make the steps. We first adjust the value column. This
is the actual parametric step, or, by contrast to variation of
the objective function, the primal parametric step. We now
obtain tableau l3.2c.

TABLEAU 13.2 C

DISPLACED OPTIMUM OF THE SAME

PARAMETRIC VARIATION EXAMPLE.

NAME!! S I S 2 S 3 S 5 II VALUE CL=0.33)

----------------------------------------------
X I I! I • a a 0.50 -1.00
S "
1.50 -0.50 II 1.50
X 2 !I 1.00 -2 2 ED I. a a II -0.00
X 3 ! !-I. a a 1.50 I 0.50 -0.50 II I. 17
X 4 ! ! 2 -3.00 1.00 -2.00 I II 0.33
X 5 !I I .00 -1.50 1.00 -1.50 0.50 II 0.50
----------------------------------------------
T !I I .00 1.00 1.00 1.00 1.00 II 0.33

We then need an ordinary step, to obtain a new optimal and

feasible vertex obtaining tableau l3.2d.

TABLEAU 13.2 D

THE SAME VERTEX AS IN TABLEAU 13.2 C# NOW

PRESENTED AGAIN ttl OPTIMAL AND FEASI BLE rOF.M.

NAME I S I S 2 S 3 X 2 S 5 II VALUE CL"0.33)

X I 1.50 -0.50 0.50 0.00 II 1.50

D. a a
S
x "
3
- 0.33
-0.83
0.67 -0.67 -0.33 -0.33
1.17 1.33 0.17 -0.33
II
II 1.17
X
X 5 " 1.33 -1.67 -0.33 -0.67
0.50 -0.50
0.33
0.00 -0.50 -0.00
II
II
0.33
0.50

T 1.33 0.33 1.67 0.33 1.33 II 0.33

280 CHAPTER XIII

If the constant of the s4-restriction is increased beyond the

value b 4 = 1/3, this only leads to an increase in the value
of the s4 slack-variable, and nothing else happens. In this
case, the possibility to increase the parametric variable A
ad infinitum without reducing any variable to zero, arose
because the variation-direction was to increase the constant
of a single restriction, and that restriction was no longer
binding. It can also arise because change of the right-hand
side is matched by a combination of basic variables. We may
illustrate this case by using the same unit-vector as variation-
vector, but in the opposite direction.

The initial result of a reduction of b 4 below zero is that at

A = -2/3, xl ceases to be positive.

The optimal tableau has then become:

TABLEAU 13.2 E

LEFT-DISPLACED OPTIMUM OF THE SAME

PARAMETRIC VARIATION EXAMPLE.

NAME II 5 1 5 2 5 3 S 4 S 5 II VALUE (L--0.67)

----------------------------------------------
X 1 I! I 0.50@ 1.50 -0.50 II -0.00
X 2 ! ! I -2 2 -3 1 II 3
X 3 ! I -I 1.50 1 0.50 -0.50 I! 0.67
X 4 !I 2 -3 1 -2 I II 2.33
X 5 !! 1 -I. SO 1 -1.50 0.50 II 2

T ! I I! -0.67

To eliminate xl without losing optimality, we need to bring

s3 into the list of basic variables, and we obtain our next
tableau:

TABLEAU 13.2 F

THE SAME VERTEX AS IN TABLEAU 13.2 E. NOW

PRESENTED AGAIN IN OPTIMAL AND FEASIBLE FORM.

NAME l S I S 2 X 1 S 4 S 5 II VALUE (L=-0.67)

S 3 -I -0.50 -1 -1.50 0.50 II 0

X 2 3 -1 2 0 II 3
X 3 -0 2 1 2 -1 II 0.67
X 4 3 -2.50 1 -0.511 0.50 II 2.33
X 5 2 -1 1 0 0 II 2

T 2 1.50 2.50 0.50 II -0.67

PARAMETRIC VARIATION OF THE L.P. PROBLEM 281

We will now wish to investigate the implications of even more

negative values of A. To this end it is useful to re-define
the A-variable. Or rather, we express the original variable
as the sum of the already attained value and any further
change

\ + Al (13 • 2 • 9)

for A 2/3 (13.2.8) is now rewritten as:

I
s3 + lIs - I
xl + sl + 2s2 2 4 2SS =-0 + H Al

x2 + sl - 2s2 + 2s 3 - 3s 4 + Ss 3 - 3 Al

sl + HS 2 +
I
x3 s3 + !s4 - 2SS 2/3 + Al

x4 + 2s l - 3s 2 + s
3
- 2s 4 + Ss 2 1/3 -n 1
Xs + sl - HS 2 + s
3
I
- HS 4 + 2SS 2 - H Al

sl + s2 + s3 + s4 + Ss + T -2/3 .+ Al

The updating rules for Simplex tableaux were derived from the
rules for solving systems of equations by elimination of the
variable associated with the pivotal column from all other rows.
They are therefore equally applicable to the parametric problem.

Thus, we could write our tableaux for the last parametric step
as in tableau l3.2g
(To distinguish the parametric steps from ordinary steps, we mark
the "parametric-pivots" by a square.)

TAELFAU 13.2 G

TAELEAt.:'-PRESTENTATION OF A PAPAMETRI C STEP

-L= 0
N~.ME 5 1 5 2 5 3 S 4 S 5 II VALUE -L 0

;-~-----;-----~:~~-:;-----;:~~-:~:~~-~~--------i~:~~1
x 2 1 -2 2 -3 1 II -3
X 3 -1 1.50 1 0.50 -0.50 II 0.50
X 4 2 -3 1 -2 1 II -2
X 5 1 -1.50 1 -1.50 0.50 II -1.50

T II 0
282 CHAPTER XIII

This presentation allows application of the usual rule of the

smallest quotient, to determine the leaving variable, which is
in this case xl'

Now re-adjust right-hand side and redefine parameter, to

obtain tableau l3.2h.

TAE'LEAU 13.2H

DISPLACED OFTIMCM. WITH PAFAMETRIC COLl~N.

-1.=0.67
NAME! S I 5 2 5 3 5 4 5 5 II VALl'E -Li
----------------------------------------------------
X I I 0.50G]) 1.50 -0.50 II -0.00 1.50
X 2 I -2 2 -3 I II 3 -3
X 3 -I I. SO I 0.50 -0.50 II 0.67 0.50
X oil 2 -3 I -2 I II 2.33 -2
X 5 I -I. SO I -1.50 0.50 II 2 - 1.50

T II -0.67

Make one ordinary step, introducing s3 as a basic variable to

eliminate the now negative variable Xl' The next tableau ready
for a newparametric step is as follows:

TABLEAU 13.2 I

ADJUSTED VERTEX. READY FOR THE NEXT PARAMET?l C STEP.

-1.:0.67
NAME! 5 I 5 2 X I S oil S 5 I I VAL UE -1. I
----------------------------------------------------
S 3 -I -0.50 -I -1.50 O. SO II 0 -I SO
X
X
2
3
3 -I
2
2
I 2 -I
0
I!
II 3
0.67
-
[g]
X 4 3 -2.50 I -0.50 0.50 II 2.33 -0.50
X 5 2 -I 0 0 II 2 0

T 2 1.50 2.50 0.50 II -0.67 2.50

The lowest ratio at which the next basic variable is eliminated

by further change in the parameter is now obtained for x3
at -AI = 0.67/2 = 0.33.

We now apply the same logic again. We set the attained value
of the parameter at

A = A + A
o 1
= -0.67 - 0.33 -1
PARAMETRIC VARIATION OF THE L.P. PROBLEM 283

and indicate any further change in the parameter as

After this adjustment of the right-hand side, the tableau

becomes

TABLEAU 13.2 J

DISPLACED OPTIML~,RELATIVE TO TABLEAU 13.2

-L= I
NAME I S I S 2 X I S 4 5 5 I I VALUE -L2

5 3 -I -0.50 -I -I. SO 0.50 II 0.50 -I. SO

X 2 3 -I 2 0 II 3
X 3 2 I 2 8) II - 0.00 2
X 4 3 -2.50 I -0.50 0.50 II 2. SO -0.50
X 5 2 -I 0 0 II 2 0

T 2 1.50 2.50 0.50!1 -1.50 2.50

We now make another "ordinary" step, introducing s5 as a basic

variable against x 3 . The tableau now becomes:

TABLEAU 13.2 K

NEXT FOLLOWlNG VERTEX. READY FOR THE PARAMETRIC STEP.

-L= I
NAME I S I S 2 X I S 4 X:3 !! VALUE -L2

S 3 -I 0.50 -0.50 -0.50 0.50 II 0.50 -0.50

X 2 3 -1 2 0 0 ! ! 3 0
S 5 0 -2 -1 -2 -I II 0 -2
X 4 3 -1.50 1.50 0.50 0.50 ! ! 2.50 ~
X 5 2 -I 1 0 0 11 2 0

T 2 2.50 1.50 3.50 0.50 !! -1.50 3.50

The length of the parametric step ~s rather greater this time,

L..
~ e - A2 = 21/1
2 2 = 5•

Thus, we subtract the A2 column from the value column with a

factor 5. The next tableau with x 4 just negative, is:
284 CHAPTER XIII

TABLEAU 13.2 L

THE LAST OF THE LEFT-ADJUSTED OPTIMA.

-L"6
NAME I S I S 2 X I S 4 X 3 II VALUE -L3
----------------------------------------------------
S 3 -I 0.50 -0.50 -O.SO 0.50 II 3 -0.50
X 2 3 -I 2 0 0 II 3 0
S 5 0 -2 -I -2 -1 ! ! 10 -2
X 4 3 ~ 1.50 0.50 0.50 ! ! -0.00 0.50
X 5 2 -1 I 0 0 ! I 2 0

T 2 2.S0 I.SO 3.50 0.50 11-19 3.50

Introducing s2 as a basic variable to eliminate the negative

x 4 -variable, the next tableau, ready for further parametric
adjustment (if possible), is:

TABLEAU 13.2 M

AN llNBOL'NDED PARAMETRIC TABLEAU.

-L= 6

-- -- -- - - --- --- ----- - -- ----- ---_ .. _- - --- -- -- - - -- -----

NAME ! S I X 4 X I S 4 X 3 II VALUE -L3

S 3 0 0.33 0 -0.33 0.67 II 3 -o.n

X 2 1 -0.67 I -0.33 -0.33 II 3 -D.n
S 5 -4 -1.33 -3 -2.67 -1.67 II 10 -2.67
S 2 -2 -0.67 -1 -0.33 -0.33 II 0 -0.33
X 5 0 -0.67 0 -0.33 -0.33 II 2 -0.33

T 7 1.67 4 4.33 1.33 11-19

Except for the fancyhigh artificial upper limits (see section

10.3), reference to which has been suppressed, the parametric
variation problem is now unbounded. Further increase in the
value of -A i.e. A falling below A = -6, leads only to
indefinite increases in all basic variables, no basic variable
being driven towards zero.

Our researches into the effects of changes in b 4 on the optimal

solution may be summarised in a graphical mapping.

We plot the value of the objective function T as a function of

the parametric variable A. Each vertex is represented by a
segment of uniform slope, i.e. a straight line. The slope of
this line is the amount of change in the value of the objective
function per unit of change of the parametric variable.
 PARAMETRIC V ARIAnON OF THE L.P. PROBLEM 285

It is characteristic of the class of convex problems that these

segments form a continuous "curve" with a mountain-shape. The
linear combination of any two points of the curve generally
lies below the curve itself.

For the LP problem, we may state (and prove) this property

also in algebraic terms. To do this we first need to express
the notion of a combination algebraically. Consider the range
of optimal solution values for values of A within the interval
A* ~ A :5..A**. We express the notion that A is a convex
combination of A* and A** as

A = pA* + (l-p)A** (0 ~ p ~ 1)

For a particular value of p (and hence of A) the value of the

optimal solution lS

T (A)

-1 -1
p(~~ All (~l + A*~l» + (l-p)(~i All (~l + A**~l»

(l3. 2.10)

We assume that the LP problem is non-empty for all values of A

in the range A* ~ A ~ A**.

If the same vertex is applicable for all values of A within the

interval we obviously have

T (A) = pT (A*) + (1-p) T (A"'''') (13.2.11)

the expressions within brackets (~~A~i (~l +A*~l)) and (w' A-I (b +
-1 11 -1
A'~"'~1)) being nothing else but T(A*) and T(A**). In that case
we are on a straight section, i.e. (13.2.11) describes T(A)
as a linear function of p and hence of A.

The more general relationship is (the notion of the mountain-

shape)

T(A) ~ pT(A*) + (l-p)T(A**) (13.2.12)

To prove this we invoke the Duality Theorem, i.e. we note that,

irrespective of the value of A,
-1
u' -~iAll )
-1
) (13.12.13)
u' 0 )
-2
286 CHAPTER XIII

is a feasible solution of the dual problem minimise (~' + AV' ).u,

subject to A'u > w, u > O. It follows that the optimal and - -
feasible soluti;n-of-b~h the primal and dual problems obey the
restriction
-1
,(A) 2..~iAl1 (~l + A~l) (13.2.14)

where the = equality sign will hold if All gives the appropriate
partitioning and in general the < inequality sign will hold if,
for the value of A which is currently chosen, All is not
associated with a feasible solution.

Combining (13.2.10) (applied for the initially chosen value of A

(e.g. "in the middle") with (13.2.14) applied for A = A* and
A = A**, we find confirmation of (13.3.12).
q.e.d.

We now illustrate the property by way of graphical mapping of

the relationship between, and b 4 in the example used before.

This has been done in graph 13. 2a. The points marked in this
graph with circles correspond to the vertices where, at certain
critical values of A, a change of basis took place. Note, that
the scales of the axes have been adjusted by a factor two, to
accommodate the point A = -6, ' = -19.

The objective function mapped in relation to changes in b 4 •

PARAMETRIC VARIATION OF THE L.P. PROBLEM 287

TobJ. f.
2
param.
-6 -J., -2 2
graph 13. 2 a
Illustration 0 f
the convexity
property of
parametric
variation.
-6

-10

-12

-14

-16

-18

-20

-22

In the above example, the parametric varicotion problem became

eventually unbounded in both directions. There are only a finite
number of vertices in any LP problem, hence. the parametric search
operation must sooner or later fail, as it did in this example,
or else an abnormal exit of the LP algorithm must be activated
on re-entry. For parametric variation of the right-hand side that
abnormal exit of the LP algorithm itself arises when an empty
problem has been generated.
288 CHAPTER XIII

Suppose, for example, that we had added an additional sixth

restriction to the demonstration problem, i.e.

x3 + 2x4 - 3x 5 > -10

(-x 3 - 2x 4 + 3x 5 < 10)

Then obviously, for A = -10, the now endless bottom left-hand

segment of the graph would end.

Thus, the segment with s2' s3' s5' x 2 and x5 as the basic
variables, would run from A = -10 to A = -6, instead of from
A = -00 to A = -6 as is the case, without this additional
restriction. For A < 10, the restrictions

and

would come to contradict each other.

If an infeasible solution is generated, there is no point in

exploring even higher values of A. Finding a feasible solution
after increasing A still further would in that case contradict
the convexity property of the set of vectors x, A obeying
(13.2.1). It follows that once we find the limit of the feasible
space area all problems characterised by even higher values of A
are empty problems.

The case of generating an unbounded ordinary LP problem, after

having found that the originally specified problem has a finite
optimal and feasible solution - does not arise. It would imply
an empty dual problem and finding a finite optimum for the
originally specified problem proves the dual (of which only the
objective function changes with A) to be non-empty.

If parametric variation is implemented in conjunction with the

linear programming procedure discussed in section 12.3, one would
normally end up with an empty problem, rather than the parametric
search operation itself failing. The reason is that all variables
are assumed to have implicit upper limits and if they are also
sign restricted the problem is bounded in all directions.

One would need a special instruction to signal the fact that a

fancy-high upperbound had become binding and that the problem
was in an intrinsic sense unbounded.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 289

Exercise

Repeat the operations, performed as example in this section for

changes in the constant of the fourth restriction, for the first
restriction, i.e. replace the originally specified first
restriction by xl + x 2 - Xs ~ 1 + A.

13.3 Parametric variation of the objective function in an

L.P. problem

We will indicate the originality specified objective function as

T(X) as before, the parametric component as T*(X), and the total
value of the objective function as T**(X). We are then concerned
with finding solutions, for various val~es of the parameter A,
to the following problem:

Maximise

w'X + A~'~ (13.3.1)

Subject to

A x < b (~ ~ 0) (7.2.2)

In (13.3.1) g' is the parametric variation vector for the

objective function, e.g. gl=l and other elements of ~ at zero
would mean that we investigate the sensitivity of the solution
to changes in wI' Application of the Duality Theorem converts
this problem into the following dual problem

Maximise

- b' u (13.3.2)

Subject to

(13.3.3)

Apparently we can, with the appropriate adjustments of the signs,

use the results of the previous section 13.2 to obtain solutions
for various alternative objective functions.

If the right-hand side is changed, there generally is a range of

variation within which the same vertex stays valid and only the
solution vector is adjusted. The end of this range of parametric
variation is defined by that vertex ceasing to be feasible.
290 CHAPTER XlII

If the objective function is changed, there is a range of

variation, within which the same vertex stays valid, without any
change in anything else other than the updated form of the
objective function row.

The end of that range of parametric variation is in this case

defined by the problem ceasing to be optimal at that vertex.

After a (primal) parametric variation step of the right-hand side

one obtains an adjacent vertex, if there is one, by re-entering
the Phase I part of the L.P. algorithm.

In the case of parametric variation of the objective function,

one obtains the adjacent vertex (if there is one), by re-
optimizing.

One could, if one wished, solve the problem of finding the whole
family of solutions for a particular dimension of change of the
objective function, by actually punching the dual problem and
solving the family of problems associated with the corresponding
variation of what then becomes the right-hand side. But it is
more convenient to perform search operations on a parametric
row instead.

Example

We take the same demonstration example as in the previous

section. We now carry an additional parametric row rather than
an additional parametric column. And we assume that we wish to
analyze ·the sensitivity of the solution for changes in wI.

Accordingly, our initial equations-system is:

xl + x2 Xs + sl 1

xl + 2x2 3xS + s2 0

x3 + x4 Xs + s3 1

x3 + 2x 4 3x S + s4 0

xl + 3x 2 + x3 + 4x4 l2xS + Ss -2

-3x
1
- 7x2 3x 3 7x4 + 20x S + T** 0

-xl + T* 0

This system is tabulated in the usual way, see tableau l3.3a.

PARAMETRIC VARIATION OF THE L.P. PROBLEM 291

TAELEAU 13.3 A
SET-UP TABLEAU FOR PARAMETRIC VARIATION OF
THE OBJECTIVE-FUNCTION COEFFICIENT FOR X 1.

NAME X 1 X 2 X 3 X 4 X ·5 VALUE
---------------------------------------------
S 1 1 -1 1
S 2 2 -3 0
S 3 1 -1 1
S 4 2 -3 0
S 5 4 4 -12 -2

-3 -7 -3 -7 20
T* !-1 o

The initial value of the parameter is again zero and the

corresponding initial optimal tableau is tableau l3.3b.

We mark the dual parametric step in a way which is broadly

similar to the primal parametric steR discussed in the previous
section 13.2. We put a square marker[] on the critical element
of the T* row, the element which first causes the corresponding
figure in the T** row to become negative.

TABLEAU 13.3 B

ILLUSTRATION OF THE DUPL PARAMETRIC STEP.

NAME 1 S 1 S 2 S 3 S 4 S 5 VALUE
---------------------------------------------
X 1 J 1 0.50 -I 1.50 -0.50
X 2 I 1 -2 2 -3 I
X 3 I -I 1.50 I 0.50 -0.50
X 4 I 2 -3 I -2 I
X 5 .1 1 -I. SO -1.50 0.50
---------------------------------------------
I I I I I a CL" 0)
O.sor::IJ 1.50 -0.50 I 1

This tableau indicated that T* row can be added to the complete

objective-function row, with a factor of not more than 1. for
A = 1, the shadow price of s3 changes sign.
292 CHAPTER XIII

Accordingly we make a parametric step, and obtain the following

no longer optimal tableau:

TABLEAU 13.3 C

ILLUSTRATION OF THE ORDINARY STEP WHICH

FOLLOWS AFTER A DUAt. PARAMETRIC STEP.

NAME l S I S 2 S 3 S 4 S 5 VALUE

X 1 I 0.50 -I I. SO -0.50
X 2
X 3
I
-I
-2 ®
I. SO I
-3 I
O. SO -0.50
X 4 2 -3 I -2 I
X 5 I -1.50 I -I. SO 0.50

I -0.00 I I I CL-I)
0.50 -I 1.50 -0.50 I

At this point it is useful to mention that the computational

implementation of param~tric programming offered in this book
includes a call to the ordering procedure discussed in
section 12.3. Until now, examples have often been un-ordered
by file-editing or by suppressing the call to the ordering
procedure.

This will not be done anymore now, and the next tableau,
re-ordered according to codes, is tableau l3.3d

TABL EAU 13.3 D

THE SECOND DUAt. PARAMETRIC ST t:P.

NAME X 2 S S 2 S 4 S 5 VAt.UE
---------------------------------------------
X I 0.50 1.50 -0.50 a I. SO
X 3 -0.50 -1.50 2.50 2 -I 0.50
X 4 -0. SO 1. SO -2 -0. SO 0.50 0.50
X 5 -0..50 0.50 -0.50 -0 0.50
5 3 0.50 0'.50 -I -1.50 0.50 0.50
a 2 1.50 2.50 0.50 1 CL-I)
O. so I. so 1- 0.501 - 0 1.50

This tableau indicates that the full objective function has

attained a maximum value of T** = 1, while the parametric
component is not at its maximum value.

The full objective function is (for A = 1), to maximise

T** = 4xl + 7x2 + 3x3 + 7x4 - 20x S ' and this function attains a
maximum value of 1 for Xl = I!, x 2 = 0, x 3 -x
- 4 -- x S -- 2·
I
PARAMETRIC VARIATION OF THE L.P. PROBLEM 293

The parametric component, i.e. to maximise xl' is not at its

maximum as may be seen from the negative entry in the T*/SZ cell.

The next parametric step has been marked [ ]

For A = 1 + Al = 1 + 3 = 4, the shadowprice of Sz will change
sign and a new ordinary optimizing step will be needed to
restore optimality.

Just as in the case of parametric variation of the right-hand

side, a series of parametric steps may end in two ways.

If the parametric component of the objective function is at a

maximum at the current vertex, i.e. there are no negative entries
in the T* row, the search for a column to be marked [ ] as the
next incoming variable will fail.

13.4 Parametric adjustment of mixed systems

In this section we survey the complications which arise if

parametric change/sensitivity analysis is practised in conjunction
with a system which contains equations as well as inequalities,
variables with and without non-negativity restrictions.

First of all we must clarify the definition of a parametric

variation step.

The parametric variation step finds the smallest (change in the)

value of the parameter, which causes the current vertex to be
no longer optimal and feasible.

Therefore, a parametric change of the right-hand side vector b,

which causes a variable to which no non-negativity restriction
applies, to become negative, it not the end of a parametric
step.

Example

Consider again the problem we first introduced in section l3.Z,

but let us now assume that xl is an "absolute" or "free" variable,
whereas x z , x 3 ' x 4 and Xs are restricted to non-negative values
only.

In that-case xl may not be eliminated from the list of basic

variables, and the original vertex stays valid until another
variable than xl is eliminated. We again consider the parametric
reduction of the constant of the fourth restriction. The
resulting parametric step has been marked in tableau l3.4a.
294 CHAPTER XIII

TAELEAU 13.4 A

XI IS NOT ELIGEAELE AS (PARAMETRIC) PIVOTAL ROW.

(L= 0)
NAME! 5 I 5 2 5 3 5 4 5 5 !I VALUE -LO
----------------------------------------------------
X I I 0.50 -I 1.50 -0.50 ! I
I. SO
X 2 I -2 2 -3 I II -3
X 3 -I 1.50 I o. so -0.50 II ~
X 4 2 -3 I -2 I I! -2
X 5 I - I• so I -I. so 0.50 II so
-I.
----------------------------------------------------
T I I 0

Here Xl is allowed to become negative, and the length of the

parametric step is instead determined by the non-negativity of
x 3 . Implementation of this step leads to tableau 13.4h

TAELEAU 13.4 B

DISPLACED OPTIMUM WITH A NEGATIVE

NON-ZERO VALUE FOR A FP-EE VARIABLE.

(-L=2)
NAME 5 I S 2 5 3 5 4 5 5 I I VAL UE -L I
----------------------------------------------------
X I I 0.50 -I 1.50 -0.50 II -2 1.50
X 2 I I -2 2 -3 I II 7 -3
X 3 !8) 1.50 I 0.50 -0.50 II -0.00 0.50
X 4 I 2 -3 I -2 I fI 5 -2
X 5 I I -1.50 I -1.50 0.50 II 4 -1.50

T II -2

As before, an ordinary step will be need to find a new optimal

and feasible solution. The x3-variable is eliminated and Xl
stays at its (negative) value of -2.

TAELEAU 13.4 C

NEW VERTEX Oil TH NEGA1"l VE-VALUED FREE VAPIAE·LE)

THE PARAMETRI C PPOBL EM IS NOW UNSOt'NDED. (R E-ORCERED)

(-L= 2)
NAME X 3 5 2 5 3 5 4 S 5 II VALt'E -LI
------------------------------------------------------
X I I 2 2 -I II -2 2
X 2 I -0.50 3 -2.50 0.50 II 7 -2.50
X 4 2 3 -I II 5 -I
X 5 I 2 -I II 4 -I
S I -I -1.50 -I - 0.50 0.50 I! 0 -0.50

T 2. so 2 1.50 0.50 II -2 I. so
PARAMETRIC VARIATION OF THE L.P. PROBLEM 295

At that vertex, the parametric variation problem becomes

unbounded, unless upper limits on the variables are considered.

This vertex is valid for all values of A < - 2.

The same logic applies to equations in the case of parametric

variation of the objective function.

The shadow-prices of equations are allowed to change sign.

Therefore the same vertex stays optimal, until the shadow-price
of a non-negativity restriction or an inequality changes sign.

This restrictive definition of the parametric step may imply a

loss of information, compared to what the user would like to
get out of parametric sensitivity analysis.

One may have defined a variable as not restricted in sign, but

would nevertheless wish to see a record of the values of other
variables at the point where a particular variable changes sign.

Similarly, one may have defined a restriction as an equation, but

wish to record the shadow-prices of other restrictions associated
with the value of the parameter, at the point where the shadow-
price of an equation changes sign. One could obviously solve
this problem by defining the parametric step in terms of a
variable (or shadow-price) attaining a zero value. Definitions
are to some extent a matter of convention and convenience not
of a priori logic.

The computational implementation of parametric sensitivity

analysis (to be discussed in the next section), uses the more
restrictive definition given above, and it is useful to mention
that this problem may be circumvented.

If one wants a record of a variable to become zero, one should

declare it as a sign-restricted variable, and declare minus the
same variable as another variable, on the lines of the opening
paragraph of section 10.1. The same approach may be followed
with respect to equations.

Example

Consider again the same problem, where it it now assumed that

the third restriction is an equation, but that we wish to keep
track of any change in the sign of its shadow-price during
parametric sensitivity analysis. The direction of parametric
adjustment is assumed to be the same, i.e. increased preference
for xl'
296 CHAPTER XIII

We re-formulate the original problem, as summarized in

tableau l3.4d.

TAELEAU 13.4 0

SET-UP TAELEAV FOR PARAMETRIC VARIATION OF THE

OEJECTIVE-FUNCTION COEFFICIENT FOR X I. WITH
~qUATION-DUPLICATION OF THE S3 RESTRICTION.

NAME I X I X 2 X 3 X 4 X 5 II VALUE

S I 1 -1 II I
S 2 2 -3. II 0
S 3 I I -1 II 1
S 4 -I -1 1 II -0.999
S 5 1 2 -3 II 0
S 6 4 I 4 -12 II -2
T -.3 -7 -3 -7 20 II
T* I - I II o

Restrictions 3 and 4, i.e.

X3 + x 4 - Xs < 1 and

x3 + x 4 - Xs > 0.999

amount for practical purposes to

X3 + x 4 - Xs 1

i.e. the old third restriction re-defined as an equation. The

new restrictions Sand 6 are the old restrictions 4 and S.

It is necessary to put a small, technically non-zero margin

between the two restrictions. If rows 3 and 4 are exactly the
same except in sign, a small calculation error may lead to an
empty problem.

Thus

-X 3 - x 4 - Xs < -1.000000000000001

are contradicting restrictions. Errors of this order of magnitude

would easily arise from rounding and the problem might be found
empty. The figure of 0.001 for the margin as chosen in this
example is st ill unduly high. I t was taken in order to highlight
the fact that it is there at all. In practice, a much smaller
margin will be sufficient.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 297

The corresponding central optimal and feasible solution-tableau

is given below with the first parametric adjustment step, i.e.
the shadow-price of the third restriction indicated in the
usual way 0
TABLEAU 13.4 E

CENTRAL OFTIMUM, WITH DUAL PARAMETRIC STEP.

NAME ! S 1 S 2 5 3 S 5 S 6 II VALUE
----------------------------------------------
1 0.50 -1 1.50 -0.50 II 1
X 1
X 2 1 -2 2 -3 1 II 1
X 3 -1 1. SO 1 0.50 -0.50 II 1
X 4 2 -3 1 -2 1 II I
X 5 1 -1.50 1 -1.50 0.50 II I
5 4 1 II 0.001
----------------------------------------------
1 1 1 1 1I a
T
T* o. soD 1.50 -0.50 II 1

The parametric adjustment step is now made and we obtain a

tableau which requires re-optimization, tableau 13.4f.

TABLEAU 13.4 F

EQUATION-DUPLICATION SIGNALS THE POINT AT

WHICH THE SHADOWPRICE CHANGES SIGN,THEN
LEADS TO A ZERO LENGTH ORDINARY STEP.
NAME l S I S 2 S 3 S 5 S 6 II VALUE

X 1 1 0.50 -1 1.50 -0.50 II 1

X 2 1 -2 2 -3 1 II 1
X 3 -1 1.50 1 0.50 -0.50 II 1
X 4 2 -3 1 -2 1 II 1
X 5 1 -1.50 1 -1.50 0.50 II 1
S 4 CD I I 0.001
----------------------------------------------
2 1.50 -0.00 2.50 0.50 II 1 (L-l)
1 0.50 -1 1.50 -0.50 II 1

Note again that, whereas the figure of 0.001 has been used to
emphasis that there is no actual degeneracy, one should think in
terms of an E -difference, hence the heading of tableau 13. 4f,
which speaks of a step of zero length.
298 CHAPTER XIII

The "ordinary" step which is now needed to regain optimality

has not much intrinsic significance, but signals the fact that
the shadow-price of the original third restriction has changed
sign. Rounded to two decimals, the tableaux 13.4f and 13.4g
below, are only distinguishable in the s3/s4 columns.

TABLEAU 13.4 G

NEV OPTIMAL AND FEASIBLE SOLUTIONI VITH ONLY AN

INSIGNIFICANTLY SMALL CHANGE IN THE VALUE COLUMN.

NAME I 5 I 5 2 S 4 S 5 5 6 II VALUE

X 1 I 0.50 1 1.50 -0.50 II 1.001

X 2 1 -2 -2 -3 1 ! I 0.998
X 3 -1 1.50 -I 0.50 -0.50 II 0.999
X 4 2 -3 -1 -2 1 ! I 0.999
X 5 1 -1.50 -I -1. SO O. SO I! 0.999
5 3 o o -0 II 0.00 I

2 1.50 0 2.50 0.50 II 1. 001 (L= 1)

1 0.50 I 1.50 -0.50 !! 1.001

At this point s6 is the next variable for which the shadow-price

is to change sign. Clearly that would have happened in the first
place, if the s3 restriction had formally been treated as an
equation. And this would be the case for A = 2 in either case.

13.5 Treating the parameter as a variable

To cope efficiently with upper bounds, we introduce a method of

making parametric steps which is somewhat different from the one
outlined in the two previous sections.

There is no conceptual or definitional problem concerning upper

limits. Upper limits on specified variables are restrictions, and
they may become binding. Or parametric variation of the objective
function may cause the shadow-prices u[ upper bounds to become
negative, and the upper bounds may cease to be binding, and the
corresponding variables may again be represented by normal
entries in the list of basic variables. However, the practical
computational implementation of parametric sensitivity analysis
makes use of an adaptation of the linear programming procedure,
and certain special complications arise from the desirability to
use basically the same algorithm for parametric as well as for
"ordinary" steps.

Recall the double search operation for the pivot row, i.e. for
the elimination of a variable or alternatively the slack of its
upper bound restriction, which was discussed in section 10.3.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 299

We could duplicarl:! this search operation in connection with

parametric variation of the right-hand side. But that is not the
end of the complications. The re-entry of the linear programming
procedure can cope with a minus zero (e.g. -0.000001), in the
value column, indicating a violated restriction, but not with
negative entries in the updated upper bounds column. The point
is simply that the "Phase I" part of this procedure addresses
itself only to negative elements in the main value column. If
someone initially supplied a negative upper limit this would in
general lead to malfunctioning of the procedure as well.

An adaptation of the LINP procedure (developed in the context

of integer programming), in which Phase I also deals with
negative upper limit distances, is in fact offered in this book
as well. However, the solution opted for here, is to introduce
the parameter into the basis as a variable, and to re-define
its value afterwards, as being -0.0000001, causing its
elimination by the normal "Phase I" block of the basic LP
algorithm.

To actually make the step in which the parameter is brought into

the basis as a variable, is necessary, only if the leaving
variable is an upper limit distance. The method is, however,
generally valid, and is illustrated here, without reference to
upper limits.

To illustrate this version of the algorithm, we tabulate

(13.2.7), with v' = (0,0,0,-1,0), - for the same example as
illustrated in tableau l3.2g and following-,
Note the change in the sign of the unit vector due to rn .ng
the term AV to the left-hand side.

TABLEAU 13.5 A
SET-UP TABLEAU WITH A PARAMETRic ACTIVITY
(L= 0)
NAME! XI X2 X3 Xii X5 t LO ! I VALUE ! D15T
------------------------------------------------------
51
52
I
2
-I I
-3
II I
I
--
X
!I 0 X
53
S4
I
2
-I
-3
I - I!
II
I
0
X
X
55 4 4 - 1.1 -2
-12 I X
------------------------------------------------------
T -3 -7 -3 -7 20 0 !I X
BOUND! 100 100 100 100 100 I X II X X
300 CHAPTER XIII

As mentioned in section 12.3 (in the procedure-text itself), the

number 100 represents a "fancyhigh" number where no intrinsic
upper limit is intended, and the computational implementation
normally uses 1 000 000, a million.

Compared with section 13.2, where A was on the right-hand side,

this presentation implies an implicit change of the sign of the
parametric column.

In short, this example concerns the s4 restriction being pushed

parametrically inwards.

During the "normal" optimizing phase, the parametric column is

exempt from search operations, but not from updating operations.
The resulting initial optimum is eiven below in tableau l3.5b.

TABLEAU 13.5 B

INI TIAl OPTIMt..'M. WI TH UPCATED. BUT lJNACTl VATED PARAMETRIC

ACTIVITY.

(L= 0)
NAME! S 1 S 2 S 3 S 4 S 5 LO 11- VALUE r,. ST
-----------------------------------------------------------
X 1 1 0.50 -1 1.50 -0.50 [hl] II 99
X 2 1 -2 2 -3 1 -3 II 99
X 3 -1 1.50 1 0.50 -0.50 0.50 II 99
X 4 2 -3 1 -2 1 -2 II 99
X 5 1 -1.50 1 -1.50 0.50 -1.50 II 99

T 1 1 1 1 1 1 II o X
BOUtIDI X X X X X X II X X

Note, that there is no general rule that the shadow-price of

the parametric variable is positive.

The parametric variable becomes incoming variable, irrespective

of its shadow price. The parametric step is now maae as a normal
step, leading to tableau l3.5c.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 301

TABLEAU 13.5 C

THE PARAMETRIC ACTIVITY HAS ENTERED THE BASIS.

(L"0.67)
NAME! SIS 2 S 3 S 4 S 5 X 1 I I VAL UE 01 ST
~~------~:~;--~:;;-:~:~;-------:~:;;----~:~;-~;---~:~;----;---
X 2 3 -I 0 2 II 3 97
X 3 -1.33 1.33 1.33 -0.33 -0.33 II 0.67 99.33
X 4 3.33 -2.33 -0.33 0.33 1.33 II 2.33 97.67
X 5 2 -I 0 0 1 II 2 98

T 0.33 0.67 1.67 0 1.33 -0.67!1 -0.67 X

BOL'NDI X X X X X 100 II X X

The actual variation of the problem arises when A ~s redefined

to be minus zero, and we re-enter Phase I to make an ordinary
step.

TABLEAU 13.5 0

TABLEAU VITH ARE-DEFINED FARAMETRIC VARIABLE.

(L~0.67)
NAME! SIS 2 S 3 S 4 S 5 X 1 II VAL UE DI ST
~~---~--~:~;--~:;;~--;----:~:;;----~:~;-~~--:~:~~----;---
X2 I 3 -1 - - a 2 II 3 97
X 3 I -1.33 1.33 1.33 -0.33 -0.33 II 0.67 99.33
X 4 I 3.33 -2.33 -0.33 0.33 1.33 II 2.33 97.67
X 5 2 -1 0 0 1 II 2 98

T I 0.33 0.67 1.67 0 1.33 -0.67 II -0.67 x

BOtNDI X X X X X 100 It X X

The question arises, whether the procedure outlined in this

section, is fully equivalent with the one outlined in section
13.2. This is .indeed the case. Both versions of the parametric
variation algorithm consist of pairs of steps, parametric
variation steps and ordinary steps, and despite the different
presentation of the parametric variation step, we develop
after each pair of steps, substantially the same tableau
by either method.
302 CHAPTER XIII

When we actually make the step marked in tableau l3.5d, we

develop substantially the same tableau as in section 13.2,
i.e. tableau 13.2 i, the ordering and the classification of A
as a variable being the only difference. The selection of the
same incoming variable in the ordinary step (s3 in the example),
by both versions of the algorithm, is systematic, as may be
shown by expressing the dual ratio of a tableau as developed
according to the rules of this section in the coefficients as
they would arise ~n the version of the algorithm discussed in
section 13.2.

t"<
m+l,j = _
t* . .
~,J
(tm."i - tm+ 1 ,n • t.~ ,J. It.~ ,n)
t. . It.
~,J ~,n

= - (tm.',i · 'i,n -tm."n) (13.5.1)

(j = 1, ... n-1)

In (13.5.1), t l ' is the objective function row/incominR

m+ ,J
variable column cell figuring in the search operation for making
the ordinary step, made according to the rules of section 13.2
(j=3, m=5, t6 3=1 for the L/s 3 cell in tableaux l3.2g and l3.2h),
ti,j is the corresponding pivotal row element (-1 for x Is 3 )
cell, i=l), the corresponding starred expressions are t~e updated
cells, following the parametric step made according to the rules
of this section (i.e. tg 3 = 1.67 and tt 3 = -0.67 in tableau
13.5d). ' ,

The index n refers to the parametric variable, i.e. assumes the

ordering used in this section.

The second expression in (13.5.1) gives the division of the two

updating expressions, the third expression is a simplification
of the second one.

Since t. is the pivot in the parametric step we have t. > O.

It fol16~2 that the expressions ~,n

t
m+ 1 ,J. tm+l,j * ti,n
and -
t*
m+l,j
t. . t .. t* . .
~,J ~,J ~,J

all give the same ranking, at least as far as they are positive.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 303

Eligible pivots are in fact found, only in those columns for

which all these expressions are positive in both tableau-
presentations. There is no change of sign between ti J" and t* "'
, ~ ,J
but the transition from t 1" to t* 1 " needs further
discussion. m+ ,J m+ ,J

It is true that the parametric step, performed according to the

rules of this section may lead to the emergence of negative
entries in the objective function row, thereby causing the
expression given by (13.5.1) to change sign. However, such a
change in the sign of an element of the objective function row
is of necessity associated with a positive element in the
pivotal row, i.e. related to a variable which is not eligible
as incoming variable (in a tableau which represents a non-
feasible solution), in either version of the algorithm.

The dual ratio is therefore the effective column-selection rule

in both versions of the algorithm, and (13.5.1) indicates that
both dual ratio search operations will come up with the same
incoming variable. The steps are effectively the same as in
section 13.2, and it follows that any negative entries in the
objective function row which may be developed by the parametric
step, when the rules of this section are followed, will again
disappear when the ordinary step is made. The stated equivalence
property includes the case j=n, to which (13.5.1) is not
applicable e.g. xl in the example. The tt n element is the
reciprocal of the parametric pivot hence positive and not
eligible as pivot in a Phase I tableau. Since the same
vertex is developed by both methods (e.g. tableau l3.2i) and
the use of the dual ratio in section 13.2 guarantees finding a
new optimum (unless the problem has become empty), this will
also be so when the rules of this section are applied.

Note that no amendment of the basic Phase I search operations as

laid down in sections 11.4 and 12.3 is needed. These rules
already ensure that the dual ratio is the effective column-
selection criterion in the situation discussed in section 12.3
and, for the reasons given above also if the parametric variable
is re-defined after entering the basis.

While the above observations merely prove the equivalence of the

two methods, the rationale of the method outlined in this section
is that parametric steps can be made as "ordinary" steps even if
the leaving variable is an upper limit distance and that
essenti,ally the same programme code can be used to perform the
search operation.

In the case of variation of the objective function, we have a

parametric restriction, which is adjusted to requir~ that the
parametric component of the objective function is slightly more
304 CHAPTER XIII

than it is in the current vertex. Phase I of the linear

programming algorithm is then re-entered, with recognition of
a small -0 entry as a violated restriction. This -0 entry is
offered to the search-loop of Phase I, irrespective of the true
value of the parametric component of the objective function,
which is restored just before a step is actually made.

Example

Consider the same numerical example as discussed in section 12.3.

We write the initial tableau as in tableau l3.5e below

TABLEAU 13.5 E

SET-UP TABLEAU WITH THE PARAMETRIC ROW

SLOTTED AS THE LAST RESTRICTIONI INDEX M.

NAME I X I X 2 X 3 X II x 5 II VALUE

S I I -I II I
S 2 2 -3 ! I 0
S 3 I I -I ! ! I
S II -I -I I II -I
S 5 I 2 -3 ! I 0
S 6 II I II -12 II -2

I -I II o
T !-3 -7 -3 -7 20 II

The corresponding initial optimum tableau is now presented as in

tableau l3.5f below

TABLEAU 13.5 F

INITIAL OPTIMUM TABLEAU .. ITH THE PARAMETRIC

ROW SLOTTED AS THE LAST RESTRICTIONI INDEX M.

NAME I S I S 2 S 3 S 5 S 6 II VAr.UE
----------------------------.-----------------
x I I 0.50 -I 1.50 -0.50 II I
X 2 I -2 2 -3 I II I
X 3 -I I. SO I 0.50 -0.50 II I
X II 2 -3 I -2 I II I
X 5 I -1.50 1 -1.50 D.SO II 1
S II I II 0.00

0.50 -I 1.50 -0.50 II

T 11 o
PARAMETRIC VARIATION OF THE L.P. PROBLEM 305

The parametric adjustment of the objective function is now

made effective by requiring the T* restriction to be binding.
We artificially activate Phase I, i.e, we declare the T*
restriction to be violated for the purpose of search operations,
but restore the correct entry before the resulting step is
actually made. The resulting parametric step is marked
in tableau l3.5g below.

TAEJ...EAl! 13.5 G

THE DUAL PARAMETRIC SEARCH. ACTIVATED BY

A JUST-VIOLATED PARAMETRIC RESTRICTION.

NAME I S I 5 2 S 3 S 5 S 6 II VALUE
----------------------------------------------
I 0.50 -I 1.50 -0.50 ! I
X I
X 2 I -2 2 -3 I I!
X 3 -I 1.50 I 0.50 -0.50 II
X II 2 -3 I -2 I II I
X 5 I - 1.50 I -1.50 0.50 II 1
S II I II O. 00
----------------------------------------------
1* 0.50D 1.50 -0.50!1 -0.00
----------------------------------------------
T II Q

The requirement that the parametric component of the objective

function increases by an ~-amount (printed as 0.00 in the tableau)
is either attainable in one step, and selection of the incoming
variable according to the dual ratio may be left to a "normal"
Phase I search operation or alternatively if the parametric
component of the objective function is already at its maximum,
an empty problem is indicated.

At this point it may be observed that an upper limit is not an

acceptable leaving variable in a dual parametric step. Either
the parametric restriction's "slack variable" is signalled as
the leaving variab~e, or no parametric step can be made at all,
the parametric component of the objective function being already
at its maximum.

This consideration makes it possible to proceed, from this point

onwards, very much in the same way as in section 13.3, i.e.
update the T** row only, place a -~ entry in the i** row/
incoming variable column cell, and make the corresponding
ordinary step.

The only difference is that the T*-row is slotted in position

m as an "ordinary" row of the tableau, permitting the use of the
"normal" Phase I search operations in order to identify the
incoming variable and the appropriate dual ratio.
306 CHAPTER XIII

The parametric step is, however, not made in the usual

"normal" LP sense, we only update the T**-row, to obtain
tab leau l3. Sh.

TABLEAU 13.5 H

ORDINARY STEP, FOLLOVING THE DUAL PARAMETRIC

STEP. (EQUIVALENT TO TABLEAU 13.3 C)

NAME ! S I S 2 S 3 S 5 S 6 , I VALUE

X , I. SO -0.50 ,,
----------------------------------------------
X 2
I 0.50 -I
I -2 2 -3 1 ,,
X
X
3
4
- 1
,2
I • SO
-3
I
I
O. SO -0.50
-2 1 ,,
! ,

X 5 -1. SO 1 -1. SO 0.50 , !

S 4 0 CD 0 -0 , !
----------------------------------------------
T* O. SO -1 1. SO -0.50 ! ,

1.50 -0.00 2.50 0.50 II

The method of carrying on from this point onwards will be obvious.

13.6 Computational implementation of parametric LP

Before addressing ourselves to the details of the computational

implementation, it is useful to recapitulate the salient features
of the parametric linear programming algorithm.

Parametric steps corne in pairs, i.e. a parametric adjustment step

and, after re-definition of the problem, an ordinary step, to
regain an optimal and feasible solution.

For variation of the right-hand side the parametric variation

step may, or may not consist of entering a parametric activity
as a basic v~riable irrespective of its shadow-price, in which
case the adjustment redefines the value of that variable as minus
zero, and the ordinary step eliminates the parametric variable.

For variation of the objective function, the parametric variation

step consists of finding an incoming variable which increases
the parametric component of the objective function, at the lowest
relative loss in terms of the value of the specified objective
function.

The search operations for variation of the right-hand side and

the objective function are sufficiently similar to each other
and to "normal" LP search operations, to justify their
integration in an adaptation of the LINP procedure.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 307

The parametric variation step is initiated by temporarily

putting a small negative entry in the tableau, this minus zero
(actually -0.0000001) is replaced, just before the step is
actually made, by the "true" figure.

To this end, the linear programming code from section 12.5 and
the LINP procedure from section 12.3 were suitably amended.

These amendments cover the following points: firstly, two rather

than one point of call to the linear programming procedure are
required, one for calculating the initial optimal and feasible
solution, one for the variation exercise. Also, the linear
programming procedure was actually re-named i.e., LINP still
is the version from section 12.3, and we now need SIMP for
simplex procedure.

The simplex procedure differs from the original linear

programming procedure on the following points: there are two
additional procedure-parameters of type integer, called PR and
PC.

These variables communicate the presence of a parametric

A-variable or a parametric restriction from the calling main
programme to the simplex procedure. Inside the simplex
procedure, reference to these variables occurs in particular
in the loops which control search operations.

PR stands for parametric row, and this variable will be one

when there is a parametric T* row and otherwise zero; PC stands
for parametric column and this variable will be one when there
is a parametric column and otherwise zero.

On normal entry of the procedure (REENTRY = 0), the parametric

row/column are not included in search operations.

Thus for REENTRY = 0, PR = PC = 0, the action of the simplex

procedure is substantially the same as for the LINP procedure.

For REENTRY = 0, PR = 0, PC = 1, the last mth "normal" column

of the tableau is updated in the usual way, but it is not
considered as pivot-column. In other words, the A-column is
updated, but does not enter the list of basic variables. This
is meant for the initial calculation of an optimal and feasible
solution, while parametric variation of the right-hand side is
to be analyzed with help of the re-entry call.

For REENTRY = 0, PR = 1, PC = 0, the last nth "normal" row is

the parametric restriction and is not included in the search
operations.
308 CHAPTER XIII

Besides these amendments to the search operations we also need

some additional control loops to initiate parametric steps in
the first place, to regulate the succession of a parametric
variation step by a "normal" step, and to signal the various
exits of the parametric adjustment algorithm, and to decide
whether the parametric step is made by full tableau-updating,
or by vector-adjustment only.

Parametric steps are initiated by the calling main programme.

In the case of parametric variation of the right-hand side,
the indication of the parametric A-variable as a pivot-column
is set by temporarily substituting a negative number for its
shadow-price, and the exclusion of A as pivot-column variable
is removed by entering the procedure with REENTRY = 2.

In the case of variation of the objective function the similar

adjustment is substituting a very small negative number for the
slack of the parametric restriction. This ensures that the
pivotal column is selected by the dual ratio criterion.

The normal search-loops (which are largely the same as in the

LINP procedure), now establish which is the correct parametric
variation step. A further special control loop then restores
the numerical content of the problem as it was before the
parametric re-entry. This is done at the start of making the
step, just below the label MAKE THE STEP.

To make (or attempt to make) both a parametric adjustment step

and the ordinary step, on the basis of a single re-entry call,
there is a parametric loop in the "check for status", part of
the procedure. In the case of a dual parametric step the similar
control loop occurs earlier in the procedure, inhibiting tableau-
updating in the parametric step itself.

Both loops control the actual adjustment of the problem, and a

"GOTO" instruction to Phase I, as well as, where appropriate
an instruction tc exit.

Only two steps are allowed on parametric reentry.

The A-variable, once it has again been eliminated should not

re-enter the list of basic variables before the parametric step
has been reported to the main programme. Yet its shadow-price
may be negative, i.e. it could be that the parametric variation
of the right-hand side was in a direction which increases the
objective function. During the normal entry call to the simplex
procedure (REENTRY = 0), this variable was slotted in the last
column, and could not become a basic variable because the end-
index of the column-search loop is adjusted. This method of
protecting the A-variable against undesired activation does not
 PARAMETRIC VARIATION OF THE L.P. PROBLEM 309

work during the re-entry-call itself. To ensure exit to the main

programme, it is therefore protected by allowing only two steps
on parametric reentry.

Some fairly obvious minor changes in the code relate to the

upper limit on the A-variable: we do not recognize an upper
limit on A.

Unbounded parametric problems may then appear in two forms.

One or more constants of amply fulfilled restrictions may be

adjusted outwards, in which case no pivotal row may be found
in the A-column.

This will be signalled by the "unbounded" exit of the Simplex

procedure itself.

We may however, also have what might be called "lack of

meaningful boundedness" i.e. a "fancyhigh" non-meaningful
upper limit or one of the elements of x becomes binding.
Formally this is not unboundedness at all, and it is possible
to generate further reentry calls, until an empty problem is
generated.

We now first of all list the main programme, which is an

adaptation of the main LP programme listed in section 12.5.

TEXT-LISTING OF THE PARAMETRIC LP FROGRP~ME.

'BEGIN' 'INTEGER' M,N,NAV,NEQ, FR, PC, I .J. REENTRY;

'PROC ECUP. E' S I MP( T. ~l, N. N EQ. NAV. RO\>.'LST. COLLST. FR •. PC. REEN TRY);
'ARRAY' T; 'IlnEGER' ~l,N.NE".fIAV.PR.PC.EEENTRY; ,
'INTEG ER' 'ARRAY' RO\..'LST. COLLST; 'ALGOL ';

'PROCEDURE' ORDR(T,M.N.ER.RH.ROWLST.COLLST);
'APRAY' Tj 'ItJTEGEIi' M.N,ER.RH; 'ItJTEGEP' 'ARRAY' ROWLST.COLLSTJ
'ALGOL·';

'PROCEDURE' MATICMATR,MB.NB.FP..FC)j
'ARRAY'MATR; 'INTEGER' MB,NE.FR,FC; 'ALGOL';

'PROC EDULE' TASO (MATR. M. N. sn. s C, RH, ER. RO WL ST, COLL ST);
'ARRAY'MATR; 'INTEGER' M.N.SR.SC.RH,ER;
'INTEGER' 'ARRAY' ROlr:LST.COLLST; 'ALGOL';
310 CHAPTER XIII

• COMMENT·
LINEAR PROGRAMMING BY THE SIMPLEX ALGORITHM.
WITH POST-OPTIMAL PARAMETRIC VARIATION.
FOR DETAILS OF THE ALGORITHM SEE THE TEXT OF THE SIMP PROCEDURE.
PRESENTATION OF DATAl
FIRST THE NUMBER OF RESTRICTIONS AND DUMMY-RESTRICTIONS. I.E. M.
INCLUDING THE PARAMETRIC ROW. IF ONE IS THERE.
THEN THE NUMBER OF VARIABLES AND DUMMY-VARIABLES. I.E. N.
INCLUDING THE PARAMETRIC COLUMN. IF ONE IS THERF..
FOLLOWED BY THE NUMBER OF EQUATIONS. NEQ.
FOLLOWED BY. NAV. THE NUMBER OF VARIABLES
TO WHICH THE TACIT (NON-NEGATIVITY) RESTRICTION DOES NOT
APPLY.
THEN PUNCH PRo
WHICH IS ZERO IF NO PARAMETRIC VARIATION OF THE OBJECTIVE
FUNCTION IS ASKED FOR. AND ONE IF IT IS ASKED FOR.
FOLLOWED BY PC.
WHICH IS ZERO IF NO PARAMETRIC VARIATION OF THE RIGHT-HAND
SIDE IS ASKED FOR. AND ONE IF IT IS ASKED FOR.
THEREAFTER PUNCH EACH ROW OF THE COMPOSITE MATRIX
A B
W 0
U 0

TO REPRESENT A.X < OR ~ B.

MAXIM -W·. X
AND X < OR = U

THE NUMERICAL CONTENT OF THE PARAMETRIC ROW IN NON-UPDATED FORM.

I F PRESENT.
SHOULD BE SUPPLIED AS THE LAST ROW OF A. THE ASSOCIATED
LAST ELEMENT OF B BEING ZERO.
THE NUMERICAL CONTENT OF THE PARAMETRIC COLUMN. IF SUPPLIED.
SHOULD BE PUT IN THE LAST COLUMN OF A. WITH INVERTED SIGN.
THE CORRESPONDING ENTRIES IN THE TARGET-ROW AND UPPERBOUNDS
VECTOR BEING ZERO.

MI-READJ NI=READJ NEQI=READJ NAVI-READ.

PRI"READ' PCI"READ'
REENTRYI-O.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 311

'EEGIN' 'ARRAY' TACI:M+2,I:N+2);

'I tlTEG ER' 'ARRAY' RO"'l. C1: M], COLL [II N);
MATICTA,M+2,N+I,0,0);
SIMPCTA,M,N,NE~,NAV,R01t1L,COl.l.,PF.,PC,REENTRY)J

PRINT RESUL.TSI
'IF' N < 14 'DR' M+N < 40 'THEN' TABOCTA,M,N,0,0,2,2,RO~~,COl.l.)
'El.. S E' 'EEGI N'
TABOCTA,M,O,O,N,O,I,RO~~,COl.l.);
TASOCTA, O,N,M, 0, I, O,RO~~,COl.l.)J 'END';

CHECK fOR INTRINSI C BOUNDEDNESS:

',fOR' J:=I 'STEP' I 'UNTIL' N '00'
'IF' CoLLCJl > 10000 'AND' TACM+2,J] > 999999 'THEN' 'BEGIN'
SIGNAL INTRItJSIC UNEOUNDEDNESS:
N£1,.~INE( I);
\.'RI TET DCTC ' C ' fArJCYH I GH %UPPER%LI MI T%BI N[)l NG') , );
NEI."LINE( I);
'GoTo' END OF FARALP; 'END';

'If' FR:O 'AND' FC:O 'THEN' 'GoTo' END Of PARALP;

'If' REENTRY' 0 'THEN' 'GoTo' END Of PARPLP;

SET REENTRY:
RE£N TRY : : 2;

' I f ' PC:! 'THEN' 'BEGIN'

'COMMENT'
NOV ACTIVATE THE RIGHTHAND SIDE VARIATION ACTIVITY;
TACM+2,N] : : TACM+I,N); TACM+I,N]:: -0.0000001; 'END';

' I f ' PR=I 'THEN' 'BEGIN'

'COMMENT'
NO~ ACTIVAT~ TARGET R01t1 VARIATION 'VECToR;
TACM,N+2]: =TACM,tH!); TACM,N+! l: "-0'.00000000 I; 'END';

REORDER PARAMETRIC VECTORS T01t1ARI:S END .of TABLEAUI

'fOR' J:"I 'STEP' I 'UNTIL' N 'DO'
'IF' COl.l.[Jl " N 'AND' PC:I 'THEN' 'COl.l.CJ] p. 20000;

'IF' PC=I 'AND' 'NOT' COl.l.CN]=N 'THEN'

ORDRCTA,M,N,2,2,Ro~"L,COl.L);
' I f ' CoLLCN],,20000 'AND' PC" I 'THEN' COLLCN] :: N;
' I f ' PR:I 'P.NO' 'NOT' RO\.~CM]=!OOO+M 'THEN'
OROr.CTA,M,tJ, 2, 2, POO"'l., COLl.);
'GOTO' PRINT RESULTS;

END Of PARALP:

'END'; 'END'
312 CHAPTER XIII

The above programme-text differs from the one listed in

section 12.5, on some additional points besides the ones
already mentioned.

There is the obvious need to declare, and to read the values of

the PR and PC variables. We also need some loops to signal and
effectuate the end of the algorithm.

The basic structure is to call the simplex procedure again in

the re-entry mode, i.e. to ask for the next parametric step,
unless an empty or an unbounded problem is met.

That refers to the two abnormal exits of the linear programming

(simplex) procedure, and also to intrinsically unbounded
problems which are technically bounded.

Whenever a "fancyhigh" upper limit on one of the specified

variables is found binding, this is equivalent to an unbounded
problem. We will refer to this situation of a technically
optimal solution which has hit the ceiling as "substantially
unbounded" •

In terms of their significance, the following possibilities

arise:

a) the problem is found empty, unbounded or substantially

unbounded on return from the first normal call to the
simplex procedure. The significance of that result is
obvious, i.e. the originally specified problem is empty
or unbounded.

b) the problem is found empty, on return from a re-entry call

for parametric adjustment of the right-hand side. In that
case the problem has become empty, for some result of
parametric variation. (This would arise after making the
parametric step itself). -----

c) An empty problem arises, when calling the simplex procedure

in the re-entry mode, for parametric variation of the
objective function. The empty problem is in that case not
the main linear programming problem itself, but the
redefined problem with the additional restriction, on T*
the parametric component of the objective function. When
we put a negative (-0.0000001) entry in the value column
entry of the T* row, this means that an additional increase
(of at least 0.0000001) in the value of T* is required.
When T* is already at a maximum value, that requirement
cannot be met.
PARAMETRIC VARIATION OF THE L.P. PROBLEM 313

d) An unbounded or substantially unbounded problem arises

when calling the simplex procedure in the re-entry mode
for parametric variation of the right-hand side. In that
case the parametric variable A is unbounded. If there are
variables without non-negativity restrictions in the problem,
this may also be the apparent cause of unboundedness, i.e.
no parametric step has been made but the unbounded loop of
the procedure has been activated. This case is limited to
the specified variables either not to change, or not to be
restricted in the sign, in which case there is no upper
limit either. (The A-variable itself has its upper limit
removed in the procedure itself.)

Normally unboundedness will be revealed as substantial

unboundedness, by the calling main programme, i.e. some
specified variable hits the ceiling. This condition is
signalled, only after a parametric variation step and a
further normal step have technically been completed.

d) The problem may be found unbounded, after calling the

simplex procedure in the re-entry mode, for parametric
adjustment of the objective function. In that case the LP
problem has become unbounded, i.e. for some finite value
of A, the full objective function T** becomes unbounded.

Again, unboundedness will normally be revealed by the

main programme as substantial unboundedness, rather than
the alarm exit of the simplex procedure being activated.

A full listing of the simplex procedure is not given here, this

would involve too much duplication of copied parts of the
linear programming procedure.

Instead, we list the file of editing instructions which converts

LINP into SIMP.

The editing instructions follow the text in the same order as

reading, and have the following significance:

TC/OLD/

Copy text until, somewhere in the text, the characterstring

"OLD" as its first non-blank text is found.

TS/OLD/

Copy entire lines, until a card-line starting with "OLD" as its

first non-blank text is found.
314 CHAPTER XIII

R/OLD/NEW/

In the next card-line, replace the character-string "OLD", by

the characterstring "NEW",

I? NEW?

Insert text, as quoted.

PC/OLD/

Skip and delete text, until (somewhere ~n the text), the

characterstring "OLD" is found.

Tl - Copy one line

PI - Skip and delete one line

T.? OLD?

In the next line, copy characters until the characterstring

"OLD" is found.

P.? OLD?

In the next line, skip and delete characters, until the

characterstring "OLD" is found.

LISTING OF THE FILE OF EDITitiG INSTRL'CTlONS \.1IICH CONVERTS

LINF INTO SIMP:

TC/'PROCEDURE' LINF(/.R/LINF/SIHF/
T.?REENTRY>?I?FR.rC.?
TC/'INTEGEE'M.N.NEG:./.T.?REENTRY1.I?FR.FC.1
TS/'COMMENT' LINEAR/.R/././.TI
11 Io.'ITH FOST-oPTIMAL PARAMETRIC VARIATIDrJ.

M RESTRI,GTIONS AND N VARIABLES.

ltJCLUDING ONE EXTRA VECTOR. THE FARAMETEI C ROIJ/COLUMN
(I F SUFFLI ED) •

FOR THE SIGNIFICANCE OF THE PARAHETERS.T.H.N.NEQ.NAV.

ROlJLST II.ND CoLLST. SEE THE TEXT OF THE LINF PROCEDURE.
THE PROVISIONS FOR UPPER LIMITS ON OFlDINARY VARIABLES
ARE ALSO COMMON EEHIEEN SIMP ArJD LUIF. AND ARE DESCRI BED
 PARAMETRIC VARIATION OF THE L.P. PROBLEM 315

IN THE CDMMENT AT THE HEAD DF LINP.

THE ABBREVATIONS PR AND PC INDICATE

'PARAMETRIC ROW' AND 'PARAMETRIC COLUMN'.
IF THE PROCEDURE IS ENTERED WITH REENTRY-D.
I.E. NORMAL OPTIMIZING.
WITH PC=1 AND PR=O FOR ADJUSTMENT OF THE RIGHTHAND SIDE.
DR WIHT PR=1 AND PC=O.
FOR VARIATION OF THE OBJECTIVE FUNCTION.
THE LAST 'NORMAL' COLUMN OR ROW DF THE TABLEAU
(NDT COUNTING THE VALUE COLUMN. TARGET ROW ETC.).
IS UPDATED IN THE USUAL WAY.
BUT IS NDT INCLUDED IN SEARCH OPERATIONS.
GENERALLY. FOR REENTRY < -PR+1 AND ALSO < -PC+1.
PR ROWS AND PC COLUMNS ARE NOT INCLUDED IN SEARCH
OPERA TI ON S •

FOR REENTRY=2. THE CONTROL SWITCHE5 FOR PARAMETRIC

ADJUSTMENT ARE ACTIVATED. AS FDLLoWS:
FOR PARAMETRIC VARIATION OF THE RIGHT-HAND SIDE.
THE PRDCEDURE SHOULD BE ENTERED WITH THE TRUE SHADOWPRICE
DF THE PARAMETRIC COLUMN SAVED IN THE M+2 ND ENTRY
IN THE (N TH) PARAMETRIC COLUMN.
THE SHADoWPRICE ITSELF IS THEN SET AT A NEGATIVE VALUE.
THE PARAMETRIC SEARCH WILL THEN BE MADE. BUT JUST BEFORE THE
STEP IS ACTUALLY MADE. THE TRUE SHADOWPRICE IS RESTORED.
AFTER EACH PRIMAL PARAMETRIC VARIATION STEP. THE STATUS-
LOOP DIRECTS THE PROGRAMME TO ELEMINATE THE PARAMETRIC
VARIABLE.
FOR PARAMETRIC VARIATION OF THE OBJECTIVE FUNCTION.
THE PROCEDURE SHOULD BE RE-ENTERED. WITH REENTRY-2.
AND THE PARAMETRIC RESTRICTION ADJUSTED AS JUST VIOLATED.
THE TRUE VALUE OF THE PARAMETRIC OBJECTIVE FUNCTION
SHOULD TEMPORARILY BE STORED IN COLUMN N+2 OF THE PARA-
METRIC ROW.
IN THAT CASE. THE PARAMETRIC ADJUSTMENT STEP IS MADE
AS A SEPERATE ROW-VECTOR OPERATION. ONLY THE SECOND
(ORDINARY) STEP IS MADE BY THE MAIN STEP-MAKING
LDOP DF THE PROCEDURE.
THE INTEGER VARIABLES PR AND PC (PARAMETRIC ROW AND
PARAMETRIC COLUMN) ARE NORMALLY SUPPLIED AS ZERO.
EXCEPT WHEN THE PROCEDURE IS CALLED IN THE CONTEXT OF
PARAMETRIC PROGRAMMING DR SENSITIVITY ANALYSIS.
PR SHOULD BE SUPPLIED AS ONE IN THE CASE OF
PARAMETRIC ADJUSTMENT OF THE OBJECTIVE FUNCTION. WHILE
PC SHOULD BE SUPPLIED AS ONE IN THE CASE OF PARAMETRIC
ADJUSTMENT OF THE RIGHT-HAND SIDE. ;
?
316 CHAPTER XIII

Pcn hPI
TS/' COMMENT' I. T1
I? FOR REENTRY=O THE NORMAL LP ALGORITHM IS FOLLOWED
FROM THE START.
INCLUDING THE FILLING OF THE NAMELISTS.
OTHERWISE THE PROCEDURE EXPECTS AN ALREADY UPDATED TABLEAU.
THE FOLLOWING VALUES OF THE REENTRY-PARAMETER ARE
ACCOMODATED IN THE RE-ENTRY MODE:
REENTRY = I.
IF THIS VALUE IS SUPPLIED. THE ALGORITHM IS ENTERED
AGAIN. WITHOUT SPECIAL FEATURES.
REENTRY = 2. WITH PC I.
PARAMETRIC VARIATION OF THE RIGHT-HAND SIDE.
REENTRY = 2. WITH PR = I.
PARAMETRIC VARIATION OF THE OBJECTIVE FUNCTION.
THE SIGNIFICANCE OF THE EXIT-VALUES OF THE REENTRY-
PARAMETER IS THE SAME AS FOR LINP.
?
pcn hPI
TS/RETURN IN INVERSION:I.TS/'FOR' 1:=11
R/'UNTIL' M/'UNTIL' M-PRI
TS/ORDER:I.TI.TS/OR/.
R/.M.I.M-PR.I.R/N./N-PC.I.R/2/2+PR/.H/2/2+PCI
TS/PHASE I:I.TS/'FOR' II=I.R/'UNTIL' M/'UNTIL' M-PR.I
I?
M 'STEP' I 'UNTIL' M-3+PR+REENTRY1
T.?'THEN'?P.E.TI.T.1FEASIBLE?I7'TH~~' 7
TS/MAXIMIZEI/.TS/'FOR'I
R/'UNTIL' N/'UNTIL' N-PC.I
I?
N 'STEP' I 'UNTIL' N-3+PC+REENTRY?
TS/INITIALIZE SUBSTIT/.TS/'FOR' 11=1
R/'UNTIL' M/'UNTIL' M-PR.I.P.E.TI
I? M 'STEP' I 'UNTIL' M-3+PR+REENTRY 'DO'
7
TS/SEARCH FOR SMALLEST QUO WITH JTH COLUMN:I
TS/'FOR' I:=NAV+II
R/'UNTIL' M/'UNTIL' M-PR.I.I1
M 'STEP' I 'UNTIL' M-3+PR+REENTRY 1
TS/'IF' ROWLSTCIJ/.R/IOOO/N/.T2
TC/ROWLSTCIJ < 1000/.R/IOOO/NI
TS/TRY UPPER BOUND:I.TS/'IF' QUO> 9991
11 'IF' QUO> 1000000000 'THEN' 'BEGI N'
'IF' J=N 'AND' REENTRY=2 'AND' PC=I
'THEN' 'BEGIN'
NEWLINE( I);
WRITETEXT('('PARAMETRIC%REENTRY%PROBLEM%UNBOUNDED%%')');
'GOTO' UNBOUNDED; 'END'; 'END';
'IF' J=N 'AND' PC=I 'THEN' 'BEGIN'
RI=TRYR; KI=COLNI=N; ROWN:=TRYN; VNBVI=QUO;
I 'GOTO' MAKE THE STEP; 'END';

?
PARAMETRIC VARIATION OF THE L.P. PROBLEM 317

TS/MAKE THE STEPI/.TI

11

CONSULT PARAMETRIC REENTRY SWITCHES:

'IF' REENTRY=2 'AND' COLN=N 'AND' PC=I 'THEN' 'BEGIN'
TCM+I.KJ:=TCM+2.KJ;
'IF' 'NOT' R=O 'THEN' 'BEGIN'
'IF' ROWLSTCRJ=ROWN 'THEN' 'BEGIN'
QUO: =VNBV;
'GOTO' ADJUST RIGHTHAND SIDE AND UPPER BOUNDS COLUMN;
'END'; 'END'; 'END';
'IF' REENTRY=2 'AND' ROWN=IOOO+M 'AND' PR=l
, THEN' 'BEG IN'
TCM.N+IJ:=T[M.N+2J;
'FOW JI=I 'STEP' 1 'UNTIL' K-I. K+I 'STEP' 1 'UNTIL' N+I
'DO' T[M+I.JJ:=T[M+I.JJ-T[M.JJ*TCM+l.KJ/T[M.KJ;
REENTRY:=l;
TCM+l.KJ := -0.0000001; 'GOTO' PHASE U 'END';
1
TS/CONSIDER ONE COLUMN UPDATE:/
11 CONSIDER RH ONLY UPDATE:
'IF' REENTRY=2 'AND' PC=l 'AND' 'NOT' R=O 'THEN' 'BEGIN'
'IF' ROWN=ROWLST[RJ 'THEN' 'BEGIN'
TIR.N+2J:=TIR.N+2J-T[R.N+lJ;
TIR.N+lJ:=-O.OOOOOOI;
REENTRY: = 1 ;
'GOTO' PHASE H 'END'; 'END';
1
TS/CHECK FOR STATUS:/.Tl
11 'IF' COLN=N 'AND' PC=I 'THEN' 'BEGIN'
FEASIBLE := 'FALSE';
TIR,N+lJ:=-O.OOOOOl.O.OOOOOl; 'END';
'IF' PC=l 'AND' ROWN=N 'THEN' 'BEGIN'
REENTRY:=O; 'GOTO' END OF SIMP; 'END';
1
TC/END OF LINP;/.R/LINP/SIMP/
TS/ORDER FOR EXIT:/.TS/ORDL(/
11 'IF' PR=I 'OR' 'NOT' COLLSTINJ=N 'THEN'
1
R/l//.T1
11 'ELSE' ORDL(T.M.N-PC.2.2+PC.ROWLST.COLLST);
1
TS/END OF LINP/.R/LINP/SIMP/
TE.E
Partill
SOME GENERAL MATHEMATICAL PROGRAMMING
NOTIONS AND RELATED MATRIX ALGEBRA

TOPOLOGY OF FEASIBLE SPACE AREAS AND ITS RELATION

TO DEFINITENESS 319

14.1 The mathematical programming problem 319

14.2 Convex and non-convex problems 320
14.3 Tangential subspaces 334
14.4 Extrema and convexity properties of
quadratic functions 339
l4.S Subspaces and partial inversion of
definite matrices 3S0
14.6 Definite matrices and diagonal pivoting 3S2
14.7 The factorization of a semidefinite matrix 356
14.8 The constrained maximum of an anti-convex
function, with linear side-conditions 3S8

CHAPTER XV

OPTIMALITY CONDITIONS 163

lS.l The additive properties of restrictions 363

lS.2 Combined restrictions and their
tangential equivalent 36S
lS.3 The Lagrangean expression and the
conditions on its derivatives 368
lS.4 Dual variables and right-hand side
variations 377
lS.S Constrained second order conditions:
subspace convexity 383
lS.6 Non-linear duality 397
318
CHAPTER XIV

TOPOLOGY OF FEASIBLE SPACE AREAS AND ITS RELATION TO

DEFINITENESS

14.1 The mathematical programming problem

The theory of mathematical programming is traditionally stated

in terms of inequalities. In fact, mixed nonlinear systems with
some of the restrictions being linear equations do not give rise
to more (extra) complications than those that were discussed in
Chapter X with respect to linear programming. However, to avoid
unnecessary complications at the level of theory, I shall follow
the tradition, and introduce mixed systems of equations and
inequalities only at a later stage in an applied context.

We state a general (possibly non-linear) mathematical

programming problem as follows:

Maximise

T = T (x) (14.1.1)

Subject to

f. (x) > 0 (14.1.2)

1

(i = 1,2 m)

Although formulated in a particular way, with the inequality

sign in the> direction and a zero on the right-hand side,
(14.1.2) is In fact quite generally an inequality. Any inequality
can be put in that form.

For example

can be re-written as

and in that form the restriction conforms (14.1.2).

The definition of a function (and hence of a mathematical

programming problem) does not require continuity, but we will
normally assume that both the objective function T and the
restricting functions fi are continuous and differentiable.

319
320 CHAPTER XIV

Just as in linear programming, it ~s conventional to add the

non-negativity restriction

x > 0 (14.1.3)

i.e. to assume that all specified variables are restricted to

non-negative values.

14.2 Convex and non-convex problems

Some, but not all programming problems are specially complicated

because they are non-convex.

That term refers to, for example the shape of a lens used in a
pair of spectacles. If the lens is thin at the edge and thick
in the middle, it is a convex lens, if it is thick at the edge
and thin in the middle it is a concave lens. (See graph l4.2a)

graph 14.2 a
8 convex area.

The term convex restriction includes the ordinary linear

inequality i.e. a flat surface. We already gave some formal
definitions in section 6.5 which are therefore not repeated
here. Note that we may speak of an anti-convex restriction
(= curved away from the feasible space area), but of a non-convex
TOPOLOGY OF FEASIBLE SPACE AREAS 321

area, which may have any shape provided it is not convex.

Non-convexity is a serious complication mainly because it may

give rise to a number of local maxima rather than to a single
optimum.

Example

Maximise T = 2x l + 3x 2

Subject to 2xl ~ (x 2 - 2)2 -1

(xl' x 2 ~ 0)

(See also graph l4.2b).

2.'>

TX, graph 14.2 b

\ a non-convex problem
with two loca~ solutions.
\ 2.0
\
\
\
1.

-0.5 \.5 2.0 2.5

-0.5
322 CHAPTER XIV

In this problem, there are two local maxima, one at xl = 1.5,

x2 = 0, and one at Xl = 0, x2 = 1. They are in this case of
equal value, 2xl = 2 * 1.5 = 3 and 3x2 = 3 * 1 = 3. But this
can only be established by identifying both of them in the first
place. And that's precisely the problem. With a large number of
restrictions there could be a large number of local solutions
and one might be forced to calculate the values of the variables
for each one of them.

In a convex problem on the other hand, it is enough to establish

the fact that every movement away from a particular point leads
to a reduction of the value of the objective function.

Not surprisingly, most (but not all) operationally effective

mathematical programming algorithms, refer to convex systems.
The same problem i.e. a motley collection of local maxima, may
also arise as a result of the specification of the objective
function.

Example

Maximise - Xl • x 2

Subject to Xl + x 2 - 10 > °
(Xl' x 2 > 0)

Here the shape of the feasible space is still just convex

(the surface is linear), but there are nevertheless two local
maxima Xl = 10, x2 = °
and Xl = 0, x2 = 10.

We will indicate such an objective function also as non-convex.

We will now survey the problem of convexity versus non-convexity,
i.e. state some definitions, theorems etc., and generally analyze
the implications of various properties of restricting functions
and objective functions.

A function f(~) may be indicated as properly convex, if and only if

f(p . x* + (l-p). ~**) ~ p . f(~*) + (l-p). f(~**) (14.2.1)

is true for all x* and ~**, and (their non-negative linear

combinations i. e-:-) for

° ~ p.::.1 (14.2.2)

If the strict inequality

f(p . x* + (l-p) ~**) > p. f(x*) + (l-p) f(~**) (14.2.3)

TOPOLOGY OF FEASIBLE SPACE AREAS 323

applies for 0 < p < 1, we might indicate the function as

properly and strictly convex.

The obvious example of a properly convex function which is not

strictly convex, is an ordinary linear function. However, a
function which is a linear in some segments but curved or kinked
at other points can also be convex, and not properly and strictly
convex.

If the inequality signs in (14.2.1) and (14.2.3) are in the

reverse direction i.e. < and <, the function is properly
anticonvex (for the < case), or properly and strictly
anticonvex (for the < case).

For restricting functions and objective functions the weaker

properties of peripheral and directional convexity* are relevant,
and are satisfied by a wider group of functions.

These weaker properties were the reason for naming the stronger
property of "proper" convexity as "proper'" in the first place.
Unless otherwise stated the term "convex" will in the rest of
this book be understood as meaning "properly" convex.

We now give some definitions

A restriction is peripherally convex

if and only if

f(x*) .:. 0 )
(14.2.4)
f (x**).:. 0 )

(i.e. the points x* and x** both satisfying the restriction),

implies that a no~-negatIve linear combination also satisfies
the same restriction

*Theterms "peripheral" and "directional" convexity are used here,

but they are not conventional terms. It was felt that these
terms convey their meaning even without exact definition, and
for that reason they will be used in this book. To the extent
that there is a tradition, the term quasi-convex (which is the
same as directionally convex) is the more conventional one.
Note, that not all peripherally convex restrictions are
associated with a directionally convex restricting function.
Contra-examples with submerged islands cannot be excluded.
For reference see Mangasarin [27] Ch.9, as well as
Ponstein [30] •
324 CHAPTER XIV

I f x* and x** satisfy (14.2.lt), then

f(p ~* + (1 - p) ~**).::.. 0 )
)
)
is also true for ) (14.2.5)
)
o < p 2. 1

(the set of vectors satisfying f(~) ~ 0 is a convex set).

For a peripherally convex restriction, the line which marks the

restriction (the periphery of the set of points which satisfy
it), is flat or curved towards the inside of the feasible space
area, but the function does not satisfy the definition of a
(properly) convex function.

The terms peripheral and directional convexity often occur in

association with a specified domain.

The term domain means also satisfying other restrictions as

specified. For example, xl • x2 ~ 6 is peripherally convex in
the xl' x2 ~ 0 domain. The function y = xl • x2 - 6 is not
properly convex, and neither is xl • x2 .::.. 6 convex for all
values of xl and x2. That restriction is satisfied for
xl = x2 = -4. and for xl = x2 = 4, but not for xl = x2 = o.
That, even within the xl' x2 ~ 0 domain, the restricting
function is not properly convex, is illustrated as follows:

For xl = x 2 = 1, we find

f (x, , x 2 ) = f(l, 1) = 1 x 1
...
- 6 - 5,

and for xl = x 2 = 3, we find

f(x l , x 2 ) = f (3, 3) = 3 x 3 - 6 = 3

The function value of the average of the two vectors

f(x l x 2 ) = f(2, 2) = 2 x 2 - 6 =- 2
is less than the average of the two function values which is - 1.
The function is therefore shown to be not properly convex. Yet
the shape of the hyperbola xl • x2 > 6 is neatly convex. And if
we perform the same calculation on the half-way linear
combination of

f(l, 6) 1)(6-6 o
TOPOLOGY OF FEASIBLE SPACE AREAS 325

and

f(x l , x 2 ) = f(6, 1) = 6 - 1 - 6 = O.

i.e. two points on the periphery, the result is a confirmation

of the convexity property.

This half-way combination is

f(x l , x 2 ) = f(3.5,3.5) = 3.5 x 3.5 - 6 = 6.25

which is well above the average of the two function values.

The concept of peripheral convexity is relevant, mainly for

functions which define restrictions.

For objective functions the stronger property of directional

convexity is more relevant.

A function f(x) is directionally convex (some domain)

if and only if

(14.2.6)
f(~**) > A

implies (for all A, ~ being in the specified domain, if one is

specified), that for 0 ~ p ~ 1, we find

f(p . x* + (1 - p) . ~** ) > A (14.2.7)

(The set of vectors satisfying f(~) > A is convex).

To make the difference quite clear, we give an example of a

function which is peripherally convex, but not directionally
convex.

We define f(x l , x 2 ) as follows:

f(x l , x 2 ) - 1, if xl + x 2 > 5

f(x l , x 2 ) 1, if xl < 2 and x 2 < 2

and

f(x l , x 2 ) = 10, in all other cases.

Thus, the restriction f(Xl, x2) ~ 0 is convex, but f(xl' x2) ~ 5

is not convex. The square in the bottom lefthand side of graph
l4.2d makes a dent in the restriction f(x l , x 2 ) ~ 5. This
326 CHAPTER XIV

graph 14.2 C
illustration of
peripheral convexity.

1 ~
I

graph 14.2 d
illustration of peripheral
convexity, not associated
with directional convexity.
TOPOLOGY OF FEASIBLE SPACE AREAS 327

non-convex "island" does not invalidate the peripheral

convexity of f(xl' X2) ~ 0 as peripheral convexity is defined
in terms of f(xl, x2) ~ o. (See also graph l4.2d).

Directional convexity is thus a stronger property than

peripheral convexity,

Theorem

Let f(~) be a continuous and directionally convex function.

Then (even if f(x) is not properly convex), there exists a

properly convex function g(x), which gives a similar ordering
of the valuation of all possible vectors~, as does f(~), i.e.

g(~**) ~ g(~*)

if and only if

f(~**) ~ f(~*)

and the same for the >, < and < signs.

Substitute of a proof

Consider a series of level curves f(~) = A for various values

of A, say A = - 100 to A = 100, calibrated at suitable small
intervals e.g. A = - 100, A = - 99,99 etc., with increments
of initially 0.01 between two curves.

Take some vectors x* and x** and a number p in the interval

o < p < 1. We will-indicate the higher (not lower) valued of the
tw; ve~tors as ~**, the other as ~*, i.e. assume

(14.2.8)

For p = 0 and p = 1 the proper convexity property (14.2.1) is

trivial but satisfied irrespective of the shape of the function.
We can therefore limit our investigations to the interval

o < p < 1 (14.2.9)

For f(x**) f(x*) the property of proper convexity is implied

by the-property-of directional convexity, hence we can limit our
investigation to

f(x**) > f(~*) (14.2.10)

328 CHAPTER XIV

A positive linear combination of x** and x* can logically

satisfy one and only one of the following-relations:

either

a) f(p x* + (1 - p) • ~**) <

or

b) f(p . x* + (1 - p) . ~**) f(~*)

or

c) f(~*) < f(p . x* + (1 - p) . ~**) < f(~**)

or else

d) f(~**) ~ f(p . ~* + (1 - p) . ~**)

Case a) contradicts the assumption of directional convexity,

and must be assumed not to arise.

We may illustrate this point by drawing some level curves for

a non-convex function, e.g. (see graph l4.2e)

f(x l , x 2 ) = - xl • x 2

The point

x** = 5, 0.2 1S on the - xl • x 2 = - 1 level curve, and

~* = 1, 2 is on the - Xl x2 = - 2 level curve. Both

points are marked 0 in the graph.

Linear combinations of these two points are on the "wrong"

side of the - Xl • x2 = - 2 level curve i.e. at points where
- xl' x2 is less than this proves that the set of points
satisfying - x • x 2 ~ - 2, is not a convex set.

Therefore, f(x l , x 2 ) = - Xl • x 2 1S not a directionally convex

function.

End of illustration.

Exi t case a)
TOPOLOGY OF FEASIBLE SPACE AREAS 329

graph 14.2 e
a non-convex curve
crosses a link with
6
a more outward one.

-2

2 5 6
330 CHAPTER XIV

Case b) does not directly contradict the definition of convexity,

although it contradicts the definition of strict convexity. It
is however a property of convex sets (strictly convex or not)
that a linear combination of some points in the set, including
a non-zero contribution of an interior* point, is an interior
point.

By assumption the set of points satisfying f(x) > f(x*) is

convex and x = x** is an interior point of that set.-Therefore
case b) contradicts the assumed directional convexity.

Exit case b)

Case d) would indicate that, at that particular point, f(~)

itself satisfies the property of convexity as defined in
(14.2.1). It does not require further examination.

Exit case d).

We now concentrate on case c) where the straight line between

x* and x** does not cross either of the two level-curves.

We denote the indicator of the relative valuation of the two

vectors as q. i.e. q is the number which fits

f(p . x* + (1 - p) ~**) q . f(~*) + (1 - q) • f(x**)

(14-:-2.11)

Should

q ~ p (14.2.12)

be true, there is no problem as proper convexity is not

contradicted, at this point for f(~) itself.

For q < p we re-define the function, i.e. we now define a new

function g(x).

g(~) = (1 -~) . f(p . ~* + (1 - p) ~**) + ~ • f (x)

p
p

if f(~) < f(p . x* + (1 - p) ~**)

and

g(~) = (1 _ 1-q)
1-p
. f(p x* + (1 - p)
-
. ~**)
1-q
+ -1- f (x)
-p
if f(~) > f(p x* + (1 - p) ~**) (14.2.13)

* No proof of this
point is offered
TOPOLOGY OF FEASIBLE SPACE AREAS 331

We may assume that this transformation ~s in first instance

performed with the vectors x* and x** and the number p and q,
for which the lowest ratio q/p occ~rs. The transformation leaves
the shape and the ordering of all the level curves intact.

It is kinked at the level-curve through x = p . x* + (1 - p) x*.

There the function values of g(x) and f(x) are the same, but the
differences from this value are-scaled. -

The scale-factor is ~ for values below f(p x* + (1 - p) . x**)

p
l-q
and -1-- for higher values of the function.
-p
The corresponding "backward" transformation is

f(x) = ( 1 - E) . g(p . x* + (1 - p) . x**) +

q
E
q
if
g (~) < g (p • x* + (1 - p) x'"* )

and
l-p l-p
f (x) = (1 - - ) . g (p . x* + (1 - p) • x**)+
l-q l-q
if
g(~) > g(p . x* + (1 - p) . x**)
(14.2.14)

Substitution of the equivalent value in g(x) for f(~), f(~*) and

f (x*"'), according to the backward transformation is now used to
express the linear combination in g(x). Application of (14.2.14)
to all three terms of (14.2.11) yields:

g (p x* + (1 - p) ~**)

q (1 E) g (p x* + (1 - p) ~**) + p g(~*)
q
+ (1 - q) ( 1 _ l-p) g(p x* + (1 - p) x**) + (1 - p)
l-q -

g (~**) p g(~"') + (1 - p) . g(x*"')

- (14.2.15)

The transformation "blows up" differences in function value below

the kink, and reduces differences above the kink.
 332 CHAPTER XIV

Example

Suppose it is desired to re-name a mid-point to being 60% from

the lowest point and 40% from the highest point, we simply
adjust the marker points. Starting with f(x*) = 0.0, f(x**) 1.0,
f(0.6 x* + 0.4 x**) = 0.5, - -

~o.~ + 0.4~*~
f (x*) r f(x**)
I I T
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

re-scale by 0.5/0.6 re-scale by 0.5/0.4

L.__- 4____L -__ __ __ ~ ~ ~ ___ L_ _ _ _~L__ _ _ _~_ _ _ _~_ _ _ _~

new scale:

g'(~*) g(~**)
I I
-0.1 0.0 o. 1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

To comply with the mid-point as point of common reference (at

the kink), i.e.

f(0.6 x* + 0.4 ~**) = g(0.6 ~* + 0.4 ~**) = 0.5,

it is necessary to change all other marker points.

Thus f(~*) is 0.0, but g(~*) becomes -0.1.

Relative differences for other points are affected in the same

systematic way. For example, the evaluation of the point x z
is assumed to be f(~) = 0.78. With P = 0.6, q = 0.5 and
f(0.6 x* + 0.4 x**) = 0.5, g(z) becomes (apply (14.2.13),
second-part) -

g(~) = ( 1 - g:~). 0.22 + g:~ . 0.78 0.73

End of Example.

Generally, the transformation does not create new violations of

(14.2.1), but on the contrary makes-rhat condition easier to
satisfy for other linear combinations than just the one to which
it was written. Should any violations be left, the procedure can
be repeated.
TOPOLOGY OF FEASIBLE SPACE AREAS 333

End of proof-substitute

We now refer back to our illustration of peripheral convexity

earlier in the chapter. Clearly we can replace a peripherally
convex function by a directionally convex function with the
same level curve f(x) = o. One simply erases any non-convex
islands which may exist at other levels than the critical value
of zero.

Therefore our theorem has the following

Corollary

If f(x) > 0 is peripherally convex, then, even if f(x) is not

properlY-convex (not even directionally convex), there exists
an equivalent restriction g(x) > 0, where g(x) is properly
convex. The generalisation of these theorems-to proper and
strict convexity will be obvious.

Ordinary algebraic restrictions don't normally satisfy the

definition of peripheral convexity for all values of x, but
there are restrictions which are peripherally convex In a
specified domain, and an often-specified domain is the x > 0
domain. In practice, with continuous and differentiable-
functions expressed in neat algebraic formulae, it is often
possible, (but not so often advantageous) to perform the
transformation needed to turn directional or peripheral convexity
into proper convexity, by an algebraic reformulation. Thus
f(x,y) = x y is directionally convex in the x, y > 0 domain,
while log f = log x + log y is properly convex and gives the same
ordering. A further consequence of the theorems in this section
is the following:

A mathematical programming problem in which the objective

function is directionally convex and the restrictions
peripherally convex, can be reformulated, and in the reformulated
problem, which has the same solution, the objective function as
well as the restricting functions will be properly convex.
Whether a restricting function is properly convex, or only
peripherally convex, is a matter of putting the restriction in
a particular form. It makes no difference to the set of vectors
satisfied by the restriction. Similarly, a directionally convex
objective function gives the same ranking of all solution vectors,
as a corresponding properly convex one. If a mathematical
programming problem is characterized by a properly convex
objective function and properly convex restricting functions,
we call such a problem a properly convex problem. If the
objective function is directionally convex and/or - one or more
of the restricting functions peripherally convex (the rest
being properly convex). we speak of a quasi-convex problem.
334 CHAPTER XIV

Exercise 14.2 (convexity)

The following restrictions on xl and x 2 are listed:

2
+ X2 > 16 (1; 2)

(3; 4)

(xl + 1)(x 2 + 1) > 10; (xl + 1)(x 2 + 1) .::.-10 (5; 6)

.:: 10; (x + 1) (x 2 - 1) < -10 (7; 9)

1 -

Each of these restrictions is considered in isolation from the

others, i.e. each defines its own set of pairs xl' x 2 which
satisfy the restriction in question.

Which of these sets is

a) properly convex by its restricting function?

b) quasi (peripherally) convex within the xl' x 2 > 0 domain,

without being properly convex?

Which of the corresponding restricting function is

c) anti-convex

Write each of the restrictions in the conventional form of

(xl' x 2 ) 2:.. o.

Hint: Make use of graphical analysis.

* * ** **
(xl' x 2 ) = p(x l , x 2 ) + (1-p) (xl' x 2 ) is the line which joins
* * ** **
(xl' x 2 ) and (xl' x 2 ).

14.3 Tangential subspaces

Consider a functional relationship

(14.3.1)

Assuming that f(~) is continuous and differentiable, we may

determine for any x x* the vector of first-order differentials

v if
ax (x*)
- (14.3.2)

'~a4d~q~ JO pua 4~ 8-S ~oJ sqd~~~

~ £'2 s~u :(~ ~ssauA4dllia JO 4unO~~E uo A4~1~~A~~4
pu~
Ul 9 OSl E '8 pu~ S s~u :(q ~~ pu~ £'1 s~u :(~ 2'~1 dlaq-~a~suv
TOPOLOGY OF FEASIBLE SPACE AREAS 335

For the same particular vector x x* we may also determine

the inner product

c = v'x* (14.3.3)

We will indicate the relation

v'x = c = v'x* (14.3.4)

as a tangential approximation of f(x) = o.

The linear approximation may be different for different points
on the graph of f(x) = O. We therefore say that (14.3.4)
gives the tangential approximation f(x) = 0 at the point x x*.

Example (See also graph l4.3a)

2
f(x l , x 2 ) = xl + !x 2 - 5 = 0

This functional relation is satisfied by

At that point, the first-order derivatives are

)( 2 1

From (14.3.3) we evaluate the constant as

c = 1 . xl + 1 . x2 = 4 + 2 = 6
and the linear approximation is found to be

There are other tangential approximations of the same function.

For example

f(x l , x 2 ) = xl + !x 2 2 - 5 =0
1S also satisfied by xl 1• 4.
336 CHAPTER XIV

At that point, the same procedure results in a different

tangential approximation, as follows
af
aX l = 1

af
aX 2 = 2x2
I = x 4 2

Apply (14.3.4), to find the constant

c = 1 . xl + 2 . x 2 = 1 + 4 x 2 9

and the tangential approximation at x I I , x 2 = 4 is found to

be xl + 2 x 2 = 9.

Before formulating theorems and proofs concerning linear

approximations, we introduce a related concept.

A linear inequality restriction

v' x > c (14.3.5)

is said to be the tangential approximation (at the point

x = ~*), of

f(~) > 0 (14.3.6)

If x = x* satisfies the binding form of (14.3.5), i.e. the

tangential approximation

v' x c (14.3.7)

as well as (14.3.6) the restriction itself. If the set of points

which satisfies the tangential approximation of a restriction
includes the set of points satisfied by the restriction itself
we also refer to it as a tangential equivalent.

In graph l4.3a., which illustrates the tangential approximation

of xl + !X2 2 - 5 ~ 0, both sets are in the left-hand bottom
side of the equation-lines, and the strictly convex set
xl + !x2 2 - 5 ~ a just touches xl + X2 = 6. Thus xl + x2 ~ 6
is the tangential equivalent of xl + iX22 - 5 ~ 0, at the
point xl = 4, x2 = 2. Just as the tangential approximations, the
corresponding tangential equivalents may be different for
different points of the periphery of the restriction.

At xl = 1, x 2 = 4, the tangential equivalent is xl + 2x2 < 9.

TOPOLOGY OF FEASIBLE SPACE AREAS 337

7
T graph 14.3 a
a quadratic function
Xl with two tangentiaL
approximations to it.
6

3
,
2

1 -

o o~----~------~----~ 6 7
2 3
338 CHAPTER XIV

Theorem

Let
the set of vectorpoints satisfying;

(14.3.6)

be a convex set, f (~) being continuous and differentiable.

Then
every finite boundary point of that set is also an outward
point.

Proof

We may, without loss of generality assume that f(x) is properly

convex. (See the previous section)

Now consider the function

(14.3.8)

where v ' i s a finite vector and A a finite positive number.

The necessary first-order conditions for a maximum of z(~) are

~ = - + Alf = 0
ax -v' ax (14.3.9)

For a convex function the necessary first-order conditions are

also sufficient for a maximum. This is true, even if f(~) itself
does not have a finite maximum.

But clearly these conditions are satisfied by any linear

approximation, with A = 1. (Just compare (14.3.2) with (14.3.9».

This shows that, for

v' af (_x*) (14.3.10)

ax

z(x) = - ~' ~ + f(~) (14.3.11)

attains a maximum value at x x*.

i. e.

- v' x + f(x) < - v' x* + f(x*) (14.3.12)

is true for all ~.

TOPOLOGY OF FEASIBLE SPACE AREAS 339

But f(x*) is zero, because that is the definition of a boundary

point.-Therefore (14.3.12) reduces to

- ~'~ + f(~) < - v'x* (14.3.13)

From (13.3.13) we immediately infer that, because of the non-

negativity of f(~),

- v' x < - v' x* (14.3.14)

is true for all x which satisfy

(14.3.5)

This is the definition of an outward point with - v' as the

direction to be maximized ~.e. x x* is an outward point.
q.e.d.

Furthermore, comparing (14.3.14) with (14.3.6), it is clear

that we have the following

Corollary (also assuming differentiability)

The tangential approximation of a convex restriction - based on

a differentiable restricting function - gives the tangential
equivalent, at the point of approximation. This is so at each
point where the restriction is binding and a tangential
approximation is taken. (A tangential approximation of a convex
restriction is satisfied by each vector point which satisfies
the original restriction, and is a tangential equivalent.)

Furthermore, it may be observed that if T(X) is a convex

function, -T(X) is an anti-convex function: this follows simply
from changing-the signs in the definitions of (anti) convexity.
From thls we have the further

corollary

A tangential approximation of an anti-convex restriction ~s

violated by each vector-point that violates the original
restriction.

14.4 Extrema and convexity properties of quadratic fUIlctions

A quadratic vector function of n variables may be specified as:

¢(x) = ¢ + a' x + x' B x (14.4.1)

- 0 - -- - -

Here, ~ is an n-dimensional vector, the elements of which are

the arguments of the function.
340 CHAPTER XIV

The n-dimensional row-vector a' indicates the linear

component of the function. For ~ = 0 the function-value
becomes cp = cp , hence the constant cp indicates the function-
value at the ~oint of x = o. 0

The matrix B is square (of order n by n), and is assumed to be

known. An expression like x' B x is named a quadratic form.
The quadratic form x' B x gives-the quadratic component of the
function, the two other terms, CPo + a' x constitute the linear
component. (See also section 2.11)

Example
2 2
cp = (xl - 3) + (x 2 - 5)2 + (2 xl - x )
2
2
36 - 6 x - 10 x 2 + 5 xl + 2x2 2 - 4 xl x
1 2
leads to

cp
o
= 36 and a' = [-6 -lOJ
Unless a further condition is imposed, the matrix B is not
fully determined by the function.

We could write it as

I 5
B , or as B
l-4
or as any linear combination of these two matrices.

This is because a quadratic form is its own transpose

x' B x = x' B' x

Unless the associated matrix contains other information which

requires a non-symmetric presentation, a quadratic form is
conventionally presented in symmetric form.

We may, in effect take the average of the two presentations

offered.

This convention determines the quadratic form in the example as

B = r -2l
5

1-2 2J
TOPOLOGY OF FEASIBLE SPACE AREAS 341

The existence of extrema and the convexity of quadratic

functions is to a considerable extent determined by the
properties of the matrix B.

In this connection the term definiteness occurs. We distinguish

positive definite, positive semidefinite, indefinite, negative
semidefinite, and negative definite matrices.

The definitions of these terms are

"
positive definite: x'Bx > 0, for all x 0.

positive
x'Bx > 0, for all x.
semi -defini te:

indefinite: x'Bx < 0, for some x, but also

x'Bx> 0, for some 'Other x.

negative
x'Bx S- 0, for all x.
semi-definite:

negative definite: x'Bx < 0, for all0.

x "
The definiteness of a square matrix (which by convention is
assumed to be a symmetric matrix) is in its turn determined
by the determinantal equation (normally referred to as the
characteristic equation)

IB-lAl =0. (14.4.2)

In view of the definition of singularity, the requirement

(14.4.2) implies, for some non-zero vector v , the equivalent
definition of singularity.

Bv = Av. (14.4.3)

Equation (14.4.2) is called the characteristic equation, the

vector v, associated in (14.4.3) with any particular root,
is the corresponding characteristic vector, the number A itself
is also known as a latent root or eigenvalue of the matrix.

For a symmetric matrix all the roots of the characteristic

equation are equal.

Readers who are not familia~with the arithmatic of complex

numbers may wish to take this statement on trust, and only
refer to the example below instead, but a proof is here
provided for the benefit of those who are.
*For further referer,e (in climbing order of algebraic sophisti-
cation) see: Theil [33], Chapter 1, Hohn (21], Chapter 10, and
Parlett [29]
342 CHAPTER XIV

Proof

Lennna

If a + Si is a root, a - Si is also a root, and the corresponding

characteristic vectors may be required to be v + u i where v
and u are real vectors, i = I=i (i.e. to consist also of pairs
of complex numbers).
(e
Pro'of (of the lennna)

Suppose

B (~ + u i) (v + u i)(a + Si) va - uS + (~S + ua) i.

(14.4.4)

We must then assume the real and the irrational parts in

(14.4.4) to balance separately, i.e.:

Bv ~a - ~S, as well as

Bu i (vB + ~a) i (and obviously Bu = vS + ~a).

Upon taking the difference rather than the sum of the real
and the irrational parts of (14.4.4), we obtain

B(v - ~i) (v - ~i)(a - Si) = va uS - (~S + ua) i

-(14.4.5)

q.e.d. (for the lemma).

(This lennna applies to all square matrices, not just to

synnnetric matrices).

However, for a synnnetric matrix, transposition of (14.4.5) yields

(~. -~' i) B = (a - Si) (~' - ~'i). (14.4.6)

Therefore, pre-multiplication of (14.4.4) by ~' - ~'i yields:

(~' - u' i) B(~ + ~i) v'Bv + u'Bu (~'~ + u'u)(a + ~i).

- - (14.4.7)

In (14.4.7) the expression in the middle, v'Bv + u'Bu, is real,

therefore the righthand side expression (~T~ + ~'~)(a + Bi)
must also be real.
TOPOLOGY OF FEASIBLE SPACE AREAS 343

We must also exclude v'v + u'u = 0; v = 0, u = 0 would imply

that a + Bi was not req~ired to be a-root, (14.4.4) becoming
trivial. Therefore, we must assume B = 0 for the righthand side
to be real.
q.e.d. (for the theorem itself).

Note

On casual reading of this proof, it may not be immediately

obvious where the symmetry enters the proof.

This is in fact the transposition from (14.4.5) to (14.4.6):

the transposition symbol for B itself has been omitted, writing
B instead of B'.

Example (of the characteristic equation and the real roots

theorem)

The characteristic equation therefore is:

5-)"
1 -21 = (5-)..) (1-)..) -4 O.
-2 1-)"

This equation solves as

6 ± 136 - 4
3 ± 18,
2

both roots being of necessity real.

Once we solve the roots of the characteristic equation, we have

no difficulty in identifying the definiteness of the matrix.

The following rules apply:

Positive definite matrices have only positive roots.

(x'Bx > 0 for all x # 0 if and only if Bv = ~).., v # 0 implies
)..-> o.)
Positive semi-definite matrices have only non-negative roots.
(x'Bx > 0 for all x if and only if Bv ~).., v # 0 implies
)..-> 0.)
344 CHAPTER XIV

° °
Indefinite matrices have both positive roots and negative roots.
(~' Bx > for some ~, and ~'By < for some y, if and only if

Bv AV, for A>O, v l' 0, and also

Bu y~for y<O,u1'O.)

°
Negative semi-definite matrices have only non-positive roots
(~'B~~ for all x if and only if

Bv implies A .::. 0).

Negative definite matrices have only negative roots (~'B~'::' 0

for all x l' 0 if and only if

Bv A~, ~ l' 0 implies A < 0).

(Proofs of these rules follow from evaluation of the appropriate
expressions v'Bv = AV'V, and a systematic analysis of the
confirmations and contradictions of the various definiteness
properties which they imply).

Examples

I -~ has the roots 3 + rs > °and 18 >

l-2
5 3 - 0
IJ
and ~s therefore positive definite both roots being positive

(the quadratic form [xl x21

may also be written as x 1 2 + (2x l - x 2 )2, confirming its

positive definiteness.)

[-23 -2J1
characteristic equation

I
3-A
-2 I (3-A) (I-A) -4
2
A -4A + 3 - 4
2
A -4A -1 o.
-2 I-A

4 ± 116 + 4
2 ± /5
2
TOPOLOGY OF FEASIBLE SPACE AREAS 345

2 + /5 > 0, 2 - Is < 0, the matrix is therefore indefinite,

one root being positive, one root being negative.
(N.B.: A different method of recognising definiteness will be
discussed in the next section).

Theorem

A quadratic function ~(~) = ~

o
+ a'x + x'Bx

is strictly convex if and only if B is negative definite.

Proof

The definition of strict convexity, expressed for a quadratic

function by substitution of the right hand side of (14.4.1)
into (14.2.3) is:

~ + a' (px* + (l-p) ~**)

o - -
+ (p~* + (l-p) ~**)'B(p~* + (l-p)~**) >

p(~ + a'x* + x*'Bx*) + (1-p) (~ + a'x*'~ + _x**'B_x**)

o 0 --
(14.4.8)

to be true for all x* ~ ~**, p in the interval 0 < p < 1.

(In (14.4.8) the linear terms cancel immediately, leaving the

equivalent condition

(p~* + (l-p) ~**'B (p~* + (l-p)~**) >

p~*' B~ + (l-p) x**' Bx** (14.4.9)

(for all x* ~ ~**, p in the interval 0 < p < 1)

working out (14.4.9) we obtain

p2~*'B~* + 2p(1-p) x*'Bx** + (1_p)2 x**'Bx** >

p~*'B~* + (l-p) x**'Bx**.

or equivalently

p(p-l) ~*'B~* - 2p(p-I)~*'B~** + p(p-l) x**'Bx** > 0

or equivalently

p(l-p)(~* - ~**) 'B (~*-~**» 0 (14.4.10)

(for all x* ~ ~**, p in the interval 0 < p < 1).

346 CHAPTER XIV

In view of the condition on p, the factor p(l-p) is positive,

and we end up with the definition of strict convexity being
equivalent to the negative definiteness of B.
q.e.d.

Theorem

A quadratic function

cP + a'x + x'Bx
o
attains a unique maximum if and only if B ~s negative definite.

Proof

Assume the existence of a maximum, therefore, for some

vector x = x* (which mayor may not be the unique maximum)
the necessary first order conditions

l<t a + 2Bx o (14.4.11)

dX

are satisfied.

Substitution of -2B~* for a in the function expression (14.4.1)

permits us to express the quadratic function as

cP (x) =
CPo - 2x*'Bx + x'Bx

cp - 2x*'Bx* - 2x"<' B (~ - ~*) + x'Bx

0

= cp
0
- x*'Bx* + (~ - ~*)'B(~ - ~*) (14.4.12)

Once a first-order solution to (14.4.11) is shown to exist,

(14.4.12) proves that, if the first-order solution corresponds
to a unique maximum, B must be assumea co be negative definite
(i.e. the quadratic form (x - x*) 'B (x - x*) to be negative
for a 11 ~ - ~* of o. -
Conversely if B is negative definite, all its roots are negative,
therefore none is zero and a unique solution to the first-order
conditions (14.4.12) illay be found by inversion of B.

Once the expression (14.4.12) exists, it also proves that

negative definiteness of B implies the existence of a unique
max~mum, with x x* being the point where it is obtained.
q.e.d.
TOPOLOGY OF FEASIBLE SPACE AREAS 347

The two theorems together imply that all three of the following
statements are equivalent:

1) A unique maximum exists;

2) The function is strictly convex;
and
3) B is negative definite.

The similar set of statements concerning the existence of a

unique minimum of a strictly anti-convex function, i.e. the
equivalence between the statements:

1) A unique minimum exists;

2) The function is strictly anti-convex;
and
3) B is positive definite,
will be obvious.

Note, however, that a more complex situation arises in the

case of semi-definiteness.

Of the two main theorems discussed in this section, the one

on convexity permits immediate generalization to the semi-
definite case.

A quadratic function, ~(x)

-
= ~
0
+ a'x + x'Bx is (properly)
-- --
convex if and only if B is negative semi-definite.

(The proof is analageous to the strictly convex/negative

definite case, and the reader is invited to go through this
proof again, and to change, where appropriate, > signs into
> signs, < signs into < signs, the words "strictly convex"
to "convex" and "definite" to "semidefinite".)

The generalization of the other theorem on the existence of a

maximum is, however, slightly different:

A quadratic function, ~(x) = ~o + a'x + x'Bx attains a maximum

only if B is negative semi-definite.-

Not only has the word "unique" disappeared and the word "semi"
appeared, but the words "if and" have disappeared as well;
there is no longer full equivalence between the conditions.

There are functions which are characterized by a semi-definite

matrix B, which have no finite maximum.

The obvious example of such a function is a linear function:

a zero matrix is negative semi-definite. (~'B~2.0 for all
348 CHAPTER XIV

x is satisfied for a zero matrix as x'Bx - 0). Other examples

may also be given, e.g.
2 2 2
~ = xl + x 2 - (xl - x 2 ) xl + x 2 - xl + 2xI x 2 - x 2

Here B

-1+2
r: -:]
1
has one negative root,

1 1
, A -2,
I -1+2 1 1

and one zero root.

The existence of a finite maximum depends in such cases also

on the linear component of the function.

has a maximum value of = 0.25, which is attained whenever

xl and x 2 satisfy the condition xl - x 2 0.5.

In terms of the formal algebra, we note that if B is singular

(14.4.11) defines requirements which may be contradictory, or
consistent, in which case they do not determine a unique max~mum.

We must begin by assuming the existence of a first order solution,

a point which follows in the strictly definite case from the
invertability of B.

A strictly convex quadratic function always has a finite maximum;

for a convex but not strictly convex function, we can only say
that a solution which satisfies the necessary first order
conditions is a (non-unique) maximum.

The indefinite and positive (semi) definite cases on the other

hand, are clear, at least as far as (the absence of) maxima is
concerned.

If B is indefinite, then, even if there is a first order solution,

this is a saddle-point.

If B is indefinite, but none of its roots is zero, we can be sure

of th'e existence of one unique first order solution, which may be
found by inversion of B. The expression (14.4.12) will then
exist' but disprove that the solution thus found is a maximum, as
(~* - ~)'B(~* - ~) may be positive.
TOPOLOGY OF FEASIBLE SPACE AREAS 349

Example

a' = [1 lJ '

Here B is indefinite, as may be illustrated by

0] [1
< 0, but also >0

(The roots are -! + ! 15 > 0, and -1 _1

2 2 15 < 0.)

The first order conditions are

= 1 - 2xl + 2x2 = o.

These conditions permit a unique first order solution,

Xl = 1, x 2 = -0.5.

This solution indicates a function value of

¢ = 1 - 0.5 -1 -1 -1.5,

which is not, however, a maximum, as may be illustrated by

¢(O,O) 0, and also by ~(1,1000) = 1 + 1000 -1 + 2000 = 3000.
Nor is it a minimum, as may be illustrated by ~(1000, 0)
= 1000 - 1000000 = - 999000.

If B is indefinite, but one of the roots is zero, there is

either no first order solution at all, or if there is one, it
cbrresponds to a horizontal ridge, i.e. there is a range of
vectors which all satisfy the first order conditions but do not
indicate a maximum or a minimum.

If B is positive semi-definite, there may, or may not be, a

minimum, but there is no finite maximum. (And no saddle-point
either). Any solution which satisfies the first order conditions
permits the expression (14.4.12) to exist, and if B is positive
semi-definite, this means that the solution in question is a
minimum.
350 CHAPTER XIV

If B is also positive definite, such a minimum is also unique.

14.5 Subspaces and the partitial inversion of definite matrices

The association between a definite matrix and an extreme point of

a quadratic function allows us to state and prove an algebraic
theorem, the proof of which would be much more complicated, if it
had to be provided by strict algebraic means

Theorem

Let

(14.5.1)

be a (positive) definite matrix Bll and B22 being square diagonal

blocks

Then

(1) Bll and BZ2 are also positive definite

and
-1
(2) B22 - B2l Bll B12 exists and is also positive definite.

Proof

First, we partition the quadratic form.

By assumption (B being positive definite)

x' B x

(14.5.2)

is true for all x ~ o.

For ~2 = 0, (~l ~ 0) (14.5.2) reduces to

~i BU ~l > 0 (14.5.3)

for all ~l ~ 0, showing Bll to be pos~t~ve definite. By the

same argument, applied for xl = 0, (x 2 ~ 0) B22 is positive
definite, q.e.d. ad (1). - -

Concerning (2), we denote as x = x* the vector at which a

quadratic function attains its (unconstrained) minimum.
TOPOLOGY OF FEASIBLE SPACE AREAS 351

We consider the minimum value of the quadratic function

<P (x) = <P + a' x + x' B x (14.4.1)

- 0

where ~ is subject to the restriction

(14.5.4)

i.e. Bl ~ is re-named as Z and required to maintain the value

this vector attains for x*, the point at which a quadratic
function is maximal. The-partitioned equivalent of (14.4.1) is

+ X ' + X '
-1 -1

(14.5.5)

Since Bll has been shown to be positive definite, this block

is inverfable. From (14.5.4) we have the following expression
for ~l.
-1 -1
~l = Bll Z - Bll B12 ~2 (14.5.6)

Substitution of the righthand-side of (14.5.6) for ~l into

(14.5.5) yields after transposition and re-grouping of some
terms:

+ ~l
, -1
+ Z' Bll
-1
<P (~) <Po Bll Z Z
, -1
+ [a ' - a
-2 -1 Bll B121~2
, -1
+ ~2 [B22 - B2l Bll B12J~2 (14.5.7)

This is a quadratic function in ~2.

Within the Bl ~ = Z subspace, ~2 determines the value of the

function completely. Because B as a whole is positive definite,
the function has a unique minimum and this minimum is in the
Bl ~ = Z subspace. Then by the theorem shown in the previous
section such a minimum implies that the matrix [B22- B2l Bll-1B12J,
which arises in the quadratic forms is positive definite.
q . e . d. ad ( 2) .

Since a positive definite matrix B is associated with a

negative definite matrix -B; we have the following:

Corollary:

If B LS negative definite, then Bll and B22 are also negative

352 CHAPTER XIV

-1
definite, and BZZ-BZ1B11 BIZ exists and is negative definite.

14.6 Definite matrices and diagonal pivoting

Theorem

Let D be a square, symmetric and positive-definite matrix.

v' D v > 0 (14.6.1)

true for all v ~ 0

Let 0 be a positive non-zero number

o > 0 (14.6.Z)

Then, irrespective of the content of the vector z

-1
[
D + ~ 0 ~'
M = (14.6.3)
Z I 0- 1

is a positive-definite matrix.

Proof

Suppose by contra assumption

[~1 < 0 (14.6.4)

to be true for some~, A, not ~ = 0, A = O.

Upon working out the expression (14.6.4), we obtain

v' D v + v' z 0- 1 z' v + Z v' z 0- 1 A + AZ 0- 1 < 0

-(14.6.5)

To present (14.6.5) more compactly, we may denote

~' ~ = y 14.6.6)

and (14.6.5) is written as

-1 Z Z
v' D v + 0 (y + Z YA + A ) < 0 (14.6.7)
TOPOLOGY OF FEASIBLE SPACE AREAS 353

or equivalently

v' D v + O-l(y + \)2 :5... 0 (14.6.8)

Since 0 is by assumption positive non-zero, we infer from

(14.6.8)

v D v < 0 (14.6.9)

Unless we assume the v is a null vector, (14.6.9) amounts to a

direct contradiction of (14.6.1), the positive-definiteness of
D.

This would show that the contra assumption (14.6.4) was untrue,
except possibly for v = o.

However, for ~ = 0, (14.6.5) reduces to

\2 0 -1< 0 (14.6.10)

which can only be true for \ = 0 (0 being positive non-zero).

Since v = 0, 0 = 0 is not part of the contra-assumption (14.6.4),

that contra-assumption is untrue in all cases and M must be
assumed to be positive definite,
q.e.d.

The above theorem is relevant, largely because it shows a way

in which a positive definite inverse may be built up.

Let

(14.6.11)

be a partially inverted matrix. D being already a positive

definite sub-inverse, 0 (0 > 0) being the next pivot. Then M
is the inverse, and is shown to be positive definite.

Example

D
[: J. 0 1, zI [1 1J

-i-1] ' [3i

1 2

lJ
Q 2
M =
3
= [:
1 1

Both D and M positive definite.

354 CHAPTER XIV

(The roots of a non-singular matrix and its inverse are each

others reciprocals, as may be seen by inverting Au ~A, to
obtain ~ = A-l~A, therefore A-l~ = ~A-l.)

The following corollae are stated here without further proof.

For

Q , with Dll and D22 both positive

definite,

M = [Dll + ZD 22 -1 Z I
(14.6.12)
ZID 1
22
is positive definite.

If D in (14.6.11) is negative definite and 0 negative non-zero,

M is also negative-definite.

Example

D
r-: -:J Zl [1 lJ , 0 -1

Q
n -3
1
1
-1]
-1
-1
M

Similarly, if in (14.6.12) Dll and D22 are both negative

[-i
-1
1
-4
-1
-1]
-1
-1

definite, M is also negative definite.

These theorems allows us to test the (positive) definiteness

of symmetric matrices with positive non-zero diagonal elements,
as well as the positive semi-definiteness of matrices with non-
negative diagonal elements. If a matrix can be inverted by
positive pivots along the main diagonal, it is a positive
definite matrix. As soon as a negative number appears on the
main diagonal, it is found to be indefinite. If a zero appears
on the main diagonal, the matrix may be positive semi-definite,
or indefinite. These two cases may be distinguished as follows:

If a zero appears on the diagonal while other non-zero diagonal

pivots further to the bottom-right hand side are available,
re-order the quadratic form, taking the next positive non-
zero diagonal pivot.
TOPOLOGY OF FEASIBLE SPACE AREAS 355

If a two by two block

appears, with t .. t J~
.. '" 0
~J

this proves indefiniteness, since

= 2 t.. x. x. 2 t .. x. x. < 0
~J ~ J J~ ~ J
is true whenever x. and ~j a:e bot~ non-zero and opposite in sign,
whereas the same e~press~on ~s pos~tive non-zero whenever x. and
x. are non-zero and of the same sign. ~
J
Exercise 14.6a

Test the definiteness of:

A
[~
-1
~2 -~J3
Exercise 14.6b

Test the existence of extrema of the following functions:

2 2
¢l (x l ,x 2 ) xl + 2x2 + 2xl + 2x l x 2 + 4x2
2 2
¢2(x l ,x 2 ) x - x 2 + 2xl - 2x l x 2 + 4x2
1
2 2
¢3(x l ,x 2 ) xl - 4xl + 4x l x 2 - x2
2 2
¢4(x l ,x 2 ) xl - 2x
2
- xl + 4x l x 2 - 4x2

(Formulate the function as a linear funcLLon plus a quadratic

form associated with a symmetric matrix, and extract the roots
from the matrix. Cross-check by inversion. Verify the first-order
conditions only in the case of semi-definiteness.)

(There are answers to these exercises at the end of the

chapter)
356 CHAPTER XIV

14.7 The factorization of a semidefinite matrix

Theorem

Let D be a square, symmetric and positive semi-definite matrix

of order n.

(~' D ;!... ~ 0 for all ;!...)

Then there exists a real matrix A of the same order as D,

satisfying

A' A = D (14.7.1)

where A may be required to be lower-triangular, with each

diagonal element a .. being non-negative.
1.1.
Proof

We first deal with two trivial cases, viz:

a) n 1

Then A

q.e.d. for case a

b) D is a zero matrix

Then A is also a zero matrix.

q.e.d. for case b.

Furthermore d .. = 0 (i=1,2 •... n), d .. # 0 (some i # j) implies

. d e f"1.n1.teness,
1.n 1.1. see sect1.on
. 14 .6. 1.J

In the general non-trivial case, D may now be partitioned as

D (14.7.2)

where d 22 > 0 applies.

(Not finding a positive non-zero diagonal element would imply

that one of the special cases discussed above is applicable.)
We now assume the theorem to be true for matrices of order n-1.
TOPOLOGY OF FEASIBLE SPACE AREAS 357

We also note that positive semi-definiteness of D implies

positive semi-definiteness of Dll - ~12 d 22 - 1 ~i . (See again
section 14.6). Therefore, by application of the theorem for a
matrix of order n-l, there exists a lower-triangular matrix B,
satisfying

(14.7.3)

with each diagonal element of B being non-negative.

We also denote

~2 c (14.7.4)

We now express D as

t' I
~12

c
c-l
k+ J
r"
-1 -1 -1

I ::1
- ~12 d 22 ~i2 + ~12 c c ~12

~' 12

[u J i

d 22
l
(14.7.5)

~i2
Therefore, for matrices of order n, the assumption that the
theorem holds for matrices of order n-l, permits us to express A
as

A
=[:~] (14.7.6)

and the proof follows by recursive induction.

q.e.d.
358 CHAPTER XIV

Example

[4J - [2] 0.25 [2] [3]

c = 2

The following corollae will be obvious:

If D
is positive definite, IDr"
0, all the diagonal elements of A
may be required to be non-zero, since they may already be
required to be non-negative, we conclude to a .. > 0.
1.1.
If D is negative semidefinite D may be expressed as

D = -A' A (14.7.7)

with the same conditions on A.

14.8 The constrained maximum of an anti-convex function, with

linear side-conditions

Recall section 8.11. A linear function constrained by linear

side-conditions attains its maximum (within the specified
feasible space area), at a vertex. This theorem is also applicable
to anti-convex functions, (of which the linear function is the
borderline case).

Theorem

Let x be an n-dimentional vector.

Let T = T(~) be an anti-convex function.

Let a feasible space area be defined by x. > 0, j=1,2, ... , n,

and a! x. < b. i=l, 2, ... , m. 1.
-1.-1. - 1.
Then

The maximum value of T(X) inside the feasible space' area (if
finite) is attained at a
point x = x*, which is a vertex i.e.
the number of non-zero elements-of x* is not greater than the
number of exactly met restrictions a!x* = b. and the non-zero
* -1.- 1.
s~b-vector ~l=~l may_be descri~ed by the inver~e of a block-
p1.vot ~s ~l = ~! = Ali~l' prov1.ded the system 1.S ordered
accord1.ngly.
TOPOLOGY OF FEASIBLE SPACE AREAS 359

Proof

If ,(x) is anti-convex, T(X) ~ t(x*) is an anti-convex

restrIction. Therefore (see section 14.3) the tangential

equivalent :::!...' (~-~*) = [~~ (~*)J '(~-~*) ~ ,(x*) is violated for

all points for which T(X) < T(X*) is true. Therefore, if t(x)
attains (within the specified feasible space area) its maxImum
at ~ = ~* the linear approximation of the objective function

T*(~) =[~~(~*)J' (~-~*) + T(~*) also attains its maximum at ~ = x*.

(All points at which T*(X) > T*(X*) is true also satisfy

T(x» T(X*); since ,(x*) is the-maximum feasible value, such
poInts must be infeasible points.) Therefore, x = x* is an
optimal solution of the problem of maximizing T*(X), subject to
the same side-conditions. Thus, the familiar L.P.-property of
the optimum being of a vertex is applicable.
q.e.d.

The following notes recapitulate the proof as given in part II

in the linear case, now extended to the general case:

An anti-convex function does not attain an unconstrained maximum,

neither in the feasible space area as a whole, nor in any linear
sub-space. (See also section 6.7 where the equivalent lemma was
stated and proved for the case of a linear objective function
and side-restrictions of an unspecified type.) Only a strictly
convex function attains an unconstrained maximum. We postulate
the possibility to successively identify additional restrictions
that are binding on the optimal and feasible solution. Within
each linear subspace it is possible to pivot a non-basic
variable against the identified restriction. This defines a
residual problem, except when the last restriction has been
identified and remaining non-basic variables are optimal at their
non-negativity restrictions (if present). In each residual
problem a new binding restOriction may be identified. Therefore
the total number of identifiable and invertable binding
restrictions equals the number of non-zero variables in the
optimal solution q.e.d.

N.B.
A theorem which is very similar to the one discussed in this
section, but more general in considering non-linear side-
conditions, will be discussed in section 15.5.

Answer l4.6a

Nr 2 is positive semi-definite, the others are indefinite.

360 CHAPTER XIV

Answer l4.6b

For <PI we find B zJ ; roots 3 + h > 0 and 3 - h > 0

Therefore, this is a positive definite matrix and the function

has a unique minimum.

For <P 2 , we find B =[2

-1
-z] ; which has the same roots as for

<PI and the same conclusion follows.

; roots A1 = - 5 , o

Therefore, this is a negative semi-definite matrix, and we need

to verify the first-order conditions, which are

Cl<P3
Clx 2

Substitution of 2xl for x 2 by the condition on x2 reduces the

condition on xl to 1 = 0, therefore these conditions cannot be
satisfied, and <P3 has no finite extremum value.

For <P 4 we find B = [-~ -zJ

This matrix has the same roots as found above for <P 3 ; we
therefore need once more to verify the first-order conditions
which are:

Substitution of 2x2 + 0.5 for xl into the condition on x 2 now

causes the latter condition to reduce to the trivial requirement
o = 0, and we find that <P4 attains a maximum value, whenever xl
and x 2 satisfy the condition xl = 2x2 + 0.5.
TOPOLOGY OF FEASIBLE SPACE AREAS 361

''1
I I
I

1141
IX 1
9 r a ph
I I
I I 1 4 2 e x s u b 5
1121
I I
I I
Pe r p he r a L L Y
1'0
C 0 n vex
IS
I
I
1&
I

14
I

'2
I

- 6 ______
-4 -2 L '0- - n- - t<.- -16

I
12
I

,, ~4
I
\
!~

01
-
I
I
I
II~
I
9 rap h
, I 1 4 2 e x sub 6
, '12
VI,
,, I

a n e m p t y set
110

+,
, I
obeys aLL
IS requirements.
""
x/
,
16
'-- /

.... /
/
I
..... I
,,4
"/
I.. -1-... ;"
/

,'2
I
X2
'--~--~~4---~--~~--~--~--~~--~)
_6 -4 -2 10 12 14 16
------r
--- ------
.-
/
/
362 CHAPTER XIV

, [-
: ~ITXl 9 r a ph
I II
1 4 2 e x sub 7
II
112 ,
~
I I[
I II n 0 t c 0 n vex
1'0 i-
a t a l l

12
X2
.,. LJ-_=~='='-~_.J...L~.
- - '- - - - ' - -I:l--'f-----'----- _<
' _.c u)_
I 0 -=-~.= - - - - _- 16
~~ __ L
~2_ _____________________
I

" , ~l,
I

" \
-6 -

9 rap h
1 4 2 e x sub 8

per pherally
convex

"'I
XI
'- I

-.. I
I

.,. I
I
"
,,/
'- + ./
/

-2 1 2 10 12 '" 16 18
~----------------------
-2
---
CHAPTER XV

OPTIMALITY CONDITIONS

15.1 The additive properties of restrictions

Consider a set of vector-points, e.g. xl' x x 3 ' which are

required to satisfy a combination (i.e. all) of a series of
z'
restrictions. For example xl' xz' x3 may be required to satisfy

+ 5 > 0

- Xz + x3 - 10 > 0

It follows that xl' xz' x3 must also satisfy

Z
Pl(x l + Xz + x3 + 5) + PZ(x l - Zx Z) + P3(-x Z + x3 - 10) > 0

where PI and p are non-negative numbers, and PZ' which is

associated wit~ the equation xl=Zx Z ' can be any real number.

For example, we may require Pl=l, PZ=-l, P3=3, i.e.

Z
(xl + Xz + x3 + 5) - (xl - Zx Z) + 3(-x Z + x3 - 10)
Z
x3 + 3x 3 - Z5 ~ 0

The reason is that each of the separate terms In this expression

lS required to be non-negative, i.e.

o
and
3(-x Z + x3 - 10) ~ o.
We therefore have the following properties for various classes
of restrictions:

Class a): inequalities of type> 0

If the vector x satisfies

f. (x) > 0
l - -
(15.1. 1.)
i=l,Z ... m

363
364 CHAPTER XV

then

x also satisfies

m
L p •• f. (x) > 0 (15.1. 2)
1 1 - -
i=l

with p. > b for i=1,2 ... m

1-

Class b): inequalities of type ~ 0

If the vector x satisfies

f. (x) < 0
1 - -
(15.1.3)
i=1,2, .•. m

then

x also satisfies

m
L p. . f. (x) < 0 (15.1.4)
1 1 - -
i=l

with p. > 0 for i=1,2 .•. m

1 -

Class c): equations

If the vector x satisfies

f. (x) = 0 (15.1.5)
1 -

~i=l, 2 •.• m)

then

x also satisfies

m
L p . • f. (x) o (15.1.6)
1 1 -
i=l

where p. can be any numbers, positive, zero or negative.

1

These properties are hardly theorems. They follow simply from

the summation of the terms in (15.1.2), (15.1.4) and (15.1.6).
If (for certain values of the arguments-vector) each term is
non-negative, the sum is also non-negative. However, the
OPTIMALITY CONDITIONS 365

non-negativity of a sum of terms, does not require that each

term is non-negative. In the example at the beginning of this
section, x3 ~ 5 satisfies the combined restriction
x3 2 + 3x3 - 25 ~ 0, irrespective of the values of xl and x 2 •
One can obviously state values of xl and x 2 which contradict
any of the individual restrictions. For xl ~ -50, x 2 ~ 0,
x3 ~ 5, none of the individual restrictions is met, but their
combination is satisfied.

15.2 Combined restrictions and their tangental equivalent

Consider the following combination of restrictions on two

variables xl and x 2

xl + !x 2 2 - 5 < 0
2
xl - 2 x 2 + 4x2
I
< 0

Application of the combination-property discussed in the

previous section tells us that xl and x 2 should also satisfy

PI (xl + !x 2 2 - 5) + P2(x l - 2x2 + !x22) < 0

for PI ~ 0, P2 ~ O.

Taking (for example) Pl~3, P2~5 this combined restriction is

2
8x l - 10x 2 + 2x 2 - 15 < 0
2
Since both xl + !x2 - 5 ~ 0 and x - 2x2 + !x 2 < 0 are convex,
a combination of these two restrictions is also convex.

Theorem

Let

f.(x)
1
be a series of properly convex functions.

(i~l ,2 ... m)

Then
m
g(x) L p. f. (x) )
1 1-
i~l )
) (15.2.1)
p. > 0 (i~l, 2 ... m) )
1

is also properly convex.

366 CHAPTER XV

Proof

Recall the definition of a properly convex function. To avoid

a possible confusion with multipliers for functions we now use
the letter w for the relative weight of a particular point.
From Section 14.2 we copy and adapt (the definition of proper
convexity). f.(x) is properly convex if and only if
1 -

is true for all x* and x** and (their non-negative combinations,

i.e.)

(15.2.3)

The corresponding generalization to a combined restriction now

follows from application of the additive property discussed
in the previous section to all inequalities which state the
convexity of the m restrictions by (15.2.2)

m m
E p .. f.(w.x* + (1-w)~**).::. E (w.f.(x*) + (l-w).f.(x**»
i=l 1 1 - i=l 1 - 1 -
(15.2.4)

or equivalently, considering (14.2.1) the definition of the

combined function:

g(w.~* + (l-w).~**) > w.g(~*) + (l-w) g(~**) (15.2.5)

which is the definition of proper convexity for g(~).

q.e.d.

Since all vectors which satisfy a convex restriction also

satisfy any tangential equivalent of that restriction, we have
the following~
Corollary

If x satisfies a series of convex restrictions, that vector

satisfies any tangential equivalent of any non-negative
combination of them.

(Refer back to the example at the beginning of this section:

. of numbers xl and x2 wh1ch
Eac h pa1r . ..
sat1sf1es xl + 4I x2 2 - 5 ~ °
as well as xl - 2x2 + 4x22 ~ 0, also satisfies 8x l - 10x 2 + 2x 2 2
- 15 < 0, because that is a positive combination of the two.
Therefore each pair of numbers which satisfies the two original
restrictions, also satisfies xl < 3.44, because that is the
OPTIMALITY CONDITIONS 367

7
graph 15.2 a
aLL soLutions which satisfy
severaL restrictions, aLso
6 satisfy a combination of them,
and if they are convex, aLso
a tangentiaL equivalent of
the combinat ion.
5

3 5
368 CHAPTER XV

2
tangential equivalent of 8x l ~ 10x2 + 2x2 - 15 ~ 0, at the point
x l =3.44, x 2 =2.5.)

15.3 The Lagrangean expression and the conditions on its

derivatives

A combination of the objective function with one or more of the

restricting functions is indicated as a Lagrangean expression
after the French mathematician La Grange or de la Grange. [251
m
T (~) + L p. . f. (x) (15.3.1)
i=l 1 1 -

(p.1>- 0, i=1,2 .•• m)

The Pi are indicated under various names. In calculus, they are

usually indicated as "undetermined multipliers". In the theory
of mathematical programming they are indicated as dual variables.
This is by analogy to their equivalents in linear programming.
If the theory of optimality conditions (i.e. the Kuhn-Tucker
theorem, see below) is applied to a linear problem, the p. turn
out to be the variables in the dual problem. Because of tReir
economic interpretation they are also sometimes indicated as
"shadowprices". I shall, by and large, follow the general
mathematical programming tradition and indicate these multipliers
as dual variables. The use of the letter p is, however, a
reference to their economic interpretation.

The theorems by John (22] and by Kuhn and Tucker [24] concern
the existence, the derivatives and the extremum properties of
combinations of the objective function and restricting functions.

John's theorem is the most general. Kuhn and Tucker's refers to

convex problems only, and generally assumes stronger conditions
leading to stronger results.

We first form the more general (John's) expression

m
J(x) = p • T(X) + L p. f. (x)
- 0 - i=l 1 1-
(15.3.2)
(p. ~ 0, i=0,1,2 ..• m)
1

where Po may be zero - as distinct from the Lagrangean

expression, where Po is equal to one.

This is not particularly apparent in John's original publication,

but Kuhn and Tucker [24] give an example where John's expression
involves a zero multiplier for the objective function. Kuhn and
Tucker's contra-example involves non-convexity and non-zero
third-order derivatives, but contra-examples with only convex
OPTIMAUTY CONDITIONS 369

functions and no third-order derivatives exist as well. The

Kuhn-Tucker theorem is therefore associated with a constraint
qualification condition.

The constraint qualification condition essentially requires the

stronger result Po> 0 to hold. In practice, the contra-examples
are rather special and one normally assumes the Lagrangean
expression to exist.

Kuhn and Tucker's main theorem is the following

Let

Subject to

f. (x) > 0
1 - -

(i=1,2 .•. m)

x> 0

be a convex programming problem satisfying the constraint

qualification condition.

And let

x = x*
be an optimal and feasible solution to that problem

Then, there exists multipliers Pi ~ 0, such that

a) aL < 0 (15.3.3)
a~ -
is true,* together with the complementary slackness conditions.
viz:

b) p . • f. (x) = 0 (all i) (15.3.4)

11-

p. is positive non-zero, only if f.(x) 0, the restriction

1 1 -
f.(x) > 0 being binding
1 -

*Some authors also mention a similar condition with respect to

the multipliers, with the opposite sign, i.e. ~ > 0
dpi-
(i=1,2 ... m), but this is simply a re-statement of the original
restrictions, i.e. f.(x)
1-
> O.
370 CHAPTER XV

and
dL
c) x.
d x.
o (all j) (15.3.5)
J
J

i.e. the derivative is negative rather than zero, only if the

variable is zero. The conditions (15.3.3) are known as the
dual requirements or optimality conditions, (15.3.4) and (15.3.5),
as the complementary slackness condition. The combination of the
binding restrictions, weighed according to the corresponding
dual variables,
m
l: p. f. (x) > 0 (15.3.6)
i=l ~ ~-

may be indicated as the aggregate restriction.

Example

Maximise

T = xl + x 2

Subject to
2 <
xl + x 2 - 5
< 2

(x 1 -> 0, x2 > 0)

We first put the restrictions in the .:':..0 form

2
-xl - x2 + 5 > 0

+ 2 > 0

+ 3 > 0

The associated Lagrangean expression ~s

+ 2)

- x2 + 3)

The theorem does not instruct us how to find an optimum solution.

OPTIMALITY CONDITIONS 371

Rather it tells us that, when an optimum has been found, certain

conditions will be satisfied by the solution. For this example,
we find a solution by graphical mapping and substitution.

Graphical mapping of the restrictions (see graph l5.3a), and

consideration of xl' x 2 (top-righthand) as the desirable
direction identifies

and

x < 2
1 -
as :he bindin~ restrictions. Hence x l =2. Substitution of 2 for
xl ~n xl + x 2 = 5 then gives us X2 as x2 = 13 = 1.73.

Note the way in which the aggregate restriction is a kind of

"compromise" between the binding restriction.

Accordingly, the Kuhn-Tucker theorem tells us that the Lagrangean

expression is effectively reduced to

,.5

,. a
TXl graph \5.3 a
I LLustrat ion 0 f op-
.lr timality conditions.
3 5
+
/

3.0

U'
XI

X
/'.J

I{\

•. 5
 372 CHAPTER XV

The last term P3(-x + 3) drops out since the restriction is not
binding in the optimum, and (15.3.4) tells us that in that case
the associated multiplier is P3=0.

We are now in a position to establish the numerical values for

PI and P2· From (15.3.3) we obtain

l-p -p =0
1 2
and

For x = 13 = 1.73 the condition on aL gives us 1 - 2 x 1.73 PI

~ 1 - 3.46 PI o. aX2

Hence we are able to solve PI as PI = 1/3.46 = 0.29. The other

multiplier then is P2 = 1 - PI = 0.71. Therefore, in this example,
the Lagrangean is
2
L = xl + x 2 + 0.29 (-xl - x 2 + 5)

+ 0.71 (~xl + 2)
2
x 2 - 0.29 x 2 + 2.87
2
- 0.29 (x 2 - 1.73) + 3.73

The first-order derivative of this expression with respect to xl

does not exist, i.e. is identically equal to zero. This is
because the objective function and the restrictions are all
linear in xl. The derivative with respect to x 2 vanishes for
x 2 =l. 73.

The Lagrangean L = -0.29 (x 2-l.73)2 + 3.73 is a convex function.

This is systematic, whenever the Lagrangean is a combination of
convex functions. For a problem which is formulated in terms
of properly convex functions, the Lagrangean always attains
an unconstrained maximum at the optimal point. For numbers xl
and x 2 which satisfy the stated restrictions we establish the
inequality

The restriction xl + x 2 ~ 3.73 can also be obtained in a

somewhat different way, as follows:
OPTIMALITY CONDITIONS 373

Take the combined restriction or aggregate restriction

2
PI (-xl - x 2 + 5) + P2(-x 2 + 2)

= 0.29 (-xl - x 2 2 + 5) + 0.71 (-xl + 2) ~ 0

2
This combined restriction xl + 0.29 x2 ~ 2.87 has been drawn
in the graph (without shading). The linear restriction
xl + x 2 < 3.73 (again without shading) has been drawn in the
same graph as well, and is its tangential equivalent.

Some, but not all, of the above properties are true for the
optimal solution of a non-convex problem.

John's theorem is more gener~l than the Kuhn-Tucker theorem, but

relative to weaker conditions. Unlike the Lagrangean, John's
expression may involve a zero multiplier for the objective
function.

m
P T (x) + L: p. f. (x) (15.3.2)
a - ~ ~-
i=l

(p > 0, p. > 0 i=1,2 ... m not all p.=zero)

a - ~ - ~

If the multiplier of the objective function is zero, at least

one of the multipliers of the restrictinB functions may be
required to be non-zero. (A Lagrangean expression can be just
the objective function, i.e. none of the restrictions is
binding.)

Otherwise the content of the theorem is the same, i.e. if there

is an optimal and feasible solution the first-order derivatives
of J(x) will vanish for the optimal x=x* and the appropriate
values of the multipliers. But since-John does not require
convexity, the first-order condition
m df.
aJ dT ~
= Po -- + L: p. ax
ax ax i=l ~

does not guarantee a max~mum.

It does not follow that a first-order stationary solution of

the Lagrangean is a maximum of the Lagrangean. Nor does it
follow that (in a non-convex problem) a first-order solution
indicates the optimum of the programming problem.
374 CHAPTER XV

Tx,
S.O

Example graph 15.3 b

,.S
a loea l max imum
Max. T = 3x I + x 2 without a global one.
'.0
Subj. to xl < 4 4
3. s

Xl' x 2 < 4
3.0

(xl' x 2 -> 0)
2. 'i
Form the
Lagrangean 2.0

L 3X 1 + X2 1.5

+ PI (4-x l )
1.0

+ P2(4-x l o x 2 )
O.'i

There is a lo-
cal maximum ~0~0--~0~.S~~I~.0~-71.7S--~2~.~0--~2~.S~~l~.O~-7l.~S---'~.~0~u
4.S
at x l =4, x 2 =1,
and we fino multipliers in the usuCll way.
(3L
= 3 - PI - P2 . x 2 = 0 (xl > 0)
dX l

o (x 2 > 0)

For x l =4, x2=1 these equations reduce to

1 - P2 4 = 0 --?- P2 0.25

3 - PI - P2 . 1 = 3 2.75

Therefore the Lagrangean expression is

3 xl + x 2 + 2.75(4-x l ) + 0.25(4-x l ·x 2 )

12 + 0.25 xl + x 2 - 0.25 xl . x 2

This is not a convex function, and it has no finite maximum, only

a local saddle-point at x l =4, x2=1. And it does not represent a
global optimum. The problem is in fact unbounded, because it
admits infinite values of x 2 . However, if the top of the non-
convex "spine" in the x 2 -direction is cut off, the same
optimality conditions may represent a local maximum whi dl is
also the global maximu~
OPTIMALITY CONDITIONS 375

For example, if the restriction x 2 2 4 is added to the problem.

x l =4,.X2=1, with Pl=2.75, P2=O.25 (as above) is the optimal
solut10n.

The relation between convexity of the Lagrangean and convexity

of the programming problem is further complicated by the
possibility of directional and/or peripheral convexity (see
Section 14.2). A programming problem may in substance "be convex",
but if the objective function is formulated as directionally
convex but not properly convex, and/or some of the restricting
functions as only peripherally convex, the Lagrangean function
may be non-convex.

Example

Maximise

Subject to

This is substantially a linear programming problem with the

maximisation of x l +x 2 as objective function. The optimal
solution obviously is xl=l, x 2 =1. The associated dual variables
are

PI = P2 = 4.

The Lagrangean expression

1S not a convex function.

It does not even have a local maximum at xl=l, x 2 =1. Yet xl=l,
x 2 =1 is the true and only optimal solution of the programming
problem.

Now suppose that a programming problem is properly convex, both

in the objective function and in the restricting functions.
In that case the Kuhn-Tucker conditions (15.3.3) to (15.3.5),
are sufficient to identify a feasible solution as being the
optimum. We saw in section 14,.4 that, for a properly convex
function, the first-order conditio"s are sufficient to identify
a (possibly non-unique) maximum. The Lagrangean is a linear
combination of (by assumption) properly convex functions, i.e. the
376 CHAPTER XV

objective function and properly convex restricting functions,

therefore the Lagrangean is a properly convex function.

It follows that the Kuhn-Tucker conditions identify a maximum

of the Lagrangean. We indicate i.e. denote the maximum value
of the Lagrangean as L(~*)
m
L(~) + L p . . f.(x) < L(~*) (15.3.7)
i=l 1. 1. -

For vectors x which satisfy the specified side-conditions each

term p. (x) is non-negative. (The restriction is defined as the
non-ne~ativity of f. (x).) Hence we establish the inequa1ity*
1. -
m
L (x) + L p. f. (x) < L(~*) (15.3.8)
1. -
i=l 1.

The complementary slackness condition implies that in the optimum,

the terms p . . f. (x) all vanish. Hence the optimal value of the
objective fGncti~n-and the Lagrangean are the same.

Exercise

Solve the contrained maximisation problem:

Maximise L = xl + X z
Subject to xl .2.. 3, x z .2.. 4, xl .. x Z .2.. 4 (xl' x z .:.. 0)

z
Find the optimal values of xl and X by graphical mapping. Form
the Lagrangean expression, and derive the optimality conditions.

Solve the corresponding values of the dual variables

(answer at the head of a further exercise, at the end of the
next section.)

*Generallythis inequality is also valid where, in a non-convex

problem, a local maximum is found to be the global maximum. But
since the optimality conditions do not themselves tell us
whether or not, in a non-convex problem, a local maximum, is the
global maximum, it is nevertheless necessary to verify by other
means whether the global optimum has been attained. Furthermore,
in a non-convex problem L(x*) either must be understood to be
the maximum value of the Lagrangean attained by any feasible
vector x, or simply as the value of the Lagrangean associated
with the optimal solution of the programming problem. In a non-
convex problem (but not in a convex problem) there will generally
be (non-feasible) vectors ~, for which L(~) > L(~*) is true.
OPTIMALITY CONDITIONS 377

15.4 Dual variables and right-hand side variations

The dual variables are indicators of the changes in the solution

value, which arise as a result of changes in the constants of
the restrictions. To show this relationship it is useful to
separate exogenous increments in these constants from the
initially specified functions.

Hence we denote

z.(x)
1 -
= f.1 (x)
-
+ d.
1
(15.4.1)

Above, fi(x) are the originally defined restricting functions,

and the d.-are later changes in the constants of these functions,
causing tfle z. (x) to become slightly different from the originally
specified f.(~)~
1 -

Now consider the optimal solution of an altered programming

problem.

Maximise T(~)

Subject to z. (x) > 0 (15.4.2)

1 -

(i=1,2 ..• m, ~ ~ 0)

We formulate the Lagrangean

m m
I: I: p. (f. (x) + d.)
11- 1
i=l i=l
(15.4.3)

The inequality-relationship (15.3.8) may now be copied and

adapted. For the "new" problem this inequality will be

m m
I: p. f. (x) +
1 1-
I: p. d.
1 1
2 L(~*)
i=l i=l

m m
T(~*) + I: p. f. (x*) +
1 1-
I: p. d. (15.4.4)
1 1
i=l i=l

The inequality (15.4.4) does not by itself depend on the

optimality conditions.

It depends on two things, (a) that (for certain multipliers) a

maximum of the Lagrangean relative to x has been established.
That is, for those particular values of the Pi' no other vector ~
378 CHAPTER XV

yields a higher value of L(x), than the figure indicated by the

right-hand side of (15.4.4)~ (b) That the solution is feasible,
i.e. p. f. (x) > 0 is true for all i.
~ ~-

Granted these two points, the inequalities

m
T (x) < L(~) T (~) + L p. (f. (x) + d.) (15.4.5)
~ ~ - ~
i=l

i.e. the first part of (15.4.4)

will stay binding as long as the complementary slackness

condition is maintained.

Hence we assume that for the changed solution it will still be

possible for the same multipliers and the same collection of
binding restrictions-to maintain

f. (x) = 0 (p. > 0)

~ - ~ -
(15.4.6)
x > 0

By assumption all Pi fi(~*) are zero, and the second set of

inequalities from (15.4.4) reduces to
m m
T(~) + L p.(f.(x) + d.) < T{x*) + L p. d. (15.4.7)
i=l ~ ~ - ~ - - i=l ~ ~

The non-active (=amply fulfilled) (non-negativity) restrictions

will remain fulfilled even after some changes in the constant
non-zero numbers d i . That, at least, is the case for sufficiently
small numbers d .. If variations in the restrictions i.e. in their
constants are c6nsidered, the Lagrangean is not only a function
of the vector x, but also of the specified shifts in the constants.
Thus, (15.4.7)-should be re-written as:

m m
L p.(f.(x) + d.).::. T(~*) + L p. d.
~ ~ - ~ ~ ~
i=l i=l
(15.4.8)

We now evaluate the Lagrangean expression, by first and second-

order approximation.
m
L(~*) + 6x + elL d. + !6~'
L
i=l (ld i ~ [:~:]
(15.4.9)
OPTIMALITY CONDITIONS 379

The first term on the right-hand side of (15.5.9), L(x*) gives

the value of L(~,~) for ~=~*. The second term ~; I fI~ gives the

first-order effect of any changes in the vector x, fix standing

for the increment of x. This term is in fact zero because
aL m aL
is a zero vector. The third term L d. gives
ax . 1 ad. 1
1= 1

(the first-order effect of) the changes in the constants of the

equations. From (15.5.9) we see that there is no o~her
first order effect as a result of non-zero values of the d ..
1
m
This term is in fact L p.d ..
i=l 1 1

The last term gives the second order effect and for a convex
problem it will be negative (non-positive). Thus, (15.5.9)
effectively reduces to

[:~~] .~
m
L(~*) + L p.d. + !flx' (15.4.10)
1 1 -
i=l

The Lagrangean is (by first-order approximation) insensitive

to any changes in the vector x, but it is dependent on changes
in the constants of the equations. And for small changes in the
m
constant, the (linear) first-order term p.d. will dominate L
i=l 1 1
over the second-order effect or any other higher-order effects.
If it is possible to find a corresponding adjustment in the
variables-vector x which keeps the same collection of
restrictions binding, thus maintaining the complementary
slackness condition, the same conclusion holds for the objective
function.

Therefore the dual variables indicate the changes in the

objective function which will arise in response to small changes
in the constants of the restrictions.

Example

Recall our earlier example (at the beginning of Section 15.3),

i.e.

Maximise xl + x 2
2
Subject to xl + x 2 < 5

xl < 2
x2 < 3
380 CHAPTER XV

Now suppose that the binding upper bound on xl is moved outwards

to xl ~ 4. We found the dual variable associated with this
restriction to have the value P2 = 0.711. The original optimum
function valu.e was L = xl + x 2 = 3.73. We would therefore expect
the new function value to be close to L = 3.73 + 2xO.7l = 5.15.
The value of L = xl + x2 = 5.15 is the optimal solution of the
linear programming problem

Maximise L Xl + x 2

Subject to xl + 3.464 x 2 < 8

< 4
X2 < 3

This is the original problem with the following modifications

(a) xl < 3 has been substituted for xl ~ 2, and

2
(b) xl + 3.464 x 2 ~ 8 has been substituted for xl + x 2 ~ 5.

The dual variable gives the cost in terms of the stated objective
function, should a unit of some resource be lost or diverted to
other purposes. This is typically the economist's notion of the
(marginal) opportunity cost of a unit of the resource. For this
reason dual variables are also indicated as imputed prices or
shadow prices.

The linear restriction xl + 3.464 x2 ~ 8 is the tangential

equivalp.nt of the originally specified nonlinear restriction
xl + x 2 2 < 5, the tangential being taken at x l =2, x 2 =13=1.732.

The optimal solution of this linear problem is xl=4, x 2 =1.lS.

However, the solution follows the inwardly curved surface of
the restriction xl + x 2 2 ~ 5 and the new nonlinear optimum is
xl=4, x 2=1, and the optimal function value is only 5.00 instead
of 5.15. Thus the second-order effect is found to be (minus)
0.15.

At this point, it is appropriate to comment on the economic

interpretation of dual variables. In economic allocation
problems, the maximand is either a utility function, i.e.
the problem is consumers' maximization of their personal
satisfaction from a given set of availabilities, or a profit
function, i.e. the problem is the firm's maximization of
profit within the technical limits of its production installations,
or a'social welfare function, i.e. the problem is society's
maximization of the collective supply of goods and services,
within a given set of natural and other resource limits.
OPTIMALITY CONDITIONS 381

graph 15.4 a
X disp laced opt imum and
5.
second order effect.
s.

~.

I.j.

3.

3.

2.

2.
X I 4C. 2
1.
/ X
IV

1.
/' 1(\
o.

4.5

In each case the dual variables indicate by how much the

specified objection function would increase, if the supply
of a particular resource were increased by one unit.
 382
CHAPTER XV

4.5
T X1 graph 15.4
part-answer to an
b
ear Li er exerc i se and
onset to the one in
~.O
this section.

:3.5 optimaL soLution:

Xl = 1, x2 =47
with
pl = 0, p2=0.75 1

p3=0.25.
2.S

2.0

1. S

1.0

0.'5

o0 o.s 1.0 1.5 2..0 2.5 "3.0 3.5 4.0 4.5

Exercise and answer to a previous exercise

The optimal solution of the exercise at the end of seccion 15.3

(but not the optimality conditions), is given in Graph l5.4b.
Reperform that exercise, changing

xl • x 2 ~ 4, to xl • x 2 ~ 8.

In this particular case the second-order term ~n (15.4.10) is

systematically zero.
OPTIMALITY CONDITIONS 383

Check that the change in solution value between the two

versions of the exercise matches the product of the dual variable
of the xl' x 2 ~ 4 restriction (0.25), and the shift in its
constraint (there being no second-order effect).

15.5 Constrained second order conditions: subspace convexity

The necessary first-order conditions discussed in section 15.3

may, or may not indicate a true optimum. Only in convex problems
may this be assumed without further investigation, as may be
illustrated by reference to the example below.

Example l5.5a
2 2
Maximise T - xl - x2

Subject to (xl - 3)2 + (x 2 - 4)2 > 25 ,

(xl' x 2 ) ~ °
In this problem there are four points for which the first-order
conditions may be satisfied, viz. (xl.=.6, x 2 = 0); (xl = 0,
x 2 = 8), (xl = 0, x 2 = 8), and the or~g~n, (xl = 0, x 2 = 0). If
we add a restriction to exclude the origin, e.g. xl + x2 ~ 1
(xl = 6, x 2 = 0) is the true optimum with T = -36. The objective
function ~s in fact the minimization of (the square of) the
distance from the origin, and the non-linear restriction is the
non-convex outside of a circle with radius 5 and its centre in
the point (3,4).

The points (6,0) and (0,8) - and without the additional

restriction also (0,0) - , are local maxima i.e. points where
any change in the solution vector is either associated with
losing objective function value or else with moving out of the
feasible space area. A formal definition is as follows:
384 CHAPTER XV

graph 15.5 a

illustration 0 f
loea l maxima and sadd lepoints

A point x = x* is a (strict) local maximum, if it is true for

all ~x~O~ and £ > 0, but approaching zero, that
either,

T(~ + £ ~~) < T (~) (15.5.1)

~, for some i:

°
f. (x + £ ~~) < (15.5.2)
1 -

(x x* being a feasoble solution)

OPTIMALITY CONDITIONS 385

Thus, at (0,8) we may find ~x' 6,-8

satisfying

- 36 > - 64, and

fl (~ + ~~) = 0, but that point is nevertheless a local maximum

because when moving from (0,8) to (6,0) we must go through the
inadmissable inside of the circle or alternatively, when going
around it, begin with moving away from the origin. (The global
optimum is also a local maximum, but the use of the term is
usually reserved for non-convex problems). The conditions to be
discussed in this section do not necessarily distinguish
between different local maxima in terms of identifying the true
optimum, but they distinguish betweel local maxima and
constrained saddle points, where we may start to "climb" at
once e.g. at (xl = 6, x 2 = 8). (The first order conditions
generally are sufficient to exclude points where one may at once
start to climb "steeply"). A constrained saddle point is thus
a point where the first order conditions are satisfied, but
which is not a local maximum.

To distinguish local maxima from constrained saddle points, we

first formulate a first and second order approximation of the
Lagrangean.

The following expression is therefore exact if all functions

are quadratic, but an approximation - which is generally quite
adequate in the vicinity of a first-order solution x = x* -
if there are non-zero third and higher order derivatives.

L(x) L(x* + /',~)

L(~*) + (3L
3x
(~*» , /',x + !/',~'
[,2L (~.)] /',x
a2x

T (~*) + (~ (~*»' /',x + !/',~'

3x
2
[3 "(
(J2x
(~*)J /',x

m m 3f.
1
+ l: p. f. (x*) +
1 1 -
l: P'(-a
1 X
- (~*» , /',x
i=i t=l

m a2 f.1
+ ! l: p. /',x' (~*) /',x (15.5.3)
i=i
1
a2 x
For the sake of uniformity and simplicity of notation we will
assume that the non-negativity restrictions have also been listed
386 CHAPTER XV

as explicit restrictions. Otherwise it should be understood

that x is the vector of non-zero variables only - or that non-
negativity restrictions do not apply.

Ifx = x* satisfies the first order conditions, the first

two members-of (15.5.3) collapse to

L(x) L(x* + /I,~) (15.5.4)

(Refer also to section 2.18, concerning the rules for

differentiating matrix expressions).

If the quadratic form /I, x , (~*j /I,~ is non-positive

for all /I, x , this confirms that x = x~, is the optimum. Note that
this conclusion does not depend-on the convexity of any
individual function, as may be illustrated by referring to the
example below.

Example l5.5b

Maximise T - (x - 3)2 _ (x 2 - 4)2

1

2 2
Subject to xl + x2 > 100, x l ,x 2 > O.

Here the objective function is the minimisation of (the square

of) the distance from the point (xl = 3, x2 = 4), and the non-
linear restriction is the non-convex outside of a circle with
a radius 10 and the centre at the origin.

Compared with example I5.5a, there has been a kind of inter-

change between the objective function and the binding
restriction and (xl = 6, x 2 = 8) is now the global optimum, as
may be shown by developing the Lagrangean expression:
OPTIMALITY CONDITIONS 387

12 graph 15.5 b

Tx I
a stricLy convex obJ ective function
dominatE's the non-convexity 0 f
the binding restriction.

o 1--(-

12

3)2 _ (x - 4)2 + 2 2
L = - (x - 2 Pl (xl + x2 - 100)
1

aL -2 (x - 3) + 2 P 0 6 > 0)
dx l 1 1 xl (xl

o 8 > 0)

From which we solve for xl 0.5.

388 CHAPTER XV

This permits us to numerically evaluate the Lagrangean as:

2 2
= - ! (xl - 6) -! (x 2 - 8) - 25.

This is a strictly convex function which attains a unique

maximum of -25 at (xl = 6, x 2 = 8).

That no feasible solution vector can assign any higher value

to T, then follows immediately from the definition of a
Lagrangean function i.e. for feasible solutions the value of
the objective function does not exceed that of the Lagrangean.

Convexity of the Lagrangean throughout the coordinate space is

not, however a necessary condition for a constrained maximum,
certainly not if the term maximum includes local maxima. (see
also the example in section 15.3).

To derive necessary and sufficient conditions we must analyze

the possibility that the quadratic form in (15.5.4) is
restricted to non-negative values for those vectors ~x which
respect the side-conditions.

To this purpose we describe any vector ~x as

~ x = A v + cS.s. (15.5.5)

where v is required to satisfy (for all i)

df.1 (x*) ] ' v

[_
dX
_
- -
o (15.5.6)

and therefore, if x x* satisfies the first order conditions,

also,

o (15.5.7)

The presence of the scalars A and cS permits us to normalize

v and .s., requiring:

v' v = 1 )
)
and ) (15.5.8)
)
.s.' .s. 1 )
OPTIMALITY CONDITIONS 389

It will be noted that, in the absence of any restriction on 0,

or on the relative values of the elements of q (nor for that
matter, so far on A), no requirement on ~x has so far been
imposed.

To test whether x = x* is optimal, we now require that q does

not satisfy the equi~alent of (15.5.7), and that both A and °
approach zero, not both of them actually being zero.

These conditions make AV + oq equivalent to E ~ x # 0 as used

E > 0 and A # 0 or °
in (15.5.1) and (15.5.2)~ (The-difference between-the condition
# 0 is immaterial as no restriction on the
signs of ~x or of v and q has been put. The condition that q is

restricts the substantial case °

not merely-another-vector v is not actually necessary but -
= 0, to the formal case O. °
We now re-formulate the condition for a local maximum by
substituting AV + oq for E ~ x in (15.5.1) and (15.5.2) and
putting the quadratic-approximation used in (15.5.3). x x*
is a strict local maximum, if
either:

~T (~: (~*))' (Av + oS) + 4 (A~ + oS)' [:~~(~*) 1(A~ + e.g) < 0

(15.5.9)

or

<If.
M.
1
(ax 1 (~*)) , (A~ + os.) + 4 (A~ +

(some i)
(15.5.10)

applies for all v and q satisfying (15.5.6) and (15.5.8), but q

not satisfying the equivalent of (15.5.6) with both A and
approaching zero (but at least one of them not actually being
° -
zero).

By the first-order conditions (15.5.9) and (15.5.10)

reduce to (15.5.11) and (15.5.12)

x x* is a local maximum, if

/1T (~: (~*))' os. + 4 (A~ + os.) , [-:~: (~*) J(A~ + os.) < 0

(lS.5.U)
390 CHAPTER XV

or
af.
(--~ (x~'»' o_q +
ax - ! (A_v +

(some i)
(15.5.12)

applies for all ~ and ~ satisfying (15.5.6) and (15.5.8), q

not satisfying the equivalent of (15.5.6), A and 0 both
approaching zero, but at least one of them not actually being
zero.

These conditions may also be formulated equivalently, by their

negation:

x x* is not a (strict) local maximum if

+ oq) > 0

(15.5.13)

and

af.
M. ~
~
(--(x*»'oq
ax - - + !(A_V

(all i) (15.5.14)

applies for some v and ~ satisfying (15.5.6) and (15.5.8) A and

o both approaching zero, not both of them actually being zero.
Note that this definition does not accept non-unique local
maxima where the solution value is the same when going for some
length along a combination of binding restrictions, hence the
word strict local maximum.

We now first of all have the following

Theorem (corner theorem)

If all functions are anti-convex (the objective function as well

as the restricting functions, the matrix

ra~ (x*)
[ax - J
may be required to be square, of the same order as the number
of elements of ~ (counting non-negativity restrictions as
explicit restricting functions), and non-singular.
OPTIMALITY CONDITIONS 391

Proof

For 0 = 0, A I 0 (15.5.11) and (15.5.12) cannot be satisfied if

all the quadratic forms are positive semi-definite. Therefore
the assumption that x = x* is the optimum implies that the set
of non-zero vectors v satisfying (15.5.6) is empty.
q.e.d.

(N.B. The words "may be required" relate to the possibility that

more than n restrictions are binding on the optimum solution.)

The reader will note that the familiar LP situation, where we

only need to investigate the corners of the feasible space
area, is covered by this theorem.

On the other hand, if the solution is not ~n a corner we have

the following

Lemma

If x = x j , satisfies the first order conditions then

['-' (,*)1
ax -
q > 0

implies

e (~*)J
dXr ' 3. < 0

Proof

By the first order condition [~.i (~*)], may be expressed as

t;~ ("')] , --E' t:! (",)] , from which the result, given the

non-negat i vi t y of both

q.e.d.
E' and [:! (",)] q.

Therefore except ~n the excluded trivial case that q is in

fact another vector v, the negative formulation of the
definition of a local maximum (15.5.13) and (15.5.14) can only
be met in the case of a non-zero contribution of a quadratic
form.
392 CHAPTER XV

However, if A approaches zero, we can only assume that (15.5.13)

and (15.5.14) are met on account of a non-zero quadratic form
(and violated if a quadratic form is deleted) if the ratio
6/A also app~oaches (or becomes) zero.

Therefore a necessary condition for non-optimality ~s that (for

6 = 0)
i'lL = ~ v'

for some v satisfying (15.5.6).

Conversely for 6 = 0 the positive formulation (15.5.11) and

(15.5.12) shows that

i'lL = ~ [
a 2L
-v' ---
/
2 (x*)
-
1 v < 0
ax
to hold for all v satisfying (15.5.6) is a necessary condition
for optimality. The latter condition has therefore been shown
to be both necessary and sufficient for optimality.

It is stated here without further proof that one may normally

assume that

i'lL = !2 _v' ~a2L

2
a x

for all v satisfying (15.5.6) is a necessary (and normally

sufficient) condition for a local maximum without the word
"strict" added to it. Complications with non-zero third higher
order derivative may, however, arise.

Example l5.5c

(Of the constrained second order conditions)

Maximise

Subject to (xl - 10) (x 2 - 10) > 4

In this problem, the objective function is anti-convex, and the

one non-linear restriction is peripherally convex in the
xl ~ 10, x 2 ~ 10 domain.
OPTIMALITY CONDITIONS 393

Graphical analysis (the actual drawing is left to the reader;

the non-linear restriction is a hyperpo1a with rectangular
axes Xl = 10, and x 2 = 10, the objective function is the
(square of) the distance from the origin) indicates that the
optimum is Xl = x 2 = 8.

This is obviously a method which is not generally available in

the n-dimentiona1 case, and we seek algebraic confirmation.

We develop the Lagrangean

L = Xl 2 + x2
2
+ P1 «xl - 10)(x 2 - 10) - 4)

The zero valued multipliers associated with the amply fulfilled

linear restrictions have been left out from this expression
already.

The first order conditions are

<lL
8 > 0)
aX 1

aL
aX 2 = 2x2 + P1 (xl - 10) o (x 2 = 8 > 0)

and we solve the multiplier as P1 = 8.

The Lagrangean may now be evaluated numerically as

L = Xl 2 + x2
2
+ 8 «xl - 10) (x 2 - 10) - 4)

and the associated matrix of second order derivatives as

t:~;]
This is an indefinite matrix. To verify whether or not it is
associated with a constrained maximum, we need to define the
set of admissable vectors v to figure in (15.5.6).

The linear approximation of the side conditions at Xl 8

is

- 2 dx
1
- 2 dx
2
= 0

i.e. can only be satisfied if the changes in Xl and x 2 are of

the same absolute value but opposite sign.
 394 CHAPTER XV

We evaluate the quadratic form as

[-2

and conclude that xl = 8, x 2 = 8 is indeed a constrained (local)

maximum. Note however, that we cannot by this calculation alone
deduce that it is' the global maximum:

f the restrictions on xl and x 2 separately are replaced by

xl ~ 20, x 2 ~ 20, the global optimum is xl = x 2 = 20, but the
local maximum at x = x 2 = 8 is not affected by such a
re-definition of t~e problem, it still satisfies the conditions
for a constrained local maximum.

Before finalizing this section, it is useful to mention an

equivalent second order condition for a constrained local
maximum. There is actually an older tradition in analysis and
mathematical economics (see for example P.A. Samuelson (32]
pp 61-69), which tends to formulate the second order conditions
in terms of determinants rather than quadratic forms.

To that purpose we form the composite matrix

3f
3 2L (- -=) I
3x
3 2x
M
3f
3x

The second order condition then relates to the principal minors

(= determinants of square diagonal blocks) of M, insofar as the
corresponding minor-matrices include the zero bottom righthand
block. The second order condition for a constrained maximum
then amounts to the non-negativity of each of these principal
minors including IMI itself.

For a unique maximum we may also require IMI > 0.

OPTIMALITY CONDITIONS 395

Example

Maximise T = Xl' x 2
Subject to xl + x 2 ~ 10 (x l ,x 2 ~ 0)

This problem has an optimum of xl = x2 = 5, with PI = 5.

The corresponding matrix M for that example is as follows:

and the second order condition 1S verified as follows:

IMI 2 > o.
Apart from the trivial case of the 3,3 element (and in
general the bottom righthand block itself) which obviously
gives rise to a zero principal minor, M has 5 principal
minors viz: the 1,1 diagonal element, the 2,2 diagonal element
the determinant of top lefthand block [_ a~LI = [ 0 -11
a ~ -1 0
and the blocks which arise from deleting the first column and
row, and similarly the second column and row. The first three
of these do not count, as their minor matrices do not include
the bottom righthand element.

The remaining relevant minors are:

HI , 1
1-: :1 1 > 0, and

1
M2 , 2 =
1-: 0
1 > 0,

Confirming that xl = x 2 = 5 is a local maX1mum.

Note that we do not require IM331

1- :~~I -> 0 as indeed, is

not the case. Such a condition would imply positive semi-

definitene" of [- :::1 (. negative ,emi-definitene" of

396 CHAPTER XV

[:::] , the Lagrangean to be convex throughout the coordinate

space, that would be a too strong condition. We also observe

that the stated condition becomes trivial in the case of a
corner solution. If [~~] is square and non-singular, IMI is

evaluated as IMI = I~! I,

2
and all the relevant minors vanish

on account of the block triangular structure of the minor

matrix.

A proof of the equivalence between the determinantal condition

and the one on the constrained quadratic form may be obtained by
means of analyzing the possible pivots which might be applied to
invert M. (Readers who might wish to derive this proof may find
it helpful to re-read sections 5.8, 14.4 and 14.6, and to read
section 16.6 before pursuing this point.)

Exercise l5.5.a

The following problem is formulated:

2
Maximise T = xl + xl x 2 + x 2

Subject to 4xl + xl x 2 2.. 18

(xl' x 2 ~ 0)

It is suggested that xl = 3, x 2 = 2 is the optimum solution of

this problem, or at least a constrained local maximum.

Verify the first and second order conditions, i.e. whether

this is so.

(Answer l5.5.a at the end of the chapter)

OPTIMALITY CONDITIONS 397

15.6 Non-linear duality

In this section we discuss a t~eorem which is originally due to

Dantzig, Eisenberg and Cottle (9J , and is reported here on the
authority of A. Whinston. [40]

To conform with presentation conventions in this book, the

theorem has been re-formulated. These presentation conventions
require, inter alia,

a) that the original problem (i.e. the "primal") problem is a

maximisation problem, and that

b) the specified primal variables are indicated as ~, and the

specified dual variables are indicated as ~.

We put the usual mathematical programming problem, as follows:

Maximise

T = T (x) (14.1.1)

Subject to

f. (x) > 0
1 - -
(i 1, 2 . . . . . . . m) (14.1.2)

(x ~ 0)

The associated Lagrangean expression therefore is:

m
T (x) + L: p. f. (x) (15.6.1)
i=l 1 1-

We now formulate this primal problem as

Maximise

(15.6.2)

Subject to

dL > 0 (15.6.3)
0E. -

(x > 0, ~ > 0)
398 CHAPTER XV

dL
This is the same programming problem, because is the vector
of function-values f.(x) dE
1 -

f (15.6.4)

Hence the term -p' dL in (15. .2) reduces the Lagrangean to

- dE
the objective function

T (x) (15.6.5)

and (15.6.3) are simply the primal requirements

f > 0 (15.6.6)

which is (14.1.2) 1n vector notation.

The dual problem then is

Minimise
dL
jl = L(~, E) - x' (15.6.7)
dX

Subject to

dL < 0 (15.6.8)
dX -

(~ 2. 0, E 2. 0)

The theorem then states that the optima of these two problems
are characterized by (the complementary slackness conditions)

_p ' -
dL
dE
= ° and x'
dL
ax
0, with the same optimal solution vectors

~, E figuring 1n both problems.

OPTIMALITY CONDITIONS 399

Answer 15.5a

We first form the Lagrangean

The first order conditions are

3 > 0)

o 2 > 0)

Therefore, optimality of this solution requires (by the

condition on x 2 ): PI = (xl+l)/x l = 4/3 = 1 1/3

The solution xl = 3, x 2 = 2, PI = 1 1/3 also satisfies the

condition on xl'

2xl + x 2 - 4Pl - Pl x2 = 6 + 2 - 16/3 - 8/3 = 0

therefore the first order conditions are satisfied

We now evaluate the Lagrangean numerically as

and the matrix of second order derivatives as

2 -1/6J
[
-1/6 -

and the side-conditon o as

This condition is satisfied only by changes in xl and x 2

being in the proportions vector ~' = [1 -2J.
d 2L
The quadratic form~' ~(~*) ~ is now evaluated as:

[ 11l
-2J
[2 1/3 -1/6J 2 2/3 > 0
-2
showing non-optimality of xl 3, x 2 = 2.
Part IV
QUADRATIC PROGRAMMING

CHAPTER XVI

QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 402

16.1 Statement of the problem 402

16.2 The optimality conditions 403
16.3 Outline of the Simplex Algorithm
for quadratic programming 410
16.4 The symmetry property of the tableau
in its standard form 415
16.5 The solution value 419
16.6 The negative diagonal 420
16.7 The non-standard form tableau and its
standard form neighbours 433
16.8 Dual variable elemination and the
parameter theorems. 447
16.9 Step-length and objective function value 456
16.10 Boundedness and artificial feasibility 458
16.11 The treatment of infeasible starting solutions 473
16.12 Ordering of the quadratic programming tableau 482
16.13 Equations and variables without non-
negativity restriction 486
16.14 Lower bounds in quadratic programming 495
16.15 Commented text of a quadratic
programming code 498

CHAPTER XVII

PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 516

17.1 Postoptimal variation of the righthand-side

of a quadratic programming problem 516

400
 401

17 .2 Parametric variation of the linear

component of the objective function of
a quadratic programming problem 525
17.3 Strict convexity in the parameter subspace 530
17.4 Elimination of dual variables during
parametric variation 532
17 .5 Parametric equivalence and the number of
steps on parametric reentry 537
17.6 Parametric solution methods 543
17.7 Computational implementation of
parametric QP 547

CHAPTER XVIII

GENERAL QUADRATIC PROGRAMMING 556

18.1 Statement and discussion of the general

problem 556
18.2 Pseudo-Lagrangean and linear approximation 559
18.3 Verification and primal adjustment of
subsidiary optima 562
18.4 Reapproximations 576
18.5 The objective function limit 585
18.6 Consistency and optimality 538
18.7 The upward adjustment of dual variables 594
18.8 Loss and correction of optimal form 599
18.9 Overview of the SCM algorithm 604
18.10 Code-listing of the SCM algorithm 613
CHAPTER XVI

QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS

16.1 Statement of the Problem

Let x be an n by 1 column vector of unknown variables. w' a 1 by

n row-vector of known linear preference coefficients, D-a known
matrix of quadratic preference coefficients. Let T be a scalar
variable indicating the value of the preference-function, or
objective function.

A 8eneral quadratic objective function lS then specified as

T = w' X + ~ ~' D~ (16.1.1)

where the inner product w'x is the linear component, the

quadratic form ~ x' Dx the-non-linear component. Following
section 14.4, we assume that D is symmetric. If we had originally
specified a non-symmetric preference matrix we may always replace
this by a symmetric one, by takin8 the avera8e between the matrix
itself and its transpose. For example x 1 2 + 2x l x 2 + x 2 2 may be
written as

[: J [::1 or as

[: J [::1 and we might as well write

Generally,

x' D x = ~' UD + ~D'J~ (16.1.2)

lS true by the definition of a quadratic form. (see section 14.4).

To ensure that a first-order solution actually is the optimum,
we may (or may not) require that D is negative semi-definite.

x' D x < 0 (16.1.3)

for all x
402
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 403

This requirement of negative semi-definiteness implies that we

are dealing with a convex preference function. Some implications
of relaxing the requirement of negative semi-definiteness will
be discussed at a later stage.

The algorithm to be discussed in this chapter, deals with a

quadratic objective function, but only with linear side-
conditions. These linear side-conditions are the usual block of
inequalities, similar to the restrictions in linear programming.

A x < b (7.2.2)

and the specified programming problem is

Maximise

T = WI X + Xl D x (16.1.1)

Subject to

A x < b (7.2.2)

In (7.2.2), A will be an m by n matrix of known coefficients, b

an m by 1 vector of known constants, and we normally assume
x > O.

16.2 The Optimality Conditions

Following Dantzig [8] , section 2,4.4, Van de Panne and

Whinston [36], and Cottle [6J , we first state the optimality
conditions. An m by 1 vector of Lagrangean multipliers is
denoted by E and the Lagrangean expression is:

L = W I X + t~ I D~ + E" (~- A~) (16.2.1)

The necessary first-order conditions are then, besides (7.2.2)

and the complementary slackness condition
ClL
WI + Xl D - EIA < 0 (16.2.2)
Clx

or,
presented column-wise as a block-inequality

Dx - AlE 2.. - ~ (16.2.3)

404 CHAPTER XVI

We will assume that x is restricted to non-negative values only.

The quadratic programming problem can now be presented in

tabular form. We seek a permissable solution to the combined
system.

Dx - A'E ~ - w ) (16.2.3)
) x > 0, E ~ 0
Ax < b) (7.2.2)

The solution to this system is also required to satisfy the

remainder of the Kuhn-Tucker conditions, i.e. the complementary
slackness condition that an element of p, a dual variable, is
only allowed to attain a non-zero value-if the corresponding
primal restriction in (7.2.2) is exactly binding, and an element
of x is only allowed to attain a non-zero value if the
corresponding dual restriction in (16.2.3) is binding.

At this point it is useful to again discuss the second-order

conditions for a constrained maximum.

The requirement that D is negative semi-definite combined with

the necessary first-order conditions, is a sufficient condition
to show that a particular vector x corresponds to a maximum.
But this condition is not a necessary condition.

Consider the following example

Maximise T xl . x2

Subject to xl + x 2 < 10

(xl , x 2 -> 0)

problem has a unique optimal and feasible solution,

x 2 = 5. At this point xl • x2 = 25, whereas for example
4, x 2 = 6 yields xl • x 2 = 24.

~l
Yet D is not a ne8ative semi-definite matrix since

[:

[1 lJ
[: ~l [:]
I t is an indefinite quadratic form.
= 2 > 0
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 405

investigate is [-iJ
However, the only variation in the xl' x 2 vector which we should
i.e. xl increasing at the cost of x 2 or

vice-versa; other directions violate the binding side-condition.

Under that side-condition. the quadratic form [.x, hx2] [: ~] [::~

in indeed negative definite.

[1 -lJ
[: ~] [J -1 < 0

or, if the sign is reversed (x 2 increasing at the cost of xl)'

[-1 lJ
[: ~] [-J -1 < 0

To avoid describing local minima or constrained saddle-points,

we introduce the notion of subspace convexity.

The existence of an optimal solution to the problem as a whole

implies the existence of an optimal solution to the sub-problem.

Maximise (16.2.4)

Subject to ~l (16.2.5)

where ~l is the vector of variables which the optimal solution

describes as being non-zero, ~i the cor~esponding vector of
linear preference coefficients, Dll toe corresponding block of
quadratic preference coefficients, All the intersection of the
block-row which describes the binding restrictions, with the
block column which relates to ~l' while ~l are the corresponding
elements of b.

Now refer to section 15.5 and note that the second-order

derivatives of the Lagrangean are (with linear side-conditions),
simply those of the objective function.

The second order conditions for a constrained maximum of this sub-

problem therefore are

(16.2.6)
406 CHAPTER XVI

for all ~l satisfying

(16.2.7)

We will express this condition by saying that the objective

function is convex within the All ~l = ~l subspace.

In anticipation of finding an optimal solution which satisfies

this condition, we impose it also on each current solution, at
least as far as this concerns solutions for which the
complementary slackness conditions has been verified. Moreover,
it will be shown in section 16.6, that once we require subspace
convexity, the existence of an invertable block-pivot also
imples the more stringent condition of strict subspace convexity.

~i Dll ~l < 0,

for all ~l # 0, satisfying

(the objective function is strictly convex within the

All ~l = ~l subspace).

At this point it is useful to put the second-order condition

for a constrained maximum, as developed in section 15.5, in
the form in which this condition appears in the case of a
quadratic objective function and linear side-conditions. This
permits simpler proofs.

Theorem

Let
w be an n-dimensional vector of known coefficients, D a square
and symmetric matrix of known coefficients, D a square and
symmetric matrix of known coefficients, A an m by n matrix of
known coefficients, and b an m by 1 vector of known coefficients.

Then, .
regardless of the content of the vector b

T(X) = w'x + !~'Dx

attains a finite maximum in the Ax b subspace,

only if

v'Dv < 0

~s true for all v satisfying Av o.

QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 407

Proof

If a finite constrained maximum exists, we may express any

actual vector x as

x x* + v = x* + tJ.x (16.2.8)

where x* is a vector at which the constrained maximum ~s

attained.

Furthermore, if maximum exists, we may form the Lagrangean

expression

L = w'x + b'D~ + .£' (.t-A~) (16.2.9)

where on account of the equation-nature of the side-restriction,

p need not be non-negative. We may then (in the absence of
non-negativity restrictions) require, without regard to
complementary slackness
dL
--(x*) = w + Dx* - A'_p = 0 16.2.10)
dX - -
We now substitute x* + v for x by (16.2.8) into (16.2.9), to
obtain

+ (~' + ~*'D - .£'A')~ + !~'D~ (16.2.11)

However on account of the first-order-conditions (16.2.10), and

the assumed equality between Ax* and b (16.2.11) reduces to

L = T(X*) + !~'Dv (16.2.12)

Since for any Av ~ 0 the side-conditions remain satisfied and

the value of L remains equal to T, the result follows.
q.e.d.

N.B. :
Recapitulation of the similar theorem ~n the case of non-linear
side-conditions, as stated in section 15.5:

A constrained maximum exists, only if

~'[~~L(~*)J ~.2. 0
408 CHAPTER XVI

is true for all ~ satisfying

[ai.(x*)l v = 0
[ax - j-
The theorem stated here, relating to linear side-conditions
is included in that theorem as a special case

[~~LJ = [~~T ]= D; -A.

As in the case of convexity throughout the coordinate space the

existence of a maximum implies (subspace-)convexity but
(subspace-) convexity does not imply the existence of a maX1mum.

Example

Maximise

Subject to

The objective function is convex throughout the coordinate

space, therefore also convex in the xl=x Z subspace, but there
is no finite maximum.

Just as in the case of convexity throughout the coordinate

space (see sect ion 14.4) a stronger theorem in the if and only
if form applies in the presence of strict convexity.

There is, however, one obvious caveat, relating to non-

emptiness. We must begin by assuming that the set of vectors
satisfying Ax = b is not an empty set. This point is attended
to by putting additional conditions on the order and rank of A.

Theorem

Let A be an m by n matrix of rank m (m < n), while b is an

m-dimensional vector,D an n by n symmetric matrix, and w an
n-dimensional vector.

Then

attains a unique maximum in the Ax b subspace

if and only if

v'Dv < 0

is true for all v satisfying Av 0, and v f o.

QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 409

Proof

We first state and prove the following lemma

The assumed

v'Dv < 0

for all v ~ 0 satisfying Av = 0

implies the non-singularity of

M [: I -A']
Proof (of the lemma)

Assume by contra-assumption that for some [~' ,~'J ,; 0, we have

Dv - A'z o )
) (16.2.13)
Av o )
For [y' ,~'J ,; 0, v 0, (16.2.13) implies, by its first
block-row

A'z = 0

and therefore for m < n

a contradiction of the assumed full rank of A.
We must therefore assume v I o.

From (16.2.13) as a whole we then obtain

= v'Dv + z'Av - v'Az = v'Dv = 0

contradicting the assumed strict subspace-convexity, in

combination with the second block-row of (16.2.13),

Therefore we must assume (16.2.13) to be untrue for any

[~' ,~'J ,; o.
q.e.d. for the lemma.
410 CHAPTER XVI

The lemma proves that the composite system

Dx* - A'E -w )
) (16.2.14)
Ax* b )

can be solved by inversion, yielding a solution to (16.2.8).

Therefore even if we do not initially assume that x = x*

is an optimum, this is proved by (16.2.12); and in-the-case of
strict subspace-convexity, x x* is shown to be a unique
constrained max~mum.
q.e.d.

16.3 Outline of the Simplex Algorithm for Quadratic Programming

The algorithm discussed in this section is essentially

Cottle's [6) ,hut features which are more akin to Van de Panne
add Whinston [36J , (37] ,are introduced later in this
chapter. i.e. section 16.9 (flying through dual restrictions),
and section 16.11 (the introduction of a separate Phase I).

A difference between the algorithm offered here and the ones

suggested by these authors is that they include unit vectors ~n
their tableaux, while the "shortened" tableau discussed in
section 8.8 is used here, and indeed, even more condensed
tableau-presentations will be introduced later in this chapter.

The algorithm involves the use of a tableau which is rather

similar to a linear programming tableau, but the rules for
choosing pivots, especially the rule for choosing the pivotal
column, are different from those which apply in the case of
linear programming.

In a quadratic programming tableau the objective function row

is needed only for calculating the solution value, not for search
operations. We "present" the quadratic programming tableau as a
linear programming problem, as follows:

Maximise

]l = w' x + b' E (16.3.1)

QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 411

Subj ect to

Dx - A'E.~-w ) (16.Z.3)
) x':::" 0, E. > 0
Ax < b ) (7.Z.Z)

It will be shown in section 16.5, that, for solutions which

satisfy complementary slackness condition, the value of this
pseudo objective function is twice that of the specified
objective function, and we shall write ZT instead of ~.

We write the above system in tabular form, representing an

initial tableau of the trivial basis. We use the "shortened"
presentation of the simplex tableau as discussed for linear
programming in section 8.8, and tabulate the system as follows:

Name x E. < Value

- -

-d D -A' - w
-
.----
A
-s b
-

ZT - w'
-
-b' 0

Here d are the dual slack-variables, associated with (16.Z.3)

and ~-are the primal slack-variables associated with (7.Z.Z).

The trivial basis is d = -wand s = b.

The requirement of complementary slackness is represented
by the name-codes of the variables. The elements of x are
numbered 1 to n, the complementary dual slacks, the elements
of d from -1 to -no The elements of s are numbered from 1000 to
1000 + m, the associated dual variables from -1000 to -1000 - m.

Example

Minimise

i. e. Maximise -xl

Subject to xl + ZxZ .:::.. 3 (i.e. -xl - ZxZ ~ -3)

(xl' X z .: :. 0)

Formulate the Lagrangean expression

412 CHAPTER X\{

and obtain the dual requirements

ClL
- 1 - 2X2 + 2p < 0
Cl~2 1-

or in the tabular presentation with all inequalities in the

< form:

TABLEAU 16.3 A

THE SET-UP TABLEAU OF A QUA-

DRATIC PROGRAMMING PROBLEM.

NAME ! ! I X I X 2 P I II VAl.UE

! CODE !! 2 -1001 II

D 1 -I !! -2 I I I 1
D 2 -2 !! -2 2 ! I 1
S I 100 I !! -I -2 II -3

2T I! 3 II

The complementary slackness condition is now given as not

permitting two names of the same absolute value to be in the
list of basic variables at the same time.

If xl (name-code 1) enters the basis, then· d l (name-code-l),

being the slack in the dual restriction which refers to xl' has
to leave. Likewise PI (name-code -1001) is not allowed in the
list of basic variables, unless sl' the slack variable
associated with the corresponding primal restriction (name-
code 1001) leaves the list of basic variables.

The methods for actually finding a solution permits some tableaux

to be "not in standard form", in which case one name is in the
list of basic variables together with its complementary name.
(The complementary slackness condition is then temporarily
violated.)

The algorithm begins with selecting a "badname" " a variable

which we wish to eliminate from the list of basic variables.
(Or to use Cottle's terminology, a distinguished variable.)
We initially assume that badname-variable selection is according
to _the rule of the steepest ascent, i.e. we select the name
of the variable for which the most negative entry occurs on the
right-hand side. In our example this would be badname = 1001,
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 413

i.e. we seek to eliminate the (negative valued) slack of the

first primal restriction. The elimination of the badname-
variable is anticipated, by introducing the complementary name
in the list of basic variables. Hoping to eliminate sl' we
introduce the dual variable Pl. The complementary var~able
associated with the selected badname variable, is known as the
driving variable.

The choice of the pivotal row is more or less by the same

method as in linear programming, i.e. by the rule of the
smallest quotient. However, there are differences with that
rule, as known from LP. We accept a negative pivot in the
badname-variable row, but not in any other row.

TABLEAU 16.3 B
SET-UP TABLEAU OF A QUADRATIC PRO-
GRAMMING ROBLEM. WITH PIVOT-MARKING.

NAME !! X 1 X 2 PI! I VALUE

I CODE I! 2 -1001 II

D 1 -I I I -2 1 ! ! 1
D 2
5 I
-2
1 001
I I
I I -1
-2
-2
® I!
I!
1
-3

2T II 3 !I

In case of equal quotients, the badname-row has priority over

another row in which an equal quotient might occur. Also,
quite apart from degeneracy, there are restrictions on the
acceptability of dual variables as leaving variables; sometimes
they are allowed to become negative instead. We shall need to
come back to this question, but at this stage, we will illustrate
the algorithm, on the assumption, that the rule of the smallest
quo:ient is applied in the usual way. We find d 2 as leaving
var~able, and develop' tableau l6.3c.

TABLEAU 16.3 C

QP TABLEAU. AFTER ONE STEP. NON-

STANDARD FORM. WITH NEW PIVOT-MARKING.
NAME ! ! I X 1 X 2 D 2 II VALUE
! CODE II 2 -2 I!

D 1 -I I! -2 CD
-1
-0.50 II 0.50
P 1 !-1001" 0.50 II 0.50
51 !IOOII! - I -2 !! -3

2T I! 4 -1.50 II -1.50
414 CHAPTER XVI

Tableau l6.3c is not in standard form, i.e. fails to satisfy

the complementary slackness requirements. Since d 1 (name-code -1)
is a basic variable, xl is excluded as new pivot column, or we
would risk a second pair of alternative variables.

Further violation of the complementary slackness requirement

needs to be avoided, and we introduce the complementary name
to the just eliminated d 2 variable. (Re-introduction of d2
as basic variable would merely reverse the previous step.) We
would wish to get back to standard form.

Since d 2 (name-code -2) was just eliminated, x 2 (name-code 2)

is the new variable to enter the list of basic variables. The
new pivot-row is the d l row, (by the rule of the smallest
quotient). (That pivot was already marked in tableau l6.3c. The
new tableau is:

TABL.EAU 16.3 D

THE SAME QP PROBL.EM. AFTER TWO

STEPS. STIL.L. NOT BACK IN STAN-
DARD FORM. WITH NEW PIVOT MARKED.

NAME II X 1 D 1 D 2 II VALUE
----------------------------------------------
I CODE II -1 -2 II
----------------------------------------------
0.50
X 2 2 II -2 1 -O.SO II
P 1 !-1001 I! -2 1 II 1
S 1 11001 !I @ 2 -1 I! -2
----------------------------------------------
0.50 II -3.50
2T II 9

Note that we have two distinct rules for selecting the incoming
variable. In standard form tableaux we select the driving
variable, otherwise column-selection is by the complementarity
rule which says that the incoming variable is the complementary
variable associated with the just-eliminated variable.

In tableau l6.3d the complementary variable to the just-

eliminated dl variable is Xl. The Xl column i~dicates 3 ratios,
of which both !/-2 in the x2 row, and 1/-2 in the PI row are of
the wrong sign. Therefore, -2/-5 in the sl row indicates the
pivot-row. Note the acceptance of a negative pivot, in the
badname row. We refuse negative pivots in other rows, "flying
through" violated restrictions instead.

The sl'x element has been marked in tableau I6.3d. The step
is made lnd the next tableau is tableau l6.3e.

The tableau is now back in standard fbrm, and the right-hand

side is non-negative. Therefore Xl = 2/3, x 2 = 1 3/10 is the
optimum.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 415

TABLEAU 16.3 E

THE SAME QP PROBLEM. AFTER THREE

STEPS. NOW BACK IN STANDARD FORM.
THIS ALSO BEING THE OPTIMUM.
NAME ! ! lSI D 1 D 2 I! VALUE

X 2
I CODE II 1 001

2 I! -0.40
" -1
----------------------------------------------
0.20
-2

-0.10 II 1 .30
P 1 ! - 1001 ! ! -0.40 0.20 0.40 I! 1.80
X 1 I 1 II -0.20 -0.40 0.20 II 0.40

2T !I 1.80 -0.40 -1.30" -7.10

Exercise l6.3a

Consider the quadratic programming problem

Maximise

Subject to 2xl + x 2 _~ 6

(xl ,x 2 ~ 0)

Solve the problem by applying the algorithm, as outlined so far.

To verify the correctness of your results, also map the one

restriction and draw the (x l -5)2 + (X2-5)2 = 16.2 circle,
(~2 ~ 4.02), which is the optimal iso-objective function line.

Exercise l6.3b

If you had any difficulty in making the actual steps for this qp
problem, (as distinct from sorting out which steps to mak~ it's
time you refresh your competence at tableau-manipulation.

You should:
1) For each tableau developed in this section (which are all
shortened tableaux), write the corresponding full explicit
tableau, with the unit vectors.
2) For tableaux 16.3a, l6.3c and l6.3d, write the corresponding
systems of equations explicitly.

16.4 The Symmetry Property of the Tableau 1n its Standard Form

Some properties of the tableau are more clearly illustrated if

we re-order the tableau according to the name-codes of the
various columns and rows.
.416 CHAPTER XVI

We put each column in the position indicated by a name of the

same absolute value in the trivial basis, and re-order the rows
on the same principle (see tableau l6.4a).

TAB1.EAU 16.4 A

THE OPTIMUM TAB1.EAU OF THE

QP PROB1.EM FROM SECTION 16.3.
RE-ORDERED ACCORDING TO CODES.
NAME II 01 SIll VA1.UE
--------------------------------------------
I CODE II -1 -2 1001 "
--------------------------------------------
X 1 0.20 -0.20 II
1 " -0.40 0.40
X 2 I 2" 0 .20 -0.10 -0.40 " 1.30
P 1 !-1001" 0.20 0.40 1.80
-0.40 "
--------------------------------------------
2T "-0.40 -1.30 1.80 If -7.10

Note the semi-symmetry property. If the signs were suppressed,

the tableau would be a symmetric matrix.

To describe the signs, it is useful to classify all the names in

the tableau in two groups, primal names and dual names. Primal
names are the positive names corresponding to the elements of x
and s with positive name-codes, as well as the value-column.

Dual names are the ones corresponding to elements of p and i,

with negative name-codes, as well as the quasi objectIve
function.

It appears that a cell of the tableau, where the row and the
column belong to the same class, has the same absolute value
but the opposite sign as the symmetric entry across the
diagonal. If the row- and column-names belong to different
classes, there is full symmetry.

The bottom left-hand block with dual row-names p and ~ and dual
column-names d l and d 2 is symmetric but opposite in sign to the
top right-hand block with primal row-names xl and x z' the primal
column-names, sl and the value column.

The rest of the tableau (in this small example only the diagonal
itself) is fully symmetric. The rule applies in that trivial case
as well. Thus the rowname of the top left-hand cell is 1, the
corresponding column-name is -1, and the cell is its own full
symmetry.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 417

This symmetry property is systematic as we first show for the

(quasi) target-row and the value-column.

Consider the dual of the linear problem formulated in Section

16.3. This dual problem is:

Maximise

w' Z - b' z (16.4.1)

Subject to

- DZ - A' z < - w (16.4.2)

and

(16.4.3)

if ~, E is a solution to the primal problem, then

x (16.4.4)

and

(16.4.5)

will be a solution of the dual problem, as may be seen from

comparing (16.4.2) with (16.2.3) and (16.4.3) with (7.2.2).
(That "dual" solution will not usually be a non-negative vector,
but this does not concern us here).

We may not generally require that the target-row indicates the

same combination of binding restrictions for the dual, as does
the value column for the primal.

But the complementary slackness condition, ~.e. the requirement

of standard form, implies that in the case at hand. Therefore
we will find y as defined by (16.4.4) and z as defined by
(16.4.5) in the target-row, when ever the tableau is in standard
form.

For elements of the pivot-inverse, we also have a more direct

proof.

Consider the symmetric matrix

S (16.4.6)
418 CHAPTER XVI

which is the block-pivot with the sign of the second block-

column changed round.

The inverse of this matrix is obviously an entirely symmetric

matrix, and we find the inverse of the block-pivot by changing
the sign of the bottom block-row once more.

Denote

-1
C S

with associated name-codes

~l ~l

~l
[C ll C12]
£.1 CZl C22

and the correct inverse of the block-pivot is

~l ~l

J
xl
[C ll C12
PI -CZl -C 2Z

The proof can in fact be generalized to the full tableau. The

semi-symmetric matrix associated with the set-up tableau would
in that case be an extended basis-matrix, the difference with
the usual basis-matrix being the addition of two - I block-rows
representing the non-negatively restrictions. When these
restrictions are explicitly written as x >:0 and p>o, these
vectors match the unit vectors which represent d~al and primal
slack variables.

The curious sign-inversion rules now follow and may be

recapitulated as follows.

If the row and column name which are associated with a tableau-
cell belong to the same class e.g. Pi/dj in -CZl with an
associated cellon the other side with Xj/si in ClZ we have
sign-inversion.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 419

Inside the diagonal blocks we have cells for which the row and
column-names belong to different classes e.g. xj/d j ~n Cll and
Pi/si in -C 22 and we have full symmetry.

Exercise

The optimum tableau which you should have obtained when doing
the exercise at the end of the previous section 16.3 is given
here in part i.e. the top right-hand part only.
TABLEAU 16.4 EX

THE OPTIMUM TABLEAU OF THE

QP EXERCISE FROM SECTION 16.3.
FOR YOU TO COMPLETE.

NAME II D 1 D 2 Sill VALUE

! CODE !! -1 -2 1001 I!

Xl" -0010 0.20 0.40" 1.40

X 2 2 I! 17 -0.40 0.20 I! 3.20
P 1 !-1001" ?? 17 -0.40!l 3.60

2T II ?? ?? II 67.60

Find the correct entries to be placed in the bottom lefthand

cells marked as ??

Do not initially refer back to section 16.3, using the symmetry

properties explained in this section instead.

Then afterwards compare with section 16.3 and attend to any

corrections which may be needed in your answers to either of
the exercises.

16.5 The Solution Value

Recall, from section 16.3, the linear quasi-objective function

]J = WI X + ~' £. (16.3.1)

We will now show that, for a solution which ~s described by a

standard form tableau, this function indeed ~s twice the value
of the objective function.

We introduce equation-type equivalents of (7.2.2) and (16.2.3).

Following what we already did for individual variables, we

indicate the vector of primal slack variables as ~, and the dual
slack-variables as d. Then

Ax + S b (16.5.1)
420 CHAPTER XVI

is equivalent to (7.2.2), and

D~ - A'~ + ~ =- w (16.5.2)

is equivalent to (16.2.3). Substitute Ax + S for b into

(16.3.1), to obtain:

Jl = ~' ~ + x' A'~ + ~' ~ (16.5.3)

The complementary slackness condition requires the third term

of (16.5.3) to vanish. Substitute Dx + d + W for A'p by
(16.5.2) into (16.5.3), and suppress the last term of (16.5.3),
to obtain

Jl = ~' x + x'Dx + x'd + x'w (16.5.4)

The complementary slackness condition requires the vanishment

of yet another term, x'd. Therefore when the tableau is in
standard form

Jl = ~' ~ + x'Dx + x'w = 2 w' x + x'Dx (l6~5.5)

Comparing (16.5.5) with (16.1.1), it is clear that (when the

tableau is in standard form) Jl = 2T is indeed correct.

Note, however that for a non-standard form tableau,

Jl = 2T + x'd + ~'~ (16.5.6)

is the true relation.

16.6 The negative diagonal

In preparation for the next section on non-standard form

tableaux, we develop some theorems, concerning the pivot inverse
and its main diagonal.

These theorems and their proofs are rather detailed and a

summary of the main results is given here first, together with
some earlier results, but in advance of mor~ detailed discussion.

Our interest in this section concentrates on the d1'/x 1·, x./d.,

· 1 1
s./p., and p./s. cells of the tableau. We assume that the
t~bl~au is o~defed accordine to the (absolute values of the)
name codes and these cells therefore all occur on the main
diagonal of the tableau.

We collectively refer to them as "the diagonal cells". Note,

however, that no properties concerning the 2T/value cell is
stated in this section, even where 'that cell too may be written
on the main diagonal.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 421

The following properties apply

Property 1

If the problem is properly convex (~'D~~O for all ~),

all conceivable pivot inverses -1

represent constrained maxima in the All ~l = ~l subspace and

all diagonal cells are non-positive, and the fully symmetric
part of the tableau forms a negative semi-definite matrix.

Property 2

Regardless of the definiteness or indefiniteness of D, standard

form tableaux developed by the algorithm, describe All ~l = ~1
subspaces, in which the more restrictive condition of strict
convexity (~l' Dll ~l < 0 for all ~l satisfying All ~l = 0 and
~l I 0) is satisfied.

(A subspace is the set of points satisfying some listed

restrictions, see section 6.4).

In such tableaux the diagonal cells in dual variables' columns

(x./d. and s./p. cells) are negative non-zero, except in those
du~l ~ariabl~s,1columns in which all primal variables' row-
entries are zero.

We may in fact interpret a pos1t1ve diagonal entry as a sympton

of non-convexity, and we say that non-convexity, if present,
becomes manifest only in primal variables' columns.

An essential feature of the algorithm is therefore the

restriction of solved solution-vectors to those where the problem
may be described as being strictly convex in the All ~l = ~l
subspace. This restriction is not nearly as severe as the
word convexity might suggest.

In particular, if the objective function is anti-convex, we

know (see section 14.7), that the optimum (if one exists, will
be in a corner of the feasible space area.

In that case, the set of vectors ~l' which satisfy All ~l = 0

for ~l I 0, is an empty set, All being square and non-s1ngular.
422 CHAPTER XVI

Thus, if the problem is anti-convex, (the matrix D is positive

semidefinite), the algorithm only develops solutions for which
All is square and non-singular.

The other obvious case where we would expect this property

to be revealed is with a directionally convex objective function.

For example if we seek to maximise T = - xl + xl • x , subjectto

xl + x2 ~ 2 (xl~ x 2 ~ 0), we will exchange x 2 and tte one dual
varia~le.Pl aga1nst the one side-restriction and the dual
restr1ct10n on x 2 •

We then operate in the xl + x 2 = 1 subspace and substitution

of 2-xl. for x 2 now gives T = xl - x 1 2 , which is a properly
and str1ctly convex function of xl.

We now proceed with a more detailed discussion of the theorems.

The pivot-matrix of a quadratic programming tableau in standard

form will be

p
[~] (16.6.1)

Above, Dll is the block of D which corresponds to the basic

primal variables ~l and the associated dual slacks ~l. This
block will be square (of order nl by n l ), and symmetric, it may,
or may not be negative (semi) definite, like the full matrix D.
All is the intersection between the block-row AI', the rows
of which represent binding restrictions, and the block-column
AI' the columns of which represent basic variables.

The block All need not be square, it can contain more columns
than rows.

Theorem

Let a vertex correspond to optimal and feasible solution to

the sub-problem.

Maximise

~i ~l + (16.6.2)

Subject to

~l (16.6.3)
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 423

then, irrespective of the definiteness of Dll

the first n l leading diagonal elements of the pivot inverse

will be non-positive.

Proof

Consider any e.g. the jth column of the pivot-inverse. We will

denote the pivot inverse as Z and the jth leading column of the
inverse is indicated as ~j. By the definition of an inverse,
post-multiplication of P by z. yield a unit vector.
-J
P z.
-J
= -J
e. (16.6.4)

Therefore the quadratic form ~l' P ~l is evaluated as

z.' P z. z .. (16.6.5)
-J -J J,J
However, for j < nl' ~j yie~ds a zero product with the bottom
block-row of the p~vot-matr~x.

(16.6.6)

correspondingly, for j < n l , (16.6.5) reduces to

~l,j Dll ~l,j = Zj,j (16.6.7)

where ~l,j indicates the leading sub-vector of z., containing

the n l elements which refer to the elements of ~i rather than
Er
Therefore, Zjj > 0 would contradict the second-order conditions
for a constra~ned maximum, as discussed in section 15.5 and
recapitulated in section 16.2.
q.e.d.

The one point still to be noted here, is that for certain

columns ~l j may be a zero vector, i.e. certain dual variables
may not affect x at all.

Otherwise, for ~l,j f 0, a top left-hand diagonal element of the

pivot inverse is expressed as a quadratic form by (16.6.5) and
a unique maximum is shown to imply

z .. < 0 (16.6.8)
JJ
We now consider the question how we may recognise standard
form tableaux which correspond to solutions that are properly
and strictly convex in the Al,l ~l = ~l sub-space.
424 CHAPTER XVI

We postulate the existence of a symmetric ordering and

partitioning of the pivot matrix P, as follows:
..,
* 1,1 D*l , 2 -A*'
1,1
--
* 2,1 D*2,2 -A*'
1,2
* A~ , 2
1,1
-
(16.6.9)

for which all the diagonal blocks are square, while

P 3 , 1 = At , 1 is square and non-singular.

This partitioning is related to the previous given by

(16.6.1), as:

= Dl , 1 •

In the trivial case where no primal restriction is binding

and P is identical to Dl 1, the third-block-row and block-
column of P do not exist'at all, but otherwise the existence
of a square and non-singular block A~ 1 is a condition for the
invertability of P. (see Section 5.7)!

We are, incidentally, also in a position at this point to make

an inference concerning the number of non-zero variables in a
QP problem.

We know from section 5.7, that (if P is invertable) the rank

of Dl 1 must be at least equal to the number of columns of Al* 2'
Or, t~ put the same statement the other way round, the number'
of non-zero variables in excess of the number of binding
restrictions is at most equal to the rank of Dl l'
,

The following tableau-extract from the partitioned set-up

tableau relates some notation of vectors to the partitioning
given by(16.6.9.)
Name * * Value
~l ~2 ~l
d* *
Dl,l
*
Dl , 2 -A*' -w*
-1 1,1 -1
d*
-2 D2* , 1 D2* , 2 -A*'
1,2
-w*
-2

~l A~ , 1 A*
l ,2 E.l
2T -w*' -w*' -b'
-1 -2 -2
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 425

A corresponding extract from the current tableau, named T, is

summarized as follows:

Name
~l* ~2* ~l

~l
* Tl , 1 Tl , 2 T l ,3

~2* T2 , 1 T2 , 2 T 2 ,3

£1 T 3 ,1 T3 , 2 T 3 ,3

Theorem

A necessary and sufficient condition for P to be ass~~iated

\vith a solution where T is strictly convex in the A x = E.l
subspace, is that (for some ordering of the variabl~~)-l
T2 , 2 is negative definite, (or else to be of zero order).

Proof:

Consider the product of the two top block-rows of T, post-

multiplied by the first two block-columns of p, postmultiplied
by a vector ~~~, ~~~ (a change in the vector ~).

(16.6.10)
(The two matrices on the left of the left-hand side expression
are a block-row of p-l, postmultiplied by a block-column of P,
yielding a unit-matrix as product.)

Within the Al 1 ~l = E.l subspace, ~~l is restricted to

Al 1 ~~l = 0 'and the following relation applies, for vectors
~ ~l' wfiich obey that side-restriction.

z*
-1

z*
-2 (16.6.11)

o
426 CHAPTER XVI

In (16.6.11), ~~ and ~~ may be considered as arbitrary vectors.

The abstract form T inaicates that for ~sl=O (and ~P2=O), we
may interpret ~l* and ~2* as -~l* and -~.
*-We therefore
-
reformulate the combination of (16.6.10) and (16.6.11) as:

Tl ,31
T2 ,3J
[-~~!]
-~~l
=[~.:!.>]
~.:!.2
(16.6012)

o
Note that the condition ~sl = 0 in (16.6.12) implies that
* and ~.:!.2* obey the requirement
~.:!.l - Al,l ~.:!.l = O.

Note also that the block [Tl,l Tl ,2l

,
T2l ,
T22 J
is fully symmetric by the symmetry rules.

Therefore, for vectors I~.:!.~' ~~I which satisfy the

Al,l ~.:!.l = 0 condition, we find the following relation:

[~.:!.~]
[~d*'
-1 ~d*'J
-2 [T 1,1

T2 , 1 ~x*
-2
- ~d*' ~x* - ~d*' ~x*
-1 -1 -2 -2
(16.6.13)

However, ~~t, ~~~, ~.:!.t, ~~ are also related by a tableau

associated with a smaller block-pivot. This smaller block-pivot
is

j
r~"
p*
-A*'
1,1
(16.6.14)
Al , 1

The tableau-extract corresponding to that block-pivot p*

may be summarised as T*
~l

.:!.l* -T*' *-1 where -T*' *

2,1 Al , 1 2,1 Tl , 2
d* * T*' T*'
-2 T~,l T2 , 2 3,2
and
3,2 T~,3
[A*'lJ-l T* T* conform the symmetry rules.
1, 3,2
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 427

Note the zero top left-hand block, due to the block-triangular

structure of p*

Tableaux T and T* are related by, among other relations,

(16.6.15)

(T~ 2 is the third and last block-pivot used in inverting P to

bec6me T, T* being the preceding calculation tableau.)

We also note that both T~ 2 and T2 2 are non-singular (if P is

non-singular). ' ,

The first block-row of T* shows that the vectors ~~*1' ~~*2'

- 1 and ~x*
~x*
- 1 and ~x*2' as given in (16.6.12) are also
related by

~x* T*' ~x* (16.6.16)

- 1 2,1 -2
(again assuming ~~1 = 0)
Similarly for the second block-row

~d* -T* ~d* - T* ~x* (16.6.17)

-2 2,1 -1 2,2-2
Substitution of the right-hand sides of (16.6.16) and (16.6.17)
for ~~*1 and ~~~ into the right-hand side of (16.6.13) now
yields

[~d*',
-1
~d*
-2
'J

- ~d*' T*' ~x* + (~d*' T*' + ~x*' T*' ) ~x*

-1 2,1 -2 -1 2,1 -2 2,2 -2

~d*'
-1 ~x*'
-1
2,1
- T*'] [~d*l
-1

T~ , 2 ~~~

= ~x*' T* ~x*
-2 2,2 -2 (16.6.18)

Combining (16.6.12), (16.6.13) and (16.6.18), we find for every

~x ' = [~x*', ~x*'J, which satisfies the side-condition
-1 -1 -2
Al , 1 ~~l = 0,
428 CHAPTER XVI

[ l1x*'
-1
I1x*'J
' -2
[P 1.1
P"'J [A~]
P2 I1x*
P2 , 1 ,2 -2

[T , ., T1"j
T2 , 1 T2 , 2
r~1
I1d*
-1
I1x*' Tt2 I1x*
-2 -2

(16.6.19)

which shows that the negative definiteness of T~ 2 is a

necessary condition for the far left-hand side e~pression to
be negative for all 11~~ # O.

Furthermore, by (16.6.16) 11~ = 0 implies (for Al 1 I1x l 0)

I1xt 0, therefore 11~*1 # 0 implies (for Al,l 11~1'= 0)
I1x* # O.
-2
Since T~ 2 is non-singular we must also exclude the possibility
that bot~ sides of (16.6.19) are zero on account of sem~­
definiteness of T~ 2'
,
The right-hand side is negative non-zero for all 11~~ # 0 and
there are no vectors 11~1 # 0 with 11~~ = 0 and Al,l ~~l = O.
Therefore all the three equivalent expressions in (16.6.19)
are negative non-zero for all 11~1 # 0 satisfying Al,l 11~1 = O.

Since T~ 2 is the inverse of T2,2 its negative definiteness ~s

equivalebt to the negative definiteness of T2 2 which is
therefore also shown, proving the necessity of the stated
condition.

We also note, that for a vector 11~1 # 0 satisfying the side-

condition All 11~1 = 0, the quadr~tic f~rm l1~i Dll 11~1 is
evaluated by (16.6.19) as 11~~' T 2 ,2 11~2 < 0, proving the
sufficiency of the stated condition.
q.e.d.

Note that this theorem does not have the usual weaker sem~­
definite equivalent. The invertability of T2 2 means that
standard-form tableaux either correspond to solutions
where T is strictly convex in the All ~l = ~l subspace,
or else not convex at all!
( A *' T*2, 2 Ll~2
A * 0 , a 11 Ll~2'
A '. A *' T~ 2 11~2 = 0, some 11~2 # 0
Ll~2 ~ D~2
implies IT~ 21 = 0, and therefore
, Ipi = 0.)
 QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 429

It is, incidentally, quite possible to write standard form

tableaux which correspond to solutions of both the primal and
the dual restrictions, but which do not refer to convex
subspaces, and which display manifest non-convexity in dual
variables' columns, i.e. a positive diagonal cell.

Example

Maximise

subject to xl + x 2 < 10

TABLEAU 16.6 A

ILLUSTRATION OF NOT RESPECTING <SUBSPACE) CONVEXITY.

SET-UP TABLEAU INCORRECT 'OPTIMUM'
NAME Xl X2 PI t VALUE NAME t 01 02 PI VALUE
01 -1 1 Xl -1 1
02 -1 1 X2 1 -1 1
S1 10 Sl -1 -1 2 8

2T -10 2T -1 -1 -8 -2

(To solve this non-convex problem, one would have to add

restrictions, e.g. xl ~ 2, x 2 ~ 2. The objective function would
then be directionally convex in the feasible space area.)

The property that non-convexity is manifest only in primal

variables' columns is true for standard form tableaux developed
by the algorithm. These tableaux correspond to solutions which
are convex, and therefore strictly convex in the Al 1 ~l = ~l
subspace. '

One further comment on the precise significance of the theorem

which we might call the "negative definiteness theorem", concerns
the partitioning. It is normal that several partitionings will
permit a square and non-singular block AI l' Correspondingly,
there will be several negative-definite biocks T2 2 = T~-~.
, ,
We may not, however, require that the whole top left-hand block
of the tableau is negative definite, only negative semi-
definiteness applies for the full intersection between primal
variables' rows and dual variables' columns. And there may be
zeros.
430 CHAPTER XVI

Theorem

If a standard form tableau refers to a solution which is convex

in the All ~l = ~l subspace, all dual variables' columns display
one of the following characteristics:

Either: a) the diagonal cell associated with that column is

negative non-zero

or .b) all primal variables' row-entries in such a dual

variables' column are zero.

Proof:

By the previous theorem, the assumed convexity in the

All ~l = ~l subspace, implies, in combination with the non-
singularity of p, strict convexity (in the same subspace).

We now first consider dual slack variables dj. If there are

columns for these variables in the tableau the diagonal cell
which we refer to is an element of the pivot inverse. Conform
the notation earlier in this section we indicate the vector of
primal variables' entries of the dj column, as far as they
refer to elements Xj rather than slack variables as ~l,j.

The property of strict convexity, as proved above, now implies

~i,j DI 1 ~l,j = Zj,j < 0, for all ~l,j ~ ° (16.6.20)

No variation in ~l implies no change in s, the vector of slack-

variables, therefore ~l j = °
implies zero entries for all
primal variables' rows {n the column in question.

Concerning columns for non-basic dual variables p., we note

that these are equivalent to dual slack-variables 1 if primal
slack-variables are listed as explicit variables, with explicit
non-negativity restrictions. The same proof has then to be
carried for an enlarged pivot matrix.
-
DI , I Al , I

E -I (16.6.21)

Al , I I
-
which completes the proof for dual variables p .•
q.e.d. 1
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 431

The symmetry of the tableau now leads to the following:

Corollary:

If a standard form tableau refers to a solution which is convex

in the All ~l = ~l subspace, all primal variables' rows display
one of tfie following characteristics:

Either: a) the diagonal cell associated with the row in

question is negative non-zero

or b) all dual variables' column-entries in such a

primal variables' row are zero.

Note:

The "matrix analogy" of the above theorem:

"A dual variables' block-column is either free of non-zero

elements in primal variables' rows, or the diagonal block is
negative definite", is not true, as may be illustrated by the
following example:

Maximise

subject to

The optimum tableau of this problem is given in tableau l6.6b

below

TAB1.EAU 16.6 B
I1.1.USTRATION OF THE NEGATIVE DIAGONA1..
NAME! ! D 1 D 2 S 1 ! J VA1.UE
-------------------------------------------
X 1 II -0012 0.12 0.50 J J 0.50
X 2 JJ 001~ -0.12 0.50 JJ 0.50
p 1 ! I -0.50 -0.50 JJ 1

2T IJ -0.50 -0.50 JJ 2

There are two dual variables' columns in this tableau (dl and d Z).
The first theorem of this section tells us that they contain
non-positive entries on the main diagonal (the two entries of
-1/8, printed as -O.lZ).
432 CHAPTER XVI

The second theorem tells us that, since the excess of the

binding restrictions is 2-1 = 1, there should be a partitioning
which yields a 1 by 1 negative-definite diagonal block ~*2/~*2.
Either of the two diagonal cells of -1/8 will do.

And the last theorem tells us that in a dual variable's column

which contains non-zero entries in primal variables' rows (both
of them), the diagonal cell is negative non-zero. They both are.

The top left-hand block is nevertheless negative semi-definite

rather than negative definite - as may be illustrated by

lJ [
-1/8 1/ 8 l o
1/8 -1/8J
However, the weaker theorem:

"If a standard form tableau refers to a solution which is convex

in the All ~l = ~l subspace, the square block formed by the
intersection of primal variables' rows and dual variables'
columns in negative semidefinite."
is true. (n-dimentional generalization of (16.6.5»

Furthermore, since, as far as subspaces associated with

invert able block-pivots is concerned, subspace convexity implies
strict subspace convexity (~~i Dll ~~l ~ 0 for all ~~l
satisfying All ~~l = 0

implies ~~i Dll ~~l < 0 for all ~~i I 0 satisfying All ~~l = 0)

the negative semi-definiteness of the symmetric top left hand

block of the pivot inverse is a sufficient condition to prove
subspace convexity, and by implication strict convexity, and
hence the existence of a negative definite block T2 2 of the
appropriate order. '

Exercise

The following QP problem is given

2 2
Maximise T = xl - x 2 - xl + xl x 2 - 2x 2

Subject to xl + x 2 > 10

Solve this problem

QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 433

For all standard form tableaux developed verify the symmetry

properties, i.e. check that they conform, if necessary after
re-ordering), to the properties stated in section 16.4.

Also verify the convexity properties of the same tableaux.

Extract the block, T2 2 as well as the larger semi-definite
'
J
blOCk[Tl,l T 2 ,2

T2 , 2 T2 2
,
from each tableau and indicate which of the other non-positive
diagonal cells are non-positive by any property stated in this
section, or only because this is a problem in which the objective
function is convex throughout the coordinate space.

16.7 The non-standard form tableau and its standard form

neighbours

Most of the theorems to be discussed in this section concerns a

(2 by ~ 4 cell block of the non-standard form tableau consisting
of the intersection of the badname variable row and the driving
variable row, with the last leaving variable column and the
incoming variable column.

We call this block the non-standard form block. In association

with this block, we define 4 standard form tableaux, which
mayor may not exist.

The smaller subspace predecessor tableau is obtained by

introducing the last leaving variable back into the basis, in
exchange for the driving variable. The smaller subspace successor
tableau is obtained by introducing the incoming variable into
the basis in exchange for the driving variable. The larger
subspace predecessor tableau is obtained by introducing the last
leaving variable into the basis, in exchange for the badname
variable. The larger subspace successor tableau is obtained by
introducing the incoming variable into the basis in exchange
for the badname variable.

Together these four standard form tableaux (as far as they exist)
are indicated as the neighbouring standard form tableaux, or
standard form neighbours. The terms "smaller" subspace and
"larger" subspace refer to the situation where with an initially
feasible basis, the badname variable is a dual variable and the
driving variable is a primal variable. They will, however be used
in the general situation where the badname could be a primal
variable, i.e. a negative valued slack-variable and the
connotation of "more space" and "less space" may not be
applicable, even though the terms are used.
434 CHAPTER XVI

Theorem (non standard form block non-singularity theorem). The

non-standard form block is invertable, its determinant being
either +1 or -1.

Proof:

Suppose the theorem to be true for some non-standard form

tableau.

We now consider the implication of the column-updating rule: the

last leaving variable's column is obtained by division by minus
the pivot (see section 8.8). If (as assumed) the theorem is true
for the current non-standard form tableau, the last leaving
variable column of the non-standard form block of a non-standard
form successor tableau contains at least one non-zero element,
and either the smaller subspace predecessor tableau of the
successor tableau (= the smaller subspace successor tableau of
the current tableau) exists, or else the larger subspace
predecessor tableau of the non-standard form successor tableau
(= the larger subspace successor tableau of the current non-
standard form tableau exists.)

If the smaller subspace predecessor exists, we may summarize

its relation with the actual tableau, as follows:

Non standard form tableau

last leaving v incoming v

badname v ? ?
driving v i 0 ?

Smaller subspace predecessor

driving v incoming v

badname v ? i 0
last
leaving v i 0 ?

By the symmetry property of the standard form tableau, the badname

variable row/incoming variable column cell of the standard form
tableau is of the same absolute value as the last leaving variable
row/driving variable column cell, which is the reciprocal of the
pivot used to create the standard form tableau.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 435

The other non-zero diagonal cell now provides the other pivot
needed to calculate the inverse of the non-standard form block.

If the smaller subspace predecessor does not exist, the larger

subspace predecessor exists, and pivoting on the badname
variable row/last leaving variable column cell allows us to
invoke the same argument with respect to the symmetry property
of the larger subspace predecessor tableau.

Hence the theorem assumed to be true for any current non-standard

form tableau, implies the validity of the same theorem for its
non-standard form successor tableau (if one exists). The smaller
subspace predecessor of the first non-standard form successor
tableau of a standard form tableau is that some standard form
tableau itself.

Therefore the theorem is true for the first of any series of

non-standard form tableaux, and by implications for all its non-
standard form successor tableaux.
q.e.d.

The following corrollae - inferences of the fact that a non-

singular matrix contains at least one non-zero entry in each
row/column -, may now be stated: If the smallers subspace
predecessor tableau does not exist, then the smaller subspace
successor and the larger subspace predecessor tableaux exist.

The non-standard form block plays a crucial role in reco8nizing

the transmission of convexity properties between its standard
form neighbours. Before we proceed to stating the theorems and
their proofs, it is useful to introduce one more term. The
subspace described by standard form tableau Q is said to be
contained in the subspace described by standard form tableau T,
if T may be transformed into Q, by eliminating a primal variable
from the basis of the solution described by T, brin8ing the
corresponding dual variable into the basis instead.

There are two cases. If the eliminated variable is a primal slack-

variable, Dll remains the same but All loses a row. If the
eliminated variable is an element of ~, the order of Dll is
reduced by one, and All loses a column.
,
Since the definition of subspace convexity (~~l DIL ~l ~ 0
for all L\~l ~atisfyin8 All~l = 0) inc1~des vect~rs. ~~l. for which
one elemenE ~s zero the subspace convex~ty of q ~s ~mpl~ed by the
by the subspace convexity of T, if T obeys that property.
436 CHAPTER XVI

We now state the theorems

Theorem (smaller subspace immediate neighbour convexity

transmission theorem)

If the smaller subspace predecessor tableau exists, and describes

a-solution which is strictly convex in the All ~l = ~l subspace
then a sufficient condition for the smaller subspace successor
tableau 'to exist and obey the same convexity property is that
the entry in the driving variable row/last leaving variable
column cell is positive (non-zero), while the driving variable
row/incoming variable column cell is negative non-zero.

Proof

Existence:

Obvious: the negative non-zero number is the pivot needed to

generate it

Convexity:

The condition implies the presence of a negative non-zero

diagonal entry in the last leaving variable row/incoming variable
column cell in the smaller subspace predecessor tableau, as may
be illustrated by summarizing the transition from the non-standard
form tableau to the smaller subspace predecessor tableau

Non standard form tableau

last leaving variable incoming variable

driving
+
variable

Smaller subspace predecessor

driving variable incoming variable

last
leaving +
variable

This negative diagonal cell provides the pivot needed to transform

the smaller subspace predecessor tableau into the smaller subspace
successor tableau. If.the incoming variable is a primal variable
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 437

this cell provides a further negative pivot in T~,2 as defined

in the previous section. If the incoming variable is a dual
variable, no further proof of convexity is needed the subspace
defined by the successor tableau is contained in the one
defined by the predecessor tableau.
q.e.d.

We might formulate the theorem as stated above somewhat more

compact, by introducing the convex transition. A condition is
said to indicate the convex transition from the standard form
tableau T to the standard form tableau Q, if the condition is
sufficient to prove that Q describes a solution which is
convex in the All~l = ~l subspace, provided T does so. We will
however reserve fhe use of the term convex transition for the
discussion of examples, or where the meaning of the term is
self-evident without reference to the definition, and formulate
the theorems themselves explicitly.

The same argument concerning the implied existence of a negative

diagonal pivot gives rise to the following parallel theorems.

If the smaller subspace predecessor tableau exists, and describes

a-solution which is strictly convex in the AII~1 = ~l subspace
then a sufficient condition for the smaller subspace successor
tableau to exist and obey the same convexity property is that
entry in the driving variable row/ last leaving variable column
cell is negative (non-zero) while the driving variable row/
incoming variable column cell is positive non-zero. (Upon
developing the smaller subspace predecessor, division by a
negative pivot gives rise to a negative diagonal cell).

If the larger subspace predecessor tableau exists, and describes

a-solution which is strictly convex in the Al ~l = ~l subspace,
then a sufficient condition for the larger suSspace successor
tableau to exist and obey the same convexity property is that
the entry in the badname row/last leaving variable column cell is
positive (non-zero), while the badname row/incoming variable
column cell is negative non-zero.

Note however, that since column-updating is by division by minus

the pivot, the similar theorem concerning the relation betw~
smaller and larger subspace successors requires equal signs. If
the smaller subspace successor tableau exists and describes a
solution which is strictly convex in the AII~l = ~l subspace,
then a sufficient condition for the larger sUDspace successor
tableau to exist and obey the same convexity property is that the
entry in the driving variable row/incoming variable column cell
is negative (non-zero) while the badname-variable row/incoming
variable cell is also negative non-zero.
438 CHAPTER XVI

The following arrangement of the signs of the four elements of

the non-standard form block allows all four neighbouring non-
standard form tableaux to exist and they are either all convex,
or none of them describe convex solutions:

last leaving variable incoming variable

badname variable +

driving variable +

with all four cells containing non-zero numbers of the signs as

indicated. The complete reversal of all of these signs, i.e.

last leaving variable incoming variable

badname variable +

driving variable +

has the same implications.

The presence of one of these two arrangements (in practice only

the first one is used) is also a necessary condi~ion for the
convexity of all four neighbouring standard form tableaux.

Theorem

If both entries in the driving variable's row of the non-standard

fOrm block are non-zero and of the same sign,
then both the smaller subspace predecessor tableau and the
smaller subspace successor tableau exist , and at least one of
the two descri,bes a solution which does not obey the property
of convexity in the AII~1 = ~l subspace.

Proof

By the complementary rule, the last leaving' variable and the

incoming variable are each other's complements. Therefore at
least one of them is a dual variable. Upon 'pivoting on the primal
variable's entry in the driving variable's row, we develop a
standard form tableau, in which a positive non-zero figure, the
ration between the two, figures in the diagonal cell of a dual
variable's column, contradicting the convexity property of the
tableau as thus developed. (The existence of the other tableau
is obvious).
q.e.d.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 439

The parallel theorem concerning the badname-row and the larger

subspaces will be obvious and is not formulated explicitly here.

The parallel theorem concerning two successor tableaux (or two

predecessor tableaux) is again slightly different: if two cells
in the same column are poth non-zero, but of different signs
at least one of the tw6 1 standard form tableaux describes a
solution which is not convex in the AII~1 = ~l subspace.

We may refer to the group of theorems concerning the convex

transitions between the immediate neighbours of a non-standard
form tableau as the immediate neighbour convexity transmission
theorems.

The non-standard form block non-singularity theorem does not

require that all four elements of the non-standard form block
are non-zero. On the contrary, if a primal badname is chosen
in the set-up tableau (i.e. we wish to make a violated
restriction binding), the first smaller subspace predecessor
(the set-up tableau itself) contains a zero in the badname
row/incoming variable cell, and one always develops a non-
standard form block ~hich contains at least one zero element.

Example

Maximise T =

Subj ect to Xl + 2x 2 > 3

(Xl' x 2 ~ 0), see tableau 16.7a below

TABLEAU 16.7 A

SET-UP TABLEAU FOR ILLUSTRATING

THE NON-STANDARD FORM BLOCK.
NAME! I XI X2 PI II VALUE
Dl ! ! -2 1 1I 1
D2
51
II -2 ® II 1
! ! - 1 -2 II -3

2T !! 3 II
440 CHAPTER XVI

The first badname obviously is sl' therefore PI is the driving

variable and a non-standard form tableau is developed, which is
tableau l6.7b. Note that this tableau has been re-ordered.

TABLEAU 16.7 B

NON-STANDARD FORM T4aLEAU.

WITH NON-STANDAhD FURM BLOCK.
NAME! , Xl D2 X2 I I VALUE
----------------------------------------
Dl !! -2 -0.50 II . 0.50

!,
51 -1
--------------
-2 I I -3
Pl !! 0.50 -1 I I 0.50

----------------------------------------
2T !, -1.50 4 II -1.50

From tableau l6.7b we extract the non-standard form block as

given below.

d2 x2
last leaving v. incoming v.

sl (badname variable) -2

PI (driving variable) ! -1

The driving variable row confirms the convex transition from

the trivial basis to the x2 subspace - not surprising since
this is a convex problem. The badname row does not provide any
similar information at least not directly but the x 2 column
tells us that the x 2 subspace is convex, therefore, so is the
x 2 = I! subspace.

The following theorems relating to zeros in the non-standard

form block are now stated.

~ the driving variable row/last leaving variable column cell

contains a zero entry,
then the smaller subspace predecessor of the current non-
standard form tableau does not exist, but the smaller subspace
predecessor of the immediately preceding non-standard form
tableau does exist.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 441

Proof

We first note that the zero contradicts the development of the

current non-standard form tableau from a standard form tableau
(otherwise that cell would contain the reciprocal of the pivot).
Hence the smaller subspace predecessor does not exist and the
current tableau was developed from a non-standard form tableau.

By the column updating rule that non-standard form predecessor

tableau contains a zero in the driving variable row/incoming
variable column. The non-standard form non-singularity theorem
therefore requires the existence of a non-zero entry in the
driving variable row/last leaving variable column cell of the
non-standard form predecessor tableau, proving the existence
of the previous tableau's smaller subspace predecessor.
q.e.d.

Concerning the similar theorem for the badname-variable and the

larger subspace we note that the case where a non-standard
form tableau displays a zero in the badname row/last leaving
variable column cell, but has no non-standard form predecessor
does exist - it was illustrate~above.

Otherwise the theorem is listed above for the driving variable

is the same:

If the badname row/last leaving variable column cell contains

a zero entry while a non-standard form predecessor tableau exists,
then the larger subspace predecessor tableau of the non-standard
form block the transition from a standard form predecessor
tableau to the standard form successor tableau of the next non-
standard form tableau may be proved by way of a pair o-f---
complementary steps.

A pair of complementary steps consists of two steps which involve

pivots of the same absolute value, opposite complementary name-
codes, and a zero in one of two diagonal cells which link the
row of the one to the column of the other and vice versa.

r--
The 2 by 2 block
first incoming v. second incoming v.

second leaving v -p

first leaving v p ---- d

\
1S called the pair block
442 CHAPTER XVI

Theorem (complementary pair theorem)

If a standard form tableau (hereafter indicated as T), can be

transformed into another standard form tableau (hereafter
indicated a Q), by a complementary pair then:

either T and Q both describe solutions which are convex

~n th~ All~l ~l subspace,
Or ne~ther of them does.

Proof:

An extract from T may be summarized as follows:

d~' x. or d* d.
-2 J -2 J ~

x* -v -u
-2 T22 - -u x*
-2 T22 v

d. v' -p x. v' -p
J - J
u' p d dk u' P d
-

where the first incoming variable associated with v and -v is

a primal variable, indicated as x. (it could also be s.),-the
second one associated with u' andJ-u is then a dual vafiable.
The block T2 2 is the negatIve-definite block T2 2 as pr~ved
in section 16.6, provided T conforms to subspace 2 convex~ty.

The opposite signs of p and -p imply that of the two incoming

variables one is a primal variable and the other a dual variable;
if they were both of the same class there would be full symmetry
between the two pivots.

The alternating sign presentation of v and u follows from this.

Alternatively, if the first incoming variable is the dual variable,
we would write v and -u but the same result follows)

Concerning the convex transition from T to Q: We assume that T

describes a convex solution. If T2 2 is of zero order the
theorem is trivial since both subspaces reduce to single points.
Otherwise, the equivalent block of Q22 is expressed as:

Q22 = T2 , 2 + 2 ~~' d/p2 + ~~' d/p2 (16.).1)

if tHe first incoming variable ~s a primal variable and

2 2
T2 , 2 - 2 -u -v' dIp
I
Q2 , 2 = + v v' dIp (16.7.2)
- -
if the first incoming variable is a dual variable.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 443

For d = 0, convex transition follows in both orderings, because

Q2 2 is equal to T2 2' hence negative definite if T 2 ,2 is
negative definite. 'If we assume d < 0, convex transition
follows by inversion of the pairblock along the main diagonal

l :J
as may be summarized as follows:

[~ P/dJ d/p2
[

lId -lip

For d < 0 both pivots, d and p2/d are negative, and we therefore
have proved convex transition from T to Q for d < O. For d > 0,
convexity of T implies that the first incoming variable is the
dual variable, the second one the primal one, as manifest non-
convexity only occurs in primal variables' columns. But if so
the first one must be a dual variable, and the zero on the
diagonal implies v = 0, and convex transition follows for the
same reason as proved for d = 0, Q2 2 being identical to T2 2.
We therefore prove convex transition from T to Q, for d = 0:
for d < 0, and for d > O.

Concerning the convex transition from Q to T: The above

calculation of the inverse of the pairblock shows that this
transition is also by a complementary pair.

Having proved convex transition in both directions it also

follows that if T did not describe a convex solution in the
All~l = ~l subspace, then neither does Q, otherwise the convex
transition from Q to T would contradict the assumed non-
convexity of T. The same argument applies to non-convexity of Q.
q.e.d.

We may now use this theorem to prove convex transition from a

smaller subspace predecessor of one tableau to the smaller
subspace successor of the next tableau, without actually
developing either tableau.

Example
2
Maximise l = 3x l + xl + xl • x 2 - x 2

Subject to xl + x 2 < 10

The set-up tableau of this problem is given in tableau l6.7c,

which is given below, together with its non-standard-form
successor tableau.
444 CHAPTER XVI

TABLEAUX 16.7 C AND 16.7 0

SET-UP TABLEAU AND FIRST NON-

STANDARD FORM TABLEAU. FOR IL-
LUSTRATION OF CONVEX TRANSITION.

NAME! ! XI X2 PI II VALUE
---------------------------------------
01 ! ! 2 -I II -3
02 ! I CD - I I!
I!
I
10
51 ! II

2T !! -3 -10 ! I

NAME! I 02 X2 PI II VALUE

01
XI
! !
! I
.. -2
I
I
-I
I!
! !
-5
I
--------------
51 ! ! -I CD ! I 9

2T !! 3 -13 ! ! 3

In tableau l6.7c we choose d l as badname-variable, and we assume

that d 2 is selected as leaving variable, this row indicating
the smallest non-negative quotient.

In the resulting non-standard form tableau l6.7d, the non-

standard form block has been marked; it is also given in a
separate extract below.
d2 X2

last leaving v. incoming v.

d l badname v. -2 1

xl driving v. 1

We note that this tableau has no smaller subspace successor

tableau that is, the driving tableau xl cannot be eliminated by
the incoming variable x 2 . However, if we first develop a further
non-standard form successor tableau - by eliminating sl' then
we know that that tableau has a smaller subspace successor
tableau.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 445

This non-standard form successor tableau 1S tableau l6.7e below.

TABLEAU 16.7 E

A FURTHER NON-STANDARD FORM SUCCESSOR

TABLEAU. WITH NON-STANDARD FORM BLOCK.

NAME! I D2 51 PI !! VALUE

DI
XI
!!
!I
- I . -1
-I
.. I !
!!
-14
1
--------------
X2 !! - 1 !! 9

2T !! 4 -I -14 I! -6

The smaller subspace predecessor of the first non-standard

form tableau l6.7d is the set-up tableau l6.7c, the smaller
subspace successor tableau of the second non-standard form
tableau l6.7e may be obtained by eliminating the driving
variable xl against PI as incoming variable. This step, (which
we would not make in actual calculation) leads to tableau 16.7f
below, (re-ordered as 16.7g.)

TABLEAUX 16.7 F AND 16.7 G

SMALLER SUBSPACE SUCCESSOR TABLEAU

OF TABLEAU 16.7 E. AS DEVELOPED
AND RE-ORDERED TO STANDARD FORM.

NAME! 1 D2 51 Xl II VALUE
----------------------------------------
01 I! - 1 -1 ! ! -14
PI ! ! - I -1 I I - I
X2 I! 1 II 10

2T I I -10 -I -14 II -20

NAME II Xl D2 51 II VALUE
----------------------------------------
01 ! 1 - 1 -1 II -14
X2 ! I I 1 I I 10
PI ! 1 -1 -1 I I -1

2T ! I -14 -10 -1 II -20

446 CHAPTER XVI

Theorem (smaller subspace complementary pair transition theorem)

If two successive non-standard form tableaux, of which the first

one was transformed into the second one (by a positive pivot), do
not have a common smaller subspace successor/predecessor while
the smaller subspace predecessor of the first non-standard form
tableau describes a solution which is strictly convex in the
AIl~l = ~l subspace.

then

a sufficient condition for the smaller subspace successor

tableau of the second non-standard form tableau to obey the
similar convexity property is that the entry in the driving
variable row/last leaving variable column cell in the first
non-standard form tableau is positive non-zero, and the driving
variable row/incoming variable column cell of the second tableau
is negative non-zero. (the two other cells in the two driving
variable rows being zero).

Proof:

The assumptions stated imply the following sign and figure

arrangement in a 6 cell extract from the smaller subspace
predecessor tableau of the first non-standard form tableau.

smaller subspace predecessor tableau

(s tandard form)

driving v. 1st incoming v. 2nd incoI!ling v.

badname v ? p or -p ?

1st leaving v p 0 -q

2nd leaving v ? q d

where p is the reciprocal of the pivot used to transform the

first non-standard form tableau into its smaller subspace
predecessor - hence by assumption p > 0 -, q is the pivot used
to transform the first non-standard form tableau into the second
one - hence by assumption q > 0, and d is the leaving variable
row/next incoming variable column cell of the first non-standard
form tableau. This shows that the complementary pair theorem is
applicable and the theorem follows.
q.e.d.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 447

The similar parallel theorem with respect to the badname-

variable row and the larger subspace follows by analogy, and
will be referred to as the larger subspace complementary pair
transition theorem.

Exercise:

The following QP problem is given

2 2
Maximise T = xl - x 2 - xl + xl • x 2 - 2x 2

Subject to

Solve this problem starting with sl as badname (If you did not

already do this at the end of section 16.6 this being the same
problem). For each non-standard form tableau developed

a) Extract the non-standard form block

b) Develop the neighbouring standard form tableaux (as far as
they exist). Indicate where possible, which one of them may
be transformed into one of the others by
bl) a diagonal pivot, or
b2) by a complementary pair.

16.8 Dual variable elimination and the parameter theorems

Three criteria are relevant in deciding whether to drive a dual

variable out by the usual pivoting elimination process or
alternatively to possibly breach its non-negativity. We may
outline these criteria as convergence, boundedness and optimality;
this section is devoted to the first problem.

The issue of convergence is linked with that of maintaining

subspace convexity, as may be illustrated by the following
example, which relates to a non-convex problem.

Maximise

Subject to

We must obviously start with sl as badname but if we apply the

rule of the smallest quotient as known from LP without modification
and combine it with the complementarity rule, we get into an
endless cycle, as may be illustrated by actually going through
the cycle once.

This has been illustrated ~n the tableau-series 16.8 below.

448 CHAPTER XVI

TABLEAU-SERIES 16.8

ILLUSTRATION OF CYCLING IN A NON-CONVEX PROBLEM.

16.8 A (SET-UP) 16.8 B (NON-STAND. F.)

NA.! XI X2 P I ' VAL. NA.! XI X:: D1! VAL.

D1 ! 2 ® I PI 1 CD O.S O.S! 0.5

D2! 1 I I D2 ! -0.5 -O.S! 0.5
51 ! -2 -I -5 51 I -2 -1 -5

2T ! 5 2T ! -4 -1.5 -2.S! -2.5

16.8 C (STANDARD FORM) 16.8 D (= 16.8 B)

NA.! PI X2 D1 I VAL. NA. ! XI X2 DI I VAL.

XI I CD 0.5 0.5 I 0.5 PI I 0.5 ~, 0.5

D2 ! -0.5 -0.5 I 0.5 D2 I -0.5 -0.5 I 0.5
51 I 2 I -4 51 ! -2 -I -5

2T I 4 0.5 -0.5 I -0.5 2T I -4 -1.5 -2.5 I -2.5

16.8 E (= SET-UP)
NA.! XI X2 PI I VAL.

D1 I 2 2 1
D2 I 1 I 1
51 I -2 -1 -5

2T ! 5

In the set up tableau l6.8a, the driving variable PI is the

incoming variable, and the smallest quotient is found in the d l
row. Tableau l6.8b is a non-standard form tableau, with xl as
incoming variable, by the complementarity rule; the smallest
non-negative quotient is found in the PI row.

This elimination of the driving variable PI leads to a standard

form tableau, except that tableau l6.8c has not been re-ordered
to the standard form presentation.

We again choose sl as badname, and PI as driving variable. The

smallest non-negative quotient is now found in the xl row, and
we make the previous step in opposite direction. Tableau l6.8d
is identical to tableau l6.8b, but - coming from a different
standard form tableau, the complementarity rule now indicates
d l as incoming variable, and we also make the first step in the
opposite direction.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 449

The following exception to the rule of the smallest quotient,

as far as dual leaving variables is concerned is sufficient to
ensure convergence.

A dual variable may be refused as a leaving variable on account

of failing to satisfy the driving variable increment qualifier.
If advance calculation of the driving variable row/incoming
variable column cell indicates that accepting a particular dual
variable, would activate a complementary primal incoming
variable, which displays a positive non-zero entry in the
driving variable's row, then the dual variable in question is
refused as leaving variable. It will be noted that this qualifier
was breached in the example given above; d should have been
eliminated despite the fact that this woul~ imply that d l would
become negative. As a result, the non-convex variable xl was
activated without guarantee of developing a subspace in which
this variable would be contained, and the driving variable PI
wa~ again eliminated.

Theorem (driving variable parameter theorem)

If the driving variable increment qualifier is obeyed - and

otherwise the rule of the smallest quotient applied -,
then all non-standard form tableaux which are developed from a
standard form tableau describing a solution which is convex in
the All ~l = ~l subspace, display the following properties:

a) The entry in the driving variable row/incoming variable

column cell is non-positive, and negative non-zero, if the
similar entry in an immediately preceding non-standard
form tableau was zero.

b) The entry in the driving variable row/last leaving variable

column cell is non-negative, and positive non-zero if the
similar entry in an immediately preceding non-standard form
tableau was zero.

c) The smaller subspace predecessor and successor tableaux (as

far as existing) describe solutions that are convex in the
AII~1 = ~l subspace.

Proof:

Concerning the successor tableau of an actual standard-form

tableau:

ad a) For a primal leaving variable a positive entry in the

diagonal cell would contradict the convexity property of the
initial standard form tableau, hence a)
450 CHAPTER XVI

For a dual leaving variable a) is required by programme

provision (driving variable increment qualifier), hence a)

ad b) The cell in question is the reciprocal of the pivot;

hence b)

ad c) The smaller subspace predecessor is the actual standard

form tableau, which is assumed to obey the convexity property
referred to.

If the smaller subspace successor exists, a) and b) ensure

convex transition (see the previous section). Hence c).

Concerning the successor tableau of a non-standard form

tableau: assume the theorem to hold for the preceding non-
standard form tableau. The same theorem, as applicable to the
current non-standard form tableau, may then be shown as follows:

ad b) The non-negativity applies, because by assumption a)

applies to the previous non-standard form tableau and column-
updating by division by minus a positive pivot changes a non-
positive entry into a non-negative one.

If the entry in the previous tableau's driving variable row/

incoming variable column cell was zero, then, by the non-
standard form block non-singularity theorem, the previous
tableau's entry in the driving variable row/last leaving
variable column cell was non-zero, hence positive non-zero.
Hence b).

ad a) and c) If the smaller subspace predecessor exists, b)

as proved above implies that the transformation between the
current tableau and the smaller subspace predecessor is by way
of a positive pivot.

The proof supplied for the successor tableau of an actual

standard form tableau is therefore applicable, proving that
the driving variable row/incoming variable column cell is
non-positive. Furthermore, the (positive) non-zero entry
in the last leaving variable column implies a negative non-zero
entry in the previous tableau's driving variable row/incoming
variable column cell and no property of non-zeroness of the
driving variable row/variable column cell of the current
tableau is stated by the theorem. Hence a), if the smaller
subspace predecessor tableau exists.

Concerning c), we first note that if the smaller subspace

predecessor tableau exists, its convexity property is already
assumed, this tableau being identical to the smaller subspace
successor tableau of the previous non-standard form tableau.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 451

If the smaller subspace successor tableau does not exist, no

further proof of c) is needed. If the smaller subspace successor
tableau exists, a) and b) imply c), by the smaller subspace
immediate neighbour convexity transmission theorem. Hence c),
if the smaller subspace predecessor tableau exists.

If the smaller subspace predecessor tableau does not exist, the

driving variable row/last leaving variable column cell of the
current tableau, and the driving variable row/incoming variable
column cell of the previous tableau are both zero. Therefore
the driving variable row/incoming variable column of the
current tableau is non-zero (otherwise the non-standard form
block would be singular), and the same is true for the driving
variable row/last leaving variable column cell of the previous
tableau. The latter cell therefore is positive non-zero by
property b) assumed to be valid for the previous tableau.

To show that the smaller subspace complementary pair theorem

is applicable, we therefore still need to prove the non-
positivity of the driving variable row/incoming variable
column cell.

If the incoming variable is a primal variable this condition

applies by programme requirement - the driving variable
increment qualifier. If the incoming variable is a dual
variable this follows, because the opposite would contradict
the symmetry property of the previous tableau's smaller subspace
predecessor tableau (see also the proof of the smaller subspace
complementary pair transition theorem). Hence a) and c).

The proof now follows by recursive induction q.e.d.

Note that no programme restriction applies to the driving

variable itself, which may, in case of a dual badname (and
therefore a primal driving variable), and a non-convex
programming problem, display manifest non-convexity.

The corresponding theorem for the larger subspace and the

badname-variable is therefore weaker.

Theorem (weak badname variable parameter theorem)

If the driving variable increment qualifier is obeyed - and

otherwise the rule of the smallest quotient applied - non-
standard form tableaux which are developed from a standard form
tableau describing a solution that is convex in the All ~l = -1
b
subspace, and of which one (indicated as the current one), obeys
the following properties:
452 CHAPTER XVI

a) the entry in the badname variable row/incoming variable

column cell is non-positive (and negative non-zero if the
similar cell in any immediately preceding non-standard form
tableau was zero)

b) the entry in the badname variable row/last leaving column

cell is non-negative (and positive non-zero if the similar
entry in any immediately preceding non-standard form tableau
was zero)

Then

All non-standard form successor tableaux of the current non-

standard form tableau (if any) obey the same properties, and
furthermore, the current and its successor tableaux also obey

c) the larger subspace predecessor and successor tableaux (as

far as existing), describe solutions that are conved in the
All ~l = ~l subspace.

Summary of the proof

ad c) If all four standard form neighbours of the current

tableau exist, convex transition from the smaller subspace
tableaux to the corresponding larger subspace tableaux follows
from the appropriate immediate neighbour convex transition
theorem. If there are zeros in the non-standard form block, the
applicability of the complementary pair theorem may be proved,
showing convex transition from the smaller subspace predecessor
to the larger subspace successor tableau, or from the smaller
subspace successor to the larger subspace predecessor tableau,
as the case may be. If there is a zero in the driving variable's
row, and two non-zero elements in the badname row, convex
transition to one of the two larger subspaces follows by the
complementary pair theorem, to the other by one of the immediate
neighbour convex transition theorems.

Hence c) in all cases for the current tableau. From this point
onwards, the proof is analogous to the one supplied for the
driving variable parameter theorem.

End of proof summary.

The difference between the (strong) driving variable parameter

theorem and the weak badname variable parameter theorem arises
on account of two points:

1) there is no programme requirement with respect to the

badname-row/incoming variable cell,
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 453

and

2) in the case of a dual badname in a non-convex programming

problem there may initially be manifest non-convexity in
the diagonal badname-row/driving variable column cell,
i.e. this cell may contain a positive non-zero entry.

However, in the case of a primal badname, i.e. the negative

valued slack of a violated restriction, we have the following:

Theorem (primal badname convexity theorem)

If a series of non-standard form tableaux is developed by

selecting a primal badname-variable in a standard form tableau
which satisfies the property of subspace convexity (and in the
case of a dual leaving variable the driving variable increment
qualifier is obeyed),
then
the first non-standard form tableau satisfies properties
a) and b) listed above for the badname-row.

Proof

If the leaving variable is a dual variable the transition of the

initial standard form tableau to the non-standard form block
is summarized as follows:

Standard form tableau

driving v. next incoming v.

badname v. -/0

leaving v. o -/0

Non-standard form block

last leaving v. incoming v.

badname v. +/0

driving v. + -/0

In the standard-form tableau the leaving variable row/driving

variable column cell (marked II + "), is positive non-zero
because it is the pivot, the badname-row next incoming variable
454 CHAPTER XVI

cell then is negative non-zero by the symmetry rules. The

badname row/driving variable column cell (marked "-/0") is
non-positive because non-convexity, if present, is not manifest
in dual variables' columns. The driving variable row/next
incoming variable column cell is required to be non-positive
by programme provision, on account of the driving variable
increment qualifier.

The sign properties of the non-standard form block as marked,

now follow from the logic of the pivoting operation. If the
leaving variable is a primal variable the similar transition
is summarized as follows:

Standard form tableau

drivinf'; v. next incoming v.

badname v. dl p

leaving v. p d2

Non-standard form block

last leaving v. incoming v.

badname v.

driving v.

The assumed convexity properties of the initial standard-form

tableau imply that the block

~s
lp
rdl p] negative semi-definite
d2

Also, since both columns are dual variables columns with a non-

° °
zero entry p in a primal variable's row, not only must
d l ~ 0, d 2 ~ be assumed, but d l < 0, d 2 < as well.

The requirement of negative semidefiniteness now implies.

(after pivoting on d l )
2
d2 - P /d l < 0,
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 455

hence, after multiplication by d/p (dl/p < O),p - dl • d 2 /p < 0,

proving the assumed sign propert1es in all cases.
q.e.d.

In the case of a dual badname-variable, we may not necessarily

assume that conditions a) and b) are initially satisfied for
the badname-variable row. However, the only accepted way of
again developing an actual standard form tableau, is by
eliminating the badname-variable.

Since that is a negative-valued variable, it must be done by

way of a negative pivot, it obviously follows that for the last
non-standard form tableau, of which the new actual standard
form tableau is the larger subspace successor tableau, property
a) applies by way of a negative non-zero pivot in the badname-
row/incoming variable cell.

Convex transition to the new standard form tableau now follows

in one of the following two ways: if the corresponding smaller
subspace successor tableau also exists, convex transition from
the smaller subspace successor to the larger successor tableau
follows by the immediate neighbour theorem.

If the corresponding smaller subspace successor tableau does

not exist, convex transition from the smaller subspace
predecessor to the larger subspace successor tableau follows
by showing applicability of the complementary pair theorem.

This proves the property of subspace convexity for the next

standard form tableau in all cases, provided the previous one
obeyed that property.
q.e.d.

Exercise

The following QP problem is given

2
Maximise T =
xl + xl • x 2 - xl - 2x2

Subject to xl + x 2 > 5

3x I + x 2 < 30

(xl' x 2 -> 0)

Solve that problem.

(Answer-sheet at the end of the chapter)

456 CHAPTER XVI

16.9 Step-length and objective function value

So far, the QP problem has been discussed in terms of finding

a solution to both the primal requirements Ax < b and the
optimality conditions Dx - A'X < -w, and it has been assumed
that the rule of the smallest-q~ot{ent is used as a method to
select the leaving variable, whenever that is practicable.

If the badname variable is a primal variable, that is an

efficient procedure although there are - in a non-convex problem-
certain complications to be considered.

If the badname variable is a dual variable, there is no

convergence or boundedness problem, but the procedure is not
very efficient. This assymetry arises, on account of the
direction in which the objective function changes: it increases
in response to a dual badname variable and a primal driving
variable, but its value is reduced in the presence of a primal
badname-variable and a dual driving variable.

From (16.2.7), the Lagrangean expression

L = WI X + ! ~I D ~ + .E I (~ - A ~)

we obtain the following differential expression

lIL = w'll x + x I DlIx - .E I A I 1I~ + (~ - A ~) I 1I .E

= -d I 1I x + s'll .E (16.9.1)

The second-order term! 1I x' D 1I x has been omitted from the

right-hand side of (16.9.1) and the symbol 1I should be read as
indicating differentiation, not a vector of finite differences.
(The use of the symbol d might give rise to confusion with the
vector ~).

The complementary slackness condition, upheld for all variables

except the basic pair, causes (16.9.1) to collapse into only
one term.

If the badname-variable is a negative shadowprice d., we have:

J
lIL = - d. 1I x. (16.9.2)
J J
Since no other term of the Lagrangean expression is present in
(16.9.2), we innnediate infer

liT = - d. 1I x. (16.9.3)
J J
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 457

if dj or Xj is the badname variable. The case of an element of x

becoming the badname variable does not arise when applying the
algorithm when starting at the trivial solution but could arise
when re-entering a modified problem.

In that case (16.9.2) and (16.9.3) are also applicable.

The ith slack variable may be re-specified as an explicit

variable, bringing a new additional slack-variable into the
problem. Thus, for example, the problem

Maximise

Subject to

may be re-specified as
2
Maximise T = xl + x 2 - xl + x3

Subject to xl + x 2 + x3 < 10

(xl' x 2 ' x3 ~ 0)

Obviously the shadowprice d 3 is equal to PI' as may be seen

by writing out the dual requirement on x 3 .

Therefore, by generalization of (16.9.3), we also have

(16.9.4)

if p. or s. is the badname variable.

~ ~

If the badname-variable is a dual variable (d. or p.), we

obviously have d. < 0 or p. < 0 and the valueJof th~ objective
function increasJs with ev~ry increase in the driving variable.

If the badname variable is a primal variable, this is not the

case, on the contrary, generally d· > 0 or p. > 0 and the value
of the objective function drops with every ificrease (towards
zero) of the badname-variable.

A similar conclusion is also apparent directly from (16.10.1),

which becomes in the case of a primal variable s. being
~
badnarne-variable.

lIL = s. II p.
~ ~
(16.9.5)
458 CHAPTER XVI

For s. < 0, 6L is obviously non-positive, and negative non-zero

for 6 1 p. # O. The Lagrangean is in that case not identical with
the obj~ctive function, but the two functions are equal to each
other in a standard form tableau, and both are continuously
falling (or not increasing) if the badname-variable is a primal
variable.

Therefore, having postulated boundedness of the primal feasible

space area in all directions, we accept a dual variable (other
than a dual badname-variable, as leaving variable, only if the
badname-variable is a primal variable.

The relative advantage of respecting dual restrictions when

eliminating a primal badname variable is most apparent with a
convex problem which is already in optimal form: the righthand-
side stays non-negative in all dual variables.

16.10 Boundedness and artificial feasibility

Earlier in this chapter we postulated that each incoming variable

always find a leaving variable. As far as the version of the
algorithm which has been outlined so far is concerned, it does
not in fact meet that obvious requirement of being always
effective.

Example

Maximise

Subject to

The set-up tableau of this problem 1S given 1n tableau l6.l0a

TABLEAU 16.10 A

ILLUSTRATION OF SPURIOUS UNBOUNDEDNESS

NAME II X 1 P 1 P 2 !! VALUE

D 1 " 2 - 1 II I
S 1 II 1 II 2
S 2 II -1 II -1

2T II -2 II
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 459

The negative-valued slack variable s2 is the obvious badname

variable, but the P2 column can only be brought into the basis
if d l is accepted as leaving variable, in breach of the driving
variable increment qualifier. (If this step is made in breach
of the qualifier, subsequent application of the complementarity
rule leads to cycling.)

An incoming variable column is said to be unbounded if no

leaving variable may be found against which the corresponding
variable can be exchanged. If, on the other hand, the rules
permit the elimination of some variable (possibly among others),
we say that the incoming variable is bounded by that variable as
leaving variable (or blocked, according to Cottle's terminology).

If an incoming variable is not bounded by any other variable as

leaving variable, despite the existence of an optimal and
feasible solution, we speak of a spuriously unbounded column.

In the example given, xl. 2 is the optimal and feasible solution,

therefore the P2-column 1S spuriously unbounded. There are twp
obvious approacnes towards dealing with this problem, i.e.
avoiding activation of spuriously unbounded columns, even where
they exist, and the imposition of upper limits in all variables.

In a convex problem, the first approach is normally effective

if preference is given to badname-variables which are associated
with negative non-zero elements on the main diagonal.

Example

Maximise

Subject to

The set-up tableau of this problem is given 1n tableau l6.l0b

below.

TABLEAU 16.10 B
PI IS UNBOUNDED. Xl IS BOUNDED

NAME !I X 1 P 1 ! I VALUE

D 1 ! ! -2 ! ! -1
5 2 II -1 1·1 -2

2T II -1 2 II
460 CHAPTER XVI

If sl is selected as badname-variable, we activate the

unbounded Pl-column, but if d l is selected as the first badname-
variable we activate the x 2-column as incoming variable column
and the problem is readily solved in two steps.

In a non-convex problem, such avoidance of spuriously unbounded

columns in this way is not always possible, and we even lack
a proof in the convex case.

However, if the current solution is primal feasible, we have

the following

Theorem

If the current basis represents a feasible solution - vector,

not finding a leaving variable, after activating the incoming
variable column, implies that the QP problem which one attempts
to solve, permits an infinite increase in the value of the
objective function, in a feasible direction (i.e. is unbounded).

Proof

In a tableau which represents a feasible solution the badname-

variable is a dual variable, and the driving variable is a
primal variable. It was shown in the previous section that an
increase of a primal driving variable is linked - by (16.9.3)
or (16.9.4), depending on whether the badname-variable is a
dual slack-variable dj or a specified dual variable Pi -,
to a corresponding increase in the value of the objective
function. Therefore, if the incoming variable is the driving
variable, or if a non-zero (and therefore negative non-zero)
entry occurs in the driving variable row/incoming variable
column cell, no further proof is needed.

Hence, if there is spurious unboundedness, this assumes that the

incoming variable is some other variable than the driving variable
while (in a non-standard form tableau), the driving variable
row/incoming variable column cell contains a zero entry.

The non-standard form block non-singularity theorem then tells

us that there is a non-zero entry in the badname-row/incoming
variable column cell.

Now consider the transition of the non-standard form block, to

the corresponding extract of the smaller subspace predecessor
tableau, which is summarized below, assuming that the
determinant of the non-standard form block is -1, as follows:
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 461

Non-standard form tableau

last leaving v. incoming v.

badname v. ? p

driving v. (~) 0

Smaller subspace predecessor

driving v. incoming v.

badname v. ? p

last leaving v. p o

If the entry in the badname-row/incoming variable column cell

(marked "p") is in fact negative non-zero no further proof
is needed either, as the incoming variable is bounded by the
badname-variable (the marking assumes a positive non-zero
entry) .

Since no dual variables are permitted as leaving variables in

the presence of a dual badname-variable, the incoming variable
(in a non-standard form tableau activated by the complementarity
rule) is a primal variable.

It is now readily verified that the two equal elements in the

off-diagonal cells of the transformed non-standard-form block
(marked "p") cannot, in fact, be equal to each other; they
should be equal in absolute value but opposite in sign.

In that case the summary of the non-standard form block 1S in

fact the following:

last leaving v. incoming v.

badname v. ? -p

driving v. lip o
462 CHAPTER XVI

On account of the driving variable parameter theorem we must

then assume p > 0, and the incoming variable is found to be
bounded by the badname variable as leaving variable. Therefore
only the case of genuine unboundedness with a negative non-zero
entry in the driving variable row/incoming variable column cell
actually exists.
q.e.d.

To accommodate the situation where (most of the) elements of

the initial vector d are non-negative and we wish to maintain
this initial non-negativity of the dual solution in the presence
of a starting solution which is not primal feasible, we
postulate that each element of ~, and each element of p has an
upper limit. -

This does not require any significant additional tableau-space

as we may generalize the device of the double value column
(introduced in section 10.3 for the L.P. problem), to quadratic
programming.

The interpretation of upper limits on the elements of x is

straightforward, but the interpretation of upper limits on dual
variables needs further discussion.

It is in fact readily verified that they correspond to

artificial primal variables. At the cost of a huge penalty it
is permitted to breach the primal restrictions, and the penalty
coefficients are the upper limits on the dual variables.

For example if we suppress the non-active upper limit in xl' the

last example may now be written as
2
Maximise xl xl 100 a l

Subject to -Xl - a l < -2

(Xl' a l > 0)

and the dual restriction on a l is ~n fact Pl ~ 100.

In the interest of being able to elucidate the significance

this problem is written here in the form of an ~xplicit set-up
tableau (tableau l6.l0c).

Here ~l ~s the slack-variable associated with the dual

restr~ct~on on the artificial variable.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 463

TABLEAU 16.10 C

EXPLICIT FORMULATION OF
AN ARTIFICIAL VARIABLE.

NAME!! XI A2 PI !! VALUE

DI !! -2 ! ! -I
Z2 ! ! II 100
SI ! ! -I -I !! -2

2T !I -I 100 2 ! !

In the rest of this section the following conventions with

respect to upper limit distances and their associated dual
variables will be used. Artificial variables are indicated as
a., the index i being also associated with the i th primal
1 . .
restrlctlon.

The dual slack associated with the optimality condition on the

artificial variable a. is then indicated as zi (z 1 in the
example above). 1

Distances between x. (elements of x) and their upper limits are

indicated as bj' coJform section 10.3, for the LP problem.
Bound-distances bj should not be confused with the elements of b
in A~~ ~, but this should be clear from the context.

Dual variables, associated with binding upper limit restrictions

are indicated as u.. (For the sake of completeness the associated
numerical codes arJ also given here:

j for x., 1000 + i for s., 2000 + j for b.,

J 1 J
3000 + i for

-j ford., -lOOO-i for p., -2000 -j for u.,

J 1 J
-3000 - i for z.; these codes are the computational means to
keep track of tEe complementary slackness condition and the
complementarity rule.)

Just as in the LP case, one may economize on tableau-space, by

suppressing the storage of unit vectors. This, however, gives
rise to additional complications in the manipulation of tableaux.
An example of the application of the upper limit facility
(together with the full tableaux) is given here. It concerns the
example used in the beginning of this section.
464 CHAPTER XVI

TABLEAU 16.10 D (SERIES)

ILLUSTRATION OF THE COMPACT FORM OF THE QP-TABLEAU.
EXPLICIT TABLEAUX CU~DENSED TABLEAUX
NA.' XI A2 PI P2 'VAL NA.' XI PI P2 'VAL DIS
Dl 2 -1 I 1 Dl I 2 -1 1 X
D2 CD 100 51
52 I
I 1
-1
2
-1
X
51 1 2 X
52 -1 -1 -1
2T' 1 -2 1 I X
2T I 100 -2 UB I 1 0 0 lOb @]) I x X

BRING IN THE DRIVING VARIABLE

NA.' XI A2 PI D2 'VAL
01 2 -I -I -99
P2 I 100 NO EQUIVALENT.
51 1 2 OF THESE
S2 -1 8) -I TABLEAUX
ON ACCOUNT
2T , 100 -2 -I !-IOO OF THE
STANDARD FORM
NOW ELEMINATE 52 BY A2 DOUBLE STEP
NA.' Xl S2 PI 02 'VAL
DI 2 -I -1 I - 99
P2 1 100
51 2
A2 -1 1

.2T ! -99 100 -2 -1 !-200

RE-ORDER TO STANDARD FORM VECTOR-UPDATES ONLY
NA.' Xl D2 PI S2 I VAL NA. I Xl PI Z2 I VAL DI5
DI 2 -I -I -99 Dl 2 -1 -1 I -99 X
A2 CD1 -I 1 51 ! I 2 X
51 I 2 A2 ! CD X 1
P2 100 ------------------------------
------------------------------
2T ! -99 -I -2 100 1-200 UB ,
2T I -99
100
-2
100
-I
X
'-200
X
X
X
01 IS BADNAME. ELEM. A2 BY Xl DITTO
NA.' A2 02 PI S2 I VAL NA. , A2 PI Z2 VAL DIS
------------------------------ ------------------------------
DI -2 8) -I 2 '-101 DI -2 -1 8) 1-101 X
Xl
51
1
-I
-I I 1
1
51
XI ,
! -I I 1
1 99
X
P2 100 ------------------------------
2T ,
------------------------------
99 -1 -2 ! -I 01
2T I
UB I
99
X
-2
100
-I
X
'-10'
I X
X
X
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 465

NOW ELEMINATE 01 BY 02 01 BY Z2. HE-FORM P2 ANO S2

NA. I A2 01 PI S2 I VAL NA.I S2 PI Dl I VAL DIS

02 2 -I -2 101 P2 I 2 -1 -1 I 01
XI I -I I SI I 1 1 X
SI -I I I XI I -I 1 99
P2 -2 -I 2 - I
2T I -I -I -1 X
2T I 101 -I -I -1 UB I X 100 X X X

RE-ORDER AGAIN TO STANDARD F. RE-ORDER ALSO (NOT THE SAME)

NA.! D'I A2 PI S2 I VAL NA.I 01 PI S2 I VAL DIS

XI - I 1 Xl I -I 1 99
D2 -I 2 -2 10 I SI I CD 1 X
SI -I CD I P2 I -I 2 -1 101
P2 -2 -I 2 - I
2T I -I -1 -1 X
2T -I 101 -1 -I UB I X 100 X X X

P2 IS BADNAME. ELEM. SI BY S2 DITTO. ELEMINATE SI BY S2

NA.I DI A2 PI SI I VAL NA.I DI PI SI VAL DIS

XI I 2 XI I 2 98
D2 -I 2 103 S2 I 1 1 X
S2 -I I I P2 I -2 -3 103
P2 c:::D -2 -3
2T I -I -I I I X
2T I -I 100 -I UB I X 100 X X X

NOW ELEMINATE P2 BY PI DITTO. ELEMINATE P2 BY PI

NA.IOI A2 P2 SI I VAL NA.I 01 P2 SI VAL DIS

Xl 2 XI I 1 2 98
D2 100 S2 I 1 1 X
S2 -I 1 I PI I -1 -1 2 3 97
PI -I -I 2 3
2T -2 -I 3 4 X
2T I -2 100 -1 3 UB! X 100 X X X

RE-ORDER TO STANDARD FORM RE-ORDER ALSO (NOT THE SAME)

NA.I DI A2 SI P2 I VAL NA.I DI SI P2 I VAL DIS

XI 2 XI 1 2 98
02 100 PI I -I 2 -1 3 97
PI -I 2 - I 3 S2 I 1 1 X
S2 -1 1 I ------------------------------
2T -2 3 -I 4 X
2T I -2 100 3 -I 4 UB I X X 100 X X

WHICH IS THE OPTIMUM

466 CHAPTER XVI

To contain the size of the explicit tableau to some extent,

only the active upper limit on P2 is included in the tableaux.

The following notes arise in connection with the tabulation

l6.l0d.

First of all, not all explicit tableaux have obvious condensed

equivalents. If a row and and column are "flicked over", this
is computationally mos~asily handled in one operation, but
in terms of solution-interpretation, there are separate steps.
If two steps of this type form a complementary pair linking
two standard form tableaux, we speak of a standard form double
step.

One further computational detail is useful to mention at this

point.

Upper limits and upper limit distances have been marked with a
cross X indicating "not applicable" where they are ignored
by the search operations. In fact figures are entered in these
cells and they are used when rebuilding the corresponding
"normal" variable. For example, z2 is the distance between P2
and its upper limit. As z2 enters the basis under circumstances
where the non-negativity of P2 is not protected at that stage
of the algorithm, z2 has no upper limit, but the value of P2
is in fact stored in the upper limit cell of the z2 vector.
The upper limit facilities solve the problem of boundedness, so
to say "by brute force".

with such strong restrictions in the specification of the

problem, we have little difficulty in proving that all incoming
variable columns are bounded. This proof is as follows:

If the incoming variable is a primal variable: obvious. Even

if the incoming variable itself is a slack-variable which has
no upper limit, a change in a primal slack-variable requires
a change in at least one specified primal variable x., which
is driven either towards its upper limit, or towardsJits lower
limit of zero.

If the badname-variable is a primal variable, there is a non-

positive entry in the driving variable row/incoming variable
column cell, as well as in the badname-variable row/incoming
variable column cell, at least one of them being negative non-
zero. Therefore, either the badname-variable is driven towards
zero and can be eliminated at that point, or the driving
variable is driven against its upper limit.

Having proved boundedness of primal variables as well as of

all variables in the case of a primal badname-variable, we still
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 467

need to prove boundedness of a dual incoming variable in the

presence of a dual badname-variable. For this we refer to the
proof supplied for the case of a feasible solution-vector, as
the same argument (contradiction of the symmetry property is
also applicable in the general case of a dual badname-variable
and a dual incoming variable.

End of proof summary

Note that the above proof implies that the algorithm will
always converge to the development of an end-of-algorithm
standard-form tableau with a non-negative (primal and dual)
solution vector.

If the meaningful problem is empty, this will therefore become

apparent because artificial variables are left in the end-of-
algorithm basis. If the meaningful problem is unbounded, this
will become apparent because artificial fancy-high upper limits
on primal variables are binding on the optimal solution.

That a binding artificial upper limit on a primal variable

implies unboundedness of the meaningful problem is obvious.
The relation between an artificially feasible end-of-algorithm
solution and emptiness of the meaningful problem needs, however,
further investigation.

We partition A in two block-rows. One block-row refers to the

satisfied restrictions, the other to the ones in which artificial
variables remained in the basis; the vectors band E are
partitioned accordingly.

If an artificially feasible solution is developed the optimality

conditions as satisfied by the artificially feasible solution are

d (16.10.1)

the optimality-conditions on the meaningful variables (with only

those elements of d which refer to non-basic variables being in
fact non-zero),
and.

o= -p
-2
+ f
-2 (16.10.2)

the optimality conditions on the active artificial variables,

!2 being the fancy-high upper limits on P2'

If the meaningful problem was indeed empty, the primal solution

would be equal to the primal solution of the following problem~
468 CHAPTER XVI

Maximise

Subject to A x < b
1 - --1
implying inter alia a solution in the corner of the feasible
space area.

The dual requirements of this problem are that for some E** ~ 0,

-A' f - A' p** < 0 (16.10.3)

2 -2 1 -
We would like to assume that (16.10.1) could be obtained as the
summation of

d* (16.10.4)

and

d**
-
= A'1 p** + A'2 -2
-1
P (16.10.5)

where g*, 2t
is the dual solution associated with the (block
triangular) block-pivot which emerges if the violated
restrictions are removed from the problem (not normally an
optimal solution, i.e. not ~* ~ 0, pt ~ 0) and d** are the
slack-variables in (16.10.3), d** > O.

If the dual variables are of a different (greater) order of

magnitude than the primal variables, it is safe to assume that
(16.10.1) is in practice equivalent to (16.10.5).

To allow this condition of dominance of (16.10.1) by the dual

variables' terms ~, E, and Ai Ai
E to be met, we must ensure
that incoming variables which are activated before all dual
badnames have been dealt with are not only technically bounded,
but also bounded even without the artificial upper limits on
primal variables. Otherwise we may after all develop an
artificially feasible "optimum" for a non-empty problem.

Example
2 2
Maximise xl + x 2 - xl + x2

xl > x 2 + 2, xl' x 2 > 0

We put the artificial limits all at 100

(xl' x 2 .::. 100, PI -< 100) .

If this problem is solved by selecting badname-variables in
the (inappropriate) order, d 2 , d l , sl' we develop the series of
tableaux as summarised in the tabulation l6.l0e.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 469

TABLEAU 16.10 E

IN A NON-CONVEX PROBLEM WITHOUT A FEASIBLE STARTING

SOLUTION. INAPPROPRIATE BADNAME-SELECTION LEADS TO
AN ARTIFICIALLY FEASIBLE SOLUTION WITH A HIGH PO-
SITIVE SOLUTION VALUE.

NAME II X I X 2 P I II VALUE DIST

-------------------------------------------------
D 1 I! -2 1 II -I X
D 2 !I 2 -1 !I - I X
S I II -I I II -2 X
-------------------------------------------------
2T ! ! -I -I 2 II X
BOUND I ! 100 C!]]) I 00 ! ! X X

NAME I! X B 2 P I !I VALUE DIST

-------------------.-----------------------------
D I ! ! @) ! ! - I X
U 2 ! ! 2 !, 201 X
S I !, -I -I II -102 X
-------------------------.-----------------------
2T II -I 201 102 I! 20200 X
BOUNDII 100 100 100 II X X

NAME ! I D I B 2 P II VALUE DIST

-------------------------------.-----------------
X I II -O.~O -0.50 II 0.50 99. SO
U 2 ! ! 2 I II 201 X
S I !I -0.50 -I -0.50 II -101.50 X
-------------------------------------------------
2T !I -0.50 201 101.50 ! I 20200.50 X
BOUNDII 100 100 ill]) I! X X

NAME ! I D 1 B 2 Z; 1 II VALUE DIST

-------.-----------------------.-----------------
X 1 ! I -0.50 0.50 ! ! 50.50 49.50
U 2 II 2 -1 II I0I X
A I II 0.50 I -0.50 II 51.50 X
-------------------.-----------------------------
2T II 50.50 - 10 I 51.50 II 4900.50 X
BOUND! I 100 100 100 II X X
470 CHAPTER XVI

N.B.
Readers who might experience difficulty in following the
tableau-manipulation in this example, may wish to work through
the corresponding explicitly written problem, for which the
set-up tableau is given here, with suppression of the never-
active upper limit on xl'

TABLEAU 16.10 F

THE PROBLEM GIVEN IN TABLEAU 16.10 E.

WITH AN ARTIFICIAL VARIABLE FOR RESTRICT-
ION ONE. AND AN EXPLICIT UPPER LIMIT ON X2.

NAME I XI X2 AI PI U2 I VALUE
--------------------------------------------
DI -2 -1
D2 2 -1 -1 -1
ZI 1 100
51 -I -I -2
82 100
--------------------------------------------
2T -I -I 100 2 -100

(Note that the example contains two standard form doublt steps;
one for x 2 reaching its upper limit, and one for Pl reaching
its upper limit. When the tableaux are written in full, these
double steps obviously become two separate steps. Furthermore,
in the absence of the actual unity-pivots, step-markings for
upper limits have been made on the upperbounds row.)

The true optimal and feasible solution of this problem is,

however, x] = 2, x 2 = 0, and if either d l or sl is selected as
the first Dad name-variable, that solution is found as the
end-of-algorithm solution.

To ensure "meaningful boundedness", i.e. boundedness even in

the absence of (artificial) upper limits of all incoming
variables activated by a dual badname-variable, it is sufficient
that the standard-form tableau in which the badname-variable
is selected contains a negative non-zero cell in the diagonal
badname-variable row/incoming variable column cell.

Theorem (Convex primal boundedness theorem)

If the badname-variable is a dual variable (and therefore the

driving variable is a primal variable) and the driving variable
displays (the strict convexity property of) a negative non-zero
diagonal cell in the standard-form tableau (in which the
badname-variable is selected)
then all incoming variables are bounded by the badname-variable
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 471

as leaving variable.

Proof

The condition is equivalent to strict convexity in an

All ~l = ~l subspace, which may be developed by direct exchange
of the driving variable against the badname-variable. The
tableau which is developed by that step, is either the next
actual standard form tableau (in which case no further proof
is needed), or it is the larger subspace predecessor tableau
of the first non-standard-form tableau developed.

The existence of, and the convexity properties of some other

standard form neighbours of the current tableau is now shown
by the following summary of the relationship between the current
tableau and its larger subspace predecessor.

Non-standard form tableau

last leaving v. incoming v.

driving v. a 1/b + ac

badname v. bc

Larger subspace predecessor

badname v. income v.

driving v. -alb 1/b

last
1/b c
leaving v.

We begin with denoting the figures in the last leaving variables

column of the current non-standard form tableau as a and b, as
marked.

By assumption, the larger subspace predecessor exists, and

describes a solution which is (strictly) convex in the
All ~l = ~l subspace.
472 CHAPTER XVI

For b=O the theorem follows immediately from the properties of

the non-standard form block as discussed in sections 16.7 and
16.8, we now assume b > O.

Upon developing the larger subspace successor tableau we

calculate the cells marked -alb and lib in the badname variable
column. The badname-variab1e is a dual variable containing a
non-zero entry in a primal variable's row (lib in the last
leaving variable's row), therefore the diagonal cell is
negative non-zero.

- alb < 0,

therefore, since b > 0, a > o.

Similarly, the current incoming variable is a dual variable
containing a non-zero entry in a primal variable's row (lib in
the driving variable's row). Therefore (denoting the diagonal
cell as c)

c < o.
The other cells in the current tableau are now calculated
backwards. The results obtained so far, a > 0 and c < 0 imply
bc < 0, showing that the initial assumptions (convexity in
the Al1~1 = ~1 subspace, b > 0) imply:

(1) the existence and convexity of the larger subspace

successor tableau - thus permitting recursive
application of the same argument - and

(2) boundedness of the incoming variable's column by the

badname-variab1e as leaving variable.
q.e.d.

Furthermore, since the similar property a > 0 is valid for the

successor tableau, we must also assume

lib + ac < 0

i.e.

all four standard form neighbours exist.

In other words, a negative non-zero cellon the main diagonal

in the initially selected badname-row/driving variable column
cell ensures non-standard form blocks which contain four
non-zero elements.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 473

Exercise 16.10

Solve the LP problem given in the example in section 9.5

(artificial variables in LP), as a quadratic programming
problem. Do this first with two explicitly written artificial
variables, and subsequently with artificial upper limits on the
dual variables, using the tableau-presentation discussed in
this section.

16.11 The treatment of infeasible starting solutions

In this section, we outline, in effect, three alternative

versions of the QP algorithm. We call them the convex mode of
operation, the non-convex mode of operation, and the amended
convex mode of operation.

The convex mode of operation relies on the selection of badname

variables, which can be either primal or dual variables and uses
the complementarily rule to drive them out. The non-convex mode
of operation is characterized by two separate stages of
calculation, the initial Phase I being primarily devoted to
finding a feasible solution, by means of complementary pairs of
steps. The third version of the algorithm is to some extent a
compromise between these two, and will therefore be discussed
last. We first outline the convex and the non-convex modes of
operation.

In the convex mode of operation, we proceed as follows

1) Select as badname-variable a basic variable which is

associated with a negative entry in the value column and
preferably also a negative entry in the diagonal cell
(thereby avoiding the activation of artificial upper limits
as far as possible). The criterion function suggested here
is the product of the entry in the valule column (if
negative) multiplied by the sum of the diagonal cell and a
small negative number. The function chosen was in fact:

T[I, N+lJ * (T[I,I] - 0.000001),

For a violated primal restriction this function is of

necessity positive, a manifestly non-convex driving variable
is therefore always a primal variable (associated with a
dual badname-variable), and such a choice is made only if
the solution is primal feasible.

2) Bring in the driving variable as incoming variable.

474 CHAPTER XVI

3) In standard form tableaux and non-standard form tableaux

alike select the leaving variable as follows:
If the badnarne-variable is a dual variable apply the rule
of the smallest quotient over the badname variable row,
and primal variable's rows. (Accept a negative pivot only
in the badname variable row, fly through other violated
restrictions.)

If the badname variable is a primal variable then dual

variables also qualify as leaving variables and are
included in the search for the smallest quotient, subject
to the driving variable increment protection qualifier.

4) If no negative entries in the value column are left, the

problem is solved, otherwise return to 1.

This is a convergent procedure. For each selected primal

badname-variable at least one violated primal restriction
becomes satisfied in a finite number of steps, (either
artificially or also meaningfully); once only dual badname
variables are left the value of the objective function increases
by a finite amount in every two steps made. Its main drawback
is, however, that an artificially feasible end of alsorithm
standard form tableau, lS not necessarily evidence of emptiness.

In the non-convex mode of operation, there are two distinct

phases, "Phase I" and "Phase II", where Phase I concerns
finding a feasible solution, just as in LP, Phase II the search
for the optimum, a feasible solution having been found already.
An essential characteristic of Phase I is that, - just as in
LP - only the corners of the feasible space area are investigated.
This means that standard form tableaux maintain the characteristic
block triangular structure with zero entries in all primal
variable row/dual variable column cells throughout Phase I.

We proceed as follows:

Phase I

1) Select as badname-variable a primal variable which is

associated with a (the most) negative entry in the value
column. If no negative-valued slack-variable is found,
proceed to Phase II.

2) Select an incoming variable column associated with a

negative entry in badname row/income variable column cell
(if none is found this proves the problem to be an empty
one).
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 475

The recommended selection criterion within this category

of incoming variables is as follows:

First, preferred columns are those which are also associated

with a negative valued dual slack-variable, and among those
the one with the largest absolute value of the sum of the
two negative coefficients, the one in the s. row/x. column
cell and the one in the dj row/value column~cell. J In the
absence of preferred incoming variable columns choose the
incoming variable by the criterion of the dual ratio. In a
QP tableau in standard form, the search operation for the
smallest dual ratio may be performed as a search for the
smallest critical ratio with the driving variable column as
tentative incoming variable column.

The primal variable associated with the dual variable

tentatively indicated as leaving variable, is then the
incoming variable.

Disregard the driving variable increment qualifier (illustration

example to be given below). Do not however, bring in the
driving variable as incoming variable at this stage, we need
a primal incoming variable.

3) Select a primal variable as leaving variable, by applying

the rule of the smallest quotient. The eligible non-
negative quotients relate to two categories of primal
variables viz:

a) Non negative valued primal variables, whose non-

negativity needs to be protected.

b) The (negative valued) badname variable, but only if

one of the two following conditions is satisfied. If no
pivot of satisfactory absolute value has been found in
category a), or if a higher solution value is attained by
selecting the badname variable instead. (See also the
discussion on "flying through" violated restrictions in
an LP problem in Chapter IX.

Dual variables do not at this point qualify as leaving

variables. Since the badname row/incoming variable column
cell contains a negative non-zero cell, the issue of
unboundedness does not arise here.

4) Make the step

5) In the resulting non-standard form tableau select

(disregarding the value column), as incoming variable the
complement of the incoming variable selected under 3, as
leaving variable the complement of the leaving variable
476 CHAPTER XVI

selected under 3, thus completing the step made under 4

as a complementary pair.

6) Make the step selected under 5

This step does not cause any change in any primal
variable's entry in the value column all primal variable's
row/incoming variable column cells containing zero entries.

On account of the block triangular structure of the block-

pivot, the leaving variable row/next incoming variable column
cell of the standard form tableau in which the search operations
under 2 and 3 were performed, contains a zero entry. The
figures in the next incoming variable column are therefore
unchanged between the standard form and the non-standard
form tableau.

7) If the leaving variable selected under 3 was the badname-

variable, or if the badname-variable has ceased to be
negative-valued (by flying through" its non-negativity),
return to 1, otherwise to 2.

Phase II:

8) Proceed as in the convex mode.

Note that in this version of the algorithm, dual variables are

only accepted as leaving variables, if they are either: the
complement of a previously introduced primal variable (in the
second step of a pair made under 6 in Phase I), or: the badname-
variable (in Phase II).

It follows that the only way in which we may develop a solution-

vector in which the number of basic variables xj exceeds the
number of eliminated slack variables si is by direct exchange
of a primal driving variable Xj or si against the badname
variable dj or Pi, a situation which can only arise in Phase II.

The convex mode of operation as outlined above is basically

a formalization of an algorithm which has been discussed and
illustrated extensively in the previous section, but we now
give an illustration of the non-convex mode of operation.

Example

Maximise

Subject to
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 477

The set-up tableau of this problem is given below as

tableau l6.11a.

TABLEAU 16.11 A

ILLUSTRATION-PROBLEM CONCERNING
THE NON-CONVEX MODE OF OPERATION.

NAME ! I XI X2 PI P2 II VALUE

DI
D2
II
II
-2
2 "" -I-2
I
- I
-I
2
- I

SI
S2
II
Q)
I'
-I I
-2 ""
2T 'I -I -I ,,
---------------------------------------
2

We now proceed as follows:

Phase I

1 Choose sl as badname-variable. (in the convex mode: d l )

2 Choose Xl as incoming variable (preferred)

3 Choose s2 as leaving variable (the pivot is actually the

same as in the convex mode with d l as badname-variable.

4 Make the step, to develop tableau l6.llb

(The tableau is at this point also re-ordered. This re-ordering

is organized in such a way as to permit return to a correctly
ordered standard form tableau, by the second step of the pair,
without further re-ordering.)

TABLEAU 16. II B

NON-STANDARD FORM TABLEAU. FOLLOWING

THE FIRST (PRIMAL) STEP OF A COMPLEMEN-
TARY PAIR.

NAME II P2 X2 PI S2 II VALUE
XI II -2 II
D2 II 2 2 -I II -I
SI
DI
II
! I 8)
-I
-4
"
---------------------------------------
I
2 " -2
-I

2T I' -3
"
2
478 CHAPTER XVI

5 To get back to standard form, we echange P2 against d l

as leaving variable (step still the same as in the convex
mode).

6 Develop the next standard form tableau, which 1S

tableau l6.l1c.

TABLEAU 16.11 C

STANDARD FORM TABLEAU. 51 STILL IS THE

BADNAME-VARIABLE.

NAME II 01 X2 PI 52 II VALUE

Xl II -2
1 "
D2 -6 -3
" 2 4 "
51 ! ! c::D -1
1 "
-2
1
P2 ! I -1 4 -2 "

2T II -3 2
"
7 sl is still badname variable and has not so far been
eliminated therefore return to 2 (in the convex mode d 2
would now become the badname-variable)

2 Select x2 as incoming variable (preferred) (in the convex

mode x2 would be the driving variable, hence also the
incoming variable).

3 Select sl' the badname variable, as leaving variable, this

being the only primal variable which gives rise to a quotient
of the appropriate sign. (in the convex mode d 2 would become
the leaving variable).

4 Develop the non-standard form tableau l6.lld

TABLEAU 16.11 D
NON-STANDARD FORM TABLEAU. FOLLOWING
THE FIRST (PRIMAL) STEP OF THE SECOND
COMPLEMENTARY PAIR.

NAME I! 01 PI 5 1 52 II VALUE
---------------------------------------
Xl ! I -2 -1 ! ! 4
X2 ! ! -1 -1 ! ! 2
02 ! I 2 CD -6 -2 I !
-7
9
P2 ! ! -1 -1 4 2 I I

2T !! 2 -3 -2 I! 6
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 479

5 Select as incoming variable PI and as leaving variable d 2

6 Develop the tableau l6.lle

TABLEAU 16.11 E

A FEASIBLE (AND OPTIMAL) TABLEAU

NAME I I DI 02 S I S2 II VALUE
---------------------------------------
XI ! ! -2 4
-I I I
X2 ! ! -I -I I I 2
PI I I 2 -6 -2 I! 9
P2 I I 1 -2 I I 2
---------------------------------------
2T ! I -4 -2 9 2 I! -12

which is the optimum tableau.

At this point it is useful to summarize the relative(dis)-

advantages of the two methods, in relation to certain groups
of problems. The various cases are first summarized as follows:

convex problem non-convex problem

The trivial basis makes no difference the two

lS primal feasible methods become identical In
(not dual feasible) that case

The trivial basis Use the convex mode; ? ?? ? ? ?

is dual feasible optimal form lS The convex mode
(not primal feasible) maintained through- may get there
out quicker, but the
non-convex mode
is safer

The trivial basis Ifmany dual restrict- Use the non-

lS neither primal lons are violated, use convex mode - no
feasible nor dual the non-convex mode: problem of spur-
feasible it can seek out pre- ious emptiness -
ferred columns

In a non-convex problem with a dual feasible trivial basis (a

"minimization" problem), there may, or there may not be a trade
off between computational efficiency in the well-behaved case,
and proof supported effectiveness.
480 CHAPTER XVI

The problem of "running away" in the artificially feasible area

may arise despite the initial optimal form, because satisfied
dual restrictions may 'be flown through in the wrong direction
first, on account of the driving variable increment qualifier
and then activate directions in which the quadratic component
of the objective function comes to dominate over the penalty
coefficient which are supposed to drive the artificial variables
out.

In addition to this the non-convex mode can also be more

effective as an optimizing algorithm than the convex mode. In
the convex mode, the value of the objective function consistently
drops when a violated primal restriction figures as badname.
The non-convex mode, with its L.P. style Phase I on the other
hand, can in fact gain solution value during Phase I.

Even if a drop in the value of the objective function is

unavoidable, the non-convex mode has greater freedom in
selecting columns and the criterion of the dual ratio does to
some extent contain the drop in the solution value.

The amended convex mode is an attempt to improve on the convex

mode as an ef,ficient procedure to solve problems in which the
trivial basis violates primal as well as dual requirements. It
does so, by preferably selecting as badname variables shadow-
prices of variables which would qualify as "preferred" incoming
variables in Phase I of the non-convex mode, in preference to
other dual variables.

Example
2 2
Maximise T = xl - x 2 + 2x3 - xl - 2x2

Subject to -xl - x 2 + x3 < -2

xl - 2x3 < 1

(xl' x 2 ' x3 ~ 0)
For this example we only give the set up tableau, and arguments
for choosing a particular first pivot.

If sl were selected as badname variable in the convex mode, we

woula be forced to introduce x2 into the basis as the first
primal incoming variable, despite the fact that X2 leads to a
reduction in the value of the objective function.

In fact sl is selected as badname variable only in the non-convex

mode. In the convex mode shadow prices of the strictly convex
variables xl and x3 both have preferences over sl.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 481

TABLEAU 16.11 F

PIVOT-SELECTION UNDER THE AMENDED CONVEX MODE.

NAME II Xl X2 X3 PI P2 II VALUE

~~---!!---~----~:-----------!----:~---::---:!--
03 !I -2 -1 2 II -2
51 II -1 -1 1 II -2
52 II 1 -2 II 1
------------------------------------------------
2T !I -1 -2 2 -1 II

In the convex mode as it was outlined above, d3 would be chosen.

This is a pity because xl leads to an increase in the objective
function and to a less negative value of sl at the same time.
The non-convex mode will seek it out for that reason. The
convex mode as amended does the same by selecting dl as badname
variable, in preference to the more negative valued d3 variable.
This is done by mUltiplying the criterion-function for selecting
the badname variable with another load-factor, which is in fact
0.01 plus the sum of the absolute values of all the negative
entries in the intersections of violated restriction rows and
the driving variable column i.e. in the example at hand 1.01
for dl and 0.01 for d3 (and 0.01 for directly tackling sl
as badname variable).

The subject of this section, the treatment of infeasible

starting solutions is a major point of distinction between the
QP algorithms offered in this ~ook, and their closest relatives
as previously developed by other authors. Having covered this
point, it is therefore appropriate to briefly review these
related QP algorithms, in particular Cottle's principal
pivoting method and the a-symmetric version of the Van de Panne
and Whinston method.

The symmetric variant of Van de Panne's method will be discussed

in the next chapter, whilst other QP algorithms, although
existing*, differ so substantially from the algorithms offered
here that a more extensive discussion would be needed, and we
shall refrain from such a more extensive summary.

*This refers in particular to Beale's [3] method.

482 CHAPTER XVI

Cottle's algorithm is most akin to what has been described

here as the convex mode of operation. Its convergence rests
on regularly reducing the number of violated restrictions
whether they are primal or dual restrictions. For this reason
it has been suggested (by Van de Panne, Linear and ~uadratic
Programmin8 p 323) that this method should be more appropriately
called the index-method, rather than the principal pivoting
method, as Cottle names this method. It is designed for convex
problems only, and it applies the rule of the smallest quotient
in the more strict sense of protecting the non-negativity
of both primal and dual variables at all stag"es of calculation.
Basically it is the algorithm which we began to outline in
section 16.3.

The driving variable increment qualifier - and hence the

possibility to apply the convex mode of operation to non-convex
problems is, as far as I know, novel to this book. If the
original version of Cottle's method is applied to a problem
where the objective function is not properly convex, it may
break down in cycling, the convex mode of operation can be
shown to converge in all cases even where the end of algorithm
tableau may represent (in the non-convex case) a local rather
than the global maximum and even possibly a local maximum ~n
an artificially feasible region giving rise to a spurious
indication of emptiness.

The two-phase method on the other hand is directly due to

Van de Panne. Indeed the non-convex mode of operation comes near
to being an implementation of the Van de Panne and Whinston
method.

The one difference is that, for similar reasons as advocated

in Chapter IX for the L.P. case, there is no complete
separation between the two phases i.e. optimality is considered
already during Phase I, giving rise to the notion of "preferred"
columns, to the possibility to "fly through" violated primal
restrictions, and to the use of the dual ratio as a selection-
criterion for choosing between non-preferred columns. Van de
Panne (Linear and Quadratic Programming p 279) recommends the
minimization of the sum of all artificial variables by an
LP algorithm as Phase I for the QP problem, a procedure which
has the same limitations as using artificial variables in the
LP case.

16.12 Ordering of the quadratic programming tableau

Recall the symmetry properties of the quadratic programmin8

tableau, especially when in standard form (see section 16.4).
For a number of reasons, it is desirable to keep these symmetry
properties apparent in the tableaux developed by the algorithm.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 483

Firstly, a symmetrically ordered tableau is the most suitable

way of presenting the end result. Writing a programme-code is
also simplified, if it is known where the rows and columns of
the tableau may be found. Unfortunately, simplex steps tend to
bring the tableau in a random non-ordering, in quadratic as well
as in linear programmine.

This problem was solved, for linear programming (see sections 12.2
and 12.3), by re-ordering the tableau afterwards.

With quadratic programming it is more practical to maintain

ordering as far as possible at the time of making the steps.

This may be done, by the following two devices, to be applied at

each step, except when the tableau is coming back to standard
form.

a) Interchange the pivot column with the "alternate" of the

pivot row. This alternate ~s the next pivot column.

b) Interchange the pivot row with the "badname" row.

Application of these re-ordering rules may be illustrated by

writing an example, as follows:
2 2
Maximise T = xl - 5x 2 + xl - x2

Subject to xl + x 2 > 2

xl + x 2 < 10

TABLEAU 16.12 A

SET-UP TABLEAU OF THE EXAMPLE ON ORDERING.

t-JAME I ! ! XI X2 PI P2 !! VALUE

! CODE !! 2 -IDOl -1002 !!

01 -I ! ! 2 I -I ! ! -I
02 -2 ! ! -2 Q) -I ! ! 5
SI 100 I ! ! - I -I I! -2
S2 1002 ! ! I I I! 10
--------------------------------------------------
2T ! ! -I 5 2 -10 ! !
484 CHAPTER XVI

The set-up tableau of this problem is given in tableau l6.l2a

and we now proceed as follows:

Re-order, to maintain ordering in the next tableau

a) Interchange the PI column (pivot column) with the x 2 column,

x 2 being the complementary variable to the pivot-row
variable d 2 .

b) Interchange the d 2-row (pivot) with the sl (badname) row.

The resulting tableau has been named l6.l2b.

TABLEAU 16.12 B

INITIAL TABLEAU. RE-ORDERERED FOR THE FIRST STEP.

NAME I

I CODE II
" Xl PI

-1001
X 2

2
P2

-1002
I I VALUE

"
Dl I -1 I I 2 -1
51 I 1001 " -1 - -1
"
"
-1
-2
D2
52
I
!
-2
1002
I I
II
CD -2
1
-1 I I 5
" 10
2T " -1 2 5 -10
"
This re-ordering ensures that the new basic variable (the pivot
column) and the just eliminated pivot-row variable (which may
stay a non-basic variable) are stored in their "natural"
positions.

TABLEAU 16.12 C

NON-STANDARD FORM TABLEAU. WITH ONLY

THE "WRONG" NAMES OUT OF ORDERING.

NAME

I CODE II
" Xl D2

-2
X2

2
P2

-1002
II

"
VALUE

Dl -1 I I 2 -1 2 " -6
51 I 1001" -1 8) " -2
PI 1-1001 I I -2 -1
52 ! 1002 I I 1
" 5
" 10
2T " -1 -2 9 -8 ! I -10
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 485

The d2 column in the new tableau comes on place 2, and only the
x 2 column, which is to become pivot column immediately afterwar~s
is in a "wrong" place. the PI row in the new tableau comes in
place (minus) 1001, and only the badname-row (to be eliminated
in due course) is in the "wrong" place.

The next tableau, numbered l6.l2d, is ln standard-form and no

re-ordering is needed. Here we select d l as the badname-variable.

TABLEAU 16.12 D

STANDARD FORM TABLEAU. ORDERED ACCORDING TO CODES.

NAME I II XI 02 51 P2 II VALUE
--------------------------------------------------
! CODE II -2 IDOl -1002 "
--------------------------------------------------
-10
01 -I II -1 2 II
X2 2 II CD -I
-2 -I
II
II
2
9
PI ! -I 001 II 2
52 I 1002 II II 8
--------------------------------------------------
2T II -10 -2 9 -8 II -28

Before actually making the next step, we again re-order, to

maintain ordering in the next tableau, which is once more in
non-standard form. To develop tableau l6.2e, we proceed as
follows:

a) Interchanee the pivotal xl - column with the (next pivotal)

d 2 - column.

b) Interchange the pivotal x 2 - row with the d l badname - row.

TABLEAU 16.12 E

THE SAME TABLEAU. RE-ORDERED FOR THE- NEXT STEP

NAME I II 02 XI 51 P2 II VALUE

I CODE II -2 -2 1 001 -1002 II

--------------------------------------------------
X2
01
2 I!
-1 II -I
CD -1
2
II 2
II -10
PI ! -100 I II I 2 -2 -1 II 9
52 ! 1002 II 1 II 8

2T II -2 -10 9 -8 II -28
486 CHAPTER XVI

As a result of the ordering of tableau 16.12e, the successor

tableau only has the badname-row i.e. d l and the complement
of the current pivot row i.e. d2 out of ordering. After making
the step, we develop tableau 16.12f.

TABLEAU 16.12 F

AGAIN. THE 'WRONG' NAMES ARE IN THE WRONG SLOTS.

NAME I II D2 X2 SI P2 I I VALUE
------------------------------------------------~-
I CODE II -2 2 1001 -1002 II
--------------------------------------------------
XI I I! -I II 2
DI I -I II -I 2 " -10
PI 1-1001" I -2 -I II S
S2 I 1002 " II 8
--------------------------------------------------
2T 10 -I -8 II -8
" -2

The next tableau will be in standard form, therefore no

reordering before making the next step is needed. We develop
tableau 16.12g.

TABLEAU 16.12 G

STANDARD FORM TABLEAU. ALREADY CORRECTLY ORDERED.

NAME !

I CODE II
" DI

-1
X2

2
SI

1001
P2

-1002
II VALUE

"

""
Xl 1 II -1 2
D2 I -2 II -1 -2 10
-s
"
PI 1-1001 II 1 -'2 2 -1
S2

2T
I 1002

II -2 10
"
---------~----------------------------------------
-s -8 II 12
1 II 8

This particular problem of manipulating the, QP tableau may now

be considered as having adequately illustrated and we will not
pursue the example any further.

16.13 Equations and variables without non-negativity-restriction

There is a strong analogy between an equation and a variable

without non-negativity requirement. The one is the dual problem
of the other. A variable which is not subject to a non-negativity
restriction, but is allowed to attain negative values, is brought
into the list of basic variables with priority. It then stays a
basic variable irrespective of its sign. Its associated dual
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 487

restriction therefore is an equation. Likewise the dual

variable associated with an equation has to be brought into
the list of basic variables, and then stays there. It does
not become bad name and is exempt from the search operation
for a pivot row and is allowed to become negative instead.

In the simplex algorithm for quadratic programming which is

the subject of this chapter, both problems are handled along
the lines followed in section 10.2 for equations in linear
programming.

The equation is presented with a negative constant, i.e. a

violated restriction. The "conservative" r ..Jle for choosing the
pivot-row is applied insofar as equation-slacks are concerned,
i.e. they are not allowed to become positive. Once the equation
is binding, the associated dual variable is or becomes a basic
variable. The dual variable of an equation is initially brought
into the vector of basic variables with a positive sign but is
allowed to become negative later on.

Example

Maximise 1.4 xl + 2 x 2

Subject to xl + x 2 = 3

(xl' x 2 > 0)

We write the equation as - xl - x 2 = - 3, rather than as

xl + x 2 = 3, and, just as in the LP-case, we start with a
set-up tableau which contains in fact a legative-valued slack-
variable, i. e. sl = -3.

This set-up tableau is given as tableau 16.13a below.

TABLEAU 16.13 A

SET-UP TABLEAU WITH AN EQUATION. PRESENTED

AS A VIOLATED INEQUALITY.

NAME I II X I X 2 P I VALUE
-------------------------------------------------
I CODE II 2 -1001
-------------------------------------------------
D I - 1 I I ~ -1.~0

D 2 -2 II I ~ I -2
S 1 1001 I I -I -1 I = I -3
-------------------------------------------------
2T II -1 .~O -2 3
488 CHAPTER XVI

To facilitate following the ordering

tableau l6.l3a and its successor tableaux are presented with
numerical name-codes as well as alphanumerical names.

Note, that this is really an LP problem, which is solved by a

QP algorithm.

The solution of this problem, by the non-convex mode of

operation, is given below. We first exchange the (preferred)
x 2 -variable against the undesired sl-slack, and develop
tableau l6.l3b.

TABLEAU 16.13 B

THE EQUATION IS NOW SATISFIED.

NAME I II X 1 P 1 S 1 VA1.UE
-------------------------------------------------
I CODE !I -1001 1001
-------------------------------------------------
D 1 -1 I I -1.40 :::
X 2 2 II -1 I ::: I 3
D 2 -2 II I ::: I -2
-------------------------------------------------
2T , I 0.60 3 -2 6

To get back to standard form, we complete the pair of steps,

exchangingly Pl against d 2 . We develop tableau l6.l3c.

TABLEAU 16.13 C

THE DUA1. VARIABLE OF AN EQUATION MAY ATTAIN

A NEGATIVE VA1.UE.

NAME I II X 1 D 2 S 1 VA1.UE

! CODE II -2 1001

D 1 -1 II -1 ::: 0.60
-1
=
X2 I 211 I ::: I 3
~ 1 1-1001 II I I -2

2T
I' 0.60 -3 -2 12

The exchange of one dual variable against another, irrespective

of the non-negativity of either the incoming variable, or of any
other dual variable, is a normal feature of the non-convex
mode of operation.
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 489

However, since PI is the only negative-valued variable,

tableau l6.l3c represents the optimum. Dual variables of
equations are not restricted in sign.

Note that this fact affects the possibility to interpret the

corresponding row of the tableau as an inequality. For example,
the dl-row of tableau l6.l3c can be read as

-d 2 2.°. 60
The slack of this restriction is the dl-variable. The similar
reading of the PI-row,

d <-2
2 -
does not apply, as the "slack-variable" of this restriction, PI'
does not have to be non-negative, and no restriction on d 2 is
given by the PI-row.

The treatment of variables without sign restriction is on the

same lines. "Free" variab les, are initially presented with a
violated dual restriction, i.e. as "desirable" variables. Once
a free variable is in the list of basic variables it is exempt
from search operations.

Example

Maximise

Subject to 2Xl + x 2 < 10

> -2

< 1

(X 2 ~ 0, but xl not restricted in sign)

This too, is a linear programming problem, which we solve by

the quadratic programming algorithm, in tableau l6.l3d to
l6.l3h, as follows:

As in the case of equation-requirements on primal variables,

writing the "-" sign in the d l and d 2 rows of tableau l6.l3d
~ni:ially, an~ inde~d as long as d l an~ d 2 are bas~c ~aria~les,
~nd~cates an ~ntent~on, not a met requ~rement, as ~s ~n th~s
e~ampl: the case with the sl' s2 and s3-rows where the "<"
s~gn f~gures.
490 CHAPTER XVI

TABLEAU 16.13 D

SET-UP TABLEAU WITH FREE VARIABLES.

PRESENTED AS 'DESIRABLE' VARIABLES.

NAME! XI X2 PI P2 P3 ! VALUE
-------------------------------------------
D1 -2 -I = -I
D2 -I = - I
5-1 2 ~ I0
52 -I ~ 2
53 CD ~ 1

2T -1 -I -10 -2 -I

In order to drive out the negative-valued dl-variable, d l is

chosen as the first badname-variable and xl becomes the
driving variable. The rule of the smallest quotient then
indicates sl as leaving variable.

The resulting non-standard form tableau is not reported explicitly

here, instead we 8ive the next successor tableau, which is again
a standard form tableau, tableau l6.l3e.

TABLEAU 16.13 E

XI IS NOW A BASIC VARIABLE.

AND WILL STAY IN THE BASIS.

NAME! DI X2 PI P2 53 ! VALUE

XI :: I
D2 -I = -1
51 CD -2 ~ 8
3
52 I ~
P3 -I 2 -I ~ I

2T -I - I -8 -3 2

In tableau l6.13e we repeat the same procedure, d 2 is chosen as

badname, and x 2 is the next pivot column. The rule of "the smallest
(only) quotient indicates sl as pivot row.

The immediate successor tableau of tableau l6.l3e again is a

non-standard form tableau, which is not printed. The comple-
mentarity rule activates PI as incoming variable, and the
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 491

badname-variable d 2 can at that stage be eliminated. The then

resulting standard form tableau is given as tableau l6.l3f
below.

TABLEAU 16.13 F

X2 IS NOW ALSO A BASIC VARIABLE.

AND WILL STAY IN THE BASIS AS WELL.

NAME DI D2 51 P2 53 ! VALUE

XI I ::
X2 -2 ~ 8
PI -I ~ I
52 Q) ~ 3
P3 -I 2 -I :: - I
2T - 1 -8 -3 - 1 18

In tableau l6.l3f we select P3 as badname-variable, s3 becomes

the driving variable. The rule of the smallest quotient is now
applied to the s3-column, with exclusion of the quotient
in the xl~row, al:hough that quotient is actually the smallest
non-negaE~ve quot~ent.

The result of the choice of s2 with a quotient of 3 and the

rejection of xl as leaving variable is that xl now becomts
negative in taBleau l6.l3g.

TABLEAU 16.13 G

XI HAS BEEN ALLOWED TO BECOME NEGATIVE.

NAME I DI D2 51 52 P2 ! VALUE

XI -1 ~ -2
X2 2 ~ 14
PI -1 ~ I
P3
53
-I 2 c::D ~ - I
3
~

2T -1 -8 -3 21
492 CHAPTER XVI

The negative entry for xl in the value column does not

constitute a violation of any restriction, as the non-
negativity requirement does not apply to xl.

We continue with the next step. The alternative of the just

eliminated s2 - variable (P2) becomes a pivot column. There is
only one p'ositive quotient and the badname - variable P3
is eleminated.

TABLEAU 16.13 H
AN OPTIMAL AND FEASSIBLE SOLUTION.
DESPITE THE NEGATIVE VALUE OF XI.

NAME 01 02 51 S2 P3 , VALUE
- - - - - - - - - - - -.- - - - - - -'!' - - - - - - - - - - - - - - - - - - - - - - - -

XI -I ~ -2
X2 2 ~ 14
PI -I ~ I
P2 -2 -I ~ I
S3 ~ 3
------------------------------------------_.
2T 2 -14 -3 24

The solution given by tableau l6.l3h is the optimal and feasible

solution, i.e. no badname is found among those variables to
which the non-negativity requirement applies.

The one other problem to be noted in connection with mixed

systems is that it may be necessary to "turn round" an equation
(or to re-define the sign of a variable). This problem is
illustrated here in relation to the elimination of an equation-
slack in the convex mode it does not arise in the same way
in the non-convex mode·~
2 2
Example: Maximise 4x l - xl + 4x2 - x 2

subject to xl + x 2 = 1

(xl' x 2 .?: 0)
(see tableau l6.l3i)

We select the shadowprice of the strictly convex variable xl

as badname-variable. Under the rules of the convex mode of
operation, the positive quotient -1/-1 in the sl-row does not
qualify as being included in the search for the smallest
quotient. Column-updating by a negative pivot would generate
a non~standard form tableau which does not obey the recognizeable
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 493

convexity properties discussed in Sections 16.6-16.8. (The

dl/Pl cell is positive and does not permit the elimination of
d1 by PI while assigning a non-negative value to Pl·).
Tfiere are two possible ways of handling this problem.

TABLEAU 16.13 I

SET-UP TABLEAU WITH AN EQUATION

PRESENTED AS A SINGLE INEQUALITY.
NAME I! Xl X2 PI ! I VALUE
Dl
D2 "" -2
-2
II
II
-4
-4
51

2T
"II -1

-4 -4
-1 II

II
-1

We may read the tableau as if the equation was written in both

directions, e.g. although we actually write only one s -row,
our tableau interpretation is as given in tableau l6.l3j below.

TABLEAU 16.13 J

SET-UP TABLEAU WITH AN EQUATION

PRESENTED AS A DOUBLE INEQUALITY.

NAME II Xl X2 Pll P12 II VALUE

Dl
" -1
-2 II -4
D2
511
"" -1
-2
-1
-1
"
II
-4
-1
512 II <D 1
" 1
------------------------------------------------
2T II -4 -4 -1
"

The quotient for both presentations of the equation is obviously

the same, as we put the equation in the xl + X 2 S 1 form, once
the smallest quotient is found for that equation.

Upon selecting sJ as leaving variable, the tableau is written

as tableau l6.l3R below.
494 CHAPTER XVI

TABLEAU 16.13 K

RE-INTERPTETED TABLEAU WITH AN EQUATION

PRESENTED AS A SATISFIED RESTRICTION.

NAME! ! XI X2 PI2 II VALUE

DI I! -2 -I I! -4
D2 II -2 -I II -4
SI2 II <D I
----------------------------------------
I! I

2T ! I -4 -4 -I ! !

and from that point onwards the algorithm works in the usual way.

The alternative possibility is to fly through an equation and

turn it round afterwards. After exchanging x against d l , we
initially develop tableau 16.131, in which tfie equation appears
as an amply fulfilled restrictions, but before selecting a
badname in the next standard-form tableau, (in this case
immediately), we change the presentation of the tableau by

TABLEAUX 16.13 L AND 16.13 M

EQUATION-PRESENTATION EQUATION-PRESENTATION
AFTER FLYING THROUGH AFTER TURNING ROUND

NA.I DI X2 PII !VAL DI X2 F'I2 IVAL

----~-----------------------
----------------------------
XI ! -0.5 -0.5 I 2 XI I -0.5 0.5 I 2
D2 I -2 I I -4 D2 I -2 -I I -4
S II I -0.5 -I -0.5 I I S121 0.5 -0.5 I -I
---------------------------- ----------------------------
2T I -2 -4 -I 8 2T I -2 -4 8

turning it round as in tableau l6.l3m.

If the latter device is applied, sl will of necessity be

selected at some stage as badname-variable.

The same choice between "flicking round" at the stage of pivot

selection or at a later stage is also present for variables
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 495

without non-negativity restriction, at least when the convex

mode of operation is applied only to convex problems. When the
convex mode of operation is used on a non-convex problem the
qualifier may cause dual restrictions t.o be flown through.
There is also the complication that with a dual badname-
variable the normal procedure is to fly through dual
restrictions anyhow.

The code offered in Section 16.15 therefore treats the two

problems in a different way, flicking round equations at the
stage of pivot-selection and changing the presentation of
variables without non-negativity restriction just before
selecting a badname-variable. This is actually done for
equations as well but that loop is activated only if an
equation initially written in a form in which its constant
is positive.

16.14 Lower bounds in quadratic programming

The device of preliminary adjustment of the set-up tableau, as

discussed in Section 10.4 for the LP problem may also be
applied to a quadratic programming problem. The one point where
simply copying of the tableau-manipulation from Section 10.4
produces a result which is actually incorrect relates to the
value of the objective function, a point to be discussed 1n
more detail later in this section.
2 2
Example: Maximise xl - xl + 2x2 - x 2

subject to xl + x 2 > 3

5 < xl < 100, 2 ~ x 2 < 100

Tableau l6.l4a below gives the set-up tableaux for this problem,
without the lower limits, and with the LP-style adjustment

The adjusted tableau is already in optimal form, confirming

the fact that xl = 5, x 2 = 2, is the optimal and feasible
solution of the stated problem. We need, however, to analyse
the interpretation of the dual restrictions and the solution
value in more detail.

We apply the same device as used 1n Section 10.3 and put

496 CHAPTER XVI

TABLEAU 16.14 A

LP-TYPE LOWER-BOUNDS ADJUSTMENT IN QP

SET-UP TABLEAU WITHOUT ADJUSTED ACCORDING TO

THE LIMITS SECTION 10.4

NAME I Xl PI I VALUE NAME I Yl Y2 PI I VALUE

01 -2 -1 01 -2 9
02 -2 -2 02 -2 2
51 -1 -1 -3 51 -1 -I 4
----------------------~------
2T I -1 -2 3
-----------------------------
2T -1 -2 3 9
UB I 100 100 100 X UB 95 96 100 X
LB 5 2 0 X

Re-numbering the one initially specified explicit restriction

as s3' the set-up tableau now becomes the following:

TABLEAU 16.14 B

QP PROBLEM WITH EXPLICIT LOWER LIMITS.

NAME II XI X2 Yl Y2 PI P2 P3 II VALUE
----------------------------------------------------------
II -1
01 ! ! -2
02 ! I -2 II -2
03 !I -1 II
04 II -1 II
51 ! I -I II -5
52 II -1 II -2
53 II -1 -1 II -3
----------------------------------------------------------
2T ! I -1 -2 5 2 3 11

Note that the d 3 and d 4 restrictions simply say -PI + d 3 0

and -P2 + d 4 = 0, i.e. d 3 (= the shadowprice of the non-
negativity of YI) is the same variable as PI (= the dual
variable associated with the equation xl = YI + 5). As a
shadowprice of an equation PI is as such allowed to beeome
negative but this will not occur on account of the non-
negativity of d 3 ; the same argument applies to P2 and d 4 .
Substitution of d 3 .for PI and d 4 for P2 into the d l and d 2
equations results 1n:
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 497

-Zx
1
+ d
3
+ p
3
= -1 for d
3
= P
1
and

z
Furthermore, xl and X themselves may be treated as variables
which are free of non-negativity restrictions for the same
reasons as already mentioned in Section 10.3 for the LP case.
This turns the dual requirements associated with xl and X z
into equations but we can, in practice, interpret d3 and
d 4 as slack-variables.

Which gets us, as far as the restrictions is concerned, to a

direct generalisation of Section 10.4 to the QP case.

For the value of the objective function this simple procedure

gives rise to an erroneous result. The calculated result so far
is actually (once) the linear component of the ohjective
function, ~'~, i.e. xl + ZxZ = 5 + Z x Z = 9. Reference to
(16.5.5) shows that, to actually obtain ZT as the ZT entry for
a tableau in standard form, this figure needs to be doubled
and twice the value of the quadratic component of the objective
function needs to be added, i.e. the cur~ent initial value for
x = 5 and X = Z is: ZT = Z x 9 - Z x 5 - Z x ZZ = -40.
D~noting theZlower bounds vector (at which the solution is
initialised as ~*, we obtain by application of (16.5.5)

ZT* = Zw'x* + x*'Dx* (16.14.1)

The correct set-up tableau for the re-formulated problem is

given below as tableaux l6.l4c. This tableau also contains
adjustments to the rest of the ZT-row, conform the symmetry-
rules.

TABLEAU 16.14 C
CORRECT LOWER-BOUNDS ADJUSTMENT IN QP.

NAME I! Yl Y2 PI II VALUE

"
Dl -2 II 9
D2
SI
!!
! ! -1 " -2
-1
------------------------------------------
II
2
II

2T ! ! 9 2 -4 II -40
UB ! ! 95 98 100 II X

"
LB 5 2 0 II X
498 CHAPTER XVI

16.15 Commented text of a quadratic programming code

Two bits of code are offered in this section. They are: The
procedure QUAD which contains the actual quadratic programming
algorithm, a main programme which reads the data and converts
the tableau on the lines explained in section 16.14. A solution
reporting procedure which prints the solution of the specified
problem, is not listed: The procedure REPQ is simply an
adaptation of the LP reporting procedure REPO from section
10.4, the adaptation consisting in considering the more
complicated coding of a QP tableau.

The effect of the adjustment device is to make bounded variables

to be in effect restricted to the interval

lb. < x. < ub. (16.15.1)

J J - J
As the QUAD procedure could malfunction if negative upper limit
distances were generated, the main programme contains a check
on the non-negativity of the interval spanned by (16.15.1).

For the same reasons as given in section 10.4 for the L.P.
case it is inadvisable to declare variables with a "-ex'"
(e.g.-1000 000) lower limit. Variables without meaningful bounds
should be declared as variables without non-negativity
restriction.

The algorithm as offered in the QUAD procedure is a combination

of the amended convex mode of operation and the non-convex mode
of operation, the "mix" of the combination being .controlled l:>y
the parameter-variable NNEGD.

This is the acceptable number of negative dual variables.

If this number is supplied in the interval -2 < NNEGD < N+M, the
choice between the two modes of separation is determined as
follows:

At the moment of calling the convex mode of operation is

presumed but a switch to the non-convex mode of operation takes
place if either of the following two conditions is encountered:

a) the actual number of negative-valued dual variables (other

than equations-shadow prices) is found to exceed the
specified permitted number NNEGD, or
b) any sign of non-convexity is observed.

In the interest of being able to apply the non-convex mode of

operation also on re-entry of the algorithm, the code also
contains a provision for verifying the condition of its
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 499

applicability, i.e. the zero diagonal entry in the badname row.

If NNEGD is supplied negative the obvious implication is that

the non-convex mode is followed in any case. This would be
done, for example if the problem were known to be non-convex
despite the lack of manifest non-convexity. If NNEGD is supplied
as NNEDG N+M, the convex mode is followed regardless any symptoms
of the problem being in fact non-convex. If NNEGD<-2 is supplied,
the compulsory use of the non-convex mode is assumed.

Use of these 'non-choice' facilities carries a restriction

which is not strictly necessary but which was found useful
when using a modified of this algorithm in the context of a
more general kind of quadratic programming in Chapter XVIII.

If either mode is employed "perforce" in this way and a

shadowprice of a binding inequality is found to display manifest
non-convexity by way of a positive non-zero diagonal entry,
the inequality stays binding as an equation. i.e. its
shadowprice is not selected as badname-variable.

Technically this means that the problem is not actually solved.

Note, however, that supplying NNEGD as NNEGD = -1 also
effectively makes the use of the non-convex mode obligatory:
there are no tableaux in which -1 negative-valued dual
variables occur, but the restriction on not permitting a non-
convex direction does not apply in that event.

The intertwining of the two modes of operation as well as the

complications associated with the upper limit facilities do
unavoidably give use to a somewhat complicated code and
although some major elements of the 0P algorithm may still
clearly be recongized in the code-listing it becomes in
practice difficult to read and follow the code in its
entirety.

The alphanumerical labels and the comment may however go some

way towards relieving this problem.

The listings are now given, as follows:

T~XT-LISTING OF THE QUADRATIC PORGRAMMING PORCEDURE.

'PROCEDURE' QUAD<T,M,N,NEQ,NAV,NNEGD,ROWLST,COLLST,REENTRY);
'ARRAY'T; 'INTEGER' M,N,NEQ,NAV,NNEGD.REENTRY;
'INTEGER' 'ARRAY' ROWLST.COLLST;
'BEGIN' 'INTEGER' NAME,BADN.COLN,ROWN.B,BB.D,I,II,J.K.KK.R,
N OF ACT NEG D. N OF P BAS V,RR. NON SQUARE;
'REAL' ASC.QUO.PIV.COP.NUM.FANCYHIGH,PENALTY,CRIT;
'BOOLEAN' CONVEX, CONVEX MODE, FLICKED. LOOKED FOR PAIR;
500 CHAPTER XVI

'COMMENT'
QUADRATIC PROGRAMMING ALGORITHM.

THE SHORTENED TABLEAU WITHOUT UNIT VECTORS IS USED.

A MIXED SYSTEM OF EQUATIONS AND INEQUALITIES IS EXPECTED.
THE NON-NEGATIVITY RESTRICTION APPLIES TO SOME. BUT NOT TO
ALL SPECIFIED VARIABLES.
ALL SPECIFIED VARIABLES. PRIMAL AS WELL AS DUAL VARIABLES
HAVE UPPER BOUNDS.
THE UPPER BOUNDS ON THE DUAL VARIABLES CORRESPOND TO UNPUNCHED
UNIT VECTORS. REPRESENTING ARTIFICIAL FEASABILITY VARIABLES.
THE DUAL UPPER LIMITS ARE. HOWEVER. ONLY ACTIVATED
WHEN THEY RELATE TO A DUAL DRIVING VARIABLE. I.E. IF
THE DUAL VARIABLE IN QUESTION IS THE SHADOWPRICE OF A
VIOLATED PRIMAL RESTRICTION.
OTHER DUAL UPPER LIMITS ARE CALCULATED. BUT IGNORED BY
THE SEARCH OPERATIONS.
THE VALUES OF THE DUAL UPPERBOUNDS. I.E. THE PENALTY-
COEFFICIENTS FOR THE INTRODUCTION OF ARTIFICIAL VARIABLES
IN THE VECTOR OF BASIC VARIABLES. ARE SET BY THE PROGRAMME.
THE FIGURE AT WHICH THE DUAL UPPERBOUNDS ARE SET
IS ASSIGNED TO A VARIABLE. CALLED PENALTY.
PRIMAL UPPERBOUNDS FOR WHICH A ZERO IS SUPPLIED ARE
REPLACED BY A SUITABLE HIGH NUMBER. CALLED FANCYHIGH.
USER TO DECLARE PREVIOUSLY THE TABLEAU AND THE TWO NAMELISTS.
THE EXTENDED TABLEAU-MATRIX IS OF ORDER M+N+3 BY M+N+2.
AND THE NAMELISTS ARE BOTH OF ORDER M+N.
UNLESS THE RE-ENTRY MODE IS USED.
THE TABLEAU SHOULD BE PART-FILLED BEFORE ENTRY. AS FOLLOWS:
THE SYMMETRIC MATRIX D. OF ORDER N BY N IN THE TOP-LEFTHAND
CORNER. TO INDICATE THE QUADRATIC COMPONENT OF THE MAXIMAND
W TRANSPOSE X + 1/2 X TRANSPOSE D X.
THE BOTTOM PART OF THE LEFTHAND BLOCK-COLUMN SHOULD CONTAIN
THE COEFFICIENTS MATRIX A. REPRESENTING THE LINEAR RESTRIC-
TIONS A X LESS THAN OR EQUAL TO B.
THE INTERSECTION OF THE TOP BLOCK-ROW WITH
THE RIGHT-HAND N+M+I TH COLUMN SHOULD CONTAI~ MINUS THE
VECTOR W. THE LINEAR COMPONENT OF THE PREFERENCE FUNCTION.
THE BOTTOM BLOCK-ROW PART OF THAT SAME RIGHT-HAND COLUMN
SHOULD CONTAIN THE VECTOR B.
THE LEFT-HAND SIDE SUB-VECTOR OF THE BOTTOM M+N+2 NO ROW
SHOULD CONTAIN THE SPECIFIED UPPER BOUNDS.
THE SIMILAR RESERVATION FOR LOWER BOUNDS IN ROW M+N+3
IS NOT ACTUALLY USED BY THE QUAD-PROCEDURE. BUT IT IS
SUGGESTED THAT THIS ROW BE FILLED BEFORE ENTRY AS WELL.
THE FIRST NEQ ROWS OF A ARE RESERVED FOR EQUATIONS. AND
THE FIRST NAV COLUMNS ARE RESERVED FOR VARIABLES
WITHOUT SIGN-RESTRICTION.
QUADRATIC~ROGRAMMING WITH LINEAR RESTRICTIONS 501

THE NAMELISTS ARE FILLED BY THE PROGRAMME.

EXCEPT IF THE RE-ENTRY MODE IS USED.
NAME-CODES ALLOCATED BY THE PROGRAMME ARE AS FOLLOWSI
PRIMAL NAMES HAVE POSITIVE CODES. DUAL NAMES HAVE NEGATIVE
CODES.
THE SPECIFIED ACTIVITIES HAVE CODES EQUAL TO THEIR
INDICES. I.E. FROM 1 TO N.
THE SLACKS OF THE RESTRICTIONS. HAVE NAME-CODES EQUAL TO
THEIR INDICES PLUS
I
1000. I.E. FROM 1001 TO 1000+M.

DUAL VARIABLES HAVf NEGATIVE NAME-CODES. EQUAL IN ABSOLUTE

VALUE TO THE CODES, OF THE ASSOCIATED PRIMAL VARIABLES.
HENCE THE SHADOWP~~'CES OF THE SPECIFIED ECONOMIC VARIABLES
HAVE NAME-CODES FR M -1 TO -N. AND THE SHADOWPRICES OF THE
RESTRICTIONS HAV~ AMECODES FROM -1001 TO -1000 - M.
DURING CALCULATI dN:~,. THE PROGRAMME MAY GENERATE NAMECODES
WITH ENLARGEMENT$.,~ .E. WITH AN INCREASED ABSOLUTE VALUE.
UPPER BOUNDS HAVE NAME-CODES EQUAL TO THE NAMES OF THE
CORRESPONDING VARIABLES. INCREASED BY AN ENLARGEMENT OF 2000.
HENCE 2033 IS A PRIMAL VARIABLE. INDICATING THE DISTANCE
OF VARIABLE 33 FROM ITS SPECIFIED UPPER BOUND.
THE DUAL VARIABLE ASSOCIATED WITH THIS UPPER BOUND
<WHEN BINDING). IS THEN CODED AS -2033.
DUAL UPPERBOUNDS AND ARTIFICIAL FEASABILITY VARIABLES HAVE
NAMECODES WHICH ARE 2000 HIGHER IN ABSOLUTE VALUE THEN THE
CORRESPONDING ORDINARY VARIABLES.
FOR EXAMPLE. 1002 IS THE SLACK OF THE 2ND RESTRICTION. -1002
IS THE ASSOCIATED DUAL VARIABLE.
THEN 3002 IS THE ARTIFICIAL VARIABLE WHICH MAKES THE SECOND
RESTRICTION FULFILLED AT THE COST OF A PENALTY LOSS IN VALUE
OF THE OBJECTION FUNCTION. AND -3002 IS THE CORRESPONDING
UPPER BOUND ON THE DUAL VARIABE ASSOCIATED WITH THE
SECOND RESTRICTION.
ANOTHER TYPE OF ENLARGEMENT RELATES TO A MIXED SYSTEM.
FOR EQUATIONS. ANDIOR VARIABLES WITHOUT NON-NEGATIVITY
RESTRICTION THERE MAY BE ENLARGEMENTS OF 500.
THESE ENLARGEMENTS OF 500 RECORD THE FACT THAT A PARTICULAR
VARIABLE (OR EQUATION). IS PRESENTED WITH THE OPPOSITE SIGN
COMPARED TO WHAT THE USER PRESENTED AS INPUT-INFORMATION.
HENCE 50~ IS THE ~ TH VARIABLE. WHICH IS OF TYPE ABSOLUTE.
WITH THE SIGN INVERTED. I.E. MINUS X(4).
SIMILARLY. -1507 IS THE DUAL VARIABLE ASSOCIATED WITH THE
SEVENTH RESTRICTION. WHICH IS AN EQUATION. AND IT HAS BEEN
NECESSARY TO INVERT THE SIGN OF EVERY COEFFICIENT IN THAT
EQUATION. MAKING THE DUAL VARIABLE INTO MINUS THE
ORIGINALLY SPECIFIED ONE.
ONCE A VARIABLE OF TYPE ABSOLYTE. OR A DUAL VARIABLE OF AN
EQUATION IS A BASIC VARIABLE IN A STANDARD FORM TABLEAU.
THE PROGRAMME RECONVERTS SUCH A VARIABLE (IF IT CARRIES THE
ENLARGEMENT OF 500). TO THE PRESENTATION WHICH
CORRESPONDS TO THE INPUT-DATA.
ENLARGEMENTS OF AN ABSOLUTE VALUE OF 500 THEREFORE DO NOT
OCCUR IN THE OPTIMAL TABLEAU.
502 CHAPTER XVI

FANCYHIGHI=10000; PENALTYI=1000;
CONVEX 1= 'TRUE'; CONVEX MODE 1= 'TRUE';
N OF ACT NEG 0 1= 0;

, IF' REENTRY= 1 'THEN' 'BEGIN'

'COMMENT'
THE PARAMETER REENTRY SHOULD BE SUPPLIED AS ZERO FOR NORMAL
QUADRATIC PROGRAMMING. AND AS ONE. IF RE-ENTRY AFTER
ADJUSTMENT OF THE RIGHTHAND-SIDE IS ASKED FOR.

ON EXIT. THE NORMAL VALUE OF THE REENTRY-PARAMETER IS ZERO.

REENTRY=-l INDICADES THAT THE PROBLEM HAS BEEN FOUND
EMPTY IN THE NON-CONVEX MODE OP OPERATION.
(THE CONVEX MODE OF OPERATION REACTS TO EMPTYNESS BY
GENERATING AN ARTIFICIALLY FEASIBLE SOLUTION.)
OTHERWISE. WHEN A FEASIBLE SOLUTION HAS BEEN FOUND.
REENTRY=100 INDICATES THAT THE PROBLEM IS NOT CONVEX,
THIS MEANS IN GENERAL THAT THE REPORTED SOLUTION IS
A LOCAL RATHER THAN THE GLOBAL MAXIMUM.

'GOTO' START; 'END';

COMPLETE TABLEAUI
TRANSPOSE A TO MINUS A TRANSPOSEI
'FOR' 11=1 'STEP' 1 'UNTIL' M 'DO'
'FOR' JI=l 'STEP' 1 'UNTIL' N 'DO'
T(J.I+NJ 1= - HI+N.JH

TRANSPOSE MINUS W TO MINUS W TRANSPOSEI

'FOR' JI=l 'STEP' 1 'UNTIL' N 'DO' T(M+N+1.JJI=T(J.M+N+1);
TRANSPOSE B TO MINUS B TRANSPOSE I
'FOR' 1:=1 'STEP' 1 'UNTIL' M 'DO'
T(M+N+1.N+Il 1= -T(N+I.M+N+11;

FILL BOTTOM RIGHT ZERO BLOCKI

'FOR' 11= N+1 'STEP' 1 'UNTIL' N+M 'DO'
'FOR' J:= N+1 'STEP' 1 'UNTIL' N+M 'DO' T(I.JJI=O;
SET FANCY UPPER BOUNDSI
'FOR' JI"l 'STEP' 1 'UNTIL' N 'DO' 'IF' T(M+N+2.Jl=0
'THEN' T(M+N+2.J11=FANCYHIGH;
'FOR' 11=1 'STEP' 1 'UNTIL' N 'DO' T( I.N+M+2JI=PENALTY
-T( I.N+M+1];
'FOR' JI=N+1 'STEP' 1 'UNTIL' M+N 'DO' 'IF' T(N+M+2.Jl=0
'THEN' T(M+N+2.J11=PENALTY;

FILL DUMMY UPPER BOUNDS WITH ZEROSI

'FOR' II=N+1 'STEP' 1 'UNTIL' M+N+1 'DO' T(I.M+N+2Jp,0;
ATTEND DEGENERACY:
'FOR' 11=1 'STEP' 1 'UNTIL' M+N 'DO' 'IF' T(I.M+N+"=O
'THEN' 'BEGIN'
NUMI =1;
'FOR' J:=l 'STEP' 1 'UNTIL' M+N 'DO' 'IF' T(I.Jl < 0
'THEN' NUM:=NUM-T(I.Jl;
'IF' I < NAV+1 'THEN' NUMlc-NUM;
'IF' I>N 'AND' I<N+NEQ+1 'THEN' NUMI=-NUM;
TU.M+N+1J := 0.0000000001 * NUM; 'END';
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 503

FILL NAMELlaTSI
'FOR' JI=l 'STEP' 'UNTIL' N 'DO' 'BEGIN'
COLLST[JJI=J; ROWLST[JJI=-J; 'END';
'FOR' 11=1 'STEP' 1 'UNTIL' M 'DO' 'BEGIN'
COLLST[N+IJI=-1000-I; ROWLST[N+IJI=1000+I; 'END' ;

START:
STANDARD FORM:
BADNI =0;

ATTEND MIXED SYSTEMI

ATTEND ABS VI
'FOR' 11=1 'STEP' 1 'UNTIL' NAV 'DO'
'IF' CCROWL5T[IJ < 0 'AND' -ROWLST[IJ < NAV+1)
'OR' C-ROWLST[IJ > 500 'AND' -ROWLST[IJ < SOl+NAV»
'AND' 'NOT' T[I.N+M+1l < 0 'THEN' 'BEGIN'
'IF' T[I.N+M+1J =
0 'THEN' T[I.N+M+1J := 0.00000001;

TURN ABS VAR ROUNDI

'FOR' J:=l 'STEP' 1 'UNTIL' N+M+1 'DO' 'BEGIN'
TCI.JJ := -TCI.J); TCJ.IJ:= -TCJ.IH 'END';
'IF' COLLST[IJ < 500 'THEN' COLLSTCIJI=COLLSTCIJ+SOO
'ELSE' COLLSTCIJI=COLLSTCIJ-SOO;
ROWLSTCIJ := -COLLST[IJ; 'END';

PUT ABS VAR RIGHT AGAINI

'FOR' 11=1 'STEP' 1 'UNTIL' NAV 'DO'
'IF' ROWLST[IJ=SOO+I 'THEN' 'BEGIN'
ROWLSTCIl:=ROWLSTCIJ-SOO;
COLLSTCIJI=-ROWLST[IJ;
T[I.N+M+2J 1= FANCYHIGH+T[I.N+M+1J;
'FOR' JI=l 'STEP' 1 'UNTIL' N+M+1 'DO' 'BEGIN'
TCI.JlI=-T[I.JH T[J.Ill=-TCJ.IH 'END'; 'END';

ATTEND EQUATIONI
'FOR' II=N+1 'STEP' 'UNTIL' N+NEQ 'DO'
'IF' CCROWLSTCIJ > 1000 'AND' ROWLSTCJ) < 1001 + NEQ)
'OR' CROWLSTUJ > 1500 'AND' ROWLST[J) < lS01+NEQ»
'AND' 'NOT' T[I.N+M+1J < 0 'THEN' 'BEGIN'
'IF' T[I.N+M+1J = 0 'THEN' T[I.N+M+1J 1= 0.00000001;

TURN EQUATION ROUNDI

'FOR' JI=l 'STEP' 1 'UNTIL' N+M+1 'DO' 'BEGIN'
TC I • J J : = - T [ I • J); T [ J. I J I = - TC J. I); , END' ;
'IF' ROWLST[ll < 1500 'THEN' ROWLST[Ill= ROWLST[IJ+SOO
'ELSE' ROWLST[IJ := ROWLST[IJ-SOO;
COLLSTCIJI=-ROWLSTCI); 'END';

PUT EQUATION RIGHT AGAINI

'FOR' II=N+1 'STEP' 1 'UNTIL' N+NEQ 'DO'
'IF' COLLST[IJ=lS00+I-N 'THEN' 'BEGIN'
COLLST[IJ:=COLLST[IJ-SOO;
ROWLST[IJ:=-COLLST[IJ;
T[I.N+M+2J := PENALTY+T[I.N+M+1J;
'FOR' J:=l 'STEP' 1 'UNTIL' N+M+1 'DO' 'BEGIN'
T[I.JlI=-T[I.J); T[J.IJI=-T[J.IH 'END'; 'END';
504 CHAPTER XVI

SELECT BADNAME:
PREPARE INV OF MODEl
LOOKED FOR PAIR 1= 'FALSE';
N OF ACT NEG 0 1= 0; N OF P BAS V := 0;

INITIATE ASCENT LOWI

ASC:=-IOOOOOOOOOOOOOOO;
'FOR' 1:=1 'STEP' 1 'UNTIL' M+N 'DO' 'BEGIN'

'IF' TCI.I] > 0 'THEN' 'BEGIN'

'IF' TCI.I] < 0.0000001 'THEN' TCI.I] 1= 0;
'IF' TCI.Il > 0 'THEN' CONVEX MODE := 'FALSE';
'IF' NNEGD> N+M 'THEN' CONVEX MODE 1= 'TRUE'; 'END';

COUNTI
'IF' I > NAV 'AND' 'NOT'
(I>N 'AND' I<N+NEQ+I) 'THEN' 'BEGIN'
'IF' ROWLST[J) < 0 'AND' TCI.N+M+I] <
'THEN' N OF ACT NEG D 1= N OF ACT NEG D + I; 'END';

'IF' ROWLSTCJ] > 0

'THEN' N OF P BAS V := N OF P BAS V + I;

ABS VAR STAYS:

'IF' I < NAV+I 'AND' (ROWLSTCI] = I 'OR' ROWLSTCIl=500+I)
'THEN' 'GOTO' END OF BADNAME SELECTION LOOP;

DUAL VAR EQUATION STAYS:

'IF' I > N 'AND' I < N+NEQ+I
'AND' (-ROWLSTCIl = 1000+I-N 'OR' -ROWLSTCJ] 1500+I-N)
'THEN' 'GOTO' END OF BADNAME SELECTION LOOP;

CRIT := I;
'IF' ROWLSTCI) < 0 'THEN' 'BEGIN'

REFUSE NON CONVEX SLACKS WHEN MODE FORCED:

'IF' -ROWLSTCIl > 1000+NEQ 'AND' -ROWLSTCI)<IOOI+M
'AND' (NNEGD>N+M 'OR' NNEGD<-l> 'THEN' 'BEGIN'
'IF' TCI.I) > 0
'THEN' 'GOTO' END OF BADNAME SELECTION LOOP; 'END';

'FOR' II:=NAV+l 'STEP' 'UNTIL' N+M 'DO'

'IF' ROWLSTCII) > 0 'AND' TCII.N+M+I] < 0
'AND' TCIl.I) < 0 'THEN' CRIT:=CRIT-TClI.IH 'END';

'IF' TCI.N+M+l] < 0 'AND' CRIT > ASC 'THEN' 'BEGIN'

BADN:=RO.WLSTCJ]; Bp=r; ASC:=CRIT;
'END' ;

END OF BADNAME SELECTION LOOP: 'END';

'IF' TCB.B) > 0 'THEN' CONVEX:= 'FALSE';

'IF' NNEGD > N+M 'THEN' CONVEX MODE := 'TRUE';
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 505

'IF' BADN=O 'THEN' 'GoTo' END OF QUAD;

'IF' 'NOT' CONVEX 'DR' N OF ACT NEG D > NNEGD

'THEN' CONVEX MODE := 'FALSE';
'IF' NNEGD > N+M 'THEN' CONVEX MODE := 'TRUE';

'IF' CONVEX MODE 'DR' LOOKED FOR PAIR 'THEN' 'GoTo' PHASE II;
'IF' BADN>O 'AND' TCB.B)=O 'THEN' 'GoTo' PHASE Il

'FOR' I:=NAV+I 'STEP' 1 'UNTIL' N.

N+NEQ 'STEP' 1 'UNTIL' N+M 'DO'
'IF' TCI.M+N+I) < 0 'AND' RoWLSTCIl > 0 'AND' TCI.I)=O
'THEN' 'GoTo' PHASE I SEARCH;
'GoTo' PHASE II;

PHASE I SEARCH:
B:=O; BADN 1= 0; ASC := 0;
'FOR' I 1= NAV+I 'STEP' 1 'UNTIL' N.
N+ 1 'STEP' 1 'UNTIL' N+M 'DO'
'IF' RoWLSTCI) > 0 'AND' TCI.N+M+IJ < 0 'THEN' 'BEGIN'
'IF' -TCI.N+M+I) > ASC 'THEN' 'BEGIN'
B:=J; ASC := -TCI.N+M+I); BADN:=RoWLSTCI); 'END'; 'END';

PHASE I:
CoLN :=0; K:=O; ASC:=O;
'FOR' J:=I 'STEP' 1 'UNTIL' N+M 'DO'
'IF' CoLLSTCJ) > 0
'AND' CTCB.J) < 0 'DR' J < NAV+I) 'THEN' 'BEGIN'

'IF' J<NAV+I 'THEN' 'BEGIN'

'IF' 1000*ABSCHB.J) > ASC 'THEN' 'BEGIN'
K:=J; CoLN := CoLLSTCK); ASCI=IOOO*ABSCTCB.K);
'END'; 'END';

'IF' -TCB.J»O 'THEN' 'BEGIN'

'IF' TCJ.N+M+I) < 0 'THEN' 'BEGIN'

'IF' 10*C-TCB.J)-TCJ.N+M+I) > ASC 'THEN' 'BEGIN'
ASC := -IO*CTCB.J)+TCJ.N+M+I);
K:=J; CoLN:=CoLLSTCJH 'END'; 'END'

'ELSE' 'BEGIN'
'IF' TC J.N+M+I )<0 ,0000001
'THEN' T[J.N+M+I):=O.OOOOOOI;
'IF' -TCB.J)/T[J.N+M+I) > ASC 'THEN' 'BEGIN'
K:=J; CoLN:=CoLLSTCK);
ASC:=-TCB.J)/TCJ.N+M+I); 'END';
• END'; • END';
'END' ;

'IF' CoLN = 0 'THEN' 'BEGIN'

'COMMENT'
SIGNALLING OF AN EMPTY PROBLEM IN THE NON-CONVEX MODE;
REENTRY: = -I;
NEWLINEC 1 );
WRITETEXTC'C'RoW%')');
WRITEC30.FoRMATC'C'S-NDDDD')').BADN);
WRITETEXTC'('%%~oUND%IMPoSSIBLE%To%SATISFY')');
'GoTo' FINAL END OF QUAD; 'END';
506 CHAPTER XVI

ATTEND FREE VAR IN NON CONVEX MODE:

'IF' (CoLN>O 'AND' K<NAV+1 'AND' T(B.KJ>O)
, THEN' , BEG 1 N '
'FOR' JI=l 'STEP' 1 'UNTIL' N+M 'DO' T(K.JJp'-T(K.JJl
NUM 1= T(K.N+M+1J; T(K.N+M+1J:=T(K.N+M+2J;
T(K.N+M+2J := NUM;
'FOR' 11=1 'STEP' 1 'UNTIL' N+M+1 'DO'T(I.KJI=-T(I.KJ;
'IF' COLN < 500 'THEN' COLN:=COLN+500
'ELSE' COLNI=COLN-500;
COLLST( KJ: =CoLN; ROWLST( KJ I =-COLN; 'END';

'GOTO' INITIATE SEARCH FOR PIVOT ROW;

PHASE 11:
D:=KI=B; COLNI=-BADN;

NON STANDARD FORMI

'IF' ABS(ROWN) < 2000 'THEN' FLICKED 1= 'FALSE';

INITIATE SEARCH FOR PIVOT ROWI

QUO:=1000000000000; ROWNI=O;

CHECK ON ROUNDING NON CONVEXITY:

'IF' T(B.KJ > 0 'THEN' 'BEGIN'
'IF' T(B.KJ < 0.00000001 'THEN' T(B.KJ 1= 0; 'END';

'IF' 'NOT' CONVEX 'OR' N OF ACT NEG D > NNEGD

'THEN' CONVEX MODE := 'FALSE';
'IF' NNEGD> N+M 'THEN' CONVEX MODE := 'TRUE';
'IF' N OF P BAS V = M 'AND' 'NOT' CONVEX MODE
'AND' BADN > 0 'THEN' 'GOTO' SEEK SMALLEST QUOTIENT;
TRY THE BADNAME ROW:
BUT NOT IN NON CONVEX MODE:
'IF' T(B.KJ < -0.0000000001 'THEN' 'BEGIN'
'COMMENT'
THE BADNAME-ROW HAS PREFERENCE. ACCEPT A NEGATIVE PIVOT;
R:=B; ROWNI=BADN; QUO:=TCB.M+N+1J/TCB.KJ; 'END';

SEEK SMALLEST QUOTIENT:

'FOR' 1:=1 'STEP' 1 'UNTIL' M+N 'DO'
'IF' ABS(T(I.KJ) > 0.000000001 'THEN' 'BEGIN'

CONSIDER UPPERB ON DUAL VARI

'IF' ROWLST(IJ < 0 'AND' ROWLST(IJ = -BADN
'AND' T(I.XJ < 0 'THEN' 'GOTO' TRY UPPER SOUND;

KEEP SHADOWPRICE EQUATION BASIC VARI

'IF' (ROWLST(IJ < -1000 'AND' ROWLST(IJ > -1000-NEQ-1)
'OR' (ROWLSTCIJ < -1500 'AND' ROWLSTCIJ > -1500-NEQ-l)
'THEN' 'GOTo' DONE;

CONSIDER EQUATION SLACK EVEN IF NEGI

'IF' «ROWLST(IJ > 1000 'AND' ROWLST(IJ<1001+NEQ)
'DR' (RoWLSTCIJ > 1500 'AND' ROWLST(IJ < 1501+NEQ»
'AND' T(I.KJ<O 'AND' T(I.N+M+1J<0
'THEN' 'GOTo' TRY VALUE COL;
 QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 507

INVESTIGATE UPPERB PRIMAL VARI

'IF' RoWLSTCIJ > 0 'AND' RoWLSTCI1 < N+I
'AND' TCI.K1 < 0 'THEN' 'GOTO' TRY UPPER BOUND;

KEEP ABS VARIABLE BASIC:

'IF' (RoWLSTCI1 > 0 'AND' RoWLSTCIJ < NAV+I)
'OR' (RoWLSTC I 1 > 500 'AND' ROWLSTC I 1 < NAV+501)
'THEN' 'GOTO' DONE;

ATTEND SPECIAL FEATURES OF NON CONVEX MODEl

'IF' N OF P BAS V = M 'AND' 'NOT' CONVEX MODE
'AND' BADN> 0 'THEN' 'BEGIN'
'IF' ROWLSTC 11<0 'THEN' 'GOTD' DONE;
'IF' ROWLSTCIJ=BADN 'AND' TCI.K1<0
'AND' 'NOT' TCI.N+M+I1>0 'THEN' 'GOTO' TRY VALUE COL;
'END';

REFUSE QUOTIENT WITH THE WRONG SIGN:

'IF' TCI.K1 < 0 'OR' TCI.M+N+I1 < 0 'THEN' 'GOTO' DONE;
TRY VALUE COLI
'IF' TCI.M+N+l1/TCI.KJ < QUO 'THEN' 'BEGIN'

'IF' ROWLSTCI1<0 'THEN' 'BEGIN'

DO NEED TO ELEMINATE:
'IF' BADN < 0 'THEN' 'GOTO' DONE;

CHECK SUBSPACE CONVEXITY:

WHEN INCOMING VAR IS DRIVING VAR:
'IF' COLN=-BADN 'AND' TCI.I1 > 0 'THEN' 'GOTO' DONE;

FOR DRIVING VAR IN NON STAND F:

'IF' 'NOT' COLN=-BADN
'AND' TCD.I1-TCD.K1*TCI.I1/TCI.K1>0
'THEN' 'GOTO' DONE; 'END';
RI=I; ROWN:=ROWLSTCIJ; QUO:=T[I.M+N+l1/TCI.K1; 'END';
'GOTO' DONE;
TRY UPPER BOUNDI
'IF' ABS<-TCI.M+N+21/TCI.K1) < ABS(QUO) 'THEN' 'BEGIN'
R:=I; QUOI=-TCI.M+N+21/TCI.K1;
NAME:=ABS<ROWLSTCI1);
'IF' NAME < 2000 'THEN' ROWNI=NAME+2000
'ELSE' ROWN:=NAME-2000;
'IF' ROWLSTCI1 < 0 'THEN' ROWN:=-ROWN; 'END';

DONE: , END';

'IF' T[B.B1=0 'AND' 'NOT' CONVEX MODE

'AND' BADN > 0 'AND' COLLSTCBJ=-BADN 'THEN' 'BEGIN'
'IF' TCB.K1 > -0.0000000001 'THEN' 'GOTO' FLY THROUGH;
'IF' R=O 'THEN' 'GOTO' ACCEPT BV;
'IF' 'NOT' TCB.N+M+I1/T[B.K1 < QUO 'THEN' 'BEGIN'
'IF' N OF P BAS V = M 'THEN' 'GOTO' NOT SMALLER;
'GOTO' ACCEPT BV; 'END';
'IF' ABS<TCR.K1) < 0.01 'THEN' 'GoTO' ACCEPT BV;
'IF' -QUO*TCN+M+I.K1 > -<TCB.N+M+I)/TCB.K1)*TCN+M+I.B1
'THEN' 'GOTO' FLY THROUGH;
ACCEPT BV:
ROWN:=BADN; R:=B; QUO:=TCB.N+M+I1/T[B.KJ;
508 CHAPTER XVI

NOT SMALLER:
FLY THROUGH:
, IF' N OF P BAS V > M ' THEN' 'BEG IN'
'IF' 'NOT' ROWN=BADN 'THEN' 'BEGIN'
LOOKED FOR PAIR :- 'TRUE'; 'GOTO' PHASE IU 'END';
NON SQUARE :- N OF P BAS V - MJ
N OF P BAS V := MJ 'END'J 'END'J

ATTEND UPPER BOUND COLUMN VAR:

BIND SPECIFIED VAR:

'IF' COLN > 0 'AND' COLN < N+I 'THEN' 'GOTO' UPPERB ALLOWEDJ

BIND UPPER LIMIT DISTANCE:

'IF' COLN > 2000 'AND' COLN < 2001+N
'THEN' 'GOTO' UPPERB ALLOWEDJ

BIND DUAL DRIVING VARIABLE:

'IF' CCOLN<O 'AND' COLN"-BADN) 'THEN' 'BEGIN'
'IF' ABSCCOLN)<N+I 'THEN' TCN+M+2.Kl:=PENALTYJ
'GOTO' UPPERB ALLOWEDJ 'END'J
UPPERBOUND NOT ALLOWED:
'GOTO' CHECK FOR BOUNDEDNESSJ
UPPERB ALLOWED:

'IF' ROWN=O 'OR' QUO> TCM+N+2.Kl 'THEN' 'BEGIN'

START OF VECTORS ONLY UPDATING LOOP:
MARK NO PIVOT ROW:
R := 0;

QUO := TCM+N+2.KlJ

ASSEMBLE ROWN:
'IF' COLN < 0 'THEN' NAME := -COLN 'ELSE' NAME := COLNJ
'IF' NAME < 2000 'THEN' ROWN := NAME+2000
'ELSE' ROWN:=NAME-2000;
'IF' COLN < 0 'THEN' ROWN := -ROWNJ
COLLSTCKl:=ROWN;

UPDATE VECTORS:
'FOR' 1:"1 'STEP' 1 'UNTIL' M+N+I 'DO' 'BEGIN'
TCI.M+N+Il:=TCI.M+N+Il-TCI.Kl*QUOJ
TCI.M+N+2ll=TCI.M+N+2l+TCI.Kl*QUO;
TCI.Kl := -TCI.KlJ 'END';

'IF' CABSCNAME) = ABSCBADN»

'OR' C'NOT' CONVEX MODE 'AND' N OF P BAS V M
'AND' BADN>O 'AND' ROWN>O) 'THEN' 'GOTO'
STANDARD FORM DOUBLE STEP;
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 509

ATTEND COMPLEMENTI
'IF' ROWLSTCK1 .. -COLN 'THEN' 'BEGIN'
ATTEND COMPLEMENTARY ROWI
ROWLSTCK11=-ROWN;
'FOR' J:=I 'STEP' I 'UNTIL' N+M 'DO' TCK.J11--TCK.JH
COPI=TCK.M+N+Il; TCK.M+N+I11=TCK.M+N+21;
TCK.M+N+211"COP; 'END' 'ELSE' 'BEGI N'

ATTEND COMPLEMENTARY COLUMNI

COLLSTCRR1:=-ROWN;
'FOR' 11=1 'STEP' I 'UNTIL' M+N+I 'DO'
TCI.RR11--TCI.RRJJ
ATTEND 2T ROW OF THE DUAL OF THE UPPERBI
'IF' COLN < 0 'THEN' TCM+N+I.RR11=TCM+N+I.RR1+TCM+N+2.Kl
'ELSE' TCM+N+I.RR11=TCM+N+I.RR1-TCM+!II+2.KH 'END';

STANDARD FORM DOUBLE STEP:

'IF' (ABS(NAME) = ABS(BADN»
'OR' ('NOT' CONVEX MODE 'AND' N OF P BAS V .. M
'AND' BADN>O 'AND' ROWN>O)
'THEN' 'BEG IN'
'FOR' J:o.I 'STEP' 'UNTIL' N+M+I 'DO' TCK.J11=-TCK.Jl;

'IF' ROWN < 0 'THEN'

TCM+N+I.M+N+IJI=TCM+N+I.M+N+Il+TCM+N+I.KJ.QUO
'ELSE' TCM+N+I.M+N+IJI=TCM+N+I.M+N+Il-
TCM+N+I.Kl*QUO;

'FOR' 11=1 'STEP' 'UNTIL' M+N 'DO'

'IF' ROWLSTCIl < 0 'THEN' TCM+N+I.I11=TCI.M+N+11
'ELSE' TCM+N+I.I11=-TCI.M+N+11;
ROWLSTCKl 1= -ROWN;

'IF' 'NOT' ABS(ABS(BADN)-ABS(ROWN) )=2000 'THEN' 'BEGIN'

'COMMENT'
PAIR IN PHASE I IN NON-CONVEX MODE;
CHECK ON FLYING THROUGHI
'IF' 'NOT' TCK.N+M+Il < 0 'THEN' 'GOTO' STANDARD FORM;
'GOTO' PHASE 1J 'END';

'GOTO' STANDARD FORM) 'END';

ADJ COLLST AND PERM UPPERBOUND COLUMNS I

COLLST[Bl 1= ROWN; COLLST.CKl 1" -ROWN;
'FOR' 11=1 'STEP' I 'UNTIL' M+N+2 'DO' 'BEGIN'
COP:=TCI.BlJ TCI.B11=TCI.KlJ TCI.KlI=COP; 'END';
COLN:=-ROWNJ
'GOTO' CHECK FOR STATUS;
END OF VECTORS ONLY UPDATING LOOPI 'END';
S10 CHAPTER XVI

CHECK FOR BOUNDEDNESS:

'IF' ROWN-O 'THEN' 'GOTO' UNBOUNDED;
SAVE INDICES:
KK := K; RR :- R; BB:= B;
'IF' 'NOT' CONVEX MODE 'AND' N OF P BAS V M
'AND' BADN>O 'THEN' B:=R;
ATTEND NON-NEGATIVITY EQ SL:
'IF' 'NOT' ROWN=BADN 'AND'
CCROWN>IOOO 'AND' ROWN<IOOI+NEQ)
'OR' CROWN> I 500 'AND' ROWN<ISOI+NEQ»
'THEN' 'BEGI N'
'IF' TCR.N+M+Il<O 'AND' TCR.Kl<O 'THEN' 'BEGIN'
ADJUST CODE:
'IF' ROWN<lS00 'THEN' ROWLSTCR1:=ROWN:~ROWN+SOO
'ELSE' ROWLSTCR1:-ROWN:-ROWN-SOO;
COLLSTCK1:=-ROWN;
FLICK ROUND:
'FOR' 11"1 'STEP' I 'UNTIL' N+M+I 'DO' 'BEGIN'
TCI.Rl: .. -TCI.RH TCR.Il: .. -TCR.IH 'END'; 'END'; 'END';
PERMUTE COLUMNS:
'IF' C"NOT' CONVEX MODE 'AND' N OF P BAS V - M
'AND' BADN > 0>
'OR' CROWNIBADN 'AND' ABSCROWN>IABSCBADN)+2000)
'THEN' 'BEGIN'
'FOR' 11=1 'STEP' 1 'UNTIL' M+N+2 'DO' 'BEGIN'
COP:=TC I.K]; TC I.KH-T[ I.RH TC I.RlI-COP; 'END';
COLLSTCK1:-COLLSTCRlJ K: .. R; 'END';
ADJUST COLLST:
COLLSTCK1:-ROWN;
ATTEND UPPERBOUNDS OF PIVOTAL PAIR:
'IF' ROWNIROWLSTCRJ
'THEN' 'GOTO' PRESENT NEWLY BINDING UPPERBOUND AS ROW;
NUM:-TCR.M+N+IJ+TCR.M+N+21;
'IF' ROWN>2000 'THEN' NUM:-TCR.N+M+21-TCR.N+M+l1;
TCR.M+N+21:-TCM+N+2.Kl-QUO;
TCM+N+2.Kl:=NUMJ
PRESENT NEWLY BINDING UPPER BOUND AS ROW:
'IF' ROWN I ROWLSTCRl 'THEN' 'BEGIN'
FLICKED :- 'TRUE';
ROWLSTCR1:=ROWN;
'FOR' J:-I 'STEP' I 'UNTIL' M+N 'DO' TCR.Jl:"'-TCR.Jll
NUM:~TCR.M+N+l1; TCR.M+N+l1:-TtR.M+N+21J TCR.M+N+21:=NUM;

GENERATE COMP UL COLUMN:

'I F' ABSCROWN) IABSCBADN )+20 0 0 'THEN' 'BEGIN'
'FOR' 11 .. 1 'STEP' 1 'UNTIL' M+N+I 'DO' Tt I.KK1: .. ~T[ I.KKH
COLLSTC KK1:=-ROWN;
'IF' ROWN<O 'THEN' TCN+M+I.KK1:-TtN+M+I.KK1+
TCR.N+M+Il+TCR.N+M+21
'ELSE' TCN+M+I.KK1:~TCN+M+I.KK1-
TC R.N+M+ Il-TCR.N+M+2H 'END' J
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 511

STILL ATTEND UPPERBOUNDS OF PIVOTAL PAIR:

NUM:=T(R.M+N+11+TCR.M+N+21;
T(R.M+N+21:=TCM+N+2.KJ-QUO;
T(M+N+2.KJ I=NUM;
'END';

PERMUTE ROWS AND ADJUST ROWLST:

'IF' 'NOT' CONVEX MODE 'AND' N OF P BAS V = M
'AND' BADN>O 'AND' COLN>O 'THEN' B:=KK;
'FOR' J:=I 'STEP' I 'UNTIL' M+N+2 'DO' 'BEGIN'
COP:=TeR.JH TCR.JJ:=TCB.JH TCB.JJ:=COP; 'END';
ROWLSTCRJ:=ROWLSTCBJ; ROWLSTCBJ:=COLN;

UPDATE:
PI V: =TeB.KJ;

CLEAN AND UPDATE PIVOT ROW:

'FOR' J:=I 'STEP' I 'UNTIL' M+N+I 'DO' 'BEGIN'
'IF' J < M+N+I 'AND' ABS(TCB.JJ) < 0.0000000001
'THEN' T(B.JJ:=O;
'IF' T(B.JJ I 0 'THEN' TCB.JJ:=TCB.JJ/PIV; 'END';
ADJUST COEFFICIENTS MATRIX:
'FOR' JI=I 'STEP' I 'UNTIL' K-I.
K+I 'STEP' I 'UNTIL' M+N+I 'DO'
'IF' TCB.JJ I 0 'THEN'
'FOR' 1:=1 'STEP' I 'UNTIL' B-1.
B+I 'STEP' I 'UNTIL' M+N+I 'DO'
'IF' TeI.KJ 10 'THEN' TCI.JJ:=TeI.J1-TCB.JJ*TCI.KlJ

'FOR' 11=1 'STEP' I 'UNTIL' B-I.B+I 'STEP' I 'UNTIL' M+N+I

'DO' 'BEGIN'
TCI.M+N+2J:=T(I.M+N+2J+T(B.M+N+IJ*TCI.K1;
T( I.KJI=-TC I.l<l/PIV; 'END';
TCB.K1:=I/PIV;

REPRESENT NEW UPPERBO BASIC VAR NORMALLY:

'IF' ABS<COLN) < 1000
'THEN' 'GOTO' EXTRA STEP BACK TO STANDARD Fl

'IF' COLN > 2000 'OR' COLN < -3000

'OR' (COLN < -2000 'AND' 'NOT' FLICKED) 'THEN' 'BEGIN'
'COMMENT'
SPECIFIED VARIABLES. I.E. PRIMAL NAMECODES ENTERING AS
2000 + J. AND DUAL UPPERBOUNDS ENTERING AS -3000 -I.
AND UPPER LIMITS ON DUAL SLACKS ENTERING AS -2000-J.
FOLLOWING THE ELEMINATION OF ARTIFICIAL NEGATIVES OF
ORDINARY VARIABLES (PERMITTED ONLY IN THE REENTRY MODE).
ARE TO BE RE-DEFINED.
THEY THEN BECOME ORDINARY VARIABLES. CODED AS J FOR THE
J TH PRIMAL VARIABLE AND -1000 -I FOR THE I TH DUAL
VARIABLE. OR -J FOR THE J TH DUAL SLACK,

'IF' COLN > 0 'THEN' NAMEI=COLN-2000

'ELSE' NAME:=COLN+2000;
ROWLSTC B1: =NAME;
'FOR' 11=1 'STEP' I 'UNTIL' M+N 'DO' TCB.I1:=-TCB.I1J
NUM:=TCB.M+N+IH T(B.M+N+I1:=TCB.M+N+21; TCB.M+N+21:=NUMJ
512 CHAPTER XVI

'IF' COLN - -COLLSTCBl 'THEN' 'BEGIN'

'FOR' II-I 'STEP' 1 'UNTIL'M+N+l 'DO' TCI.Bll--T(I.Bl;
COLLST[Bll--NAME;
'IF' NAME> 0 'THEN' TCM+N+l.Bl I" TCM+N+l.Bl
-T[B.M+N+Il-TtB.M+N+2l 'ELSE'
Tt M+N+ I. Bl I-TtM+N+ 1. Bl+Tt B.M+N+ 1 l+Tt B.M+N+2lJ 'END';
END OF INCOMING VARIABLE LI~IT RENORMALISATIONI 'END';

EXTRA STEP BACK TO STANDARD FI

UPPER LIMIT ELEMINATION TO ST FI
'IF' ABSCROWN) - ABSCBADN)+2000 'THEN' 'BEGIN'
'FOR' JI-I 'STEP' 1 'UNTIL' N+M+I 'DO' TCR.Jll .. -TtR.JlJ
, IF' BADN > 0 ' THEN '
TtM+N+I.M+N+lll-TCM+N+l.M+N+ll-PENALTY.TtR.M+N+ll
'ELSE' TCM+N+l.M+N+lll-TtM+N+l.M+N+ll +
T[M+N+2.Kl.TtR.M+N+ll;

'FOR' Jp'l 'STEP' 1 'UNTIL' M+N 'DO'

'IF' ROWLST[Jl < 0 'THEN' TCM+N+l.JlI-TtJ.M+N+l l 'ELSE'
TCM+N+l.Jll--TtJ.M+N+ll;
'IF' BADN > 0 'THEN' ROWLSTtRl I- BADN+2000
'ELSE' ROWLSTtRl I- BADN-2000;
'GOTO' STANDARD FORM; 'END';

CHECK FOR STATUSI

ATTEND PAIR IN NON CONVEX MODEl

'IF' 'NOT' CONVEX MODE 'AND' N OF P BAS V - M 'THEN' 'BEGIN'
'IF' ElADN> 0 'AND' COLN<O 'THEN' 'BEGIN'
'IF' COLN--BADN 'THEN' 'GOTO' STARTJ
'IF' TtBB.N+M+ll > 0 'THEN' 'GOTO' STANDARD FORM;
N OF P BAS V I- N OF P 8AS V + NON SQUARE;
BI-88; 'GOTO' PHASE H 'END';
'IF' BADN > 0 'AND' COLN> 0 'THEN' 'BEGIN'
'IF' R-O 'THEN' RRI-K;
'IF' R-O 'THEN' KKI-K;
81-RI-RR; KI-KK;
NAMEI-ROWN; ROWN:--COLN; COLNI--NAME;
'GOTO' ADJUST COLLST; 'END'; 'END';
'IF' 8ADN-ROWN 'THEN' 'GOTO' START;
PREPARE NON STANDARD FORM STEPI
'IF' R-O 'THEN' KKI-K;
'IF' R-O 'THEN' R:=B;
K:-KK; 81 a R; COLNI=-ROWN;
'GOTO' NON STANDARD FORM;

UN80UNDED:
NEWLINEC 1)J
WRITETEXTC'C'UNBOUNDED%COLUMN')');
WRITEC30.FORMATC'C'S-NDDDDDDDDD')').COLN);
NEWLINEC 1 )J

END OF QUADI

'IF' 'NOT' CONVEX 'THEN' REENTRY: - 100;

FINAL END OF QUAD: 'END';
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 513

TEXT-LISTING OF THE CONTROLLING MAIN PROGRAMME.

'BEGIN' 'INTEGER' M.N.NAV.NEQ.REENTRY.I.J.NNEGD.

'PROCEDURE' QUAD(T.M.N.NEQ.NAV.NNEGD.ROWL.COLL.REENTRY>.
'ARRAY' T. 'INTEGER' M.N.NEQ.NAV.NNEGD.REENTRY.
'INTEGER' 'ARRAY' ROWL.COLLl 'ALGOL'.

'PROCEDURE' MATI(MATR.M.N.SR.SC)l
'ARRAY' MATR. 'INTEGER' M.N.SR.SC' 'ALGOL';
'PROCEDURE' TABO(MATR.M.N.SR.SC.RH.ER.ROWL.COLL>l
'ARRAY' MATR. 'INTEGER' M.N.SR.SC.RH.ER.
'INTEGER' 'ARRAY' ROWL.COLL. 'ALGOL';
'PROCEDURE' REPQ(T.M.N.NEQ.NAV.ROWL.COLL);
'ARRAY' TJ 'INTEGER' M.N.NEQ.NAV;
'INTEGER' 'ARRAY' ROWL.COLL; 'ALGOL';
'COMMENT'
SIMPLEX ALGORITHM FOR QUADRATIC PROGRAMMING.
FOR DETAILS OF THE ALGORITHM SEE THE TEXT OF
THE PROCEDURE QUAD.

PRESENTATION OF DATAl
FIRST THE NUMBER OF RESTRICTIONS I.E. M.
THEN THE NUMBER OF VARIABLES. I.E. N.
FOLLOWED BY THE NUMBER OF EQUATIONS. NEQ.
THEREAFTER NAV. THE NUMBER OF ('FREE') VARIABLES
TO WHICH THE NON-NEGATIVITY RESTRICTION DOES NOT APPLY.
THEREAFTER NNEGD.
THE NUMBER OF NEGATIVE-VALUED DUAL VARIABLES WHICH IS PER-
MITTED UNDER THE CONVEX MODE OF OPERATION.

THEREAFTER PUNCH EACH ROW OF THE COMPOSITE MATRIX

D -W
A B
U' 0 OR
L' 0
TO REPRESENTI
MAXIM W' X + 1/2 X' 0 X
A.X < OR B =
AND X < OR U
AND X > OR = L

READ ORDER PARAMETERS ETCI

M: =READ; NI =READ' NEQ I =READ; NAV: =READ; NNEGD: =READ'
NEWLINECI )J
'IF' NEQ > M 'THEN'
WRITETEXT('('YOUIHAVEIERRONEOUSLYISPECIFIEDIMOREI
EQUATIONSITHANIRESTRICTIONS')');

NEWLINEC 1).
'IF' NAV> N 'THEN' WRITETEXTC'('YOUIHAVEIERRONEOUSLYI
SPECIFIEDIMOREIFREEIVARIABLESIr'HANIVARIABLES' >')J
REENTRY: = 0 ;
514 CHAPTER XVI

'BEGIN'
'ARRAY' TA[ 1 :M+N+3.1 :M+N+2H
'INTEGER' 'ARRAY' ROIolL.COLL[l:M+N);

READ MAIN TABLEAU MATRIX:

MATICTA.N+M+2.N+I.O.O);

REORDER TO APPROPRIATE BLOCKSI

'FOR' JI=I 'STEP' 1 'UNTIL' N+I 'DO'
'FOR' 11=2.1 'DO'
TA[M+N+I+I.JJ:=TA[M+N+I.JJ;
'FOR' 11=1 'STEP' 1 'UNTIL' M+N+I 'DO' 'BEGIN'
TA[I.M+N+IJ:=TA[I.N+IH TA[I.N+IJ:=O; 'END';

RE INTERPRET:

INITIAL VALUE:

FOR LINEAR PART OF INITIAL VALUEI

'FOR' JI=I 'STEP' 1 'UNTIL' N ,~O'
TACN+M+I.N+M+IJ 1=
TACN+M+I.N+M+IJ -2*TACJ.N+M+IJ*TACN+M+3.Jl;

FOR QUADRATIC PART OF INITIAL VALUE:

'FOR' JI=I 'STEP' 1 'UNTIL' N 'DO'
'FOR' 11=1 'STEP' 1 'UNTIL' N 'DO'
TACN+M+I.N+M+IJ:=
TACN+M+I.N+M+Il + TACN+M+3.IJ*TACI.Jl*TACN+M+3.Jl;

REST OF VALUE COLI

'FOR' J:=I 'STEP' 1 'UNTIL' N 'DO'
'FOR' 1:=1 'STEP' 1 'UNTIL' N+M 'DO'
TACI.N+M+I JI=TACI.N+M+IJ-TACI.Jl*TACN+M+3.Jl;

UPPER LIMITS:
'FOR' J:=I 'STEP' 'UNTIL' N 'DO' 'BEGIN'
'IF' TACN+M+2.JJ=O 'THEN' TA[N+M+2.JJ:=IOOOOO;
TA[N+M+2.JJ := TA[N+M+2.Jl-TA[N+M+3.JJ;
'IF' TA[N+M+2.JJ < 0 'THEN' 'BEGIN'
NEIoiLINE( I);
IoIRITETEXT('( 'YOU%HAVE%SUPPLIED%A%LOIolER%LIMIT%IN%
EXCESS%OF%THE%CORRESPONDING%UPPER%LIMIT')');
NEIoiLINE( 1 );
WRITETEXTC'('THE%QP%PROBLEM%IS%THEREFORE%EMPTY.')');
'END';
'END';

NOIoi SOLVE:
QUADCTA.M.N.NEQ.NAV.NNEGD.ROIolL.COLL.REENTRY);

REPORT SOLUTION:
REPQ(TA.M.N.NEQ.NAV.ROIolL.COLL);
'IF' M+N < 13
'THEN' TABOCTA,M+N,M+N,O,O,2,2,ROIolL,COLL)
'ELSE' TABO(TA,M+N,O,O,M+N,I,O,ROIolL,COLL);
NEIoiLINE( I);
IoIRITETEXTC'C'SOLUTION%VALUE%IF%IN%STANDARD%FORM')');
WRITEC30,FORMATc'C'S-NDOOO.DDO')'),TA[M+N+I,M+N+Il/2);
NEIoiLINEC I);
• EN D • ; I EN D •
QUADRATIC PROGRAMMING WITH LINEAR RESTRICTIONS 515

ANSWER-SHEET 16.8 EX

NAME I X 1 X 2 P 1 P 2 VAl.UE

D 1 2 1 -3 1 SET-UP TABl.EAU.
D
S
2
1
1
-1 -1
CD -1 2
-5
51 IS BADNAME. PI DRIVING VAR.
REFUSE Dl AS l.EAVING VARIABl.E.
S 2 3 1 30 ON ACCOUNT OF THE QUAl.IFIER.
D2 BECOMES l.EAVING VARIABl.E.
2T 2 5 -30

NAME X D 2 X 2 P 2 VAl.UE
IN THE RESUl.TING NON-
D 1 1 -1 1 -2 -1 STANDARD FORM TABl.EAU.
S 1
P 1
-1
1
B -1
-5
2
X2 BECOMES INCOMING VAR ••
BY THE COMPl.EMENTARITY RUl.E.
S 2 3 30 51 15 NOW l.EAVING VARIABl.E.

"'aT -4 -5 2 -25 -10

NAME X D 2 S P 2 VAl.UE

D 1 -1 1 -2 -6 WE NOW HAVE A STANDARD FORM

X 2 1 -1 5 TABl.EAU.
P 1
S 2
CD
2
-1 2
25
Dl BEC[lI(ES BADNAME.
Xl IS DRIVING VARIABl.E.
PI IS SEl.ECTED AS
2T -6 -5 2 -25 -20 l.EAVING VARIABl.E.

NAME S D 2 P P 2 VAl.UE

X 1 1 1 -1 2 IN THIS NON-STANDARD FORM

X 2 -1 -1 -1 1 3 SI IS INCOMING VARIABl.E
D 1 1 -1 -2 -6 BY THE COMPl.EMENTARITY RUl.E.
5 2 CD -2 -2 2
21
----------------------------------------
52 15 l.EAVING VARIABl.E. BY
THE RUl.E OF THE SMAl.LEST
2T 2 6 -31 -8 CONl.Y) QUOTIENT.

NAME I P 2 D 2 P S 2 VAl.UE

X 1 -1 1 1 2 ANOTHER NON-STANDARD FORM

X 2 3 -3 -3 1 24 TABl.EAU. IN WHICH THE
S 1 2 -2 -2 1 21 COMPl.EMENTARITY RUl.E
D 1 e> 1 2 -27
-1
----------------------.-----------------
ACTIVATES P2 AS INCOMING
VARIABLE. Dl 15 THE
2T -35 5 10 -2 -50 l.EAVING VARIABl.E.

NAME D D 2 P S 2 VAl.UE
----------------------------------------
X 1 -0.25 0.75 0.50 0.25 8.75 WE HAVE NOW REACHED
X 2 0.75 -2.25 -1.50 0.25 3.75 THE OPTIMUM.
S 1 0.50 -1.50 -1 0.50 7.50
P 2 -0.25 -0.25 -0.50 0.25 6.75

2T -8.75 -3.75 -7.50 6.75 I 186.25

CHAPTER XVII

PARAMETRIC METHODS IN QUADRATIC PROGRAMMING

17.1 Postoptimal variation of the right-hand side of a

quadratic programming problem

We now discuss the following problem:

Maximise

, =, o + w'x + ~ x'Dx (17.1.1)

Subject to

Ax < b + Av (13.2.1)

(~ ~ 0 unless otherwise stated)

(17.1.2)

The main block of linear side-restrictions (and the parametric

variation of the righthand-side of these restrictions),
(13.2.1), are the same as those arising in the case of a
linear programming problem, and we will make use of the same
notation as developed in Chapter XIII.

The non-negativity restriction on the objective function, i.e.

(17.1.2) is a novelty which will cause some complications.

These complications are not particularly serious and the

restriction of a quadratic objective function to non-negative
values falls within the general notion of sensitivity analysis.

The addition of a constant ' 0 to the objective function, i.e.

(17.1.1), rather than (16.1.1) is related to this non-
negativity requirement on the objective function. The non-
negativity requirement on ,(x) can be made into a ~edundant
restriction, by specifying a-large initial value for ,. Our
approach to solving the parametric quadratic programming
problem is via extending the algorithm which we developed for
parametric variation of (the right-hand side of) a linear
programming problem to the quadratic case.

Recall section 16.3.

The optimal and feasible vertex of a quadratic programming

problem is also a feasible solution to the linear programming
problem specified in section 16.3, and modified as follows:
516
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 517

Maximise

~ = w'x + ~'E. (16.3.1)

Subject to

Dx - A'E. 2- -w (16.2.3)

Ax < b + AV (13.2.1)

If we followed the analogy of section 16.3 completely, the

optimal and feasible vertex of the adjusted quadratic
programming problem would have as objective function of the
linear equivalent

(17.1.3)

We shall, however, initially be content with the objective

function indicated by (16.3.1), as a feasible vertex which also
satisfies the complementary slackness condition (i.e. the
requirement of standard form), will be sufficient, without
regard to optimal form.

As in the LP case, parametric steps may be made either by

exogenous adjustment of the righthand-side - a QP generalization
of the approach outlined in section 13.1, or the parameter
may be treated as a variable and re-defined after having entered
the basis - a QP generalization of the procedure outlined in
section 13.5 for the LP-case -

To illustrate the procedure, we slightly modify an example of

a quadratic programming problem, which we already presented in
section 16.3.

We put the initial value of the objective function at 10, and

define the parametric activity as moving the one restriction
outwards.

The parametric problem therefore is:

Maximise

Subject to
518 CHAPTER XVII

As in the linear programming case, the computational

implementation is facilitated, if the parametric activity is
brought over to the lefthand side, and grouped with the other
variables for purposes of updating. This is so, irrespective
of the method of actually making the parametric step. The
resulting set-up tableau 17.la is given below, with deletion
of the fancyhigh upper limits on xl and x 2 .

TABLEAU 17.1 A

SET-UP TABLEAU FOR PRIMAL PARA-

METRICVARIATION OF A QP-PROBLEM.

NAME ! II X 1 X 2 l. P I ! I VALUE

! CODE II 2 3 -1001 I!
----------------------------------------------------
D 1 -1 II -2 1 II 1
D 2 -2 11 -2 2 II 1
D 3 -3 II 1 11 0
5 1 1 001 ! I -1 -2 -1 !! -3

2T I! 3 I! 20

The corresponding initial optimum is given below in tableau

17.lb.

TABLEAU 17.1 B

INITIAL OPTIMUM OF THE PRIMAL PARAMETRIC

VARIATION QP PROLEM GIVEN IN TABLEAU 17.1 A

NAME ! !! D 1 D 2 l. 5 II VALUE

! CODE ! ! -1 -2 3 1001 II
----------------------------------------------------
X 1 1 ! ! -0.40 0.20 (0.20/ -0.20 II 0.40
X 2 2 !I 0.20 - 0.10 0.40 -0.40 I! 1 .30
D 3 -3 I! -0.20 -0.40 -0.40 0.40 ! ! -1.80
P 1 1-1001 ! I 0.20 0.40 0.40 -0.40 ! ! 1.80

2T II -0.40 -1.30 -1.80 1.80 ! I 12.90

The dual variable d3 is negative, indicating that the value of

the objective function will actually increase, if the parametric
variable is brought into the basis.

However, in a non-linear problem, the value of the objective

function ~s a non-linear function of the value of the parameter.
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 519

How non-linear this relationship is, is indicated by the

diagonal d 3 /A entry of -2/5.

The dual variable is the first-order derivative of the

objective function with respect to A, and this first-order
derivative is itself a function of A.

If we follow the convention that the parameter is always stored

as the last specified variable we may indicate this dual
variable as d , and the diagonal entry (-2/5 in the d 3 /A cell
in this case)nas t .
n,n
In the context of the kth parametric step, these two numbers
are related to each other by the following equation

d - t (17.1.4)
n n,n

In (17.1.4), d (A k ) is the value of the nth dual slack-variable

(shadowprice),nexpressed as a function of the kth increment of
the parameter. The term d is the current value of this
shadowprice, i.e. -1.80 iR the example; t is the nth diagonal
cell of the tableau -0.40 in the example.n,n

The shadowprice of the A-column remains minus the first-order

derivative of the objective-function when A is increased
exogenously. Assuming that we are not concerned with the value
of the objective, function, the usual search operation indicates
A=2 as the critical value at which xl will be eliminated. If
the right-hand side is adjusted exogenously, the parametric
step results in tableau l7.lc.

TABLEAU 17. I C

DISPLACED OPTIMUM OF THE PRIMAL PARAMETRIC

VARIATION QP PROLEM GIVEN IN TABLEAU 17.1 A

(L=2>
NAME ! ! ! D I D 2 LI S I !! VALUE
----------------------------------------------------
! CODE !! -I -2 3 1001 !!
----------------------------------------------------
X I I I! -0.40 0.20 0.20 -0.20 I! -0.00
X 2 2 II 0.20 -0.10 0.40 -0.40 ! ! 0.50
D 3 -3 ! ! -0.20 -0.40 -0.40 0.40 ! ! -I
P I ! -100 I ! ! 0.20 0.40 0.40 -0.40 ! ! I
----------------------------------------------------
711
2T ! ! 0 -0.50 -I ! !
520 CHAPTER XVII

The usual updating procedure is not applicable to the 2T-row,

if we only changed the 2T/value cell, the old pseudo-objective
function ~ (see 16.3.1) would stand. The entries in this row
have been obtained by copying from the value column, with the
sign-adjustments indicated in section 16.4. The 2T/value cell,
marked ??? in tableau l7.lc is so far unknown.

The fact that the d 3 variable is the shadow-price of the

redefined A-variable throughout the step, may be expressed
algebraically by the following equation:

(17.1.5)

Substitution of the right-hand side of (17.1.4) for dn(A k )

in (17.6.5) gives the following result:

(17.1.6)

The use of a partial derivative for ~~ in both (17.1.5) and

(17.1.6) relates to the assumption that other non-basic
variables stay at their pre-assigned values of zero, i.e.
these relations apply for a particular vertex. Integration
of (17.1.6) with respect t~ Ak , from zero to a specific value
of Ak , to be indicated as Ak yields:

\k \k Ak
,h
f TI (A k ) dAk -d
n J dAk + t n,n J AkdAk
0 0 0

Evaluation of the integrals yields:

T (\k) - T = -d \ + ! t (\ )2
k (17.1.7)
n k n,n
Since Ak is normally used to indicate a specific parametric
increment, we can now dispense with the distinction between Ak
(any value) and the \k' the position at the next vertex and
re-write (17.1.7) as
2
= T - d Ak + ! t n,n . Ak (17.1.8)
n 2

We may use (17.6.8), to impose (if so desired) a non-negativity

requirement on T.
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 521

We find a zero value of T(A k ) for

d
n
±Vctn 2 -2T t
n,n
(17.1.9)
t
n,n

The parameter is increased from zero to a positive value, and

the parametric step is only made if the initial value of the
objective function is non-negative.
(If no non-negativity requirement on T is intended, (17.1.9)
may be made redundant, by specifying a large initial value
of T.)

For t 0, T is after all a linear function of A, otherwise

n,n
the larger of the two roots is positive and applicable, the
smaller is negative. Thus Ak is restricted to the interval
between zero and the value indicated by the positive root
indicated by (17.6.9).

Since t non positive (negative), the upper limit on Ak is,

°
~s
for t n,~
n,n

d
_ y{2 _ 2T t
n n n,n
(17.1.10)
t
n,n

The quadratic term in (17.1.8) may in fact be zero, in which

case (17.1.9) and (17.1.10) are replaced by simpler linear
formulae.

For tn n > 0, including the linear case t = 0, (17.1.10) is

not appliZable. For tn,n ~ 0, d n ~ 0, the¥enis no upper limit
at all, and the parametric search operation may (or may not)
become unbounded. For the example at hand, we find:

A < -1.80 - v'<1.80)2 + 12.90 x 0.40 = 11.75

- -0.40
We now adjust the initial tableau with resvect to two things.
We mark the upper limit on A. Also, we implement the calculation
of the first terms of (L7.l.8) with respect to 2T, i.e. two
terms of:

2T(A k ) = 2T - 2 . d n . Ak + tn,n A~ (17.1.11)

We double the 2T/A entry, from -1.80 to -3.60 and "normal"

updating will amend the 2T entry to become 2T - 2.d . Ak . The
upper limit is not binding, and the corrected table~u is at
522 CHAPTER XVII

this stage tableau l7.ld.

TABLEAU 17.1 D
INITIAL OPTIMUM OF THE PRIMAL PARAMETRIC
VARIATION QP PROLEM GIVEN IN TABLEAU 17.1 A.
WITH THE 2T/L CELL SPECIALLY ADJUSTED.
NAME I II D 1 D 2 L S 1 I I VALUE

I CODE I I -1 -2 3 1001 ! !
----------------------------------------------------
X 1 1 II -0.40 0.20 -0.20 ! ! 0.40 ~
X 2 2 II 0.20 -0.10 0.40 -0.40 I I 1.30
D 3 -3 II -0.20 -0.40 -0.40 0.40 I I -1.80
P 1 1-1001 I I 0.20 0.40 0.40 -0.40 II 1.80
-----------------------------.----------------------
2T I II -0.40 -1.30 -3.60 1.80 I I 12.90
BOUNDI II X X 11.75 X ! ! X

The quadratic term in (17.1.11) will have to be attended

separately. If the parameter is treated as a variable, i.e. an
explicit step is made, we now obtain tableau l7.le.

TABLEAU 17.1 E
NEW VERTEX TABLEAU. NOT IN STANDARD FORM.

NAME I II D 1 D 2 X 1 S 1 I I VALUE

I CODE II -1 -2 1001 II

L 3 II -2 1 5 -1 II 2
X 2 2 II 1 -0.50 -2 II 0.50
D 3 -3 II -1 2 II -1
P 1 1-1001 II 1 -2 ! ! 1

2T II -7.60 2.30 18 -1.80 II 20.10

Since the upper limit on A is no longer relevant at this stage,

we have suppressed reference to it. Tableau l7.le contains
incorrect entries in the 2T row. The 2T/value cell needs to be
corrected, i.e. the third, quadratic term of (17.1.11) should
be taken into account. For t = -2/5 and A=2, this term is
n,n
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 523

evaluated as -1.60 and the 2T entry in the value column should

be 20.10 -1.60 = 18.50.

It is readily verified that for xl = 0, x2 = 0.50,

T = 10 - 0.5 - 0.5 2 = 9.25, i.e. 2T = 18.50 is indeed the
correct entry.

The other cells in the 2T row will be attended to when the

tableau is back in standard form.

We now adjust the prearranged value of the parameter, i.e.

integrate A = 2 with the value column and put the variable
itself re-defined as Al at -0.00.

Having eliminated xl' d l becomes the next pivot column by the

complementarity rule.

To maintain ordering we therefore interchange the xl an~ d l

columns. Since we wish to eliminate the parameter aga~n at

the next step, A becomes badname and stays in the "wrong"

slot. The resulting tableau l7.lf is given below.

TABLEAU 17.1 F

NEW VERTEX TABLEAU. ADJUSTED

AND READY FOR THE ORDINARY STEP.

(1.=2)
NAME , ! ! X I D 2 D I S I 'I VALUE
, CODE , ! -2 -I IDOl ! !
----------------------------------------------------
LI 3 I I 5 I , !@) -I -0.00
X 2 2 ! ! -2 -0.50 I I I 0.50
D 3 -3 ! ! 2 -I I! -I
P I ! -I 001 I! -2 I ! I I

2T ! , 18 2.30 -7.60 -1.80 II 18.50

The normal quadratic programming step is now made, leading to

tableau 17. 19.
524 CHAPTER XVII

TABLEAU 17.1 G

NEW OPTIMAL AND FEASIBLE TABLEAU,

WITH AN INCONSISTENT 2T-ROW.
(L .. 2>
NAME I II X 1 D 2 L1 S 1 I I VALUE

I CODE I I -2 3 1001 I!
----------------------------------------------------
D 1 -1 II -2.50 -0.50 -0.50 0.50 I I 0
X 2 2 I I 0.50 0.50 -0.50 II 0.50
D 3 -3 II -0.50 -0.50 -0.50 0.50 II -1

"I
P 1 1-1001 0.50 0.50 0.50 -0.50 II 1
-----------------~----------------------------------
2T -1 I-1.50 -3.80 2 18.50
"
This is a standard form tableau and it is now possible to correct
the 2T row. We can use the symmetry property (see section 16.4),
and copy from the value column, while inverting the sign of the
x 2 entry. We also add an upper limit on Al, calculated afresh
from (17.1.10). The resulting tableau l7.lh is given below.

TABLEAU 17.1 H

NEW OPTIMAL AND FEASIBLE TABLEAU,

READY FOR THE NEXT PARAMETRIC STEP.

.Cl.=2 >
NAME I ! I X 1 D 2 Ll S i l l VALUE
------------------~--------------------------------
I CODE II -2 3 1001 II

"
D 1 -1 -2.50 -0.50 -0.50 0.50 I I 0
X 2 2 II 0.50 (0.50) -0.50 I I 0.50
D 3 -3 II -0.50 -0.50 -0.50 0.50 ! ! -1
P 1 1-1001 I I 0.50 0.50 0.50 -0.50 II 1
---------------------------------------------------
2T I
BOUNDI
I I 0
I I
-0.50 -1
100 X
38.50
3.64 X "
II X

The upper limit on A is now

A < -1 - vii + 18.50 x 0.50 = 3.64

1 -0.50
It will be noted that, with an "ordinary" leaving variable. as
distinct from an upper limit distance (e.g. xl)' the exogenous
adjustment of the value column and the 2T-row (e.g. calculating
tableau l7.lc and putting the 38.50 in the cell marked "???")
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 525

is the simpler method compared with introducing A into the

basis (e.g. calculating tableau l7.le).

Since Al is limited by the normal rule of the smallest quotient

(on the x 2 row), to be not more than 1, a further parametric
step can De made, and A will become 2 + 1 = 3.

Exercise

Continue the example, until at A = 3, xl = x 2 0, the para-

metric adjustment problem becomes unbounded.

17.2 Parametric variation of the linear component of the

objective function of a quadratic programming problem

We are now concerned with finding the family of solutions to

the problem

Maximise

T (~, A) w'x + A ~'~ + ~ x'Dx (17.2.1)

Subject to

Ax < b (7.2.2)

(~~ 0)

Now recall section (15.6), concerning non-linear duality. We

formulate the Lagrangean expression

x'Dx + .£.' (E. - ~) (17.2.2)

and the dual requirements are

aL
ax = ~ + A~ + D~ - A'.£. ~ 0 (17.2.3)

.
Th e expreSS10n L - x ,aL f d • (15 .6. 7) turns out to
be
- --ax re erre to 1n
aL
L(x, .£.' A) - x' -ax
- =

-w'x - A~'X - x'Dx

E.'.£. - ! x'Dx (17.2.4)

526 CHAPTER XVII

The dual of a quadratic programming problem therefore lS

Maximise

x'Dx (17.2.5)

Subject to

Dx - A'E ~ - w - Ag (17.2.6)

In quadLatic programming there is an even closer analogy between

the "primal" and the "dual" parametric variation problem, than
is the case with linear programming.

We proceed as follows:

Firstly, a parametric restriction, specifying that a certain

linear function of the variables is to be greater than or equal
to zero, is entered as an (mth) dummy-restriction.

(17.2.7)

As in the linear programming case (compare Section 13.5), this

"restriction" is initially exempt from search operations. Its
associated row serves as an operator. The transposition and
inversion of the sign of that row as part of the set-up tableau
automatically leads to the generation of a tableau in which we
may interpret the dual requirements as the equivalent of
(17.2.6), i.e.

(17.2.8)

Example

Maximise
2
T** x
1
-

Subject to

(This is substantially the same example as used in the previous

sect ion) .
 PARAMETRIC METIIODS IN QUADRATIC PROGRAMMING 527

The initial set-up tableau is as follows:

TABLEAU 17.2 A

SET-UP TABLEAU FOR DUAL PARA-

METRIC VARIATION OF A QP-PROBLEM.
NAME ! ! ! X I X 2 P I l. !! VALUE
---------------------------------------------------
! CODE I I 2 -1001 -1002 I!
---------------------------------------------------
0 I -I II -2 ! !
0 2 -2 II -2 2 ! ! I
S 1 1001 ! ! -I -2 ! ! -3
T* 1002 I ! - I I! 0
---------------------------------------------------
2T , ! 3 ! !

The vector g is a unit vector ~l' i.e. the unity element in

position 1.-

In the QP case it is in fact practical (though by no means the

only possible method to never actually activate the parametric
restriction in the sense~aking it binding.

The above dual problem shows that we may extend the approach
which we followed in section 13.3 for linear programming, to the
quadratic case. All that is needed is to properly account for
the change in the right-hand side of the dual requirements. The
interpretation which we mentioned in section 13.5, i.e. the
parametric component of the objective function to attain a
certain value is also applicable in both cases.

Using only the convex mode of operation, the similarity between

the parametric variation of the right-hand side of the primal,
or the dual requirements, is enhanced even more, because in
quadratic programming we actually start a quadratic programming
step, by bringing a variable into the list of basic variables.

That variable is in the case of parametric variation of (the

linear component of) the objective function, the dual variable
associated with the parametric restriction. The parametric
restriction itself is in fact never activated.

The example's initial optimal tableau, with the first

parametric pivot marked c::J ,
is given below, in tableau l7.2b.
528 CHAPTER XVII

TABLEAU 17.2 B

INITIAL OPTIMUM OF THE DUAL PARAMETRIC VA-

RIATION QP-PROBLEM. GIVEN IN TABLEAU 17.2 A.

NAME II D 1 D 2 S 1 L II VALUE

I CODE II -1 -2 1001 -1002 II

X 1 1!1 -0.40 0.20 -0.20 -0.40 II 0.40

X 2 I 2 I! 0.20 -0.10 - 0.40 10.201 I I 1.30
P 1 1-1001 ! I 0.20 0.40 -0.40 0.20 II 1.80
T* ! 1'002 I I -0.40 0.20 -0.20 -0.40 II 0.40

2T II -0.40 -1.30 1.80 -0.40!1 -7.10

We do not, in the case of variation of the objective function,

implement a non-negativity restriction on the objective
function, therefore there is no upper limit on P2' except the
fancyhigh upper limit implied in the basic quadratic programming
algorith. (There would be no problem in extending the analogy
with variation of the right-hand side to cover this point as
well.) We do, however, need a generalization of (17.1.8) to
calculate the correct value of the objective function.

T**(A k ) = T** + T* . Ak - ! t m+n,m+n (17.2.9)

Here, T* is the current value of the parametric component of

the objective function (2/5 in the example) and t m+n, m+n is
the m+n, m+n cell of the tableau. This assumes
that the parametric restriction is stored - as in the case
of linear programming - as the last restriction. Note the change
in sign, compared with (17.1.8). This is because a generalization
of (17.1.6), would refer to dT/dA being +T*, instead of -d ,
as in (17. 1. 6) • n

The computational implementation of the parametric variation of

the objective function is largely analogous to the procedure
discussed in the previous section for the variation of the right-
hand side. Thus, the A entry in the 2T row is doubled, becoming
-0.80. Reference to the 2T/A cell of the tableau rather than to
the value of T* automatically covers the difference in the sign
of the linear terms of (17.1.8) and (17.2.9). The parametric
step is now made, yielding incorrect results for the 2T-row.
Note the difference with section 13.3, where, in the LP case
dual parametric steps are made by updating the objective
function row only. In a QP problem, a change in the objective
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 529

function generally leads to a change in the primal solution

vector, and an upper limit distance could be eliminated. We
develop tableau l7.2c.

TABLEAU 17.2 C

INTERMEDIATE TABLEAU OF THE DUAL PARAMETRIC

VARIATION QP-PROBLEM. NOT IN STANDARD FORM.
NAME I I I D I D 2 S X 2 II VALUE
I CODE II -I -2 100 I 2 II

X I I I II -I 2 II 3
L 1-1002 II -0.50 -2 5 I I 6.50
P I I-IDOl !! 0.50 -I II 0.50
T* I 1002 !! -I 2 II 3

2T ! ! 0.40 -1.70 0.20 4 II -1.90

The correction for the quadratic component turns out to be

(for 2T) 0.40 x 6.50 2 = 16.90. We now re-define the objective
function, thereby eliminating P2 (=A) without affecting the
Lagrangean. The tableau now becomes (after column-reordering):

TABLEAU 17.2 D

RE-DEFINED TABLEAU OF THE DUAL PARAMETRIC

QP-PROBLEM. READY FOR THE ORDINARY STEP.
(L=6.50>
NAME I I I D I X 2 S I D 2 II VALUE

I CODE I I -I 2 100 I -2 II

X 1 ! ! 2 -I ! ! 3
LI !-1002 I I 5 -2 (-0.50) I I - 0.00
P 1 1- 1 001 ! ! -I 0.50 I I 0.50
T* I 1002 I I 2 - I I I 3
----------------------------------------------------
2T ! ! 0.40 4 0.20 -I .70 I I IS
530 CHAPTER XVII

After making the ordinary step, we develop tableau l7.2e.

TABLEAU 17.2 E

NEW STANDARD FORM TABLEAU OF THE

DUAL PARAMETRIC VARIATION QP PROBLEM.
(L=6.50)
NAME ! ! ! D I X 2 S I LI ! I VALUE

! CODE !I -I 2 1001 -1002 "

X I I ! ! 2 -I ! ! 3
D 2 -2 ! ! -2 -10 4 -2 I! 0
P I !-IOOI ! ! I 4 -2 I II 0.50
T. ! 1002 ! ! 2 -1 ! I 3

2T !! -3 -13 7 -3.40 !! 15

Tableau l7.2e is a standard form tableau, except for the

pseudo-objective function row. To obtain the correct 2T** row,
one needs to re-form this row from the value-column, with the
help of the symmetry rules. (The reference to this row as the
"2T" row is in fact incorrect.)

Exercise:

Continue the example, until at A = 7.00, xl 0, the

parameter-direction becomes unbounded.

17.3 Strict convexity in the parameter subspace

The examples discussed so far have not involved the diagonal

cell of the parametric variable itself as pivot.

If we follow the normal QP rules (of the convex mode) as near

as possible this would be an acceptable parametric pivot.
Whether or not we accept it is basically a question of whether
we want the information provided by such a step - the next
solution-point is not affected. This point arises in connection
with parametric variation of the (primal) righthand-side
vector b.

A parametric variation of the righthand side may be associated

with an increase in the value of the objective function, and
in the presence of strict convexity in the A-subspace, this
increase (if present) ceases at some point, without change in
the collection of binding restrictions.
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 531

Example
2 2
Maximise T - xl x2 + 15 x - 4x2 + 500
1
Subject to xl + x 2 > 10 + >..)
) xl' x 2 > 0
xl < 9 + 3>..)

The initial optimal solution of this problem.is xl = 9~ x 2 1,

and the tableau, inclusive of the x3 = >.. varlable and ltS
dual restriction is the following

TABLEAU 17.3 A

THE SHADOWPRICE OF THE PARAMETER BEING ELEMINATED.

NAME !! D 1 D 2 L S 1 S 2 !! VALUE
-------------------------------------------------
X 1 I ! -3 I I 9
X 2 ! ! 2 -I -I ! ! 1
D 3 ! I 3 -2 1-261 4 10 ! ! -3
P 1 ! ! 1 4 -2 -2 ! ! 6
P 2 ! ! -1 1 10 -2 -4 I ! 3

2T ! I -9 -I -3 6 3 I! 1098

By the normal rules of the QP algorithm the d 3 !>..-cell is the

obvious pivot. If >.. is then re-defined as being minus zero, and
declared to be the badname-variable, the same step is made ln
opposite direction and the next solution is as given in
tableau l7.3b.

TABLEAU 17.3 B

PARAMETRICALLY ADJUSTED SOLUTION, MADE ON

INDICATION OF THE OBJECTIVE FUNCTION ONLY.

(L=0.12)
NAME! ! D 1 D 2 LI S 1 S 2 !! VALUE

X ! ! -3 ! ! 9.35
X 2 ! ! 2 -I -I ! ! 0 .77
D 3 ! ! 3 -2 -26 4 10 ! ! 0 .00
P 1 !! 4 -2 -2 ! ! 5.54
P 2 ! ! -1 10 -2 -4 ! ! 1 .85
-------------------- -------------------------------
2T !! -9.35 -0.77 5.54 1.85!! 1098.35
532 CHAPTER XVII

This tableau differs from the previous one, only in the

righthand-side and the 2T-row.

If A is increased further, the d 3 row is no longer eligible

as pivotal row, provided the entry for d 3 (written as 0.00),
is classified as positive. The dual variable P2 is then
eliminated instead. Had the d 3 row been excluded in the first
place, the same result would nave arisen in only one parametric
step.

It is in fact a question of signalling the point at which the

value of the objective function changes sign.

In the case of parametric variationoL-the objective function,

the similar issue arises wi_thre-spect to the value of the
parametric componente>f-ihe objective function. In the QP
case, the incoming variable in the parametric step is selected
by the complementarity rule, and there is no need to put any
other figure in the T* row/value column cell, than the actual
value of T*. Elimination of this figure by applying the rule
of the smallest quotient, then indicates the value of A, for
which T* changes sign, from negative to positive.

17.4 Elimination of dual variables during parametric variation

The rules concerning the elimination of dual variables are

slightly different when making parametric steps as compared
to the normal rules on this point.

This is because the argument raised in Section 16.9, - when

increasing the value of a primal driving variable in the
presence of a dual badname variable it is efficient to fly
through dual restrictions -, is not applicable for a parametric
step.

The object of making parametric steps is to obtain information

about the solution at suitable values of the parameter, not
simply to ~ncrease the value of the parameter. Values of the
parameter at which exclusion of certain variables from the list
of basic variables ceases to be optimal would appear to come
under the general heading "suitable" values of the parameter
and dual variables will be accepted as leaving variables.

In a convex problem, the strict interpretation of the rule of

the smallest quotient, as outlined above, ensures that, if
parametric variation is initiated at an optimal and feasible
solution, all variables stay non-negative all the time, except
for a "parametric zero" entry of -0.00 for one variable at each
change of vertex.
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 533

However, in a non-convex problem, it may happen that the

critical value of A, at which a particular dual variable ceases
to be positive, does not mark a new solution which is convex
in some All~l = ~l subspace, but a local minimum or a saddle-
point. It mayor may not be possible then to exchange the dual
variable in question against the corresponding primal variable,
but if we wish to maintain the property of subspace convexity,
we must either stop at such a point, or alternatively, breach
the dual restriction in question.

This point is best discussed in the context of the "explicit"

method of making parametric steps, this permits the generalization
of the results of Chapter XVI to the parametric variation
problem.

The parameter is the first variable which is brought into the

basis in a standard form tableau, i.e. we treat it as the
driving variable. By implication, the variable which is
complementary to the parameter i.e. the shadowprice of the
parameter in the primal parametric variation problem, and the
value of the parametric component of the objective function
in the case of dual parametric variation, is the badname
variable. If the problem were pursued on the lines discussed
in section 13.5 for the LP case, the normal search operations
of the basic QP algorithm would assign that status to the
variables in question in any case.

We may then invoke the driving variable increment qualifier,

i.e. a dual variable is not acceptable as leaving variable, if
subsequent application of the complementary rule were to lead
to the selection of a primal incoming variable which would cause
a reduction in the value of the parameter. Then, if this
qualifier is accepted, we may be sure that the re-defined
parameter can be increased (from -0.00 to exactly zero), if not
in the immediately succeeding step, then in the next step,
permitting return to standard form in at most two steps.

The term driving variable' increment qualifier may not apply if

the direct method of making parametric steps by changing the
righthand side exogenously is employed, but the need to avoid
the development of "improper vertices" where the objective
function is not conved in the All ~l = b subspace, exists for
both methods of making parametric steps.

Example
2
Maximise T = Xl - 20 Xl + 500

Subject to Xl > 1 + A )
Xl < 12
) Xl > 0
)
534 CHAPTER XVII

Starting at A = 0, we note that this problem has two local

optima~ xl =.l.and xl = 12. Between these t~o local optimal,
there ~s a mlcn~mum, at xl = 10. Normal appl~cation of the basic
QP algorithm leads to the global optimum, which is xl = 1. The
initial optimum tableau is given below in tableau l7.4a.

TABl.EAU 17.4 A

INITIAL OPTUMUM OF A NON-CONVEX

PARAMETRIC QP-PROBl.EM.

NAME! I D 1 L 5 1 P 2 II VALUE

XI!! -1 - 1 ! I 1
D 2 ! I 2 2 -1 II 18
PI" 2 2 -1 ! I 18
5 2 !! 1 1 I! 11

2T " -1 18 18 -11 II 962

Now let us assume that the degeneracy is broken in favour of

eliminating PI' and that this variable is accepted as leaving
variable without paying attention to convexity.

If the parametric step is then made by exogenous adjustment of

the righthand-side, we develop tableau l7.4b, if the step is
made explicitly, we develop tableau l7.4c.

If the usual feature of declaring the zero which is to be

eliminated as being a very small negative number is applied here,
neither presentation actually permits its elimination. If the
entries of -0.00 are replaced by exact zeros which are then
classified as positive numbers, and the qualifier is again
disregarded, both methods permit return to standard form,
developing a "quadratic programming" tableau which describes
the local minimum at xl = 10.

The ambiguous status of dual variables in a non~convex problem

gives rise to the concepts of normal, proper, and improper
(non-optimal) boundedness (or unboundedness). A" parametric
variation column associated with a (locally or globally)
optimal and feasible solution basis is normally bounded, if we
may develop (without violating subspace convexity), for some
positive value of the parameter, without further change in the
actual value of any primal variable, another (locally or
globally) optimal and feasible solution basis, for which the
collection of primal basic variables differs from the one ~n
the present tableau either by the exchange of two primal
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 535

TABLEAUX 11.4 BAND C

DISPLACED OPTIMUM OF A NON-CONVEX QP PROBLEM.
NOT RESPECTING THE DRIVING VARIABLE INCREMENT
QUALIFIER.

11.4 B EXOGENOUS ADJUSTMENT.

CL=9)
NAME !! D 1 LI S 1 P 2 !! VALUE
X 1 ! ! -I -I ! ! 10
D 2 ! ! 2 2 -I ! !
P 1 ! ! 2 2 -I ! ! -0.00
S 2 ! ! 1 1 ! I 2

2T !! -10 -2 !I 800
11.4 C EXPLICIT STEP.

CL=9)
NAME !! D 1 P 1 S 1 P 2 !! VALUE
X 1 ! ! 0.50 0.50 -0.50 ! ! 10
D 2 ! ! -I ! !
LA ! ! 0.50 0.50 -0.50 ! ! -0.00
S 2 ! ! -0.50 -0.50 0.50 I! 2

2T !! -10 -9 -2 !! 800

variables, or by one primal variable leaving or entering the

basis. The new optimal and feasible solution, thus identified
is then associated with a normal successor tableau. If the
leaving variable identified by the parametric variable is a
primal variable, and if that primal variable cannot be
eliminated because the problem becomes empty at that value of
the parameter, the parametric column is said to be bounded
by the end of the feasible space area.

(In the example this is the case for A = 11, s2 becomes

negative, but cannot be eliminated, xl being already at its
maximum value of xl = 12.)

Boundedness at the end of the feasible space area gives rise

to an empty end of algorithm tableau. In the example given
above, the empty end of algorithm tableau would be either
tableau l7.4d or tableau 17.4e, depending on the method of
making the parametric step.
536 CHAPTER XVII

TABLEAUX 11.4 0 AND E

EMPTY END OF ALGORITHM TABLEAUX.

11.4 0 EXOGENOUS ADJUSTMENT

CL= 11 )
NAME!! 0 1 Ll S 1 P 2 !! VALUE
X 1 !! -1 -1 !I 12
0 2 !! 2 2 -1 I! -4
P 1 !! 2 2 -1 !! -4
S 2 !! 1 1 !! -0.00
-------------------------------------------
2T !! -12 -4 -4 0 ! ! 808
11.4 E EXPLICIT STEP METHOD

(L- 1 1 )
NAME II 0 1 S 2 S 1 P 2 I! VALUE
-------------------------------------------
XI!! 1 ! I 12
o 2 !! -2 -1 ! I -4
PI!! -2 -1 ! I -4
Ll !! 1 I! -0.00

2T !I X X X X II X

The computational implementation offered here would normally

supply the equivalent of tableau 17.4d, but if the leaving
variable in the parametric search operation is an upper limit
distance, the equivalent of tableau 17.4e would be supplied,
the technical cause of the end of the algorithm being that the
P2 column activated by the complementary rule, is unbounded.
(The 2T row has been marked with X entries as this row would
contain invalid entries on account of the coefficient doubling
device).

If an empty end algorithm tableau describes a solution vector

where all dual requirements are satisfied, the corresponding
parametric column is said to be properly bounded by the end
of the feasible space area.

If the empty end of algorithm tableau describes a solution which

violated one of more dual restrictions (e.g. as is the case with
the PI restriction in the example), the corresponding parametric
column in the preceding tableau is said to be improperly (or
non-optimally) bounded by the end of the feasible space area.

The notion of proper and improper (non-optimal) boundedness

PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 537

may obviously be extended to non-empty successor tableaux and

indeed to unbounded columns. A parametric column is improperly
bounded by a non-empty successor solution if it is not normally
bounded but if some other primal feasible solution which is
convex in some All ~l = ~l subspace exists, (which differs in
only one variable from the current one, in the same way as
defined above for normal boundedness), but which breaches one
or more dual restrictions.

In a convex problem, there is never any need to breach dual

restrictions it therefore follows that, in a convex problem,
boundedness implies proper boundedness.

The similar distinction also applies to unbounded parametric

columns. If all entries in primal and dual variabl'es' rows are
non-positive (and, as explicitly written upper limit
restrictions are obtained by turning the signs around), zeros
in all rows associated with bounded variables), the parametric
column is unbounded. If there are positive entries in one or
more dual variables rows, but these dual variables cannot be
eliminated on account of the qualifier, we have an improperly
unbounded column.

A solution vector at which a dual variable changes sign, but

where it is not possible to introduce the corresponding primal
variable at zero value without violating subspace convexity
might be called an improper vertex. (An improper vertex is
therefore a solution basis which describes a local minimum or
a saddle-point).

It is obviously possible to activate the basic QP algorithm

just beyond an improper vertex (e.g. in tableau l7.4b one would
re-enter with A = 9 and select PI as badname-variable).

The typical parametric feature of exchanging the solution basis

without actually changing the primal solution vector at the
point where one changes basis, would not, however apply in
that case.

17.5 Parametric equivalence and the number of steps on

parametric re-entry

We have so far assumed that the normal way of making parametric

steps is to change the solution vector during the parametric
variation step, and then to make one ordinary step which
merely effectuates a change in the solution basis without
actually causing any change in the solution vector.

In fact two ordinary steps may be needed besides the parametric

variation step, and one of these ordinary steps may involve a
538 CHAPTER XVII

change in the dual solution vector even where neither of the

two causes any further change in the primal solution vector.

We now state four alternative methods of making parametric

steps and their relationship to each other, somewhat more
formally, as follows:

Method 1

Bring the parameter into the solution-basis, treating it as the

driving variable. (Exit if the parameter is found unbounded).
Select leaving variables by the rule of the smallest quotient,
subject to the driving variable increment qualifier, and incoming
variables by the complementarity rule. Re-define the value of
the parameter to Ak = -0.00 immediately after its entry into
the solution basis, and exit as soon as the parameter has again
been eliminated.

Method 2

Change the right-hand side exogenously, until one variable,

which is also acceptable as leaving variable under method 1,
ceases to be non-negative, now being observed as being -0.00
instead. Then eliminate the now negative-valued variable either
by one step with a negative pivot on the main-diagonal, or by
the application of a complementary pair.

The first incoming variable of the pair is the complement of

the parametrically eliminated variable, and the corresponding
leaving variable is selected by the rule of the smallest
quotient, but the driving variable increment qualifier lS
replaced by a verification of the applicability of the
complementary pair situation.

Method 3 (not recommended)

Change the righthand-side as in method 2, and find a new optimal

and feasible solution by re-entering the basic QP algorithm
in the convex mode of operation. Exit on return to standard form.

Method 4

As method 3, but use the non-convex mode of operation.

All four methods have the search operation for the parametrically
eliminated leaving variable in common, and if none is found all
four versions of the algorithm terminate.

In a convex problem, this alarm exit obviously corresponds to

either an unbounded parametric variation problem (in the case
PARAMETRIC METIIODS IN QUADRATIC PROGRAMMING 539

of primal parametric variation), in the same way as discussed

in section 13.2 for the LP case or, (with an unbounded
shadowprice of the parametric restriction), to having attained
the maximum value of the parametric component of the objective
function. No statement concerning the non-existence of solutions
for higher values of A, is, however, applicable in the case of
an improperly unbounded parametric column, although such a
solution - if existing - would not be obtainable by any of the
methods listed under 1, 2, 3 or 4.

The following properties concerning the 4 stated methods apply

therefore under the prerequisite that the parametric column is
bounded (properly bounded or improperly bounded). We also
exclude the complement of the parameter as leaving variable,
although it should be remembered that this case, as discussed
~n section 17.3, exists and presents no particular problems.

In the interest of concise formulation the actual formulation

of the relevant properties is preceded by a tableau summary
which refers to the results of the parametric search operation:
We first consider that search operation as such

Name corn Value

plv d v

We shall use the abbreviation plv("Parametric leaving variable")

to refer to the variable which is driven to zero by the parametric
variation step, and corn ("complement") for its associated
complementary variable; the current value of plv variable is
indicated as v; p is the pivot found in the parametric search
operation; d is the diagonal cellon the intersection of the
plv-row and the corn-column. Note that these abbreviations will
be used in relation to all four methods, even though they
derive their significance "mainly from their role in method 1.

We may distinguish two cases, d = 0, and d < 0. The case d > °

does not arise; if plv is a primal variable it does not exist
at all, as it would contradict the subspace convexity properties
of the current solution"basis, if plv is a dual variable its
selection as leaving variable is prohibited by the driving
variable increment qualifier.

We shall refer to the case d < 0, as "the (negative) definite

case" and the case d = 0, as "the semi-definite case".

We now state some properties concerning the relations between

the four algorithms that were described.
540 CHAPTER XVII

Equivalence property concerning the negative definite case

In the negative case, d < 0, all four methods effectuate a

return to a standard form successor solution-basis, in just
one step, following the parametric variation-step.

(The proof of this statement is obvious; the one pivot is

dip in method 1, eliminating A, and d in method 2, 3 and 4
eliminating plv).

Before stating a similar property relating to the semi-

definite case, it is useful to give some further tableau-
summaries.

Name com oiv A Value

semi-definite tableau
plv 0 -a [i] v
on entry to the
olv a c ? ?
parametric step

Name com oiv plv Value

semi-definite tableau
Ak 0 -alp lip -0.00 in exit from the
parametric variation
olv a ? ? ?
step by method 1

Name com oiv A Value

semi-definite tableau
plv 0 -a p -0.00
exit from the para-
olv a c ? ?
metric variation step,
by method 2, 3 or 4
Here olv ("ordinary leaving variable") is the variable selected
as leaving variable by the application of the rule of the
smallest quotient, in the first step after the parametric step,
following method 1. oiv ("ordinary incoming variable") is the
variable which is thereupon activated by the co~plementarity
rule.

Note that the fully symmetric arrangement

Name com oiv A Value

plv 0 a [EJ v
olv a c ? ?

i.e. assuming the parametric and the ordinary leaving variable

to be of the same class, does not apply. It would contradict
the driving variable parameter theorem-(Recapitulation of the
proof on that point: the assumption that both plv and olv are
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 541

primal variables contradicts the semi-definite case; the

assumption that both are dual variables is wrong because the
qualifier inhibits that choice of pivots, i.e. olv would be
refused) .

The very writing of the applicable summaries for the semi-

definite case assumes that a further step by method 1 can be
made.

This is not a trivial condition, because primal parametric

variation may result in the formulation of an empty problem,
emptiness becoming apparent when a primal plv variable - or
the parameter - has the value -0.00, and cannot be increased
to exactly zero. Method 1 will then exit with the com variable
being found unbounded, the boundedness proof of section 16.10
does not apply because the variable which is technically the
badname-variable is not eligible as leaving variable. Methods
2 and 4 will react to emptiness by not finding a primal
incoming variable.

However, the following statement is true:

Equivalence property for the semi-definite case;

If the parametric variation step has been made, and has not
resulted in the formulation of an empty problem, while the
semi-definite case is applicable, then irrespective of the
convexity or non-convexity of objective function methods 1, 2
and 4 each find a new solution basis in two ordinary steps
(following the parametric variation step). Of these two steps
only one involves a non-trivial search for a smallest ratio,
the other is of zero length and involves either a trivial
search (in method 1), or no search at all (the second step of
a pair in methods 2 and 4).

In the absence of degeneracy (other than the parametric pseudo

zero), all three methods lead to the same successor solution.
This statement appears to follow almost immediately from the
summaries, but the significance point is the selection of
the same olv. variable by all three methods.

The search for the smallest dual ratio (methods 2 and 4)

involves actually the same numbers as the search for the
smallest quotient in method 1. This is because the parametric
variation step leaves the com variable's column unaffected,
insofar as its non-zero elements are concerned.

For a non-convex problem, the new solution-basis may be non-

optimal, if the driving variable increment qualifier became
active in the parametric variation step, but the equivalence
542 CHAPTER XVII

applies to improper boundedness as well. (This point has been

discussed already in the section 17.4).

But even if optimal form is maintained, the equivalence property

does not necessarily apply to method 3. This is because, under
metho~ the driving variable increment qualifier is applicable
with respect to the com variable as driving variable, and for
c > 0, this may inhibit the activation of the complementary
pair used in method 2. This will generally mean that a different
local solution is reached, and it may take more than 2 steps
to get there.

Example
2 2
Maximise T xl + x2 - 20 xl - 15 x 2 + 500

Subject to xl + x2 > 1 + A

xl > x - 1 + A
- 2
o~ xl < 100; 0 < x 2 ~ 2. (A ~ 100)

A local initial optimum (xl = 0, x 2 = 1) is developed by

normal application of the basic QP algorithm, this solution
is summarized in tableau l7.5a below.

TABLEAU 17.5 A

ILLUSTRATION OF THE BREAKDOWN OF METHOD 3.

NAME!! X 1 D 2 LA S 1 P 2 !! VALUE DIST

-------------------------------------------------------
D 1 ! ! lj -1 -2 -2 2 ! ! 7 X
X 2 ! ! 1 -1 -1 ! ! 1
D 3 ! ! -2 2 2 -2 ! ! 13 X
P 1 ! ! -2 2 2 - 1 ! ! 13 X
S 2 ! ! -2 2 1 ! ! X
-------------------------------------------------------
2T ! ! 7 -1 13 13 ! ! 972 X
BOUND! ! 100 X 100 x X ! ! X X

If the zero in the s2 row/value column cell is parametrically

adjusted to -0.00 and the convex mode is activated (method 3),
the P2 column is bounded only by its artificial upper limit,
the dl restriction needs to be breached on account of the
driving variable increment qualifier.

In this small example, we then still get at the same solution in

two steps, but we must obviously consider the possibility that
the dual restriction associated with another non-basic variable
is eliminated at this point, leading to a non-optimal successor
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 543

tableau, i.e. in effect selecting the wrong complementary pair.

Methods 2 and 4 on the other hand, both implement the comple-

mentary pair which exchanges xl against s2' and interchanges
their associated dual variables at the next step.

There is, in fact no meaningful distinction between methods 2

and 4, at least no~if we assume that the previously existing
solution basis was an optimum, method 4 will activate the
complementary pair mentioned for method 2.

In activating the non-convex mode in the reentry-mode it is

obviously necessary that one verifies whether the complementary
situation is applicable; the "ordinary" QP code of section
16.15 contains that provision already.

Note also that method 1 is equivalent to methods 2 and 4,

despite the fact that it calls for verifying the driving
variable increment qualifier at the stage of searching for a
leaving variable in the first ordinary step.

This is also apparent from the summaries given above: the

qualifier merely serves to exclude the selection of a second
dual variable as ordinary leaving variable (with the plv
variable already being a dual variable) and is otherwise always
satisfied.

The computational implementation offered in this chapter uses

method 1 if the parametrically eliminated variable is an upper
limit distance, otherwise method 4.

17.6 Parametric solution methods

Parametric methods can be used to solve quadratic programming

problems in the first place.

In the computational context offered here this is not a

practical proposition, if only because parametric steps are made
by way of making ordinary steps. The topic has, however, had
a certain amount of attention in the literature in particular
by van de Panne, and it provides certain useful interpretations.

The two main issues are the representation of the driving

variable as a parameter, and the artificially optimal and
feasible starting solution.

The first issue is obviously relate,d to the driving variable

parameter theorem.
544 CHAPTER XVII

The driving variable parametric equivalence method

(or ,- as van de Panne-rJ5Ji Section 9.2), would say, the
symmetric variant of the s~mplex method for Quadratic
programming), may be summarized as follows:

We may select a badname-variable and a driving variable - by

any criterion -. Instead of bringing in the driving variable
as a basic variable we specify a parametric activity the
coefficients of which are the same as for the selected driving
variable (except for a change in sign if the parameter is
presented on the right-hand side). We then make parametric
steps until the badname variable has been eliminated. We must
obviously restrict the method to those cases where the driving
variable can be shown to be bounded - and then exchange the
driving variable itself for the parameter.

Depending on the precise version of the convex mode of operation

which is used, we may develop the same solutions as in the method
outlined in Chapter 16. It is in fact a different presentation
of the tableaux.

This is illustrated below in the tabulation l7.6a. The parameter

has been presented on the right-hand side of the equation sign,
the entries in the parameter column therefore have the same
absolute value, but are of the opposite sign as the corresponding
entries in the driving variable's column in the same tableau.

The one exception to that property is the 2T-row/A column entry,

where proportionality with the incoming variable column cell
in the direct tableaux has been maintained.

Except in the second step of a parametric pair, the incoming

variable in the direct presentation of the tableau is
proportional to the parametric column. Both columns indicate
the change in the solution vector, but the parametric column
gives the change per unit of increase of the driving variable,
the ordinary incoming variable per unit of the incoming
variable. (The reason that this property is not automatically
maintained in the 2T-row of the parametric tableaux, is that the
2T-row is re-built at each step from the value column.)

A computational disadvantage of the driving variable parametric

equivalence method as a computationally applied method is that
pivots are taken on the main diagonal whenever a negative
non-zero entry occurs in the diagonal cell of the leaving
variable's row, irrespective of the absolute value of this cell.
That may not be a particularly obvious drawback in the
example given, but the issue of near zero pivots arises.
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 545

TABLEAU 17.6 A

ILLUSTRATION OF DRIVING VARIABLE PARAMTERIC EQUIVALENCE.

DIRECT TABLEAUX PARAMETRIC TABLEAUX

1..=0 1..=0.5
NA! Xl X2 PI VAL NA! Xl X2 PI VAL I.. VAL
-------------------------- --------------------------------------
Dl -2 1 1 Dl -2 1 1 -1 0.5
D2 ! -2
~ 1 D2 ! @ 2 1 IT) -0.0
SI ! -I -2 -3 SI ! -I -2 -3 -3

2T ! 3 2T ! 3 -3 -1.5

1..=0.5 1..=1
NA I XI D2 X2 VAL NA! XI D2 PI VAL 1..1 V(~L

-------------------------- --------------------------------------
DI
SI
-2
! -I
-0.5 C0
-2 I
0.5
-3
DI ! @
X2 ! -0.5 -I
0.5 EiJ
0
-0.0
1 0 .5
PI ! 0.5 -I 0.5 SI I -I -1 -2 -3 2 -2
-------------------------- --------------------------------------
2T ! -1.5 4 ! -I .5 2T ! 0.5 0.0 3 ! -2 -4 -3.5

1..=1 1..=1 .8
NA ! DI D2 XI !VAL NA ! DI D2 PI VAL 1..2 VAL
--------------------------
51 2 -I G -2 XI -0.5 -0.5 0.0 0.5 0.4
X2! I - 0• 5 - 2 0•5 X2 ! - 0 .5 - I ! 0•5 1 1 .3

::-~-=~::-::---~~:~-:-=~----!~::~--~:~
PI! I -2 I

21 ! -4 0.5 9 ! -3.5 2T! 0.0 -0.5 2 -3.5 -4.5 -7.1

1..= 1.8 1..=0

NA! D! D2 SI VAL NA! DI D2 SI VAL 1..3 VAL
--------------------------------------
,
XI -0.4 0.2 -0.2 0.4 XI -0.4 0.2 -0.2 0.4 0.4
X2! 0.2 -0.1 -0.4! 1.3 X2 ! 0.2 - 001 -0.4 ! I .3 1.3
PI ! 0.2 0.4 -0.4! 1.8 PI ! 0.2 0~4 -0.4 ! o .0 -1 I .8

2T ! -0.4 -1.3 1.8! -7.1 T I -0.4 -1.3 1.8! -7.1 -7 .1

546 CHAPTER XVII

In particular, if the solution is in a corner of the

feasible space area, and a primal variable is eliminated by a
primal driving variable, mistaking a rounding error for a non-
zero number may totally destroy the accuracy of the tableau.

Example

Maximise T 2xl + 4x2

Subject to 3x l + 2x2 ~ 11, 2xl + 3x 2 < 4

This is in reality an LP problem.

Now suppose we first exchange xl against s2' - by whatever

method, parametric or not -, and then parametrically introduce
x 2 into the basis, in fact against xl as leaving variable. This
is not an efficient path anyhow, the assumption merely serves
to illustrate a potential risk of the parametric method. ~he
resulting situation is summarized in tableau l7.6b.

TABLEAU 17.6 B

CHOICE OF AN UNDESIRABLE DIAGONAL PIVOT.

LA=O LA=I.33
NAME! I D I X 2 P I S 2 !! VALUE LA VALUE
---------------------------------------------------------------
X I II ~ 1.50 0.50 II 2 1- 1 • 50 1 -0.00
D 2 ! ! -1.50 2.50 I I -I - I
S I II -2.50 - 1.50 I! 5 2.50 1.67
P 2 ! I -0.50 1.50 II 1 I
---------------------------------------------------------------
2T ! I -2 -1 -5 I! 6 2 10.67

A computational implementation of the driving variable

parameter method would therefore need some special cross-check
on the solution-structure, i.e., whenever a corner solution ~s
developed (the number of elements of x which have entered
the basis, being equal to the number of elements of s which
have left the basis), one would replace all intersections of
primal variable's rows with dual variable's columns by exact
zeros.

Van de Panne does not, however, put the emphasis on this method
for computational reasons - he provides his proofs and theory
in terms of the symmetric method, and then shows that the a-
symmetric case is equivalent; we came the other way.

Another parametric solution method, developed independently

by van de Panne [35] , section 11.2, and by Goncalves [17]
might be called the artificial righthand-side method. In this
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 547

method, one superimposes arbitrary positive numbers on the

negative elements of the righthand side, and parametrically
moves from the artificial solution (which is therefore optimal
and feasible at the outset), to the specified right-hand side.

The same drawback as mentioned above for the driving variable

parameter method, also applies to this method.

17.7 Computational implementation of parametric QP

As in the LP case no full listing of the parametric vers~on

of the quadratic programmin8 procedure is given; this would not
be justified in view of the large overlap with the listing in
section 16.15.

We also observe that, although it was possible to generate some

example tableaux in section 17.6, the procedure offered here
does not cater for parametric solution methods as such.

We now give a listing of the file of editing instructions which

converts the procedure listed in section 16.15 into an
equivalent procedure which caters for parametric re-entry,
and a corresponding calling main programme.

TEXT-LISTING OF THE QUAD/QUAP EDITOR-FILE.

TC/'PROCEDURE' QUADC/,R/QUAD/QUAP/
T.?REENTRY?,I?PR,PC,???,T.?)?,I?,LAMBDA?
TC/' INTEGER'/
I?'VALUE' NNEGD;
?
T.?REENTRY?,I?PR,PC,?
T.E,I???'REAL' LAMBDA;?
TC/'BEGIN'/,TC?;?,T.?;?,I?,NAC?
TS/'REAL'/,T.?;?,I?,TWOT1
TS/'BOOLEAN'/,T.?CONVEX?,I1PARAMETRIC STEP,?

POST PARAM ST?

TS/' COMMENT' /
TI,PC/;/
I? QUADRATIC PROGRAMMING, WITH POSTOPTIMAL VARIATION.

FOR AN OUTLINE AND DESCRIPTION OF THE MAIN ALGORITHM,

SEE THE QUAD PROCEDURE.
THE DIFFERENCES BETWEEN QUAP AND QUAD ARE AS FOLLOWS:
548 CHAPTER XVII

PR AND PC STAND FOR PARAMETRIC ROW AND PARAMETRIC COLUMN.

FOR PC=I AND PR=O AN EXTRA COLUMN, THE PARAMETRIC COLt~N
IS INCLUDED.
FOR REENTRY=O, THE PARAMETRIC VARIABLE IS NOT INCLUDED IN
THE SEARCH OPERATIONS,
BUT A PARAMETRIC "STEP MAY BE MADE, BY SUBSEQUENTLY
ENTERING THE PROCEDURE WITH REENTRY=2.
FOR PREI AND PC=O, THE SAME APPLIES FOR THE PARAMETRIC
RESTRICTION, WHICH REPRESENTS PARAMETRIC VARIATION IN
THE LINEAR COMPONENT OF THE OBJECTI VE FUNTION.
FOR PR=PC=REENTRY=O, THE ACTION OF QUAP IS THE SAME AS
THE ACTION OF OF QUAD.

THE VARIABLE NAC IS AN AUXILIARY VARIABLE,

INDICATING THE NUMBER OF ACTIVE COLUMNS, WHICH ARE
TO BE INCLUDED IN THE SEARCH OPERATIONS.
1
TS/FANCYHIGH:=/
11 PARAMETRIC STEP := 'FALSE'; CONVEX:= 'TRUE';
POST PARAM ST := 'FALSE'; NAC:=N-PC;
?
TS/'IF' REENTRY=I/,T.?REENTRY?,I?'NOT' ?,R/I/O/
TS/QUADRATIC/,T.1RE-ENTRy?,I1NORMAL ?
TS/ADJUSTMENT/,R/./,/,TI,PC/;/,PI
I? AND AS TWO IF PARAMETRIC REENTRY IS ASKED FOR.

ENSURE CONVEX MODE IN DUAL PARAMATRIC LOOP:

'IF' REENTRY=2 'AND' PR=I 'THEN' NNEGD:=M+N+IJ
'IF' REENTRY=2 'THEN' PARAMETRIC STEP := 'TRUE';
'IF' REENTRY=I 'THEN' 'GOTO' START;
'I F' PR= I 'THEN' 'BEGIN'
MARK DUAL OF PARAMETRIC ROW:
BADN := 1000+M; COLN := -IOOO-M;
QUO : = I 0 0 0 0 0 0 0 0 0 0 0 0; RO WN : = 0 ;
K: .. B:=N+M;
ATTEND DUAL LA DERIVATIVE OF 2TI
TCN+M+ I.K): = 2*TC N+M+ I. KJJ
'GOTO' PREPARE INV OF MODE; 'END';

'I F' PC= I 'THEN' 'BEGIN'

MARK PARAMETRIC COLUMN VARIABLE:
BADN : = -N; COLN 1 = N;
QUO := 1000000000000; ROWN 1= 0;
D:=K:=B:=N;
ATTEND PRIMAL LA DERIVATIVE OF 2TI
TCM+N+I.K) := 2*TCN+M+I.KJJ

ACCEPT PRESET UPPER LIMIT:

'IF' TCN+M+2.K) > 0
'THEN' 'GOTO' PREPARE INV OF MODE;
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 549

SET UPPER LIMIT ON PARAMETERI

'IF' TCN.Nl>O 'AND' TCN.N+M+Il>O 'THEN' 'BEGIN'
TCN+M+2.Kl := I 000 000;
'GOTO' PREPARE INV OF MODE; 'END';
'I F' TCN.Nl < 0 'THEN'
~CN+M+2.Kl 1= TCN.N+M+Il/TCN.Nl -
SQRTCTCN.N+M+Il$TCN.N+M+ll-TCN.Nl.TCN+M+I.N+M+ll)
ITCN.Nl
'ELSE'
~CN+M+2.K11=0.S*TCN+M+I.N+M+Il/TCN.N+M+11;
'IF' TCN+M+2.Kl < 0 'THEN' TCN+M+2.K11=1000000;
'GOTO' PREPARE INV OF MODE; 'END';

?
TS/COMPLETE TABLEAUI/
I? INITIATE LAMBDA:
'IF' 'NOT' PARAMETRIC STEP 'THEN' LAMBDA := 0;

?
TS/SELECT BADN/
TI.I? 'IF' CPC=I 'AND' ROWLSTCN1=N) 'THEN' NAC:=N;
?
TS/'FOR' 11=/.1'1
I? 'FOR' 1:=1 'STEP' 'UNTIL' NAC.
N+ I 'STEP' I 'UNTIL' N+M-PR '00' 'BEGIN'

1
TS/ABS VAR ST/
11 'IF' PARAMETRIC STEP
'THEN' 'GOTO' END OF BADNAME SELECTION LOOP;

?
TC/END OF QUAD/
11 'IF' PARAMETRIC STEP
'THEN' 'GOTO' TRY THE BADNAME OR PAR ROW;

1
R/OF QUAD/OF QUAP/
TS/'FOR' II=NAV+I/
R/'UNTIL' N/'UNTIL' NAC/.TC/N+M/.R/N+M/N+M-PR/
TS/PHASE I SEARCHI/
TS/'FOR'I/.P2
11 'FOR' I:=NAV+I 'STEP' 'UNTIL' NAC.
N+I 'STEP' I 'UNTIL' N+M-PR 'DO'
1
TS/PHASE 1: /
TS/'FOR' JI=/.PI
11 'FOR' J:"I 'STEP' 'UNTIL' NAC.
N+I 'STEP' I 'UNTIL' N+M-PR 'DO'

?
TC/END OF QUAD;/.R/OF QUAD/OF QUAP/
TS/PHASE III/.T2
11 'IF' POST PARAM ST 'THEN' 'BEGIN'
'IF' PC=I 'THEN' D:aN 'ELSE' D:=N+M; 'END';
1
550 CHAPTER XVII

TC/TRY THE BADNAME/.R/BADNAME/BADNAME OR PAR/

TC/-O.OOO/
T. l ' THEN' 'BEGIN'?
I?'AND' 'NOT' TCB.N+M+IJ>O??1
I? 1

TS/SEEK SMALLEST QUo/.TS/ ',oR' 1/

PI.I? ',oR' 1:=1 'STEP' I 'UNTIL' NAC.
N+ I 'STEP' I 'UNTIL' N+M-PR 'DO'
?

TS/Do I NEED TO ELEMINATE: /

TS/'I,' BADN < 0 'THEN' 'GoTo' DoNE/
T.?'THEN'1.I1'AND' REENTRY < 2?
TS/UPDATE VECToRS:/
I? ATTEND 2T AND LAMBDA IN UB PARSTEP:
' I , ' PARAMETRIC STEP 'THEN' 'BEGIN'
' I , ' PC=I 'THEN'
TCN+M+I.N+M+IJ:=TCN+M+I.N+M+IJ+QUo*QUo*TCK.KJ
'ELSE'
TCN+M+I.N+M+IJ:=TCN+M+I.N+M+IJ-QUo*QUo*TCK.KJ;
LAMBDAI=LAMBDA+QUo; 'END';

1
TS/STANDARD ,0RM DOUBLE STEP:/
TC/' ,0R'/
TS/'I,' RoVN < 0/.T.1'THEN'?
I?'AND' 'NOT' (PR=I 'AND' Ro1JN=-3000-M)?P.E
T I. T.? T?
I 1 ' TH EN' ? T. E. I 1; ? • T I
R/' ELSE' / /
I ? ' I , ' RoWN>O 'AND' 'NOT' (PC=I 'AND' ROVN=2000+N)?
I?
'THEN'?
TS/'I,' 'NOT' ABS(/
11 ' I , ' PARAMETRIC STEP 'AND' PC=I 'AND' ROVN=2000+N
'THEN' 'GOTo' END 0, QUAP;

' I , ' PARAMETRIC STEP 'AND' PR=I 'AND' ROVN=-3000-M

'THEN' 'GOTo' END 0, QUAP;

1
TS/PERMUTE CoLUMNS:/
I? ATTEND PARAMETRIC REENTRY:

'I,' PARAMETRIC STEP 'THEN' 'BEGIN'

,I " PC= I 'TH EN '
TCN+M+I.N+M+IJ:=TCN+M+I.N+M+IJ+QUO*QUO*TCB.BJ
'ELSE'
TCN+M+I.N+M+IJ:=TCN+M+I.N+M+IJ-QUo*QUO*TCB.BJ;
LAMBDAI=LAMBDA+QUO;

PARSTEP BY VECTORS ONLY:

' I , ' (ABS(ROWN) < 2000 'AND' 'NOT' ROVN=BADN
'AND' TCR.RJ < -0.000001 )
'OR' (RoVN>O 'AND' ROVN=R01JLSTCRJ)
'THEN' 'BEGIN'
',oR' 1:=1 'STEP' I 'UNTIL' N+M+I 'DO'
' I , ' 'NOT' TCI.KJ=O 'THEN' 'BEGIN'
TCI.N+M+IJ:=TCI.N+M+IJ-TCI.KJ*QUo;
TC 1. N+M+2J: =TC I .N+M+2 J +TC I. KJ* QUO; 'END';
TCR.N+M+IJ := -0.000000001;
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 551

'FOR' J:=I 'STEP' 'UNTIL' N+M 'DO'

'IF' ROTJLSTCJl < 0 'THEN' TCN+M+I,Jl:=TCJ,N+M+Il
'ELSE' TCN+M+I,Jl:=-TCJ,N+M+Il;

PARAMETRIC STEP 1= 'FALSE'; REENTRY : = I;

TCN+M+2,Kl := TCN+M+2,Kl - QUO;
BADN:=ROWN; B:=R;
'IF' BADN>O 'AND' TCB,Bl > -0.000001 'THEN' 'BEGIN'
TCB,Bl:=O; 'GOTO' PHASE I ; 'END';
'GOTO' PHASE II; 'END';
'END' ;

?
TS/CHECK FOR STATUS:I
TI
I?
'IF' PARAMETRIC STEP 'AND' PC=I 'AND' ROWN=-N
'THEN' 'BEGIN'
TCN,N+M+Il:=-O.OOOOOOI; BADN:=N; COLN:=-N; K:=R:=B:=N;
PARAMETRIC STEP := 'FALSE';
'GOTO' TRY THE BADNAME OR PAR ROW; 'END:;

TERMINATE PARAMETRIC REENTRY:

'IF' (PC=I 'AND' ROWN=N) 'OR' (PR=I 'AND' ROWN=-IOOO-M)
'THEN' 'GOTO' END OF QUAP;
?
TS/ATTEND PAIR IN NON CONVEX MODE:I
TS/'IF' 'NOT' CONVEX MODE/,T.? 'THEN' 'BEGIN '?
I?
'AND' 'NOT' PARAMETRIC STEP?
TS/PREPARE NON STANDARD FORM STEP:I,TI
I?
'IF' PARAMETRIC STEP 'THEN' 'BEGIN'
'FOR' J:=I 'STEP' I 'UNTIL' N+M+2 'DO' 'BEGIN'
COP:=TCB,J); TCB,Jl:=TCR,JH TCR,Jl:=COP; 'END';
NAME: = ROWLSTC RlJ ROTJLSTC Rl: = ROWLSTC B); RO \JLSTC B 1 : =NAME;
PARAMETRIC STEP := 'FALSE';
POST PARAM ST := 'TRUE';
TWOT 1= TCN+M+I,N+M+Il;
REENTRY: = I; BADN: =COLN; B:=R;
TCR,N+M+Ill=-O.OOOOOOI; 'END';
?
TC/END OF QUAD:I,R/END OF QUAD/END OF QUAP/,Tl
TC/FINAL END OF QUADI
I? 'FOR' 1:=1 'STEP' I 'UNTIL' N+M 'DO'
'IF' ROWLSTCI)<O 'THEN' T[N+M+I,Il:=T[I,N+M+ll
'ELSE' TCN+M+I,Il:=-T[I,N+M+I);
'IF' PR=I 'OR' PC=I 'THEN' 'BEGIN'
'IF' POST PARAM ST 'THEN' T[M+N+I,N+M+Il:=TWOT;
• END';

?
R/END OF QUAD/END OF QUAPI
TE,E
552 CHAPTER XVII

TEXT-LISTING OF THE PARAMETRIC Qp MAIN PROGRAMME.

'BEGIN' 'INTEGER' M.N. NAV.NEQ. REENTRY. I .J. PRINT. pR. PC.

NNEGD; 'REAL' FANCYHIGH.INITIAL VALUE. LAMBDA;
'BOOLEAN' TRULY FEASIBLE. TRULY BOUNDED;
'PROCEDURE' QUAp(T.M.N.NEQ.NAV.NNEGD.ROWLST.COLLST.pR.PC.
REENTRY. LAMBDA) J
'ARRAY' T; 'INTEGER' M.N.NEQ.NAV.NNEGD.pR.pC.REENTRY;
'REAL' LAMBDA;
'INTEGER' 'ARRAY' ROWLST. COLLSTl
'ALGOL' ;

'PROCEDURE' MATI(MATR.MB.NB.FR.FC);
'ARRAY' MATR; 'INTEGER' MB.NB.FR.FC; 'ALGOL' ;

'PROCEDURE' TABO(MATR.M.N.SR.SC.ER.PR.ROWLST.COLLST);
'ARRAY' MATR; 'INTEGER' M.N.SR.SC.ER.RH;
'INTEGER' 'ARRAY' ROWLST.COLLST; 'ALGOL';
'PROCEDURE' REpQ(T.M.N.NEQ.NAV.ROWL.COLL);
'ARRAY' Tl 'INTEGER' M.N.NEQ.NAV;
'INTEGER' 'ARRAY' ROWL.COLLJ 'ALGOL';

'COMMENT'
SIMPLEX ALGORITHM FOR QUADRATIC PROGRAMMING.
WITH pOSTOpTIMAL PARAMETRIC VARIATION.

FOR DETAILS OF THE ALGORITHM.

SEE THE TEXT OF THE PROCEDURE QUAI'.
PRESENTATION OF DATAl
FIRST THE NUMBER OF RESTRICTIONS I.E. M.
IF PARAMETRIC VARIATION OF THE OBJECTIVE FUNCTION
IS INTENDED. A PARAMETRIC ROW SHOULD BE INCLUDED AS A DUMMY-
RESTRICTION. BOTH IN THE NUMBER OF RESTRICTIONS AND IN THE
COEFFICIENTS MATRIX. AS THE LAST RESTRICTION.
THE NEXT NUMBER TO BE OFFERED IS THE NUMBER OF VARIABLES. N.
IF PARAMETRIC VARIATION OF THE RIGHTHAND-SIDE IS·INTENDED.
A DUMMY-ACTIVITY SHOULD BE INCLUDED IN THE NUMBER OF VA-
RIABLES
AND -AS LAST COLUMN-. IN THE COEFFICIENTS MATRIX.
THESE NUMBERS SHOULD BE FOLLOWED BY
THE NUMBER OF EQUATIONS. NEQ.
AND NAV. THE NUMBER OF VARIABLES TO WHICH
THE TACIT (NON-NEGATIVITY) RESTRICTION DOES NOT APPLY.
THE SERIES OF INFROMATION-PARAMETERS IS THEN CLOSED BY
THE INTEGER NUMBERS pR AND PC.
FOR PR"PC= D.
THE PROGRAM OPERATES AS. A 'NORMAL' QUADRATIC PROGRAMMING
ALGORITHM.
FOR pR~1 WITH PC-D. PARAMETRIC VARIATION OF THE OBJECTIVE
FUNCTION IS ASKED.
AND FOR pR=O WITH pC"I.
PARAMETRIC VARIATION OF THE RIGHT-HAND SIDE
IS ASKED.

BEFORE THE MAIN BODY OF NUMERICAL INFORMATION.

ONE SHOUD SUPPLY A SINGLE REAL NUMBER.
THE CONSTANT TO BE ADDED AS INITIAL VALUE TO
THE OBJECTIVE FUNCTION.
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 553

THEREAFTER PUNCH THE TABLEAU ITSELF.

THE REQUIRED PRESENTATION OF THE TABLEAU-MATRIX
IS THE SAME AS FOR 'ORDINARY' QUADRATIC PROGRAMMINGI
EXCEPT FOR THE INCLUSION OF AN ADDITIONAL VECTORI TO
CONTAIN THE PARAMETRIC ACTIVITY/RESTRICTION.

FANCYHI GH I'" 100;

MI=READ; NI=READ; NEQI=READJ NAV:=READ;
NNEGDI= READ; PRI=READJ PCI=READJ
INITIAL VALUE 1= READ;
REENTRY: = 0;
'BEGIN' 'ARRAY' TAC I:M+N+311:M+N+2lJ
'INTEGER' 'ARRAY' R01olLSTICOLLSTC\:M+NH
MATI(TAIM+N+2IN+II0 1 0);
PRINTI=TACM+N+IIN+Il;

REORDER TO APPROPRIATE BLOCKS:

'FOR' J:=I 'STEP' 1 'UNTIL' N 'DO'
'FOR' 11=21\ 'DO'
TACM+N+I+IIJllcTACM+N+IIJ];
'FOR' 11=1 'STEP' 1 'UNTIL' M+N+\ 'DO' 'BEGIN'
TA[lIM+N+Ill=TACIIN+IH TAClIN+'ll:=O; 'END';
TACM+N+IIN+Ill=O;
RE INTERPRET:
'FOR' J:=I 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'
'FOR' 1:=1 'STEP' 1 'UNTIL' N+M+\ 'DO'
TACIIN+M+\l:=TACIIN+M+Il~TACIIJl*TACN+M+3IJl;
'FOR' 11=1 'STEP' 1 'UNTIL' N 'DO'
TACN+M+IIN+M+Il:=
TACN+M+IIN+M+ll+TACIIJl*TACJIIl*TACN+M+3.Jl;
TACN+M+IIN+M+Ill=
TACN+M+IIN+M+Il-TACN+M+\IJl*TACN+M+3.Jl;
'IF' TACN+M+2IJl=0 'THEN' TACN+M+2IJl:=FANCYHIGH;
TACN+M+2IJl 1= TACN+M+2IJl-TACN+M+3.J];
'IF' TACN+M+2IJl < 0 'THEN' 'BEGIN'
NEWL IN E( 1 )J
VRITETEXT('C'YOU%HAVE%SUPPLIED%A%LOVER%LIMIT%IN%
EXCESS%OF%THE%CORRESPONDING%UPPER%LIMIT') ');
NEWLINECI )J
VRITETEXT( '( 'THE%QP%PROBLEMUS%THEREFORE%EMPTY. ') ');
• END';
• END'J

NOW SOLVE:
FIRST CALL:
QUAP(TAIMININEQINAVINNEGDIROWLST.COLLSTI
PR.PC.REENTRYILAMBDA);

CORRECT 2T FOR INITIAL VALUE:

TACM+N+IIM+N+Il := TACM+N+\IM+N+Il + 2*INITIAL VALUE;

'IF' NNEGD > N 'THEN' NNEGD := N;

554 CHAPTER XVII

POINT OF OUTPUT:
CHECK ON STANDARD FORM:
'FOR' 1:=1 'STEP' I 'UNTIL' N+M '00'
'IF' 'NOT' RoWLSTClJ=-CoLLSTCIJ 'THEN' 'BEGIN'
TRULY FEASIBLE := 'FALSE';
'GoTo' SIGNAL UPPER LIMITS; 'END';

'IF' REENTRY=O 'THEN' 'BEGIN'

CORRECT 2T ROW:
'FOR' 1:=1 'STEP' I 'UNTIL' M+N 'DO' 'IF' RoWLSTCIJ>O
'THEN' TACM+N+I.IJI=-TACI.M+N+IJ
'ELSE' TACM+N+I.IJ:=TACI.M+N+IH 'END';

'IF' 'NOT' CPR=O 'AND' PC=O) 'THEN' 'BEGIN'

NEWLINEC I);
WRITETEXTC'('PARAMETER %VALUE') ');
WRITEC30.FoRMATC 'c 'S-NDDDDD.DD') '>'LAMBDA); 'END';

CHECK FOR ARTIFICIAL FEASIBLITY:

TRULY FEASIBLE := 'TRUE';
, I'tJ R' 1: = I ' STEP' I ' UN TIL' M+ N ' DO' , 1 F' Ro WL S T C1 J
> 3000 'THEN' 'BEGIN'
TRULY FEASIBLE 1= 'FALSE';
NEWLI NEC I);
WRITETEXTC'C'RESTRICTIoN') ');
WRI TEC 30. FoRMATC 'C 'SNDDDDS') '). Ro"!LSTC 1 J -30 0 0);
WRI TETEXTC 'c 'oNLY%ARTI F! CI ALLY%SATI SF! ED') '); 'END';

DITTO FOR SPECIFIED VARIABLES:

'FOR' 1:=1 'STEP' I 'UNTIL' N+M 'DO'
'IF' RoWLSTCIJ>2000 'AND' RoWLSTClJ<3000 'THEN' 'BEGIN'
TRULY FEASIBLE := 'FALSE';
NEWLINEC I)J
WRITETEXTC'C 'VARIABLE') ');
WRITEC 30. FORMAT<' C 'SNDDDDS')'). RoWLSTC I J-2000);
WRI TETEXTC 'c 'oNLY%ARTI F! CI ALLY%POSI TI VE') , ); 'END';

SIGNAL UPPER LIMITS:

TRULY BOUNDED := 'TRUE';
'FOR' J:=I 'STEP' I 'UNTIL' M+N 'DO' 'IF' COLLST[JJ
> 2000 'AND' COLLSTCJJ < 3000 'THEN' 'BEGIN'
'IF' TA[M+N+2.JJ = FANCYHIGH
'THEN' TRULY BOUNDED := 'FPLSE';
'IF' CoLLST[JJ = 2000+N 'THEN' TRULY BOUNDED := 'FALSE';
'END' ;

REP!: CTA. M. N. NEQ. NAV. RO!,,'!. ST. COLL ST);

'IF' PRINT> 0 'OR'
M+N < 14 'THEN' TAEoCTA.M+N.M+N.O.O. I. I.RoWLST.COLLST)
'ELSE' TABoCTA.M+N.O.O.M+N.O.I.ROWLST.COLLST);

NE!,,'LINEC I)J
WRI TETEXT C ' C 'SOL UTI ON% VALUEU FU N%STAN DAPD% FORM') ');
WRI TE( 30. FORMAT( , C 'S-NDDDD. DDD' ) • ). TA[ M+N+ I. M+N+ 1 ] 12);
NEWLINEC I);
PARAMETRIC METHODS IN QUADRATIC PROGRAMMING 555

ENDLOOP FOR NORMAL QUADRATIC PROGRAMMING:

'IF' PF=O 'AND' PC=O 'THEN' 'GOTo' END OF PAFQP;

ENDLOOP FOR PARAMETRIC VARIATION:

'IF' REENTRY=-I 'THEN' 'GOTO' END OF PARQP;
'I F' 'NOT' TRULY FEASI BLE 'OR' 'NOT' TRULY BOUNDED
'THEN' 'GOTO' END OF PAPQP;

CHECK POSITIVE VALUE QUADRATIC FUNCTION:

, IF' PC= 1 'AND'
'NOT' TACM+N+I.M+N+Il > 0.0000000001 'THEN' 'BEGIN'
NEWLINEC I)J
WRITEiEXTC'C'QUADRATIC%FUNCION%ZERO%OR%NEGATIVE%') ');
NE1,.'L I NEC I);
'GOTO' END OF PARQP; 'END';

PARAMETRIC REENTRY CALL:

REENTRY: =2;
QUAPCTA,M.N.NEQ.NAV.NNEGD.ROWLST.COLLST.
PR.PC.REENTRY.LAMBDA);
'GOTO' POINT OF OUTPUT;

END OF PARQP:
'END'; 'END'
CHAPTER XVII I

GENERAL QUADRATIC PROGRAMMING

18.1 Statement and discussion of the general problem

In this chapter we discuss a method of solving the following

problem

Maximise

T w' x + x' D x (18.1.1)

<r-

Subj ect to

(18.1.2)

(k = 1, 2, ... m)

In practical numerical application it is extremely wasteful of

both tableau-space and computational effort to treat linear
restrictions as quadratic restrictions with a zero quadratic
component. Thus, in practice (18.1.2) is split in

c' x < b (18.1. 2a)

- k- - k
(k = 1, 2, .... , ml )

and

c' x - 1 x'D x < b (18.1. 2b)

- k- 2 - k - - k

(k = ml+l, ... m)

The objective function is similar to the one used in Chapter 16,

but now the side-conditions are also quadratic.

Example

Maximise

Subject to

(xl - 2)2 + (x 2 - 3)2 ~ 4

(0 ~ xl ~ 10, 0 ~ x 2 ~ 10)

556
GENERAL QUADRATIC PROGRAMMING 557

We first of all make a graphical mapping of the stated problem,

with deletion of the upper limits. The one restriction is a
circle, with a radius of 2, and the point xl = 2, x 2 3 as
centre. The objective function is indicated by a series of
parabolae (in fact, three) for T = 0 and T = 1, and T = 3.1.
The highest one of these, insofar as it still touches the
feasible space area (the circle) is to be chosen. In this
example, that is also visually the highest but more generally,
we should choose the highest value of T. To conform to (18.1.1)
and (18.1.2) it is necessary to reformulate the objective
function as well as the restriction, i.e. work out the
expressions within brackets.

Maximise

T =
2
xl + 2x2 - x 2 - 1

Subject to

i.e.
2 2
-4x - 6x + x + x <-9
1 2 1 2 -
Thus our initial set-up information is

w'

c'
[1

[-4 -6J
2J, D
0

Dl
[: J
-1
[-' OJ
o -2
bl -9
558 CHAPTER XVIII

graph 18.1
5.5 illustration of the general
quadratic programming problem:
5.0
find the highest fe~sibl~
iso-objective functIon lrne.
~.5

4.0

3.5

3.0

2.5

2.0

4.5 S.O 5 5

While this example is neatly convex, it is important to note

that the efficacy of the van de Panne and Whinston/Cottle
algorithm (on which this chapter heavily depends) does not
require a convex preference function as such.

By and large, it is enough that each ordinary quadratic

programming problem which will be generated possesses a proper
optimum, and this requires convexity within the subspace defined
by the binding linear restrictions, not convexity of the
objective function as such. The algorithm to be developed in
this chapter will therefore also be effective in many cases
where some or all of the matrices D and Dk are indefinite,
rather than negative (semi) definite.

Furthermore, it will be seen below, that with only a slight

adaptation of the subsidiary quadratic programming algorithm
we can in practice solve problems in which some of the
restrictions are peripherally convex, or indeed anti-convex.
GENERAL QUADRATIC PROGRAMMING 559

lS.2 Pseudo-Lagrange an and Linear Approximation

The general quadratic programming algorithm revolves around

the maximisation of a function which we might indicate as the
pseudo-Lagrangian.

When the algorithm has converged the pseudo-Lagrangean is the

Lagrangean function, less the terms which refer to the linear
restrictions.

We do not so far know the values of any dual variables, and

cannot directly verify whether the first-order optimality
conditions are satisfied.

We therefore start with some provisional guesses at what the

dual variables might be.

The pseudo-Lagrangean is the objective function, plus a non-

negative combination of the quadratic restricting functions.
As in the case of a proper Lagrangean, positive multipliers
are restricted to restrictions which have been identified
as possibly binding. In practice this means either violated
or binding.

In the initial (trivial) solution we do not as yet identify

any restriction as binding or violated and the pseudo-
Lagrangean is the objective function.

The algorithm consists of, inter alia, a number of optimising

phases. In each optimising phase we solve an "ordinary"
quadratic programming problem, which we will indicate as a
subsidiary problem. In this subsidiary "ordinary" quadratic
programming problem, the objective function is the pseudo-
Lagr angean .

The side conditions are 3 categories of linear restrictions,

v~z:

a) Any linear restrictions arising from (lS.1.2a)

b) Linear approximations of quadratic side-conditions, and

c) Sometimes a "safety restriction" the nature of which will

be discussed later on.

The presence of these additional linear restrictions (classes b

and c) distinguishes the algorithm which is offered here from the
group of algorithms surveyed by Fiacco and
560 CHAPTER XVIII

McCormick [lOJ. Their algorithms are called sequential

unconstrained minization techniques. By contrast the present
algorithm might be called a sequentially constrained maxi-
mization method. We should, however, also mention a different
method due to M.J.D. Powell [31].which is rather more similar
to the one offered here. Although the programmed algorithm
technically caters for "free" variables, this feature of the
algorithm has not been tested, and indeed could give rise
to boundedness problems.

Our discussion of the algorithm will assume that restrictions-

class a) includes non-negativities and upper limits on the
specified variables.

The "safety restriction" c) may not always be present (for a

start it is not present in the initial problem) and the
linear approximations may be totally inadequate initially.

To prevent unboundedness in the case of a semi-definite (or

linear) objective function, we need to ensure that the feasible
space area is bounded even without the quadratic restrictions.

We nm. discuss the linear approximations. Any linear approxi-

mation refers to vectors in the vicinity of some previously
solved vector x = x*. The initial linear approximation has to
be supplied by-the-user, one would normally initiate at x* = O.
The actual value of x is then the sum of x* and an increment,
indicated as 6x.

x = x* + 6x (18.2.1)

Substitution of the right-hand side of (18.2.1) for x into

(18.1.2) yields the following expression

~k(~* + ~~) - !(~*' + ~~')Dk(~* + ~~) < b k (18.2.2)

Partial working out of (18.2.2) gives us:

~(~* + ~x) - ~*'Dk~~ - !~*'Dk~* - !~~'Dk~x ~ b k

(18.2.3)
GENERAL QUADRATIC PROGRAMMING 561

We add a further term -~~*'D~* to both sides of (18.2.3) causing

the term -~~*'Dk~* on the left-hand side to become a full term
-~*'Dk~*' and bring the term ~6~'Dk6~ to the other side.

We again write x for x* + 6~, and obtain after re-ordering:

(~ - ~*'Dk)~ ~ b k - ~~*'Dk~* + ~6~'Dk6~ (18.2.4)

The approximation consists in neglecting the term ~6~'Dk6~,

and the approximation is

(18.2.5)

Thus, in our example, our first approximation 1S

and in general, for x* 0, the initial approximation is

(18.2.6)

For a properly convex quadratic restricting function Dk is

negative semi-definite, hence the term ~ ~'Dk6~ is non-
positive. But if the function is non-convex (or directionally
convex), 6X'Dk6X may be positive. Thus for a convex restricting
function the linear approximation is more liberal than the true
quadratic restriction. The discrepancy between the true
restriction and its approximation may go as far as finding
the approximation not to be a meaningful restriction on the
objective function at all, as may be illustrated in graph l8.2a
below.

If the restriction is not convex, or only peripherally convex

in the ~ ~ 0 domain, the linear approximation may actually
cut into the feasible space area. This may lead to complications.
The problem may appear to be empty, despite the existence of a
proper optimal and feasible solution in the true quadratic
problem.

Example

Maximise
2 2
T = -(xl - 2) - (x 2 -3)

Subject to
562 CHAPTER XVIII

5.0;

TX,
graph 18.2 a
5.0
inadequate initial approximation
4.5

4.0

3.5

3.0

This problem has a proper solution, the unconstrained maximum

of the objective function, xl = 2, x 2 = 3. However, the linear
approximation of the hyperbola, taken at the point xl = x 2 = 0,
is much more restrictive than the true non-linear
restriction, it is 0 xl + 0 x 2 2 -4.

18.3 Verification and primal adjustment of subsidiary optima

When a subsidiary quadratic programming problem of maximising

a pseudo-Lagrangean has been dealt with in the first instance,
the linear approximations of non-convex restrictions may need
adjustment. It should be borne in mind that when using the
convex mode of operation the ordinary quadratic programming
algorithm will corne up with a "solution" irrespective of
whether the problem is intrinsically solvable or not. For
example, the first subsidiary optimisation problem tor the
quasi-convex example at the end of the previous section is

Maximise
2 2
x - x 2 (- l3)
1
GENERAL QUADRATIC PROGRAMMING 563

subject to

o < - 4

and the "solution" is xl = 2, x 2 = 3, with the dual variable

PI at its fancy high upper limit, the slack-variable of the one
restriction having been replaced by an artificial variable. The
fancyhigh dual variable is not, however, particularly informative.
In this somewhat odd and trivial example, the adjustment
substantially amounts to the neutralisation of a nonsense-
restriction. The "adjustment" will change the "restriction"

o < -4 into 0 < o.

We will move a linear approximation restriction outward,
whenever we find one binding, (or artifitially satisfied), while
the slack of the true quadratic restriction turns out to be
positive. The operation need not be as apparently simple and
trivial as in the example given above.

Consider the following example,

Maximise

T = -(x 1 - 1)2 - (x
2
- 2)2

Subject to

-xl - x 2 - 2xl . x 2 ~ -4

i.e. 2(x l + !)(x 2 + !) ~ 4!

The subsidiary optimization problem, i.e.

Maximise
2 2
T = -(xl - 1) - (x 2 - 2)

Subject to

is now first of all solved.

The set-up tableau and the subsidiary optimum tableau are given
below in the tableaux·18.3a and l8.3b.
 564 CHAPTER XVIII

graph 18.3 a

overtight approximation of a
per iphera LLy convex restr i ct ion:
the ?pti~um may be between the ap-
proximation and the true restriction.

2.0

1. S _

1.0

0.5

0.5 1 0 1.5 2.0 S.O

TABLEAUX 18.3 A AND 18.3 B

A SUBSIDIARY PROBLEM WITH AN OVERTIGHT OPPROXIMATION.

SET-UP TABLEAU SUBSIDIARY OPTIMUM
NA.! XI X2 PI !VAL. NA.I DI D2 SI IVAL.
DI I -2 -2 XI -0.25 0.25 -0.5 I 1.5
02 I -2 ! -4 X2 I 0.25-0.25-0.512.5
S 1 I -I -I ! -4 PI ! 0.5 0.5 -I I I
2T ! -2 -4 4 ! -10 2T I -1.5 -2.5 I-I
GENERAL QUADRATIC PROGRAMMING 565

We are now in a position to give precise operational

significance to the notion of pushing the linear approximation
restriction outwards. It means parametrically increasing the
constant of the sl-restriction, until the quadratic restriction
becomes binding rather than amply fulfilled.

The sl-column tells us that for every unit that the restriction
is shifted outwards (back to the origin in this case), xl and
x 2 will each be reduced by!.

In other words, if A is the length of a parametric step in

the indicated direction, we can put:

lIx l = -! it.. , lIx 2 = -! A.

Confirm the notation of section 17.2 (parametric variation), we

indicate a variation vector as v , but in this case v is an
updated vector, i.e.

x x* + VA (18.3.1)

and

lIx v A (18.3.2)

xl I! - !A

x2 2! - !A

in the particular example.

We first implement the approximation-formula (18.2.5) at the

current solution vector, and then proceed as follows:

Substitution of the right-hand side of (18.3.1), i.e. x* + AV

for x, and of AV, the right-hand side of (18.3.2) for lIx,
into-a binding form of (18.2.4) gives the following result:
2
(~ - ~*'Dk)(~* + ~A) = b k - !~*'Dk~* + !~'Dk~A (18.3.3)

which may be re-ordered as

-c'x* +
-=k-
~x*'D
-
x*
k-
=0 (18.3.4)

A numerical evaluation of (18.3.4) is by far the most easily

handled, if the vector [~ - ~*'Dk]' and the quadratic forms
566 CHAPTER XVIII

~'D v and ~*'D~* are calculated first, hence the implementation

of ~8.2.5) prior to (18.3.4).

To"represent the term 2x l x 2 as !~'D~, we need D

and we calculate:

(c' - x*'D )
-k - k [-1 -1] - [I! 2!J [: :] [-6

The quadratic form ~'D~~ is evaluated as

H -I] [: :] [~:] " 1

and similarly

x*'D x*
- k- [I! 2!J
~ :] [::] = 15.

Accordingly, (18.3.4) is evaluated as

1;\2
2 - [-6 -4J
C~] A - 4 - [-1 -lJ
G!J + n 0

or

1A2
2 - 5A - 4 + 4 + n 0

or

We now solve A as

;\ 5 ± 125 - 15 = 5 ± /DO = 5 ± 3.162

8.162,

Both roots correspond to an exact solution of the binding form

of the non-linear restriction. For the larger of the two roots,
we find:

Xl = 1. 50 - 4.08 -2.58

and

2.50 - 4.08 -1.58

GENERAL QUADRATIC PROGRAMMING 567

This point neatly satisfies the specified restriction - but

not in the x > 0 domain! When two positive roots are solved from
(18.3.4) the smaller of the two is applicable.

If a restriction is non-convex rather than peripherally convex

in the x > 0 domain, a second positive root may also refer to a
point in the x ~ 0 domain. We then obviously take the one with
the lowest absolute value.

The other case which may quite well arise in connection with a
peripherally convex restriction is that we find a positive and
a negative root. For example, if the restriction

is added to the previous restriction, the example becomes

Maximise
2 2
T = -(xl - 1) - (x 2 - 2)

Subject to

- xl - x 2 - 2x l x 2 ~ -4

The first subsidiary problem is in that case the following:

Maximise

T = -(x - 1)2 - (x - 2)2

1 2
Subject to

and its optimal solution is

Parametric variation of the constant of the second restriction

now means to slide laterally along the binding restriction
568 CHAPTER XVIII

j Xl + x 2 = 2. This line actually cuts the quadratic

restriction at two points, on account of the curvature of the
quadratic restriction. Our underlying problem is that possibly
the solution might be in the area which is satisfied by the true
quadratic restriction, but not by the linear approximation.
(This has been illustrated in graph l8.3a.) - Therefore, we want
to relax the linear approximation, not to make it a more
stringent restriction. This means algebraically that when one
of the roots is negative and one positive, the positive root is
the relevant one; see also graph l8.3b.

The roots of (18.3.4) may also be complex and their real parts
may be positive or negative. Complex roots with a negative
real part though possible, are not very useful, and one can in
such cases, only hope that after a fresh start, new, more
satisfactory approximations will avoid the problem. The case of
a pair of complex roots with a positive real part is, however,
useful to persue.

Example

Maximise T = -(x 1 - 3)2 _ (x - 2)2

1
Subject to 2xl x2 < 0
2
xl - x 2 + 2x2 < 2

It will be noted that the quadratic restriction in this

example is non-covex, and indeed, anticonvex. This problem,
nevertheless, has a unique optimal and feasible solution. The
quadratic restriction is not even binding on the optimum, and
in any case, even if it was e.g. if the linear restriction
2xl - x 2 ~ 0 were left out, the strict convexity of the
objective function would still ensure a unique optimum. The
initial linear approximation of the quadratic restriction is
xl + 2x 2 ~ 2 and the first subsidiary optimum is
3 3
xl = 8' x 2 = 4 . The optimal quadratic programming tableau
corresponding to the maximization of -(X l -3)2_(X 2-l)2 +
10 = -xi + 6x l - x; + 2x 2 , subject to 2x l - x 2 ~ 0 and
xl + 2x2 ~ 2 is given below, with the parametric column and the
corresponding displaced solution. (Tableau l8.3c). Non-
meaningful upper limits, as well as parametric variations in the
objective function have been replaced by xx.
GENERAL QUADRATIC PROGRAMMING 569

7
graph 18.3 b

illustration 0 f outward

-
6 adj ustment I
a positIve and a negative root.

0.5 1.0 1.5 '.0 4.5 5.0 5.5

570 CHAPTER XVIII

TABLEAU IB.3 C

A SUBSIDIARY OPTIMUM. WITH PARAMETRIC ADJUSTMENT.

BASED ON A PAIR OF COMPLEX ROOTS OF THE LIMIT EQUATION.

NAME! ! D 1 D 2 S 1 S 2 !! VALUE L VACL"'I.13>

-------------------------------------------------------------
X 1 ! I 0.40 0.20 ! I 0.40 0.20 0.63
X 2 ! ! -0.20 0.40 ! ! 0.80 0.40 1.25
P 1 ! ! -0.40 0.20 -0.40 ! I 1.60 1.60
P 2 II -0.20 -0.40 -0.40 ! I 2 -0.40 1.55

2T !! -0.40 -0.80 1 .60 2 ! I 9.60 XX xx

UB !! XX XX XX XX ! I XX 1.13 xx

Relaxation of the binding s2-restriction now means that for

every unit that the s2-restriction is pushed outwards, xl

increases by t and x 2 by i· This is due to movine along the

binding linear restriction, 2xl - x 2 o. Inspection of

· restr~ct~on
graph l8.3c indicates that t h ~s 2.. never meets t h e
quadratic restriction.

Application of (18.3.4) for:

[~l
v

c'
-k 2J

x* bk 2

[;]
yields
~ ;\2 9 ;\ + 16
0
25 25 25
GENERAL QUADRATIC PROGRAMMING 571

This equation yields a pair of complex roots

9
8 ± 8
50

~. s
graph 18.3 c

, slid i ng'
l.O
the true res-
trictIon may
never be
reached.

\ S

1.0

OS ·

-0 S 1. 5 2 0 2. 5 ; .0
572 CHAPTER XVIII

3.5
graph 16.3 d

adjusted old
approximation,
3.0 cutting true
restriction
at an angle.

I. 5 -

1.0

-0.5 0.5 1.0 1.5 2.0 2 5 3 0

GENERAL QUADRATIC PROGRAMMING 573

The algebraic equivalent of this situation is finding a pair

of complex roots to (18.3.4).
The problem is solvable nevertheless.

When we meet a pair of complex roots in this way we take the

real part only or what amounts to the same, we put the first-
order differential of the left-hand side of (18.3.4) at zero

(c - x*'D )v
-k - k-
(18.3.5)
v'D v
- k-

We then subsequently take a nrw lin~ar approximation and repeat

the operation. Thus for A = 18 we f:tnd x = 0.63, x 2 = 1. 25.
The new graphical mapping below gives both the adjusted linear
approximation and the new one, taken at xl = 0.63 and x 2 = 1.25.

The new linear approximation mayor may not come to be used at

that point. The reason for this qualification is that the
calculated value of A, whether obtained as a real root or as the
real part of a pair of complex roots is a maximum value. It
mayor may not happen that the ordinary quadratic programming
vertex changes before the full adjustment is reached. There
may be another restriction in between, or the adjusted solution
may not be optimal. The parametric adjustment algorithm of
Chapter 17 is applicable, and only one parametric step is made.
In the example at hand, the adjusted vertex xl = 0.63 and
x 2 = 1.25 is the new subsidiary optimum. Accordingly, the next
subsidiary problem is

Maximise

Subject to

2xl - x 2 ~ 0

xl - 0.50x 2 < 0.44

The optimum solution of this problem is

xl = 1.4 x2 = 2.8 with PI = 1.6, P2 = 0

The quadratic restriction, as well as its approximation are

now amply fufillled and no dual variable has so far been
introduced into the pseudo Lagrangean. Hence this particular
subsidiary optimum is actually the true global optimal and
feasible solution. The problem at hand has been solved.
574 CHAPTER XVIII

One related issue which comes up under the heading of verifying

the solution of a subsidiary problem is the recognition of
emptiness.

The obvious way of recognising an empty problem is by way of

finqing a subsidiary problem empty. The following statements
apply in that connection.

If a restricting function is properly convex the set of vectors

which satisfy the corresponding true restriction is entirely
included in the set of vectors which satisfy an approximation
(the approximation is more liberal than the true restriction).

Therefore, in the convex case, the feasible space area of the

true problem is entirely included in the feasible space area of
any subsidiary problem and we may conclude to emptiness of the
true problem, as soon as we find a subsidiary problem to be
empty.

Note however, that this indication does not always arise in

the first subsidiary problem.

Example
2
Maximise T = xl + (x 2 -1)

Subject to xl > 5

(x - 2)2 + (x - 3) 2 .2. 4
1 2
(0 < x < 10, a < x 2 .2. 10)
- 1-
This problem is a modification of the one put at the opening
of this chapter, the modification consisting of the introduction
of the linear restriction xl ~ 5, making the problem empty.

Whilst the linear restriction contradicts the true quadratic

restriction, it does not contradict the initial approximation
of the quadratic restriction (4xl + 6x 2 ~ 9), and the initial
subsidiary problem attains an optimal and feasible solution at
xl = 10, x 2 = 1.

For the convex case, convergence of the sequentially constrained

maximization method arises on account of progressive reduction
of the attainable value of the objective function, a topic to
be discussed in more detail later in this chapter.

The algorithm has two permitted endings, finding the optimum,

and finding the true problem empty, by way of finding a
subsidiary problem empty. (The issue of unboundedness does not
 GENERAL QUADRATIC PROGRAMMING 575

arise on account of the presence of (artificial) upper limits on

all specified primal variables). Finding the optimum of the
true problem is impossible for an empty problem, it therefore
follo~s that emptiness will be signalled in the way indicated,
if not for the initial subsidiary problem then for a later
subsidiary problem.

There are, however, complications for non-convex problems or

indeed, as soon as not all functions are properly convex. It is
in this connection necessary to differentiate between true linear
restrictions and linear approximations. This was in fact, done
by employing the convex mode of operation in solving subsidiary
problems including and indeed in particular with non-convex
problems.

The method of dual upper limits and artificial variables then

permits to put the penalty-coefficient for not meetins a true
linear restriction an order of magnitude higher than the penalty
coefficient for not meeting an approximation restriction.

If we then find that a true linear restriction is met only

artificially, we assume that the true linear restrictions
contradict each other, and conclude to emptiness, irrespective
of convexity (we strictly lack proof on this point, see
section 16.10).

For quadratic restrictions the adjustments discussed earlier

in this section then comes before we investigate emptiness.
This obviously leaves the possibility that emptiness of a non-
convex problem leads to non-convergence.

We adjust one restriction, formulate a new subsidiary problem,

with the same pseudo-Lagrangean, and find that the modified
problem again leads to a solution where (the same or another)
true restriction is amply fulfilled by a solution vector
associated with a binding or artificially met approximation.
Adjustment is then again called for once more.

Whether there are artificial variables left in association with

restrictions where the approximation and the true restriction
are violated, is then never investigated.

However, emptiness will be recognized, even in a non-convex

problem, if an artificially feasible "optimum" arises, for
which all binding or artificially met approximations relate to
binding or violated true restrictions.
576 CHAPTER XVIII

lB.4 Reapproximations

Let us now suppose that we have found the optimal and feasible
solution of a subsidiary problem, and that we have verified the
solution-structure (or if necessary re-defined the problem)
to the extent that no approximation-restriction is binding on
a solution for which the true quadratic restriction is amply
fulfilled.

In such a situation there are generally restrictions for which

the true quadratic restriction is violated, while the
approximation is binding or amply fulfilled. One possible
approach towards getting nearer to the true solution is to
impose new approximations as additional restrictions, or as we
will call such restrictions reapproximations. There are two
versions of this technique. If the current approximation of a
quadratic restriction is not binding on the optimum of the
subsidiary problem (while the true quadratic restriction is
violated, the value of the restricting function being negative
non-zero), we speak of a loose approximation.

If the current approximation is binding, but we expect to be

able to replace it by a new approximation, we speak of
superimposing a new approximation. In general, reapproximation
means that we do not (as yet) formulate a new subsidiary problem
to be solved from the trivial basis onwards, but re-enter the
ordinary QP algorithm in the re-entry mode, with a new violated
approximation restriction. The loose approximation is the simplest
of the two cases, and will be discussed first.

We again take the same example i.e.

2
Maximise T = xl - (x 2 - 1) , subject to
2 2
(xl - 2) + (x 2 - 3) < 4

(0 ~ xl ~ 10, 0 ~ x2 < 10)

The p = 0 subsidiary optimum, with the so-far unused

"extra" restriction slot is given below as tableau lB.4a.

As we saw already earlier in this chapter the initial

approximation 4xl + 9x 2 ~ 4 is not binding on the optimal
solution, the positive value of sl = 37 confirms this. In the
interest of containing the number of restrictions to be
administered, and of keeping the option of imposing more in
future we simply discard a loose approximation, and put a new
one in its place.
GENERAL QUADRATIC PROGRAMMING 577

TABl.EAU 18.4 A

ILLUSTRATION OF LOOSE APPROXIMATION.

NAME' , B1 D2 PI P2 II VAl.UE DIST

-----------------------------------------------------
U1 ! ! -4 "1 X
X2 I' -0.50 -3 II 1 9
Sl ,i 4 -3 -18 'I 31 X
S2 I' 'lOX

2T " 1 -1 -37 I' 22 X

US 'I 10 X 100 100 II X X

Application of (18.2.5), for ~*' = [10, lJ

yields a new approximation, which is l6x l - 4x2 ~ 92

(The reader is invited to verify this calculation referring to

18.2.5)

Formulae for calculating a currently updated part of the

tableau from a known block-pivot, (the inverse of which may be
extracted from the current tableau), and the non-updated form
of the tableau, may be obtained by seneralisation of the
similar problem as it arises in the LP case.

From section 11.1 we derive the following formulae for

calculating those elements of a "missing" row which refer to
non-basic (primal and dual) slack variables.
-1
.£' 2 , 1 P (18.4.1)

Here t' is a row of the second block-row, first block-column

block-ot'the current tableau. The composite vector [~'2,1' OJ

arises because in the QP case, the rows which relate to primal

restrictions need to be extended with zeros, insofar as dual
variables are concerned. (The straightforward application of the
matrix-formulae from section 11.1 to the vectorial case would be

t' =- a' A -1
- 2,1 - 2,1 1,1
In applying (18.4.1) in the presence of a binding upper-limit
restriction we must also consider the unwritten Xl-row, which
is a unit-vector, with the unity element in the b l column.
578 CHAPTER XVIII

The full pivot inverse therefore is

dl dZ bl

xl 1

X z -0.50

ul -1

Application of (18.4.1) therefore results in

dl dZ b.
X z ul

[ ]
xl ~

-sl [16 -4 0] xl dl dZ bl
X z -0.50 sl [0 -z -16J

ul

However, since the d l column is not stored as a currently updated

column, we do not need to calculate an additional element to it,
which is anyhow systematically zero. Also multiplication by zero
in relation with dual variables elements of the non-updated form
of the row may be suppressed and the non-trivial part of this
calculation is

-4J

The calculation of an additional column is in a QP tableau

facilitated by the symmetry rules; having done the row already
we just copy with, where appropriate, a change in sign.

Only for one element do we need a further application of the

matrix formulae from section 11.1, i.e. to calculate an element
of a non-basic variable's column which does not figure as an
eliminated slack-variable in the pivot inverse.

In view of the symmetry between ~Z 1 as non updated row and the

corresponding dual variable's column, the formulae for the
GENERAL QUADRATIC PROGRAMMING 579

diagonal element is

[~2,1' OJ
(18.4.2)

In practice, we obtain this result as the inner product between

the already updated row i.e. -[~'2 1 I O]p-l, and the unupdated
form of the dual variables' column.'

In this example that calculation ~s

[: ]
Since the dual variables' column contains zero entries for all
primal variables' rows, this calculation can be restricted to
its non-trivial part only and becomes

Similar calculations for the value column cell are possible,

but not necessary. Reference to (18.2.4) makes clear, that, at
x = x* itself the current slack of the true restriction and the
approximation are the same. Since we needed to calculate the
quadratic slack in the first place in order to es~ablish its
sign, it is now known (to be - 64).

The tableau now becomes as given in tableau l8.4b.

TABLEAU 18.4 B

REENTRY TABLEAU FOR IMPOSING THE REAPPROXIMATION

OF A LOOSE APPROXIMATION RESTRICTION.
NAME! I BI D2 PI P2 'I
-----------------------------------------------------
VALUE DIST
UI ! ! 16 ! I X
X2 ! I -0.50 -2 , I 9
SI ! ! -16 -2 -8 ! ! -64 X
S2 ! ! ! ! 0 X
-----------------------------------------------------
2T I I I -I 64 o !! 22 X
UB !! 10 X 100 100 !! X X
580 CHAPTER XVIII

Normal re-entry of the ordinary QP algorithm now leads to a new

optimal and feasible solution, for which the calculation tableau
is given in tableau l8.4c

TABLEAU 18.4 C
A REOPTIMIZED SUBSIDIARY PROBLEM. AFTER IMPOSING THE
REAPPROXIMATION OF A LOOSE APPROXIMATION RESTRICION.
NAME II D1 D2 Sl P2 II VALUE DIST
Xl " -0.03 -0.12 0.06 II 6.03 3.97
X2 II -0.12 -0.50 " 1.13 8.88
PI " -0.06 II 0.06 X
S2 II X
" 0
2T II -6.03 -1.13 0.06 0 II 14.03 x
UB I! X X X 100 X
" X

The approximation of the quadratic restriction is now binding

and we cannot simply discard it as not being useful.

What we can, however still do, is to impose a new approximation

as well. We therefore now occupy the (m+l)th (second)
restriction slot.

We apply (18.2.5) again, now for x*' [6.03, 1.l3J and find
8.06 xl - 3.75 x 2 < 28.64.

The same procedure of updating an additional restriction now

results in tableau l8.4d

TABLEAU 18.4 D
ILLUSTRATION OF THE SEARCH FOR SUPERIMPOSITION.
NAME II D1 D2 Sl P2 II VALUE DIST
-------------------------------------------------------
""
Xl II -0.03 -0.12 0.06 -0.22 6.03 3.97
X2 !I -0.12 -0.50 -0.87 1.13 8.88
PI
S2 "" -0.06
-0.22 -0.87 -0.50
0.50 0.06
-1.50 II -15.77" X
X

2T !I -6.03 -1.13 0.06 15.77 II 14!03 x

UB II X X X 100 II X X
GENERAL QUADRATIC PROGRAMMING 581

At this point w~ investigate whether or not there is a

reasonable prospect of actually replacing the old approximation
by the new one. In the general m-variable case, we might wish to
repeat the operation for several restrictions and this is
possible, only if after re-entry of the ordinary QP algorithm,
we again have one binding approximation for one restriction,
in which case the now amply fulfilled old approximation can be
discarded.

To assess the likelihood of that to happen we investigate the

implications of two tentative pivots, by comparing their
associated ratios with the value column.

These two pivots are: the n + m + l/n + m ~ 1 cell i.e. the

elimination of (the s2 variable) the slack variable of the new
approximation by its own dual variable, and the m + 2/n + m + 1
cell, i.e. the elimination of the dual variable of the old
approximation of the offending rth restriction by the dual
variable of the new approximation of the same quadratic
restriction, slotted in place n + m + 1.

In the example, this leads to comparison of the ratios

- 15.77/-1.50 = 10.5 in the s2 row and 0.06/0.50 = 0.12 in the
PI - row. The one in the PI row is the smaller one. Therefore
we can at least be sure that re-entry of the ordinary QP
algorithm will not result in simply exchanging the slack of the
new approximation against the dual variable of the new approxi-
mation, by way of activating the diagonal pivot. This would
a fortiori apply in the case of a zero diagonal cell, and the
ratio is classified as infinite in that case. We do not
investigate any other ratios and accept the new approximation
as probably fully superimposable.

In anticipation of subsequently discarding the old approximation,

we reorder the tableau, now classifying the new approximation
as 'the "proper" one, and the old one as the "additional" one.
This re-ordering results in tableau l8.4e.
582 CHAPTER XVIII

TABLEAU I B.4 E

REE~TRY TABLEAU FOR SUPERIMPOSING

NAME! ! DI D2 PI 52 !! VALUE DIST

-------------------------------------------------------
3.97
XI !! - 0 .-03 - 0.12 -0.22 0.06 ! ! 6.03
X2 !! -0. I 2 -0.50 -0.87 ! ! I .13 8.88
51 !! -0.22 -0.B7 - I .50 -0.50 ! ! -15.77 X
P2 ! ! - 0 .06 o .50 ! ! 0.06 X

2T !! -6.03 -1.\3 15.77 o .06 !! 14.03 x

US !! X X 100 X !! X X

The resulting new optimal and feasible solution (not summarised

in detail) is indeed characterized by an amply fulfilled s2
restriction this being the old approximation 16x I - 4x2 < 92.

We then say that the new approximation (8.06x I - 3.75x I < 28.64),
has been fully superimposed over the old one. If this were not
the case, i.e. if both approximations were binding on the
optimal solution we would speak of a blocked superimposition
and the operation could not be repeated, for lack of a slot
to put a new re-approximation in.

Note however, that the old approximation may sometimes again be

unblocked, by imposing a new approximation of a different
restriction, where the previous approximation became loose in
the process of superimposing.

If we carryon superimposing new approximations we run the

obvious risk of "slicing off" smaller and smaller areas of the
feasible space area. The computational implementation offered
here contains a search operation for the most violated
restriction for which a probably fully superimposable approxi-
mation may be identified, and a limit of at most 2 q (twice
the number of quadratic restrictions) superimpositions for
any pseudo-Lagrangean, superimposing new approxima~ions whenever
the (m+l)th slot is free.

In the example at hand is limit is 2, and it was operative, i.e.

a second probable fully superimposabJe approximation of the salT'e
rescrict~on is idencified and actually leads to superimposing
the new approximation once more, but thereafter no more
are made until a new pseudo-Lagrangrean has been formed.
GENERAL QUADRATIC PROGRAMMING 583

One further refinement of the approximation-technique which

is useful to discuss at this point, is the inwardly adjusted
reapproximation, and more generally the adjusted reapproximation.

In section 18.3 we assumed that the whole tableau and solution-

vector was adjusted, using the parametric variation method from
chapter 17. Although a drastic change in the slope of an
approximation-restriction is generally undesirable it is not
necessary that reapproximations refer to solution-vectors
which satisfy the (approximations of) other restrictions.
Furthermore, it is at least as important that they come in some
sense near the restriction to which they refer and this may
well be achieved by taking approximations of different
restrictions at different points. For reasons to be explained
later on in this chapter this is especially important in the
case of peripheral convexity where it is desirable to obtain
tangential approximations. The technique of inward adjustment
is applicable on both properly convex and on peripherally
convex restrictions. We now illustrate it in relation to
proper convexity.

Example (the one with which this chapter was opened)

2
Maximise T xl - (xl - 1)
2 + (x - 3) 2 < 4
Subject to - 2) 2

(0 2-. xl 2-.10, ° 2-. x 2 < 10)

The first subsidiary problem has the optimal solution Xl 10,

x2 = 1, which is nowhere near the true solution. We put a new
linear approximation

in place of the initially loose approximation.

This restriction is binding on the subsidiary optimum reported

tableau l8.4e (xl = 6.03, x 2 = 1.13). We initially discussed
superimposition with respect to a ne," approximation taken at that
point. The further refinement of (inwardly) adjusted
reapproximation consists of taking the new approximation to be
superimposed, not at (xl = 6.03, x 2 = 1.13), but at
(xl = 2.70, x 2 = 1.13), the solution which is obtained by
adjusting the existing approximation-restriction,
584 CHAPTER XVIII

~. O
graph 16. ~ a
TXl
i 0;
1 illustration of
).0 inward adj ustment.

• s
1

6 l<
} 0; =:)e
10 '-! 1- l

\ 61-\
} S

1 0

I ;

I 0

;
X2
~

~
O. S 1.0 I. S ~. 5 5.0 5 5 6.0
 GENERAL QUADRATIC PROGRAMMING 585

16 xl - 4 x 2 ~ 92, inwardly to 16 xl - 4 Xz ~ 38.64, where

the Erue quadratic restriction is exactly fulfilled.

Note (see graph 18.4 a), that the adjusted old approximation
e.g. 16 xl - 4 x 2 ~ 38.64 generally cuts through the true
quadratic restriction, the new approximation is tangential to
it. As in the case of outward adjustment only the smaller of
two positive roots of the quadratic adjustment equation
(18.3.4) is applicable. The adjustment of the solution-
vector from xl = 6.03, x 2 = 1.12 is marked in the graph by
the downwards pointing arrows, the adjustment ends in fact
at the end of the line of bigger arrows, the continuation
merely indicates the significance of the other root.

Before we finalize this section, we may note that there are

two versions of the (inwardly) adjusted reapproximation.

The new approximation may be brought in by superimposing it

over the old one, or we may formulate new approximations for
all restrictions and solve the same problem again from the
trivial basis. The latter version will apply if a return to
start has already been indicated for some other reason. In
that case new approximations will be taken with inward
adjustment for violated true restrictions for which approxi-
mations are binding, and with outward adjustment if any
overtight approximations are encountered.

Hence the general name adjusted reapproximation.

18.5 The objective function limit

Let us denote as x = x**, the optimal solution vector of a

particular subsidiary-problem, in which

P(~) = T(~) + E' f figures as the objective function.

The corresponding vector of true slacks (the values of the

restricting functions f. (x**) is indicated as f**.
1 -

Let us further assume that the problem is property convex,

in T(~) as well as in each f. (x). We also assume that the
particular solution vector x 1 =---x** satisfies the
condition of optimal form pi f~* < 0, (p. > 0), all i (we
cannot achieve the complementafy slackne§s-condition as such,
p. = °
if f. > 0, f. = °
if p. > 0, until we have solved the
pfoblem). W~ then h~ve, (for ~ convex problem when a subsidiary
problem has been found to s.tisfy the condition of optimal
form), the followine
586 CHAPTER XVIII

Theorem

In a convex problem all vectors x which satisfy the true

restrictions f.(x) > 0, all i, also satisfy
1 - -

[~
ax
(x**)J' x <
- - -
[~(x**)]'
ax -
x** + £.' f** (18.5.1)
- -

i.e. a linear approximation of

(18.5.2)

taken at x = x**.

Proof 1 (assuming optimal form)

Consider the mathematical programming problem (hereafter named

the displaced and constrained problem).

Maximise

Subject to

f. (x) > f. (x**)

1 - - 1-

as well as the restrictions of the subsidiary problem.

(If the latter group of restrictions is absent, we will speak
of the displaced problem.)

The Lagrangean expression, associated with the optimal solution

of the displaced and constrained problem, differs from the
Lagrangean expression which is associated with the optimal
solution of the subsidiary problem only in its constant term.

We simply re-group each term Pi f., as figuring in the

objective function of the subsidi~ry problem with a term
-p.f~*, to obtain the corresponding term of the Lagrangean
as§oeiated with the optimal solution of the displaced and
constrained problem.

It follows that x = x** also is an optimal solution of the

displaced and constrained problem, the feasible space area of
which includes that of the true problem entirely. A
tangential approximation of the aggregate restriction of the
displaced and constrained problem is:

aT (x**)], x < [aT (x**)], x**

[ ax (18.5.3)
- - - ax - -
- -

This restriction (18.5.3) is also obtainable as a non-

negative combination of the restrictions of the old subsidiary
problem (with the corresponding linear shadowprices as
GENERAL QUADRATIC PROGRAMMING 587

multipliers), and tangential approximations of f. (x) > fi(~**)'

(with the p. as multipliers). The similar combikatio~ of the
restriction§ of the old subsidiary problem, and new approxi-
mations of the true restricting functions f.(x) > 0, results
in (18.5.1). 1--

q.e.d.

Note that if T(X) is a convex function an upper limit on the

objective functTon i.e. (18.5.2) is an anti-convex restriction
and its tangential approximation (18.5.1) is more not less
restrictive than an upper limit on T(X) as 8iven in (18.5.2)
itself. All x satisfying (18.5.1) also satisfy (18.5.2), but
some ~ may satisfy (18.5.2), without satisfying (18.5.1).

It follows that if (18.5.1) is added as an additional

restriction to the next subsidiary problem, such a device
ensures, for p' f** < 0, an actual reduction in the value of
the objective-fu~ction between one subsidiary problem and its
successor.

We will therefore, in the rest of this chapter, refer to

(18.5.1) as "the objective function limit" as it implies
(in the convex case), (18.5.2).

Note

The above proof becomes invalid if the assumption of optimal

form is dropped. This is because the feasible space area of
the displaced and constrained problem includes that of the
true problem, only if optimal form is assumed.

If some of the displaced restrictions (associated with Pi > 0)

require f. (x) > f.(x**) > °
we have to consider the possibility
that the 6ptim~1 §olution of the displaced and constrained
problem is associated with a solution value which is below
the value of the true optimum.

In fact (18.5.1) is still valid provided the restrictions are

convex. In this connection we supply

Proof 2 (not assuming optimal form)

Since P(x) is the constrained maximum of the pseudo-Lagrangean,

(18.5.4)

holds for all feasible solution vectors of the subsidiary

problem and therefore by implication, for all feasible
solution vectors of the true problem.
588 CHAPTER XVIII

Since p' f is required to be non-negative (for a feasible

solution)-the non-linear form of the objective function limit
(18.5.2) follows. The equivalence between the two inter-
pretations of (18.5.3) - a tangential approximation of an
upper limit on the objective function, or of the aggregate
restriction of the displaced and constrained problem - holds
irrespective of optimal form. It follows that (18.5.1),
understood as a non-negative combination of the restrictions
of the subsidiary problem and of new approximations of the
quadratic restrictions, taken at x x** also applies
irrespective of optimal form.
q.e.d.

The validity of the objective function limit in the presence

of peripheral convexity will be discussed in the next section.

18.6 Consistency and optimality

The optimal solution of a subsidiary problem is said to be

consistent with the true problem if the solution vector x = x**
results in f.(x) > 0, for all i, in other words if it satisfies
the primal r~strictions of the true problem.

The conditions of optimal form (p. f. < 0, Pi ~ 0), and

° °
consistency (f. > 0) together amoQnt~to the familiar comple-
mentary slackn~ss conditions (p. = if f. > 0, f. = if
p~ = 0, p. f. > 0). ~ ~ ~
L ~ ~-

Despite the fact that there may be binding linear restrictions

in the optimum of the subsidiary problem, a consistent
solution also is an optimal solution of the true problem, at
least as far as the primal solution vector x is concerned.

The following cases arise:

Case a:

No binding restrictions in the optimal solution of the subsidiary

problem

If the optimum solution of the subsidiary Rroblem is the

unconstrained maximum of the pseudo-Lagrangean, the pseudo-
Lagrangean is revealed to be the optimal Lagrangean of the
true problem.

Case b:

Some approximation-restrictions are binding, the objective

function limit is not binding

Since loose approximations are always replaced by new

GENERAL QUADRATIC PROGRAMMING 589

approximations, consistency implies that for all restrictions

for which dual variables Pi have been stated, there are
corresponding exactly met linear approximations.

Comparison of (18.2.4) and (18.2.5) now shows that consistency

implies

6 x'D. 6 x = (x - x**) , D. (x - x**) o (18.6.1)

1 - - -1 1 - -1
For all i for which p. > applies.
1

(This quadratic form is the error term in the approximation

formula, if the approximation fits exactly the error is by
implication zero).

Reference to (18.2.4) also shows that, if the error term

vanishes conform (18.6.1), the true restricting function and
its approximation have the same vector of differentials.

There are only two possibilities for 6x'D.6x to vanish:

Either 6 x = 0 (the approximation was takeb at x = x** itself),
or 6 x is a characteristic vector associated with a-zero
latent root of D .. (see also sections 14.4 and 14.7).
1

In either case, we find, denoting the restricting function

associated with an approximation restriction as f.* (x.)
1 -1
af.
1
* af.
1
ax ax
The dual variables associated with the true optimal solution
are then related to the ones assigned to the pseudo-Lagrangean
by the following relationship.

p.** p. + p~ (18.6.2)
1 1 1

where p~* are the true dual variables, p. are the (estimates
of the)1 dua1 variables that were assigned to the pseudo
Lagrangean, and pt are the shadowprices of the linear
approximation restrictions as obtained from the dual solution
of the optimum of the subsidiary problem.

The left-hand side of (18.6.2) gives the dual variables as they

figure in the Lagrangean of the true problem, the righthand
side gives two separate terms as figuring in the Lagrangean
associated with the subsidiary optimum, where a term p. f.
is grouped with the objective function and p. f* with 1 1
the rest of the Lagrangean. 1 1
590 CHAPTER XVIII

Case c:

The objective function limit is binding

We only have a meaningful use of the objective function limit

if certain properties of convexity are satisfied.

We postulate that each approximation restriction which is

subsumed in the objective function limit (which is a non-
negative combination of approximation-restriction), permits
an area which includes all of the area permitted by the
corresponding true restriction. (For properly convex
restricting functions, this assumption is always satisfied,
but we shall need to discuss the situation which arises in the
case of peripheral convexity in more detail).

With that provisio, a binding objective function limit implies

that all previous approximation-restrictions that are subsumed
in the objective function limit, are exactly met. (The stated
convexity property means that none of their slacks can be
negative, the corresponding true restricting functions being
non-negative on account of the assumed consistency, if their
sum is zero each separate term must also be zero).

The argument put forward under case b, may now be extended to

case c. Note, however, that (18.6.2) will in case c, with a
binding objective function limit, understate the true value of
the dual variable pt*, as shadowprices of binding old
approximation restrictions may be subsumed in the shadowprice
of the binding objective function limit.

We now come to discuss the issue of the applicability of the

objective function limit in the presence of peripheral
convexity, and its implications for the optimality theorem
stated in this section. By keeping a track-record of the
applications of (18.2.2) we may establish whether or not a
particular restriction is a tangential approximation. If
(18.2.2) were in fact binding at x = x* itself - as -would, ~n
particular, be the case if the applic~tion of (18.2.2) was
made either in the context of an inwardly adjusted
reapproximation, or after outward adjustment, following the
discovery of overtightness, we would be entitled to treat such
a tangential approximation of a peripherally convex restriction
on the same basis as an approximation of a convex restriction.

Under those conditions, the linear form of the objective

function limit is also a tangential approximation of a
peripherally convex restriction, i.e. of (18.5.2) interpreted
as the aggregate restriction of the displaced and constrained
problem displaced once more by adding the term £' f**.
GENERAL QUADRATIC PROGRAMMING 591

If a fully non-convex restriction - or a non-tangential

approximation of a peripherally convex restriction has become
active, the formulation of a new objective function limit is
inhibited, and the issue of it being binding does not arise.
We may therefore conclude that finding a solution-vector
which satisifes both the requirement of optimal form, and is
consistent with the true problem, is equivalent to having
found the primal solution of the true problem.

18.7 The upward adjustment of dual variables

There are two noticeably distinct ways of changing the estimates

of the dual variables assigned to the pseudo-Lagrangean, which
we will indicate as normal transition under optimality, and
correction of optimal form.

If the optimal solution of the last subsidiary problem satisfies

the condition of optimal form, we assume that the collection
of restrictions for which there are dual variables and/or
binding approximation restrictions is the collection of
restrictions which is binding on the optimum solution of the
true problem. In that case we adjust the dual variables
upwards, according to the rules to be laid down in this
section. If optimal form has been lost we proceed in a
quite different way, adjusting more downwards than upwards,
a subject to be discussed in more detail in the next section.

The formulation of a pseudo-Lagrange an in which in particular

convexly-shaped restrictions are represented by pos1t1ve non-
zero dual variables, is of particular importance if the
solution-structure of the optimal solution of the true problem
is characterised by an excess of the number of non-zero
elements of x, over the number of binding restrictions, while
the objective-function is linear or anti-convex.

Example

Maximise
2 2
Subject to -4x l - 4x2 + xl + x 2 < -7

(0 .::. xl .::. 100, 0.::. x 2 .::. 100)

Any subsidiary problem which consists of maximising the

objective function, subject to a linear approximation of the
one restriction will put either xl or x 2 at zero. Yet the
true solution (xl = x 2 = 1-12) assi.gns non-zero values to
both variables. The non-linear component of the restriction
is essential.
592 CHAPTER XVIII

We now address ourselves to the question of formulating

suitable rules for adjusting dual variables upwards under
normal transition. It is intuitively obvious that the shadow-
prices of binding approximation restrictions are relevant
information here. We may also observe that once a consistent
solution without a binding objective function limit has been
found the problem is essentially solved. We only need to add
the shadowprices of the approximation restrictions to the dual
variables, i.e. we apply (18.6.2).

It would therefore be desirable that any rules to be laid down

would converge to (18.6.2) in the vicinity of the optimal
solution of the true problem - or nearly so, for reasons which
will become clearer in the next section.

This desideratum is achieved by proceeding on the assumption

that the current solution vector x = x** is the optimal solution
of the displaced problem, i.e. the true problem modified by
displacing the quadratic restrictions to f.(x) > f**. = f.(x**),
and that the approximation-restrictions ar~ proporti5nal 1 -
to new approximations of f. (x) > f. (x**) , taken at x = x**.
1 - - 1-

This assumption leads to a generalization of (18.6.2), namely

df.
1
P**i = Pi + P*i/ db*. (18.7.1)
1

wheredf./db*i may be obtained by evaluating the change in the

true re§tricting function f., per unit of increase in b*.,
as given by the f*. column 5f the optimal tableau of the 1
subsidiary problem~

If that maintained assumption is actually true, the current

solution vector is (at least in the convex case), the
unconstrained maximum of the new pseudo-Lagrangean, but a
further approximation to the true optimal and feasible
solution is obtained by introducing new approximation-
restrictions, which do not admit x x** as a feasible solution-
vector.

The relation (18.7.1) may be indicated as the correction

equation, and the reciprocal of the differential df./db*. is
the correction-factor - if (18.7.1) is applied in tRat f5rm.

In fact, certain modifications of the correction equation

may be desirable, in view of the fact that the maintained
assumptions that the current solution is the optimum of the
displaced problem and that the approximation-restrictions are
proportional to (= geometrically coincide with) tangential
approximations of the displaced quadratic restrictions, may
not be true.
GENERAL QUADRATIC PROGRAMMING 593

6.0

IX
graph 18.7 a
5.5 1
Inward adjustment would
5.0 Increase the distance from
the true restriction.
4.5

1,.0

3.5

3.0
594 CHAPTER XVIII

The following notes summarise modifications to (18.7.1) and

the reasons, therefore, in order of obviousness and importance.

Firstly, it is even possible for the differential df./db*.

l. l.
to be negative

Example

Minimise xl + 3x 2
2 - 3)2 2-
Subject to (xl - 2) + (x 4
2
(0 2- xl < 100, o < x2 2- 100)
This problem has been graphically summarised in graph l8.7a.
The initial approximation is 4x l + 6x2 ~ 9 and the initial
subsidiary optimum is found at xl = 2.25, x 2 = O.

If the initial approximation is adjusted inwards in the sense

of making it more stringent, by adjusting its constant we
increase the distance from the true restriction.

This problem of the adjustment going the wrong way is solved

in the case of primal adjustment by simply not adjusting (see
sections 18.3 and 18.4). That will not do for the dual; as the
example also makes clear, we need some non-zero value for the
dual variable of, in particular, a-convex restriction. When
the calculated correction factor is found to be negative, we
substitute an arbitrary positive number for it; the number 0.5
was chosen.

Secondly, unduly big calculated correction factor~ may arise.

In the extreme, we could theoretically meet the borderline-
case between a positive and negative correction-factor, which
would in this case be infinite, with df./db*. being zero.
1 1

We obviously need to impose an upper limit. This limit was

eventually set at 0.999, just below the figure which still
permi ts (18.7.1), as modified, to converge 'to (18.6.2) for
a consistent solution.

18.8 Loss and correction of optimal form

The rules for adjusting dual variables upwards, which were

discussed in section 18.7, are, as far as practicable, designed
to ensure that no estimate of a dual variable is formulated
which is actually in excess of the true value. These are,
however, not proof-supported features of the algorithm.
GENERAL QUADRATIC PROGRAMMING 595

If, despite these attempts, an estimate of a dual variable is

assigned to a quadratic restriction and entered into the pseudo
Lagrangean, and such an estimate exceeds the true value of the
dual variable in question as it relates to the optimal solution
of the true problem, we say that the dual variable in question
has been overstated.

We may distinguish two distinct varieties of overstating. If we

have assigned a dual variable to a restriction which is not
binding on the true optimal solution of the true problem, and
there ought therefore not to be a non-zero dual variable for the
restriction in question at all, we speak of overstatement by
mis-identification.

Over-statement of a dual variable of a restriction which is

binding on the true optimum, is then indicated as simple
overstatement.

The calculation of correction factors as discussed in the

previous section, may, or may not, lead to avoiding simple
overstatement. There are, however problems for which over-
statement by mis-identification cannot well be avoided.

Example

Maximise T 2 xl + x 2
2 2
Subject to xl -2 xl + x 2 - 2 x 2 < 7
2 2
xl + 2 xl + x 2 + 2 x 2 < 34
(0 ~ xl < 100, a ~ x2 ~ 100)

This problem is graphically surveyed in graph l8.8a. The first

restriction is equivalent to (xl - 1)2 + (x - 1)2 ~ 9 and
is represented in the graph by a circle wit~ radius 3, and the
centre at the point xl = I, x 2 = 1. The second restriction is
equivalent to (x 2 + 1)2 + (x 2 + 1)2 ~ 36 and is represented
by a circle with radius 6, and the centre at the point -I, -1.
596 CHAPTER XVIII

5. S graph IB.B a

illustration of loss
of optimal form.
5. 0

I, . S

1, . 0

3. 5
the vertex between two
approximations lies out-
side both 0 f the true
restrictions.
3.0

X2
2. S ~~--~~~~~~~~~~--~--~--~--~--~~~ ~
0. 6 0 .8 1.0 1.2 1.1, 1.6 1.6 2.0 2.2 2 .1, 2.6 2.8 3. 0
GENERAL QUADRATIC PROGRAMMING 597

The graph makes it quite clear that the second restriction is

in fact redundant. However, a finite maximum of the linear
objective function requires 2 binding restrictions, and the
direction of the objective function does not make it likely
that the non-negativities of xl and x 2 itself will become
binding. Indeed, the initial subsidiary problem, modified by
a series of reapproximations, finds its optimum solution at
xl = 4.53, x 2 = 1.72, the objective function at this point
being contained by two approximation restrictions, one for each
quadratic restriction.

Both of the true restrictions are violated by the subsidiary

optimum xl = 4.53, x 2 = 1.72 and dual variables are stated for
both restrictions (PI = 0.069, P2 = 0.097).

We now have a strictly convex pseudo Lagrangean and one binding

linear restriction is sufficient to contain the pseudo-
Lagrangean. The second subsidiary problem finds its optimal
solution, at xl = 3.82, x 2 = 2.43, and this time only the
approximation of the first restriction is binding, and we find
fl = -0.999, f2 = +1.

We now establish loss of opitmal form (P2 = 0.097, f2 = 1) .

There are a number of modalities of loss of optimal form , we
may meaningfully distinguish the following:

Subdominant loss of optimal form: £.' f < 0, some p. £. > 0

1 1

dominant " " " " £.' f > 0, some p. £.

1 1
> 0

total loss of optimal form all p. f. -> 0, some Pi £.1

1 1
> o.
Subdominant loss of optimal form still permits the formulation
of a new objective function limit. This may also apply for
dominant loss of optimal form.

For p' f > 0, a new objective function limit does not require
a reduction of the value of the objective function limit below
its value in the last subsidiary problem but that level itself
may be well below the value required by the previous objective
function limit.

Nevertheless, when optimal form has been lost, and in particular

when dominant or total loss of optimal form have arisen, we can
no longer look for a continuous reduction in the objective
function value as our guarantee of not repeating the same
solutions again. Instead, we require a continuous reduction
in the sum of all dual variables.
598 CHAPTER XVIII

To analyse the problem of optimal form, we denote

as x = x***, the primal solution vector of the true problem

as x = x**, the primal solution vector of the subsidiary problem.
as p***-the vector of true dual variables, and as p the vector
of estimates assigned to the pseudo-Lagrangean. -

Absence of overstatement is then given as p < p***. In fact,

there are complications with non-unique subsidiary optima,
precisely for p = p***, and we will for the moment assume
E 2 E***' p =f E:***-:-

Theorem

In a convex problem p. < p***, if p*** > 0 ensures that

optimal form is maint~ined: 1

Proof

We first state a much weaker theorem, which refers to total

loss of optimal form.

Lemma

Total loss of optimal form implies that one of the two

following conditions is present

Either (a) E ~ E***' E ,; E***

at least one dual variable has been overstated.

or

(b) E = E***
Proof (of the lemma)

If (a) applies, no further proof is needed.

Now assume not (a) and in particular absence of overstatement
by mis-identification.

We then find

P(~***) = T(~***) = L(~***) (18.8.1)

We also find, since x = x*** maX1m1ses L(~),

L(x**) T (x**) + E**1<' i (~**) < T (~***) L(x***)

- (18.8.2)
GENERAL QUADRATIC PROGRAMMING 599

We also find, since x*** is a feasible solution of the

subsidiary problem, and x** the constrained maximum of P(~),

(18.8.3)

Consolation of (18.8.1), (18.8.2) and (18.8.3) yields:

T (~**) + 1:***' i(~**) .::. T (~***) .::. T (~**) + 1:' i(~**)

(18.8.4)

From which

1:***' i(~**) .::. 1:' i(x**) (18.8.5)

Since total loss of optimal form is defined as fi(~**) > 0

if p. < 0, we find that (18.8.4) can only be satisfied in the
pres~nce of total loss of optimal form, if

1:***' .::. 1:' (18.8.6)

applies, which is equivalent to (a) or (b)

q.e.d. (for the lemma)

The more general theorem now follows by induction.

If loss of optimal form is not total there are some i for which
p. < 0, f. < 0 and therefore binding approximation restrictions.
Within th~ subspace of those binding approximation restrictions
any loss of optimal form is total loss of optimal form.
q.e.d.

We now come to discuss the problem of the non-unique subsidiary

optimum. In a problem which is convex, but not strictly convex,
the true Lagrangean often has no strict maximum. The most
extreme case arises with a linear problem.

Example

Maximise T = xl + x 2

Subject to xl < 1; x 2 .::. 1

(0 < xl ~ 100, 0.::. x 2 .::. 100)

If the upper limit on the correction factor for calculating

dual variables were set at 1 instead of at 0.999, our second
subsidiary problem would be
600 CHAPTER XVIII

Maximise P(x l x 2 ) = xl + x 2 + 1 (1 - xl) + 1(1 - x 2 ) - 2

Subject to xl -< 1, x2 < 1 (xl' x 2 ..:.. 0)

xl + x 2 < 2 (objective function limit)

The pseudo-Lagrangean reduces to a constant, and the origin

is co-equally optimal with the optimum of the true problem.

It is clearly undesirable to treat linear restrictions as

quadratic in this way, but one assumes that this problem can
also arise with semi-definiteness. Hence the downward bias in
calculating estimates of dual variables. We normally do not
calculate exactly the true vector p***, but continue to obey
the condition Pi < p~**, if p~** >-0, and find the problem
solved when both con§istency 5f the primal solution and zero
residual shadowprices of approximation restrictions are
satisfied with a set tolerance, we accept not exactly meeting
these latter conditions. Note also that in a non-convex problem,
there may be several local optima.

Regardless of the fact that we lack a proof, the plausible

weaker theorem that the pseudo-Lagrange an attains a local
maximum which respects optimal form does not inform us to which
local maximum is the "right" one. It certainly is not always
the global maximum of the pseudo-Lagrangean.

Example
2
Maximise T = 5x l + 4x2 - x 2

Subject to
2 2
lOx 1 - 10x 2 ~ x 2 - xl - 16

(0 ~ xl ~ 4; 0 ~ x 2 ~ 100)

Here the one restriction is peripherally convex in the xl ~ 4

domain as may be seen by wr1t1ng it as (x2 - xl)(lO - xl - x 2 )
..:.. 16; the graph (not given) is a rectangular hyperbola with
asymptotic axes xl x 2 and xl + x 2 = 10, the' upper part where
x 2 - xl < O.a~d 10 - xl - x 2 ~ 0 may coincide is excluded by
tfie upper l1m1t on xl'

The second subsidiary problem, with PI = 0.50 is:

2 2
Maximise P(~) = 9x 2 + 0.50 xl - 1.50 x 2

Subject to 8.06 x - x 2 < 3.31

1
(0 ~ xl < 4; 0 < x 2 < 100)
GENERAL QUADRATIC PROGRAMMING 601

The global optimum of this problem is xl 3 with

fl = 55 > O.

There is however, a second constrained local maximum, x] = 2.9,

x2 = 4.5, with fl = - 11.84 which is rather more helpful.
(The true optimum is xl = 0.56, x 2 = 3.08, with PI = 0.56).
We may thus classify loss of optimal form as being due to
three possible causes, viz: Overstatement of dual variables by
mis-identification, simple overstatement of dual variables,
and (in a non-convex problem only) getting at the "wrong;" local
solution of a subsidiary problem. Overstatement by mis~
identification cannot be avoided, we can only correct it
afterwards.

Simple overstatement is, we hope, normally prevented by the

calculation of correction-factors, for normal transition
under optimality, but it can also be corrected, should it arise.

There are a number of features of the SCM algorithm - and of

the adaptation of the ordinary QP algorithm employed by it,
which are aimed at developing local optima in which approximation-
restrictions are bindin~, in preference to other restrictions.
However, we lack a proof on that point, and at the same time
correction of optimal form is, in effect adapted to ameliorate
the first two causes of loss of optimal form only: we start
to reduce dual variables, assuming that some are overstated.

In short the SCM algorithm although clearly capable of solving

some non-convex problems, is proof-supported, only in the
convex case. We now discuss the correction of optimal form
in more de tai 1.

We proceed as follows:

We first formulate a "correction restriction". This correction

restriction is a linear approximation of a combination of
restrictions £. < 0
~ -

i.e.

L: m. £. < 0 (18.8.7)
1 ~

where

m. if f. (x**) > 0
~ Pi' ~ -

m.
~
0 , if f.(x**) < 0
~ -
602 CHAPTER XVIII

The calculation of this correction-restriction is performed at

the start of a correcting dual adjustment round (before each
separate correcting dual adjustment round). Each correcting
dual adjustment round then consists of:

(1) A semi-proportional downward adjustment of all dual

variables p., for which f. (x1,*) > 0 applies.
1 1 -

The first downward adjustment is associated with dual parametric

variation of the subsidiary optimum, the linear approximation of
(18.8.7) being the parametric restriction. (See also section 17.2).

The parameter A, ( = dual variable of (18.8.7)) is limited

by an upper limit, which is A < 2, if no other limit applies.
In the absence of approximation~error A = 1 corresponds to
complete removal of reference to the "offending" group of
restrictions from the Pseudo-Lagrangean. We would therefore
not normally expect this limit to be reached.

The parameter A is also restricted by the parametric displacement

leading to one of the amply fulfilled restrictions becoming
exactly fulfilled A 2.. \., Ak = A (f k = 0).

This gives rise to a search operation. For each i for which

f. (x**) > 0, p. > 0 applies we evaluate the roots of (18.3.4),
if any positiv~ root is found, or the smallest of two positive
roots, or the real and positive part of a pair of complex root -
for a non-convex restriction, the calculated root is compared
with the existing limit on A, if a lower limit is identified
1n this way, we conclude k = i, and reduce the limit on A.

If the actual parametric step results in A > 1, A = 1 is

substituted for it.

The parameter, now interpreted as a proportionality factor,

is now adjusted upwards according to the following correction
formula

A(C) A sum t/sum p (18.8.8)

where sum t is the total sum of all the (estimates of) the dual
variables, sum p is the sum of all those dual variables p., for
which f. > 0 applies. The first downward adjustment is no~
perform~d on those p. for which f. (x**) > 0 applies, as follows:
1 1 -

for i =k (the restriction which eventually limited A, if any)

(18.8.9)
GENERAL QUADRATIC PROGRAMMING 603

substitute zero instead, if Pk(new) found negative,

and for i '" k,

Pi (new) = (1 - 0.5 A(c» Pi (old) (18.8.10)

substitute zero instead, if p. (new) found negative.

~

Note that this first downward correction never leads to the

complete removal of all dual variables.

If there was no total loss of optimal form some dual variables

are left-undisturbed, if there was total loss of optimal form,
we find sum t = sum p, hence A(c) = A < 1, and not more than
25% is taken of Pk' and not more than 50% of any other p ..
We then have a second (middle) upward correction. The du1l
variables of which f. < 0 applies are adjusted upwards in the
same way as would ap~ly in the case of normal transition under
optimality.

Finally, the adjustment round is ended with a concluding

proportional downward adjustment. The total of all dual variables
as it was at the start of the round (or at the end of the previous
round, see below), is compared with the total of all dual
variables as it has become.

If the new sum is 90% of the old sum or less, the round of
adjustment is complete, otherwise all dual variables are adjusted
proportionally downward, making the new sum equal to 90% of the
old sum. The "old sum" is in this connection:

either the sum calculated when optimal form was found to be

violated for the first time

or the figures assigned to the old sum at the end of the round,
which is either the new sum at that stage or 90% of the old
sum, as the case may be. Adjustments of dual variables by other
loops of the programme (e.g. increasing the dual variable of a
particular restriction when a second leg of its approximation
is found to be binding) are then classified as being part of
the middle correction.

When a round of correcting optimal form is complete and no new

objective function limit has been formulated at its start, we
generally refer back to older approximations, rather than
forming new ones. We start with the approximations that were ~n
force when the last objective function limit was formulated
(which are recorded at that moment).

If any current approximations (current at the moment of

establishing loss of optimal form) are found to describe a
604 CHAPTER XVIII

permitted area which is entirely included in the one permitted

by the older one, the old approximation is replaced by the
current one.

The same applies when optimal form is lost in first instance

(as distinct from not as yet having restored it) for tangential
approximations, and more generally for approximations taken
nearer to the true restriction than the older one.

The latter feature ensures in particular that a restriction

which is instrumental in limiting the dual parametric adjustment,
is represented by a tangential approximations at least if this
happens in the first round.

This procedure ensures that, if several rounds of parametric

adjustment be needed the restrictions which were in force at
the start of the first round continue to be obeyed.

A proof of convergence to optimal form (and thereby in the

convex case to being able to identify a new objective function
limit), may now be stated, as follows:

Suppose by contra-assumption, that an endless series of

correcting dual adjustments rounds took place. The primal
solution vector would then converge to the constrained maximum
of the objective function (relative to the set of restrictions
stated at the beginning of the first round or in a more limited
feasible space area).

Since new dual variables, not approaching zero would be stated

in the middle upward correction round, the correction factor
sum t/sum p in (18.8.4) would approach infinity. Therefore any
remaining dual variable associated with an amply fulfilled
restriction would be completely removed, and given the con-
vergence of the primal solution to the constrained maximum of
the objective function only, optimal form would then be
restored.

18.9 Overview of the SCM algorithm

In this section we recapitulate the sequentially constrained

maximization algorithm and its capabilities. We first offer an
outline scheme. This outline primarily covers its application
to convex problems. It also contains some loops which specifically
relate to non-convexity, but only insofar as they are
computationally integrated with the main body of the algorithm.

There are also some adaptations of the "ordinary" QP algorithm,

as described in Chapters 16 and 17; these are discussed later
in this ~ection.
GENERAL QUADRATIC PROGRAMMING 605

The outline scheme is as follows:

Stage 0: Initiate

Set all matrices and vectors which do not contain initial data
at zero, and all logical variables in their "normal"
preferred state.

Set the namelists at zero, thus indicating that there are no

binding restrictions. Set any relevant indices at zero.

Stage 1: Approximate

Apply (18.3.4) afresh for each quadratic restriction. Quadratic

restrictions which are currently associ~ted with binding
approximations are made with adjustment. For fi > 0,
x = x*. is calculated with outward adjustment, for f. < 0,
-
x>',:
x = - ~ is calculated with inward adjustment. If th~re is
no binding approximation, x*. is taken to be the current
solution vector x = x**, the~same applies for f. = 0, or if
the approximation restriction is artificially s~tisfied.

Stage 2: Start a new or modified subsidiary problem

Assemble the pseudo-Lagrangean. Enter the primal and the dual

restrictions and the objective function limit, if any, in the
ordinary QP tableau. Set the upper limits of the dual variables.
Set the re-entry-parameter at REENTRY = 0; marking the normal
entry-mode, starting from the trivial basis.

Stage 3: Reapproximate and solve

When coming here in the normal re-entry-mode (REENTRY = 1),

reapproximate any quadratic restrictions which are not
represented by binding approximations, and enter their updated
form in the appropriate slots of the ordinary QP tableau.
Enter the ordinary QP algorithm, record the primal solution
vector obtained, and calculate the corresponding vector i(~).

Stage 4: Verify the primal solution vector

°
If the ordinary linear restrictions cannot be met, declare the
problem empty and exit. If f. > applies in relation with some
binding approximation-restri~tion, search for the "most
overtight" one (= maximal value of f.(x) • (l + p.». Move
the most overtight approximation par~metrically oGtwards, record
the solution vector obtained, and if the corresponding true

step, adjust dual variables upwards as follows: If f. <°

restriction is actually reached by the parametric adjustment

applies both before and after the parametric step, iAcrease

606 CHAPTER XVIII

the dual variable by 50% of what would apply with normal

transition under optimality. For the adjusted restriction
itself, use only 25% of the correction factor. If any overtight
approximation restriction has been signalled go back to
Stage 1. If, having established absence of overtightness there
are nevertheless artificial variables in the basis, declare
the true problem empty and exit.

Stage 5: Check on looseness

If, for some i, we find fi < °

in association with a non-
binding approximation-restriction, set REENTRY = 1, and go
back to Stage 3.
th
Stage 6: Check status (m+l) restriction slot

I f t h e ( m+ 1) th restr1ct10n
. . .1S b'1nd"1ng, 1n assoc1at10n
... W1t h
IN-INDEX> ° (= a second leg of a blocked superimposition),
increase the dual variable in question in the same way as
applies for normal transition under optimal form (see
section 18.7), and go back to Stage 2. If the (m+lr-restriction
is not binding, in association with IN-INDEX> 0, set
IN-INDEX = 0, put the updated form of the objective function
limit in the (m+l)th slot, set REENTRY = 1, and go back to
Stage 3. If the (m+l)th restriction is binding, in association
with IN-INDEX = 0, (= the objective function limit), proceed
directly to Stage 8.

Stage 7: Superimpose

If CUTN = 2 • Q (= the number of superimpositions made since

last going through either Stage °
or Stage 8 already is twice
the number of quadratic restrictions), proceed directly to
Stage 8.

If DUAL R I > 0, and for R = DUAL R I we find fr(x) < 0,

(= optimal form has not yet been restored after loss of optimal
form, and this restriction was critical in limiting the downward
adjustment of the dual variables, by becoming exactly fulfilled
in a parametric step) superimpose a new approximation of
fr(~)' regardless of the probability of successful super-
imposition. If DUAL R I = 0, search for the most violated
true restriction for which successful superimposition appears
probable. If no superimposition is indicated, proceed to
Stage 8, otherwise put the old binding approximation in the
(m+l)th slot, the new approximation in the "regular" slot for
the restriction to be superimposed, set IN INDEX according to
the restriction in question, set REENTRY = 1, and go back to
Stage 3.
GENERAL QUADRATIC PROGRAMMING 607

Stage 8: Verify optimal form

Set CUTN = O.

If optimal form is confirmed, set OPTIMAL = true

DUAL R I = 0, DUAL REDUCED = false. Copy the-a;ailable set of
current approximations in the matrix of back-copies, and
proceed to stage 9.

If optimal form is not confirmed, set OPTIMAL = false (do not

as yet disturb the logical variable DUAL REDUCED, this permits
to distinguish between not yet having restored optimal form and
having just lost optimal form).

Stage 8a: Correction of optimal form

If DUAL REDUCED = false (= optimal form was lost after last

going through Stag~as distinct from not yet having
restored it), set OLDSUM as being the sum of all dual variables.
Adjust the dual variables that are associated with f. < 0
upwards in the same way as would apply for normal tr~n;ition
under optimal form (=·middle round, but for technical reasons,
i.e. needing the undisturbed ordinary QP tableau, this is done
first) .

Form the correction restriction, put the updated form of its

approximation in slot m+2, make the dual parametric step, and
perform the first semi-proportional downward adjustment as
outlined in Section 18.8. If a particular f. is critical
in limiting the downward adjustment of the aual variables,
assign its index to DUAL R I, and make a new (tangential)
approximation.

Set NEWSUM equal to the sum of all dual variables. If

NEWSUM < 0.9 x OLDSUM, set OLDSUM equal to NEWSUM, otherwise
mUltiply all dual variables by 0.9 x OLDSUM/NEWSUM and
OLDSUM by 0.9.

Assemble a combined set of approximations from the current set

and the back-copy as follows: If DUAL REDUCED = false, choose
for each ith restriction the approximation take~the lowest
absolute value of f. (thus preferably selecting a tangential
approximation). Oth~rwise start with the back-copy, and replace
individual approximations by newer ones, only if a scan of their
coefficients proves that they exclude all vectors x excluded
by the old one. Put the selected set of approximations in the
back-copy. Set DUAL REDUCED = true, and go back to Stage 2.
608 CHAPTER XVIII

Stage 9: Test for final optimality

If any f. is found to be f. < -EPS (= negative and differing

from zer6 with a set toler1nce, e.g. EPS = 0.000001) proceed
to Stage 10.

If any residual shadowprice of a binding approximation-

I
restriction is found to be P*i I > EPS, also proceed to
Stage 10, the same applies to the shadowprice of the objective-
function limit, if binding. If none of these conditions is
present, declare the problem solved and exit. (N.B. p*. < 0
is possible with a non-convex pseudo-Lagrangean.) ~

Stage 10: Attend to the objective function limit

If the convexity requirements for formulating an objective

function limit are met (= all restrictions convex or peripherally
convex with a tangential approximation), copy the current set
of approximations in the back-copy, and write a new objective
function limit. If OPTIMAL = false, go back to Stage 8a.

Stage 11: Normal transition under optimality

Adjust the dual variables upwards, conform Section 18.7. Go

back to Stage 1.

END OF OUTLINE SUMMARY

It will be noted from the above outline-summary that there is

no exit on indication of unboundedness. The ordinary QP
algorithm has such an exit but in fact its activation is limited
to problems which contain variables of type absolute, "free"
variables. The computational implementation of the SCM algorithm
offered in the next section carries the facility to declare
variables "free" over into the SCM algorithm, but its use is
not part of the proof-supported application of the algorithm.

Equations can be used readily as far as linear equations are

concerned, but quadratic restrictions are inequalities only!
Initial approximations can be wide-out and to declare them as
equations gives rise to obvious problems of spurious emptiness.

We now corne to discuss non-convexity and the adaptations in the

ordinary QP algorithm made to the purpose of solving non-
convex problems. The QURO procedure (= Quadratic Restrictions
and Objective function), as listed in the next section, employs
an adaptation of the QUAP procedure from Chapter 17. These
adaptations relate not only to non-convexity but also to the
dependence of the objective function limit on the approximation
restrictions.
GENERAL QUADRATIC PROGRAMMING 609

We now summarise these adaptations as follows:

Firstly, when entering the adapted QUAP procedure in the normal

mode with REENTRY = 0, PR = PC = 0 (= asking for essentially
the same action as QUAD, solving a QP problem, starting from
the trivial hasis), the primal restrictions are divided into
three distinct categories, viz., the proper linear restrictions,
the approximations of the quadratic restrictions, and the
objective function limit. The main QURO procedure calls in
the compulsory "forced" convex mode, but the non-convex mode
of operation is employed nevertheless, until all linear
restrictions are satisfied.

During this "non-convex mode Phase I", the two other classes
of restrictions are totally disregarded by the search
operations. They may at this stage be flown through in either
direction, from satisfied to violated or vice versa.

Once the proper linear restrictions are satisfied, the convex

mode of operation is resumed. The result of this separate
treatment of proper linear restrictions is that emptiness of
a subsidiary problem comes in two modalities. If the proper
linear restrictions cannot be met, exit from the adapted QUAP
procedure is via the alarm exit for emptiness with REENTRY = -1.
Emptiness of the true problem is then established immediately,
irrespective of (non) convexity. If the approximation
restrictions or the objective function limit cannot be met
exit from the adapted QUAP procedure is technically successful,
but artificial variables are left in the basis. It is then
left to the main QURO procedure to analyse the implications
of this situation.

In a convex problem emptiness is also proven, but in a non-

convex problem the issue of over-tightness arises, and we may
try to have a go at getting better approximations. The
differential treatment of the approximations on the one hand
and the last restriction on the other hand is mainly motivated
by reasons which are at least as pertinent to convex problems
as to non-convex ones.

In the vicinity of the optimum, the objective function limit

converges to a linear combination of the other restrictions.
This gives rise to two problems, viz. (in)accuracy and a
danger of non-convergence. The accuracy problem might arise
if the objective function limit.became binding as well as the
restrictions on which it is (almost) dependent. At the optimal
solution itself such a solution cannot be developed as zero
pivots are not accepted, but, with incomplete convergence, an
ill-conditioned basis might be developed, giving rise to
accuracy problems.
610 CHAPTER XVIII

The other problem which might arise is that a consistent

solution might be developed, where on account of rounding
error, the objective function limit was more stringent than
the corresponding combination of the other restrictions.

One could end up with the pseudo-Lagrangean being constrained

by the objective function limit only, yet no final optimality
as a positive shadowprice of the objective function limit
indicates understatement of dual variables. (This problem can
only arise with a consistent solution as otherwise some
approximation would be classified as loose.)

Of these two problems the first one did actually arise during
testing with elements of the QP tableau-matrix indicating
multipliers above 1000 and falling out of the format-range
normally catered for by the tableau-printing procedure; the
second did not.

To counter both problems, the last restriction is (during

normal entry) excluded as a pivotal row as well as badname-
variable, until at the very end of the call to the ordinary
QP algorithm, when it is, if violated, declared badname.

However, if the absolute value of the slack is then found to

be less than a set tolerance, it is declared to be a zero
irrespective of its sign and return to the main algorithm
follows, without actually making the last restriction binding.

The third adaptation concerns the problem of getting the

"right" local optimum of the pseudo-Lagrangean. This is
understood as a local optimum in which approximation
restrictions rather than non-negativities or proper linear
restrictions are binding.

In terms of the internal structure of the ordinary QP algorithm,

there are two groups of textual adaptations which relate to
this problem. One is the selection of the badname variable:
shadow-pricesof approximation-restrictions are not selected
as badname-variable until all negative valued slack-variables
(other than the objective function limits), and negative-
valued shadowprices of non-negativities or proper linear
restrictions have been eliminated or made positive. The other
group of textual adaptations relates to "flying through". The
version of the ordinary QP algorithm discussed in Chapter 16,
permits and indeed prefers, to turn negative valued variables
directly into positive-valued variables.

The adaptation then consists of applying the "conservative"

version of the rule of the smallest quotient, insofar as
approximation-restrictions are concerned, i.e. to eliminate
GENERAL QUADRATIC PROGRAMMING 611

rather than turn positive, negative-valued slacks of

approximation-restrictions.

Finally by calling the ordinary QP algorithm in the "forced"

convex mode of operation, shadowprices of approximation-
restrictions are permitted to stay negative, if the restriction
in question displays manifest non-convexity in its slack-
variable subspace, by way of a positive non-zero diagonal
cell of the QP tableau.

This ~s a feature of the QUAP procedure in the version listed

in Chapter 17 as well, but it has been taken out as far as
proper linear restrictions and the last restriction (which can
be the second leg of an approximation in the re-entry mode)
are concerned.

For approximation-restrictions this device comes into its own;

these negative shadowprices are used in exactly the same way
as positive ones, leading, of course, to a reduction in the
dual variable in question.

In the convex case, convergence of the SCM algorithm is a

fairly obvious result of the presence of the objective function
limit and the ensured return to optimal form. We therefore end
this section by reviewing the likely effectiveness of the SCM
algorithm on non-convex problems where it lacks the support of
a proof of its effectiveness.

If we forget the constraint qualification problem, the necessary

and sufficient conditions for a constrained local maximum are:
complementary slackness (= consistence + optimal form), the
first-order conditions (= zer·o residual shadowprices of
approximation restrictions), and convexity within a subspace
of tangential approximations, which is ensured by finding a
locally constrained maximum of the subsidiary problem.

In short, activation of the successful exit of the SCM algorithm

proves that a local maximum of the true problem has been found.
There is, however, the problem of actually getting there. To be
precise, there are two problems on that score, viz. spurious
emptiness and lack of convergence.

The first problem arises because an approximation of a non-

convex problem can actually face the "wrong" way, i.e.
impose a requirement not to get anywhere near the true
solution.
612 CHAPTER XVlII

Example (from Section 15.5)

Maximise

Subject to

Xl + X z > 4

(0 < xl < 100, o ~ Xz ~ 100)

If the restriction xl + Xz
~ 4 is removed from this problem
(it was added in Section 15.5, solely to exclude the point
z
xl = X = 0), the trivial basis of the first subsidiary
problem is correctly identified as an optimum. But if the
linear restriction is put, starting as one normally would at
z
xl = X = 0, leads to finding the true problem empty.

The first subsidiary problem then is

z
Maximise - X z
Subject to xl + X z~ 4

6x + 8x Z < 0
1 -

(0 ~ xl ~ 100),

Even though the ordinary QP procedure, as adapted will satisfy

the linear restriction xl + X z ~ 4, this results in 6xl + 8x Z ~ °
being left with an artificial variable a 2 = 24 (xl = 4).
Since, at xl = 4, Xz
= 0 the true restriction is amply
fulfilled, no overtightness is diagnosed and the problem is
declared empty.

If, however, a starting solution is taken in the area

z
xl > 3, X > 4, this problem does not arise and we obtain a
solution. Which solution depends obviously on the particular
starting solution chosen.

To counter this possibility of spurious emptiness, the code

has been amended at a late stage suppressing the initialization
of x as x = 0, thus permitting the user to suggest a different
starting-solution.

Concerning the other problem of a possible lack of convergence

we may observe that despite the absence of an objective function
limit and therefore of a convergence proof in the non-convex
case, all examples both in this chapter and in chapter 15 were
duly solved.
GENERAL QUADRATIC PROGRAMMING 613

Exercise 18.9

The following general QP problem

(from section 15.3) is given

Maximise T = xl + x 2
2
Subject to x 1 -< 2, x 2 .2. 3, xl + x 2 < 5

(xl' x 2 -> 0)

Starting with x* = 2, x* = 2
1 2
apply the algorithm discussed in this chapter to its solution,
until neither the primal nor the dual solution can within the
tolerance of rounding to two digits after the decimal point,
be distinguished from the true solution xl = 2, x 2 = 1.73,
PI = ?7~, P3 = 0.29, (P2 =0) with the correct set of binding
restr~ct~ons.

Answer with tableau-listings at the end of this chapter; note

the implicit upper limits i.e. (0.2. xl .2. 2; 0 .2. x 2 .2. 3);
instead of explicit linear restrictions, the index of the
quadratic restriction therefore becomes 1.

18.10 Code listing of the SCM algorithm

Two stretches of programme-text are listed in this section.

They are a calling main programme, and the QURO procedure.
No text-listing of the modified version of the ordinary QP
algorithm is provided, but the substance of the difference
between the QUAP procedure as reported in section 17.7 and
the version employed here was surveyed in section 18.9.

The main programme listed below contains the lower limit

reinterpretation facility, as discussed in section 10.4 for LP
and in section 16.14 for ordinary QP. The generalizations of
this device to quadratic side-restrictions is catered for by
the code-text following the label REINTERPRET. The call to the
QURO procedure as surveyed in the previous section then relates
to the problem in the y variables with the lower limits at
zero.

The required presentation of the input-data ~s specified ~n

the comment in the main programme.

The output comes in two stages. First the dual variables of the
quadratic restrictions are reported by the main programme.
Thereafter the result reporting procedure discussed in
section 16.15 is called in unamended form, i.e. the primal and
614 CHAPTER XVIII

dual solutions of the last subsidiary problem are reported,

with re-interpretation according to the true lower limits. The
primal solution of this problem is identical to the solution-
vector of the true problem. However, the dual solution is
identical only insofar as this refers to the shadowprices of
linear restrictions be it non-negativities of specified lower
or upper limits, or binding linear side-restrictions. The
reported dual variables associated with the linear approximations
would normally be zero within the confines of rounding; the
exception would be where the problem was assumed to be empty.

The text listings indicated are now given, as follows:

TEXT-LISTING Of THE MAIN GENERAL QP PROGRAMME.

'BEGI N' 'I NTEGER' M. Q. N. NAV. NED. REENTRY. R. I.J. PC;

'BOOLEAN' OPTIMAL.FEASIBLE.EMPTY;

'PROCEDURE' QURO(M.Q.N.NEQ.NAV.PC.T.AB.D.X.U.P.ROWLST.COLLST,
OPTI MAL. FEASI. BLE. EMPTY);
'INTEGER' M.Q.N.NEQ.NAV.PC; 'ARRAY' T.AB.D.X.U.P;
'INTEGER' 'ARRAY' ROWLST.COLLST;
'BOOLEAN' OPTIMAL. FEASIBLE. EMPTY;
'ALGOL';

'PROCEDURE' MATI(MATR.MB.NB.FR.FC);
'ARRAY' MATR; 'INTEGER' MB.NB.FR.FC; 'ALGOL';
'PROCEDURE' TABO(MATR.M.N.SR.SC.RH.ER.ROWL.COLL);
'ARRAY' MATR; 'INTEGER' M.N.SR.SC.RH.ER;
'INTEGER' 'ARRAY' ROWL.COLL; 'ALGOL';
'PROCEDURE' REPQ(T.M.N.NEQ.NAV.ROWL.COLL);
'ARRAY' T; 'INTEGER' M.N,NED.NAV;
'INTEGER' 'ARRAY' ROWL. COLLI 'ALGOL';

'COMMENT'
GENERAL QUADRATIC PROGRAMMING ALGORITHM.
CONVEX QUADRATIC OBJECTIVE FUNCTION AND LINEAR AS WELL AS
CONVEX QUADRATIC SIDE RESTRICTIONS.
WHEN USED ON NON-CONVEX PROBLEMS. THE ALGORITH NOR-
MALLY WORKS. BUT THERE IS NO PROOF-SUPPORT IN THE
NON-CONVEX CASE.

FOR DETAILS OF THE ALGORITHM SEE THE TEXT OF THE PROCEDURE QURO.
PRESENTATION OF DATA:

FIRST THE TOTAL NUMBER OF RESTRICTIONS I.E. M.

THEN THE NUMBER OF QUADRATIC RESTRICTIONS, I.E. Q.
THfN THE NUMBER OF VARIABLES. 1. E. N.
FOLLOWED BY THE NUMBER OF EQUATIONS. NEQ.
 GENERAL QUADRATIC PROGRAMMING 615

THE NUMBER OF EQUATIONS SHOULD NOT EXCEED THE NUMBER OF

LINEAR RESTRICTIONS.
AS ONLY LINEAR EQUATIONS ARE ACCOMMODATED.

THEREAFTER. PUNCH NAV. THE NUMBER OF VARIABLES TO "'HICH

THE TACIT (NON-NEGATIVITY) RESTRICTION DOES NOT APPLY.
(IF NAV>O IS SUPPLIED. THE USER SHoL~D ENSURE
BOUNDEDNESS OF ALL SUBSIDIARY PROBLEMS. AS MALFUNCTIONING
MIGHT OTHERSISE ARISE.)

THEREAfTER PUNCH EACH ROW OF THE

M+3 BY N+2 COMPOSITE MATRIX
ABO
C F O. -I. OR +1
-W 0 0
U 0 0
L 0 0

THE M-Q BY N+ 1 LEADING BLOCK A. B REPRESENTING

A.X < OR = B
THE LINEAR RESTRICTIONS. (THE ZERO BEING IRRELEVANT)

AND THE Q BY N+I BLOCK C. F REPRESENTING

C.X < OR = F.
THE LINEAR COMPONENT OF THE QUADRATIC RESTRICTIONS.
THE K TM QUADRATIC RESTRICTION BEING
C(K)' • X 0.5 X'. D(K). X < OR = F<K).

HERE THE (N+2)ND ELEMENT HAS SIGNIFICANCE. AS FOLLOWS:

o CONVEXITY OF THIS RESTRICTION UNKNOWN.
CONVEXITY ASSUMED UNTIL SHOWN TO BE NON-CONVEX.
-I THIS RESTRICTION IS CONVEX.
INVESTIGATION OF CONVEXITY INHIBITED.
+1 THIS RESTRICTION IS NOT CONVEX.
THE M+I TH ROW REPRESENTS THE LINEAR COMPONENT OF THE
OBJECTIVE FUNCTION.
MAX I MI Z E W· • X + O. 5 X' • D • X

HERE THE (N+I)TH CELL CONTAINS THE INITIAL VALUE OF THE

OBJECTIVE FUNCTION. THE (N+2)ND CELL IS IRRELEVANT.

THE M+2 ND ROW OF THE LINEAR BLOCK IS RESERVED FOR THE

VECTOR U. REPRESENTING UPPER LIMITS

X < OR = U. (CELLS N+I AND N+2 IRRELEVANT)

THE M+3 0 ROW IS RESERVED FOR THE VECTOR L. OF LOWER

LIMITS. REPRESENTING
x > OR = L. (CELLS N+ 1 AND N+2 IRRELEVANT)
616 CHAPTER XVIII

THEREAFTER, NOT AS PART OF THE MAIN INPUT-MATRIX,

PUNCH THE SUGGESTED STARTING SOLUTION FOR X.

THEN PUNCH THE Q+ I SQUARE MATRI CES

0, DCI), O(2), •••••••• OCQ),
REPRESENTING THE QUADRATIC COMPONENTS OF THE OBJECTIVE FUNCTION,
ANO THE Q QUADRATIC RESTRICTIONS.

THESE MATRICES SHOl~D BE PUNCHED WITH THE SIGNS IN SUCH A

WAY, THAT THEY ARE NEGATIVE-SEMIDEFINITE IN THE CONVEX
CASE.

M:=REAO; Q:-REAO; N:=REAO; NEQ:=REAO; NAV:=REAO;

NEWLINEC I);
WRITETEXTC 'C'NUMBERIOFILINEARIRESTRICTIONS') ');
WRITEC30,FORMATC'C'S-NODOO')'),M-Q);
NEWLINEC I);
WRITETEXTC'C'OFIWHICHIEQUATIONS') ');
WRITEC30,FORMATC'C'S-NOOOO')'),NEQ);
SPACEC 5);
'IF' NEQ > M 'THEN' WRITETEXTC'C 'TOOIMANYIEQUATIONS') ');
NEWLINEC I);
WRI TET'EXTC 'C 'NUMBERIOFIQUAORATI CIRESTRI CTI ONS') ');
WRITEC30,FORMATC'C'S-NODOO')'),Q);
SPACEC 5);
'I F' Q > M
'THEN' WRITETEXTC'C'TOOIMANYIQUAORATICIRESTRICTIONS')');
NEWLINEC 2);
WRITETEXTC'C'NUMBERIOFIVARIABLES')');
WRITEC30,FORMATC'C'S-NOOOO')'),N);
NEWLINEC I);
WRITETEXTC 'C'OFIWHICHIOFITYPEIABSOLUTE') ');
WRI TEC 3 0, FORMATe' ( 'S-NOOOO') 'bNAV);
SPACEC 5);
'IF' NAV > N
'THEN' WRITETEXTC'C'TOOIMANYIOFITYPEIABSOLUTE')');
NEWLI N EC I);
'IF' NAV > 0 'THEN' WRITETEXTC'C'CAUTIONI%VARIABLES WITHOUTI
SIGNIRESTRICTIONlOOINOTIHAVEIUPPERILIMITS')');

'BEGIN' 'ARRAY' TAC 1:N+M+4, I:N+M+4],

ABC 1 : M+3, I: N+2] ,X, U,LC I: N]' PC I: M], DC I: N, 1: N, 0: Q+ I ]I

'INTEGER' 'ARRAY' ROWLST,COLLSTC 1:N+M+211

READ LINEAR PART:

MATICAB,M+3,N+2,0,0);
'FOR' J:=I 'STEP' I 'UNTIL' N '00' X[J]:=REAO-ABCI1+3,JlI

READ QUADRATIC PART:

 GENERAL QUADRATIC PROGRAMMING 617

'FOR' RI"O 'STEP' I 'UNTIL' Q ,~O' 'BEGIN'

MATI(TAININIOIO);
'FOR' 11 .. 1 'STEP' I 'UNTIL' N '00'
'FOR' JI .. I 'STEP' I 'UNTIL' N ,~O'
OCIIJIRll'"TACIIJH 'END';

PUT FANCYHIGH UPPERBOUNOS:

'FOR' JI=I 'STEP' I 'UNTIL' N '00'
'IF' ABCM+2IJ]=0 'THEN' ABCM+2 I JJ := 1000;

PUT LOWER LIMItS IN PLACEI

'FOR' J:=I 'STEP' I 'UNTIL' N '~O' LCJl := ABCM+3IJH

RE INTERPRET:

LINEAR PART OF INITIAL VALUE OF OBJ F:

'FOR' JI-I 'STEP' I 'UNTIL' N 'DO'
ABCM+IIN+I] 1= ABCM+IIN+ll - ABCM+I.JJ*LCJ];

QUADRATIC PART OF INI TIAL VALUE OF OBJ F:

'FOR' JI=I 'STEP' I 'UNTIL' N '~O'
'FOR' 1:=1 'STEP' I 'UNTIL' N 'DO'
ABCM+IIN+I] := ABCM+I.N+I] + O.S*LCIJ*DCIIJIOl*LCJ];

DIFFERENTIAL PART OF OBJ F:

'FOR' J:=I 'STEP' I 'UNTIL' N ,~O'
'FOR' 11=1 'STEP' I 'UNTIL' N 'DO'
ABCM+IIJl 1- ABCM+IIJ] - LCIJ*OCIIJIO);

LINEAR PART OF RHS OF RESI

'FOR' R:=I 'STEP' I 'UNTIL' M '00'
'FOR' J:-I 'STEP' I 'UNTIL' N ,~O'
ABCRIN+I) 1= ABCRIN+IJ - ABCRIJ]*LCJJ;

QUADRATIC PART OF RHS OF RESTR:

'FOR' RI=M-Q+I 'STEP' 1 'UNTIL' M 'DO'
'FOR' JI=I 'STEP' I 'UNTIL' N 'DO'
'FOR' 11 .. 1 'STEP' I 'UNTIL' N ,~O'
ABCRIN+I) 1= ABCRIN+I] + O.S*LCI)*DCIIJIR-M+Ql*LCJ);

DIFFERENTIAL PART OF RESTRI

'FOR' RI=M-Q+I 'STEP' I 'UNTIL' M '00'
'FOR' JI .. I 'STEP' I 'UNTIL' N ,~O'
'FOR' 11=1 'STEP' 1 'UNTIL.' N '00'
ABCRIJ] : .. ABCRIJJ - L.CI)*OCIIJIR-M+Q);

REINTERPRET UPPER LIMITSI

'FOR' Jp=1 'STEP' 1 'UNTIL.' N '~O' 'BEGIN'
ABCM+2IJ) 1= ABCM+2IJ)-ABrM+3IJJ;
'IF' ABCM+2IJ] < 0 'THEN' 'BEGIN'
NEWLINEC I);
WRITETEXTC'C'YOU%HAVE%SUPPL.IEO%A%LOWER%L.IMIT%IN%
EXCESS%OF%THE%CORRESPONOING%UPPER%L.IMIT')');
NE1rILINEC I);
WRI TETEXTC 'C 'THE%GENERAL%
QP%PROBL.EM%IS%THEREFORE%EMPTY.')');
'END';
'ENO'J
618 CHAPTER XVIII

PUT UPPERBOUNDS IN PLACE:

'FOR' J:=I 'STEP' I 'UNTIL' N '00' UCJJ:=ABCM+2,JH

NOW SOLVE:
QURO(M,Q,N,NEQ,NAV,PC,TA,AB,D,X,U,P,RoWLST,
COLLST,oPTIMAL, FEASIBLE, EMPTY);

POINT OF OUTPUT:
INTERPRET BACK:
'FOR' J:=I 'STEP' 'UNTIL' N 'DO'
TACN+M+PC+4,JJ:=LCJJ;

NEWLINE<1 )J
'FOR' 1:=1 'STEP' I 'UNTIL' M '00' 'BEGIN'
NEWLINE(I); WRITETEXT( '('DUAL%VARIABLE') ');
WRITE( 30, FoRMAT(' (' SNDDDD') '), I)J
WRITE( 30, FoRMAT(' (' S-NDDDD. DDD') '), PC I J); 'END';

'I F' 'NOT' FEASI BLE 'THEN'

WRITETEXT('('No%FEASIBLE%SoLUTION') ');

'IF' N+PC+M < 14

'THEN' TABO(TA,M+PC+N+I,M+PC+N+I,O,O,I,I,RoWLST,COLLST)
'ELSE' TABO(TA,M+PC+N+I,O,O,M+PC+N+I,O,I,RoWLST,CoLLST);

NEWLINE( I);
WRITETEXT('('SOLUTIoN%VALUE%IF%IN%STANDARD%FoRM%
AND%FEASIBLE')');
WRITE(30,FORMAT('('S-NDDDD.DDD') '),TACM+N+PC+2,M+N+PC+2J/2);
NEWLINE( I);

REPORT SOLUTION WITH REINTERPRETATION:

REPQ(TA,M+I,N,NEQ,NAV,RoWLST,COLLST);

'END'; 'END'

TEXT-LISTING OF THE QURo PROCEDURE.

'PROCEDURE' QURO(M,Q,N,NEQ,NAV,PC,T,AB,D,X,U,P,RoWLST,COLLST,
OPTIMAL, FEASI BLE, EMPTY);
'INTEGER' M, Q,N, NEQ,NAV, PC; 'ARRAY' T, AB, D,X, U, P;
'INTEGER' 'ARRAY' ROWLST,COLLST;
'BOOLEAN' OPTIMAL, FEASIBLE, EMPTY;

'BEGIN' 'INTEGER' I ,J, R,K, REENTRY, PR,NAME, CNAME,LAST, RR,

IN IND,II,JJ,DUAL R I, NNEGD, CUTN, DIRECTION;
'BOOLEAN' ADJUST, DUAL REDUCED;
'REAL' FANCYHIGH,LAMBDA,EPS,MOST NEGATIVE,LEAST POSITIVE,
MOST POSITIVE,CHECKNUM,COP,NUM,DISC,
OLDSUM,NEWSUM, NEXT NEGATI VE;
'ARRAY' S,CVCM-Q+I:MJ,APPR,oLDAPPRCM-Q+I:M+2,I:N+3J,
QUEQCO:2J,ROOTCI:2J;
GENERAL QUADRATIC PROGRAMMING 619

'PROCEDURE' QUAG(T.M.N.NEQ.NAV.NNEGD.ROVLST.COLLST.PR.PC.
NLINRES.REENTRY.LAMBDA);
'VALUE' NNEGD;
'ARRAY' T; 'INTEGER' M.N.NEQ.NAV.NNEGD.PR.PC.
NLI NRES. REENTRY;
'REAL' LAMBDA;
'INTEGER' 'ARRAY' ROWLST.COLLSn
'ALGOL' ;

'PROCEDURE' APPROXIMATE(R.PUT.ADJUST);
'VALUE' R; 'INTEGER' R.PUn 'BOOLEAN' ADJt'ST;
'BEGIN' 'INTEGER' TAKE;
'IF' R=O 'THEN' R:=M+II
TAKE := R;
'IF' R>O 'AND' R<M+I 'THEN' R:=R-M+Q;
'IF' R=M+I 'THEN' R:=O;
'IF' R=M+2 'THEN' 'BEGIN'
R:=Q+I;
'GOTO' NOW PUT THE APPROXIMATION; 'END';

'I F' R > 'AND' R < Q+I 'THEN' R:=R+M-Q;

'IF' ADJUST 'THEN' 'BEGIN'

'IF' SCRJ > 0 'THEN' DIRECTION := -I
'ELSE' DIRECTION := 11
'IF' PC=I 'OR' PR=I 'THEN' DIRECTION:=II
TCN+M+FC+PR+3.N+PC+RJ := 1 000 000 OOO*DIPECTION;
LIMIT(N+PC+R.R.DIRECTION);
, FO R ' I: = 1 's T EP' 1 ' UN TIL' N 'DO'
'IF' ROVLSTCIJ=I 'AND' TEI.N+PC+RJ>EPS
'THEN' 'BEGIN'
'IF' XCIJ/TCI.N+FC+RJ < TCN+FC+M+PP+3.N+PC+RJ
'THEN' TCN+PC+M+PR+3.N+PC+RJ:=XCIJ/TCI.N+PC+RH 'END';
, FO R' I: = 1 'S T EP , 1 ' UN TIL' N ' DO '
'IF' ROVLSTCIJ=I 'THEN' XCIJ := XCIJ-TCI.N+PC+RJ*
TCN+PC+M+PR+3.N+PC+RJ; 'END';

APPRCPt'T.N+3J := APPRCPUT.N+2J;

, Fo R' J: = 1 'S T EP , 1 ' UN TIL' N + 1 ' DO '

APPRCPUT.JJ:=ABCTAKE.JJ;

'IF' R> M-Q 'AND' R < M+I 'THEN' R:=R-M+Q;

NOW PUT THE APPROXIMATION:

'FOR' 1:=1 'STEP' 1 'UNTIL' N 'DO'
'FOR' J:=I 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'
APPRCPUT.JJ:=APPPCPUT.JJ - XCIJ*DCI.J.RJ;
APPRCPUT.N+IJ:=APPRCPUT.N+IJ-0.5*XCIJ*DCI.J.RJ*XCJJ;
, END';

'I F' R > 'AND' R < Q+I 'THEN' R:=R+M-Q;

'I F' ADJUST 'THEN' 'BEGIN'

'FOR' I: = 1 'STEP' 1 'UNTIL' N 'DO'
'IF'ROWLSTCIJ=I 'THEN' XCIJ := XCIJ+TCI.N+PC+RJ*
TCN+PC+M+PR+3.N+PC+RJ; 'END';
620 CHAPTER XVIII

'IF' PUT < M+I 'THEN' 'BEGIN'

APPRCPUT,N+2] := APPRCPUT,N+I];
'FOR' J:=I 'STEP' 1 'UNTIL' N '00'
APPRCPUT,N+21 := APPRCPUT,N+2]-APPRCPUT,J]*XCJ]; 'END';

'IF' R>O 'AND' R<M+I

'THEN' 'GoTo' END OF APPROXIMATION;

'IF' 'NOT' PUT=M+I 'THEN' 'GoTo' END OF APPROXIMATION;

'FOR' J:=I 'STEP' 1 'UNTIL' N 'DO'

APPRCM+I,Jl := -APPRCM+I,JH
APPRCM+I,N+Il := 0;
'FOR' J:=I 'STEP' 1 'UNTIL' N '00'
APPRCM+I,N+Il := APPRCM+I,N+Il+XCJ1*APPRCM+I,Jl;
'FOR' R:=M-Q+I 'STEP' 1 'UNTIL' M 'DO'
APPRCM+I,N+Il:=APPRCM+I,N+I] + PCR1*SCR1;
APPRCM+I,N+21 := APPRCM+I,N+Il;

END OF APPROXIMATION: 'END';

'PROCEDURE' UPDATE(R.PUT);
'INTEGER' R,PUT;
'BEGIN'
PUT AS COLUMN AND AS ROW:
'FOR' 1:=1 'STEP' 1 'UNTIL' N 'DO'
TCI,PUT]:=TCPUT.I]:=APPRCR,Il;
'FoR'I:=N+I 'STEP' 1 'UNTIL' N+PC+M+PR+2 'DO'
TCI.PUT]:=TCPUT,I]:=O;

POSTMULTIPLY BY MINUS THE PIVOT INVERSE:

'FOR' J ..·I 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'
'IF' CoLLSTCJl = -J 'THEN' TCPUT.Jl:=O;
'IF' CoLLSTCJl = 2000+J
'THEN' TCPUT.Jl := -TCPUT,Jl1 'END';

'FOR' J:=I 'STEP' 'UNTIL' N+M+PR+PC+I 'DO'

'FOR' 1:=1 'STEP' 'UNTIL' N '00'
'IF' RoWLSTCI] = I 'THEN'
TCPUT,J) := TCPUT.J) - TCI.Jl*TCI.PUT);

ATTEND DIAGONAL CELL:

TC PUT, PUT] : = 0;
'FOR' 1:=1 'STEP' 'UNTIL' N '00' 'IF' RoWLSTCI1=I
'THEN' TCPUT,PUTl := TCPUT,PUTl - TCPUT,IJ*TCl.PUTH

REVERSE SIGN FOR ARTIFICIAL VAR COLUMNS:

'FOR' 1:=1 'STEP' 1 'UNTIL' M 'DO'
'IF' Ro1JLSTCN+!] = 3000+1 'THEN'
TCPUT.N+!) := -TCPUT.N+I];

PUT CORRESPONDING COLUMN:

'FOR' 11=1 'STEP' 1 'UNTIL' N+PC+M+PR+I 'DO'
'IF' RoWLSTCI] > 0 'THEN' TCI.PUT]:=TCPUT,I)
'ELSE' TCI,PUT] 1= -TCPUT.I];
GENERAL QUADRATIC PROGRAMMING 621

PUT DUAL UL:

TCN+PC+M+PR+3.PUTJ:=SQRTCFANCYHIGH);

PUT UPDATED RHS :

RHS OF NORMAL RESTR:

'IF' R>M-Q 'AND' R<M+I
'THEN' 'BEGIN'
TCPUT.N+PC+M+PR+2]:=APPRCR.N+2J;
TCN+PC+M+PR+2. PUT]: =_oAPPRC R. N+2]; 'END';

END OF UPDATE: 'END';

'PROCEDURE' PREPARE ADJUSTMENTCPUT.R);

'INTEGER' PUT.R;
'BEGIN'
ATTEND COLUMN:
'FOR' 1:=1 'STEP' 'UNTIL' M+N+3 'DO'
'FOR' J:=PUT 'STEP' I 'UNTIL' N+M+3 'DO'
TCI.N+M+4+PUT-JJ:=TCI.N+M+3+PUT-J];
, 1 F' PUT=N + I ' TH EN '
'FOR' 1:=1 'STEP' I 'UNTIL' N. N+2 'STEP' 'UNTIL'R-I.
R+I 'STEP' I 'UNTIL' N+M+3 'DO' TCI.N+I]:=-TCl.N+I+R];

ATTEND ROW:
'FOR' J:= I 'STEP' 'UNTIL' N+M+4 'DO'
'FOR' I:=PUT 'STEP' I 'UNTIL' N+M+3 'DO'
TCN+M+4+PUT-I.JJ:=TCN+M+3+PUT-I.J];
'IF' PUT=N+I 'THEN'
'FOR' J:=I 'STEP' I 'UNTIL' N. N+2 'STEP' I 'UNTIL' R.
R+2 'STEP' I 'UNTIL' N+M+4 'DO' TCN+I.JJ:=-TCN+I+R.J];

'IF' PUT=N+I 'THEN' TCN+I.N+IJ:=TCN+I+R.N+I+RH

ADJUST NAMELI STS:

'FOR' II=N+M+2 'STEP' -I 'UNTIL' PUT+I 'DO' 'BEGIN'
ROWLSTC 1 J :=ROWLSTC I-I]; COLLSTC 1] I=COLLSTC 1-1 J; 'END';
'IF' PUT=N+I 'THEN' 'BEGIN'
COLLSTCN+IJI=N+I; ROWLSTCN+IJ:=-N-I; 'END'
'ELSE' 'BEGIN'
COLLSTCPUTJ:=-IOOO-PUT+N;
ROWLSTCPUTJ:=IOOO+PUT-N; 'END';

END OF ADJUSTMENT PREPARATION: 'END';

'PROCEDURE' LIMITCK.R.DIREC);
'I NTEGER' K. RI 01 REC;
'BEGIN'

'FOR' JI=O.I 'DO' QUEQCJJ := 0; QUEQ[2J := APPRCR.N+IJ;

, FO R' 1: = I ' 5 T EP , , UN TIL' N ' DO '

'IF' ROWLSTCI J=I 'THEN'
'FOR' J:=I 'STEP' 'UNTIL' N 'DO'
'1 F' ROWLSTCJ J=J 'THEN'
QUEQC011=QUEQCOl + 0.5*TCI.KJ*DCI.J.R+Q-Ml*TCJ.Kl;
622 CHAPTER XVIII

'FOR' J:= I 'STEP' I 'UNTIL' N '00'

'IF' ROWLST[JJ=J 'THEN'
QUEQ[IJ:=QUEQ[ll + APPR[R.Jl*T[J.KJ;

'FOR' J:=I 'STEP' I 'UNTIL' N 'DO'

QUEQ[2J := QUEQ[2J - APPR[R.JJ*X[JJ;

'IF' ABS(QUEQ[OJ) < 0.000000001 'THEN' 'GOTO' LIN ADJ;

CALCULATE 01 SC I
DISC := QUEQ[IJ*QUEQ[ll - 4*QUEQ[Ol*QUEQ[2l;

'I F' 01 SC < 0 'THEN' 'GOTO' NEAREST TO;

ROOT[I] := 0.5*(-QUEQ[ll+SQRT(DISC»/QUEQ[Ol;
ROOT[2l 1= 0.5*(-QUEQ[ll-SQRT(DISC»/QUEQ[OJ;

'IF' ROOT[1l*DIREC < 0 'THEN' ROOT[ll := ROOT[2];

'IF' ROOT[2l*DIREC < 0 'THEN' ROOTC2] := ROOT[I];
'IF' ROOT[ll*DIREC < 0 'AND' PR=I
'THEN' 'GOTO' END OF UPPER LIMIT CALCULATION;
'IF' ROOT[2l*DIREC < ROOT[ll*DIREC
'THEN' ROOT[1J:=ROOT[2lJ
'IF' ROOT[ll*DIREC < T[N+PC+M+PR+3.Kl*DIREC
'THEN' T[N+PC+M+PR+3.Kl :=ROOTr IlJ
'GOTO' END OF UPPER LIMIT CALCULATION;

LIN ADJI
'IF' QUEQ[ll = 0 'THEN' 'BEGIN'
'IF' 'NOT' PR=I 'THEN' T[N+PC+M+PR+3.Kl:=0;
'GOTO' END OF UPPER LIMIT CALCULATION; 'END';
'IF' DIREC*QUEQ[2l/QUEQ[ll > 0 'THEN' 'BEGIN'
'I F' 'NOT' PR= 1 'THEN' T[N+PC+M+PR+3.Kl : = 0;
'GOTO' END OF UPPER LIMIT CALCULATION; 'END';
'IF' -DIREC*QUEQ[21/QUEQ[ll < T[N+PC+M+PR+3.Kl*DIREC
'THEN' T[N+PC+M+PR+3.Kll= -QUEQ[2l/QUEQ[ll;
'GoTo' END OF UPPER LIMIT CALCULATION;

NEAREST TO:
'IF' -DIREC*QUEQ[ll/QUEQ[Ol < 0 'THEN' 'BEGIN'
'IF' 'NOT' PR=I
'THEN' T[N+PC+M+PR+3.Kl := 0;
'GoTo' END OF UPPER LIMIT CALCULATION; 'END';
'IF' -DIREC*0.5*QUEQ[IJ/QUEQ[01 < T[N+PC+M+PR+3.Kl*DIREC
'THEN' T[N+PC+M+PR+3.Kl:=-0.5*QUEQ[ll/QUEQ[Ol;

END OF UPPER LIMI T CALCULATION: 'END';

'REAL' 'PROCEDURE' CoRRECTION(R.I>;

'INTEGER'R.I;
'BEGIN' 'REAL' cn
CF := 0;
'FOR' JI=I 'STEP' 'UNTIL' N 'DO'
'IF' ROW'LST[Jl=J 'THEN' CF:=CF+AB[R.Jl*T[J.N+IlJ

'FOR' 111=1 'STEP' 'UNTIL' N 'DO'

'FOR' JJ:=I 'STEP' 'UNTIL' N 'DO'
'IF' RoW'LST[JJ1=JJ 'THEN'
CF := CF - X[IIl*D[II.JJ.R-M+Ql*T[JJ.N+Il;
'IF' CF > 0.5 'THEN' CF 1= I/CF 'ELSE' 'BEGIN'
'IF' CF < 0 'THEN' CF:=O.S 'ELSE' CF:=0.999; 'END';
GENERAL QUADRATIC PROGRAMMING 623

'IF' CF>O.999 'THEN' CF:=O.999;

CORRECTl ON : = CFJ
END OF THE CORRECTION CALCULATION: 'END';

'PROCEDURE' RECORD;
'BEGIN'
'FOR' J:=I 'STEP' 'UNTIL' N 'DO' 'BEGIN'
'IF' R01JLST[JJ=J 'THEN'
X[JJ:=T[J.N+PC+M+PR+2J;
'IF' ROIJLST[JJ = -J 'THEN' X[JJ:=O;
'IF' COLLST[JJ=2000+J 'THEN' X[JJ:=UCJH 'END';

CALCULATE QUADRATIC SLACKS:

'FOR' R:=M-Q+I 'STEP' 1 'UNTIL' M 'DO' 'BEGIN'
S[RJ :=AB[R.N+ 1 J;
'FOR' J:=I 'STEP' 1 'UNTIL' N 'DO'
S[RJ:=S[RJ-X[JJ*AB[R.JJ;
'FOR' 1:=1 'STEP' 1 'UNTIL' N 'DO'
'FOR' JI'=I 'STEP' 1 'UNTIL' N 'DO'
S[RJ := S[RJ + O.5*X[IJ*D[I.J.R+Q-MJ*X[JH 'END';

'FOR' I:=M-Q+I 'STEP' 1 'UNTIL' M 'DO'

'IF' SrIJ > EPS 'AND' 'NOT' ROIJLST[N+PC+IJ=IOOO+I
'AND' AB[I.N+2J=O 'THEN' AB[I.N+2J := S[IH

END OF PRIMAL SOLUTION RECORDING: 'END';

'PROCEDURE' COpy BEST;

'BEGIN'
'FOR' I := M-Q+I 'STEP' 'UNTIL'M 'DO' 'BEGIN'

'I F' 'NOT' DUAL REDUC ED 'AND'

ABS(APPR[I.N+2)<ABS(OLDAPPR[I.N+2J)
'THEN' 'GOTO' COpy;
'IF' NAV > 0 'THEN' 'GOTO' DO NOT COPY;
'IF' 'NOT' APPR[I.N+IJ*OLDAPPFHI.N+IJ>O
'THEN' 'GOTO' DO NOT COpy;

'FOR' J:=I 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'

'IF' APPR[I.JJ=O 'AND' 'NOT' OLDAPPR[I.JJ=O

'THEN' 'GOTO' DO NOT COPY;

'.IF' APPR[I.JJ=O 'THEN' 'GOTO' NEXT;

'IF' APPR[I.N+IJ > 0 'AND' APPR[I.JJ > 0 'AND'
AP?R[I.N+IJ/APPR[I.JJ>OLDAPPR[I.N+IJ/OLDAPPR[I.JJ
'THEN' 'GOTO' DO NOT COPY;

'IF' APPR[I.N+IJ < 0 'AND' APPRrI.JJ < 0 'AND'

AP?R[I.N+IJ/APPR[I.JJ<OLDAPPR[I.N+IJ/OLDAPPR[I.JJ
'THEN' 'GOTO' DO NOT COpy;
'IF" APPR[I.N+IJ > 0 'AND' APPR[I.JJ < 0 'AND'
APPR[I.N+IJ/APPR[I.JJ<OLDAPPR[I.N+IJ/OLDAPPR[I.JJ
'THEN' 'GOTO' DO NOT COPY;
'IF" APPRrI.N+l) < 0 'AND' APPR[I.JJ > 0 'AND'
APPR[I.N+I]/APPR[I.JJ>OLDAPPR[I.N+IJ/OLDAPPR[I.JJ
'THEN' 'GOTO' DO NOT COPY;
624 CHAPTER XVIII

NEXT: 'END';

COPY:
'roR' J:a·1 'STEP' I 'UNTIL' N+3 'DO'
OLDAPPRCI.JJ:=APPRCI.Jl;

DO NOT COPY: 'END';

'FOR' J: .. I 'STEP' I 'UNTIL' N+3 'DO'
'FOR' I:=M-Q+I 'STEP' I 'UNTIL' M '00'
APPRCI.JJ:=OLDAPPRCI.Jl;
END OF COPY BEST: 'END';

'COMMENT'
QUADRATIC PROGRAMMING ALGORITHM.
THE ABBREVARION QURO STANDS FOR
QU ADRATIC RESTRICTIONS AND o BJ ECTI VE FUNCTI ON.
N SPECIFIED VARIABLES.
M RESTRICTIONS. OF WHICH M-Q LINEAR RESTRICRIONS.
AND Q QUADRATIC RESTRICTIONS.

THE SIGNIFICANCE OF THE PARAMETERS M.N.NEQ.NAV.ROWLST AND

COLLST WILL BE OBVIOUS.
WHEN SEEN IN COMBINATION WITH THE QUAG PROCEDURE.
AB.D. AND U ARE INPUT-ARRAYS. X AND P ARE OUTPUT-ARRAYS.
AS FAR AS THE OPERATIVE USE BY THE QURO-PROCEDURE
IS CONCERNED (AS DISTINCT I:ROM STORING LIMITS).
AB IS ASSUMED TO BE OF ORDER M+I BY N+2.
THE (N+2)ND COLUMN IS OPERATIVELY USED. ONLY FOR THE
QUADRATIC RESTRICTIONS. TO STORE INFORMATION CONCERNING
THEIR CONVEXITY.
ABCR.N+2l < 0 MEANS CONVEX. NO VERIFICATION
ABCR.N+2J .. 0 MEANS PROBABLY CONVEX. TO BE VERIFIED
ABCR.N+2l > 0 MEANS NON-CONVEX.

THE LEADING M-Q BY N+I BLOCK-ROW IS RESERVED FOR THE

LINEAR RESTRICTIONS.
THE NEXT Q BY N+ I BLOCK-ROW roR THE LINEAR PART OF THE
QUADRATIC RESTRICTIONS.
THE 'EXTRA' ROW (INDEX "M+l) IS RESERVED
FOR THE LINEAR COMPONENT OF THE OBJECTIVE FUNCTION.

D IS ASSUMED TO BE A THREE-DIMENTIONAL ARRAY.

OF ORDER N BY N BY Q+2.
EACH OF THE Q+l SQUARE 'N BY N MATRICES. INDICES ZERO iO Q.
IS EXPECTED TO CONTAIN AN INPUT MATRIX D.
THE FIRST OF THESE MATRICES (INDEX ZERO) REPRESENTS THE
QUADRATI C COMPONENT OF THE OBJ ECTI VE FUNCTION.
MAXIMISE W' X + O.S X' D X.
THE OTHERS (INDICES I TO Q). REPRESENT THE QUADRATIC
COMPONENTS OF THE QUADRATIC RESTRICIONS
B + C' X + 0.5 X' D X > OR .. 0
THE LAST OF THE MATRI CES D. INDEX Q+ I. I S RESERVED FOR
INTERNAL USE BY THE QURO-PROCEDURE.
GENERAL QUADRATIC PROGRAMMING 625

THE N DIMENTIONAL ARRAY U IS ASSUMED TO CONTAIN THE VECTOR OF

UPPERBOUNDS ON X.
THE ARRAYS X AND P ARE SPACE-RESERVATIONS FOR THE VECTORS
OF PRIMAL AND DUAL VARIABLES.
OF THESE, X MUST BE PRE-FILLED BY THE USER, PIS INI-
TIALIzED AT ZERO BY THE PROCEDURE.

DUAL REDUCED := 'FALSE';

FANCYHIGH:=!OOOOJ EPS:=O.OOOOOIJ
DUAL R I := 0; OLDSUM:=O;

STAGE 0:
INITIATE AT ZERO:
'FOR' 1:=1 'STEP' 'UNTIL' N+M+2 'DO' R01olLSTCIJ:=0;
'FOR' J:=I 'STEP' 'UNTIL' N+M+2 'DO' COLLSTCJJ:=O;
'FOR' 1:=1 'STEP' 'UNTIL'M 'DO' PCIJ:=O;
, FO R' I: = M- Q+ I ' STEP' I ' UN TIL' M 'DO' SCI J : = 0;
ADJUST := 'FALSE'; PR:=PC:=OJ
'FOR' I:=M-Q+I 'STEP' I 'UNTIL' M+2 'DO'
, FO R' J: = I 'S T EP , 1 ' UN TIL' N + 3
'DO' OLDAPPRCI,JJI=APPRCI,JJ:=O;

STAGE I:
APPROXIMATE RESTRICTIONS:
IN IND:=O;
DUAL R I := CUTN := 0;
'FOR' R:=M-Q+I 'STEP' I 'UNTIL' M 'DO' 'BEGIN'
'IF' CoLLSTCN+PC+Rl = 1000+R
'THEN' ADJUST := 'TRUE' 'ELS.E' ADJUST := 'FALSE';
'IF' ADJUST 'THEN' APPROXIMATECR,R, 'FALSE');
APPRoXIMATEC R, R, ADJUST); 'END';

STAGE 21
SET THE PSEUDO LAGR:
FEASIBLE 1= 'TRUE';
PC: =PR: =RR: = 0;

CLEAN TABLEAU:
'FOR' 11=1 'STEP' 'UNTIL' N+M+4 'DO'
'FOR' JI=! 'STEP' 'UNTIL' M+N+4 'DO' TCI,JJ:=O;

ASSEMBLE PSEUDO LAGRANGEAN:

LINEAR PART OF PSEUDO L:

'FOR' 1:=1 'STEP' 1 'UNTIL' N 'DO' TCI,M+N+2J:=ABCM+I,IlJ
'FOR' RI=M-Q+I 'STEP' I 'UNTIL' M 'DO'
'FOR' 1:=1 'STEP' I 'UNTIL' N 'DO' TCI,M+N+2J:=
TCI,M+N+2J + PCRJ*ABCR,IJ;

QUADRATIC PART OF PSEUDO L:

, FO R ' I: =! ' STEP' I ' UN TIL' N 'DO'
'FOR' J:=I 'STEP' I 'UNTIL' N 'DO' 'BEGIN'
TCI,JJ := DCI,J,OJ;
, FO R' K : = I 'S T EP , I ' UN TIL' Q 'DO'
T(l,JJ := TCI,JJ + PCK-Q+MJ * D(l,J,KJJ 'END';
626 CHAPTER XVIII

ASSEMBLE REST OF TABLEAU:

PUT LINEAR RESTRI

'FOR' JI=I 'STEP' 'UNTIL' N '00'
'FOR' 1:=1 'STEP' 'UNTIL' M-Q '00' TCN+I~JJ := ABCI~J];
'FOR' 1:=1 'STEP' 'UNTIL' M-Q 'DO'
TCN+I~N+M+2J := ABCI~N+IJ;

PUT QUADRATIC APPROXIMATIONS AND EXTRA RESTR:

'FOR' JI=I 'STEP' I 'UNTIL' N '00'
'FOR' II=M-Q+I 'STEP' I 'UNTIL' M+I 'DO'
TCN+I~JJ 1= APPRCI~JJ;
'FOR' I:=M-Q+I 'STEP' I 'UNTIL' M+I 'DO'
TCN+I~N+M+2J 1= APPRCI~N+IJ;

PUT INITIAL VALUEI

TCN+M+2~N+PC+M+2J:=ABCM+I~N+IJ*2;

PUT UPPER LIMITS:

'FOR' JI=I 'STEP' I 'UNTIL' N '00' TCN+M+3~JJ:=UCJ];
'FOR' I:=N+I 'STEP' I 'UNTIL' N+M+I 'DO'
'IF' I<N+M-Q+I 'THEN' T[N+M+3~IJ:=FANCYHIGH
'ELSE' TCN+M+3~IJ:=SQRT(FANCYHIGH);

REENTRY:=O; LAMBDA:=O;

STAGE 3:
SOLVE:
NNEGD 1= N+M+IOO;

'IF' OPTIMAL 'AND' REENTRY> 0 'THEN' 'BEGIN'

'FOR' RI=M-Q+I 'STEP' I 'UNTIL' M 'DO'
'IF' ROYLSTCN+PC+RJ=IOOO+R
'AND' TCN+PC+R~N+PC+M+2]>0 'AND' SCR]<O
'THEN' 'BEGIN'
ADJUST := 'FALSE';
APPROXIMATE(R~R~ADJUST);
UPDATE(R~N+PC+R)J 'END'; 'END';

OPTIMAL := 'TRUE';

RECALL QUAGI
'IF' REENTRY=2 'THEN' NNEGD 1= 0;
QUAG(T~M+PR+I~N+PC~NEQ~NAV~NNEGD~ROWLST~COLLST~O~PC~
M- Q~ REENTRY ~ LAMBDA);

'IF' REENTRY=IOO 'THEN' 'BEGIN'

'COMMENT'
A NON-CONVEX SUBSIDIARY PROBLEM HAS BEEN ENCOUNTERED;
NEYLINE( I)J
YRITETEXT('('CARRY%ON%YITH%SOLUTION%OF%A%
NON%CONVEX%PROBL EM') , )J
'END';

'IF' ROWLSTCN+PC+M+I] = M+IOOI

'AND' 'NOT' (PC=I 'AND' ROYLSTCN+ I ]=N+ 1>
'AND' TCN+PC+M+I~N+PC+M+2] < 0 'THEN' 'BEGIN'
REENTRY I = I;
QUAG(T~M+PR+I~N+PC~NEQ~NAV~NNEGD~ROYLST~COLLST~O~PC~
M-Q~REENTRY.LAMBDA); 'END';
GENERAL QUADRATIC PROGRAMMING 621

COPY NEXT X:
'FOR' I:=M-Q+I 'STEP' I 'UNTIL' M 'DO' CVCIl:=SCIH
RECORD;

'IF' PC=I 'THEN' 'BEGIN'

'IF' ABS(SCK]) < 0.000001 'THEN' 'BEGIN'
'FOR' R:=M-Q+I 'STEP' I 'UNTIL' K-I.
K+I 'STEP' I 'UNTIL' M '~O'
'IF' SCR]<O 'AND' CVCR]<O
'AND' ROWLSTCN+I+R] = -IOOI-R 'THEN'
PCR] := PCRl + 0.5*TCN+R+I.N+M+3]*CORRECTION(R.R+I);
'IF' ROWLSTCN+I+K]=-IOOO-K 'THEN'
PCK] := PCK] + 0.25*TCN+I+K.N+M+3]*CORRECTIONCK.K+I);
'EN"O'; 'GOTO' STAGE); 'END';

STAGE 4:
CHECK ON OVERTI GHTNESS AND EMPTINESS:

CHECK EMPTINESS ON LINEAR RESTR:

'IF' REENTRY = -I 'THEN' 'BEGIN'
'COMMENT'
EMPTINES ESTABLISHED BY ORDINARY QP ALGORITHM
IN THE NON-CONVEX MODE. THIS RELATES SPECIFICALLY TO
LINEAR RESTRICTIONS.;
EMPTY 1= 'TRUE';
'GOTO' F'l NAL END OF QURO; 'END';
EMPTY := 'FALSE';
'FOR' 11=1 'STEP' I 'UNTIL' M-Q '~O'
'IF' ROWLSTCN+I]=3000+1 'THEN' EMPTY:='TRUE';
'IF' EMPTY 'THEN' 'GOTO' END OF QURO;

CHECK ON OVERTIGHTNESS OF BINDING APPR:

MOST POSITIVE := -I;
'FOR' I:=M-Q+I 'STEP' I 'UNTIL' M '00' 'BEGIN'
'IF' ROWLSTCN+I] = 3000+1 'AND' SCI]>EPS
'THEN" GOTO' STAGE );
'IF' ROWLSTCN+I] = -1000-1
'AND' SCI]>EPS 'AND' SCI]*<1+PClJ»MOST POSITIVE
'THEN' 'BEGIN'
K:=I; MOST POSITIVE := SCK]*C I+PCK]); 'END'; 'END';

'IF' MOST POSITIVE> 0 'THEN' 'BEGIN'

ABCK;N+2] : = );
PREPARE ADJUSTMENTCN+I.K);
TCN+M+4.N+IJ := I 000 000 000;
ADJUST I'" 'FALSE';
APPROXIMATE(K.K.AOJUST);
PC:=); LIMIT(N+I.K.I);
REENTRY : = 2;
'GOTO' RECALL QUAG; 'END';

CHECK EMPTINESS ON QUADRATIC RESTR:

EMPTY := 'FALSE';
'FOR' I:=M-Q+I 'STEP' I 'UNTIL' M '~O'
'IF' ABCl.N+2] > 0 'AND' ROWLSTCN+M+I] 3001+M
'THEN' 'BEGIN'
APPRCM+I.N+I] := I 000 000 000;
'GOTO' STAGE); 'END';
628 CHAPTER XVIII

'FOR' I:=M-Q+I 'STEP' 1 'UNTIL' M 'DO'

'IF' R0lo1LST[N+I]=3000+I 'AND' S[I]>O 'THEN' 'GOTO' STAGE J;
'FOR' I:=M-Q+I 'STEP' 1 'UNTIL' M+I 'DO'
'IF' ROlo1LST[N+I] = 3000+1 'THEN' EMPTY := 'TRUE';
'IF' EMPTY 'THEN' 'GOTO' END OF QURO;

STAGE 5:
CHECK ON LOOSE APPR:
'FOR' KI=M-Q+I 'STEP' 1 'UNTIL' M 'DO'
'IF' R0lo1LST[N+K]=1000+K 'AND' S[K] < -EPS
'AND' T[N+K~N+M+2] > EPS
'THEN' 'BEGIN'
REENTRY: = J; 'GOTO' STAGE 3; 'END';

STAGE 6:
CHECK ON M PLUS 1 RESTR:

'IF' ROlo1LST[N+M+I] = -IOOI-M 'AND' IN IND > 0

'THEN' 'BEGIN'
PrIN IND] 1= PrIN IND] +
T[N+M+I~N+M+2]*CORRECTION(IN IND~M+I);
'GOTO' STAGE 2; 'END';

'IF' R0lo1LST[N+M+J] = -IOOI-M 'THEN' 'GOTO' STAGE 8;

'IF' IN IND > 0 'THEN' 'BEGIN'

UPDATE(M+I~N+M+I);
IN IND := 0;
'I F' T[N+M+ I.N+M+2] < 0 'THEN' 'BEGIN'
REENTRY := J; 'GOTO' RECALL QUAG; 'END'; 'END';

STAGE 7:
SUPERIMPOSE:
NE>CT NEGATIVE := -I 000 000 000;
START OF APPROXIMATION SEARCH:
RI=O; MOST NEGATIVE p= -IO*EPS;

'IF' CUTN < 2*Q 'THEN' 'BEGIN'

'FOR' I:=M-Q+I 'STEP' 1 'UNTIL' M 'DO' 'BEGIN'
'IF' 'NOT' SrI] < MOST NEGATIVE
'THEN' 'GOTO' DO NOT IMPROVE;
'IF' 'NOT' SCI] > NE>CTNEGATIVE
'THEN' 'GOTO' DO NOT IMPROVE;
'IF' ABrI~N+2] > 0 'THEN' 'GOTO' DO NOT IMPROVEl
R:=I;
MOST NEGATIVE := S[RB 'END';

SAVE THE Ol.D ONE:

'FOR' J:=I 'STEP' 1 'UNTIL' N+I 'DO'
APPR[M+2~J]:=APPR[R~J];

ADJUST: ='TRUE';
APPROXIMATE(R~ R~ 'FALSE');
APPROXIMATE(R~R~AI>JUST); UPDATE(R~N+M+I);
'IF' ABS(T[N+M+I~N+M+I]) < EPS*EPS 'THEN' 'BEGIN'
T[N+M+I~N+M+I] := 0;
'GOTO' NOW PUT IMPROVEMENT; 'END';
'IF' T[N+R~N+M+I] < EPS 'THEN' 'GOTO' TRY NE>CT;
GENERAL QUADRATIC PROGRAMMING 629

'IF' R .. DUAL R I 'THEN' 'GOTO' NOW PUT IMPROVEMENT;

'IF' 'NOT' TCN+R.N+M+2J/TCN+R.N+M+Il <
TCN+M+I.N+M+2J/TCN+M+I.N+M+IJ
'THEN' 'GOTO' TRY NEXT;

NOW PUT IMPROVEMENTI

CUTN 1= CUTN+1I IN INDI=RJ
ROWLSTCN+RJI=IOOO+RJ ROWLSTCN+M+IJI=-IOOI-MJ
COLLSTCN+RJI=-IOOO-RJ COLLSTCN+M+I1:=1001+MJ

'FOR' .11=1 'STEP' 1 'UNTIL' N+M+3 '00' 'BEGIN'

NUMI.TCN+R.J1J TCN+R.J1:-TCN+M+I.J1J
TCN+M+I.J1I-NUM; 'END'J
'FOR' .11-1 'STEP' I 'UNTIL' N+M+3 'DO' 'BEGIN'
NUMI=TCJ.N+R1J TCJ.N+RJ:=TCJ.N+M+I1J
TCJ.N+M+ 11: "NUM; 'END' J

REENTRYI-I; NNEGDI=N+M+1000J
'GO TO , RECALL QUAG;

00 NOT IMPROVEI 'END';

TRY NEXTI
'IF' NEXT NEGATIVE> MOST NEGATIVE 'THEN' 'BEGIN'
NEXT NEGATI VE : = MOST NEGATI VEJ
RESTORE THE OLD ONEI
'IF' R>O 'THEN' 'FOR' J:=I 'STEP' I 'UNTIL' N+I 'DO'
APPRCR.J1 1= APPRCM+2.J1J

'GOTO' START OF APPROXIMATION SEARCHJ 'END'J

STAGE 81
VERI FY COMPLEMENTARY SLACKNESS:
OPTIMAL 1= 'TRUE'J
'FOR' R:=M-Q+I 'STEP' I 'UNTIL' M 'DO'
'IF' PCRl*SCR1 > EPS 'THEN' OPTIMAL:='FALSE';

'IF' OPTIMAL 'THEN' 'BEGIN'

SAVE OLD APPRI
'FOR' .11=1 'STEP' I 'UNTIL' N+3 '00'
'FOR' I:=M-Q+I 'STEP' 1 'UNTIL' M 'DO'
OLDAPPRCI.J1 1= APPRCI.JJJ
DUAL REDUCED I" 'FALSE' J
OLDSUM 1= OJ 'GOTO' STAGE 9J 'END'J

REDUCE DUAL:
NEWSUMI=OJ
DUAL R I I" OJ

INITIATE CORRECTION RESTR AT ZERO:

'FOR' JI=I 'STEP' 1 'UNTIL' N+I 'DO' APPRCM+2.J 1: = OJ
'FOR' Ire I 'STEP' I 'UNTIL' N '00 '
'FOR' .1:-1 'STEP' I 'UNTIL' N '00 ' DCl.J.Q+I1:=OJ

ASSEMBLE CORRECTION RESTR:

'FOR' RI=M-Q+I 'STEP' 1 'UNTIL' M '00' 'BEGIN'
CVCRl := OJ
'I F' 'NOT' DUAL REDUCED 'THEN' OLDSUMI=OLDSUM+PCRlJ
'IF' SCR1 > 0 'AND' 5CR1*PCR1 > 0 'THEN' 'BEGIN'
630 CHAPTER XVIII

'FOR' J:=I 'STEP' I 'UNTIl.' N+I 'DO'

APPRCM+2~JJ:=APPRCM+2~JJ-ABCR~JJ*PCRJ;
'FOR' 11=1 'STEP' I 'UNTIl.' N 'DO'
'FOR' JI=I 'STEP' I 'UNTIl.' N 'DO'
DC I ~J~Q+ I J I"DC I ~J~ Q+I J-DC I ~J~ Q-M+RJ*PCRJJ 'END ,;

'IF' SCRJ<O 'AND' ROWl.STCN+RJ=-IOOO-R 'THEN'

CVCR] I " TCN+R~N+M+2]*CORRECTIONCR~R)J 'END';

PREPARE ADJUSTMENTCN+M+2~0)J

PR:-I; APPROXIMATECM+2~M+2~'FAl.SE');
UPDATECM+2~N+M+2);
l.EAST POSITIVE 1= TCN+M+4~N+M+2] 1= 2;
COMPl.ETE LAST ELEMENT OF UPDATE:
TCN+M+2~N+M+3J 1= 0;
'FOR' Rp'M-Q+1 'STEP' I 'UNTIL' M '00' 'BEGIN'
'IF' SCR] > 0 'AND' SCRJ*PCRl > 0 'THEN'
TCN+M+2~N+M+3l p. TCN+M+2~N+M+3] - PCRl*SCRlJ 'END';

'FOR' RI"M-Q+I 'STEP' I 'UNTIL' M 'DO'

'IF' ROWLSTCN+Rl=IOOO+R 'AND' SCRl*PCRl>EPS
'THEN' 'BEGIN'
ADJUST := 'FAl.SE';
APPROXIMATECR~R~ADJUST);
LIMITCN+M+2.R.I);

'IF' TCN+M+4.N+M+2l < l.EAST POSITIVE

'THEN' 'BEGIN'
LEAST POSITIVE := TCN+M+4.N+M+2l;
DUAL R I I = R; , EN D ' ; , EN D ' ;

CALL QUAG FOR DUAl. ADJUSTMENT:

NNEGD :e 1000;
REENTRYI=2; PR:=I; LAMBDAI=O;
QUAGCT.M+2~N+PC~NEQ~NAV~NNEGD~ROWl.ST~COl.l.ST~PR~PC~
M-Q~ REENTRY ~LAMBDA);

'IF' LAMBDA> I 'THEN' l.AMBDA 1= I;

LAMBDA I - LAMBDA*OLDSUM;
'FOR' RI=M-Q+I 'STEP' I 'UNTIL' M '00' 'BEGIN'
'IF' SCRl*PCRl>EPS 'THEN' 'BEGIN'
'I F' 'NOT' R = DUAl. R I
'THEN' PCR] := PCRl - l.AMBDA*PCR1/OLDSUM
'ELSE' PCRl := PCR] - 0.5*l.AMBDA*PCRl/0l.DSUM; 'END';
'IF' CVCR] > 0 'THEN' PCRl:=PCR1+CVCRJJ
'IF' PCR] < 0 'THEN' PCRl 1= 0; 'END';

SELECT BEST APPRI

'IF' DUAL R I> 0 'THEN' 'BEGIN'
RECORD;
APPROXIMATECDUAl. R I~DUAl. R l~ 'FAl.SE'); 'END';
COPY BEST;
CUTN 1= 0; DUAl. REDUCED 1= 'TRUE';
GENERAL QUADRATIC PROGRAMMING 631

'FOR' R:=M-Q+I 'STEP' I 'UNTIL' M '00'

NEWSUM : = NEWSUM+PC RH
'IF' NEWSUM < O.9*oLOSUM 'THEN' 'BEGIN'
oLOSUM := NEWSUM; 'GoTo' STAGE 2; 'END';
'FOR' R:=M-Q+I 'STEP' I 'UNTIL' M '~O'
PCRl := O.9*PCRl*oLOSUM/NEWSUM;
oLOSUM := O.9*oLOSUM;

'GO TO , STAGE 2;

ST·AGE 9:
INVESTIGATE FINAL OPTIMALITY:
OPTIMAL: = FEASI BLE: =' TRUE'; EMPTY: = 'FALSE';
'FOR' R:=M-Q+I 'STEP' I 'UNTIL' M '~O' 'BEGIN'
'IF' SCRl < -EPS 'THEN' FEASIBLE := 'FALSE';
'I F' RoWLSTCN+Rl< 0 'AND'
ABS(TCN+R.N+M+2l) > EPS 'THEN' oPTIMAL:='FALSE';
• END';

'IF' RoWLSTCN+M+ll = -IOOI-M

'AND' ABS(TCN+M+I.N+M+2l) > EPS
'THEN' OPTIMAL := 'FALSE';

'IF' OPTIMAL 'AND' FEASIBLE 'THEN' 'GoTo' END OF QURo;

OPTIMAL := 'TRUE';

STAGE 10:
CHECK APPLICABILITY OF oBJ F LIM:
'FOR' R:=M-Q+I 'STEP' I 'UNTIL' M '~O'
'IF' RoWLSTtN+Rl = -IOOO-R 'AND' ABCR.N+2l > 0
'THEN' 'GoTo' STAGE II;
'FOR' R:=M-Q+I 'STEP' I 'UNTIL' M '~O'
'IF' RoWLSTCN+Rl = -IOOO-R 'AND' ABCR.N+2l < 0
'AND' ABS(APPRCR.N+2l) > EPS
'THEN' 'GoTo' STAGE III

CALCULATE NEW OBJECTIVE FUNCTION LIMIT:

COPY 01.0 APPR IF LAST ONE oPTIM:
'FOR' J:=I 'STEP' I 'UNTIL' N+3 '00'
'FOR' I:=M-Q+I 'STEP' I 'UNTIL' M '~O'
oLOAPPRCI.Jl:=APPRCI.Jl;
ADJUST := 'FALSE';
APPRoXIMATE(O.M+I.ADJUST);

STAGE II:
ADJUST DUAL VARIABLES:

'FOR' 1:=1 'STEP' I 'UNTIL' M '~O' 'BEGIN'

'IF' 'NOT' I>M-Q 'THEN' 'BEGIN'

STATE SHAOoWPRICE OF LINEAR RESTR:
'IF' ROWLSTCN+Il = 1000+1 'THEN' PCI1:=0;
'IF' RoWLSTCN+Il=-IOOO-I 'THEN'
PCIl:=TCN+I.N+M+2l;
'GoTo' END Of' PRICE ADJUSTMENT LOOP; 'END';
632 CHAPTER XVIII

ATTEND UPWARD ADJUSTMENT FOR A QUADRATI C PRI CEs

"IF" SCIJ<EPS "AND" COL.L.STCN+IJ .. 1000+1
"THEN" PCIls=PCIl+TCN+I.N+M+21*CORRECTIONCI.I);
"IF" PCIl<O "THEN" PCIl:=O;
END OF PRICE ADJUSTMENT L.OOPs "END";

"IF" ROWL.STCN+M+Il=-IOOI-M "AND" IN IND > 0 "THEN" "BEGIN"

PCIN IND] s= PCIN IND] +
CORRECTIONCIN IND.M+I)*TCN+M+I.N+M+2l;
"IF" PCIN IND] < 0 "THEN" PUN INDJ Se 0; "END";

"GOTO" STAGE 1J

END OF QURO;

"IF" EMPTY "THEN" "BEGIN"

NEWLINEC 1 H
WRITETEXTC "C "PROBL.EM%ASSUMED%TO%BE%EMPTY%ON%ACCOUNT%
OF%OCCURRENCE%OF%ARTIFICIAL%VARIABL.ES") "); "END";

REAPPROXIMATE L.INEAR SL.ACKS:

"FOR" RI-M-Q+I "STEP" 1 "UNTIL." M "DO"
"IF" ROWLSTCN+RJ"IOOO+R "THEN" "BEGIN"
ADJUST :- "FALSE";
APPROXIMATEC'R. R.ADJUSTH UPDATEC R.N+RH "END";

CORRECT SOL.UTION VALUE FOR CONSTANTS:

"FOR" I ;"M-Q+I "STEP" 1 "UNTIL." M "DO"
TCN+M+2.N+M+2J;=TCN+M+2.N+M+2J+ABCI.N+IJ*PCIJ*2;

CORRECT SOL.UTION VALUE FOR INITIAL. VALUE:

TCN+M+2.N+M+21 := TCN+M+2.N+M+21 + 2*ABCM+I.NJ;

FINAL END OF QURO: "END" ;

To help the reader of this book to read back the text-

listing of the QURO procedure, it may be useful to point out
some subordinate internal procedures, whose significance may
be s~mmarized as follows:

The procedures approximate and update:

To calculate a linear approximation, and to calculate its

current form in the updated QP-tableau.

The procedure prepare adjustment:

To re-order the tableau making room for another activity or

restriction.

The procedure limit:

To calculate A, as defined by (18.3.4) and implement

appropriate modifications of the parametric adjustment
technique.
GENERAL QUADRATIC PROGRAMMING 633

Answer to exercise lB.9

Stages 0 - 3

Our first subsidiary problem is:

Maximise xl + x 2

Subject to xl + 4 x 2 < 9 (the linear approximation)

(0 < xl ~ 2, 0 ~ x2 < 3)

The objective function limit is at this

stage trivial (0 ~ 0).

Stages 4 - 6

The optimum solution of the first subsidiary problem (See

tableau lB exc 1) xl at this upper limit of 2; x 2 = 1.75.

There are no overtightness, no loose approximation, and no

binding extra-restriction.

Stage 7

The (one) most violated true restriction is re-approximated,

with adjustment to xl = 2, x 2 = 1.73, the true quadratic
restriction being binding at that point. The new approximation
is xl + 3.46 x 2 ~ B.

Tableau lB exc 2 gives the new approximation as violated

restriction in the sl-slot, the binding old approximation in
the P2-slot. The new approximation is probably superimposable,
on account of the zero in the sllPl cell. Hence we return to
stage 3.

Stages 3 - 5

After re-entering the ordinary QP algorithm, the optimum of

the modified subsidiary problem is found at the consistent
solution vector xl = 2, x 2 = 1.73. (See tableau 18 exc. 3).

There is no overtightness and no loose approximation.

Stage 6

We replace the old approximation xl + 4 x 2 ~ 9 which is now

amply fulfilled by the objective function limit. As the latter
so far has the trivial form 0 ~ 0, no listing of this
modification of the subsidiary problem is given.
634 CHAPTER XVIII

Stage 7

No superimposition is called for, the solution-vector being

consistent.

Stage 8

We find the condition of optimal form satisfied.

Stage 9

We proceed to stage 10, on account of the linear shadowprice

pi = 0.29.

Stage 10

We form the objective function limit xl + X2 < 3.73.

Stage 11

The upper limit on the correction-factor is binding therefore

PI 0+ 0.999 pi = 0 + 0.999 x 0.29 = 0.29

(As the current primal solution is a consistent one, cf = 1

would have let to hitting the true dual variable exactly,
as it is, the figure cannot be distinguished within the two
decimals accuracy of presentation). We return to stage 1.

Stages 1 - 3

The second subsidiary problem is

2 2
Maximise xl + x 2 + 0.29(5-x l -x 2 ) = O.71x l + x 2 -0.58x l + 1. 45

Subject to xl + 3.46 x 2 2. 8 (app roxima ti on)

x 2 + x 2 < 3.73 (objective function limit)

(0 < xl < 2;

Stages 4 - 9

The optimum solution vector of the second subsidiary problem

(See tableau 18 exc 4) is found to be

Xl at its upper limit of 2; x 2 = 1.73

pi = 0.00
 GENERAL QUADRATIC PROGRAMMING 635

No further action is called for until stage 9, when we

conclude to final optimality.

(The programmed code, operating with a finer tolerance still

finds p* = 0.00029 at this point, and makes one round more,
but wit~in the two decimals fixed point presentation, the optimal
tableau cannot be distinguished from tableau 18 exc.4).

ADDENDUM TO THE ANSWER OF EXCERCISE 16.9

SOME TABLEAUX DEVELOPED IN SOLVING EXCERCISE 16.9

(NON-MEANINGFUL UPPER BOUNDS REPLEACED BY XX)

16 EXC I 18 EXC 2
FIRST SUBSIDIARY OPTIMUM. NR I, WITH NEW APPR ADDED.

NA.! BI D2 51 P2 VAL. NA.! BI D2 P2 S2 VAL.

UI I - 0.25 0.75 Ul ! - 0.25 0.13 ! 0.75

X2 !-0.25 0.25 1.75 X2 !-0.25 0.25! 1.75
PI ! - -0.25 0.25 SI ! - 0.13 -0.87!-0.06
S2 ! - 0.00 P2 ! - -0.25 0.87 I 0.25

2T 0.75 -1.75 0.25 -O.OO! 3.50 2T 0.75 -1.75 0.06 0.25! 3.50
UB ! 2 XX XX XX! XX UB ! 2 XX XX XX! XX

16 EXC 3 18 EXC 4
OPTIMUM OF MODI F1 ED PROBL EM. THE FINAL OPTIMUM.

NA.! BI D2 51 P2 I VAL. NA.! BI D2 SI P2 ! VAL.

--------------------------------- ---------------------------------
UI ! - 0.29 - O. 15 ! 0.71 UI !-0.05 0.29 O. 05 0.71 ! 0.71
X2 !-0.29 0.29 1.73 X2 ! - 0.29 0.29 1.73
PI
52
! -
! 0.15
-0.29
- I • 15
I. 151 0.29
0.07
PI
S2
! 0.05 -0.29
1-0.71
-0.05
- 0.29
0.29 ! 0.00
0.00
--------------------------------- ---------------------------------
2T 0• 7 I - I • 73 0.29 -0.07! 3.46 2T 0.71 -1.73 0.00 -O.OO! 7.46
UB ! 2 XX XX XX! XX UB I 2 XX XX XX! XX
Part V
INTEGER PROGRAMMING

CHAPTER XIX

INTEGER PROGRAMMING AND SOME OF ITS APPLICATIONS 637

19.1 Some integer programming terms 637

19.2 Fixed costs and decision variables 638
19.3 Non-convex restrictions and dummy variables 641

CHAPTER XX

BRANCHING METHODS 656

20.1 Branching in the order of increasing indices 656

20.2 Branching in the general mixed integer case 668
20.3 Branching methods developed by other authors 689
20.4 Text-listing of a recursive branching procedure 690

CHAPTER XXI

THE USE OF CUTS 702

21.1 Elementary cuts, augmented cuts and combined

cuts 702
21.2 Classes of cuts 714
21. 3 I The subsidiary cut 717
21.4 An integer programming adaptation of the
linear programming procedure 724
21.5 The coding of integer requirements and cuts 739
21.6 Summary of the Cutting Algorithm 741
21.7 Commented text of a mixed integer
programming procedure 752
636
CHAPTER XIX

INTEGER PROGRAMMING AND SOME OF ITS APPLICATIONS

19.1 Some integer programming terms

The term integer programming is normally applied in the context

of linear programming. An integer programming problem differs
from the corresponding "normal" linear programming problem, by
the additional condition that some (or all) variables must have
integer values in any admissable solution.

For example, we may have the following problem

Maximise

T = xl + 1.5x 2 + 0.5x 3

Subj ect to

xl + 0.5x 2 < 5

0.5x 2 < 5

(Xl' x 2 ' x3 ~ 0, xl integer-valued)

This is a mixed integer programming problem

If the requirement that variables should attain integer values

applies to all variables, we speak of an all integer
programming problem.

In the mixed integer programming problem the integer

requirement applies to some, but not necessarily to all
variables, (i.e. to xl but not to x 2 and x3 in the example).

Obviously the mixed integer problem is the general case. Any

algorithm which is effective with respect to the mixed integer
problem would cope with "ordinary" linear programming
problems, and also with all-integer problems as special cases
of mixed integer problems.

In a mixed integer problem the variables may be separated into

two groups, the integer-restricted and the continuous variables.

637
638 CHAPTER XIX

A special type of integer-restricted variable is the zero-one

variable. As the name indicates, a zero-one variable is a
variable for which only two values are admitted, zero and one.

A linear programming problem, which differs from a (mixed)

integer programming problem, only by the variables not being
required to attain integer values, is called the corresponding
continuous problem. The optimal and feasible solution of such
a corresponding continuous problem (if there is an optimal
solution) is then called the (corresponding) continuous
optimum.

It should obviously be understood that, in a corresponding

continuous problem, zero-one variables are restricted to the
interval from zero to one. The restrictions ~hich are written
explicitly with coefficients, e.g. xl + x 2 + x3 ~ 20,
xl + 0.Sx 2 ~ 5 etc., are indicated as the specified restrictions,
to distinguish them from the integer requirements, i.e. the
requirements that integer-restricted variables attain integer
values and of course, also from the tacit non-negativity
requirements.

A solution-vector which satisfies the specified restrictions

(and any non-negativity requirements) is then a feasible
solution of the continuous problem or for short a solution of
the continuous problem.

A genuinely feasible solution which satisfies not only the

specified restrictions but also the integer requirements is then
called an integer solution.

We will use the term "integer solution" irrespective of the

fact that in a mixed integer programming problem, some variables
are not restricted to integer values and may attain (in an
"integer solution" fractional values.

The true optimum may then also be indicated as the integer

optimum, to distinguish it from the continuous optimum.

19.2 Fixed costs and decision variables

Mixed integer problems in which the integer-restricted

variables are zero-one variables, are the most frequently
applied kind of integer programming problem.

They arise mainly from two types of problems which the non-
specialist will not immediately recognise as integer
programming problems.
INTEGER PROGRAMMING 639

The case of fixed costs ("overheads") is relatively

straightforward.

In the fixed-cost problem, the integer variable is a decision

variable which indicates whether or not a particular activity,
e.g. to build a factory, to operate a machine, etc., is at all
engaged. Once the factory is built, the machine bought, etc.,
certain fixed costs are incurred. These fixed costs are
independent of the activity level. (See also Westphal [39])

For example, a firm may be required to transport lS,OOO tonnes

of coal per month.

This will involve costs of transport and the method of trans-

portation will have to be decided. Obviously, one will wish to
minimize the cost of transport. If the coal is sent by road,
the firm can use the services of road hauliers and no fixed
costs for the firm arise. For bulk-transport the use of a
railway or a ship may be cheaper, but would require that a
rail-connection to the factory, or a loading berth be built
first. Once these investment decisions were made, regularly
recurring costs of interest and amortization would arise,
irrespective of the amount of coal actually transported. Total
transport costs are then the sum of the cost of road-transport
(per tonne only), the fixed cost of building the railway (if
built), the fixed cost of building the loading berth for mari-
time transport (if the firm chooses to use this mode of
transport at all), and the variable costs of both rail and
maritime transport. The problem of minimizing transport cost
might now, for example be

Minimise

Subject to

70,000x 4 > Xs
xl + x3 + Xs ~ lS,OOO

(xl' x 2 ' x 3 ' x 4 ' Xs > 0, x 2 and x 4 zero or one)

The interpretation would be (for example) as follows: xl' x3

and Xs are the quantities transported by road (xl)' by
rail (x 3 ), and by sea (x S ). These involve variable costs of
2.S, 1.S and 0.4 units of money per tonne. The decision
variable x 2 is then the decision to build the railhead. If
640 CHAPTER XIX

x2 is zero the 600 units of fixed costs are not involved, but
then no rail transport is possible. If the railhead is built
(x2 = 1), the 600 units of fixed costs are incurred, enabling
transport by rail up to the railhead's maximum capacity of
9000 tonnes. The same principle applies to maritime transport.
If the loading berth is not built (x4 = 0) the restriction that
x5 be not greater than 70,000 x prevents a non-zero value of
x5. The above illustrated use 01
decision variables represented
as zero-one variables can obviously be combined with restrictions
of a different nature, e.g. market limits, relations with other
production processes, etc.

The one additional point which is also useful to mention here ~s

the treatment of general increasing returns to scale. Suppose
the relationship between production capacity and cost of
investment is verified for certain realistically considered
sizes of a particular installation

Cost of Investment Capacity

Size No.1 500 1000

Size No.2 700 2100
Size No.3 900 3500

This example differs from the slightly simpler case of "set-up

costs" which are independent from the scale of operations.

If the fixed costs were set up cost, plus a linear function of

the scale, the tabulation could, for example be as given in the
"proportional capacity" version of the tabulation below

Initial Proportional Total Cost Propor- Actual

Set-up Additional of tional Capacity
Cost Cost Investment Capacity

Size No.1 340 160 500 1000 1000

Size No.2 340 360 700 2250 2100
Size No.3 340 560 900 3500 3500

i.e. the cost of investment ~s 340 units plus 16 per 100 units
of capacity.

As the example was actually put, there are increasing returns to

scale at all investigated scales, and size No.2 only has 2100
units of capacity for 700 units of cost. With generally increasing
returns to scale, we need a decision variable for every
considered size. One simply writes the appropriate relations
three times, as if it were seriously considered to build all
three installations of sizes No.1, 2 and 3. The three decision
INTEGER PROGRAMMING 641

variables will be indicated as, for example xl' x 2 ' x 3 . If it

is simply uneconomic to build two or three similar installations
that is all, only one will figure in the optimum solution. If
it is physically impossible to build two similar installations
on the same site, one may add that restriction to the model,
i.e., xl + x 2 + x3 ~ 1.

19.3 Non-convex restrictions and dummy variables

One may also use dummy variables to represent a general non-

convex restriction.

Example

Maximise

'[ 2y - x

Subject to

y 1 + 4x - 4x 2 + x 3

x < 3 (x > 0, y ~ 0)

The true shape of the restriction y < 4x - 4x 2 + x 3 + 1 has

the typical S-shape of a cubic restrIction and is clearly
non-convex.

Provided the true non-linear restriction does not have any

near-vertical segments, we may approximate it to any degree of
required precision by means of linear segments of predetermined
length. The more precise approximation we require, the more
segments will be needed. For example, with intervals of 0.5
for the value of x, we have the following tabulation (see also
the associated graph).

x = 0.00 y 1.00

Yl ~ 1 + 2.25 xl (0 < xl < 0.50)

x = 0.50 y = 2.13

Y2 ~ 2.25 - 0.25 x 2 (0.50 < x 2 < 1. 00)

x = 1.00 y = 2.00

Y3 ~ 3.25 - 1. 25 X z (1.00 < x3 < 1. 50)

642 CHAPTER XIX

graph 19.3 a

J.O
y approximat ion
of the cubic funct ion
y ~ 1 + ~x - 4x 2 + x 3
in six linear ~e9ments.
2.5

2.0

1.5

1.0

0.5

0.5 1.0 I.S 2.0 2.5 3.0

 INTEGER PROGRAMMING 643

x = 1. 50 y = 1. 37

Y4 .2. 2.50 - 0.75 x 4 (1. 50 .2. x 4 < 2.00)

x = 2.00 Y = 1.00
y - 1. 50 + 1. 25 x5 (2.00 .2. x5 < 2.50)
5
x = 2.50 y = 1.63

y 6 .2. -10.25 + 4.75 x6 (2.50.2. x6 < 3.00)

x = 3.00 y = 4.00

There remains the problem of actually linking the "true"

values of x and y to these segments. We will discuss this issue
at first, only for x. We introduce a series of zero-one
restricted variables, d l , d 2 ..... d 6 , and a series of
requirements

x - x <
1
x - x < etc.
2
To identify x with just one segment we add the equation

~.e. five of the dummy variables are equal to one, and one of
them is zero.

The restriction is formally written in terms of the dummy-

representations of x ~.e.

y < 2.25 xl + 1

y .2. -0.25 x 2 + 2.25

y < -1.25 x3 + 3.25

y < -0.75 x 4 + 2.50

y < 1. 25 x - 1. 50
S
y < 4.75 x6 - 10.25
The dummy representations of x are defined by the intervals, e.g.

0.2. Xl .2. 0.5,0.5 .2.~.2. 1.0, 1.0.2.~.2. 1.5 etc.

644 CHAPTER XIX

The variable x has to be equal to one of them, i.e. if the

corresponding zero-one variable d 2 is zero the pair of
restrictions

-99 d 2 - x 2 + x < 0 )
)
-99 d 2 + x 2 - x < 0 )

requires that x is equal to x2, when d2 is zero but when d 2

is one, this pair of restrictions requires vi~tually nothing.

The number 99 has been taken to represent a "very high number"

and could very well be taken higher, except that for typo-
graphical reasons, the 99 was more convenient.

For a convex restriction we would not need the integer

programming technique at all. This only arises because the
linear segments corresponding to the non-convex (upwardly
curved) parts of the curve actually cut into the admitted
area.

Accordingly, some condensation of the problem is possible, if

several consecutive segments together form a convex segment.
In the example at hand, this is the case for the first three
segments. Algebraically this is recognised, by means of
comparing the slope-coefficients.

The first segment (0 < x < 0.5) has a slope-coefficient of

2.25 the slope coefficient of the next segment (0.5 < x < 1.0)
is only -0.25, the slope of the third segment (1.0 <-x <-1.5)
is even less - going down even steeper - i.e. -1.25~

After that the slope increases again, i.e. becomes less

negative 0.75. Therefore, the convex segment 0 ~ x ~ 1.5
requires only one dummy. Thus, the three segments now become

y < 2.25 xl + 1

y < -0.25 xl + 2.25

y < -0.75 xl + 3.25

and xl now covers the interval 0 < xl < 1.50.

The problem is now tabulated.

The programmed procedure listed at the end of this section

actually produces an end result which includes the prob1em-
reinterpretation discussed in section 10.4, but the tabulation
given in tableau 19.3a contains the original consrants of the
various restrictions, with all variables set at zero.
."
~

INTEGER R. CONTINUOUS VARIABLES

01 02 D3 XI X2 X3 X4 x 04 y VALUE
~
. -----------------~------------------------------------ ------------
-I -I -I -I -3 OUMMI ES-EQ.
oJ =
III
D
a: -99 -2.25 ~ 1 CON-
Do -99 0.25 2.25 VEX
~
t!l -99 1.25 ~ 3.25 SEGMENT
2
«
...
:E -99 0.75 2.50 SEPERATE
~
:E
-99 -1.25 ~ -1.50 NON-CONVEX
M
«0:
. t!l -4.75 -99 ~ -10.25 SEGMENTS
0- D
0:
Do -99 -I I S
::> 0: -99 -I I ~ RELATIONS
w -99 -I I ~ BETWEEN X
i5 t!l
oJ -I I -99 ~ ANO THE
III ....W
z -99 -1 ~ SEGMENTAL
....« -99 -1 ~ X - OUMMI ES
tJ -99 -1 ~ XI TO X4
...
III -1 -99 ~
::>
tJ ----------------------.-----------------------------------------.-
w -2 OBJECTI VE F
" ....x 1.50 2 2.50 3 3 4 X UPPER LIMITS
1.50 2 2.50 LOWER LIMITS
~
I
8
~
,.,
~
~

"~
~
646 CHAPTER XIX

Methods of integer programming as such, i.e. finding solutions

to already stated integer programming problems are the subject
of the next two chapters. At this point it is, however useful
to comment on the possibility to automate the assembly of the
tableau, and to eliminate redundant restrictions and variables.

The procedure offered below writes the restrictions which arise

from the segmentation of one polynomial restriction of the type.

pm
y ~ I c
p
(19.3.1)
p = 0

It is obviously possible to have several restrictions of the

similar type in one problem, and have them written in the same
tableau calling the procedure several times. The call to the
procedure to be listed below should therefore supply for a
polynomial restriction:

the full size of the tableau

the index of the column which refers to y
the index of the column which refers to x
the numbers characterising
the polynomial function, i.e. the index mp (maximum
power) and mp + 1 coefficients c p giving the coefficients
associated with x O (the constant) until xmp.
the number of segments
the index of the column where space lS reserved for the
first zero one variable
the index of the column where space is reserved for the
first column representing x in a particular interval
the upper and lower limits on x

We offer the text of a procedure which assembles the

appropriate part of the tableau on the indication of that
information.

The listed procedure does more than was indicated in the text
so far. The two additional points are the following: Firstly,
there is the issue of near-equivalence of two or more adjoining
segments. If the number of segments is put fairly large
initially, it may easily happen that the approximation would
not become noticeably worse, if some segments were merged, i.e.
were replaced by only one linear segment.

In the example, segments 3 and 4 (1.0 < x < 1.5 and 1.5 < x < 2)
came most near to that possibility. Th; criterion which the
procedure applies is that the difference between the slope-
coefficients of two adjoining sections, multiplied with their
total length, is less than a given tolerance, which is set at
INTEGER PROGRAMMING 647

0.01. Thus segments 3 and 4 have slope coefficients of -1.25

and -0.75 respectively, i.e. their difference is 0.50. The
total length of the two segments is 0.50 + 0.50 = 1 full unit
of x. Hence the tolerance would have to be set at just above
0.50 before these two segments are to be merged.

If the tolerance were set at 0.51 rather than at 0.01, the two
segmented restrictions would be replaced by a single one

y .::.. 3.00 - 1.00x (1.00 .::.. x .::.. 2.00)

linking the points x = 1.00, y = 2.00 and x = 2.00, y = 1.00

directly. The main reason why this particuiar loop of the
programmed procedure (situated in the programme-text below
the label MERGE ALTOGETHER:), did not become operative in the
example is that only 6 segments is a rather crude approximation
in the first place.

The other point which has not been discussed so far is the
redundancy of the integer (zero-one) requirement on the "last"
one of any series of dummy variables.

In the example (as it became after condensation) d l , d 2 and d 3

are only allowed to attain the values zero or one. There also
is a requirement d~ + d 2 + d 3 + d 4 = 3. It is therefore
sufficient to requ1re that d 4 is 1n the interval between zero
and one, and it is not necessary to verify that d 4 is in fact at
one of the two ends of this interval. In this connection, the
tableau is re-ordered once more, and the "extra" zero-one
variable is put at the end of the tableau rather than with the
integer-restricted variables.

TEXT-LISTING OF THE MAIN PROGRAMME USED TO ASSEMBLE AND

SOLVE THE CUBIC INTEGER PROGRAMMING PROMLEM.

'BEGIN' 'INTEGER' N.NAV.I.J.MM.NN.RN.RBN.NLINRES.NEQ.NPOLYRES.P.

ROWINDEX.COLINDEX.RESTRN.IFZOV.IFDX.IFDXDF.MEQ.NZOV.
EXITTYPE.IX.IY.EQN.MMM.NNN. OUT M. OUT N;
'REAL' ALPHA;

'PROCEDURE' POLY<T.COEFVEC.ALPHA.M.N.MAXPOW.NSE.LOVX.HIVX.
MINUMX.MAXUMX.EQN.RESTRN.IFZOV.IX.ly.IFDX.IFDXDF.RN);
'INTEGER'M.N.MAXPOW.NSE.EQN.RESTRN.IFZOV.
IX.ly.IFDX.IFDXDF.RN;
'REAL' ALPHA.LOVX.HIVX.MINUMX.MAXUMX;
'ARRAY' T.COEFVEC;
'ALGOL';
648 CHAPTER XIX

"PROCEDURE" ZoLPCINTA.oUTA.OUTM.OUTN.T.M.N.NEQ.NAV.NZOV.
oUTROWLST.RoWLST.CoLLST.EXITTYPE.NSL)J
"INTEGER" oUTM.OUTN.M.N.NEQ.NAV.NZoV.EXITTYPE.NSLJ
"ARRAY" INTA.oUTA.TJ
"INTEGER" "ARRAY" oUTROWLST.ROWLST.CoLLSTJ
"ALGoL"J

"PROCEDURE" MATICMATR.MB.N8.FR.FC)J
"ARRAY" MATR) "INTEGER" M8.N8.FR.FCJ "ALGOL";

"PROCEDURE" TABOCMATR.M.N.SR.SC.RH.ER.ROWLST.CoLLST)J
"ARRAY" MATR) "INTEGER" M.N.SR.SC.RH.ERJ
"INTEGER" "ARRAY" ROWLST.COLLSTJ "ALGOL"J

"COMMENT"
PROGRAMME TO MAXIMIZE A LINEAR 08JECTIVE FUNCTION.
SUBJECT TO NLINRES LINEAR RESTRICTIONS. AND NPOLYRES
POLYNOMIAL RESTRICTIONS.
EACH POLYNOMIAL RESTRICTION LINKS ONE OF THE VARIA8LES
TO A POLYNOMIAL FUNCTION OF ONE OF THE OTHER VARIA8LES"
OF THE RESTRICTIONS. NEQ ARE ALLOWED TO BE EQUATIONS.
ALL THE EQUATIONS MUST BE BE LINEAR EQUATI~~S.

OF THE N VARIABLES. NAV ARE ALLOWED TO 8E OF ABSOLUTE TYPE.

I.E. NOT TO BE ASSOCIATED WITH NON NEGATIVITY RESTRICTIONS.

THE PROGRAMME DOES THIS 8Y ASSEMBLING AN MM BY NN INTEGER

LINEAR PROGRAMMING TABLEAU. IN WHICH EACH POLYNOMIAL
RESTRICTION IS REPRESENTEP BY LINEAR SEGMENTS.
FOR DETAILS OF THIS LINEAR PRESENTATION. SEE THE TEXT OF
THE PROCEDURE POLY.

READ MAIN PROBLEM PARAMETERS a

NLINRESa-READJ NEQa-READ; NPOLYRESa-READJ NasREAD; NAVa=READJ

"8EGIN"
"ARRAY" LoVX.HIVX.MINUMX.MAXUMXClaNPOLYRESlJ
"INTEGER" "ARRAY" MAXPOW.NSEClaNPOLYRESlJ

READ MAIN PARAMETERS OF SEPERATE POLYNOMIALSa

"FOR" RNa-l "STEP" 1 "UNTIL" NPOLYRES "DO" "BEGIN"

READ THE INDEX FOR THE HIGHEST POWERa
MAXPOWCRN la-READJ

HOW MANY SEGMENTS FOR THIS RESTRICTIoNa

NSEC RN la-READJ

WHICH INTERVAL a
LOVXCRNla=READJ
".\
HIVX(RNla=READJ
INTEGER PROGRAMMING 649

WHICH TOTAL INTERVALI

MINUMXCRNJI=READ; MAXUMXCRNJI=READ;
'END';

CALCULATE ORDER PARAMETERS OF MAIN TABLEAU I

MMI=NLINRES; NNI=N;
'FOR' RNI"1 'STEP' 1 'UNTIL' NPOLYRES 'DO' 'BEGIN';
MM I =MM+ 1 +3*NSEC RN J; NN: =NN+2*NSE( RN]; 'END';

'IF' NEQ > NLINRES 'THEN' 'BEGIN'

NEWLINECl H
WRITETEXTC'C'TOO%MANY%EQUATIONS')');
NEWLINEC 1 H
WRITETEXTC'C'THE%NUMBER%OF%EQUATIONS%EXCEEOS%THE%
NUMBER%OF%LINEAR%RESTRICTI ONS' ) 'H 'END'
'ELSE' MEQI=NEQ;

'BEGIN' 'ARRAY' T( IIMM+3.IINN+2H

INITIATE TABLEAU AS ZERO MATRIXI

'FOR' 11=1 'STEP' 1 'UNTIL' MM+3 ,~O'
'FOR' .J1=1 'STEP' 1 'UNTIL' NN+2 '~O' T(I • .JJI=O;

PUT LINEAR INPUT DATA TEMPORARY IN TOP BLOCK ROWI

MATICT.NLINRES+3.N+l.0.0);
TRANSPORT TO APPROPRIATE BLOCKSI
'FOR' I1al+MEQ 'STEP' 1 'UNTIL' NLINRES+3 '~O'
'FOR' .JI-l 'STEP' 1 'UNTIL' N+l ,~O' 'BEGIN'
HMM-NLINRES+I • .JJI=Tr I • .J]; T( I • .JJI=O; 'END';

'FOR' II-I 'STEP' 1 'UNTIL' MEQ.

MM+MEQ-NLINRES+l 'STEP' 1 'UNTIL' MM+3 'DO'
'FOR' .J1-NAV+l 'STEP' 1 'UNTIL' N+l ,~O' 'BEGIN'
H I.NN-N+.JJI=T[ I • .JJ; T[ I • .JJI=O; 'END';

ATTEND THE POLYNOMIAL RESTRICTIONSI

INITIATE INDICES FIRST BLOCKI

ROWINOEXI=MEQ+NPOLYRES; COLINOEXI-NAV; NZOVI=O;
EQNI=MEQ; IFZOVI=NAV;

'FOR' RNI:l 'STEP' 1 'UNTIL' NPOLYRES '~O' 'BEGIN'

EQNI&EQN+l; MEQI-MEQ+l;
RESTRNr-ROWINOEX; ROWINDEXr-ROWINDEX+NSECRNJ;
IFDXDFr-ROWINDEX; ROWINDEX:=ROWINDEX+NSE[RNJ;
COLINDEXr-COLINDEX+NSECRNJ; NZOVr-NZOV+NSECRNJ;
IFDXr-COLINDEX; COLINDEXr-COLINDEX+NSECRNJ;
'BEGIN' 'ARRAY' COEFVECCOIMAXPOWCRNJJ;
READ AND ASS INFORM PARTICULAR RESTRr

ALPHAlaREAD;
 650 CHAPTER XIX

IXa-REAOJ
'IF' IX>NAV 'THEN'
'FOR' RBNa-1 'STEP' 1 'UNTIl.' NPOI.YRES ,~O'
IXa-IX+2*NSECRBN1;
IYa-REAOJ
'IF' IY>NAV 'THEN'
'FOR' RBNa-1 'STEP' 1 'UNTIl.' NPOI.YRES 'DO'
IYa-IY+2*NSECRBN1J
'FOR' Pa-O 'STEP' 'UNTIl.' MAXPOWCRN1 'DO'
COEFVECCP1a-REAO;
NOW ASSEMBI.E THE RN RESTRa
MMMa-MM; NNNa-NNJ
POI.Y<T,COEFVEC,AI.PHA,MM,NN.MAXPOWCRN1,NSECRN1,
1.0VXCRN1,HIVXCRN1,MINUMXCRN1,MAXUMXCRN1.EQN,RESTRN,
IFZOV,IX,IY,IFOX.IFOXOF,RN);
RESET INOI CES a
ROWINOEXa-ROWINOEX+MMM-MM; COI.INOEXa=COI.INOEX+NNN-NN;
NZOVa-NZOV-ENTIER<O,5*<NNN-NN+1»-1;
'END'; 'END';

'BEGIN' 'ARRAY' TAC 1aMM+3, 1aNN+21,

OUTAC1aMM+NN+NZOV,la11J
'INTEGER' 'ARRAY' ROWI.STC 1aMM1,COI.I.STC 1aNN1,
OUTROWI.STC 1 a NZOV+MM+NN1;
PUT ZEROS TO PREVENT OVERFI.OWa
'FOR' II-I 'STEP' 1 'UNTIl.' MM+NN+NZOV ,~O'
OUTACI,11a=OJ
'FOR' II-I 'STEP' 'UNTIl.' MM ,~O' ROWI.STC I lI=O;
'FOR' I a -I 'STEP' 'UNTIl.' NN '~O' COI.I.STC I 1a =0;
'FOR' 1,,'1 'STEP' 'UNTIl. ' NZOV+MM+NN '~O'
OUTROWI.STCI1a-OJ
'FOR' la-I 'STEP' 'UNTIl.' MM+2 '~O'
'FOR' Ja-1 'STEP' 'UNTIl.' NN+1 '~O' TACI,J1a=0;
ZOI.P<T,OUTA,OUTM,OUTN,TA,MM,NN,MEQ.NAV,NZOV,
OUTROWI.ST, ROWI.ST. COI.I.ST, EXITTYPE, 100);
'IF' EXITTYPE-O 'THEN' TABO<OUTA,OUT M,O,O.O,l,O,
OUTROWI.ST,COI.I.ST)
'EI.SE' TABO<TA,OUT M,OUT N,O,O,I,l,ROWI.ST,COI.I.ST);
'END'; 'END'; 'END'; 'END'
INTEGER PROGRAMMING 651

TEXT-LISTING OF THE PROCEDURE WHICH SEGMENTS ONE SEPERATE

POLYNOMIAL RESTRICTION.
'PROCEDURE' PoLY(T.COEFVEC.ALPHA.M.N.MAXPoW.NSE.LOVX.HIVX.
MINUMX.MAXUMX.EQN.RESTRN.IFZOV.IX.IY.IFDX.IFDXDF.RN)J
'INTEGER' M.N.MAXPoW.NSE.EQN.RESTRN.IFZoV.
IX.IY.IFDX.IFDXDF.RNJ
'REAL' ALPHA.LDVX.HIVX.MlNUMX.MAXUMX;
'ARRAY' T.CoEFVECJ
'eEGIN' • INTEGER' I.J.II.JJ.NSEC.SE.P;
'REAL' LDWV.HIGV.X.Y.XDIF.XP.DYDX.FANCYHIGH.COP.SLDIF.MAXUMYJ
'COMMENT'
PROCEDURE TO ASSEM.LE A SEGMENTED APPROXIMATION OF THE POLY-
NOMIAL RESTRICTION
ALPHA • Y < DR • PoLINOMIAL FUNCTION OF X.
THE FUNCTION IS EXPECTED TO eE NON-CONVEX.
IN EACH SEGMENT. X IS REPRESENTED ey ASSOCIATED DUMMY VARIABLES.
THESE DUMMY VARIABLES ARE RESTRICTED TO A PARTICULAR
INTERVAL.
ALSO ASSOCIATED ARE ANOTHER SERIES OF DUMMY VARIABLES.
THOSE DUMMY VARIABLES ARE ZERO-ONE VARIABLES. ONE OF THEM
WILL BE ZERO AND IDENTIFY AND MAKE EFFECTIVE A PARTICULAR LINEAR
SEGMENT OF THE FUNCTION. WHILE THE OTHERS ARE AT THE VALUE ONE.
THE SIGNIFICANCE OF THE PARAMETERS IS AS FOLLOWS,
T IS AN M ey N LINEAR PROGRAMMING TYPE TABLEAU. OR M+2 BY
N+2. IF THE MARGINS ARE INCLUDED.
CoEFVEC IS THE COEFFICIENTS VECTOR WHICH CHARACTERIZES THE
POLYNOMIAL FUNCTION.
THE TERMS BY WHICH X POWER ZERO (THE CONSTANT). X POWER ONE.
X POWER TWO. UNTIL X POWER MAXPOW.
ARE TO BE MULTIPLIED. ARE STORED IN THIS VECTOR.
THEIR PLACES IN THE COEFFICIENTS VECTOR ARE THE EXPONENTS. I.E.
THE CONSTANT IN CELL ZERO. AND THE COEFFICIENT WHICH IS ASSO-
CIATED WITH THE HIGHEST (MAXIMUM) POWER. IS STORED IN THE CELL
WITH INDEX MAXPOW.
NSE IS THE NUMeER OF (LINEAR) SEGMENTS.

LOVX IS THE LOWER VALUE OF X. I.E. THE LOWER LIMIT OF SEGMENT L

PRESENTATION OF X.
HIVX IS THE HIGHER VALUE OF x. I.E. THE HIGHER LIMIT OF
SEGMENTAL PRESENTATION OF X.
THE FIRST AND THE LAST SEGMENT ARE. HOWEVER.
EXTENDED ON THE INDICATION OF THE VARIABLES MINUMX AND MAXUMX.
THE FIRST SEGMENT OF X STRETCHES FROM MINUMX (THE MINIMUM
VALUE OF X). TO LOVX +l/NSE • CHIVX-LOVX).
SIMILARLY. THE LAST SEGMENT OF X STRETCHES FROM
HIVX - l/NSE • (HIVX-LOVX). TO MAXUMX.
652 CHAPTER XIX

OTHERWISE THE JTH SEGMENT OF APPROXIMATION COVERS THE INTERVAL

OF X BETWEEN LOVX + (J-l)/NSE * (HIVX-LOVX).
AND LOVX + JINSE * (HIVX-LOVX).
EQN IS THE INDEX OF THE ROW OF T. WHERE THE EQUATION-RESTRICTION
ON THE TOTAL VALUE OF THE ZERO-ONE VARIABLES IS TO BE STORED.

RESTRN IS THE INDEX OF THE ROW OF T. WHERE THE REPRESENTATION

OF THE RESTRICTION ITSELF IS TO BE STORED. I.E.
RESTRN+I IS THE ROW-INDEX OF THE I TH SEGMENT.
IFZOV+J IS THE INDEX OF THE COLUMN OF T WHICH IS TO
REPRESENT THE JTH ZERO-ONE VARIABLE.
IX IS THE INDEX OF THE COLUMN OF T. WHICH IS TO REPRESENT X.
IY IS THE INDEX OF THE COLUMN OF T. WHICH IS TO REPRESENT Y.

IFDX+J IS THE INDEX OF THE JTH DUMMY-VARIABLE. REPRESENTING X.

IFDXDF+I IS THE INDEX OF THE ROW OF T.
WHERE THE ITH DEFINITIONAL RESTRICTION.
LINKING X TO ITS DUMMY REPRESENTATIVE
IN THE ITH (OR I-NSE)TH SEGMENT. IS TO BE STORED.
J

PUT FANCYHIGH FANCYHIGH.

FANCYHIGH.-99J
INITIATE MAXIMUM Y LOW.
MAXUMy.--l000000J

WRITE EQUATION RESTRICTION ON THE INT VAR.

'FOR' J.=1 'STEP' 1 'UNTIL' NSE 'DO' TCEQN.IFZOV+J11=-IJ
TCEQN.N+l1.a-NSE+1J

PUT LIMITS ON X.
'IF' TCM+2.IX1-O 'OR' MAXUMX<TCM+2.IXl
'THEN' TCM+2.IX1.aMAXUMXJ
'IF' MINUMX > TCM+3.IXl 'THEN' TCM+3.IX11=MINUMXJ

RELATE X TO SEGMENTED X AND DUMMIESI

'FOR' II-I 'STEP' 1 'UNTIL' NSE 'DO' 'BEGIN'
TCIFDXDF+I.IFZOV+Il 1= -FANCYHIGHJ
TCIFDXDF+I.IXl I- lJ
TCIFDXDF+I.IFDX+Il 1= -1;
TCIFDXDF+NSE+I.IFZOV+l1 1= -FANCYHIGHJ
TCIFDXDF+NSE+I.IXl 1= -IJ
TCIFDXDF+NSE+I.IFDX+Il I- lJ 'END';

PUT LIMITS ON X DUMMIESI

XDIF 1 8 (HIVX-LOVX)/NSEJ
'FOR' II-I 'STEP' 1 'UNTIL' NSE 'DO' 'BEGIN'
TCM+2.IFDX+Il I- LOVX+I*XDIF;
TCM+3.IFDX+Il 1= LOVX+<I-l)*XDIFJ 'END'J
INTEGER PROGRAMMING 653

TtIFDXDF+NSE+NSE+l.N+ll s= -MINUMXi
TtM+2.IFDX+NSEls= MAXUMXi
RELATE Y TO X PER SEGMENTs
XS=LOVXi
'FOR' 11=1 'STEP' 1 'UNTIL' NSE 'DO' 'BEGIN'
'IF' 1>1 'THEN' LOWVs=HIGV 'ELSE' 'BEGIN'
LOWVs=Oi XPs=li
'FOR' Ps-O 'STEP' 1 'UNTIL' MAXPOW 'DO' 'BEGIN'
LoWVs=LOWV+COEFVECtPl*XPi
XPs=XP*Xi 'END'i 'END'i
Xs=X+XDIFi
HIGVs=Oi XPs=li
'FOR' Ps=O 'STEP' 1 'UNTIL' MAXPOW 'DO' 'BEGIN'
HIGVs=HIGV+COEFVECtPl*XPi
XPS=XP*Xi 'END'i
DYDX := CHIGV-LOWV)/XDIFi

TtRESTRN+I.IFDX+Il:=-DYDXi
TtRESTRN+I.IYls=ALPHAi
TtRESTRN+I.IFZoV+Il:=-FANCYHIGHi
TtRESTRN+I.N+ll:=HIGV-X*DYDXi
PUT LIMIT ON Y:
'IF' LOWV>MAXUMY 'THEN' MAXUMYs=LOWVi
'IF' HIGV>MAXUMY 'THEN' MAXUMY:=HIGVi 'END'i
SET UPPER LIMIT ON Y AS FOUND:
'IF' TtM+2.IY1-0 'OR' MAXUMY<TtM+2.IYl
'THEN' TCM+2.IYl:=MAXUMYi
PUT LIMITS ON ZERO ONE VAR:
'FOR' J:=l 'STEP' 1 'UNTIL' NSE 'DO' 'BEGIN'
TtM+2.IFZOV+Jls=I.0000000Ii TCM+3.IFZQV+Jls=Oi 'END'i
REDUCE SIZE IF CONVEX OR NEAR EQUIVALENT:
II:-li JJs=2i NSECs=NSEi
'FOR' SEs=2 'STEP' 1 'UNTIL' NSE 'DO' 'BEGIN'
Ils"II+li
HIGV:-TCM+2.IFDX+JJli LoWVs=-TtIFDXDF+2*NSEC+JJ-l.N+l1i
XDIF:-HIGV-LOWVi
SLDIF:-CTCRESTRN+lI-l.IFDX+JJ-ll-TtRESTRN+II.IFDX+JJ1)*XDIFi
'IF' SLDIF < 0.001 'THEN' 'BEGIN'
'IF' SLDIF < -0.001 'THEN' 'GOTO' MERGE DUMMIESi
MERGE SEGMENT:
DYDX:-C-TCRESTRN+II.IFDX+JJ1*HIGV+TtRESTRN+II.N+ll +
TtRESTRN+II-l.IFDX+JJ-ll*LOWV-TCRESTRN+II-l.N+ll)/XDIFi
TCRESTRN+II-l.IFDX+JJ-ll:=-DYDXi
TCRESTRN+II-l.N+ll:=
-TCRESTRN+II.IFD~+JJl*HIGV+TCRESTRN+II.N+Il-DYDX*HIGVi
'FOR' I:=RESTRN+II 'STEP' 1 'UNTIL' M+2 'DO'
'FOR' Js=l 'STEP' 1 'UNTIL' N+l 'DO' TtI-l.JlI=Ttl.JH
654 CHAPTER XIX

SEI=SE-1; NSEI=NSE-1J MI=M-1; IFOXOFI=IFOXOF-1;

III-II-lJ

MERGE DUMMIESI
ADJUST NUM8ER OF NON ZERO DUMMIESI
TCEQN.N+1ll=TCEQN.N+1l+1;

MERGE COLUMNSI
'FOR' II-RESTRN+II. IFDXOF+JJ. IFDXDF+NSEC+JJ.
IFDXDF+2*NSEC+JJ. M+2 'DO' 'BEGIN'
TCI.IFZOV+JJ-1ll=TCI.IFZOV+JJlJ
TCI.IFDX+JJ-1ll=TCI.IFDX+JJH 'END';

SHIFT COLUMNSI
'FOR' 11-1 'STEP' 1 'UNTIL' M+3 '~O' 'BEGIN'

SHIFT FOR MERGED DI

'FOR' J"'I FZOV+JJ+ 1 'STEP' 1 'UNTIL' N+1
'DO' TCI.J-1ll-TCI.JlJ

SHIFT FOR MERGED XI

'FOR' JI-IFDX+JJ 'STEP' 1 'UNTIL' N
'DO' TCI.J-1ll=TCI.JlJ
'END' J

ADJUST COLUMN INDICESI

NI-N-2J I FDXI", I FDX-1 J
'IF' IX > IFZOV 'THEN' IXI=IX-lJ
'IF' IY > IFZOV 'THEN' IYI=IY-!J
'IF' IX> IFDX 'THEN' IXI=IX-1J
'IF' IY > IFDX 'THEN' IYI=IY-1J

SHIFT ROWSI
'FOR' JI"1 'STEP' 1 'UNTIL' N+1 'DO' 'BEGIN'

SHIFT UPW FOR MERGED UPPER XDFI

'FOR' II-IFDXDF+JJ 'STEP' 1 'UNTIL' M+3
'DO' TCI-1.Jll-TCI.JlJ

SHIFT UPW FOR MERGED LOWER XDFI

'FOR' I"'IFDXDF+NSEC-1+JJ 'STEP' 1 'UNTIL' M+3
'DO' TCI-1.Jll=TCI.JlJ 'END';

ADJUST ROW INDICESI

MI-M-2J NSEC"'NSEC-lJ 'END'
'ELSE' JJI-JJ+lJ 'END';

MOVE EXTRA ZERO ONE VARI

'FOR' 11-1 'STEP' 1 'UNTIL' M+3 'DO' 'BEGIN'
COPI-TCI.IFZOV+NSEClJ
'FOR' JI-IFZOV+NSEC 'STEP' 1 'UNTIL' N-RN 'DO'
TCI.Jll-TCI.J+1l;
TC I.N-RN+1ll-COPJ 'END';
INTEGER PROGRAMMING 655

END OF POLYI

REINTERPRET TO LOWER LIMIT DISTANCESI

'FOR' JI-1 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'
TCM+2.Jl I- TCH+2.Jl-TCH+3.Jl)
'FOR' 11-1 'STEP' 1 'UNTIL' H+1 'DO'
TC I.N+1] I- TC I.N+1 ]-TCH+3.J]*TC I.JlJ 'END')

FINAL END OF POLYI 'END')

CHAPTER XX

BRANCHING METHODS

20.1 Branching in the order of increasing indices

The algorithm offered in this section is mor,e primitive than

the ones normally recommended in the literature.

It is specifically concerned with zero-one variables. It serves

the purpose of introducing the idea of branching and it has the
further attraction of a relatively simple programme-code.

It solves the (zero one variables) mixed integer programming

problem by means of a series of ordinary linear programming
problems. In this particular method, these linear programming
problems refer to tableaux from which columns which relate to
zero-one variables are progressively removed.

A branch is a consecutively solved series of "ordinary" L.P.

problems which arise from the introduction of additional
requirements, requiring successive variables to attain
specified integer values. If the variable is zero we remove the
column altogether, if the variable is one, we subtract it from
the value column. It seems natural to assume that a branch is
developed in the order of the indices of the variables. Thus
we would begin with specifying a value for the integer variable
with the lowest index. In the example from section 19.3 we could
begin with requiring that d l should be zero. In that example,
there is an equation requirement d l + d 2 + d 3 + d 4 = 3. The
requirement that d l should be zero, comoined with upper limits
on d 2 , d 3 and d 4 , Eherefore automatically implies d 2 = d 3 = d 4 = 1
This is not necessarily so in the general mixed integer
programming problem, and one may have to impose further
restrictions on other integer-restricted variables (e.g. d ,
d 3 , etc), until an admissable solution to the specified integer
programming problem is reached, or until a particular branch
is terminated (abandoned) for other reasons. We will obviously
need some sort of system to ensure that all appropriate
combinations are investigated. We do not start with requiring
that all integer-restricted variables actually attain an
integer value.

The two main reasons for this practice are a) It is systematic

rather than accidental that additional integer requirements are
sometimes met without imposing a requirement to that effect on
all variables in question. b) Non-integer solutions may reveal
the fact that there is no prospect of finding the optimum
in a particular branch.
656
BRANCHING METHODS 657

We will come back to this problem, when we have discussed the

coding system. There is no inherent reason why problems should
be coded in a particular way and no other, provided some basic
desiderata are met. Code-numbers (declared as integer variables)
should enable us to identify integer restrictions in a parti-
cular problem. They should also enable us to relate the code-
number to the order in which the various problems are tackled.
The formula

niv
Code \
L (1 + rVk ) 3niv- k (20.1. 1)
k=l

meets these desiderata, and will be used here.

In (20.1.1) rv is the restriction value, i.e. rVk = 0 indicates

a restriction which says that the kth zero-one variable is
required to be zero, and rvk=l indicates that the kth zero-one
variable is required to be one. It is assumed that variables
to which no integer restriction is actually applied are not
included in the summation, i.e. the effective value of rVk is
-1 in that case, l+rvk attaining the value zero.

That this coding formula meets the two basic requirements of a

coding system is illustrated below by tabulating its effective
significance for the case of three. zero-one variables.
The various LP problems are in fact tackled in the order of
increasing numerical value of the code-number (see tabulation 20.1).

In the case of three zero-one variables we may (at most) have to

investigate 15 LP problems. There is however, a possibility that
certain branches may not need to be investigated, at least not
beyond the problem at the head of a particular branch.

In fact, the problem "at the head" defines the branch, by a

particular series of requirements.

dk = 0,1 (k 1, 2, ... index, index < ni v) (20.1.2)

In (20.1.2), niv is the number of integer restricted variables,

i.e. 3 in the tabulated example.

Note that we do not investigate problems where the integer

restrictions on variables with higher indices are implemented,
while the similar restrictions on variables with lower indices
are not implemented. (Problems 1-8 , 10, 11, 19 and 20).

A branch, or the search for a solution of the problem as a whole

658 CHAPTER XX

Tabulation 20.1

Coding summary for 3 zero-one variables

Inv. ~'R. R. R. Code Code Code Value

order on on on rV 1 as rV 2 as rV3 as of
d1 d2 d3 Code

1 ? ? ? 0
Never ? ? 0 1x30=1 1
Never ? ? 1 2X30=2 2
Never ? 0 ? 1x3=3 3
Never ? 0 0 1x3=3 1x30=1 4
Never ? 0 1 1x3=3 2x30=1 5
Never ? 1 ? 2x3=6 6
Never ? 1 0 2x3=6 1x30=1 7
Never ? 1 1 2x3=6 2x3O=2 8
2 0 ? ? 1x3 2=9 9
Never 0 ? 0 1x3 2=9 1x30=1 10
Never 0 ? 1 1x3 2=9 2 x 30=2 11
3 0 0 ? 1x3 2=9 Ix3=3
12 } d 1 =d 2 =0
13
4 0 0 0 1x3 2=9 1x3=3 1x30=1
5 0 0 1 1x3 2=9 1x3=3 2x30=2 14 Branch
6 0 1 ? 1x3 2=9 2 x 3,;,,6 15} d 1 =0
7 0 1 0 1x3 2=9 2x3=6 1x30=1 16 d 2 =1
8 0 1 1 1x3 2=9 2x3=\6 2x3O=2 17 Branch
9 1 ? ? 2x3 2=18 18
Never 1 ? 0 2x3 2=18 1x30=1 19
Never 1 ? 1 2x3 2=18 2x30=2 20
10 1 0 ? 2x3 2=18 1x3=3 21}d 1 =1
11 1 0 0 2x3 2=18 1x3=3 1x30=1 22 d 2 =0
12 1 0 1 2x3 2=18 1x3=3 2x3O=2 23 Branch
13 1 1 ? 2x3 2=18 2x3=6 24} d 1=d 2
14 1 1 0 2x3 2=18 2x3=6 1x30=1 25 =1
15 1 1 1 2x3 2=18 2x3=6 2x 3O=2 26 Branch
BRANCHING METHODS 659

ends, when one of the following four conditions ~s met:

a) Full Exhaustion

All relevant problems have been solved, i.e. integer restrictions

and all possible combinations of zeros and ones) have been
imposed on all variables.

b) Optimality without Exhaustion

A (sub)problem in which integer restrictions,have not so far

been imposed on all variables (only on those below a certain
index) happens to have an optimal solution which satisfies all
the remaining integer restrictions as well. Thus, in the
example from section 19.3, there are dummy-variables which are
subject to an integer restriction of zero-one type. The
requirement dl = 0, combined with d l + d 2 + d 3 + d 4 = 3 and
the upper limits of d 2 ~ 1, etc., implies d 2 = d 1 = d 4 = 1.
Obviously it is then unnecessary to investigate the combination
d l = 0, d 2 = o. Finding an all-integer solution as optimum of the
problem at the head of a branch terminates the branch.

c) An empty problem is met

If the problem at the head of a branch is empty, the whole

branch is abandoned.

d) An Unbounded problem is met

This is in fact only possible with the initial continuous

problem. When the continuous problem has been solved, i.e. is
found to have a finite optimal solution, subsequent branching
may still lead to empty problems, but not to unbounded problems.

e) Sub-optimality

The optimal solution of the problem at the head of the branch

(irrespective of further integer requirements), indicates a
lower solution value than a previously found integer solution.

Conditions b), c), d) and e) all cause a reduction in the number

of problems to be solved, they are premature terminations.

The most extreme form of premature termination is obviously

to find the continuous problem empty, unbounded or all-integer
in its optimum.

Similarly a branch ~s terminated, as soon as the problem at the

head of the branch is either found to be wanting in some
660 CHAPTER XX

respect, i.e. empty or sub-optimal, or to have an all-integer

optimum. Recall the tabulation of the three integer restricted
variables case:

If problem 18 (d l 1, d 2 and d 3 so far not integer restricted)

is empty, problems 21 to 26, whose solutions would be feasible
solutions of problem 18, need no investigation.

Likewise, if we already found a feasible integer solution,

there is a possibility that one or more remaining branches may
be found sub-optimal by way of investigating the problems at
their heads. Suppose for example that we already recorded an
optimal and feasible solution of the d l = °branch (for
example) a solution value of 53 for problem 14. (d l = d 2 = 0,
d3 1).

We will need to investigate the rest of the d l = 1 branch,

(problem 21 to 26), only if problem 18 is found to have an
optimal and feasible solution, with a solution value of more
than 53. But even then we may find the sub-branch d 1 = 1,
d2 = °sub-optimal. If problem 21 (dl = 1, d 2 = 0, a3 not so

that the optimum is not in the d 1 = 1, d 2 = °

far integer restricted), has a solution value of 49, we ~now
branch. All
feasible solutions of problems 22 and 23, including the optima,
are also feasible solutions of problem 21, hence their optimum
solution values cannot exceed the optimum of problem 21. The
other way in which the number of actually investigated problems
may be reduced, arises in connection with non-required part-
integer solutions.

For reasons which tend to be somewhat more complicated* than

in the all-integer case - but not basically different -, this
possibility is to some extent systematic rather thar. incidental.
We do not need to record all partly integer solutions. Thus,
the optimum of problem 0, may turn out to satisfy the
requi:ements of problem 5. (d l and d 2 are not integer-
restr1cted, d 1 =1), for example d l = 0.22, d 2 = 0.73, d 3 = 1.
Because we ao not intend to investigate problem 5 in any case,
we may as well classify the solution of problem zero as being
non-integer as soon as d l is found to have a ~on-integer value.
But if the optimum of problem zero turns out to be d l = 1,
d2 = 0, d3 = 0.43, we may record the fact that problems 18 and
21 have already been solved. It is in fact sufficient to record
the attainment of an optimum solution to problem 21.

* See Weingartner (38]

BRANCHING METHODS 661

A prematurely obtained solution of problem 21 implies that

it also is the optimum of problem 18. We do not tackle any
sub-problems of the d l = 1 branch before we enter it at the
head. Thus we will record the code 21 and a corresponding
solution value in a solutions list.

When we come to tackle problem 18, we can identify 21 as being

a sub-problem of the d l = 1 branch.

The relevant numerical condition of the code is

o < recorded code - co d e < 3niv-index (20.1. 3)

Above, recorded code (obviously) is the code of the previously

recorded solution, and code is the code of the problem which
is currently being tackled.

If the previously recorded code is less than the code of the

currently tackled problem, the previously recorded code refers
to a problem which belongs to an already exhausted branch.
(Problem 17 does not belong to the d l = 1 branch). The problem
at the head of the branch has the lowest code of all the
problems in the branch. Since codes of lower values than the
code of the currently tackled one, are codes of problems
already dealt with the programme developed in association with
this section, will wipe out such codes, whenever they are met,
just to save list-space. On the other hand, problems with a
code which is at least 3niv-index greater than the problem
which is being tackled, belong to the next branch, or possible
to one which is still further ahead.

For example, we may find the optimal solution of the d l = 1

branch (whether obtained by fresh calculation or from earlier
recording) to be an improvement relative to anything found in
the d l = 0 branch. We therefore enter the d J = 1 branch further,
and tackle problem 21. The index up to whicfi the variables are
integer-restricted is then 2, niv-index is 3-2=1, and (20.1.3)
reveals that problems whose codes are at least 3 1 =3 higher than
21 (i.e. 24, 25, 26), do not belong to the d l = 1, d 2 = 0
branch.

The computational implementation of the branching algorithm

depends to some extent on whether one is interested in the
shadow prices of the integer restrictions. In this section it
is assumed that the user is not interested in those shadow
prices. Each sub-problem which is tackled, is started by
partly copying the initial tableau of the continuous problem.
Columns which refer to variables on which integer restrictions
are actually imposed are not copied. If the integer value is
zero, the corresponding column is simply suppressed, if the
integer value is one, the corresponding column is subtracted
662 CHAPTER XX

from the value column, i.e. the variable lS brought over to the
right-hand side.

The integer programming procedure is now listed as follows:

TEXT-LISTING OF THE ZERO-ONE VARIABLES INTEGER PROGRAMMING

PROCEDURE.

'PROCEDURE' ZOLPCINTA.OUTA.OUTM.OUTN.T.M.N.NEQ.NAV.NZOV.
OUTROWLST.ROWLST.COLLST.EXITTYPE.NSL)}
'INTEGER' OUTM.OUTN.M.N.NEQ.NAV.NZOV.EXITTYPE.NSLJ
'ARRAY' INTA.OUTA.TJ
'INTEGER' 'ARRAY' OUTROWLST.ROWLST.COLLSTJ

'BEGIN' 'INTEGER' I.J.P.NEWCODE.NAME.II.JJ.R.K.ROWN.COLNJ

'REAL' SOLUTION VALUE.NUM.QUO.ZERODIS.ONEDISJ
, BOOLEAN' INTGJ
'ARRAY' SOLUTION VALUES LISTCIINSLlJ
'INTEGER' 'ARRAY' CODELISTCIINSLl.DCNAV+IINAV+NZOVlJ

'PROCEDURE' LINPCT.M.N.NEQ.NAV.ROWLST.COLLST.REENTRY)J
'ARRAY' T; 'INTEGER' M.N.NEQ.NAV.REENTRYJ
'INTEGER' 'ARRAY' ROWLST.COLLST;
'ALGOL'J
'PROCEDURE' INTPCT.M.N.NEQ.NAV.ROWLST.COLLST.
IROWLST.R.K.ROWN.COLN.REENTRY)J
'ARRAY' TJ 'INTEGER' M.N.NEQ.NAV.R.K.ROWN.COLN.REENTRYJ
'INTEGER' 'ARRAY' ROWLST.COLLST.IROWLSTJ
'ALGOL'J
'PROCEDURE' BRANCHCCODE.RE VALUE. INDEX.
TO BE SOLVED FROM NEW)J
'VALUE' CODE.RE VALUE.INDEX.TO BE SOLVED FROM NEWJ
'INTEGER' CODE.RE VALUE.INDEXJ
'BOOLEAN' TO BE SOLVED FROM NEWJ
'BEGIN'
CHECK IF INVESTIGATED BEFORE:
'FOR' PI=I 'STEP' 1 'UNTIL' NSL 'DO' 'BEGIN'
'IF' CODELISTCPl < CODE 'THEN' CODELIST(Pl:=-lOJ
'IF' CODELIST(Pl-CODE < 3 t CNZOV-INDEX) -0.5
'AND' 'NOT' SOLUTION VALUES LIST(Pl > SOLUTION VALUE
'THEN' 'BEGIN' CODELISTCPl:=OJ
'GOTO' END OF BRANCHJ 'END'J 'END'J

'IF' 'NOT' TO BE SOLVED FROM NEW

'THEN' 'GOTO' RESTRICT ADDITIONALLYJ
PREPARE AN LP PROBLEM:

PUT INITIAL RHS:

'FOR' 1:=1 'STEP' 1 'UNTIL' M+I 'DO'
T(I.N+I-iNDEXl:=INTA(I.N+IlJ

PUT HIGH UPPERLIMITDISTANCES FOR SLACKS:

'FOR' 1:=1 'STEP' 1 'UNTIL' M 'DO'
T(I.N-INDEX+2l 1= fOOOOOOOOOOOOJ
BRANCHING METHODS 663

ATTEND TO REST OF TABLEAU I

JJI"O;
'FOR' J:"I 'STEP' I 'UNTIL' N '~O' 'BEGIN'
'IF' J > NAV 'AND' 'NOT' J > NAV+INOEX 'THEN' 'BEGIN'
'IF' OCJJ-I 'THEN' 'GOTO' BRING TO RHS
'ELSE' 'GOTO' NEXT COLUMN; 'END';
COpy JTH COLUMNI
JJI-JJ+U
'FOR' II-I 'STEP' 'UNTIL' M+2 'DO'
TCI.JJJI=INTACI.JJ;
'GOTO' NEXT COLUMN;
BRING TO RHSI
'FOR' 11=1 'STEP' I 'UNTIL' M+I 'DO'
TCI.N+I-INOEXJI=TCI.N+I-INOEX1-TCI.Jl;
NEXT COLlI'lN I
'END';
EXITTYPEr=O;
LINP<T.M,N-INDEX.NEQ.NAV.ROWLST,COLLST.EXITTYPE);
TO BE SOLVED FROM NEW 1= 'FALSE';
'GOTO' CHECK FOR NO SOLUTION OF PARTICULAR PROBLEM;
RESTRICT ADDITIONALLY:
'FOR' II "NAV+I 'STEP' I 'UNTIL' M 'DO'
'IF' ROWLSTC I l=NAV+I 'THEN' 'GOTO' PUT THE CUT;
CHECK AND ATTEND NON BASIC VARI
'FOR' KI"I 'STEP' I 'UNTIL' N-INOEX+I 'DO'
'IF' COLLSTCK1-NAV-INOEX 'THEN' 'BEGIN'
'IF' OCNAV+INDEX1"0 'THEN' 'GOTO' REDUCE
'ELSE' 'GOTO' PREPARE AN LP PROBLEM; 'END';
CHECK AND ATTEND BINDING UPPER LIMIT:
'FOR' KI"I 'STEP' 1 'UNTIL' N-INDEX+I 'DO'
'IF' COLLSTCKJ=IOOOO+NAV+INDEX 'THEN' 'BEGIN'
'IF' OCNAV+INOEX1=1 'THEN' 'GOTO' REDUCE
'ELSE' 'GOTO' PREPARE AN LP PROBLEM; 'END';
PUT THE CUT:
'IF' OCNAV+INOEXJ"O 'THEN'
TCI.N-INOEX+31 := 0.0000001-TCI.N-iNOEX+21
'ELSE' TCI.N-INOEX+21 1= TCI.N-INOEX+21-0.999999;
PUT FANCYHIGH UPPER LIMITS ON SLACKSI
'FOR' 11=1 'STEP' I 'UNTIL' M 'DO'
'IF' ROWLSTCIl > 1000 'THEN' TCI.N+2J 1= 1000000000000;
'FOR' J:"I 'STEP' I 'UNTIL' N-INOEX+l 'DO'
'IF' COLLSTCJl > 1000 'AND' COLLSTCJJ < 1001 + M
'THEN' TCM+2.Jl 1= 1000000000000;
664 CHAPTER XX

CALL INTPI
EXITTYPElcl; RlcKI=ROWNI=COLNI=O;
INTP(T.M.N-INDEX+I.NEQ.NAV.ROWLST.COLLST.ROWLST.
R.K.ROWN.COLN.EXITTYPE);
'IF' EXITTYPE = -10 'THEN' 'GOTO' CALL INTP;
'IF' 'NOT' EXITTYPE-O 'THEN' 'GOTO' END OF BRANCH;
'FOR' JI-I 'STEP' I 'UNTIL' N-INDEX+I 'DO
'IF' (COLLSTCJJ<IOOO 'OR' COLLSTCJJ>NEQ+IOOO)
'AND' TCM+I.JJ<O 'THEN' 'GOTO' CALL INTP;
'FOR' KI=I 'STEP' I 'UNTIL' N-INDEX+I 'DO'
'IF' COLLSTCKJ~NAV+INDEX 'OR' COLLSTCKJ-IOOOO+NAV+INDEX
'THEN' 'GOTO' REDUCE;
REDUCEI
REMOVE KTH COLUMNI
'FOR' 11=1 'STEP' I 'UNTIL' M+2 'DO'
'FOR' J:-K+I 'STEP' I 'UNTIL' N-INDEX+3 'DO'
TC I.J-I JI-TC I.JlJ
REMOVE KTH NAME FROM LIST OF COLUMNAMESI
'FOR' JI-I 'STEP' I 'UNTIL' N-INDEX+I 'DO'
'IF' COLLSTCJJ > NAV 'AND' 'NOT'
(COLLSTCJJ > 1000 'AND' COLLSTCJJ < 1001 + M)
'THEN' COLLSTCJJ 1= COLLSTCJJ - II
'FOR' II-NAV+I 'STEP' I 'UNTIL' M 'DO'
'IF' ROWLSTCIJ < N 'THEN' ROWLSTCIJI-ROWLSTCIJ-I;
'FOR' JI-K+I 'STEP' I 'UNTIL' N-INDEX+I 'DO'
COLLSTCJ-IJI=COLLSTCJJI
CHECK FOR NO SOLUTION OF PARTICULAR PROBLEMI
'IF' EXITTYPE'O 'THEN'
'GOTO' END OF BRANCH;

CHECK WHETHER WORTHWHILE TO PERSUEI

'IF' 'NOT' TCM+I.N+I-INDEXJ > SOLUTION VALUE
'THEN' 'GOTO' END OF BRANCH;
CHECK IF ADDITIONAL INTEGER RESTRICTIONS ARE METI
INTGI -'TRUE' I
'FOR' PI-INDEX+I 'STEP' I 'UNTIL' NZOV 'DO' 'BEGIN'
NUMI-OI
'FOR' 11"1 'STEP' I 'UNTIL' M 'DO'
'IF' ROWLSTCIJ=NAV+P-INDEX 'THEN' NUM:=TCI.N+I-INDEXJ;
'FOR' J"'I 'STEP' I 'UNTIL' N-NZOV-INDEX 'DO'
'IF' COLLSTCJJ-NAV+P-INDEX+IOOOO 'THEN'
NUMI~lJ
'IF' NUM < 0.000001 'THEN' NUMI=O;
'IF' NUM > 0.99999 'THEN' NUMI=I;
'IF' 'NOT' (NUM=O 'OR' NUM=I) 'THEN' INTGI='FALSE';
DCNAV+PJI"NUM;
'END' ;
 BRANCHING METHODS 665

'IF' 'NOT' INTG 'THEN' 'GOTO' MARK NON INTEGER 501.;

MARK NEW IMPROVED INTEGER S01.UTIONI
S01.UTION VA1.UEI-TCM+I,N+I-INDEXlJ
PUT eRANCHED ZO VARI
'FOR' It-I 'STEP' I 'UNTI1.' INDEX 'DO' 'BEGIN'
OUTROW1.STCIll-NAV+IJ
OUTAC 1,1 ll-DCNAV+IlJ 'END'J
INITIATE EXIT TAB1.EAU PARAMETERSI
OUT N I- N-INDEXJ III-INDEXJ
PUT OTHER ZO VARI
'FOR' NAMEI-NAV+INDEX+1 'STEP' 1 'UNTI1.' NAV+NZOV 'DO'
'BEGIN'
IlI-ll+lJ
OUTROW1.STCIIll~NAMEJ
'FOR' II-NAV+I 'STEP' I 'UNTI1.' M 'DO'
'IF' ROW1.STCIl-NAME-iNDEX
'THEN' DUTACII,lll-TCI,N-iNDEX+ll;
'FOR' JI-I 'STEP' I 'UNTI1.' N-INDEX 'DO'
'I F' C01.LSTC Jl-NAME 'THEN' 'OUTAC I I, I 1 1-0;
'FOR' JI-I 'STEP' 1 'UNTI1.' N-INOEX 'DO'
'IF' C01.LSTCJl-IOOOO+NAME -INDEX
'THEN' OUTACII,lll-iNTACM+2,NAMElJ
'END' J

PUT VAR OF TYPE A8S1

'FOR' NAMEI-I 'STEP' I 'UNTIL' NAV 'DO' 'BEGIN'
OUTROWLSTCNZOV+NAMEl I- NAMEJ
OUTACNZOV+NAME, I 1 I-TCNAME,N+1-INDEX]; 'END';
III-NZOV+NAVJ
PUT ORDINARY VARIABLES WITH COLUMN NAMESI
'FOR' NAMEI-NZOV+NAV+I 'STEP' I 'UNTI1.' N 'DO' 'BEGIN'
'FOR' 1I-NAV+1 'STEP' I 'UNTIL' M 'DO'
'IF' ROWLSTCIl-NAME-iNDEX 'THEN' 'BEGIN'
III-II+I;
OUTROW1.STCIIll-NAMEJ
OUTAC 11,1 1 I-TC I, N+ I-iNDEXl+ INTACM+3, NAME]; 'END';
'FOR' Jlal 'STEP' I 'UNTI1.' N-INDEX 'DO'
'IF' C01.1.STCJl-NAME+IOOOO-iNDEX 'THEN' 'BEGIN'
IlI-ll+lJ
OUTROWLSTCII ll-NAMEJ
OUTACII,lll-iNTACM+2,NAMEl+INTACM+3,NAME1J 'ENO'J
'END' J
PUT S1.ACKSI
'FOR' NAMEI-IOOO+I 'STEP' 1 'UNTI1.' 1000+M 'DO' 'BEGIN'
'FOR' 1I-NAV+1 'STEP' I 'UNTI1.' M ,~O'
'IF' ROWLSTCIl-NAME 'THEN' 'BEGIN'
111-U+lJ
OUTROW1.STCII llaNAME;
OUTAC 1 I, Ill-TC I,N+I-INOEXH 'END'; 'END';
666 CHAPTER XX

OUT M a- II)
'GOTO' END OF BRANCH;

MARK NON INTEGER SOLa

NEWCODEa =CODEJ
'FOR' Pa-INDEX+I 'STEP' I 'UNTIL' NZOV 'DO'
NEWCODEa-NEWCODE+CI+DtNAV+Pl)*3 f CINDEX-P)J
'FOR' Ja-I 'STEP' I 'UNTIL' NSL 'DO' 'BEGIN'
'IF' CODELISTtJl-O 'THEN' CODELISTtPla=NEWCODEJ
'GOTO' 8RANCH AGAINJ 'END'J

WARNING FOR EXHAUSTING SOLUTIONLISTa

'IF' P> NSL 'THEN' 'BEGIN'
NEWL I NEC I )J
WRITETEXTC'C 'SOLUTIONSILISTIEXHAUSTED') '); 'END'J

BRANCH AGAINa

'IF' INDEX-NZOV 'THEN' 'GOTO' END OF BRANCH;

'BEGIN' 'REAL' SAVED ONEJ

CALCULATE MAXIMUM ONEBRANCHI
ONEDISI-I-TtNAV+I.N-INDEX+I1J
QUOI-IOOOOOOOOOJ
'FOR' Ja-I 'STEP' I 'UNTlL"N-lNDEX 'DO'
'IF' 'NOT' CCOLLSTtJ]>IOOO 'AND' COLLSTtJl < IOOO+NEQ+I)
'AND' TtNAV+I.Jl < 0 'THEN' 'BEGIN'
'IF' -TtM+I.Jl/TtNAV+I.Jl < QUO 'THEN'
QUOa"-TtM+I.Jl/TtNAV+I.JlJ 'END')
SAVED ONE I- TtM+I.N-INDEX+Il - QUO*ONEDISJ
CALCULATE MAXIMUM ZEROBRANCHI
ZERODISI-TtNAV+I.N+I1J
QUOI-IOOOOOOOOOJ
'FOR' JI-I 'STEP' I 'UNTIL' N-INDEX 'DO'
'IF' 'NOT' CCOLLSf[Jl > 1000 'AND' COLLSf[Jl < IOOO+NEQ+I)
'AND' TtNAV+I.Jl > 0 'THEN' '8EGlN'
'IF' TtM+I.Jl/TtNAV+I,Jl < QUO 'THEN'
QUol-TtM+I.Jl/TtNAV+I.JlJ 'END'J

ZERo8RANCHI
RE VALUE I- OJ
'IF' 'NOT' TtM+I.N-INDEX+Il - QUO*ZERoDIS >
SOLUTION VALUE 'THEN' 'GOTO' ONEBRANCHJ
FORKa
DtNAV+INDEX+I11-RE VALUEJ
NEWCODEI-CoDEJ
NEWCoDE.-NEWCODE+CI+RE VALUE)*3fCNZOV-lNDEX);
BRANCH(NEWCoDE.RE VALUE.INDEX+I.TO BE SOLVED FROM NEW);
TO BE SOLVED FROM NEW a= 'TRUE')
BRANCHING METHODS 667

ONEBRANCHI
'IF' RE VALUE"O 'THEN' 'BEGIN'
'IF' 'NOT' SAVED ONE> SOLUTION VALUE
'THEN' 'GOTO' END OF BRANCH;
RE VALUE 1= 11 'GOTO' FORKJ 'END'; 'END';

END OF BRANCHI 'END'J

PARTLY FILL EXIT NAMELISTI

'FOR' 11=1 'STEP' 1 'UNTIL' NZOV 'DO' OUTROWLSTCI JI"NAV+r;
'FOR' 11=1 'STEP' 1 'UNTIL' NAV 'DO' OUTROWLSTCNZOV+IJI=r;

FILL CODELIST AND SOL LIST WITH DUMMIESI

'FOR' PI=l 'STEP' 1 'UNTIL' NSL 'DO' 'BEGIN'
SOLUTION VALUES LI STC P J 1= 0; CODELI STC P J 1--1 0; 'END';

INITIATE SOLUTION VALUE LOWI

SOLUTION VALUE 1= -1000000000J

INITIATE TABLEAU ORDERI

OUT M 1= MJ OUT N I " N-NZOV;
BRANCHeo.o.o.'TRUE')J

REGISTER WHETHER SOLVED:

'IF' SOLUTION VALUE> -99999999 'THEN' EXITTYPEI=O
'ELSE' EXITTYPEI=-lJ

END OF ZOLPI 'END'J

668 CHAPTER XX

20.2 Branching in the general mixed integer problem

The general mixed integer problem is here defined as follows

Maximise T = WI X

Subject to Ax < b (7.2.2)

1. < x. < h. (j n l , n l + 1, ... , n) (20.2.1)

J J J
and

x. 1. + 1, 1. + 2 h. )
J J J J )
(j nl + 1, n l + 2 ..... , n l + nirv) ) (20.2.2)

Here n is the total number of elements of ~, n l is the number

of "free" variables which are not subject to any lower or upper
bound, an nirv is the number of integer-restricted variables.

It is more or less conventional to restrict integer-restricted

variables to non-negative valued integers (Xj = 0, 1, 2 .... ),
but this convention has no meaning in connection with the
algorithm offered here, because the upper and lower limit
facilities of sections 10.3 and 10.4 will be employed in any
case. Note, however, that while upper limits may be put at
some fancyhigh figure if no meaningful upper limit is intended,
one should try to supply a realistic lower limit, for the
reasons explained in section 10.4.

The notion of the zero-branch and the one-branch, as discussed

in section 20.1 may now be replaced by that of the lower branch
and the higher branch.

Branching consists of adding additional restrictions to the

problem, the branching restrictions, or to be more precise in
redefining upper and lower limits. We may branch by inequalities
or by equations. If we branch by means of inequalities, we
branch (split) the problem for some variable x. into two rump-
problems. J

In the x. lower branch problem, x. 1S restricted to

J J

x. < int (x~) (20.2.3)

J
I J
where xi is the current (fractional) value of x., int(x~) is
the next lower integer number. J J
BRANCHING METHODS 669

The x. higher branch problem, then contains the branching

J
restriction

X. > int (x~) + 1 (20.2.4)

J J

As we may find need to subsequently investigate different

integer values of x., we also refer to the problems restricted
by (20.2.3) or (20.2.4), as x. higher main branch and the
J
x. lower main branch problems.
J

There is no general rule which says that either (20.2.3) or

(20.2.4) is binding on the integer optimum.

Example

Maximise T 9 x -
1
Subject to 3 x2 > 7 xl' 10 xl + 11 x3 ~ 25, 0 < xl < 100,

o < x2 ~ 100, 0 < x3 ~ 100,

Xl and x 2 integer-restricted, x3 continuous.

The continuous optimum of this problem is xl = 2!, x 2 = 5 5/6,

x3 = 0, with a solution value of T = 5.

The restriction 3 x 2 ~ 7xl is binding in the continuous

optimum, and it effectively reduces the amount of solution
value to be obtained from a unit of xl' from 9 to 2.

The integer requirements on xl and x 2 imply however, more than

3 units of x 2 per 7 units of x 2 ' except for exact multiples of
xl = 3, x 2 = 7. The resu~t is that the continuous variable
x3 becomes a more econom~cal way of utilizing the 25 units of
b 2 , and the integer optimum is xl = x 2 = 0, x3 = 2 1/11, with
T = 4 2/11.

A point in the algorithm where a previously existing LP problem

is split into one or more sub-problems which differ only in
the branching restriction on one variable, is called a node.

There is no uniform convention with respect to equations or

inequalities. Land and Doig {26J are generally credited with
670 CHAPTER XX

having pioneered the branching method, and they use equations.

Hence in our example, branching on x 2 would give rise to the
x = 0, x 2 = 1, and x 2 = 2 branches - thereafter, for x 2 > 2!,
tte problem becomes infeasible. Dakin (7J on the other
hand uses inequalities.

Partly for this reason the term "branch and bound method" which
is otherwise usual is not used here. There is an element of
ambiguity in this term because the word bound could refer either
to a bound on the objective function or to a bound on a variable;
we will simply speak of a branching method.

A major problem in all branching algorithms is how to keep

record of the de1eniation between those parts of the continuous
feasible space area that have already been explored, and the
areas that still need investigating and which may conceivably
contain the optimum.

Dakin solves this record-keeping problem by keeping a list

of so-far unexplored problems. Every time a higher branch
problem is chosen, the corresponding lower branch problem is
added to the list of unexplored problems and vice versa. Land
and Doig on the other hand, solve sub-problems strictly in order
of declining solution value; as a result of this procedure their
algorithm tends to involve a substantial number of sub-problems,
which are in the process of further branching at the same time.

The algorithm which is offered here, solves the record-keeping

problem by postulating a re-ordering of the variables, using
just two sub-problems per variable at the same time. It is
agreed by all those who have tried their hands on integer
programming that it is computationally efficient to branch
on a variable for which the integer restriction is in some
sense more binding than for other variables.

The criterion opted for here is to branch on the variable

which is furthest away from any integer value, that criterion
also appears to be the one used in most other algorithms. The
ordering system is as follows:

The number of variables, which are already subject, or being

subjected to branching restrictions, is called the node-index,
BRANCHING METHODS 671

or the node level. The variables to which branching restrictions

apply, are then called the variables at or below the node index.
The variables at or below the node index are considered as
ordered according to the time of branching. The variables at or
below the node index may further be distinguished as follows:

The variable on which branching took place in the continuous

problem before there were any branching restrictions on other
variables is indicated as the leading variable, the variable on
which branching is now taking place or has just taken place is
indicated as the variable at the node index.

The variables on which branching took place previously and on

which branching restrictions are still in force in the current
problem are together indicated as the variables below the node-
index. There is no ordering of the variables to which no
branching restrictions apply, and these (unordered) integer-
restricted variables on which no branching restrictions apply
in the current problem are called the variables above the node-
index.

The ordering of the variables below tne node-index permits us

to develop additional sub-problems with branching restrictions
which differ only in the constant, in a systematic way.

Suppose, for example, that we are dealing with a problem in

which there are 4 integer restricted variables. We will indicate
them as xl' x 2 ' x3 and x 4 ' but it should be borne in mind that
the first integer restricted variable may be preceeded by any
number of free variables and that the last integer restricted
variable may be followed by any number of bounded continuous
variables. (We impose the convention that free variables come
before integer-restricted variables, and that bounded continuous
variables come after integer restricted variables.)

Variable lower branch higher branch current branch

x3 3 7 higher

x2 5 exhausted lower

x4 unexplored 4 higher

xl unexplored unexplored ?

node-index 3.
672 CHAPTER XX

This record sheet gives us the following information:

The variable on which branching took place for the first time,
was x~, i.e. x3 is the leading variable. Sub-problems which
conta1ned the integer requirements x3 = 4, x3 = 5, and x 1 = 6,
have already been fully explored; if there was an indication
that the integer optimum could be characterized by x3 = 4,
x3 = 5, or x3 = 6, a best so far attained integer solution with
4 ~ x 1 ~ 6 is already noted separately, to be available for
output if it still is the best solution at the end of the
algorithm. The x3 = 3 sub-problem has been found feasible
(otherwise the x lower branch would be indicated as being
exhausted), but ihis sub problem has not so far been explored
further, because the solution value of the x3 = 7 sub problem
was found to be higher. The problem which is currently being
investigated, contains the branching restrictions x3 = 7,
x 2 = 5, x 4 = 4, while no branching restriction applies as yet
to xl'
The x~ = 7, x 2 = 5, x 4 = 4 sub problem has been developed, by
treat1ng the x3 = 7 sub-problem as an integer programming
problem in its own right, with x 2 ' x 4 and xl being the inter-
restricted variables. We distinguish branching in a sub
problem and thereby developing branching restrictions on more
variables, from branching on the leading variable by using
the term further branching. All branching takes place on the
variable which is furthest away from any integer value. If
therefore follows that x 2 was the variable furthest away from
any integer value in the continuous optimum of the x3 = 7 sub
problem, and x 4 was found as the variable furthest away
from any integer value in the continuous optimum of the x3 = 7
x 2 = 5 sub problem.

The value of x 4 in the optimum solution of the x3 = 7, x 2 = 5,

sub-problem must have been in the interval 3 < x 4 < 4, otherwise
the x3 = 7, x 2 = 5, x 4 = 4 sub problem would not be the first
problem to be developed when branching on x 4 . The variables
x 3 ' x 2 and x4 (ordered in that way) are the variables at or
below the node-index, xl is the one variable above the node-
index; x3 and x 2 are the variables below the node-index, x 4
is the variable at the node-index.

To state the algorithm somewhat more formali"y, it may be useful

to introduce some additional notation and terminology.

xj(k) is the jth element of ~, on which branching took place

at node level k.
BRANCHING METHODS 673

The sub-problem which contains the branching restriction

Xj(k) = int (x!(k) + f + 1) (20.2.5)

(and no branching restrictions above node level k) is called

the fth further out sub-branch problem on the Xj (k) higher
branch.

The similar problem for f = 0, is called the opening problem

of the Xj(k) higher main branch. Similarly, for the lower
branch, the opening problem contains the equation-equivalent
of (20.2.3), and the fth further out problem on the lower
branch contains the restriction

Xj (k) = int(x!(k) - f) (20.2.6)

A sub branch problem which differs from a previously developed

one on the same main branch, only by the fact that f has been
increased by one, is called the next further out problem on
the main branch ~n question.

The exploration of both main branch problems and of sub branch

problems takes place under the side condition that the value
of the objective function must exceed a previously set figure.
That figure is initially - 00 (in computational implementation:
- 10 24 ) but as feasible integer solutions are found the
required minimum is increased each time an improved solution
is identified.

A sub branch problem which, though possessing an optimal and

feasible solution, does not attain the required solution value
is called a sub-optimal sub-branch problem.

A sub branch problem is said to be fully explored if we have

found the integer optimum of the sub branch problem, but also
if it is found to be empty, empty of integer solutions, sub-
optimal or empty of non sub-optimal integer solutions. Since
sub-branch problems are solved by treating them as integer
programming problems in their own right, we refer to them ~n
that capacity as head problems, i.e. a head problem is a
problem which is split into sub-problems by branching.

A main branch is said to end if we establish that the integer

optimum (if it exists) does-not obey the restrictions of any
of its remaining unexplored further-our problems.

This may occur on the following indications:

a) A sub branch problem is found to be empty

We may interpret the br~nching restriction on the variable at

674 CHAPTER XX

the node index as an inequality even where it is transformed

from a binding inequality into an equation, when the sub
branch problem is redefined as a head problem in order to
branch further on the remaining variables above node-index.

Further out problems may then be said to be developed from

each other by parametrically displacing the constants in
(20.2.3) and (20.2.4). Once such a displacement results in
the generation of an empty problem, this automatically
implies the emptiness of all remaining further out problems.
(See also Chapter XIII).

b) A sub-branch problem is found to be sub-optimal

The same logic as explained under a) for an empty problem

applies here. If the required value of the objective function
is added as a restriction, cases a) and b) become formally
undistinguishable.

c) The (unbranched) solution of the sub-branch problem lS

found to be integer

This condition is verified only if the solution is not sub-

optimal; therefore a newly identified integer solution is by
implication an improved integer solution, and identifies a
new lower limit of all acceptable solution values.

Sub-optimality of the remaining further-out problems is then

implied.

Note that all three of these end of main branch indications

relate to the unbranched solution of a sub branch problem of
the main branch itself, not to sub-problems of these sub-
branch problems. For example an integer programming problem may
contain an equation restriction let us say 3 x2 = 7 x 5 ' and
we may branch on x2 at x 2 = 4.5. As x 2 = 5 implies x5 = 2 1/7,
the x = 5 sub problem is empty of integer solutions, this
fact aoes not contradict the possibility that further out
problems, e.g. the x 2 = 7 sub-problem may yet yield an
integer solution!

One computational point is useful to mention here, before we

outline the algorithm in more detail. This relates to re-entry
of the LP algorithm with an infeasible solution. We can specify
both upper and lower limits simply by putting negative entries
in the appropriate tableau-cells, and in the case of a lower
limit also altering the non-updated limit.
BRANCHING METHODS 675

The following tableau - which is in fact simply a set of figures

rather than the solution of a specific problem may illustrate
the point.

TABLEAU 20.2 A

RECAPITULATION OF LIMITS.

NAME ! ! S 1 S 2 ! ! VALUE DIST.

--------------------------------------
0.70 ! ! 2.15 97.85
X ! !.
1 -1-10
X 2 ! ! -0.10 0.50 ! ! 7.65 92.35
--------------------------------------
T ! ! 3.50 1 .1 7 ! ! 5.40 X
UB ! ! X X ! ! X X
LB ! ! 0 0 ! ! X X

It will be recalled (see section 10.3 and 10.4) that lower

limits are not part of the LP algorithm as such, and that they
are - in the computational implementation offered in this book
conventionally stored in the m + 3 d row of the tableau, and stay
in their original place. Hence the two zeros of the lower bound
row indicate that xl and x 2 as reported in the tableau itself
are the true values of the variables their lower limits being
zero.

The upper limits on the other hand go with the variables and are
re-formed when a variable is eliminated as the sum of its old
value and the distance from its upper limit. The computational
implementation of the branching method offered here uses an
integer programming adaptation of the basic LP algorithm,
called INTP which is discussed in more detail later on in
Section 21.4. (The same procedure INTP is also employed by the
cutting algorithm.)

Itis assumed that the upper limits on Xl and x 2 are so far both
at 100. If we branch on x 2 ' the x 2 ~ 8 restricEion is put by
the simple device of replacing the entry of 7.65 in the x 2 -row/
value column by 7.65 - 8 = -0.35 while at the same time
setting the un-updated lower limit at 8.
676 CHAPTER XX

The tableau now becomes:

TABLEAU 20.2 B

LOWER LIMIT REENTRY TABLEAU.

NAME I ! 5 1 5 2 II VALUE DIST.

--------------------------------------
X 1 ! ! -1 .1 0 0.70 I! 2.15 97.85
Y 2 ! I -o.} 0 0.50 II -0.35 !l2.35
--------------------------------------
UB
T I!
! !
3.50
:x
1 .17
X "
II
5.40
:x
)(
)(
LB ! ! 0 8 I! X )(

where y? ~s (see section 10.4) ·the.d~st~nce between X and i7s z

lower 11m1t. As far as Lhe lower 11m1t 1S concerned, the bas1c
LP algorithm as discussed in Part I and written down as the
LINP procedure would suffice. (The phase I substitute
objective function does not distinguish between slack-variables
and elements of x, it just picks negative valued variables).
The one further bit of programming done to serve this aspect
of the lower limit is an adaptation of the reporting procedure
of section 10.4 which distinguished between branched limits in
an exit-tableau, and initial limits in the separately stored
set-up tableau.

The upper limit is dealt with in an ana1ogeous way, the

restriction x2 ~ 7 is put in the tableau, simply by reporting
its upper limit distance to be 7 - 7.65 = - 0.65, and the
opening problem of the x 2 lower branch is written as follows:

TABLEAU 20.2 C

UPPER LIMIT REENTRY TABLEAU.

NAME
"II -1 .10 0.70 II 2.15 97.85
5 1 5 2
--------------------------------------
X 1
II VALUE DIST.

X
" 3.50 1 • 17 II" 5.40 X
2 -0.10 0.50
--------------------------------------
7.65 -0.65

T
UB "" X X II X X
LB II 0 0 II X X
BRANCHING METHODS 677

i.e. x 2 is in excess ~f its upper limit, the excess being 0.65.

The integer programming adaptation of the LP algorithm INTP as
distinct from LINP) will react to this tableau by generating
the appropriate substitute objective function, flicking the
x 2 -row round as a b 2 row and make the required step.

TABl.EAUX 20.2 D AND E

El.EMINA Tl ON OF AN UPPER l.IMIT.

NAME II S 1 S 2 II VAl.UE D1ST.

--------------------------------------
2.15 97.85
"
X 1 -1 .1 0 0.70 II
B 2 II 0.10 (-0.50) II -0.65 7.65
--------------------------------------
T II 3.50 1 .17 II 5.40 X
UB II X X II X X
l.B II 0 0 II X X

" 1.24 98.76

NAME II S 1 B 2 VAl.UE DIST.
--------------------------------------
X 1 II -0.96 1 .40 II
S 2 II -0.20 -2 II 1 .30 X
--------------------------------------
"
2.34 3.88
"
T 3.73 X
UB II X 7 II X X

"
l.B 0 0 I! X X

The same facility also makes it possible to generate the next

further out problem, by subtracting the branching-restriction
column from the value column and adding it to the upper limit
distance column (and changing the separately stored constant
of the upper limit restriction).
678 CHAPTER XX

Hence a re-entry tableau of the x 2 6 sub-problem ~s written

as follows:

TABLEAU 20.2 F
A FURTHER-OUT REENTRY TABLEAU.

NAME! ! S 1 B 2 I! VALUE DlST.

--------------------------------------
-0.16 100.16
X 1 ! ! -0.96 ! ! 1.40
5 2 ! I - 0.20 -2 I! 3.30 X
--------------------------------------
T ! ! 3.73 2.34 II 0.54 X
UB ! ! X 6 II X X
LB ! ! 0 0 II X X

and the same method would always produce a tableau which can
be handled by the Phase I of INTP, even if the result would
include negative distances from upper limits.

We are now in a position to summarise the algorithm in some

detail. The branching procedure may be called at any node-level.
It performs the following operations

it reserves memory space for an upperbranch tableau and a

lower branch tableau, which are released when returning to the
point of call
it branches on one variable, and for the appropriate values
of that variable generates a call to itself, one node-level
higher
it writes any integer solutions of which the solution value
exceeds the previously attained one, to an output-buffer
tableau.
it makes appropriate adjustments to the branching record.

Since each call generates further calls until the problem is

exhausted, the effect of a single call is to ask for solution
of the integer programming problem in which the variables
below and at the node-index are set at specific integer values,
whereas the variables above the node-index are specrfied as
integer restricted variables, but will generally be fractionally
valued in the solution offered at the moment of calling.

We now proceed as follows:

Preliminary

Apply the Simplex Algorithm to the continuous problem (If an

empty or unbounded problem is met at this stage, terminate the
BRANCHING METHODS 679

algorithm).

Set the minimum required solution value at _10 24 .

Call the branching procedure at node-level 1, offering the

continuous problem as head problem.

Once inside the branching procedure, our operations-summary

is as follows:

Stage 1

Search for a variable to branch on

Search for the integer restricted variable x'(k)' which is

furthest away from any integer value. If nonJ is found, all
integer-restricted variables being integer valued, declare the
current head problem the integer optimum and exit to the point
of call. (This can only happen at this stage if the head problem
is the continuous problem).

Stage 2 high

Open the higher branch

Mark the higher branch as being opened, and the lower branch
as unexplored. Copy the current head problem into the higher
branch tableau and make the adjustments needed to convert it
into a re-entry tableau of the opening problem of the higher
main branch.

Stage 2a high

Re-enter the Simplex Method (higher branch)

If the current higher branch sub-problem is found empty or its

solution sub-optimal, mark the higher branch as exhausted, and
go to stage 3, (otherwise proceed to stage 2b high).------

Stage 2b high

Check for integer solution (higher branch)

If the current solution is integer in all integer-restricted

variables, write the current tableau to the output-buffer, set
the required solution value at its new higher figure and mark
the higher branch as exhausted. (This is of necessity the case
at maximum node-level, hence the branching process does not
carryon indefinitely). Go to Stage 3.
680 CHAPTER XX

Stage 2 low

Open the lower branch

Mark the lower branch as being opened.

Copy the current head problem into the lower branch tableau
and make the adjustment needed to convert it into a re-entry
tableau of the opening problem of the lower main branch.

Stage 2a low

Re-enter the Simplex Method (lower branch)

If the current lower branch sub-problem is empty, or its

solution sub-optimal, mark the lower branch as exhausted,
and go to stage 3, otherwise proceed to stage 2b low.

Stage 2b low

Check for integer solution (lower branch)

If the current subproblem is integer in all integer-restricted

variables, write the current tableau to the output-buffer, set
the required solution value at its now higher figure and mark
the lower branch as exhausted.

Stage 3

Choose what to do next

If both branches are exhausted, exit to the point of call, i.e.

to the end of the algorithm in the main programme if the node-
level is one, and otherwise to the end of one of the two sides
of stage 4, one node-level lower.

If the lower branch is unexplored, go to stage 2 low to open

the lower branch.

If the higher branch is exhausted go to stage 4 low; if the

lower branch is exhausted, go to stage 4 high.

If the solution value of the current lower branch sub-problem

exceeds that of the current higher branch sub-problem, go to
stage 4 low, otherwise proceed to stage 4 high.

Stage 4 high

Branch further (high)

Offer the current higher branch problem as head-problem, with

BRANCHING METHODS 681

all branching restrictions (at or below the node-index, there

are no other), coded as equations, as head-problem to the
branching procedure itself, one node-level higher, thereby
again entering the branching procedure at stage 1.

Stage 4a high

(When getting at this point the last-formulated higher branch

problem has been fully explored)

Push upwards

Adapt the value-column, the upper limit distances column and

the lower limit on Xh(k) , to form a re-entry tableau for the
next further-out problem on the higher branch.

Go to stage 2a high, to solve the new further-out problem.

Stage 4 low

Branch further (low)

Often the current lower branch problem as head-problem, with all

branching restrictions coded as equations, as head problem to
the branching procedure itself, one node-level higher than the
current one.

Stage 4a low

Push downwards

Adapt the value column, the upper limit distances column and
the upper limit on Xj(k)' to form a re-entry tableau for the
next further-out problem on the lower branch.

Go to stage 2a low, to solve the new further-out problem.

End of branching procedure.

We now illustrate the algorithm, by reference to the small 3-

variable problem which was stated earlier in this section but
which is repeated here.

Maximise T = 9x - 3x 2 + 2x3
1
Subject to 7x l - 3x3 < 0

10x l + llx 3 < 25

(0 < x
- 1
< 100, o .::. x 2 < 100, o <
-
x3 -
< 100 , x l ,x 2 integer)
682 CHAPTER XX

The following operations are involved

Preliminary

Solve the continuous problem, finding an optimal and feasible

solution, xl = 2.50, x 2 = 5.83, x3 = 0, T = 5.

Enter the branching procedure with node-index 1

Stage 1

Find xl as xj(k)' to be branched on.

Stage 2 high

Form the re-entry tableau as follows

TABLEAU 20.2 G

OPENING TABLEAU OF THE XI HIGHER BRANCH (EMPTY)

NAME! ! S I S 2 X 3 ,! VALUE DIST.

Y I !! 0.10 lol0!! -0.50 97.50

X 2 !! -0.33 0.23 2.57 ! I 5.83 94.17

T ! I 0.20 0.20!! 5 X
UB !! X X 100 I! X X
LB ! I 3 I! X X

(Conform section 10.4 lower bounds are stored in the cells

which correspond to the variable's original position, the
difference between the zero in the set up tableau and the 3
in the node 1 tableau, permits us to distinguish y, as a
branched variable from the original xl variable even where
their numerical codes are identical).

Stage 2a high

On re-entering the Simplex Algorithm the problem is found to

be empty. (in the small example this is obvious without
reference to a tableau, xl -> 3 contradicts lOx1+ llx3
-
< 25,
x3 :=.. 0).

We mark the xl higher branch as exhausted.

BRANCHING METHODS 683

Stage 3

Since the lower branch 1S unexplored, we go to stage Z low.

Stage Z low

We form the re-entry tableau

TABLEAU 20.2 H

OPENING TABLEAU OF THE XI LOWER ~RANCH

NAME! ! S I S 2 X 3 !I VALUE DIST.

X I !I 0.\ 0 1 • 10 ! ! 2.50 -0.50
X 2 !I -0.33 0.23 2.57 ! ! 5.B3 94.17
------------------------------------------------
T ! ! 1 0.20 o .20 I! 5 X
UB !I X X 100 ! ! X X
LB !I ! ! X X

Stage Za low

On re-entering the Simplex Method, we find a new optimal

tableau, which is

TABLEAU 20.2 I

CONTINUOUS OPTIMUM OF Xl LOWER BRANCH HEADPROBLEM

NAME! ! S 1 S 2 B 1 !! VALUE DIST.

X 3 !! 0.09 -0.91 II 0.45 99.55

X 2 !! -0.33 2.33 !! 4.67 95.33

T !! 0.\8 0018 !! 4.91 X

UB !! X X 2 !! X X
LB !! ! ! X X

This solution is not suboptimal, we proceed to stage Zb low.

Stage Zb low

We find that the solution 1S not integer 1n x Z ' therefore we

go to stage 3.
684 CHAPTER XX

Stage 3

Since the xl higher branch is empty, (has been exhausted)

further branching now takes place on the last developed problem
on the xl lower branch.

Stage 4 low

Offer the xl = 2 sub-branch problem as head problem, for

further branching at node-level 2.

Stage 1

We find x 2 as the next variable to branch on.

Stage 2 high

We form the re-entry tableau

TABLEAU 20.2 J

REENTRY TABLEAU OF THE XI = 2. X2 ~ 5 PROBLEM

NAME II S I S 2 B 1* II VALUE DIST.

X 3 I! 0.09 -0.91 II 0.45 99.55

Y 2 ! ! (-0.33) 2.33 II - 0 .33 93.33
------------------------------------------------
T ! ! 1 O. 18 0.18 ! ! 4.91 X
UB II X X 2 II X X
LB I! 5 II X X

B1 NOT TO ENTER THE BASIS

*

N.B. In actual computational implementation xl would be

temporarily recoded as sl~and sl and s2 as s2 and s3,
to conform with the coding of equations discussed in.
section 10.2.

Stage 2a high

We find a new optimum tableau as follows:

BRANCHING METHODS 685

TABLEAU 20.2 K

OPTIMUM OF THE XI 2. X2 ~ 5 PROBLEM

NAME II Y 2 S 2 B 1* II VALUE DIST.

X 3 " 0.09 -0.91 " 0.45 99.55

S 2 II -3 o -7 II 1 X

T II 3 0.18 7.18 It 3.91

UB II 95 X 2 II X X
LB I I 5 I I X X

* Bl NOT TO ENTER THE BASIS

This solution is not sub-optimal, hence we proceed to stage 2b

high.

Stage 2b high

We find the solution integer, therefore xl = 2, x 2 = 5,

T = 3.91 is listed as the best so far obtained integer solution.
We mark the x higher branch (of the xl = 2 subproblem) as
exhausted, ana require all future solutions to satisfy the
condition T > 3.91.

Stage 3

Since the x 2 lower branch is so far unexplored, we go to

stage 2 low.

Stage 2 low

We form the re-entry tableau as follows

TA8LEAU 20.2 1.

REENTRY TABLEAU OF THE Xl = 2. X2 ~ 4 PROBLEM

NAME II S 1 S 2 D 1*!! VALUE DIST.

------------------------------------------------
X 3 ! I 0.09 -0.91 !I 0.45 99.55
X 2 ! I -0.33 2.33 II 4.67 -0.67
------------------------------------------------
T II 0.18 0.18 II 4.91 x
U8 II x X 2 !! X X
1.8 II II X X
686 CHAPTER XX

This is an empty problem, the substitute objective function

is in this case:
sum of infeasibilities = b 2 = -0.33 sl + 2.33 bt and since b l
is an equation slack,no permitted incoming variable is found.
We mark the x 2 lower branch (of the xl = 2 subproblem) as
exhausted.

Stage 3

Since both main branches at node-level 2 are exhausted we

return to the point of call which is the end of stage 4 low,
having found the integer optimum of the xl = 2 sub-problem.

Stage 4a low (node-level 1)

We develop the lower branch re-entry tableau as follows

TABLEAU 20.2 M

REENTRY TABLEAU OF THE XI =1 FURTHER OUT PROBLEM

NAME! ! S I S 2 B I*!! VALUE DIST.

X 3 !! 0.09 -0.91 II I .36 98.64
X 2 !! -0.33 2.33 !! 2.33 97.67

T I! 0.18 0.18 !! 4.73 X

UB II X X I !! X X
LB ! I I! X X

~ BI NOT TO ENTER THE BASIS

Stage 2a low

We find the re-entry tableau already optimal and feasible

without any steps, and the solution is not sub-optimal
T = 4.73 > 3.91.

Stage 3

Since the xl higher branch is exhausted, we go to stage 4 low.

Stage 4 low

We offer the xl = I sub problem as head-problem, entering

the branching procedure at node-level 2.

Stage 1

We find x 2 as the next variable to branch on.

BRANCHING METHODS 687

Stage 2 high

The higher branch re-entry problem re-entry tableau is as

follows:

TABLEAU 20.2 N

REENTRY TABLEAU OF THE Xl =1. X2 ~ 3 PROBLEM.

NAME !! S 1 S 2 B 1. I! VALUE DIST.

X 3 !! 0.09 -0.91 !! 1.36 98.64
Y 2 !! -0.33 2.33 I! -0.67 97.67

T ! ! 0.18 0-18 ! ! 4.73 X

UB ! ! X X ! ! X X
LB ! ! 3 ! ! X X

• BI NOT TO ENTER THE BASIS

Stage 2a high

Xl = 1, x 2 = 3, (Y2 = 0), T = 2.73

is found as optimal and feasible solution of the specified
sub-problem. The solution of this problem is, however, sub-
optimal, in view of the earlier found solution xl = 2,
x 2 = 5, T = 3. 91 •

We mark the x 2 higher branch as exhausted, and go to stage 3.

Stage 3

Since the x 2 lower branch (of the xl = 1 sub-branch problem)

is still unexplored, we go to stage 2 low.

Stage 2 low

The re-entry tableau for the xl = 1, x 2 < 2 sub problem is

developed, but in the interest of avoiai~g tedious repetition
it is not copied here.

Stage 2a low

The xl = 1, x 2 ~ 2 subproblem is found empty (contradicts the

restriction 3x 2 ~ 7).
688 CHAPTER XX

We mark the Xz lower branch of the xl 1 subbranch problem

as exhausted, and go to stage 3.

Stage 3

Having exhausted both the higher and the lower x 2 main branch
problems, we return to the point of call i.e. tne end of
stage 4 low of node-level 1.

Stage 4a low

The next further-out problem on the xl lower branch is the

xl = 0 subbranch problem, the re-entry tableau of this
problem is as follows

TABl.EAU 20.2 0

REENTRY TABl.EAU OF THE Xl a 0 SUB-PROBl.EM.

NAME II S 1 S 2 B .. II VAl.UE DIST.

-------------------------------------------------
X 3 II 0.09 -0.91 II 2.27 97.73
X 2 II -0.33 2.33 II 100
-------------------------------------------------
T II 0.18 0.18 II 4.55 x
UB II X X II X X
1.B II ! I X X

• Bl NOT TO ENTER THE BASIS

The zerq values of x 2 arises in this case because, with Xl at

the value xl=O, the Einding restriction 7x l 23x2 determines
the corresponding value of xZ.

In fact the tableau print was given as -0.00 and a further

step was needed.

Stage 2a low

We find Xl = 0, Xz = 0 as solution, this solution is not

sub-optimal. It was not actually the same vertex, the value
of x 2 was exchanged against the slack of the restriction
7xl - 3~2 2 0, the latter restriction now being classified as
not bind1ng with the slack-variable entering the basis of the
value sl = O.

This solution is not sub-optimal, we therefore proceed to

stage 2b low.
BRANCHING METHODS 689

Stage 2b low

The current sub-problem is found integer. We record the new

solution xl = 0, x2 = °
as the best so far found one and set
the required solution value as T > 4.55, and mark the xl lower
branch as exhausted.

Stage 3

Having exhausted both the xl higher branch and the xl lower

branch we exit from the branching procedure.

°
For the leading variable xl that is also the exit from the
algorithm, xl = 0, x 2 = with x3 2.27 is the integer optimum.

20.3 Branching methods developed by other authors

We now briefly discuss the two main published branching

algorithms (Dakin's [7] and Land and Doig's (261 .

We take Dakin's first.

As already indicated, Dakin uses inequalities. Accordingly,

in the example-problem discussed above, the first sub-problem
is the xl ~ 3 sub problem. This is empty, and further branching
takes place on the basis of the solution of the xl ~ 2 sub
problem, for the same reasons as listed above for selecting
the xl = 2 subproblem for further branching on other variables
i.e. x 2 . The notion of developing further-out problems i.e.
exploring other integer values of variables which have already
been branched upon is dropped, in favour of that of the
alternative problem.

This feature of Dakin's algorithm is activated when the

xl ~ 2, x 2 ~ 5 subproblem has been solved.

This problem has a solution and it is in fact an integer

solution, i.e. xl = 2, x 2 = 5. Despite the fact that this is
an integer optimum of a sub-problem, the alternative problem
xl ~ 2,:X" ~ 4 is tackled - as was done above with xl = 2,
x 2 = 4 sutproblem, the difference as yet still being solely
the use of inequalities.

The equivalent subproblem when using equations is empty but

when inequalities are used, the solution xl = 1 5/7,
x 2 = 4, x3 = 5/8, with T = 4 6/7, is found, a solution which
satisfies the x-I ~ 2 restriction, but not the xl = 2
restriction. Again branching on xl in the xl ~ 2, x 2 ~ 4
sub-problem makes the development of further out problems with
different values of xl superfluous. The xl ~ 1, x 2 ~ 4
690 CHAPTER XX

subproblems is in due course developed as a sub-problem of

the xl 2 2, xl ~ 4 subproblem, rather than as a further out
problem of the xl lower branch.

The obvious drawback of this is that it involves the need to

store information about the optima of a largish and generally
unpredictable number of alternative problems.

This is in fact the very point on which Dakin criticised the

Land and Doig procedure where the same problem arises in a
different way.

Land and Doig develop, in effect further out problems in much

the same way as is done by the algorithm offered in this
book.

They do not, however, exhaust a subproblem before they develop

the next subproblem at the same mode-level.

This makes it necessary to preserve information about solutions.

If all other subproblems are subsequently found to be either
empty of integer solutions or alternatively to possess lower
solution values it may be necessary to use the solution of a
particular subproblem to the purpose of splitting it into
new ones by further branching.

Exhaustion of a particular subproblem as practiced by the

algorithm offered in this book avoid this problem: either the
sub-problem is empty of non-suboptimal integer solutions or
we record a new best-so-far attained inter solution.

20.4 Text-listing of a recursive branching procedure

We now give the usual text-listing of a procedure-code ..

It follows the outline given earlier in section 20.2 fairly

closely, a feature which was enhanced by editing the text
accordingly and inserting the labels stage 1, etc., even when
they are not actually used by the code.
BRANCHING METHODS 691

'PROCEDURE' BRANCH<T.M.N.NEQ.NAV.NIRV.MINV.ROWL.COLL.
EXT.IROWL.EXROWL.EXCOLL.NODE.EXNODE.RECORD)J
'ARRAY' T.EXTJ
'INTEGER' H.N.NEQ.NAV.NIRV.NODE.EXNODEJ
'INTEGER' 'ARRAY' ROWL.COLL.IROWL.EXROWL.EXCOLL.RECORDJ
'REAL' HINVJ
'BEGIN'
'INTEGER' I.J.R.K.NEXTJ.ROWN.COLN.II.REENTRY.
'REAL' DIST.NUMJ
'PROCEDURE' INTP<T.H.N.NEQ.NAV.ROWLST.COLLST.
IROWLST.R.K.ROWN.COLN.REENTRY)J
'ARRAY' TJ 'INTEGER' H.N.NEQ.NAV.R.K.ROWN.COLN.REENTRYJ
'INTEGER' 'ARRAY' ROWLST.COLLST.IROWLSTJ
'ALGOL';
'BEGIN'
'ARRAY' LT.HT[1IM+3.1IN+2lJ
'INTEGER' 'ARRAY' LROWL.HROWL[1IHl.LCOLL.HCOLL[1INlJ
'COMMENT' CODING CONVENTIONI
THE BRANCHING RECORD IS KEPT IN THE ARRAY RECORD.
ROW 0 OF THIS ARRAY IS RESERVED FOR THE NAME-CODES OF
THE VARIABLES THAT HAVE BEEN BRANCHED ON.
ROWS 1 AND 2 OF THE ARRAY RECORD CONTAIN THE RECORD OF
THE STATUS OF THE VARIOtiS BRANCHES. AS FOLLOWS I
o UNEXPLORED.
1 PRELIMINARY EXPLORED. I.E. BEING OPENED.
2 • BEING EXPLORED FURTHER OUTWARDS. I.E. FULLY OPEN.
3 EXHAUSTED.
ROW 3 IS RESERVED FOR THE EQUATION NAME-CODES OF THE
BRANCHED VARIABLES ASSOCIATED WITH ANY BEST-SO-FAR-FOUND
INTEGER SOLUTION. IN ASSOCIATION WITH THE OUTPUT-BUFFER
TABLEAU. WHERE THESE UPPER AND LOWER LIMITS ARE CODED AS
EQUATIONS.
THIS REFERS TO THE MOMENT OF BRANCHING. OR OF WRITING TO
THE EXIT-TABLEAU.
THE ACTUAL VALUES OF THE LIMITS AT WHICH THE BRANCHING
RESTRICTIONS ARE SET. ARE STORED AS NORMAL UPPER AND
LOWER LIMITS ON THE VARIABLES IN QUESTION.

INITIATE AS UNEXPLORED I
'FOR' 11'·0.1.2 'DO' RECORD[I.NODEll=O;
STAGE 11
FIND A VARIABLE TO BRANCH ONI
NEXTJI-OJ DISTI-O;
692 CHAPTER XX

'FOR' I.-NAV+l 'STEP' 1 'UNTIL'M 'DO'

'IF' ROWLC Il < NAV+NIRV+l 'THEN' 'BEGIN'
'COMMENT'
INTEGER RESTRICTED VARIABLES MUST BE PLACED DIRECTLY
AFTER THE FREE VARIABLES. THEREFORE WE RECOGNIZE THEM
BY THIS CONDITION ON THEIR NAME-CODES. J
NUM .- TCI.N+ll-ENTIER(TCI.N+ll)J
'IF' NUM>0.999 'THEN' TCI.N+ll.-ENTIER(TCI.N+ll+0.002)J
'IF' NUM<O.OOl 'THEN' TC I.N+ll.-ENTIER(TC I.N+ll';
NUM .- TCI.N+ll-ENTIER(TCI.N+ll)J
'IF' NUM > 0.5 'THEN' NUM.=l-NUMJ
'IF' ABS(NUU > 0.00001 'AND' NUM > DIST 'THEN' 'BEGIN'
III=U NEXTJ,"ROWLCIH DIST.'"NUMJ 'END'; 'END'J
'IF' NEXTJ-O 'THEN' 'BEGIN'
'FOR' J.-l 'STEP' 1 'UNTIL' N+2 'DO'
'FOR' 11"1 'STEP' 1 'UNTIL' M+3 'DO' EXTC I.Jl.-TC I.Jl
'FOR' II-I 'STEP' 1 'UNTIL'M 'DO' EXROWLCIl,"ROWLCIl
'FOR' J.-l 'STEP' 1 'UNTIL' N 'DO' EXCOLLCJll-COLLCJl
EXNODE I- OJ MINV I- TCM+l.N+llJ
'GOTO' END OF BRANCHINGJ 'END'J
RECORDCO.NODEJ I- NEXTJJ
STAGE 2 HIGHI
Copy HEAD PR TO HIGHER BRI
'FOR' JI-l 'STEP' 1 'UNTIL' N+2 'DO'
'FOR' II-I 'STEP' 1 'UNTIL'M+3 'DO' HTCI.Jll-TCI.JlJ
'FOR' II-I 'STEP' 1 'UNTIL' M 'DO' HROWLEIlI-ROWLCIH
'FOR' JI-l 'STEP' 1 'UNTIL' N 'DO' HCOLLEJlI=COLLEJJJ
OPEN HIGHER BRANCHI
RECORDC1.NODEJ I" IJ RECORDC2.NODEl I- 0;
HTCM+3.NEXTJll-HTCM+3.NEXTJJ+IJ
HTCII.N+lll-HTCII.N+ll-ENTIER(HTCII.N+ll)-lJ
STAGE 2A HIGHI
REENTER HIGHER BR.
'FOR' II-NAV+l 'STEP' 1 'UNTIL' M 'DO' 'BEGIN'
NUM,"HTCI.N+IJ+HTCI.N+2lJ
'IF' ABS(NUM) < 0.0000001
'THEN' HTCI.N+2ll-o.o0oooo1-HTCI.N+ll; 'END';
'FOR' JI-l 'STEP' 1 'UNTIL' N 'DO'
'IF' (HCOLLCJJ>NAV 'AND' HCOLLCJl<NIRV+l)
'OR' (HCOLLCJl>1000o+NAV 'AND' HCOLLCJl<10001+NIRV)
'THEN' 'BEGIN'
NUM 1= HTCM+2.Jl-ENTIER(HTCM+2.Jl);
'IF' NUM>O.999
'THEN' HTCM+2.Jll-ENTIER(HTCM+2.Jl+O.002)J
'IF' NUM<O.Ool
'THEN' HTCM+2.JlI=ENTIER(HTCM+2.Jl)J 'END';
REENTRYI"l; RI-KI=O; ROWNI"COLN:=O;
INTP(HT.M.N.NEQ+NODE-l.NAV.HROWL.HCOLL.IROWL.R.K.ROWN.
COLN.REENTRY';
BRANCHING METHODS 693

CHECK ON EMPTYNESS OF HBa

'IF' REENTRY· -I 'THEN' 'BEGIN'
'COMMENT'
EMPTY PROBLEM ON HIGHER BRANCH;

'IF' MINV < -993 000 000 'AND' NODE> £KNODE

'THEN' 'BEGIN'
'COMMENT'
NO INTEGER SOLUTION HAS BEEN FOUND SO FAR.
THIS IS SO FAR THE HIGHEST BRANCHED SOLUTION.
JUST IN CASE NOTHING BETTER IS FOUND.
RECORD THIS SOLUTION FOR DIAGNOSTIC PURPOSES;
'FOR' Ja"l 'STEP' I 'UNTIL' N+2 '~O'
'FOR' II-I 'STEP' I 'UNTIL' M+3 ,~O' EXTC I.JlI=HTC I.JJ
'FOR' 11=1 'STEP' I 'UNTIL' M 'DO' EXROWLCIJ"'HROWLCIJ
'FOR' Ja-I 'STEP' I 'UNTIL' N ,~O' EXCOLLCJJ"'HCOLLCJJ
'FOR' Ja-I 'STEP' I 'UNTIL' NODE 'DO'
RECORO[3.JJ a- RECORO[O.JJJ
EXNOOE a- NODE; MINV a- -995 000 000; 'END';

RECOROCI.NOOEJa=3;
'GOTO' CHOOSE; 'ENO'J

HB MAY NEED ANOTHER S.TEPa

'IF' 'NOT' K-O 'THEN' 'GOTO' REENTER HIGHER BR;

HIGHER BR NON EMPTYa

'IF' 'NOT' HTCM+I.N+Il > MINV 'THEN' 'BEGIN'
'COMMENT'
SUBOPTIMAL SOLUTION. TREAT AS EMPTY;
REENTRY a- -IJ
'GOTO' CHECK ON EMPTYNESS OF HBJ 'END';

'IF' RECOROCI.NOOEJ - 2
'THEN' 'GOTO' FIND WHETHER HB INTEGER;

PUT EQUATION CODE IN HBa

Ka-O;
'FOR' Ja-I 'STEP' 1 'UNTIL' N 'DO'
'IF' HCOLLCJJ-NEXTJ 'THEN' 'BEGIN'
Ka-J; HCOLLCJlI-I000+NOOEJ 'END';
'IF' K-O 'THEN' 'BEGIN'
NEWLINE( I H
WRITETEXT('('EQUATION%FOR%NEW%HIGHER%BRANCH%
ON%X')'); PRINT(NEXTJ.5.0);
WRI TETEXT( '( '%%NOT%FOUNO' ) , ); 'END';

'FOR' Ja-l 'STEP' 1 'UNTIL' K-l.

K+l 'STEP' 1 'UNTIL' N '~O'
'IF' HCOLLCJJ>999+NOOE 'AND' HCOLLCJJ<1001+NOOE+M
'THEN' HCOLLCJJa-HCOLLCJJ+l;
'FOR' 1,,'1 'STEP' 1 'UNTIL' M '~O'
'IF' HROWLCIl>999+NOOE 'THEN' HROWLCIJa=HROWLCIJ+l;

STAGE 2B HIGHa
FIND WHETHER HB INTEGERa
'FOR' U-NAV+l 'STEP' 1 'UNTIL'M '~O'
'IF' HROWL[Il < NAV+NIRV+l 'THEN' 'BEGIN'
NUM a- HTCI.N+IJ-ENTIER(HrCI.N+IJ);
694 CHAPTER XX

'IF' NUM>O .999

'THEN' HTCI.N+IJa-ENTIERCHTCI.N+IJ+0.002)J
'IF' NUMcO.OOI 'THEN' HTCI.N+IJI=ENTIERCHTCI.N+Il)J
HUM a- HTCI.N+IJ-ENTIERCHTCI,N+Il)J
'IF' ABSCNUM) > 0.00001
'THEN' 'GO TO , CHOOSEJ 'END' ;

RECORD HB OUTCOMEa
MINV 1= HTCM+I,N+Il; EXNOOE a- NODE;
'FOR' Ja-I 'STEP' I 'UNTIL' N+2 ,~O'
'FOR' II-I 'STEP' I 'UNTIL' M+3 ,~O' EXTCI,Jla=HTCI,Jl
'FOR' 11"1 'STEP' I 'UNTIL' M ,~O' EXROWLCIlp'HROWLCIJ
'FOR' Ja-I 'STEP' I 'UNTIL' N 'DO' EXCOLLC J11-HCOLLC JJ
'FOR' J ... I 'STEP' I 'UNTIL' NODE 'DO'
RECOROC3,Jl .= RECOROCO,Jl;
RECOROCI,NOOEJI-3J
'GOTO' CHOOSE;
STAGE 2 LOW.
COpy HEAD PR TO LOWER BRa
'FOR' J.=I 'STEP' I 'UNTIL' N+2 ,~O'
'FOR' 11"1 'STEP' I 'UNTIL' M+3 ,~O' LTCI,JlI .. TCI,Jl
'FOR' la-I 'STEP' I 'UNTIL' M 'DO' LROWLCIlI=ROWLCIl
'FOR' JI-l 'STEP' I 'UNTIL' N ,~O' LCOLLCJlI=COLLCJl
OPEN LOWER BRANCHI
RECOROC2,NOOEJ.=lJ
LTCII,N+211=LTCII.N+21-ENTIERCLTCII,N+21)-1;
STAGE 2A LOWa
REENTER LOWER BRI
'FOR' Ip'NAV+I 'STEP' I 'UNTIL' M ,~O' 'BEGIN'
NUMa=LTCI,N+l1+LTCI,N+21J
'IF' ABS(NUM) c 0.0000001
'THEN' LTCI.N+211-0.0000001-LTCI,N+Il; 'END';
'FOR' J.-l 'STEP' 1 'UNTIL' N ,~O'
'IF' CLCOLLCJJ>NAV 'AND' LCOLLCJlcNIRV+l)
'OR' (LCOLLtJJ>10000+NAV 'AND' LCOLLCJlcl000l+NIRV)
'THEN' 'BEGIN'
NUM I- LTtM+2,Jl-ENTIERCLTCM+2,Jl)J
'IF' NUM>0.999
'THEN' LTCM+2,Jl.-ENTIERCLTCM+2,JJ+0.002);
'IF' NUMcO.OOl
'THEN' LTCM+2,JlI-ENTIER(LTCM+2,Jl>J 'END';

REENTRy.-1J RI-KI-O; ROWN.=COLNI-O;

INTPCLT,M,N,NEQ+NOOE-I,NAV,LROWL,LCOLL,IROWL,R,K,ROWN,
COLN,REENTRYH
CHECK ON EMPTYNESS OF Lal
'IF' REENTRY· -1 'THEN' 'BEGIN'
'COMMENT'
EMPTY PROBLEM ON LOWER BRANCH;
BRANCHING METHODS 695

'IF' MINV < -993 000 000 'AND' NODE> EXNODE

'THEN' 'BEGIN'
'COMMENT'
NO IfiTEGER SOLUTION HAS BEEN FOUND SO FAR~
THIS IS SO FAR THE HIGHEST BRANCHED SOLUTION.
JUST IN CASE NOTHING BETTER IS FOUND~
RECORD THIS SOLUTION FOR DIAGNOSTIC PURPOSES;
'FOR' Jlsl 'STEP' 1 'UNTIL' N+2 'DO'
'FOR' 11-1 'STEP' 1 'UNTIL' M+3 'DO' EXTCI~Jl"'LTCI~JH
'FOR' U-l 'STEP' 1 'UNTIL'M 'DO' EXROWLCIll .. LROWLCIlJ
'FOR' JI-l 'STEP' 1 'UNTIL' N 'DO' EXCOLLCJl.~LCOLLtJl;

'FOR' JI-l 'STEP' 1 'UNTIL' NODE 'DO'

RECORDt3~Jl I- RECORDtO~JlJ
EXNODE I- NODEJ MINV I- -995 000 OOOJ' 'END'J
RECORDt2~NODEll-3J
'GOTO' CHOOSEJ 'END';
LB MAY NEED ANOTHER STEPI
'IF' 'NOT' K-O 'THEN' 'GOTO' REENTER LOWER BR;
LOWER BR NON EMPTYI
'IF' 'NOT' LTtM+l~N+ll > MINV 'THEN' 'BEGIN'
'COMMENT'
SUBOPTIMAL SOLUTION~ TREAT AS EMPTYJ
REENTRY I - -IJ
'GOTO' CHECK ON EMPTYNESS OF LB; 'END';
'IF' RECORDt2.NODEl • 2
'THEN' 'GOTO' FIND WHETHER LB INTEGER;
PUT EQUATION CODE IN LBI
KI-OJ
'FOR' Jl a l 'STEP' 1 'UNTIL' N 'DO'
, IF' LC01.Lt Jl-NEXTJ+ 10000 'THEN' 'BEGIN'
K.-JJ l.COLLtJlao'1000+NODEJ 'END'J

'IF' K-O 'THEN' 'BEGIN'

NEWLlNEC 1 H
WRITETEXTC'('EQUATIONIFORINEWILOWERIBRANCHI
ONIX')')J PRINT(NEXTJ~5~0)J
WRITETEXTC' ('IIINOTIFOUND')')J 'END'J

'FOR' JI=1 'STEP' 1 'UNTIL' K-l~

K+l 'STEP' 1 'UNTIL' N 'DO'
'IF' LC01.LtJl>999+NODE 'AND' LCOLLtJl<1001+NODE+M
'THEN' LCOLLtJll-LCOLLtJl+IJ
'FOR' 11-1 'STEP' 1 'UNTIL' M 'DO'
'IF' LROWLtIl>999+NODE 'THEN' LROWLtlllcLROWLtIl+l;
STAGE 2B LOW.
FIND WHETHER LB INTEGER.
'FOR' U-NAV+l 'STEP' 1 'UNTIL' M 'DO'
'IF' LROWLtIl < NAV+NIRV+l 'THEN' 'BEGIN'
NUM ,- LTtI~N+ll-ENTIER(LTCI~N+ll)J
696 CHAPTER XX

'IF' NUM:>O.999
'THEN' LTCI.N+1ll=ENTIER(LTCI.N+1l+0.002);
'IF' NUM<O,OOl 'THEN' LTCI.N+1ll=ENTIER(LTCI.N+ll)i
NUM I- LTCI.N+1l-ENTIER(LTCI.N+ll);
'IF' NUM :> 0.00001
'THEN' 'GOTD' CHOOSE; 'END';
RECORD LB OUTCOMEI
MINV I- LTCM+1.N+1l; EXNODE I- NODE;
'FOR' JI=l 'STEP' 'UNTIL' N+2 'DO'
'FOR' 11-1 'STEP' 'UNTIL' M+3 'DO' EXTCI.Jll=LTtI.Jl
'FOR' 11=1 'STEP' 'UNTIL' M 'DO' EXROWLCIlI=LROWLtIl
'FOR' JI-1 'STEP' 'UNTIL' N 'DO' EXCOLLtJll=LCOLLtJJ
'FOR' J.Dl 'STEP' 'UNTIL' NODE 'DO'
RECORDC3.Jl I- RECORDCO.JJ;
RECORDC2.NODEll=3J

STAGE 3:
CHOOSEI
'IF' (RECORDCl.NODEJ=3 'AND' RECORDt2.NODEJ=3)
'THEN' 'BEGIN'
'COMMENT' HEAD PROBLEM EXHAUSTED;
'GOTO' END DF BRANCHING; 'END';
'IF' RECORDC2.NODEl = 0
'THEN' 'GOTO' COpy HEAD PR TO LOWER BR;
'IF' RECORDtl.NODEJ=3 'THEN' 'BEGIN'
'COMMENT'
CHOOSE THE LOWER BRANCH BY DEFAULT;
'GOTO' BRANCH FURTHER IN LB; 'END';
'IF' HTtM+l.N+lJ ,. LTCM+l.N+ll 'OR' RECORDt2.NODEl=3
, THEN' , BEG IN'
'COMMENT'
CHOOSE THE HIGHER BRANCH;
'GOTO' BRANCH FURTHER IN HB; 'END';
'GOTO' BRANCH FURTHER IN LB;

STAGE II HIGHI
BRANCH FURTHER IN HB:
RECORDC1.NODE11-2;
'IF' NODE" NIRV 'THEN' 'GOTO' RECORD HB OUTCOME;
BRANCH(HT.M.N.NEQ.NAV.NIRV.MINV.HROWL.HCOLL.
EXT.IROWL.EXROWL.EXCOLL.NODE+l.EXNODE.RECORD);
STAGE 4A HIGH:
PUSH UPWARDS:
RECORDC 1 • NODE 1 :.. 2;
K: =OJ
'FOR' J:Dl 'STEP' 1 'UNTIL' N 'DO'
'IF' HCOLL(Jl=1000+NODE 'THEN' KI=JJ
BRANCHING METHODS 697

'IF' K-O 'THEN' 'BEGIN'

NEW1.INEC 1 )J
WRITETEXTC'C'%INTEGER%REQUIREMENT%ON%X')');
PRINTCNEXTJ.5.0);
WRITETEXTC'C'%%%IN%THE%HIGHER%BRANCHIII
NOTIFOUND' ) '); 'END';

'FOR' II-I 'STEP' 1 'UNTI1.' M+1 'DO' 'BEGIN'

HTCI.N+1J:~HTCI.N+1J-HTCI.KJ;
HTCI.N+2J:-HTCI.N+2J+HTCI.KJ; 'END';

HTCM+3.NEXTJJ:-HTCM+3.NEXTJJ+1;
NUM := HTCM+3.NEXTJJ-ENTIERCHTCM+3.NEXTJJ);
'IF' NUM>O.999
'THEN' HTCM+3.NEXTJJ:-ENTIERCHTCM+3.NEXTJJ+0.002)J
'IF' NUM<O .001 'THEN' HTCM+3.NEXTJJ :"ENTIERCHTCM+3.NEXTJJ)J
'GOTO' REENTER HIGHER BR;

STAGE II 1.0W:
BRANCH FURTHER IN 1.B:
RECORDC2.NODEJ:~2;
'IF' NODE - NIRV 'THEN' 'GOTO' RECORD 1.B OUTCOME;
BRANCHC 1. T. M. N. NEQ. NAV. N.I RV. MI NV • 1.ROW1.. 1.C01.1..
EXT.IROW1..EXROW1..EXC01.1..NODE+1.EXNODE.RECORD);

STAGE IIA 1.0W:

PUSH DOWNWARDS:
RECORDC2.NODEJ:-2J
K:-OJ
'FOR' J:=l 'STEP' 1 'UNTI1.' N 'DO'
'IF' 1.C01.1.CJJ~1000+NODE 'THEN' K:-J;

, IF' K- 0 ' THEN' , BEG IN'

NEW1.INEC 1 );
WRITETEXTC'C'IINTEGERIREQUIREMENTION%X')')}
PRINTCNEXTJ.5.0)J
WRITETEXTC'C'%IIINITHE%1.0WERIBRANCHI%%
NOTIFOUND')'); 'END';

'FOR' II-I 'STEP' 1 'UNTI1.' M+1 'DO' 'BEGIN'

1.TCI.N+1J:-1.TCI.N+1J-1.TCI.KJ;
1.TC I.N+2J:-1.TC I.N+2J+1.TC I.KJJ 'END';

1.TCM+2.KJ:-1.TCM+2.KJ-1;
'GOTO' REENTER 1.0WER BRJ

END OF BRANCHING:
END OF BRANCHING PROCEDURE:

'END'; 'END';
698 CHAPTER XX

TEXT-LISTING OF THE MAIN PROGRAMME. WHICH SOLVES THE CONTINUOUS

HEADPROBLEM. AND SETS THE CALL TO THE BRANCHING PROCEDURE.
'BEGIN' 'INTEGER' M.N.NAV.NEQ.REENTRY.
NIRV.EXNODE.I.J.JJ.R.K.ROWN.COLNl
'REAL' MINVI
'PROCEDURE' BRANCHCT.M.N.NEQ.NAV.NIRV.MINV.ROWL.COLL.
EXT. IROWL. EXROWL. EXCOLL.NODE.EXNODE. RECORD)}
'ARRAY' T. EXTJ
'VALUE' NODEI
'INTEGER' M.N.NEQ.NAV.NIRV.NODE.EXNODEJ
'INTEGER' 'ARRAY' ROWL.COLL.
IROWL.EXROWL.EXCOLL.RECORD;
'REAL' MINV;
'ALGOL';
'PROCEDURE' INTPCT.M.N.NEQ.NAV.ROWLST.COLLST.
IROWLST.R.K.ROWN.COLN.REENTRY);
'ARRAY' TJ 'INTEGER' M.N.NEQ.NAV.R.K.ROWN.COLN.REENTRY;
'INTEGER' 'ARRAY' ROWLST.COLLST.IROWLST;
'ALGOL'}
'PROCEDURE' REPOCT.M.N.NEQ.NAV.ROWL.COLL);
'ARRAY' T; 'INTEGER' M.N.NEQ.NAV;
'INTEGER' 'ARRAY' ROWL.COLL;
'ALGOL';
'PROCEDURE' IREPCTRIVT.T.M.N.NEQ.NAV.ROWL.COLL);
'ARRAY' TRIVT.T; 'INTEGER' M.N.NEQ.NAV;
'INTEGER' 'ARRAY' ROWL.COLL;
'ALGOL' ;
'PROCEDURE' MATICMATR.MB.NB.FR.FC);
'ARRAY' MATR; 'INTEGER' MB.NB.FR.FC; 'ALGOL' ;
'PROCEDURE' TABOCMATR.M.N.SR.SC.RH.ER.ROWLST.COLLST);
'ARRAY' MATR; 'INTEGER' M.N.SR.SC.RH.ER;
'INTEGER' 'ARRAY' ROWLST. COLLS!> 'ALGOL';
'COMMENT'
MIXED INTEGER PROGRAMMING. ADMITTING ANY INTEGER VALUES.
J3Y BRANCHING.
PRESENTATION OF DATA:
FIRST THE NUMBER OF RESTRICTIONS I.E. M.
THEN THE NUMBER OF VARIABLES. I.E. N.
FOLLOWED BY THE NUMBER OF EQUATIONS. NEQ.
FOLLOWED BY NAV. THE NUMBER OF VARIABLES
TO WHICH THE TACIT CNON-NEGATIVITY) RESTRICTION DOES NOT
APPLY.
THEN AS LAST INTEGER PARAMETER. NIRV. THE NUMBER OF
INTEGER RESTRICTED VARIABLES,
BRANCHING METHODS 699

THEREAFTER PUNCH EACH ROW OF THE COMPOSITE MATRIX

A B
'01 0
U 0
L 0

TO REPRESENT A.X < OR .. B.

MAXIM -W.X
X < OR .. U
AND X ,. OR - L

THE PROGRAMME READS ALL THE ELEMENTS OF THE UPPER BOUNDS VECTOR
U. AND iHE LOWER BOUNDS VECTOR L.
DESPITE THE FACT THAT FOR VARIABLES WITHOUT NON-NEGATIVITY
RESTRICTION. THE MAIN PROCEDURE USES THESE NUMBERS ONLY FOR
BOUNDED VARIABLES.

MINV I - -1 000 000 000)

H:-READ; N:=READ; NEQ:-READ; NAVI-READJ NIRV:=READJ
'BEGIN'
'ARRAY' TAC1IM+4.1:N+2J.EXTC1:M+3.1IN+2JJ
'INTEGER' 'ARRAY' ROWL.EXROWL.IROWl.C 1IHJ.COl.L.EXCOl.LC 1INJ.
RECORDCOI3.1INIRVJ;

PREFIl.l. T WITH tEROS:

'FOR' JI'"l 'STEP' 1 'UNTIL' N+2 'DO'
'FOR' 11-1 'STEP' 1 'UNTIL' M+3 'DO' TACI.JJI=OJ

MATICTA.M+3.N+1.0.0»)
'FOR' JI=l 'STEP' 1 'UNTIl.' N 'DO'
'FOR' 1:'"1 'STEP' 1 'UNTIL' H+1 'DO'
TACI.N+1J 1= TACI.N+1J-TACM+3.JJ;
'FOR' J:=l 'STEP' 1 'UNTIl.' N 'DO'
TACH+2.JJ 1= TACM+2.Jl-TACM+3.Jl;
'FOR' J:=l 'STEP' 1 'UNTIL' N 'DO'
TACM+4.Jl 1= TACM+2.Jl;

, FOR' I 1 = 1 'S TEP' 'UNTIl.' M+3 'DO' TACI.N+2ll=1000000;

, FOR' 11 = 1 ' STEP' 'UNTIL' M 'DO' IROWLC Ill=};

REENTRYI-O;

REENTER:

RI-O; KlsO; ROWN:=O; COLN:=O;

INTPCTA.M.N.NEQ.NAV.ROWL.COl.l..IROWl..R.K.ROWN.
COl.N.REENTRY);
700 CHAPTER XX

CHECK ON EMPTYNESS:
'IF' REENTRY" -I 'THEN' 'BEGIN'
'COMMENT'
EMPTY PROBLEM;
NEWLINEC 1 );
WRITETEXTC' C'MAINIPROBLEMIEMPTY.IEVENIWITHOUTI
INTEGERIRESTRICTIONS')');
'GOTO' REPORT ON SOLUTION; 'END'J

'IF' REENTRY" 1 'THEN' 'BEGIN'

'COMMENT'
UNBOUNDED PROBLEM;
NEWLINEC 1 );
WRITETEXTC'C'UNBOUNDEDIPROBLEM')')J
'GOTO' REPORT ON SOLUTIONJ 'END';
MAY NEED ANOTHER STEP:
'IF' 'NOT' K=O 'THEN' 'BEGIN'
REENTRY I" I; 'GOTO' REENTERJ 'END';

NON EMPTY:
BRANCH NOWI
REENTRY I- 0;
BRANCHCTA.M.N.NEQ.NAV.NIRV.MINV.ROWL.COLL.
EXT.IROWL.EXROWL.EXCOLL.I.EXNODE.RECORD);

OUT:

REENTRYI-I;
'IF' MINV > -999 000 000 'THEN' REENTRY :- 0;
'IF' REENTRY=O 'THEN' 'GOTO' CONVERT OPTIMUM;
NEWLINEC 1 )J
WRITETEXTC'C'PROBLEMIEMPTYIOFIINTEGERISOLUTIONS')');
NEWLINEC 1 )J
'IF' MINV > -990 000 000 'THEN' 'GOTO' CONVERT OPTIMUM;
'IF' N < 14 'OR' M+N < 40
'THEN' TABOCTA.M.N.O.O.I.I.ROWL.COLL)
'ELSE' 'BEG IN'
TABOcTA.M.O.O.N.O.I.ROWL.COLL);
TABOCTA.O. N.M.O.I.O.ROWL.COLLH 'END';
'GOTO' REPORT ON SOLUTION;

CONVERT OPTIMUM 1

'FOR' JI-I 'STEP' 1 'UNTIL' N 'DO'

'IF' EXCOLLeJJ>IOOO 'AND' EXCOLLeJJ<IOOI+EXNODE
'THEN' 'BEGIN'
JJ I" RECORDe3.EXCOLL(JJ-IOOO);
EXCOLL(JJ 1= JJJ
'IF' EXT(M+3.JJJ=TA(M+3.JJJ
'THEN' EXCOLLC JJ 1 =EXCOLL( JJ+ 10000; 'END';
BRANCHING METHODS 701

'FOR' Js-1 'STEP' 1 'UNTIL' N ,~O'

'IF' EXCOLLCJl>1000+EXNOOE 'AND' EXCOLLCJl<10000
'THEN' EXCOLLCJls-EXCOLLCJl-1;
'FOR' 11-1 'STEP' 1 'UNTIL' M 'DO'
'IF' EXROWLCIl > 1000 'THEN' EXROWLClll-EXROWLCIl-EXNOOEJ
REPORT ON SOLUTIONs
'IF' ABSCREENTRY)-l 'THEN' REPOCTA.M.N.NEQ.NAV.ROWL.COLL)
'ELSE' IREPcTA.EXT.M.N.NEQ.NAV.EXROWL.EXCOLL);

END OF PROGRAMMEI
'END'; 'END'
CHAPTER XXI

THE USE OF CUTS*

21.1 Elementary cuts, augmented cuts and combined cuts

The solution of integer programming problems by means of cuts

has been pioneered by R. Gomory [.15], (16] , and has been
surveyed more recently at textbook level by Zionts [411 . The
treatment of the material in this and the following sections
differs from the work of these authors on the following points:
Firstly, the interpretation of a combined cut as a cut on a
combination of variables is believed to make the logic of
"deeper" cuts more easy to appreciate than the algebraic
approach followed in these source-texts. Some other points are
of a more substantial nature. They concern the anticipation of
fractional value of integer-restricted non basic variables, the
permanent incorporation of upper limits into the problem itself,
and formulation of cuts before the continuous problem has been
solved.

A cut is an additional restriction, obtained on the basis of

information contained in an updated tableau, which excludes an
area in which no integer solution exists, from the feasible
space area.

We will discuss the procedure of cutting, at this stage on the

assumption that the continuous problem is solved first. If a
solution is fractional, one "cuts" successive slices off from
the feasible space area, until the integer optimum is found
in a corner of the re-defined feasible space area. To understand
why we can make cuts it is useful to interpret a vertex, e.g.
the continuous optimum as a linear programming problem in its
own right.

The relevant coordinate directions are the variables, which are

associated with the columns of an updated tableau e.g. the
continuous optimum. (See also section 8.11). Slacks of equations
are not considered as variables in this context. They might as
well be removed from the tableau, as they do not figure in
the integer programming problem at all.

*A number of example-tableaux in this chapter are typed rather

than computer-file listed. It was felt that exact fractions
e.g. 1/11 instead of 0.09 are more useful in illustrating cuts
than the two-digit rounded figures produced by the tableau-
printing procedure.
702
mE USE OF CUTS 703

We may distinguish several types of cuts.

An elementary cut is a restriction which excludes from the

feasible space area a sub-area, the corners of which are the
origin, and in each coordinate direction a point where a
particular variable attains the next nearest integer value.

Example:

Consider the following integer programming problem:

Maximise T = 3 xl + x2

Subject to 8 xl + 6 x 2 < 15
<
2 xl 4 x2 -1

(0 < xl :5.- 100, a :5.- x 2 < 100, xl' x 2 integer restricted)

This small example will be used in this and in some of the

succeeding sections to illustrate the various types of cuts.

The "fancyhigh" upper limits have been set at a figure of 100

rather than the million used so far. This is because of the
need to contain rounding errors not only relative to the
rounded numbers, but also the absolute magnitude of the rounding
errors themselves.

The effectiveness of an integer programming algorithm which uses

cuts and does calculations in floating point numbers, is
critically dependent on the precision of its calculations. If
the correct value of some integer-restricted variable at a
certain point of calculation is Xk = 1600, but because of
rounding it is reported as 1599.983 we should either set our
tolerance for accepting a number as integer at a difference in
excess of 0.01 or we would risk to miss the correct solution
altogether, as the next cut might well require x k :5.- 1599.
To set the tolerance at a fairly narrow margin, and then not
dealing with excessively high values of variables at all, is
simpler in terms of programming, than to have a special loop
to calculate the tolerance.

The continuous optimum of the demonstration problem is

xl = 1 5/22 and x 2 19/22.

We can map the non-negativity of xl and x 2 ' as well as some

704 CHAPTER XXI

TABLEAU 21.1 A

CONTINUOUS OPTIMUM OF THE EXAMPLE

NAME !! S I S 2 VALUE
-------------------------------------
X I ! ! 0.09 o .I 4 ! :: ! I .23
X 2 !! o .05 - 0.16 l :: ! 0.86
-------------------------------------
T ! I 0.32 o .23 ! Ij .5<;

DITTO, NON-DECIMAL PRESENTATION.

NAME! ! S I S 2 VALUE
--------------------------------------
X I ! ! III I 3/22 ! :: ! I 5/22
X 2 !! 1122 -aliI! ::! 19/22
--------------------------------------
T !! 7/22 5/22 ! ! 4 6/11

graph 21.1 a

illustration 0 f
Integer-va lue cuts.

-2
THE USE OF CUTS 705

other lines e.g. Xl = 1 and x 2 = 2 in the sl' s2 coordinate

plane. This has been done in graph 2l.la. Tne fancyhigh
limits which are not operative in this example, have been left
out of the illustrations.

The equations of these lines in the sl' s2 coordinate place are

0: 1/11 sl + 3/22 s2 1 5/22

1: 1/11 sl + 3/22 s2 5/22

0: 1/22 s - 19/22
1

1: 1/22 s - -3/22
1
2: 1/22 s - -1 3/22
1
(See also tableau 2l.a, as well as graph 2l.a).

hie normally cut on a variable i.e. a cut is designed to

exclude a sub-area in which a particular variable attains only
fractional values. If the row of the associated Simplex tableau,
which describes a variable, contains only positive entries or
zeros the making of a cut is easy.

Thus the Xl-row reads 1/11 sl + 3/22 s2 + Xl = 1 5/22 giving

rise to the restriction 1/11 sl + 3/22 s2 ~ 1 5/22, the non-
negativity of Xl. The slack of this restriction i.e. Xl cannot
exceed the number 1 5/22, hence Xl = 2 (or Xl = 3, 4 eEc.) is
outside the feasible area.

The next integer value to be investigated is therefore x I I ,

and a cut on Xl will require Xl ~ 1.

A somewhat similar situation arises when a row contains only

negative elements, except in the value column. For example Xl
is implicitly assumed to have an upper limit which we
conventionally put, in the absence of a specified upper limit
at 100 . If that upper limit restriction is written explicitly,
it reads: -1/11 sl - 3/22 s2 + b l = 98 17/22 and quite clearly
b l is an integer restricted variable which can be restricted to
b l > 99.

Conversely, if we had found a restriction which reads (for

example) -2.3 sl - 1.5 s2 + Xl = 5 1/3 we would have known
that XI would have to satisfy x'l ~ 6 (and the associated upper
limit Dl ~ 94.)
706 CHAPTER XXI

It is not always possible to write a cut-restriction which

admits for such a simple interpretation. In the example, x
can be either greater than, or smaller than its current vatue
of 19/22.

We may therefore distinguish at this stage two classes of cuts

e.g. limit cuts which impose a specific limit (to be not
greater than or not smaller than a certain integer number) on
some variable, and elementary cuts.

Elementary cuts are cuts which we may write, even if we cannot

associate a specific limit on a variable with the cut.

We can always calculate the coefficients of a cut-restriction

by a method which follows the definition of a cut as given
above. In the demonstration-example we can write an elementary
cut on x 2 • Inspection of the optimum tableau shows that
introduction of sl in the basis reduces x2 by 1/22 per unit of
sl. Therefore in the sl-direction, x 2 will attain an integer
value (zero), for sl = 19/22: 1/22 = 19.

In the s2-direction, x 2 will attain an integer value for

s2 = 3/22: 2/11 = 3/4 . In both cases, we need the absolute
value of the coefficient, but for a negative coefficient the
numerator is the difference with the next integer value of the
variable which is being cut on.

The figure 19 and 3/4 indicate the values which sl and s2 may
attain before any solutions with integer values for x 2 come
in the excluded area. We will therefore indicate them as the
limiting value. More precisely, 19 is the limiting value of
sl with respect to x 2 becoming zero, and 3/4 is the limiting
value of s2 with respect to x 2 becoming unity.

The limiting value of a non-basic variable is the value which

that non-basic variable may attain without a particular basic
variable (which we cut on) to exceed an adjoining integer value.
When the tableau-cell which describes the effect of a non-
basic variable on a variable to be cut on, is positive,
(e.g. 1/22), the limiting value is the quotient of the
fractional part of the current value of the basic variable
cut on, divided by the particular tableau-cell.

L = (x. - entier (x.» /t .. (21. 1. 1a)

J 1 1 1J

If the column for a non-basic variable contains a negative

entry in the appropriate row (e.g. -2/11), the limiting value
is the absolute value of the difference between the current
value of the basic variable being cut on and the next higher
integer, divided by the absolute value of the negative
coefficient.
THE USE OF CUTS 707

t. = (entier (X. + 1) - x.)/(-t .. ) (2l.l.lb)

J 1 1 1J
The area to be excluded, is the area spanned by the origin
(e.g. sl = s2 = 0) and the points on the coordinate axes
indicated by the limiting values (e.g. s2 = 0, sl = 19, in the
sl-direction, and sl = 0, s2 = 3/4 in the s2-direction.)

Any solution vector which is a convex combination of the current

solution and the points on its coordinate axes where the
limiting values occur, will be characterized by values of the
variable cut on, which either are between two integer values
(e.g. x 2 between 1 and zero in the example), or, if only one
integer number is involved, between that oqe integer and the
current value.

The exclusion of this convex combination is made effective

by requiring

n
E - 1ft. v. < - 1 (21.1.2)
j=l J J

where tj is the limiting value of the jth variable, and t. the

jth var1able itself. In the example, the cut on x 2 theref6re
becomes

- 1/19 sl - 1 1/3 s2 ~ - 1

As was already indicated above, a cut according to (21.1.2) with

limiting values according to the definition given above, may
be indicated as an elementary cut. If an elementary cut can be
interpreted as requiring the variable which is cut on to attain
the next lower integer value (e.g. xl < 1) we may speak of an
upper limit cut.

The meaning of the analogous term lowerlimit cut will be obvious.

It should also be borne in mind that we intend to employ a
version of the basic Simplex LP algorithm, in which every
variable has an upper and a lower limit. Limits on integer
restricted variables will obviously be set at an integer value.
Thus upper limit cuts are lower limit cuts on upper limit
distances. If no information to the contrary is stated at the
same time, it will be ~ssumed that limit cuts are on specified
variables.

Rows which refer to specified variables and only contain

positive (negative) entries and zeros, give rise to limit cuts
on these variables. We may, if we so wish, calculate and
present a limit cut in precisely the same way as for an
elementary cut which is not a limit cut.
708 CHAPTER XXI

For example the upper limit cut on xl could be calculated as

follows (see also tableau 2l.la):

The limiting value of sl' relative to xl becoming unity

is 5/22: 1/11 = 212
The limiting value of s2' relative to xl becoming unity
is 5/22: 3/22 = 1 2/3

The reciprocals of 2! and 1 2/3 are 2/5 and 3/5. Therefore the
elementary cut on xl is

- 2/5 sl - 3/5 s2 < -1.

If this restriction is multiplied by 5/22, we obtain

- 1/11 s 1 - 3/22 s2 2.. - 5/22

Addition of this restriction to the xl-restriction itself

1/11 sl + 3/22 s2 + xl = 1 5/22

yields:

There is, however, one essential point of difference between

the two presentations of a limit cut. The slack-variable of

-3/22 s2 < - 5/22

is the difference between the value of xl which we now aim for,

and any possible later value, and this slack is an inte~er­
restricted variable. The slack-variable of

- 2/5 sl - 3/5 s2 2.. - 1 is not integer-restricted. We

shall come back to th1s point at a later stage.

We shall now pay some attention to the effectiveness of

different types of cuts. A cut is more effective than another
c~t if it if it excludes all the fractional solutions which
the other cut excludes, and some more as well. This definition
of effectiveness if relative i.e. we cannot really say how
effective a cut is. We cannot even always say that one cut is
more effective than another. We will however also use the term
in a wider, vaguer sense i.e. we may indicate that we expect
a cut to lead quickly to the solution. Elementary cuts which
are not also limit cuts are not a particular effective device
for solving integer programming problems. They will be used only
if other more effective types of cuts cannot be made. This is
THE USE OF CUTS 709

apparent from the graph where the elementary cut on x 2 does

not touch the integer optimum.

There are several things which can be done to increase the

effectiveness of cuts as a means to quickly obtaining an
integer solution.

The two devices which are the subject of this section are the
augmentation of the limiting value of an integer-restricted
non-basic variable (to unity), and the combined limit cut. The
issue of augmentation is best discussed with reference to the
limiting value of s2 relative to x 2 becoming unity. In the
original LP-tableau, the s2-restriction contains only integer
coefficients, all of which refer to integer-restricted
variables, s2 is itself an integer-restricted variable.

All points in the area excluded by the elementary cut on x 2

conform to the restriction 0 2 s2 2 3/4.

If we push the limiting value of s2 outwards to become s2 = 1

this may bring some points with inEeger values for x in the
excluded area. But that will all be points with fractional
values of s2' hence those points will not include the integer
optimum, where s2 will be integer.

Whenever a limiting value of an integer restricted non-basic

variable is found to be less than unity, we will substitute
unity for it.

Instead of the elementary cut on x 2 we should have written - if

nothing more effective appeared to be possible -

A cut of this type, which is based on the augumentation of

limiting values of integer restricted variables to unity, will
be indicated as an augmented cu~. Augmentation alone will not
remedy the basic weakness of an elementary cut which is not
also a limit cut, neither in general, nor in the particular
example. (The true solution is sl = s2 = 1 and this is still
inside the amply fulfilled cut -1/19 sl - s2 2 - 1). If we
cannot make a limit cut on a specified variable, the next
best possibility is a limit cut on an integer combination of
integer restricted variables. In an all integer problem, which
the example is, this is always possible, but in the general
mixed integer case this may, or may not be possible.
710 CHAPTER XXI

To relate the appropriate combination to a specified variable

we will use an asterisk. Thus, x*2 is the auxiliary variable
developed in combination with x2' There are several possible
combinations in the example, which would enable us to make a
limit cut on a combination of integer restricted variables. For
example, we might define an auxiliary variable

x*2 = x 2 + 2 xl

and the x*2 - row of the tableau would be found by adding twice
the x]-row to the x2-row. The significance of the auxiliary
variaole row would in that case be

5/22 sl + 1/11 s2 + x*2 = 3 7/22

From the definition of x*2 as x 2 + 2 xl i.e. an integer

combination of integer restricted variables it follows that x*2
is also an integer restricted variable. And we can write an
upper limit cut, the effective significance of which is

x*2 .2. 3.

For program-technical reasons we shall restrict the use of this

device to combination of a specified variable (or its upper
limit distance) with non-basic variables (or their upper limit
distances).

To make an effective use of the combination-device in connection

with basic variables we would have to perform a systematic
search for the appropriate combination. For non-basic variables
the equivalent of the updated row is a (negative) unit vector
i.e. the non-negativity of sl would be written explicitly as
- s .2. O. This effectively means that we can, for the purpose
of ~eveloping a cut, add or subtract integer numbers to or
from the coefficients which refer to integer-restricted non-
basic variables. This includes integer-restricted slacks.
(See also section 21.4 for the treatment of upper limits on
slack-variables. Thus, from the x 2-row

1/22 sl - 2/11 s2 + x 2 = 19/22

we infer

-21/22 sl - 2/11 s2 + x*2 = 19/22 (x*

2
and we may make a lower limit cut on x*2 (x*2 > 1).

We may also develop auxiliary variables which are integer-

restricted without being sign-restricted. Consider, for example,
the difference between x 2 and s2' We subtract the unwritten
THE USE OF CUTS 711

s2-row from the x 2 -row, i.e. we add unity to the s2- coefficient
(the s2-row reads - s2 ~ 0).

The auxiliary variable equation then is

1/22 sl + 9/11 s2 + x*2 = 19/22 (x*2 = x2 - s2)

and we can write an upper limit cut on x*2 ~ o. We will see

later in this chapter that a systematic use of upper limits
allows us to interpret even such rows as being associated with
sign-restricted variables. Cuts of this type, i.e. cuts on a
specified basic variable and an integer combination of integer
restricted nort~basic variables will be indicated as combined
cuts.

In some cases a combined cut may be useful for a different

reason than solely for being able to make a limit cut, or even
in a situation where it would not be needed for that reason.

For example, a restriction (the~l~row) might read:

1 1/10 sl + 1/5 s2 + x3 = 1 2js.

We can in that case immediat~ly formulate an upper limit cut

on x 3 . Assuming that the slack of the sl-restriction is integer-
restricted, the combined restriction

gives rise to a more effective cut, as may be seen by comparing

the two cut-restrictions.
\~
We perform this comparison, for the mfrment with both cuts in
the elementary form

-2 3/4 sl - -12 s2 < - 1

or

- 3/4 sl - 12 s2 < - 1

The point is simple that any set of figures which will satisfy
the cut on x3 itself, also satisifes the cut on x*3 = Xl + sl'
but not the other way round. The systematic fact is that
the limiting value of sl with respect to x*3 is greater than
the similar figure wi~h respect to x 3 . Reduction of the
coefficient for sl in the original restriction from 1 1/10
increases the limiting value b'y the inverse ratio from 4/11 to
4. Although the limiting value can be augmented to 1.00 the
712 CHAPTER XXI

augmented cut

is still less effective than the combined cut.

The choice of the most appropriate combination is to some

extent influenced by the form in which cut-restrictions are
presented. Cuts should if possible (which, in an all-integer
prob~emmeans always) be written in a form in which the slack-
variable of a cut-restriction is itself integer-restricted.
Thus, in the example above, we would develop an x*3-cut

- 1/10 sl - 1/5 s2 ~ - 2/5 (x* 2 - cut)

The restriction is equivalent with

(x** - cut)
2
except for the integer form. i.e. the integer requirement
on the slack-variable.

Throughout the rest of this chapter, cuts with integer-

restricted slacks will be indicated with single asterisks, cuts
with continuous slacks by letters with double asterisks.

There is in general a variety of combinations .to be considered.

In practice we can sort out the most effective combination
implicitly by means of the following rule concerning the
limiting values of integer restricted non-basic variables.

The limiting value of the x. non-basic variable, relative to

the x. basic variable or anJassociated combination integer,
is th~ highest of the following numbers:

a) The fractional part of the basic variable xi divided by the

fractional part of the tableau element (coefficient) c .. if c ..
is positive, (xi = 2 2/3, c .. = 3 3/4 limiting value 1J 1J
= 2/3: 3/4 = 8/9), or the ff!ctional part of the basic variable
x., divided by the absolute value of the difference between c ..
aBd the next lower integer number, if c .. is negative 1J
(x. = 2 2/3, c .. - 21 limiting valu~J= 2/3: (3 - 21) =
2/3: 3/4 = 8/9~J

The implied combination contains in these two examples for

a) the terms 3 x. and - 3 x. respectively, leaving in both cases
a fractional ter~ 3/4 x. inJthe auxiliary expression.
J
b) The absolute value of the difference between the value
of the basic variable and the next higher integer, divided by
THE USE OF CUTS 713

the absolute value of the difference between the coefficient

and the next higher integer, if Cij is positive
(xi = 2 2/3, Cij = 3 3/4 limiting value = (3 - 2/3):
(4 - 3 3/4) = 173: ! = 1 1/3), or the absolute value of the
difference between the value of the basic variable and the next
higher integer, divided by the fractional part of the absolute
value of the coefficient, if the coefficient is negative.

(x.
~
= 2 2/3, c .. = -
~J
2! limiting value = (3 - 2 2/3):

= 1/3: ! = 1 1/3).

The combination of variables in the implied auxiliary variable

contains for the two examples under b) the terms 4 x. and
- 2 Xj' leaving in both examples a fractional term _J! x. in
the auxiliary expression. J

c) The number 1.00 = unity, if no higher value has been found

under a) or b).

The generalization of the above stated rules to non-basic

slack variables with integer requirement (including slacks
of limit-cuts) will be obvious. The operative significance
of these rules is here illustrated with the following:

Example:

Maximise T 3 xl + x2 x3
8 xl + 6 x 2 + 3x 3 < 15

2 x - 4 x2 + x3 < - 1
1
(0 < x < 100 , 0 < x2 < 100 , 0 < x3 < 100 ) x 2 ' x3 integer
restri~ted). -

This is a modification of our earlier demonstration example.

We have dropped the integer requirement on xl' and by
implication also on sl and s2' and added the third variable x 3 ·

The corresponding continuous optimum is given below

Name sl s2 x3 Value
xl 1/11 3/22 9/22 < 1 ~/22

x2 1/22 -2/11 -1/22 < 19/22

T 7/22 5/22 2 2/11 4 6/11

714 CHAPTER XXI

Becaus: Xl is now a continuous variable, sl and s2 also loose

their lnteger restricted nature.

There are now two cuts in association with x 2 .

One is the elementary cut on x 2 itself:

- 1/19 sl - 1 1/3 s2 - 1/3 x3 < - 1

The other cut is on a combination

1/22 sl - 2/11 s2 + 21/22 x3 + x*2 19/22 (x*

2
and gives in first instance rise to

-1/19 sl - 1 1/3 s2 - 1 2/19 x3 < - 1

This combination does not look very effective at first sight

since 1/22, the coefficient in the x 2 restriction itself is a
rather small number, but we cannot be sure on this point without
further investigation. However the coefficient for x3 is based
on a limiting value of 19/22: 21/22 = 19/21, which snould be
augmented, and the second cut becomes

- 1/19 sl - 1.1/3 s2 - x3 ~ - 1

The cut on x 2 it~elf is, however found to be the most effective,

hence the cut

- 1/19 sl - 1.1/3 s2 - 1/3 x3 < - 1

is eventually put in the tableau.

We are however, not only interested in the effectiveness of

individual cuts, but also in formulating cuts with integer,
restricted slacks. rhis implies that we wish to form com-
binations on which a limit cut can be written, and certain
restrictions on the choice of combinations arise from that
desideratum.

21.2 Classes of cuts

Because there are different types of cuts to be made, we must

state some priority rules as to which cuts to make in a
particular tableau. It would be a waste of tableau-space and
of calculations if all possible cuts were made.

At this point in our discussion, we state three main priority-

classes of cuts or rather of auxiliary variables.
THE USE OF CUTS 71S

* Specified variables which permit a limit cut to be written

on the specified variable itself.

As was explained in the previous section, limit cuts are more

likely to be effective cuts than other types of cuts. We
recognize a variable which permits a limit cut by the fact that
the non-zero entries in its row (other than in columns which
refer to equations slacks), are either all positive (upper limit
cut) or all negative except the entry in the value column (lower
limit cut). Therefore we will cut on or in association with a
variable on which a limit cut can be made, in preference to a
variable on which we cannot make a limit cut. Note that this
does not always imply that we will write the limit cut.
Even for a variable which permits a limit cut, a combination
may lead to a more effective cut. For example, suppose (for
the sake of argument) that the xl-row reads:

< Value

1/11 1 1/22

Like the row which is actually there, these figures allow an

uppe r limi t cut.

Upper limit cuts on specified variables can in fact be made,

simply by entering the appropriate negative entry in the upper
limit distances column, i.e.

sl s2 <
-
Value Distance

xl 1/11 ! 1 1/22 - 1/22

If s2 is integer-restricted, we cannot only augment its

limiting value from 1/11 to 1, we can also write a combined
cut on x*l = xl + s2'

The associated auxiliary variable restriction is

1/11 sl - ! s2 + x*l = 1 1/22

and the limiting value relative to x*l becoming 2 is

21/22 : ! = 1 10/11.

In this case we will not use the combination, because it would

lead to loss of integer form (and require an explicit restriction,
716 CHAPTER XXI

rather than simply changing the upper limit). ,But even if we

were to use the combination, the x*l cut belongs to the top
priority class. This does, incidentally imply that it would be
written as a negative upper limit distance rather than as an
explicit restriction.

*Specified variables which permit a limit cut to be written

on an auxiliary variable associated with them.

When no limit cut on a specified variable itself can be written,

a limit cut on an auxiliary variable has priority over a cut
which is not a limit cut at all.

We recognize a variable of this "priority two" class by the

lack of sign homogeneity between the coefficients in its row,
in so far as these coefficients refer to integer restricted
non basic variables, while there is sign-homogeneity between
the coefficients which refer to continuous variable.

*Specified variables which do not permit a limit cut of any

type to be written on or in association with them.

We recognize a variable of this type by the fact that its

associated row contains, among the coefficients which refer
to continuous non-basic variables, positive non-zero as well
as negative non-zero numbers.

A cut on a variable of this class is made only when no cut

or in combination with a variable of one of the two higher
priority classes can be made.

There is a fourth, non-priority class of variables. It is not

recommended to write cuts on slack variables, even where some
slack variables are integer-restricted. This is because
integer values for all integer-restricted slack-variable is a
necessary rather than a sufficient condition for all integer
solution, whereas integer values for all specified variables
is sufficient.

In the continuous optimum of the demonstration example, the

two slack-variables are in fact integer valued (zero), but
xl and x 2 'are both fractiona1. If one were to cut on a slack
variable one might find that an integer value of the slack-
variable is duly attained, while none of the specified variables
attain an integer value.

Within each class of variables, we still have to decide which

variable to cut on. The algorithm which we shall actually
implement, uses for various priority-classes, three methods
of resolving this choice, viz:
THE USE OF CUTS 717

a) To cut on the variable with the lowest index.

b) To cut, space-reservation in the tableau permitting, on all
variables which come in a particular priority-class, and
c) To sort out - for a particular incoming variable - the basic
variable for which a cut is likely to lead to the highest
limiting value.

21.3 The subsidiary cut

A major problem in integer programming is that a cut on one

integer-restricted variable may lead to another integer-
restricted variable, which so far had an integer value to
become fractionally valued.

This is most obvious in the case of an integer-restricted non-

basic variable. Zero is an integer number, and if an integer-
restricted variable enters the basis at the fractional value
an integer-restricted variable becomes fractionally valued. A
related problem is that a single cut generally leads to one
integer-restricted variable becoming integer-valued, we really
want all integer-restricted variables to become integer-valued.

Our general approach to these problems is to anticipate these

undesired possibilities, in particular when low-priority cuts
have been used. When we have been forced to use a non-limit
cut, we scan adjoining vertices, without fully making the
corresponding steps. We perform the search operation for a
pivot, and update the value and upper limit columns, but don't
so far update the full tableau. The updating of the solution-
vector is done outside the main tableau - which is therefore
preserved - and we speak of a hypothetical step. A preliminary
cut which is made on the basis of information obtained by way
~a hypothetical step. That hypothetical step is the one
indicated by a search operation in an attempt to find a
solution which satisfies a cut or some cuts already made
before.

A cut which is made in the main tableau itself, by reference

to the solution-vector which that main tableau describes, is
then indicated as a main cut. Limit cuts, when found will be
made and the corresponding steps will also be actually made.
No subsidiary cuts will therefore arise from limit cuts, and
we will indicate limit cuts also as preliminary cuts. However,
in this example we will treat them as main cuts as otherwise
no subsidiary cuts could be illustrated in a small example.

In ou~ demonstration example, the cut on xl indicates s2 as

pivot column variable, and the slack of the cut x*l as pivot
row. We assume, for the sake of argument, that the x*l - cut
is treated as a main cut, rather than as an initial cut
718 CHAPTER XXI

i.e. we will now consider the next vertex as hypothetical

rather than making a step forthwith.

The search operation for a "hypothetical" pivot could initially

be the same as normal re-entry of Phase I of the L.P. algorithm,
but the following special features of the L.P.-search operation
in an integer programming context become relevant once the
first "hypothetical" step has been indicated:

a) The "efficient" rule of selecting the pivotal row to match

a particular pivotal column is applied (see section 9.2).
For the standard L.P. problem we compromised this rule,
because it results in a bias for small pivots.

b) Once we have established that a particular column contains

a negative entry against a violated restriction, and the
column could give rise to additional subsidiary cuts, we
will re-enter the "Phase I" search operation, on the basis
of that particular column as pivotal column.

The significance of these rules is illustrated below, as

follows:

Hypothetical step Partial update

(column extract)

Name sl s2 < Value x* < Value

1

xl 1/11 3/22 1 5/22 xl 1 1

x2 1/22 -2/11 19/22 x2 -1 1/3 1 1/6

x*
1 -1/11 t-3/2?) -5/22 s2 -7 1/3 1 2/3

T 7/22 5/22 4 6/11 T 1 2/3 4 1/6

The updated column-extract may now serve to indicate a variable

which is fractionally valued in the hypothetical vertex e.g. x 2 .
A subsidiary cut is now developed with the help of a partial
row-update. Just as in the case of main cuts we need only to
cut on specified variables, i.e. only on x 2 .
THE USE OF CUTS 719

Partial update (row-extract) with cut on x 2

Name sl x* < Value

1

x2 1/6 -1 1/3 1 1/6

s2 2/3 (-7 l/;D 1 2/3

x* -1/6 -2/3 -1/6

2

T 1/6 1 2/3 4 1/6

The x*2 - cut in this example ~s a limit-cut on the combination

x*2 = x 2 - 2 x*l'

To make the cut comparable with the main vertex we now make the
backward step, i.e. we re-introduce x*l as a basic variable,
and eliminate the s2-slack. This backward step can be performed
on the row-extract. The operation reverses the initial hypo-
thetical step (implemented on the extracts). Since the extract
was initially updaLed we call this operation unupdating.

Unupdated row-extract with subsidiary cut

Name sl s2 < Value

x2 1/22 -2/11 19/22

x* -1/11 -3/22 -5/22

1
x* -5/22 -1/11 -7/22
2

T 7/22 5/22 4.6/11

The unupdated X*2 cut-restriction is now amalgamated with the

main tableau, which becomes:

Name sl s2 < Value

xl 1/11 3/22 1 5/22

x2 1/22 -2/11 19/22

x* -1/11 -3/22 -5/22

1
x*
2
-5/22 El/l~ -7/22

T 7/22 5/22 4 6/11

720 CHAPTER XXI

When after making a subsidiary cut, we come to scan the

hypothetical incoming variable column, once more in search for
a Phase I pivot, we speak of a secondary reentry column.

At this point it is essential that the search for pivots uses

the "efficient" rule of the largest quotient when selecting
between a number or rows representing violated restrictions,
and not the "conservative" rule of the smallest quotient. Thus
when s2 is again chosen as pivot column, the newly made x*2-cut
becomes pivot row, rather than x*l - row, which would lead to
the ~ subsidiary cut as before.

This requires some adaptation of the standard L.P. algorithm

where the "efficient" rule of row-selection is not always
used.

As in the "Phase I" of the normal L.P. algorithm, a column is

not chosen or even considered as pivot column, unless it
contains negative elements which correspond with negative
entries in the value column.

We therefore know that secondary re-entry columns are always

bounded. Of the eligible minus-minus pairs, we will select
the one which gives the biggest value of the incoming variable.

The re-entry of the pivotal column mayor may not decide Which
subsidiary cut is going to match a particular column-variable.
This will be the case if one of the following circumstances
arises

a) The pivot row is a normal basic variable rather than a

violated cut
b) The next hypothetical vertex is integer-valued

In the first case there is little point in making a further

subsidiary cut in association with that particular column, in
the second case the job has been done, as far as that column
is concerned.

In order to verify that entry of a particular variable into

the list of basic variables avoids (as far as possible)
fractional values of previously integer valued variables we
will re-enter the same column after making a subsidary cut
and if necessary, add a new subsidiary· cut which gives rise to
a higher limiting value. The one exception to this rule arises
in the case of a zero in the extract.

If a column-extract contains a zero in the pivotal column, no

subsidiary cut on the variable associated with the corresponding
row is developed, even it if is a fractionally valued integer-
restricted variable.
THE USE OF CUTS 721

Theorem

When we re-enter a column to search for a Phase I pivot after

writing a subsidiary cut in association with a hypothetical
step (the column being indicated by the previous entry of the
same column), the pivotal row is either not a cut, or the last-
written subsidiary cut is the restriction which gives rise to
the largest value of the variable associated with the column,
and hence is indicated as pivotal row.

Example in lieu of a proof:

The sign-arrangement +

~s systematic for the "unupdating" step. e.g.

x* Value
1

s2 -7 1/3 1 2/3

x* -2/3 -1/6
2

The number -7 1/3 in the s2' x*l cell is negative because it

is reciprocal of a negative pivot (the -3/22). The number
1 2/3 is positive because it is the value of s2 and the
entries of - 2/3 and - 1/6 are negative because all entries
in a cut-restriction row are negative. In the tableau containing
the "main" cut, the slack of the new subsidiary cut is
necessarily more negative than in the tableau-extract which
was used to formulate it.

Column updating in the backward step, i.e. division by minus the

pivot also maintains the negative element in the cut-restriction/
pivot column cell, and the sign arrangement

+ always results in

The -1/11 in the x*?' s2 cell means that there is a critical

ratio for which the-restriction becomes binding.

By definition (the definition of a cut as excluding the current

solution), a cut restriction is not satisfied at the
(hypothetical) vertex for which the cut was formulated. By
assumption, i.e. the "efficient" rule of row selection, all
722 CHAPTER XXI

other cuts (e.g. the main cut and any other subsidiary cuts)
are satisfied at a hypothetical vertex. The one exception to
this rule other than elimination of a normal variable i.e. not
reaching a cut, relates to zero coefficients. In either case,
we will refrain from formulating a subsidiary cut.

Therefore the last formulated subsidiary cut is the most

binding restriction. (Thus, in the example, the x*2 - cut with
a critical ratio of 7 is more binding on s2' than the previously
formulated x*l - cut, and this is because the x'*2 - cut was
formulated at a vertex where the x*l - cut was satisfied. In
the interest of restricting the total number of cuts we will
therefore discard a subsidiary cut, whenever the immediately
following re-entry of the same column gives rise to a new
subsidiary cut.

We now re-enter the "Phase I" search operation for a hypothetical

pivot, but restrict ourselves in first instance to the s2 -
column.

The purpose of such a "restricted re-entry" is to see whether

we can perhaps replace the last-formulated subsidiary cut by
one which is associated with a higher limiting value for the
s - variable. In order to limit the number of re-entries of
tte same column, we will resort to an unrestricted search for
a hypothetical pivot, whenever the value of the incoming
variable in the previous hypothetical vertex is at least equal
to 10.

This qualification of the column-restriction is not present and

we proceed as before.

Column-extract Updated column-extract giving

hypothetical vertex

Name < Value Name x* < Value

2

3/22 1.5/22 3/4

-2/11 19/22 -2

x*
1
-3/22 -5/22 x*
1 -H
x* -7/22 -11
2

T 5/22 4 6/11 212 3 3/4

THE USE OF CUTS 723

We have indeed reached the feasible space in one step. If this

were not the case we would develop no further subsidiary cuts
in association with this column. As it is the hypothetical
vertex indicates a cut on xl' and again we use a row-extract
to implement that cut.

Row-extract with pivotal row

and xl-row (to be cut on)

Name sl s2 < Value Name sl x* 2 < Value

xl 1/11 3/22 1 5/22 xl

_1
4
q 3/4

x*2 -5/22 (-l/ly 1 7/22 82 2~ -11 3!

x* -3/4 _1 -3/4
T 7/22 5/22 4 6/11 1 2

_1 3 3/4
T 4 2~

Updated row-extract with cut

Name sl s2 < Value

Xl 1/11 3/22 1. 5/22

x* -5/22 -1/11 -7/22

2
x* -7/11 -1/22 -10/11
1

T 7/22 5/22 4 6/11

When, as in this case, we have found a higher limiting value

for a particular variable by cutting after "restricted" re-entry,
we overwrite the previous subsidiary cut. The previously developed
subsidiary cut in association with a hypothetical pivot in the
s2-column was (see tableaux on previous page) the x~ cut, we
tfierefore are left with two xt cuts, the main cut and a
subsidiary cut.

The successor tableaux are:

724 CHAPTER XXI

Main tableau with hypothetical Corresponding updated co1umn-

pivot extract

Name < Value Name < Value

xII/II (3/22) 1 5/22 7 1/3 9

x2 1/22 -2/11 19/22 1 1/3 212

x* -1/11 -3/22 -5/22 x* 1 1

1 1
x* -7/11 -1/22 -10/11 x* 1/3 -1
2
1 1

T 7/22 5/22 4 6/11 T -1 2/3

The new hypothetical vertex is not feasible, and there is no

point in the development of a further subsidiary cut in
association with the s2 - column.

On leaving the choice of the column free, sl is indicated as

pivotal column, against the first x*l - cut as pivotal row.

Main tableau with hypothetical Corresponding updated co1umn-

pivot extract

Name < Value Name x* < Value

1

1/11 3/22 1 5/22 1 1

1/22 -2/11 19/22 3/4

x*l (1/11) -3/22 -5/22 -11 212

x* -7/11 -1/22 -10/11 x* -7 15/22

1 1

T 7/22 5/22 4 6/11 T 3 3/3

We will now develop a subsidiary cut on (a combination in

association with) x 2 ' which will not be illustrated further
here.

21.4 An integer programming adaption of the linear programming

procedure

The procedure INTP offered In this section is a modification

of the LINP procedure from section 12.3. The modification
THE USE OF curs 125

concerns the already mentioned "efficient" rule of row

selection, the "search only", "column-restricted" and "isolated
step're-entries an extension of the upper limit facilities, and
the fact that outside Phase 0, only one step per call is made.

The first of these points largely relates to the internal

working of the procedure. Thus, the instructions

REENTRY:= 0;
LINP CT, M, N, NEQ, NAV, ROWLST, COLLST, REENTRY);
or alternatively
REENTRY .= 0;
IN: 'IF' REENTRY = - 10 'THEN' REENTRY: = 1;
R .= K := ROWN := COLN .= 0;
INTP CT, M, N, NEQ, ROWLST, COLLST, R, K, ROWN, COLN, REENTRY);
'IF' REENTRY = - 10 'THEN' 'GOTO' IN;

both amount to asking for the solution of an ordinary LP

problem. If LINP is employed, a compromise-rule of row
selection is applied in doing so, if INTP is applied, the
"efficient" rule of row-selection is employed in doing so, and
only one step at a time is made. The exit value of - 10 for the
re-entry parameter indicates the fact that a step has actually
been made. The second point, the "search only" re-entry relates
to hypothetical steps.

On exit, R will indicate the index of the last selected

pivotal row, K the index of the last-selected pivotal column
and ROWN and COLN the names of the pivot row and pivot column
variable. On "normal" exit after normal entry, this is not a
particularly useful information.

However, if the procedure is entered with the re-entry-parameter

set a 2, no step will actually be made.

The instructions

REENTRY : = 2; R: = K : = .ROWN : = COLN: = 0;

INTP CT, M, N, NEQ, NAV, ROWLST, COLLST, R, K, ROWN, COLN, REENTRY);

amount to a call for a "hypothetical" step. The rules for pivot-

selection are Cfor K = 0), the rules which this version of the
procedure employs normally. If K is initiated at a non-zero
number, we ask only for a pivot in that particular column. In
short, this is the call for a secondary re-entry. Thirdly,
there is the facility of making an isolated prescribed step.
If the re-entry-parameter is on re-entry set at 1, and
726 CHAPTER XXI

R, K, COLN AND ROWN are set at non-zero values, the step as

indicated by the values of R, K, COLN, and ROWN will be made.
This is for upadating tableau-extracts according to a
previously indicated step, and for implementing earlier choices.

If the re-entry-parameter is 1, and the other parameters are

zero, a single step, as indicated by the usual rules is made.
The fourth point, the extension of upper limit facilities
relates in particular to upper limit cuts. This version of the
procedure handles upper limits on slack-variables and notes
negative entries in the upper-limit distances column as
violated upper limit restrictions. It accordingly activates
Phase 1.

Note that this is a basic re-specification of the LP problem.

The linear programming problem is now specified as

Maximise w' x
Subject to d<Ax<b)
) (21.4.1)
R, < x < u )

The relation between (21.4.1) and the upper limits on the

slack-variables is in fact

s = -A x + b < b - d (21.4.2)

Just as in the case of upper limits on the specified variables

(elements of x), we would put a fancyhigh number as upperbound,
where no meaningful upper limit on a slack-variable is
intended.

These facilities make it possible to formulate an upper limit

cut, by the simple process of putting the difference between
the current fractional value of a variable and the next lower
integer value, in the distances column. Thus, following the
similar procedure as discussed in section 20.2 for branching
restrictions the example in section 21. 3 is written
as follows:

Name sl s2 < Value Distance

-
xl 1/11 3/22 1 5/22 -5/22

x2 1/22 -2/11 19/22 99 3/22

T 7/22 5/22 4 6/11

Upper b 99 99
Lower b 0 0
THE USE OF curs 727

The upper limits on x 2 ' sand s2 are "fancyhigh" limits, but

the violated upper limit ~istance represents a limit cut.

The search operations would then signal the upper limit on xl'
coded as 10001 as pivotal row against sl as pivotal column.

A secondary advantage of this way of recording limit cuts is that

irrespective of whether they are generated as main cuts or as
subsidiary cuts, we can keep any reduced upper limits. (Explicit
cuts may have to be discarded when they turn out to be amply
ful£i lIed.)

Because of the relative ease with which one can make a limit
cut, we will actually make limit cuts on all integer-restricted
variables at each vertex, actual or hypothetical whenever that
is possible.

The textual listing of the procedure INTP is now given as

follows:

'PROCEDURE' INTP(T.M.N.NEQ,NAV,ROWLST.COLLST,
IROWLST,R,K,ROWN.COLN.REENTRY)J
'ARRAY' TJ 'INTEGER' M,N,NEQ.NAV,R.K,ROWN.COLN,REENTRY;
'INTEGER' 'ARRAY' ROWLST.COLLST.IROWLSTJ

'BEGIN' 'INTEGER' I.J,TRYR.TRYN,FIRST.LAST,PRESCRIBED ROW,

UNIFORM,ZJ
'ARRAY' WORKSPACEClI2.1a2lJ
'REAL' ASC.HIG,QUO.PREV POS.TQUO,PIV,COP.NUM,PIWEIGHT,
SAVED ASCJ
'800LEAN' INVERTED.FEASIBLE,UPPERBOUND,POSITIVE,MEASURED,
SEARCH ONLY, PRESCRIBED COLUMNJ
'REAL' 'PROCEDURE' INPROD(I,A,B,X,Y);
'VALUE' A.BJ 'INTEGER' I.A,BJ 'REAL' X.YJ 'ALGOL';

'PROCEDURE' PRODUCT(A,B.CH 'REAL' A.B,C;

'BEGIN'
WORKSPACECl,lllaAJ
WORKSPACECl,2ll-BJ WORKSPACEC2.2ll c C;
Aa-INPROD(Z,l,2.WORKSPACECl.Zl.WORKSPACEC2.Zl)J
END OF PRODUCT a 'END';

'PROCEDURE' ORDL(T,M,N.ER.RH,ROWLST.COLLST);
'ARRAY' TJ 'INTEGER' M,N,ER.RH;
'INTEGER' 'ARRAY' ROWLST,COLLSTJ 'ALGOL';
728 CHAPTER XXI

'COMMENT' LINEAR PROGRAMMING PROCEDURE.

ADAPTED FOR USE IN THE CONTEXT OF INTEGER PROGRAMMING.
INPUT SUPPLY AND SIGNIFICANCE OF THE OPERANDS AS IN
LINP. EXCEPT FOR THE FOLLOWING DIFFERENCES.
INTP HAS A THIRD LIST OF NAME-CODES AS PROCEDURE PARAMETER.
THIS IS THE LIST OF INTEGER RESTRICTED ROWNAMES.
THIS LIST IS NOT GENERATED BY THE PROCEDURE ITSELF.
THE FUNCTION OF IROWLST IN THE INTP PROCEDURE IS TO INHIBIT
UNDESIRABLE SETTING OF FANCYHIGH UPPER LIMITS.
IF THE I-TH ELEMENT IF IROWLST IS NOT A ZERO.
THE UPPER LIMIT DISTANCE OF THE I-TH SLACK-VARIBLE MAY
LEGITIMABLY BE ZERO. AND NO F~~CYHIGH NUMBER IS PUT
INSTEAD.
THIS INFORMATION IS SUPPLIED ACCORDING TO THE FOLLOWING
CONVENT I ON.
o NOT INTEGER RESTICTED. 10 INTEGER RESTRICTED.

INTP RECOGNIZES UPPER LIMITS ON SLACK-VARIABLES. AND

THE NUMERICAL CONTENT OF THE N+2ND COLUMN.
THE UPPERBOUND DISTANCES COLUMN FOR BASIC VARIABLES
SHOULD ALSO BE SUPPLIED.
TO ACCOMODATE UPPER BOUNDS FOR SLACKS WHERE NO MEANINGFUL
UPPER BOUND IS INTENDED. A ZERO MAY BE SUPPLIED INSTEAD.
AND THE PROGRAMME WILL SUBSTITUTE A MILLION FOR IT.
THE CODING OF UPPER LIMITS ON SLACK-VARIABLES IS SIMILAR
TO THOSE ON ORDINARY VARIABLES. I.E. BY WAY OF AN
ENLARGEMENT OF 10000.
THE UPPER LIMIT ON THE I-TH SLACK-VARIABLE IS THEREFORE
CODED AS 11000+1.

, IF' K e O ' THEN' , BEG IN'

NEWLINE( 1 H
WRITETEXT('('YOU SERRONEOUSLYSPRESCRIBEDSA
NEGATIVESCOLUMN')')J
NEWLINE( 1 H
WRITETEXT('('KSS=SSS')');
PRINT(K.S.O)J 'END'J
'IF' K > N 'THEN' 'BEGIN'
NEWLINE( 1 )J
WRITETEXT('('YOUSERRONEOUSLYSPRESCRIBESASCOLUMN-INDEXS
INSEXCESSSOFSTHESTABLEAU-SIZE.') ' ) J
NEWLINE<1 H
WRITETEXT('('KSS-SSS')')J
PRINT(K.S.OH 'END';
'IF' K > 0 'AND' (ReI 'OR' R>M)
'AND' 'NOT' (R- 0 'AND' ABS (ROWN-COLN) -10000) • THEN' • BEGIN'
NEWLINE( 1 )J
WRITETEXT('('YOUSERRONEOUSLYSPRESCRIBESASROW-INDEXS
OUTSIDESTHESTABLEAU.')');
NEWLINE( 1 )J
WRITETEXT('('RSS=SSS')');
PRINT(R.S.O); 'END';
THE USE OF CUTS 729

PRESCRI8ED ROW .- RJ SEARCH ONLY

'IF' PRESCRIBED ROW-O 'AND' ROWNIO
.= 'FALSE'J
'THEN' PRESCRIBED ROW.-ROWNJ
INVERTED .- 'TRUE'} FIRST.-l; LAST.=N;
PRESCRIBED COLUMN .- 'FALSE';
'IF' 'NOT' REENTRY - 0 'THEN' 'BEGIN'
'COMMENT'
FOR REENTRY-O. THE NORMAL LP ALGORITHM IS FOLLOWED FROM THE
START. INCLUDING THE FILLING OF THE NAMELISTS.
PHASE 0 IS GONE THROUGH. AND 0 N E NORMAL STEP IS MADE.
OTHERWISE. FOR NON-ZERO VALUES OF REENTRY. THE PROCEDURE
EXPECTS AN ALREADY UPDATED TABLEAU. WITH NAMELISTS FILLED.
WHICH NEEDS RE-OPTIMIZING AND/OR THE FINDING OF A NEW
FEASIBLE SOLUTION.

RECOGNIZED RE-ENTRY MODES (OTHER THAN THE NORMAL

ENTRY MODE)
ARE REENTRY=l AND REENTRY=2.

FOR REENTRY-l JUST 0 N E STEP IS MADE.

DEPENDING ON THE INITIAL VALUES OF THE ROW AND COLUMN-
INDICES RAND K. THIS STEP MAY BE EITHER FREELY CHOSEN
ACCORDING TO THE NORMAL SEARCH CRITERIA. OR RESTRICTED.
OR EVEN PRESCRIBED.
FOR R-O. K-O. THE STEP IS FREELY CHOSEN.
FOR R=D. KIO. THE STEP HAS TO BE FOUND IN THE INDICATED
COLUMN. OR NONE AT ALL IS MADE.
THE PROBLEM MAY AT THAT POINT BE FOUND TO BE EMPTY OF
A FEASIBLE SOLUTION IN THAT PARTICULAR COLUMN.
FOR RIO. ROWNIO • KIO. NO SEARCH IS PERFORMED. ONLY A
PREDETERMINED STEP IS MADE.
THE NAMECODES ROWN AND COLN SHOULD IN THAT CASE ALSO HAVE
BEEN INITIALIZED AT'THEIR APPROPRIATE VALUES.
FOR REENTRY=2 NO ACTUAL STEP IS MADE AT ALL. BUT THE VALUES
OF THE ROW-AND COLUMN INDICES AND THE NAMECODES ARE
TRANSMITTED TO THE MAIN PROGRAMME.
THE REENTRY PARAMETER ALSO SERVES AS EXIT PARAMETER.
ALARM MESSAGES FOR EMPTY AND UNBOUNDED PROBLEMS ARE NOT
PRINTED BY THE INTP PROCEDURE ITSELF. INFORMATION TO THE
MAIN PROGRAMME IS RECORDED VIA THE REENTRY PARAMETER.
FOR NORMAL. I.E. OPTIMAL AND FEASIBLE EXIT.
OR EVEN IF EMPTINESS OR UNBOUNDEDNESS. THOUGH PRESENT.
IS NOT SO FAR REVEALED.
THE REENTRY PARAMETER BECOMES EITHER ZERO OR MINUS 10.
EVEN IF ITS VALUE ON ENTRY WAS DIFFERENT.
AN UNBOUNDED PROBLEM IS INDICATED BY REENTRY 1.
AN EMPTY PROBLEM BY REENTRY = -1.
 730 CHAPTER XXI

REENTRY • 0 INDICATES AN ALREADY FEASI8LE PROBLEM.

REENTRY· -10 INDICATES A PROBLEM WHICH IS NOT SO FAR
FEASIBLE. AND REQUIRES MORE STEPS TO BE MADE. QUITE
APART FROM OPTIMALITY.
THIS VERSION OF THE PROCEDURE. WHICH IS ADAPTED TO INTEGER
PROGRAMMING HAS THE FOLLOWING SPECIAL FEATURESI
NOT ONLY SPECIFIED VARIABLES. BUT ALSO SLACK-VARIABLES.
HAVE UPPER LIMITS.
ON FIRST ENTRY. THE UPPERBOUNDS DISTANCES COLUMN SHOULD
THEREFORE BE PRE-FILLED.
THE NUMBERS SUPPLIED AS UPPERLIMIT 0 1ST A N C E S
FOR THE SLACK-VARIABLES. INDICATE THE INTERVAL OVER WHICH
THE SPECIFIED COMBINATION OF VARIABLES IS ALLOWED TO VARY.
THUS THE RESTRICTION IS
SIGMA A I.J LT OR EQ B I. AND
SIGMA A I.J GT OR EQ B I L I.
WHERE L I IS THE LIMIT DISTANCE SPECIFIED FOR THE IrH
SLACK-VARIABLE.
GENERALLY. A SINGLE CALL TO THE INTEGER PROGRAMMING ADAPTATION
OF THE LINEAR PROGRAMMING PROCEDURE. CAUSES THE MAKING OR
ON CERTAIN TYPES OF CALL-S ONLY THE SEARCH AND INDICATION.
OF 0 N ESTEP.
THE EXCEPTION IS WHEN THE PROCEDURE IS CALLED WITH REENTRY
SET AT ZERO. I.E. IN THE NON-REENTRY MODE. AND THERE ARE
VARIABLES OF TYPE A8SoLUTE. WITHOUT NON-NEGATIVIY RESTRICTION.
PHASE 0 IS ALWAYS COMPLETED. EXCEPT IN THE CASE OF A SYSTEM
OF INADEQUATE RANK. ONE MORE STEP AFTER PHASE ZERO IS M DE IN
THAT CASE.

'IF' REENTRY=2 'THEN' SEARCH ONLY 1= 'TRUE';

'IF' 'NOT' K=O 'THEN' '8EGIN'

PRESCRI8ED CoLUMNI='TRUE'; FIRSTI=LAST:=Kl 'END';
'GOTO' PHASE IJ 'END';
FILL NAMELISTSI
'FOR' JI-l 'STEP' 'UNTIL' N 'DO' COLLSTCJJ 1= J;
'FOR' 11=1 'STEP' 'UNTIL' M 'DO' ROWLSTCIJ 1= 1000+1;
SET UPPER BOUNDS AND FILL DUMMY ENTRIESI
'FOR' JI .. NAV+I 'STEP' 1 'UNTIL' N 'DO' 'IF' TCM+2.JJ=0
'THEN' TCM+2.JJI=1000000;
TCM+I.N+IJI=TCM+2.N+IJI=TCM+I.N+21:=TCM+2.N+2J:=O;
'FOR' 11=1 'STEP' 1 'UNTIL' M 'DO'
'IF' TCI.N+2J=O 'AND' IRoWLSTCIJ=O
'THEN' TCI.N+2J:=1000000000;
'FOR' JI=I 'STEP' 1 'UNTIL' NAV 'DO' TCM+2.JJI=0;
ATTEND PRIMAL DEGENERACY I
'FOR' 11=1 'STEP' 1 'UNTIL' M 'DO' 'IF' TCI.N+IJ=O
'AND' tROWLSTCIl=O 'THEN' 'BEGIN'
TC I. N+ 1 J I =1 ;
'FOR', Jizi 'STEP' 1 'UNTIL' N 'DO' 'IF' TCI.JJ < 0 'THEN'
TCI.N+IJI=TCI.N+IJ-TCI.JJ;
TCI.N+IJ:=O.OOOOOOOOOOOOOOI*TCI.N+IJ;
'IF' 1< NEQ+I 'THEN' TCI.N+IJI=-TCI.N+IJ; 'END';
TIlE USE OF curs 731

ATTEND DUAL DEGENERACY.

'FOR' JI-l 'STEP' 1 'UNTIL' N 'DO'
'IF' TCM+l.Jl=O 'THEN' '!lEGIN'
TCM+l.JH=I;
'FOR' 1 ..-1 'STEP' 1 'UNTIL' M 'DO' 'IF' TCI.Jl > 0 'THEN'
TCM+l.JJ.=TCM+l.JJ+TCI.JJ;
TCM+l.JJI=O.OOOOOOOOOOOOI.TCM+l.Jl;
'IF' J < NAV+l 'THEN' TCM+l.JJI--TCM+l.JlJ 'END';
'IF' 'NOT' K=O 'THEN' 'GOTO' PHASE Jj
PHASE 01
ENTER VARIA!lLES WITHOUT SIGN RESTRICTION:
INVERTEO:='FALSE';
RETURN IN INVERSION:
K.=K+lJ
'IF' K > NAV 'THEN' 'BEGIN'
INVERTED.='TRUE'; 'GOTO' ORDER; 'END';
COLN:=KJ QUO.=1000000000000000; ROWN:=O; ,
'FOR' 1:=1 'STEP' 1 'UNTIL' M' 'DO'
'IF' ROWLSTC I J > NAV 'AND' 'NOT' TC I.Kl o 'THEN' 'BEGIN'
TQUOI=TCI.N+IJ/TCI.KJ;
'IF' TQUO < 0 'THEN' TQUO.=-TQUO;
'IF' TQUO < QUO 'THEN' 'BEGIN'
QUOI-TQUOJ RlaJj ROWNI-ROWLSTC IlJ 'END';
'I F' ROWN=O 'THEN' 'GOTO' UNBOUNDED;
QUO :. TCR.N+l J/TCR.KH 'GOTO' MAKE THE STEP;

ORDERI
ORDL(T.M.N.2.2.ROWLST.COLLST);

PHASE I: FEASIBLE 1= 'TRUE';

R 1= PRESCRIBED ROW;
'IF' KIO 'AND' (PRESCRIBED ROW I 0 'OR' ABS(COLN-ROWN)=10000)
'THEN' 'GOTO' RESTORE CORRECT VALUE OF QUO;
CHECK ON EMPTYNESS BY CONTRADICTION:
'FOR' II-NAV+l 'STEP' 1 'UNTIL' M 'DO'
'IF' TCI.N+IJ + TCI.N+2J < 0 'THEN' 'BEGIN'
'IF' TC I .N+l J+TC I.N+2l >-0. 000 1 'THEN' 'BEGIN'
'FOR' J:=1.2 'DO'
'IF' TCI.N+JJ<O 'AND' TCI.N+JJ>-O.OOOI
'THEN' TCI.N+Jl:=O;
TCI.N+2l:=-TCI.N+ll+0.000001; 'END'
'ELSE' 'GOTO' EMPTY; 'END';
FIND WHETHER A FEASIBLE SOLUTION EXISTS:
'FOR' I :=NAV+l 'STEP' 1 'UNTIL' M ,~O' 'IF' T[ I.N+1l < 0
'OR' TCI.N+2l < 0 'THEN' FEASIBLE:='FALSE';
MAXIMIZE: HIGI=O; COLNI=O;
'FOR' J:= FIRST 'STEP' 1 'UNTIL' LAST 'DO'
'IF' COLLSTCJJ < 1000 'OR' COLLSTCJl > NEQ + 1000 'THEN' 'BEGIN'
732 CHAPTER XXI

INITIALIZE SUBSTITUTE PREFERENCE FUNCTIONI

ASCI-OJ UNIFORM 1=0;
'IF' FEASIBLE 'THEN' ASCI~-TCM+I.JJ 'ELSE'
'FOR' tr-NAV+I 'STEP' I 'UNTIL' M 'DO' 'BEGIN'
SAVED ASC ,- ASC;
'IF' TCI.N+IJ < 0 'THEN' ASCI-ASC-TCI.JH
'IF' TCI.N+2J < 0 'THEN' ASCI-ASC+TCI.JJ;

'IF' UNIFORM c 0 'AND' 'NOT' ASC - SAVED ASC

'THEN' UNIFORM I=IJ
'IF' UNIFORM" I 'THEN' 'BEGIN'
'IF' TCI.N+IJ < 0 'AND' TCI.JJ > 0.000001
'THEN' UNIFORM 1= -I;
'IF' TCI.N+2J < 0 'AND' TCI.JJ < -0,000001
'THEN' UNIFORM := -I; 'END'; 'END';
REFUSE UNDERSIZE PHASE I PIVOTSI
'IF' 'NOT' FEASIBLE 'AND' ASC < 0.0000001
'THEN' 'GOTO' FINISHED WITH THIS COLUMN;
SELECT BETWEEN NON PREFERRED COL BY DUAL RATIO:
'IF' ('NOT' FEASIBLE 'AND'TCM+I.JJ > 0 'AND' ASC" 0)
'THEN' ASC 1= O.OOI*ASC*ASC/TCM+I.JJ;
'IF' 'NOT' ASC > 0 'THEN' 'GOTO' FINISHED WITH THIS COLUMN;
PUT PRFERENCE FOR UNIFORM COLUMNSI
'IF' UNIFORM - 1 'THEN' ASC 1= 10000000*ASC;
PUT PREFERENCE FOR PREFERRED COLUMNSI
'IF' 'NOT' FEASIBLE 'AND' TCM+I.JJ < 0 'THEN'
ASCI-ASC-IOOOOOOOOOOOO*TCM+I.JJ;
SEARCH FOR QUO IN JTH COLUMN:
QUOt-PREV POSI=TCM+2.JJi TRYRI=O;
'IF' COLLSTCJJ<IOOOO 'THEN' TRYNI=COLLSTCJJ+IOOOO
'ELSE' TRYNI=COLLSTCJJ-IOOOO;
'IF' UNIFORM -I 'THEN' ASC 1= ASC*IOOO;
'FOR' II"NAV+I 'STEP' 1 'UNTIL' M 'DO' 'BEGIN'
MEASURED I- 'FALSE'; POSITIVE I" 'FALSE';

VIOLATED RESTR ON VALUE COLI

EQNI
'IF' ROwLSTCIJ>IOOO 'AND' ROWLSTCIJ<IOOI+NEQ
'AND' TC I.JJ<O 'AND' TC I.N+I J<O
'THEN' 'BEG IN'
'IF' TCI.N+IJ>-O.OOOOOOI 'THEN' TCI.N+IJ:=-O.OOOOOOI;
POSITIVE 1= 'TRUE';
'GOTO' CHECK QUOTIENT; 'END';
'IF' TCI.JJ<O 'AND' TCI.N+IJ<O
'THEN' 'GOTO' CHECK QUOTIENT;
THE USE OF CUTS 733

VIOLATED UPPERBOUND DISTANCEI

'IF' T[ I#Jl>O 'AND' T[ I#N+21<0
'THEN' 'GOTO' TRY' UPPER BOUND;

POSITIVE RATIO WITH UPPER LIMIT DISTANCEI

'IF' T[I#Jl < 0 'AND' 'NOT' TCI#N+21 < 0
'THEN' 'GOTO' TRY UPPER BOUND;
POSITIVE RATIO WITH VALUE COLUMNI
'IF' TCI#Jl>O 'AND' 'NOT' TCI#N+11<0
'THEN' 'GOTO' CHECK QUOTIENT;
'GOTO' FINISHED WITH THIS PIVOT;
REMEASUREI
'IF' POSITIVE 'THEN' 'GOTO' FINISHED WITH THIS PIVOT;
MEASURED 1= 'TRUE';
VALUE COLUMN AFTER UPPER LIMITI
'IF' TCI#Jl > 0 'AND' 'NOT' TCI#N+11<0
'THEN' 'GOTO' CHECK QUOTIENT;
UPPER LIMIT AFTER VALUE COLI
'IF' TCI#Jl < 0 'AND' 'NOT' TCI#N+21<0
'THEN' 'GOTO' TRY UPPERBOUND;
'GOTO' FINISHED WITH THIS PIVOT;
CHECK QUOTI ENT I
UPPERBOUNDI='FALSE'; TQUOI=TCI#N+11/TCI.Jlj
'IF' TQUO-O 'THEN' TQUOa c O.0000000000001/TCI#Jl;
'IF' 'NOT' TCI#N+11 < 0 'THEN' POSITIVEI·'TRUE'
'ELSE' POSITIVE I- 'FALSE';
MEASURE I
'IF' ('NOT' UPPERBOUND 'AND' TCI#Jl<O 'AND' TCI#N+11<0)
'OR' (UPPERBOUND 'AND' TCI#Jl>O 'AND' TCI#N+21<0)
'THEN' POSITIVEI.'FALSE';
'IF' ROWLSTCI1>1000 'AND' ROWLSTCI1<1001+NEQ
'THEN' 'BEGIN'
POSITIVEI.'TRUE'; UPPERBOUND:='FALSE';
'GOTO' RECORD; 'END';
IN PREFERRED COLUMNS ACCEPT ONLY POSITIVEI
'IF' TCM+1#Jl < -0.000001 'AND' 'NOT' POSITIVE
'THEN' 'BEGIN'
'IF' MEASURED 'THEN' 'GOTO' FINISHED WITH THIS PIVOT
'ELSE' 'GOTO' REMEASURE; 'END';

ACCEPT POSITIVE AND ONE NEGATIVEI

'IF' POSITIVE 'AND' TQUO<QUO 'THEN' 'GOTO' RECORD;
'IF' 'NOT' POSITIVE 'AND' TQUO<PREV pas
'AND' QUO· PREV pas 'THEN' 'GOTO' RECORD;
DO NOT ACC£PT IF EXCEEDING EARLIER POSITIVEI
'IF' TQUO > PREV pas 'THEN' 'BEGIN'
'IF' MEASURED 'THEN' 'GOTO' FINISHED WITH THIS PIVOT
'ELSE' 'GOTO' REMEASURE; 'END';
 CHAPTER XXI
734

DO NOT INCREASE IN NDN PREFERRED DIRECTloNI

'IY' POSITIVE 'AND' TQUD> QUO 'AND' TCM+I#Jl>O
'THEN' 'BEGIN'
'IY' MEASURED 'THEN' 'GOTO' YINISHED WITH THIS PIVOT
'ELSE' 'GOTO' REMEASUREJ 'END"
NORMALLY YLY THROUGH NON LIMIT CUTSI
'IY' REENTRY-I 'AND' TRYN>2*N 'AND' TRYN<3*N+I
'THEN' 'BEGIN'
'IF' LAST>FIRST 'AND' 'NOT' POSITIVE 'THEN' 'BEGIN'
'IY' 'NOT' MEASURED 'THEN' 'GoTo' REMEASURE 'ELSE'
'GOTo' FINISHED WITH THIS PIVOT' 'END" 'END"
DO NOT ACCEPT ALL NEGATIVEI
'IY' 'NOT' POSITIVE 'AND' 'NOT' QUO - PRE V POS
'THEN' 'BEGIN'
'IY' 'NOT' MEASURED 'THEN' 'GOTO'REMEASURE
'ELSE' 'GOTO' FINISHED WITH THIS PIVOT' 'END"
'IY' ('NOT' UPPERBoUND 'AND' TCI#Jl<O 'AND' TCI#N+Il<O)
'DR' (UPPERBoUND 'AND' TCI.Jl>O 'AND' TCI#N+21<0)
'THEN' POSITIVEI-'YALSE' 'ELSE' POSITIVEI-'TRUE"
RECoRDI
'IY' POSITIVE 'AND'
TQUo*ASC<HIG 'THEN' 'GoTo' FINISHED WITH THIS COLUMN.
'IY' 'NOT' TQUO < PREV POS
'DR' ('NOT' POSITIVE 'AND' ABS(TCI#Jl»TCM+2#Jl)
'DR' ('NOT' POSITIVE 'AND' I/ABS(TCI#Jl»TCM+2.Jl)
'DR' ('NOT' POSITIVE 'AND' TCM+I#Jl<O)
'THEN' 'BEGIN'
'IY' MEASURED 'THEN' 'GoTo' YINISHED WITH THIS PIVOT
'ELSE' 'GoTo' REMEASURE' 'END"
TRY TO AVOID oUTSI~E PIVoTSI
'IY' (ABS(TCI#Jl) '- 0.1 'DR' ABS(TCI#Jl) > 10)
'AND' 'NOT' (RoWLSTCI1>IOOO 'AND' ROWLSTCI1<IOOI+NEQ)
'THEN' 'BEGIN'
'IY' 'NOT' POSITIVE
'THEN' 'BEG IN'
'IY' MEASURED 'THEN' 'GoTo' YINISHED WITH THIS PIVOT
'ELSE' 'GOTo' REMEASURE' 'END" 'END"
QUol-TQUo, TRYRI-I' TRYNI-RoWLSTCI1,
'IF' UPPERBoUND 'THEN' TRYNI-TRYN+IOOOO'
'IY' POSITIVE 'THEN' PREV PoSI-QUo,
'IY' 'NOT' MEASURED 'THEN' 'GOTo' REMEASURE
'ELSE' 'GOTO' FINISHED WITH THIS PIVOT.
THE USE OF CUTS 735

TRY UPPER BOUND I

UPPERBOUND I- 'TRUE', TQUO 1= -TCI.N+2l/TCI.Jl;
'IF' TQUO=O 'THEN' TQUO ..'-O .000000000000 1/TC I.JlJ
'IF' 'NOT' TCI.N+2l e 0
'THEN' POSITIVE 1= 'TRUE'
'ELSE' POSITIVE 1= 'FALSE';
'GOTO' MEASUREJ

FINISHED WITH THIS PIVOTI 'END'J

PIWEIGHTI=10;

'IF' 'NOT' TRYR=O 'THEN' 'BEGIN'

PIWEIGHT I- ABS(TCTRYR.Jl)J
'IF' PIWEI GHT> 1 'THEN' PI WEI GHT I" 1 IPI WEI GHTi
'IF' PIWEIGHTeO.1 'THEN' PIWEIGHT:=PIWEIGHT*PIWEIGHT
*PIWEIGHTJ 'END';
'IF' QUo*ASC*PIWEIGHT > HIG 'THEN' 'BEGIN'
HIGI=QUO*ASC*PIWEIGHT;
RlaTRYR;
ROWN:=TRYN; KI=J; COLNI=COLLSTCJlJ 'END';

FINISHED WITH THIS COLUMNI 'END';

'IF' COLN = 0 'THEN' 'BEGIN'

'IF' FEASIBLE 'THEN' 'GOTO' OUT 'ELSE' 'GOTO' EMPTY; 'END';
RESTORE CORRECT VALUE OF QUO:
'IF' ABS(ROWN-COLN)-10000
'THEN' RI=OJ
'IF' R=O "THEN' QIJOI"TCM+2.Kl 'ELSE' 'BEGIN'
'IF' ROWLSTCRl=ROWN 'THEN' QUOI=TCR.N+1l/TCR.Kl
'ELSE' QUO 1= -TCR.N+2l/TCR.Kl; 'END';

MAKE THE STEPI

'IF' REENTRY=2 'THEN' 'GOTO' our;
COLLSTCKll =ROIIIN;
'FOR' 11=1 'STEP' 1 'UNTIL' R-l. R+l 'STEP' 1 'UNTIL' M+1 'DO'
'IF' 'NOT' T[ I.Kl=O
'THEN' 'BEGIN'
PRoDUCT(TCI.N+1l.TCI.Kl.-QUo);
PRODUCT(TC I.N+2l.TC I.KJ.QUo); 'END';
'IF' ABS(RoIIIN-COLN)=10000 'THEN' 'BEGIN'
'COMMENT'
DIRECT HIT OF THE UPPER BOUND ON THE COLUMN-VARIABLE.
NO FULL UPDATING OF THE TABLEAU IS REQUIRED;
'FOR' 1:=1 'STEP' 1 "UNTIL' M+1 'DO'
'IF' TCI.KlIO 'THEN' TCI.Kll=-TCl.KlJ
'GOTO' CHECK FOR STATUS; 'END';
ATTEND UPPERBOUNDS OF PIVOTAL PAIR:
COPI=TCM+2.Kl; NUMI=TCM+2.Kll=TCR.N+1]+T[R.N+2];
TCR,N+1l 1= QUO; T[R.N+2] 1= COP-QUO;
736 CHAPTER XXI

REFORMULATE ROW WITH UPPER BOUND NAME.

, IF' ROWN > 1 0000 'THEN'
'FOR' J.-l 'STEP' 1 'UNTIL' N 'DO' TCRIJJ.=-TCRIJH

UPDATE.
PIV.=TCRIKH

UPDATE THE PIVOTAL ROW.

'FOR' J.-l 'STEP' 1 'UNTIL' N 'DO' 'IF' 'NOT' TCRHJJ-O 'THEN'
TCRIJll-TCRIJJ/PIV;
UPDATE THE OTHER ROWS.
'FOR' J.-l 'STEP' 1 'UNTIL' K-lIK+l 'STEP' 'UNTIL' N 'DO'
'IF' 'NOT' TCRIJl - 0 'THEN'
'FOR' 11-1 'STEP' 1 'UNTIL' R-lIR+l 'STEP' 'UNTIL' M+l 'DO'
'IF' 'NOT' TCIIKJ -0 'THEN'
PRODUCTCTCIIJJI-TCRIJJ,TCI,KJ);

CLEAN AND UPDATE PIVOTAL COLUMN.

'FOR' 11-1 'STEP' 1 'UNTIL'M+l 'DO' 'BEGIN'
'IF' I < M+l 'AND' ABSCTCIIKJ) < 0.0000001 'THEN' TCI,KJI=OJ
'IF' 'NOT' TCI,KJ=O 'THEN' TCIIKJI--TCI,KJ/PIVJ 'END';
TCR,KJ.-l/PIVl

ROWLSTC R J • -COLNJ

IF NECESSARY REFORMULATE NEW ROW WITH UPPER BOUND NAME.

'IF' COLN > 10000 'THEN' 'BEGIN'
ROWLSTCRJ.=COLN-l0000;
'FOR' J •• l 'STEP' 1 'UNTIL' N 'DO' 'IF' 'NOT' TCR,JJ=O
'THEN' TCRIJJI=-TCR.JJJ
COP.-TCR,N+1JJ TCRIN+1J.-TCR.N+2JJ TCRIN+2J.=COPJ 'END'J

CHECK FOR STATUS.

'IF' 'NOT' INVERTED 'THEN' 'GOTO' RETURN IN INVERSION;
'GOTO' OUT;

EMPTY.
REENTRY • - - 1 ;
'GOTO' END OF INTP;
UNBOUNDED I
REENTRY .- 1;
NEWLINEC 1 )J
WRITETEXT('C 'UNBOUNDEDICOLUMN') ')J
PRINT(COLLSTCJJI7.0)J
NEWLINE( 1 H
'GOTO' END OF INTPJ

OUT.
REENTRY.· 0;
'IF' FEASIBLE 'THEN' 'GOTO' END OF INTP;
'FOR' I.-NAV+l 'STEP' 1 'UNTIL' M 'DO'
'IF' (TCI,N+l1 < 0 'OR' TCI,N+21 < 0)
'THEN' REENTRY I- -10;
END OF INTPI 'END'J
THE USE OF CUTS 737

The following comments and elucidations seem appropriate in

connection with the textual listing of INTP as given above.

INTP is an adaptation of the LINP procedure from section 12.3,

and the textual differences are mainly the following.

a) The name itself, i.e.

LINP and END OF LINP: at the end

has been replaced by

INTP and END OF INTP:

b) Some additional parameters

The reasons for turning R, K, ROWN and COLN which are internally
declared variables in LINP (and hence not accessible outside
the procedure body), into parameters of the procedure were
already discussed above.

There also is the integer array IROWLST, the list of integer

restricted rownames. While the normal list of rownames is filled
-(by INTP as well as by LINP) on the first "normal" entry,
IROWLST should be pre-filled before the first entry.

The last point relates to only one instruction in the programme,

the settling of a "fancy-high" upper limit where a zero is
supplied on first entry. This is because upper limits on integer
restricted slack-variables have to be integer numbers, and we
cannot exclude the possibility that the main mixed integer
programming procedure would come up with the integer number zero.
This would in effect turn the restriction into an equation, and
such a zero should obviously be respected.

The technical side of the coding of integer-restricted slacks

will be discussed in more detail in the next section.

c) Full inclusion of the upperbounds distances column in the

"fancy-high" initial value loop, the anti-degeneracy-Ioop and
the "Phase I" search operations. This latter point, combined
with the "efficient" rule of row-selection, made it necessary
to re-write the row-selection loop more or less entirely, and
accounts easily for the longest stretch of amended text.

d) Removal of the "wrong sign" equation loop. Turning round the

sign of a row representing an equation complicates the inter-
pretation of the correspo~ding entry in the upper limits distances
column. It was considered' more simple to leave this point to the
user. Thus, if an equation is offered with a positive constant
738 CHAPTER XXI

it may in effect stay an amply fulfilled restriction.

e) During testing, some further modifications were introduced

in connection with the problem of numerical accuracy.

In particular, when as a result of cuts, an integer vertex

is reached, one has to be sure that the solution vector is
also found integer, within the specified tolerance, even
where it is, due to rounding error, not exactly integer.
The following points are to be mentioned in this connection.

(el) Calculation in double length precision. Storage of

numbers is in single length precision floating point,
but the usual combination of multiplication and
subtraction in the context of tableau-updating is done
in double length precision. The significance of the
internal procedure PRODUCT is simply to set the call to
a library procedure which does the double length
calculation.

(e2) The rule of the highest step is modified. Pivots of

which the absolute value is close to unity are
preferred over pivots which are either smaller or
bigger in absolute value. This is not an absolute
preference, rather the choice of pivots is loaded in
that way. This is the significance of the real
variable PIWEIGHT.

f) Preference for "uniform" columns

This point relates to Phase I, i.e. to tableaux in which

negative elements occur either in the value column or in
the upper limit distances column.

Uniform columns are those columns for which not only some,
but all negative slacks become less negative or at least
not more negative on entering the corresponding variable
in the list of basic variables.

The reason why INTP prefers uniform columns to non-uniform

columns is that additional violated restrictions (cuts) may
be introduced between steps.

Strictly speaking we lack a convergence proof; since the

LP problem is changed between steps.

But obviously, as long as all slack-variables are made less

negative at each step, this-problem does not arise - if it
is a problem at all.
THE USE OF CUTS 139

21.5 The coding of integer requirements and cuts

The following information needs to be coded

whether an initially specified variable is continuous

or integer restricted

whether a subsequently developed cut has a continuous

or an integer-restricted slack.

The first problem, to have a record of the type of variable

initially arises as soon as the problem is specified at the
outset.

We need to keep a record, not only of which variables are

specified as continuous but also which slack-variables are
integer-restricted by implication. It is here assumed that
the initial information is given in the tableau with other
"ordinary" linear programming information. A variable will hold
to be integer restricted if that variable is

a) sign-restricted, i.e. restricted to be not below a lower limit

b) all coefficients in its associated column, including the
preference coefficient and the upper limit are supplied
integer-valued.

During later iterations these integer properties of the initial

tableau will be lost, and we need to code them for later
reference.

To this end we can duplicate the name-lists which are associated

with the set-up tableau and replace the names in the duplicate
list which refer to continuous variables by zeros.

Example

Name x2 Value Distance

xl -<
code 1 2

i code 1 2

sl 1001 1001 8 6 15 100

s2 1002 1002 2 -4 -1 100

-3 -1 a
Upper b 101 100 x
Lower b a a x
740 CHAPTER XXI

for our demonstration example with both variables integer

restricted. On the other hand, the initial tableau-matrix

8 6 0 15

2 -4 1 -1

-3 -1 10 0

1000 1000 1000.1 X

would imply that the third variable was continuous. As a result

the second slack would be continuous, but in the first
:estriction t~ere is a zero coefficient for x 3 , and s2 is still
1nteger restr1cted.

The initial coding would therefore be

Name x2 Value Upper b

xl x3 -<
Code 1 2 3

i Code 1 2 0

sl 1001 1001 8 6 0 15 1000

s2 1002 0 2 -4 1 -1 .
1000

T -3 -1 10 0 X

Upper b 1000 1001 1000.1 X X

Lower b 0 0 0 X X

The second problem, the recognition of previous cuts relates to

augmentation and combination. We need to know whether these
operations are appropriate i.e. whether the slack of a particular
cut-restriction is integer restricted.

Cuts which are written as explicit restrictions, are therefore

coded with an enlargement. For limit cuts this enlargement is
equal to n, the number of specified variables, irrespective of
whether the cut is on the variable itself or on a combination.
The associated textual name is x*k. Hence, if x5 is an integer
restricted variable (k = 5) and there are 10 specified variables
 THE USE OF CUTS 741

(n= 10), x*S is the integer-restricted slack of a cut on or in

association with x 5 ' and this slack is coded as 15.

For cuts which do not have integer-restricted slacks, the

corresponding enlargement is 2 n, e.g. 25 for x**S if n is 10.

21.6 Summary of the Cutting Algorithm

We are now in a position to solve a problem by an algorithm

which is substantially the one which we will actually implement,
and summarise the algorithm in the process. The commented text
in the next section contains some further refinements, but the
summary in this section provides the main structure.

The problem which we take is similar to one which we used before,

except for the sign of the x3 preference coefficient.

Maximise

T = 3 xl + x 2 + x3

Subject to

x < -1
3-
(0 < xl 2 100, a2 x 2 2 100, a2 x3 2 100)

Xl' x 2 integer-restricted x3 continuous.

Stage 1

Put the initial tableau and fill the integer-lists

Name x2 Value
xl x3 -<

code to be filled by
INTP
i code 1 2 a

sl - 0 8 6 3 15 100

s2 - a 2 -4 1 -1 100

X X -3 -1 -1 - -
Upper b X X 100 100 99.99 X X

Lower b 0 a a X X
742 CHAPTER XXI

Stage 2

Make an initial step.

Attempt, and if indicated, make one step. The LP algorithm is.

entered with the re-entry parameters as zero, hence before
the one step is made we fill the ordinary name-lists and enter
any variables of "absolute" type. "Phase 0" of INTP is in this
respect comparable with LINP. The one step is the following:

Name x2 Value Distance

xl x3 -<
Code 1 2 3

i code 1 2 0

sl 1001 0 8 CD 3 15 100

s2 1002 0 2 -4 1 -1 100

T -3 -1 -1 - -
upper b 100 100 100 x x

lower b 0 0 0 x x

Name xl sl x3 <
-
Value Distance

code 1 1001 3

i code 1 0 0

x2 2 2 1 1/3 1/6 1/2 212 92!

s2 1002 0 7 1/3 2/3 3 9 90

T -1 2/3 1/6 -12 2!

upper b 100 ll5 99.99 x x

lower b 0 0 0 x x

Stage 3

Check and limit

THE USE OF CUTS 743

Check

Establish whether the current solution is:

a) in optimal form
b) feasible in relation to the already present restrictions
c) integer-valued, and
d) whether any new cuts are being made at this stage; these are
of course unsatisfied, i.e. violated restrictions

Limit

Make preliminary(limit) cuts.

At this stage we make cuts of the following types, in that order

of priority

1) Upper and lower limit cuts on specified variables. (See

sections 10.3, 10.4 and 20.3 concerning the computational
implementation of such restrictions, by putting a negative
entry in one of the two value columns.)

2) Upper and lower limit cuts on integer-restricted slack-

variables. This is also done by adjustment of one of the two
value-columns, and no record is kept of the true values of
slack-variables which may therefore be under-stated.

Since cuts of priority classes 1) and 2) do not require

additional tableau-space, all possible cuts of this type are
made.

3) Limit cuts on combinations

We reserve space for at most n+l explicit cut-restrictions. This

1S one more than the m~ximally possible number of binding cuts,
but we may come back at this stage for a number of successive
steps, developing several cuts on the same variable. It is
therefore possible that we shall have to refrain from further
cutting simply because the space reservation has been exhausted.

We find that the solution is

a) not optimal
b) feasible in relation to the existing restrictions
c) not integer, and
d) not satisfying all cuts (we are making a new cut on x 2 ).

The x 2 -variable is recorded as being ~ unit over its new upper

limit of 2.
744 CHAPTER XXI

The tableau becomes:

Name xl sl x3 -< Value Distance

code 1 1001 3

i code 1 0 0

x2 2 2 1 1/3 1/6 ! 212 -12

s2 1002 0 7 1/3 2/3 3 9 90

T -1 2/3 1/6 -12 212

Upper b 100 US 99.99

Lower b 0 0 0

Stage 4

Attempt, and if indicated, make one step (The problem may be

found empty at this stage, this obviously would terminate the
algorithm). In view of the particular features of the amended
LP algorithm coming into play at this point we will describe
the rules for making this one step in detail. The substitute
objective function is in the first instance obtained by (the
summation of all violated restrictions), the violated b2-row.

-1 1/3 1/6 1/2

-1 1/3 -1/6 -1/2

The taking of the squares of these numbers is.in this case

irrelevant. The xl and x3 columns are "preferred" columns
because they indicate an increase in both the substitute
objective function and the specified objective function.

In a "preferred" column, violated restrictions are disregarded

and we exchange xl against s2' on the indication of the rule of
the highest step , applied to the specified preference function.
THE USE OF CUTS 745

The tableau becomes

Name s2 sl x3 <
- Value Distance

code 1002 1001 3

icode 0 0 0

x2 2 2 -2/11 1/22 -1/22 19/22 1 3/22

xl 1 1 3/22 1/11 9/22 1 5/22 98 17/22

T 5/22 7/22 2/11 4 6/11

Upper b 99 115 99.99

Lower b 0 0 0

Stage 5

Consult the logical findings from stage 3. If the problem was

found to be: not optimal, or not feasible, or not satisfying a
cut, return to stage 3.

We are referred back to Stage 3 on two separate indications

a) the problem was not optimal at that stage, and

d) a new cut was made (the upper limit cut on x 2 )·

Application of the check and limit procedure now indicates that

the problem now is

a) in optimal form
b) feasib Ie
c) not integer (both xl and x 2 are fractional)
d) not satisfying all cuts, because a new cut (a reduced upper
limit on xl) is introduced. The tableau becomes
746 CHAPTER XXI

Name s2 sl x3 <
-
Value Distance

code 1002 1002 3

icode 0 0 0

x2 2 2 2/11 1/22 -1/22 14/22 1 3/22

xl 1 1 3/22 1/11 9/22 1 5/22 -6/22

T 5/22 7/22 2/22 4 6/11

Upper b 99 115 99.99 x x

Lower b 0 0 0 x x

Stage 4 does indicate a step; x3 becomes a basic variable and

the upper limit on xl becomes binding.

This involves an activation of an upper limit facility of the

LP algorithm which is slightly unusual and we will illustrate
it.

First the xl-row is replaced by the bl-row, the tableau now

becomes:

Name s2 sl Value Distance

x3 -<

code 1002 1001 3

icode 0 0 0

x2 2 2 -2/11 1/22 -1/22 19/22 1 3/22

bl 10001 1 -3/22 -1/11 E9/ip -5/22 1 5/22

T 5/22 7/22 2/22 4 6/11

Upper b 99 115 99.99 x x

Lower b 0 0 x x
0,
I
THE USE OF CUTS 747

The step is now made in the normal way and the tableau becomes:

Name s2 sl bl < Value Distance

-

code 1002 1001 10001

i code 0 0 0

x2 2 2 -1/6 1/18 -1/9 8/9 1 1/9

x3 3 0 1/3 2/9 -2 4/9 5/9 99 4/9

T
13/66 59/198 2/9 4 49/99
Upper b 99 115 1

Stage 5 once more indicates a referral to Stage 3, but this time

solely on the indication that d) there was an unsatisfied cut
when stage 3 was left the last time.

In stage 3 we now find that the solution ~s

a) optimal
b) feasible
c) not integer, and
d) satisfying all cuts

(x 3 is continuous and no limit cut on x 2 can be made). The

re-entry of the LP algorithm in Stage 4 now is trivial - no
step is indicated - In Stage 5, we are now no longer referred
back to Stage 3.

Stage 6

Remove any amply fulfilled explicit cuts from the tableau. (None).

Stage 7

Check for normal end of the algorithm.

We only get at this stage, if the problem, as currently

formulated, is optimal and feasible. Therefore, if the solution
is integer, the normal exit is activated. For the present example,
this is not yet the case, hence we proceed to the next stage.

Stage 8

Write a main cut on a still frac~ionally valued variable. As

the priority classes of cuts have been exhausted, this is not
748 CHAPTER XXI

a limit cut. However, it is not necessarily as elementary cut.

The tableau becomes the following:

Name s2 sl bl Value Distance

code 1002 1001 1001

icode 0 0

x2 2 2 -1/6 1/18 -1/9 8/9 1 1/9

x3 3 0 1/3 2/9 -2 4/9 5/9 99 4/9

x**2 8 X @ -1/16 -1 -1 100

T 13/66 59/198 2/9 4 49/99

Upper b 99 115 1 x x

Lower b 0 0 0 x x

Stage 9

Search for a pivot. We may enter this stage with or without a

column-restriction. Initially we always enter without a column
restriction, but at a later stage the choice of pivots may be
restricted to a particular column. Try to find a hypothetical
pivot in the main tableau. (The problem may be found empty - of
integer solutions - ) in which case the algorithm is terminated
prematurely. We may also on re-entering this stage, find the
problem empty, as far as pivots in a particular column is
concerned. We re-enter this stage with column-restriction
after making a subsidiary cut. If no pivot in that column may
be found the column-restriction is to be removed. In fact a
hypothetical pivot is indicated. The s2-variable is to be
introduced into the basis, making the x**2-cut binding.

Stage 10

Make a column-extract, according to the indicated column.

THE USE OF CUTS 749

Name Value Distance

s2 -<
code 1002

icode 0

x2 2 2 -1/6 8/9 1 1/9

x3 3 0 1/3 5/9 99.44

x** 2 8 X @ -1 100

T 13/66 4.49/49

Upper b 99 x x

Lower b 0 x x

Stage 11

Make a step in the column extract (This is always possible when

this stage is entered). The updated column-extract becomes

Name x** 2 Value Distance

-<

code 8

icode X

x2 2 2 -1/9 1 1

x3 3 0 2/9 1/3 99.67

s2 1002 0 -2/3 2/3 98 1/3

T 13/99 4 36/49
Upper b 99
Lower b
750 CHAPTER XXI

Stage 12

Go to a stage, as indicated by the updated column-extract and

the index of the solution column. If the (hypothetical)
solution is integer, or not feasible even after updating (while
the column was chosen freely) go to Stage 3. If the hypothetical
solution is integer, or not feasible even after updating, but
the column was prescribed, remove the column-restriction, and
go to stage 9. An integer-valued hypothetical vertex means that
the cuts have been successful, as far as activating the parti-
cular column is concerned. If the solution cannot be made
feasible by that column only, there is no purpose in developing
a (further) subsidiary cut at least not in connection with this
particular variable becoming a basic variable.

We are referred to stage 3, on the indication of an integer

solution-vector (x3 is continuous, i.e. is allowed to remain
fractional).

In stage 3, we find that the main solution-vector is

a) in optimal form
b) not feasible
c) not integer
d) not satisfying a cut (the x**2-cut)

No priority cut is indicated

In stage 4, we make a step, which is the same as already
indicated as hypothetical step in Stage 9. The tableau
becomes:

Name x**2 sl bl < Value Distance

-

code 8 1001 1001

icode 0 0 1

x2 2 2 -1/9 1/16 1 1

x3 3 0 2/5 5/24 -2 1/3 113 99 2/3

s2 1002 0 -2/3 1/24 2/3 2/3 98 1/3

T 13/99 51/176 1/11 4 36/49

Upper b 99 115 1 x x

Lower b 0 0 0 x x
TIlE USE OF CUTS 751

In stage 5 we are once more referred back to stage 3, as on the

previous exit of stage 3 the solution was not feasible and there
was an unsatisfied cut. Back in stage 3, we find that the
solution is

a) in optimal form
b) feasible
c) integer and
d) satisfying all cuts.

No new cuts are called for.

In stage 4 the entry of the LP algorithm is riow trivial.

In stage 5, we are not referred back to stage 3.

In stage 6, we do not need to remove any cuts.

In stage 7, we conclude that the problem has been solved.

In this particular example, no stage beyond stage 12 was

activated. We need at least two integer-restricted basic
variables for that, not counting variables which are already
at upper limits.

However, in the general case, there are more stages, i.e.

Stage 13

We only enter stage 13 with a fractional hypothetical vertex.

Accordingly, find the name of the first integer-restricted, but
fractionally valued variable in the hypothetical vertex, as
updated in the column-extract. This is the variable on which a
subsidiary cut is to be developed. We do not cut on the variable
which has already been cut on in the same column-extract. The
development of such a cut in a row-extract would require several
steps; and it becomes more practical to refer back to the full
tableau.

Stage 14

Make a row-extract. Depending on the name of the last subsidiary

cut, this row-extract initially may contain one, or two ordinary
rows from the main tableau. The row which was selected in stage 9
as pivotal row is always in the extract. If the selected cut
relates to the variable selected in stage 9 (i.e. the one
column in the unupdated form of the column extract), the row
extract contains only one ordinary row. Otherwise find the row
to which the subsidiary cut last selected in stage 13 refers.
This is one of the rows of the main tableau in that case, and
752 CHAPTER XXI

it should be copied into the extract.

Stage 15

Implement the hypothetical step suggested in stage 9, in the

row-extract.

Stage 16

Cut on the variable indicated as to be cut o~.

Stage 17

Make the backward step in the row extract

Stage 18

Assemble the new subsidiary cut with the main tableau. If the
subsidiary cut arose from entering the same column in the
main tableau as was already used to form a previous subsidiary
cut immediately before, overwrite the previous one; otherwise
extend the tableau.

Go to stage 9.

As the various exits of the algorithm are already indicated in

earlier stages this is the last stage.

21.7 Commented text of a mixed integer programming procedure

The programmed procedure offered in this section broadly follows

the outline scheme discussed in Section 21.6.

The following points, which cropped up at the stage of programme-

testing are useful to eludicate on.

First of all, recall Sections 19.2 and 19.3. It was there assume~
that when the significance of a zero-one variable really is to
indicate whether or not a certain relationship is at all
applicable, a "fancyhigh" coefficient is written.

These "fancyhigh" numbers can, however, become a major problem

when the method of cuts is employed to solve the problem, for
the reasons indicated at the end of Section 21.4.

The first part of the interlude between stages 1 and 2 reduces

such numbers to a figure which is (in absolute value) just one
unit higher than what is needed to "set free" a particular
relation.
THE USE OF CUTS 753

The second part of the interlude will be more or less obvious.

Secondly, there are more indications for the generation of cuts

than discussed in the previous section.

Most of them are controlled by the incoming variable intercept,

and are largely designed to avoid certain steps altogether.

The following two points are mentioned here:

Cuts on incoming variables, creating degeneracy

If an integer-restricted incoming variable is provisionally

indicated as a candidate for entering the basis (with a
fractional value) we develop the hypothetical vertex and cut
with the help of a row-extract, even if no main cut is so far
present. It may then well happen that the unupdated form of such
a cut is satisfied in degeneracy by the integer value zero.
Therefore, subsequent searches for pivotal columns will give
preference to other columns, if any are available. The rule
of the highest step or the version of that rule which is
implemented in INTP will try to avoid steps of zero length.

Pivot-size cuts

This refers to what is essentially a diversionary device. If a

candidate-pivot of undesirable absolute value is thrown up, a
further cut is made, simply to perturb the problem and to allow
INTP to have a fresh go at perhaps finding a pivot of a more
desirable order of magnitude.

'PROCEDURE' MIXI<T.M.N.NEQ.NAV.ROWLST.COLLST.NCUTS);
'VALUE' MJ 'INTEGER' M.N.NEQ.NAV.NCUTS;
'ARRAY' T; 'INTEGER' 'ARRAY' ROWLST.COLLST;

'BEGIN'
'INTEGER' I.J.R.K.RBWN.COLN.CUTN.NAME.REENTRy.II.JJ.MM.
MMM.DNAV.KK.KKK.F.P.CW.RR.DNCUTS.LOOPLENGTH.NIRV.CUTR.
SAVED R T.SAVED R E. CHECKI;
'REAL' QUO.LV.NLV,NUM.COP.FANCYHIGHJ
'BOOLEAN' OPTIMAL.FEASIBLE.INTEGER.SATISFIED.POS.NEG.MAIN.
RETURNED.RECALLING TO STAGES 3 AND 4. INTERCEPT. DEGENERATE.
OVERWRITING FOR COMBINATION.PIVOT SIZE CUT,
STEP ALREADY KNOWNJ
'INTEGER' 'ARRAY' IROWLSTC1IMJ.ICL.ICOLLSTC1INJ.DLSTClI3J.
FDLSTC1IM+N+N+3JJ
'ARRAY' ROWEXTRClI6.1IN+2J.COLEXTRC1IM+N+N+4.113J;

'PROCEDURE' INTP<T.M.N.NEQ.NAV.ROWLST.COLLST.
IROWLST.R.K.ROWN,COLN,REENTRY)J
'ARRAY' T1 'INTEGER" M,N.NEQ·,NAV,R,K,ROWN,COLN.REENTRY;
'INTEGER' 'ARRAY' ROWLST.COLLST,IROWLST;
'ALGOL;
754 CHAPTER XXI

'PROCEDURE' ORDLCT.M.N.ER.RH.ROWLST.COLLST)J
'ARRAY' TJ 'INTEGER' M.N.ER.RHJ
'INTEGER' 'ARRAY' ROWLST.COLLSTJ
'ALGOL'J

'COMMENT'
MIXED INTEGER PROGRAMMING BY CUTS.
THE TABLEAU SHOULD BE SUPPLIED IN THE USUAL LP-PRESENTATION.
WITH ONE MODIFICATION,
UPPER LIMITS ON EACH SUMMATION-EXPRESSION PER I-TH ROW
THE MAXIMUM VALUE OF SIGMA -ACI.Jl • XCJl.
SHOULD BE ENTERED IN THE UPPERBOUNDS DISTANCES COLUMN.
SIGN-RESTRICTED VARIABLES C J > NAV ) •
FOR WHICH EACH ELEMENT OF THE CORRESPONDING CULUMN OF THE
TABLEAU. INCLUDING PREFERENCE COEFFICIENT AND UPPER LIMIT.
IS SUPPLIED INTEGER. ARE INTERPRETED AS INTEGER-RESTRICTED.
OTHER SPECIFIED VARIABLES ARE ASSUMED TO BE CONTINUOUS.
ON EXIT. NCUTS ROWS WILL HAVE BEEN ADDED TO THE TABLEAU.
THE TABLEAU SHOULD BE DECLARED AS BEING OF ORDER M+N+NIRV+4
BY N+2.
HERE NIRV IS THE NUMBER OF INTEGER RESTRICTED VARIABLE.
THIS VARIABLE IS NOT DEqLARED EXPLICITLY. BUT IS IMPLIED BY
THE NUMBER OF ALL-INTEGER COLUMNS. WHICH MAY GIVE RISE TO
THE DEVELOPMENT OF CUTS.
THE LIST OF ROWNAMES SHOULD BE DECLARED AS BEING OF
ORDER M+N+NIRV+2,

THE LIST OF COLUMN-NAMES SHOULD BE DECLARED AS BEING OF

ORDER N.

STEP ALREADY KNOWNI-RETURNEDI='FALSE';

SAVE COPIES OF PARAMETERS:
MMM:-MJ DNAVI-NAVJ

INITIATE DLST AND ROW EXTRACT WITH COPYABLE NONSENSE:

'FOR' 11-1.2.3 'DO' DLSTCIl:--12345J
'FOR' 11-1.2.3.4.5 'DO'
'FOR' JI-l 'STEP' 1 'UNTIL' N+2 'DO' ROWEXTRC I.Jll-123.45;
STAGE 11
ATTEND UPPER LIMITS AND CODE INTEGER REQUIREMENTSI
'BEGIN' 'COMMENT'
NOTE THAT THE TABLEAU SHOULD BE PRE-FILLED BEFORE ENTRY.

FANCYHIGH:=100;

'FOR' J.= NAV+l 'STEP' 1 'UNTIL' N 'DO'

'IF' TCM+2.Jl = 0 'THEN' TCM+2.Jl IS FANCYHIGH;
nIE USE OF CUTS 155

'FOR' 1,-1 'STEP' 1 'UNTIL' M 'DO' 'IF' TCI.N+2J .. O 'THEN'

'FOR' J,-l 'STEP' I" 'UNTIL' N 'DO' 'BEGIN'
'IF' TCI.JJ-O 'OR' (J>NAV 'AND' TCI.JJ>O)
'THEN' 'GOTO' END OF UPPERBOUND ADJUSTMENT LOOP;
'IF' J>NAV 'THEN' LV,-TCM+2.Jl 'ELSE' LVI- FANCYHIGH;
TCI.N+2J,-TCI.N+2J+ABS(TCI.Jl*LV);
END OF UPPERBOUND ADJUSTMENT LOOP, 'END';
CODE INTEGER REQUIREMENTS,
'FOR' J,-l 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'
ICOLLST[JJ'-J;
'IF' J< NAV+l 'THEN' 'BEGIN'
'COMMENT'
FREE VARIABLES ARE NEVER INTEGER RESTRICTEDI
ICOLLSTCJJI-O; 'END'
'ELSE' 'FOR' II-I 'STEP' 1 'UNTIL' M+2 'DO'
'IF' 'NOT' TCI.Jl"ENTIER(TCI.JJ) 'THEN' ICOLLSTCJJ,-O;
'END'J

'FOR' II-I 'STEP' 1 'UNTIL' M 'DO' 'BEGIN'

IROWLSTCIl'"'1000+IJ
'IF' 1 < NEQ+l 'THEN' 'BEGIN'
'COMMENT'
EQUATION-SLACKS ARE INTEGER RESTRICTED TO THE
INTEGER VALUE OF ZERO; 'END'
'ELSE' 'FOR' JI-l 'STEP' 1 'UNTIL' N 'DO'
'IF' 'NOT' TCI.Jl=O 'AND' ICOLLSTCJ1"O
'THEN' IROWLSTCIllcOJ
'END'}
NIRVI-QJ
'FOR' J,-l 'STEP' 1 'UNTIL' N 'DO'
'IF' ICOLLSTCJ1=J 'THEN' NIRV:-NIRV+1J
NCUTS,-OJ
END OF STAGE 11 'END'I
INTERLUDE,
REDUCE FANCYHIGH COEFFICIENTS,
'FOR' RI-l 'STEP' 1 'UNTIL' M 'DO'
'FOR' Klal 'STEP' 1 'UNTIL' N 'DO'
'IF' ICOLLSTCK1=K 'AND' TCR.Kl < -2 'AND' TCM+2.Kl=1
'AND' TCM+l.Kl-O
'THEN' 'BEGIN'
'FOR' JI-l 'STEP' 1 'UNTIL' NAV 'DO'
'IF' TCR.J1IO 'THEN' 'GOTO' END OF COEFFICIENT REDUCTION;
TCM+3.N+l1,-TCR.N+l1J

'FOR' JI-l 'STEP' 1 'UNTIL' K-l.K+l 'STEP' 1 'UNTIL' N 'DO'

'IF' TCR.Jl>O
'THEN' TCM+3.N+~~I=TCM+3.N+ll-TCR.Jl*TCM+2.J1J
'IF' TCR.Kl<TCM+3.N+l1-1
'THEN' 'BEGIN'
TCR.KJ,-TCM+3.N+l1-1J
'IF' TCR.Kl>O 'THEN' TCR.K11-OJ
'GOTO' REDUCE FANCYHIGH COEFFICIENTS; 'END'J
756 CHAPTER XXI

END OF COEFFICIENT REDUCTIONI 'END';

TAKE OUT COMMON INTEGER FACTORSI

'FOR' FI-2 'STEP' 1 'UNTIL' 1000 'DO' 'BEGIN'
FIND A PRIME FACTOR FI
'FOR' PI-2 'STEP' 1 'UNTIL' F-l '00'
'IF' 'NOT' F/P - ENTIERCF/P) 'THEN' 'GOTO' NEXT F;

'FOR' II- NEQ+1 'STEP' 1 'UNTIL' M 'DO' 'BEGIN'

'FOR' JI-NAV+1 'STEP' 1 'UfliTIL' N 'DO'
'IF' 'NOT' TtI.JJ-O 'AND' 'NOT' ICOLLSTtJJ"O
'THEN' 'GOTO' START OF DIVISIONLOOP;
'GOTO' END OF DIVISIOfliLOOPJ

START OF DIVISIONLOOPI
'FOR' JI- NAV+1 'STEP' 1 'UNTIL' N 'DO'
'IF' 'NOT' TtI.JJ/F-ENTIERCTtI.JJ/F)
'AND' 'NOT' ICOLLSTtJJ=O
'THEN' 'GOTO' END OF DIVISIONLOOP;
NOW TAKE F OUTI
'FOR' JI-1 'STEP' 1 'UNTIL' N+2 'DO' Tt I.JJI=Tt I.J]/F;
'IF' 'NOT' IROWLSTtlJ .. 0 'THEN' 'FOR' JI-N+1.N+2 '~O'
TtI.JJI-ENTIERCTtI.JJ);
'GOTO' START OF DIVISIONLOOP;
END OF DIVISIONLOOPI 'END';
NEXT FI 'END' J

STAGE 21
MAKE AN INITIAL STEPI
'BEGIN' 'COMMENT'
THE INITIAL CALL TO THE LP ALGORITHM FILLS THE ORDINARY
NAMELISTS. ATTENDS DEGENERACY -BOTH PRIMAL AND DUAL-.
PIVOTS THE 'ABSOLUTE' VARIABLES INTO THE BASIS.
AND MAKES. IF POSSIBLE. ONE NORMAL STEP.

RI-KI-ROWNI-COLNI-REENTRYI-OJ
INTPCT.M.N.NEQ.NAV.ROWLST.COLLST.IROWLST.
R.K.ROWN.COLN.REENTRY)J
'IF' REENTRY - -1 'THEN' 'GOTO' EMPTY;
'IF' REENTRY - 1 'THEN' 'GOTO' UNBOUNDED;
END OF STAGE 21 'END';

STAGE 31
MAIN I- 'TRUE'; CUTN I- OJ
CHECK AND LIMITI
'8EGIN' 'COMMENT'

THIS BLOCK OF PROGRAMME CHECKS WHETHER THE CURRENT SOLUTION

lSI

A IN OPTIMAL FORM
THE USE OF CUTS 757

B FEASIBLE WITH RESPECT TO THE SO FAR SPECIFIED

RES TR I CTI ON S
C INTEGER TO A TOLERANCE OF 0.001
D SATISFIED.
THE SOLUTION IS HELD TO BE NOT SATISFIED WITH RESPECT TO
CUTTING REQUIREMENTSI
IF NEW CUTS ARE BEING FORMULATED BY THIS BLOCK.
THE FOLLOWING PRIORITY CLASSES OF CUTS ARE FORMULATED BY THE
CHECK AND LIMIT BLOCK
1 UPPER AND LOWER LIMIT CUTS ON INTEGER RESTRICTED
SPECIFIED VARIABLESI
BY WAY OF ADJUSTING THE UPPERBOUNbs DISTANCES COLUMN
OR THE VALUE COLUMN AND THE NON-UPDATED LOWER LIMIT.

2 UPPER OR LOWER LIMIT CUTS ON OTHER INTEGER RESTRICTED

VARIABLESI E.G. INTEGER-RESTRICTED SLACK-VARIBALES
OR CUTS ON COMBINATIONS OF VARIABLES.
ALSO BY WAY OF ADJUSTING THE UPPERBOUNDS DISTANCES COLUMN.
3 LIMIT COTS ON COMBINATIONSI
ASSOCIATED WITH SPECIFIED VARIABLES.
THE LAST CLASS OF CUTS HAS THE LOWEST PRIORITY AND
IS MADE ONLY WHEN NO CUT OF A HIGHER PRIORITY CLASS HAS BEEN
MADE IN THE PARTICULAR RUN THROUGH THE BLOCK SO FAR.
AN AMPLY FULFILLED EXPLICIT CUT MAY BE OVERWRITTEN BY A LATER
CUT ON THE SAME VARIABLE -OR ON ITS CUT-.

CLEAN OUT ROUNDING ERRORS:

'FOR' 11=1 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'FOR' JI=1 'STEP' 1 'UNTIL' N 'DO'
'IF' ABS(TCIIJl) < 0.0000001 'THEN' TCIIJl:=O;
CHECK FOR OPTIMALITY:
OPTIMAL := 'TRUE';
'FOR' Jl c l 'STEP' 1 'UNTIL' N 'DO'
'IF' 'NOT' (COLLSTCJl > 1000 'AND' COLLSTCJl < 1001+NEQ)
'AND' TCM+NCUTS+IIJl < 0 'THEN' OPTIMAL P' 'FALSE';
CHECK FOR BEING FEASIBLE:
FEASIBLE :- 'TRUE';
'FOR' II"NAV+l 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'IF' TC IIN+ll < 0 'OR' TC IIN+2l < 0
'THEN' FEASIBLE :- 'FALSE'; .
758 CHAPTER XXI

CHECK FOR BEING INTEGER AND LIMITI

INTEGER I - SATISFIED la 'TRUE';
LOOPLENGTH ID M+NCUTSJ
'FOR' II- M+NCUTS+2. NAV+I 'STEP' 'UNTIL' LOOPLENGTH
'DO' 'BEGIN'
'IF' I-M+NCUTS+2 'THEN' 'GOTO' BEGIN OF COMPARING;
DO NOT CUT WITH EQUATIONSI
'IF' ROWLSTCIl>IOOO 'AND' ROWLSTCll<IOOO+NEQ
'THEN' 'GOTO' END OF LIMITLOOPJ
'IF' ROWLSTCIl > N 'THEN' 'GOTO' SLACK OR CUT;
'IF' ICOLLSTCROWLSTClll • 0
'THEN' 'GOTO' END OF LIMITLOOP;
ONE MAIN LIMIT CUT ON ONE VARIABLE IS ENOUGH.
'IF' MAIN 'AND' ROWLSTCll<N+I 'THEN' 'BEGIN'
'FOR' 111"1 'STEP' I 'UNTIL' M+NCUTS 'DO'
'IF' ROWLSTClll-ROWLSTCll+N
'AND' CTCII.N+Il < 0 'OR' TCII,N+2l < 0)
'THEN' 'GOTO' END OF LIMITLOOP; 'END';

SLACK OR CUT.
DO NOT CUT ON NON LIM,IT CUTSI
'IF' ROWLSTCll > 2*N 'AND' ROWLSTCll < 1001
'THEN' 'GOTO' END OF LIMITLOOP;
DO NOT CUT ON AN UNSATISFIED LIMIT CUTI
'IF' ROWLSTCIl>N 'AND' ROWLSTCIl<2*N+I
'AND' CTCI.N+Il<O 'OR' TCI.N+2l<0)
'THEN' 'GOTO' END OF LIMITLOOP;
'IF' ROWLSTCIl < 1001
'THEN' 'GOTO' START INTEGERCHECK CUM LIMIT;
'IF' IROWLSTCROWLSTCIl-IOOOl D 0
'THEN' 'GOTO' END OF LIMITLOOPJ
START INTEGERCHECK CUM LIMIT:
'IF' ABSCENTIERCTCI.N+Il+O.OOI) - TCI.N+Il) < 0.002
'AND' MAIN 'THEN' 'BEGIN'
'FOR' JI-I.2 'DO' TCI.N+Jl,aENTIERCTCI.N+Jl+0.003);
'FOR' J.al.2 'DO' 'IF' TCI.N+Jl"'O
'THEN' TCI.N+Jll-O.OOOOOOOOOIJ
'GOTO' END OF LIMITLOOPJ 'END'J
'IF' ROWLSTCIl < N+I 'THEN' INTEGER 1= 'FALSE';
'IF' TCI.N+Il < 0 'OR' TCI.N+2l < 0
'THEN' 'GOTO' END OF LIMITLOOP;
ONE CANNOT ADJUST ON A NON BASIC VARIABLEI
'IF' 'NOT' MAIN 'AND' I-I 'THEN' 'GOTO' END OF LL;
CHECK WHETHER UPPER LIMIT IS POSSIBLE.
'FOR' J.-I 'STEP' I 'UNTIL' N 'DO'
'IF' 'NOT'. CCOLLSTCJl > 1000 'AND' COLLSTCJl < 1001+NEQ)
'AND' TCI.Jl < 0 'THEN' 'GOTO' END OF ULJ
THE USE OF CUTS 759

UPPER LIMITI
'IF' TeI.N+2l > TeI.N+Il
'THEN' TeI.N+2l 1= ENTIERCTeI.N+Il) + 0.0000001 - TeI.N+Il
'ELSE' TeI.N+2l 1= TeI.N+2l - ENTIERCTeI.N+2l) -0.9999999;
SATISFIED 1= 'FALSE';
'GOTO' END OF LIMITLOOP;
END OF ULI

CHECK WHETHER LOWER LIMIT IS POSSIBLEI

'FOR' JI-I 'STEP' I 'UNTIL' N 'DO'
'IF' 'NOT' CCOLLSTCJJ > 1000 'AND' COLLSTeJJ < 1001+NEQ)
'AND' TCI.JJ > 0 'THEN' 'GOTO' END OF LL;
LOWER LIMITI
'IF' ROWLSTC I J < N+I 'THEN' 'BEGIN'
ADJUST NON UPDATED FORM OF LOWER LIMITI
TeM+NCUTS+3.ROWLSTCIJJ I-
TCM+NCUTS+3.ROWLSTe IJ J + ENTIERCTCI.N+I J+I); 'END';
'IF' TeI.N+IJ < TCI.N+2J
'THEN' TeI.N+IJ I- TeI.N+IJ - ENTIERCTCI.N+IJ) -0.9999999
'ELSE' TeI.N+2J I- ENTIERCTeI.N+2l) + 0.0000001 - TeI.N+IJ;
SATISFIED I- 'FALSE';
'GOTO' END OF LIMITLOOP;

END OF LLI
'IF' ROWLSTeIJ > 2*N 'THEN' 'GOTO' END OF LIMITLOOP;
BEGIN OF LOWER LIMIT LOOPI
'I F' ROWLSTC 1 J < N+ I 'THEN' 'BEGIN'
'FOR' JI-I 'STEP' I 'UNTIL' N 'DO'
'IF' 'NOT' CCOLLSTCJJ > 1000 'AND' COLLSTCJJ < 1001+NEQ)
'AND' TCI.Jl > 0 'THEN' 'GOTO' BEGIN OF COMBLOOPJ
'GOTO' PREPARE LLJ 'END'J

DO NOT ADJUST ON LOWER LIMIT NON BASIC VARa

'IF' 'NOT' MAIN 'AND' I-I 'THEN' 'GOTO' END OF LIMITLOOP;

INVESTIGATE LOWER LIMIT ON CUTSLACKI

'FOR' JI-I 'STEP' I 'UNTIL' N 'DO'
'IF' 'NOT' CCOLLSTeJJ > 1000 'AND' COLLSTCJJ < 1001+NEQ)
'AND' T[ I.JJ > 0 'THEN' 'GOTO' BEGIN OF COMBLOOP;

WRITE LL ON CUT BY ADJUSTING VALUE COLI

'IF' TCI.N+IJ > TCI.N+2J
'THEN' T[I.N+IJI-TeI.N+IJ - ENTIERCT[I.N+Il) - 0.9999999
'ELSE' TeI.N+IJ I- ENTIERCTeI.N+2J) + 0.0000001 - T[I.N+2l;
SATISFIED I - 'FALSE'J
'GOTO' END OF LIMITLOOPJ

PREPARE LLI
'IF' ROWLST[Il > N 'THEN' 'BEGIN'
RI-IJ 'GOTO' REWRITE L CUTJ 'END';
NAME I- ROWLST[Il;
760 CHAPTER XXI

'IF' 'NOT' MAIN 'THEN' 'GoTo' NO SKIPPING;

OVERWRITE OUTSTANDING NON LIMIT CUTI
OVERWRITE SATISFIED LIMIT CUTI
'FOR' III-NAV+I 'STEP' I 'UNTIL' M+NCUTS 'DO'
'IF' RoWLSTCIIl-NAME+2.N
'DR' (RoWLSTCIIl-NAME+N 'AND'
TCII.N+ll> 0 'AND' TCII.N+2l > 0) 'THEN' 'BEGIN'
RI-IIJ 'GoTO' WRITE LL cun 'END';
NO SKIPPINGI
'IF' NCUTS>N+NIRV 'THEN' 'GoTo' END OF LIMITLooPJ
'FOR' III- 3.2 'DO' 'FOR' JJI-I 'STEP' 1 'UNTIL' N+2 'DO'
TCM+NCUT5+II.JJll-TCM+NCUTS+rI-I.JJl;
NCUTSI-NCUTS+IJ RI-M+NCUTSJ
REWR ITE LCUT I
WRITE LL CUTI
'IF' RoWLSTCI)<N+1 'THEN' RoWLSTtRl I- RoWLSTCll+NJ
TCR.N+2ll-TCI.N+2lJ
'IF' TCR.N+2l > Ttl.N+ll
'THEN' TCR.N+lll-TCI.N+ll - 0.9999999 - ENTIERCTtl.N+ll)
'ELSE' TtR.N+ll I- ENTIERCTCI.N+2l) +0.0000001 - TCI.N+2l;
'FOR' JI-I 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'
'IF' TC I.Jl-O
'DR' CCDLLSTCJl > 1000 'AND' CDLLSTCJl < 1001+NEQ)
'THEN' 'BEGIN'
TtR.Jll=OJ 'GoTo' LCUT CoEFF WRITTEN; 'END'J
TCR.Jll-TC I.JH
'IF' TCI.Jl - ENTIERCTCI.Jl) < 0.0000001
'THEN' TCR.Jll-ENTIERCTCI.Jl)J
'IF' TCI.Jl - ENTIER~TCI.Jl) > 0.9999999
'THEN' TCR.Jl I- 1 + ENTIERCTtI.Jl)J
'IF' ABS(TCR.J)-TCI.Jl) < 0.0000001
'THEN' TCI.Jll-TCR.Jl;
'IF' ICLCJ)-O 'DR' ABSCTCI.Jl) < 1 'THEN' TCR.Jll=TCI.Jl
'ELSE' 'BEGIN' TCR.Jll-TCI.J)-ENTIERCTCI.Jl)J
'IF' TCR.J) < 0 'THEN' TCR.Jll- 1 + TCR.JlJ
TCR.N+2) I- TCR.N+2l+ABSCTCR.Jl-TCI.Jl).TCM+NCUTS+2.JJJ
'END' J
LCUT CoEFF WRITTENI 'END'J
SATISFIED I- 'FALSE'J
'GoTo' END OF LIMITLooPJ
BEGIN OF CoMBLooPI
'IF' ROWLSTCll > N 'THEN' 'BEGIN'
RI-IJ 'GoTo' BEGIN OF COMPARINGJ 'END';
'IF' NCUTS-N+I 'THEN' 'GoTo' END OF LIMITLoOPJ
'IF' 'NOT' SATISFIED 'THEN' 'BEGIN'
'FOR' III=NAV+l 'STEP' I 'UNTIL' M+NCUTS 'DO'
'IF' RoWLSTCIll-RoWLSTCll+N
'THEN' 'GoTo' END OF LIMITLooP; 'END'J
BEGIN OF CoMPARINGI
PoSI-NEGI- 'TRUE')
THE USE OF CUTS 761

'FOR' JI .. 1 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'

NAME I" 0;
'IF' C01.1.STCJl < N+l 'THEN' NAME 1=' C01.1.STCJH
'IF' C01.1.STCJl > 10000 'AND' C01.1.STCJl < 10001 +N
'THEN' NAME I" C01.1.STCJl - 10000;

'IF' 'NOT' NAME-O 'THEN' 'BEGIN'

'IF' 'NOT' IC01.1.STCNAMEl-O 'THEN' 'GOTO' RESTRICT
'ELSE' 'GOTO' CHECK SIGN; 'END';

'IF' C01.1.STCJl > 1000 'AND' C01.1.STCJl < 1001+M

'THEN' NAMEI-C01.1.STCJl - 1000;
'IF' C01.1.STCJl > 11000 'AND' C01.1.STCJl < 11001 + M
'THEN' NAME I- C01.1.STCJl - 11000;

'IF' 'NOT' NAME"O 'THEN' 'BEGIN'

'IF' 'NOT' IROW1.STCNAMEl"O 'THEN' 'GOTO' RESTRICT
, ELSE' , GOTO' CHECK SIGN; 'END';
'IF' C01.1.STCJl < 2*N+l
'OR' CC01.1.STCJl>10000+N 'AND' COl.1.STCJl<10001+2*N)
'THEN' 'GOTO' RESTRICTJ

CHECK SIGNI ICl.CJll-O;

'IF' TCIIJJ < 0 'THEN' POSI"'FA1.SE'
'IF' TCIIJJ > 0 'THEN' NEG:='FA1.SE'
'IF' 'NOT' CPOS 'OR' NEG) 'THEN' 'GOTO' END OF 1.IMIT1.0OP;
'GOTO' END OF SIGN CHECK;
RESTRICTI IC1.CJJI-C01.1.STCJJJ
IF NOT INTEGER CLEAN OFF ROUNDING ERRORI
TCM+NCUTS+2IJJI"ENTIERCTCM+NCUTS+2IJJ+O,5);
END OF SIGN CHECKI 'END';

'IF' I-M+NCUTS+2 'THEN' 'GOTO' END OF 1.IMIT1.00P;

'IF' CPOS 'OR' NEG) 'THEN' 'BEGIN'

IF APPROPRIATE OVERWRITE FOR COMBINED CUTI

OVERWITE SATISFIED 1.IMITCUT BY ITS OWN CUT:

'IF' ROW1.STCIl>N 'THEN' 'BEGIN'
RI-IJ 'GOTO' WRITE A COMBINED CUT; 'END';

LOOK FOR AN OLD CUT TO OVERWRITEI

OVERWRITING FOR COMBINATION 1= 'FALSE';
'FOR' IlI"NAV+l 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'IF' ROW1.STCIIl-NAME+2*N
'OR' CROW1.STCIIl-NAME+N 'AND'
TCIIIN+ll > 0 'AND' TCIIIN+2J > 0) 'THEN' 'BEGIN'
OVERWRITING FOR COMBINATION I" 'TRUE';
RI-IU 'GOTO' WRITE A COMBINED cur; 'END';

CHECK SPACE FOR COMBINED CUTI

'IF' MAIN 'AND' NCUTS>N+NIRV
'THEN' 'GOTO' END OF 1.IMIT1.00P;
OVERWRITING FOR COMBINATION I- 'FALSE';
 762 CHAPTER XXI

WRITE A COMBINED CUTa

'IF' NEG 'THEN' POSa='FALSE';

'IF' ROWLSTCIl > N

'OR' OVERWHITING FOR COMBINATION 'THEN' 'GOTO' WRITE;

ENLARGE a
'FOR' lIas 3.2 'DO'
'FOR' Ja-1 'STEP' 1 'UNTIL' N+2 'DO'
TCM+~CUTS+II.Jla.TCM+NCUTS+II-1.Jl;
NCUTSa-NCUTS+1; Ra=M+NCUTS;

WRITEI
'IF' ROWLSTCI1<N+1 'THEN' ROWLSTCRla=ROWLSTCIl+N;
'FOR' J 1= N+1.N+2 'DO' TCR.Jl a= TCI.Jl;
'IF' POS 'THEN' JJa"'N+1 'ELSE' JJa=N+2;
'FOR' JI=l 'STEP' 1 'UNTIL' N 'DO' 'BEGIN'
TCR.Jla"NUMa"TCI.JH 'IF' NUM < 0 'THEN' NUMa=-NUM;
CWI-ENTlER(NUMH
'IF' 'NOT' NUM=O 'AND' 'NOT' ICLCJl=O 'THEN' 'BEGIN'
'IF' NUM - CW > 0.9999999 'THEN' CW := l+CW;
'IF' ABS(NUM-CW) < 0.0000001 'THEN' NUMa=CW;
'I F' TC 1. Jl > 0 'THEN' TC I. JJ a = NUM
'ELSE' TCI.Jl a= -NUM;
'IF' CPOS 'AND' TC I.JJ < 0)
'OR' (NEG 'AND' TCI.JJ > 0)
'THEN' CW: =CW+ 1;
'IF' TCI.JJ < 0 'THEN' NUM := -NUM;
'IF' TCI.JJ < 0 'THEN' CW a= -CW;
TCR.JJJ:=TCR.JJJ+TCM+NCUTS+2.JJ*ABSCCW);
NUM:=NUM-CW; TCR.JJa=NUM; 'END'; 'END';
'IF' POS 'THEN' JJ:=N+2 'ELSE' JJI=N+1;
TCR.JJ1 1= TCR.JJl - ENTIERCTCR.JJ1) -0.999999;
SATISFIED 1= 'FALSE';
END OF COMBINED CUT WRITING: 'END';

END OF LIMITLOOPI 'END';

END OF INTEGERCHECK CUM LIMITI

'IF' 'NOT' MAIN 'THEN' 'GOTO' STAGE 16;

RECALLING TO STAGES 3 AND 4 a= 'FALSE';
END OF STAGE 3a 'END';

INCOMING VARIABLE INTERCEPTI

CORRECT DEGENERATE INFEASIBILITYa
'FOR' Ira1 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'IF' 'NOT' (ROWLSTC 11>1 000 'AND' ROWLSTC 1 J<l 001+NEQ) 'THEN'
, FOR' J a = 1 • 2 'DO' .
'IF' TCI.N+J1 > -0.0001 'AND' TCI.N+J1 < 0.0000000001
'THEN' TCI.N+J11=0.0000000001;
THE USE OF CUTS 763

'IF' NeUTS > N+NIRV-l 'THEN' 'GOTO' STAGE 4;

RI=KI·ROWNI=COLNI=O;
REENTRYI-2;
INTPCT.M+NCUTS.N.NEQ.NAV.ROWLST.COLLST.IROWLST.
R.K.ROWN.COLN.REENTRY);
INCOMING VARIABLE INTERCEPT REENTRY 1
PIVOT SIZE CUT 1= 'FALSE';
'IF' COLN=O 'OR' R=O 'THEN' 'GOTO' STAGE 4;
'IF' COLN<3*N+l 'OR' CCOLN>lOOOO 'AND' COLN<lDOOl+2*N)
'OR' ABSCTCR.KJ»lO 'OR' ABSCTCR.KJ)<O.l
'THEN' 'BEGI N'
'IF'
CICLCKJ=COLLSTCKJ 'AND' TCM+NCUTS+2.Kl<1.001)
'OR' CCOLN>N 'AND' COLN <3*N+l)
'OR' ABSCTCR.KJ»lO 'OR' ABSCTCR.KJ)<O.l
'THEN' 'BEGIN'
'IF' CABSCTCR.KJ)<O.l 'OR' ABSCTCR.KJ»IO)
'AND' 'NOT' NCUTS > N+NIRV
, THEN' , BEG IN'
RECALLING TO STAGES 3 AND 4 1= 'TRUE';
PIVOT SIZE CUT 1= INTERCEPT := 'TRUE';
'GOTO' STAGE B; 'END' ;
'IF' ROWN=ROWLSTCRJ 'THEN' QUO:=TCR.N+IJ/TCR.KJ
'ELSE' QUOI=-TCR.N+2J/TCR.KJ;
'IF' QUO<O.OOI 'OR' QUO>TCM+2.KJ-O.00I
'THEN' 'GOTO' STAGE 4;
SET FOR AUXILIARY CUT:
KKI=K; RECALLING TO STAGES 3 AND 4 1= 'TRUE';
INTERCEPT 1= 'TRUE';
'GOTO' STAGE 10; 'END'; 'END';
STAGE 41

ATTEND DEGENERACY 1
DEGENERATE 1= 'FALSE';
'FOR' 11=1 'STEP' 1 'UNTIL' M+NCUTS 'DO'
, FOR' J I" 1. 2 'DO '
'IF' ABSCTCI.N+JJ)<O.OOOOOOOOOI 'THEN' 'BEGIN'
DEGENERATE 1= 'TRUE'; TCI.N+JJI=O.OOOOOOOOOlJ
'IF' ROWLSTCIJ>1000 'AND' ROWLSTCIJ<1001+NEQ
'THEN' TCI.N+JJ:=-TCI.N+JJJ 'END';
'IF' DEGENERATE 'THEN' 'GOTO' INCOMING VARIABLE INTERCEPT;
TRY A STEP:
'IF' STEP ALREADY KNOWN 'THEN' 'GOTO'
KNOWN STEP REENTRY OF STAGE 4;
R:=K:=ROWN:=COLNI=O;
764 CHAPTER XXI

KNOWN STEP REENTRY OF STAGE 41

STEP ALREADY KNoWNI-'FALSE'J
REENTRY.-U
INTPCT.M+NCUTS.N.NEQ.NAV.ROWLST.CDLLST.
IROWLST.R.K.RoWN.COLN.REENTRY);
'IF' REENTRY· -1 'THEN' 'GoTo' EMPTYJ
'IF' 'NOT' MAIN 'AND' REENTRY· -10
'THEN' 'GOTO' IMPLEMENTATION REENTRY OF STAGE 9J
CUTSHARPER.
'IF' ROWN>N 'AND' RoWN<3*N+1 'THEN' 'BEGIN'
'IF' ROWN>2*N 'THEN' NAMEI-ROWN-2*N
'ELSE' NAMEI-ROWN-N}
'FOR' 1.-1 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'IF' ROWLSTCIl-NAME 'AND'
ABSCENTIERCTCI.N+1l+0.001)-TCI.N+1l)<0.002 'THEN' 'BEGIN'
TCI.N+1l.-ENTIERCTCI.N+1l+0.002)J
TCI.N+2l.·ENTIERCTCI.N+2l+0.S)J
'IF' TCI.N+1l-0 'THEN' TCI.N+1l.-0.0000000001J
'IF' TCI.N+2l-0 'THEN' TCI.N+2ll-0.000000000lJ 'END'}
'END';
'IF' RETURNED 'THEN' 'BEGIN'
RETURNED .- 'FALSE'; 'GOTO' STAGE 3J 'END';
STAGE S.
MAKE LIMITCUTS BINDING WITH PRIORITY:
'FOR' 1.-1 'STEP' 1 'UNTIL' M+NCUTS ,~O'
'IF' ROWLSTCIl>N 'AND' ROWLSTCIl<2*N+1
'AND' CTCI.N+11<0 'DR' TCI.N+2l<O)
'THEN' 'GOTO' STAGE 3;
LOOK FOR SUPPLEMENTARY CUTS ON NON LIMIT CUTS:
'IF' NCUTS<NIRV 'THEN' 'BEGIN'
'FOR' 11-1 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'IF' ROWLSTCIl>2*N 'AND' ROWLSTCll<3*N+1
'AND' (TCI.N+1l<O 'DR' TCI.N+2l<0)
'THEN' 'GOTO' STAGE 9; 'ENO'J
REACT ON CHECK AND LIMIT.
'IF' 'NOT' FEASIBLE 'DR' 'NOT' SATISFIED
'DR' 'NOT' OPTIMAL
'THEN' 'GOTO' STAGE 3;
'IF' OPTIMAL 'AND' FEASIBLE 'AND' INTEGER
'THEN' 'GOTo' END OF MIXI;
STAGE 61
REMOVE AMPLY FULFILLED CUTS:
'FOR' III-NAV+1 'STEP' 1 'UNTIL' M+N+NIRV+1+1 'DO' 'BEGIN'
'IF' I I :> M+NCUTS
'THEN' 'GOTo' TRY IF MORE SPACE WILL DO IT;
 THE USE OF CUTS 765

'IF' ROW1.STCII] < N+1 'OR' ROWLSTCII] > 3*N

'THEN' 'GOTO' END OF ROW REMOVING LOOP;

REMOVE ONLY IF SPACE IS PRESSING:

'IF' NCUTS < N+NIRV 'THEN' 'GOTO' END OF ROW REMOVING LOOP;
'IF' ROWl.STCIIl > S*N 'THEN' 'GOTO' REMOVE NOW;

REMOVE ONLY ONE LIMIT CUT IN FIRST INSTANCE:

'FOR' 11"11+1 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'IF' (ROWl.STCIl>N 'AND' ROWl.STCIl<S*N+1)
'THEN' 'GOTO' END OF ROW REMOVING LOOP)
REMOVE NOWI
NCUTS.-NCUTS-1;
'FOR' II-II 'STEP' 1 'UNTIL' M+NCUTS 'DO'
ROWl.STCIla-ROWl.STCI+1l1
'FOR' II-II 'STEP' 1 'UNTIL' M+NCUTS+S ,~O'
'FOR' Ja-1 'STEP' 1 'UNTIL' N+S ,~O' TCI.Jl:=T[I+1.JlJ
HAVE ROMOVEO ONE CUTa
END OF ROW REMOVING LOOP: 'END';

TRY IF MORE SPACE WILL DO ITa

'IF' NCUTS<NIRV+N-1 'AND' 'NOT' OPTIMAL 'THEN' 'GOTO' STAGE 3;
'IF' 'NOT' OPTIMAL 'THEN' 'GOTO' STAGE 3;

STAGE 7a
CHECK FOR NORMAL END OF ALGORITHM:
'IF' INTEGER 'THEN' 'GOTO' END OF MIXX;

STAGE 61
RRa=NAV+1;
CUTa
CUTNa=O; NUMa-O.0011
'IF' MAIN 'AND' 'NOT' OPTIMAL 'THEN' NUM:=O.OOS;
'FOR' IaeRR 'STEP' 1 'UNTIL' M+NCUTS '~O'
'IF' ROWLSTC I] < N+1 'THEN' 'BEGIN'
'I F' 'NOT' I COl.l.STC ROWLSTC Ill" 0 'THEN' 'BEGIN'
'IF' 'NOT' ABS(ENTIER(TCI.N+1l+0.S*NUM)-TCI.N+1]) < NUM
'THEN' 'BEGIN'
R:=IJ
CUTNa=ROWl.STCR]+S*N;
NUM 1= ABS(ENTIER(TCI.N+1l+0.S*NUM)-TCI.N+ll);
'END'; 'END'; 'END';

'IF' MAIN 'AND' CUTN-O 'THEN' 'BEGIN'

'IF' PIVOT SIZE CUT 'THEN' 'GOTO' STAGE 9
'ELSE' 'GOTO' STAGE 4; 'END';
766 CHAPTER XXI

'IF' NCUTS < N+NIRV 'THEN' 'GOTO' ENLARGE FOR NON LCUT;
'FOR' II-RR 'STEP' I 'UNTIL' M+NCUTS 'DO'
'IF' ROWLSTCI1-CUTN "AND' TCI,N+Il>O 'AND' TCI,N+21>0
'THEN' 'BEGIN' CUTRI~IJ 'GOTO' WRITE THE CUT; 'END';

ENLARGE· FOR NON LCUTI

NCUTS,-NCUTS+1J CUTR,-M+NCUTSJ
'FOR' 1'- 2,1 ,~O' 'FOR' JI-l 'STEP' I 'UNTIL' N+2 'DO'
TCM+NCUTS+I,Jl,-TCM+NCUTS+I-I,Jl;
WRITE THE CUT·,
SATISFIED ,- 'FALSE';
ROWLSTCCUTR11=ROWLSTCR1+2*N;
TCCUTR,N+Il I- -0.99999999J TCCUTR,N+21,-FANCYHIGH;
RR.-RJ
'FOR' •.11-1 'STEP' 'UNTIL' N 'DO' 'BEGIN'
TCCUTR,J11-OJ
'IF' TCR,Jl-O
'OR' CCOLLSTCJl > 1000 'AND' COLLSTCJl < 1001+NEQ)
'THEN' 'GOTO' COEFFICIENT WRITTENJ
NUMI-ABSCTCR.Jl)J
TRY COMBINATION.
'IF' 'NOT' NUM < 0.999999 'AND' 'NOT' ICLCJ1-0 'THEN' 'BEGIN'
NUM,-NUM-IJ 'GOTO' TRY COMBINATIONJ 'END'J
'IF' ABSCNUM) < 0.000002 'THEN' 'GOTO' COEFFICIENT WRITTEN;
QUOI-TCR,N+ll-ENTIERCTCR,N+ll)J
'IF' TCR,Jl < 0 'THEN' QUOI-l-QUO;
QUO I- QUO/NUMi
'IF' 'NOT' ICLCJ1-0 'AND' QUO < I 'THEN' QUO.-U
TCCUTR,Jl,--I/QUO;
COEFFICIENT WRITTEN. 'END';

'IF' PIVOT SIZE CUT 'AND' CUTN-O

'THEN' 'GOTO' STAGE 9J
'IF' PIVOT SIZE CUT
'THEN' 'GOTO' INCOMING VARIABLE INTERCEPT;

'IF' 'NOT' MAIN 'THEN' 'GOTO' STAGE 17;

'IF' RECALLING TO STAGES 3 AND 4 'THEN' 'GOTO' STAGE 4J
'IF' NCUTS-N+NIRV+1 'THEN' 'GOTO' STAGE 31
'IF' NCUTS-N+NIRV 'THEN' 'GOTO' STAGE 9;
IMPLEMENTATION REENTRY OF STAGE 91
STAGE 91
KKI-KKKI-OJ
TRY A HYPOTHETICAL STEP.
KI-KKJ
R,-ROWN.-COLN.=OJ REENTRY,-2J
INTPCT,M+NCUTS.N,NEQ,NAV.ROWLST,COLLST,
IROWLST,R,K,ROWN.COLN,REENTRY)J
SAVED R T ,oo RJ
THE USE OF CUTS 767

'IF' REENTRY - -1 'AND' KK-O 'THEN'

'GOTO' EMPTY OF INTEGER SOLUTIONS;

'IF' R-O 'THEN' 'BEGIN'

'COMMENT'
HITTING THE UPPER LIMIT OF THE INCOMING VARIABLE MEANS
THIS IS A CHEAP STEP;
FEASIBLE 1= SATISFIED I- 'FALSE';
MAIN I - RETURNED 1= 'TRUE';
'GOTO' KNOWN STEP REENTRY OF STAGE 4; 'END';
'IF' PIVOT SIZE CUT 'THEN' 'BEGIN'
PIVOT SIZE CUT 1= 'FALSE';
RRI-RI KKI=K;
RECALLING TO STAGES 3 AND 4 1= 'TRUE';
'GOTO' STAGE 10; 'END';
'IF' CREENTRY = -1 'OR' TCRIN+1l > 10)
'AND' KKIO 'THEN' 'BEGIN'
KKI-OJ 'GOTO' Tny A HYPOTHETICAL STEP; 'END';
'IF' 'NOT' CROWN>2*N 'AND' ROWN<3*N+ 1) 'THEN' 'BEGIN'
'COMMENT'
EXISTING CUTS AT THE MAIN VERTEX ARE NOT BEING REACHED
BY A SINGLE STEP.
UNDER THOSE CIRCUMSTANCESI THERE IS NO POINT IN MAKING
HYPOTHETICAL CUTS

FEASIBLE I- SATISFIED I- 'FALSE';

MAIN I- RETURNED I- 'TRUE';
STEP ALREADY KNOWN 1= 'TRUE';
'GOTO' INCOMING VARIABLE INTERCEPT REENTRY; 'END';

STAGE 101
DNCUTSI"NCUTS;
MAKE COLUMN EXTRACT I
'FOR' 11"1 'STEP' 1 'UNTIL' M+NCUTS-NAV 'DO'
FDLSTCIJI=ROWLSTCI+NAVl;
'FOR' 11,,1 'STEP' 1 'UNTIL' M+NCUTS-NAV+2 'DO' 'BEGIN'
COLEXTRC I I III =TCNAV+ I IKH
'FOR' JI-112 'DO' COLEXTRC Ill+JlI=TCNAV+IIN+JH 'END';
DLSTClll=COLLSTCKJ;

STAGE 111
UPDATE COLEXTRI
SAVED R T 1= R;
RI=ROWNI=O; Kl a l; REENTRYI=I;
INTPCCOLEXTRIM+NCUTS-NAVIIINEQIOIFDLSTIDLSTI
IROWLST1RIKIROWNICOLNIREENTRY);
STAGES 12 AND 131
CUTNI=O; LVI=O;
'FOR' II-I 'STEP' 1 'UNTIL' M+NCUTS-NAV 'DO'
'IF' COLEXTRCII2l :'~ 0 'OR' COLEXTRCII3l < 0
'THEN' 'GOTO' NO SOLUTION IN THIS COLUMN;
768 CHAPTER XXI

'GOTO' CONTINUE WITH COLUMN AS INDICATED;

NO SOLUTION IN THIS COLUMN:
'IF' RECALLING TO STAGES 3 AND 4 'AND' INTERCEPT
'THEN' 'GOTO' CONTINUE WITH COLUMN AS INDICATED;
'IF' KK=KKK 'OR' KKK"O 'THEN' 'BEGIN'
RI-SAVED R T; KI=KK; MAIN:='TRUE';
STEP ALREADY KNOWNI='TRUE';
'GOTO' INCOMING VARIABLE INTERCEPT REENTRY; 'END';
KKKI-KK; KKI=O; 'GOTO' STAGE 9;
CONTINUE WITH COLUMN AS INDICATEDI
CHECK INTEGER NATURE HYP VERTEX:
INITIATE LIMITING VALUE:
LV I- 0;
'FOR' 11=1 'STEP' 'UNTIL' M+NCUTS-NAV 'DO'
'IF' FDLSTC I J < N+l 'THEN' 'BEGIN'
AVOID ZERO:
'IF' COLEXTRCI.l J=O 'THEN' 'GOTO' NEXT ROW;
CHECK WHETHER FRACIONALI
'IF' ABSCENTIERCCOLEXTRCI.2J+0.001) - COLEXTRCI.2J) < 0.002
'THEN' 'GOTO' NEXT ROW;
CHECK WHETHER INTEGER RESTRICTED:
'IF' ICOLLSTCFDLSTCIJJ" 0 'THEN' 'GOTO' NEXT ROW;
PREFERENCE ON INCOMING VARIABLEI
'IF' FDLSTCIJ .. COLLSTCKKJ 'OR' FDLSTCIJ=COLLSTCKKJ-I0000
'THEN' 'BEGIN'
'COMMENT'
A CUT ON THE HYPOTHETICAL INCOMING VARIABLE IS INDICATED.
A ROW EXTRACT OF INITIALLY ONLY ONE ROW IS NEEDED;
CUTRI-NAV+!J
CUTN,-FDLSTC 1 H
'GOTO' STAGE 14; 'END';
DO NOT CUT OVER AGAINI
'FOR' CHECK 1 '" NAV+l 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'IF' ROWLSTCCHECKIJ=FDLSTCIJ+N 'OR' ROWLSTCCHECKIJ
=FDLSTCIJ+2*N 'THEN' 'BEGIN'
'IF' TCCHECKI.N+IJ<O 'OR' TCCHECKI.N+2J<0 'THEN'
'GOTO' NEXT ROW; 'END';
'IF' COLEXTRCI.IJ > 0
'THEN' NLV 1= CCOLEXTRCI.2J-ENTIERCCOLEXTRCI.2J»
/COLEXTRCI.IJ
'ELSE' NLV 1= CCOLEXTRCI.3J-ENTIERCl+COLEXTRCI.3J»
/COLEXTRC I. 1 J;
'IF' NLV>LV 'THEN' 'BEGIN'
CUTN:"'FDLSTCIH CUTRp'!J LVI=NLV; 'END';
NEXT ROW: 'END';
THE USE OF CUTS 769

'IF' 'NOT' CUTN=O 'THEN' 'GOTO' STAGE 14;

'IF' RECALLING TO STAGES 3 AND 4 'AND' INTERCEPT
, THEN' , BEG IN'
RECALLING TO STAGES 3 AND 4 •• INTERCEPT 1= 'FALSE';
'GOTO' STAGE 4; 'END';
'IF' KK=KKK 'THEN' 'GOTO' INCOMING VARIABLE INTERCEPT;
KKK.·KK; KK.=O; 'GOTO' TRY A HYPOTHETICAL STEP;
STAGE 14.
MAKE A ROWEXTRACTI
'FOR' J'''1 'STEP' 1 'UNTIL' N '00'
FDLSTCJJ.=COLLSTCJl;
'FOR' J'-1 'STEP' 1 'UNTIL' N+2 'DO'
ROWEXTRC1.JJ.=TCCUTR.JJ;
'IF' CUTNICOLLSTCKKJ 'AND' CUTNICOLLSTCKKJ-IOOOO
'AND' 'NOT' R=O
, THEN' 'BEG IN'
'COMMENT'
TWO SEPERATE ROWS WILL BE NEEDED;
R"'NAV+R; 'GOTO' COpy; 'END';
MM:=I;
RENAME ROWINDEX TO BE FIRST ROW OF EXTRACT I
RI=\;
'GOTO' COpy ROWNAME;

COPYI MMI"2;
DLSTC2ll-ROWLSTCRll
'FOR' JI-l 'STEP' I 'UNTIL' N+2 'DO' ROWEXTRC2.JlI=TCR.JH
RENAME ROWINDEX TO BE SECOND ROW OF EXTRACT.
R.=2;

COpy ROWNAME'
DLSTC1 JI-ROWLSTCCUTRJ;
COpy LOWER BORDERI
'FOR' 11"1.2.3 'DO' 'FOR' JI"1 'STEP' 1 'UNTIL' N+2 'DO'
ROWEXTRCMM+I.Jll=TCM+DNCUTS+I.JJ;

STAGE lSI
UPDATE ROWEXTR.
KI.KK; COLN.=COLLSTCKKl; REENTRYI=I;
SAVED REI- R;
INTP(ROWEXTR.MM.N.NEQ.O.DLST.FDLST.IROWLST.
R.K.ROWN.COLN.REENTRY);

PUT ROWEXTRACT IN T AND SAVE LEADING ROWS OF TI

'FOR' II-I 'STEP' 1 'UNTIl.' MM+4 'DO'
'FOR' JI-l 'STEP' 1 'UNTIL' N+2 'DO' 'BEGIN'
COPI-TCI.Jl; TCI.JJ.-ROWEXTRCI.Jl;
ROWEXTRC I. J lI"COP; 'END';
770 CHAPTER XX

'FOR' II-I 'STEP' 1 'UNTIL' MM+l 'DO' 'BEGIN'

NAMEI-ROWLSTCIl; ROWLSTCIJI-DLSTCIl;
DLSTC I ll-NAME; 'END';
FDLSTCKll=COLLSTCKJ; COLLSTCKll-ROWN;
MI-MM; NAVI-NCUTS:=O;
MAIN:-'FALSE';
'GOTO' CHECK AND LIMIT;
STAGE 16: 'IF' SATISFIED 'THEN' 'GOTO' STAGE 8;
STAGE 171
PUT TABLEAUSHUFFLE BACKI
'FOR' 11'"1 'STEP' 1 'UNTIL' MM+3 'DO'
'FOR' JI=1 'STEP' 1 'UNTIL' N+2 'DO' 'BEGIN'
COP:=TCI.Jl; TCI.Jl:=ROWEXTRCI.Jl; ROWEXTRCI.Jl:=COP; 'END';

'FOR' 11=1 'STEP' 1 'UNTIL' MM+l 'DO' 'BEGIN'

NAMEI-ROWLSTC I JJ ROWLSTC I ll-DLSTC I lJ DLSTC 1]1 =NAME; 'END';
COLLSTCKll=FDLSTCKl; FDLSTCKll=ROWN;

RESTORE PARAMETERS:
M:=MMM; NAVI-DNAV;
PUT NEW AND CORRECT ORDER PARAMETERS I
MM:=MM+NCUTS; NCUTS:=DNCUTS+NCUTS;
BACKWARD STEP IN ROWEXTR:
R:-SAVED R E; REENTRY:=I;
INTP<ROWEXTR.MM.N.NEQ.O.DLST.FDLST.IROWLST.
R. K. COLN. ROWN·. REENTRY);

MAKE CUT ON INCOMING VARIABLE MORE EXACT:

'IF' ~=1 'AND' MM=2 'THEN' 'BEGIN'
'IF' DLSTC2l=FDLSTCKl+N
'OR' DLSTC2l=FDLSTCKJ+2*N
'OR' DLSTC2l=FDLSTCKl-I0000+N
'OR' DLSTC2l=FDLSTCKl-I0000+2*N
'THEN' ROWEXTRC2.N+ll:=O.OOOOOOOOOlJ 'END';
STAGE 18:
ASSEMBLE WITH MAIN TABLEAU:

'IF' NCUTS=O 'THEN' 'GOTO' ATTEND LIMITCUTS BY COLUMNADJUSTM;

ROWLSTCM+NCUTSll=DLSTCMM);
EXTEND TABLEAU FOR SUBSIDIARY CUT:
'FOR' II=M+NCUTS+3. M+NCUTS+2. M+NCUTS+l 'DO'
'FOR' JI=1 'STEP' 1 'UNTIL' N+2 'DO' TCI.Jl:=TCI-l.Jl;
COpy THE SUBSIDIARY CUTI
'FOR' JI=1 'STEP' 1 'UNTIL' N+2 'DO'
TCM+NCUTS.JJ:=ROWEXTRCMM.JJ;
ATTEND LIMITCUTS BY ~OLUMNADJUSTMI
'FOR' 11=1 'STEP' 1 'UNTIL' M+NCUTS 'DO'
'IF' ROWLSTCIJ<N+l 'AND' ROWLSTCIJ" DLSTCIJ 'THEN' 'BEGIN'
TIlE USE OF CUTS 771

DO NOT ADJUST IF DIFFERENCE COULD BE ROUNDING.

'IF' ABSCTCI.N+ll-ROWEXTRC1.N+ll)
+ABSCTCI.N+2l-ROWEXTRC1.N+2l) < 0.001
'THEN' 'GOTO' ATTEND UPPER LIMITS;
TCI.N+ll.-ROWEXTRC1.N+ll; TCI.N+2J.-ROWEXTRCl.N+2l;
'GOTO' ATTEND UPPER LIMITSJ 'END"

ATTEND UPPER LIMITS.

'FOR' 1.- 1.2 'DO' 'FOR' J.-l 'STEP' 1 'UNTIL' N+2 'DO'
TCM+NCUTS+I.Jl.-ROWEXTRCMM+I.JlJ

'IF' RECALLING TO STAGES 3 AND 4 'THEN' 'BEGIN'

MAIN .- 'TRUE'J INTERCEPT.·'FALSE'J
R. -SAVED R TJ

'GOTO' INCOMING VARIABLE INTERCEPT; 'END"

'GOTO' STAGE 9;

UNBOUNDED.
NEWLINEC 2)J
WRITETEXTC'C'UNBOUNDEOISOLUTIONIOFITHEICONTINUOUSIPROBLEM')');
NEWL INEC 2 )J
WRITETEXTC'C'COLUMN')'); PRINTCCOLN.7.0);
WRITETEXTC 'C'UNBOUNDEO') ')J
NEWLINEC2"
'GOTO' END OF MIXII
EMPTY.
NEWLINEC 2)J
WRITETEXTC'C'EHPTYIPROBLEM')');
NEWLINE(2)J
'IF' 'NOT' SATISFIED 'THEN' 'GOTO' NO INTEGER SOLUTION;
'GOTO' END OF MIXIJ
EMPTY OF INTEGER SOLUTIONS.
NEWLINEC2U
NO INTEGER SOLUTION.
WRITETEXTC'C'PROBLEH EMPTYIOFIINTEGERISOLUTIONS')');

END OF MIXI.

'IF' 'NOT' FEASIBLE 'OR' 'NOT' INTEGER

'THEN' 'GOTO' FINAL END OF MIXII
ORDER FOR FINAL EXIT.
ORDLCT.M+NCUTS.N.2.2.ROWLST.COLLST)J

REMOVE EVENTUALLY SUPERFLUOUS CUTS.

'FOR' 11.-1 'STEP' 1 'UNTIL' M+3*N 'DO' 'BEGIN'
'IF' II>M+NCUTS 'THEN' 'GOTO' FINAL END OF MIXI}
'IF' ROWLSTCIIl > 3*N 'THEN' 'GOTO' FINAL END OF MIXI;
'IF' ROWLSTCIIl < N+l 'THEN' 'GOTO'
END OF SURPLUS ROW REMOVING LOOP;
772 CHAPTER XXI

NCUTSI=NCUTS-1J
'FOR' 11=11 'STEP' 1 "JNTIL' M+NCUTS 'DO'
ROWLSTCI11=RDWLSTCI+11J
'FOR' JI=l 'STEP' 1 'UNTIL' N+2 'DO'
'FOR' II-II 'STEP' 1 'UNTIL' M+NCUTS+2 'DO'
TCI~J11·TCI+1~J1J
111=11-1;
END OF SURPLUS ROW REMOVING LOOPI 'END';
FINAL END OF MIXII 'END';
REFERENCES

[1] Abadie, J. (Editor)

Nonlinear Programming
Amsterdam, North Holland Publishing Co., 1967
Referred to: This bibliography [3J, [40].

(2) Baumol, W.J.

Economic Theory and Operations Analysis
Prentice Hall, !-.IewJersey, U.S.A., 1951, 1965, 1972
Referred to: 11.2
[3] Beale, E.M.L.
An Introduction to Beale's Method of Quadratic Programming
In: Abadie (Editor) [1], 1967, pp. 143-150
Referred to: 16.11

[4] Charnes, A.
Optimality and Degeneracy in Linear Programming.
Econometrica, 20 (1952), pp. 160-170
Referred to: 8.10

[5] Charnes, A., Cooper, W.W., and Henderson, A.

An Introduction to Linear Programming.
New York, John Wiley, 1953, 1958
Referred to: 7.1, 8.10

[6] Cottle, R.W.

The Principal Pivoting Method of Quadratic Programming.
In:
Dantzig, G. B., and Veinott, A. F. (Editors)
Mathematics of the Decision Sciences.
(Proceedings of the 1967 Summer School)
American Mathematical Society, 1968
Referred to: 16.2, 16.3

773
774 REFERENCES

[7] Dakin, R.J.

A Tree-Search Algorithm for Mixed Integer Programming
Problems.
The Computer Journal, 8 (Apr 1965), pp.250-255
Referred to: 20.2, 20.3

[8] Dantzig, G.B.

Linear Programming and Extensions.
Princeton University Press, Princeton, U.S.A., 1963
Referred to: 11.3,13.1,16.2

[9] Dantzig, G., Eisenberg, A. and Cottle, R.W.

Symmetric Dual Nonlinear Programs
Operations Research Center, University of California
Researcg Report No. 38
Subsequently published in:
Pacific Journal of Mathematics ~, 1965, pp.809-8l2

[101 Fiacco, A.V., and McCormick, r..P.

Non-Linear Programming; Sequential Unconstrained
Maximization Techniques.
New York, Wiley, 1968.
Referred toc 18.2

[11] Garvin, W.W.

Introduction to Linear Programming.
New York, McGraw Hill, 1960
Referred to: 8.1,9.5,11.2,13.1

[12] Gass, S.l., and Saati, T.

Parametric Objective Function.
Journal of t.he Operations Research S-ociety of America, 2
(l9541, pp.3l6-3l9
Referred to: 13.1

[13] Gass, S.l., and Saati, T.

The Computadonal Algorithm for the Parametric Objective
Function.
Naval Research Logistics. Quarterly, 2 (19551, pp. 39."."5
Referred to! 13.1

[14] Geary, R.C.

Elements of Linear Programmin;g.With Economic Applications
Griffins Statistical Monographs and Courses Nr. 15
Griffin, London 1973
Referred to: 8,10
REFERENCES 775

[15] Gomory, R.
An Algorithm for Integer Solutions to Linear Problems
In:
Graves, R.L., and Wolfe, P. (Editors)
Recent Advances in Mathematical Programming.
McGraw Hill, 1963
Referred to: 21.1

[16] Gomory, R.
An Algorithm for the Mixed Integer Problem.
Rand Corporation p. 1885, Santa Monica, Cal. U.S.A.,
June 1960
Referred to: 21.1

[17] Goncalves, A.
Primal-Dual and Parametric Methods in Mathematical
Programming.
Thesis, Ph.D., Birmingham, Dept. of Mathematical Statistics,
1970.
Referred to: 17.6

[18] Heesterman, A.R.G.

Allocation Models and their Use in Economic Planning.
Reide 1, Dordrecht, Netherlands, 1971
Referred to: 8.10

[19] Heesterman, A.R.G.

Forecasting Models for National Economic Planning.
Reidel, Dordrecht, Netherlands, 1970, 1972
Referred to: 4.1

[20] Heesterman, A.R.G.

Special Simplex Algorithm for Multi-Sector Problems.
Numerische Mathematik 12 (1968) pp. 288-306 (In English)
Referred to: 8.2, l3.Z-

[21] Hahn, F.
Elementary Matrix Algebra.
The Macmillan Company, New York,
& Collier Macmillan, London, U.K., and Toronto, Canada 1973
Referred to: 14.4

[22] John, F.
Extremum Properties with Inequalities as Subsidiary
Condi tions.
(Presented to R. Courant)
New York, Interscience Publishers, 1968.
Referred to: 15.3
776 REFERENCES

[23] Kim, C.
Introduction to Linear Programming.
New York, Rinehart and Winston, 1971
Referred to: 11.3

[24] Kuhn, R.W., and Tucker, A.W.

Nonlinear Programming.
Tn:
Se.cond Berkeley SY.lllPosium on Mathematics. and Probability
Theory.
Univers.ity of California Press, Berkeley, Calif., U. S .A. ,
1951, pp. 481-492
Also reprinted in:
Ne~an, P. (Editor)
Reading& in Mathematical Economics.
The Johns Hopkins Press, Baltimore, U.S.A., 1968
Vol. 1, pp.3-l4.
Referred to: 15.3

[25] La Grange (de la Grange)

Mechanique Analytique (Section Quatri~e)
Paris, Veuve Desaint, 1788
Reprinted in:
Oeuvres de la Grange
Paris, Gauthier et fils, 1888 (Tome XL)
Referred to: 15.3

[26] Land, A.R., and Doig, A.G.

An Automatic Method for Solving Discrete Programming Problems.
Econometrica 28, pp.497-520, 1960
Referred to: :20.2, 20.3

°
[27] .Mangasarin, .L.
Nonlinear Programming.
New York, McGraw Hill, 1969
Referred to: 14.2
[28] Maurer, S.
Pivotal Theory of Determinants.
In:
Balinski, .M.L.
Pivoting and Extensions: In Honour of A.W. Tucker.
Amsterdam, North Holland Publishing Co., 1974
Referred to: 5.5

[29] Parlett, B.N.

The Symmetric Eigenvalue Problem.
Prentice Hall, Englewood Cliffs, New Jersey, U.S.A., 1980
Referred to: 14.4
REFERENCES 777

[30] Ponstein, J.
Seven Types of Convexity.
Siam Review g (1967), pp.115-ll9
Referred to:- 14.2

[31] Powell, M.J.D.

A Fast Algorithm for Nonlinearly Constrained Optimization
Calculations.
In:
Numerical Analysis
Conference, Dundee, 1977, Siam
Springer Verlag
Referre.d to: 18.2

[32] Samuels.on, P.A.

Foundation~ of Economic Analysis.
Harvard University Press, Cambridge, Mass., U.S.A., 1967
Referred to: 15.5

[ 33 J Theil, H.
Principles of Econometrics.
New York, John Wiley & Amsterdam, N.Holland Pubg. Co., 1971.
Referre.d to: 14.4

[34] Tucker, A.W.

Dual Sys.tems of Homogeneous Linear Inequalities.
In:
Kuhn, H.W.. , and Tucker, A.W. (Editors}
Linear Inequalities and Related Systems.
Princeton Uniyersity Press, New Jersey, U.S.A., 1965
Referred to: 11.3

[35] Van de Panne, C.

Methods of Linear and Quadratic Programming.
Amsterdam, North Holland Publi.s.hing Co., 19]5
Re.ferred to: 8.10,17.6, lJ.6

[36] Van de Panne, C., and Whinston, A.

The Simplex and Dual Met.hod for Quadratic Programming.
Operational Research Quarterly, .!2, Nr. 4, pp. 355-38]
Referred to: 16.2, 16.3

[37] Van de Panne, C., and Whinston, A.

Simplicial Methods in Quadratic Programming.
Naval Res.earch Logistics Quarterly, 2 (1964) pp. 2]3-302
Referred to: 16.3
77 8 REFERENCES

[38] Weingartner, M.
Mathematical Programming and the Analysis of Capital
Budgeting Problems.
Prentice Hall, Englewood Cliffs, New Jersey, U.S.A., 1963
Referred to: 20.1

[39] Westphal, L.E.

Planning Investment Decisions with Economies of Scale.
North Holland Publishing Co., 1971
Referred to: 19.2

[40] Whinston, A.
Some Implications of the Conjugate Function Theory
to Duality.
In Abadie [1], 1967, pp.77-96
Referred to: 15.6

[41] Zionts, S.
Linear and Integer Programming.
Prentice Hall, Englewood Cliffs, New Jersey, U.S.A., 1974
Referred to: 21.1

[42] Zoutendijk, G.
Mathematical Programming Methods.
Amsterdam, North Holland Publishing Co., 1975
Referred to: 8.10
 I N D E X

Accuracy 166
addition of matrices 9
of several matrices 18
additive property of restrictions 363
adjoint, all-integer elimination and 101
aggregate restriction 370, 373, 586
aggregation matrix 63
ALGOL 60 ix
all integer programming problem 637
amended convex mode operation 473, 480
ample fulfillment of restriction 148
anticonvex function, constrained maximum of 358
restriction 320, 587
approximation, initial, inadequate 562
loose 576
overtight 564
tangential 583, 590
artificial right-hand side 546
variables 181, 199
Ay = Bx system 40

Badname restriction 197, 238

row 197
variable 412
basic variable 150
basis matrix 151, 161
inverse of 162
basis variables, free 249
binding restriction, linear 570
in Lagrangeans 378

779
780 INDEX

parametric variation and 567

block-colwnn 21
blocked incoming variable see bounded incoming variable
block elimination 53
equations 33
inversion 54
pivot 151
pivoting 54
with inequalities 223
block-row 21
boundary point 140, 142
bounded incoming variable 458
boundedness and artificial feasibility 458
branch, definition 656
end 673
higher 668
lower 668
main, higher and lower 669
branching algorithm commentary 678-689
method, Dakin's 689
Land and Doig's 690
methods 656
mixed-integer programming and 668
procedure, recursive 690
restriction 668

Canonical form 149, 151

characteristic equation 341
characteristic vector 341
circle 134
code (in branching problem) 661
lowest 661
recorded 661
wipe out of 661
cofactors 69
column of matrix 5
combination of rows 33
of vectors 124
combined restrictions, tangential equivalent of 365
complementarity rule 414
complementary slackness condition 369, 370
complex roots in quadratic programming 568, 573
composite matrix 22
composite vector 22
computational requirement, product-form inverse, revised simplex
algorithm 272
computer handling of large matrices 29
computer efficiency, degeneracy and 168
conservative smallest quotient rule 191
consistency and optimality 588
INDEX 781

constrained maximum of anticonvex function 358

second-order conditions 404
constrained problem 586
constraint qualification condition 369
continuous optimum 638
continuous problem, feasible solution 638
continuous variable 637
convergency 166
convexity
of Lagrangean 388
of parametric variation 287
of quadratic function 339
convex
complication 125
mode operation 473
primal boundedness 470
programming problems 320
restriction 320
set 124, 125
transition 437
coordinate plane 45
spaces 115
system, secondary 137
correcting dual adjustment 602
correction equation 592
correction restriction 601
corresponding continuous problem (in integer programming) 638
costs, fixed 638
Cottle's algorithm 482
cubic function 133
cubic integer programming 645
cuts 702
augmented 702, 709
classes- of 714
combined 702, 711
elementary 702, 706
integer-value 704
limit 706
lower limit 707
main 717
on variables 705
pre liminary 71 7
priority rules 714
subsidiary 717
upper limit
cutting algorithm summary 741

Decision variables 638

decomposition of a determinant 82
definite matrix and diagonal pivoting 352
782 INDEX

subspace and partial inversion of 350

definiteness and topology of feasible spaces 319
degeneracy 164
dual 240
problems of 166
resolution of 169
determinantal equation (see also characteristic equation) 341
determinants 68
calculation of 87
decomposition of 82
of structured matrices 91
permutation of 75
product of two square matrices 108
diagonal matrix 20
diagonal pivoting and definite matrices 352
diagonal vector 19
differentiation of inverses 58
of matrix expressions 25
directional convexity 323, 325
displaced and constrained problem 586
displaced problem 586
distinguished variable see badname variable 412
division, multiplication instead of 48
domain 324
downward adjustment 602
driving variable 413
parameter theorem 449
parametric equivalence method 544
dual degeneracy 240
dual feasibility, computational advantage 236
duality 223
nonlinear 397
theorem 228
application of 233
dual parametric step 291
dual problem 230
ratio 237
requirement condition 370
simplex method 234
variables see also shadow prices 368, 377
as indicators of change in object function 379
as leaving variable 458
elimination of 447
in parametric variation 532
economic interpretation 380
upward adjustment of 591
dummy variable 641

Efficient row selection rule 251

eigenvalue of a matrix 341
INDEX 783

element of a matrix 5
elimination 34
all-integer adjoint and 101
all-integer method 48
computerized procedure 37
emptiness 547
in nonconvex problems 575
empty problem 184, 659
equation without nonnegativity restriction 486
equations 207, 238
simultaneous linear 3
examples 10, 11, 12, 13, 14, 16, 17, 19, 20, 21, 22, 23, 24, 27,
34, 35, 37, 38, 40, 43, 44, 47, 48, 49, 51, 55, 56, 59, 62,
66, 68, 73, 82, 83, 85, 94, 95, 96, 103, 111, 117, 118, 120,
123, 127, 130, 134, 138, 145, 154, 165, 177, 179, 182, 188,
189, 202, 205, 208, 209, 210, 217, 218, 231, 232, 235, 240,
268, 269, 270, 271, 276, 287, 290, 293, 295, 304, 319, 321,
322, 332, 335, 340, 343, 344, 349, 353, 354, 370, 374, 375,
379, 383, 386, 392, 395, 404, 408, 411, 429, 439, 443, 4S8,
459, 462, 468, 476, 480, 487, 489, 492, 495, 526, 531, 533,
542, 546, 556, 561, 563, 567, 568, 574, 576, 583, 591, 594,
595, 599, 600, 612, 637, 639, 640, 641, 669, 703, 711, 713,
739
exercises 11, 13, 18, 25, 29, 41, 47, 59, 67, 75, 82, 86, 87, 99,
100(2),105,111,116,139,159,194,204,217,240,289,
334,355(2),382,396,415(2),419,432,447,455,473,525,
530, 613 (answer 633)
extrema of quadratic function 339
extreme point 140

Factorization of semidefinite matrix 356

feasible area 148
corners of 150
feasible solution 144, 153
search for 181, 250
column selection in 237
free variable entering 249
feasible space area
end bounding by 535
topology of 319
feasibility, artificial, boundedness and 458
fixed costs 638
Fortran, limitations of, in L.P. 243
free variable (= unconstrained variable) 145
full exhaustion 659
fully explored problem 673
subbranch 673
function 116
argument of 116
linear, graph of 117
784 INDEX

value 116
further out subbranch problem 673

Half space 128

head problem 673
highest step rule 163, 251
hyperbola 136
hypothetical step 717

Indefinite matrix 341

roots 344
indifferent restriction 141
inequalities,' block pivoting with 223
infeasible starting solution 473
ill-conditioned systems 34
immediate neighbour convexity transmission 439
improper vertex 537
imputed prices ~ dual variables
index, increasing order of, branching in 656
integer optimum 638
integer programming V111, 636
applications 637
terminology 637
integer requirement 638
integer-restricted variable 637
integer solution 638
interior point 139
inverse matrix 41
definition 43
differentiation of 58
inversion and reduction 47
by row operators 84
of matrices 33
of recursive products 56
of transposes 56
partial, of definite matrices 350
row permutation during 51
investment cos·ts 640
inward adjustment 584, 593
Kuhn-Tucker theorem 368

Lagrangeans 368
convexity of, in coordinate space 388
latent root of a matrix 341
leading variable 671
linear approximation 559
of cubic function 642
relaxation of 568
linear programming vii
concepts 144
INDEX 785

described procedure for 249

graphical solution 146
matrix notation 145
on computer 242
parametric variation 273
program text 264
linear restrictions, quadratic restrictions with 402
linear subspaces 118
linear transform of coordinates 137
lower bounds 205, 217
in quadratic programming 495

Mathematical programming, general 318

problem 319
matrices, computer handling of large 29
definitions and conventions 5
notation, purpose 5
operations vii
scalar product with 14
square 6
symmetric 7
vector product with 11
maximizing algorithm 181
meaningful boundedness 470
minors 68
minor of 70
mixed integer programming problem 637
mixed systems 205
parametric adjustment of 293
most negative element 152
multiplication by partitioning 22
instead of division 48
of matrices 10

Name codes 242

ordering convention 244
name lists 242
re-ordering conventions 244
negative definite matrix 341
roots 344
negative diagonal 420
pivot 437
negative definiteness 425
negative semidefinite matrix 341
roots 344
node index 67C
nonadmissible restrictions 148
nonconvex mode operation 473, 474
problem 575, 600
programming problems 320
restriction 567, 641
786 INDEX

set 125
nonlinear duality 397
programming viii
nonfeasible solution 144
nonnegativity restriction 486
nonstandard form block 433
tableau 433
non unique subsidiary optimum 599
normal transition under optimality 591

Objective function 144

limit 585
linear component, parametric variation of 525
parametric variation of 289
strictly convex 387
value, step length and 456
opening problem 273
operator 20, 62
optimal form condition 585
correction of 591, 601
loss of 236, 569
loss and correction of 594
subdominant 597
total 598
optimality 363
computational advantage of 236
condition 370
consistency and 588
without exhaustion 659
optimal solution 144
optimum solution 158
order parameter of a matrix 6
outward adjustment 569
outward point 140, 142
overheads see fixed costs 639
overstatement of dual variable 595
by misidentification 595
simple 595
overtight approximation 564

Partitioning, multiplication by 22
of a vector 22
of matrices 21
parameter subspace, strict convexity in 530
parameter theorems 447
parameter treatment as variable 298
parametric equivalence 537
driving variable method 544
linear programming, computer implementation 306
methods in quadratic programming 516
INDEX 787

Quasiconvexity see directional convexity

Rank 68
full 92
of structured matrices 91
reapproximation 576
adjusted 583, 585
inwardly adjusted 583
reentry tableau for 579
recursive product 17
inversion of 56
references 773ff.
related problem 200
relation 116
restriction 124, 150
additive property of 363
nonconvex 641
nontrivial 124
polynomial, segmentation of 646
specified 638
revised simplex algorithm 267
explicit inverse without row updating 269
with row updating 267
product form inverse 270
row of a matrix 5
operators, inversion by 84
permutation during inversion 51

Scalar 7
matrix product with 14
secondary reentry column 720
second order condition, constrained 383
second order effect, displaced optimum
segmentation of polynomial restrictions 646
semidefinite matrix factorization 356
sensitivity analysis 273
sequentially constrained maximization algorithm 560, 601
adaptations of 609-614
discussion 604-608
set, convex 124
of vectors 124
shadowprice of variable 159
sign inversion rule 418
simplex algorithm 144
for quadratic programming 410
outline 149
revised 267 see also revised simplex algorithm
simplex step 156
tableau 151
matrix notation 160
788 INDEX

naming of rows and columns 242

nonupdated computerized 267
ordering of 244
packed storage of 267
printing 258
reordering convention 244
shortened 162
vector spaces and 171
singularity 33
slack variable 145
elimination 208
smallest quotient rule 153, 190, 250
conservative 190
solution vector 33
verification 574
specified restrictions 638
spuriously unbounded variable column 459
square matrix 19
standard form double step 466
steepest ascent principle 152, 197
step length and objective function value 456
strict convexity in parameter subspace 530
strictly anticonvex function 323
strictly convex function 323
quadratic 348
strict subspace convexity 406
suboptimality 659
suboptimal subbranch problem 673
subspace convexity viii, 383
subspace, Euclidean 122
linear 118
of definite matrices 350
substitute objective function 181
nonupdating of 202
substitution, algebraic, in matrix notation 15
subtraction of matrices 9
subvector 22
sum count column 39
summation vector 62
superimposition of approximations 576, 580
blocked 582
full 582
unblocked 582

Tableau
calculation of currently updated 577
larger subspace predecessor 433
neighbouring standard form 433
nonstandard form 436
and standard 433
INDEX 789

(nonstandard form)
predecessor 441
normal sucessor 535
quadratic programming, ordering of 482
smaller subspace predecessor 433, 436
successor 433
use of 4
tangential approximation 335
equivalent 336
of combined restrictions 365
subspace 334
theorems 22, 45, 46, 66, 70, 73, 75, 76, 77, 78, 79, 81, 92, 96,
98,108,119,121,125,126,141,173, 228ff.(Duality), 327,
338, 341, 345, 346, 350, 352, 356, 358, 365, 369 (Kuhn-Tucker),
390(Corner), 406, 408, 422, 425, 430, 432, 434(Nonstandard
form block nonsingularity), 436(Smaller subspace immediate
neighbour convexity transmission), 438,440, 442(compliment-
ary pair), 446(Smaller subspace complementary pair transit-
ion), 449 (Driving variable parameter), 451 (Weak badname
variable parameter), 453 (Primal badname convexity), 460, 470
(Convex primal boundedness), 586, 598, 721
topology of feasible space, definiteness and 319
transformed objective function 159
transpose of a matrix 8
inversion of 56
triangular matrix 21
lower 21
upper 21
two-value columns 209
type absolute variable 145

Unbounded column 155

incoming variable 459
unbounded problem 155, 659
unit vector 20
un upda ting 719
upper bounds 205, 209

Van de Panne's algorithm 482, 558

variable, continuous 637
decision 638
dependant 116
dummy 641
explanatory 116
integer-restricted 637
leading 671
parameter treatment as 298
without nonnegativity restriction 486
without Sign restriction 205
zero-one 638
790 INDEX

vector 115, 118

column 6
combination of 124
composite 22
matrix product with 11
parametric variation 274
parti tioning 22
permutation 64
proportionality 78
row 6
solution 33
spaces, simplex tableau and 171
summation 62
unit 20
-vector multiplication
verification of subsidiary optima 562

Weak badname variable parameter 451

Whinston/Cottle algorithm 558

Zero-one variable 638

mixed integer problem 656