Nonlinear matrix algebra and engineering applications.

Part 1 : Theory and linear form matrix

C. L. Wu (*) a n d R. J. Adler (**)

ABSTRACT
A matrix v e c t o r formalism is d e v e l o p e d for systematizing the m a n i p u l a t i o n o f sets o f non-
linear algebraic e q u a t i o n s . In this formalism all m a n i p u l a t i o n s are p e r f o r m e d b y multiplication
with specially c o n s t r u c t e d t r a n s f o r m a t i o n matrices. For m a n y i m p o r t a n t classes o f nonlinear-
ities, algorithms based on this formalism are p r e s e n t e d for rearranging a set o f e q u a t i o n s so
that their solution m a y be o b t a i n e d b y numerically searching along a single variable. T h e o r y
developed proves that all solutions are obtained.

1. INTRODUCTION of a problem. Often specific numerical procedures

The problem of solving sets of simultaneous, non- must be designed for each new problem on a "cut
linear algebraic equations by manipulation is consider- and try" basis.
ed in this paper. Simultaneous nonlinear algebraic The introduction of matrix vector techniques makes
equations arise naturally in many fields of engineer- it possible to treat four or five simultaneous equa-
ing and science, both as an original expression of a tions by hand in a period of a few hours. Algorithms
physical problem and as a finite difference approxim- based on these same techniques can treat any number
ation t.o differential equations. of equations, but the practical implementation of
A matrix vector formalism is developed for system- these algorithms for large sets of equations depends
atically manipulating nonlinear algebraic equations upon the future development of computers to per-
and eliminating variables. This formalism consists of form the symbolic manipulations, that is, to perform
expressing the problem in matrix vector notation, the tedious algebra.
and performing operations with specially constructed For broad classes of problems, the final results of
transformation matrices. A vital organization is applying these matrix vector techniqfies is a single
brought into manipulations in this manner, and the equation in a single variable. This single equation is
theory can be compactly stated. The gain and loss then solved numerically. In certain instances, matrix
of solution sets is controlled. vector techniques permit the transformation of a set
Mathematicians concerned themselves with the of equations into another form from which the solu-
manipulative solution of simultaneous nonlinear tion sets are easily obtained without further manipul-
algebraic equations in the latter half of the nine- ation. In general, the techniques presented are most
teenth century, but usually worked with only two suited to the treatment of equations which contain
equations in two variables due to the tedious nature only multinomial terms, that is, terms of the form
of the manipulations involved. Various references n
(1, 2, 4, 5) describe procedures such as Bezout's K 7r x~ ~
method (2), Sylvester's determinant (1), etc., which i=1
were for the most part developed by 1900. These where
methods were rather tedious and unrelated due to
the lack of a general framework such as provided by K is a constant
this paper. Since about 1900 activity in this area o.i are integers
has been at a minimum. More recently, the develop- x i are variables
ment of the high speed digital computer and numer-
ical methods has made numerical iterative procedures Nonlinearities in the form of transcendental functions
the popular method of solving nonlinear equations. may sometimes be handled, depending upon the
However, iterative numerical procedures leave some- nature of the problem. The presence of one variable
thing to be desired. They sometimes have difficulty as the argument of any number of transcendental
in converging to a solution. Perhaps more serious is functions can always be handled easily.
the difficulty of locating all solutions, when several There are three important concepts upon which every-
are present, and unusual solutions such as a contin- thing rests. First, that matrix vector notation is a
uous arc or region. In short, numerical methods give generalizing concept which systematizes and simplifies
little insight into, and understanding of, the character the manipulation of equations. Second, that all

(*) T e x a c o Research L a b o r a t o r y , Beacon, N . Y .

(**) Chemical Engineering D e p a r t m e n t , Case Institute o f T e c h n o l o g y , Cleveland, Ohio.

Journal of Computational and Applied Mathematics, volume I, no 1, 1975. 25

manipulations are performed by multiplications [fij] [xjl + [ c i l = 0 (2-1c)
with specially constructed transformation matrices•
Third, that the gain and loss o f solutions depends where
only upon the nature o f the transformation matrices i=1,2 .... n
used. j=1,2 .... n

2. CLASSIFICATION AND REPRESENTATION Polynomial form equations

Quite often a set o f n nonlinear equations in n un- m m-1
knowns can be represented in one or more of the g11xk + g12xk +..-+glmXk+d 1 =0
following three forms. m m-1
g21xk + g22xk + . . . + g2mXk + d 2 = 0
Linear Form
n
I~ fijxj + Ci = 0 i = 1, 2 . . . . , n (2-1a)
j=l m m-1
g n l x k + gn2Xk +'"+gnmXk + dn=0
where
(2-2h)
xj (j - 1, 2 . . . . , n) are unknowns
have the representation
Ci (i = 1, 2 . . . . . n) and fij are either constants
[gij] [Xkn + l - j ] + [di] = 0 (2-2c)
or functions of one or more o f the unknowns,
restricted only in that C i and fij are contin-
uous functions. where
i=1,2 ..... n
Polynomial Form j=1,2 ..... m
m m+l-j FinaUy, group form equations
j = l gijxk + di = 0 i=1,2 ..... k..... n
(2-2a) f l l h l + f12h2 + . . . + f l ~ h £ + c 1 = 0
where
f21hl + f22h2 + •.. + f2£h£ + c 2 = 0
x k is one of the n unknowns
(2-3h)
d i (i = 1, 2 . . . . , n) and gij are defined in the
same way as fij with the additional restriction f lhl + fn2h2 + . . . + fn~h~ + c n = 0
that d i and gij are not functions o f x k.
have the representation
Group Form
[fijl [hjl + [cil = 0 (2-3c)
Z fijhj + C i = 0 i = 1, 2 . . . . . n (2-3a) where
j=l
i= 1,2,...,n
where
j=1,2 ..... £
hj (j = 1, 2 . . . . . £) are any functional groups o f
Equations (2-1c), (2-2c) and (2-3c) are all o f the
one or more unknowns
form
Ci (i = 1, 2 . . . . . n) and f~ are as defined in equa-
BX + C = 0. (2-4)
tion (2-1a).
Definition
Each o f these forms is useful for attacking different
The matrix B is called a coefficient matrix. The
types o f equations. Linear form is useful for simul-
matrix formed by joining the column C to the
taneously eliminating linear variables; polynomial form,
columns of B by bordering on the right, is called an
for eliminating polynomial variables, and group form,
augmented matrix A.
for eliminating groups o f variables. The formal opera-
All A and B matrices may be classified on the basis
tions associated with each o f these three forms are
o f the number o f variables contained in them. It is
discussed separately in a series of several papers.
convenient to call a matrix a function o f those var-
Matrix:vector notation is well suited to representing
iables which appear in the elements o f that matrix.
and treating these three forms. Linear form equa-
Throughout this paper, unless specifically mentioned
tions
otherwise, aid elements contained in matrices are
t711Xl + f 1 2 x 2 + " " + f l n X n + Cl = 0' restricted to continuous functions.
f21xl + f22x2 +"" + f2nXn + c2 = 0 3. EQUIVALENCE OF EQUATIONS
In order to solve a set o f nonlinear equations it is
(2-1b) often desirable to manipulate or transform them into
f n l x l + fn2X2 + " " + fnnXn + Cn = 0 a more convenient form. These manipulations may
introduce or delete solution sets. It is, therefore, of
have the matrix-vector representation great interest to consider a very useful class of trans-

Journal o f Computational and Applied Mathematics, volume I, no 1, 1975. 26

formations and the conditions under which these notation
transformations add or delete solution sets. This
F 1 = 0 D F2 = 0
leads naturally to the notion of equivalent, subordin- If=O]
ate, or d o m i n a n t equations.
is defined to mean that F 2 = 0 is subordinate to
Before proceeding to an examination o f these trans-
formations, several definitions are given which make F1 = 0 and if any missing solution sets exist, they
the Theorems given later in this section more precise must satisfy f = 0, where f is a continuous function.
and succinct. Definition
A n y function which is zero f o r all sets o f values o f
Definition its variables is said to be identically zero.
An equation vector F = [fl ] is an nxl matrix re-
presenting the left hand side o f any set o f n non- Definition
linear equations containing n variables. A transformation matrix P _~ [Pij ] is any nxn matrix
whose elements Pij are either constants or continuous
fl = 0
functions o f one or m o r e variables such that IP[ is
f2-_ o not identically zero.

THEOREM 3-1
fn=0 I f (1) P is a transformation matrix
(2) F is an equation vector
Definition then
A set o f equations F 2 -- 0 is said to be equivalent F=0 C PF~_0
(N) to a set o f equations F 1 = 0 if and only if the [IPI=0]
solution sets o f F 2 = 0 are identical with the solu-
tion sets o f F 1 = 0. In symbolic notation
Proof
The p r o o f consists o f two parts. First it is proved
F 1 = 0 ~ F 2 ~ 0. that PF = 0 is d o m i n a n t t o F = 0. Second it is prov-
ed that any additional solution sets, if t h e y exist,
Definition must satisfy I PI = 0.
A set o f equations F 2 = 0 is said to be subordinate Clearly every solution set o f F _- 0 is also a solution
(D) to a set o f equations F 1 = 0, if and only if the set o f PF = 0; thus, PF = 0 is d o m i n a n t to F = 0.
solution sets o f F 2 = 0 are a subset o f the solution If there exists any additional solution set, i. e., a
solution set which satisfies PF = 0 but which does
sets of F 1 = 0 . In symbolic notation not satisfy F = 0, let it be d e n o t e d b y 7- Substitut-
F 1 = 0 D F 2 -__ 0. ing 7 into PF = 0 and formally using Cramer's rule
yields
Definition IP ( ' / ) I F j ( 7 ) = 0 j = 1, 2 , . . . , n
A set o f equations F 2 = 0 is said~to be dominant
(C) to a set o f equations F 1 = 0, if and only if the As IP('/)[ =/=0, one has a system o f homogeneous
equations with a solution different from null vector
solution sets o f F 1 = 0 are a subset o f the solution
0. Hence IP(~/)] - 0.
sets of F 2 = 0. In symbolic notation

FI=0CF2=0. Corollary
If (1) P is a transformation matrix
Definition (2) F is an equation vector
If a set o f equations F 2 = 0 is d o m i n a n t to a set o f (3) IPI ~ 0 for any set o f values o f the variables
equations F 1 = 0, any solution sets o f F 2 = 0 which contained in P,
are not solution sets o f F 1 = 0 are called additional then
solution sets (with respect to F 1 = 0). The notation F=0~PF=0
A n y transformation matrix P has a formal inverse
F1 = 0 C F2 = 0
p - 1 with the property
If=01
is defined to mean that F 2 = 0 is d o m i n a n t to pp-1 = p-1p = I
F 1 = 0 and if any additional solution sets exist, where I is the unit matrix. The construction o f p - 1
they must satisfy f = O, where f is a continuous 'is the same whether P contains elements which are
function. constants or functions. The inverse o f any transform-
ation matrix P can be always constructed w i t h o u t
Definition
assigning a set o f values to the variables contained
If a set o f equations F 2 = 0 is subordinate to a set
inP.
o f equations F 1 = 0, any solution sets o f F 1 = 0 There m a y exist sets o f values for the variables con-
which are n o t solution sets o f F 2 = 0 are called tained in P such that I P l - - 0 . Thus p - 1 usually con-
missing solution sets (with respect to F 1 = 0). The tains elements which are discontinuous functions.
Journal o f Computational and Applied Mathematics, volume I, no 1, 1975. 27
The following Theorem deals with the special matrix Since ID I =# 0 identically, D-1 exists. Premultiplying
p-1 whose elements may not be continuous func- on the left by D -1 yields
tions, but whose inverse P has elements which are Q = D-1p
continuous functions.
In order to illustrate the application of Theorem
THEOREM 3-2 3-3, consider the following case when the Q matrix
If (1) P is a transformation matrix is used for the premultiplication of F = 0. Then
(2) F is an equation vector QF=0~D-1pF=0 C PF=0 ~ F=0
then [IDI=O] [IN=O]
F=O D p-1F= 0
according to Theorems 3-3, 3-2 and 3-1.
[IPI]=0

Proof 4. LINEAR FORM MATRICES

Let G = [Gi] be the nxl matrix and G = p-1F. Linear form matrices are useful for eliminating var-
iables which appear linearly in sets of equations.
According to Theorem 3-1
Whenever it is possible to select m equations which
G=O C PG=0 contain m variables in linear fashion only, it is poss-
.[IVl]=0 ible to reduce n equations in n variables to n-m equa-
tions in n-m variables.
Premultiplying G = p - 1 F by P yields PG = F.
Section 4.1 describes the various operations which
Substituting these two identities into the above
may be performed on linear form matrices. Section
equations yields 4.2 develops the concepts of rank, linear dependence
P - 1F = 0 ~"- F=0 and nonlinear dependence. Sections 4.3, 4.4 and 4.5
[IPI] = 0 describe three techniques of elimination. Systematic
elimination is based upon a generalization of Cramer's
THEOREM 3-3
rule. Singular elimination is based upon the concept
Any matrix Q can always be expressed as the pro- of nonlinear dependence. Triangular elimination is
duct D-1P, where D and P are transformation ma- the most mechanical of these elimination techniques.
trices, and D is diagonal.
4.1 Linear form matrix operations
Proof Before formal rules are given for the manipulation of

Q = [qiJ] =
[,,]
Any matrix Q can be represented by linear form matrices, an important difference between
sets of linear equations and sets of nonlinear equa-
tions should be noted. The linear form coefficient
and augmented matrices B and A are unique for a
set of fLxed linear equations, but are not unique for
where qij may not be continuous, but where qij and a set of fLxed nonlinear equations. The non-unique-
q'.. are constants or continuous functions. The q'.. are ness of B, for example, is illustrated with the follow-
B . . . B ing set of nonlinear equations.
restricted to be not Identically zero.
Let the elements of the diagonal matrix D=[diirij ] 2
be given by XlX 2 + x 2 + x 2 x 3 - 12 = 0 ] -
n
XlX 2 + x 2 x 3 - x22x 3 + 4 ~t (4.1-1)
d i i = k =II 1 qik
' = qil
' qi2 . . . . q i n
xI + x22 x3 2
The elements dii are continuous since the q~k are
continuous. The linear form coefficient matrix B may take a
The ij th element of DQ, (DQ)ij, is number of forms, three of which are
n
I! 2
(DQ)ij = r =I
Z dii ~ir qrj = dii qij 1 x2 t/]Ix22 1 x2211x22 1 x2-I
2

= I~ qik --qij x2-1 !11 x2 -1 _111 x2 -I J

k =1 qij k-4:j
(4.1-2)
These elements are constants or continuous lunc-
h The last linear form coefficient matrix in (4.1-2) is
tions since the factors of qij klI=l qik are constants or generally preferred, since coefficient matrix B is then
a function of only one unknown.
k~j
continuous functions. Since the elements (DQ)ij de- 4. 1.1. R E A R R A N G E M E N T OF ELEMENTS
fine a P matrix, it has been shown that All of the possible forms of the matrix B are of course
DQ=P equivalent, but for ease of solution, B should be chosen

Journal of Computational and Applied Mathematics, volume I, no 1, 1975. 28

so as to contain the smallest possible number of 4.1.3. ROW OPERATIONS
variables. The various equivalent forms of B may be Row operations may be performed on linear form
obtained from each o~her by application of the augmented matrices. Since it is sometimes desirable
following obvious matrix rearrangement rule. to multiply or divide a row by a function, it is use-
ful to distinguish between equivalent, subordinate
Linear form matrix rearrangement rule and dominant row operations.
The following rule can be applied to any element fij
of a linear form augmented matrix A or coefficient Definition
matrix B. Any element fij (i, j = 1, 2 . . . . , n) can be Any row operation performed on a linear form
replaced by zero providing fij is multiplied by xj, augmented matrix which produces an equivalent,
divided by x k, and added to fik (k ~< n). subordinate or dominant augmented matrix is said
The rule is useful whenever fij can be factored into to be an equivalent, subordinate or dominant row
operation.
the form ~ijXk. In this paper it is assumed that the
linear form matrix rearrangement rule has been ap-
Equivalent row operations
plied to obtain matrices which contain the smallest The following are equivalent row operations :
possible number of unknowns. 1. The interchange of two rows.
2. The mukiplication of the elements of a row by a
4. 1.2. COLUMN OPERA TION constant other than zero.
For the augmented matrix A, a useful extension of 3. The addition, to the elements of a row, of a con-
the above rearrangement rule can be stated. stant times the elements of another row.
4. The addition, to the elements of a row, of ~b
Column operation rule
times the corresponding elements of another row,
The entire jth column (1 <~j ~ n) of the linear form where ¢ is a function of one or more variables,
augmented matrix A may be replaced by zeros restricted only in that ~b is a continuous function.
providing the entire jth column is multiplied, element
by element, by xj and added, element by element, to Subordinate row operation
the last column. The division of each element of a row by q~k, provid-
The column of zeros introduced by a column oper- ing ~bk is a common factor of each element of that
ation may be deleted if desired, reducing the number row.
of columns in the matrix by one. The validity of the
rule is obvious, since the operation performed is Remark
equivalent to a simple rearrangement of the original By Theorem 3-2, any missing solution set must
set of nonlinear equations. satisfy ~k = 0.
Column operations can be performed on any nx(n+l)
linear form augmented matrix A by post multiplica- Dominant row operation
tion with an ( n + l ) x ( n + l ) matrix R. The matrix The following are dominant row operations :
R = [rij ] defined by 1. The multiplication of the elements of the k th row
by a function ~bk, defined as above.
[rij] = 18i. - E ~ikSkj + k~=s,~ik~(n+ 1)j x ~ 2. The addition, to ~k times the elements of the k th
LJ k = s , t . . . . . .
row, of ~bj times the elements of the jth row,
(4.1.2-1) where ~bk and ~bj are defined as above.
operates on the columns s, t . . . . of A. For example,
ehe R matrix which operates only on the s and t Remark
columns of A is By Theorem 3-1, ff any additional solution sets are
1 0- I
\ produced by either operation 1) or 2), these solu-
tion sets must satisfy ~k = 0.

1
0

\
C 0
Xs
0
S rOW

(4.1.2-2)
4.2. Rank, linear dependence, and nonlinear depend-
ence
A few useful concepts are singled out for special
emphasis here. The concepts of rank, linear depend-
ence, and nonlinear dependence help to clarify the
algorithms of solution described in the next section.

1 0 4.2.1. RANK

C) ",iJ
0 Xt t row
In linear algebra the concept of rank permits elegant
OI statement of the conditions under which a set of
linear equations has a solution. This concept of rank
can readily be extended to sets of nonlinear equa-
tions represented in linear form. Since linear form
matrices may contain one or more unknowns, it is

Journal of Computational andApplied Mathematics, volume I, no 1, 1975. 29

necessary to distinghuish between unconditional Definition
rank and conditional rank. Consider any linear form coefficient or augmented
matrix containing I variables, say Xl, x 2 . . . . . x i ( I ~ n ).
Definition For each numerical set of these variables (a 1, a 2, . . . ,
The unconditional rank of a linear form coefficient ai) w h e r e a i ~ 0 f o r i = 1 , 2 . . . . . j a n d a i = 0 for
or augmented matrix is the order o f the largest i = j + 1, j + 2 . . . . . I (0 ~ j ~ I), the conditional rank
square array in that matrix (formed by deleting o f the matrix is defined as the order of the largest
certain columns and rows) whose determinant does square array whose determinant does not vanish,
not vanish identically. where the array is formed from the matrix after sub-
stituting the numerical values (al, a 2 . . . . . ai) and
Example deleting the (j + 1)st, (j + 2)nd . . . . . Ith columns.
The three equations in three unknowns
r 2 2 4 ] Example
!x2 x 1 + x 2 + ( x 2 + x 2 ) x 3 - 3 = 0 The conditional rank o f (4.2.1-2) with respect to
3 2
! x 2 = 0 is two after substituting x 2 value and deleting
x2x I + x ~ + ( x 2+x2) x3+3=0 1 (4.2.1-1) the second column.
t
2 ! 4.2. 2. LINEAR DEPENDENCE
xI - x2+(x 2- 1) x 3 - 3=0J
The concept o f linear dependence can be extended
are associated with the linear form augmented to nonlinear sets o f equations written in linear form.
matrix The concept applies equally well to linear form co-
efficient and augmented matrices. Let the row vectors
x~ x2 x2+x 2 - 3 o f the coefficient matrix B
2 3 2 I
x2 x2 x2 + x2 3 [ (4.2.1-2) fll f12 "'" f l n

o I f21 f22 . . . . f2n

1 -1 x~- 1 - 31
-.-I

The unconditional rank of the linear form coefficient

matrix is two, since its determinant (which can be fn2 ....
formed by deleting the last column) vanishes ident-
be denoted by X i where i = 1, 2 . . . . , n. Since the
ically, and the determinant o f any 2 x 2 square array
elements fij {J = 1, 2 . . . . . n) may contain variables,
does not vanish identically. However, the uncondit-
two types o f linear dependence are distinguished.
ional rank o f the linear form augmented matrix
(4.2.1-2) is three, since the determinant of 3 x 3
square arrays (which can be formed by deleting the
Definition
A set of m row vectors ( m E n)X 1, X 2 . . . . . X m is
third column) does not vanish identically. It should
said to be unconditionally linearly dependent if there
be pointed out that in linear algebra the rank o f the
exists a set of constants 0l1, 0t2, . . . , ~cn (at least one
coefficient matrix must be equal to the rank o f the
of which is not zero) such that
augmented matrix for a solution set to exist. It is
not necessarily so in nonlinear algebra. A solution a l X 1 + a 2 X 2 + . . . + ~mXm = 0 (4.2.2-1)
set, namely, x 1 = 2 , x 2 = - 1 a n d x 3 = 3 , d o e s
identically.
exist for the system of equations (4.2.1-1), while
the unconditional ranks of the coefficient and
augmented matrices are not equal.
Example
The three row vectors
Remark X 1 = (3x 1, 2x 2, 4x 1)
The unconditional rank o f a linear form matrix is
invariant under premultiplication or post multiplica- X2 = ( x 1, x 2 , 3 x 1)
tion b y any conformable square matrix whose deter-
minant does not vanish identically. The proof is bas- X3 = (2x 1, x 2, x 1)
ed on the Binet-Cauchy Theorem concerning pro- when multiplied b y 41 = 1, 0~2 = -1, a 3 = -1 and
ducts o f compound matrices and is well known (3). summed are equal to zero identically, and are there-
Of course, the use of the matrix rearrangement rule fore unconditionally linearly dependent.
and column operations may affect the unconditional
rank o f a linear form matrix. Definition
However, it should be noted in the above example A set o f m row vectors (m ~ n)X 1 (x 1, x 2 . . . . . xi),
that for certain numerical choices o f x 2 the rank o f X2(Xl, x2, . . . , xi) , . . . Xm(Xl, x 2 . . . . . xi) is said
the above mentioned coefficient and augmented to be conditionally linearly dependent with respect
matrices may change. These possibilities lead to the to a set o f constants al, a 2 . . . . . a k (k ~ I) ff there
notion o f conditional rank. exists a set of constants Ol, ~2, " " , %n (at least one

Journal o f Computational and Applied Mathematics, volume I, no 1, 1975. 30

of which is not zero) such that tinuous functions or constants 41, 42 . . . . . 4m (at
least one o f which contains at least one variable)
C~lXl(al . . . . . a k, Xk+l . . . . . x I) such that
41X 1 + 4 2 X 2 + . . . + 4 m X m=0
+ ~2X2(al . . . . . ak,Xk+ 1. . . . . Xl) + . . .
identically.
+~VmXm(al . . . . . ak, X k + l , . . . , X l ) = 0
Example
when the constants a 1, a 2 . . . . . a k have been sub- The three row vectors of the coefficient matrix
stituted for the variables x 1. . . . . x k which appear (4.2.1-2) when multiplied by 41 = - 1 , 42 = 1,
in the row vectors. 43 = x 2 - x 2 and summed are equal to zero identic-
ally, and are therefore nonlinearly dependent.
Example
Let the first two row vectors of the linear form
THEOREM 4-2
augmented matrix (4.2.1-2) be denoted by X 1 (x2)
The p row vectors of any pxn matrix are nonlinearly
and X2(x2). If x 2 = -1, there exist ~1 = 1, c~2 = l dependent if and only if the unconditional rank of
such that the matrix is less than p. The p row vectors o f a
X1(_1 ) + X 2 ( - I ) = 0. pxn matrix are nonlinearly independent if and only
if the unconditional rank of the matrix is p.
Therefore X 1 and X 2 are conditionally linearly de-
pendent with respect to x 2 = -1.
The following Theorem, based on the concept of
Proof
The proof o f this Theorem is quite lengthy and is
linear dependence or conditional rank, is useful for
therefore not given in this paper (for proof see Ref
testing the validity and character of a numerical solu-
(6)).
tion set.
THEOREM 4-1 4.3. Systematic elimination
Let A and B be n x ( n + l ) and nxn linear form aug-
Systematic elimination is a formal linear elimination
mented and coefficient matrices associated with a
technique which can often be used to eliminate
set of n nonlinear equations with n unknowns. Let
certain linear variables and to reduce the number o f
these matrices contain I variables xi(j = 1, 2 . . . . . I)
equations which must be solved simultaneously. The
where I <~ n. Let A c and Bc be theJcorresponding
technique is useful whenever it is possible to select
matrices after column operations on all j columns.
m equations which contain m variables in linear
A numerical set of values for xj, say 7j(j = 1, 2 . . . . ,I)
fashion only, from the original n equations in n vari-
(1) is not a part o f any solution set o f the original ables. After application o f this technique, n-m equa-
nonlinear equations if and only if the conditional tions in n-m variables remain to be solved simultane-
rank of A c :# the conditional rank of Be. ously.
(2) is a part o f a unique solution set o f the original
nonlinear equations ff and only if the conditional Algorithm
rank o f A c = the conditional rank of Bc = n - I. The algorithm for performing systematic elimination
(3) is a part o f an ( n - I - r ) fold infinity o f solutions is as follows.
o f the original nonlinear equations if and only if (1) The augmented linear form matrix A which re-
the conditional rank o f A c = the conditional presents the set of n equations in n unknowns is
rank o f B c = r, where r < n - I. written down.
All of the conditional ranks referred to are o f course (2) The linear form matrix rearrangement rule (Sec-
with respect to the set o f constants 7j. tion 4.1.1) is used to minimize the number o f
variables which appear in A.
Proof (3) Select R rows which contain n-R or fewer un-
Substitution o f 7; for x] in A c and Bc yields matrices knowns so as to maximize R.
3
with constant elements. These constant matrices Ac (4) Form a new R x ( n + l ) matrix A' from these R
and B c have the dimensions n x ( n + l - I ) and nx(n-I). rOWS.

Thus the nonlinear equations have been reduced con- (5) Perform a column operation on A' for each vari-
ditiona~y to a set o f n linear equations with n-I un- able contained in A', and delete the column o f
knowns. The Theorem now follows directly from zeros created. If x 2 appears in A' operate on and
classical linear algebra. then delete the second column, etc. If the result-
ing matrix contains more than R + I columns,
4.2. 3. NONLINEAR DEPENDENCE continue column operation and subsequent dele-
The concept o f linear dependence presented in Sec- tion on arbitrarily selected columns until exactly
tion 4.2. 2 can be extended to nonlinear dependence. R + I columns remain. Through step (5) strict
equivalence has been maintained.
Definition (6) Using the generalized Cramer's rule (see Theorem
A set o f m row vectors X 1, X2, . . . , X m is said to 4-3, below) solve for the R variables which cor-
be nonlinearly dependent if there exists a set of con- respond to the first R columns in terms o f the

Journal o f Computational and Applied Mathematics, volume I, no 1, 1975. 31

 remaining n - R variables. This step often intro- which, if they exist, must satisfy IAdj P I= 0.
duces additional and missing solution sets as In order to use the generalized Cramer's rule to solve
shown in the statement following Theorem 4-3. for the vector X of the equation BX + C = 0, two
(7) Substitute the results from step (6) into the un- operations are required, premultiplication by Adj B
used n-R equations, thus eliminating R variables. followed by premultiplication with
The n-R equations obtained contain n-R vari-
ables.
(8) These n-R equations, which must be solved
simultaneously using some other technique, to-
gether with the R equations from step (6), form
IBI -iI (4.3-I)

a new set of equations which is dominant or

subordinate to the original set of equations. IBI
Cramer's rule may be generalized in nonlinear algebra.
Theorem 4-3 shows that the use of Cramer's rule By Theorem 3-1 or 4-3, premultiplication by Adj B
may introduce additional solution sets. The state- is a dominant operation and any additional roots if
ment following the Theorem applies specifically to introduced must satisfy [Adj BI = 0. By Theorem
step (6) of the systematic elimination algorithm. 3-2, premultiplication by (4.3-1) is a subordinate
operation, and any missing solution sets must satisfy
Definition IB[ = 0.
If P is a square transformation matrix, the matrix
obtained from P by replacing each element by its Remark
cofactor and then interchanging rows and columns If no variables are cancelled after premultiplying by
is called the adjoint of P, denoted by Adj P. (4.3-1), there is no possibility of missing any solu-
The following is considered to be the nonlinear tion sets.
generalization of Cramer's rule.
Example
THEOREM 4-3 The three equations in three unknowns
If (1) P is a transformatio'n matrix xy 2 + y + yz = 0]
(2) F is an equation vector (4.3-2)
then xy +y2z+ 1 =01
x +y2 _ z -2=0
PF : 0 ...C IPIF = 0
[IAdj PI=0] are associated with the linear form augmented matrix
Proof y2 1 y 0
PF = 0 in expanded form is y 0 y2 1 (4.3-3)

.... PI<FFI l L 1 y -1 -2j

Operating on the second column yields
P21 P22 P2n IF2]
=0
• :

Y y2 1 (4.3-4)
Pnl Pn2

Premultiplying by Adj P = [Pji] yields

Pn 1
y,
-1 y2-2
J
Using the generalized Cramer's rule on, e. g., the first
and third equations yields

I Pll P21 . . . . Pnl

P12. P22. Pn2.

Pll P12 " " P l n

P21 P22 " " P 2 n

-1:1"

F2
r

=0~
i -Y
[(2-y2)
Y I
=1
_y3
+ y
y2 y y2 + y
1 -1 (4.3-5)
LPIn .P2n Pnn Pnl Pn2 "'" Pnn Fn
y2 -y

1ili02
PI z = 1 (2-y2) = y4_2y2_y

\o IPI
=0
y2
1 -1
y y2 + y

Substituting into the second as yet unused ec uation

yields the algebraic equation in y
Ipl (_y3 + y ) y + y2(y4 2y2_y) +1=0
Premultiplication of PF = 0 by Adj P, according to (y2 + y) (y2 + y)
Theorem 3-1, may introduce additional solution sets
Journal of Computational and Applied Mathematics, volume I, no 1, 1975. 32
 which may be factored into don can eliminate certain nonlinear variables. The
y(y+l) (y+l) (y-l) (y2-y-1) technique is based on the notion of nonlinear depend-
= 0. (4.3-6) ence already described in Section 4.2.3.
y(y+l) The principle of singular elimination is as follows.
According to the statement and remark following Consider the n x ( n + l ) linear form augmented matrix
Theorem 4-3, equations (4.3-5) and (4.3-6) are A associated with n nonlinear equations in n un-
dominant to (4.3-2) and the following numerical knowns. After any number n + l - m of column oper-
values of y obtained from the numerator of (4.3-6) ations ( 0 < m <~ n + l ) , the equivalent nxm matrix Ac
contain all of the solutions o f the original problem takes the form
(4.3-2). ~ -

~
f
y=0
y = -0.618
y=-I y=+l y = 1.618

if any additional solution sets are present in (4.3-7)

(4.3-7)
fll

fkl
f12 " " " f l m - 1

f k 2 " ' " ff~n-1 %

lm

[i1 f

f
lm

they must satisfy [Adj B[ = 0 which in this example = k km

[xo
ist

f, f, f, f ~
nl n2" " " nm-1 nm 1
Thas the roots y = 0 and y = -1 may correspond (4.4-1)
to additional solution sets. These two values are now
where Xi (i = 1, 2 . . . . . n) are the n row vectors fil'
checked by means of Theorem 4- !. Substituting
f'i2 . . . .. f'im-l" If there exists a set of functions or
y = 0 into (4.3-4) yields
constants @i (i = I, 2 . . . . . m) as defined in Section
I: 0 01 0 1 (4.3-9)
4.2.3, i. e.,

~1 X1 + @2X2 + • • + @mXm = 0,
L1 -1 -2J then
Since the conditional rank of the coefficient matrix
of (4.3-9) is one, and the conditional rank of the X1 f'lm' -X1 f'lm
augmented matrix is two, y = 0 is not a solution set
of the original problem (4.3-2). Substituting y = -1 " C
into (4.3-4) yields Xk [@k'=ol Xk-1 fk-lm
0 @1f,lm+@2f2m •
, +" "+@mf,nm
t
-hi - 11
-1
- 11
-1
Since the conditional rank of the coefficient matrix
of (4.3-10) equals the conditional rank of the aug-
mented matrix of (4.3-10) which equals one, y = - 1
is part of a one-fold infinity of solutions of the
(4.3-10)

iX!o Xk+l

Xn

so that all solution sets of A c must satisfy

fk+l

f'nm
(4.4-2)

original problem (4.3-2). By inspection this solution

set is @1f,lm + ~2 f2m
, + . . . + @mf'nm = 0 (4.4-3)
x-z-l=0 y=-I (4.3-11) The validity of this statement is assured by Theorem
which represents a straight line. 3-1, since the matrix on the right can be obtained
The remaining roots of (4.3-7), y = 1, y = 1.618 from the matrix on the left by premultiplication
and y = -0.618, when substituted into (4.3-5) yield with the transformation P matrix,
the remaining solution sets
x=0 y=l z=-i
1 ~ . . ~ 1
x = 20.618 y = 1.618 z = 0 (4.3-12)
x = 1.618 y =-0.618 z = 0
@1 . . . . @k-1 @k @k+l . . . . em 0 . . . 0 J
Thus the original problem (4.3-2) has four solution
sets as expressed by (4.3-11) and (4.3-12). t

I
4.4. Singular elimination
Singular elimination is a flexible elimination tech- (4.4-4)
nique which in favorable cases is more powerful than whose determinant is @k"
systematic elimination. Occasionally singular elimina- Clearly is @k is a non-zero constant the matrix on

Journal o f Computational and Applied Mathematics, volume I, no 1, 1975. 33

the right is equivalent to the matrix on the left. Theorem 4.2. In this case it is necessary to select
Thus, if one or more of the ¢'s are non-zero con- the r row vectors o f (4.4-5) which are nonlinearly
stants, it is possible to choose the k th row in such independent, to add an r + l TM row, and to search
a way that no additional roots are introduced. If for a set o f functions ¢1' ~b2. . . . . Cr such that
none o f the ¢'s are non-zero constants, then it is
usually desirable to assign the subscript k to the ¢1X1 + ¢2X2 + . . . + CrXr = X r + 1 (4.4-7)
simplest ¢. The solution for ~1, ~2, " " , ~br is unique, since if it
The existence o f a set of ~b's for m row vectors X i is assumed that another set exists, say ~1' ~b2. . . . . Cr'
is assured since any m row vectors with m-1 com-
it must satisfy
ponents are nonlinearly dependent, according to
Theorem 4-2. ~1 X1 + ¢2 X2 + "'" + ¢~ Xr = X r + l (4.4-8)

Subtracting (4.4-8) from (4.4-7) yields

[¢1¢2""¢ml 2 =0'i'e"~X=0
(~b1-¢2)X1+(¢2 ~
¢2) X 2 + " "
'
+(¢r-¢r) ~ X
r=0
But all o f the coefficients ( ¢ i - ¢ i ) ' i = 1, 2 . . . . . r,
l lm must be zero because if any one should be non-zero,
that would indicate that the set o f row vectors X 1,
Taking the transpose of this equation yields X 2 . . . . . X r is nonlinearly dependent, in contradic-
xT~bT = 0, i. e., tion with the original premise.
Since the ~b's are unique they may be conveniently
f'ml ~b1 found from equation (4.4-7),
fll " " " fkl " " "

X1
X2
fi2 . , •
fi 2 "'"
f,
m2 k
=0
[¢1 ¢ 2 " " ¢ r - 1 ] . = 0, i. e., ~bX = 0.

flm-1 . . . fi m-1
" " "
b [
ram-1 i
Cm
Xr
L Xr+lJ (4.4-9)
(4.4-5) Taking the transpose of (4.4-9) yields

Since (4.4-5) consists of m-1 equations (rows) and X T c T = 0.

m unknowns (¢1, ~b2, • "" , ~m), one o f the O's, say Since the matrix X T is of dimension ( m - l ) x (r + 1),
~bk, is not determined and may be assigned the m - l - r rows o f X T are deleted such that the deter-
arbitrary value ~k = -1. Placing the kth column o f minant o f the first r columns and the r remaining
(4.4-5) on the right and denoting the new O's with rows does not vanish identically. Then ~1,~b2..... Cr
primes yields :~ may be solved for by the same procedure used for
- --[ - --
¢ i "'" ~bk-1 ¢ k + l "'" Cm"
fll • " " fk-ll fk+li "'f' ml The process of singular elimination is usually perform-
ed on a linear form n x ( n + l ) augmented matrix after
: l n - m + l column operations which leaves a nxm
, , f, .. f'
f12 " " " fk-12 k+12 " m2
:+111= column operated augmented matrix. This latter matrix
can be treated by singular elimination to yield the

flm'-I" fk-lm-1 f'

"k+ lln'-'l f'mm-I J result shown in (4.4-2). The treatment can be repeat-
ed on the matrix on the right hand side of (4.4-2) by
choosing any m row vectors X i which are not zero.
In this fashion, the treatment can be repeated n - m + 1
(4.4-6)
The k th row is selected to insure that the coefficient times, yielding n - m + 1 equations o f the form of
matrix o f (4.4-6) is non-singular, i. e., has an uncon- (4.4-3). After these n - m + l treatments the resulting
ditional •rank of m - l , ff that is possible. If the co- matrix contains n - m + 1 zero row vectors X i.
efficient matrix is non-singular, its inverse exists. If the number o f column operations performed is
equal to the number I of variables which appear in
PremuMplying by this inverse yields a solution
the linear form augmented matrix, and the column
$1 . . . . . ¢k-1' ¢ k + 1 . . . . . Cm" If any o f these terms
operations have been performed on the columns
are not continuous, the offending factors may be corresponding to these I variables, then n - m + l = I.
multiplied through, rendering the set o f cb's, ~b1, Thus singular elimination can reduce the problem of
~b2. . . . , Om, continuous functions. solving n equations in n unknowns simultaneously to
If the coefficient matrix o f (4.4-6) is singular, i. e., the problem of solving I equations in I unknowns
if its unconditional rank is less than m - l , say r, then simultaneously.
m - l - r o f the row vectors of (4.4-5) are nonlinearly The reason for not allowing a zero vector to be
dependent on r row vectors, according to the chosen as one o f the m row vectors X i in the above

Journal of Computational and Applied Mathematics, volume I, no 1, 1975. 34

procedure is that the unconditional rank of the Forming the transformation matrix of the form of
matrix formed from these m row vectors is often
m-1. In this case, ff the kth row vector is zero the
only set of O's which exist, namely 01 . . . . . Ok-1
0 1 -1 y
]
(4.4-4) by letting k = 2 yields

(4.4-10)
= Ok+l ..... Om = 0 and Ok = any non-zero
function or constant, is not useful for further reduc- 0 0
tion by singular elimination. Premultiplying (4.3-4) by (4.4-10) yields the
equivalent matrix

[: y:l[
Example
The same problem solved by systematic elimination
in Section 4.3 is now treated by singular elimination. y2 y2 ~" 0
~ (y+l)(y-Yl)(y2-y-11
xy 2 + y +yz =0"1
-I y2_~ i -1 y2-2 J
xy + y 2 z + I ~f (4.3-2)
x +y2-z-2 = so that all of the solution sets of the original prob-
lem must satisfy
The linear form augmented matrix corresponding to
(y+l) (y-l) (y2-y-1) = 0.
these equations is
This result is identical with the results obtained when
y2 1 y 0 using systematic elimination, as shown in Section 4.3.
y 0 y2 (4.3-3)
4. 5. Triang,11ar elimination
i y -i
Triangular elimination is an alternate reduction tech-
Operating on the second column yields nique for treating linear form matrices. As with
y2 y y ] singular elimination this technique can also reduce n
equations in n variables to I equations in I variables,
Y y2 1 J (4.3-4) where I is the number of variables contained in the
1 -1 y2_2 linear form matrix. Triangular elimination is a repet-
itive procedure mechanically similar to the Gauss
A set of 0's for carrying out the singular elimination
reduction in linear algebra and may therefore be the
is defined by best of the linear form reduction techniques for
computer implementation.
y2 Y _t Algorithm
[¢i 02 ¢31 Y y2 = 0
1 -1 The algorithm for performing triangular elimination
is as follows :
Before solving for the O's, it is convenient to re- (1) The augmented linear form matrix which re-
arrange the above matrix equation to presents the set of n equations in n unknowns
is written down.
(2) The linear form matrix rearrangement rule (Sec-
1 -1 1 tion 4.1.1) is used to minimize the number of
[03 02 01 ] y y2 = 0,
variables which appear in the matrix.
y2 y (3) Perform any number of column operations (Sec-
tion 4.1.2). It is usually advantageous to perform
Taking the transpose and assuming 02 = - 1 yields
I column operations on the columns correspond-
the following matrix associated with the unknowns ing to the I variabhs which appear in the matrix.
03 and OI (4) Rearrange the rows such that the 1, 1 element of
the matrix becomes the simplest possible non-
_ y y2 zero function, preferably a non-zero constant.
Denote this column operated augmented matrix
Reducing this matrix by a process similar to Gauss- by A~1] .
Jordan reduction yields
1 0 ....

I
so that
0 1
411J 1 41J ....
01 = 1 411 ,i 1 ....
02 = -1
03 = y_y2
nl .... tim nxm

Journal of Computational and Applied Mathematics, volume I, no 1, 1975. 35

 (5) Prernultiplying (4.5-1) by the following trans-
formation matrix
1
I 0
1 o _f[21 f[21
-4f ~ 3 2 22~f~ (4.5-5)
~l] ] 0 (4.5-2)
-41] 0 _f[21
n2 O ]

_L(~?
)roduces the result
d nxm
yields

1 fl] ] 0 0 o 4f f[21
23
4lj ,~] 0 f[1]
21 f[l] fill
22 23 ,2t~ -~3f4f Iti '3(f ,~2j f[21
3m
_f[1131O~ fill fll]32 fi~ ]
31
Cf f[21

_-'fnl
[ 1] fill L[l] ~[1]
nm
nlTl

12 13 "'"
23 "'"
o f;2 f;.3 f,
2m i o4]] ,[~1
5'3
4~
f" "" 3m

2 3 f, 4 J" (4.5-6)
3m
0 0 f"n3 "'" f"nm~
0 f' f' f~ where (4.5-4) C (4.5-6).
- n2 n3 nl-n (4.5-3)
where .[21 o]
t22 =
(4.5-1) ,,,C (4.5-3)
lql]=0, (8) Continuing in this manner it is possible to trans-
form the original matrix (4.5-1) into a dominant
matrix which has only zeros in its first m-1 col-
(6) Rearrange the 2nd through nth rows of the umns below the principle diagonal. Thus
matrix on the fight hand side of (4.5-3) such W"

that the 2, 2 element becomes the simplest poss- fill f[1] f[ll f[l]
ible non-zero function, preferably a non-zero 12 13 "'" lm
constant. Denote this matrix by A[2]. C
C
'"]~.~ 4]? ,~]l... 4~ ~,iI]f[2]
22 "'"
• .. ,i~]] 31 32 "'" f[m-ll
m-lm-1
- Ol
-
0 f[21224 ~ ] •.. 4f" fill fill ~ ] fill
nl n2 "'" nm
f[2] f[2] f[2] (4.5- 4)
32 33 "'" 3n

o ff ~[3]... f[2]
nlTl

(7) Premultiplying (4.5-4) by the transformation

matrix

Journal of Computational and Applied Mathematics, volume I, no 1, 1975. 36

 --I

~11 12 "'" lm-1 FI 0 0-[1 -1 y2-21

-y i °IY y2
23 "" 2m-1
0 1
-1
iy 2 Y yJ
f[31 f[3t f[3]
33-"" 3m-1 om
1 -1 y2_1
I

C "'4.
rim:l] .
m-xm-i
f[m-1 ]
m-lm
fire1
mm
I y2+y
y2+y
l_(y2_2)y
y_y2 (y2_2)

Thkd, premultiply by a constant transformation

1 (4.5-10)

matrix

L f[m]
(4.5-7)
mm --I 0 0 1 -1 y2_1
If in step (3) I column operations are performed,
then the solution sets of the matrix on the right l~ 1 0 y2+y l_(y2_2)y
hand side of (4.5-7) are found by solving the -1 1_ y2+y y_y2(y2_2 )
n - m + t = I equations containing t variables
£[m~ = 0
.rD_rD_ y2-i
f[m] = 0 (4.5-8) "" 0 y2+y i-(y2-2)y (4.5-11)
m+lm
t0 0 -(y+ l)(y-l)(y2-y-1)
fim] = 0 Since it has been possible to maintain equivalence
throughout these transformations, the solution sets
and substituting back into the matrix on the right of the original problem must satisfy
hand side of (4.5-7), providing none of the diagonal
dements Ln the first m-1 columns of the matrix on (y + 1) (y- 1)(y2-y- 1) = 0 (4.5-12)
the right hand side of (4.5-7) are zero. i. e.,
(9) Any solution set of (4.5-8) which makes one or
more of the diagonal elements of the matrix on y = 1 y = -1 y = 1.618 y = -0.618
~che right hand side of (4.5-7) zero is tested by Substituting into the matrix on the right hand side
substituting it back into the original column of (4.5-11) yields the four solution sets expressed by
operated augmented matrix A[cl]. Such solution (4.3-11) and (4.3-12).
sets of (4.5-8) are accepted or rejected on the
basis of rank according to Theorem 4-1. 9. BIBLIOGRAPHY
I. Uspensky, J. V., "Theory of equations", First Edition,
Example McGraw-Hill Book Company, Inc., N e w York (1948).
The same problem solved by systematic elimination 2, Burnside, W. S., Panton, A. W., "Theory of equations",
in Section 4.3 and singular elimination in Section Third Edition, Hodges, Figgs, and Co., Dublin (1892).
4.4 y2 y 3. Aitken, A. C., "Determinants and matrices", Ninth Edi-
tion, Oliver and Boyd, Section 41, pp. 96-9? (1958).
4. Griffiths,L. W., "Introduction to the theory of equa-
tions", Second Edition, John Wiley and Sons, N e w York,
t_ 1 -1 y2_ 2 (4.3-4) pp. 218-220 (1947).
5. Chrystal, G., "Algebra", Third Edition, A d a m and Charles
is now treated by triangular elimination. Black, London (1892).
First, the order of the rows is changed to place the 6. Wu, C. L., "Nonlinear matrix algebra and engineering
third row first applications", Ph.D. Thesis, Case Institute of Technology
(1964).
1 -1 y2_ 2-
y y2 (4.5-9)
y2 Y Y
A
Second, premultiplying by a transformation matrix

Journal of Computational and Applied Mathematics, volume I, no 1, 1975. 37