1

Matrix Algebra Review

Introduction

This is the matrix review for A Student’s Guide to Vectors and Tensors
(SGVT). It is not meant to be a thorough introduction to the theory and
practice of matrix algebra, but rather a review of the matrix concepts
you should understand when learning about vectors and tensors.
A matrix is defined as an array of numbers. Some authors restrict the
term “matrix” to refer only to square arrays of real numbers, but in com-
mon usage a matrix is a rectangular (m x n) array of complex numbers
or “elements,” where “m” represents the number of rows, “n” represents
the number of columns, and “complex” means that the numbers may be
purely real, purely imaginary, or mixed. Thus
8
 
   0 
3 −2    
A= B=  −2  C = 9 15 3 −8 12 −2

1 0  1 
5
and
5 − 2i
 
3i  
1 0 0  
 0 15  0 0
D=
 −4i
 E= 0 1 0  F =
2 + 8i  0 0
0 0 1
12 i−1
are all matrices. In these examples, matrix A has “dimensions” or “or-
der” (rows x columns) of 2x2, B of 5x1, C of 1x6, D of 4x2, E of 3x3,
and F of 2x2.
2 Matrix Algebra Review

If both m and n for a matrix are equal to one, that matrix is called
a scalar (because it’s just a single value), and if either (but not both)
of m and n are one, that matrix may be called a vector. So in the
examples shown above, B and C are vectors; in some texts you’ll see
B called a column vector and C called a row vector. A square matrix
(that is, a matrix with m=n) with ones on the diagonal and zeroes in
all off-diagonal elements (such as matrix E above) is called the “unit”
or “identity” matrix, and a matrix with all elements equal to zero (such
as matrix F above) is called the “null” or “zero” matrix.
Two matrices are said to be equal only if they have the same number
of rows as well as the same number of columns and if every element in
one matrix is equal in value to the corresponding element in the other
matrix. Matrices are usually denoted using uppercase letters or bold font,
often surrounded by square brackets (such as [A]), and the elements are
often written using lowercase letters with subscripts. So you may see the
elements of matrix [A] written as aij , although some authors use Aij or
[A]ij to refer to the elements of A (but be careful to note that Aij may
also refer to the “Matrix of Cofactors” of matrix A, which you can read
about in Section E of this document).
So does terminology such as “row vector” mean that matrices, vectors,
and tensors are the same thing? Not really. It’s certainly true that the
vectors and tensors in SGVT are often represented using matrices, but
it’s important to remember what those matrices stand for. Those arrays
of values represent the components of vectors and tensors, and those
components have meaning only when associated with the basis vectors
of a particular coordinate system. Since the basis vectors are not always
shown, it’s convenient to think of the matrix as representing the vector
or tensor itself, and that’s fine as long as you remember that the actual
vector or tensor has existence independent of any coordinate system.

A) Matrix addition, multiplication by a scalar, and subtraction

Matrices may be added only if both the row dimension (m) and their
column dimension (n) are equal (such matrices are said to be “of the
same order”). The addition is accomplished simply by adding each el-
ement of one matrix to the corresponding element of the other matrix.
For example:
     
5 2 −3 1 2 3
+ =
−3 0 0 4 −3 4
 Matrix Algebra Review 3

or in general
     
a11 a12 b11 b12 a11 + b11 a12 + b12
+ = .
a21 a22 b21 b22 a21 + b21 a22 + b22
Note that the result is a matrix of the same order as the matrices being
added. Note also that matrix addition is commutative, so [A]+[B]=[B]+[A].
Multiplication of a matrix by a scalar is straightforward; you simply
multiply each element of the matrix by the scalar. Thus
   
3 −2 9 −6
3A = 3 =
1 0 3 0
and generally
   
a11 a12 ka11 ka12
kA = k =
a21 a22 ka21 ka22
You can use the rules for addition of matrices and scalar multiplication
to see that subtraction of matrices is accomplished simply by subtracting
the corresponding elements. Thus [A]-[B]=[A]+(-1)[B]. So if
 
a11 a12
A=
a21 a22
and
 
b11 b12
B=
b21 b22
then
   
a11 + (−1)b11 a12 + (−1)b12 a11 − b11 a12 − b12
A−B = = .
a21 + (−1)b21 a22 + (−1)b22 a21 − b21 a22 − b22
Just as for addition, subtraction of matrices only works for matrices of
the same order.

B) Matrix multiplication

There are several different ways to multiply matrices; the most common
and most relevant to the vector and tensor concepts in SGVT is to
multiply two matrices (call them A and B) by multiplying the elements
of each row in A by the elements of each column of B and then summing
the results. So if matrix A is given by
 
a11 a12 a13
a21 a22 a23
4 Matrix Algebra Review

and matrix B is given by

 
b11 b12 b13
 b21 b22 b23 
b31 b32 b33

then the product A times B is given by

 
  b11 b12 b13
a11 a12 a13
× b21
 b22 b23 
a21 a22 a23
b31 b32 b33

 
a11 b11 + a12 b21 + a13 b31 a11 b12 + a12 b22 + a13 b32 a11 b13 + a12 b23 + a13 b33
= .
a21 b11 + a22 b21 + a23 b31 a21 b12 + a22 b22 + a23 b32 a21 b13 + a22 b23 + a23 b33

This result is achieved by multiplying the elements of the first row of

A by the elements of the first column of B, summing the results, and
placing the sum in the first row and first column of the result matrix,
as shown in Figure ??.

Sum products
Multiply first row
by first column Place result in first row
and first column
{

b11 ( ) ( )
a11 a12 a13 a11b11+ a12b21+ a13b31 ( ) ( )
x b21 ( ) ( ) =
( ) ( ) ( ) ( ) ( ) ( )
b31 ( ) ( )

Figure 1.1 Multiplying first row by first column

The next step is to multiply the elements of the first row of A by the
elements of the second column of B, summing the products, and placing
the sum in the first row and second column of the result matrix, as
shown in Figure ??.

After multiplying the first row of A by the third column of B and placing
the sum in the first row and third column of the result matrix, the
same procedure is done with the second row of A - those elements are
multiplied by the first column of B, summed, and placed in the second
 Matrix Algebra Review 5

Sum products
Multiply first row
by second column
Place result in first row
and second column

{
( ) b12 ( )
a11 a12 a13 ( ) a11b12+ a12b22+ a13b32 ( )
x ( ) b22 ( ) =
( ) ( ) ( ) ( ) ( ) ( )
( ) b32 ( )

Figure 1.2 Multiplying first row by second column

Multiply second row

by first column

b11 ( ) ( ) ( ) ( ) ( )
( ) ( ) ( )
x b21 ( ) ( ) =
a21 a22 a23 a21b11+ a22b21+ a23b31 ( ) ( )
b31 ( ) ( )
Place result in second row
{

and first column

Sum products

Figure 1.3 Multiplying second row by first column

row and first column of the result matrix, as shown in Figure ??.

Notice that the matrix that results from multiplying a 2x3 matrix (A)
by a 3x3 matrix (B) is a 2x3 matrix - the result has the number of rows
of the first matrix and the number of columns of the second matrix. You
should also note that this type of matrix multiplication only works when
the number of columns of the first matrix (3 in the case of A) equals the
number of rows of the second matrix (also 3 in the case of B).
To understand why matrices are multiplied in this way, consider the
sorting of five types of toy marbles (Bowlers, Cat’s Eyes, Steelies, Aggies,
and Commons) into four sizes of package (Small, Medium, Large, and
Extra Large). The number of each type of marble in each size package
6 Matrix Algebra Review

is shown in the following table:

Bowlers Cat’s Eyes Steelies Aggies Commons

Small package 1 3 2 5 4
Medium package 2 5 3 7 6
Large package 5 10 5 12 10
Extra Large package 10 15 12 16 14

This array of numbers can be put into a matrix - call it “P” the package
matrix:
 
1 3 2 5 4
 2 5 3 7 6 
P =  5 10 5 12 10 


10 15 12 16 14

Now imagine three containers (Cans, Boxes, and Crates) with different
numbers of packages in each type:

Small Pkgs Medium Pkgs Large Pkgs Extra-Large Pkgs

Can 10 8 5 2
Box 15 12 7 3
Crate 40 25 15 10

Put this array into a matrix called “C” the container matrix:
 
10 8 5 2
C =  15 12 7 3 
40 25 15 10

If you wish to find the number of each type of marble in each type of
container, you could do it like this:

Bowlers per Can = 10 small packages per can × 1 Bowler per small
package
+ 8 medium packages per can × 2 Bowlers per medium package
+ 5 large packages per can × 5 Bowlers per large package
+ 2 extra-large packages per can × 10 Bowlers per extra-large package
 Matrix Algebra Review 7

= (10)(1)+(8)(2)+(5)(5)+(2)(10)=71 Bowlers per Can

Likewise, you can find the number of Cat’s Eyes per can using

Cat’s Eyes per Can=10 small packages×3 Cat’s Eyes per small package
+ 8 medium packages × 5 Cat’s Eyes per medium package
+ 5 large packages × 10 Cat’s Eyes per large package
+ 2 extra-large packages × 15 Cat’s Eyes per extra-large package
= (10)(3)+(8)(5)+(5)(10)+(2)(15)= 150 Cat’s Eyes per Can

And if you wished to find the number of Bowlers per Box, you could use

Bowlers per Box = 15 small packages × 1 Bowler per small package

+ 12 medium packages × 2 Bowlers per medium package
+ 7 large packages × 5 Bowlers per large package
+ 3 extra-large packages × 10 Bowlers per extra-large package
= (15)(1)+(12)(2)+(7)(5)+(3)(10)=104 Bowlers per Box

If you compare the numbers in these calculations to the values in the P

and C matrices, you’ll see that in each case you’re multiplying the row
elements of C by the column elements of P, which is exactly how matrix
multiplication works:
 
  1 3 2 5 4  
10 8 5 2  2 5 3 7 6  71 150 93 198 166
CP =  15 12 7 3   5 10 5 12 10 
 =  104 220 137 291 244 
40 25 15 10 265 545 350 715 600
10 15 12 16 14
So the product matrix shows you how many of each type of marble are
in each type of container:

Bowlers Cat’s Eyes Steelies Aggies Commons

Can 71 150 93 198 166

Box 104 220 137 291 244
Crate 265 545 350 715 600

In addition to showing why the rows of the first matrix are multi-
plied by the columns of the second matrix, this example also illustrates
the point made earlier: when you multiply two matrices, the number of
columns in the first matrix must equal the number of rows in the sec-
8 Matrix Algebra Review

ond matrix. So in this case, multiplication works only if you multiply C

by P, not P by C. The larger point is that matrix multiplication is not
commutative, so even if you’re able to multiply two matrices in either
order (which you can do, for example, with square matrices), the answer
is not the same. So in matrix world, AB is in general not equal to BA.
Another difference between matrix multiplication and multiplication
of numbers is that it’s possible to get zero as the result of multiplying
two matrices, even if neither matrix is zero. For example,
     
2 1 1 0 0 0
× = .
0 0 −2 0 0 0
If you study vectors and tensors, you’re likely to come across the mul-
tiplication of a row vector times a column vector or vice versa, and you
should understand that such products use the same process described
above for matrix multiplication, although that may not be immediately
obvious when you look at the result. For example, multiplying a row
vector A by a column vector B gives
 
  b11
a11 a12 a13 ×  b21  = [a11 b11 + a12 b21 + a13 b31 ] .
b31
This scalar result (essentially the dot product between A and B) comes
from multiplying the first (and only) row of A by the first (and only)
column of B and adding the sums. Likewise, if A is a column vector and
B a row vector, the product AB is
   
a11   a11 b11 a11 b12 a11 b13
 a21  × b11 b12 b13 =  a21 b11 a21 b12 a21 b13 
a31 a31 b11 a31 b12 a31 b13
Once again, the same rule of matrix multiplication has been applied.
The first row of A (just a11 in this case) is multiplied by the first column
of B (just b11 in this case), and since there are no other elements in the
first row of A or the first column of B, there is nothing to add, and the
result (a11 b11 ) is written in the first row and column of the result matrix.
Then the first row of A (again, only a11 ) is multiplied by the second row
of B (only b12 ), and the result is written in the first row, second column
of the result matrix. After doing the same for the first row of A and the
third column of B, you then multiply the second row of A (which is just
a21 in this case) by the first column of B (again just b11 ) and write the
result in the second row, first column of the result matrix. So although
 Matrix Algebra Review 9

you get a scalar when you multiply a row vector by a column vector
(sometimes called the “inner product” of the matrices) and a matrix
when you multiply a column vector by a row vector (sometimes called
the “outer product” of the matrices), the process is exactly the same in
both cases.
Although matrix multiplication is not commutative (so AB is not nec-
essarily equal to BA), matrix multiplication is associative and distribu-
tive over addition, so for matrices A, B, and C

(AB)C = A(BC)

and
A(B + C) = AB + AC

as long as you remember not to reverse the order of any of the products.
You may be wondering if there’s ever a need to multiply each element
of one matrix by the corresponding element of an equal-size matrix.
There certainly is (for example, when applying a two-dimensional fil-
ter function to an image). This process is called “element-by-element”
or “entrywise” matrix multiplication and the result is sometimes called
the “Hadamard product.” In MATLAB, such multiplication is denoted
A.*B, where the decimal point before the multiplication symbol signifies
element-by-element multiplication.

C) Transpose and trace of a matrix

The transpose of a matrix is accomplished by interchanging the rows

with the columns of that matrix. This is usually denoted by placing the
superscript “T” after the matrix name, so the transpose of matrix A is
written as AT (MATLAB uses A0 ). So if the matrix A has elements
 
a11 a12 a13 a14
A=
a21 a22 a23 a24
then the transpose of A is given by
 
a11 a21
a12 a22 
AT = 

.
 a13 a23 
a14 a24

So (Aij )T = Aji ; notice that the indices have been switched. The trans-
pose of the product of two matrices is equal to the transpose of each
10 Matrix Algebra Review

matrix multiplied in reverse order, so

(AB)T = B T AT .
so long as the dimensions of the matrices allow such products to be
formed.
For square matrices, the trace of the matrix is given by the sum of the
diagonal elements. The trace is usually denoted by “Tr” so the trace of
matrix A is written as Tr(A). So if the matrix A is given by
 
a11 a12 a13 a14
 a21 a22 a23 a24 
A=  a31 a32 a33 a34 


a41 a42 a43 a44

the trace of A is
X
T r(A) = a11 + a22 + a33 + a44 = aii .

D) Determinant, minors, and cofactors of a matrix

The determinant of a square matrix is a scalar calculated by multiplying

and subtracting various elements of the matrix. If A is a 2x2 matrix
 
a11 a12
A=
a21 a22
the determinant of A (denoted using vertical bars on each side of A) is
found by cross-multiplying the upper-left element times the lower-right
element and then subtracting the product of the lower-left element times
the upper-right element:
 
 a11 a12 
|A| = 
  = a11 a22 − a21 a12 .
a21 a22 
Hence if A is
 
4 −2
A=
3 1
the determinant of A is
 
 4 −2 
|A| =   = (4)(1) − (3)(−2) = 4 − (−6) = 10.
3 1 
To find the determinant of higher-order matrices, you must use the
 Matrix Algebra Review 11

“minors” and “cofactors” of the determinant of the matrix. The minor

for each element of the determinant of a square matrix is found by
eliminating the entire row and the entire column in which the element
appears and then writing the remaining elements as a new determinant,
with row and column dimensions reduced by one (so the minor of each
element of a 3x3 determinant is a 2x2 determinant). So for the 3x3
determinant of matrix A
 
 a11 a12 a13 
 
|A| =  a21 a22 a23 
 a a32 a33 
31

the minor of element a11 is

 
 − − −  
  a22

 a23 
 − a22 a23 =
  a32
 a33 
 − a32 a33 

obtained by eliminating the first row and column of the determinant of

matrix A (since the element for which this minor is taken (a11 ) is in
the first row and the first column). Likewise, the minor for element a12
is found by eliminating the row and column in which a12 appears (first
row and second column):
 
 − − −   
 a21 − a23  =  a21 a23  .
   
   a31 a33 
 a
31 − a33 
For element a22 , the minor is
 
 a11 − a13  
  a11

 a13 
 − − − =
  a31 .
 a33 
 a31 − a33 

Before you can use minors to find the determinant of a higher-order

matrix, you have to turn each minor into a “cofactor.” The cofactor of
an element is just the minor of that element multiplied by either +1 or
-1. To know which of these to use, just determine whether the sum of
the element’s row index and its column index is even or odd. If even (as
it is, for example, for the element in the first row and first column, since
1+1=2 which is even), the cofactor equals the minor of that element
multiplied by +1. If odd (for example, for the element in the first row
and second column, since 1+2=3 which is odd) the cofactor equals the
minor of that element multiplied by -1. So for element aij with row index
12 Matrix Algebra Review

of i and column index of j,

Cofactor of (aij ) = (−1)(i+j) Minor of (aij ).

So for the minors shown above, the cofactors are

 
(1+1)  a22 a23 

Cofactor of (a11 ) = (−1)  a32 a33  = (1)(a22 a33 − a32 a23 )

 
 a a23 
Cofactor of (a12 ) = (−1)(1+2)  21 = (−1)(a21 a33 − a31 a23 )
a31 a33 
and
 
 a a13 
Cofactor of (a22 ) = (−1)(2+2)  11 = (1)(a11 a33 − a31 a13 ).
a31 a33 
With cofactors in hand, the determinant of a 3x3 matrix is straight-
forward to find. Simply pick one row or column, and multiply each of
the elements of that row or column by its cofactor and sum the results.
So, for example, if you choose to use the first row of the 3x3 matrix A,
the determinant is given by

|A| = a11 (Cofactor of a11 ) + a12 (Cofactor of a12 ) + a13 (Cofactor of a13 ).

or

|A| = a11 (1)(a22 a33 −a32 a23 )+a12 (−1)(a21 a33 −a31 a23 )+a13 (1)(a21 a32 −a31 a22 ).

The value of the determinant would have been the same if you had used
a different row (or any of the columns) of matrix A, as long as you
multiply each element in your selected row or column by its cofactor
and sum the results.
The same approach can be used to find the determinant of 4x4 and
higher-order matrices; you just have to expand each of the determinants
using elements and cofactors until you get down to the 2x2 level. Of
course, the calculation becomes a bit tedious, so it’s worth your time to
learn to find determinants using a program like MATLAB.
Some useful characteristics of determinants are that the determinant
of the transpose of a matrix equals the determinant of the matrix (so
|AT | = |A|) and the determinant of the product of two matrices is the
same as the determinant of the reverse product (so |AB| = |BA|) pro-
vided that the dimensions of A and B allow both of these products to
be made.
You may also find it useful to know that if a matrix has two identical
 Matrix Algebra Review 13

rows or columns, or if one row or column is an integer multiple of an-

other row or column, the determinant of that matrix must be zero.

E) Inverse of a matrix

As indicated in Section B of this document, matrix multiplication is

quite different from ordinary multiplication of numbers. One example of
this is that the product of two matrices may be the zero matrix even if
both of the matrices are non-zero (so you can’t simply divide both sides
of a matrix equation such as AB = 0 by A to get B = 0). Additionally,
the matrix equation AB = AC does not mean that matrix B equals
matrix C (so dividing both sides of this equation by A does not work
for matrices).
Differences such as this suggest that matrix division has little in com-
mon with ordinary division of numbers. About the closest you can get
to a process similar to division for matrices comes about by considering
the matrix equation
AX = B

where A, X, and B are all matrices. If you wish to find matrix X from
this equation, here’s one thing you definitely cannot do:
AX B
=
A A
B
X=
A
because this type of division does not work for matrices.
What you can do to find X is this: you can try to find a matrix A−1
(called the “inverse” or “reciprocal” of A) with the following property:

A−1 A = I

where I represents the identity matrix (ones on the diagonal and zeroes
everywhere else). Now you can find X rather easily:

AX = B
−1
A (AX) = A−1 B
−1
(A A)X = A−1 B
IX = A−1 B
X = A−1 B
14 Matrix Algebra Review

since IX = X (multiplying the identity matrix times any matrix does

not change that matrix).
So although you haven’t really divided matrix B by matrix A, you’ve
used the matrix equivalent of multiplying by the reciprocal of a number
to achieve the same result as dividing.
The question is, does every matrix have an inverse (that is, another
matrix with which it multiplies to give the identity matrix), and if so,
how do you find it? The first part of this question is easy to answer:
any matrix that has a non-zero determinant (|A| = 6 0) has an inverse.
Matrices that have no inverse ((|A| = 0) are called “singular” matrices,
and matrices that have an inverse are called “non-singular” matrices.
But if a certain matrix has an inverse, how do you find that inverse?
There are several ways to go about this, but one approach is to use
the concepts of cofactors, transpose, and determinant described above.
Those concepts appear in the following equation for the inverse of matrix
A:
(Matrix of cofactors of A)T
A−1 =
|A|
in which the “Matrix of cofactors of A” is a matrix in which each element
of A is replaced by that element’s cofactor.
Here’s how that works for a 3x3 matrix. If matrix A is the usual
 
a11 a12 a13
A =  a21 a22 a23 
a31 a32 a33
then the matrix of cofactors (MoC) for A is
 
a22 a33 − a32 a23 −(a21 a33 − a31 a23 ) a21 a32 − a31 a22
MoC (A) =  −(a12 a33 − a32 a13 ) a11 a33 − a31 a13 −(a11 a32 − a31 a12 )  .
a12 a23 − a22 a13 −(a11 a23 − a21 a13 ) a11 a22 − a21 a12
The equation for the inverse of A shown above requires that you take
the transpose of this matrix, which is
 
a22 a33 − a32 a23 −(a12 a33 − a32 a13 ) a12 a23 − a22 a13
[MoC (A)]T =  −(a21 a33 − a31 a23 ) a11 a33 − a31 a13 −(a11 a23 − a21 a13 )  .
a21 a32 − a31 a22 −(a11 a32 − a31 a12 ) a11 a22 − a21 a12
Dividing this matrix by the determinant of A provides the inverse of A.
Fortunately, the inverse of non-singular matrices is easily found using
most scientific calculators or computers, usually by defining a matrix A
and then raising A to the power of minus one.
 Matrix Algebra Review 15

For a diagonal matrix, the inverse is simply another diagonal matrix

in which each diagonal element is the reciprocal of the corresponding
element in the original matrix. So
−1  1 
0 0

a11 0 0 a11
1
 0 a22 0  =   0 0 .

a22
1
0 0 a33 0 0 a33

F) Simultaneous linear equations and Cramer’s Rule

The relationship between matrices and simultaneous linear equations

can be understood by considering equations such as

2x + 5y − z = 12
−3x − 3y + z = −1
x + y = 1.

Using the rules of matrix multiplication, this system of equations can be

written as a single matrix equation:
     
2 5 −1 x 12
 −3 −3 1  ×  y  =  −1  .
1 1 0 z 1

In general, three linear equations in three unknowns (x1 , x2 , x3 ) can be

written as

a11 x1 + a12 x2 + a13 x3 = b1

a21 x1 + a22 x2 + a23 x3 = b2
a31 x1 + a32 x2 + a33 x3 = b3

or
     
a11 a12 a13 x1 b1
 a21 a22 a23  ×  x2  =  b2  .
a31 a32 a33 x3 b3

If you define the matrices

 
a11 a12 a13
A =  a21 a22 a23 
a31 a32 a33
16 Matrix Algebra Review

and
 
x1
x =  x2 
x3
and
 
b1
b =  b2 
b3
then the system of equations can be written as a single matrix equation:

Ax = b

which can sometimes be solved using the inverse of matrix A:

x = A−1 b.

The reason that the word “sometimes” appears in the previous sentence
is that there are several conditions which will prevent this approach from
succeeding. For example, the system of equations may be inconsistent -
that is, there may be no set of values for x1 , x2 , x3 that satisfy all three
equations. In that case, you will not be able to find the inverse of A
because it will be a singular matrix. And if matrix b equals zero, you
have a system of homogeneous linear equations, which means there will
be only the trivial solution (x1 = x2 = x3 = 0) if A is non-singular or
an infinite number of solutions if A is singular.
In cases for which A is non-singular and b does not equal zero, you
can find the values of x1 , x2 , and x3 by finding the inverse of matrix
A and multiplying that inverse by matrix b, or you can use Cramer’s
Rule. In that approach, the first unknown (x1 in this case) is found by
replacing the values in the first column of the coefficient matrix (A) with
the elements of matrix b and dividing the determinant of that matrix by
the determinant of A. Here’s how that looks:
 
 b1 a12 a13 
 
 b2 a22 a23 
 
 b a
32 a33

3
x1 =  .
 a11 a12 a13 
 
 a21 a22 a23 
 
 a
31 a32 a33 

Likewise, to find the second unknown (x2 in this case), replace the values
 Matrix Algebra Review 17

in the second column of A with the elements of b:

 
 a11 b1 a13 

 a21 b2 a23 

 a b3 a33 
31
x2 =  
 a11 a12 a13 

 a21 a22 a23 

 a a a 
31 32 33

and to find x3 use

 
 a11 a12 b1 
 
 a21 a22 b2 
 
 a
31 a32 b3

x3 =  .
 a11 a12 a13 
 
 a21 a22 a23 
 
 a
31 a32 a33 

Thus for the equations given at the start of this section,

 
2 5 −1
A = −3
 −3 1 
1 1 0

   
x1 x
x =  x2  =  y 
x3 z

and
   
b1 12
b =  b2  =  −1  .
b3 1

Hence
 

 12 5 −1 


 −1 −3 1 

 1 1 0 
x = x1 =  

 2 5 −1 

 −3 −3 1 
 1 1 0 
18 Matrix Algebra Review

so
12[(−3)(0) − (1)(1)] + 5(−1)[(−1)(0) − (1)(1)] − 1[(−1)(1) − (1)(−3)]
x=
2[(−3)(0) − (1)(1)] + 5(−1)[(−3)(0) − (1)(1)] − 1[(−3)(1) − (1)(−3)]
−12 + 5 − 2
= = −3.
−2 + 5 − 0
Proceeding in the same way for y and z gives
 
 2
 12 −1 

 −3 −1 1 
 
 1 1 0 
y = x2 =  
 2
 5 −1 
 −3 −3 1 

 1 1 0 
so
−2 + 12 + 2
y= =4
−2 + 5 − 0
and
 

 2 5 12 


 −3 −3 −1 

 1 1 1 
z = x3 =  

 2 5 −1 

 −3 −3 1 
 1 1 0 
so
−4 + 10 + 0
z= = 2.
−2 + 5 − 0

G) Matrix diagonalization using eigenvectors and eigenvalues

In the matrix equation discussed in the previous section

Ax = b
the matrix A operates on matrix x to produce matrix b. If x and b
are vectors, this equation represents the transformation by matrix A
of vector x into vector b. In some cases, the operation of A on the
components of x produces the components of a vector b that is a scaled
(but not rotated) version of x. In such cases, the equation becomes
Ax = λx
 Matrix Algebra Review 19

where λ represents a scalar multiplier (and scalar multipliers can change

the length but not the direction of a vector).
Any vector x which satisfy this equation for a matrix A is called an
“eigenvector” of matrix A, and the scalar λ is called the “eigenvalue”
associated with that eigenvector.
The eigenvalues of a matrix can be very useful in finding a diagonal
version of that matrix, so you may wish to understand how to find the
eigenvalues of a given matrix. To do that, write the previous equation
as
Ax − λx = 0

which, since Ix = x, can be written as

Ax − λ(Ix) = 0

or
(A − λI)x = 0.

which means that either x = 0 (which is a the trivial case) or

|A − λI| = 0.

This equation is called the “characteristic equation” for matrix A, and

for a 3x3 matrix it looks like this:
   
 a11 a12 a13   1 0 0 
   
 a21 a22 a23  − λ  0 1 0 =0
   
 a
31 a32 a33   0 0 1 

or
 
 a11 − λ a12 a13 
 
 a21
 a22 − λ a23  = 0.

 a
31 a32 a33 − λ 

This expands to

(a11 − λ)[(a22 − λ)(a33 − λ) − a32 a23 ]

+ a12 (−1)[a21 (a33 − λ) − a31 a23 ]
+ a13 [a21 a32 − a31 (a22 − λ)] = 0.

Finding the roots of this polynomial provides the eigenvalues (λ) for
matrix A, and substituting those values back into the matrix equation
Ax = λx allows you to find eigenvectors corresponding to each eigen-
value. The process of finding the roots is less daunting than it may
20 Matrix Algebra Review

appear, as you can see by considering the following example. For the
3x3 matrix A given by
 
4 −2 −2
A =  −7 5 8 
5 −1 −4
the characteristic equation is
 
 4−λ −2 −2 
 
 −7 5 −λ 8 =0
 
 5 −1 −4 − λ 
or
(4 − λ)[(5 − λ)(−4 − λ) − (−1)(8)]
− 2(−1)[(−7)(−4 − λ) − (5)(8)]
− 2[(−7)(−1) − (5)(5 − λ)] = 0.
Multiplying through and subtracting within the square brackets makes
this
(4 − λ)(λ2 − λ − 12) + 2(7λ − 12) − 2(5λ − 18) = 0
or
−λ3 + 5λ2 + 12λ − 36 = 0.
Finding the roots of a polynomial like this is probably best left to a
computer, but if you’re lucky enough to have a polynomial with integer
roots, you know that each root must be a factor of the term not involving
λ (36 in this case). So (+/-) 2,3,4,6,9,12,and 18 are possibilities, and it
turns out that +2 works just fine:
−(2)3 + 5(22 ) + 12(2) − 36 = −8 + 20 + 24 − 36 = 0.
So you know that one root of the characteristic equation (and hence one
eigenvalue) must be +2. That means you can divide a factor of (λ − 2)
out of the equation and try to see other roots in the remainder. That
division yields this:
−λ3 + 5λ2 + 12λ − 36
= −λ2 + 3λ + 18.
(λ − 2)
The roots remaining polynomial on the right-hand side of this equation
are +6 and -3, so you now have
−λ3 + 5λ2 + 12λ − 36 = (λ − 2)(6 − λ)(λ + 3) = 0.
 Matrix Algebra Review 21

So matrix A has three distinct eigenvalues with values +6, -3, and +2;
these are the factors by which matrix A scales its eigenvectors. You
could find the eigenvectors of A by plugging each of the eigenvalues
back into the characteristic equation for A, but as long as you can find
N distinct eigenvalues for an NxN matrix, you can be sure that A can
be diagonalized simply by constructing a new diagonal matrix with the
eigenvalues as the diagonal elements. So in this case, the diagonal matrix
(call it A0 ) associated with matrix A is
 
6 0 0
A0 =  0 −3 0  .
0 0 2

To see why this is true, consider the operation of matrix A on each of

its three eigenvectors (call them e, f , and g):

Ae = λ1 e
Af = λ2 f
Ag = λ3 g

Now imagine a matrix E whose columns are made up of the eigenvectors

of matrix A:
 
e1 f1 g1
E =  e2 f2 g2 
e3 f3 g3

where the components of eigenvector e are (e1 , e2 , e3 ), the components

of eigenvector f are (f1 , f2 , f3 ), and the components of eigenvector g are
(g1 , g2 , g3 ). Multiplying matrix A (the original matrix) by E (the matrix
made up of the eigenvectors of A), you get
   
a11 a12 a13 e1 f1 g1
AE =  a21 a22 a23  ×  e2 f2 g2 
a31 a32 a33 e3 f3 g3

which is
 
a11 e1 + a12 e2 + a13 e3 a11 f1 + a12 f2 + a13 f3 a11 g1 + a12 g2 + a13 g3
AE =  a21 e1 + a22 e2 + a23 e3 a21 f1 + a22 f2 + a23 f3 a21 g1 + a22 g2 + a23 g3 
a31 e1 + a32 e2 + a33 e3 a31 f1 + a32 f2 + a33 f3 a31 g1 + a32 g2 + a33 g3

The columns of this AE matrix are the result of multiplying A by each

of the eigenvectors. But you know from the definition of eigenvectors
22 Matrix Algebra Review

and eigenvalues that

   
e1 λ1 e1
Ae = λ1 e = λ1  e2  =  λ1 e2 
e3 λ1 e3
and
  
f1 λ2 f1
Af = λ2 f = λ2  f2  =  λ2 f2 
f3 λ2 f3
and
   
g1 λ3 g1
Ag = λ3 g = λ3  g2  =  λ3 g2  .
g3 λ3 g3
This means that the product AE can be written
 
λ1 e1 λ2 f1 λ3 g1
AE =  λ1 e2 λ2 f2 λ3 g2  .
λ1 e3 λ2 f3 λ3 g3
But the matrix on the right-hand side can also be written like this:
     
λ1 e1 λ2 f1 λ3 g1 e1 f1 g1 λ1 0 0
 λ1 e2 λ2 f2 λ3 g2  =  e2 f2 g2  ×  0 λ2 0 
λ1 e3 λ2 f3 λ3 g3 e3 f3 g3 0 0 λ3
This means that you can write
 
λ1 0 0
AE = E  0 λ2 0 
0 0 λ3
and multiplying both sides by the inverse of matrix E (E −1 ) gives
   
λ1 0 0 λ1 0 0
E −1 AE = E −1 E  0 λ2 0  =  0 λ2 0  .
0 0 λ3 0 0 λ3
If you tracked the discussion of the similarity transform in Section 6.1,
you’ll recognize the expression E −1 AE as the similarity transform of ma-
trix A to a coordinate system with basis vectors that are the columns
of matrix E. Those columns are the eigenvectors of matrix A, and the
matrix that results from the similarity transform (call it A0 ) is diagonal
and has the eigenvalues of A as its diagonal elements.
 Matrix Algebra Review 23

References

Pettofrezzo, A., Matrices and Transformations, Prentice-Hall 1966.

Riley, K., Hobson, M.P., and Bence, S.J. Mathematical Methods for
Physics and Engineering, Cambridge 2006.

Schwartz, J.T., Introduction to Matrices and Vectors, McGraw-Hill 1961.