1.

1 Matrices and Vectors De…ned I

An mxn matrix, A, is a rectangular array of numbers arranged into m

rows and n columns and is written as
2 3
a11 a12 . . . a1n
6 a21 . . . . a2n 7
6 7
6 . . . . . . 7
6 7
A =6 .6 . . . . . 77. (1.1)
(mxn ) 6 . . . . . . 7
6 7
4 . . . . . . 5
am1 . . . . amn

The element of A in row i, column j is denoted aij ,where the …rst

subscript identi…es the row and the second subscript identi…es the
column to which the element belongs.

(Institute) Lecture Notes Monash University 4 / 156

1.1 Matrices and Vectors De…ned II

The number of rows in A and the number of columns in A are

respectively referred to as the row dimension and the column
dimension of A. If A contains m rows and n columns A is called an
m by n matrix (written mxn).
If A has the same number of rows and columns, A is called a square
matrix.
A matrix with one row and n columns is called a1xn row vector. For
example,
a0 = a11 a12 . . . a1n
(1xn )

is a 1xn row vector.

(Institute) Lecture Notes Monash University 5 / 156

1.1 Matrices and Vectors De…ned III
A matrix with one column and m rows is called an mx1 column
vector. For example, 2 3
a11
6 a21 7
6 7
6 . 7
a =6 6 .
7
7
(mx 1 ) 6 7
4 . 5
am1
is a mx1 column vector.
A vector is a special case of a matrix.
We will adopt the convention of using upper case letters to denote
matrices and lower case letters to denote vectors.
We will also adopt the convention of using a prime to denote a row
vector. For example, a denotes a column vector and a0 denotes a row
vector.
(Institute) Lecture Notes Monash University 6 / 156
1.1 Matrices and Vectors De…ned IV

Whenever a vector has n elements, it is called a vector of order

n. Both the 1xn row vector a0 and the nx1 column vector a are of
order n.

(Institute) Lecture Notes Monash University 7 / 156

1.2 Elementary Matrix Operations I
1.2.1 Matrix addition and subtraction

Two matrices A and B may be added together or subtracted from

each other if and only if both matrices have the same row dimension
and the same column dimension. When this condition is satis…ed, A
and B are said to be conformable for addition.
Matrix addition involves adding the corresponding elements of the
two matrices. For example, if

a11 a12 b11 b12

A = and B = ,
(2x 2 ) a21 a22 (2x 2 ) b21 b22

then
a11 + b11 a12 + b12
A+B = .
a21 + b21 a22 + b22

(Institute) Lecture Notes Monash University 8 / 156

1.2 Elementary Matrix Operations II
1.2.1 Matrix addition and subtraction

In general, if
C = A+B
then
cij = aij + bij .
For conformable matrices A and B, the operations of matrix addition
and subtraction have the following properties:

P1 Matrix addition is commutative. That is,

A + B = B + A.

P2 Matrix addition is associative. That is,

A + (B + C ) = (A + B ) + C .

(Institute) Lecture Notes Monash University 9 / 156

1.2 Elementary Matrix Operations III
1.2.1 Matrix addition and subtraction

P3 Cancellation property.

A + B = A + C ) B = C.

P4
A B = A + ( B ).
For example,

a11 a12 b11 b12

A = and B = ,
(2x 2 ) a21 a22 (2x 2 ) b21 b22

then
a11 b11 a12 b12
A B= .
a21 b21 a22 b22

(Institute) Lecture Notes Monash University 10 / 156

1.2 Elementary Matrix Operations I
1.2.2 Matrix multiplication

In scalar algebra any two numbers may be multiplied together.

However, not all matrices may be multiplied together.
The matrix product AB is de…ned if and only if the column
dimension of A (called the lead matrix i.e. the …rst matrix in the
product) is the same as the row dimension of B (called the lag
matrix i.e. the second matrix in the product).
The requirement that the column dimension of the lead matrix must
be equal to the row dimension of the lag matrix is known as the
conformability condition for matrix multiplication. When this
condition is satis…ed, A and B are said to be conformable for
multiplication.

(Institute) Lecture Notes Monash University 11 / 156

1.2 Elementary Matrix Operations II
1.2.2 Matrix multiplication

For example, let

a11 a12 b11 b12 b13

A = and B = .
(2x 2 ) a21 a22 (2x 3 ) b21 b22 b23

The matrix product

C = A B
(2x 2 )(2x 3 )

is de…ned because the column dimension of A (2) is the same as the

row dimension of B (2). However, the matrix product

D= B A
(2x 3 )(2x 2 )

is not de…ned because the column dimension of B (3) is not the same
as the row dimension of A (2).

(Institute) Lecture Notes Monash University 12 / 156

1.2 Elementary Matrix Operations III
1.2.2 Matrix multiplication

In general, if A is mxn and B is pxq, then:

The product
C = A B
(mxn )(pxq )

is de…ned if and only if

n = p.
The product
D= B A
(pxq )(mxn )

is de…ned if and only if

q = m.
Notice that it is possible for the product AB to be de…ned (n = p)
but the product BA not to be de…ned (q 6= m).
When the product AB is de…ned, we say that A is post-multiplied B
and B is pre-multiplied by A.
(Institute) Lecture Notes Monash University 13 / 156
1.2 Elementary Matrix Operations IV
1.2.2 Matrix multiplication

Consider the special case in which A and B are vectors. In particular,

let
A = a0 = a11 a12 . . a1m
(1xm )

and 2 3
b11
6 b21 7
6 7
B= b =6 6 . 7.
7
(mx 1 ) 4 5
.
bm1
Since the column dimension of a0 = the row dimension of b, a0 and b
are conformable for matrix multiplication. The product

c = a0 b
(1x 1 ) (1xm )(mx 1 )

(Institute) Lecture Notes Monash University 14 / 156

1.2 Elementary Matrix Operations V
1.2.2 Matrix multiplication

is de…ned as the sum of each element of a multiplied by the

corresponding element of b (…rst element of a multiplied by the …rst
element of b, second element of a multiplied by the second element of
b etc.). That is,

c = a0 b
(1x 1 ) (1xm )(mx 1 )
2 3
b11
6 b21 7
6 7
= a11 a12 . . a1m 6 . 7
6 7
4 . 5
bm1
= (a11 b11 + a12 b21 + ... + a1m bm1 ). (1.2)

(Institute) Lecture Notes Monash University 15 / 156

1.2 Elementary Matrix Operations VI
1.2.2 Matrix multiplication

The result in (1.2) may be written more compactly as

m
c
(1x 1 )
= a0 b
(1xm )(mx 1 )
= ∑ a1i bi 1 (1.3)
i =1

Notice that the product of a row vector postmultiplied by a

conformable column vector is always a scalar.
Having de…ned the product of a row vector postmultiplied by a
conformable column vector, we can now de…ne the product of two
conformable matrices.

(Institute) Lecture Notes Monash University 16 / 156

1.2 Elementary Matrix Operations VII
1.2.2 Matrix multiplication

Let A and B be mxn and nxq matrices respectively and let

C = A B . (1.4)
(mxq ) (mxn )(nxq )

Then
2 3 2 3
c11 c12 . . c1q a10 b1 a10 b2 .. a10 bq
6 c21 c22 . . c2q 7 6 a20 b1 a20 b2 .. a20 bq 7
6 7 6 7
C =6 6 . . . . . 7=6
7 6 . . .. . 7 7.
(mxq ) 4 5 4
. . . . . . . .. . 5
cm1 cm2 . . cmq 0 b
am a 0 b .. am0 b
1 m 2 q
(1.5)
where ai0 denotes the ith row of A and bj denotes the jth column of
B.That is,
cij = ai0 bj . (1.6)

(Institute) Lecture Notes Monash University 17 / 156

1.2 Elementary Matrix Operations VIII
1.2.2 Matrix multiplication

The result in (1.6) states if C is the matrix we obtain when A is

post-multiplied by B, then the element in row i, column j of C is the
product of the ith row of A post-multiplied by the jth column of B.
For example,
c21 = a20 b1 .
Notice from
C = A B , (1.4)
(mxq ) (mxn )(nxq )

that C inherits its row dimension from A (the lead matrix) and
its column dimension from B (the lag matrix).

(Institute) Lecture Notes Monash University 18 / 156

1.2 Elementary Matrix Operations IX
1.2.2 Matrix multiplication

Example (1.1)
Let 2 3
2 3 1 5
7 8 2 5 6 8 3 7
A =4 4 1 4 3 5 and B = 6 4 9
7.
(3x 4 ) (4x 2 ) 2 5
10 2 8 9
10 6
Notice that since the column dimension of A equals the row dimension of
B, the product AB is de…ned and is given by

(Institute) Lecture Notes Monash University 19 / 156

1.2 Elementary Matrix Operations X
1.2.2 Matrix multiplication

Example (1.1 continued)

C = A B
(3x 2 ) (3x 4 )(4x 2 )
2 3 2 0 0 3
c11 c12 a1 b 1 a1 b 2
4 c21 c22 5 = 4 a0 b1 a0 b2 5
2 2
0 0
c31 c32 a3 b 1 a3 b 2
2 3
7x1 + 8x8 + 2x9 + 5.10 7.5 8.3 + 2.2 + 5.6
= 4 4.1 + 1.8 + 4.9 + 3.10 4.5 1.3 + 4.2 + 3.6 5
10.1 + 2.8 8.9 + 9.10 10.5 2.3 8.2 + 9.6

(Institute) Lecture Notes Monash University 20 / 156

1.2 Elementary Matrix Operations I
1.2.2 Matrix multiplication

Example (1.1 continued)

2 3
7 + 64 + 18 + 50 35 24 + 4 + 30
C = 4 4 + 8 + 36 + 30 20 3 + 8 + 18 5
(3x 2 )
10 + 16 72 + 90 50 6 16 + 54
2 3
139 45
= 4 70 3 5
44 82

(Institute) Lecture Notes Monash University 21 / 156

1.2 Elementary Matrix Operations II
1.2.2 Matrix multiplication

Notice that in this example even though the product AB is de…ned,

the product
D= B A
(4x 2 )(3x 4 )

is not de…ned because the conformability condition for matrix

multiplication is violated.

(Institute) Lecture Notes Monash University 22 / 156

1.2 Elementary Matrix Operations III
1.2.2 Matrix multiplication

When we postmultiply the 1xm row vector a0 by the mx1 column

vector b we obtain the scalar

c = a0 b
(1x 1 ) (1xm )(mx 1 )
2 3
b11
6 b21 7
6 7
= a11 a12 . . a1m 6 . 7
6 7
4 . 5
bm1
= (a1 b1 + a2 b2 + ... + am bm ) (1.7)

The operation of matrix multiplication can be shown to have the

following properties:

(Institute) Lecture Notes Monash University 23 / 156

1.2 Elementary Matrix Operations IV
1.2.2 Matrix multiplication

P1 If both AB and BA are de…ned then, in general,

AB 6= BA.

P2 Matrix multiplication is associative. That is, for conformable matrices

A, B, C
(AB )C = A(BC ).
P3 Matrix multiplication is distributive. That is, for conformable matrices
A, B, C
(A + B )C = AC + BC .

(Institute) Lecture Notes Monash University 24 / 156

1.2 Elementary Matrix Operations V
1.2.2 Matrix multiplication

P4 For any scalar c the matrix cA is obtained by multiplying each

element of the matrix A by the scalar c. For example, if

a11 a12
A=
a21 a22

then
ca11 ca12
cA = .
ca21 ca22

(Institute) Lecture Notes Monash University 25 / 156

1.2 Elementary Matrix Operations VI
1.2.2 Matrix multiplication

P5 For conformable matrices A and B

AB = 0 ; A = 0 or B = 0.

That is, if the product of two matrices is a null matrix, we cannot

conclude that one of the matrices is necessarily a null matrix. For
example, let
1 0 0 0
A= and B = .
1 0 2 2
Then
0 0
AB = .
0 0

(Institute) Lecture Notes Monash University 26 / 156

1.2 Elementary Matrix Operations VII
1.2.2 Matrix multiplication

P6 For any square matrix A

A2 = 0 ; A = 0.

For example, let

1
1 2
A= .
2 1
Then
1 1
1 1
A2 = 2 2
2 1 2 1
1 1
1 1 2+ 2
=
2 2 1+1
0 0
= .
0 0

(Institute) Lecture Notes Monash University 27 / 156