Lecture I.

Matrix and Matrix Operations

Linear Algebra mainly focuses on the study of linear equations. A mathematically elegant way of
describing linear equations, as we will see, is via vector/matrix notation. We shall then introduce matrices
and related algebraic operations, which will be proved useful in studying the solvability of linear equations.

1.0. Prelude: Why Vector and Matrix?

You have to work on a set of real numbers {x1, , x n }   , say, a data record of local temperature
measured at n weather stations in Hsinchu, at 8:00 am. Just little you can do with a set proper. So be wise to
 x1 
 
put the n scalars into a vector x     ; in this way, you associate {x1, , x n } with a geometric object in
 
x n 
the Euclidean space. Consider a toy example with n  3 and {10,15,12} . The temperature data is now
10
 
 
represented by a vector x  15 in  3 . Likewise, suppose you acquire another temperature data set
 
12
 
25
 
 
{25,27,29} , gathered at 3 stations in Tainan, and in turn represented by the vector y  27  . Think about the
 
29
 
benefit of using “vectors”: geometry comes into play, and things become visualizable!

Suppose the measurement process is conducted on an hour-by-hour basis. Hence, you come up with a data set
per day of size 24n , stored into 24 vectors, say, {x1, , x24 } for Hsinchu in one day while {y1, , y24 }

for Tainan. Now you are confronted with two sets of vectors, and a standard arrangement of a collection of
vectors is to stack them into a matrix:

X  x1  x24  and Y  y  y 


24 

  1 
n24 n24

1
Is there a geometry for matrices, the likes that geometry of vectors in Euclidean space? The answer is yes, and
will be elaborated more in this course. Before that, you have to master algebraic manipulations of matrices,
some of which you might have been familiar with during high school.

1.1. Definition of Matrix

A matrix is a rectangular array of numbers. If a matrix has m rows and n columns, it is said to be of
dimension m  n .
Example 1.1.1:
 1 2 1 0 0.8 1
     
      2
 3 0 , 2 1 0 3 ,  0 3 2  ,   , 6 

□
      
1 4 a 14 matrix  2 1 6   5  a 11 matrix, scalar
 
  
 
a 32 matrix a 33 matrix a 31 matrix

A general m  n matrix A can be written as

 a11  a1n 
 
A      

 
am 1  amn 
The element aij is the entry lying on the ith row and jth column, and is called the (ij)th entry of A. If the
matrix A is square, that is m  n , then aii is called the ith diagonal entry of A.

Notation:
mn denotes the set of all m  n matrices with real-valued entries.
mn denotes the set of all m  n matrixes with complex-valued entries.
 We will focus mainly on mn in our text.
We use A  mn to indicate that A is an m  n matrix with real-valued entries, and
A  [aij ]  mn to further emphasize that the (ij)th entry is given by aij whenever needed (note:

clever use of mathematical symbols and notation).

 A zero matrix is a matrix whose entries are all zero.

 0 0
 0 0  


e.g.   ,  0 0  could be square or non-square.

 0 0  

 0 0
1 , i  j
A square matrix is called an identity matrix if aij  
0 , i  j .

Usually an n  n identity matrix is denoted by In .

2
 1 0 0 
1 0   
   
e.g. I2    , I3  0 1 0  .
0 1   
0 0 1 
 

 A square matrix is diagonal if aij  0 for all i  j .

 3 0 0
   1 0
   
e.g.  0 2 0 ,  0 1
   
 0 0 4 
  an identity matrix is diagonal

1.2 Matrix Operations

1. Two matrices are said to be equal if
(1) They have the same number of rows and the same number of columns.
(2) The corresponding entries are equal.

 2 1 2x  1 y 
e.g. A    , B= 


4 3  z  y w
   
2  2x  1




 1y
A  B   , yielding (x , y, z , w )  (0.5,1, 3, 3)

 4  z y



 3w


2. Matrix Addition:

Let A  [aij ] and B = [bij ] be two m  n matrices. Then the sum A  B is again an m  n
matrix whose (ij)th entry is defined as aij  bij , that is, A  B  [aij  bij ]  mn .
1 1 2  5 1 0 
e.g. A    ,B = 
2  1 1.2   3 4 4 
   
1  5 0 2 
A  B   

 5 3 5.2 
2 1 
If C    , then A  C is undefined!

1 0
 
Note: Only matrices with the same dimension can be added.

3. Scalar Multiplication:

Let A  aij  be an m  n matrix and a be a scalar. To multiply A by a , we multiply each entry

3
 of A by a . The matrix aA  [aaij ] is called a scalar multiple of A .
1 2  2
4
e.g. A    , a  2 , 2A 


6

 3 4  8 
 
00
a  0 , 0A   
0
0 
1 2
a  1 , (1)A     A .

 3 4 
4. Matrix Subtraction:

Let A  aij  and B  bij  be m  n matrices. The difference A  B is again an m  n matrix

and is defined by A  B  [aij  bij ] ; notably, A  B  A  (B) .

2 3 4 0 2 7  9 6 3 
e.g. A    , B= 
 1 3 5
 ,C
 3 0 12

 1 3 1     
1  4 6 8  0 2 7  3 2 1  7 2 2 
Then 2A  B  C   
 

 

 
.

3 2 6 2  1 3 5 1 0 4 4 3 11
       
1 0  3  0   3 
e.g. A    , B=   and 3A  2B          .
2 1 6 2  8

5. Row-Column Product:
 c1 
 
Let r  [a1,..., aN ] be a 1  N row matrix (vector) and c=    be a N  1 column matrix (vector).
 
cN 
N
Then the row-column product is a scalar which is defined by rc  a1c1  a2c2    aN cN   aici .
i 1
 3
   1
   
e.g. r  1 2 3 , c  2 , rc  1  3  2  2  3  1  10. If c1    , rc1 is undefined.
   
 1  4
 
6. General Matrix Product:

Suppose A and B are two matrices. The product AB is defined if and only if the number of columns
of A equals the number of rows of B . Hence, if A   pq and B  mn , AB is defined if and
only if q  m . Now consider A  aij    pq , B  bij   qn , and C  AB   pn . Then cij ,
the (ij)th entry of C , is the product of the i th row of A and the j th column of B , that is,

4
 b 
 1j  q
   
cij  ai1  aiq       aikbkj .
   k 1
Ai bqj 

Bj

Hence
a11 a12  a1q 
  c11  c1 j  c1n 
a21 a22  a2q  b11 b12  b1 j  b1n   
       
   
  b21 b22  b2 j  b2n    
  ci1  cij  cin  pn
AB   a a  a  
   C .
 i1 i2 iq         
       
     bq 1 bq 2  bqj  bqn   
    cp1  cpj  cpn 
a p1 a p 2  a pq   
 
Write A in terms of its rows and B in terms of its columns, and then we can express the product AB
as
 A1   A1B1 A1B2  A1B j  A1Bn 
   
   
 A2   A2 B1 A2 B2  A2 B j  A2 Bn 
   
        
   
AB     B1 B2  B j  Bn      C. (1.2.1)
 Ai    A B A B  A B  A B 
   i 1 i 2 i j i n 
        
  
   
A  A B A B  A B  A B 
 p   p 1 p 2 p j p n


Example 1.2.1:
1 2 1
2 1 1   2 5 5
 
(1) A      23 , B 

33
1 1 2   , AB   
2 2 4 , but BA is not
1 2 1 
1

  
 2 1
defined.
 3  3 6 9
   
  31    
(2) A= 1   , B= 1 2 3   , AB  1 2 3   33 , BA=11  11 .
13
     
2  2 4 6
   
AB  BA since they are matrices with different dimensions. Note that in general AB  BA .
1 1 1 1  0 0
(3) A     0 , B= 
 1 1
  0 , and AB   
 0 0 . Hence A  0 and B  0 does not
1 1    
necessarily imply AB  0 !
1 3 2  3     
    1 3 2   3  2
        
(4) A   1 2 3 and B= 1 , which is a column vector. Then AB   1 2 3 1  5
        
2 1 2 2  2 1 2  2   9 
         
is again a column vector.

5
Important observation: In the previous example we have

  1  3 2
2      
       
5  3  1   (1) 2  2 3
       
9 2 1  2
  
     
a linear combination of
columns of A with
coefficients 3,  1, 2.
Let us look at another example:
 
2 1 3  x1  2x1  x 2  3x 3 
    
 4 6 1 x 2    4x  6x  x 
 
 x 3   1 2 3

:A  
2x1   x 2   3x 3  2  1  3 .
          x1    x 2    x 3  
4x 6x x 4 6 1
 1   2   3     
  
Again, a linear combination of
columns of A with
coefficients from the vector x !

 a11 a12  a1n   x1 

   
 a21 a22  a2n  x2 
In general, for A     mn , x     n1 , we have
      
 
a  x 
 m 1 am 2  amn   n 

 a11x1  a12x 2    a1n x n   a11   a12   a1n 

       
 a21x1  a22x 2    a2n x n   a21   a22   a2n 
Ax   
  x1    x 2    x 
 n 


          
a x  a x    a x  a  a  a 
 m 1 1 m2 2 mn n 
  m 1   m 2   mn 

a linear combination of columns of A
with coefficients from entrices of x

Hence, multiplying a matrix A from the right by a column vector x (if the multiplication is defined) can
be done by performing a linear combination of the columns of A with coefficients from the entries of x .

Remark: How about multiplying a matrix A from the left by a row vector y ?
 Performing a linear combination of the rows of A with coefficients from the entries of y !
Example:
 2 1 3
y y       3y1    4y2 6y2 y2 
 1 2   4 6 1  2y1  4y2 y1  6y2 3y1  y2   2y1 y1   
 
 y1 2 1 3  y2  4 6 1 .
   

6
7. Matrix Product as Linear Combinations of Columns (or Rows):

Let A   pq , with Ai  1q being the i th row, 1  i  p .

B  qm , with B j  q1 being the j th column, 1  j  m .

From 6 we know how to perform matrix multiplication entry-by-entry. In fact, multiplication can be done
column-by-column (or row-by-row). To see this we recall that

 A1   A1B1 A1B2  A1B j  A1Bn 

  
   
 A2   A2 B1 A2 B2  A2 B j  A2 Bn 
   
        
  
AB    B1 B2  B j 
 Bn    
 Ai    A B A B  Ai B j  Ai Bn   Ai B
   i 1 i 2 
        
  
   
A  A B A B  Ap B j  Ap Bn 
 p   p 1 p 2


AB j

Hence
AB  A  B1 B2  Bm    AB1 AB2  ABm 
   
 A1   A1B 
   
   
 A2   A2 B 
   B   

    
   
Ap   A p B
   

 j th column A
The   of AB is a linear combination of columns of    
  rows    with coefficients from the
 i th row    
B 
     
 j th column B

 

  of   .
 i th row  
A
    
1 2 1
2 1 1   2 5 5
e.g. A      23 , B  1 1 2   33 , and C  AB   
1 2  1    2 2 4 .
  1 2 1  
 
2 2  1 1
The 1st column of C     1    (1)    1   .
2 1 2 1
5 2  1 1
The 2nd column of C     2    (1)    2   .
2 1 2 1
 

7
 5 2  1 1
The 3rd column of C     1    2    1   .
 4  1 2 1
1 2 1
2 1 1   2 5 5
A      23 , B  1 1 2   33 , and C  AB   
1 2  1   2 2 4 .
  1 2 1  
 
The 1 row of C  2 5 5  2 1 2 1  1 1 1 2  1 1 2 1 .
st    
       
The 2nd row of C  2 2 4   1 1 2 1  2 1 1 2  (1) 1 2 1 .
       

8. Matrix Product as Column-Row Expansion:

We see in (1.2.1) the matrix product AB entry-wise in terms of row-column product (row of A
multiplied by column of B). What happens if we otherwise write A in terms of its columns, say, A j ’s, and B
in terms of its rows, say, Bi ’s? Then you check this (something like a “vector-version” of row-column
product that is hereafter called “column-row expansion”):

 B1 
 
 
 B2 
 
  
  n
AB   A1 A2  A j 
 An      Ak Bk .
   B  k 1
 i
  
 
 
 Bn 
 

 1
2 1 1  1 2
 
Consider again the example A      23 , B 
 1 1 2   33 . We have
1 2 1  
  1 2 1
 
2  1 1 2 5 5
C  AB    1 2 1    1 1 2    1 2 1   
1
  2 
 
 1   2 2 4
   

Some Properties of Matrix Product:

Theorem 1.2.2: Suppose a A  aij    pq

, , and B  bij   qn . Then
a AB  aA B  A aB , hereafter denoted as aAB .
[Proof]: Let a AB  C , aA B  D , A aB  E . To show C  D  E , by definition we have to
check (i) they have the same dimension (O.K.)
(ii) the (ij)th entries of the three matrices are the same.

8
 b1 j  b1 j 
   
b  b 
   2j  q
   2j  q
 cij  a ai1 ai 2  aiq      aaikbkj , dij  aai1 aai 2  aaiq      aaikbkj
     k 1      k 1
   
bqj  bqj 
   
 ab1 j 
 
ab 
 2j  q
eij  ai1 ai 2  aiq      aaikbkj
     k 1
 
 abqj 
 
Hence we have cij  dij  eij ,  1  i  p , and  1  j  n . This implies C  D  E . □

Theorem 1.2.3: Suppose A  aij   mp , B  bij    pq , and C  cij   qn .
Then AB C  A BC , i.e., the associative law holds.
[Proof]: Let D  AB , E  BC , hence F  AB C  DC , G  A BC  AE .
Check: F  G , i.e., fij  gij for all i, j . Note that
q p
fij   dikckj and dik   ailblk ,
k 1 l 1
p q
gij   ailelj and elj   blkckj .
l 1 k 1
p q 
Now we have gij   ail   blkckj  .
l 1
k 1 
q  p  q p p q p q 
Also fij    ailblk ckj    ailblkckj    ailblkckj   ail   blkckj   gij . □

k 1 l 1
 k 1 l 1 l 1 k 1  
l 1 k 1

Theorem 1.2.4: A  aij   mn , B  bij   mn , C  cij   nq , D  dij    pm . Show that
A  B C  AC  BC and D A  B  DA  DB .
[Proof]: Exercise. □

Fact 1.2.5: If either A=0 or B=0, then AB=0. However, it is possible that A  0 , B  0 but AB=0. □
1 1 1 1  0 0
e.g. A   , B

 
1 1 , AB 
 
 0 0 .
1 1    

8. Transpose of a Matrix:

Let A  mn . The transpose of A, denoted by AT , is an n  m matrix whose (ij)th entry is the (ji)th
entry of A.

9
 a 
a11 a12 a13   11 a21 
e.g. A  a    23 , AT  a12 a22    32
 21 a22 a23   
a13 a23 
A  a1 a2 a 3  , a row vector.
 
a 
 1
A  a2  , a column vector.
T

 
a 3 

Facts 1.2.6: The following results hold.

(a). AT   A .
T

T
(b). aA  aAT , for all a   .
T
(c). A  B  AT +BT .
T
(d). AB  BT AT , provided that AB is defined.
[Proof]: We only prove (d); (a)~(c) are left as exercises. Suppose A  aij    pq , B  bij   qn , and
again let AB=C, BT AT  D .
Check: CT  D
q
The (ij)th entry of C is cij   aikbkj , and this is by definition the (ji)th entry of CT .
k 1
Hence it suffices to show d ji  cij .
Indeed, d ji   j th row of BT  i th column of AT 
ai1 
 
ai 2  q q
   
 b1 j b2 j  bqj 
      b a
kj ik   aikbkj  cij . □
  k 1 k 1
entries from the a 
j th column of B  iq 

entries from the
i th row of A

9. Symmetric matrices:

A square matrix A is said to be symmetric if A  AT , or equivalently, aij  a ji .

1 1 2 
 
 
e.g. A  1 3 3 is symmetric.
 
2 3 2
 

Note: If A  nn and B  nn are symmetric and a, b   , then so are A  B , aA , aA  b B .

But AB is NOT necessarily symmetric!

10
 1 2  1 1 1 1
e.g. C     
 1 1   1 1

2 3   

A  AT B BT CCT

10. Lower triangular and upper triangular matrices:

A square matrix A  aij   nn is said to be lower triangular if aij  0 for all i<j, that is, the entries

above the main diagonal are all zero. A square matrix A is upper triangular if AT is lower triangular.

 1 0 0  1 3 5
   
   
e.g. 2 4 0 is lower triangular.  0 4 9 is upper triangular.
   
 3 5 6  0 0 6
   

11. Trace of a square matrix:

N
Let A  aij   nn . Then the trace of A , denoted by tr (A) , is defined by tr (A)   aii , that is, the
i 1

sum of all entries on the main diagonal of A .

1 1 2 
 
 
e.g. For A  1 3 3 , we have tr (A)  1  3  2  6 .
 
2 3 2
 
Remark: The trace of a non-square matrix is not defined.
Exercise: Let A  mn and B  nm . Prove that tr (AB)  tr (BA) .

12. Simple Functions of Matrices:

(1) Let A  nn . Then for a positive integer l  1 , Al  A A

   A

l times
nn l
(2) Let A   and p x   a 0  a1x    al x be a polynomial of order l.
l nn
Then p A  a 0 In  a1A    al A   .

1.3 Complex-Valued Matrices

A  aij   mn is an m  n matrix whose entries are complex numbers. The conjugate of A, denoted
by A , is an m  n matrix with (ij)th entry equal to a ij . The “conjugate transpose” of A, denoted by A* , is
defined to be A*  A . If A is a square matrix and A  A* , then A is said to be Hermitian.
T

11
  1 i 1  i 

 
e.g. A   i 5 2  i  (you can verify that A  A* ).
 
1  i 2  i 3 
 
*
Note: A  A  aij  a ji .

12