Mathematical Foundations

BITS Pilani MFDS Team

Pilani Campus
 BITS Pilani
Pilani Campus

* ZC416, Lecture 0
Agenda

• Matrices and their types

• REF and RREF

• Rank, its computation and properties

• Determinant, its computation and properties

• Consistency and inconsistency of linear systems

• Nature of solutions of linear systems

BITS Pilani, Pilani Campus

Matrices
• A matrix is a rectangular array of numbers or functions
which we will enclose in brackets. For example,
 a11 a12 a13 
0.3 −5  a a23  ,
1
 0 −0.2 16  ,  21 a22
   a31 a32 a33  (1)

e  4
−x
2x 
2
1
 6x ,  a1 a2 a3  ,
e 4x   
2
• The numbers (or functions) are called entries or, less
commonly, elements of the matrix.
• The first matrix in (1) has two rows, which are the
horizontal lines of entries.
BITS Pilani, Pilani Campus
Matrix – Notations

• We shall denote matrices by capital boldface letters A, B,

C, … , or by writing the general entry in brackets; thus A =
[ajk], and so on.
• By an m × n matrix (read m by n matrix) we mean a matrix
with m rows and n columns—rows always come first! m
× n is called the size of the matrix. Thus an m × n matrix
is of the form

(2)

BITS Pilani, Pilani Campus

Vectors

• A vector is a matrix with only one row or column. Its

entries are called the components of the vector.
• We shall denote vectors by lowercase boldface letters a,
b, … or by its general component in brackets, a = [aj], and
so on. Our special vectors in (1) suggest that a (general)
row vector is of the form

A column vector

BITS Pilani, Pilani Campus

Equality of Matrices

• Two matrices A = [ajk] and B = [bjk] are equal, written A = B,

if and only if (1) they have the same size and (2) the
corresponding entries are equal, that is, a11 = b11, a12 = b12,
and so on.

• Matrices that are not equal are called different. Thus,

matrices of different sizes are always different.

BITS Pilani, Pilani Campus

 Algebra of Matrices
1. Addition of Matrices
• The sum of two matrices A = [ajk] and B = [bjk] of the same size is written A + B
and has the entries ajk + bjk obtained by adding the corresponding entries of A
and B. Matrices of different sizes cannot be added.

2. Scalar Multiplication (Multiplication by a Number)

• The product of any m × n matrix A = [ajk] and any scalar c (number c) is written
cA and is the m × n matrix cA = [cajk] obtained by multiplying each entry of A by c.

(a) A+B = B+A (a) c( A + B) = cA + cB

(b) (A + B) + C = A + (B + C) (written A + B + C) (b) (c + k )A = cA + kA
(c) A+0 = A (c) c( kA) = (ck )A (written ckA)
(d) A + ( −A ) = 0. (d) 1A = A.

• Here 0 denotes the zero matrix (of size m × n), that is, the m × n matrix with all
entries zero.

BITS Pilani, Pilani Campus

Matrix Multiplication
Multiplication of a Matrix by a Matrix
•The product C = AB (in this order) of an m × n matrix A = [ajk]
times an r × p matrix B = [bjk] is defined if and only if r = n
and is then the m × p matrix C = [cjk] with entries

(3)

• The condition r = n means that the second factor, B, must

have as many rows as the first factor has columns, namely n.
A diagram of sizes that shows when matrix multiplication
is possible is as follows:
A B = C
[m × n] [n × p] = [m × p].
BITS Pilani, Pilani Campus
Matrix Multiplication

EXAMPLE 1

 3 5 −1 2 −2 3 1  22 −2 43 42 
    
AB =  4 0 2   5 0 7 8  =  26 −16 14 6  
−6 −3 2  9 −4 1 1 −9 4 −37 −28 

•Here c11 = 3 · 2 + 5 · 5 + (−1) · 9 = 22, and so on. The entry

in the box is c23 = 4 · 3 + 0 · 7 + 2 · 1 = 14.

•The product BA is not defined.

BITS Pilani, Pilani Campus
Matrix Multiplication
Matrix Multiplication Is Not Commutative, AB ≠ BA in
General
• This is illustrated by Example 1, where one of the two
products is not even defined. But it also holds for
square matrices. For instance,
 1 1   −1 1  0 0 
100 100   1 −1 = 0 0 
    
 −1 1   1 1   99 99 
but     =  .
 1 −1 100 100   −99 −99 
• It is interesting that this also shows that AB = 0 does
not necessarily imply BA = 0 or A = 0 or B = 0.

BITS Pilani, Pilani Campus

Transposition of Matrices &
Vectors
• The transpose of an m × n matrix A = [ajk] is the n × m
matrix AT (read A transpose) that has the first row of A as its
first column, the second row of A as its second column, and
so on. Thus the transpose of A in (2) is AT = [akj], written out

• As a special case, transposition converts row vectors to

column vectors and conversely.

BITS Pilani, Pilani Campus

Transposition of Matrices

• Rules for transposition are

(a) ( A T )T = A
(b) (A + B)T = A T + BT (5)
(c) ( cA ) T = cA T
(d) ( AB)T = BT A T .

CAUTION! Note that in (5d) the transposed matrices are in reversed order.

BITS Pilani, Pilani Campus

 Special Matrices
é 1 1 -1 ù
ê ú
• Symmetric: aij = aji Eg: ê 1 2 0 ú
ê -1 0 5 ú
ë û

é 0 1 -2 ù
ê ú
• Skew Symmetric : aij = - aji Eg: ê -1 0 3 ú
ê 2 -3 0 ú
ë û

• Triangular: Upper Triangular → aij = 0 for all i > j

Lower Triangular → aij = 0 for all i < j

• Diagonal Matrix: aij = 0 for all i ≠ j Eg:

• Sparse Matrix: Many zeroes and few non-zero entities

BITS Pilani, Pilani Campus

Positive Definite matrix

BITS Pilani, Pilani Campus

Elementary Row Operations

Given a matrix A, the following operations are

called Elementary Row Operations

• Interchange of two rows

• Addition of a constant multiple of one row to another row

• Multiplication of a row by a non-zero constant c

CAUTION! These operations are for rows, not for columns!

BITS Pilani, Pilani Campus

Row Echelon Form (REF) of a
matrix

BITS Pilani, Pilani Campus

Reduced Rpw Echelon Form
(RREF)
• We say that a matrix is in Reduced Row Echelon Form if
it is in Echelon form and additionally,
1. The leading entry in each row is 1.
2. Each leading 1 is the only non zero entry in its
column

BITS Pilani, Pilani Campus

Uniqueness of Row Reduced
Echelon Form
• We can transform any matrix into a matrix in reduced
row echelon form by using elementary row operations.

• No matter what sequence of row operations we use each

matrix is row equivalent to one and only one reducedrow
echelon matrix

BITS Pilani, Pilani Campus

REF of a Matrix
Swap rows
1 and 2

Replace R3
by R3+1.R1
Replace R3 by
R3+(-10).R2

Row Echelon Form

BITS Pilani, Pilani Campus

Rank of a matrix

• The number of nonzero rows, r, in the reduced row (or

row echelon form) coefficient matrix R is called the rank
of R and also the rank of A.

• The rank is invariant under elementary row operations:

BITS Pilani, Pilani Campus

Determination of Rank

3 0 2 2 
A =  −6 42 24 54  (given)
 21 −21 0 −15 

3 0 2 2 
= 0 42 28 58  Row 2 + 2 Row 1
0 −21 −14 −29  Row 3 − 7 Row 1

3 0 2 2 
= 0 42 28 58 
1
Row 3 + Row 2.
2
0 0 0 0 
• The last matrix is in row-echelon form and has two nonzero rows.
Hence rank A = 2.
BITS Pilani, Pilani Campus
Minor
 a11 a12 a13 

A = a21 a22 
a23 
a31 a32 a33 
Each element in A has a minor.
Delete first row and column from A. The determinant of the remaining
2x2 submatrix is the minor of a11 which is a22a33 – a23a32

a22 a23
m11 =
a32 a33
BITS Pilani, Pilani Campus
Minor

a21 a23
m12 =
a31 a33
And the minor for a13 is:

a21 a22
m13 =
a31 a32

BITS Pilani, Pilani Campus

Cofactor

The cofactor Cij of an element aij is defined as:

Cij = (−1)i + j mij

When the sum of a row number i and column j is even, cij = mij and
when i+j is odd, cij =-mij
c11 (i = 1, j = 1) = (−1)1+1 m11 = + m11
1+ 2
c12 (i = 1, j = 2) = (−1) m12 = −m12
1+ 3
c13 (i = 1, j = 3) = (−1) m13 = + m13
BITS Pilani, Pilani Campus
Determinant

The determinant of an n x n matrix A can now be defined as

A = det A = a11c11 + a12c12 +  + a1nc1n

The determinant of A is therefore the sum of the products of the

elements of the first row of A and their corresponding cofactors.
(It is possible to define |A| in terms of any other row or column
but for simplicity, the first row only is used)

BITS Pilani, Pilani Campus

Determinant -Example
Therefore the 2 x 2 matrix :
Example 1:
 a11 a12  3 1 
A= 
A=  1 2 
a21 a22 
Has cofactors :
A = (3)(2) − (1)(1) = 5
c11 = m11 = a22 = a22
And:
c12 = −m12 = − a21 = −a21
And the determinant of A is:

A = a11c11 + a12c12 = a11a22 − a12a21

BITS Pilani, Pilani Campus
Determinant -Example
 a11 a12 a13 
A = a21 a22 a23 
a31 a32 a33 
The cofactors of the first row are:
a22 a23
c11 = = a22a33 − a23a32
a32 a33
a21 a23
c12 = − = −(a21a33 − a23a31 )
a31 a33
a21 a22
c13 = = a21a32 − a22a31
a31 a32 BITS Pilani, Pilani Campus
Determinants – Example
A = a11c11 + a12c12 = a11a22 − a12a21
Which by substituting for the cofactors in this case is:
A = a11(a22a33 − a23a32 ) − a12 (a21a33 − a23a31) + a13 (a21a32 − a22a31)
 1 0 1
Example 2:
A =  0 2 3
− 1 0 1

A = a11(a22a33 − a23a32 ) − a12 (a21a33 − a23a31) + a13 (a21a32 − a22a31)

A = (1)(2 − 0) − (0)(0 + 3) + (1)(0 + 2) = 4

BITS Pilani, Pilani Campus

Adjoint
The adjoint matrix of A, denoted by adj A, is the transpose of its
cofactor matrix
adjA = C T

It can be shown that:

A(adj A) = (adjA) A = |A| I
 1 2
Example: A= 
 − 3 4 
A = (1)(4) − (2)(−3) = 10
 4 − 2
adjA = C = T

 3 1 
BITS Pilani, Pilani Campus
Adjoint

 1 2 4 − 2 10 0 
A(adjA) =     =  = 10 I
− 3 4 3 1   0 10 

4 − 2  1 2 10 0 
(adjA) A =     =  = 10 I
3 1  − 3 4  0 10 

BITS Pilani, Pilani Campus

Properties of Determinants

1. det(AB) =det(A) * det (B)

2. det(A) nonzero implies there exists a matrix B such that AB=BA=I

3. Two Rows Equal➔ det = 0(Singular)

4. Ri and Rj swapped ➔ det gets a minus sign (i ≠ j)

5. det(A) = det(AT)

6. Ri  cRj ➔det A c detA

BITS Pilani, Pilani Campus

Inverse

• A-1 = adj(A) where det(A) ≠ 0

det(A)
Reiterate det(A) ≠ 0 → A is Non singular

é 3 1 ù
A=ê ú
ë 2 1 û
-1
é 1 -1 ù
A =ê ú
ë -2 3 û

BITS Pilani, Pilani Campus

Inverse – 2x2 Example
Example
 1 2
A= − 3 4
 

−1 1 4 − 2 0.4 − 0.2
A =  =

10 3 1  0.3 0.1 
To check AA-1 = A-1 A = I
 1 2 0.4 − 0.2 1
−1 0
AA =     =  =I
− 3 4 0.3 0.1  0 1
−1 0.4 − 0.2  1 2 1 0
A A=    =  =I
0.3 0.1  − 3 4 0 1
BITS Pilani, Pilani Campus
Inverse – 3x3 Example
Example 2
3 − 1 1 
A = 2 1 0 
1 2 − 1
The determinant of A is
|A| = (3)(-1-0)-(-1)(-2-0)+(1)(4-1) = -2

The elements of the cofactor matrix are

c11 = +(−1), c12 = −(−2), c13 = +(3),
c21 = −(−1), c22 = +(−4), c23 = −(7),
c31 = +(−1), c32 = −(−2), c33 = +(5),
BITS Pilani, Pilani Campus
Inverse – 3x3 Example
The cofactor matrix is therefore
− 1 2 3
C =  1 − 4 − 7
− 1 2 5 

so
− 1 1 − 1
adjA = C T =  2 − 4 2 
 3 − 7 5 

and
− 1 1 − 1  0.5 − 0.5 0.5 
−1 adjA 1   = − 1.0 2.0 − 1.0 
A = = 2 − 4 2
A −2    
 3 − 7 5  − 1.5 3.5 − 2.5
BITS Pilani, Pilani Campus
Inverse
The result can be checked using
AA-1 = A-1 A = I

The determinant of a matrix must not be zero for the inverse to

exist as there will not be a solution
Nonsingular matrices have non-zero determinants
Singular matrices have zero determinants

BITS Pilani, Pilani Campus

Inverse -Simple 2 x 2 case
So that for a 2 x 2 matrix the inverse can be constructed
in a simple fashion as
é d -b ù
ê ú
é a b ù ê A A ú 1é ù
-1
A =ê =
ú êê ú = ê d -b ú
-c a ú A ë -c a û
ë c d û ê A A ú
ë û
•Exchange elements of main diagonal
•Change sign in elements off main diagonal
•Divide resulting matrix by the determinant

BITS Pilani, Pilani Campus

Inverse -Simple 2 x 2 case
Example
2 3
A= 
 4 1
−1 1  1 − 3 − 0.1 0.3 
A =−  =
10 − 4 2   0.4 − 0.2


Check inverse
A-1 A=I

1  1 − 3 2 3 1 0
−     =  =I
10 − 4 2  4 1 0 1

BITS Pilani, Pilani Campus

Linear System
• A linear system of m equations in n unknowns x1, … ,
xn is a set of equations of the form

(1)

• The system is called linear because each variable xj

appears in the first power only, just as in the equation of
a straight line.
BITS Pilani, Pilani Campus
Linear System

(1)

•a11, … , amn are given numbers, called the coefficients of

the system.
•b1, … , bm on the right are also given numbers.
•If all the bj are zero, then (1) is called a homogeneous
system.
•If at least one bj is not zero, then (1) is called a non-
homogeneous system.
BITS Pilani, Pilani Campus
Coefficient Matrix
Matrix Form of the Linear System (1).
•From the definition of matrix multiplication we see that
the m equations of (1) may be written as a single vector
equation
Ax = b (2)
where the coefficient matrix A = [ajk] is the m × n matrix
are column vectors.

BITS Pilani, Pilani Campus

Augmented Matrix
Matrix Form of the Linear System (1). (continued)
• We assume that the coefficients ajk are not all zero, so that A is not a
zero matrix. Note that x has n components, whereas b has m
components. The matrix

is called the augmented matrix of the system (1).

• The dashed vertical line could be omitted, as we shall do later. It is
merely a reminder that the last column of à did not come from
matrix A but came from vector b. Thus, we augmented the matrix A.

BITS Pilani, Pilani Campus

Solution to System of Linear
Equations

• A linear system (1) is called overdetermined if it has more

equations than unknowns, determined if m = n, and
underdetermined if it has fewer equations than unknowns.

• A linear system is consistent if rank(A) = rank(Ã)

• A consistent system has at least one solution (thus, one

solution or infinitely many solutions), but inconsistent has no
solutions at all, as x1 + x2 = 1, x1 + x2 = 0.

BITS Pilani, Pilani Campus

Solution

The method for determining whether Ax = b has solutions

and what they are:

(a) No solution. If r is less than m (meaning that R actually

has at least one row of all 0s) and at least one of the numbers
fr+1, fr+2, … , fm is not zero, then the system Rx = f is
inconsistent: No solution is possible. Therefore the system
Ax = b is inconsistent as well.

BITS Pilani, Pilani Campus

Solution

If the system is consistent (either r = m, or r < m and all the

numbers fr+1, fr+2, … , fm are zero), then there are solutions.

(b) Unique solution. If the system is consistent and r = n,

there is exactly one solution, which can be found by back
substitution.

(c) Infinitely many solutions. To obtain any of these

solutions, choose values of xr+1, … , xn arbitrarily. Then solve
the rth equation for xr (in terms of those arbitrary values),
then the (r − 1)st equation for xr−1, and so on up the line.

BITS Pilani, Pilani Campus