Lecture 05

Matrices – Matrix
Algebra
1. Introduction

We have learnt quite a bit about vectors. Before that, we already knew about scalars. Scalars and
vectors are part of mathematical structures that help us to handle information in the form of data.
So far, we have been dealing with information in the form of numbers only. As systems become
more complicated, we will be dealing with numerical and non-numerical data in the same
problem. That is where we will extend our mathematical structures from scalars, vectors to
matrices and tensors. In the next few lectures we will learn about matrices,

A matrix is a set of elements written in a rectangular pattern. Therefore

(1,1) (1,2) (1,3
0.2 1 6 sin 𝑥 cos 𝑥
(2,1) (2,2) (2,3)
6.5 3.2 2 −cos 𝑥 sin 𝑥
(3,1) (3,2) (3,3)
𝑑
𝑒 2𝑥 (𝑎 𝑏 𝑐 ) 𝑒
𝑒 4𝑥 𝑓
are examples of matrices. Matrices are enclosed in parenthesis – square or round. Generally,
matrices are denoted by boldface – A, a, etc. Therefore
𝑎 𝑏 𝑐
𝐀= (1)
𝑑 𝑒 𝑓
is a representation of a matrix.

One world of caution: A matrix does not have any value. We begin with a physical problem. For
sake of convenience, we rewrite the problem in terms of matrices. We do algebra with the
matrices. Finally, we go back to the physical problem. The new matrix form, hopefully, helps us
to find a solution that we are looking for easier.

In (1), the entities a, b, c, d, e, f are referred to as elements of the matrix A. The entities
𝑎 𝑏
(𝑎 𝑏 𝑐) and (𝑑 𝑒 𝑓) are referred to as rows of the matrix A; and the entities , and
𝑑 𝑒
𝑐
𝑓 are referred to as columns of the matrix A. Therefore, the matrix A can also be written as
𝐫𝟏
𝐀 = 𝐫 , where 𝐫𝟏 = (𝑎 𝑏 𝑐 ) and 𝐫𝟐 = (𝑑 𝑒 𝑓). The matrix can also be written as 𝐀 =
𝟐
𝑎 𝑏 𝑐
𝐜
( 𝟏 𝐜 𝟐 𝐜𝟑 ), where 𝐜𝟏 = , 𝐜𝟐 = and 𝐜𝟑 = 𝑓 . One question would definitely arise.
𝑑 𝑒
Should we write the matrix in the row for or column form. The problem would determine that.

The size of a matrix is indicated as its order, and is specified by number of rows times the
number of columns. Therefore, the order of matrix A above is 2  3, ad is read verbatim as ‘two
by three’.

It is also possible to write matrices using indicial notations. Therefore, matrices can often be
found written as A = (aij). The first index i refers to the row position, and the second index j
refers to the column position.
2. Some terminologies

A matrix whose all elements are zero is called a zero matrix, or null matrix.

A matrix whose number of rows and columns are same is called a square matrix.
3 6 −2
𝐀= 1 0 3 (2)
5 4 −4
A is a 3  3 square matrix The diagonal from top left to bottom right of a square matrix is called
simply the diagonal of a matrix. If all the elements of a diagonal matrix are 1, the matrix is
called a unit matrix. Unit matrices are indicated by I
1 0 0
𝐈= 0 1 0 (3)
0 0 1
In a square matrix, if all the elements below the diagonal are 0, the matrix is referred to as an
upper triangular matrix. Similarly, if all the elements of a square above the diagonal are zero, the
matrix is called a lower triangular matrix. Traditionally, an upper triangular matrix is indicated
by U, and a lower triangular matrix is indicated by L.
6 4 9 −2 0 0
𝐔 = 0 2 −4 𝐋 = 5 −1 0 (4)
0 0 −1 6 2 8
Are examples of an upper-triangular and a lower-triangular matrices.

3. Transpose, symmetric, skew-symmetric matrix

For a matrix A, the transpose of the matrix, indicated by AT is obtained by interchanging the
rows and columns of the matrix.

Example 01
3 6 −2 3 1 5
𝐓
𝐀= 1 0 3 𝐀 = 6 0 4
5 4 −4 −2 3 −4
T
In terms of indices, for a matrix A = (aij), A = (aji)

For a square matrix, if AT = A, the matrix is a symmetric matrix. If AT = −A, the matrix is skew-
symmetric matrix.

Example 02
5 −1 2
𝐀 = −1 2 −3
2 −3 −1
is a symmetric matrix.
 0 1 −2
𝐀 = −1 0 3
2 −3 0
is a skew-symmetric matrix

Note: By algebraic necessity, the diagonal elements of a skew-symmetric matrix would be 0.

4. Matrix Algebra

Equality of matrices

Two matrices would be equal if (i) the matrices are of same order, and (ii) the individual
elements are same.

Example 03
6 −4 𝑏 𝑏
If 𝐀 = is equal to 𝐁 = then b11 = 6, b12 = −4, b21 = 3 and b22 = 2.
3 2 𝑏 𝑏

Scalar multiplication of a matrix

A matrix is multiplied by a scalar by multiplying the elements of the matrix term-by-term with
the scalar. In terms of indices, if A= (aij), kA = (kaij).

Example 04
6 2 −3 30 10 −15
𝐀 = −2 4 8 5𝐀 = −10 20 40
4 −4 5 20 −20 25

Addition of two matrices

Two matrices A = (aij) and B = (bij), both of order m  n are added by adding the elements term-
by-term. Therefore A + B = (aij + bij). Caution: If the matrices are not of same order, they cannot
be added.

Example 05
3 −2 4 −3 7 −5
Given 𝐀 = 4 2 and 𝐁 = 1 5 ,𝐀+𝐁 = 5 7
5 −3 −4 6 1 3

Exercise

Let
 1 2 4 0 5 2 5 2 −4 2
𝐀= 6 5 4 ,𝐁= 4 3 5 .𝐂= −2 4 , 𝐃 = 5 1
1 2 −2 −2 3 −2 1 −2 2 −1
0 2 1.5 −1 −5
𝐄= 3 4 ,𝐮= 0 ,𝐯= 3 ,𝐰= −10
−3 −1 −3 2 15

Find the following expressions, or give reasons why they are not defined

1. 2A + 4B, 4B + 2A, 0A + B, 0.4B – 4.2A

2. 3A, 0.5A, 3A + 0.5B, 3A + 0.5B + C
3. (4 ⋅ 3)A, 4(3A), 14B – 3B, 11B
4. 8C + 10D, 2(5D + 4C), 0.6C – 0.6D, 0.6(C – D)
5. (C + D) + E, D + (E + C), 0(C – E) + 4D, A – 0C
6. (2 ⋅ 7)C, 2(7C), – D + 0E, E – D + C + u
7. (5u + 5v) – ½w, –20(u + v) + w, E – (u + v), 10u + v) + w
8. (u + v) – w, u + (v – w), C + 0w, 0E + u – v
9. 15v – 3w – 0u, –3w + 15v, D – u + 3C, 8.5w – 11.1u + 0.4u

5. Matrix multiplication

We are attempting to multiply one matrix with another matrix. The definition is standard but
looks artificial. Thus, you have to study matrix multiplication carefully, practice few
multiplications until you understand it. At the onset, it is advisable to approach this mechanically.
We will discuss the motivation for matrix multiplication later.

The compatibility of two matrices that can be multiplied with each other is very strict. If we want
to multiply matrix A with B, then number of columns of matrix A has to be equal to the number
of rows of matrix B. In other words, if matrix A is of order m  n, then matrix B must be of the
order n  l. The resulting matrix would be of order m  l.

Using indices let us define matrix A of order m  n as A = (aij), and matrix B of order n  l as
B = (bjk). The product C is defined as

𝐂 = 𝐀𝐁 = (𝑐 ) = 𝑎 𝑏 (5)

The entry cik in (5) is obtained by multiplying each entry of ith row of A by the corresponding
entry in the kth column of B and then adding these n product. Strictly speaking, the cikth element
of the matrix C is the dot-product of the ith row of A with the kth column of B. We illustrate this
 𝑎 ⋯ 𝑎 𝑏 ⋯ 𝑏 ⋯ 𝑏 𝑐 ⋯ 𝑐
⋮ ⋯ ⋮ ⋯ ⋮ ⎞ ⎛ ⋮ ⋯ ⋮
⎛ ⎞⎛ ⋮ ⋯ ⋮ ⎞
𝑎 ⋯ 𝑎 ⋯ ⋮ ⎟=⎜ ⋮ 𝑐 ⋮
⎜ ⎟⎜ ⋮ ⋯ ⋮ ⎟
⋮ ⋯ ⋮ ⋮ ⋯ ⋮ ⋯ ⋮ ⋮ ⋯ ⋮
⎝𝑎 ⋯ 𝑎 ⎠ ⎝𝑏 ⋯ 𝑏 ⋯ 𝑏 ⎠ ⎝𝑐 ⋯ 𝑐 ⎠
where

𝑐 =𝑎 𝑏 +𝑎 𝑏 + ⋯+ 𝑎 𝑏 = 𝑎 𝑏

=𝐫 ∙ 𝐜

Example 06
3 5 −1 2 −2 3 1 22 −2 43 42
4 0 2 5 0 7 8 = 26 −16 14 6
−6 −3 2 9 −4 1 1 −9 4 −37 −28
Here c11 = 32 + 55 + (–1)9 = 22. c23 = 43 + 07 + 21 = 14, etc.

Generally, matrix multiplication is not commutative, meaning AB  BA.

Example 07
1 1 −1 1 0 0
=
100 100 1 −1 0 0
but
−1 1 1 1 99 99
=
1 −1 100 100 −99 −99

Example 08
1 2 3 2 −5 1
𝐀 = −4 −4 −4 𝐁 = 0 3 −2
5 6 7 1 2 −4
5 7 −15 27 30 33
𝐀𝐁 = −12 0 20 𝐁𝐀 = −22 −24 −26
17 7 −35 −27 −30 −33

Exercise

Let
4 −2 3 1 −3 0
𝐴 = −2 1 6 𝐵 = −3 1 2
1 2 2 5 1 4
1 −1 3
𝐶= 3 2 𝑎 = (1 −2 4) 𝑏= 1
−2 4 −1
Calculate the following results, or give reasons why they are undefined.
1. AB, ABT, BA, BAT
2. AAT, A2, BBT, B2
3. CCT, BC, CB, CTB
4. 3A – 2B, (3A – 2B)T, 3AT – 2BT, (3A – 2B)TaT
5. Aa, AaT, (Ab)T, bTAT
6. BC, BCT, Bb, bTB
7. ABC, Aba, Abb, CaT
8. ab, ba, aA, Bb
9. 1.5a + 3.0b, 1.5aT + 3.0b, (A – B)b, Ab – Bb
10. bTAb, aBaT, aCCT, CTba

6. Some application of matrix multiplication

1. Computer Graphics: In computer graphics, matrices are used to represent

transformations such as translation, rotation, scaling, and projection. Matrix
multiplication is essential for efficiently applying these transformations to vertices of
3D objects.
2. Deep Learning and Neural Networks: Matrix multiplication lies at the heart of deep
learning algorithms, particularly in the training and inference stages of neural networks.
The weights and biases in neural networks are typically represented as matrices, and
matrix multiplication is used to propagate signals through the network.
3. Signal Processing: In signal processing applications such as image processing and
audio processing, matrix multiplication is used for operations like filtering, convolution,
and Fourier transforms.
4. Robotics: In robotics, matrices are used to represent the transformations between
different coordinate systems, such as the position and orientation of a robot arm. Matrix
multiplication is used to calculate these transformations efficiently.
5. Optimization: Matrix multiplication is often used in optimization problems, such as
linear programming and regression analysis. For example, in linear regression, matrix
multiplication is used to calculate the predictions based on the model parameters.
6. Physics Simulations: In physics simulations, matrices are used to represent physical
systems and their dynamics. Matrix multiplication is used to update the state of the
system over time, making it crucial for real-time simulations.
7. Financial Modeling: In finance, matrix multiplication is used in portfolio optimization,
risk management, and pricing of financial derivatives. For example, matrix
multiplication is used in the calculation of returns and risk measures for investment
portfolios.
8. Image and Video Processing: In applications like image and video compression,
matrix multiplication is used in operations such as discrete cosine transform (DCT) and
discrete wavelet transform (DWT) to analyze and compress visual data efficiently.