Contents

Matrices and Linear Algebra

Quantitative methods for Economics and Business

University of Ferrara

Academic year 2017-2018

Matrices and Linear Algebra

 Contents Matrices and Linear Algebra

Matrices and Linear Algebra

1 Basics

2 Matrix operations

3 Determinant of a matrix

4 Inverse matrix

5 Eigenvalues of a matrix

Matrices and Linear Algebra

 Contents Matrices and Linear Algebra

Matrices and Linear Algebra

1 Basics

2 Matrix operations

3 Determinant of a matrix

4 Inverse matrix

5 Eigenvalues of a matrix

Matrices and Linear Algebra

 Contents Matrices and Linear Algebra

Matrices and Linear Algebra

1 Basics

2 Matrix operations

3 Determinant of a matrix

4 Inverse matrix

5 Eigenvalues of a matrix

Matrices and Linear Algebra

 Contents Matrices and Linear Algebra

Matrices and Linear Algebra

1 Basics

2 Matrix operations

3 Determinant of a matrix

4 Inverse matrix

5 Eigenvalues of a matrix

Matrices and Linear Algebra

 Contents Matrices and Linear Algebra

Matrices and Linear Algebra

1 Basics

2 Matrix operations

3 Determinant of a matrix

4 Inverse matrix

5 Eigenvalues of a matrix

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Part I

Matrices and Linear Algebra

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Definition
A matrix is an m × n array of m · n scalars from R. The individual values
in the matrix are called entries.
Examples:
   
2 −1 3 1 2
A= B=
1 4 −2 3 4
The size of the array is written as m × n, where
m is the number of rows
n is the number of columns
Notation:  
a11 a12 ... a1n
 a21 a22 ... a2n 
A= .. .. ..
 

 . . . 
am1 am2 ... amn
aij := the entry on the row i and on the column j
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Specific cases
A matrix with dimension 1 × n is called row vector:
x = (x1 , x2 , . . . , xn )
A matrix with dimension m × 1 is called column vector or simply vector:
 
y1
 y2 
y= . 
 
 .. 
ym
A unit vector is a vector of ones:
 
1
 1 
..
 
 
 . 
1

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Special matrices
If m = n, the matrix is called square. In this case we have:
A matrix A is said to be diagonal if
aij = 0 for i 6= j
A diagonal matrix A may be denoted by diag (d1 , d2 , . . . , dn )
where
aii = di and aij = 0 for i 6= j.
The diagonal matrix diag (1, 1, . . . , 1) is called the identity matrix
and is usually denoted by
 
1 0 ... 0
 0 1 
I = .
 
 .. . .. 

0 1
The diagonal matrix O = diag (0, . . . , 0) is called the zero matrix.
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

A square matrix L is said to be lower triangular if

lij = 0 for i < j

that is  
l11 0 ... 0
 l21 l22 
L= .. ..
 
.. 
 . . . 
ln1 ln2 ... lnn
A square matrix U is said to be upper triangular if

uij = 0 for i > j

that is  
u11 u12 ... u1n
 0 u22 u2n 
U= ..
 
.. 
 . . 
unn
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Definition

Let A be an m × n matrix.
Define the transpose of A, denoted by AT , to be the n × m matrix (i.e.
n rows and m columns) with entries

(AT )ij = aji .

In other words:
   
a11 a12 ... a1n a11 a21 ... am1
 a21 a22 ... a2n   a12 a22 ... am2 
A= . .. .. AT =  .. .. ..
   
 ..
 
. .   . . . 
am1 am2 ... amn a1n a2n ... amn

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Examples:

 
  2 1
2 −1 3
A= AT =  −1 4 
1 4 −2
3 −2
   
1 2 T 1 3
B= B =
3 4 2 4
We have that
(AT )T = A

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

A square matrix A (i.e. an n × n matrix) is called symmetric if

aij = aji

that is
AT = A
Example: The matrix
 
0 1 2
A=  1 −1 −4 
2 −4 0

is symmetric.

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Equality, Addition, Scalar multiplication

Two matrices A and B are equal if and only if they have the same size
and
aij = bij for all i, j.
If A and B are matrices of the same size then the sum of A and B is
defined by C = A + B, where

cij = aij + bij for all i, j.

If A is any matrix and α ∈ R then the scalar multiplication B = αA is

defined by
bij = αaij for all i, j.

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Examples

     
2 1 6 −1.5 8 −0.5
 −3 4 + 0 1  =  −3 5 
7 0 1 π 8 π

   
1 2 −3 −6
−3 =
3 4 −9 −12

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Matrix addition “inherits” many properties from R.

Theorem: If A, B, C are m × n matrices and α, β ∈ R, then

A + B = B + A (commutivity)
A + (B + C ) = (A + B) + C (associativity)
α(A + B) = αA + αB (distributivity of a scalar)
if B = O (a matrix of all zeros) then A + B = A + O = A
(α + β)A = αA + βA
α(βA) = αβA
0A = O
αO = O
(A + B)T = AT + B T

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Inner or scalar product

Let x = (x1 , . . . , xn ) and y = (y1 , . . . , yn ) be two vectors.

The scalar or inner product of x and y is given by

hx, yi = x1 y1 + x2 y2 + · · · + xn yn

It is a scalar (i.e. a number)!!!

Remark: Alternative notation for the scalar product is hx, yi = x · y.

Scalar product is defined only for vectors of the same length!!!

Example: let x = (1, 0, 3, −1) and y = (0, 2, −1, 2). Then

hx, yi = 1 · 0 + 0 · 2 + 3 · (−1) + (−1) · 2 = −5

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Matrix product

Assume that A is m × n and B is n × p.

Denote by
ri (A) the i−th row of A
cj (B) the j−th column of B

The product D = AB is the m × p matrix defined by

dij = hri (A), cj (B)i

that means

dij = ai1 b1j + ai2 b2j + · · · + ain bnj , for 1 ≤ i ≤ m, 1 ≤ j ≤ p

Remark: In order to perform matrix multiplication, we need that the

number of columns of A is equal to the number of rows of B!

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Example: 

  2 1
1 0 1
A= B= 3 0 
3 2 1
−1 1
Set C = AB, then

d11 = hr1 (A), c1 (B)i = 1 · 2 + 0 · 3 + 1 · (−1) = 1

d12 = hr1 (A), c2 (B)i = 1 · 1 + 0 · 0 + 1 · 1 = 2
d21 = hr2 (A), c1 (B)i = 3 · 2 + 2 · 3 + 1 · (−1) = 11
d22 = hr2 (A), c2 (B)i = 3 · 1 + 2 · 0 + 1 · 1 = 4

We obtain  
1 2
D = AB =
11 4

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Properties of matrix product

If AB exists, does it happen that BA exists and AB = BA?
The answer is usually no.
First AB and BA exist if and only if A is m × n and B is n × m.
Even if this is so, the sizes of AB and BA are different (AB is m × m
and BA is n × n) unless m = n.
Example:
   
−1 −1 −2
A = (1, 2) B = AB = (1) BA =
1 1 2
However even if m = n we may have AB 6= BA.
Example:    
1 2 −1 1
A= B=
3 4 0 1
   
−1 3 2 2
AB = BA =
−3 7 3 4
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

If AB = O, does it happen that A = O or B = O?

The answer is usually no.

Example: Let    
1 0 0 0
A= B=
0 0 0 1
It happens that  
0 0
AB =
0 0
but  
0 0
A 6=
0 0
and  
0 0
B 6=
0 0

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Theorem (Matrix Multiplication Rules).

Assume A, B, and C are matrices for which all products below make
sense. Then
A(BC ) = (AB)C (associativity)
A(B + C ) = AB + AC and (A + B)C = AC + BC
AI = A and IA = A
α(AB) = (αA)B
AO = O and OB = O
(AB)T = B T AT

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

The simple case of 2 × 2 matrices

Consider a 2 × 2 matrix
 
a11 a12
A=
a21 a22

The determinant of A is defined as

det(A) = a11 a22 − a12 a21

Example:  
1 2
A=
3 4
det(A) = 1 · 4 − 2 · 3 = −2

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

The case of 3 × 3 matrices

Consider a 3 × 3 matrix
 
a11 a12 a13
A =  a21 a22 a23 
a31 a32 a33
The determinant of A is defined as
det(A) = a11 ·(−1)1+1 det(A11 )+a12 ·(−1)1+2 det(A12 )+a13 ·(−1)1+3 det(A13 )
where
     
a22 a23 a21 a23 a21 a22
A11 = A12 = A13 =
a32 a33 a31 a33 a31 a32
Then
det(A) = a11 · det(A11 ) − a12 · det(A12 ) + a13 · det(A13 )
= a11 (a22 a33 − a32 a23 ) − a12 (a21 a33 − a31 a23 ) + a13 (a21 a32 − a31 a22 )
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

The same rule may be carried out by selecting any row or column of A:

for each row index i (i = 1, 2, 3), we have

det(A) = ai1 ·(−1)i+1 det(Ai1 )+ai2 ·(−1)i+2 det(Ai2 )+ai3 ·(−1)i+3 det(Ai3 )

for each column index j (j = 1, 2, 3), we have

det(A) = a1j ·(−1)1+j det(A1j )+a2j ·(−1)2+j det(A2j )+a3j ·(−1)3+j det(A3j )

where Aij is the submatrix obtained from A after deleting the i−th row
and the j−th column of A itself.

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

For instance, consider

 
1 2 1
A =  3 −1 4 
1 −2 1

Then we have
     
−1 4 3 4 3 −1
det(A) = 1 · det − 2 · det + 1 · det
−2 1 1 1 1 −2

It follows that

det(A) = (−1 + 8) − 2(3 − 4) + (−6 + 1) = 4

Any row or column can be selected in order to evaluate the determinant:

try to work on the third column and verify the result!

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

The general case of n × n matrices

Consider a square n × n matrix
 
a11 a12 ... a1n
 a21 a22 ... a2n 
A= . ..
 
 ..

. 
an1 an2 ... ann
The determinant of A is defined as
det(A) = ai1 ·(−1)i+1 det(Ai1 )+ai2 ·(−1)i+2 det(Ai2 )+· · ·+ain ·(−1)i+n det(Ain )
for each row index i (i = 1, 2, . . . , n), or equivalently as
det(A) = a1j ·(−1)1+j det(A1j )+a2j ·(−1)2+j det(A2j )+· · ·+anj ·(−1)n+j det(Anj )
for each column index j (j = 1, 2, . . . , n), where

Aij is the submatrix obtained from A after deleting the i−th row and the
j−th column of A itself.
n n
X XMatrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Some properties of the determinant

Consider an n × n matrix A.
If two rows of A are interchanged to obtain B, then

det(B) = − det(A)

If any row of A is multiplied by a scalar α, the resulting matrix B

has determinant
det(B) = α det(A)
If any two rows of A are equal, then

det(A) = 0

If A has two rows equal up to a multiplicative constant, then

det(A) = 0

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

If B is obtained by summing to a row or a column of A a linear

combination of the other rows or columns of A, respectively, then

det(B) = det(A)

det(AT ) = det(A)
det(AB) = det(A) · det(B)
det(αA) = αn det(A)
If A = diag (d1 , d2 , . . . , dn ), then det(A) = d1 · d2 . . . dn .
It follows that
det(I ) = 1 · 1 . . . 1 = 1

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Definition

Let A be an n × n square matrix.

A is invertible if there exists a matrix B that satisfies the following
relationship
AB = BA = I
Under the assumption that B exists, it is unique!!! We denote B by

A−1 inverse of A.

Then
AA−1 = A−1 A = I

It is possible to prove that:

A is invertible (i.e. A−1 exists) ⇔ det(A) 6= 0
in that case, A is non singular.

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Simple case: 2 × 2 matrices

Consider n = 2 and  
a11 a12
A=
a21 a22
Assume that det(A) 6= 0. Then A is invertible and
 
−1 1 a22 −a12
A =
det(A) −a21 a11

Example:  
2 −1
A= det(A) = 10
4 3
   3 1

−1 1 3 1 10 10
A = =
10 −4 2 − 52 15

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Some properties of the inverse

Consider two n × n matrices A, B and a scalar α ∈ R.

(AB)−1 = B −1 A−1
(αA)−1 = α−1 A−1
det(A−1 ) = 1/ det(A) (why?)

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

An application: solution of linear systems

Data: n × n matrix A, n × 1 column vector b
Unknown:: n × 1 column vector x
Problem: Find x such that
Ax = b
that is equivalent to solve the following system of linear equations


 a11 x1 + a12 x2 + · · · + a1n xn = b1
 a21 x1 + a22 x2 + · · · + a2n xn = b2

..


 .
an1 x1 + an2 x2 + · · · + ann xn = bn


Assume that A is non-singular (i.e. det(A) 6= 0), then the previous

system has got a unique solution, which is obtained as
A−1 Ax = A−1 b
thus
x = A−1 b
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Example: The following system


3x1 + 2x2 = 1
4x1 + 3x2 = −2

is equivalent to
Ax = b
with    
3 2 1
A= b=
4 3 −2
We have det(A) = 1 6= 0, then A is invertible and
 
3 −2
A−1 =
−4 3

The solution of Ax = b is
    
−1 3 −2 1 7
x=A b= =
−4 3 −2 −10

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Definition
If a vector v 6= 0 satisfies the equation
Av = λv,
for some scalar λ, then λ is said to be an eigenvalue of the matrix A, and
v is said to be an eigenvector of A corresponding to the eigenvalue λ.
Example: If  
2 3
A=
3 2
and  
1
v=
1
then    
5 1
Av = =5 = 5v
5 1
So λ = 5 is an eigenvalue of A and v is an eigenvector corresponding to
this eigenvalue.
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

The definition of eigenvector requires that v 6= 0.

The reason for this is that if v = 0 were allowed, then any number λ
would be an eigenvalue since the statement A0 = λ0 holds for any λ.
On the other hand, we can have λ = 0 and v 6= 0.

Example: If  
1 1
A=
3 3
and  
1
v=
−1
then  
0
Av = = 0v
0
So λ = 0 is an eigenvalue of A and v 6= 0 is an eigenvector corresponding
to this eigenvalue.

Matrices and Linear Algebra

 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

How do we find the eigenvalues and eigenvectors of a

matrix?
Suppose v 6= 0 is an eigenvector of A. Then for some λ ∈ R, we have
Av = λv. Then
Av − λv = 0
or, equivalently,
(A − λI )v = 0
This happens when
det(A − λI ) = 0

Then:
λ is an eigenvalue of A ⇔ det(A − λI ) = 0

p(λ) = det(A − λI ) is a polynomial function of λ.

p(λ) = 0 is called the characteristic equation of the matrix A
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

An example
Consider  
1 3
A=
3 1
Then  
1−λ 3
A − λI =
3 1−λ
and
p(λ) = det(A − λI ) = (1 − λ)2 − 9 = λ2 − 2λ − 8 = (λ − 4)(λ + 2)
From p(λ) = 0 we obtain that λ1 = 4 and λ2 = −2 are the eigenvalues
of A.

As an exercise, we want to find an eigenvector corresponding to λ1 . Thus

we have to solve the linear system (A − 4I )v = 0, i.e.
   
1−4 3 0
v=
3 1−4 0
Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

It is equivalent to
    
−3 3 v1 0
=
3 −3 v2 0

This leads to the two equations −3v1 + 3v2 = 0 and 3v1 − 3v2 = 0.
Notice that the first equation is a multiple of the second one, so there is
really only one equation to solve

3v1 − 3v2 = 0

The general solution to the homogeneous system is given by

v1 = v2 = c

therefore, all vectors v such that

   
c 1
v= =c (where c 6= 0 is arbitrary)
c 1

are eigenvectors of A corresponding to λ1 = 4.

Matrices and Linear Algebra
 Basics
Matrix operations
Determinant of a matrix
Inverse matrix
Eigenvalues of a matrix

Conclusion:

In general, eigenvectors are not unique! If v is an eigenvector for A

corresponding to a given eigenvalue λ, then so is cv, for any number
c 6= 0.

As an exercise, find the eigenvectors of A corresponding to the other

eigenvalue λ2 = −2 (in the previous example).

Matrices and Linear Algebra