Scribd
Upload
178 views

Lecture 3

This document defines and describes vector and matrix norms. It begins by defining what a norm is for vectors, noting three key properties. It then discusses important vector norms like the 2-norm and how they can be used to define induced matrix norms. Matrix norms measure how a matrix stretches vectors. The Frobenius norm is introduced as a useful matrix norm not induced by a vector norm. Properties like bounding products of norms and invariance under orthogonal transformations are also covered.

Uploaded by

lvrevathi
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
178 views

Lecture 3

This document defines and describes vector and matrix norms. It begins by defining what a norm is for vectors, noting three key properties. It then discusses important vector norms like the 2-norm and how they can be used to define induced matrix norms. Matrix norms measure how a matrix stretches vectors. The Frobenius norm is introduced as a useful matrix norm not induced by a vector norm. Properties like bounding products of norms and invariance under orthogonal transformations are also covered.

Uploaded by

lvrevathi
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

Lecture 3 - Norms

OBJECTIVE:
The notions of size and distance in a vector space are
described by norms;
e.g., approximations and convergence are measured by
norms.
3 VECTOR NORMS
Denition 1. A norm is a function : R
m
R
that assigns a length to a vector.
For this to make sense, a norm must satisfy the
following 3 conditions for all vectors x, y and scalars :
1. x 0, and x = 0 if and only if x = 0
2. x +y x +y (triangle inequality)
3. x = ||x
What do these conditions say?
1. the length of a non-zero vector must be positive
2. the length of a vector sum cannot exceed the sum
of the individual vector lengths
3. scaling a vector scales its length by the same amount
The most important vector norm in this course is the
(unweighted) 2-norm
x
2
=
_
m

i=1
x
2
i
_
1
2
=

x
T
x.
But we will also use
x
1
=
m

i=1
|x
i
| and x

= max
1im
|x
i
|.
3MATRIX NORMS INDUCED BY VECTOR NORMS
You can view an m n matrix as an mn-dimensional
vector and use vector norms to dene norms of
matrices.
It turns out that there are other special vector norms
that can be dened for matrices that are more
useful. These are called induced matrix norms because
although the norms are intended for matrices, they
only use vector norms in their denition.
An induced matrix norm is dened in terms of the
behaviour of a matrix as an operator between its
normed domain and range spaces.
Specically, let
(n)
be a norm on the domain of A
(because x R
n
). Let
(m)
be a norm on the range
of A (because Ax R
m
).
Then, the induced matrix norm A
(m,n)
is the
smallest number C such that
Ax
(m)
Cx
(n)
for all x R
n
.
In other words, A
(m,n)
is the maximum value of
Ax
(m)
x
(n)
,
i.e., the maximum factor by which A can stretch any
vector x.
We say
(m,n)
is the matrix norm induced by the
(vector) norms
(m)
,
(n)
.
Because stretching factors are independent of the
size of the thing they are stretching, it is equivalent
(and convenient) to dene the induced matrix norm as
A
(m,n)
= max
xR
n
x=0
Ax
(m)
x
(n)
= max
xR
n
x
(n)
=1
Ax
(m)
.
Example 3.1 Consider A =
_
1 2
0 2
_
, which maps
R
2
to R
2
(so it is convenient to draw pictures!).
Matlab demo: Lecture3demo.m
Example 3.2 NORMS OF A DIAGONAL MATRIX
Let D be a diagonal matrix
D =
_

_
d
1
d
2
.
.
.
d
m
_

_
.
Then
D
2
= max
1im
|d
i
|,
but the same is true for D
1
, D

, or any
p
.
Example 3.3 THE 1-NORM OF A MATRIX
If A R
mn
, then
A
1
= max (absolute column sum) = max
1jn
a
j

1
.
Example 3.4 THE -NORM OF A MATRIX
If A R
mn
, then
A

= max (absolute row sum) = max


1im
a
T
i

1
.
Let A =
_
1 2 3
4 5 6
_
.
Then A
1
= 9 , A

= 15.
3 CAUCHYSCHWARTZ INEQUALITY
A useful result to bound the size of inner products: for
any vectors x,y
|x
T
y| x
2
y
2
Example 3.5 THE 2-NORM OF A ROW VECTOR
Consider a matrix A comprised of a single row;
i.e., A = a
T
, where a is a vector.
The CauchySchwartz inequality allows us to nd A
2
in this case.
For any x, we have Ax
2
= |a
T
x| a
2
x
2
A
2
= max
Ax
2
x
2
= a
2
Example 3.6 THE 2-NORM OF AN OUTER
PRODUCT
Let u R
m
, v R
n
, and A = uv
T
. (This is called a
rank-one outer product.)
Now,
Ax
2
= uv
T
x
2
= u
2
|v
T
x| u
2
v
2
x
2
A
2
= u
2
v
2
3 BOUNDING AB IN AN INDUCED MATRIX
NORM
Let
(l)
,
(m)
,
(n)
be norms on R
l
, R
m
, R
n
.
Let A R
lm
, B R
mn
.
Then, for all x R
n
,
ABx
(l)
A
(l,m)
Bx
(m)
A
(l,m)
B
(m,n)
x
(n)
.
(Treat Bx as a vector and use denition of induced
matrix norm.)
AB
(l,n)
A
(l,m)
B
(m,n)
3 GENERAL MATRIX NORMS
In general, matrix norms do not always have to be
induced by vector norms:
a matrix norm must merely satisfy the 3 conditions
of a vector norm applied in the mn-dimensional vector
space of matrices;
i.e.,
1. A 0, and A = 0 if and only if A = 0
2. A+B A +B
3. A = ||A
The most important matrix norm (not induced by a
vector norm) is the Frobenius norm (also called the
HilbertSchmidt norm):
A
F
=
_
_
m

i=1
n

j=1
a
2
ij
_
_
1/2
,
i.e., the square root of the sum of the squares of all
the elements
the 2-norm of the matrix viewed as an mn-
dimensional vector.
Another useful way to write A
F
is in terms of 2-
norms of the rows or columns. e.g.,
A
F
=
_
_
n

j=1
a
j

2
2
_
_
1/2
This can be expressed compactly as
A
F
=
_
tr(A
T
A) =
_
tr(AA
T
)
where tr() is the trace of a matrix: the sum of its
diagonal elements.
It can also be shown (p.23) that
AB
2
F
A
2
F
B
2
F
3 INVARIANCE UNDER ORTHOGONAL
MULTIPLICATION
Like the vector 2-norm, the matrix 2-norm is invariant
under multiplication by orthogonal matrices.
The same property holds for the Frobenius norm.
Theorem 1. For any A R
mn
and orthogonal
Q R
mm
, we have
QA
2
= A
2
and QA
F
= A
F
Note 1. This theorem still holds if Q R
pm
with
p > m.
i.e., Q consists of m orthonormal columns of size p.
The results also hold for multiplication on the right by
Q; i.e., for matrices with orthonormal rows.

You might also like