Scribd
Upload
81 views28 pages

Orthogonality

The document discusses orthogonality and orthogonal subspaces in linear algebra. It defines the scalar product, orthogonal vectors and subspaces, orthogonal complements, and orthonormal sets. It presents the Gram-Schmidt process for obtaining an orthonormal basis from a given basis through orthogonalization.

Uploaded by

Belsty Wale Kibret
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
81 views28 pages

Orthogonality

The document discusses orthogonality and orthogonal subspaces in linear algebra. It defines the scalar product, orthogonal vectors and subspaces, orthogonal complements, and orthonormal sets. It presents the Gram-Schmidt process for obtaining an orthonormal basis from a given basis through orthogonalization.

Uploaded by

Belsty Wale Kibret
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

Chapter 5

Orthogonality
1 The scalar product in Rn
The product xTy is called the scalar product of x and y.
In particular, if x=(x1, …, xn)T and y=(y1, …,yn)T, then
xTy=x1y1+x2y2+‥‥+xnyn

The Scalar Product in R2 and R3

Definition
Let x and y be vectors in either R2 or R3. The distance
between x and y is defined to be the number ‖x-y‖.
Example If x=(3, 4)T and y=(-1, 7)T, then the distance
between x and y is given by
‖y-x‖= 5
Theorem 5.1.1 If x and y are two nonzero vectors in either
R2 or R3 and θ is the angle between them, then
(1) xTy=‖x‖‖y‖cosθ

Corollary 5.1.2 ( Cauchy-Schwarz Inequality)


If x and y are vectors in either R2 or R3 , then
(2) ︱xTy︱≤‖x‖‖y‖
with equality holding if and only if one of the vectors is 0 or one
vector is a multiple of the other.
Definition
The vector x and y in R2 (or R3) are said to be orthogonal if
xTy=0.
Example
(a) The vector 0 is orthogonal to every vector in R2.
 3   4
(b) The vectors  2  and  6  are orthogonal in R2.
   
 2  1
(c) The vectors   and   are orthogonal in R3.
  3 1
 1  1
   
Scalar and Vector Projections

x
z=x-p
y

p=αu
θ
u

x y cos xT y
  x cos  
y y
The scalar  is called the scalar projection of x and y, and
the vector p is called the vector projection of x and y.
Scalar projection of x onto y:
T
x y

y
Vector projection of x onto y:

1 xT y
p  u   y T y
y y y
Example The point Q is the point on the line y  1 x that is
3
closet to the point (1, 4). Determine the coordinates of Q.

(1, 4)
1
y x
v 3

w
Q
Orthogonality in Rn

The vectors x and y are said to be orthogonal if xTy=0.


2 Orthogonal Subspaces
Definition
Two subspaces X and Y of Rn are said to be orthogonal if
xTy=0 for every x∈X and every y∈Y. If X and Y are
orthogonal, we write X⊥Y.

Example Let X be the subspace of R3 spanned by e1, and


let Y be the subspace spanned by e2.

Example Let X be the subspace of R3 spanned by e1 and e2,


and let Y be the subspace spanned by e3.
Definition
Let Y be a subspace of Rn . The set of all vectors in Rn that
are orthogonal to every vector in Y will be denoted Y⊥. Thus
Y⊥={ x∈Rn︱xTy=0 for every y∈Y }
The set Y⊥ is called the orthogonal complement of Y.

Remarks
1. If X and Y are orthogonal subspaces of Rn, then X∩Y={0}.
2. If Y is a subspace of Rn, then Y⊥ is also a subspace of Rn.
Fundamental Subspaces
Theorem 5.2.1 ( Fundamental Subspaces Theorem)
If A is an m×n matrix, then N(A)=R(AT) ⊥ and N(AT)=R(A) ⊥.

Theorem 5.2.2 If S is a subspace of Rn, then


dim S+dim S⊥=n. Furthermore, if {x1, …, xr} is a basis for S and
{xr+1, …, xn} is a basis for S⊥, then {x1, …, xr, xr+1, …, xn}
is a basis for Rn.
Definition
If U and V are subspaces of a vector space W and each
w∈W can be written uniquely as a sum u+v, where u∈U and
v∈V, then we say that W is a direct sum of U and V, and we
write W=U V.

Theorem 5.2.3 If S is a subspace of Rn, then Rn=S S⊥.

Theorem 5.2.4 If S is a subspace of Rn, then (S⊥) ⊥=S.


Theorem 5.2.5 If A is an m×n matrix and b∈Rm, then
either there is a vector x∈Rn such that Ax=b or there is a
vector y∈Rm such that ATy=0 and yTb≠0.

1 1 2
 
Example Let A  0 1 1
1 3 4
 

Find the bases for N(A), R(AT), N(AT), and R(A).


4 Inner Product Spaces
Definition
An inner product on a vector space V is an operation on V
that assigns to each pair of vectors x and y in V a real
number <x, y> satisfying the following conditions:

Ⅰ. <x, x>≥0 with equality if and only if x=0.

Ⅱ. <x, y>=<y, x> for all x and y in V.

Ⅲ. <αx+βy, z>=α<x, z>+β<y, z> for all x, y, z in V and all

scalars α and β.
The Vector Space Rm×n

Given A and B in Rm×n, we can define an inner product by

m n
A, B   aijbij
i 1 j 1
Basic Properties of Inner product Spaces

If v is a vector in an inner product space V, the length or norm


of v is given by
v  v, v

Theorem 5.4.1 ( The Pythagorean Law )


If u and v are orthogonal vectors in an inner product space V,
then

uv  u  v
2 2 2
1 1   1 1 
   
Example If A  1 2 and B   3 0
3 3   3 4
   

then A, B  6

A 5
B 6
Definition
If u and v are vectors in an inner product space V and v≠0,
then the scalar projection of u onto v is given by
u，v

v

and the vector projection of u onto v is given by

 1  u, v
p    v   v
 v  v, v
Theorem 5.4.2 ( The Cauchy- Schwarz Inequality)
If u and v are any two vectors in an inner product space V, then

u, v  u v

Equality holds if and only if u and v are linearly dependent.


5 Orthonormal Sets
Definition
Let v1, v2, …, vn be nonzero vectors in an inner product
space V. If <vi, vj>=0 whenever i≠j, then { v1, v2, …, vn} is
said to be an orthogonal set of vectors.

Example The set {(1, 1, 1)T, (2, 1, -3)T, (4, -5, 1)T} is an
orthogonal set in R3.

Theorem 5.5.1 If { v1, v2, …, vn} is an orthogonal set of


nonzero vectors in an inner product space V, then v1, v2, …,vn
are linearly independent.
Definition
An orthonormal set of vectors is an orthogonal set of unit
vectors.

The set {u1, u2, …, un} will be orthonormal if and only if


u i , u j   ij
where 1 if i  j
 ij  
0 if i  j

Theorem 5.5.2 Let { u1, u2, …, un} be an orthonoemal basis


n
for an inner product space V. If v  c u
i 1
i i
, then ci=<v, ui>.
Corollary 5.5.3 Let { u1, u2, …, un} be an orthonoemal basis
n n
for an inner product space V. If u  a u
i 1
i i
and v  b u
i 1
i i
, then

n
u, v   ai bi
i 1

Corollary 5.5.4 If { u1, u2, …, un} is an orthonoemal basis


n
for an inner product space V and v  c u
i 1
i i
, then

n
v   ci2
2

i 1
Orthogonal Matrices
Definition
An n×n matrix Q is said to be an orthogonal matrix if the
column vectors of Q form an orthonormal set in Rn.

Theorem 5.5.5 An n×n matrix Q is orthogonal if and only if

QTQ=I.

 cos  sin 
Example For any fixed  , the matrix Q   
 sin cos 

is orthogonal.
Properties of Orthogonal Matrices
If Q is an n×n orthogonal matrix, then
(a) The column vectors of Q form an orthonormal basis for Rn.
(b) QTQ=I
(c) QT=Q-1
(d) det(Q)=1 or -1
(e) The thanspose of an orthogonal matrix is an orthogonal
matrix.
(f) The product of two orthogonal matrices is also an orthogonal
matrix.
6 The Gram-Schmidt Orthogonalization
Process

Theorem 5.6.1 ( The Gram-Schmidt Process)

Let {x1, x2, …, xn} be a basis for the inner product space V. Let
 1 
u1    x1

 x1 

and define u2, …, un recursively by


1
u k 1  ( x k 1  p k ) for k=1, …, n-1
x k 1  p k
where
pk=<xk+1, u1>u1+<xk+1, u2>+‥‥<xk+1, uk>uk
is the projection of xk+1 onto Span(u1, u2, …, uk). The set
{u1, u2, …, un}
is an orthonormal basis for V.
1  1 4
 
1 4 2
Example Let A
1 4 2
 
1  1 0 

Find an orthonormal basis for the column space of A.

You might also like