Scribd
Upload
142 views36 pages

05 Orthogonalization

This document discusses various methods for orthogonalization in linear algebra, including orthogonal matrices, the Gram-Schmidt process, Givens rotations, and Householder transformations. It provides examples and algorithms for applying each method. The Gram-Schmidt process constructs an orthogonal basis from a set of vectors. Givens rotations and Householder transformations are used in QR factorizations to decompose a matrix into an orthogonal matrix and upper triangular matrix.

Uploaded by

Alexandru Daniel Lacraru
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
142 views36 pages

05 Orthogonalization

This document discusses various methods for orthogonalization in linear algebra, including orthogonal matrices, the Gram-Schmidt process, Givens rotations, and Householder transformations. It provides examples and algorithms for applying each method. The Gram-Schmidt process constructs an orthogonal basis from a set of vectors. Givens rotations and Householder transformations are used in QR factorizations to decompose a matrix into an orthogonal matrix and upper triangular matrix.

Uploaded by

Alexandru Daniel Lacraru
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 36

Linear algebra

Orthogonalization
Marta Jaroov
http://homel.vsb.cz/~dom033/
Outline
Orthogonal matrices
Gram-Schmidt process
Givens rotations
Householder transformation

Euclidean inner product
Euclidean inner product (dot product) is a mapping
u R
n
, v R
n
(u, v) R

It is defined for two vectors u, v by
(u, v) = u
1
v
1
+ + u
n
v
n
= u
T
v

It satisfies for each u, v, w a R the following properties












Orthogonal matrices
Square matrix Q satisfying
Q
T
Q = I
is called orthogonal matrix. This matrix has
orthogonal columns and it holds
Q
-1
= Q
T
.


The columns form an orthogonal set of vectors, i.e.

QR factorization
For an arbitrary matrix there exist an orthogonal
matrix and an upper triangular matrix ,
such that
A = QR.
QR factorization
m = n:




The columns of A can be written as a linear combination of
the columns of Q




Gram-Schmidt process
Example:
For matrix




Find the columns and the elements of matrix R.
Gram-Schmidt process
Gram-Schmidt process
Gram-Schmidt process
Gram-Schmidt process
Gram-Schmidt process
Gram-Schmidt process
Gram-Schmidt process
Problem: Find the columns and the elements of
matrix R

Process:


Let us consider that we already know

(*)
||a||
2
= (a,a)
Gram-Schmidt process
Requirement: is orthogonal to

for



Since
for
|| v
j
||
2
= r
jj
2
q
j
T
q
j
= r
jj
2

Algorithm
function [Q,R] = my_gram_schmidt(A)
n = size(A,1); Q = zeros(n); R = zeros(n);
for j=1:n
v=A(:,j);
for i=1:j-1
R(i,j)=Q(:,i)'*A(:,j);
v=v-R(i,j)*Q(:,i);
end
R(j,j)= norm(v);
Q(:,j)=v/R(j,j);
end
Remark: The Gram-Schmidt process can be stabilized by a small modification
modified Gram-Schmidt, which gives the same result as the original formula in
exact arithmetic and introduces smaller errors in finite-precision arithmetic.
r
jj
= || v
j
||
modify A(:,j) to v
for more accuracy
Givens transformation
Let us consider Givens matrix (rotation matrix)



which rotates a vector (a,b)
T

in the xy-plane through
an angle about the origin.

We will use a notation

x = (a,b)
T

Gx
Example in Matlab: givens_rotation

Givens transformation
We can use matrix G to zeroing elements. Let us
consider that



It is easy to see that




Givens transformation
Generalization for vectors of the order n





To zeroing the element in the j-th row we have
Givens QR method
Form of the rotation matrix to zeroing element in
the i-th row is


Using the rotation matrices we will edit matrix A:
where
Example
Example:


Solution:
To set A(3,1) = 0, we need to build matrix G
1
(2,3)

G(i-1,i) =
i-1
i
Example



To set A
1
(2,1) = 0, we need to build matrix G
2
(1,2)

Example
To set A
2
(3,2) = 0, we need matrix G
3
(2,3)

Algorithm
function [Q,R] = my_givens_QR(A)
n = size(A,1); Q=eye(n); R=A;
for j=1:n
for i=n:(-1):j+1
x=R(:,j);
if norm([x(i-1),x(i)])>0
c=x(i-1)/norm([x(i-1),x(i)]);
s=-x(i)/norm([x(i-1),x(i)]);

G=eye(n); G([i-1,i],[i-1,i])=[c,s;-s,c];
R=G'*R;
Q=Q*G;
end
end
end
Householder transformation
For vector x we are able to find its reflection Px to
axis x






Both vectors have the same length
Householder transformation
Px is mirror image of x with axis H. H is orthogonal
to vector
Reflection matrix:
Function sign
sign(x)
Householder transformation
The image of x is not unique: for all
For numerical stability we choose

Householder QR method
Using the reflections P we can modify matrix A
A = = R
Example
Example:


Solution:
Example
Example
Algorithm
function [Q,R] = my_householder_QR(A)
n = size(A,1); Q=eye(n); R=A; I = eye(n);
for j=1:n-1
x=R(j:n,j);
v=-sign(x(1))*norm(x)*eye(n-j+1,1)-x;
if norm(v)>0,
v=v/norm(v);
P=I; P(j:n,j:n)=P(j:n,j:n)-2*v*v';
R=P*R;
Q=Q*P;
end
end

QR factorization
m > n:
Full QR factorization




Reduced QR factorization
Matlab function: qr
[Q,R] = qr(A), where A is m-by-n, produces
an m-by-n upper triangular matrix R and an m-by-
m unitary matrix Q so that A = Q*R.

[Q,R] = qr(A,0) produces the "economy size"
decomposition. If m>n, only the first n columns of
Q and the first n rows of R are computed. If m<=n,
this is the same as [Q,R] = qr(A)
For more details see: help qr
References
Gilbert W. Stewart, Matrix Algorithms: Basic decompositions
(available on Google books)

Lloyd Nicholas Trefethen, David Bau, Numerical linear algebra
(available on Google books)

Kozubek, Brzobohat, Hapla, Jaroov, Markopoulos:
LINERN ALGEBRA S MATLABEM, http://mi21.vsb.cz/
(in Czech)

Gram-Schmidt in 9 Lines of MATLAB
http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-
2010/related-resources/MIT18_06S10_gramschmidtmat.pdf

You might also like