0% found this document useful (0 votes)
41 views5 pages

Engg Maths

The document describes the Gram-Schmidt orthogonalization process, which takes a set of linearly independent but non-orthogonal basis vectors and constructs an orthonormal basis from them. It does this by successively orthogonalizing each new basis vector against the previous ones using a subtraction process. The document provides an example applying this to the polynomial basis to construct the Legendre polynomials. It notes that while Gram-Schmidt works, a modified approach yields a more efficient algorithm for generating orthogonal polynomials.

Uploaded by

king
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
41 views5 pages

Engg Maths

The document describes the Gram-Schmidt orthogonalization process, which takes a set of linearly independent but non-orthogonal basis vectors and constructs an orthonormal basis from them. It does this by successively orthogonalizing each new basis vector against the previous ones using a subtraction process. The document provides an example applying this to the polynomial basis to construct the Legendre polynomials. It notes that while Gram-Schmidt works, a modified approach yields a more efficient algorithm for generating orthogonal polynomials.

Uploaded by

king
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Jim Lambers

MAT 415/515
Fall Semester 2013-14
Lecture 3 Notes

These notes correspond to Section 5.2 in the text.

Gram-Schmidt Orthogonalization
We have seen that it can be very convenient to have an orthonormal basis for a given vector
space, in order to compute expansions of arbitrary vectors within that space. Therefore, given a
non-orthonormal basis, it is desirable to have a process for obtaining an orthonormal basis from it.
Fortunately, we have such a process, known as Gram-Schmidt orthogonalization. Suppose that
we have a linearly independent, but not orthonormal, set of functions {χ1 , χ2 , . . .} that span a given
vector space V . To construct an orthonormal set {ϕ1 , ϕ2 , . . .} from this set, we proceed as follows.
First, to obtain ϕ1 , we simply normalize χ1 :
χ1
ϕ1 = .
kχ1 k

Next, to obtain ϕ2 , we need to ensure that it is orthogonal to ϕ1 , and then normalize it.
As an intermediate step, we seek a function ψ2 of the form

ψ2 = χ2 + c12 ϕ1

such that hϕ1 |ψ2 i = 0. Then, we can set ϕ2 = ψ2 /kψ2 k. Taking the scalar product of both sides of
the above equation with ϕ1 , we obtain

0 = hϕ1 |ψ2 i = hϕ1 |χ2 i + c12 hϕ1 |ϕ1 i.

Because the ϕj are orthonormal, it follows that

c12 = −hϕ1 |χ2 i.

We conclude that ϕ2 can be obtained as follows:

ψ2 = χ2 − hϕ1 |χ2 iϕ1


ψ2
ϕ2 = .
kψ2 k

We now have a set of two functions that is orthonormal.


Now, to obtain ϕ3 , we must ensure that it is orthogonal to ϕ1 and ϕ2 , and then normalized.
To that end, we seek a function ψ3 of the form

ψ3 = χ3 + c13 ϕ1 + c23 ϕ2

such that hϕ1 |ψ3 i = hϕ2 |ψ3 i = 0. Then, we can set ϕ3 = ψ3 /kψ3 k. Taking the scalar product of
both sides of the above equation with ϕ1 , and then separately, ϕ2 , we obtain

0 = hϕ1 |ψ3 i = hϕ1 |χ3 i + c13 hϕ1 |ϕ1 i + c23 hϕ1 |ϕ2 i
0 = hϕ2 |ψ3 i = hϕ2 |χ3 i + c13 hϕ2 |ϕ1 i + c23 hϕ2 |ϕ2 i.

1
Because the ϕj are orthonormal, it follows that

c13 = −hϕ1 |χ3 i, c23 = −hϕ2 |χ3 i.

We conclude that ϕ3 can be obtained as follows:

ψ3 = χ3 − hϕ1 |χ3 iϕ1 − hϕ2 |χ3 iϕ2


ψ3
ϕ3 = .
kψ3 k

We now have a set of three functions that are orthonormal.


Continuing this process, we see that we can obtain each function ϕj as follows:
j−1
X
ψj = χj − hϕk |χj iϕk
k=0
ψj
ϕj = .
kψj k

This yields a set of functions {ϕ1 , ϕ2 , . . .} that is an orthonormal basis of the space spanned by
{χ1 , χ2 , . . .}, with respect to the scalar product that is used.
Example We wish to obtain a set of orthonormal polynomials with respect to the scalar product
Z 1
hf |gi = f ∗ (s)g(s) ds.
−1

This will be accomplished by applying Gram-Schmidt orthogonalization to the set {1, x, x2 , x3 , . . .}.
Setting χj (x) = xj for j = 0, 1, 2, . . . , our orthogonal set {ϕj }, j = 0, 1, 2, . . . ,, is obtained as follows:

ψ0 (x) = χ0 (x)
= 1,
ψ0 (x)
ϕ0 (x) =
kψ0 k
1
=
h1|1i1/2
1
= hR i1/2
1
−1 1 ds
1
= √ ,
2
ψ1 (x) = χ1 (x) − hϕ0 |χ1 iϕ0 (x)
 
1 1
= x − √ x √
2 2
Z 1
1
= x− s ds
2 −1
1
= x− 0
2
= x,

2
ψ1 (x)
ϕ1 (x) =
kψ1 k
x
=
hx|xi1/2
x
= hR i1/2
1 2
−1 s ds
r
3
= x,
2
ψ2 (x) = χ2 (x) − hϕ0 |χ2 iϕ0 (x) − hϕ1 |χ2 iϕ1 (x)
  *r + r
1 1 3 2 3
x2 − √ x2 √ −

= x x x
2 2 2 2
1 1 2
Z Z 1
2 3
= x − s ds − x s3 ds
2 −1 2 −1
12 3
= x2 − − 0
23 2
1
= x2 − ,
3
ψ2 (x)
ϕ2 (x) =
kψ2 k
x2 − 13
= 1/2
x2 − 1 x2 − 1


3 3
x2 − 13
= hR i1/2
1 1 2

−1 x − 3 ds
r  
5 3 2 1
= x − .
2 2 2

Continuing this process, we obtain


r  
7 5 3 3
ϕ3 (x) = x − x .
2 2 2
In general, r
2n + 1
ϕn (x) = Pn (x),
2
where Pn (x) is the Legendre polynomial of nth degree. We will learn more about these orthogonal
(but not orthonormal) polynomials later in this course. 2
While Gram-Schmidt orthogonalization can be applied to the monomial bais {1, x, x2 , x3 , . . .}
to obtain an orthonormal sequence of polynomials, it can be quite cumbersome, as can be seen from
the preceding example. However, a modification of this procedure can yield a much more efficient
approach.
Suppose that we have already generated a sequence of n orthonormal polynomials ϕ0 , ϕ1 , ϕ2 , . . . , ϕn−1
with respect to some scalar product
Z b
hf |gi = f ∗ (x)g(x)w(x) dx,
a

3
where ϕj is of degree j for j = 0, 1, 2, . . . , n − 1. Then, to obtain ϕn , which of degree n, we
orthogonalize xϕn−1 (x), which is of degree n, against ϕ0 , ϕ1 , . . . , ϕn−1 using the same approach as
in Gram-Schmidt orthogonalization. That is, we compute
n−1
X
ψn (x) = xϕn−1 (x) − hϕj |xϕn−1 iϕj (x),
j=0
ψn (x)
ϕn (x) = .
kψn k
Now, consider the scalar product hϕj |xϕn−1 i. Using the properties of the scalar product, we
have
hϕj |xϕn−1 i = hxϕj |ϕn−1 i.
However, because ϕ0 , ϕ1 , . . . , ϕn−1 are orthonormal, hp|ϕn−1 i = 0 if p(x) is any polynomial of
degree less than n − 1. Because xϕj (x) is of degree j + 1, it follows that hϕj |xϕn−1 i = 0 whenever
j + 1 < n − 1, or j < n − 2. Therefore, our orthogonalization procedure simplifies to

ψn (x) = xϕn−1 (x) − hϕn−2 |xϕn−1 iϕn−2 (x) − hϕn−1 |xϕn−1 iϕn−1 (x),
ψn (x)
ϕn (x) = .
kψn k
That is, any family of orthogonal polynomials satisfies a three-term recurrence relation, in which
each polynomial depends on the previous two. Table lists several families of orthogonal polynomials
that can be generated from such a recurrence relation; we will see some of these families later in
the course.
Polynomials Scalar Product
R1
Legendre Pn (x)Pm (x) dx = 2δmn /(2n + 1)
R−1
1 ∗ ∗ (x) dx = δ
Shifted Legendre P (x)Pm mn /(2n + 1)
R01 n
Chebyshev, first kind Tn (x)Tm (x)(1 − x )−1/2 dx = δmn π/(2 − δn0 )
2
R−1
1 ∗ ∗ (x)[x(1 − x)]−1/2 dx = δ
Shifted Chebyshev, first kind T (x)Tm mn π/(2 − δn0 )
R01 n 2 )1/2 dx = δ
Chebyshev, second kind R−1 U n (x)U m (x)(1 − x mn π/2
∞ −x
Leguerre R0∞ Lnk (x)Lm (x)e dx = δmn
Associated Laguerre L (x)L k (x)e−x dx = δ
mn (n + k)!/n!
R0∞ n m
−x 2 √
Hermite −∞ Hn (x)Hm (x)e dx = 2n δmn πn!

As can be seen in the following example, Gram-Schmidt orthogonalization can be applied to


vectors in any inner product space, such as vectors in Rn .
Example Given the vectors in R3 ,
     
1 1 0
|a1 i =  1  , |a2 i =  2  , |a3 i =  1  ,
−2 −3 1

we will use Gram-Schmidt orthogonalization to obtain an orthonormal set of vectors, {|b1 i, |b2 i, |b3 i}.
We have
|a1 i
|b1 i =
ha1 |a1 i1/2

4
1
= |a1 i
61/2
 
1
1
= √  1 ,
6 −2
|b02 i = |a2 i − hb1 |a2 i|b1 i
   
1 1
9 1
=  2 − √ √  1 
−3 6 6 −2
 
−1/2
=  1/2  ,
0
|b02 i
|b2 i =
hb2 |b02 i1/2
0

1
= |b02 i
1 1/2

2
 
−1
1 
= √ 1 ,
2 0
|b03 i = |a3 i − hb1 |a3 i|b1 i − hb2 |a3 i|b2 i
     
0   1 −1
1 1 1 1
=  1  − −√ √  1 − √ √  1 
1 6 6 −2 2 2 0
     
0 1 −1
1 1
=  1 +  1 −  1 
6 2
1 −2 0
 
1
2 
= 1 ,
3
1
|b03 i
|b3 i =
hb03 |b03 i1/2
1
= |b03 i
4 1/2

3


1
1
= √  1 .
3 1

We conclude that our orthonormal set of vectors is


     
1 −1 1
1 1 1
|b1 i = √  1  , |b2 i = √  1  , |b3 i = √  1  .
6 −2 2 0 3 1
2

You might also like