Numerical Analysis

Orthogonal polynomials

Instructor: Yifan Chen

Courant Institute, New York University

2025 Spring

1/17
Review

Goal: approximating functions by linear combination of

simpler basis functions

We have covered
▶ Polynomial interpolation
▶ Continuous least squares and orthogonal polynomial

Today: more general orthogonal polynomials

2/17
Weighted least squares

Often in applications the given data are not equally important

everywhere. For example, we may know that the data are
more accurate in some regions than in others.

3/17
Weighted least squares

Often in applications the given data are not equally important

everywhere. For example, we may know that the data are
more accurate in some regions than in others.

The weight function is then used to indicate and take into

account where f (x) should be respected more and where it
matters less. A weight function on an interval [a, b] must be
nonnegative, w(x) ≥ 0, and it may vanish only at isolated
points.

3/17
Weighted least squares
The basic approximation problem is now
Z b
min ψ(c) = w(x)(f (x) − v(x))2 dx,
c a

where still
n
X
v(x) = cj ϕj (x).
j=0

4/17
Weighted least squares
The basic approximation problem is now
Z b
min ψ(c) = w(x)(f (x) − v(x))2 dx,
c a

where still
n
X
v(x) = cj ϕj (x).
j=0

with the modification, the normal equation reads,

Z b Z b
B̃j,k = w(x)ϕj (x)ϕk (x)dx, b̃j = w(x)f (x)ϕj (x)dx.
a a

To make B̃ diagonal, the orthogonality condition is

generalized into
Z b
w(x)ϕj (x)ϕk (x)dx = 0, j ̸= k.
a
4/17
Last Class: Three Term Recurrence

For orthogonal polynomial

xϕk (x) = ck+1 ϕk+1 (x) + ck ϕk (x) + ck−1 ϕk−1 (x), k ≥ 1.

5/17
Gram–Schmidt Process and Three Term Recurrence

Given an interval [a, b] and a weight function w(x), set

ϕ0 (x) = 1,
ϕ1 (x) = x − β1 ,
ϕj (x) = (x − βj ) ϕj−1 (x) − γj ϕj−2 (x), j ≥ 2,

where Rb
a xw(x) [ϕj−1 (x)]2 dx
βj = R b 2
, j ≥ 1,
a w(x) [ϕj−1 (x)] dx
Rb
a xw(x)ϕj−1 (x)ϕj−2 (x)dx
γj = Rb 2
, j ≥ 2.
a w(x) [ϕj−2 (x)] dx

6/17
Legendre polynomials

For w(x) ≡ 1 and [a, b] = [−1, 1],

ϕ0 (x) = 1

ϕ1 (x) = x

2j + 1 j
ϕj+1 (x) = xϕj (x) − ϕj−1 (x), j ≥ 1.
j+1 j+1

7/17
Laguerre polynomials

For w(x) = exp(−x) and [a, b] = [0, ∞)

ϕ0 (x) = 1

ϕ1 (x) = 1 − x

2j + 1 − x j
ϕj+1 (x) = xϕj (x) − ϕj−1 (x), j ≥ 1.
j+1 j+1

8/17
Hermite polynomials

For w(x) = exp −x2 and [a, b] = [−∞, ∞)

ϕ0 (x) = 1

ϕ1 (x) = 2x

ϕj+1 (x) = 2xϕj (x) − 2jϕj−1 (x), j ≥ 1.

9/17
Chebyshev polynomials

1
For w(x) = (1 − x2 )− 2 and [a, b] = [−1, 1]

ϕ0 (x) = 1

ϕ1 (x) = x

ϕj+1 (x) = 2xϕj (x) − ϕj−1 (x), j ≥ 1.

10/17
Chebyshev polynomials

The Chebyshev polynomials can also be elegantly defined as

ϕj (x) = Tj (x) = cos(j arccos x)

To verify orthogonality,
Z 1 Z 1
Tj (x)Tk (x)
w(x)Tj (x)Tk (x)dx = √ dx
−1 −1 1 − x2
π
®
0, j ̸= k,
Z
= cos(jθ) cos(kθ)dθ = π
0 2 , j = k.

11/17
Chebyshev polynomials

Figure: Chebyshev polynomials of degrees 4, 5, and 6.

12/17
Chebyshev polynomials

The n zeros (roots) of Tn (x) are called the Chebyshev points.

From the definition, they are the first n roots of the equation

cos(nθ) = 0, θ = arccos x,
(k−1/2)
and hence θk = n π. Consequently, the roots xk = cos (θk )
are given by

2k − 1
Å ã
xk = cos π , k = 1, . . . , n,
2n

and satisfy in particular 1 > x1 > · · · > xn > −1.

13/17
Chebyshev polynomials

Moreover, the n + 1 extremum points of Tn (x) on the interval

[−1, 1] (these are the n − 1 points where the first derivative
Tn′ (x) vanishes, plus the interval ends) are at
Å ã
k
ξk = cos π , k = 0, 1, . . . , n,
n

and thus they satisfy

Tn (ξk ) = (−1)k ,

14/17
Chebyshev Min-Max Property
Recall that we wanted to solve
n
Y
min max | (x − xj )|
xj x∈[−1,1]
j=1

15/17
Chebyshev Min-Max Property
Recall that we wanted to solve
n
Y
min max | (x − xj )|
xj x∈[−1,1]
j=1

From the recursion formula, the leading coefficient of Tn (x)

equals 2n−1 . Thus, the monic (i.e., polynomial with leading
coefficient 1) Chebyshev polynomials are defined as
1
Ten (x) = Tn (x)
2n−1
n
Q
Let xj be the Chebyshev points, Ten (x) = (x − xk )
k=1

15/17
Chebyshev Min-Max Property
Recall that we wanted to solve
n
Y
min max | (x − xj )|
xj x∈[−1,1]
j=1

From the recursion formula, the leading coefficient of Tn (x)

equals 2n−1 . Thus, the monic (i.e., polynomial with leading
coefficient 1) Chebyshev polynomials are defined as
1
Ten (x) = Tn (x)
2n−1
n
Q
Let xj be the Chebyshev points, Ten (x) = (x − xk )
k=1
The Min-Max property:
1
= max |Ten (x)| ≤ max |Pn (x)|
2n−1 x∈[−1,1] x∈[−1,1]

Pn (x) belongs to the set of all monic polynomials of degree n. 15/17

Chebyshev Min-Max Property

Recall that
Tn (ξk ) = (−1)k ,
such that,
1
Ten (ξk ) = (−1)k ,
2n−1

16/17
Chebyshev Min-Max Property

Recall that
Tn (ξk ) = (−1)k ,
such that,
1
Ten (ξk ) = (−1)k ,
2n−1
Let’s assume that Pn (x) is a monic polynomial of degree n
such that maxx∈[−1,1] |Pn (x)| < 21−n .

Let
q(x) = Ten (x) − Pn (x)
®
1 k < 0, k even
q (ξk ) = n−1 (−1) − Pn (ξk )
2 > 0, k odd
for k = 0, 1, . . . , n.

16/17
Chebyshev Min-Max Property

As such, q(x) changes sign at least n times on the interval

[−1, 1], and from the intermediate value theorem it has at
least n zeros.

However, this is impossible, as q(x) is a polynomial of degree

n − 1, the fundamental theorem of algebra tells us that it has
at most n − 1 roots. This implies that q ≡ 0, and hence
Pn ≡ Tn .

This proves the Min-Max property of Chebyshev polynomials

17/17