Math 128A Spring 2002 Handout # 19

Sergey Fomel April 2, 2002

Answers to Homework 7: Approximation: Polynomial Approximation

1. (a) Is the collection of functions φ1 (x) = 1, φ2 (x) = x, and φ3 (x) = sin x orthogonal with
respect to the inner product
Zπ
< f , g>= f (x) g(x) d x ? (1)
−π

If not, find the corresponding orthogonal functions using the Gram-Schmidt orthogonal-
ization process

φ̂1 (x) = φ1 (x) ; (2)

k−1
X <φk , φ̂i >
φ̂k (x) = φk (x) − φ̂i (x) , k = 2, 3, . . . (3)
i=1
<φ̂i , φ̂i >

Answer: These functions are not orthogonal, because

Zπ Zπ
<φ2 , φ3 > = x sin x d x = − x cos x|π−π + cos x d x
−π −π
= (sin x − x cos x)|π−π = −2 π cos π = 2 π 6= 0.

Using the Gram-Schmidt orthogonalization process, we get

φ̂1 (x) = 1 ;
Rπ
x dx
−π
φ̂2 (x) = x − π =x;
R
dx
−π
Rπ Rπ
sinx d x x sinx d x
−π −π 2π 3x
φ̂3 (x) = sin x − − x = sin x − x = sin x − .
Rπ Rπ π2
3π
2 3
dx x 2d x
−π −π

1
 (b) Using the Gram-Schmidt process, find the first three orthogonal polynomials with respect
to the inner product
Z∞
< f , g>= w(x) f (x) g(x) d x , (4)
0
where w(x) = e (a > 0).
−a x

Hint: Use the equality (for integer n)

Z∞
n!
x n e−a x d x = .
a n+1
0

Answer:
φ̂1 (x) = 1 ;
R∞
w(x)x d x
0 1
φ̂2 (x) = x − =x− ;
R∞ a
dx
0
R∞ R∞
w(x)x 2 d x w(x)x 2 x − a1 d x 



0 0 1
φ̂3 (x) = x − 2
− x−
R∞ R∞
2 a
dx x − a1 dx
0 0
6−2  
2 2 a4 1 4x 2
= x − 2− 2−2+1
x− = x2 − + 2.
a a a a
a3

2. (a) Prove that the constant function f (x) = a that fits inconsistent measurements f 1 , f 2 , . . . , f n
in the least-squares sense corresponds to the mean value
n
1X
a= fk . (5)
n
k=1

Answer:
We need to minimize the least-squares norm
n
X
F(a) = (a − f k )2 .
k=1

Differentiating with respect to a leads to the linear equation

n
X
0
F (a) = (a − f k ) = 0 ,
k=1

whose solution is
n
X n
X
fk fk
k=1 k=1
a= n = .
X n
1
k=1

2
(b) Prove that, if the measurements f 1 , f 2 , . . . , f n are taken at the integer values xk = k,
k = 1, 2, . . . , n, the linear function f (x) = a x +b that fits the data in the least-squares sense
corresponds to the values

" n n
#
6 X X
a = 2 k f k − (n + 1) fk ; (6)
n (n 2 − 1)
k=1 k=1
n n
" #
2 X X
b = (2 n + 1) fk − 3 k fk . (7)
n (n − 1)
k=1 k=1

Answer:
We are minimizing the least-squares norm
n
X
F(a, b) = (a k + b − f k )2 .
k=1

Differentiating with respect to a and b leads to the system of two linear equations
n n n n
∂F X X
2
X X
= k (a k + b − f k ) = a k +b k− k fk
∂a
k=1 k=1 k=1 k=1
n
n (n + 1) (2 n + 1) n (n + 1) X
= a +b − k fk = 0 ,
6 2
k=1
n n n n
∂F X X X X
= (a k + b − f k ) = a k +b 1− fk
∂b
k=1 k=1 k=1 k=1
n
n (n + 1) X
= a +bn − fk = 0 .
2
k=1
In the matrix form, these equations are
 n
X

 
n (n + 1) (2 n + 1) n (n + 1)    k fk 
6 2 a  k=1 
.
=
 
n (n + 1) n
b
   X 
n  fk 
2
k=1

The determinant of the system matrix is

n (n + 1) 2 n 2 (n 2 − 1)
 
n (n + 1) (2 n + 1)
n − = ,
6 2 12
and the solution of the system is
" n n
# " n n
#
12 X n (n + 1) X 6 X X
a = n k fk − fk = 2 k f k − (n + 1) fk ;
n 2 (n 2 − 1) 2 n (n 2 − 1)
k=1 k=1 k=1 k=1
n n
" #
12 n (n + 1) (2 n + 1) X n (n + 1) X
b = fk − k fk
n 2 (n 2 − 1) 6 2
k=1 k=1
n n
" #
2 X X
= (2 n + 1) fk − 3 k fk .
n (n − 1)
k=1 k=1

3
 3. Chebyshev polynomials Tk (x) can be defined by the recursive relationship

T0 (x) = 1 (8)
T1 (x) = x (9)
Tk+1 (x) = 2 x Tk (x) − Tk−1 (x) , k = 1, 2, . . . (10)

One can evaluate the Chebyshev polynomial representation

n
X
f (x) = ck Tk (x) (11)
k=0

efficiently with the algorithm

C HEBYSHEV S UM(x, c0 , c1 , . . . , cn )
1 ĉ1 ← 0
2 ĉ0 ← cn
3 for k ← n − 1, n − 2, . . . , 0
4 do
5 t ← ĉ1
6 ĉ1 ← ĉ0
7 ĉ0 ← ck + 2 x ĉ0 − t
8 return (ĉ0 − x ĉ1 )

Design an analogous algorithm for the Hermite polynomial representation

n
X
f (x) = ck Hk (x) . (12)
k=0

Hermite polynomials Hk (x) satisfy the recursive relationship

H0 (x) = 1 (13)
H1 (x) = 2 x (14)
Hk+1 (x) = 2 x Hk (x) − 2 k Hk−1 (x) , k = 1, 2, . . . (15)

Answer:
Define a family of coefficients ĉk (x) such that ĉn+1 = 0, ĉn+2 = 0, and

ck = ĉk (x) − 2 x ĉk+1 (x) + 2 (k + 1) ĉk+2 (x) , k = 0, 1, . . . , n

Then
n
X n
X n
X n
X
f (x) = ck Hk (x) = ĉk Hk (x) − 2 x ĉk+1 Hk (x) + 2 (k + 1) ĉk+2 Hk (x) .
k=0 k=0 k=0 k=0

4
 Shifting the indeces in the last two sums, we obtain
n
X n
X n
X n
X
f (x) = ck Hk (x) = ĉk Hk (x) − 2 x ĉk Hk−1 (x) + 2 (k − 1) ĉk Hk−2 (x)
k=0 k=0 k=1 k=2
n
X  
= ĉ0 H0 (x) + ĉ1 H1 (x) − 2 x ĉ1 H0 (x) + ĉk Hk (x) − 2 x Hk−1 (x) + 2 (k − 1) Hk−2 (x) .
k=2

The last term is zero because of the recursive property of Hermite polynomials. Finally,
f (x) = ĉ0 H0 (x) + ĉ1 H1 (x) − 2 x ĉ1 H0 (x) = ĉ0 + 2 x ĉ1 − 2 x ĉ1 = ĉ0 .
The recursion for computing ĉ0 is
ĉn+1 = 0
ĉn = cn
ĉk = ck + 2 x ĉk+1 − 2 (k + 1) ĉk+2 k = n − 1, n − 2, . . . , 0 ,
which results in the following algorithm:

H ERMITE S UM(x, c0 , c1 , . . . , cn )
1 ĉ1 ← 0
2 ĉ0 ← cn
3 for k ← n − 1, n − 2, . . . , 0
4 do
5 t ← ĉ1
6 ĉ1 ← ĉ0
7 ĉ0 ← ck + 2 x ĉ0 − 2 (k + 1) t
8 return ĉ0

Note: Hermite polynomials are orthogonal with respect to the inner product
Z∞
2
< f , g>= e−x f (x) g(x) d x .
−∞

4. (Programming)

(a) Write a program for evaluating Chebyshev polynomial representation using the algorithm
above. Test your program by approximating the infinite sum
∞
1−t x X
2
= t k Tk (x) (16)
1−2t x +t
k=0

with the finite sum

n
X
t k Tk (x) . (17)
k=0
Plot (or tabulate) the absolute error on the interval −1 ≤ x ≤ 1 for t = 1/2 and n = 5, 10, 15.
Answer:

5
0.035

0.03

0.025

0.02

0.015

0.01

0.005

0
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1
−3
x 10
1

0.8

0.6

0.4

0.2

0
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1
−5
x 10
3.5

2.5

1.5

0.5

0
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

6
Solution:
C program:
#include <stdlib.h> /* for allocation */
#include <stdio.h> /* for output */
#include <assert.h> /* for assertion */
#include <math.h> /* for math functions */

/* Function: chebyshev_sum
-----------------------
Evaluates Chebyshev polynomial representation
f(x) = sum(k=0,n-1) C_k*T_k(x)
n - number of coefficients
x - where to evaluate
c[n] - coefficients
*/
double chebyshev_sum (int n, double x, double c[])
{
int k;
double t, c0, c1;

c1 = 0;
c0 = c[n-1];
for (k=n-2; k >= 0; k--) {
t = c1;
c1 = c0;
c0 = c[k] + 2.*x*c0 - t;
}

return (c0 - x*c1);

}

int main (void) {

int i, k, n[] = {5,10,15}, nx=401;
const double t=0.5;
double *c, *x, *f, e;

x = (double*) malloc (nx*sizeof(double));

f = (double*) malloc (nx*sizeof(double));
assert (x != NULL && f != NULL);

for (k=0; k < nx; k++) {

/* regular grid for plotting */
x[k] = -1. + 2.*k/(nx-1.);
/* true function values */
f[k] = (1.-t*x[k])/(1.-2.*t*x[k] + t*t);
}

c = (double*) malloc ((n[2]+1)*sizeof(double));

assert (c != NULL);

/* chebyshev coefficients c_k = t^k */

for (e=1., k=0; k <= n[2]; k++, e *= t) {
c[k] = e;
}

/* three cases */
for (i=0; i < 3; i++) {
for (k=0; k < nx; k++) {

7
 /* absolute error */
e = fabs(f[k] - chebyshev_sum (n[i]+1, x[k], c));
/* output table */
printf("%d %f %g\n",k,x[k],e);
}
}

exit (0);
}

(b) Write a program for evaluating Hermite polynomial representation using your algorithm
from problem 3. Test your program by approximating the infinite sum
∞ k
t (2x−t)
X t
e = Hk (x) (18)
k!
k=0

with the finite sum

n
X tk
Hk (x) . (19)
k!
k=0
Plot (or tabulate) the absolute error on the interval 0 ≤ x ≤ 1 for t = 1/2 and n = 5, 10, 15.

Answer:

−3
x 10
5

4.5

3.5

2.5

1.5

0.5

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

8
 −6
x 10
3

2.5

1.5

0.5

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
−10
x 10
5

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Solution:
C program:

#include <stdlib.h> /* for allocation */

#include <stdio.h> /* for output */
#include <assert.h> /* for assertion */
#include <math.h> /* for math functions */

/* Function: hermite_sum
---------------------
Evaluates Hermite polynomial representation
f(x) = sum(k=0,n-1) C_k*H_k(x)
n - number of coefficients
x - where to evaluate
c[n] - coefficients
*/
double hermite_sum (int n, double x, double c[])

9
{
int k;
double t, c0, c1;

c1 = 0;
c0 = c[n-1];
for (k=n-2; k >= 0; k--) {
t = c1;
c1 = c0;
c0 = c[k] + 2.*(x*c0 - (k+1)*t);
}

return c0;
}

int main (void) {

int i, k, n[] = {5,10,15}, nx=401;
const double t=0.5;
double *c, *x, *f, e;

x = (double*) malloc (nx*sizeof(double));

f = (double*) malloc (nx*sizeof(double));
assert (x != NULL && f != NULL);

for (k=0; k < nx; k++) {

/* regular grid for plotting */
x[k] = k/(nx-1.);
/* true function values */
f[k] = exp(t*(2.*x[k]-t));
}

c = (double*) malloc ((n[2]+1)*sizeof(double));

assert (c != NULL);

/* hermite coefficients c_k = t^k/k! */

for (e=1., k=0; k <= n[2]; k++, e *= (t/k)) {
c[k] = e;
}

/* three cases */
for (i=0; i < 3; i++) {
for (k=0; k < nx; k++) {
/* absolute error */
e = fabs(f[k] - hermite_sum (n[i]+1, x[k], c));
/* output table */
printf("%d %f %g\n",k,x[k],e);
}
}

exit (0);
}

10
5. (Programming) The following table contains the number of medals won by the United States at
the winter Olympic games:
Year Location Gold Silver Bronze Total Points
1924 CHAMONIX 1 2 1 4 8
1928 SAINT MORITZ 3 2 2 7 15
1932 LAKE PLACID 6 4 2 12 28
1936 GARMISH PARTENKIRCHEN 1 0 3 4 6
1948 SAINT MORITZ 3 5 2 10 21
1952 OSLO 4 6 1 11 25
1956 CORTINA D’AMPEZZO 2 3 2 7 14
1960 SQUAW VALLEY 3 4 3 10 20
1964 INNSBRUCK 1 2 4 7 11
1968 GRENOBLE 1 4 1 6 12
1972 SAPPORO 3 2 3 8 16
1976 INNSBRUCK 3 3 4 10 19
1980 LAKE PLACID 6 4 2 12 28
1984 SARAJEVO 4 4 0 8 20
1988 CALGARY 2 1 3 6 11
1992 ALBERTVILLE 5 4 2 11 25
1994 LILLEHAMMER 6 5 2 13 30
1998 NAGANO 6 3 4 13 28
2002 SALT LAKE CITY 10 13 11 34 67
The points are computed with the formula

Points = 3 × Gold + 2 × Silver + Bronze .

Using the method of least squares, find linear trends of the form f (x) = a + b x for the functions

(a) Points(Year)
(b) Points(Gold)

In each case, find a, b and the Olympic games with the largest and smallest least-squares errors.
Answer:
Points(Year) ≈ −532.218037 + 0.281527 Year
The smallest least-squares error (0.181477) is in 1960 games. The largest error (1267.494312)
is in 2002.

Points(Gold) ≈ 1.736067 + 5.300210 Gold

The smallest least-squares error (0.928761) is in 1924 games. The largest error (150.352466) is
in 2002.
For both “dependencies”, the 2002 games appear to be the least predictable.

11
Solution:
C program:

#include <stdio.h> /* for output */

/* Function: least_squares
-----------------------
Finds straight-line least-squares fit
f(x) ~= a[0] + a[1]*x
to a table of f(x)
n - number of points
x[n] - table of variable
f[n] - table of function
a[2] - output line coefficients
*/
void least_squares(int n, const double *x, const double *f, double *a)
{
int k;
double sx, sx2, sf, sxf, det;

sx=sx2=sf=sxf=0.;
for (k=0; k < n; k++) {
sx += x[k];
sx2 += x[k]*x[k];
sf += f[k];
sxf += x[k]*f[k];
}
det = sx2*n - sx*sx;

a[0] = (sx2*sf - sx*sxf)/det;

a[1] = ( n*sxf - sx*sf )/det;
}

/* number of nodes */
#define NODES 19

/* main program */
int main (void)
{
const int n = NODES;
int k;
double years[] = { 1924.,1928.,1932.,1936.,1948.,1952.,1956.,1960.,1964.,
1968.,1972.,1976.,1980.,1984.,1988.,1992.,1994.,1998.,
2002.};
double points[] = { 8., 15.,28.,6. ,21.,25.,14.,20.,11.,
12.,16.,19.,28.,20.,11.,25.,30.,28.,
67.};
double golds[] = { 1.,3.,6.,1.,3.,4.,2.,3.,1.,
1.,3.,3.,6.,4.,2.,5.,6.,6.,
10.};
double a[2], e;

least_squares(n, years, points, a);

printf("a=%f b=%f\n",a[0],a[1]);
for (k=0; k < n; k++) {
/* error */
e = points[k] - a[0] - a[1]*years[k];
/* output table */

12
 printf("%d \t %d \t %f \t %f\n",k,(int) years[k],e,e*e);
}

least_squares(n, golds, points, a);

printf("a=%f b=%f\n",a[0],a[1]);
for (k=0; k < n; k++) {
/* error*/
e = points[k] - a[0] - a[1]*golds[k];
/* output table */
printf("%d \t %d \t %f \t %f\n",k,(int) years[k],e,e*e);
}

return 0.;
}

13