An Introduction to Scientific Computing with MATLAB and Python Tutorials 1st Edition Sheng Xu download
An Introduction to Scientific Computing with MATLAB and Python Tutorials 1st Edition Sheng Xu download
https://ebookmeta.com/product/an-introduction-to-scientific-
computing-with-matlab-and-python-tutorials-1st-edition-sheng-xu/
https://ebookmeta.com/product/introduction-to-engineering-and-
scientific-computing-with-python-1st-edition-david-e-clough/
https://ebookmeta.com/product/python-for-astronomers-an-
introduction-to-scientific-computing-3rd-edition-imad-pasha/
https://ebookmeta.com/product/linux-mint-20-self-help-guide-2nd-
edition-unofficial-and-compatible-with-all-linux-
mint-20-versions-partridge/
Archaeological Theory in Practice 2nd Edition Patricia
A Urban Edward Schortman
https://ebookmeta.com/product/archaeological-theory-in-
practice-2nd-edition-patricia-a-urban-edward-schortman/
https://ebookmeta.com/product/everyday-data-cultures-1st-edition-
jean-burgess/
https://ebookmeta.com/product/zeb-police-officer-virgin-curvy-
woman-suspense-rescued-diamond-ridge-mountain-men-book-1-1st-
edition-brynn-hale-hale/
https://ebookmeta.com/product/the-trouble-with-her-forbidden-
love-4-1st-edition-kat-t-masen/
https://ebookmeta.com/product/inside-afghanistan-routledge-
contemporary-south-asia-series-1st-edition-timor-sharan/
The Bathysphere Book 1st Edition Brad Fox
https://ebookmeta.com/product/the-bathysphere-book-1st-edition-
brad-fox/
An Introduction to
Scientific Computing
with MATLAB® and
Python Tutorials
An Introduction to
Scientific Computing
with MATLAB® and
Python Tutorials
Sheng Xu
MATLAB ® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks
does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion
of MATLAB ® software or related products does not constitute endorsement or sponsorship by The
MathWorks of a particular pedagogical approach or particular use of the MATLAB ® software.
© 2022 Sheng Xu
Reasonable efforts have been made to publish reliable data and information, but the author and pub-
lisher cannot assume responsibility for the validity of all materials or the consequences of their use.
The authors and publishers have attempted to trace the copyright holders of all material reproduced
in this publication and apologize to copyright holders if permission to publish in this form has not
been obtained. If any copyright material has not been acknowledged please write and let us know so
we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.
com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA
01923, 978-750-8400. For works that are not available on CCC please contact mpkbookspermis-
[email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are
used only for identification and explanation without intent to infringe.
DOI: 10.1201/9781003201694
Publisher’s note: This book has been prepared from camera-ready copy provided by the authors
To my family and friends.
Contents
Preface xiii
Author xv
2 Taylor’s Theorem 9
2.1 Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.1 Polynomial Evaluation . . . . . . . . . . . . . . . . . . . . 10
2.2 Taylor’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.1 Taylor Polynomials . . . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Taylor Series . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.3 Taylor’s Theorem . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Alternating Series Theorem . . . . . . . . . . . . . . . . . . . . . 19
2.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5 Programming Problems . . . . . . . . . . . . . . . . . . . . . . . 22
vii
viii Contents
6 Interpolation 131
6.1 Terminology of Interpolation . . . . . . . . . . . . . . . . . . . . . 132
6.2 Polynomial Space . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.2.1 Chebyshev Basis . . . . . . . . . . . . . . . . . . . . . . . 135
6.2.2 Legendre Basis . . . . . . . . . . . . . . . . . . . . . . . . 138
6.3 Monomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . 141
6.4 Lagrange Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 143
6.5 Newton’s Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 145
6.6 Interpolation Error . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Contents ix
Appendices 305
Index 379
Preface
Sample Syllabi
Below is a sample syllabus for a two-semester course sequence.
Semester 1
1. Appendix A or B: MATLAB or Python Tutorial
xiii
xiv Preface
Semester 2
1. Chapter 4: Direct Methods for Linear Systems
2. Sessions 5.6–5.7: Nonlinear Systems and Optimization
3. Sessions 6.6–6.7: Spline Interpolation and DFT
4. Sessions 7.7–7.8: Gaussian Quadrature Rules
5. Chapter 9: IVPs and BVPs
6. Chapter 10: Iterative Methods for Linear Systems
7. Chapter 11: LS Problems
8. Chapter 12: Monte Carlo Methods and Parallel Computing
Acknowledgments
We acknowledge our colleagues Daniel Reynolds and Johannes Tausch for detailed
critiques of this first edition. We would appreciate any comments, suggestions and
corrections that readers may wish to send us using the email address [email protected].
Author
xv
1
An Overview of Scientific Computing
In this chapter, we address what scientific computing is about and why it is important.
We give a brief introduction to algorithms and errors. We emphasize the importance
of error upper bounds.
Example 1.2 (Particle chasing) Two particles A and B move from the same location
along a straight path in the same direction. The particle A moves with the constant
speed 2, and the particle B starts with a zero speed and accelerates with a time-
dependent speed et − 1, where t is the time. How long does the particle B take to catch
the particle A?
[Solution:] When the particle B catches up with the particle A at t = T > 0, they
travel the same distance. TheRdistance traveled by the particle A is 2T , and the distance
traveled by the particle B is 0T (et − 1)dt. The time T therefore satisfies
Z T
2T = (et − 1)dt, T >0
0
DOI: 10.1201/9781003201694-1 1
2 An Overview of Scientific Computing
which gives
3T + 1 = eT , T >0
This equation cannot be solved analytically using elementary functions, but we
clearly know it has a positive real solution from the physics intuition or the plots of
f (T ) = 3T + 1 and g(T ) = eT in Fig. 1.1.
20
y=3t+1
18 y=e
t
16
14
12
10
y
0
0 0.5 1 1.5 2 2.5 3
t
FIGURE 1.1
Graphs of f (T ) = 3T + 1 and g(T ) = eT for T ∈ [0, 3].
the interval [1, 3] by the middle point 2 and look at the two subintervals [1, 2] and
[2, 3]. We can use the same strategy to narrow down the solution to one subinterval,
in this case [1, 2]. We can repeatedly apply this strategy until we find a subinterval
that contains the solution and is small enough such that its middle point is a good
approximation of the solution. Actually, this procedure is called bisection iteration,
which will be learned in more details later in Section 5.2 of Chapter 5.
Scientific computing, theoretical study and experiments are three pillars to support
scientific research and technological applications. The diagram in Fig. 1.2 shows
where scientific computing comes into play.
Experiments Computers
modeling scientific
Real−world Problems Mathematical Models Numerical Solutions
computing
FIGURE 1.2
Scientific computing in solving real-world problems.
π = 3.14159 · · ·
The approximation A is
A = 3.14
The absolute error |E| in the approximation is
• Modeling errors: the errors due to the simplifications and hypotheses in the
modeling process that convert a real-world problem into a mathematical model.
For example, the error due to the neglect of air resistance in the apple’s free fall
problem above. Modeling errors are not in the scope of scientific computing.
• Mathematical approximation errors: the errors due to the approximation of an
actual quantity by an approximate formula.
Example 1.5 We know that f 0 (a), the derivative of f (x) at x = a, is the slope of
f (a + h) − f (a)
the tangent line to the curve y = f (x) at x = a; and , where h is
h
finite, is the slope of the secant line through the points (a, f (a)) and (a + h, f (a +
h)) on the curve y = f (x). If we approximate the tangent slope T = f 0 (a) by
f (a + h) − f (a)
the secant slope A = , we have the mathematical approximation
h
error:
f (a + h) − f (a)
|T − A| = f 0 (a) −
h
Later, we will analyze how this error depends on the value of h using Taylor’s
theorem and introduce the big-O notation to describe this dependence.
• Roundoff errors: the errors due to finite-precision representation of real numbers.
Example 1.6 Let x̂ = 1.2345679 be a 8-digit representation of x = 1.23456789.
The absolute error |x − x̂| = 0.00000001 = 10−8 is caused by rounding x to x̂ and
is the roundoff error in the representation. Later, we will describe how computers
represent real numbers and learn that roundoff errors are inevitable in computer
representations.
1.4 Exercises
Exercise 1.1 A1 is an approximation of T1 = 1 with the absolute error 1, and A2 is
an approximation of T2 = 100 with the absolute error 1. Which approximation (A1 or
A2 ) has better accuracy? Why?
Exercise 1.2 (1) Suppose the true value T = 2, and the relative error of its approx-
imation A is less than 5%. What is the range for A? (2) Suppose an approximation
A = 99, and the relative error in A is 10%. What are the possible values of the true
value T ?
1 n
Exercise 1.4 Euler’s number e can be defined as e = lim 1 + . It is a famous
n→∞ n
transcendental number with infinitely many digits e = 2.718282828459.... Suppose
1 2
one approximates e by A = 1 + with n = 2. (1) Find a meaningful upper bound
2
for the absolute error of the approximation. (2) Find a meaningful upper bound for
the relative error of the approximation.
Exercise 1.6 Let Ah be an approximation of the true value T . Suppose the approxima-
tion Ah depends on a small parameter h, and the absolute error |T − Ah | depends on h
as |T − Ah | = Ch p , where C > 0 and p > 0 are constants. (1) To make the error small,
do we want a small or large value of p? (2) If we double the value of h, how many
times larger does the absolute error become for p = 1, p = 2 and p = 3, respectively?
Exercise 1.7 Count the total number of multiplications in the nested loops of the
following pseudocodes.
(1)
for m = 1 to 20 do
for n = 1 to 20 do
bm = bm − Gm,n bn
end for
end for
(2)
for m = 1 to 20 do
for n = m to 20 do
bm = bm − Gm,n bn
Exercises 7
end for
end for
Exercise 1.8 If you are asked to evaluate p4 (x) = 1 + 2x + 3x2 + 4x3 + 5x4 at a given
value of x, how many multiplications do you use? (Note that evaluating xn as x ·x · · · · ·x
needs n − 1 multiplications.) Can you use less?
2
Taylor’s Theorem
In this chapter, we presents Taylor’s theorem, which is used later for development
and analysis of some numerical methods. We show how to approximate functions by
Taylor polynomials and how to analyze the errors in such approximations. The big-O
notation is introduced to describe efficiency and accuracy.
2.1 Polynomials
A polynomial pn (x) in the variable x of degree n, where n is a non-negative integer,
can be written in the general form as
Note that pn (x) is built from constant coefficients and the variable x using only
addition, multiplication and exponentiation of x to non-negative integer powers (re-
peated multiplication). So polynomials can be easily evaluated, differentiated and
integrated, and are good candidates to approximate functions or data.
The polynomial pn (x) can be regarded as a linear combination of the monomial
basis functions 1, x, x2 , ..., xn−1 and xn using the constant weights c0 ,c1 , ..., cn−1 and
cn , respectively. Later we will construct a polynomial of degree at most n as a linear
combination of other basis functions.
A polynomial centered at the number a has the form
DOI: 10.1201/9781003201694-2 9
10 Taylor’s Theorem
Remark The roots can be repeated and can be complex numbers. The theo-
rem can be proved using complex analysis.
where only 4 multiplications are required. The number of additions is still 4. The
evaluation procedure using nested multiplication is known as Horner’s method.
The above analysis can be easily extended to a polynomial of degree n, pn (x) =
∑nk=0 ck xk , for which the naive method needs
n
n(n + 1)
∑ k = 0+1+2+3+···+n = 2
(2.5)
k=0
w = O( f (n)) (2.6)
which means that the growth of the computational time w is bounded from above for
large enough n as
w ≤ C f (n) (2.7)
where C is a positive constant. For example, the computational time of the naive
method for polynomial evaluation is O(n2 ), while Horner’s method is O(n), where n
is the degree of a polynomial.
The polynomial pn (x) centered at a in the form of nested multiplication is
which can be evaluated by Horner’s method by starting with the innermost parentheses
and working outward. The pseudocode of Horner’s method is
The Taylor polynomial Tn (x) is constructed using the derivatives of f (x) at a such
that Tn (x) and f (x) satisfy the following matching conditions at a
Example 2.1 (Taylor polynomials) Find the Taylor polynomial of degree 2 for
f (x) = 1 + 2x + 3x2 about a = 1.
[Solution:] In this problem, f (x) = 1 + 2x + 3x2 and a = 1. The Taylor polynomial is
f 00 (1)
T2 (x) = f (1) + f 0 (1)(x − 1) + (x − 1)2
2!
We have
Taylor’s Theorem 13
f 00 (a) ∞
f (k) (a)
f (x) ∼ f (a) + f 0 (a)(x − a) + (x − a)2 + · · · ≡ ∑ (x − a)k (2.14)
2! k=0 k!
Note that
(1) the Taylor series (on the right) can be regarded as limn→∞ Tn (x), and a Taylor
polynomial is a truncated Taylor series;
(2) the symbol ∼ is used to indicate that the Taylor series may not equal f (x); If the
Taylor series converges to f (x) for any x in an interval, we can replace ∼ by =
with the specification of the convergence interval;
(3) the Taylor series is also called the Maclaurin series if a = 0.
Below are some familiar convergent Maclaurin series with their convergence
intervals.
x2 x3 ∞ k
x
ex = 1 + x + + +··· ≡ ∑ , |x| < ∞ (2.15)
2! 3! k=0 k!
x3 x5 x7 ∞
(−1)k x2k+1
sin x = x − + − +··· ≡ ∑ , |x| < ∞ (2.16)
3! 5! 7! k=0 (2k + 1)!
x2 x4 x6 ∞
(−1)k x2k
cos x = 1 − + − +··· ≡ ∑ , |x| < ∞ (2.17)
2! 4! 6! k=0 (2k)!
∞
1
= 1 + x + x2 + x3 + · · · ≡ ∑ xk , |x| < 1 (2.18)
1−x k=0
x3 x5 x7 ∞
(−1)k x2k+1
tan−1 x = x − + − +··· ≡ ∑ , |x| ≤ 1 (2.19)
3 5 7 k=0 2k + 1
x2 x3 x4 ∞
(−1)k−1 xk
ln(1 + x) = x − + − +··· ≡ ∑ , −1 < x ≤ 1, (2.20)
2 3 4 k=1 k
In the last binomial series, the power p is real, and the binomial notation is defined by
p p(p − 1) · · · (p − k + 1)
:= (2.22)
k k!
We may substitute x by new variables in the above series or integrate or differenti-
ate the above series to obtain new series.
Example 2.2 Substitute x by −t 2 in the Maclaurin series for ex , we get
2 t4 t6 ∞
(−1)k t 2k
e−t = 1 − t 2 + − +··· ≡ ∑ , |t| < ∞ (2.23)
2! 3! k=0 k!
Example 2.3 Substitute 1 + x by t in the Maclaurin series for ln(1 + x), we get the
Taylor series of lnt about a = 1 as
(t − 1)2 (t − 1)3 (t − 1)4
lnt = (t − 1) − + − +···
2 3 4
∞
(−1)k−1 (t − 1)k
≡ ∑ , |t − 1| < 1 (0 < t < 2) (2.24)
k=1 k
-1
y
-2
-3
-4
-3 -2 -1 0 1 2 3
x
FIGURE 2.1
Approximations of y = cos x, x ∈ [−π, π] by its Taylor polynomials of different
degrees.
n+1
Theorem 2.2 (Taylor’s Theorem) Assume f (x) ∈ C[α,β ]
, and let a be a number
in (α, β ). Then
where
n
f (k) (a)
Tn (x) = ∑ (x − a)k (2.26)
k=0 k!
is the n-th order Taylor polynomial of f at a, and Rn (x) is called Taylor’s
remainder (or the error term in approximating f (x) by Tn (x)). Taylor’s remainder
Rn (x) in Lagrange mean value form is
f (n+1) (c)
Rn (x) = (x − a)n+1 (2.27)
(n + 1)!
which is a function of t that satisfies g(a) = g0 (a) = · · · = g(n) (a) = 0 and g(x) =
0.
Remark Taylor’s remainder Rn (x) also has an integral form as
Z x n+1
f (t)
Rn (x) = (t − x)n dt (2.28)
a n!
which can be revealed using integration by parts as follows
Z x Z x
f (x) = f (a) + f 0 (t)dt = f (a) + f 0 (t)(t − x)0 dt
a a
Z x
= f (a) + f 0 (a)(x − a) − f 00 (t)(t − x)dt
a
0
(t − x)2
Z x
0 00
= f (a) + f (a)(x − a) − f (t) dt
a 2
(x − a)2 (t − x)2
Z x
= f (a) + f 0 (a)(x − a) + f 00 (a) + dt f 000 (t)
a 2 2
0
(x − a)2 (t − x)3
Z x
0 00 000
= f (a) + f (a)(x − a) + f (a) + f (t) dt
2 a 3!
(x − a)2 (x − a)3 (t − x)3
Z x
0 00
= f (a) + f (a)(x − a) + f (a) + f 00 (a) + f (4) (t) dt
2 3! a 3!
Z x n+1
f (t)
= · · · = Tn (x) + (t − x)n dt
a n!
f (x) − f (a)
f (x) = f (a) + f 0 (c)(x − a) or = f 0 (c) (2.29)
x−a
Taylor’s Theorem 17
which is the mean value theorem (MVT). The MVT has a clear geometry
interpretation: existence of a tangent line parallel to the secant line through the
two ending points.
• Let h = x − a, then x = a + h and we can write Taylor’s theorem in the form in
terms of h as
n
f (k) (a) k f (n+1) (c) n+1
f (a + h) = ∑ h + h (2.30)
k=0 k! (n + 1)!
where c is an unknown point between a and a + h.
Similarly, if we let h = a − x, then x = a − h (and x − a = −h) and we can write
Taylor’s theorem in the form in terms of h as
n
f (k) (a) f (n+1) (c)
f (a − h) = ∑ (−h)k + (−h)n+1 (2.31)
k=0 k! (n + 1)!
f (n+1) (c) ec
| f (x) − Tn (x)| = (x − 0)n+1 = |x|n+1
(n + 1)! (n + 1)!
α a x
β
FIGURE 2.2
Relative positions of a, x and c if a ∈ [α, β ] and x ∈ [α, β ].
ec e2
| f (x) − Tn (x)| = |x|n+1 < 2n+1
(n + 1)! (n + 1)!
18 Taylor’s Theorem
or
∞
−b0 + b1 − b2 + b3 − · · · ≡ ∑ (−1)k+1 bk (2.36)
k=0
and
lim bk = 0 (2.38)
k→∞
Then
|S − Sn | ≤ bn+1 (2.41)
20 Taylor’s Theorem
The Maclaurin series mentioned above satisfy the condition of the alternating
series theorem for x in the specified intervals and are therefore convergent (as we have
already known from Taylor’s theorem).
The alternating series theorem provides an upper bound bn+1 for the error in
approximating the sum S by the partial sum Sn of a convergent alternating series that
satisfies the conditions in the theorem.
which converges to the sum S(x) = cos x. Suppose we stop the partial sum Sn (x) right
at the term (−1)n x2n /(2n)! (included). (Note that Sn (x) is the Taylor polynomial of
degree 2n for f (x) = cos x about a = 0.) By the alternating series theorem, we have
|x|2(n+1)
|S(x) − Sn (x)| = | cos x − Sn (x)| ≤
(2(n + 1))!
So the absolute error in approximating cos 1 by the partial sum Sn (1) is bounded as
1
| cos 1 − Sn (1)| ≤
(2n + 2)!
Let 1/(2n + 2)! < 10−6 , we obtain n ≥ 4. Note that S4 (x) and S4 (1) are
x2 x4 x6 x8 1 1 1 1
S4 (x) = 1 − + − + , S4 (1) = 1 − + − +
2! 4! 6! 8! 2! 4! 6! 8!
2.4 Exercises
Exercise 2.1 Write the polynomial p4 (x) = 1 + 5x + 4x2 + 3x3 + 2x4 in the form
of nested multiplication. Then evaluate p4 (2) using the nested form (i.e. Horner’s
method). How many multiplications and additions are used in your evaluation?
Exercise 2.2 How to evaluate p(x) = 2 + 3x3 − 4x6 + 8x9 − 11x12 at a given value of
x using only 6 multiplications?
Exercise 2.3 Count how many multiplications (in terms of n) are used in the following
pseudocode (at the beginning of next page). Write down the number of multiplications
in the big O notation. Hint: ∑nk=1 k2 = n(n + 1)(2n + 1)/6.
Exercises 21
for k from 1 to n − 1 do
for i from k + 1 to n do
for j from k + 1 to n do
Ai j ← Ai j − Ak j Aik /Akk
end for
end for
end for
Exercise 2.4 Find the Taylor polynomial T3 (x) of degree 3 for f (x) = 1 + 4x + 3x2 +
2x3 about a = 1. Show that T3 (x) = f (x) in this case by simplifying T3 (x) or by
applying Taylor’s theorem.
√3
Exercise 2.5 Find the Taylor polynomial T1 (x) of degree 1 for f (x) = x5 about
a = 0. Sketch the graphs of T1 (x) and f (x) in the same plot. Can you find the Taylor
polynomial T2 (x) of degree 2 for f (x) about a = 0?
Exercise 2.6 Write down the Taylor polynomial T4 (x) of degree 4 for f (x) about a.
(k)
Verify that T4 (a) = f (k) (a) for k = 0, 1, 2, 3, 4. Note that we define the 0-th derivative
of a function as the function itself.
Exercise 2.7 Let Tn (x) be the Taylor polynomial of degree n for the polynomial
pn (x) = c0 + c1 x + c2 x2 + · · · + cn xn about a. Prove that Tn (x) = pn (x).
Exercise 2.8 Derive the Maclaurin series for f (x) = ln(1 − x) by two ways: (1)
finding f (k) (0)/k!, k = 0, 1, 2, 3, . . .; (2) using a substitution in an existing Maclaurin
series. State the range of x on which the series converges to f (x) = ln(1 − x).
Exercise 2.9 Let θ = (1/55 · · · 5)◦ be a small angle in degrees, where the denominator
has n fives. (1) Compute sin θ for n = 3, 5, 7, 10 and write down the results in scientific
notation. Do you see something unusual? (2) Why sin θ has more and more same
decimal digits in the same order as π when n increases? (Hint: What is the truncated
degree-1 Maclaurin series of sin x? Is x in the series in degrees or radians?)
Exercise 2.10 Let Tn (x) be the n-th order Taylor polynomial for f (x) = ex about
a = 0. (1) Find the expression for Tn (x). (2) If T9 (x) is used to approximate f (x) = ex
for −2 ≤ x ≤ 1, find an upper bound of the error. (3) If Tn (x) is used to approximate
f (x) = ex for −1 ≤ x ≤ 1 with the absolute error at most 10−3 , how large should n
be?
Exercise 2.11 Let f (x) = sin x, a = π. (1) Derive the degree-3 Taylor polynomial
T3 (x) for f (x) at a. (2) If f (x) is approximated by T3 (x) for π − 1 ≤ x ≤ π + 1, find
an upper bound for the error of this approximation using Taylor’s Theorem.
Exercise 2.12 Derive the degree-2 Taylor polynomial T2 (x) for f (x) = ln x at a = 1. If
f (x) is approximated by T2 (x) for 0.5 ≤ x ≤ 1.5, find an upper bound for the absolute
error of this approximation using Taylor’s remainder.
22 Taylor’s Theorem
Exercise 2.13 Prove Taylor’s theorem. (Hint: See remarks for Theorem 2.2.)
Exercise 2.14 Show 2n+1 /(n + 1)! is decreasing with n for n = 1, 2, 3, . . . using proof
by induction.
Exercise 2.15 Use the alternating series theorem to determine the value of n in the
Taylor polynomial
n
(−1)k x2k
T2n (x) = ∑
k=0 (2k)!
for cos x about 0 such that cos 1 is approximated by T2n (1) with the absolute error less
than 10−6 .
Exercise 2.16 Approximate T = f (a) by A = f (a + h), where h is small, and f (x)
has continuous first derivative in an interval containing a and a + h. (1) Use Taylor’s
theorem to find an expression of the absolute error |T − A| in terms of h. (2) Write the
absolute error in big-O notation. (3) What is the limit of the error as h → 0?
Exercise 2.17 If f (a) is approximated by
f (a + h) + f (a − h)
f (a) ≈
2
where h is a small value, and f (x) has continuous second derivative everywhere, (1)
find the absolute error of the approximation in terms of h using Taylor’s theorem
f = Tn + Rn with n = 1; (2) write the absolute error in big-O notation.
Exercise 2.18 f (x) ∈ C2 near a. Approximate f 0 (a) as
f (a) − f (a − h)
f 0 (a) ≈
h
where h is a small positive value. Use Taylor’s theorem to show that the error of the
approximation is O(h).
Problem 2.2 Let Tn (x) be the Taylor polynomial of degree n for f (x) = ln(1 + x) at
a = 0. Write a MATLAB m-script to use your function nest in the previous problem
to evaluate T4 (x) and T9 (x) at
(1) Plot f (x), T4 (x) and T9 (x) in one figure using the MATLAB command plot.
(2) Plot the Taylor’s remainders (the error terms) | f (x) − T4 (x)| and | f (x) − T9 (x)|
in another figure using the MATLAB command semilogy (use help semilogy to
learn what semilogy does and how to use it).
(3) Derive an upper bound of | ln(1 + x) − T9 (x)| for −0.5 ≤ x ≤ 0.5 using Taylor’s
theorem. Is the absolute error | ln(1 + x) − T9 (x)| in your second plot less than your
derived upper bound?
3
Roundoff Errors and Error Propagation
In this chapter, we explains why roundoff errors are inevitable in scientific computing
and how roundoff errors are propagated in arithmetic and algorithms. We use exam-
ples to illustrate and analyze the stability of algorithms.
3.1 Numbers
We have seen the following different types of numbers.
• natural numbers: N = {1, 2, 3, . . .}
• integers: Z = {. . . , −3, −2, −1, 0, 1, 2, 3, . . .}
• rational numbers: Q = {p/q|, p ∈ Z, q ∈ Z, q 6= 0}
DOI: 10.1201/9781003201694-3 25
26 Roundoff Errors and Error Propagation
• real numbers: R = set of all the numbers on the real number line, including
√ rational
numbers (such as 0.1 and 1/3) and irrational numbers
√ (such as 2, π and e).
• complex numbers: C = {a + bi|a ∈ R, b ∈ R, i = −1}
The above sets of numbers expand from top to bottom: N ⊂ Z ⊂ Q ⊂ R ⊂ C.
A computer stores a number in a physical unit such as registers, RAMs, or disk
drives which may be regarded as an ordered list of switches. Each switch has 2
statuses: on and off. We assign the on and off statuses the values 1 and 0, respectively,
and call such a switch a bit. So a computer represents a number as a pattern of ordered
bits (for example 01101010). The mapping between a bit pattern and a number is
determined by the IEEE (Institute of Electrical and Electronics Engineers, pronounced
as I-triple-E) standards. Below we describe how a computer represents integers and
real numbers.
3.1.1 Integers
Signed integers are commonly used in computations. An 8-bit (1-byte) signed integer
is stored as the bit pattern
b7 b6 b5 b4 b3 b2 b1 b0
which corresponds to the value
(−b7 )×27 +b6 ×26 +b5 ×25 +b4 ×24 +b3 ×23 +b2 ×22 +b1 ×21 +b0 ×20 (3.1)
With 8 bits, only 28 = 128 signed integers −128, −127, . . . , 126, 127 can be repre-
sented.
We also use unsigned integers in indexing (for example, row and column indices
of a matrix) and in the representing real numbers (soon later). An 8-bit unsigned
integer is stored as the bit pattern
b7 b6 b5 b4 b3 b2 b1 b0
b7 × 27 + b6 × 26 + b5 × 25 + b4 × 24 + b3 × 23 + b2 × 22 + b1 × 21 + b0 × 20 (3.2)
With 8 bits, only 28 = 128 unsigned integers 0, 1, . . . , 254, 255 can be represented.
T = σ · T̄ · 10e (3.3)
where
• σ = +1/ − 1: sign of T
• 1 ≤ T̄ < 10: significand/mantissa of T
• e: integer exponent
T̄ = d1 .d2 d3 · · · (3.4)
A = σ · Ā · 10E (3.5)
but the number of digits in the mantissa is limited. If the mantissa Ā of a floating-point
representation allows only t decimal digits, then it can be written as
then its floating-point representation A can be obtained by rounding. The rule for
rounding is “round to nearest, ties to even” by comparing the digit dt+1 (the first digit
to be discarded) with 5 (half of the base value 10) as follows.
• “Round to nearest”: If dt+1 < 5, round T̄ down to Ā by simply chopping off
all the less significant digits to the right of dt . If dt+1 > 5, round T̄ up to Ā by
discarding all the digits to the right of dt and adding 1 to the digit dt , which
28 Roundoff Errors and Error Propagation
may lead to carrying (or even the adjustment of the exponent e to E = e + 1). If
dt+1 = 5 but is not the rightmost nonzero digit (i.e. there are other nonzero digits
after it), round T̄ up.
• “Ties to even”: If dt+1 = 5 and is the rightmost nonzero digit, then when dt is
even, round T̄ down; and when dt is odd, round T̄ up. In either case, the digit d̃t
is even.
+3.1416 × 100 , −1.2463 × 102 , −4.3652 × 101 , 2.375 × 10−3 , −2.385 × 10−1
respectively. If the mantissa of a floating-point representation (in base 10) can have
only 3 significant digits, then the floating-point representations of these numbers are
obtained by rounding as
+3.14 × 100 , −1.25 × 102 , −4.37 × 101 , 2.38 × 10−3 , −2.38 × 10−1
respectively.
Page 375, ligne 16, Mais Mme de Marelle et Mme Walter s’étant
saluées, Clotilde prononça:
—Si nous le gardions près de nous madame, il nous nommera les
tireurs et les gens connus. Il peut bien demeurer debout au coin de
ce banc.
Page 380, ligne 12, Mme de Marelle, car Mme Walter les
connaissait presque tous. C’étaient...
BEL-AMI .
«C’est une œuvre très forte, très puissante, mais d’une vérité
cruelle et légèrement répulsive, le Bel-Ami de M. de Maupassant...
Ce misérable réussit avec une chance si constante et il accepte le
succès comme chose due avec une si imperturbable sérénité que
cela en devient exaspérant... Et cependant, une fois ce livre bleu
entre les mains, je ne l’ai pas lâché, mais j’ai lu tout d’une haleine,
non pas le dévorant, mais le savourant. Que voulez-vous? Cela est à
la fois irritant et exquis.»
Nouvelle Revue, juin 1885 (Francisque Sarcey).
Updated editions will replace the previous one—the old editions will
be renamed.
1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside the
United States, check the laws of your country in addition to the
terms of this agreement before downloading, copying, displaying,
performing, distributing or creating derivative works based on this
work or any other Project Gutenberg™ work. The Foundation makes
no representations concerning the copyright status of any work in
any country other than the United States.
1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if you
provide access to or distribute copies of a Project Gutenberg™ work
in a format other than “Plain Vanilla ASCII” or other format used in
the official version posted on the official Project Gutenberg™ website
(www.gutenberg.org), you must, at no additional cost, fee or
expense to the user, provide a copy, a means of exporting a copy, or
a means of obtaining a copy upon request, of the work in its original
“Plain Vanilla ASCII” or other form. Any alternate format must
include the full Project Gutenberg™ License as specified in
paragraph 1.E.1.
• You pay a royalty fee of 20% of the gross profits you derive
from the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
about donations to the Project Gutenberg Literary Archive
Foundation.”
• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.
1.F.
1.F.4. Except for the limited right of replacement or refund set forth
in paragraph 1.F.3, this work is provided to you ‘AS-IS’, WITH NO
OTHER WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.