Numerical Analysis FIF EH EDITION Richard L. Burden Youngstown State University J. Douglas Faires Youngstown Staie University & PWS PUBLISHING COMPANY BOSTONContents Mathematical Preliminaries 1 Ll Review of Calculus 2 1.2 Round-Off Errors and Computer Arithmetic 12 13. Algorithms and Convergence 24 14 Numerical Software 32 nN Solutions of Equations in One Variable 39 2.1 The Bisection Method 40 2.2 Fixed-PointIteration 46 2.3 The Newton-Raphson Method 56 2.4 Error Analysis for Iterative Methods 70 2.5 Accelerating Convergence 78 2.6 Zeros of Polynomials and Milller’s Method 82 2.7 Survey of Methods and Software 93 Qo Interpolation and Polynomial Approximation 95 3.1 Interpolation and the Lagrange Polynomial 98 3.2 Divided Differences 112 3.3 Hermite Interpolation 123 3.4 Cubic Spline Interpolation 130 35 Parametric Curves 148 3.6 Survey of Methods and Software 154 Numerical Differentiation and Integration 156 4.1 Numerical Differentiation. 157 4.2 Richardson’s Extrapolation 168 43 Elements of Numerical Integration 174Contents 4.4 Composite Numerical Integration 184 45 Adaptive Quadrature Methods 192 46 Romberg Integration 199 4.7 Gaussian Quadrature 205 4.8 Multiple Integrals 2/1 49 Improper Integrals 224 4.10 Survey of Methods and Software 230 5 Initial-Value Problems for Ordinary Differential Equations 232 5.1 Elementary Theory of Initial-Value Problems 234 5.2 Enler’s Method 239 53 Higher-Order Taylor Metbods 248 5.4 Runge-Kutta Methods 254 55 Error Control and the Runge-Kutta-Feblberg Method 263 5.6 MultistepMethods 270 5.7 Variable Step-Size Multistep Methods 282 5.8 Extrapolation Methods 288 5.9. Higher-Order Equations and Systems of Differential Equations | 294 5.10 Stability 30 5.11 Stiff Differential Equations 314 5.12 Survey of Methods and Software 321 6 Direct Methods for Solving Linear Systems 323 6.1 Linear Systems of Equations 324 6.2 Pivoting Strategies 338 63. Linear Algebra and Matrix Inversion 345 64 The Determinant of ¢ Matrix 358 6.5 Matrix Factorization 362 66 Special Types of Mattices 371 6.7 Survey of Methods and Software 380 7 Nerative Techniques in Matrix Algebra 389 7.1 Norms of Vectors and Matrices 390 7.2. Eigenvalues and Eigenvectors 401 7.3. Iterative Techniques for Solving Linear Systems 406Contents 7.4 Error Estimates and lerative Refinement 424 7.5 Survey of Methods ond Software 434 S Approximation Theory 435 8.1 Discrete Least-Syuares Approximation 436 8.2 Orthogonal Polynomials and Least-Squares Approximation 449 8.3. Chebyshev Polynomials and Economization of Power Series 459 84° Rational Function Approximation 469 8.5 Trigonometrie Polynomial Approximation 180 8.6 Fast Fourier Transforms 485 8.7. Survey of Methods and Software 496 9 Approximating Eigenvalues 497 9.1 Linear Algebra and Eigenvalues 498 9.2 The Power Method 506 9.3. Housebelder’s Method 523 94 The QR Algorithm 530 9.5 Survey of Methods and Software 54J 10 Numerical Sotutions of Nonlinear Systems of Equations 543 10.1 Fixed Points for Functions of Several Variables 544 10.2 Newton's Method 553 10.3 Quasi-Newton Methods 560 10.4 Steepest Descent Techniques 568 10.5 Survey of Methods and Software 575 ]] Boundary-Value Problems for Ordinary Differential Equations 577 11.1 The Linear Shooting Method 578 11.2 The Shooting Method for Nonlinear Problems 585 11.3 Finite-Difference Methods for Linear Problems 592 114 Finite-Difference Methods for Nonlinear Problems 598 115. Rayleigh-Ritz Method 605, 11.6 Survey of Methods and Software 620[2 Numericai Solutions to Partial-Differential Equations 622 12.1 Elliptic Partial-Differential Equations 625 12,2 Parabolic Partial-Differential Equations 635, 12.3. Hyperbolic Partial-Differential Equations 649 12.4 An Introductionto the Finite-Element Method 657 12.5 Survey of Methods and Software 672 Bibliography 675 Answers to Selected Exercises 685 Index 761Preface augue eee About the Text sso Ste a Raa RE ‘We have developed the material in this text for a Sequence of courses in the theory and application of numerical approximation techniques. The text 1 designed primarily tor junior-leve! mathematics, science, and engineering majors who have completed at least the first year of the standard college calculus sequence and have some knowledge of a high- evel programming language. Familiarity with the fundamentals of matrix algebra and differential equations is also iseful, ht adequate introductory material on these topics is presented in the text, Previous editions of the book have been used in @ wider variety of situations than we originally intended. In some cases, the mathematical analysis underlying the development of approximation techniques is emphasized, rather than the methods themselves; in ‘ters, the emphasis is reversed. The DOoK has also been used as the core reference for ‘courses at the beginning graduate level in engineering and computer science programs, as the basis for the actuarial examination in numerical analysis, where self-study is common, and in first-year courses in introductory analysis offered at international universities. We have tried to adapt the book to fit these diverse requirements without compromising our original purpose: to give an introduction to modern approximation techniques; to expla how, why, and when they can be expected to work; and to provide a firm basis for future study in numerical analysis. ‘The book contains sufficient material for @ full year of study, but we expect many Feaders 10 use the text for only a single-term course. In such a course, students learn to identify the type of problems that require numerical techniques for their solution, see examples of error propagation that can occur when numerical methods are applied, and accurately approximate the solutions of some problems that cannot be solved exactly. ‘The remainder of the text serves a5 a reference for methods that are not discussed in the course, Bither the full-year or single-course treatment is consistent with the purpose of the text. ‘Virtually every concept in the text is illustrated by example, and this edition contains more than 2000 class-tested exercises. These exercises range from elementary applications of metods and algorithms to generalizations and extensions of theory. In addition, the exercise sets include a large number of applied problems from diverse areas of engineeringPreface 485 well as from the physical, computer, biological, and social sciences, The applications chosen concisely demonstrate how numerical methods can be (and are) applied in “real- 1ife” situations, eee ia a New for This Edition «In the introductory chapter, we added a section that discusses the various software packages commonly used in professional environments for numerical approximation We include sources for obtaining the software and provide information for relevant reference material, with particular emphasis on public domain software. This section also deserines how professional software packages differ from software available from other sources. * At the end of Chapters 2 through 12 we.edlded a section entitled “Survey of Methods and Software.” These sections summarize the methods developed in the chapter and recommend strategies for choosing techniques to use in various situations. These sec- tions also reterence appropriate programs in the International Mathematical and Statis- tics (IMSL) and Numerical Algorithms Group (NAG) libraries and refer to other professional sources when they ate pertinent to the material in the chapter. © New algorithms are included for the Method of False Position, Bézier curve generation, Gaussian quadrature for double and triple integrals, Padé approximation, and Chebyshev rational fonction approximation. The editotial commentsin all the algorithms have been Tewritten, when appropriate, to more closely correspond to the discussion in the text. * The Method of False Position (Regula Falsi) has been included in Chapter 2 since this type of bracketing technique is commonly used in professional software packages. ‘+ The presentation of Lagrange interpolation in Chapter 3 has been streamlined for better continuity. We also reduced the discussion of Taylor polynomials in this chapter 10 make it clerer thes Taylor polynomials are not used for interpolation. * Chapter 3 now concludes with a section on parametric curves. In this section we de- scribe how interactive computer graphic systems permit curves to be drawn and quickly modified. Many of our students are familiar with computer graphic software that permits freehand curves to be quickly drawn and modified. This section describes how culbic Hermite spline. functions and Bézier polynomials make this possible. Although the section is intended to be informational rather than computational, we have included an algorithm for generating Bézier curves, * Chapter 5 contains a new series of examples that better illustrate the methods used for solving initial-value problems. We use a single initial-value problem to illustrate all the standard techniques of approximation, which permits the methods to be compared more effectively. + The review material on linear algebra presented in Chapter 6 and at the beginning of ‘Chapter 7 has been condensed and teorganized to better reflect how this material is taught at most institutions. ‘© We reworked and reordered the exercise sets to provide better continuity both within the scotions aul from sectiun to sectiou. New exercises have been alded to make the bookPreface mote flexible, and the more difficult theoretical exercises have been modified and ex- panded to make them accessible to @ wider range of student, * The bibliography has been supplemented with the relevant references for general- purpose software packages. When appropriate, older citations have been updated and replaced With more current sources, Library of Congress numhers ate now included ‘with all text and joumal references. This makes it easier to find material in the bibliog- raphy when doing a library search. We hope that this feature will become standard in mathematical books that contain extensive bibliographic material eran su ORT I ENS Algorithms {As in the previous editions, we give a detailed, structured algorithm without program listing for each method in the text, The algorithms are in a form that students with even limited programming experience can code. A Student Study Guide is available with this edition; it includes solutions to representative exercises and a disk containing programs written from the algorithms, The programs are written in hoth FORTRAN and Pascal and the disks are formatted for a DOS platform. The publisher can also provide instructors with a complete solutions manual that provides answers and solutions to all the exercises in the book as well as a copy of the disk that is included in the study guide. All the results in the Solutions Manual were regenerated for this edition using both the FORTRAN and Pascal programs on the disk. ‘The algorithms in the text lead to programs that give correct results for the examples and exercises in the text, but no attempt was made to write general-purpose professional software, In particular, the algorithms are not always written in the form that leads to the most efficient program in terms of either time ot storage requirements. When a conflict occurred between writing an extremely efficient algorithm and writing a slightly different one illustrating the important features of the method, the latter path was invariebly taken Suggested Course Outlines ‘The text has been designed to allow instructors flexibility in the choice of topics as well as in the level of theoretical sigor aud in he emphasis on applivativus. Iu line with these aims, we provide detailed references for the results that are not demonstrated in the text as well as for the applications that are used to indicate the practical importance of the meth- ods. The references cited are those most likely to be available in college libraries. For the most recent methods we also include original research paper quotations when we feel this material is accessible to our andience. The flow chart on page xiv indicates chapter prerequisites. The only deviation from this chart is described in the foomote at the bottom of the first page of Section 3.4. Most of the possible sequences that can be generated from this chart have been taught by the authors at Youngstown State University.| Chapter 1 [ Chapter? | | ‘Chapters | | Chapter ¥ 7 ¥ ¥ ¥ ¥ He [See] [Bont] [eies | [emmet)] [Gans | ¥ Chapier 9 Chapter 11 ac Chapter 12 EORTC Acknowledgments Unfortunately, it is not feasible to give individual recognition to all those who have made valuable suggestions for improving our presentation of numerical analysis. These include our own students and those at other schools, our colleagnes who teach namerical analysis courses, and the editorial and production personnel at PWS-KENT. ‘We would like to personally thank the reviewers for this edition: George Andrews, Oberlin College Richard O. Hill, Jr., Michigan John E. Buchanan, Miami University State University Richard Franke, The Naval Postgraduate School Leonard J. Lipkin, The University Richard E. Goodrick, Evergreen State College of North Florida Nathaniel Grossman, The University of California at im Ridenhour, Austin Peay State Los Angeles University Max Gunzburger, Virginia Polytechnic and state Steven E. Rigdon, Southern Tili- University nois University David R. Hill, Temple University In particnlar, we thank Phillip Schmidt of the University of Akron, Phil is both a good friend and, wheu appropriate, a must critical reviewer, We were again fortunate to have an excellent team of student assistants, headed by Sharyn Campbell. Included in this group were Genevieve Bundy, Beth Eggens, Melanie George, and Kim Graft. Thanks for keeping us honest. Finally, we would like to thank Chuck Nelson of the English Department at Youngstown State University forhis assistance on the variety of equipment in the Professional Design and Production Center. Richard L. Burden J. Donglas Faires| Cuaprer 1 Mathematical Preliminaries Es beginning enemistry courses, students ae confronted with area tionship known as the ideal gas law, : DV = NRT, which relates the pressure P, volume V, temperature 7, and number of moles. of an “ideal” gas, The R in this equation is a constant that depends only on the measurement system being used. Suppose two experiments are conducted to test this Jaw, using the same gas in each case, In the first experiment, V = 0.100 m, R = 0.08206. P N 1.00 atm, 0.00420 mole, ‘Using the ideal gas law, we predict the temperature of the gas to be PY —C.00}0100) — _ 999,15 Kelvin = 17° Cesius T= Nak ~ (010082000.08206) = ‘When we measure the temperature of the gas, we find that the true temperatureis 15°C.CHAPTER 1 + Mathematical Preliminaries ‘The experiment is then repeated, using the same values of K and LN, but increasing the pressure by a factor of two while reducing the volume by the same factor. Since the product PV remains the same, the predicted temperatnre would still he 17°C, but now we find that the actual temperature of the gas is 19°C. Clearly, the ideal gas law is suspect when an error of this magni- tude is obtained. Before concluding that the law is invalid in this situ- ation, however, we should examine the data to determine whether the error can be attributed to the experimental results. Ifo, it would be of interest to determine how much more accurate our experimental results would need (o be to ensure that an error of this magnitude could not oceur. Analysis of the error involved in calculations is an important topic in numerical analysis and will be introduced in Section 1.2. This articular application is considered in Exercise 24 of that section. ‘This chapter contains a short review of those topies from elementary single-variable calculus that will be needed in later chapters, together ‘with an introduction to the terminology used in discussing conver gence, error analysis, and the machine representation of numbers. 1.1 Review of Calculus : Definition 1.1 Definition 1.2 Fundamental to the: study of calculus are the concepts of limit and continuity of a function. Let fbe a function defined on a set X of real numbers; fis said to have the limit L at xo, written lim,.,,, f(2) = L, if, given any real number € > 0, there exists a real number 8 > 0 such thot |f(x) — L| < e whenever xe Xand0<|x — xq| <8. (Scc Figure 1.1.) Let f be # function defined on a set X of real numbers and xy € X; f is said to be continuous 2t x5 if lim,» f(x) = f(%o). The function f is said to be continuous on X ifitis continuous at each number in X. see CCX) denotes the set of all functions coutinuous on X. When X is an interval of the real line, the parentheses in this notation will be omitted. For example, the set of all functions continuous on the closed interval [a, b] is denoted Cla, b]. ‘The limit of a sequence of real or complex numbers can be defined in a similar manner.Figure 1.1 Definition 1.3 Theorem 1.4 Definition 1.5 fe Let {x,}£., be an infinite sequence of real or complex numbers. The sequence is said to converge to a number x (called the limi) if, for any € > 0, there exists a positive integer ‘N(e) such that n > Me) implies | x, — x| <¢. The notation lit, sa, = % OFX, > x as n> ©, means that the sequence {x, }5.., converges to x. oo ‘The following theorem relates the concepts of convergence and continuity. If f is a function defined on a set X of real numbers and x9 € X, then the following are equivalent: a. f is continuous at x9; b. if (x, Jf_1 is any sequence in X converging to xp, then dimif@n) = feeq). The functions we shall be considering when discussing numerical methods are as- sumed to be continuous, since this is 2 minimal requirement for predictable behavior, Functions that are not continuous can skip over pointe of interest, which is not a catisfac- - tory trait when attempting to approximate @ solution. More sophisticated assumptions about a function generally lead to better 4pproximation results. For example, a function with a smooth graph would normally behave more predictably than one with numerous jagged features. The analytic definition of smoothness relies on the concept of the ‘derivative. If f is a function defined in an open interval containing xo, f is said to be differentiable atxoif tim £2 = feo) im E> exists. When this limit exists it is denoted by (x0) and is called the derivative of f at Xo. A function that has a derivative at each number in a set X is seid to be differentiable4 CHAPTER 1 + Mathematical Preliminaries on X. The derivative of f at xo is the slope of the tangent line to the graph of f at (qf (%o)), as shown in Figure 1.2. ae Figure 1.2 y Tangent tne, spe f*(3) Fe) Co fed) y= G9) % % Theorem 1.6 Ifthe function f is differentiable at xo, then f is continuous at xo. ere ‘The set of all functions that have n continuous derivatives on X is denoted C"(X), and the set of functions that have derivatives of all orders on X is denoted C°(). Polynomial, rational, trigonometric, exponential, and logarithmic fonctions are in C*(X), where X consists of all numbers at which the functions are defined. When X is an interval of the real line, we will again omit the parentheses in this notation, ‘The next theorems are of fundamental importance in deriving methods for error estimation. The proofs of these theorems and the other unreferenced results in this section can be found in any standard calculus text. Theorem 1.7 (Rolle’s Theorem) Suppose fe C[a, b] and f is differentiable on (a, b). If f(a) = f(b) = 0, then a number c in (a, b) exists with f’(c) = 0. (See Figure 1.3.) Figure 1.3 i) SeemaTheorem 1.8 g a R Theurens 1.9 Definition 2.10 LL Review of Calculus an Vaine Theorem) life Cla, b] and f is differentiable on (a,b), then a number c in (a, b) exists with -s Pe AO a2 ee (See Figure 1.4.) _ Parallel ines | ! S00) ~ $a) | : t a xixeue Value Theorem) If f © Cla, b), then cy, c € (a, b] exist with f(c,) f(x) = f(c) for each x [a, b). If, in addition, f is differentiable on (a, b), then the numbers c, and c, occur either at endpoints of [a, 8] or where f” is zero. Ee ‘The other basic concept of calculus that will be used extensively is the Riemann integral. The Riemann integral of the function f on the interval (a, b] is the following limit, provided it exists: [roa- tm Spends, ras; 90 fet where the mumbers 29,2)... 5%, Satisfy a = ag S2, ~~~ =, = Land where, for each i= 1,2...” Ax = x; ~ 252), and z is arbitrarily chosen in the interval Leena] eee A function f that is continuous on an interval [a, b] is Riemann integrable on the interval, This permits us to choose, for computational convenience, the points x; to be equally spaced in (a, b] and for each ¢ = 1,2,...,m,to choose g = x, In this case, 2 [ fovae= tim ®=23 peea, bra where the numbers shown in Figure 1.5 as.x,ere2, = a + [i(b — a)]/n eeeFigure 1.5 Theorem 1.11 Figure 1.6 CHAPTER J + Mathematical Preliminaries } y= fo) at G= HM ee ey Meee Tuy D ‘Two other results will be needed in our study of numerical methods. The first is a ‘generalization of the usual Mean Vaiue Theorem for Integrals (Weighted Mean Value Theorem for Integrals) Ife Cla, b}, g is imegrabie on (a, b], and gx) dues not chauge sign on [a 6, then there exists a number cin (a, 6) with [ FO)g@) de = FO) f gt) de. ae When g(x) = 1, this theorem gives the average value of the function f over the interval (a, 6). (See Figure 1.6.) This average value is pon ef sean ‘The proof of Theorem 1.11 is not generally given in a basic calculus course but can be found in most analysis texts (see, for example, Fulks (59], p. 162). Fee) faTheorem 1.12 Theorem 1.43 Figure 1.7 EXAMPLE 1 LL Review of Calculus 7 The other theorem we will need that is not generally presented in a basic calculus course is derived by applying Rolle’s Theorem successively to fj", ... and finally to Pp. (Generalized Roles Theorem) Let fe C[a, b) be m times differentiable on (a, 5). If f vanishes at the n + 1 distinct numbers xq --. x, in [a, b), then a namber c in (a, b) exists with fc ‘The next theorem presented is the Intermediate Value Theorem. Although its state- ‘ment is intuitively clear, the proof is beyond the scope of the usual calculus course. The proof can be found in most analysis tests (see, for example, Fulks [59], p. 67). Gtermediate Value Theorem) fe Cla, b) and K is any number between f(a) and f(Q), then there exists c in (a, b) for which f(c) = K. (See Figure 1.7.) 7 eos Show that x> — 2x3 + 3x? — 1 = Qhas a solution in the interval (0, JJ. Consider f(x) = x5 — 2x° + 3x7 - 1. The function f is a polynomial and is continuous on (0, 1]. Since sO) = -1<0<1 =f), e Value Theorem implies that there is a number x in (0, 1) with x5 — 2x3 0. eee the Interm +37 = 1 As seen in Example 1, the Intermediate Value Theorem is important as an aid to ‘determine when solutions to certain problems exist. It does not, however, give a means For finding these solutions. This topic is considered in Chapter 2. ‘The final theorem in this review from calculus describes the development of the Taylor polynouiials. The importance of the Taylor polynomials to the study of numerical analysis cannot be overemphasized. and the following result will be used repeatedly.8 Theorem 1.14 EXAMPLE 2 CHAPTER 1 * Mathematical Preliminaries (faylor’s Theorem) Suppose f € C" [a, bl, f"*” exists on [a, b], and xo € [a, 6]. For every xe [a, bl, there exists €(2) between x9 and x with SG) = Pie) + Ri), where a P,0) = fla) + Fete ~ x) + 28D — age 4 Po ( ~ x0)" and er PEG), a R= Tp 70) Here, P,(x) is called the nth Taylor polynomial for f about xo and R,(x) is called the remainder term (or truncation error) associated with P,,(x). The infinite series obtained by taking the limit of P,(x) as n —» is called the Taylor series for f about xo, In the case Xo = 0, the Taylor polynomial is called a Maclaurin polynomial and the Taylor series is called a Maclaurin series. ane ‘The term truncation error generally refers to the error involved in using a truncated or finite summation to approximate the sum of an infinite series. This terminology will be reintroduced in subsequent chapters. Determine (2) the second and (b) the third Taylor polynomials for f(x) = cos x about Xo = 0, and use these polynomials to approximate cos(0.01). (c) Use the third Taylor polynomial and its remainder term to approximate {3 cos x dx. Since f © C*(R), Taylor's Theorem can be applied for any > 0. a. Forn = 2and.xo = 0, we have cos z = 1~ fx? + x3 sin £00), where (x) is a number between 0 and x. (See Figure 18.) With x = 0.01, the Taylor polynomial and remainder term is 1 ~ 4(0.01 + $(0.01) sin £(2) 0.99995 + 0.16 x 10° sin (2), where 0 < £(x) < 0.01. (The bar over the six in 0.16 is used to indicate that this digit repeats indefinitely.) Since |sin €(x)| <1, we have Jcos 0.01 ~ 0.99995] = 0.18 x 10-8, u cos 0.01 1 so the approximation 0,99995 matches at least the first five digits of cos 0.01. Using standard tables we find that the value of cs 0.01 to 11 digits is 0.99995000042, which gives agreement through the first nine digits.LL Review of Calculus 9 Figure 1.8 ra Since f”(0) = the third Taylor polynomial and remainder term about xp = 0 is cosa = = fart gbxt cos E(), where 0 < & (x) < 0.01. The approximating polynomial remains the same, and the approx- imation is still 0.99995, but we now have much better accuracy assurance since |&x* cos €@)| = 40.01)°0) ~ 4.2 x 107". The first two parts of the example nicely illustrate the two objectives of numerical analysis. The first is to find approximation, which both Taylor polynomials provide, The second objective is to determine the accuracy of the approximation. In this case the third Taylor polynomial was much more informative than the second, even though both poly- nomials gave the same approximation. fe. Since cos x = 1 — 4x? + dhx* cos E(x), on a1 1 [ cova = [ a-p aed 008 Es) dr 01 Bsa xt cas E(x) dx 03 0.1 = 2004) + af, x“ cos E(x) dx. So (on cos x dx = 0.1 — 3 (0.17 = 0.09983. ‘A bound for the e1ror in this approximation can be determined from the integral of the ‘Taylor remainder term:10 CHAPTER 1 * Mathematical Preliminaries 1 1 on & if x* cos fo ax] «af x* |oos Folar=a[ xtdx = 83x 107% Since the true value of this integral is Fate the error for this approximation is within the bound, Bee sin 0.1 ~ 0.099833417, EXERCISE SET 11 1. Show thet the following equations have at least une solution in the given intervals: a xcosx— 2? + 3x-1=0, — (0.2,0.3] and [12,13] b. @-2%-Inx=0, [1,2] and fe,4] & Wxcos2e- G2? =O, (2, 3land [3,4] a x daxy¥ 0, (4,51 2, Find intervals thet contain solutions to the following equations: a r-3%=0 : be. dx? ~ er = 0 oe Pb —det3=0 a. x7 + 4.0012? + 4.002x + 1.101 = 0 3. Show that the first derivatives of the following functions are zero at least once in the given intervals: a f@=1—e*+(e-1)sina/29), (0,11 b. f@) = @~ Itanx + xsin 7x, (0,1). f@sxsnax-@-Dinx (1,2. a. f@=@—Dsinxine +2, [~1.3]. 4, Find max | f(2)| for the following functions and intervals: a. f@)=2-e% +2)/3, [0,1] bf) — 4 — 3)/G? 2), [05,1] & FQ = Recos(2s) ~ (x 2%, 24 @ Feit eV, 1,2 ef () = 160 cos(2x) — 64x sin(2x), (0,04) £ f() = (ny [4,51 5. Use the Intermediate Value Theorem and Rolle’s Theorem to show that the graph of f(x) = x? + 2x + k crosses the x-axis exactly once, regartiless of the value of the constant k, 6 Suppose f © C [a, b] and f'(z) exists on (a, 6), Show that if f'(x) ¥ 0 for all x in (@, 6), then there can exist at most one number p in (a, b] with f(p) = 0. 7. Find the second Taylor polynomial P,(x) for the function f(x) Xp 0. a, Use P,(0.5) to approximate f(0.5). Find an upper bound for the error | (0.5) ~ P2(0.5)| using the error formula and compere it to the actual error. b. Flug e bound for the exror |G) — F2(2)| in using FG) (© aporunimate f@) ow the interval (0, 1) eos x expanded boutLL Review of Calculus w ec. Approximate [5 (2) dx using J§ PG) dx d. Find an upper bound for the error in (c) using J} |e(x)| dx and compere the bound to the ecwual error. 8. Repeat Exercise 7 using xp = 7/6. 9. Find the thira Taylor polynomial P(x) for the Munction (x) = Gr ~ 1) Inxexpandea about x= ‘a. Use Px(0.5) to approximate /(0.5). Find an upper bound for the error | (0.5) (0.5) using the error formula end compere it to the actual error, b. Find a bound for the error | f(x) — P,(2)| in using P.(x) to approximate f(x) on the interval (0.5, 1.3] Approximate J3$,/() deusing J2§ Ps) dz. 4. Find an upper bound for the error in (c) using [43 {Ra(x)| dx and compare the bound to the actual error. 10. Let f(s) = 2x cos(2x) ~ (¢ ~ 2 and x = 0. a. Find the third Taylor polynomial Py(x) and use it to approximate f (0.4) b. Use the error formula in Taylor's Theorem to find an upper bound for the error | #(0.4) ~ Px(0.4)|. Compute the actual error. & Find the fourth Teylor polynomial P(x) and use itt appronimatc f(04). 4. Use the exror formula in Taylor's Theorem to find an upper bound for the exror | (0.4) ~ P(0.4)}, Compute the actual error. 11. Find the fourth Taylor polynomial P,(x) for the function f(z) = xe* expanded sbout xo 0. a. Find an upper bound for | f(2} ~ Pe(a)\ for 0 = x= 0.4. b. Approximate [$4 f(x) dx using [3* P.(x) da. ¢. Find an upper bound for the error in (b) using [64 P(x) dx. @. Approximate #"(0.2) using P,(0.2) and find the error. 12. Use the error term of 2 Taylor polynomial to estimate the error involved in using sin x~ x to approximate sin 1°, 13. Use 2 Taylor polynomial about 77/4 to approximate cos 42° to an accuracy of 106 14. Let FG) = (1 ~ 29°! and xo = 0. Find the nth Taylor polynomial P,(x) = #() expanded about xq. Find the value of n necessary for P,(4) to approximate (2) to within 107% on (0,051. 1S. Let f(x) = e* and xp = 0. Find the nth Taylor polynomial P,(x) for (2) expanded about o.Find the value of n necessary for P,() to approximate f(z) to within 10°® on (0, 0.5]. 16. Find for an arbitrary positive integer n, the nth Maclaurin polynomial P,(2) for f) = arctan. 17. The polynomial P,(2) = 1 — $32 is used to approximate f(2) ~ cos xin {—f,!]. Find a ound for the maximum error. 18 The nth Taylor polynomial for a function f et.xq is sometimes referred.to as the polynomial of degree at most n that “best” approximates f neat Xe - a. Explain why this description is accurate , Find the quadratic polynomial that best approximates 2 function fnear x) ~ 1, ifthe function has as iis tangent line the line with equation y = 4x ~ 1 whenx = { and hes £7) = 6.12 CHAPTER 1 * Mathematical Preliminaries 19, 21. ‘A Maclautin polynomial for e* is used to give the approximation 2.5 to e. The error bound in this approximation is established to be £ = % Find a bound for the error in E, ‘The error function defined by [ora gives the probability thet any one of a series of trials will lie within x units of the mean, assuming that the trials have @ standard normal distribution. This integral cannot be evalu ated in terms of elementary functions, so an approximating technique must be used. Integrate the Maclaurin series for e~"” to show that b. Theerror function can also be expressed in the form tee 2 oH = ae aS aD ‘Verify thatthe two series agree for k = 0, 1, 2, 3, and 4, (Hint: Use the Maclaurin series tore" © Use the series in part (a) to approximate erf(1) to within 10~7. 4, Use the same number of terms used in part (€) to approximate erf(1) with the series in part). fe. Explain why difficulties qeeur ing the series i part (0) to approximate er), Suppose f € Cla, b], that x, and x, are in [a, b], and that cy and ¢, are positive constants. Show that a number Zexists between x, and.x, with of) + aft abe fo Use the Mean Value Theorem to show the following: a. [cosa —cosb|s|a—b| —b. |sina + sinb|=|a +d] A function f: [a, b] > R is said to satisfy a Lipschitz condition with Lipschitz constant L on [a, B] if, for every x, » ¢ a, by | FO) — f()| = Le 9h 8, Show that if f satisfies a Lipschitz condition with Lipschitz constant Zon an interval (a, B), then f = C [a,b]. b. Show that if f has a derivative that is bounded on [a, 5] by L, then f satisfies a Lipschitz condition with Lipschitz constant L on (a, b]. ©. Give an example of a function thet is continuous on a closed interval but does not satisfy 4 Lipschitz.condition on the interval 1.2 Round-Off Errors and Computer Arithmetic ‘The arithmetic performed by a calculator or computer is different from the arithmetic that ‘we use in our algebra and calculus courses. From our past experiences we expect that we will always have as true statements such things as 2 + 2 = , 2 = 16, and (Vy = 3,12 RoundOjf Errors end Computer Arithmetic 3 Jn standard computational arithmetic we will have the first two, but not the third. To understand why this is true we must explore the world of finite-digit aritmetic. In our traditional mathematical world we permit numbers with en infinite number of nonperiodic digits. The arithmetic we use in this world defines V3 as that unique posi- tive number that when multiplied by itself produces the integer 2. In the computational ‘world, however, each representable number has only a fixed, finite number of digits. Since V3 does not have a finite-digit representation, it is given an approximate representation ‘within the machine, one whose square will not be precisely 3, although it will likely be suf- ficiently close to 3 to be acceptable in most situations. In most cases, then, this macbine epresentation and arithmetic 15 satisfactory and passes without notice or concern, ‘but we must be aviare that this is not always true and be alert to the problems that it can produce. Round-off error occurs when a calculator or computer is used to perform real-number calenlations. ‘This error arises hecause the arithmetic performed in a machine involves numbers with only a finite number of digits, with the result that calculations are performed ‘with approximate representations of the getual numbers. In a typical computer, only @ relatively small subset of the real number system is used for the representation of all the real numbers. This subset contains only rational numbers, both positive end negative, and stores a fractional part, called the mantissa, togetier with an exponential part, called the characteristic. For example, ¢ single-precision floating-point number used in the IBM 3000 and 4300 series consists of a 1-binary-digit (bit) sign indicator, « 7-bit exponent with a base of 16, and 2 24-bit mantissa ce 24 binary digits correspond to between 6 and 7 decimal digits, we can assume that this number bas at least 6 decimal digits of precision for the floating-point number system. The exponent of 7 binary digits gives 2 range of 0 to 127. However, using only positive integers for the cbaracteristic does not permit an adequate representation of num- bers with small magnitude. ‘ ensure thet numbers with smell magnitude are equally representable, 64 is subtracted from the characteristic, so the range of the exponential part is actually from — 64 to +63. Consider, for example, the mechine number © | 1900010 | 1011001 1000001 0000000000, L.@_] 1000010 _| 1011001 10000010000000000_} ‘The leftmost bitis 2 zero, which indicates that the number is positive. The next seven bits, 1000010, are equivelent to the decimal number 12940298 40-2 406% 4 OE 1-2 40-2 = 66 and are used to describe the characteristic, 16°. The final 24 bits indicate that the mantissa is 1+ (Wy) + 1+ GAP + 1 iat + L- 2y + 1 C2) + 1-2). As a consequence, this machine number precisely represents the decimal number + (G/2)! + (2 + (DS + C2? + IDS + DV HIG-* = 179,015625. ‘However, the next smallest machine number is 0 | 000010 | ioisoorio000001i11111111 | = 179.0156097412109375,14 CHAPTER 1 * Mathematical Preliminaries and the next largest machine number is © | 1000010 | 1011001 10000010000000001_| = 179.0156402587890625. ‘This means that onr original machine number rust represent not only 179.015625, bnt many real numbers that are between this number and its nearest machine number neigh- bors. ‘To be precise, the original machine number is used to represent any real number in the interval 1179.01561737060546875. 179.01563262939453125). ‘TD ensure uniqueness of representation and obtain all the available precision of the system, a normalization is imposed by requiring that at least one of the four leftmost bits, of the mantissa of a machine number is a 1. Consequently, 15 x 2* numbers of the form # Odyda... day X 160% are used by this system to represent all real numbers. With this representation, the number of binary machine numbers used to represent [16", 16"*"] is constant independent of n within the limit of the machine; that is, for — 64 = n = 63, This requirement also implies that the omallest normalized, positive machine number that can be represented is, © | c000000 | 000100000000000000000000_| = 16- = 10", while the largest is O | anandaa | aininaaagraaiininaai1iia_|~ 16 ~ 1076, Numbers occurring in calculations that have a magnitude of less than 16~® result in what is called underflow and are often set to zero, while numbers greater than 16% result in overflow and cause the computations to halt, The arithmetic used on microcomputers differs somewhat from that used on main- frame computers. In 1985, the IEEE (Institute for Electrical and Electronic Engineers) published a report called Binary Floating Point Arithmetic Standard 754-1985. In this report, formats were specified for single, double, and extended precisions, and these stan- dards are generally followed by microcomputer manufacturers who use fioating-point hardware. For example, the numerical coprocessor for IBM-compatible microcomputers implements a 64-bit representation for a real number, called a Jong real. The first bit is a sign indicator denoted s. This is followed by an 11-bit exponent c and a 52-bit mantissa f ‘The base for the exponent is 2 and, to obtain numbers with both large and small magni- tude, the actual exponent isc — 1023. In addition, a normalization is imposed that requires that the units digit be 1, and this digit is not stored as part of the 52-bit mantissa. Using this system gives a floating-point number of the form (Hype +P), which provides between 15 and 16 decimal digits of precision and arange of approximately 10-** to 108. This is the form that compilers use with the coprocessor and refer to as double precision. ‘The use of binary digits tends to conceal the computational difficulties tat occur when a finite collection of machine numbers is used to represent all the real numbers. ToEXAMPLE £ Definition 115 1.2 Round-Off Errors and Computer Artibmetic 3 explain the problems that can arise, we will now assume, for simplicity, that machine numbers are represented in the normalized decimal floating-point form + 0d, d; 4X10, 15459 054 <9, for each i= 2,...,&, where, from whet we beve just discussed, the IBM mainframe machines have approximately k ~ 6 and ~78 y, have the k-digit representations Sle) = Od de. ty Qpur Open 10", and AC) = O.ds dy... dy Bp Bys2 +++ Be X 10", The floating-point form of x ~ yis ACA) — AC) = Oop; Opea--- OE X 10%, where O.0p 41 Ope2 + 84 =O. s1 Gye Be ~ OByy1 Boa Be The fioating-point number used to represent x — y has only k — p digits of significance. However, in most calculation devices, x — y will be assigned k digits, with the last p being either zero of randomly assigned. Any further calculations involving x — y retain the problem of having only k — p digits of significance, since a chain of calculations cannot be expected to be more accurate than its weakest portion. Ifa finite-digit representation o calculation introduces an error, further enlargement of the error occurs when dividing by a number with small magnitude (or equivalently, when multiplying by a number with large magnitude). Suppose, for example, that the number z has the finite-digit approximation z + 8, where the error 6 is introduced by representation or by previous calculation. Dividing by e+ 0 results in the approximation © cfem ae + )/ (HO). ‘Suppose € = 10°", where n > 0. Then c/e=zx 10" and AG + /AB) = @ + HX 10". Thus, the absolute exror in this approximation, |6| x 10", is the original absolute error |3| multiplied by the factor 10". The loss of accuracy due to round-off error can often be avoided by a careful se- quencing of operations or reformulation of the problem, as ilustrated in the final two examples, ‘The quadratic formula states that the roots of ax? + bx + ¢ = 0, whena #0, are an ao and Consider x? + 62.10x + 1 a quadratic equation with roots approximately x, = =0.01610723 and x, = ~62.08300. In this equation, }? is much larger than dac, so the numerator in the calculation for 21 involves the subiraction of nearly equal numbers. Suppose we perform the calculations for x, using four-digit rounding arithmetic, First we have1.2 Round-Off Errars and Computer arithmetic 19 VP = 4ac = V(G2.10)" = 4.000 = V3856. — 4.000 = V385; 62.06. So b+ VP = 4ac _ 62.10 + 62.06 _ ~0.04000 Ale) = I = rg agg = 0.02000 is an approximation tox, = ~0.01611, whose relative error is large 01611 + 0.02000) [-0.01611] =24x 1071 On the other hand, the calculation for x, involves the addition of the nearly equal numbers band — Vi" = 4ac and presents no problem: VP = 4ae _ - 62.10 ~ 62.06 124.2 Alles) = 7 = agg 7 20, an approximation to x, = —62.08, with relative error 62.08 + 62.10) ; ]— 62.08} 32x 10 To obtain a more accurate fo form of the quadratic formula by “ digit rounding approximation for x, we change the rationalizing the numerator’ @ = 4ac) 2a(—b — VP = gac)" which simplifies to a2 Using (1.2) gives = 2.000 2.000 62.10 + 62.06 ~ 124.2 with the small relative error 6.2 X 1074. ‘The “rationalization” technique can also be applied to give the alternate form for x3: ad =2¢ 2 b= VB = dae ‘This is the form to use if b is a negative number. In our problem, however, the use of this tormula results in not only the subtraction of nearly equal numbers, but also the division by the small result of this subtraction. The inaccuracy that this combination produces, fled = = -0.01610, as) has the large relative error 1.9 x 107). Eres20 EXAMPLE 5 Table 13 CHAPTER 1 * Mathematical Preliminaries Evaluate f(x) = x? - 6x* + 3x - 0.149 atx = 4.71 using three-digit arithmetic, ‘Table 1.3 gives the intermediate results in the calculations. Note that the three-digit chopping values simply retain the leading three digits, with no rounding involved, and differ significantly from the three-digit rounding values. x & FP xt Be Exact 47 104487111 133.1046 14.13 ‘Three-digit (chopping) 47 104, 132. 141 ‘Threedigit (rounding) 4 105. 133. 141 Exact: F(471) = 104.487111 — 133.1046 + 14.13 - 0.149 = —14.636489; ‘Three-digit (Chopping): (4.71) = 104. - 132, +7141 - 0.149 = -14.0; Three-digit (rounding): (4.71) = 105. - 133. + 14.1 - 0.149 = - 140. ‘The relative error for both the three-digit methods is 14.636489 + 1400 ~ 008 = 14636489 | ~° Asan alternative approach, f(x) could be written in a nested manner as FG) = x9 = 6x? + 3x — 0.149 = (& - Ox + 3x - 0.149. This gives ‘Three-digit (chopping): f(4.71) = (4.71 — 64.71 + 34.71 - 0.149 = -14.5, and a three-digit rounding answer of — 14.6. The new relative errors are. — 14.636489 + 14.5) ‘Three-digit (chopping) jeans ~ 0.0093; = 14636489 + 14.6 ‘Three-digit (rounding): sees = 0.0025. Nesting has reduced the relative error for the chopping approximation to less than one-fourth that obtained initially. For the rounding approximation the improvement has been even mote dramatic. The errorin this case has been reduced by more than 90%. Bes Polynotnials should always be expressed in nested form before performing an evalu ation, since this form minimizes the number of required arithmetic calculations. The decreased error in Example 5 is due to the fact that the number of computations has been.12 Round-Off Errors and Computer Arithmetic 21 reduced from four multiplications and three additions to two multiplications and three additions. One way to reduce round-off error is to reduce the number of error-producing computations. EXERCISE SEF 12 1 4. 10. Find the largest interval in which py? mnst He to approximate p with relative error at mast 10" ifpis aon be a Vi a Vi Suppose p* must epproximete p with relative error at most 10°*, Find the interval in which p® must hie ifp is a 150 Bb, 900, 1500 a 90 Compute the absolute error and relative exrorin the following approximations of p by p*: a po mp = 22/7 db. p= mp* = 31416 c p=ep* = 2718 a. p= V3p" = 1414 e p= el pt = 20000 f p= 10,p* = 1400 g p= 8I,p* = 39900 bh p= Opt = Vi8a@/e? Perform the following esmputations (9 exacdy, (fusing three-digit chopping arithmetic, (iii) using three-digit rounding arithmetic. (iv) Compute the relative errors in parts (ii) end (0, 442 rat a 343 boos G-B+4 @ G+9-8 value determined to at least five digits, compute the abgolure error and relative error. a. 133 + 0.921 b. 133 — 0.499 e, (121 — 0.327) - 119 @. (121 ~ 119) — 0.327 & = 107+ 60-3 -2 i ‘Repeat Exercise 5 using four-digit rounding arithmetic. git chopping arithmetic. Repeat Exercise 5 using four-digit chopping arithmetic. ‘The first three noszero terms of the Maclaurin series for the arctangent function are Repeat Exercise 5 using three- x $234 bx! Compute the absolute errorand,elative error in the following approximations of zrusing the polynomial in place of the arctangent: a. 7~4[artea$ + arctan] b. 7=t6arcten} - 4 arctan dy > de where nt ‘The number e is sometimes defined by € n(n 1)++-2- 1 ifm #0,CHAPTER 1 + Mathematical Preliminaries and 0! = 1, Compute the absolute error and relative error inthe following approximations of Pi ay 1 ae Da bende LL. Using four-digit vounding arithiuevc aud dhe Foumutas of Exemple 4, flnd Whe most eccunate approximations to the roots of the following quadratic equations. Compute the absolute errors and relative errors a dx? -1Bx+3=0 b. dts Heb =0 c. 1,002x? ~ 11.01x + 0.01265 = 0 a. 1,002x7 + 11.01x + 0.01265 = 0 12. Repeat Exercise 1] using four-digit chopping arithmetic. 1B, Using the IBM mainframe format, find the decimal equivalent of the following floating-point machine numbers; a 0 1000011 10101001001 1606000000000 bot 1000011 101010010011000000000000 fe. 0 OLI1I11 —010001111000000000000000 dO o1nilt 910001111 e00000000000001 14, Find the next largest and smallest machine numbers in decimal form for the mumbers given in Exercise 13, 15. Suppose two points (xp, yo) and (%}, }) are on a straight Ine. Two formulas are available 0 find the x-intercept of the line: pe BAl gy pe gy STE, : "3 ne a Show that Lott fousudles are ulgebraivally correct, b. Using the dats (zo, 94) = (1.31, 3.24) and (ry, 93) = (1.93, 4.76) and chree-cigit rounding arithmetic, compute the x-intercept both ways. Which method is better and why? 16, The Taylor polynomial of aegree n or fu) = eis D Use the Taylor polynomial of degree nine and three-digit chopping arithmetic to find an approximation to e~® by: sey cus mot me a E An approximate value of e* correct to three digits is 6.74 x 107. Which formule, (a) or (b), gives the most accuracy, and why? ‘The two-by-two linear system art bye, i atd=f where a, b,c, de, f are given, can be solved for x and y as follows: setm= e/a, provided a + 0; a, = d= mb;18, 19. 20. (e- by)/a Solve the following linear systems using four-digit rounding arithmetic. a, 1.130x — 6.990y = 14.20 b. L013: ~ 6,09 = 14.22 8.110x + 12.209 = —0.1370 18.1 + 12.ay = -0.1376 Repeat Exercise 17 using four-digit chopping arithmetic 8. Show thatthe polynomial nesting technique described in Example 5 ean also be applied wo the evaluationof FO) = LOT = 4628 = 3.11 4 1720 = 1.99) b. Use three-digit rounding arithmetic, the assumption that el“? = 4.62, end the fact thet e™ = (@°Y" to evaluate f(1.53) as given in part). Redo the calculation in part (b) by first nesting the calculations. 4. Compare the approximations in parts (b) and (¢) to the true three-digit result f(1.53) = “7.61 ‘A rectangular parallelepiped has sides 3 em (centimeters), 4 cm, and 5 cm, measured only to the nearest centimeter. What are the best upper and lower bounds for the volume of this parallelepiped” What ere the best upper and lower bounds for the surface area? Suppose thas fy) is a k-digit rounding approximation to y. Show thet A) [Hints If dy, <5, then fy) = x 10° + 10") ‘The binomial coefficient My dy dy X ONAL dey 25, hem fy) = Od, dy. dy ()- me eseribes the number of ways of choosing a subset of & objeots from a set af m elements. a Suppose decimal machine mumbers are of the form 404, ddd X10, 1sds9, 05459, and ~1Ssns15, ‘What is the lergest value of m for which the binomial coefficient ('f) can be computed by the definition without causing overflow? b. Show that ('t) can also be computed by ()-@GH)- C44) ¢, What is the largest value of m for which the binomial coefficient (#"} can be computed by the formula in part (b) without causing overflow? @. Using four-digit chopping arithmetic, compute the number of possible 5-card hands in a 52-cand deck. Compute the actual nd relative error, Let f © Cla, 6} be & function whose derivative f* exists on (a, 6). Suppose fis to be evalutted at xy in (a, 6), but instead of computing the actual value (x), the approximate value, F'(xo)is the eotual value of f at xo + E:thatis, Fla) = Fla + 8.24 CHAPTER 1 * Mathematical Pretiminaries a. Use the Mea Value Theorem to estimate the absolute error |/(va) ~ Ftvg)| and ehe [io = Lea essaming flr) #0. relative exo [SE b. Ife = 5 x 10°F and xo = J, find bounds for the absolute and relative erors for Fey =e" Hf) = sin Repeat part (b) with € = (5 * 10"S)xg and x9 = 10. 24, The opening example to this chepter described @ physical experiment involving the temper ature of a gas under pressure. In this application. we were given P = 1.00 atm. V = 0.100m°. N = 0,00420 mole, and R = 0.08206. Solving for Tin the ideal ges law gives PY ___(1.00X0.100) T= In (@:004203(0.08206) ~ 290-15 = 17°C. Tn the laboratory, i€ wus found dhet 7 = 15°C under dhese conditions, wan! when the pressure was doubled and the volume halved, 7 = 19°C, Assuming that the data are rounded values accurate 1 the places given, show that both laboratory figures are within the bounds of accuracy for the ideal gas law. 1.3 Algorithms and Convergence: ‘The cxamples in Scetion 1.2 demonstrate ways that machine calculations involving ap- ptoximations can result in the growth of round-off errors. Throughout the text we will be examining approximation procedures, called algorithms, involving sequences of calcula- tions. An algorithm is a procedure that describes, in an unambiguous manner, a finite sequence of steps to be performed in a specified order. The object of the algorithm is to implement a numerical procedure to solve problem or approximate a solution to the problem, A pseudocode is used for describing the algorithms. This pseudocode specifies the form of the input fo be supplied and the form of the desived output. Not all numerical procedures give satisfactory output for arbitrarily chosen input. As a consequence, a stop ping technique independent of the numerical technique is incorporated into each algorithm so that infinite loops are unlikely to occur. ‘Two punctuation symbols are used in the algorithms: the period (.) indicates the termination of a step, while the semicolon (;) separates tasks within a step. Indentation is used to indicate that groups of statements are to be treated as a single entity. Looping techniques in the algorithms are either counter controlled, for example, Leen : Set =a + irk For or condition controlled, such as, While i < N do Steps 3-6. ‘To allow for conditional execution, we use the standard If... thenEXAMPLE £ EXAMPLE 2 13 Algorithms and Convergence 25 or If. .: then else constructions. The steps in the algorithms generally follow the rules of structured program construc- tion. They have been arranged so that there should be minimal difficulty translating pseu- docode into any programming language suitable for scientific applications. However, many of the more advanced topics in the latter balf of the book are difficult to program in certain languages, especially if large systems ot complex arithmetic are involved. ‘The algorithms are liberally laced with comments. These arc written in italics and contained within parentheses to distinguish them from the algorithmic statements. An algorithm to compate Pmt tay, where N and the numbers x,, x2... . 2 are given, is described by the following: INPUT Nyx yx... sane OUTPUT SUM = > x, bast Step I Set SUM = 0. Step2 Fori=1.2,...,Ndo set SUM = SUM + x, Step 3. OUTPUT (SUM) STOP, eee ‘The Taylor polynomial P,, (x) for f(2) = In.x expanded about x = lis Pyoy= mo @- 1h ‘The value of In 1.5 to eight decimal places is 0.40546511, Suppose we want to compute the minimal value of N required for [In 1.5 - Py(1.5)} < 10-5, without,using the Taylor polynomial truncation error formula. From calculus we know that if a, is an alternating series with limit A whose terms decrease in magnitude, then a N A and the Nth partial sum A, = >, a, differ by less than the magnitude of the N + 1th term; that is, \A - Ayl S lay,26 Definition 1.16 CHAPTER 1 * Mathematical Preliminaries An algorithm to solve this problem is INPUT value x, tolerance TOL, maximum number of iterations M. OUTPUT degree N of the polynomial or a message of failure. 4 Step1 SetN y SUM = 0; POWER = y; TERM SIGN = Step 2 While N = M do Steps 3-5. Step 3 Set SIGN = ~SIGN; SUM = SUM + SIGN: TERM; POWER = POWER+y, TERM = POWER; (N +°1). Step 4 If|TERM| < TOL then OUTPUT (N); STOR Step 5 SN =N +1; Step 6 QUTPUT (‘Method Failed’); STOP, ‘The input for our problem would ve x = 1.5, TOL = 1075, and perhaps M = 15. This choice of M provides an upper bound for the number of calculations we ate willing to have performed, recognizing that the algorithm is likely to fail if this bound is exceeded ‘Whether the output is a value for NV or the failure message depends on the precision of the computational device being used. moe We are interested in choosing methods that will produce dependably accurate results for a wide range of problems. One criterion we will impose on an algorithm whenever possible is that small changes in the initial data produce correspondingly small changes in the final results. An algorithm that satisfies this property is called stable; it is unstable when this criterion is not fulfilled. Some algorithms will be stable for certain choices of initial data but not for all. choices. We will characterize the stability properties of algo- rithms whenever possible. ‘To consider further the subject of round-off error growth and its connection to algo- rithm stability, suppose an error with magnitude Zp is introduced at some stage in the calculations and that the magnitude of the error after n subsequent operations is denoted by Ey. The two cases that arise most oftén in practice are defined as follows. Suppose that E,, represents the magnitude of an error after n subsequent operations. If E, ~ Cn Ep, where C ig a constant independent of n, the growth of error is said to be Mnear. If £, ~ C"Ep, for some C > 1, the growth of error is called exponential,Figure 1.9 EXAMPLE 3 13 Algorithms and Convergence wy q Linear growth of error is usually unavoidable and when C and Eo are small the results are generally acceptable. Exponential growth of error should be avoided, since the term C* becomes large for even relatively small values of n. This leads to unacceptable inaccu- racies, regardless of the size of Ey As a consequence, an algorithm that exhibits linear growth of error is stable, while an algorithm exhibiting exponential error growth is unstable. (See Figure 1.9.) Exponential growth "Ey © Linear growth en tf | B= OnE ‘The sequence p, = ()", n > 0, can be generated recursively by letting py = 1 and defining p, = 4p, -,, for n > 1. If the sequence is generated in this manner using five-digit rounding arithmetic, the results are 0.10000 x 10', 0.33333 x 10%, 0.11111 x 10°, 0.37086 x 10-', 0.12345 x 107, ‘The round-off error introduced by replacing } by 0.33333 produces an error of only (0.33333Y' x 10° in the nth term of the sequence. This method of generating the se quence is stable. Another way to generate the sequence is to define py = 1, p, = 4, and compute, for each n = 2, ad) Pr = "BPy-1 ~ Prod Table 1.4 lists the exact and five-digit rounding results using this formula. This method is unstable. Note that formula (1.4) is satisfied whenever p, is of the form Pe = C4)" + C3"‘Table 1.4 CHAPTER 1 * Mathematical Preliminaries for any pair of constants C, and C2. To verify this, Brrr — Pron = PLC Gy! + e"'] - [eo Gy? + car] = [RG ~ GA] + [Patt 3] = C, (3)" + C3 Pa To have py = 1 and p, = } in Eq, (1.4), the constants C, and C, must be chosen as 0. However, in the five-digit approximation, the first two terms are 0.10000 X10" and p, = 0.33333. 10°, which requires a mochtication of these constants to C, = 0.10000 x 10! and C, "= —0,12500 x 10%. This small change in C, results in a round-off error of 3"(— 0.12500 x 10-‘) when calculating p,. As a conse- quence, an exponential growth of esror results, which is reflected in the extreme inaccu- racies found in the entries of Table 1.4. mee n Computed p, Correct value p, o 0.10000 x 10° 0.10000 x 10° 1 0.33333 « 10° 0.33333 x 10” 2 0.11110 x 10° OIL x 10 3 037000 x 10°" (0.37087 x 107 4 0.12230 X 107" 0.12346 x 107 5 0.37660 x 10" 0.41152 x 10°? 6 032300 x 10° 0.13717 x 107 7 =0.26893 x 10 0.45725 x 107° 8 musE8T2 XIU U.15242 x 10 To reduce the effects of round-off error, we can use high-order digit arithmetic such 2s the double- or multiple-precision option available on most digital computers. A disad- vantage in using double-precision arithmetic is that it takes more computer time, and the growth of round-off error is not eliminated but only postponed until subsequent computa- tions are performed. One approach to estimating round-off error is to use interval arithmetic (that is, to retain the largest and smallest possible values at cach step) so that, in the end, we obtain an interval that contains the true velue. Unfortunately, we may have to find a very small interval for reasonable implementation. It is also possible to study ervor from a statistical standpoint, This study, however, involves considerable analysis and is beyond the scope of this text. Hemrici [72], pages 305-309, presents a discussion of a statistical approach to estimate accurnulated round-off error. Since iterative techniques involving sequences are often used, the section concludes with a brief discussion of some terminology used to describe the rate at which convergence ‘occurs when employing @ numerical technique. In general, we would like the technique to converge as rapidly as possible. The following definition is used to compare the conver ‘gence rates of various methods,EXAMPLE 4 ‘Table 1.5 EXAMPLE 5 13. Algoritoms and Convergence 29 Suppose {,) is a sequence known to converge to zero and (a,)".., converges to a number a. Ifa positive constant K exists with la, - a} = K]B,| for large n, then we say that {a,,]%., converges to a a with rate of convergence 0(8,). (This is read “big ob of 8, .°) This is indicated by writing a, = a + O(8,) or a, > a with rate of convergence O(6,,), a Suppose that the sequences {a} and {é,} are described by a, — (2 + 1)/n? and & — (w+ 3)/7? for each integer n = 1. Although lim... @, = 0 and lim,_,.. @, = 0, the sequence {4} converges to this limit much faster than the sequence {a,,}. In fact, using five-digit rounding arithmetic gives the entries in Table n 1 3 4 5 6 7 co, — 2,00000 0.75000 0.44444 0.31250 0.24000 0.19444 0.16327 , 4.00000 0.62500 0.22222 0.10938 0.064000 0.041667 0.029155, If we let 8, = 1/nand 8, = 1/1? for each n, we see that n+1lontna 1 la, ~ 0) = 5 ~ = 2B, 7 n +3 and la, - 01 =" 7 (1\ 1\ so a= 0+ (7) vile g, = 0 + O(-5) \n n The race of convergence of {a} to zer0 is similar to the convergence of {1,/'n}t0 zero, while {4,} converges to zero at a rate similar to the more rapidly convergent sequence {1/n?}. BEE ‘We also use the “hig ob” notation to describe the rate at which functions converge Suppose it is known that lim G(h) = 0 and that lim F(t) = L. If a positive constant K id ho exists with as) |FQ) ~ Li = KG(h) for sufficiently small h, then we write F(h) = L + O(G(h). From Example 2(b) of Section 1.1 we imow that by using 2 third Taylor polynomial + Ei cos Eh)30 CHAPTER 1 + Matbematical Preliminaries for some number é (h) between zero and h. Consequently, cosh + $1? = 1 + dh cos Eh). This implies that cosh + 4H = 1 + OC0), since |(cos h + 4 a = [Arcos Eula = Spat, ‘The implication is that cos h + 3h converges to its limit, 1; at least as fast as h* ‘converges 10 zero. BEE EXERCISE SET 13 » 1. a Use three-digit chopping arithmetic to compute the cum D+ fret by} 4 b+ + pigand then by ig + + ++ « + 4. Which method is mere accurate and why? . Write an algorithm fo sum the finite series 5x, in reverse order. a 2 The number eis defined by e = >) 4, where nt = n(n — 1)- + -2- 1, form Oand 0! = 1. i) Use four-digit rounding arithmetic to compute the following approximations to e, Also ‘compute absolute and relative errors. a exe on 3. The Maclaurin series for the arctangent function converges for all values of x and is given yy 5 ror =e . ZO aaa arctan x Recall that tan a /4 = 1, a, Determine the aumber of terms of the series that need to be summed to ensure that | arctan | — x] < 10-3 b. Tho single precision version of the sciontifie programming language FORTRAN re- quires the value of 7 to be within 10°”. How many terms of this series must be summed to obtsin this degree of accuracy? 4, Bxorsise 3 dotils a rather inefficiont means of obtaining sn epproximation to 7. The method can be improved substantially by observing that » /4 = arctan } + arctan } and evaluating the series for arctan at J and at 4, Determine the number of terms that must be summed t0 ensure an approximation 7 to within 107%,13 i rR B. 14. Algorithms and Convergence 3h Another formula for computing 7 can be deduced from the identity a /4 = 4 arctan arctan 345, Determine the number of terms that must be summed to ensure an approximation to to within 107° Find the rates of convergence of the following sequences as n — =; + ma sa(?) = win sn(2) = lim sit? () =0 4. Jim lagu + 1) ~ Inf) = 0 sink — cosh l= Tim SAA 0 b. Tim -1 sso moh in tt a. tim SSF 2g son 0) a, How many multiplications and additions are required to determine a sum of the form “no b. Modify the sum in part (2) to an equivalent form that seduces the number of computations, Let PO) = a,x" + Gy.[x"") ++ + asx + dy be 2 polynomial and let xp be given. Construct an algorithm to evaluate P(x) asing nested muttiplication, Example 4 of Section 1.2 gives alternate formulas for te roots x; and x3 of ax? + bx + ¢ = 0. Construct an algorithm with input a, b, ¢ and output x, x2 that computes the roots xy and x, (which mey be equal or complex conjugates) so as to employ the best formula for each. Construct an algorithm that has as input an integer n= I,m + 1 points x5,2),... 4% and a point x and which produces as outpat the product P = ( ~ xp)(x - 2) *+- e ~ > Assume that 2x ioe, ee 142s lteex for x <1, fx = 0.25, write and execute an algorithm that determines the number of terms needed on the left side of the equation so that the left side differs from the right side by less than 10%, ‘The sequence {F,} described by Fy = 1,F, = 1,andF,,. =F, + F,.1, ifn = 0, is called 8 Fibonacci sequence. Is terms occur nacaraly in many botanical species, particularly those ‘with petals or scales arranged in the form of a logarithmic spiral. Consider the sequence fx,}, Where x, = Fyy,/F,- Assuming that lim,... x, = x exists, show that x = (a'+ V5) /2. This namber is called the golden ratio. ‘An equilateral triangle has sides of unit length. Another equilateral triangle is constracted within the first by placing its vertices at the midpoints of the sides of the first triangle. A third equilateral triangle is constructed within the second in the same manner, and so on (see the figure on page 32). a, What is the eum of the perimeters ofthe tiangles? b. Whats the sum of tne areas of the triangles?32 CHAPTER 1 * Mathematical Preliminaries fy 15. A white square is divided into 25 equal squares. One of these squares is blackened. Each of the remaining smaller white squares is then divided into 25 equal squares and one of each is blackened (see the figure). If this process is continued indefinitely, how many sizes of black squares are necessary to blacken a total of at Teast 80% of the original white square? 16. Suppose that as x approaches 220, ‘ FQ) = 1, + 06%) and AR) = L, + OW"). Let c; and ¢, be nonzero constants and define Fe ‘Show that if y = minimum {a, f}, then as x approaches zero, GA@ + oF and GG) = Fea) + Few). a. FQ) = chy took + OG") and ob. GG) = Ly +L + OG"). 1.4 Numerical Software Computer software for approximating the numerical solutions to problems is available in { many forms. In the Student Supplement to this book we have provided programs waitten in Pascal and FORTRAN 77 that can be used to solve the problems given in the examples and exercises, These programs will give satisfying results for most problems that you will likely need to solve, but they are what we call special-purpose programs, We use this term to distinguish these programs from those available in the staudaid mathematical subroutine * libraries. The programs in these packages will be called general purpo:14 Numerical Software ‘The programs in general-purpose software packages differ in their intent from the algorithms and programs provided with this book. General-puxpose software must take into consideration ways to reduce errors due to machine rounding underflow and overflow. It must also describe the renge of input that will be expected to lead to results of a certain specified accuracy. Since these are machine-dependent chrrseteristies, general-pimpose software involves parameters thet describe the floating-point characteristics of the machine being used for calculations. Tp illustrate some differences between programs included in general-purpose pack- ge and a program that we would provide for use in this book, let us consider an algorithm ‘that computes the Euclidean norm of an n-dimensional vector x = (X,,%3.---4X,)- This norm is often required within larger programs and is given by Isl = [Se]. where 2,2, ..., ¥, are either real or complex numbers. The norm gives ¢ measure for the distance from the vector x to the zerer vector. For example, the vector x = (2, 1, 3, —1y has xi) = 2 + 4 Fh + (2% + (17)? = V9 so its distance from 0 = (0, 0, 0, 0, 0) is VI9 ~ 4.36. An algorithm of the type we would present for this problem is given here. It is easy to follow and will give accurate results “most of the time.” This algorithm includes no ‘machine-dependent parameters and provides no accuracy assurances INPUT: 1 4,22,-++ 428 OUTPUT: NORM Step 1 Se SUM = 0 Step 2 Fori = 1,2,...,nsetSUM = SUM + 2. Step 3 Set NORM = SUM*®. Step 4 OUTPUT (NORM); STOR A program based on this algorithm is easy to write and understand and would give Satisfactory 1esults for most problems. Howeves, the program could fail to give sufficient accuracy for a number of reasons, For example, the magnitude of some of the numbers might be too large or too small to be accurately represented in the computer's flozting- point system, Also, the normal order for performing the calculations might not produce the most accurate results, or the standard software square-root routine might not be the best available for the problem. Matters of this type are considered by algorithm designers when writing programs for general-purpose software. These programs are often used as subprograms for soiving larger problems, so they must incoxporate controls that we will not need. Let us uow cousider an algorithm for @ general-puspose software program for com- puting the Euclidean norm, First, it is possible thet although ¢ component x, of the vectorCHAPTER 1 * Mathematical Preliminaries is within the range of the machine, she square of the component is not. This can occur when some |:,| is so small that x causes underflow, or when some || is so large that x} cases over flow. It is also possible for all these terms to be within the range of the machine but for overflow to occur from the addition of a square of one of the terms to the previously computed sum. Since accuracy criteria depend on the machine on which the calculations are being performed, machine-dependent parameters must be incorporated into the algorithm, Sup- pose we are working on a hypothetical computer with base 10, having i digits of precision, ‘zminimum exponent emin, and 2 meximum exponent emax. Then the set of floating-point numbers in this machine consists of zero and the numbers of the form x=f-l0, where f= +(fl0" + 4107? + +++ + £10", 15f<9and0 Sf 59, foreachi = 2,.... 4, and where emin A causes overflow, Whén under flow occurs the prograrn will continue, often without a significent loss of accuracy, but if overflow occurs, the program will fail. The algorithm assumes that the floating-point characteristics of the machine are de- seribed using parameters N, s, S, y, and ¥. The maximum number of entries thet can be summed with 2t least 1/2 digifs of accuracy is given by N. This implies the algorithm will proceed to find the norm of a vector ¥ = (x1, x2... , x_) only if n = N. To resolve the underflow-overflow problem, the nonzero floating-point nmmhers are partiioned into three groups: small-megnitude numbers x, those satisfying 0 < |x| < y; medium- megnitude numbers x, where y = |x| < ¥; and large-magnitude numbers x, where ¥ = |x|, The parameters y and Y are chosen so that there will be no underflow-overflow problem in squering and summing the medium-magnitude numbers, Squaring small- ‘magnitude numbers can cause underflow, so a scale factor S much greater than | is used ‘so that (Sx)? avoids the underflow even when x? does not. Summing and squaring numbers having ¢ large magnitude can cause overflow, so in this case & positive scale factors much smaller than 1 is used to ensure that (sx)? does not cause overflow when calculated or incorporated into e sum, even though x? would. t To avoid unnecessary scaling, y end Y are chosen so that the range of medium- megnitude numbers is as large as possible. The algorithm is e modification of one de- seribed by Brown (20}, page 471. It incorporates a procedure for scaling the components of the vector thet are small in magnitude until a component with metium magnitude is encountered. It then unscales the previous sum and continues by squaring end summing | small and medium numbers until a component with « large magnitude is encountered. ‘Once a component with large magnitude appears, the algorithm scales the previous sum and proceeds to scale, square, and sum the remaining numbers. The algorithm assumes; thet, in transition from small to medium numbers, unsealed small numbers are negligible | when compared to mediam numbers. Similarly, in transition from medium to lerge num- bers, unsealed medium numbers are negligible when compared to large numbers. Thus, the choices of the scaling parameters must be made so thet numbers are equated to zero only when they are truly negligible. Typical relationships between the machine characier- istics es described by & 0, A, emin, emax, and the algotithin parameters N, 5, $, y, and ¥ are given after the algorithm,ats is ng om, 1es ale as, 0 ca iY 14 Numerical Software ‘The algoridlun uses three flags to indicate the Vatious stages in the sumunttion proces. ‘These flags are given initial values in Step 3 of the algorithm. FLAG 1 is 1 until amediam or large component is encountered and then is changed to 0. FLAG 2 is 0 while small numbers are being summed, changes to 1 when @ medium number is fizst encountered, and changes back to 0 when @ large number is found. FLAG 3 is initially 0 and changes to 1 when a large number is first encountered. Step 3 also introduces the flag DONE, which is 0 until the calculations are complete end then changes to 1. INPUT N.8,S.y.¥oRs M15 80000 Sn OUTPUT NORM ur as appropriate erior message Step 1 VE n <0 then OUTPUT (‘The integer m must be positive.’); STOP. Step 2 Tin & Nthen OUTPUT (“The integer m is too Large.”); STOP. Step 3 SetSUM FLAGL FLAG2 = 0; FLAG3 = 0, DONE = 0; i= Stop 4 While (i < n and FLAG} — 1) do Step 5. Step 5 lf|x;| nthensetNORM = (SUM/S)'?, DONI 1 else set SUM = (SUM/5)/5; FLAG2 = 1. Step 7 While (i $n and FLAG2 = 1) do Step 8. Step 8 If |x;| < ¥then set SUM = SUM + x?; iziel else set FLAG2 Step 9 If DONE = O then if > then set NORM = (SUM)"* ; DONE =1 else set SUM = (SUM)s)s FLAG3 = 1. Step 10 While (i Sn and FLAG3 = 1) do Step 11. Stop 11, Set SUM = SUM + (=)? 5 sitl wCHAPTER 1 * Mathematical Preliminaries Step 12, If DONE = 0 then if SUM"? < S$! then set NORM = (SUM / 5)! DONE = 1 else set SUM Step 13. If DONE = 1 then OUTPUT (‘Nom is’, NORM) : else OUTPUT (‘Norm =", NORM, ‘overflow’) é Step 14 STOP. ‘The relationships between the machine cheracteristics f, o, hs emtin, emaz, and the algorithm parameters N, s, $, y, and ¥ were chosen by Brown as follows: N= 10%, — where ey @-2)/ s= 10", where (t — 2)/2], the greatest integer less than or equal to [Cmax + ey) /2; S= 10%, — where eg = [(1, — emin)/2], the smallest integer greater than or equal to (1 — emin) /2; y= 10% where ey = [(emin + t ~ 2)/2}; Y= 10%, — where ey = ((emax ~ ey) /2}. The reliability built into chis algorithm has greatly increased the complexity over the algorithm given earlier in the section. ‘There are many forms of general-purpose numerical software available commercially and in the public domain, Most of the early software was written for mainframe com- puters. A good reference for this software is Sources and Development of Mathematical Software, edited by Wayne Crowell (34]. Now that the desktop computer has become sufficiently powerful, much of the standard numerical software is available for personal computers and workstations. Most of this numerical software is written in FORTRAN 77, although some packages are written in C and Pascal, In Wilkinson and Reinsch {157}, ALGOL procedures were presented for matrix com- putations. A package of FORTRAN subroutines based mainly on the ALGOL procedures was developed into the EISPACK routines. These routines are documented in the manuals published by Springer-Verlag as part of their Lecture Notes in Computer Science series. (See Smith, et al. 139} and Garbow, et a. [60].) ‘The FORTRAN subroutines are used to compute eigenvalues and eigenvectors for a ‘variety of types of matrices. LISPACK is available from Argonne National Laboratory tree of charge. The EISPACK project was the first large-scale numerical software package to be made available in the public domain and led the way for many packages to follow. i LINPACK is a package of FORTRAN subroutines for analyzing and solving various systems of simultaneous Tinear algebraic equations and linear least squares problems. LINPACK is also available from Argonne National Laboratory. The documentation for this package is contained in Dongarra et al [45}. A step-by-step introduction to LINPSCK, EISPACK, and BLAS (Basic Linear Algebra Subroutines) is given in Coleman and Van Loan (32]. The new package LAPACK is a transportable library of FORTRAN 77 subrou- tines that has been designed to supercede LINPACK and EISPACK by integrating these two sets of algorithms into a unified and improved package. The software has been re-14. Numerical Software 37 structured to achieve greater efficiency on vector processors and other high-performance or shared-memory multiprocessors. LAPACK is accessible through netlib for specified subroutines or from NAG for the complete librery, The LAPACK. User's Guide by E, Anderson etal. should be available by the time this book is printed. Other packages for colving specific types of problems are available in the pub- lic domain. Information about these programs can be obtained through electronic mail by sending the line “help” to one of the following Internet addresses: neilib@research att.com, netlib@ornl gov, [email protected], or [email protected] or to the uuep address: uunet!research!neilib. These packages are considered by experts to be highly efficient, accurate. and reliable. They are thoroughly tested and documentation is readily available. Although the packages are portable, itis 2 good idea to investigate the machine dependence and thoroughly read the documentation. The programs test for al- most all special contingencies that might result in erzor and failures. At the end of each chapter we will disenss some of the appropriate general-parpase packages Commercially available packages also represent the state of the art in numerical meth- ods. Their contents are often based on the public-domain packages but inciude methods in libraries for almost every type of problem.” IMSL (international Mathematical Software Library) consists of the libraries MATH, SYAT, and SFUN for numerical mathematics, statistics, and special functions, respec tively. These libraries contain over 700 subroutines written in FORTRAN 77, which solve ‘most common numerical analysis problems. In 1970 IMSL beceme the first large-scale scientific library for mainframes. Since thst time the libraries have been made available for computer systems ranging from supercomputers to personal computers. The libraries are available commercially from IMSL, 2500 Park West Tower One, 2500 City West Boulevard, Houston, TX 77042-3020, The packages are delivered in compiled form with extensive documentation. There is an example program for each routine as well as back- ground reference information. IMSL contains methods for inear systems, eigensystem analsis, interpolation and approximation, integration and differentiation, differential equations, transforms, nonlinear equations, optimization, and basic matrix/vector opera- tions. The library also contains extensive statistical routines. ‘The Namerical Algorithms Group (NAG) has been in existence in the United King- dom since 1971. NAG offers over 600 subroutines in a FORTRAN 77 library for over 90 different computers. Subsets of their library are available for IBM personal computers (the PC50 Libiary consists of 50 of the most frequently used routines) and workstations (the Workstation Library contains 172 routines). The NAG users manual includes instructions and examples along with sample output for each of the routines. A useful introduction to the NAG routines is Phillips {111}. The NAG Ubrary contains routines to perform most standard numerical analysis tasks in 2 manner similar to those in the IMSL. It also includes some statistical routines and a set of graphic routines. The library is commercially avail- able from Numerical Algorithms Group, Inc., 1400 Opus Place, Suite 200, Downers Grove. IL 60515-5702. ‘The IMSL and NAG packages are designed for the mathematician, scientist, or engi- neet who wishes to call high-quality FORTRAN subroutines from within a program. The documentation available with the commercial packages illustrates the typical driver pro- gram required to use the library routines. The next two software packages are stand-alone environments, When activated, the user enters commands to cause the package to solve a problem. However, each package allows programming within the command language. TheCHAPTER 1 * Mathematical Preliminaries incorporated languaye resembles Pescel, but iis mipussible w call external routines which, consist of compiled FORTRAN or C subprograms. ‘MATLAB is a matrix laboratory that was originally Fortran program published in Moler {99}. The laboratory is based mainly on the EISPACK and LINPACK subroutines although functions such as nonlinear systems, numerical integration, cubie splines, curve fitting, optimization, ordinary differential equations, and graphical tools have been incor- porated. MATLAB is currently written in C and assembler, and the PC version of this package requires a numeric coprocessor. he basic structure is to perform matrix opera- tions, such as finding the eigenvalues of a matrix entered from the command line or from. an external file Via function calls. This is a powerful self-contained system that is espe- cially useful for instruction in an applied linear algebra course. A well-written textbook by David Hill [74], with a complete solutions manual, is also available. MATLAB has been. available since 1985 and may be purchased from The MathWorks Inc., Cochituate Place, 24 Prime Park Way, Natick, MA 01760. The MATLAB software is designed to run on many computers, including IBM PC compatibles, APPLE Macintosh, and SUN work- stations. A student version of MATLAB has been recently released that does not require a coprocessor but will use one if it is available.“ ‘The second package is GAUSS, a mathematical and statistical system for IBM pes- sonal computers produced by Lec E. Ediefson and Samuel D. Jones in 1985. It is coded mainly in assembler and based primarily on EISPACK and LINPACK. As in the case of MATLAB, integration/differentiation, nonlinear systems, fast Fourier transforms, and stapbics are available. GAUSS is less oriented toward instruction in linear algebra and more oriented to statistical analysis of data. This package also uses a numeric coprocessor if one is available. It can be purchased from Aptech Systems, Inc., 1914 N. 34th St,, Suite 301, Seattle, WA 98103. ‘There are numerous packages available that can be classified as supercalculator pack- ages for the PC. These should not be confused, however, with the general-purpose soft- ware listed here. If you have an interest in one of these packages, you should read ‘Supercalculators on the PC by B. Simon and R. M. Wilson [136]. In the past decade a number of software packages have been developed to pro- duce symbolic mathematical compvtations. Predominant among them are MACSYMA, DERIVE, Maple, and Mathematica. There are versions of the software packages for most common computer systems and student versions of some are available at very reasonable prices. Although there are significant differences among the packages, both in perform- ance and price, they all can perform standard algebra and calculus operations. Having a symbolic computation package available can be very useful in the study of, approximstion techniques. The results in most of ovr examples and exercises have been generated using problems for which exact values can be determined, since this permits the performance of the approximation method to be monitored. Exact Solutions can often be obtained quite easily using symbolic computation. In addition, many numerical techniques have error bounds that require bounding a higher ordinary or partial derivative of a func- tion. This can be a tedious task, and one that is not particularly instructive once the ‘ectmiques of calculus have been mastered. These derivatives can be quickly obtained symbolically, and ¢ little insight will often permit a symbolic computetion to aid in the bounding process as well besSolutions of Equations in One Variable Fra growth otiarge popsatons can be modeled over sort periods of time by assuming that the popnlation grows continnonsly with time at arate proportional to the nnmber of individnals present at that time, If we let V() denote the number of individnals at time ¢ and let A , bs]. This produces the method described in Algorithm 2.1, (See Figure 2.1.) Bisection Tp find a solution to f(x) = 0 given the continuous function f on the interval [a, b] where F(a) and f(b) have opposite signs INPUT endpoints a, b; tolerance TOL; maximum number of iterations Np. OUTPUT approximate solution p or message of failure. Step1 Seti= 1. Step 2 While i = No do Steps 3-6. Step3 Setp=a+(b-a)/2. (Computep;.) Step 4 Iéf(p) = Oor(b ~ a)/2 O then set a else set b = p. (Compute ay, bj.) Step 7 OUTPUT (‘Method failed after Np iterations, No (Procedure completed unsuccessfully.) STOP MN Listed next are some otber stopping procedures that can be applied in Stop 4 of ‘Aigorithm 2.1, each of which applies to any iterative technique considered in this chapter. Select a tolerance € > 0 and generate p),.... , py until one of the following conditions is met: en pw = Pwarl = & 22) pw = pwail < ¢ py#0, oF tow! 23) Voml No, as was done in Step 2 of Algorithm 2.1 Note that to start the Biseotion Algorithm, an interval [a, b] must be found with F(a) - f(B) < 0. At each step the iength of the interval known to contain a zero of f is reduced by a factor of two; hence it is advantageous to choose the initial interval [a, 5] as small as possibie. For example, if f(x) = 2x3 — x? + x — 1, then F(-4)-f4)<0 and — fF) -f1) <0, so the Bisection Algorithm could be used on either of tite intervals (-4, 4] or [0, 1]. Starting the Bisection Algorithm on (0, 1] instead of [~4, 4] will reduce by three the number of iterations required to achieve a specified accuracy. ‘To illustrate the Bisection Algorithm, consider the foliowing example. The iteration in this exampic is terminated when the relative enor 2 =e < 10 Ip ‘The equation f(z) = x3 + dx* — 10 = 0 has a root in (1, 2] since f(1) = ~5 and £2) = 14 Iis easily seen that there is only one root in [1, 2]. The Bisection Algorithm ives the values in Table 2.1.Table 24 Theorem 2.1 21 The Bisection Method 4B ® a bn Pn fen 1 20 13 2375 2 13 1.25 -1.79687 3 13, 1375 0.1621 4 1.375 1.3125 0.84839 5 1375 1.34375, 0.35098 6 1.375 1.359375, — 0.09641 7 1.359375 1.375 1.3671875 0.03236 8 1.3995/5 13671875 1.36528129 0.03215 9 136328125 1.3671875 1365234375 0.000072 10 136328125 1365234375 1,364257813, 0.01605 n 1364257813 1.365234375 1364746094 —0,00799 12 1364746094 1,365254375 1,364990235, ~ 0.00396 13 1.364990235 1,305234375 1505112505 0.00194 After 13 iterations, ps Ip ~ Pisl <[bia ~ a4 Since |a,4| <|pl, 365112305 approximates the root p with an error = |1.365234375 ~ 1365112305] = 0.000122070. aud 9.0 x 10-4, ii so the approximation is correct to at least four significant digits. The correct value of p, to nine decimal places, is p = 1.365230013. Itis interesting to note that pis closer to p than is the final approximation pys, but there is no way of determining this unless the true answer is known. BEE The Bisection method, though conceptually clear, bas significant drawbacks. It is very siow in converging (that is, W may become quite large before |p — ps is sufficiently small) and, moreover, a good intermediate approximation may be inadvertently discarded. ‘However, the method has the important property that it always converges to a solution and for that reason is often used as a “starter” for the more efficient methods presented later in this chapter. Let f € Ca, b] and suppose f(a) - f(b) < 0, The Bisection method (Algorithm 2.1) generates a sequence {p,} approximating p with the property b-a@ en lp PLES H onel Proof Foreachn = 1, we have Ja) and De (aby barEXAMPLE 2 CHAPTER 2 * Solutions of Bquations im One Vartable Since p, = }(a, + b,), for all n = 1, it follows that bra F According to Definition 1.17, this inequality implies that {p,}j=1 converges to p with rate of convergence O(2~”), since by (2.4) tp, — pl=2@, — 4) Itis important to realize, however, that theorems such as this give only bounds for errors in approximation. For example, this bound applied to the problem in Example 1 ensures only that, ppl s25A~2 x 10°, iP ut the actual error is much smaller: lp — pol = |1.365230013 — 1.365234375|~ 4.4 x 107°, ‘determine the number of iterations necessary to solve f(x) = x? + 4x? — 10 = Owith accuracy € = 10°? using a, = 1 and b; = 2 requires finding an integer N that satisfies Ipy — p| S27" -— a) = 27" < 1077. ‘D determine N, we will use logarithms. Although logarithms to any base would suffice, we will use base 10 logarithms since the tolerance is given as a power of 10, Since 2°" 10° implies that logy 27" < logy 10"? = —3, =Niog2<-3 or N> ~ 9.96. logy 2 Hence, no more than ten iterations are required to obtain an approximation accurate to within 107°. Table 2.1 shows that the value of py = 1.365234375 is accurate to within 1074, It is important to keep in mind that the error analysis gives only a bound for the number of iterations necessary, and in many cases this bound is much larger than the actual numnber required. see EXERCISE SET 21 1. Use the Bisection method to find solutions accurate to within 10°? for x? — 7x? + 14x ~ 6 on a. (0,1) b. 13.2] ce (32,4) 2. Use the Bisection method to find solutions accurate to within 10°? for x* =~ 2x3 ~ 4x? + 440m a [-2-1 be 10,2) ce [23] 4& [-1,03 4 10. 1 om 3B. cee The Bisection Method 45 Use the Bisection method to find a solution accurate to within 10~? for tan xon [4,45]. Use the Bisection method to find a solution accurate to within 10-? for 2 + cos(e* ~ 2) — e* = 0on 05, 1.5} Use the Bisection method to find solutions accurate to within 10-$ for the following problems, a x-2¥%=0 fordsxsl, bo f= 43x-2=0 for0 I,butthat |p — p,| < 10? requires that n> 1000 ‘A trough of length L has a cross section in the shape of a semicircle with radius r (see figure con page 46). When filled with water to within a distance h of the top, the volume V of water is 1) = 0, but (p)\ < 10-8 whenever V = L[0.5ar? — r? aresin (h/t) — AC? - PY?) Suppose 4 = 10 it, 7 = 1 ft, and V = 12.4 ft? Kind the depth of water in the trough to within 0.01 ft.46 CHAPTER 2 © Solutions of Equations in One Variable 16. A particle starts at rest on a smooth inclined plene whose angle @ is increasing et a constant rate Ea an® ‘At the end off seconds, the position of the objectis given by a ee sty (C5E soe) Suppose the particle has moved 1.7 ft in 1 s. Find, to within 10°, the rate w at which 6 changes if wy = —O.5 anda = —0.L. Assume thet g = —32.17 15% AO 2.2 Fixed-Point Iteration A fixed point for a given function g is a number p for which g(p) = p. In this section we will consider the problem of finding solutions to fixed-point problems and the connection Detween these problems and the rovt-finding problems we wish Ww sulve. Root-finding problems and fixed-point problems are equivalent classes in the following sense: Given a root-finding problem f(p) = 0 we can define a function g with a fixed point at p in a number of ways, for example, as g(x) = x — f(a) oF as g(t) = x + 3f(). Conversely, if the function g has a fixed point at p, then the function defined by f(x) = x — g(x) has a zero at p. Although the problems we wish to solve are in the root-finding form, the fixed-point form is easier to analyze and certain fixed-point choices lead to very powerful root-finding techniques. ‘Our first task is to become comfortable with this new type of problem and to decide when a function has a fixed point aud how the fixed points can be deveruined. (La duis sense, “determined” means that it can be approximated to within a specified accuracy) isoribarietetnttinintont2.2 Fixed-Point Iteration - a EXAMPLE 1 a. The function g(x) = x, 0 <2 = 1, bas a fixed point ateach xin (0, 1) b. The function g(x) = x — sin wx bas exactly two fixed points in [0, 1], x = Oand x= 1. (See Figure 2.2. eee Figure 2.2 ata) The following theorem gives sufficient conditions for the existence and uniqueness of a fixed point Theorem 2.2 If g © Cla, b] and g(x) [a, b] for all x € [a, b), then g has a fixed point in (a, b). Suppose, in addition, that g'(z) exists on (a, b) and that a positive constant k < 1 exists with 5) lp'@lsk<1, — forallx < (a,b) ‘Then the fixed point in [a, b] is nique. (See Figure 2.3, page 48.) Proof If g(a) = aor g(b) = b, the existence of a fixed point is clear. Suppose nor; then it must be true that g(a) > a and g(b) <>. Dofine h() ~ g(s) ~ = Then h is continuous on [a, 6), and Ha) = g(a) ~ a>0, —WQ) = 6) -— b<0. The Intermediate Value Theorem implies that there exists p € (a, b) for which h(p) = 0. Thus, g(p) — p = Uand pisa fixed point of g. Suppose, in addition, that inequality (2.5) holds and that p and q are both fixed points in [a, b] with p » g. By the Mean Value Theorem, a nunber Eexists between p and g, and hence in (a, b), with a z a = #'&.48 CHAPTER 2 * Solutions of Equations in One Vartable Figure 2.3 ‘Then Ip ~ a] = |g(p) ~ 8@)| = |8'@)| |p — al Alp ~ al <|p - ab which is a contradiction. This contradiction must come from the only supposition, p # ¢. Hence, p = g and the fixed point in (a, b] is unique. see EXAMPLE 2 a, Let g(x) = (x? ~ 1)/3 on [~1, 1]. Using the Extreme Value Theorem, it is easy to show that the absolute minimum of g occurs at x = O and g(0) = ~} Similarly, the absolute maximum of g occurs atx = +] and has the value (£1) = 0. Moreover, g is continuous and | 22 3 '@l = + forall [-1, 1, a so g satisfies the hypotheses of Theorem 2.2 and has a unique fixed point in {-1,1. In this example, the unique fixed point p in the interval [~1, 1} can be determined algebraically. If L avai P= atp) 7 then =p? ~ 3p -1 which, by the quadratic formula, implies that =16- VB. i i i i Note that g also has a unique fixed point p = }(3 + V13) for the interval [3, 4]. However, g(4) = 5 and g/(4) = $> 1; so g does not satisfy the hypotheses of ‘Theorem 2.2. This shows that the hypotheses of Theorem 2.2 are sufficient to | guarantee a unique fixed point, but are not necessary. (See Figure 2.4.}(Sian Serine ee 2.2 Fixed-Point Iteration 49 Figure 2.4 | b. Let ga) = 3°, Since g'G) = —3°* In 3 <0 on (0, 1], the function g is decreasing on {0, 1}. Hence, (1) = $< g(x) =1 = g(0) for0 Sx = 1. Thus for x € (0, 1], g(x) € [0, 1], and g has a fixed point in (0, 1}. Since g'() = —In3 = ~1098612289, \g’@)| 1 on {0, 1] and Theorem 2.2 cannot be used to determine uniqueness. However, g is decreasing so itis clear that the fixed point must be unique. (See Figure 2.5.) Bog Figure 2.5 i yCHAPTER 2 * Solutions of Equations in One Variable “To approximate the fixed point of a function g, we choose an initial approximation pp and generate the sequence {p,}*-0 by letting p, ~ §( 1-1) for each n > 1. If the sequence converges to p and g is continuous, then, by Theorem 1.4, p= lim p, = lim g(p,-1) = g{ lim py-1) = ap). and a solution to x = g(x) is obtained. This technique is celled fixed-point iteration, or functional iteration. The procedure is detailed in Algorithm 2.2 and illustrated in Figure 26. yh an 7) ae Ps g(a) uP) en) P2* 80) upd Tg rim ald) y= 8) - + Pi Ps Pr Po = @ ® fo auconrran { 2.2 Fixed-Point iteration Th find a solution to p = g(p) given an initial approximation pet INPUT initial approximation po; tolerance TOL; maximum number of iterations No. OUTPUT approximate solution p or message of failure. Step 1 Seti = 1. Step 2. While i = No do Steps 3-6. Stop 3 Setp = g(pq). (Compute pj.) Step 4 \f\p — po| < TOL then OUTPUT (ps (Procedure completed successfully.) STOP. Step 5 Seti Step 6 Setpy=p. (Update py.) +1 ess{ ' | | en eniaten ota sown EEnEE EXAMPLE 3 2.2 Fixed-Point Iteration Step 7 OUTPUT (‘Method failed after No iterations, Np = *, No)s (Procedure completed unsuccessfully.) STOP, ‘To illustrate the technique ot tunctional iteration, consider the following example. The equation x* + 4x* — 10 = Ohas unique root in (1, 2). There are many ways to change the equation to the form x = (x) by simple algebraic manipulation, For example, to obtain the function g described in (¢), we can manipulate the equation x? + 4x7 — 10 = 0as follows: 4x? = and ‘To obtain a positive solution, ga(x) is chosén-as shown, It is not important to derive the functions shown here, but you should verify that the fixed point of each is actually a solution to the original equation. a x= giQ) =x - 4274 10, A =a) =} 00-24, a v= = ( 2) ; e xe gio ax- 2H With po = 1.5, Table 2.2 lists the results of the fixed-point iteration method for all five choices of g n @ © © © © ° 1s 1s 15 1s 15 1 -0.875 0.8165 1.286953768 1348399725 1373383333 2 6.732 2.9969 1,402540804 1367376372 1365262015 3 -469.7 8.65)" 1345458374 1.3649570151.365230014 4 1.03 x 10° 1375170253 1365264748 1365230013, 5 1360094193 1365225594 6 1,367846968 1365230576 7 1.363887004 1,365229942 8 1.365916734 1.365230022 9 1364878217 1.365250012 10 1.3654100621.365230014 5 1.365223680 1365230013 20 1,365230236 5 1.365230006 30 1.36523001352 CHAPTER 2 * Solutions of Equations in One Variable ‘The actual root is 1.365230013, as was noted in Example 1 of Section 2.1. Comparing the results to the Bisection Algorithm given in that example, it can be seen that excellent results have been obtained for choices ((), (4), and (@), since the Bisection method requires 27 iterations for such accuracy. It is interesting to note that clioice (a) led to divergence | and that (b) became undefined because it involved the square root of a negative number. ao oo Even though the various functions in this example are fixed-point problems for the same ryot-fiuding problem, they differ vastly as techniques for approximating the solution to the root-finding problem, Theit purpose is to ilustrate the true question that needs to be answered: How can we systematically determine fixed-point problems that rapidly approximate the solution to a given root-finding problem? The following result gives us some clues concerning the paths we should pursue and, perhaps more importantly, some we should reject. Theorem 2.3 (Fixed-Point Theorem) : Let g ¢ Ca, b] and suppose that g(x) € (a, b] for all x in (a, b]. Suppose, in addition, that g' exists on (a, b) with 2.6) le)|=k<1, forall (ab). Ifpois any number in (a, b], then the sequence defined by Po = Pu» nel F converges to the unique fixed point p in fa, 6]. : Proof By Theorem 2.2, a unique fixed point exists in fa, 6]. Since g maps [a, b] into itself, the sequence {p,}i7_9 is defined for all n = 0 and p,, € [a,b] for all n. Using inequality (2.6) and the Mean Value Theorem, we have 1) |p — PL = [e(Pn-a) — 8€P)| = 18'©)I [Pot — Pl = leas = Ph where Le (a,b), Applying this inequality inctively gies: 2.8) (Pn — PLS lp — P] SF [Pen — pis ++ Sk" ~ Ph : Sincek <1, : lim |p, — p| = lim "| p9 — p| = i and {p, Yq Converges to p. tte i Corollary 2.4 If g satisfies the hypotheses of Theorem 2.3, bounds for the error inyolved in using p, to approximate p are given by lpn — p| Sk" max {pp — a, b ~ pot » and Ip. ~ Pp } bPo for alln = 1. |2.2. Fixed-Point Iteration 53 Proof The first bound tollows trom inequality (2.8): [Pn — pl =k" |po — p] =k" max {po ~ a. — poh sincep {a,b} For n = 1, the procedure used in the proof of Theorem 2.3 implies that Iasi ~ Pal = [e(Pd — 8 Pad Seip Pr-al S++ Sk" Ip, = pol. Thus, form> n= 1, [Pe ~ Pal = Pm ~ Peat + Pat = 11+ + Past — Pal [Pm Pmoil + nar Bmeal + °° + oer — Pal Sk 'lpy — pol + Alps — pol +--+ + HIP — Pol SRL tke Re ep, — pol By Theorem 2.3, lim, . Py, = P, SO Ip — pal = tim py ~ Pal =p: — pol SF Since > k’is a geometric seties with constant ratio k, we have the second bound: . ee \ ype Pol Both inequalities in the corollary relate the rate at which {p,,} coverges to the bound k on the first derivative. The rate of convergence depends on the factor k”/ (1 — 1); the smaller f, the faster the convergence, The convergence may be very slow if kis clase 70 1. In the following example, the fixed-point methods in Example 3 are reconsidered in light of the results described in Theorem 2.3. @ When g(x) = x — x3 — 4r* + 10, ei(x) = 1 — 3x° — 8x. There is no interval [a, 5] containing p for which | g{(x)| < 1. Although Theorem 2.3 does not guar- aniee that the method must fail for this choice of g, there is no reason to expect convergence, 1b. With go(x) = ((10/x) — 4x]*, we can see that gp does not map [1, 2] into (1, 2] and the sequence {p,}5.o Is not defined with po = 1.5. Moreover, there is no interval containing p such that [a@Q/<1, since |2i(p)| = 3.4. For the Luuetion 339 = 3 (10 — 2°)", ao) = -b?a0-<4y <0 on[l. 2h, 80 g3 is strictly decreasing on [1, 2]. However, |g5 (2)] ~ 2.12, so inequality (2.6) doee not hold on (1, 2]. A clover examination of the cequence {p,}7ag starting with py = 1.5 shows that it suffices to consider the interval [1, 1.5] instead of54 CHAPTER 2 * Solutions of Equations in One Variable [1, 2]. On this interval it is still true that gi(x) < 0 and g; is strictly decreasing, but, additionally, 1< 1.28 » gs(1.5) = gu(x) = gA(l) = 15 for all x < [1, 1.5]. This shows that g, maps the interval [1, 1.5] into itself. Since itis also true that [g{(x)| = |gh(1.5)] ~ 0.66 on this interval, Theorem 2.3 con- firms the convergence of which we were already aware. o a. rersie = ( 10 ) ; Tee Jex| = |] < 2 <01s, foratre 11,21 Ao" Wo + 8] Vio? mead ‘The bound on the magnitude of gi(x) is much smaller than the bound on the magnitude of g/(2), which explains the more rapid convergence using gy. The other part of Example 3 can be handled in a similar manner. neg EXERCISE SET 2.2 1 5. Use slgcbraie maniptlation to show that cach of the following fumetions has a fixed point at p precisely when f(p) = 0, where f(x) =x + 2x7 ~ x= 3. 1 +3 7 a 9,0) = +x 20 be gts) = p=] ia _fx43 oy aoe tet? eee [a al OO "Be aem I a, Perform four iterations, if possible, on each of the functions g defined in Exercise 1, Let Po = 12nd Pya, = g(p,) forn = 0, 1,2,3 b. Which fanction do you think gives the best approximation to the solution? ‘The following three methods are proposed to compute 211°. Rank them in order, based on ‘their apparent speed of convergence, assuming pp = 1. 2p. +21 fr Boa = 21 7 py = Meet +2 phas b De Poo Pins = phen 2d ‘The following four methods are proposed to compute 7"°. Rank them in order, based on their apparent speed of convergence, assuming pp = |. (+) 7 Py = Py — 12 Use a fixed-point jteration method to determine a solution accarate to within 10°? for 2sin ae + x= Oon [1,2]. Use pp = 122 10. 1 2 1% Fored-Point jteration Solve x? ~ 2 — 1 = O for the soot in [1,2], using fixed-point iteration, Obtain an approxi- mation to the soot accurate 10 within 10°” Use Theorem 2.2 to show that g(x) = or + 0.5 sin xhas a unique fixed point on (0, 2-7]. Use fixed-point iteration to find an approximation t» the fixed point that is accurate % within 102, Use Corollary 2.4 to estimate the number of iterations required to achieve 107? accu- racy, and compare this theoretical estimate to the number actually needed. ‘Use Theorem 2.2 to show that g(x) = 27 has a unique fixed point on [4, 1], Use fixed-point iteration to find an approximation to the fixed point accurate to within 10™*, Use Corollary 2.4 to estimate the number of iterations required to achieve 10~* accuracy, and compere this theoretical estimate to the number actually needed. ‘Use a fixed-point iteration procedure to find an approximation to ‘V3 that is accurate to within 10-*. Compare your result and the number of iterations required with the answer obtained in Exercise 8 of Section 2.1 Use 2 fixed-point iteration procedure to find an approximation to ¥/25 that is accurate to within 10°*. Compare your result and the number of iterations required with the answer obtained in Exercise 7 of Section2.1, For each of the following equations, determine an interval (a, b] on which fixed-point itera~ tion will converge. Estimate the number of iterations necessary to obtain approximations accurate to within 10°, and perform the esleulations. 2- er ex a eet ete x= (e/3! e ree fre OSiine + coe) For each of the following equations, determine a function g and an interval (a, 6] on which fixed-point iteration will converge to a positive solution of the equation. a) b. x cox =0 Find the solutions to within 10° Find all the zeros of f (2) = x* + 10 c08 x by using the fixed-point iteration method for an appropriate iteration function g. Find the zeros accurate to within 10~¢ ‘Use a fixed:point iteration method to determine a sotution accurate to within 107* for xatnx4sx<5 Find a function g defined on (0, 1] that satisfies none of the hypotheses of Theorem 2.2, but still has a unique fixed point on [0, 1]. a, Show that Pheorem 2,2 is true if inequality (2.5) is replaced by g(x) =k <1 forall x6 (@,8), b, Show that Theorem 2.3 may not hold if inequality (2.6) is replaced by the hypothesis in part(a). ‘a, Use Theorem 2.3 to show that the sequence defined by 1 Xp = Fee in = tend forn= 1, converges to V2 whenever x9 > V2. Db, Usethe fact that 0 < (xq — V2) whenever xy ~ V2 to show that if 0 V2. ‘Use he results ef pasts (4) aun (Q) uo slow that the sequeuce i (a) wouverges uo V2 whenever xq > 0.56 CHAPTER 2 = Solutions of Equations in One Variable 18, a, Show thatif A is any positive number, then the sequence defined by i n= pHa + 5 forn= L converges to VA whenever xy > 0. b, What happens ifx» <0? 19, Replace the assumption in inequality (2.6) of Theorem 2.3 with ‘g satisfies a Lipschitz condition on the interval (a, 6] with Lipschitz constant L < 1." (See Exercise 23, Section 1.1.) Show that the conclusions of this theorem are still valid, : 20, Suppose that g is continuously differentiable on some interval (c, d) that contains the fixed point p of g. Show that if [g'(p)| <1, then there exists a 6 > 0 such that the fixed-point iteration converges for any initial approximation pp, whenever [pp - p| =. 21, An object falling vertically through the air is subjected to viscous resistance as well as to the forse of gravity. Assume that an object with mass m is dropped from a height Sp and thet the height of the object after r secontis is eq em, , where g = —22.17 fs! and k represents the coefficient of air resistance in Ib-sift. Suppose Sq = 300 ft, m = 0.25 fb, and k = 0.1 1b-s/ft. Find, to within 0.01 , the time it takes this quarter-pounder to hit the ground. 22, Let g © Cla, b] and suppose g' exists on (a, b). Let p be in (a, b) with g(p) = p and |2'(p)| > 1. Show that there exists a & > 0 such that if 0 < [pp —.p| < 6, then po ~ pl <[ps ~ pl. Thus, no matter how close the initial approximation po is top, the next iterate pis farther away, 50 the fixed-point iteration does not converge if py = p. = 5 48 50 = & +7 2.3 The Newton-Rapbson Metho ‘The Newton-Raphson (or simply Newton’s) method is one of the most powerful and well-known numerical methods for solving a root-finding problem f(x) = 0. There are ‘many ways of introducing Newton's method; the most common is to consider the technique graphically. Another possibility is to derive Newton's method as a simple technique to obtain faster convergence than offered by other types of functional iteration. This will be done in Section 2.4. A third means of introducing Newton's method, which is discussed next, is based on Taylor polynomials. Suppose that fe C? [a, b). Let¥ € (a, b] be an approximation to p such that f’() # O and | 5 — pis “smell.” Consider the first Taylor polynomial for f(2) expanded about, =x 2 FM =fO+e-D/O|+ LEG), where (x) lies between x and x. Since f(p) this equation, with x — p, gives | | a 29) 0=F@ + @—af'@ +2 pep.itz on ed int he he se ea 3 a x a nd. en at d 2.3 23. The Newton-Rapbson Metbod 57 Newiou's aiethod is devived by assuming thal, since |p — 5 is Small, the tenn invelving (p> is negligible and that 2.0) O~f@ + (p— PF". Solving for p in this equation gives 7-28. pee re ‘This sets the stage for the Newton-Raphson method, which starts with an initial approxi- mation po and generates the sequence {p,), defined by mic) Fo, Figure 2.7 dlustrates bow the approximations are obtained using successive tengents. (See also Exercise 7.) Starting with the initial approximation pg, the approximation p, is the x-intercept of the tangent line to the graph of f at ( pp, foo). The approximation p, is the x-intercept of the tangent line to the graph of f at (p:, f(p))), and so on. Algorithm 2.3 fellows this procedure. Qi) Pr = Prot Slope f"¢p1), vf) _- Slope "(pa i (Cofpad) Newton-Raphson Tp find a solution to f(x) = C given an initial approximation po: INPUT initial approximation pp; tolerance TOL; maximum number of iterations No. OUTPUT _ approximate solution p or message of failure.58 CHAPTER 2 * Solutions of Equations in One Variable Step Seti= 2 Step 2 While i = No do Steps 3-6. = ts Step 3 Setp = po — Fipad/F'(po). (Compute p.) Step 4 IE|p — pol < TOL then OUTPUT (ps Procedure completed successfully.) t STOR, e id StepS Saimitl £ Step6 Setpy=p. (Update po.) = Step 7 OUTPUT (‘Method failed after No iterations, Np = *, Nos (Procedure completed unsuccessfully.) STP The stopping-technique inequalities given with the Bisection method are applicable to Newton’s method. Thatis, select a tolerance € > 0 and construct p,, ...,pyuntiL (2.12) Ipy ~ pral<é 3) + Bum pul cg py sH0, : lew! 7 ee z (2.14) Soom 0 such that Newton's method generates a sequence {p,}§_ converging to p for any initial approximetion pyc [p — 8p + 8]. Proof The proof is based on analyzing Newton's method as the functional iteration scheme Py = 8(Pq-1), for n = 1, with fe) fey The objectis to find, for a value k in (0, 1), an interval [p — 6, p + 8] such that g maps the interval [p — 8,p + 8] into itself and |g'(z)| Sk< 1 forxe [p - 6p + 6) Since f'(p) # 0 and f" is continuous, there exists 5, > 0 such that f" (x) + 0 for x e [p — 6p + 6) Cla, bl]. Thus, g is defined and continuous on [p — 6,p + 6). Also, FOF OQ = fore _ fore [fear [Fr@r tor x [p — yp + oJ; since fe C?[a, b], we have g € C'lp — dp + di). BY assumption, f(p) = 0, 80 gy x e'@) _ fF") FO TPF Fitemenceoia aN NRleR NY : i ‘ ' | | | 23. The Newton-Rapbson Method 6h Since g' is continuous, this implies that for any positive k <1 0<6<6,,and there exists a 6, with \e'@W| Sk forzelp- Bp + Jl. Ttremains to show that g: [p — 8p + 8] > [p — 8p + O]. xe (yp — 6p + 8), the Mean Value Theorem implies that, for some number € between x and p, |g(x) — g(p)| = |g'(O) |x - pl. So le@) — P| = |e) — a¢P)| = [9 lz - pl = klx — pl <|x— al. Since x © [p — 8p + 6}, it follows that |x — p| < 6 and that |g(z) — p| < 6 This implies g: [p — 6.p + 6] > [p — 6, p + 8). ‘All the hypotheses of the Fixed-Point Theorem are now satisfied for g(x) = x — £0) /f' G), s0 the sequence {p,}"_ defined by = Pp, forn = 1,2,3, converges to p for any po€ [p - 8,p + 4] een ‘Theorem 2.5 states that, under reasonable assumptions, Newton's metbod converges provided a sufficiently accurate initial approximation is cbosen. It also implies that the constant k that bounds the derivative of g, and, consequently, indicates the speed of con vergence of the method, decreases to zero as the procedure continues, Newton's method is an extremely powerful tecbnique, but it has a mejor difficulty: the need to know the value of the derivative of f at each approximation, Frequently f"(x) is, far more difficult and needs more arithmetic operations to calculate than f(x). As a simple example, consider f(x) = x73* cos 2x, Then f"(x) = 2x3* cos 2x + x°3* (cos 2x) n 3 — 2x°3* sin 2x, which is tiresome to evaluate. Tp circumvent the problem of the derivative evaluation in Newton's method, we derive a slight variation. By definition, S'Pnatd = 2.15) Pa = Prot — Treen Pad) a) not The technique using this formula is called the Secant method and is presented in Algo- rithm 2.4. (See Figure 2.10, page 62.) Starting with the two initial approximations p, and i the approximation pz is the x-intercept of the line joining (pe. f(p9)) and (;,.FCpi)) The approximation ps is the x-intercept of the line joining (p;, f(p;)) and (p>, fp). and so on,62 Figure 2.10 f i I ArconiTas. 24 CHAPTER 2 * Solutions of Equations in One Variable *t Ps Po_P2 Pe rx Secant To find a solution to f(x) = 0 given initial approximations po and p;: INPUT _ initial approximations po, p1; tolerance TOL; maximum number of iterations No, OUTPUT approximate solution p or mescage of failure. Step 1 Seti = 2; 40 = F (Pos a= fed Step 2. While i Noo Steps 3-6 Step 3 Setp = pi — ax(Pi — Pod/ (i ~ a). (Compute p,.) Step 4 Tf |p ~ p,| < TOLthen OUTPUT (py; (Procedure completed successfully.) STOP. Step5 Seti= itl Step 6 Setpo= pis (Update py qo Pu 4) = U5 Pi= Pi a = FCP). Step 7 OUTPUT (‘Method failed after No iterations, No ~ ’, No)s (Procedure completed unsuccessfully) STOP. ‘The next example involves a problem considered in Example 1(e), where we used Newton's method with po = 77/4. sensei src ler nm ipl aa bMPLE 2 ‘able 2.4 23. The Newton-Raphson Method 63 Find a zero of f(x) = oo x — x, using the Secant method. In Example 1 we used the initial approximation py = 7/ 4. Here we need two initial approximations, Table 2.4 lists the calculations with pp = 0.5, p; = 77/4, and the formula (y-1 ~ Pr-2XCO8 Pn-1 = Pri) 08 Py=1 — Pri) ~ (COB Pye Pa = Pro from Algorithm 2.4. ene ” bn 0 0. 1 07853981685 2 0.7363841388 3 0.7390381992 4 0.7390851493, 5 0.7390851332 By comparing the results here with those in Example 1, we see that ps is accurate to the tenth decimal place, It is interesting to note that the convergence of the Secant method is slightly slower in this example than that of Newton's method, which obtained this degree of accuracy with ps. This is generally true. (See Exercise 12 of Section 2.4.) ‘The Secant metaod (or Newton's method!) is often used to refine an answer obtained by another technique, such as the Bisection method. Since these methods require a good first approximation but generally give rapid convergence, they serve this purpose well. Each successive pair of approximations in the Bisection method brackets a root p of the equation; that is, for each positive integer m, a root lies between p, and p,_. This amplies that for each n the Bisection method iterations satisty Pe ~ BI<|Pa~ Pr-ah which provides an easily calculated error bound for the approximations. Root bracketing {is not guaranteed for either Newton's method ar the Secant method. Table 2.3 contains results from Newton's method applied to f(a) = cos x — x, where an approximate root was found to be 0.7390851332. Notice that this root is not bracketed by either po, p, of Pi, Pr» The Secant method approximations for this problem are given in Table 2.4. The initial approximations pp, p; were chosen to bracket the root, but the subsequent pairs of approximations p,, po and pz, ps fail to do So. The method of False Position (2lso called the Regula Falsi method) generates ap- proximations in the same manner as the Secant method, but it provides a test fo ensure that the root is bracketed between successive iterations. We first choose initial approximations Po and p; with f( po} - f(p;) < 0. The approximation p, is chosen in the same manner as in the Secant method, as the x-intercept of the line joining ( po, f(po)) and (p, f(p;)). To decide which secant line to use to compute ps, we check f(p2) ° f(p,). If this value is negetive, then py, pz bracket a root and-we choose p, as the x-intercept of the line joining (p;, f(p))) and (p2, f(p2)). If not, we choose p3.as the 2-intercept of the Tine Joining (po, f(pp)) and (p>, f(p2)) and then interchange the indices on pp and py. Ina64 CHAPTER 2 = Solutfons of Equations tn One Variable § similar unanmer, once ps is found, the sign uf ((p3) ° ((p2) determines wheter we use p, and ps or ps and p, to compute p,. In the latter case e relabeling of p» and p, is performed. The relabeling ensures thet the root is bracketed between successive iterations. The process is described in Algorithm 2.5 and Figure 2.11 shows how the iterations can differ from those of the Secant method. In this illustration, the first tbree approximations are the same, but the fourth approximations differ. Figure 2.11. Secant method ‘Method of False Position y=f) y= fd G ancontum —Meibod of False Position 25 ‘To find a solution to f(x) = 0 given the continuous function f on the interval (po, p1) > where f(pq) and f(p:) have opposite signs: = INPUT endpoints po, ps; tolerance TOL; maximum number of iterations No. i be k i i m OUTPUT approximate solution p or message of failure. fl Step1 Seti=2; ui go = FPos ‘ ie a = f(p)- : a i Step 2 While i = No do Steps 3-7. | i Step 3 Setp = pr — gulps — po)/ Gs ~ 40). (Compute p,.) ! i Step4 |p — pi] 0 on [a, 8]. Further, suppose there exists one p € [4, 6] such that S(B) = 0.Let po € (a, b) be arbitrary. Let p, be the point at which the tangear ine to fat (bo. (pe) crosses the z-axis. For each m = 1, let p, be the x-intercept of the line tangent to f at(Dy-s,f (Pe-1)). Derive the formula describing this method. Use Newton's method to solve the equation 1 xsine ~3e082e with po = 2 Tterate using Newton's method until an accuracy of 10~> is obtained, Explain why the results seem unusual for Newton’s method, Also, solve the equation with pp = Sar and po = 1077. The fourth-degree polynomial Fle) = 230x4 + 18x? + 9x? — 291x = 9 haas two real zer05, one in [~ 1, 0] and the other‘in [0, 1]. Attempt to approximate these zeros to within 10-€ using a, the method of False Position b. the Secantmethod ¢. the Newton's method Use the endpoints of each interval 2s the initial approximations in (a) and (b) and the mid- points as the initial approximation in (c). ‘The function f(a) = tan 4x — 6 has a zero at (1/77) arctan 6 ~ 0,447431543, Let pp = 0 and p, = 0.48 and use ten Hterations of each of the following methods To approximate this root. Which method is most successful and why? a. theBisection method b. the method of FalsePosition e. the Secant method ‘The iteration equation for the Secant method can also be written in the simpler form _ fPactPaen ~ fPaea)Peor L(Pu-) ~ F(Pa-2) Explain why, in general, this iteration equation is likely to be less accurate than the one given 4m Algorithm 2.4 ‘The equation x4 — 10 cos x = 0 has two solutions, 13793645 and — 1.3793646. Use Newton's method to approximate the solutions to within 10° with the following values of Do: a. py = ~100 bpp = ~50 p= -25 a ps py = 50 £ py — 100 Use all applicable methods to find all solutions accurate to within 10° for the problem x? + 10cos r= 0. Use all applicable methods to find four solutions accurate to within 10° for the problem 4xcos 2x — (x — 2) = 0, ford Sx 8. Pn perecinseetitnmeteet5 hr isha aR dR i | x ¥ : : 16. WW. 18 19. 20. The Newton-Rapbson Method 67 ‘The function desesibed by f(a) = In @? + 1) = ef cos ux hes a infinite uuuber of ze08. ‘8, Determine, within 10°, the only negative zero. b, Determine, within 106, the four smaliest positive zeros €. Determine a reasonable initial approximation to find the nth smallest positive zero of f [Hint: Sketch a rough graph of f] 4, Use part {c) to determine, within 10°, the 25th smallest positive zero of f Find an approximation for A, accurate to within 10°¢, for the population equation 1,564,000 = oe ,000,000e" + -, x ‘iscussed in the introduction to this chapter. Use this value to predict the population at the fend of the second year, assuming that the immigration rate during this year remains at 435,000 individuals per vear. ‘The sum of two numbers is 20. If each number is added to its square root, the product of the two sums equals 155.55. Determine the owo numbers to within 10~* ‘The accumulated value of a savings account based on regular pesiodic payments can be decermined from the annuity due equation, Pe A=f(+p"- 1. In this equation A is the amount in the account, P is the amount regularly deposited, and 1 is the rate of interest per period for the n deposit periods ‘An engineer would like to have a savings account valued at $75,000 upon retirement in 20 years and can afford to put $150 per month toward this goal. What is the minimal interest fate at which this amount can be deposited, assuming that che interest is compounded monthly? Problems involving the amount of money required to pay off a mortgage over a fixed period. of time involve a formula, aq=l0-a+9", mown as an ordinary annuity equation. In this equation A is the amount of the mortgage, P is the amount of each payment, and / is the interest rate per period for the m payment petiods. ‘Suppose that a 30-year home mortgage in the amount of $75,000 is needed and that the borrower can affard house payments of at most $625 per month. What is the maximal interest rate the borrower can afford to py’? ‘A drug administered to a patient produces a concentration in the blood stream given by o() = Ate mg/ml, thours afterA units have been injected. The maximum safe concentra tion is 1 mg/ml. a. What amount should be injected ¢o reach this maximum safe concentration and when does chis maximum occu? b, An additional amount of this drug is 10 be administered co the patient after the concen- tration falls to 0.25 mg/ml. Determine, to the nearest minute, when this second injection should be given68 CHAPTER 2 © Solutions of Equations in One Variable 21. 2. 23. ce. Ascuming that the concentration from consteutive injections ic additive and that 75% of the amount originally injected is administered in the second injection, when is it time for the third injection? ‘The logistic population growth model is described by an equation of the form Ea To ce** PQ) where P,, ¢, and & are constants and P() is the population at time ¢, P, represents the limiting value of the population since lim,..« P(0) =P, provided that k > 0. Use the census data for the years 1950, 1960, and 1970 listed in the table on page 95 to determine the constants Pz, ¢, and k for a logistic growth model. Use the logistic mode! to predict the population of the United States in 1980 and in 2000, assuming t = 0 at 1950. Compare the 1980 prediction to the actual value. The Gompertz population growth model is described by PO = Pye, where P;, ¢, and k are constants and P(}) i$ the population at time z. Repeat Exercise 21 using the Gompertz growth mode! in place of the logistic model. Player A will shut out (win by a score of 21-0) a player B in » game of racquetball with probability puite(_p_\" 2 \i-pt ep) where p denotes the probability A will win any specific rally (independent of the server) (See Keller (88}, p. 267.) Determine, to within 10°?, the minimal value of p that will ensure that A will shut out B in atleast half the matches they play. In the design of terrain vehicles, itis necessary to consider the feilure of the vehicle when ‘attempting to negotiate two types of obstacles. One type of failure ie called hang up failure ‘and typically occurs when the vehicle attempts to cross an obstacle that causes the bottom of the vehicle to touch the ground (or the obstacle). The other type of failure is called nose- in failure and commonly occurs when the vehicle descends into a ditch and its nose touches the ground. ‘The following figure, adapted from Bekker [12], shows the components associated with the nose-in failure of a vehicle, In that reference it is shown that the maximum angle «, which can be negotisted by 2 vehicle when A is the maximum engle at which hang-up failure does nor occur, satisfies the equation Asina cos @ + Bsin’ a — Ccosa- Esina where A= sin By, B= 100s By, C= (h + 05D) sin B, ~ 05D tan By, (h + 05D) cos B, ~ OSD. a. It is stated that when = 89 in,, = 49 in., D = 55 in,, and B, = 11.5%, angle ais approximetely 33°. Verify this result. 1b. Find efor the situation when /, h, and By are the same as in part (2) but D = 30 in, il anata 4 t t 4 } : t ft 3 i i | | | | |Pee ences tatiana iit 23 25. 26. g The Newton-Rapbson Metbod In g paper entitled “Holdup ard Axial Mixing in Bubble Columns Containing Screen Cylin- ders” [29], B. H, Chen computes the gas holdup in a gas-liquid bubble column by first approximating the quantity < __Sysin S, SEM a 23 pts eo | - OM 4], where t and M are physical parameters and the S,"s are the smallest values (in magnitude) satisfying Sr a S,tan (5) a se(&) a, Assuming M = 37, find S,, Sy, 5s, and S, Db. Use the results in part (a) to approximate the sum in Eq. (1) when t = 0. M, — when nis odd, and =M, when nis even. “To fine approximations for the tension T and angle of inclination from the horizontal at a particular point on a cable or pipeline run underwater, a number of equations of the form _1 | Tosin by | Tyc0s by “I, -_ ® 1= [(Tosin 6 ~ F)? + (To.c0s by ~ GI? must be solved for ¢ and T (see Wang [152]). The functions F and G in (1) and (2) both involve ¢ and have the form a o i g [S(o) sin do — fe) sin d + gly) COs by + KO) cos GI — ws and ~FLMby) 008 bp + $06) 208 6 + sXe) sin by + (6) sin 4] for given tangential and normal hydrodynamic loading functions f and g. Suppose that dp = 77/2, s = 0.1, Ty = 2, w = 1, and the loading functions f and g are given by f@) = Find an approximation to ¢ atarting with initial approximations clove to a /2. What type of accuracy for 6 is sufficient? .02cos@ and gd) = 0.98sin? d + 0.02 sind.70 2.4 Error Analysis for Iterative CHAPTER 2 © Solutions of Equations in One Variable Methods Definition 2.6 EXAMPLE i This section is devoted (o investigating the order of convergence of functional iteration schemes and, as a means of obtaining rapid convergence, rediscovering Newton’s method. ‘We will also consider ways of accelerating the convergence of Newton's method in special circumstances. Before these methods can be presented, we need a procedure for measuring how rapidly a sequence converges. Suppose {p,}2.9 is a sequence that converges to p. If positive constants A and a exist with then {p,}n0 is said to converge top of order a, with asymptotic error constant A. An iterative technique of the form p, = g(p,-s) is said to be of order a if the sequence {p,,} converges to the solution p = g(p) of order a In general, a sequence with a high order of convergence converges more rapidly than a sequence with a lower ordet. The asymptotic constant affects the speed of convergence bbut is not as important as the order. Two cases of order are given special attention. 1. Ifa = 1, the method is called linear. 2. Ifa = 2, the method is called quadratic. ‘The next example compares a linearly convergent method to one that is quadratically convergent aud demonstiates why we will Le tying to find higher-order convergent ‘methods. Suppose {p,} and (,} ate sequences converging o zero, {p,) i linear with vim Pett 9 5, ae [Dal and {f,} is quadratic with the same asymptotic error constant, tim Weal — 95 aoe IPA ‘Suppose also, for simplicity, that and Beal os For the linearly convergent scheme, this means that wa | ~ O5|e4-1] ~ ©.5)'Ipp-2| > * = (0.5)"Lpals iar nectar shh psBicetatitadscatman inn ' i | | Table 2.6 Theorem 2.7 24. Error Analysts for erative Methods 7 while the quadiatically convergent procedure has (p.| = 0.5)6,-1P ~ 0.5[0.5}p,-27 F = ©.5)1p,-2/* = O5)[O.5)bnsPF = OS), -al = ++ = 0." Bol” Table Z.6 ilustrates the relative speed of convergence of the sequences to zero. Linear Convergence Quadratic Convergence Sequence {p.} Sequence {f,} n 5)" ost 1 5.0000 x 10 5.0000 x 107? 2 2.5000 107» 1.2500 %¢ 107? 3 1.2500 x 107? 7.8125 x 10°* 4 6.2500 x 107? 3.0518 x 10° 5 3.1250 x 10° 4.6566 x 107” 6 1.5625 x 107° 1.0842 x 10°? 7 78125 10° 5.8775 x 10-9 ‘The quadratically convergent sequence is within 10°** of zero by the seventh term. Al least 126 terms ate needed ty cusute this accuracy for the linewly convergent sequence. BEE Quadraticelly convergent sequences generally converge much more quickly than those. that converge only linearly, but many techniques that generate convergent sequences do so only linearly. Let g © C[a, b] and suppose that g(x) € [a, b] for all x € (a, 6]. Suppose, in addition, that ' is continuous on (a, 6) with lek) |Sk<1, — forallxe (a,b). If g’(p) #0, then for any number py in [a, 5), the sequence Px= 8(P-av 2=I, converges only linearly to the unique fixed point in (a, b1. Proof We know from the Fixed-Point Theorem that the sequence converges to p. Since 4’ exists on (a, 6], we can apply the Mean Value Theorem to g to show that for any n, Pari = a(p,.) — ap) = (EM ra ~ PheR CHAPTER 2 * Solutions of Equations in One Variable where & is between p, and p. Since {p,}..9 converges t0 p, { & }2q also converges 10 p. Since g’ is continuous on [a, 6], we have lim g'(&) = g'(p). : 7 | ' . lim g'(&) and 4 noe PrP noe ; Heuve, fixed-point iteration exhibits near vouvergeuve if g'(p) #0. eed i Theorem 2.7 implies that higher-order convergence for fixed-point methods can occur = i only when g'(p) = 0. The next result describes additional conditions that ensure the quadratic convergence we seek. t Theorem 2.8 — Letp bea solution of the equation x = g(x). Suppose that g’(p) = 0 and g” is continuous ' and sttictly bounded by M on an open interval J containing p. Then thete exists a 8 > 0 such that, for po € [p — 8,p + 6], the sequence defined by p, = g(p,-1), when n = 1, converges at least quadratically to p, Moreover, for sufiiciently large values of n, M 2 bass — PL ipa abe Proof Choose 8> 0 such that on the interval [p — 8,p + 6), contained in I,| g'(2)| = k< 1 and g" is continuous. Since | g'(x) | = k < 1, it follows that the terms of the sequence {pdza0 are contained in [p ~ 8,p + 8]. Expanding g(x) in a linear Taylor polynomial for xe [p — &p + S]gives 8G) = ap) + g'(pye ~ p) + £2 ~ Py, zi | where lies between x aul p. The hypolleses g¢p) = p and g'(p) — 0 imiply daa © a) ~p + De — py. Tn particular, when x = p,, ) poss = (0) = p + EG, — p with &, between p,, and p. Thus, &) Poti — P= FOG, ~ py Since |g'@) |< <1 on[p - 8p + 8) and g maps [p — 8,p + 8] into itself it follows from the Fixed-Point Theorom that {p,]%=9 converges to p. But ¢, is between p and p, for each n, so { €,}"_ converges to p also, andSi ht EAR ENRON Nn | L 24 Error Analysis jor Iterative A etbods 73 ras —pl _ 12° tim 4 as le) a= [Px PI 2 ‘This implies thatthe sequence {p,}%_ is atleast quadratically convergent. Since g" is continuous and strictly bounded by Mon the interval [p — 8,p + 6], this also implies that for sufficiently large values of m, M Ipuet — PI) xo xo 2 ro 2 The terms generated by Newton's method applied to f with pp = 1 are shown in Table 2.7. The sequence is clearly not quadradically convergent to zero. The grap of f is shown in Figure 2,12, Z seer 4 3 :of ae Jr na tnibinbiaipeemnet thetihinmien ple ons sa rif act, vin fis niin italia able 2.7 igure 2.22 24 Error Analysis for erative Methods ” P. ” 0 10 9 2.7750 x 10> 1 058198 101.3881 x 107% 2 051906 1 63411 x 10-* 3 0.16800 12 3.4703 x 107 4 0.08635 1B L7416 x 107 5 0.04380 14 88041 x 107% 6 0.02206 15 4.2610 x 10° 7 ooi07 16 1.9142 x 1077 8B 0.005545 HO) = FG) Af p isa root of f of multiplicity m= 1 and f(x) = @ — p)"gW), hen Ha) = (x — P) __& = plats) ma(x) + (* — pg’) also has a root at p, but of multiplicity one. Newton’s method can then be applied, function to give Hea. f/f) (FOP - Y@OILP OL @FP 8G) or (2.16)EXAMPLE 3 ‘Table 2.8 EXAMPLE 4 ‘Table 2.9 CHAPTER 2 © Solutions of Equations in One Variable If g has the required continuity conditions, functional iteration applied to g will be quadratically convergent regardless of the multiplicity of the root of f Theoretically, the only drawback to this method is the additional calculation of #"(x) and the more laborious procedure of calculating the iterates. In practice, however, the presence of a multiple root cau cause seaivus 10uud-vff problems because the denominator of (2.16) consists of the difference of two small numbers. ‘Tahle 7.8 lists the approximation to the donhle root at x = Oof fle) = e* ~ x — 1 nsing (2.16) and a calculator with 10 digits of precision. The initial approximation of py = 1 ‘was chosen so that the entries can be compared with those in Table 2.7. What Table 2.8 does not show is that no improvement to the root approximation —2.8085217 x 107? will occur in subsequent computations using (2.16) and this calculator, since both the numera- tor and the denominator approach zero. ZEB OR n Pn —2.3421061 x 10°! 84582788 x 107* — 11889524 x 1077 — 68638230 x 10°¢ 2.085217 1077 woe In Example 3 of Section 2.2 we solved f(x) = x2 + 4x? — 10 = 0 for the root p = 1.36523001. To compare convergence for a root of multiplicity one by Newton's method and the modified Newton’s method listed in Bq. (2.16), let 2 2 = @ Pa = Pay — Pest iat 10 from Newton's method Spit + 8a and, from Eq. (2.16), a (phar + 494-1 ~ 10)Gp 5-3 + BP) Gpp-1 + Bop) ~ (Baa + 4ph-1 ~ 1ONGPy-1 * 8) With po = 1.5, the first three iterates for (i) and (ii) are shown in Table 2.9. The results illustrate the rapid convergence of both methods in the case of a simple zero. Gi) Pa = Paaa @ w@ By 137333333 —(1.35689898 Ps 136526201 1.36519585 Ds 136523001 136523001tke ge See i 24 Error Analysis for Iterative Methods —_—_— EXERCISE SET 24 1 10, iL Use Newton’s method to Sind solitons accurate to within 10 to the following problems: a dette =0, for0er=l b. cose + V2) + xx/24+-VH=0, for-2Sx=-1 cP = 3x92) +34") BHO, forsxS1 a. 4 30m 2%e% = en —dn2)>= 0, for-isx so Repeat Exercise 1 using the modified Newton-Raphson method described in Eq. (2.16). Is there an improvement in speed or accuracy over Exercise 1? ‘Use Newton’s method and the modified Newton-Raphson method described in Ea. (2.16) 10 Sind a solution accurate to within 10°* to the problem e&& + 1A44le™ ~ 2.079e - 0.3330= 0 for -1 = x<0. This is the same problem as 1(4) with the constants replaced by their four-digit approxima: ‘tions. Compare the solutions to the results in 1(€) and 214). Show that the following sequences {p,} converge linearly to p before |p, - p| 5 x 10°77 ©, How large must » be . n=l be py 1 & Pee Show that the sequence defined by p, = 1/n!, n= 1, for any positive integer k, converges linearly to p = 0. For each pair of integers k and m, determine a number N for which 1/NF< 10°", a. Show that the sequence p, = 10- converges quadratically to zero. bb, Show that the sequence p, ~ 107* does not converge to zero quadratically, regardiess of the size of the exponent E> 1 & Construct a sequence that converges to zero of order 3. b. Suppose a> 1. Construct a sequence that converges to 2er0 of order: Suppose p is a r00t of multiplicity m of f(s) = 0 where f* is continuous on sn open interval containing p. Show that the following fixed-point method has e'(p) = 0: f(s) a) re Show that the Bitection Algorithm 2.1 gives a sequence with an error bound that converges linearly to zero. Suppose that f has m continuous derivatives. Show that f has a zero of multiplicity m at p if and only if OF =P a LOM), ae fp) 6. Show thatthe iterative method to solve f(z) = 0, given by the fixed-point method g(x) = 2, where i _ Hod _ ft | delice: Pa™ 8p, an py aR 1 fern =1,2,3,..., has g'(p) = e"(p) = 0. This will generally yield cubic (a = 3) convergence. Using the analysis of Example 1, compare quadratic and cubic convergence.78 CHAPTER 2 © Solutions of Equations in One Variable 12. It can be shown (see, for example, Dahlquist and Bjérck [35], p. 228-229) that if {p,}%-9 are convergent Secant method approximations to p, the solution to f(x) = 0, then a constant C exists with [pai - P| ~ Clps — pl [pe-1 — pl. for sufficiently large values of n, Assume {p,} converges to p of order aand show that a = (1 + V5}/2. This implies that the order of convergence of the Secant method is approximately 1.62. 2.5 Accelerating Convergence It is rare to have the luxury of quadratic convergence. We will now consider a technique, called Aitken’s A? method, that can be used to accelerate the convergence of a sequence ‘hat 1s linearly convergent, regardless of ts origin o application. Suppose {p,}finq 15 a linearly convergent sequence with limit p and asymptotic error constant less than 1. To motivate the construction of a sequence {j,,} that converges more rapidly to p than does {p,}, let us first assume that the signs of p, — P, Per — p, and Pro ~ pagtee and that n is sufficiently large that Pott P , Peoa ~ P * Pa PB Pat" Then (Cine ~ PP * (Psa ~ PP» = Pe 0 Phat ~ 2PraiP +B? Pas2Pn — (Pat Pra2d? + PP and (Basa + Pn ~ 2Pnst)P ~ Pasa Pn ~ Pit - Solving for p gives PusaPa ~ Past Pasa ~ wet + Pa = Pit PoPos2 + 2PaPast — 2PmPasi ~ Da ~ Past Par2 ~ 2Pnoi + Pr — DEF PaPnor ~ 2a Pao) ~ Vi = 2PoPaat Pus2 ~ 2Past + Py = pp - — West eae "Pasa 7 BPaat * Pu Pr = {AK Pd. where {A} is used to indicate that the Aitken’s A? technique is employed. Steffensen’s method constructs the same first four terms, po, py, Pa, and By. However, at this step it is assumed that fy is a better approximation to p than isp, and fixed-point iteration is applied to fp instead of pp. Using the notation in Algorithm 2.6, the sequence generated is PE P= ar), PP = ap), PP = {APP pl? = a vb), Every third term is generated using the A? technique; the others use fixed-point itecation on the previous term, Steffensen'’s ‘Tofind a solution to p = g(p) given an initial approximation po: INPUT _ initial approximation pp; tolerance TOL; maximum number of iterations Np. OUTPUT approximate solution p or message of failure, Step 1 Seti Step 2. While i = No do Steps 3-6. Step3 Setpr = spo (Compute p") Pa= ats (Compute pi?) P= Po (Pi ~ Pol /(p2 — 2pr + po» (Compute pf.) Step 4 lE|p — po| < TOL then OUTPUT (p); (Procedure completed successfully.) STOR Step5 Sei-i+l. Step 6 Setpo =p. (Update po.) Step 7 OUTPUT (‘Method failed after Np iterations, No = *, No): (Procedure completed unsuccessfully.) STOP. Note that A’p, may be zero, which would introduce a zero in the denominator of the next iterate. To avoid this, we terminate the sequence and select p{'")) as the approximate answer. sans inner leg tn gohan ania Lae qe aise Siri bee| : | | nu CS a eR i A SUE Le URE ECU NCS a EXAMBLE 2 ‘able 2.47 Theorem 3.34 2.5 Accelerating Convergence 8! ‘To solve x° + 42 — 10 = O using Steffensen's method, let x + 4x7 = 10 and sotve for x by dividing by x + 4.If then x = g(x) implies x? + 4x? ~ 10 = 0. Using pp = 1.5, Stefffenser’s procedure gives the values in Table 2.11. The iterate pf = 1,365230013 is accurate to the minth decimal place. In this example, Steffensen’s method gave about the same rate of convergence as Newton's method (ee Example 4 in Section 2.4). =e k ee we we 0 15 "1348399725 —«1.867376372 1 1365265226 1365225534 —1.365230583 2 1,365230013 7 From Example 2, it appears that Steffensen’s method gives quadratic convergence without evaluating a derivative. Theorem 2.14 verifies that this is the case. The proof of this theorem can be found in Henrici {72}, pp. 90-92, or Isaacson and Keller [78), pp. 103-107. Suppose that x = g(x) has the solution p with g'(p) » 1. If there exists a § > 0 such that g€ Cp — 8p + 6), then Steffensen’s method gives quadratic convergence for any Poe (p — Op + 4. eee 1. The following sequences are linearly convergent. Generate the fist five terms of the sequence {9,} using Aitken’s A’ method, a p= 05, Pp b. p= 075, py & po~ 05. Pp @ po= 05, py = c0sp, 2, Consider the function f(x) = e + 3(in 2)* e®* — in Be* — (In 2)°. Use Newton's method with p> = 0 to determine the root of f(2) = 0. Generate terms until Py.) — py| < 0.0002. Construct the sequence {p,). IS the convergence improved? 3. Solve x® ~ x — 1 = Oforthe root. [1,23 to an accuracy of 10~* using Steffensen's method and the results of Exercise 6 of Section 2.2. 4. Soe x — 2°* = O for the root in (0, 1] to an accuracy of 10~* asing Steffensen’s method, sand compare to the recults of Exercise 8 of Section 2.2 1,2,82 2.6 Zeros of Polynomials and CHAPTER 2 * Solutions of Equations in One Variable 5. Use Steffensen's method with po = 2 to compate an approximation to V3 accurate to within 107+. Compare this result with those obtained in Exercise 9 of Section 2.2 and Exercise 8 of Section2.1 6 Use Steffensen’s method to approximate the solutions ofthe following equations to within 108, a. ‘where g is the function in Exercise 11(a) of Section 2.2. b. x= 05(6inx + cos.x), where g is the function in Exercise 11(F) of Section 3x? — e = O, where g is the fanction in Exercise 12(a) of Section 2.2. a. x ~ cos.x = 0, where gis the function in Exercise 12(b) of Section 2.2. 7. For the following convergent sequences {p,}, use Aitken’s A? method to generate a sequence {p,) until |p, — p]=5 x 107%: a Py 8. Ascquence {p,) is seid to be snperlinearly convergent to p if im Pett 2 oe Py? =0, a. Show that ifp, —> p of order a for > 1, then {p,} is superlinearly convergent top. b. Find a sequence {p,} that is sapertinearly convergent to zeto, but does not converge to zero of order a for any a> 1 9, Suppose that {p,} i2 superlinescly convergent to p. Show that tim Pest — Pu nae [Pe P| 10. Prove Theorem 2.13. [Hint: Let 5, = (Dy21 ~ p)/ (Pp ~ p) — Aand show that Timy4= 8, ~ 0. Thon express (2, — p}/ (bx — p)in terms of By, S41, and A} UL. Let P,(z) be the nth Taylor polynomial for f(x) = e* expanded about x9 = 0. a. For fixed x show that p, = P,(2) satisfies the hypotheses of Theorem 2.13. b. Letx = 1, Use Aitken’s A? method to generate the sequence fo, ..., Bg: Dose Aisken's mathod accelerate convergence in this situation? Muiller’s Method Theorem 215 A function of the form PO) = a,x" + a, eo eax + Om where the as, called the coefficients of P, are constants and a, # 0 is a polynomial of degree n. The zero function, P(x) = 0 for all values of x, is considered a polynomial but is assigned no degree. (Fundamental Theorem of Algebra) If is a polynomial of degree n = 1, then P(x) = O has at least one (possibly complex) root. eee sient a eh| | | | } | i | i Corollary 2.16 Coroliary 2.17 Theorem 2.38 26 Zeros of Polynomials and atiller’s Method 83 Although Theorem 2.15 is basic to any study of elementary functions, the usual proof requires techniques from the study of complex-function theory. The reader is referred to Saffland Snider {124}, p. 155, for the culmination of a systematic development of the topics needed to prove Theorem 2.15 ‘An important consequence of Thearem 2 15 is the fallowing corellary. If P(x) is a polynomial of degree n = 1, then there exist unique constants x), x2,..., Xi t possibly complex, and positive integers, m, mz, ...,m, such that >” m; = n and at PO) = a — 2," @ ~ x2) — eae Coratfary 2.16 states that the collection of zer0s of a polynomial is unique and that, if cach zero x 19 counted as many times as its multiplicity m, a polynomial of degrce 1 bas exactly n zeros. Its proof is considered in Exercise 7(a), ‘The following corollary of the Fundamental Theorem of Algebra will be used often in this section and in later chapters. The broof of this result is considered in Exercise 7(b). Let P and Q be polynomials of degree at most n. Ifxy, x2)... With k > n, are distinet numbers with P(x) = Q(x,) fori = 1,2,..., &, then P(x) = O(a) for all values of x. ‘To use the Newton-Raphson procedure to locate approximate zeros of a polynomial P, it is necessary to evaluate P and its derivative at specified values. Since both P and its derivative are polynomials, computational efficiency requires that the evaluation of these functions be done in the nested manner discussed in Section 1.2. Hornes’s method incoz- porates this nesting technique and as a consequence requires only » multiplications and n additions to evaluate an arbitrary nth-degree polynomial (Horner's Method) Let Plx) = age” + aye lee tage + ag and If By = ay + Dyyto fork =n—-1,n—- 1,0, then by = P(xo). Moreover, if OCs) = bx" + By yx tt ax + Ds then PG) = (x ~ xO) + Bo» Proof By the definition of Q@), DQG) + bo =O — H)(Bax" E+ +++ + bax + by) + bo = (byt" + By yx" + +++ + box? + bz) = (Byxox" +s + byxgx + Byxq) + bo peo b+ by — Bate + by — byxd a, and by — by a% = bx" + Oa - By the hypothesis of the theorem, &, = x9Q@) + bo =P) — and — by = Pep). wee Ay $084 CHAPTER 2 * Solutions of Equations in One Variable EXAMPLE 1 Evaluate P(x) = 2x" ~ 3x? + 3x ~ 4 at xy = —2 using Horner’s method. By Theorem 2.18, we have by = 2, bp = 2(-2) + 0 = -4, by = (4-2) — 3 = 5, by = (2) +3 = 7, and finally, P(~2) = by = (-7K-2) = 4 = 10. Moreover, Theorem 2.18 gives P(x) = (& + 2)(2x3 — 4x? + Sx — 7) + 10. sna ‘When we use hand calculation in Horner's method, we first construct a table, which suggests the “synthetic division” name often applied to the technique. For the problem in the preceding example, the table appears as (Coefficient) (Coefficient , (Coefficient (Coefficient (Constant ofx! ofx? of? ofx ‘erin 3 ay = ~4 =10 bo = 14 | bas b=-7 by An additional advantage of using the Homer (or synthetic-division) procedure is that, since 10 POx) = (x = xq) Ox) + bo, where Ole) = Bx! + bye? 4 + yx + By, differentiating with respect to x gives, (2.19) P(x) = QO) + (x — x0)0'@) and P'(xo) = Oxo). When the Newton-Raphson method is being used to find an approximate zero of a poly- nomial P, both P and P’ can be evaluated in the same manner. Algorithm 2.7 computes P(xo) and P’ (xq) using Horner's method. panes fp MUCORITEN Horner's 27 ‘To evaluate the polynomial See SE TS PC) = Gg" + dq gx") FH aE + ay and its derivative at x9 INPUT degree n; coefficients ay, a),..-, ay5 20 OUTPUT y = Plan 2 = P's). q Step 1 Sety = a,; (Compute b, for P) | z= 4, (Compute b,_, for Q.) aEXAMPLE 2 2.6 Zeros of Polynomials and Miilier’s Method Step2 Forj =n -1n- wt sety = xy +a, (Compute b, for P) gamit (Compute bj; for.) Step3 Sety= xy + ay. (Compute bo for P) Step 4 OUTPUT (y, 2); STOP. Find an approximation to one of the zeros of P(x) = 2x4 — 3x? + 3x - using the Newton Raphson procedure and synthetic division to evaluate P(,) and P! for each iterate x,. With x» = —2 as an initial approximation, we obtained P(—2 Example 1 by 2 0°-3 3 -4 2-4 § -7 10=%(-2). Using Theorem 2.18 and Eq. (2.19), Oe) = 28 a a sx—7 amd P(-2) = Q-2); so P'(~2) can be found by evaluating Q{—2) in a similar manner: x= -2 [2-4 5 -7 -4 16-42 2 -8 21 -49 = O-2)= P(-2) _ PG) __,_ 10 and w= X 2- (9) ~ -1.796, Repeating the procedure to find x, “1796 {2 0 ~3 30 4 6.451 —6.197 5.742 2 3.451 -3.197 1.742 = Pix) 3592 12.902 -29.368 2 ~7184 16.353 - 32.565 = Qe, = Ply). So P( 1.796) = 1.742, P'(-1.796) = —32.565, and 1.742 32.565 An actual zero to five decimal places is — 1.73896, 8 y= ~1.796 - 1.7425.CHAPTER 2 * Solutions of Equations in One Vartable Note that the polynomial denoted @ depends on the approximation being used and changes from iterate to iterate Ifthe Mth iterate, xy, in the Newton—Raphson procedure is an approximate zero for P, then PO) = = xyJQG) + by = - xyIOG) + Ply) = @ — xO) 90.x — xy is an approximate factot of P(x). Letting £ = xy be the approximate zero of P and Q,(x) the approximate factoz, PR) ™ & — £200, wwe can find a second approximate zero of P by applying the Newton-Raphson procedure to Q\(x), If P is an nth-degree polynomial with n real zeros, this procedure applied repeat edly wil eventually result in (n — 2) approximate zeros of P and an approximate quadratic factor Q,,.(x). At this stage, Q,-2(x) = U can be solved by the quadratic formuta to tind the last two approximate zeros of P Although this method can be used to find approximate zer08, it depends on repeated use of approximations and on occasion leads to very inac- curate approximations, ‘The procedure just described is called deflation. The accuracy difficulty with defin. tion is due to the fact that, when we obtain the approximate zeros of P, the Newton~ Raphson procedure is used on the reduced polynomial Q,, ie., the polynomial having the property that 7 BB Bae ts PO) = (= BNO = Le ++ OE — HIQWLD. ‘An approximate zero £,,., of Q, will generally not approximate a root of P(x) = O.as well as it does a r00t of the reduced equation Q,(x) = 0, and inaccuracy increases as k increases. One way to eliminate this difficulty is to use the reduced equations to find approximations, £z, $y, «s Bp to the zeros of P and then improve these approximations by applying the Newton-Raphson procedure to the original polynomial P Ithas been previously noted that the success of Newton's method often depends on obtaining a good initial approximation. The basic idea for finding approximate zeros of P is as follows: evaluate P at points x, for = 1,2,...,k. If P()P(x)) < 0, then Phas a zero between x; and x. The problem becomes a matter of choosing the xs so that the chance of missing a change of sign is minimized, while keeping the number of x,'s reason- ably small. To illustrate the possibie difficulty of this problem, consider the polynomial P(x) = 16x* ~ 40x5 + Sx? + 20x + 6. lf x; is any integer, it can be shown that P(x,) > 0; hence, evaluating P(x) at this infinite number of x;'s would not locate any intervals (x;,.x,) containing zeros of P However, P does have real zeros, which we will find in Example 3. It happened that this particular choice of the x's is inappropriate for this polynomial because of the closeness of the roots. Another problem with applying Newton's method to polynomials concerns the possi bility of the polynomial having complex roots even when ali the coefficients are real ‘numbers. If the initial approximation using Newton's method is a real number, all subse- quent approximations will also be real numbers. One way to overcome this difficulty is to begin with a nonreal initial approximation and do all the computations using complex arithmetic. An alternative approach has its basis in the following theorem.| Theorem 2.19 Figure 2.13 2.6 Zeros of Polynomials and Miller's Metbod 87 Iz =a + di! is@ complex zero uf muluplicity m of the polynomial P, then 2 = a — biis also 4 zero of multiplicity m of the polynomial P and (x? — 2ax + a +B)" is a fector of eee A synthetic division involving quadratic polynomials can be devised to approximately factor the polynomial so thet one term will be a quadratic polynomial whose complex roots are approximations to the roots of the original polynomial. This technique was described in some detail in our second edition, Instead of proceeding along these lines, we will now considera method first presented by D. E. Miiller [101] in 1956. This technique can be used for any root-finding problem. ‘but is particularly useful for approximating the roots of polynomials. ‘Mitler’s method is an extension of the Secant method. The Secant method begins with two initial approximations xp and x, and determines the next approximation x2 as the intersection of the x-axis with the line through (xp, f(xo)) and (x1, f(2,)). (See Figure 2.13(@).) Miller's method uses three initial approximations xo, x, and x, and determines the next approximation x, by considering the intersection of the x-axis with the parabola through (xo, f(0)). (*1.F(21)). and (x2, f.)). (See Figure 2.13(b).) + Xo % & | \. ‘The derivation of Miiller’s method begins by considering the quadratic polynomial PQ) = ale + Bm) Fe that passes through (xy, f=) (rs, fe) and (x,, f(%)). The constants a, b, and ¢ can be determined from the conditions Fe) = al%y — X2)* + BO — #2) + 6 Flex) and Fes) = ey — x2) + BO — 2) +, “P+ bOre= aiGoniiaM BB CHAPTER 2 * Solutions of Equations in One Variable tobe (2.20) c= fle), p = Corse? Ife) ~ fled] = 6 Ufo) ~ flea Gp — Ce — #a) — 1) ane ge Siz aD LF) ~ Fo fo) (= x2)04 — *2)(0 ‘To determine x3, the zero of P, we apply the quadratic formula to P Because of round-off error problems caused by the subtraction of nearly equal numbers, however, We apply the formula in the manner prescribed in Example 4 of Section 1.2: ssteaanaisbin tance isetintes: ~26 8 GE VE aac" This gives two possibilities for x5, depending on the sign preceding the radical term in Eq. (2.21). In Milller’s method, the sign‘is chosen to agree with the sign of &. Chosen in this manner, the denominator will be the largest in magnitude and will result in x; being selected as the closest zero of P to x,. Thus, 21) cigs aR AE 2e * be sign) VE dae where a, b. and c are given in Ea. (2.20). Once x3 is determined, the procedure is reinitialized using x, x2, and x5 in place of Xo Xj, and x, to determine the next approximation x4. The method continues until a satisfactory conclusion is obtained. Since, at each step, the method involves the radical V6 — 4ac, the method can approximate complex roots when it is appropriate to do so, Algorithm 2.8 implements this procedure. w= x, tics Mitlier's ‘Tofind « solution to f(x) = 0 given three approximations x9, x), and xy: INPUT Xp, x1,.r9% tolerance TOL; maximum number of iterations No OUTPUT approximate solution p or message of failure, Step 1 Seth, = x ~ x03 fy =~ HS 8, = (Foe) ~ Feo) / hs 8 = (Fa) ~ FEDD/ has 4 =~ 6) /Mha + Ae Step 2. While #< Nodo Steps 3-7. Step 3b = Bs + Indi : W — 4f Gey)", (Note: May be complex arithmetic.)ain ain sing eof dla ical 30. 3 | i | EXAMPLE 5 Step 8 OUTPUT (‘Method falled after No iterations, No Consider tne polynomial f(4) Zeros of Polynomials and Muliler's Method Step 4 I|b- D|<|b + dj thensetE = 5 +D else set E -D. Step5 Steph = -2f(2)/E; pHyth Step 6 I \h| Xo; hy = — xy 8 = (FO) ~ FR) / hss & = (f(s) — FE) / gs d = (8; ~ 8)/ th, + hy); rai4l "Noi (Procedure completed unsuccessfully.) STOP. 16x! = 40x* + 3x7 + 20x + 6, Using Algorithm with TOL = 10°S and different values of x9, x), and x» produces the results in Table 2 Table 2.12 0. ° * fod = 0.555556 + 0.598352¢ = 0.435450 + 0.1021018 0.390631 + 0.141852¢ — 0.357699 + 0.169265 0.356051 + 0.162836 ~ 0356062 + 0.162758 = 29.4007 = 3.89872 1.35223 ~ 1.193094 0.375057 — 0.670164¢ ~ 0.146746 — 0.007446295 = 0.183868 10°? + 0.539780 x 10° 0.286102 x 10°* + 0.953674 x 10° e ae wo=05, my =10, x= 15 * Foe 1.28785 = 1.37624 1.23746 0.126941 1.24160 0.219440 x 107 1.26168 0.257492 x 10~ 1.24168 0.257492 x 107 contt90 CHAPTER 2 * Solutions of Equations in One Variable ‘able 2.12 eas He= 25, 20, m= 225 ‘ x ed 196059 0.611255 197056 0.748825 x 10°? 1 1 17044 ~ 0.295639 x 107 17044 — 0.259639 x 107 Av aw ‘The actual values for the roots of the equation are 1.241677, 1.970446, ~ 0.356062. | + 0,162758i, which demonstrates the accuracy of the approximations from Miller's method. eeu Example 3 illustrates that Muller's method can approximate the roots of polynomials with a variety of starting values. In fact, the importance of Miller's method is that the technique generally converges to the root of a polynomial for any initial approximation choice, although problems can be constructed for which convergence will not occur for certain choices of initial approximations. For example, if x) 2.1, 884-42 for some i have the property that f(x) = Fleiss) = f0ri2) # 0, the quadratic equation reduces to a nonzero constant function and never intersects the x-axis. This is not usually the case, however, and general-purpose software packages using Miller's method request only one initial approximation per root and will even supply this approximation as an option, EXERCISE SET 2.6 1. Find the approximations to within 10°* to all the resl zeros of the following polynomicls using Newton’s method: a PQ) ax = 2x7 = 5 bv. PQ) ax + 3x7 = 1 & P@=xt—x-1 a Pe) e PC) fh Po)=xS= xt + 2 3b 4 2. Find approximations to within 10° to all the zeros of exch of the following polynomials by ‘irst finding the real zeros using Newton's method and then redacing to polynomils of lower degree to determine any complex zeros: a. PG) = x4 + 5x3 — 9x7 — 85x ~ 136 b. Po@y— xt 2x3 12s? + Ter ~ 40 e P@aatexts ax? 4 ws? Pix) = xP + 1x4 — 2103 = 10x? = 2x5 e. P(x) 6x* + 88x7 + 159x7 + 76x — 240 fF. PQ) A dx? 4 § PG) ext 2h at hat 4, hb. P(x) 3 Ix? 4 lax — 6, sit Shar citable ans Snonin Daae ahees iestiairhdn osuinastntinidaaabrrerlaeotenamnterneines eo i | 10. n B. 14. Zeros of Polynomials and Miller's Metboa 91 Repeat Exercise | using Miller's method. Repeat Exercise 2 using Miller's method, Use Newton's method to ficd, within 10-3, the zeros and critical points of the féllowing functions. Use this information to sketch the greph of a fQ@)= a — 9x8 +12 db, FQ)= x 2x? Set + eS Px) = 10r? = 83x" + 2.298x — 0.21141 = 0 has a root at x = 0.29. Use Newton's ‘method with an initil approxintation x9 = 0.28 to attempt to find this root. Wht heppens? a. Prove Corollary 2.16, b. Prove Corollary 2.17. Prove Theorem 2.19, (Hint: Fist cousider the case m = 1 aud) note what happeas (9 the constants if @ ~ bi is nota 2er0 of PCx).} Prove the following theorem: Theorem Let P(x) = ax" + dy.y2") + +++ + aut + ay be a polynomial of degree n and let x9 be a positive real number with P(x) > 0. If Q(x) satisfies PG) = (= x0)0(2) F Pore) Fm eMbyat oe + Bax + bd + Pld, and b> Ofori = 1,2... then al real zeros of Pare less than ot equ to ‘The Legendre polynomials can be generated recursively by Pott) = 1, Py(a) = x and 2 med Prose) = BES apaasty ~ +4 p,¢0, n = 0. These polynomias and thei 008s wil ‘be considered in Sections 4.7 and 8.2. Table 4.11 on page 209 lists the zeros of Pa Py, Py, and Pe a, Determine these polynomials and verify that the values in the table are correct. D. Determine Pg and approximate the zeros of this polynomial to within 10°6 ‘The Chebyshev polynomials can be generated recursively by Ty(x) = 1, T(x) = x and Traal2) = 2xT,41(x) — T,@).n = 0. These polynomials and their roots will be considered. in Section 8.3. a, Determine To, 75, Ty 8nd Ts. b. Approximata, to within 10°S, the 2eroe of Ty, Za, and Ty. Show thatthe results in part (b) are consistent with the resalt in Theorem 8.9. The Laguerre polynomials can be generated recursively by L(x) = 1, Ly) = 1 = Enaalt) = Qn + 3 ~ HL qei(%) — + PLA). 2 BO, a, Determine Lo, La, Le. and Ls. B. Approximate, to within 10 the zeros of Ly, La. and Ls. ‘The Hermite polynomials can be genereted recursively by Holx) = 1, Hy(x) = 2x, and Hosa) = 2H ya (2) — Un + TAG), 2 = 0. a, Determine Ha, Hs, Ha and Hs. ‘b. Approximate, to within 104, the zeros of H,, H., and Hy. Use each of the following methods to find 2 solution accurate to within 10~ for the problem 600s" — 5507 + 200:7- 20r-1=0, ford.lsx=1. a. Bisection method, Newton's method c. Secant method method of False Position Miller's methodCHAPTER 2 + Solutions of Rquations in One Variable 18. A canin the shape of a right circular cylinder is to be constructed to contain 1000 en, The Circular top and bottom of the can must have a radius of 0.25 cm more thn the radius of the can so that the excess can be used to form a seal with the side, The sheet of material being 4 formed into the side of the can must also be 0.25 om longer than the circumference of the can so that a seal can be formed. Find, to within 10°, the minimal amount of materiel needed (o construct the can, r+ 025 16, Two ladders crisscross an alley. Each ladder reaches from the base of one wall to some point on the opposite wall. The Jaddare cross at a height H above the paveunent. Given that the“ lengths of the ladders are x, = 20 ft and x, = 30 ftand that H = 8, find W. k—w— 17, In 1224, Leonardo of Pisa, better known as Fibonacci, answered a mathemetical challenge ‘of John of Palermo in the presence. of Emperor Frederick Il. His challenge was to find a root of the equation x? + 2x? + 10x = 20. He first showed that the equation had no rational sous and ny Duolidean issational root—ihet is, no root lu one of the forms a + Vb,Va & Vb, Va & Vi or VVa * Vb, where a and b ate rational numbers. He then approxi- tated the only real root, probably using an algebraic technique of Omar Khayyam involving the intersection of a circle and a parabole. His answer was given in the base 60 namber system as 14 22(G) + 1G)" + 42(hY + 33(G)* + 4(G)° + 400)" iow accurate was his approximation?Survey of Methods and Software 93 18. In describing a mathematical model for machine-tool chatter [107], the authors need the roots of a polynomial equation of the form PB) = 1 ~ OB ~ 628 ++ by B™= 0, for varions valves of and collections of constants gy da. +.» Bay The value of n depends ‘on the emount of experimental data being used, and approximations for the constants 4, i= 1,2,..., 2nere obtained using this data end a linear least squares technigue, «topic discussed in Section 8.1 ‘In en experimental situation involving e Gisholt turret lathe, it was found that, for rr = 2, reasonable approximations for the y's at: = 18310, gy = 05218, by = 04754, by = 0.1595. ttn Use these velues to find the zems of b. Inthe same experiment, with n = 3, values of ¢, were found to be: 742, o: = ~0.0385, = ~ 08! O.= = 01061, os ie — 0.0383. ‘Use these values to find the 2er08 of Ps sxmasinetseTironsenres 2.7 Survey of Methods and Software ust In this chapter we have considered the problem of solving the equation f(x) = 0, where f is a given continuous function. All the methods begin with an initial approximation and generate a sequence that converges to a root of the equation if the method is successful. If {a, b] is an interval on which f(a) and f(b) are of opposite sign. then the Bisection method and the method of False Position will converge. However, the convergence of these meth- coxis may be slow. Faster convergence is generally obtained using the Secant method or the Newton~Rapbson methori Good initial approximations are required for these methods, ‘wo for the Secant method and one for the Newton-Raphson method, so the Bisection or the False Position method can be used as starter methods for the Secant or Newton-- Raphson method. ‘Miiller’s method will give rapid convergence without a particularly good initial ap- proximation. Muller’s method is not quite as efficient as Newton's method; its order of convergence neat a root is approximately a = 1.84 compared to the quadratic, a = 2, of ‘Newton's method. However, it is better than the Secant method, whose order is approxi- mately « = 1.62. Deflation is generally used with Miiller’s method once an approximate root has been 4etermined. After an approximation fo the root of the deflated equation bas been deter mined, use either Milller’s method or Newton's method in the original polynomial with this root as the initial approximation. This will ensure that the root being approximated is a solution fo the true equation, not to the deflated equation. We recommended Miiller’s method for finding all the zeros of polynomials, real or complex. Miller's method can alsv be used for an arbitrary continuous function, but complex arithmetic may be needed.CHAPTER 2 © Soluttons of Equations in One Variable ‘Other high-order methods are available for determining the roots of polynomials. If this topic is of particular interest, we recommend that consideration be given to Laguerre’s method, which gives cubic convergence and also approximates complex roots (see House- older [76], pp. 176-179, for a complete discussion), the Jenkins—Traub method (see Jen- Kins and Traub (811), and Brent’s method (See Brent (18]), which is based on the Bisection and False-position methods. Another method of interest, called Cauchy's method, is similar to Miller’s method but avoids the failure problem of Miiller's method when f(x) = f(xi.1) = f(%;42), for some i. For interesting discussion of this method, as well as more detail on Miller's method, we recommend Young and Gregory [160], Sections 4.10, 4.11, and 5.4. Given a specified funetion f and a tolerance, an efficient program should produce an approximation to one or more solutions of f(x) = 0, each having an absolute or relative error within the tolerance, and the results should be generated in a reasonable amount of time. If the program cannot accomplish this task, it should atleast give meaningful expla- nations of why success was not obtained and an indication of how to remedy the cause of failure. ‘The IMSL FORTRAN subroutine ZANLY uses Milller’s method with deflation to approximate a number of roots of f(x) = 0. The routine ZBREN, due to R. P. Brent, uses ‘a combination of lincar interpolation, an inverse quadratic interpolation similar to Muller’s method, and the Bisection method. I requires specifying an interval [a, 6] that contains a root. The IMSL routine ZREAL is based on a variation of Miller's method and approxi- mates zeros of a real function f when ouly poor initial approximations are available, Routines for finding zeros of polynomials are ZPORC, which uses the Jenkins—Traub method for finding zeros of a real polynomial; ZPLRC, which uses Laguerre’s method to find zeros of areal polynomial ; and ZPOCC, which uses the Jenkins~Traub method to find zeros of a complex polynomial. ‘The NAG FORTRAN subroutine COSADF uses the Bisection method in conjunction with a method based on inverse linear interpolation, similar to the False Position and Secant methods, to approximate a real zero of f(x) = 0 in the interval (a, b]. The subrou- tine COSAZE is similar to COSADF but requires a single starting value instead of an interval. The subroutine COSAGF will find an interval containing a root on its own. NAG. also supplies subroutines CO2AEF and CO2ADF to approximate all zeros of a real poly- nomial or complex polynomial respectively. The subroutine CO2AGE uses a modified Laguerre method to find the roots of a real polynomial. Within MATLAB, the function ROOTS is used to compute all the roots, both real and complex, of a polynomial. For an arbitrary function, FZERO computes a root near a specified initial approximation to within a specified tolerance. Notice that in spite of the diversity of methods, the professionally written packages are primarily based on the methods and principles discussed in this chapter. You should be able.to use these packages by reading the manuals accompanying the packages to better understand the parameters and the specifications of the results that are obiained. ts Ses being canta ear Rail em oe sates ti setaeCuarrer 3 95 interpolation and Polynomial Approximation A consuscttte Population of the United States is taken every ten years. The following table lists the population, in thousands of people from 1940 to 1990, Population Ga thousindsy| Po) Population 1940 1950 1960 1970 1980 1990 2000 t Year ‘In reviewing this data, we might ask whether it could be used to reasouably estimate the population, say, in 1965 or even tn the yearFigure 3.1 Theorem 3.1 CHAPTER 3 © Interpolation and Polynomial approximation 2000. Predictions of this type can be obtained by using a function that, fits the given data. ‘This is called interpolation and is the subject of this chapter. This popelation problem is cousidered throughout the chapter and in Exercises 18 of Section 3.1, 10 of Section3.2, and 16 of Section 34, One of the most useful and well-known classes of functions mapping the set of real numbers into itself is the class of algebraic polynomials, the set of functions of the form P,(2) = dg + ax to a2", where n is a nonnegative integer and ag .. ., a, are real constants. One reason for their importance is that they uniformly approximate continuous functions. Given any function, defined and continuous on a closed interval, there exists a polynomial that is as “close” 4 to the given function as desired, This result is expressed precisely in the following theo rem, (See Figure 3.1.) : (Weierstrass Approximation Theorem) If f is defined and continuous on [a, 6] and ¢ > 0 is given, then there exists a polynomial P, defined on [4, b], with the property that |F@) - P@)|<6 — forall xe (a, 0} for example, Bartle (11), pp. 165-172) Other important reasons for considering the class of polynomials in the approximation of functions are that the derivative and indefinite integral of any polynomial are easy to determine and the results are again polynomials. For these reasons, polynomials are often used for approximating continuous functions.‘The Taylor polynomials were introdaced in the first section of the book, where they were described as one of the fundamental building blocks of numerical analysis. Given this prominence, it would be reasonable to assume that polynomial interpolation would make heavy use of these functions. This is not, however, the case. The Taylo: polynomials agree as closely as possible with a given function at a specific point, but they concentrate their accuracy only near that point. A good interpolation polynomial needs to provide a relatively accurate approximation over an entire interval. Taylor polynomials do not gen- erally satisfy this condition. For example, suppose we calculate the first six Taylor poly- nomials about xy = U for f(x) = e*, Since the derivatives of f are all e*, which evaluated att = 0 gives 1, the Taylor polynomials are. PO =1 Aa)=l+s PG) =1+x45, Pw) 2 ‘ PQ)altet De Gtr md Pope ltet 4 * 120 ‘Table 3.1 lists the values of the Taylor polynomial for various values of x, Notice that even for the higher-degree polynomials, the error becomes progressively worse as we move away from zero. ‘Table 3.1 x Po) Pu Pac Pea) Pac Pao e 1.00000 = 1.00000 1.00000, 033833005667 0.1354 =15 1.00000 0.50000 0.62500 0.27344 0.21016 0,213, 1.0 1.00000 0.00000 0.50000 036667 0.36788 05 1.00000 050000 0.62500 0.60651 0.60653 0.00000 1.00000 1.00000 1.00000 1.00000 05 1.00000 1.50000 1.62500, 1.64870 1.64872 10 1.00000 2.00000 3.50000 2.71667 2.71828 1S 1.00006 256000 3.62500 4.18750 446172 4.49169 20 1.00000 3.00000 5.00000 6.33883 7.26667 7.38906 Although better approximations are obtained for this prablem if higher-degree Taylor polynomials are used, this is not always the case. Consider, as an extreme example, using Taylor polynomials of various degrees for f(x) = 1/2 expanded about.x) = 1 to approx- imate f(3) = §. Since f(x) = x74 f'@) = -x7% f"@) = (-1)°2+ 27? and, in general, f” (2) = (~1)”nlx~*~}, the Taylor polynomials for n = 0 are rn, . PG) = Soe - f= D-'e- DS me To approximate f(3) = by P,(3) for increasing values of n, we obtain the values in Table 3:2—vathier a dramaie failure! Since the Taylor polynomials have the property that all the information used in the approximation is concentrated at the single point x, the type of difficulty that occurs here98 Table 3.2 CHAPTER 3 * Interpolation and Polynomial Approximation Px) is quite common, This generally limits Taylor polynomial approximation to the situation in which approximations are needed only at points close to xo. For ordinary computational purposes it is more efficient to use methods that include information at various points, ‘which we will consider in the remainder of this chapter. The primary use of Taylor poly nomials in numerical analysis is not for approximation purposes, but for the derivation of numerical techniques. cost tna i pistes esse aesnessnacatesecenen 3.1 Interpolation and the Lagrange. Polynomial Since the Taylor polynomials are not appropriate for interpolation, alternative methods are needed. In this section we find approximating polynomials that can be determined simply by specifying certain points on the plane through which they must pass, ‘The problem of determining a polynomial of degree 1 that passes through the distinct Points (%s, yo) and (x, y;) is the same as approximating a function f for which f(9) = Yo and f(x,) = y; by means of a first-degree polynomial interpolating, or agreeing with, the values of f at the given points Consider the linear polynomial ~~ & =) PR) = Gay When x = xp, Plas) = 1+ yo + 0°, = and when x = x1, Ples) = O-y0 + Ley = = feeds so P has the required properties. (See Figure 3.2.) The technique used to construct P is the method of “interpolation” often applied in using trigonometric or logarithmic tables. To generalize the concept of linear interpolation, consider the construction of a polynomial of degree at most n that passes through the n + 1 points (xo, f(z), (ay fCDs + «++ Cm f(€,))- (See Figure 3,3.) The linear polynomial passing through (q f(%o)) and (x; f(%,)) is constructed by using the quotients @~ sy) & = x9) Ly - nd ~ Te ) ~ When x = xo, Dols) = 1 and L(t) = 0. When x 1 boxy) = Oand Ly(x,) r | ikeaon val ts, lye a aly net % is os fae igh | 3.1 Interpolation and the Lagrange Fotynomtal 99 Figore 3.3 For the general case, we need to construct for each k = 0,1, ..., n,a quotient L(x) with the property that L,, ,@v) = Owheni # kand L,, ,(r,) = 1. To satisfy L, 0.) = 0 for each i # kerequires that the numerator of Z.,» contain the term ap Ge mG ey) @ = EDGE Hees) “7 OH x. To satisfy L,,(%,) = 1, the denominator of L, must be equal to (3.1) evaluated at x = 74. Thus,100 Figure 3.4 Theorem 3.2 EXAMPLE 1 CHAPTER 3 * Interpolationand Polynomial Approximation ‘A sketch of the graph of a typical L,,. (in the case when n is even and k is odd) ig | shown in Figure 3.4. Labo The interpolating polynomial is easily described now that the form of L,,, is known, This polynomial, called the nth Lagrange interpolating polynomial, is defined in the following theorem, Tfxq,Xy). + 4%n fe (n + 1) distinct numbers and f is a function whose values are given at these numbers, then theré exists a unique polynomial P of degree at most n with the property that q FG) = PG) — foreach k = 0,1,...,n. This polynomial is given by G2) Pla) = FORaMbnal®) + +++ + Fl nala) = > FOdLual, 2 where @ = x6) : Xe = Xo)" 83) Bal) = Faery =) Or = Ha DE Hh " Ge= x) ie for eachk =0,1.....0 eon ‘We will write Z, (2) simply as Ly(x) when there is no confusion as to its degree, Using the numbers, or nodes, 9 = 2,2; = 2.5, andx, = 4 to find the second interpolating polynomial for f(x) = 1 /+x requires that we first determine the coefficient polynomials “| Lo, Ly, and Lo: & — 2.5) = 4) Lo) = Goa hay 7 ~ OSE + 10, _ = De= 4) _ (H4e + Dye = 32 LO=Gs—nas- a 3 .| and L:@ = Go aye as) 7 { Since f(x) = f fe) = f2.5) = 04, and f(x.) = f(4) = 0.25, 2 Pe) = 3 fede) & | = 05 ~ 65) + 10) + 04 AEB } ~45)r 45 I +005 Se + { = (0.05x — 0.425)x + 1.15 ‘ An approximation to f (3) = 3 (see Figure 3.5) is } FQ) = PA) = 0.995 1 Figure 3.5 7 i 2 ; 3 : i | q : 1 i | | | i | i | | | ‘Compare this to Table 3.2 on page 98, where no Taylor polynomial (expanded about Xp = 1) could be used to reasonably approximate f(3) = 5. ene The next step is 10 calculate a remainder term or bound for the error involved in approximating a function by an interpolating polynomial. This is done in the following ‘theorem. Theorem 3.3 If xp. x1... x, are distinct numbers in the interval fa, bland f ¢ C**fa. b], then, for each x in (a, b], and a number £(x) in (a, b) exists with fey BA) £@) = P@) + (x = xp) — xa) ta) where P is the interpolating polynomial given in Eq. (3.2).102 CHAPTER 3 * Interpolationand Polynomial Approximation Proof Note fitst tbat if x = x, fork = 0, 1,...,n, then f6%,) = Por) and choosing £(z,) arbitrarily in (@, b) yields Eq, (3.4), Ifx # x, for any k = 0, 1,...,, define the function g for rin (a, b] by (= xo\lt = ey) = Fy) (= xo) — A] XD = sl ~ PO) ~ [70 ~ PC ont 4 8 sO-fO- PO LG) Pe) Since fe C'* "fa, b], Pe C™lu, bh, and x + xg for any k, it follows What g eC" fa, B) Fort = x;, - + = (x) Gi ai) = Fox) — Pod) — FG) - po [2 = 0~ [fG) — P@]-0=0 Morcover, as) =F) ~ Pe) ~ Le) ~ Pon TT S—*2 0 — x) = FQ) — Pa) — (F0) — PO) = 0. Thus, g = C* fa, b] and g vanishes at the n + 2 distinct numbers x, x05 -+ 5 %m BY the Generalized Rolle’s thedrem, there exists in (a, b) for which aE) 0, So, nr aipaeel a9 on NH = 9 - PN 46) - Pan SL LLP] E a Since P is a polynomial of degree at most , the (n + 1)st derivative, P*®, is identically zero, Also, [T}_, {(t — x) / @ — x)]is a polynomial of degree (n + 1), 80 -| 7 * shi" + dower depeetems ina, Tho @ = #0 Hd Fe (A and at fy Goad wt aeg@— x) Th — 0" Equation (3.5) now becomes a 0 = FN — 0 - (76) — Pall FOP if Th.@-2 and, upon solving for f), porrhy OF FO) = PO + FEL — x. soe ‘The error formula in Theorem 3.3 is an important theoretical result because Lagrange polynomials are used exteusively for detiving uumedical differentiation and integra tion methods. Etror bounds for these techniques are obtained from the Lagrange error formula. i i iEXAMPLE 2 ‘Note that the error form for the Lagrange polynomial is quite similar to that for the Taylor polynomial. The Taylor polynomial of cegree n about x» concentrates all the known information at xp and has an error term of the form LMC) tt fEONe — 5 Gey eet The Lagrange polynomual of degree n uses information at the distinct numbers xo, x), ....%, and, in place of (@ — xq)", its error formula uses a product of the n + 1 terms OF xg @ — ye > HD) *E@) WED TOE HD Bd The specific use of this error formula is restricted to those functions whose derivatives have known bounds. Suppose a table 1s 10 be prepared for the function f(x) = e*,0 SxS L. Assume the number of decimal places to be given per entry is d = 6 and that the difference between adjacent x-values, the step size, is h. What should h be for linear interpolation (ie., the Lagrange polynomial of degree 1) to give an absolute error of at most 10~°? Tet xe (0, 1] and suppose j satisfies x, = x= x,,,. From Eq. (3.4), the error in linear imtexpolation is : Dg) 2 FXB)| | FG) — PO) SS le = abe = adh Gea x)@ — xj) Since the step size is f, it follows thatx, = jh). = V+ WA, and 6 [F0) ~ Pats Se — je — G+ 1 Hence, {70 — Po)| ene max |(x ~ jaxx — U + DA). She max |@~ jhe ~ (jf + 1h) seoaija1 By considering g(x) = (x — jh\(e — (j + Dh) for jh =x <(j + Dhand using techniques of calculus (see Exercise 24), max |g()| = |g + DP] Consequently, the error in linear interpolation is bounded by el [f(@) ~ Poy) =. Hence it is sufficient for h to be chosen so that eh? -6 <8 6. 2 -6 "3, PsP 2-10 W<2986 x10 or) RS 1.72 x 10 Letting h = 0,001 would be one logical choice for the step size. eons104 EXAMPLE 3 ‘Table 3.3 CHAPTER 3 © Interpolation and Polynomial Approximation ee ‘The next example illustrates interpolation techniques for a situation in which the error portion of Eq. (3.4) cannot be used. ‘Table 3.3 lists values of a function (the Bessel function of the first kind of order zero) at various points. The approximations to /(I.5) obtained by various Lagrange polynomials, will be compared. x fe) 1.0 0.765197 13 0.6200860 16 0.455422 19 0.2818186 22 0.110623 Since 1.5 is between 1.3 and 1.6, the linear polynomial will use xp = 1.3 and. x, = 1.6, The value of the interpolating polynomial at 1.5 is given by G5 — 1.6) @3-16 ‘Two polynomials of degree two can reasonably be used, one by letting x9 = 1.3, x; = 1.6, and x» = 1.9, which gives PALS) = (0.6200860) 1 (0.455402) ~ 0.5102968, (6 = 13) 7 XLS - 19) (15 = 1.3)0. 9) PAS) 7 TEAS — 19) CMM + TE Taya. 19) 48540 5 2 XLS — 1.6) (L9 — 13).9 ~ 1.6) = 05112857 (0.2818186) and the other by letting x = 1.0,x1 = 13, andx, = 1.6, in which case PLS) = 05124715 In the third-degree case there are also two choices for the polynomial. One is with Xp = 13,21 = 16,2) = 19, and.xs = 2.2, which gives P3(1.5) = 05118302. = 16,andx3 = 19, giving ‘The fourth-degree Lagrange polynomial uses all the entries in the table. With 0,2) = 1.3,x2 = L6,r3 = 19, and.x, = 2.2, itcan be shown that P15) = 0.511820. x=3.1 Interpolation and the Lagrange Polynomial Since P3(1.5), P3(1.5), and P,(1.5) all agree to within 2 x 107? units, we expect P,(1.5) to be the most accurate approximation and to be correct to within 2 x 107* units. ‘The actual value of f(1.5) is known to be 0.5118277, so the true accuracies of the approximations are as follows: \P,G.5) — f(L.5)| * 1.53 x 107%, |Ps(L.5) - fL.S)j = 542 x 10-4, |P2C.3) — fLS)|~ 644 x 1074, IPs.5) (LS) ~ 2.5 x 107%, {P3(1.5) — f(L.5)| ~ 1.50 x 1073, [Pa(l.5) — fL.S)| ~ 7.7 x 1076, P, is the most accurate approximation. However, with no knowledge of the actual value of f(1.5), P, would be accepted as the best approximation, Note that the error or remainder termn derived in Theorem 3.3 cannot be applied here, since no knowledge of the fourth derivative of f is available. Unfortunately, this is generally the case. Bee ‘There are two immediate difficulties with using Lagrange polynomials as illustrated in this example. First, the exror term is difficult to apply, so the degree of the polynomial needed for the desired accuracy is generally not known until computations are determined, ‘The usual practice is to Compute the resuits given from various polynomials until sppro- priate agreement is obtained. Also, the wérk done in calculating the approximation by the second polynomial does not lessen the work needed to calculate the third approximation; nor is the fourth approximation easier to obtain once the third approximation is known. ‘We will now derive these approximating polynomials in a manner that uses the previous calculations to greater advantage. Definition 34 Let f be a function defined at xo, x), 2, 4%, and suppose that my, mig, ..., m, are distinct integers with 0 = m, , (8) sta - art & were (7) aenores 77-55 Tete polynomials canbe wedin seonstedve proof of he Weierstrass Approtimation Theorem 3.1 (See Bartle (11)), since if f is continuous on {0, 1) and x © (0,1), then lim B,(%) = FG). a, Find B,(«) forthe functions Lf@=x rel, ff fe=1, xe} b. Show that for each k = n, mw 1) (E n ena) *\a) Xe . Use part (b) and the fics, from (ji) in part (a), that ls > (jx xy! foreach n, o\t Be) 2 ah 4, Using part (©), estimate the value of n necessary for (x) — xin (0, 1]. to show that, for f(x) = 23, 10° co hold for all