Preface About the Text We have developed this material for a sequence of courses on the theory and application of numerical approximation techniques. The text is designed primarily for junior-level math- ematics, science, and engineering majors who have completed at least the first year of the standard college calculus sequence. Familiarity with the fundamentals of matrix algebra and differential equations is also useful, but adequate introductory material on these topics is presented in the text so that those courses need not be prerequisites. Previous editions of Numerical Analysis have been used in a wide variety of situations, In some cases, the mathematical analysis underlying the development of approximation techniques was emphasized rather than the methods themselves; in others, the emphasis ‘was reversed. The book has also been used as the core reference for courses atthe beginning graduate level in engineering and computer science programs, and in first-year courses in introductory analysis offered at international universities. We have tried to adapt the book to fit these diverse users without compromising our original purpose: To give an introduction to modern approximation techniques; to explain how, why, and when they can be expected to work; and to provide a firm basis for future study of numerical analysis and scientific computing. The book contains sufficient material for a full year of study, but we expect many readers to use the text only for a single-term course. In such a course, students lea to identify the types of problems that require numerical techniques for their solution and see examples of the error propagation that can occur when numerical methods are applied. ‘They accurately approximate the solutions of problems that cannot be solved exactly and earn techniques for estimating error bounds for the approximations. The remainder of the text serves as a reference for methods that are not considered in the course. Either the full-year or single-course treatment is consistent with the aims of the text Virtually every concept in the text is illustrated by example, and this edition contains more than 2,000 class-tested exercises ranging from elementary applications of methods and algorithms to generalizations and extensions of the theory. In addition, the exercise sets include many applied problems from diverse areas of engineering, as well as from the physical, computer, biological, and social sciences. The applications chosen demonstrate concisely how numerical methods can be, and often must be, applied in real-life situations. ‘A number of software packages have been developed to produce symbolic mathemat- ical computations. Predominant among them in the academic environment are Derive”, Maple®, and Mathematica®. Student versions of these sofiware packages are available atx Preface reasonable prices for most common computer systems. Although there are significant dif- ferences among the packages, both in performance and price, all can perform standard algebra and calculus operations. Having a symbolic computation package available can be very useful in the study of approximation techniques. The results in most of our examples and exercises have been generated using problems for which exact values can be deter- mined, since this permits the performance of the approximation method to be monitored. Exact solutions can often be obtained quite easily using symbolic computation. In addition, for tiany numerical techniques the error analysis requires bounding a higher ordinary or partial derivative of a function, which can be a tedious task and one that is not particularly instructive once the techniques of calculus have been mastered. Derivatives can be quickly obtained symbolically, and a litle insight often permits a symbolic computation to aid in the bounding process as well. ‘We have chosen Maple as our standard package because of its wide distribution, but Derive or Mathematica can be substituted with only minor modifications. Examples and exercises have been added whenever we felt that a computer algebra system would be of significant benefit, and we have discussed the approximation methods that Maple employs when it is unable to solve a problem exactly. 2 New for This Edition ‘The seventh edition includes two new major sections. The Preconditioned Conjugate Gra- dient method has been added to Chapter 7 to provide a more complete treatment of the numerical solution to systems of linear equations. It is presented as an iterative approxi- mation technique for solving positive definite linear systems. In this form, it is particularly useful for approximating the solution to large sparse systems. In Chapter 10 we have added a section on Homotopy and Continuation methods. These methods provide a distinctly different technique for approximating the solutions to nonlin- ear systems of equations, one that has attracted a great deal of attention recently. We have also added extensive listings of Maple code throughout the book, since re- viewers found this feature useful in the sixth edition. We have updated all the Maple code to Release 6, the version that will be current by the time the book is printed. Those familiar with our past editions will find that virtually every page has been improved in some way. Alll the references have been updated and revised, and new exercises have been added. We hope you will find these changes beneficial to the teaching and study of numerical anal- ysis; most have been motivated by changes in the presentation of the material to our own students. Another important modification in this edition is a web site at http://www.as.ysu.edu/~faires/Numerical-Analysis/ On this web site we will place updated programs as the software changes and post responses to comments made by users of the book. We can also add new material that might be included in subsequent editions in the form of PDF files that users can download. Our hope is that this will extend the life of the seventh edition while keeping the material in the book up to date.Preface Algorithms As in previous editions, we give a detailed, structured algorithm without program listing for each method in the text. The algorithms are in a form that can be coded, even by those with limited programming experience. This edition includes a disk containing programs for solutions to representative exer- cises using the algorithms. The programs for each algorithm are written in Fortran, Pascal, and C. In addition, we have coded the programs using Maple and Mathematica, as well as in MATLAB®, a computer software package that is widely used for linear algebra applica- tions. This should ensure that a set of programs is available for most common computing systems. A Student Study Guide is available with this edition that illustrates the calls required for these programs, which is useful for those with limited programming experience. The study guide also contains worked-out solutions to many of the problems. Brooks/Cole can provide instructors with an Instructor's Manual that provides an- swers and solutions to all the exercises in the book. Computation results in the Instructor's ‘Manual were regenerated for this edition, using the programs on the disk to ensure com- patibility among the various programming systems. The algorithms in the text lead to programs that give correct results for the examples and exercises in the text, but no attempt was made to write general-purpose professional software. Specifically, the algorithms are not always written in a form that leads to the most efficient program in terms of either time or storage requirements. When a conflict occurred between writing an extremely efficient algorithm and writing a slightly different one that better illustrates the important features of the method, the latter path was invariably taken. About the Program Disk ‘The CD on the inside back cover of the book contains programs for all the algorithms in the book, in numerous formats, as well as samples of the Student Study Guide for the book in both the PostScript® (PS) and the Adobe® Portable Document (PDF) formats. For each algoithm there is a C, Fortran, Maple, Mathematica, MATLAB, and Pascal program, and for some of these systems there are multiple programs that depend on the particular version of the software that is being run. Every program is illustrated with a sample problem that is closely correlated to the text. This permits the program to be run initially in the language of your choice to see the form of the input and output. The pro- ‘grams can then be modified for other problems by making minor changes. The form of the input and output are, as nearly as possible, the same in each of the programming systems This permits an instructor using the programs to discuss them generically, without regard to the particular programming system an individual student uses. ‘The programs are designed to run on a minimally configured computer. All that is re- quired is a computer running MS-DOS®, Windows®, or the Macintosh” operating system. You will, however, need appropriate software, such as a compiler for Pascal, Fortran, and , or one of the computer algebra systems (Maple, Mathematica, and MATLAB). Therexii Preface are six subdirectories on the disk, one for each of the computer languages and the accom- panying data files. Alllof the programs are given as ASCII files or worksheets. They can be altered using any editor or word processor that creates a standard ASCII file. (These are also commonly called “Text Only” files.) Extensive README files are included with the program files so that the peculiarities of the various programming systems can be individually addressed. The README files are presented both in ASCII format and as PDF files.“As new software is developed, the algorithms will be updated and placed on the web site for the book. ee Suggested Course Outlines ‘Numerical Analysis is designed to allow instructors flexibility in the choice of topics, as ‘well as in the level of theoretical rigor and in the emphasis on applications. In line with these aims, we provide detailed references for the results that are not demonstrated in the text and for the applications that are used to indicate the practical importance of the methods. The text references cited are those most likely to be available in college libraries and have been updated to reflect the most recent edition at the time this book was placed into production. We also include quotations from original research papers when we feel this material is accessible to our intended audience. The following flowchart indicates chapter prerequisites. The only deviation from this, chart is described in the footnote at the bottom of the first page of Section 3.4, Most of the possible sequences that can be generated from this chart have been taught by the authors at Youngstown State University.Preface xiii Acknowledgments We feel most fortunate to have had so many of our students and colleagues communicate with us regarding their impressions of earlier editions of this book. Alll of these comments are taken very seriously; we have tried to include all the suggestions that are in line with the philosophy of the book, and are extremely grateful to all those that have taken the time to contact us and inform us of improvements we can make in subsequent versions. ‘We would particularly like to thank the following, whose efforts we greatly appreciate. Glen Granzow, Idaho State University Jose Miguel, Universidad Peruana Cayetano Heredia, Lima, Peru John M, Neuberger, Northen Arizona University L. G. de Pillis, Harvey Mudd College We want especially to thank our friend and former student Jim Baglama of Ball State University. Jim agreed to be an extensive reviewer for this edition and was particularly helpful in updating our survey sections and references to software. It is most gratifying to see one’s students move through the profession. ‘Also moving through his profession, but in a completly different manner, is our Editor and Publisher Gary Ostedt. Gary has been an outstanding manager of our projects and a good personal friend. We will very much miss his direction and assistance, and would like to take this opportunity to wish him all the best in his upcoming retirement from Brooks/Cole. As has been our practice in past editions of the book, we have used student help at Youngstown State University in preparing the seventh edition. Our able assistant for this edition was Laurie Marinelli, whom we thank for all her work. We would also like to express gratitude to our colleagues on the faculty and administration of Youngstown State University for providing us the opportunity and facilities to complete this project. Finally, we would like to thank all those who have used and adopted the various edi- tions of Numerical Analysis over the years. It has been wonderful to hear from so many students, and new faculty, who used our book for their first exposure (o the study of nu- merical methods. We hope this edition continues the trend and adds to the enjoyment of students studying numerical analysis. If you have any suggestions for improvements that can be incorporated into future editions of the book, we would be grateful for your com- ‘ments, We can be contacted by electronic mail at the addresses listed below. Richard L, Burden burden @math.ysu.edu J. Douglas Faires [email protected]Contents Mathematical Preliminaries 1 11 Review of Calculus 2 1.2 Roundoff Errors and Computer Arithmetic 18 1.3 Algorithms and Convergence 31 1.4 Numerical Software 40 — Solutions of Equations in One Variable 47 21 The Bisection Method 48 2.2. Fixed-Point Iteration 55 23. Newton’sMethod 66 2.4 Exror Analysis for Iterative Methods 78 2.5 Accelerating Convergence 86 2.6 Zeros of Polynomials and Muller's Method 91 2.7 Survey of Methods and Software 101 N — Interpolation and Polynomial Approximation 104 3.1 Interpolation and the Lagrange Polynomial 107 3.2 Divided Differences 122 3.3. Hermite Interpolation 133 3.4. Cubic Spline Interpolation 141 3.5 Parametric Curves 156 3.6 Survey of Methods and Software 163 UWContents 4 = Numerical Differentiation and Integration 166 4.1 Numerical Differentiation 167 4.2. Richardson's Extrapolation 178 43. Elements of Numerical Integration 186 44 Composite Numerical Integration 196 4.5 Romberg Integration 207 4.6 Adaptive Quadrature Methods 213 47 Gaussian Quadrature 220 48 Multiple Integrals 227 49 Improper Integrals 241 4.10 Survey of Methods and Software 247 5 _Initial-Value Problems for Ordinary Differential Equations 249 5.1 The Elementary Theory of Initial-Value Problems 251 5.2. Buler’s Method 256 5.3. Higher-Order Taylor Methods 266 54 Runge-Kutta Methods 272 5.5. Error Control and the Runge-Kutta-Fehlberg Method 282 5.6 Multistep Methods 289 5.7 Variable Step-Size Multistep Methods 301 5.8 Extrapolation Methods 307 5.9 Higher-Order Equations and Systems of Differential Equations 313 5.10 Stability 324 5.11. Stiff Differential Equations 334 5.12. Survey of Methods and Software 342 6 Direct Methods for Solving Linear Systems 344 6.1 Linear Systems of Equations 345 6.2 Pivoting Strategies 359 6.3 Linear Algebra and Matrix Inversion 370 64 The Determinant of a Matrix 383 6.5 Matrix Factorization 388 6.6 Special ‘Types of Matrices 398 6.7 Survey of Methods and Software 413Contents 7 oe © 10 Iterative Techniques in Matrix Algebra 417 7.1 Norms of Vectors and Matrices 418 7.2. Eigenvalues and Eigenvectors 430 7.3. Iterative Techniques for Solving Linear Systems 437, 7.4 Error Bounds and Iterative Refinement 454 7.5 The Conjugate Gradient Method 465 7.6 Survey of Methods and Software 481 Approximation Theory 483 8.1 Discrete Least Squares Approximation 484 8.2 Orthogonal Polynomials and Least Squares Approximation 498 8.3 Chebyshev Polynomials and Economization of Power Series 507 8.4 Rational Function Approximation $17 8.5. Trigonometric Polynomial Approximation 529 8.6 Fast Fourier Transforms 537 8.7 Survey of Methods and Software 548 Approximating Eigenvalues 550 9.1 Linear Algebra and Eigenvalues 551 9.2. The Power Method 560 9.3 Houscholder’s Method 577 9.4 The QR Algorithm 585 9.5 Survey of Methods and Software 597 Numerical Solutions of Nonlinear Systems of Equations 600 10.1. Fixed Points for Functions of Several Variables 602 10.2. Newton's Method 611 10.3 Quasi-Newton Methods 620 10.4. Steepest Descent Techniques 628 10.5 Homotopy and Continuation Methods 635 10.6 Survey of Methods and Software 643 =viii Contents 11 12 Boundary-Value Problems for Ordinary Differential Equations 645 il M2 n3 na 11s 116 The Linear Shooting Method 646 ‘The Shooting Method for Nonlinear Problems 653 Finite-Difference Methods for Linear Problems 650 Finite-Difference Methods for Nonlinear Problems 667 The Rayleigh-Ritz Method 672 Survey of Methods and Software 688 Numerical Solutions to Partial Differential Equations 691 12.4 122 123 124 125 Elliptic Partial Differential Equations 694 Parabolic Partial Differential Equations 704 Hyperbolic Partial Differential Equations 718 ‘An Introduction to the Finite-Element Method 726 ‘Survey of Methods and Software 741 Bibliography 743 Answers to Selected Exercises 753 Index 831CHAPTER Mathematical Preliminaries In neginning chemistry courses, we see the idea! gos law, PV = NRT, which relates the pressure P, volume V, temperature T, and number of moles N of an “ideal” gas. In this equation, K is a constant that depends on the measurement system. ‘Suppose two experiments are conducted to test this law, using the ‘same gas in each case. In the first experiment, P=1.00atm, V = 0.100 m’, N= 0.00420 mol, = 0.08206. ‘The ideal gas law predicts the temperature of the gas to be PY _ _(1.00)(0.100) NR ~ (0-00420)(0.08206) ‘When we measure the temperature of the gas, we find that the true tem- perature is 15°C. T= = 290.15K =17°C.CHAPTER 1 + Mathematical Preliminaries We then repeat the experiment using the same values of R and N, but the same factor, Since the product PY remains the same, the predicted temperature is still 17°C, but we find that the actual temperature of the ease the pressure by a factor of two and reduce the volume by gas is now 19 Clearly, the ideal gas law is suspect, but before concluding that the ji in this situation, we should examine the data to see whether the error can be attributed to the experimental results. If so, we might law is inv be able to determine how much more accurate our experimental results would need to be to ensure that an error of this magnitude could not occur, Analysis of the error involved in caleulations is an important topic in numerical analysis and is introduced in Section 1.2. This particular application is considered in Exercise 28 of that section. This chapter contains a short review of those topics from elementary single-variable calculus that will be needed in Inter chapters, together with an introduction to convergence, error analysis, and the machine representation of numbers. 1.1 Review of Calculus Definition 1.1 Definition 1.2 ‘The concepts of limit and continuity of a function are fundamental to the study of calculus. A function f defined on a set X of real numbers has the limit L at xo, written lim f(x) = L, ay ; given any real number € > 0, there exists areal number 6 > O such that | f(x) ~ LI < €, whenever x € X and 0 < |x — xol < 6. (See Figure 1.1.) . Let f be a function defined on a set X of real numbers and xo ¢ X. Then f is continuous at x9 if tim £6) = feo). ‘The function f is continuous on the set X if it is continuous at each number in X. .Figure 1.1 Definition 1.3 Theorem 1.4 1 Review of Calculus 3 | m8 mH HtS x C(X) denotes the set of all functions that are continuous on X. When X is an interval of the real line, the parentheses in this notation are omitted. For example, the set of all functions continuous on the closed interval [a, b] is denoted C[a, b}. The limit of a sequence of real or complex numbers is defined in a similar manner. Let (x,}2, be an infinite sequence of real or complex numbers. The sequence [x,}2 , has the limit x (converges to x) if, for any € > 0, there exists a positive integer N’(e) such that lx, — x] < €, whenever n > N(¢). The notation lim x, = OF Xe x as N+ 00, means that the sequence (x,}°°2, converges to x. ‘The following theorem relates the concepts of convergence and continuity. If f is a function defined on a set X of real numbers and x € X, then the following statements are equivalent: af is continuous at xo; Db. If (ap}22; is any sequence in X converging to xo, then limy+oo fn) = f (x0) . ‘The functions we consider when discussing numerical methods are assumed to be con- ‘tinuous since this is a minimal requirement for predictable behavior. Functions that are not continuous can skip over points of interest, which can cause difficulties when attempting to approximate a solution to a problem. More sophisticated assumptions about a func- tion generally lead to better approximation results. For example, a function with a smoothDefinition 15 Figure 1.2 Theorem 1.6 Theorem 1.7 CHAPTER 1 + Mathematical Preliminaries ‘graph will normally behave more predictably than one with numerous jagged features. The smoothness condition relies on the concept of the derivative. Let f be a function defined in an open interval containing x9. The function f is differen- tiable at x0 if $40) = in LOA LED at —X0 exists. The number f"(xo) is called the derivative of ff at.x9. A function that has a deriva- tive at each number in a set X is differentiable on X. The derivative of f at xp is the slope of the tangent line to the graph of f at (xo, (0), as shown in Figure 1.2. . ‘The tangent line has slope f(x) Gof) —-y2yay If the function f is differentiable at xo, then f is continuous at xo. . ‘The set of all functions that have n continuous derivatives on X is denoted C*(X), and the set of functions that have derivatives of all orders on X is denoted C™°(X). Polynomial, rational, trigonometric, exponential, and logarithmic functions are in C™(X), where X consists of all numbers for which the functions are defined. When X is an interval of the real line, we will again omit the parentheses in this notation. ‘The next theorems are of fundamental importance in deriving methods for error esti- mation. The proofs of these theorems and the other unreferenced results in this section can be found in any standard calculus text. (Rolle’s Theorem) Suppose f € Cla, b] and f is differentiable on (a, b). If f(a) = f(b), then a number c in (a, b) exists with f’(c) = 0. (See Figure 1.3.) .11 Review of Calculus 5 | Flo) = 6) | Ake. 7 Neo oF) +1. Theorem 1.8 —_ (Mean Value Theorem) If f € Cla, b] and f is differentiable on (a, b), then a number c in (a, 5) exists with £O- fay boa Figure 13 fo=s (See Figure 1.4.) . Figure 1.4 Parallel lines Slope f'(c) y=f@) Slope Loa fe Theorem 1.9 —_ (Extreme Value Theorem) If f € Cla, b), then cy, cz € [a,b] exist with f(c1) < f(x) < f(ca), for all x € [a,b]. In addition, if f is differentiable on (a, b), then the numbers c, and cz occur either at the endpoints of [a, b] or where f” is zero. (See Figure 1.5.) . As mentioned in the preface, we will use the computer algebra system Maple whenever appropriate, Computer algebra systems are particularly useful for symbolic differentiation and plotting graphs. Both techniques are illustrated in Example 1.Figure 15 EXAMPLE 1 CHAPTER 1 + Mathematical Preliminaries Find maxgexwith (plots); The commands within the package are then displayed. We define f by entering >£:= Skcos(2ex)-2axksin (24x) ; The response from Maple is Ff := Scos(2x) — 2x sin(2x) To graph f on the interval (0.5, 2], use the command >plot (f,x=0.5..2)5 ‘The graph appears as shown in Figure 1.6, and we can determine the coordinates of any point of the graph by moving the mouse pointer to the desired point and clicking the left mouse button, This technique can be used to estimate axis intercepts and extrema of func- tions. ‘We complete the example using the Extreme Value Theorem. First, consider the inter- val (1, 2]. To obtain the first derivative g = /’, enter >gindiff(t,x); Maple gives =12sin(2x) — 4x cos(2x)1.1 Review of Calculus Figure 16 aa 1 2 5 12," 14618 We can then solve g(x) = , for | fsolve(g,x,1..2); obtaining 1.358229874, and compute f(1.358229874) using >evald (subs (x=1.358229874,£)) 5 Since f (1) = -3.899329037 and f(2) = —0.241008124, we have, for the interval [1,2], a maximum value of (2) = —0.241008124 as illustrated in Figure 1.7, and a Figure 1.7 as -1 4 i 12 i is is 3 x8 CHAPTER 1 + Mathematical Preliminaries minimum of approximately /(1.358229874) = —5.675301338. Hence, max, |5 cos 2x — 2x sin 2x] * | f(1.358229874)| = 5.675301338. If we try to solve g(x) = 0, for 0.5 fsolve(g,x,0.5..1); Maple responds with fsolve(—12 sin(2x) — 4x cos(2x),*, 5.1) which indicates that Maple could not find a solution in {0.5, 1]. If you graph g, you will see that there is no solution in this interval, and the maximum occurs at an endpoint. Hence, J’ is never 0 in (0.5, 1], as shown in Figure 1.8, and, since f(0.5) = 1.860040545 and ‘{(1) = —3.899329037, we have max |Scos 2x — 2x sin 2x| = |f(1)| = 3.899329037. . oSsrsi Figure 18 The other basic concept of calculus that will be used extensively is the Riemann inte- gral. Definition 1.10 The Riemann integral of the function on the interval [a,b] is the following limit, pro- vided it exists: [re ax = lim Lie An,Figure 19 Theorem 1.11 11 Review of Calculus 9 where the numbers x0,.£1,...,%, satisfy a = x9 Say S++ S ay = b, and where Ax i — Xi-1, for each i 1,2,...,m, and z; is arbitrarily chosen in the interval bax . Every continuous function f on [a, b] is Riemann integrable on {a, b]. This permits tus to choose, for computational convenience, the points x; to be equally spaced in [a, b], and for each i = 1,2,...,1, to choose z * f(x) dx = lim where the numbers shown in Figure 1.9 as x; are x; =a +i(b — a)/n. y | y=fe) ‘Two other results will be needed in our study of numerical analysis. The first is a generalization of the usual Mean Value Theorem for Integral. (Weighted Mean Value Theorem for Integrals) Suppose f € C[a, b], the Riemann integral of g exists on (a, 5], and g(x) does not change sign on (a, 6}. Then there exists a number c in (a, b) with . ° f FH)8@), ax= fo [ g(a) dx. . When g(x) |, Theorem 1.11 is the usual Mean Value Theorem for Integrals. It gives the average value of the function f over the interval (a, 6] as (See Figure 1.10.)10 Figure 1.10 Theorem 1.12 Theorem 1.13 Figure 1.11 CHAPTER 1 + Mathematical Preliminaries ‘The proof of Theorem 1.11 is not generally given in a basic calculus course but can be found in most analysis texts (see, for example, [Fu, p. 162]). The other theorem we will need that is not generally presented in a basic calculus course is derived by applying Rolle’s Theorem successively to f, f’,... , and, finally, to yo, (Generalized Rolle's Theorem) Suppose f € C[a, b] is n times differentiable on (a, b). If f (x) is zero at the n +1 distinct numbers x0, ... , %, in [a,b], then a number c in (a, b) exists with f(c) = 0. . ‘The next theorem is the Intermediate Value Theorem. Although its statement seems reasonable, the proof is beyond the scope of the usual calculus course. It can, however, be found in most analysis texts (see, for example, (Fu, p. 67) (Intermediate Value Theorem) If f € Cla, b] and K is any number between f(a) and f(b), then there exists a number in (a, b) for which f(c) = K. . Figure 1.11 shows one choice for the number that is guaranteed by the Intermediate Value Theorem. In this example there are two other possibilities.EXAMPLE 2 Theorem 1.14 EXAMPLE 3 11 Review of Calculus " To show that x° ~ 2x3+ 3x? — 1 = O has a solution in the interval (0, 1], consider f(x) = a5 = 2x3 + 3x? — 1. Since f@)=-1<0<1=f(1) and f is continuous, the Intermediate Value Theorem implies that a number x exists with 0 < x <1, for which x5 — 2x? + 3x?-1=0. . As seen in Example 2, the Intermediate Value Theorem is used to determine when solutions to certain problems exist. It does not, however, give an efficient means for finding these solutions. This topic is considered in Chapter 2. The final theorem in this review from calculus describes the Taylor polynomials. These polynomials are used extensively in numerical analysis. (Taylor's Theorem) Suppose f € C"[a, b), that f+” exists on [a, b], and xo € [a, b]. For every x € [a,b], there exists a number (x) between xp and x with F(&) = Pax) + Ral), where £0) +2 = 20)" 7 ° ~ Xo) and _ £0) an Ral) = ep or . Here P,(x) is called the nth Taylor polynomial for f about x9, and R, (x) is called the remainder term (or truncation error) associated with P, x). The infinite series obtained by taking the limit of P,(x) as n — 00 is called the Taylor series for f about xo. In the case xo = 0, the Taylor polynomial is often called a Maclaurin polynomial, and the ‘Taylor series is called a Maclaurin series. ‘The term truncation error refers to the error involved in using a truncated, or finite, summation to approximate the sum of an infinite series. Determine (a) the second and (b) the third Taylor polynomials for f(x) = cos.x about x0 = 0, and use these polynomials to approximate cos(0.01). (c) Use the third Taylor polynomial and its remainder term to approximate f° cos x dx. Since f € C™(R), Taylor's Theorem can be applied for any n > 0. Also, f'@) = =sinx, f"(x) =—cosx, f"(x) =sinx and f(x) = cosx2 CHAPTER 1 * Mathematical Preliminaries F(0) =1, fO) =0, f"(0)=-1 and f"()=0. a. Forn =2and x = 0, we have cosx =1— ty bys cost = 1 5x7 + Gx? sing(x), where &(x) is a number between 0 and x. (See Figure 1.12.) Figure 1.12 With x = 0.01, the Taylor polynomial and remainder term are 080.01 F000? + 710.01) sing) = 0,99995 + 0.16 x 10-*sing(x), where 0 < £(x) < 0.01. (The bar over the 6 in 0.16 is used to indicate that this digit repeats indefinitely.) Since | sin€(x)| <1 for all x, we have [cos 0.01 — 0.99995] < 0.16 x 10°%, so the approximation 0.99995 matches at least the first five digits of cos0.01, and 0.9999483 < 0.99995 — 1.6 x 10-6 < cos0.01 < 0.99995 + 1.6 x 10% < 0.9999517. The error bound is much larger than the actual error. This is due in part to the poor bound we used for | sin (x)|. It can be shown that for all values of x, we have | sinx| < |x|. Since 0 < & < 0.01, we could have used the fact that sin (x)] < 0.01 in the error formula, producing the bound 0.16 x 10°,11 Review of Calculus B b. Since f’”(0) = 0, the third Taylor polynomial with remainder term about x9 is 1 cosx == 5x7 + = A cosE(x), where 0 < &(x) < 0.01. The approximating polynomial remains the same, and the approximation is still 0.99995, but we now have a much better accuracy as- surance. Since | cos §(x)| < 1 for all x, we have |zetomien| < Faonke 4210" So | cos 0.01 — 0.99995] < 4.2 x 10-", and 0.99994999958 = 0.99995 — 4.2 x 1071 < c0s0.01 < 0.99995 + 4.2 x 10" = 0.99995000042. ‘The first two parts of this example illustrate the two objectives of numerical analysis. The first is to find an approximation, which the Taylor polynomials in both parts provide. The second is to determine the accuracy of the approximation. In this case, the third Taylor polynomial was much more informative than the ‘second, even though both polynomials gave the same approximation. ¢. Using the third Taylor polynomial gives 01 oy 1 pty. [ cossas fOr )etaf x* cos (x) dx =[x-1e Pk [ tooteras 6 ty 24 So 3 af 4 cos E (1)? + 35 J) x*costx) dx. Therefore, at 1 = f osx dx = 0.1 — gour = 0.09983. 5 ‘A bound for the error in this approximation is determined from the integral of the ‘Taylor remainder term and the fact that | cos (x)| < 1 for all x: 1 or 1p ye alf x costa) ds| = 3 f *|cos§(x)| dx 1p, wos f:=cos(x); ‘Maple allows us to place multiple statements on a line and to use a colon to suppress Maple responses. For example, we obtain the third Taylor polynomial with >a3:mtaylor(£,x-0,4): p3:i=convert(s3, polynom); ‘The first part computes the Taylor series with four terms (degree 3) and remainder ex- panded about x9 = 0. The second part converts the series s3 to the polynomial p3 by dropping the remainder. To obtain 11 decimal digits of display we enter >Digits and evaluate f (0.01), P;(0.01), and | f (0.01) — P3(0.01)| with >y1:sevalf (subs (x=0.01,£)); >y2:sevalt (subs (x=0.01,p3)); >err:=abs (y1-y2); ‘This produces y; = f(0.01) = 0.99995000042, y2 = P3(0.01) = 0.99995000000, and [,f (0.01) ~ P;(0.01)| = 42 x 10 To obtain a graph similar to Figure 1.12, enter >plot({ £,p3 },x=-Pi. Pi); ‘The commands for the integrals are dqiimint (f,x=0..0.1); >q2:=int (p3,x=0..0.1); >err:=abs (qi-q2) ; which give the values 01 0 a= £0) dx = 0.099833416647 and gp = f Ps(x) dx = 0,099833333333, lo 0 with error 0.83314 x 1077. Parts (a) and (b) of the example show how two techniques can produce the same ap- proximation but have differing accuracy assurances. Remember that determining approx- mations is only part of our objective. An equally important partis to determine at least a bound for the accuracy of the approximation.MW Review of Calculus 5 eo EXERCISE SET 1.1 10. con Show that the following equations have at least one solution in the given intervals. (0.2, 0.3) and (1.2, 1.3] (1, 2] and fe, 4] (2, 3] and [3, 4) a. xcosx— 2x? +3x—1 bo -2h-In e& 2xcos(2x) ~ (= 2 4 x= (nx =0, [4,5] Find intervals containing solutions to the following equations. a x-3*=0 bo 4x?@e" =0 & Pa2F ae $3=0 4244.01 + 4.002 + 1.101 = 0 Show that /"(x) is Oat least once in the given intervals. a fa)slme + (e-Dsin(r/Dx), (0,11 @—Itanx +xsinnx, (0,1) singx ~(x~2)Inx, (1,2) af) = —2sinxInxe +2), (1,3) Find maxecss» |/(2)| for the following functions and intervals. a f@)=Q-e+20/3, 10.11 Bf) = Gr- D/O 20), (05,1) & f(a) = 2xcos2x)—- 27, (2,41 a fayatte ee, (1,2) Use the Intermediate Value Theorem and Rolle’s Theorem to show that the graph of (x) = x + 2x +k crosses the x-axis exactly once, regardless of the value of the constant Suppose f € Cla, b] and f"(x) exists on (a, 6). Show that if f"(x) # 0 for all x in (a,b), then there can exist at most one number p in fa, b] with f(p) Let fey =. ‘a, Find the second Taylor polynomial P,(x) about b. Find (0.5) and the actual error in using P3(0.5) to approximate (0.5) Repeat part (@) using x9 = 1 4. Repeat part b) using the polynomial from part (c). Find the third Taylor polynomial P,(z) for the function f(x) = VF about xp = 0. Ap- proximate /0:5 , 0.75, V1-25, and VTS using Ps(x), and find the actual errors. Find the second Taylor polynomial P(x) for the function f(x) = e* cos.x about xy = 0. ‘a, Use P,(0.5) to approximate (0.5). Find an upper bound for error |/'(0-5) ~ P2(0.5)) using the error formula, and compare it to the actual error. 1b. Find a bound for the error | f(x) — P2(x)| in using P(x) to approximate f(x) on the interval (0, 1) ¢. Approximate f) f(x) dx using j) P,(x) dx. d. Find an upper bound for the error in (c) using ' |R:(x) dx|, and compere the bound to the actual enor. Repeat Exercise 9 using x = 7/6. Find the third Taylor polynomial s(x) for the function f(x) = (x — 1) nx about x = 1.16 CHAPTER 1 + Mathematical Preliminaries 2 14. 18. 16. 17. 18, 19. 21. Use Ps(0.5) to approximate /(0.5). Find an upper bound for error |f(0.5) — P3(0.5)] using the error formula, and compare it to the actual error. b. Find a bound for the error | f(x) — Pj(x)| in using P(x) to approximate f(x) on the interval (0.5, 1.5] c. Approximate fy'g f(x) dx using 5 Py(x) dx. 4. Find an upper bound for the error in (c) using /_'; |Rs(x) dx|, and compare the bound to the actual error. Let f(x) = 2x cos(2x) — (x ~ 2)? and x0 = 0. a, Find the third Taylor polynomial s(x), and use it to approximate (0.4) 1b. Use the error formula in Taylor's Theorem to find an upper bound for the error | (0.4) — Ps(0.4)|. Compute the actual error. ©. Find the fourth Taylor polynomial P4(x), and use it to approximate f (0.4). 4. Use the error formuis in Taylor's Theorem to find an upper bound for the error | f (0.4) — ,(0.4)|. Compute the actual estor. Find the fourth Taylor polynomial P.(x) for the function f(x) = xe" about x ‘8. Find an upper bound for | (x) — P4(x)|, for 0 < x < 0.4 Approximate fp f(x) dx using Jj'* Pa(x) dex. Find an upper bound forthe eror in () using f* P(x) dx 4. Approximate "(0.2) using P{(0.2), and find the error. Use the error term of a Taylor polynomial to estimate the error involved in using sinx ~ x to approximate sin 1°. Use a Taylor polynomial about x /4 to approximate cos 42° to an accuracy of 10-6 Let f(x) = e*? sin(x/3). Use Maple to determine the following a, The third Maclaurin polynomial P(x). 1b. f(x) and a bound forthe error | f(x) — Ps(x)| on [0,11 Let f(x) = In(a? +2). Use Maple to determine the following, a. The Taylor polynomial Ps(x) for f expanded about x9 = 1. The maximum error | f(x) ~ Pa(x)), for 0 < x < 1 ¢. The Maclaurin polynomial P(x) for f. @. The maximum error jf (x) — Py(x)], for R is said to satisfy a Lipschitc condition with Lipschitz constant L on (a,b) if, for every x, y € fa, b], we have If (x) — FOI < Lix — yl a. Show that if f satisfies a Lipschitz condition with Lipschitz constant Zon an interval {a, 6}, then f € Cla, 6). 'b. Show that iff has a derivative that is bounded on (a, 6] by L, then f satisfies a Lipschitz condition with Lipschitz constant L on {a, b] Give an example ofa function that is continuous on a closed interval but does not satisfy a Lipschitz condition on the interval. Suppose f € Cla, b), that x: and xz are in (a, b], and that c, and cz are positive constants. Show that a number & exists between x; and x, with caf) + eof G2) 46) ate Let f € Cla, 6], and let p be in the open interval (a, b). a. Suppose f(p) # 0. Show that a 6 > Ocxists with f(x) # 0, for all x in [p — 8, p +8}. with [p — 5, p +8] a subset of (a, 8). b. Suppose f(p) = 0 and k > Os given, Show that a 8 > O exists with |(x)| < k, forall xin [p ~5, p +8], with [p ~ 5, p+] a subset of [ab]18 CHAPTER 1 = Mathematical Preiminaries 1.2 Roundoff Errors and Computer Arithmetic The arithmetic performed by a calculator or computer is different from the arithmetic in our algebra and calculus courses. From your past experience, you might expect that we always have as true statements such things as 2 +2 = 4, 4-4 = 16, and (V3)? = 3. In standard computational arithmetic we will have the first two but not always the third. To understand why this is true we must explore the world of finite-digit arithmetic. In our traditional mathematical world we permit numbers with an infinite number of digits. The arithmetic we use in this world defines /3 as that unique positive number that when multiplied by itself produces the integer 3. In the computational world, however, each representable number has only a fixed, finite number of digits. This means, for example, that only rational numbers—and not even all of these—can be represented exactly. Since V3 is not rational, it is given an approximate representation, one whose square will not be precisely 3, although it will likely be sufficiently close to 3 to be acceptable in most situations. In most cases, then, this machine arithmetic is satisfactory and passes without notice or concern, but at times problems arise because of this discrepancy. Roundoff error is produced when a calculator or computer is used to perform real- number calculations. It occurs because the arithmetic performed in a machine involves numbers with only a finite number of digits, with the result that calculations are performed with only approximate representations of the actual numbers. In a typical computer, only a relatively small subset of the real number system is used for the representation of all the real numbers. This subset contains only rational numbers, both positive and negative, and stores the fractional part, together with an exponential part. In 1985, the IEEE (Institute for Electrical and Electronic Engineers) published a report called Binary Floating Point Arithmetic Standard 754-1985. Formats were specified for single, double, and extended precisions, and these standards are generally followed by all. microcomputer manufacturers using floating-point hardware. For example, the numerical coprocessor for PCs implements a 64-bit (binary digit) representation for a real number, called a long real. The first bit is a sign indicator, denoted s. This is followed by an 11-bit exponent, c, called the characteristic, and a 52-bit binary fraction, f, called the mantissa. The base for the exponent is 2. Since 52 binary digits correspond to between 16 and 17 decimal digits, we can as- sume that a number represented in this system has at least 16 decimal digits of precision. The exponent of 11 binary digits gives a range of 0 to 2'' — 1 = 2047. However, using only positive integers for the exponent would not permit an adequate representation of ‘numbers with small magnitude. To ensure that numbers with small magnitude are equally representable, 1023 is subtracted from the characteristic, so the range of the exponent is actually from —1023 to 1024. To save storage and provide a unique representation for each floating-point number, a normalization is imposed. Using this system gives a floating-point number of the form (pra +f). Consider for example, the machine number © 1000000001 1 1011100100010000000000000000000000000000000000000000.1.2 Roundoff Errors and Computer Arithmetic 19 The leftmost bit is zero, which indicates that the number is positive. The next 11 bits, 1000000001 1, giving the characteristic, are equivalent to the decimal number = 1-2 40.28% 4.040.241.2412? = 1024 +241 = 1027 ‘The exponential part of the number is, therefore, 2!0?7I25 = 24. The final 52 bits specify that the mantissa is 1! 1? 1s ns nt r=¥(3) 1G) 1G) 1G) +1 G) ‘As a consequence, this machine number precisely represents the decimal number nr2a 4 y= ee. 2-08 (14(F444 44545450 2° 8 16 32 256 4096 = 27.56640625. However, the next smallest machine number is © 10000000011 1011 10010000111 1111ENTIIL00 0 L1M111111TTTTITIIITIIL, and the next largest machine number is (0 10000000011 1011100100010000000000000000000000000000000000000001 This means that our original machine number represents not only 27.56640625, but also half of the real numbers that are between 27.56640625 and its two nearest machine-number neighbors. To be precise, it represents any real number in the interval [27.566406249999998223643 16059974953532218933 10546875, 27.5664062500000017763568394002504646778 106689453125). ‘The smallest normalized positive number that can be represented has all Os except for the rightmost bit of 1 and is equivalent to 270. 2°) = 10-8, and the largest has a leading 0 followed by all 1s and is equivalent to 2104. (2 — 2-5) = 10°08 ‘Numbers occurring in calculations that have a magnitude less than 2~!% . (1 +2-**) result in underflow and are generally set to zero. Numbers greater than 2! . (2 ~ 2~S2) result in overflow and typically cause the computations to halt. ‘The use of binary digits tends to conceal the computational difficulties that occur when a finite collection of machine numbers is used to represent all the real numbers. To examine these problems, we now assume, for simplicity, that machine numbers are represented in the normalized decimal floating-point form £0.did2...de x10", 1Digits: causes all arithmetic to be rounded to t digits. For example, fI(fl(«)+ sly) is performed using f-digit rounding arithmetic by >evalf (ovals (x) +evalt (y)) ; Implementing -digit chopping arithmetic is more difficult and requires a sequence of steps ora procedure, Exercise 27 explores this problem. ‘Suppose that x = }, and that five-digit chopping is used for arithmetic calculations involving x and y. Table 1.2 lists the values of these computer-type operations on f(x 0.71428 x 10° and f1(y) = 0.33333 x 10°. . Operation Result Actual value Absolute error Relative error rey 0.10876 x 108 22/21 0.190 x 10 0.182 x 10-4 xoy 0.38095 x 10° 8/21 0.238 x 10-5 0.625 x 10° xey 0.23809 x 10° 5/21 0.524 x 10° 0.220 x 104 yox 0.21428 x 10! 15/7 0.571 x 104 0.267 x 10-*Table 13 1.2. Roundoff Errors and Computer Arithmetic. 23 Since the maximum relative error for the operations in Example 3 is 0.267 x 10-*, the arithmetic produces satisfactory five-digit results. Suppose, however, that we also have 0.714251, v = 98765.9, and w = O.L11111 x 10-4, so that f/(u) = 0.71425 x 10°, fI(v) = 0.98765 x 10°, and fi(w) = 0.11111 x 10~*. (These numbers were chosen to illustrate some problems that can arise with finite-digit arithmetic.) In Table 1.3, x © w results in a small absolute error but a large relative error. The sub- sequent division by the small number w or multiplication by the large number v magnifies the absolute error without modifying the relative error. ‘The addition of the large and small numbers u and u produces large absolute error but not large relative error. Operation Result Actual value Absolute error Relative error x6u 0.30000 x 10 0.34714 104 0.471 x 10 0.136 (ew Bw 0.29629 x 10! 0.34285 x 10! 0.465 0.136 (26x) @v 0.29629 x 10 0.34285 x 10! 0.465 0.136 ueu 0.98765 x 10° 0.98766 x 10° 0.161 x 10! 0.163 x 10 One of the most common error-producing calculations involves the cancellation of significant digits due to the subtraction of nearly equal numbers. Suppose two nearly equal numbers x and y, with x > y, have the k-digit representations {UK) = O.didy «.dp6tpyrotpa2 «ct, x 10" and FULY) = O.did..dpBp41Bpi2 +++ Be x 10" The floating-point form of x — y is SUFUR) = FLY) = 0.6 p410p42---04 x 10", where Oops open -.0e = Outpt tps2 -+ Oe — O-BpriBps2 --- Br The floating-point number used to represent x — y has at most k — p digits of significance. However, in most calculation devices, x — y will be assigned k digits, with the last p being either zero or randomly assigned. Any further calculations involving x ~ y retain the problem of having only k — p digits of significance, since a chain of calculations is no more accurate than its weakest portion. Ifa finite-digit representation or calculation introduces an error, further enlargement of the error occurs when dividing by a number with small magnitude (or, equivalently, when multiplying by a number with large magnitude). Suppose, for example, that the number z has the finite-digit approximation z + 6, where the error 6 is introduced by representation or by previous calculation. Suppose we now divide by © = 107", where m > 0. Then 20 (fI@) _ x10 xn (Z2) = (+8) x 10"EXAMPLE 4 EXAMPLE 5 CHAPTER 1 + Mathematical Preliminaries ‘Thus, the absolute error in this approximation, |3| x 10*, is the original absolute error, [5], multiplied by the factor 10". Let p = 0.54617 and q = 0.54601. The exact value of r = p—g is r = 0.00016. Suppose the subtraction is performed using four-digit arithmetic. Rounding p and q to four digits gives p* = 0.5462 and q* = 0.5460, respectively, and r* = p* — q* = 0.0002 is the four-digit approximation to r. Since Ir —r*| __ {0.00016 — 0.0002) rl 10.0016) the result has only one significant digit, whereas p* and q” were accurate to four and five significant digits, respectively. If chopping is used to obtain the four digits, the four-digit approximations to p,q, and = 0.25, rare p* = 0.5461, q° = 0.5460, and r* = p* — q* = 0.0001. This gives —r*| _ (0.00016 — 0.0001) Ir |0.00016) 0375, which also results in only one significant digit of accuracy. . ‘The loss of accuracy due to roundoff error can often be avoided by a reformulation of the problem, as illustrated in the next example. The quadratic formula states that the roots of ax? + bx +c = 0, when a # 0, are b+ VP fac —b - VP Fac n= We a = aps Using four-digit rounding arithmetic, consider this formula pplied to the equation x? + 62.10x + 1 = 0, whose roots are approximately. 41 = —0.01610723 and x2 = —62.08390. In this equation, b? is much larger than 4ac, so the numerator in the calculation for x, involves the subtraction of nearly equal numbers. Since Vb? = 4ac = V(62.10)? — (4.000) (1.000) (1.000) = /3856. — 4.000 = v3852. = 62.06, we have —0.04000 FIED = —Jo60 = ~2.000- — ~07008- a poor approximation to x; = —0.01611, with the large relative error 0.01611 + 0.02000) _ = Toor; “24% 10 On the other hand, the calculation for x2 involves the addition of the nearly equal numbers —b and —./b? — 4ac. This presents no problem since 10 62.06 _ 2.000EXAMPLE 6 Table 1.4 1.2. Roundoff Erors and Computer Arithmetic 25 has the small relative error 62.08 + 62.10) _ |~ 62.08] To obtain a more accurate four-digit rounding approximation for x1, we change the form of the quadratic formula by rationalizing the numerator: =b + VBP = fae (25s) BP - (b= 4ac) 3.2.x 10. * 2a —b— Vi Fac) ~ 2a(—b — VB which simplifies to an alternate quadratic formula —2e n= 12 b+ Ve —4ac 02 Using (1.2) gives 2.000 _ -2.000 TU = ane = or” = 7 0.01610, FIG = By I9-+ 62.06 = 1242 which has the small relative error 6.2 x 1074 . The rationalization technique can also be applied to give the following alternative quadratic formula for x2: n=—* (13) ob Ve 4ac 4 This s the form to use if is a negative number. In Example 5, however, the mistaken use of, this formula for x» would result in notonly the subtraction of nearly equal numbers, but also the division by the small result of this subtraction. The inaccuracy that this combination produces, ~2.000 2,000 2¢ SlG2) = > aaa = G10 — 6.06 = 0.04000 = ~>°- has the large relative error 1.9 x 10-1. Accuracy loss due to roundoff error can also be reduced by rearranging calculations, as shown in the next example. Evaluate f(x) =x? — 6.1x? + 3.2x + 1.5 atx = 4.71 using three-digit arithmetic. Table 1.4 gives the intermediate results in the calculations. Carefully verify these re- sults to be sure that your notion of finite-digit arithmetic is correct. Note that the three-digit chopping values simply retain the leading three digits, with no rounding involved, and dif- fer significantly from the three-digit rounding values. x 2 x 6.1 3.2r Exact 471 221841 104487111 = «135.32301 «15.072 Three-digit (chopping) 4.71 22.1 104. 134. 150 Three-digit (rounding) 4.71—22.2 108. 135. 15.126 a CHAPTER 1 + Mathematical Preliminaries Exact: f (4.71) = 104.487111 ~ 135,32301 + 15.072 + 1. —14.263899; ‘Three-digit (chopping): (4.71) = ((104. ~ 134.) + 15.0) + 1.5 = Three-digit (rounding): f(4.71) = ((10S. ~ 135.) + 15.1) + 1.5 13.5; 13.4, The relative errors for the three-digit methods are ~ 0. choppit Feaag =| * 0.08, forchopping and 4.263899 + 13.4| —14.263899 | 0.06, for rounding. ‘Asan alternative approach, f(x) can be written in a nested manner as f(x) = —6.1x? +3.2x 415 = (Cx - 6.1) + 3.2)x + 1S, This gives Three-digit (chopping): (4.71) = (4.71 — 6.14.71 + 3.2)4.71 + 15 = —14.2 and a three-digit rounding answer of —14.3. The new relative errors are =14.263899 + 14.2 ‘Three-digit (chopping) | | ~ 0.0045; —14.263899 + 14.3 —14.263899 Three-digit (rounding): | ~ 0.0025. Nesting has reduced the relative error for the chopping approximation to less than 10% of that obtained initially. For the rounding approximation the improvement has been even. more dramatic; the error in this case has been reduced by more than 95%. Polynomials should always be expressed in nested form before performing an evalu- ation, because this form minimizes the number of arithmetic calculations. The decreased error in Example 6 is due to the reduction in computations from four multiplications and three additions to two multiplications and three additions. One way to reduce roundoff error is to reduce the number of error-producing computations. EXERCISE SET 1.2 1. Compute the absolute error and relative error in approximations of p by p* a pam p= 2/7 b. paz, pt =3.1416 718 a. p= V2, p' =1.41412 ca & 9. 10, nL Roundoff Errors and Computer Arithmetic 27 e f. p= 10", p= 1400 & p=8!, p= 39900 bh. p=9i, pt = VibrO/e Find the largest interval in which p* must lie to approximate p with relative error at most 10-* for each value of p. aon be « v2 a 7 Suppose p* must approximate p with relative error at most 10, Find the largest interval in ‘which p* must lie for each value of p. a 150 b. 900 1500 a 9 Perform the following computations (i exactly, (ii) using three-digit chopping arithmetic, and (iii) using three-digit rounding arithmetic. (iv) Compute the relative errors in parts (ii) and (ii 3 as a Use three-digit rounding arithmetic to perform the following calculations. Compute the abso- lute error and relative error with the exact value determined t0 at least five digits. a. 133 40.921 b 133 0.499 © (21-0327) -119 d. (121-119) ~ 0.327 f =10n + 6¢~ 3 & 2) (9 x « (5)-G) oF Repeat Exercise 5 using four-digit rounding arithmetic Repeat Exeris Suing thee digit chopping arithmetic. Repeat Exercise $ using four-digit chopping arithmetic. The fst tree noaero terms ofthe Maclanin series forthe arcangent function are x (1/3)x3 + (1/5)x5. Compute the absolute error and relative error in the following approxi- Imation of asing the polynomial in place ofthe aretangent wem(2)-me(5)] oe) =e) ‘The number e can be defined by e = S%o(1/n!), where n! = n(n — 1) -++2-1 for #0 and 0! = 1. Compute the absolute error and relative error in the following approximations of e: a. Find lim,.o f(x). 1b. Use four-digit rounding arithmetic to evaluate f (0.1). ¢. Replace each trigonometric function with its third Maclausin polynomial, and repeat part o. 4. The actual value is f(0.1) = ~1.99899998, Find the relative error forthe values obtained in parts (b) and (c).28 CHAPTER 1 * Mathematical Preliminaries 2 1B. 14, 15. 16. 1. 18, 19. Let e fos a. Find lim, soe" — e-*)/x. 7 three-digit rounding arithmetic to evaluate f(0.1).. Replace each exponential function with its third Maclaurin polynomial, and repeat part (b), 4d. The actual value is (0.1) = 2,003335000. Find the relative error for the values obtained in parts (a) and (b). Use four-digit rounding arithmetic and the formulas of Example 5 to find the most accurate approximations to the roots of the following quadratic equations. Compute the absolute errors and relative errors. 1, 13 1 1,13 1 Va 3 yt Lay 23, a atts boge tpt gn0 & 1.002x? ~ 11.01x + 0.01265 1.002%? + 11.01 +.0.01265 = 0 Repeat Exercise 13 using four-digit chopping arithmetic. Use the 64-bit long real format to find the decimal equivalent of the following floating-point machine numbers. . 010000001010 1001001 100000000000000000000000000000000000000000000 b. 110000001010 1001001 100000000000000000000000000000000000000000000 «0. 1111111111 0101001100000000000000000000000000000000000000000000 4. 0 01111111111 0101001:100000000000000000000000000000000000000000001 Find the next largest and smallest machine numbers in decimal form for the numbers given in Exercise 15. Suppose two points (xo, yo) and (x1, yi) are on a straight line with y; # yo. Two formulas are available to find the x-intercept ofthe line: (x1 = x0) ¥0 Yo 200 and x= x= % v a. Show that both formulas are algebraically correct b. Use the data (xo, yo) = (1.31, 3.24) and (x1, y1) = (1.99, 4.76) and three-digit rounding, arithmetic to compute the x-intercept both ways. Which method is better and why” The Taylor polynomial of degree n for f(x) = e* is Sfuo(x!/i!). Use the Taylor polynomial of degree nine and three-digit chopping arithmetic to find an approximation to e~* by each of the following methods. 2 a one ay “3 se An approximate value of e~S correct to three digits is 6.74 x 10-?. Which formula, (a) or (©), gives the most accuracy, and why? “The two-by-two linear system artby=e, extdy=f. where a,b, c,d, e, fare given, can be solved for x and y as follows:1.2. Roundoff Errors and Computer Arithmetic 29 21 23. setm=£, provided a # 0; dy =d—mb; ‘Solve the following linear systems using four-digit rounding arithmetic. a 1.130x ~ 6.990y = 14.20 b. 1.013x ~ 6.099y = 14.22 & 8.110x + 12.20y = 0.1370 @ -18.1Lx + 112.2y = -0.1376 Repeat Exercise 19 using four-digit chopping arithmetic. ‘a, Show that the polynomial nesting technique described in Example 6 can also be applied to the evaluation of F(x) = LOle* ~ 4.626 — 3.11e* + 12.26" ~ 1.99. b. Use three-digit rounding arithmetic, the assumption that e'....de x10". Hdgy > 5, then f(y) = O.dydp «de x 10" + 10°F] ( ) : a) > mB! The binomial coefficient describes the number of ways of choosing a subset of k objects from a set of m elements. ‘8, Suppose decimal machine numbers are of the form £0.didadads x 10", with Sd) $9, 05d, <9, if =2,3,4 and jn} $15, ‘What is the largest value of m for which the binomial coefficient (7) can be computed for all k by the definition without causing overflow’?30 CHAPTER 1 * Mathematical Preliminaries 27. 28. 'b. Show that (7) can also be computed by ‘m) _ my (m m—Kk+1 ()-@l ) (* 1 ) €. What isthe largest value of m for which the binomial coefficient (5) can be computed by the formula in part (b) without causing overflow? 4. Use the equation in (b) and four-digit chopping arithmetic to compute the number of possible S-card hands in a S2-card deck. Compute the actual and relative errors. Let f € Cla, b] be a function whose derivative exists on (a,b). Suppose f is to be evaluated at xo in (@, 8), but instead of computing the actual value f (2a), the approximate value, f(x), is the actual value of f at xo +, that, F(x) = fo +) a. Use the Mean Value Theorem to estimate the absolute error |f(%o) ~ f()| and the relative error | f (20) ~ F4o)|/Lf Ceo)| assuming (x9) #0. b. Ife = 5 x 10°6 and xp = 1, find bounds for the absolute and relative errors for f(ayaet fi, f(x) = sinx c. Repeat part (b) with ¢ = (5 x 10°)xy and x) = 10. The following Maple procedure chops a floating-point number x 10 digits chop:=proc(x,t); if x=0 then 0 else a:etrunc(avalf (logi0(abs(x)))) ; if e>0 then e:=et1 fi; x2:sevalf (trunc(x#10°{}(t-e) 10°C} (e-t)) 4 end Verify the procedure works for the following values. a, x= 124031, 1=5 bx = 124.036, 1=5 ce x= 124.031 dx = 124.036, 1 =5 00653, fx = 0.00656, 1=2 Bx = -0.00653, bh. x = 0.00656, 1 =2 ‘The opening example to this chapter described a physical experiment involving the tempera- ture of a gas under pressure. In this application, we were given P = 1.00 atm, V = 0.100 m*, 'N =0,00420 mol, and X = 0.08206, Solving for T in the ideal gas law gives PY __(1.00)(0.100) NR (0,00420)(0.08206) re. 290.15 K = In the laboratory, it was found that Twas 15°C under these conditions, and when the pressure ‘was doubled and the volume halved, 7 was 19°C, Assume that the data are rounded values accurate to the places given, and show that both laboratory figures are within the bounds of accuracy for the ideal gas law.1.3. Algorithms and Convergence 31 1.3 Algorithms and Convergence EXAMPLE 1 ‘Throughout the text we will be examining approximation procedures, called algorithms, involving sequences of calculations. An algorithm is a procedure that describes, in an unambiguous manner, a finite sequence of steps to be performed in a specified order. The object of the algorithm is to implement a procedure to solve a problem or approximate a solution to the problem. We use a pseudocode to describe the algorithms. This pseudocode specifies the form of the input to be supplied and the form of the desired output. Not all numerical procedures give satisfactory output for arbitrarily chosen input. As a consequence, a stopping tech- nique independent of the numerical technique is incorporated into each algorithm to avoid infinite loops. ‘Two punctuation symbols are used in the algorithms: A period (.) indicates the termination of a step, a semicolon (;) separates tasks within a step. Indentation is used to indicate that groups of statements are to be treated as a single entity. Looping techniques in the algorithms are either counter-controlled, such as, For i=1,2, an Set x =aj+i-h or condition-controlled, such as While i < N do Steps 3-6. To allow for conditional execution, we use the standard If.. then, or If. .then else constructions. The steps in the algorithms follow the rules of structured program construction. They have been arranged so that there should be minimal difficulty translating pseudocode into any programming language suitable for scientific applications. ‘The algorithms are liberally laced with comments. These are written in italics and contained within parentheses to distinguish them from the algorithmic statements. An algorithm to compute 8 Yontatetay, it32 EXAMPLE 2 CHAPTER 1 * Mathematical Preilminaries where N and the numbers x1, x2,... , xy are given, is described by the following: INPUT, 21,2, OUTPUT SUM = 0%, x; Step 1 Set SUM =0. (Initialize accumulator) Xa Step2 Fort = 1,2, set SUM = ‘Step 3 OUTPUT (SUM); STOP. (Add the next term.) ‘The Nth Taylor polynomial for f(x) = In.x expanded about xp = 1 is 1 Py(x) = Yo pm), and the value of In 1.5 to eight decimal places is 0.4054651 1. Suppose we want to compute the minimal value of N required for [lm 1.5 ~ Py(1.5)| < 10-* without using the Taylor polynomial remainder term. From calculus we know that if DX, ae is an alternating series with limit A whose terms decrease in magnitude, then A and the Nth partial sum Ay = 3°, a differ by less than the magnitude of the (N + 1)st term; that is, 1A ~ Ant < lawerl- The following algorithm uses this bound. INPUT value x, tolerance TOL, maximum number of iterations M. OUTPUT degree NV of the polynomial or a message of failure. Step 1 SetN POWER = y; TERM = y; SIGN = 1. (Used to implement alternation of signs.) Step2. While N < M do Steps 3-5. Step3 Set SIGN = -SIGN; (Alternate the signs.) SUM = SUM + SIGN: TERM; (Accumulate the terms.) POWER = POWER - y, TERM = POWER/(N +1). (Calculate the next term.)Definition 1.17 EXAMPLE 3 1.3. Algorithms and Convergence 33 Step 4 If |TERM| 0 is introduced at some stage in the calculations and that the magnitude of the error after n subsequent operations is denoted by E,. The two cases that arise most often in practice are defined as follows. ‘Suppose that Ey > 0 denotes an initial error and E,, represents the magnitude of an error after n subsequent operations. If E, * CnEo, where C is a constant independent of n, then the growth of error is said to be linear. If E, © C” Eo, for some C > I, then the growth of error is called exponential. . Linear growth of error is usually unavoidable, and when C and Ep are small the re- sults are generally acceptable. Exponential growth of error should be avoided, since the term C” becomes large for even relatively small values of n. This leads to unacceptable inaccuracies, regardless of the size of Ep. As a consequence, an algorithm that exhibits linear growth of error is stable, whereas an algorithm exhibiting exponential error growth is unstable. (See Figure 1.13 on page 34.) ‘The recursive equation 10 Pn-1~ Pr-2, form has the solution34 CHAPTER 1 + Mathematical Preliminaries igure 1.13 Unstable exponential growth © E,= CE, Stable linear growth e.et E, = CnE ott 2345678 n 10 y Pa = Pana (lV? p10 1 2 f 10 =a(3) [B.g-1] +03 [F s-1] i 1 3-29) =, (1)" 4 ea" = a(G (5) +« @=a(5) +023" = Pp 1 and p; = 3, we have c; = 1 and cz = 0, so py = (4)" for all n, Suppose that git rounding arithmetic is used to compute the terms of the sequence given by this ‘equation, Then fo = 1.0000 and 6 = 0.33333, which requires modifying the constants to 4, = 1.0000 and é, = —0.12500 x 10-5. The sequence {/i,]22g generated is then given bn = 1. oo0o (3) = 0.12500 x 10°5(3)", and the roundoff error, Pn — Pn = 0.12500 x 10-5(3"), grows exponentially with n. This is reflected in the extreme inaccuracies after the first few terms, as shown in Table 1.5. ‘On the other hand, the recursive equation Pe =2Pn-1— Pars form =2,3,..Table 15 Table 1.6 1.3. Algorithms and Convergence 35 a ‘Computed Relative Error 0 0.10000 x 10! x10" 1 0.33333 x 10° 0.33333 x 10° 2 0.11110 x 10" O.LIILT x 10° 9x 10-5 3 0.37000 x 10°" 0.37037 x 10"! 1x 107 4 0.12230 x 10" 0.12346 x 107 9x 10-9 5 0.37660 x 10 0.41152 x 107 8 x 107 6 0.32300 x10 0.13717 x 107 8 x 10-! 7 0.26893 x 10°? 0.45725 x 1073 7x10 8 ~0.92872 x 10 0.15242 x 10> 6x 10! has the solution p, = c1 + can for any constants cy and c; because 2Pn~1 — Pr—2 = Aci + €2(m — 1) = (cr + c2(n — 2) = c(2 1) + c22n —2—n +2) =e; + ean = Pp and: = -3, 4, the constants in this equation become ¢) = — $n. Five-digit rounding arithmetic in this case results in fp = 1.0000 and 0.33333. As a consequence, the five-digit rounding constants are ¢, = 1.0000 and & = —0.66667. Thus, Bn = 1.0000 — 0.66667, and the roundoff error is Pn Pr = (0.6667 3) which grows linearly with n. This is reflected in the stability found in Table 1.6. . nm Computed pi, Conrect p, Relative Error oO 0.10000 x 10" 0.10000 x 10 1 0.33333 x 10° 0.33333 x 10° 2 —0.33330 x 10° =0.33333 x 10° 9x 10% 3 0.10000 x 10! 0.10000 x 10! 0 4 0.16667 x 10! 0.16667 x 10! 0 Ss 23334 x 10! 0.23333 x 10! 4x 10° 6 0.30000 x 10! —0,30000 x 10! 0 7 —0.36667 x 10" 0.36667 x 10° 0 8 0.43334 x 10" 0.43333 x 10" 2x105 The effects of roundoff error can be reduced by using high-order-digit arithmetic such as the double- or multiple-precision option available on most computers. Disadvantages in using double-precision arithmetic are that it takes more computation time and the growth of roundoff error is not eliminated but is only postponed until subsequent computations are performed.EXAMPLE 4 Table 1.7 CHAPTER 1 + Mathematical Preliminaries One approach to estimating roundoff error is to use interval arithmetic (that is, to retain the largest and smallest possible values at each step), so that, in the end, we obtain an interval that contains the true value. Unfortunately, a very small interval may be needed for reasonable implementation. Since iterative techniques involving sequences are often used, this section concludes with a brief discussion of some terminology used to describe the rate at which convergence ‘occurs when employing a numerical technique. In general, we would like the technique to converge as rapidly as possible. The following definition is used to compare the conver- gence rates of various methods. Suppose (Bn}22, is a sequence known to converge to zero, and {ay}2 ‘number a. If a positive constant K exists with 1 converges to a [dp — al = KlPal, for large n, then we say that (o7,]°° , converges to a with rate of convergence 0 (,). (This expression is read “big oh of f, .”) It is indicated by writing a, = a + O(,). . Although Definition 1.18 permits {a.,}%., to be compared with an arbitrary sequence {n}%.;, in nearly every situation we use f= for some number p > 0. We are generally interested in the largest value of p with a, = a+ O(I/n?). Suppose that, for n > 1, ntl 5 at3 = and Gy = . me Oy = me Although both lity +00, = 0 and litty.+2o dé, = 0, the sequence (&) converges to this, limit much faster than the sequence {a,}, using five-digit rounding arithmetic, as shown in Table 1’ ® 1 2 3 4 5 6 7 a, 2.00000 0.75000 0.44444 0.31250 0.24000 0.19444 0.16327 4, 4.00000 0.62500_0.22222—0.10938 0.064000 0.041667 0.029155 If we let By = 1/n and J, = 1/n?, we see that n+l ontn lo, - O| = me1.3. Algorithms and Convergence 37 80 wa0+0(!) amt a =0+0(4). ‘The rate of convergence of {ay} to zero is similar to the convergence of (1/n) to zero, whereas (4,} converges to zero at a rate similar to the more rapidly convergent sequence {1/n?). . & ‘We also use the “big oh” notation to describe the rate at which functions converge. Definition 1.19 Suppose that limo G(h) = O and limy..0 F(h) = L. If a positive constant K exists with |F(h) — L| < K|G(Q)|, for sufficiently small h, then we write F(A) = L + O(G(h)). . ‘The functions we use for comparison generally have the form G(h) = h?, where p > 0. We are interested in the largest value of p for which F(h) = L + O(h?). EXAMPLE 5 In Example 3(b) of Section 1.1 we found that the third Taylor polynomial gives 1 1 z sh = 1— sh? + ns . cos gh + gh" cos(h), for some number &(h) between zero and h, Consequently, sh tn? 1+ Lh cos: cosh + 5h? = 1+ 57h" cosh) ‘This result implies that 1 cosh + 5h? = 1+ O(h), since |(cosh + 4h) — 1| = |z cos &(h)|h* < Gh. The implication is that ash + 0, cosh + 1h? converges to its limit, 1, about as fast as h* converges to 0. . a EXERCISE SET 1.3 1. a Use three-digit chopping arithmetic to compute the sum 319, (1/i2) first by | + 1 + “+ dy and then by gig + 3 +++ + }. Which method is more accurate, and why? b. Write an algorithm to sum the finite series SO", x in reverse order 2. The number e is defined by ¢ = D22(1/n!), where n! = n(n — 1)-+-2- 1 form # 0 and 0! = 1. Use four-digit chopping arithmetic to compute the following approximations to e, and determine the absolute and relative error.38 CHAPTER 1 * Mathematical Preliminaries Sl a expt » mt « expt a 3, The Maclarinsevies forthe actangent function converges for—1 oo. a im sin =0 b. lim sin =0 1)? fs «jim (sin?) =o 4. im {Ing + 1) — In) = Find the rates of convergence of the following functions as h —+ 0. on si 1—cosh «ia ot » in =? sinh —hcosh 1-e © no ns are required to determine a sum of the form Lda st fot a, How many multiplications and addi b. Modify the sum in part (a) to an equivalent form that reduces the number of computations. Let P(x) = agx" +aq-ax""! +--+ +41 +a be a polynomial, and let x be given. Construct ‘an algorithm to evaluate P (xq) using nested multiplication, Example 5 of Section 1.2 gives altemative formulas forthe roots x1 and x3 of ax?-+bx +e = 0. Construct an algorithm with input a, 6, c and output x1, r2 that computes the roots x) and x2 (which may be equal or be complex conjugates) using the best formula for each root. Construct an algorithm that has as input an integer n > 1, numbers xo.x1.... a, and a number x and that produces as output the product (x —x0)(x — a1)-+-(x — ty). Assume that 4x3 8x7 1-2e 2x42? 1+ 2x Terte13 B. 14, 15. 16. 17. 18, Algorithms and Convergence 39 for x < 1, and Jet x = 0.25, Write and execute an algorithm that determines the number of terms needed on the left side of the equation so that the left side ders from the right side by less than 10-6. Suppose that 0 < q < p and that ay =a + O (n“?). Show that ay = a + 0 (n™) b. Make a table listing 1/n, 1/n?, 1/n*, and 1/n* form = 5, 10, 100, and 1000, and discuss the varying rates of convergence of these sequences as n becomes large. ‘a, Suppose that 0 4 digits of precision, a minimum exponent emin, and a maximum exponent emax. Then the set of floating-point numbers in this machine consists of 0 and the numbers of the form x= f-10%, where f= +(f107'+ fyl0? +--- + F109, where 1 < fi <9 and0< fi <9, foreach i = 2,... ,1, and where emin < € = emax: ‘These constraints imply that the smallest positive number represented in the machine is o = 10°""=!, so any computed number x with |x| < o causes underflow and results in x being set to 0. The largest positive number is 2. = (1 — 10!) 10°", and any com- puted number x with |x| > A causes overflow. When underflow occurs, the program will continue, often without a significant loss of accuracy. If overflow occurs, the program will fail The algorithm assumes that the floating-point characteristics of the machine are de~ scribed using parameters N, s, S, y, and Y. The maximum number of entries that can be summed with at least t/2 digits of accuracy is given by N. This implies the algorithm will proceed to find the norm of a vector x = (1, X2,... ,%,)! only ifm = N. To resolve the underflow-overflow problem, the nonzero floating-point numbers are partitioned into three groups: small-magnitude numbers x, those satisfying 0 < |x| < y; medium-magnitude numbers x, where y < |x| < ¥; and large-magnitude numbers x, where ¥ < |x|. The parameters y and Y are chosen so that there will be no underflow-overflow problem in squaring and summing the medium-magnitude numbers. Squaring small-magnitude num- bers can cause underflow, so a scale factor S much greater than 1 is used with the result that (Sx)? avoids the underflow even when x? does not. Summing and squaring numbers having a large magnitude can cause overflow, so in this case, a positive scale factor s much smaller than 1 is used to ensure that (sx)? does not cause overflow when calculated or incorporated into a sum, even though x? would. To avoid unnecessary scaling, y and Y are chosen so that the range of medium- magnitude numbers is as large as possible. The algorithm that follows is a modification42 CHAPTER 1 + Mathematical Preliminaries of one described in (Brow, W, p. 471]. It incorporates a procedure for scaling the compo- nents of the vector that are small in magnitude until a component with medium magnitude is encountered. It then unscales the previous sum and continues by squaring and summing small and medium numbers until a component with a large magnitude is encountered. Once a component with large magnitude appears, the algorithm scales the previous sum and pro- ceeds to scale, square, and sum the remaining numbers. The algorithm assumes that, in transition from small to medium numbers, unscaled small numbers are negligible when compared to medium numbers. Similarly, in transition from medium to large numbers, un- scaled medium numbers are negligible when compared to large numbers. Thus, the choices of the scaling parameters must be made so that numbers are equated to 0 only when they are truly negligible. Typical relationships between the machine characteristics as described by t,.0,, emin, emax, and the algorithm parameters Ns, S, y, and ¥ are given after the algorithm. ‘The algorithm uses three flags to indicate the various stages in the summation process. These flags are given initial values in Step 3 of the algorithm. FLAG 1 is I until a medium of large component is encountered; then it is changed to 0. FLAG 2 is 0 while small num bers are being summed, changes to 1 when a medium number is first encountered, and changes back to 0 when a large number is found. FLAG 3 is initially 0 and changes to 1 when a large number is first encountered. Step 3 also introduces the flag DONE, which is O until the calculations are complete, and then changes to 1 INPUT Ny s,S. 9, ¥odsm tis tases on OUTPUT NORM or an appropriate error message. Step 1 ifn <0 then OUTPUT (‘The integer n must be positive”); STOP, Step 2. Ifn > N then OUTPUT (‘The integer n is too large."); STOP, Step 3 Set SUM = 0; FLAGL (The small numbers are being summed.) FLAG2 FLAG3 DONE i=l Step 4 While (i n then set NORM = (SUM)'/S; DONE =1 else set SUM = (SUM/S)/S; (Scale for larger numbers.) FLAG2 Step 7 While (i m then set NORM DONE else set SUM FLAG3 (suM)'”, (SUM)s)s; (Scale the large numbers.) Step 10 While (i < n and FLAG3 = 1) do Step II. Step 11 Set SUM = SUM +(sx,)*; (Sum the large numbers.) i= +1 Step 12. If DONE =0 then if SUM"? < 4s then set NORM DONE else set SUM (The norm is too large.) Step 13 If DONE = 1 then OUTPUT (‘Norm is’, NORM) else OUTPUT (‘Norm >", NORM, ‘overflow occurred’). = (SUM)"7/s; Step 14 STOP. The relationships between the machine characteristics 1, 0, 2, emin, emax, and the algorithm parameters N, 5, S, y, and ¥ were chosen in [Brow, W, p. 471] as: N= 10%, where ey = [(¢ ~2)/2), _ the greatest imteger less than or equal to (f= 2)/2 10%, where e, = |—(emax + ey)/2]; s S= 10°, where es = [(1—emin)/2], the smallest integer greater than or equal to (I~ emin)/2; y= 10%, where ey = [(emin +1 ~ 2)/21; Y=10%, where ey = [(emax ~ ey)/2] The reliability built into this algorithm has greatly increased the complexity compared to the algorithm given earlier in the section. There are many forms of general-purpose numerical software available commercially and in the public domain. Most of the early software was written for mainframe computers, and a good reference for this is Sources and Development of Mathematical Software, edited by Wayne Cowell [Co]. Now that the desktop computer has become sufficiently powerful, standard numerical software is available for personal computers and workstations. Most of this numerical software is written in FORTRAN, although some packages are written in C, (C++, and FORTRANSO. ALGOL procedures were presented for matrix computations in 1971 in [WR]. A pack- age of FORTRAN subroutines based mainly on the ALGOL procedures was then devel- oped into the EISPACK routines. These routines are documented in the manuals publishedCHAPTER 1 = Mathematical Preliminaries by Springer-Verlag as part of their Lecture Notes in Computer Science series [Sm,B] and [Gar]. The FORTRAN subroutines are used to compute eigenvalues and eigenvectors for a variety of different types of matrices. The EISPACK project was the first large-scale numerical software package to be made available in the public domain and led the way for many packages to follow. EISPACK is maintained by netlib and can be found on the Internet at bttp://www.netlib.org/eispack. LINPACK is a package of FORTRAN subroutines for analyzing and solving systems of linear equations and solving linear least squares problems. The documentation for this, package is contained in [DBMS] and located at hutp://www.netlib.org/linpack. A step-by- step introduction to LINPACK, EISPACK, and BLAS (Basic Linear Algebra Subprograms) is given in (CV). ‘The LAPACK package, first available in 1992, is a library of FORTRAN subroutines that supercedes LINPACK and EISPACK by integrating these two sets of algorithms into a unified and updated package. The software has been restructured to achieve greater ef- ficiency on vector processors and other high-performance or shared-memory multipro- cessors. LAPACK is expanded in depth and breadth in version 3.0, which is available in FORTRAN, FORTRAN9O, C, C++, and JAVA. FORTRAN90, C, and JAVA are only available as language interfaces or translations of the FORTRAN libraries of LAPACK. ‘The package BLAS is not a part of LAPACK, but the code for BLAS is distributed with LAPACK. The LAPACK User's Guide, 3rd ed. [An] is available from SIAM or on the Internet at http://www.netlib.org/lapack/Iug/lapack Jug.html. The complete LAPACK or individual routines from LAPACK can be obtained through netlib at netlibornl.gov, netli- bresearch.att.com, or http://www.netlib.org/lapack. Other packages for solving specific types of problems are available in the public do- main. Information about these programs can be obtained through electronic mail by send- ing the line “help” to one of the following addresses: netlibresearch att.com, netlibornl.gov, netlibnac.no, or netlibdraci.cs.uow.edu.au or to the wucp address wunet!research!netlib. As an alternative to netlib, you can use Xnetlib to search the database and retrieve software. More information can be found in the article Software Distribution using Netlib by Don- garra, Roman, and Wade [DRW] These software packages are highly efficient, accurate, and reliable. They are thor- oughly tested, and documentation is readily available. Although the packages are portable, it is a good idea to investigate the machine dependence and read the documentation thor- oughly. The programs test for almost all special contingencies that might result in error and failures, At the end of each chapter we will discuss some of the appropriate general- purpose packages. ‘Commercially available packages also represent the state of the art in numerical meth- ods. Their contents are often based on the public-domain packages but include methods in libraries for almost every type of problem. IMSL (Intemational Mathematical and Statistical Libraries) consists of the libraries MATH, STAT, and SFUN for numerical mathematics, statistics, and special functions, re- spectively. These libraries contain more than 900 subroutines originally available in FOR- ‘TRAN 77 and now available in C, FORTRAN90, and JAVA. These subroutines solve the most common numerical analysis problems. In 1970 IMSL became the first large-scale scientific library for mainframes. Since that time, the libraries have been made available for computer systems ranging from supercomputers to personal computers. The libraries are available commercially from Visual Numerics, 9990 Richmond Ave $400, Houston,1.4 Numerical software 45 ‘TX 77042-4548, with Intemet address http://www.vni.com. The packages are delivered in compiled form with extensive documentation. There is an example program for each routine as well as background reference information. IMSL contains methods for linear systems, eigensystem analysis, interpolation and approximation, integration and differen- tiation, differential equations, transforms, nonlinear equations, optimization, and basic ma- trix/vector operations. The library also contains extensive statistical routines. The Numerical Algorithms Group (NAG) has been in existence in the United Kingdom since 1970. NAG offers more than 1000 subroutines in a FORTRAN 77 library, about 400 subroutines in a C library, over 200 subroutines in 2 FORTRAN 90 library, and an ‘MPI FORTRAN numerical library for parallel machines and clusters of workstations or personal computers. A subset of their FORTRAN 77 library (the NAG Foundation Library) is available for personal computers and workstations where work space is limited. The NAG C Library, the FORTRAN 90 library, and the MPI FORTRAN library offer many of the same routines as the FORTRAN Library. The NAG user's manual includes instructions and examples, along with sample output for each of the routines. A useful introduction to the NAG routines is {Ph}. The NAG library contains routines to perform most standard numerical analysis tasks in 2 manner similar to those in the IMSL. It also includes some statistical routines and a set of graphic routines. The library is commercially available from Numerical Algorithms Group, Inc., 1400 Opus Place, Suite 200, Downers Grove, IL 60515-5702, with Internet address http://www-nag.com. ‘The IMSL and NAG packages are designed for the mathematician, scientist, or engi- neer who wishes to call high-quality FORTRAN subroutines from within a program. The documentation available with the commercial packages illustrates the typical driver pro- ‘gram required to use the library routines. The next three software packages are stand-alone environments. When activated, the user enters commands to cause the package to solve a problem. However, each package allows programming within the command language. ‘MATLAB is a matrix laboratory that was originally a Fortran program published by Cleve Moler [Mo]. The laboratory is based mainly on the EISPACK and LINPACK subrou- tines, although functions such as nonlinear systems, numerical integration, cubic splines, curve fitting, optimization, ordinary differential equations, and graphical tools have been incorporated. MATLAB is currently written in C and assembler, and the PC version of this package requires a numeric coprocessor. The basic structure is to perform matrix opera- tions, such as finding the eigenvalues of a matrix entered from the command line or from an external file via function calls. This is a powerful self-contained system thet is especially useful for instruction in an applied linear algebra course. MATLAB has been available since 1985 and can be purchased from The MathWorks Inc., Cochituate Place, 24 Prime Park Way, Natick, MA 01760. The electronic mail address of The Mathworks is infomath- ‘works.com, and the Internet address is http://www.mathworks.com. MATLAB software is designed to run on many computers, including IBM PC compatibles, APPLE Macintosh, and SUN workstations. A student version of MATLAB does not require a coprocessor but will use one if it is available The second package is GAUSS, a mathematical and statistical system produced by Lee E. Ediefson and Samuel D. Jones in 1985. It is coded mainly in assembler and based primarily on EISPACK and LINPACK. As in the case of MATLAB, integra- tion/differentiation, nonlinear systems, fast Fourier transforms, and graphics are available. GAUSS is oriented less toward instruction in linear algebra and more toward statistical analysis of data. This package also uses a numeric coprocessor if one is available. It can46 CHAPTER 1 + Mathematical Preliminaries be purchased from Aptech Systems, Inc., 23804 S.E. Kent-Kangley Road, Maple Valley, WA 98038 (infoaptech.com). ‘The third package is Maple, a computer algebra system developed in 1980 by the ‘Symbolic Computational Group at the University of Waterloo. The design for the orig- inal Maple system is presented in the paper by B.W. Char, K.0. Geddes, W.M. Gen- tlemen, and G.H. Gonnet [CGGG]. Maple has been available since 1985 and can be purchased from Waterloo Maple Inc., 57 Etb Street, Waterloo, ON N21. 6C2. The elec- tronic mail address of Waterloo Maple is infomaplesoft.com, and the Internet address is hitp://www.maplesoft.com. Maple, which is written in C, has the ability to manipulate information in a symbolic manner. This symbolic manipulation allows the user to obtain exact answers instead of numerical values. Maple can give exact answers to mathematical problems such as integrals, differential equations, and linear systems. It contains a pro- gramming structure and permits text, as well as commands, to be saved in its worksheet files. These worksheets can then be loaded into Maple and the commands executed. Be- cause of the properties of symbolic computation, numerical computation, and worksheets, Maple is the language of choice for this text. Throughout the book Maple commands will ‘be embedded into the text. Numerous packages are available that can be classified as supercalculator packages for the PC. These should not be confused, however, with the general-purpose software listed here. If you have an interest in one of these packages, you should read Supercalculators on the PC by B. Simon and R, M. Wilson {SW}. Additional information about software and software libraries can be found in the books by Cody and Waite [CW] and by Kockler [Ko], and in the 1995 article by Dongarra and ‘Walker [DW]. More information about floating-point computation can be found in the book by Chaitini-Chatelin and Frayse [CF] and the article by Goldberg [Go]. Books that address the application of numerical techniques on parallel computers in- clude those by Schendell [Schel, Phillips and Freeman [PF], and Golub and Ortega [GO)CHAPTER 2 Solutions of Equations in One Variable The growth of a population can be modeled over short periods of time by assuming that the population grows continuously with time at a rate Proportional to the number present at that time. If we let N(é) denote the number at time ¢ and A denote the constant birth rate of the population, then the population satisfies the differential equation an eC ‘The solution to this equation is N(t) = Noe, where No denotes the initial population, PO) 3 uy = 1000 + 3 (es — 1) 1564 1435 Population (thousands) g Birth rate48 CHAPTER 2 = Solutions of Equations in One Variable ‘This exponential model is valid only when the population is isolated, with no immigration. If immigration is permitted at a constant rate v, then the differential equation becomes ano | Oe =ano+y, whose solution is No ae + 500% — 1). Suppose a certain population contains 1,000,000 individuals ini- tially, that 435,000 individuals immigrate into the community in the first year, and that 1,564,000 individuals are present at the end of one year. ‘To determine the birth rate of this population, we must solve for in the equation e*—1). 1,564,000 = 1,000,000¢* + Seem, ‘The numerical methods discussed in this chapter are used to approxi- ate solutions of equations of this type, when the exact solutions cannot be obtained by algebraic methods. The solution to this particular prob- lem is considered in Exercise 20 of Section 2.3. 1 The Bisection Method In this chapter, we consider one of the most basic problems of numerical approximation, the root-finding problem. This process involves finding a root, or solution, of an equation of the form f(x) = 0, fora given function f. A root of this equation is also called a zero of the function f. The problem of finding an approximation to the root of an equation can be traced back at least as far as 1700 B.C. A cuneiform table in the Yale Babylonian Collection dating from that period gives a sexigesimal (base-60) number equivalent to 1.414222 as an approximation to ¥/2, a result that is accurate to within 10-5. This approximation can be found by applying a technique described in Exercise 19 of Section 2.2 The first technique, based on the Intermediate Value Theorem, is called the Bisection, or Binary-search, method. Suppose f is a continuous function defined on the interval [a, 6}, with f(a) and f(b) of opposite sign. By the Intermediate Value Theorem, there exists a number p in (a, b) with f(p) = 0. Although the procedure will work when there is more than one root in the interval (a, b), we assume for simplicity that the root in this interval is unique. The method calls for a repeated halving of subintervals of [a, 6] and, at each step, locating the half containing p.Figure 2.1 2.1. The Bisection Method 49 To begin, set a, =a and b; = b, and let p; be the midpoint of [a, bj; that is, b-a +b raat ogo =S So If f(p:) = 0, then p = py, and we are done. If f(p:) # 0, then f(p1) has the same sign as either f(a;) or (by). When f(p,) and f(a)) have the same sign, p € (p1,51), and we set ay = p and by = by. When f(pi) and f (ay) have opposite signs, p € (a1, pi), and we set a) = a; and by = p,. We then reapply the process to the interval [az, b2]. This produces the method described in Algorithm 2.1. (See Figure 2.1.) Bisection ‘To find a solution to f (x) = 0 given the continuous function f on the interval (a, b], where f(a) and f(b) have opposite signs: INPUT endpoints a, b; tolerance TOL; maximum number of iterations No OUTPUT approximate solution p or message of failure. step 1 Seti =1; FA= f(a), Step 2 While i < No do Steps 3-6. Step 3 Set p=a+(b—a)/2; (Compute p,.) FP = f(p)50 CHAPTER 2 * Solutions of Equations in One Variable Step 4 If FP =O or (6 — a)/2 < TOL then OUTPUT (p); (Procedure completed successfully.) STOP. Step5 Seti=i+1 Step 6 If FA-FP > Othen seta =p; (Compute a;,bj.) FA= FP else set b = p. Step 7 OUTPUT (‘Method failed after No iterations, No (The procedure was unsuccessful.) STOP. . Other stopping procedures can be applied in Step 4 of Algorithm 2.1 or in any of the iterative techniques in this chapter. For example, we can select a tolerance ¢ > 0 and generate p1.... , pw until one of the following conditions is met: Ipw - Pw-il <€, Q1 \pw = Pw-i “a or Ife) <6. 23) Unfortunately, difficulties can arise using any of these stopping criteria, For example, there are sequences {p,}2. with the property that the differences p, — p»_1 converge to zero while the sequence itself diverges. (See Exercise 15.) Itis also possible for f(p,) to be close to zero while py differs significantly from p. (See Exercise 14.) Without additional knowledge about f or p, Inequality (2.2) is the best stopping criterion to apply because it comes closest to testing relative error. When using a computer to generate approximations. it is good practice to set an upper bound on the number of iterations. This will eliminate the possibility of entering an infinite loop, a situation that can arise when the sequence diverges (and also when the program is incorrectly coded). This was done in Step 2 of Algorithm 2.1 where the bound No was set and the procedure terminated if i > No. Note that to start the Bisection Algorithm, an interval (a, 6] must be found with f(a) - f(b) <0. At each step the length of the interval known to contain a zero of f is reduced by a factor of 2; hence it is advantageous to choose the initial interval {a, b) as small as possible, For example, if f(x) = 2x? — x? + x — 1, we have both f(-4)- FA) <0 and FO): f(D <0, so the Bisection Algorithm could be used on either of the intervals [—4, 4] or (0, 1]. Start- ing the Bisection Algorithm on [0, 1] instead of [~4, 4] will reduce by 3 the number of iterations required to achieve a specified accuracy.=XAMPLE 1 Table 2.1 2.1. The Bisection Method 51 The following example illustrates the Bisection Algorithm. The iteration in this exam- ple is terminated when the relative error is less than 0,0001; that is, when [P= Pal ig-4 \pl ‘The equation f(x) = x3 + 4x? — 10 = 0 has a root in [1,2] since f(1) = —5 and f(2) = 14. The Bisection Algorithm gives the values in Table 2.1. n a b, Pn S(Pn) 1 10 20 5 2375 2 10 1s 12s 1.79687 3 1.25 1s 1.375 0.16211 4 1.25 1375 13125 = 0.84839 s 13125 1375 1.34375 =0.35098 6 1.34375 1375 1.359375, ~0.09641 7 1.359375 1375 13671875 0.03236 8 1.359375 13671875 136328125 0.03215 9 1,36328125 13671875, 1,365234375 0.000072 10 1,36328125 1,365234375 1,364257813 0.01605 u 1.364257813 1,365234375 1364746004 =0.00799 12 1.364746094 1,365234375 1,364990235 0.00396 B 1.364990235 1.365234375 1365112305 0.00194 After 13 iterations, p13 = 1.365112305 approximates the root p with an error Ip — pisl < [bts — aya] = [1.365234375 — 1.365112305| = 0.000122070 Since Javai < |p|, Ip=pisl — Ipl fay so the approximation is correct to at least four significant digits. The correct value of p, to nine decimal places, is p = 1.365230013. Note that pe is closer to p than is the final approximation p;3. You might suspect this is true since | f(ps)| < |f(p13)}, but we cannot be sure of this unless the true answer is known, . The Bisection method, though conceptually clear, has significant drawbacks. It is slow to converge (that is, N may become quite large before |p ~ py| is sufficiently small), and a good intermediate approximation can be inadvertently discarded. However, the method has the important property that it always converges to a solution, and for that reason it is often used as a starter for the more efficient methods we will present later in this chapter.52 CHAPTER 2 + Solutions of Equations in One Vartable Theorem 2.1 Suppose that f ¢ C[a, b] and f(a)- f(b) < 0: The Bisection method generates a sequence {Pn}$2.; approximating a zero p of f with Since Its important to realize that Theorem 2.1 gives only a bound for approximation error and that this bound may be quite conservative. For example, this bound applied to the problem in Example 1 ensures only that 2-1 |p ~ pol = =2x 10%, but the actual error is much smaller: Ip — pol = |1.365230013 ~ 1.365234375| = 4.4 x 1076. EXAMPLE 2 ‘To determine the number of iterations necessary to solve f(x) = x? + 4x? — 10 = 0 with accuracy 10~ using a; = | and b; = 2 requires finding an integer N that satisfies *< 107. Ipw ~ pl <2-"(b—a) ‘To determine N we will use logarithms. Although logarithms to any base would suf- fice, we will use base-10 logarithms since the tolerance is given as a power of 10. Since 2-" < 10-3 implies that logyg2~™ < logy 10~? = —3, we have =Nlogip2<—3 and N> 9.96. Togio2 Hence, ten iterations will ensure an approximation accurate to within 10-3. Table 2.1 on page 51 shows that the value of py = 1.365234375 is accurate to within 10~*. Again, it2.1 The Bisection Method 53 is important to keep in mind that the error analysis gives only a bound for the number of iterations, and in many cases this bound is much larger than the actual number required. . The bound for the number of iterations for the Bisection method assumes that the calculations are performed using infinite-digit arithmetic. When implementing the method on a computer, consideration must be given to the effects of roundoff error. For example, the computation of the midpoint of the interval [4,, b,] should be found from the equation bn = Gn instead of from the algebraically equivalent equation ay thy The first equation adds a small correction, (b, — aq)/2, to the known value a,. When bq — ay is near the maximum precision of the machine this correction might be in error, but the error would not significantly affect the computed value of p,. However, when by — dy is near the maximum precision of the machine, it is possible for (ay + b,)/2 to return a midpoint that is not even in the interval (ay, b,]- Asa final remark, to determine which subinterval of [¢,, by] contains a root of f, itis better to make use of the signum function, which is defined as -1, ifx <0, sen) = 40, ifx 1, ifx>0. ‘The test sgn (f(@n)) sgn (f(Pn)) <0 — insteadof —f (an) f (Pn) < 0 gives the same result but avoids the possibility of overflow or underflow in the multiplica- tion of f (aq) and f (pn). | SSS SSS EXERCISE SET 2.1 1. Use the Bisection method to find ps for f(x) = J¥ — cos x on (0, 1) 2. Let f(x) = 3(x + 1) — 4)(x — 1). Use the Bisection method on the following intervals to find ps a [-2,15] be [-1.25,25] 3. Use the Bisection method to find solutions accurate to within 10"? for x? — 7x4 14x ~6 =0 on each interval. a (0,1) Bb (1,32) ce 3.2.4)54 CHAPTER 2 = Solutions of Equationsin One Variable 4 9%. 10. we 2 B. 14, 18. 16. a7. Use the Bisection method to find solutions accurate to within 10°? for x* — 2x? — 4x7 + 4x + 4 = Oon each interval. a [-2,-1] 10,2) © 23) a (-1,0) Use the Bisection method to find a solution accurate to within 10"? for x = tan.x on [4, 4.5]. Use the Bisection method to find a solution accurate to within 10-? for 2-+cos(e* —2)~e on 0.5, 1.5}. Use the Bisection method to find solutions accurate to within 10° for the following problems. a x-2%=0 for0 1 but that |p — ql < 10°? requires that n > 1000. Let (p,} be the sequence defined by p, = Yj. ¢- Show that (p,)} diverges even though Timy-sso(Px ~ Pr-1) =0. ‘The function defined by f(x) = sinsrx has zeros at every integer. Show that when —1 2 @ 1, if atb=2 A trough of length Z has a cross section in the shape of a semicircle with radius r (See the accompanying figure.) When filled with water to within a distance h of the top, the volume V of water is V=L[0.Smr? —r? aresin(h/r) — her? — 2)" =~:2.2 Fixed-Point iteration 55 Suppose Z = 10 ft,r = 1 ft, and ¥ = 12.4 1°. Find the depth of water in the tough to within 0.01 ft 18. A particle starts at rest on a smooth inclined plane whose angle @ is changing at a constant rate 49 Fae<0 ‘Atte end of seconds, the position ofthe object i given by xoe-35 ( - snes) 2 Suppose the particle has moved 1.7 ft in 1 s. Find, to within 10°, the rate « at which @ changes. Assume that g = 32.17 fus?. 2 2.2. Fixed-Point Iteration EXAMPLE 1 ‘A number p is a fixed point for a given function g if g(p) = p. In this section we consider the problem of finding solutions to fixed-point problems and the connection between the fixed-point problems and the root-finding problems we wish to solve. Root-finding problems and fixed-point problems are equivalent classes in the follow- ing sense: Given a root-finding problem f(p) = 0, we can define functions g with a fixed point at p in a number of ways, for example, as g(x) = x — f(x) or as g(x) = x +3f (x). Conversely, if the function g has a fixed point at p, then the function defined by f(x) x — g(x) has a zero at p. Although the problems we wish to solve are in the root-finding form, the fixed-point form is easier to analyze, and certain fixed-point choices lead to very powerful root-finding techniques. We first need to become comfortable with this new type of problem and to decide when a function has a fixed point and how the fixed points can be approximated to within a specified accuracy. ‘The function g(x) =x? — 2, for -2 < x < 3, has fixed points at x = —1 and x = 2 since a(-I)=(-1?-2=-1 and g(2)=2?-2=2. ‘This can be seen in Figure 2.2. .56 CHAPTER 2 = Solutions of Equations in One Variable Figure 22 ‘The following theorem gives sufficient conditions for the existence and uniqueness of a fixed point. Theorem 2.2 a. If g € C[a,b] and g(x) € [a,b] for all x € [a, 6), then g has a fixed point in (a,b). b. If, in addition, g’ (x) exists on (a, b) and a positive constant k < 1 exists with Is'@)| sk, forall x € (a,b), then the fixed point in {a, 5] is unique. (See Figure 2.3.) .EXAMPLE 2 2.2. Fixed-Point iteration 57 Proof a If g(a) = a or g(b) = b, then g has a fixed point at an endpoint. If not, then (a) > a and g(b) < b, The function h(x) = g(x) — x is continuous on [a, b], with h(a) ‘The Intermediate Value Theorem implies that there exists p € (a,b) for which ‘h(p) = 0. This number p is a fixed point for g since (a)—a>0 and h(b) = g(b)—b <0. O=h(p) = g(p) ~ p_ implies that g(p) = p. ‘Suppose, in addition, that |g’(x)| < k < 1 and that p and q are both fixed points in (a, b]. If p # q, then the Mean Value Theorem implies that a number & exists between p and q, and hence in (a, 6], with EP 2@) _ 9) Thus, Ip ~ 41 = Ie(P) — g@)l = Is’ @)llp — 41 Skip — 41 < |p al, which is a contradiction. This contradiction must come from the only supposition, P #4q.Hence, p = q and the fixed point in [a, 6] is unique. wae Let g(x) = (2? — 1)/3 on [-1, 1. The Extreme Value Theorem implies that the absolute minimum of g occurs at x = 0 and g(0) = —4. Similarly, the absolute maximum of g occurs at x = +1 and has the value g(:1) = 0. Moreover, ¢ is continuous and for all x € (=I, 1) So g satisfies all the hypotheses of Theorem 2.2 and has a unique fixed point in (-1, 1, In this example, the unique fixed point p in the interval (—1, 1} can be deter- mined algebraically. If P=s8(p) , then p? which, by the quadratic formula, implies that P 30 - V1). Note that g also has a unique fixed point p = 4(3 + V3) for the interval (3, 4]. However, ¢(4) = Sand g’(4) = § > 1, so g does not satisfy the hypotheses of Theorem 2.2 on [3, 4]. Hence, the hypotheses of Theorem 2.2 are sufficient to guarantee a unique fixed point but are not necessary. (See Figure 2.4.)58 CHAPTER 2 = Solutions of Equations in One Variable Figure 2.4 b. Let g(x) = 37%. Since '(x) = ing on [0, 1]. So 3-* In3 < Oon (0, 1], the function g is decreas- a= ; Sa(x)s1=8(0), for O 1. If the sequence converges to p and g is continuous, then P= lim, pe = im, g(Pra) = 8 (im, Pri) = 800), and a solution to x = g(x) is obtained. This technique is called fixed-point iteration, or functional iteration. The procedure is detailed in Algorithm 2.2 and illustrated in Figure 2.6. Figure 2.6 Pa= stp) (PoP) ps= a(n) emp) Pim apo) Pi= ato) (rw) Pi Ps Ps Po x mM Pe * @ © perms Fixed-Point Iteration pd To find a solution to p = g(p) given an initial approximation po: INPUT initial approximation po; tolerance TOL; maximum number of iterations No OUTPUT approximate solution p or message of failure. Step 1 Seti =1. Step 2 While i < No do Steps 3-6. Step 3 Set p= (po). (Compute pi.) Step 4 If |p — po| < TOL then OUTPUT (p); (The procedure was successful.)60 EXAMPLE 3 CHAPTER 2 + Solutions of Equations in One Variable STOP. Step 5 Seti +1 Step 6 Set py = p. (Update po.) Step 7 OUTPUT (‘The method failed after No iterations, Ny =’, No); (The procedure was unsuccessful.) STOP. . ‘The following example illustrates functional iteration, ‘The equation x3 +4x? — 10 = O has a unique root in [1, 2]. There are many ways to change the equation to the fixed-point form x = g(x) using simple algebraic manipulation. For example, to obtain the function g described in part (c), we can manipulate the equation x? + 4x? — 10 = 0as follows: 3 47=10-2°, so v= lao-2), and x +300 =2)2, To obtain a positive solution, g3(x) is chosen. It is not important to derive the functions shown here, but you should verify that the fixed point of each is actually a solution to the original equation, x? + 4x? — 10 = 0. —47 +10 a xa gi(t)ax— » sae) =(!-«) x= a0) = 300 = 8)2 10 \'? a x= gx) = vane =(_") reper teen 10 ES BOS 3x? + Be With py = 1.5, Table 2.2 lists the results of the fixed-point iteration for all five choices ofg. ‘The actual root is 1.365230013, as was noted in Example 1 of Section 2.1. Comparing the results to the Bisection Algorithm given in that example, it can be seen that excellent results have been obtained for choices (c), (d), and (e), since the Bisection method requires, 27 iterations for this accuracy. Itis interesting to note that choice (a) was divergent and that (b) became undefined because it involved the square root of a negative number. . Even though the various functions in Example 3 are fixed-point problems for the same root-finding problem, they differ vastly as techniques for approximating the solution toTable 22 Theorem 2.3 2.2. Fixed-Point iteration a1 n @ ®) © @ © o 1s 15 1s 1S Ls 1 0.875 0.8165 1,286953768 —‘1.348399725—1.373333333 2 6732 2.9969 1,402540804 ——1,367376372 ——_‘1.365262015 3 469.7 (-8.65)" 1.345458374 ——_1.364957015 1,365230014 4 1.03 x 108 1.375170253‘1.365264748——_‘1.365230013 5 1,360094193, 1,365225594 6 1,367846968 ——_‘1.365230576 7 1.363887004 ——_—1.365229942 8 1.365916734 ——_‘1.365230022 9 1,364878217—_1.365230012 10 1.365410062—_1.365230014 15 1,365223680 —_1.365230013 20 1,365230236 25 1.365230006 30 1.365230013 the root-finding problem. Their purpose is to illustrate the true question that needs to be answered: How can we find a fixed-point problem that produces a sequence that reliably and rapidly converges to a solution to a given root-finding problem? ‘The following theorem and its corollary give us some clues concerning the paths we should pursue and, perhaps more importantly, some we should reject. (Fixed-Point Theorem) Let g € Cla, b] be such that g(x) € [a, 6), for all x in [a, 6]. Suppose, in addition, that g” exists on (a, b) and that a constant 0 < k < 1 exists with Is'@)| 0, and p, € (a, b] for all n. Using the fact that |g'(x)| < k and the Mean Value Theorem, we have, for each n, Ipn — pl = 1¢(Pn—1) — 8(P)! = Ie'En)IPa—1 — Pl < KlPa—t ~ Phe where & € (a, 5). Applying this inequality inductively gives [Pa — PLS klpe-1 — pl SP lpa—2 — PLS +++ SR"Ipo ~ pl. (24)62 Corollary 2.4 CHAPTER 2 * Solutions of Equations in One Variable Since 0 < k < 1, we have litt... k" = Oand Jim |e — pl S Jim k"Ip0 ~ pl = 0. Hence, {Pa }puo converges to p. If g satisfies the hypotheses of Theorem 2.3, then bunds for the error involved in using py to approximate p are given by lpn — p| =" max{p9 — a, b — po} = pol, forall n>1 . Proof Since p € a, b], the first bound follows from Inequality (2.4) IPa ~ Pl = k"|po — pl = k* max{po — a, b — po}. For n > 1, the procedure used in the proof of Theorem 2.3 implies that 13(Pn) ~ 8(Pn-) S klPn — Prt) S++ Sk" |ps — Pol IPati — Pal Thus, form >n > 1, Pal Wm ~ Pal = Pm — Prem + Pent — °° + Past SIP m — Pm=1l + Pmat = Pm=al +++ Pasi — Pal Ipi — pol +R"*lpi — pol +--+ "lpi — Pol se =K'lpy— poll + kth +e Rm By Theorem 2.3, lino Pm = P, SO mone ‘o (p — pal = tim, |Pm — Pal < lim, k"Ip: ~ pol Dk 1 for all x in {1,2}. Although Theorem 2.3 does not guarantee that the method must fail for this choice of g, there is no reason to expect convergence. b. With go(x) = [(10/x) —4x}"/?, we can see that go does not map (1, 2] into (1, 2], and the sequence (p,)2%» is not defined when po = 1.5. Moreover, there is no interval containing p © 1.365 such that Ign) <1, since |gp(p)| © 3.4. There is no reason to expect that this method will converge. c. For the function g3(x) = (10 = x3)", 8h) = 3700 =)? <0 on[l,2I, 80 g3 is strictly decreasing on [1, 2}. However, |g;(2)| * 2.12, so the condition igs(x)| < k < 1 fails on [1,2]. A closer examination of the sequence {p,}729 starting with po = 1.5 shows that it suffices to consider the interval [1, 1.5] instead of [1, 2]. On this interval it is still true that g4(x) < 0 and gs is strictly decreasing, but, additionally, 1 < 1.28 © ga(1.5) < gx(x) < ga(1) = LS, forall x € [1, 1-5]. This shows that g3 maps the interval [1, 1.5] into itself. Since it is also true that |g4(x)| < |g4(1.5)| © 0.66 on this interval, Theorem 2.3 confirms the convergence of which we were already aware. 10/(4 + x)", we have | 5 {.__5 VI04+ Al ~ Y10(5)32 ‘The bound on the magnitude of g{(x) is much smaller than the bound (found in (c)) on the magnitude of g4(x), which explains the more rapid convergence using Re. <0.15, forall x € [1,2] Ieie)] The sequence defined by &) xi+4x7-10 Bs 3x? + Be ‘converges much more rapidly than our other choices. In the next sections we will see where this choice came from and why it is so effective. s et EXERCISE SET 22 Use algebraic manipulation to show that each of the following functions has a fixed point at p precisely when f(p) = 0, where f(x) =x! + 2x? ~ x — 3. “ ) ge) = (B+4~24)"" b. a= (243CHAPTER 2 = Solutions of Equationsin One Variable 3. 4 10. va 2 eg 3x4 420743 ) @ awaze a. Perform four iterations, if possible, on each of the functions g defined in Exercise 1. Let Po = Vand pays = g(r). forn =0,1,2,3. Which function do you think gives the best approximation to the solution? ‘The following four methods are proposed to compute 21", Rank them in order, based on their apparent speed of convergence, assuming py = 1. 20py-1 + 21/Pi a py et Be P= Pai — fy 21 3P a1 ena) ‘The following four methods are proposed to compute 7". Rank them in order, based on their apparent speed of convergence, assuming po = 1. TH 4 Ps © Pa = Pra — wre (t# © Pa = Pat Use a fixed-point iteration method to determine a solution accurate to within 10°? for x* — 3x? 3 = Oon[1, 2]. Use pp=1. Use a fixed-point iteration method to determine a solution accurate to within 10? for x? — x — = 0on[1, 2). Use po =1. Use Theorem 2.2 to show that g(x) = a + 0.5sin(x/2) has a unique fixed point on (0, 2] Use fixed-point iteration to find an approximation to the fixed point that is accurate to within 10-2. Use Corollary 2.4 to estimate the number of iterations required to achieve 10°? accuracy, and compare this theoretical estimate to the number actually needed, Use Theorem 2.2 to show that g(x) = 2* has a unique fixed point on {}, 1]. Use fixed-point iteration to find an approximation to the fixed point accurate to within 10. Use Corollary 2.41to estimate the number of iterations required to achieve 10~ accuracy, and compare this theoretical estimate to the number actually needed. Use a fixed-point iteration method to find an approximation to V/3 that is accurate to within 10-. Compare your result and the number of iterations required with the answer obtained in Exercise 10 of Section 2.1 ‘Use a fixed-point iteration method to find an approximation to -/25 that is accurate to within 10-+. Compare your result and the number of iterations required with the answer obtained in Exercise 11 of Section 2.1. For each of the following equations, determine an interval [a, 6] on which fixed-point iteration will converge. Estimate the number of iterations necessary to obtain approximations accurate to within 10-8, and perform the calculations. 2 bo x=542 © a@ox=5* e £ x= 0,5(inx +005) For each of the following equations, determine a function g and an interval [a,b] on which fixed-point iteration will converge to a positive solution of the equation. a 3x? et =0 be x—cosx=0 Find the solutions to within 10%22 13. 14, 15, 16. 7. 18, 1. 21. Fixed-Point iteration 6 Find all the zeros of f(x) = x? + 10cosx by using the fixed-point iteration method for an appropriate iteration function g. Find the zeros accurate to within 10-*. Use a fixed-point iteration method to determine a solution accurate to within 10~! for x = tanx, for x in 4, 5) Use a fixed-point iteration method to determine a solution accurate to within 10-? for 2sinxx +x = On [1,2]. Use po =1. Let A be a given positive constant and g(x) = 2x ~ Ax* Show that if fixed-point iteration converges to a nonzero limit, then the limit is p = 1/4, 50 the inverse of a number can be found using only multiplications and subtractions. Find an interval aboot 1/A for which fixed-point iteration converges, provided po is in that interval Find a function g defined on (0, 1] that satisfies none of the hypotheses of Theorem 2.2 but stil has a unique fixed point on (0, 1]. ‘&, Show that Theorem 2.2 is true ifthe inequality |g'(x)| < & is replaced by g'(x) < k, for allx € (a, 6). [Hint: Only uniqueness is in question.) Show that Theorem 2.3 may not hold if inequality jg’(x)| < k is replaced by g'(x) < k (dint: Show that g(x) = 1 ~ x2, for x in (0, 1], provides a counterexample.] &. Use Theorem 2.3 to show that the sequence defined by 1 1 me per to, forme, converges to /2 whenever xp > 2. b. Use the fact that 0 < (xp ~ 2)? whenever 1p 4 V2 to show that if 0 < xp < V3, then x > v2, ¢. Use the results of parts (a) and (b) to show that the sequence in (a) converges to 72 whenever x > 0 ‘8. Show that if Ais any positive number, then the sequence defined by for n> 1, Pe converges to /A whenever x > 0. b, What happens if xp < 0? Replace the assumption in Theorem 23 that “a positive number k < 1 exists with |g’(x)| < &" with “g satisfies « Lipschitz condition on the interval [a, b) with Lipschitz constant L < 1.” (See Exercise 25, Section 1.1.) Show that the conclusions of this theorem are still valid ‘Suppose that g is continuously differentiable on some interval (c, d) that contains the fixed point p of g. Show that if |p'(p)| < 1, then there exists a8 > O such that if |pp — p| < 8, then the fixed-point iteration converges. ‘An object falling vertically through the air is subjected to viscous resistance as well as to the force of gravity. Assume that an object with mass m is dropped from a height sy and that the height of the object after ¢ seconds is where g = 32.17 fus? and k represents the coefficient of air resistance in Ib-s/ft. Suppose 50 = 300 ft, m = 0.25 Ib, and k = 0.1 Ib-sft. Find, to within 0.01 s, the time it takes this ‘quarter-pounder to hit the ground66 CHAPTER 2 = Solutions of Equations in One Variable 24, Let g € C'la,b] and p be in (a, b) with g(p) = p and |g'(p)| > 1. Show that there exists a 45> Osuch that if < ipp — p| = 8, then Ipo ~ pl < |p ~ p|- Thus, no matter how close the initial approximation py isto p, the next iterate p, is farther away, so the fixed-point iteration does not converge if po # p. 2.3 Newton's Method ‘Newton's (or the Newion-Raphson) method is one of the most powerful and well-known numerical methods for solving a root-finding problem. There are many ways of introducing Newton's method. If we only want an algorithm, we can consider the technique graphically, asis often done in calculus. Another possibility is to derive Newton's method as a technique to obtain faster convergence than offered by other types of functional iteration, as is done in Section 2.4. A third means of introducing Newton's method, which is discussed next, is based on Taylor polynomials ‘Suppose that f ¢ C?[a, b]. Let ¥ € [a, b] be an approximation to p such that f’(x) # 0 and jp ~ | is “small.” Consider the first Taylor polynomial for f (x) expanded about X, & a fQ) = {M+@-D/O+ FEO), where §(x) lies between x and X. Since f(p) = 0, this equation with x = p gives o= I@+@-n/@+2o™ B® me, Newton's method is derived by assuming that since |» — ¥| is small, the term involving (p — ¥) is much smaller, so O= f@ + (p—*S'R)- Solving for p gives wz-fQ p 7@ This sets the stage for Newton's method, which starts with an initial approximation po and ‘generates the sequence {p,}32, by forn > 1. (2.5) Figure 2.7 illustrates how the approximations are obtained using successive tangents. (Also see Exercise 11.) Starting with the initial approximation po, the approximation p; is the x-intercept of the tangent line to the graph of f at (po, f (po)). The approximation py is the x-intercept of the tangent line to the graph of f at (p1, f())) and so on. Algorithm 2.3 follows this procedure.2.3. Newton's Method 67 Figure 2.7 y Srey — 9 =f) (Prfey) (o,f) Newton's To find a solution to f(x) = 0 given an initial approximation pp: INPUT initial approximation po; tolerance TOL; maximum number of iterations No, OUTPUT approximate solution p or message of failure. Step1 Seti =1 Step 2. While i < No do Steps 3-6. Step 3 Set p = po — f(po)/f'(po). (Compute p;.) Step 4 If |p — pol < TOL then OUTPUT (p); (The procedure was successful.) STOP. Step 5 Seti =i+1 Step6 Set po =p. (Update po.) Step 7 OUTPUT (‘The method failed after No iterations, No =", No): (The procedure was unsuccessful) STOP. . ‘The stopping-technique inequalities given with the Bisection method are applicable to Newton's method. That is, select a tolerance ¢ > 0, and construct p;,... py until Iw pwil 1. (2.9) 8 (Pa Suppose we would like to approximate a fixed point of g(x) = cos.x. The graph in Figure 2.8 implies that a single fixed-point p lies in (0, 7/2]. /4. The best we could Table 2.3 shows the results of fixed-point iteration with pp = conclude from these results is that p * 0.74. ‘To approach this problem differently, define f(x) = cosx — x and apply Newton’s method. Since f(x) = —sinx — 1, the sequence is generated by cos forn > 1. Prot With po = 1/4, the approximations in Table 2.4 are generated. An excellent approximation is obtained with n = 3. We would expect this result to be accurate to the places listed because of the agreement of p3 and ps .Theorem 2.5 2.3. Newton's Method 69 Table 23 Table 2.4 ” Ps ” Pn 0 0.7853981635 0 0.7853981635 1 0.7071067810 1 0,7395361337 2 0.7602445972 2 0.7390851781 3 0.7246674808 3 0.7390851332 4 0.7487198858 4 0.7390851332 5 0.7325608446 6 0.7434642113 7 0.7361282565 The Taylor series derivation of Newton’s method at the beginning of the section points ut the importance of an accurate initial approximation. The crucial assumption is that the term involving (p — =)? is, by comparison with |p — ¥|, so small that it can be deleted. This will clearly be false unless X is a good approximation to p. If po is not sufficiently close to the actual root, there is little reason to suspect that Newton's method will converge to the root. However, in some instances, even poor initial approximations will produce convergence. (Exercises 12 and 16 illustrate some of these possibilities.) The following convergence theorem for Newton’s method illustrates the theoretical importance of the choice of po. Let f € C2{a,b). If p € (a, b] is such that f(p) = 0 and f’(p) # 0, then there exists a & > O such that Newton's method generates a sequence {p,}%2., converging to p for any initial approximation po € [p — 5, p +5]. . Proof The proof is based on analyzing Newton's method as the functional iteration scheme py = g(Pn-1), for n > 1, with f@) fray Let k be in (0, 1). We first find an interval [p — 6, p +6] that g maps into itself and for which |g'(x)| < k, for all x € (p — 5, p +8). Since f’ is continuous and f'(p) # 0, part (a) of Exercise 27 in Section 1.1 implies that there exists a 6; > 0, such that /’(x) # 0 for x € [p ~ 51, p + 41] S [a,b]. Thus, & is defined and continuous on [p — 51, p + 81]. Also, 8) = ge LOLO=fOF@) _ fos"@ ror FOr for x € [p — 81, p +61], and, since f € C*[a, b], we have g € C'[p — 3), p +41]. By assumption, f(p) = 0, so ep) = LOL) ePTFECHAPTER 2 * Solutions of Equations in One Variable Since g' is continuous and 0 < k < 1, part (b) of Exercise 27 in Section 1.1 implies that there exists a 5, with O <4 <5,, and Ie’) sk, forall x €(p—3, p +4]. Itremains to show that g maps [p—6, p+8] into [p~8, p+]. Ifx € [p—8, p+], the ‘Mean Value Theorem implies that for some number & between x and p, |g(x) — g(p)| = Ig'E)Ilx ~ pl. So Ig) ~ pl = lax) — Cp) = Ia'(@ Dla — pls klx — pl < lx — pl. Since x ¢ [p ~ 4, p + 4], it follows that |x — p| < 6 and that |g(x) — p| < 6. Hence, g maps [p — 8, p + 4] into [p — 6, p + 4]. All the hypotheses of the Fixed-Point Theorem are now satisfied, so the sequence (pager defined by Ce) n= m1) = Pat — , forn>1, Pa=8Pr-t) = Prat Feary forme converges to p for any po € [p 8, p +8). wae ‘Theorem 2.5 states that, under reasonable assumptions, Newton's method converges provided a sufficiently accurate initial approximation is chosen. It also implies that the constant k that bounds the derivative of g, and, consequently, indicates the speed of con- vergence of the method, decreases to 0 as the procedure continues. This result is important for the theory of Newton's method, but itis seldom applied in practice since it does not tell tus how to determine 3. In a practical application, an initial approximation is selected, and successive approximations are generated by Newton's method. These will generally either converge quickly to the root, or it will be clear that convergence is unlikely. ‘Newton's method is an extremely powerful technique, but it has a major weakness: the need to know the value of the derivative of f at each approximation. Frequently, f(x) is far more difficult and needs more arithmetic operations to calculate than f(x). To circumvent the problem of the derivative evaluation in Newton's method, we intro- duce a slight variation. By definition, fra) = tim L~ Led) Pet = Prot Letting x = pn-2, we have f@n-2) — F(a) _ fn-1) ~ Fn. “om ® Fmd Pn-2— Pn=1 Pr-t — Pr-2 Using this approximation for f’(p,-1) in Newton's formula gives SF (Pn-1)(Pr—1 = Pn—2) n= Pr—1 . (2.10) Pa Ps FD) = Fn-2) 19) This technique is called the Secant method and is presented in Algorithm 2.4. (See Figure 2.9.) Starting with the two initial approximations pp and p1, the approximation p2 is the x-intercept of the line joining (po, /(po)) and (p1, f(p1)). The approximation p3 is the x-intercept of the line joining (p1, f(pi)) and (p2, f(p2)), and so on.2.3. Newton's Method 71 Figure 2.9 pend Secant To find a solution to f(x) O given initial approximations po and p, INPUT initial approximations po, p1; tolerance TOL; maximum number of iterations No. OUTPUT approximate solution p or message of failure. Step 1 Seti =2; qo = f (Po): a = f(pv). Step 2 While i < No do Steps 3-6. Step3 Set p = pr ~9i(Pi ~ Po)/Gi — qo). (Compute pi.) Step4 Ifip — pil < TOL then OUTPUT (p): (The procedure was successful) STOP. Step5 Seti=i+1 Step6 Set po = pr; (Update po, gor P14.) w= Hs PL=Ps a = f(p)- Step 7 OUTPUT (‘The method failed after No iterations, No =", No); (The procedure was unsuccessful.) STOP. . The next example involves a problem considered in Example 1, where we used New- ton’s method with po = 1/4.72 EXAMPLE 2 Table 25 CHAPTER 2 © Solutions of Equations in One Variable Use the Secant method to find a solution to x = cos x. In Example 1 we compared func- tional iteration and Newton's method with the initial approximation po = 1/4. Here we need two initial approximations. Table 2.5 lists the calculations with py = 0.5, p1 = 1/4, and the formula , Pa = Pri Tey from Algorithm 2.4. . 0s 07853981635 0.7363841388 0.7390581392 0.7390851493 0.7390851332 waunee By comparing the results here with those in Example 1, we see that ps is accurate to the tenth decimal place, The convergence of the Secant method is much faster than functional iteration but slightly slower than Newton's method, which obtained this degree of accuracy with p3. This is generally true. (See Exercise 12 of Section 2.4.) Newton's method or the Secant method is often used to refine an answer obtained by another technique, such as the Bisection method, since these methods require a good first approximation but generally give rapid convergence. Each successive pair of approximations in the Bisection method brackets a root p of the equation; that is, for each positive integer n, a root lies between a, and by. This implies that, for each n, the Bisection method iterations satisfy \Pn = PL < 5a bal, which provides an easily calculated error bound for the approximations. Root bracketing is not guaranteed for either Newton’s method or the Secant method. Table 2.4 contains results from Newton's method applied to f(x) = cos.x — x, where an approximate root ‘was found to be 0.7390851332. Notice that this root is not bracketed by either po, Pi OF Pi, 2. The Secant method approximations for this problem are given in Table 2.5. The initial approximations po and p; bracket the root, but the pair of approximations ps and p, fail t0 do so. ‘The method of False Position (also called Regula Falsi ) generates approximations in the same manner as the Secant method, but it includes a test to ensure that the root is brack- ted between successive iterations. Although it is not a method we generally recommend, it illustrates how bracketing can be incorporated. First choose initial approximations pp and p; with f (po) f(p1) < 0. The approxima- tion p2 is chosen in the same manner as in the Secant method, as the x-intercept of the line joining (po, f(po)) and (p1, f (p1)). To decide which secant line to use to compute p3, weFigure 2.10 2.3. Newton's Method 73 check f(p2)- f(p1). If this value is negative, then p; and p2 bracket a root, and we choose 3 a8 the x-intercept of the line joining (pi, f(p1)) and (p2, f(p2)). If not, we choose ps as the x-intercept of the line joining (po, f(po)) and (p2, f(p2)), and then interchange the indices on po and p;. In a similar manner, once ps is found, the sign of f(ps) - f(p2) determines whether we use p» and p3 or p3 and p, to compute pz. In the latter case a rela- beling of pz and p; is performed. The relabeling ensures that the root is bracketed between successive iterations. The process is described in Algorithm 2.5, and Figure 2.10 shows how the iterations can differ from those of the Secant method. In this illustration, the first three approximations are the same, but the fourth approximations differ. ‘Secant method Method of False Position » y=f@) 25 Method of False Position To find a solution to f(x) = 0 given the continuous function f on the interval [po, pi] where f (po) and f (pi) have opposite signs: INPUT initial approximations po, pi; tolerance TOL; maximum number of iterations No, OUTPUT approximate solution p or message of failure, Step 1 Seti =2; qo = f(D): n= f(r). Step2 While i < No do Steps 3-7. Step 3 Set p= pi —41(P1 — Po)/(qi — 90). (Compute p;-) Step 4 If |p — pi| < TOL then OUTPUT (p); (The procedure was successful.) STOP.74 CHAPTER 2 + Solutions of Equations in One Variable Step 5 Seti itl: F(). Step 6 fq -q, < Othen set po Step 7 Set py) = pi n=4 Step 8 OUTPUT (‘Method failed after No iterations, No =", No); (The procedure unsuccessful.) STOP. . EXAMPLE 3 Table 2.6 shows the results of the method of False Position applied to f(x) = cosx — x with the same initial approximations we used for the Secant method in Example 2. Notice that the approximations agree through p and that the method of False Position requires an additional iteration to obtain the same accuracy as the Secant method. . Table 2.6 0s 07853981635 0.7363841388 0.7390581392 0.7390848638 0.7390851305 0.7390851332 The added insurance of the method of False Position commonly requires more calcula- tion than the Secant method, just as the simplification that the Secant method provides over Newton's method usually comes at the expense of additional iterations. Further examples of the positive and negative features of these methods can be seen by working Exercises 13 and 14. SS EXERCISE SET 23 1 Let fe 2 Let f(x) used? 3. Let f(x) = x? ~ 6 With py = 3 and py = &. Use the Secant method. 1b. _ Use the method of False Position. Which of (a) or (b) is closer to 6? 4. Let f(x) = x — cos. With py = ~1 and p, = 0, find ps a. Use the Secant method. b. Use the method of False Position. — 6 and pp = 1. Use Newton’s method to find p2. — cosx and po = —1. Use Newton's method to find p2. Could py = 0 be find ps23 10. ML. 2. 1B. 14, Newton's Method 75 Use Newton's method to find solutions accurate to within 10~* forthe following problems. a P2550, [1,4] boo e32-1=0, (-3,-2) & x= cosx=0, (0,7/2] dx -08-02sinx=0, (0.2/2) Use Newton's method to find solutions accurate to within 10-S for the following problems, a ef H2*+2cosx-6=0 for I, let py be the x-intercept of the line tangent to F at (Pros f(Px-1))- Derive the formula describing this method. Use Newton's method to solve the equation 1 x = xsinx — 5 cos2x, wit $ sinx ~ 5 cos th po = 5 Iterate using Newton's method until an accuracy of 10-$ is obtained. Explain why the result seems unusual for Newton's method. Also, solve the equation with po = Sar and py = 10%. ‘The fourth-degree polynomial f(x) = 230x* + 18x? + 9x? — 221-9 thas two real zeros, one in {—1, 0] and the other in [0, 1]. Attempt to approximate these zeros to within 10 using the 8, Method of False Position b. Secant method Newton's method Use the endpoints of each interval as the initial approximations in (a) and (b) and the midpoints a the initial approximation in (c). ‘The function (x) = tan sex ~ 6 has a zero at (1/r) arctan6 ~ 0.447431543. Let po = O and Pi = 0.48, and use ten iterations of each of the following methods to approximate this root, ‘Which method is most successful and why? a. Bisection method b. Method of False Position © Secant methodCHAPTER 2 = Solutions of Equations in One Variable 15. 16. 17. 18. 19, 2. 2. ‘The iteration equation for the Secant method can be written in the simpler form paw fhe L(Pn-2)Pa-t FPn-i) = FPr-ad Explain why, in general, this iteration equation is likely to be less accurate than the one given in Algorithm 2.4. ‘The equation x? — 10cosx = 0 has two solutions, +1.3793646, Use Newton’s method to approximate the solutions to within 10~* with the following values of po. ‘Po = 100 bpp = 50 ce po=-25 dad. po = 25 e& po = 50 f. po = 100 ‘Use Maple to determine how many iterations of Newton's method with py = 1/4 are needed to find a root of f (x) = cos.x ~ x to within 10“, Repeat Exercise 17 with po = 4, py = 4, and the Secant method, ‘The function described by f(x) = In(x? + 1) ~ e* cos x has an infinite number of zeros. a. Determine, within 10-°, the only negative zero. b. Determine, within 10-6, the four smallest positive zeros. Determine a reasonable initial approximation to find the nth smallest positive zero of f. [Hint: Sketch an approximate graph of f.] d. Use part (c) to determine, within 10~*, the 25th smallest positive zero of f Find an approximation for A, accurate to within 10~, for the population equation 435,000 1,564,000 = 1,000,000e* + We = 1), discussed in the introduction to this chapter. Use this value to predict the population at the end of the second year, assuming that the immigration rate during this year remains at 435,000 individuals per year. ‘The sum of two numbers is 20. If each number is added to its square root, the product of the two sums is 155.55, Determine the two numbers to within 10 ‘The accumulated value of a savings account based on regular periodic payments can be deter- mined from the annuity due equation, A= FU +i)" 1) In this equation, A is the amount in the account, P is the amount regularly deposited, and i isthe rate of interest per period for the n deposit periods. An engineer would like to have a savings account valued at $750,000 upon retirement in 20 years and can afford to put $1500 per month toward this goal. What is the minimal interest rate at which this amount can be invested, assuming that the interest is compounded monthly? Problems involving the amount of money required to pay off a mortgage over a fixed period. of time involve the formula 1-045"), known as an ordinary annuity equation. In this equation, A is the amount of the mortgage, P iis the amount of each payment, and i is the interest rate per period for the n payment periods. Suppose that a 30-year home mortgage in the amount of $135,000 is needed and that the borrower can afford house payments of at most $1000 pet month. What is the maximal interest rate the borrower can afford to pay?23 mw 28, 27. 29. ‘Newton's Method 7 ‘A drug administered to a patient produces a concentration in the blood stream given by c(t) = Ate~"? milligrams per milliliter, t hours after A units have been injected. The maximum safe concentration is 1 mg/ml. ‘a, What amount should be injected to reach this maximum safe concentration, and when does this maximum occur? b. An additional amount of this drug is to be administered to the patient after the concen- tration falls to 0.25 mg/ml. Determine, to the nearest minute, when this second injection should be given. ©. Assume that the concentration from consecutive injections is additive and that 75% of the amount originally injected is administered in the second injection. When is it time for the third injection? Let fa) = 1-7 ‘8, Use the Maple commands solve and fs01ve to try to find all roots of f. b. Plot f(x) to find initial approximations to roots of f. ¢. Use Newton's method to find rdots of f to within 10, 4. Find the exact solutions of f(x) = 0 algebraically Repeat Exercise 25 using f(x) = 2° — 3. 7*+1 ‘The logistic population growth model is described by an equation of the form PL P= where P,,¢, and k > 0 are constants, and P(t) is the population at time t. P, represents the limiting value of the population since lim,.,. P(t) = Pz. Use the census data for the years 1950, 1960, and 1970 listed in the table on page 104 to determine the constants P;,.c, and k for a logistic growth model. Use the logistic model to predict the population of the United States in 1980 and in 2010, assuming t = 0 at 1950. Compare the 1980 prediction to the actual value. ‘The Gompertz population growth model is described by Pw = Pe where Pc, and k > Oare constants, and P(t) is the population at time 1. Repeat Exercise 27 using the Gompertz growth model in place of the logistic model. Player A will shut out (win by a score of 21-0) player B in a game of racquetball with proba- bility a rot? ( 25) . 2 \imprF where p denotes the probability A will win any specific rally (independent ofthe server. (See {Keller, J}, p. 267.) Determine, to within 10-*, the minimal value of p that will ensure that A will shut out B in at least half the matches they play. In the design of all-terrain vehicles, it is necessary to consider the failure of the vehicle when attempting to negotiate two types of obstacles. One type of failure is called hang-up failure ‘and occurs when the vehicle attempts to cross an obstacle that causes the bottom of the vehicle to touch the ground. The other type of failure is called nose-n failure and occurs when the vehicle descends into a ditch and its nose touches the ground. ‘The accompanying figure, adapted from [Bek], shows the components associated with the nose-in failure ofa vehicle. Ia tht reference itis shown that the maximum angle cr thatCHAPTER 2 + Solutions of Equations in One Variable ccan be negotiated by a vehicle when f is the maximum angle at which hang-up failure does ‘not occur satisfies the equation Asina cosa + B sina ~C cosa ~ E sina =0, where A=lsing,, B= lcosB;, C= (k+05D)sinf; -05Dtanp, and E = (h+0.5D)cos 6; — 05D. a, It is stated that when I = 89 in., A= 49 in., D = 55 in., and fy = 11.5°, angle a is approximately 33°. Verify this result . Find a forthe situation when I, fh, and fi, are the same as in part (a) but D = 30 in. 2.4 Error Analysis for Iterative Methods Definition 2.6 In this section we investigate the order of convergence of functional iteration schemes and, as a means of obtaining rapid convergence, rediscover Newton's method. We also consider ways of accelerating the convergence of Newton’s method in special circumstances. First, however, we need a procedure for measuring how rapidly a sequence converges. Suppose {pa}%2o is a sequence that converges to p, with p, # p for all n. If positive constants A and a exist with east = pl _ 28% 1p — Die then {pn}22 converges to p of order a, with asymptotic error constant 2. . An iterative technique of the form p, = g(Pn-t) is said to be of order a if the sequence (pn}$2o converges to the solution p = g(p) of order a. In general, a sequence with a high order of convergence converges more rapidly than a sequence with a lower order. The asymptotic constant affects the speed of convergence but is not as important as the order. Two cases of order are given special attention.EXAMPLE 1 Table 2.7 2.4 Error Analysis for herative Methods 79 @ Ifa = 1, the sequence is linearly convergent. (ii) Ifa = 2, the sequence is quadratically convergent. ‘The next example compares a linearly convergent sequence to one that is quadratically convergent. It shows why we try to find methods that produce higher-order convergent sequences. ‘Suppose that {pp )}®2.o is linearly convergent to 0 with tim Patt! _ 9.5 no 1Pnl and that (f,}229 is quadratically convergent to 0 with the same asymptotic error constant, im, : ae =05. For simplicity, suppose that Pett! 9.5 and [Pal For the linearly convergent scheme, this means that [Pa ~ Ol = [Pal © 0.5|Pa—tl © (0.5)"|pa-al © + © (0.5)" [pol whereas the quadratically convergent procedure has Wn — 01 =| Pal © 0.5111? © (0.5)10.51Bn-2)°F = 0.5)? [Pe % (0.5)°[.5)|Pn-sl°I* = (0.5)"|Pn-sl° we 0.5)" Bol” Table 2.7 illustrates the relative speed of convergence of the sequences to 0 when | pol = pol Linear Convergence Quadratic Convergence Sequence {Palio Sequence (F4}o n 0.5)" (0.57! 1 5,000 x 10°! 2 1.2500 x 10°! 3 a 7.8125 x 10-3 4 6.2500 x 10°? 3.0518 x 10-5 5 3.1250 x 10°? 4.6566 x 10°'!° 6 1.5625 x 107 1.0842 x 10° 7 7.8125 x 10-9 5.8775 x 10-* ‘The quadratically convergent sequence is within 10~** of 0 by the seventh term. At least 126 terms are needed to ensure this accuracy for the linearly convergent sequence.Theorem 2.7 Theorem 2.8 CHAPTER 2 = Solutions of Equations in One Variable Quadratically convergent sequences generally converge much more quickly than those that converge only linearly, but many techniques that generate convergent sequences do so only linearly. Let g € Cla, b] be such that g(x) € [2, 6), for all x € [a, b]. Suppose, in addition, that g" is continuous on (a, b) and a positive constant k < 1 exists with Ie) sk, forall x € (a,b). If g'(p) # 0, then for any number pp in (a, b}, the sequence Pa=8(Pr-1), forn> 1, converges only linearly to the unique fixed point p in (a, 6). . Proof We know from the Fixed-Point Theorem 2.3 in Section 2.2 that the sequence con- verges to p. Since g’ exists on [a, b], we can apply the Mean Value Theorem to g to show that for any n, Pati — P= 8(Pn) ~ 8(P) = 8'(En)(Pa — P)s where & is between p, and p. Since (pn}2.o converges to p, we also have {8,]22o con- verging to p. Since g’ is continuous on [a, 5}, we have im &/(G) = &'(p). Thus, +1—P ‘ , fim Patt = Pl _ yoy = and Tim [Pet1— Pl 8'Gn) = 8'(p) ig y= le Jim 2 we ppp Hence, if g’(p) # 0, fixed-point iteration exhibits linear convergence with asymptotic error constant |g’(p)|- eee ‘Theorem 2.7 implies that higher-order convergence for fixed-point methods can oc- cur only when g/(p) = 0. The next result describes additional conditions that ensure the quadratic convergence we seek. Let p bea solution of the equation x = g(x). Suppose that g'(p) = 0 and g” is continuous and strictly bounded by M on an open interval J containing p. Then there exists a5 > 0 such that, for po € [p — 6, p +6], the sequence defined by py, = g(Pn-1), when n > 1, converges at least quadratically to p. Moreover, for sufficiently large values of n, M past — Pl <1Pa~ pr. . Proof Choose k in (0, 1) and 6 > 0 such that on the interval [p — 8, p + 8], contained in I, we have |g/(x)| < k and g" continuous. Since |g'(x)| < < 1, the argument used in the proof of Theorem 2.5 in Section 2.3 shows that the terms of the sequence {Pn)22o are contained in (p — 8, p +8]. Expanding g(x) in a linear Taylor polynomial for2.4 Error Analysis for erative Methods 31 x € [p 76, p +8) gives 802) = 8(p) + 8D) ~ p) + fO. -p, where lies between x and p. The hypotheses g(p) = p and g'(p) = 0 imply that sea pte Be py. In particular, when x = Py, Poss = 800) = p+ £8, — py, with § between p, and p. Thus, pret ~ p= £22, ~ py. Since |g’(x)| < k < 1 on [p ~ 5, p +8] and g maps [p — 4, p + 8] into itself, it follows from the Fixed-Point Theorem that (p_}%2. converges to p. But §, is between p and p, for each n, $0 {§q}%25 also converges to p, and iy [Pasi ~ Dl _ le") noo [Pa — Pl? 2° This result implies that the sequence (p,}22o is quadratically convergent if g’(p) # 0 and of higher-order convergence if g”"(p) = 0. Since g" is continuous and strictly bounded by M on the interval {p — 6, p + 5], this also implies that, for sufficiently large values of n, M IPnti— pi < lpn PP eee ‘Theorems 2.7 and 2.8 tell us that our search for quadratically convergent fixed-point methods should point in the direction of functions whose derivatives are zero at the fixed point. The easiest way to construct a fixed-point problem associated with a root-finding prob- Jem f(x) = Ois to subtract a multiple of f(x) from x. So let us consider Pn=8(Pr1), forn> 1, for g in the form a(x) =x — OF), where ¢@ is a differentiable function that will be chosen later. For the iterative procedure derived from g to be quadratically convergent, we need to have g'(p) = 0 when f(p) = 0. Since 8) =1- FOS) — f'O@),82 Definition 2.9 Theorem 2.10 Theorem 2.11 CHAPTER 2 + Solutions of Equations in One Variable we have 8'(p) =1-6'(p)f(p) — f'(p)b(p) = 1 — b'(p) 0 F'(p)9(p) = and g’(p) = 0 if and only if $(p) = 1/f"(p). If we let (x) = 1/f’(x), then we will ensure that 6(p) = 1/f"(p) and produce the quadratically convergent procedure =F (P)e(). Pn = 8(Pn-t) = Pat F'n) This, of course, is simply Newton's method. In the preceding discussion, the restriction was made that f’(p) # 0, where p is the solution to f(x) = 0. From the definition of Newton's method, it is clear that difficulties might occur if f"(p,) goes to zero simultaneously with f(p,). In particular, Newton's method and the Secant method will generally give problems if f'(p) = 0 when f(p) To examine these difficulties in more detail, we make the following definition. A solution p of f(x) is a zero of multiplicity m of f if for x # p, we can write F(x) = (x = p)"q(x), where lim,» q(x) #0. . In essence, g(x) represents that portion of f(x) that does not contribute to the zero of ‘f. The following result gives a means to easily identify simple zeros of a function, those that have multiplicity one. Jf € Ca, by has a simple zero at p in (a, b) if and only if f(p)=0, but f'(p) #0. = Proof If f has a simple zero at p, then f(p) = 0 and f(x) = (x — p)q(x), where lim,,p q(x) # 0. Since f € C'fa, b}, FD) = fim fC) = Hila + ~ pa") = im g(a) # 0. Conversely, if f(p) = 0, but f’(p) # 0, expand f in a zeroth Taylor polynomial about p. Then F(x) = f(p) + FEC = p) = - P)F'ER)), where (x) is between x and p. Since f € C'{a, b}, fim £6) =f" (im 00) =f) 40. Letting g = f’ o€ gives f(x) = (a — p)q(x), where lim,.,pq(x) # 0. Thus, f has a simple zero at p. wae The following generalization of Theorem 2.10 is considered in Exercise 10. The function f € C™[a, 6] has a zero of multiplicity m at p in (a, 6) if and only if O= fp) = fp) = fp) = = FO PCD), but f(y) #0. .EXAMPLE 2 Table 2.8 Figure 2.11 2.4 Error Analysis for iterative Methods 83 The result in Theorem 2.10 implies that an interval about p exists where Newton's method converges quadratically to p for any initial approximation pp = p, provided that p is a simple zero. The following example shows that quadratic convergence may not occur if the zero is not simple. Consider f(x) = e* — x — 1. Since (0) =e? — 0-1 =O and f'(0) =e” — 1 =0, but £0) =e? = 1, f has a zero of multiplicity two at p = 0. In fact, f (x) can be expressed in the form where, by L’Hépital’s cule, lim =o The terms generated by Newton’s method applied to f with pp = 1 are shown in Table 2.8. The sequence is clearly converging to 0, but not quadratically. The graph of f is shown in Figure 2.11 . 2 Pn n Pr o 10 9 2.7750 x 107 1 058198 101.3881 x 107 2 031906 11 6.9411 x 10° 3 0.16800 2 4 0.08635 1B 5 0.04380 14 8.8041 x 10-5 6 0.02206 1S 4.2610 x 10° 7 0.01107 16 1.9142 x 10-6 8 0.005545EXAMPLE 3 Table 2.9 CHAPTER 2 = Solutions of Equations in One Variable One method of handling the problem of multiple roots is to define _ £@) HO= FG) If p is azero of f of multiplicity m and f(x) = (x — p)"q(x), then . = pa) HO) = Tea pig) + PTE) a(x) =O PngGye Pe) also has a zero at p. However, 9(p) #0, s0 gp) mq(p) + (p— p)a'(p) and p is a simple zero of .. Newton’s method can then be applied to 11 to give HO) LG)/F) _ HG) (FOF VOU" ORF OF #0, gx) =x~ or SOS) FOR- fara en If g has the required continuity conditions, functional iteration applied to g will be quadratically convergent regardless of the multiplicity of the zero of f. Theoretically, the only drawback to this method is the additional calculation of f”(x) and the more laborious. procedure of calculating the iterates. In practice, however, multiple roots can cause seri- ‘ous roundoff problems since the denominator of (2.11) consists of the difference of two numbers that are both close to 0. g(x) =x— Table 2.9 lists the approximations to the double zero at x = Oof f(x) =e" — x — 1 using Pn = 8(Pn-1), for n > 1, where g is given by (2.11). The results were recorded using a calculator with ten digits of precision. The initial approximation of pp = 1 was chosen so that the entries can be compared with those in Table 2.8. What Table 2.9 does not show is that no improvement to the zero approximation —2.8085217 x 107 occurs in subse- quent computations using this calculator since both the numerator and the denominator approach 0. . Pa =2,3421061 x 10 8.458278 x 10-> —1.1889524 x 10-% —6.8638230 x 10-6 8085217 x 10°?EXAMPLE 4 Table 2.10 2.4 Error Analysis for iterative Methods 85 In Example 3 of Section 2.2 we solved f(x) = x3 + 4x? — 10 = 0 for the zero p = 1.36523001. To compare convergence for a zero of multiplicity one by Newton's method and the modified Newton's method listed in Eq, (2.11), let Phaa +403) — 10 Pa = Prot ~ Pig E+ _ from Newton’s method and, from p, 8(Pn-1), Where g is given by Eq. (2.11), rm _ (Pans + 473-1 — 10) 3p3_1 + BPn1) Pn © PA GPE T+ 8pm)? — (Dana +4924 ~ 10K6Pna +8) With po = 1.5, the first three iterates for (i) and (ii) are shown in Table 2.10. The results illustrate the rapid convergence of both methods in the case of a simple zero. @ pr —-(1.373333331.35689898 p ‘136526201 —_—1.36519585 ps___136523001__ 136523001 EXERCISE SET 24 1. Use Newton's method to find solutions accurate to within 10~* to the following problems. a xta2ee* +e =0, forO =0, for—-1 1. Construct a sequence that converges to 0 of order 3. ‘Suppose a > 1. Construct a sequence that converges to 0 zero of order a. Suppose p is a zero of multiplicity m of f, where f” is continuous on an open interval containing p. Show thatthe following fixed-point method has g'(p) = 0: mf) FR) ‘Show that the Bisection Algorithm 2.1 gives a sequence with an error bound that converges linearly 100. ‘Suppose that /' has m continuous derivatives. Modify the proof of Theorem 2.10 to show that {bas a zero of multiplicity m at p if and only if O= fp) = fp) == FO Mp), but Fp) #0. ‘The iterative method to solve f(x) = 0, given by the fixed-point method g(x) = SF (Pra) f"(Pa-s) [ fled YP Fon) Fp) [FESS] ree bas has g/(p) = g/(p) = 0. This will generally yield cobic (a = 3) convergence, Expand the analysis of Example to compare quadratic and ubic convergence It can be shown (see, for example, [DaB, pp. 228-229]) that if (p,]329 are convergent Se- cant method approximations to p, the solution to f(x) = 0, then a constant C exists with Tess pI%€ Ip, pipes Por sficiently large vales of m. Assume (,) converges tery of order and show that = (1+ V/5)/2. (Note: This implies tha the order of conver gence of the Secant method is approximately 1.62). at) + where Pa = 8(Pa-t Pri ES 2.5 Accelerating Convergence Itis rare to have the luxury of quadratic convergence. We now consider a technique called Aitken’s A? method that can be used to accelerate the convergence of a sequence that is linearly convergent, regardless of its origin or application. Suppose (Pp), is a linearly convergent sequence with limit p. To motivate the con- struction of a sequence {fn}229 that converges more rapidly to p than does {Pp}qap, let us first assume that the signs of p,, — Pp. Pn+i — P. and pn42— p agree and that n is sufficiently large that Pri P . Pad =P Pa-P Past —P Then (Pasa — PY? (Pns2 — PYPn = Ps 50 Phas — 2PntiP + P? © Pas2Pn— (Pn + Dns)? + P*EXAMPLE 1 Table 2.11 Definition 2.12 25. Accelerating Convergence 87 and (Pasa + Pn — 2Pnei)P © Pns2Pn ~ Pras Solving for p gives pw —PataPa a Pes Pst 2Pnsi + Pr Adding and subtracting the terms p? and 2p, py+1 in the numerator and grouping terms appropriately gives 42 ~ 2Pn Pat ~ 2PaPnst ~ Pa Pr Pata — 2Prei + Pa (PR PnPat2 + 2Pn Patt) — (P2 — 2PaPnti + P241) Puta — 2Pavi + Pr _ Wns = Pn)? Pn42— 2Pntt + Pn in — PnP. = Po Aitken’s A? method is based on the assumption that the sequence {,}229, defined by 5 (Past = Pn? py oP (2.12) Pe = Pe Fa Was + Pn converges more rapidly to p than does the original sequence (p.}%2- The sequence (pq}%2,,, where Py, = cos(1/n), converges linearly to p = 1. The first few terms of the sequences (p,}%2, and {P,)%=, are given in Table 2.11. It certainly appears that (f,)S2, converges more rapidly to p = 1 than does (p,}22. . " Pr bn 1 0.54030 0.96178 2 087758 (0.98213 3 0.94496 0.98979 4 0.96891 0.99342 5 0.98007 0.99541 6 098614 7098981 ‘The A notation associated with this technique has its origin in the following definition. For a given sequence {p,}%o, the forward difference Ap, is defined by APn = Pnti— Pn» forn > 0.Theorem 2.13 CHAPTER 2 = Solutions of Equations in One Variable Higher powers, A* p,, are defined recursively by A*p_ = A(A'"'p,), fork 2 2. . The definition implies that 4? py = A(Pntt — Pn) = APnit — Pn = (Pns2 ~ Prt) — (Past ~ Pr)- So 4 Pn = Pri2 ~ 2Pati + Per and the formula for f, given in Eq. (2.12) can be written as ( 2°Pn 2 for n > 0. (2.13) B=Pn ‘To this point in our discussion of Aitken’s A? method, we have stated that the sequence {n}2g, converges to p more rapidly than does the original sequence { p,}%2.o, but we have not said what is meant by the term “more rapid” convergence. Theorem 2:13 explains and justifies this terminology. The proof of this theorem is considered in Exercise 14. ‘Suppose that (py}22 is a sequence that converges linearly to the limit p and that for all sufficiently large values of n we have (p, — p)(Pa+1 — P) > 0. Then the sequence {Bn}°25 converges to p faster than {p,}22 in the sense that By applying a modification of Aitken’s A? method to a linearly convergent sequence obtained from fixed-point iteration, we can accelerate the convergence to quadratic. This procedure is known as Steffensen’s method and differs slightly from applying Aitken’s A? method directly to the linearly convergent fixed-point iteration sequence. Aitken’s A? ‘method constructs the terms in order: Po, Pi=s(po), pr= spi. o= 1A*}(po). Ps=e(P2), Br = (A7K(PVs--- + where {A?) indicates that Eq. (2.13) is used. Steffensen’s method constructs the same first four terms, po, Pi, P2, and fo. However, at this step it assumes that fo is a better approx- imation to p than is p2 and applies fixed-point iteration to Ap instead of p2. Using this notation the sequence generated is PP, PO =8P), PP = 8, PE? =(7P), PI” = 8(P)), Every third term is generated by Eq. (2.13); the others use fixed-point iteration on the previous term. The process is described in Algorithm 2.6.EXAMPLE 2 Table 2.12 2.5 Accelerating Convergence 89 Steffensen’s To find a solution to p = g(p) given an initial approximation po: INPUT initial approximation po; tolerance TOL; maximum number of iterations No. OUTPUT approximate solution p or message of failure. Step1 Seti =1 Step 2 While i < No do Steps 3-6. Step 3 Set rp, = g(po); (Compute pi.) P2 = 8(p1)i (Compute p~?.) P= Po~(Pi— Po)? /(p2—2p1 + po). (Compute p\?) Step 4 If |p — pol < TOL then OUTPUT (p); (Procedure completed successfully.) STOP. Step 5 Seti=i+1. Step 6 Set py = p. (Update pp.) Step 7 OUTPUT (‘Method failed after Np iterations, Nj (Procedure completed unsuccessfully.) STOP. . + No)s Note that A?p, may be 0, which would introduce a 0 in the denominator of the next ccurs, we terminate the sequence and select pS"~" as the approximate To solve x3 +4x?— 10 = 0 using Steffensen’s method, let x3 -+ 4x? = 10, divide by x +4, and solve for x. This procedure produces the fixed-point method 10_\'? x09 = (2) : used in Example 3(d) of Section 2.2. Steffensen’s procedure with po = 1.5 gives the values in Table 2.12. The iterate p(? = 1.365230013 is accurate to the ninth decimal place. In this example, Steffensen’s method ‘gave about the same accuracy as Newton's method (see Example 4 in Section 24). k Pa pe pe 0 15 1348399725 __1,367376372 1 1,365265224 1,365225534_1.365230583, 2 136523001390 CHAPTER 2 © Solutions of Equations in One Variable From Example 2, it appears that Steffensen's method gives quadratic convergence without evaluating a derivative, and Theorem 2.14 verifies that this is the case, The proof of this theorem can be found in (He2, pp. 90-92] or (IK, pp. 103-107} Theorem 2.14 Suppose that x = g(x) has the solution p with g/(p) # 1. If there exists a 8 > 0 such that g € C?[p — 4, p + 4], then Steffensen’s method gives quadratic convergence for any poe (p85, +5) ES EXERCISE SET 25 1 9. 10. uu. ‘The following sequences are linearly convergent. Generate the first five terms of the sequence (Bs) using Aitken’s A? method. a po =0.5, py =(2— eh + p2,)/3, nZI be p= 0.75, Py = (1/3), BT Po =05, Pe nel d= 05, pp =C0SPe1, NZI Consider the function f(x) = e + 3(In2)%e™ — (In 8)e** — (In2)°, Use Newton's method with po = 0 to approximate a zero of f. Generate terms until [Pn 1 ~ Pal < 0.0002. Construct the sequence (f,}. 1s the convergence improved? Let g(x) = cos(x — 1) and p{” = 2. Use Steffensen’s method to find p}”. Let g(x) = 1-4 (sin)? and pl = 1. Use Steffensen’s method to find p( and Steffensen’s method is applied to a function g(x) using pf = 1 and pi? = 310 obtain pi? = 0.75, What is p?? Steffensen’s method is applied to a function g(x) using pi = 1 and p = V2 to obtain 1 = 2.7802. What is p!°? Use Steffensen’s method to find, to an accuracy of 10°4, the root of x? — in (1, 2], and compare this to the results of Exercise 6 of Section 2.2. Use Steffensen’s method to find, to an accuracy of 10~*, the root of x ~ 2~* = 0 that lies in [0, 1], and compare this to the results of Exercise 8 of Section 2.2. ‘Use Steffensen’s method with po = 2 to compute an approximation to v/3 accurate to within 10"*, Compare this result with those obtained in Exercise 9 of Section 2.2 and Exercise 10 of Section 2.1 Use Steffensen’s method to approximate the solutions of the following equations to within 10° a. x= (2—e* +39)/3, where g is the function in Exercise 11(a) of Section 2.2. b. x =0.5¢sinx +cosx), where g is the function in Exercise 11(f) of Section 2.2. ¢. 3x? —e* = 0, where g is the function in Exercise 12(a) of Section 2.2 4. x —cosx =0, where g is the function in Exercise 12(b) of Section 2.2 ‘The following sequences converge to 0. Use Aitken’s A? method to generate {fx} until (Pe! <= 5x 107 0 that lies a b2.6. Zeros of Polynomials and Miller's Method 1 12. A sequence {p,) is said to be superlinearly convergent to p if a. Show that if py —» p of order afore > 1, then (pa) is superlinearly convergent to p. 1b. Show that py = 2+ is superlinearly convergent to 0 but does not converge to 0 of order a for any a > 1 13, Suppose that (p,} is superlinearly convergent to p. Show that 14, Prove Theorem 2.13. (Hint: Let 5, = (Pest — P)/(Pa— P)~ 2, and show that lima ‘Then express (Pri — P)/(Pa ~ P) in terms of 8y, By1, and 2.) 15, Let P,(x) be the nth Taylor polynomial for f(x) = e* expanded about x» = 0. a. For fixed x, show that p, = P, (x) satisfies the hypotheses of Theorem 2.13. Db. Let x = 1, and use Aitken’s A? method to generate the sequence jip,... , Bs ©. Does Aitken’s method accelerate convergence in ths situation? NEE al 2.6 Zeros of Polynomials and Miiller’s Method Theorem 2.15 Corollary 2.16 A polynomial of degree n has the form P(x) = gx" + px") Hoe bax + do, where the a,’s, called the coefficients of P, are constants and ay # 0. The zero function, P(x) = 0 for all values of x, is considered a polynomial but is assigned no degree. (Fundamental Theorem of Algebra) If P(x) is a polynomial of degree n > 1 with real or complex coefficients, then P(x) = 0 has at least one (possibly complex) root. . Although Theorem 2.15 is basic to any study of elementary functions, the usual proof requires techniques from the study of complex-function theory. The reader is referred 10 {SaS, p. 155], for the culmination of a systematic development of the topics needed to prove Theorem 2.15. ‘An important consequence of Theorem 2.15 is the following corollary. If P(x) is a polynomial of degree n > 1 with real or complex coefficients, then there exist unique constants 2x1, x3, .... Xe» possibly complex, and unique positive integers my, m2, ++ tty, Such that S01) mj =n and P(x) = a(x — 1) OE 392) .92 Corollary 2.17 Theorem 2.18 EXAMPLE 1 CHAPTER 2 = Solutions of Equations in One Varlable Corollary 2.16 states that the collection of zeros of a polynomial is unique and that, if each zero x; is counted as many times as its multiplicity m,, a polynomial of degree n has exactly m zeros. The following corollary of the Fundamental Theorem of Algebra is used often in this section and in later chapters. Let P(x) and Q(x) be polynomials of degree at most n. If x1, x2...» x4, with k > n, are distinct numbers with P(x;) = Q(xi) fori = 1,2,... ,k, then P(x) = Q(x) forall values of x. . To use Newton's method to locate approximate zeros of a polynomial P(x), we need to evaluate P(x) and P'(x) at specified values. Since P(x) and P’(x) are both polynomi- als, computational efficiency requires that the evaluation of these functions be done in the nested manner discussed in Section 1.2. Homer’s method incorporates this nesting tech- nique, and, as a consequence, requires only n multiplications and n additions to evaluate an arbitrary nth-degree polynomial (Horner's Method) Let P(x) = gx" + agaist bax + ap, If b, =a, and bp = apt besixo, fork =n—1n—2,...,1,0, then bo = P(xo). Moreover, if Q(x) = bpx"} + by nx"? + oo + bax +b, then P(x) = (x ~ x9) O(a) + bo. . Proof By the definition of Q(x), (x = xo) Q(x) + bo = (% — x0) (bux +--+ bax +1) +0 = (bax" + By ax" too + bax? + bya) = (bpxox"@! + +++ + baxox + bix0) + bo = Bax" + Baa — baxa)a™! +--+ + (bi — boxo)x + (bo ~ bi20). By the hypothesis, by = dy and by — bu+1X0 = ax, $0 (x = x0) OG) + by = P(X) and by = Pao) see Use Homer’s method to evaluate P(x) = 2x4 — 3x? 4 3x — 4 at x9 = -2. When we use hand calculation in Homer's method, we first construct a table, which suggests the syntheric division name often applied to the technique. For this problem, the table appears as follows:2.6 Zeros of Polynomials and Miller's Method 93 Coefficient Coefficient Coefficient + Coefficient Constant of x4 of x3 of x? ofx term a =2 a,=0 a =-3 a -4 baxo bs. 8 b2x0 by = b= b bt So, P(x) = (e +:2)(2x? — 4x? + 5x —7) +10. . An additional advantage of using the Horner (or synthetic-division) procedure is that, since P(x) = (x — x9) Q@) + bo, where Q(z) = yx! bytx Pp tbat + diy differentiating with respect to x gives Pix) = Q(x) + — x) O"(X)_ and P'(40) = QC). (2.14) ‘When the Newton-Raphson method is being used to find an approximate zero of a polyno- mial, P(x) and P'(x) can be evaluated in the same manner. EXAMPLE 2 Findan approximation to one of the zeros of P(x) = 2x4 ~ 3x? 4 3x4, using Newton's method and synthetic division to evaluate P(x,) and P’(x,) for each iterate ne With xo = —2 as an initial approximation, we obtained P(—2) in Example 1 by xo = -2 2 o -3 30-4 ~4 8-10 14 2 4 5-7 10 =P(-2). Using Theorem 2.18 and Eq, (2.14), Q(x) = 2x3 - 4x7 4+5x-7 and P'(-2) = Q(-2), so P’(—2) can be found by evaluating Q(—2) in a similar manner: x=-2]2 -4 5 7 4 16-42 2-8 2 -49 Q(-2) = P'(-2)94 CHAPTER 2 + Solutions of Equations in One Variable and Repeating the procedure to find x2, -1.796 | 2 0 -3 3 -4 -3.592 6.451 -6.197 5.742 =3.592 3.451 1742 = P(x) -3.592 12.902 2 7.184 16.353 —32.565 Om) P(x). So P(~1.796) = 1.742, P’(—1.796) = ~32.565, and = =1.796 - 2 x 1.2495 me 732565 ~~ Ina similar manner, x3 = —1.73897, and an actual zero to five decimal places is —1.73896, Note that the polynomial Q(x) depends on the approximation being used and changes from iterate to iterate. Algorithm 2.7 computes P (xo) and P’(xo) using Homer's method. prion Horner's 2.7 To evaluate the polynomial P(x) = yx" + appx"! $+ + ax + ag = (x — x9) (x) + bo and its derivative at xo: INPUT degree n; coefficients ao, a1,... . an Xo- OUTPUT y = P(x); z= P'(x0). Step 1 Set y= ay; (Compute b, for P.) Z=4q. (Compute b,- for Q.) Step2 For j =n—1,n—2,...,1 set y = xy taj; (Compute b; for P.) xoz+y. (Compute bj for Q.) z Step 3 Set y = xoy +9. (Compute bo for P.) Step 4 OUTPUT (y, 2); STOP. . If the Nth iterate, xy, in Newton's method is an approximate zero for P, then PQ) = & = xy) OC) + bo = (& — rw) OH) + Pw) © (@ — xv) OQ),Theorem 2.19 2.6 Zeros of Polynomials and Milller’s Method 95 sox ~ xy is an approximate factor of P(x). Letting #1 = xy be the approximate zero of P and Q(x) = Q(x) be the approximate factor gives P(x) © (@& — 1) Qi). ‘We can find a second approximate zero of P by applying Newton's method to Q(x). If P(x) is an nth-degree polynomial with n real zeros, this procedure applied repeatedly will eventually result in (n — 2) approximate zeros of P and an approximate quadratic factor Qn-2(x). At this stage, Q,2(x) = O.can be solved by the quadratic formula to find the last two approximate zeros of P. Although this method can be used to find all the approximate zeros, it depends on repeated use of approximations and can lead to inaccurate results. ‘The procedure just described is called deflation. The accuracy difficulty with deflation is due to the fact that, when we obtain the approximate zeros of P(x), Newton’s method is, used on the reduced polynomial Q,(x), that is, the polynomial having the property that P(x) © (x — By)(a — a) +++ (= F4) On (2). An approximate zero £41 of Qx will generally not approximate a root of P(x) = 0 as well as it does a root of the reduced equation Qx(x) = 0, and inaccuracy increases as k increases. One way to eliminate this difficulty is to use the reduced equations to find approximations £2, £5, ...., % to the zeros of P, and then improve these approximations by applying Newton's method to the original polynomial P(x) ‘A problem with applying Newton’s method to polynomials concerns the possibility of the polynomial having complex roots when all the coefficients are real numbers. If the ini- tial approximation using Newton’s method is a real number, all subsequent approximations will also be real numbers. One way to overcome this difficulty is to begin with a complex initial approximation and do all the computations using complex arithmetic, An alternative approach has its basis in the following theorem, If z = a + bi is a complex zero of multiplicity m of the polynomial P(x) with real coefficients, then Z = a — bi is also a zero of multiplicity m of the polynomial P(x), and (x? — 2ax +a? + b?)" isa factor of P(x). . A synthetic division involving quadratic polynomials can be devised to approximately factor the polynomial so that one term will be a quadratic polynomial whose complex roots are approximations to the roots of the original polynomial. This technique was described in some detail in our second edition [BFR]. Instead of proceeding along these lines, we will now consider a method first presented by D. E. Milller [Mu]. This technique can be used for any root-finding problema, but it is particularly useful for approximating the roots, of polynomials. Miller's method is an extension of the Secant method. The Secant method begins with two initial approximations x9 and x; and determines the next approximation x as the intersection of the x-axis with the line through (x9, f(xo)) and (x1, f(x1)). (See Figure 2.12(a).) Miller's method uses three initial approximations, x9, x1, and x2, and determines the next approximation x3 by considering the intersection of the x-axis with the parabola through (x0, f(o))s Crs FC), and (xa, f(2)). (See Figure 2.12(b).)96 CHAPTER 2 = Solutions of Equations in One Variable Figure 212 y y % 4 Pt mo HAE f f (@) (b) ‘The derivation of Miller's method begins by considering the quadratic polynomial P(x) = ae — m2) +b —m) + that passes through (xo, f(xo)), (x1, f (#1), and (x2, f (x2). The constants a, b, and c can be determined from the conditions £0) = a(xo — 2)? + b(xo — a2) te, (15) F(@1) = aay ~ 22) + B(x) — 2) +e, (2.16) and f@_)=a-P+b-04+c5c (2.17) tobe c= fla), (2.18) p= 2) Uf Ca) = fa) = C1 = 271 (0) = fea) 2.19) (Go — 2)(a1 = 22) (0 — *1) and a LF (x0) — f@2)) — Go — x21 f 1) — f2)] (Xo ~ 2)(a1 — 22)(%0 — 11) a To determine x3, a zero of P, we apply the quadratic formula to P(x) = 0, However, because of roundoff error problems caused by the subtraction of nearly equal numbers, we apply the formula in the manner prescribed in Example 5 of Section 1.2: —_ se b+ Ve Fac Bons2.8 2.6 Zeros of Polynomials and Miller's Method 97 ‘This formula gives two possibilities for x3, depending on the sign preceding the radical term. In Miiller’s method, the sign is chosen to agree with the sign of b. Chosen in this manner, the denominator will be the largest in magnitude and will result in x3 being se- lected as the closest zero of P to x2. Thus, 26 y= ap ~ ee, b+ sgn(b)/B? — 4ac where a, b, and c are given in Bq. (2.15). Once x3 is determined, the procedure is reinitialized using x), x2, and x3 in place of Xo, X1, and x2 to determine the next approximation, xs. The method continues until a sat- isfactory conclusion is obtained. At each step, the method involves the radical VB? — 4ac, so the method gives approximate complex roots when b? — 4ac < 0, Algorithm 2.8 imple- ‘ments this procedure. Miller's To find a solution to f(x) = 0 given three approximations, xo, x1, and x9: INPUT xo, x1,.%23 tolerance TOL; maximum number of iterations No. OUTPUT approximate solution p or message of failure. Step 1 Set hy = x1 — x0; 15 (£@1) — Fx0))/has (FQ2) = fs))/ has (82 — 81)/(ha + ha); Step 2 While i < No do Steps 3-7. Step3 b= 5) + hd; D = (b ~4f(x2)d)"?, (Note: May require complex arithmetic.) Step 4 If |b — D| < |b+ Di thenset E =b+D else set E = b — D. ~2f (2) /Es ath Step 5 Set P Step 6 If |h| < TOL then OUTPUT (p); (The procedure was successful.) Step 7 (Prepare for next iteration.) 51 = (F(a) — fG0))/his98 CHAPTER 2 + Solutions of Equations in One Variable 82 = (f (2) ~ FQ) / has (82 — 61)/(ha + ha) i=i¢l Step 8 OUTPUT (‘Method failed after Np iterations, No =". (The procedure was unsuccessful.) STOP. . EXAMPLE 3 — Consider the polynomial f(x) = 16x* — 40x? + 5x? + 20x +6. Using Algorithm 2.8 with = 10° and different values of xo, x1, and x produces the results in Table 2.13 Table 2.13 a 05, x i F(a) 3 -0.595556+0.598352i —29.4007 ~ 3.898721 4 ~0.435450 + 0.1021011 1.33223 ~ 1.19309 5 -0,390631 + 0.141852 0.375057 ~ 0.670164 6 -0.357699 + 0.169926i 0.146746 ~ 0.00744629i 7 -0.356051 + 0.162856i =0.183868 x 10-? + 0.539780 x 10-7 8 0.356062 + 0.162758 0.286102 x 10-§ + 0.953674 x 10-* b %=05, m= 10, m=15 i fea 3 128785 — 1.37624 4 (1.23746 0.126941 5 1.24160 0.219440 x 10-7 6 (1.24168 0.257492 x 104 7 124168 0.257492 x 10+ xo=25, m=20, m= 225 i % FO) 3 196059 —0.611255 4 197056 0.748825 x 10-2 5 197044 0.295639 x 10-4 6 197044 —0.259639 x 10-4 We used Maple to generate part (c) in Table 2.13. To do this, we defined f(x) and the initial approximations by x-P164K" 4-404" 345472420446 >p0:=0.5; pl:=-0.5; p2:=0.0;2.6 Zeros of Polynomials and Muller's Method 99 We evaluated the polynomial at the initial values >£0:=f (pO); f1:=f(pi); £2:-£(p2); and computed ¢ = 6, b = 10, a = 9, and py = —0.5555555558 + 0.5983516452i using the Miller's method formulas: £2; ((p0=p2)~24 (f1-£2) - (p1=p2)"}2* (£0-£2) )/ ((pO-p2) * (p1-p2)+ (p0-p1)); ( (pi~p2)* (£0-£2) ~ (pO~p2) # (£12) )/ ((p0-p2)* (p1~p2)# (p0-p1)) ; >p3:=p2- (2c) /(b+(b/abs (b))#sqrt (b"2-4"atc)); The value p; was generated using complex arithmetic, as is the calculation >£3:=£(p3); which gives fy = —29.40070112 -- 3.898724738i. . The actual values for the roots of the equation are 1.241677, 1.970446, and 0.356062 +£0.162758i, which demonstrates the accuracy of the approximations from Miiller's method. . Example 3 illustrates that Muller's method can approximate the roots of polynomi- als with a variety of starting values. In fact, Miller's method generally converges to the root of a polynomial for any initial approximation choice, although problems can be con- structed for which convergence will not occur. For example, suppose that for some i we have f(x:) = f (x41) = f ti42) # 0. The quadratic equation then reduces to a nonzero constant function and never intersects the x-axis. This is not usually the case, however, and ‘general-purpose software packages using Miiller's method request only one initial approx- imation per root and will even supply this approximation as an option. ae SS EXERCISE SET 26 1, Find the approximations to within 10~* to all the real zeros of the following polynomials using, Newton's method. a f(x) be f(x) & f@) a. f(x) = x4 + 2x? - 2-3 fle) =x + 4.0012 +.4.002e + 1.101 fo f(x)ax8 ax $28 ast ted 2, Find approximations to within 10~§ to all the zeros of each of the following polynomials by first finding the real zeros using Newton’s method and then reducing to polynomials of lower degree to determine any complex zeros.100 CHAPTER 2 * Solutlons of Equations in One Vartable 3 10. uw a f(x) = x4 + Sx? - 9x? — 85x — 136 bfx) = xt — 22? - 12s? + 16x — 40 ce f@) sates 43x? 42042 @ f(x) = x5 + xt — 21x? — 10x? - 21x -5 efx) = 16x4 + 88x? + 1593? + 76x — 240 f f(yaxt— ax? 3x $5 B fedex 4 tar 44 he f(x) =x — 7x? + 14x -6 [Repeat Exercise 1 using Miller's method. Repeat Exercise 2 using Miller's method. ‘Use Newton's method to find, within 10, the zeros and critical points of the following func- tions. Use this information to sketch the graph of f. a fa) =x - 9? +12 bf) sa! 20 — 5x7 + 12x -5 £4) = 10x? — 8.32? +2.295x — 0.21141 = Ohas a root at x = 0.29. Use Newton's method ‘with an initial approximation x = 0.28 to attempt to find this root. Explain what happens. Use Maple to find the exact roots of the polynomial f(x) = x9 + 4x — 4. ‘Use Maple to find the exact roots of the polynomial f(x) = x? — 2x — 5. Use each of the following methods to find a solution in [0.1, 1] accurate to within 10~* for 600x4 — 550x? + 200x? — 20x - 1 =0. &.Bisection method b. Newton’s method . Secant method d. method of False Position Miller's method ‘Two ladders crisscross an alley of width W. Each ladder reaches from the base of one wall to some point on the opposite wall. The ladders cross at a height H above the pavement. Find W ‘given that the lengths of the ladders are x; = 20 ft and xz = 30/ft, and that H = 8 ft. x, -}—w— ‘A.can in the shape of a right circular cylinder is to be constructed to contain 1000 em?. The circular top and bottom of the can must have a radius of 0.25 cm more than the radius of the ‘can so that the excess can be used to form a seal with the side. The sheet of material being formed into the side of the can must also be 0.25 cm longer than the circumference of the can so that a seal can be formed. Find, to within 10~, the minimal amount of material needed to ‘construct the can,2.7. Survey of Methods and Software 101 12, In 1224, Leonardo of Pisa, better known as Fibonacci, answered a mathematical challenge of, ‘John of Palermo in the presence of Emperor Frederick II: find a root of the equation x° + 2x?-+ 10x = 20. He first showed that the equation had no rational roots and no Euclidean irrational root—that is, no root in any of the forms a + Vb, /@+ Vb, Via + Vb, ot y/ Ja Vb, where ‘a and 6 are rational numbers. He then approximated the only real root, probably using an algebraic technique of Omar Khayyam involving the intersection of a circle and a parabola. Hiis answer was given in the base-60 number system as 1+22( ‘How accurate was his approximation? Henne en ee 2.7 Survey of Methods and Software In this chapter we have considered the problem of solving the equation f(x) = 0, where {f isa given continuous function. All the methods begin with an initial approximation and {generate a sequence that converges to a root of the equation, if the method is successful. If [a, 6] is an interval on which f(a) and f(b) are of opposite sign, then the Bisection method and the method of False Position will converge. However, the convergence of these methods may be slow. Faster convergence is generally obtained using the Secant method or Newton's method. Good initial approximations are required for these methods, two for ‘the Secant method and one for Newton's method, so the Bisection or the False Position method can be used as starter methods for the Secant or Newton's method. Miller's method will give rapid convergence without a particularly good initial ap- proximation. It is not quite as efficient as Newton's method; its order of convergence near a root is approximately a = 1.84, compared to the quadratic, « = 2, order of New- ton’s method. However, it is better than the Secant method, whose order is approximately a = 1.62, and it has the added advantage of being able to approximate complex roots. Deflation is generally used with Muller's method once an approximate root of a poly- nomial has been determined. After an approximation to the root of the deflated equation has been determined, use either Miller's method or Newton's method in the original poly-102 CHAPTER 2 + Solutions of Equations in One Variable nomial with this root as the initial approximation. This procedure will ensure that the root being approximated is a solution to the true equation, not to the deflated equation. We recommended Miiller’s method for finding all the zeros of polynomials, real or complex. Miiller’s method can also be used for an arbitrary continuous function. Other high-order methods are available for determining the roots of polynomials. If this topic is of particular interest, we recommend that consideration be given to Laguerre’s method, which gives cubic convergence and also approximates complex roots (see [Ho, pp. 176-179] for a complete discussion), the Jenkins-Traub method (see [JT], and Brent’s method (see [Bre)). Another method of interest, Cauchy's method, is similar to Miller's method but avoids the failure problem of Miiller’s method when f(x;) = f(xi+1) = f (4:42), for some i. For an interesting discussion of this method, as well as more detail on Miiller's method, we recommend [YG, Sections 4.10, 4.11, and 5.4] Given a specified function f and a tolerance, an efficient program should produce an approximation to one or more solutions of f(x) = 0, each having an absolute or relative error within the tolerance, and the results should be generated in a reasonable amount of time. If the program cannot accomplish this task, it should at least give meaningful explanations of why success was not obtained and an indication of how to remedy the cause of failure, ‘The IMSL FORTRAN subroutine ZANLY uses Milller’s method with deflation to approximate a number of roots of f(x) = 0. The routine ZBREN, due to R. P. Brent, uses a combination of linear interpolation, an inverse quadratic interpolation similar to ‘Milller’s method, and the Bisection method. It requires specifying an interval [a, b) that contains a root. The IMSL C routine f_zeros.fen and FORTRAN routine ZREAL are based on a variation of Miller's method and approximate zeros of a real function f when only poor initial approximations are available. Routines for finding zeros of polynomials are the C routine f-zeros.poly and FORTRAN routine ZPORC, which uses the Jenkins-Traub method for finding zeros of a real polynomial; ZPLRC, which uses Laguerre’s method to find zeros of a real polynomial; and the C routine ¢.zeros.poly and FORTRAN routine ZPOCC, which use the Jenkins-Traub method to find zeros of a complex polynomial. ‘The NAG C subroutine cOSade and the NAG FORTRAN subroutines COSADF and COSAZE use a combination of the Bisection method, linear interpolation, and extrapolation. to approximate a real zero of f(x) = 0 in the interval [a, b]. The subroutine COSAGF is similar to COSADF but requires a single starting value instead of an interval and returns an interval containing a root on its own. The NAG FORTRAN subroutines COSAJF and COSAXF use a continuation method with a Secant iteration to approximate the real zero of a function. Also, NAG supplies subroutines CO2AGF and CO2AFF to approximate all zeros of a real polynomial or complex polynomial, respectively. Both subroutines use a modified Laguerre method to find the roots of a polynomial. The netlib FORTRAN subroutine fzero.f uses a combination of the Bisection and Se- cant method developed by T. J. Dekker to approximate a real zero of f(x) = 0 in the interval [a, b]. It requires specifying an interval [@, 6] that contains a root and returns an interval with a width that is within a specified tolerance. The FORTRAN subroutine sdzro.f uses a combination of the bisection method, interpolation, and extrapolation to find a real zero of f(x) = 0 in a given interval (a, b]. The routines rpzero and cpzero can be used to approximate all zeros of a real polynomial or complex polynomial, respectively. Both methods use Newton's method for systems, which will be considered in Chapter 10. All27. Survey of Methods and Software 103 routines are given in single and double precision. ‘These methods are available on the In- ternet from netlib at http://www.netlib.org/slatec/sre. Within MATLAB, the function ROOTS is used to compute all the roots, both real and complex, of a polynomial. For an arbitrary function, FZERO computes a root near a specified initial approximation to within a specified tolerance. Maple has the procedure f£s0lve to find roots of equations. For example, >fsolve(x-2 - x - 1, x); returns the numbers —.6180339887 and 1.618033989. You can also specify a particular variable and interval to search, For example, >fsolve(x°2 - x - 1,x,1..2); returns only the number 1.618033989. The command fsolve uses a variety of specialized techniques that depend on the particular form of the equation or system of equations. Notice that in spite of the diversity of methods, the professionally written packages are based primarily on the methods and principles discussed in this chapter. You should be able to use these packages by reading the manuals accompanying the packages to better understand the parameters and the specifications of the results that are obtained. There are three books that we consider to be classics on the solution of nonlinear equations, those by Traub [Tr], by Ostrowski [Os], and by Householder [Ho]. In addition, the book by Brent [Bre] served as the basis for many of the currently used root-finding methods.CHAPTER 3 Interpolation and Polynomial Approximation A\ coneu ott poptaton othe United ttesis taken every 10ers. The following table lists the population, in thousands of people, from 1940 to 1990. Year 1940 | 1980 1960 1970 | 1980 | 1990 Population 132,165 | 151,326 | 179,323 | 203,302 | 226,542 | 249,633 (in thousands) | | Pa) 2x 108 1940 1950 1960 1970 1980 1990 2000 ¢ | “earFigure 3.1 Theorem 3.1 105 In reviewing these data, we might ask whether they could be used to provide a reasonable estimate of the population, say, in 1965 or even in + the year 2010, Predictions of this type can be obtained by using a func- tion that fits the given data, This process is called interpolation and is the subject of this chapter. This population problem is considered through- out the chapter and in Exercises 24 of Section 3.1, 14 of Section 3.2, and 24 of Section One of the most useful and well-known classes of functions mapping the set of real numbers into itself is the class of algebraic polynomials, the set of functions of the form Pal) = gx" + aX" ob ayx + a9, where n is a nonnegative integer and do, ... ,dq are real constants. One reason for their importance is that they uniformly approximate continuous functions. Given any function, defined and continuous on a closed and bounded interval, there exists a polynomial that is as “close” to the given function as desired. This result is expressed precisely in the following theorem, (See Figure 3.1.) (Weierstrass Approximation Theorem) Suppose that fis defined and continuous on {a, b]. For each ¢ > 0, there exists a polyno- mial P(x), with the property that If) — P(x) <¢, forall x in[a, 6). . ‘The proof of this theorem can be found in most elementary texts on real analysis (see, for example, [Bart, pp. 165-172)). ‘Another important reason for considering the class of polynomials in the approxima- tion of functions is that the derivative and indefinite integral of a polynomial are easy to106 Figure 3.2 CHAPTER 3 = Intepolation and Polynomial Approximation determine and are also polynomials. For these reasons, polynomials are often used for approximating continuous functions. ‘The Taylor polynomials were introduced in the first section of the book, where they were described as one of the fundamental building blocks of numerical analysis. Given this prominence, you might assume that polynomial interpolation would make heavy use of these functions. However, this is not the case. The Taylor polynomials agree as closely as possible with a given function at a specific point, but they concentrate their accuracy near that point. A good interpolation polynomial needs to provide a relatively accurate approximation over an entire interval, and Taylor polynomials do not generally do this. For example, suppose we calculate the first six Taylor polynomials about ro = 0 for f(x) = e" Since the derivatives of f(x) are all e*, which evaluated at ro = 0 gives 1, the Taylor polynomials are Polx)=1, Pix) =1+x, Ppax)altxt+ a, Px) = L4+x+ eo xt wee x Paaltxt +545, md A@aleet tba Ty The graphs of the polynomials are shown in Figure 3.2. (Notice that even for the higher-degree polynomials, the error becomes progressively worse as we move away from zero.) Although better approximations are obtained for f(x) = e* if higher-degree Taylor polynomials are used, this is not true for all functions. Consider, as an extreme example, :Table 3.1 3.1. Interpolation and the Lagrange Polynomial 107 using Taylor polynomials of various degrees for f(x) = 1/x expanded about x = 1 to approximate f(3) = }. Since faa, f') =x, f"@) = (12, and, in general, FPO) = (Dkk the Taylor polynomials are a foc " Paw) = > ao donk = Dept — Ht pad fo To approximate (3) = + by P,(3) for increasing values of n, we obtain the values in ‘Table 3.1—rather a dramatic failure! a|os 6 7 -s |u| -2 | 43 | -8s Since the Taylor polynomials have the property that all the information used in the approximation is concentrated at the single point x9, the type of difficulty that occurs here is quite common and limits Taylor polynomial approximation to the situation in which ap- proximations are needed only at points close to xo. For ordinary computational purposes it is more efficient to use methods that include information at various points, which we con- sider in the remainder of this chapter. The primary use of Taylor polynomials in numerical analysis is not for approximation purposes but for the derivation of numerical techniques and error estimation. n 0 1 2 Pr) | 1 3.1 Interpolation and the Lagrange Polynomial Since the Taylor polynomials are not appropriate for interpolation, alternative methods are needed. In this section we find approximating polynomials that are determined simply by specifying certain points on the plane through which they must pass. The problem of determining a polynomial of degree one that passes through the dis- tinct points (xo, yo) and (x1, y1) is the same as approximating a function f for which ‘Jf (%o) = yo and f (x1) = y; by means of a first-degree polynomial interpolating, or agree- ing with, the values of f at the given points. We first define the functions Lox) ==) and Lyx) = =, Xoo x0 and then define P(x) = Lolx) f 0) + Lie) f(a).Figure 33 Figure 34 CHAPTER 3 = Interpolation and Polynomial Approximation Since Lo(xo) = 1, Lo(x1) =0, Lito) =0, and Li() = 1, wwe have P(xq) = 1+ Fo) + 0+ FO) = Fo) = yo and P(x) = 0+ F(x) +1 FO) = f@) = yi So P is the unique linear function passing through (xo, yo) and (x1, yi). (See Figure 3.3.) y= fe) 1 =f) Yo = Feo) y= Pox) ‘To generalize the concept of linear interpolation, consider the construction of a poly- nomial of degree at most n that passes through the n + 1 points (Go, F (Xo). (tr, FD), «+ Ons Fn). (See Figure 3.4.)3.1._Interpolation and the Lagrange Polynomial 109 In this case we need to construct, for each k = 0,1,... .7, a function Ly,e(x) with the property that Lp.x(x;) = 0 when i # k and Ly.x(x4) = 1. To satisfy Lyx(x;) = 0 for each i # k requires that the numerator of L,,.(x) contains the term (x = xo)(e = x1) ++ tea) = Xe) = En) To satisfy Ln.x(tz) = 1, the denominator of Lq,x() must be equal to this term evaluated atx = x,. Thus, (= Xo) = Xe) = Fert) = An) Oe = 0) ++ Oe tO = X41) + Oe = Xn) A sketch of the graph of a typical L,,. is shown in Figure 3.5. Ln xa) = Figure 3.5 Lys) The interpolating polynomial is easily described once the form of Ly,. is known. This polynomial, called the nth Lagrange interpolating polynomial, is defined in the follow- ing theorem. Theorem 3.2 If xo, x1,... x, aren + 1 distinct numbers and f is a function whose values are given at these numbers, then a unique polynomial P (x) of degree at most n exists with F(x) = Pox), foreach k=0,1,....1. ‘This polynomial is given by P(x) = flo) Lno(X) +--+ Fn) Ena) = DO Fennel), G1) rad where, for each k = 0,1,... (= 4) & Fett Ln. = (3.2) MO = GaN a) ENG) OD) 1&2 . Laan" Fe We will write L,,4(x) simply as Ly (x) when there is no confusion as to its degree.110 CHAPTER 3 ~ Interpolation and Polynomial Approximation EXAMPLE 1 Using the numbers (or nodes) x9 = 2, x; = 2.5, and.xz = 4 to find the second interpolating polynomial for f(x) = 1/x requires that we determine the coefficient polynomials Lo(x), L(x), and L(x). In nested form they are (& — 2.5) - Lo) = ae (%—6.5)x +10, Lyx) = LADERA _ (ote $2) 32 = O5-na5-H 3 and (45x45 3 Since f(%0) = f(2) = 0.5, fxr) = f(2.5) = 04, and f(2) = (4) = 0.25, we have PQ) = fGdLe) c= =0.5((x — 6.5)x +10) +04 tee +0, aS ton ts = (0.05x — 0.425)x + LIS. An approximation to f (3) = } (see Figure 3.6) is £(3) © PG) = 0.325 Figure 3.6 & —. Compare this to Table 3.1, where no ayer polynomial, expanded about xp = 1, could be used to reasonably approximate f (3) =Theorem 3.3 3.1. Interpolation and the Lagrange Polynomial 11 We can use a CAS to construct an interpolating polynomial. For example, in Maple we use >interp(X,Y,x); where X is the list [xo, ... , xp], ¥ is the list [f(x0), ..- » {(Gtq)], and x is the variable to be used. In this example we can generate the interpolating polynomial p = 0.05x? —0.425x-+ 1.15 with the command >p:sinterp([2,2.5,4], [0.5,0.4,0.25] ,x); To evaluate p(3) as an approximation to f(3) = }, enter >subs(x=3,p); which gives 0.325. ‘The next step is to calculate a remainder term or bound for the error involved in ap- proximating a function by an interpolating polynomial. This is done in the following theo- rem. Suppose Xo, X1,... ,%q are distinct numbers in the interval [a, b] and f = C"*![a, b]. ‘Then, for each x in [a, b], a number (x) in (a, b) exists with FOP ER) F(A) = POD + Ty A a) = de (3.3) where P(x) is the interpolating polynomial given in Eq. (3.1). . Proof Note first that if x = xy, for any k = 0,1,...,n, then f(x,) = P(x), and Olen, choosing &(x,) arbitrarily in (a,b) yields Eq. (3.3). If x # xx, for all k define the function g for ¢ in [a, b] by (= mo) 0) = (ea) A) te) a(t) = f(t) — P®) - (Ff) - PANG =x) =f) - PO) - [fF @) - roll =) Since f € C"*fa, b], and P € C™[a, b], it follows that g € C"*![a, b]. For f= x, we have Fy (k=) _ 80%) = fae) — PO) -(f@) - Po] — (f(x) — PQ) -0= 0. (x =m) Moreover, (8) =f) — P(x) -[f@)— PO) te e3 = f(x) = P(x) Lf) ~ PO) Thus, ¢ € C"*"[a, 6], and g is zero at the n +2 distinct numbers x, x0, x1, ... . Xn. By the Generalized Rolle’s Theorem, there exists a number £ in (a, 6) for which ¢*"'() = 0.12 EXAMPLE 2 CHAPTER 3 + Interpolation and Polynomial Approximation So a x = PE = fore) — nig) —(7a)— Poot | | it aa. GA) Since P(x) is a polynomial of degree at most n, the (n + 1)st derivative, P"+ (x), is identically zero. Also, []?_ol(t — x:)/(« — x,)] is a polynomial of degree (n + 1), so tt Gox) [rete | + (lower-degree terms in 1), =m) LTio@ — =) and qt x) (+I! ant i @=a) ~ Tio - =) Equation (3.4) now becomes SOME) — 0 — LF (2) — POD} and, upon solving for f(x), we have F(x) = PQ) + T]@-»- sae see ne The error formula in Theorem 3.3 is an important theoretical result because Lagrange polynomials are used extensively for deriving numerical differentiation and integration methods. Error bounds for these techniques are obtained from the Lagrange error formula. Note that the error form for the Lagrange polynomial is quite similar to that for the ‘Taylor polynomial. The nth Taylor polynomial about xo concentrates all the known infor- mation at x9 and has an error term of the form LEP EG) G@+D! The Lagrange polynomial of degree n uses information at the distinct numbers xo, x1, .%n and, in place of (x — xo)", its error formula uses a product of the n + I terms (8 20), =D aE f£OP EG) (+)! ‘The specific use of this error formula is restricted to those functions whose derivatives have known bounds. (= x9)" (a = x0)(x = a1) +++ Ot ~ ap). ‘Suppose a table is to be prepared for the function f(x) = e*, for x in {0, 1]. Assume the number of decimal places to be given per entry is d > 8 and that the difference between adjacent x-values, the step size, is h. What should h be for linear interpolation (that is, the Lagrange polynomial of degree 1) to give an absolute error of at most 10°?EXAMPLE 3 Table 3.2 3.1 Interpolation and the Lagrange Polynomial 113 Let xo, x1,-.. be the numbers at which f is evaluated, x be in [0,1], and suppose j satisfies x; 2. . Definition 3.7 implies that 1 Slim tet) EVFGH) fen te1.te a= VF On, and, in general, 1 Sin Sects ee nou) = GEV FO). Consequently, s+ De - SOF gage) gece g SEED HE tN= Dow rey Pala) = flea) +5V Gx) + >128 EXAMPLE 2 Table 3.9 CHAPTER 3 = Intepolation and Polynomial Approximation If we extend the binomial coefficient notation to include all real values of s by letting sot Io G+k-) bi _yt then Poo = Staale(-D'(5 ‘Yvroar-i( Se Fan +eu(? “Yer rose which gives the following result. Newton Backward-Difference Formula = stl) vt pox Pala) = Seal + YC HD) (Z) f(a) G.13) ‘The divided-difference Table 3.9 corresponds to the data in Example 1 First Second Third Fourth, divided divided ided divided differences differences differences differences 10 0.765197 0.4837057 13 0.6200860 0.108739 —0.5489460 16 0.455402 0.044433 0,0018251 ~0.5786120 . 19) 0.2818186 22 0.103623 Only one interpolating polynomial of degree at most 4 uses these five data points, but ‘we will organize the data points to obtain the best interpolation approximations of degrees 1, 2, and 3. This will give us a sense of accuracy of the fourth-degree approximation for the given value of x. If an approximation to f (1.1) is required, the reasonable choice for the nodes would be Xo = 1.0, x = 1.3, x2 = 1.6, x3 = 1.9, and xz = 2.2 since this choice makes the earliest possible use of the data points closest to x = 1.1, and also makes use of the fourth divided difference. This implies that h = 0.3 and s = 4, so the Newton forward divided-difference formula is used with the divided differences that have a solid underscore in Table 3.9: 1 Pa(L.l) = Pa(l.0-+ 50.3)) = 0.7651997 + $10.3)(-0.4837057) + ; (- 3) (0.3)°(—0.1087339)3.2. Divided Diferences 129 1(2\(5\ aos +4 (-3) (-3) (0.3)3(0.0658784) 1(_2\(_5\(_8 ‘, +5 (-3) (-3) (-3) (0.3)*(0.0018251) = 0.7196480. To approximate a value when x is close to the end of the tabulated values, say, x we would again like to make the earliest use of the data points closest to x. This re using the Newton backward divided-difference formula with s differences in Table 3.9 that have a dashed underscore: 100)=n(22- 203) = 0.1103623 2(1\(4 -3(3) (3) (0.3)3(0.0680685) 0.238754, . Fos -0 5715210) ‘The Newton formulas are not appropriate for approximating f(x) when x lies near the center of the table since employing either the backward or forward method in such @ way ‘that the highest-order difference is involved will not allow x9 to be close to x. A number of divided-difference formulas are available for this case, each of which has situations when it can be used to maximum advantage. These methods are known as centered-difference formulas. There are a number of such methods, but we will present only one, Stirling's method, and again refer the interested reader to [Hild] for a more complete presentation, For the centered-difference formulas, we choose x9 near the point being approxi- mated and label the nodes directly below xo as x1, X2, and those directly above as X-1) 4-2)...» With this convention, Stirling’s formula is given by h 2 Pa) = Pans) = fla) + Fle 30) + Flo.) +54? Fle. x0.¥1) B14) 1a + LoD pa, ais toot + Flx-1 40041 a2) tee $5265? = 165? = 4) 2 (8? = Om = DPV f Beams oo Sin) 1). (9? =m )nm! 2 if n = 2m +1 is odd. If n = 2m is even, we use the same formula but delete the last line. ‘The entries used for this formula are underlined in Table 3.10 on page 130. (s? + (feamaty oo im) + fleas oo mts EXAMPLE 3 Consider the table of data that was given in the previous examples. To use Stirling's formula to approximate f (1.5) with xo = 1.6, we use the underlined entries in the difference Table 3A.130 Table 3.10 Table 3.11 CHAPTER 3 - Interpolation and Polynomial Approximation First Second Third Fourth divided divided divided divided x f(x) differences__differences differences differences xa flea) Flat] xa flxal S241, 40) les, x0) fle Ho, x1] xo fled fle1 0.41) Flea, x1, 20.41, 22) fl0.%1] flea X00 X12] a fll Pleo. x1. 2] Fle, x2] v2 fel First Second Third Fourth divided divided divided divided x F(x) differences differences differences differences 10 0.7651977 =0.4837057 13 0,6200860 —0.1087339 0.5489460 0.0658784 16 0.454022 =0.0494433 0.0018251 =0.5786120 (0.0680685 1.9 0,2818186 0.018183 -0.5715210 22 0.103623 = oassaon2 + (-3) ( %) ((-0.5489460) + (—0.5786120)) 1? + (-3) (0.3)?(—0.0494433) 17.1 1)? 5 7 + z (-4) (4) - ‘) (0.3)° (0.0658784 + 0.0680685) nefy ye +(-3 ((-) -1) (0.3)*(0.0018251) = 0.5118200. .32 Divided Differences 131 SS EXERCISE SET 3.2 L 2 Use Newton's interpolatory divided-difference formula or Algorithm 3.2 to construct inter- polating polynomials of degree one, two, and three for the following data, Approximate the specified value using each of the polynomials. a. f(8.4) if f(8.1) = 16.94410, f(8.3) = 17.56492, (8.6) = 18.50515, f(8.7) = 18.82091 b. FO) if f(0.6) = —0.17694460, f(0.7) = 0.01375227, (0.8) = 0.22363362, ‘F(1.0) = 0.65809197 Use Newton's forward-difference formula to construct interpolating polynomials of degree ‘one, Wo, and three forthe following data, Approximate the specified value using each of the polynomials. a f(-4) if f(-0.75) = —0.07181250, f(-0.5) = -0.02475000, f(-0.25) = 0.33493750, # (0) = 1.10100000 b. (0.25) if f(0.1) = —0.62049958, 7 (0.2) = —-0.28398668, (0.3) = 0.00660095, £ (0.4) = 0.24842440 Use Newton's backward-difference formula to construct interpolating polynomials of degree ‘one, two, and three for the following data, Approximate the specified value using each of the polynomials. a. f(-4) if f(-0.75) = —0.07181250, f(-0.5) = -0.02475000, f(-0.25) = 0.33493750, (0) = 1.10100000 db. £(0.25) if f(0.1) = —0.62049958, (0.2) = —0.28398668, f (0.3) = 0.00660095, (0.4) = 0.24842440 a. Use Algorithm 3.2 to construct the interpolating polynomial of degree four for the un- ‘equally spaced points given in the following table: f@) 6.00000 o.1 | -5.89483 03 | -5.65014 06 | ~5.17788 Lo | —4.28172 b. Add f(1-1) = —3.99583 to the table, and construct the interpolating polynomial of degree five. Approximate (0.05) using the following data and the Newton forward divided- difference formula: 00 [02 fos 06 Jos Fea) | 1.00000 | 1.22140 | 1.49182 | 1.82212 | 2.22558 D. Use the Newton backward divided-difference formula to approximate (0.65) €. Use Stirling’s formula to approximate (0.43), Show that the polynomial interpolating the following data has degree 3.132 CHAPTER 3 + Interpolation and Polynomial Approximation 7. a. Show that the Newton forward divided-difference polynomials, P(x) = 3-2 +1) +0 + DG) + & + D@E-1) and QU) = 144 +2) -3G $+ DEED $+ DOEF DE) both interpolate the data x -2|-1/0 1/2 ye -1[ 3[1[-1[3 'b. Why does part (a) not violate the uniqueness property of interpolating polynomials? 8. A fourth-degree polynomial P(x) satisfies A‘P(0) = 24, A®P(0) = 6, and A?P(0) = where AP(x) = P(x +1) — P(x). Compute A?P(10), 9. The following data are given for a polynomial P(x) of unknown degree. x jo] 1]2 Pa@) 2|-1/4 Determine the coefficient of x? in P(x) if all third-order forward differences are 1 10. The following data are given for a polynomial P(x) of unknown degree. x [Of1] 2) 3 P@|4folis| is Determine the coefficient of x* in P(x) if all fourth-order forward differences are 1 11. The Newton forward divided-difference formula is used to approximate f (0.3) given the fol- lowing data. 02) 04| 06 f(s) | 15.0 | 21.0 | 30.0 | Si Suppose it is discovered that (0.4) was understated by 10 and (0.6) was overstated by 5. By what amount should the approximation to f(0.3) be changed? 12. Fora function f, the Newton's interpolatory divided-difference formula gives the interpolating polynomial P(x) = 1 +e + 4x(x - 0.25) + eee —0.25)(x -0.5), (on the nodes xo = 0, x1 = 0.25, x = 0.5 and x5 = 0.75. Find (0.75), 13, Fora function f, the forward divided differences are given by Ft. 1) ftom.) =F Sls x2] = 10 Determine the missing entries in the table.3.3 Hermite interpolation 133 14, a, The introduction to this chapter included a table listing the population of the United States from 1940 to 1990. Use appropriate divided differences to approximate the population in the years 1930, 1965, and 2010. . The population in 1930 was approximately 123,203,000. How accurate do you think your 1965 and 2010 figures are?” 15. Given Py(x) = flto + flto. tra ~ x0) + ag(x — x0) = 41) fay(x = ap)(x = xy) = an) +++ 4 a6 — aa )(X = 1) ee ruse P, (x3) to show that ay = fx, x1, x2) 16, Show that fog) Gt” FUR Hty eo ne for some &(x). (Aint: From Eq. (3.3), - £6) Fa) = Pala + FEE Cx — 20) — a). Considering the interpolation polynomial of degree m+ 1 on xo, 815. ++ yeu, WE have F(R) = Pre (0) = Pal) + FE, Ay oo ony AIG = Ho) (=) 17. Loti. i, ... yy be a rearrangement of the integers 0, 1, ... .. Show that flxigs i. ---s Xi] = fl%o, Xi, «4 Xq)- [Hint: Consider the leading coefficient of the nth Lagrange polynomial fom the data (x04.t1¢.0+ «e} = (pe Sigs +i] LS 3.3 Hermite Interpolation Osculating polynomials generalize both the Taylor polynomials and the Lagrange poly- nomials. Suppose that we are given n + 1 distinct numbers x9, x),... ,%» in (a, b]-and nonnegative integers mo, m1,...,™®q, and m = max{mo, m;,... mq). The osculating polynomial approximating a function f ¢ C"[a, b] at x;, for each i = 0,... ,, is the polynomial of least degree with the property that it agrees with the function and all its derivatives of order less than or equal to m, at x;. The degree of this osculating polynomial is at most M=Som+n ms because the number of conditions to be satisfied is 77g mi + (n + 1), and a polynomial of degree M has M + 1 coefficients that can be used to satisfy these condit134 Definition 3.8 Theorem 3.9 CHAPTER 3. = Interpolation and Polynomial Approximation Let x0, x1, ..- +t, be m+ I distinct numbers in {a, b] and m; be a nonnegative integer asso- ciated with x;, for = 0, 1,... ,n. Suppose that f € C"[a, 6], where m = maxoxien ™ ‘The osculating polynomial approximating f is the polynomial P(x) of least degree such that ' aA P(x) _ d*f dxt ~ dxk » foreachi=0,1,...,0° and k=0,1,...,m . Note that when 1 = 0, the osculating polynomial.approximating is the moth Taylor polynomial for f at xo. When m; = 0 for each i, the osculating polynomial is the nth ‘Lagrange polynomial interpolating f on x0,X1, +6. «Xn ‘The case when m, = 1, for each i = 0, 1... ,n, gives the Hermite polynomials. For ‘a given function f, these polynomials agree with fat xo,1,... ,n. In addition, since their first derivatives agree with those of f, they have the same “shape” as the function at (x, f(%))) in the sense that the tangent lines'to the polynomial and to the function agree. We will restrict our study of osculating polynomials to this situation and consider first a theorem that describes precisely the form of the Hermite polynomials. If f € C'[a, b] and xo, ... , , € [a, b} are distinct, the unique polynomial of least degree agreeing with f and f’ at xo, ... , %, is the Hermite polynomial of degree at most 2n + 1 given by Hansi) = > F%)Hn sj) + DO Fy) Hass i= i= where Hy 2) = (1 = 2x — x/)L4, LG, | @) and Fy, j= (x = x,)E2 (2), In this context, Ly,)(x) denotes the jth Lagrange coefficient polynomial of degree n de- fined in Eq. (3.2). Moreover, if f € C*"*[a, 5), then (= 40)... (Qn +2)! Bn 2 £ (8) = Hans (x) + LOVE), for some§ witha <§ £0) OF FA) LHD £4) -0= FO), o = te 90 Ho» +1 agrees with f at x9, x}, Xn: To show the agreement of H;,,,, with J’ at the nodes, first note that L(x) is a factor of Hy ,(x), 80 Hf, (i) = O when i # j. In addition, when i = j and L,.i(%) = 1, we have Hg a4) = 21,05) Lh Gai) + (1 = 2044 = aL, ZL ADL y 4) = 214, (4) + 20}, (xi) = 0. Hence, Hy, (xj) = 0 for all i and j. Finally, Fy (aa) = LF 00a) + Ce = x) 2La EDLY, (40 = Lp jt ba 0) +204 = L010), so HY, ,(x;) =Oifi ¢ j and Ay ,(x;) = 1. Combining these facts, we have Hyp) = Yo FO) 0+ Df’) 04 f)-1= fad j= jab Jet Therefore, Han+1 agrees with f and Hy, with f’ at xo,%1,... Xn ‘The uniqueness of this polynomial and the error formula are considered in Exercise 8. Use the Hermite polynomial that agrees with the data listed in Table 3.12 to find an ap- proximation of (1.5).136 Table 3.12 CHAPTER 3. = Intepolation and Polynomial Approximation kon £64) i) 0 13 0.6200860 0.520232 1 16 04554022. ——0.5698959 219 02818186 11ST We first compute the Lagrange polynomials and their derivatives. This gives (w—x)@—x) 50, 175, 152 100.175 Lyo(x) = RoE) rT Lyo(x) = ex — 209) = GaGa) 9 OTTO 2002) = tg 100, , 320 247 LQ) = 200, 320 9 a an) 9 and (x= x0)(@ =m) 104 : 100 faa = a) a" baal) = 9 The polynomials H,;(x) and A, ;(x) are then 2 o(X) = [= 2 ~ 1.3)(-5)] (3. 2 By 12) 9 9 50,,_ 175, 152 2 (10x, 19) (2s oo 100, 320. 247\* Hoi(x) =1 +i), ats) (S +o >) _ 50,2 _ 145 4 toe 2 Hy, 2(x) = 10(2 (Fx > 9 9) . . 50 175 412) Ago(x) = (x — 1.3) ( =x? - x ols) = ¢ ee eae), 2 Base) = = 1.6 (= +P. - 2) . and Aya) = -19(F Finally, Hs(x) = 0.6200860H, o(x) + 0.455402 Hp,1(x) + 0.2818186H2 2(x) = 0,5220232Ah o(x) — 0.5698959 Fh, (x) — 0.581157] Fh.2(x)3.3. Hermite interpolation 137 and 4 4 5 Hs(1.5) = 0.621 = ) +0.4554022 ( ~ ) +0.281 = (1.5) = 0. 0850 ( +) + 024 (3)+ sis6(=) 4 =32 0.52202 =) ~ ==) -o.s811571 32 (4s) o.seasaso (=) o.s8115 ( = 05118277, a result that is accurate to the places listed . Although Theorem 3.9 provides a complete description of the Hermite polynomials, it is clear from Example | that the need to determine and evaluate the Lagrange polynomials and their derivatives makes the procedure tedious even for small values of n. An alternative method for generating Hermite approximations has as its basis the Newton interpolatory divided-difference formula (3.10) for the Lagrange polynomial at x0, 1, -.. sn, XJ ~ 80) OF = Hs Pax) = FO + DO F001 it and the connection between the nth divided difference and the nth derivative of f, as outlined in Theorem 3.6 in Section 3.2. ‘Suppose that the distinct numbers x9, x1,... ,%, are given together with the values of J and f’ at these numbers. Define a new sequence zo, 21, ... » Zan41 BY 2 =a =H, foreachi=0,1,....m, and construct the divided difference table in the form of Table 3.7 that uses zp, 21, .--+ Zant: Since zz; = cas; = x) for each i, we cannot define f[z2, z2/+1] by the divided difference formula. If we assume, based on Theorem 3.6, that the reasonable substitution in this situation is f[2xj, zai+1] = f’(@ai) = f’(%i), we can use the entries SF (40), fC). sf On) in place of the undefined first divided differences Fiza, 21), flza, 23], «+.» Fans Zang]: ‘The remaining divided differences are produced as usual, and the appropriate divided dif- ferences are employed in Newton's interpolatory divided-difference formula. Table 3.13 shows the entries that are used for the first three divided-difference columns when de- termining the Hermite polynomial Hs(x) for xo, x1, and xp. The remaining entries are generated in the same manner as in Table 3.7. The Hermite polynomial is given by eH Hanxi (x) = Flzol + >>) Feo, +++ + Za — Zoe ~ Za) ~ B=). cot A proof of this fact can be found in [Po, p. 56]138 Table 3.13 EXAMPLE 2 Table 3.14 CHAPTER 3 = Intepolation and Polynomial Approximation First divided ‘Second divided z f@ differences differences = %9 — flzo) = f(x) fl20, 211 = f'Gxo) nam flel= Soa flzo, 21.22) Angle al Flan zal = fala fe nen flel= se) Flom tng) = Hee Les Flea, 23] = #04) 1 J [ 1 nen flel=sov Flan 25,24) = Leste = fleas flex 2 = =} wen flel= fx) Sle 24, 25) = Een) of ad Flea, zs] = f'(a) gem files) = fe) ‘The entries in Table 3.14 use the data given in Example 1. The underlined entries are the given data; the remainder are generated by the standard divided-difference formula (3.9): Hs(1.5) = 0.6200860 + (1.5 — 1.3)(—0.5220232) + (1.5 — 1.3)°(—0.0897427) + (1.5 = 1.3)7(1.5 — 1.6)(0.0663657) + (1.5 — 1.3)°(1.5 — 1.6)*(0.0026663) + (LS ~ 1.37.5 ~ 1.6)°(1.5 ~ 1.9)(—0.0027738) = 0.511827, . 13 0,6200860 =0.5220232 1.3 046200860 —0.0897427 ~ —0.5489460 0.063657 0.455402 = 0.0698330 0.002663, —0.5698959 0.067965 =0.0027738 1.6 0.455402 =0.0290537 0.0010020 ~ ~~ ~0,5786120 0.0685667 19 02818186 0.084837 — -0.5811571 19 0,2818186 The technique used in Algorithm 3.3 can be extended for use in determining other osculating polynomials. A concise discussion of the procedures can be found in [Po, pp. 53-57]3.3. Hermite interpolation 139 peas Hermite Interpolation 33 ‘To obtain the coefficients of the Hermite interpolating polynomial H(x) on the (n + 1) distinct numbers xo, ... , X_ for the function f: INPUT numbers x0, x1, ... . m3 Values (x0), +f (n) and f(x)... F'n). OUTPUT the numbers oo, Q1.1 Qinsianer Where H(3x) = Qoo + Qua — x0) + Q2.a0% — x0)? + Qa ~ x9)*(x = a1) + Qsa(x — x0)2(x = xy)? + F Qonst.anti = X09)? =P Oe = yO = Hy). Step 1 Fori =0.1,... .n do Steps 2 and 3. Step2 Set oa Za =X Quo = Fx): Qrso = fds Orisa = F%). Step 3 Iti #Othen set 0 = Qr-19 22,1 2b — 21-1 Step4 For =2,3,...,2n+1 - 0 for j=2.3,...,iset Q,, = Sut ins, ua tes Step 5 OUTPUT (Qo0, Qi.ty-++ » Qanti2nti)s STOP . 1 SE SSE SESS EXERCISE SET 33 1. Use Theorem 3.9 or Algorithm 3.3 to construct an approximating polynomial for the following data a box fy | fe 83 | 1756492 | 3.116256 os | 0.22363362 | 2.1691753 8.6 | 1850515 | 3.151762 1.0 | 0.65809197 | 2.0466965 © * fa) fi) ~0.0247500 0.1 | -0.62049958 | 358502082 ~0.25 | 03349375 | 2.1890000 0.2 | ~0.28398668 | 3.140327) 0 1.1010000 | 4.020000 0.3 | 0,00660095 | 2.66668043 0.4 | 0.24842440 | 2.16529366 2. The data in Exercise 1 were generated using the following functions. Use the polynomials constructed in Exercise 1 for the given value of x to approximate f(x), and calculate the actual error.