Chapter 1 Finite Difference Approximations Our goal is to approximate solutions to differential equations, ie., to find a function (or some discrete approximation to this function) that satisfies a given relationship between various of its derivatives on some given region of space and/or time, along with some boundary conditions along the edges of this domain. In general this is a difficult problem, and only rarely can an analytic formula be found for the solution. A finite difference method proceeds by replacing the derivatives in the differential equations with finite difference approximations. This gives a large but finite algebraic system of equations to be solved in place of the differential equation, something that can be done on a computer, Before tackling this problem, we first consider the more basic question of how we can approximate the derivatives of a known function by finite difference formulas based only on values of the function itself at discrete points, Besides providing a basis for the later development of finite difference methods for solving differential equations, this allows us to investigate several key concepts such as the order of accuracy of an approximation in the simplest possible setting. Let u(x) represent a function of one variable that, unless otherwise stated, will always be assumed to be smooth, meaning that we can differentiate the function several times and each derivative is a well-defined bounded function over an interval containing a particular point of interest ¥. Suppose we want to approximate w'(%) by a finite difference approximation based only on values of u at a finite number of points near X. One obvious choice would be to heal u& +h) —u@) i for some small value of h. This is motivated by the standard definition of the derivative as the limiting value of this expression as h —> 0. Note that D.,u(2) is the slope of the line interpolating w at the points and ¥ + h (see Figure 1.1) The expression (1.1) is a one-sided approximation to w’ since w is evaluated only at values of x >. Another one-sided approximation would be Dyuli ay (1.2)4 Chapter 1. Finite Difference Approximations slope Dy.u(z) “Slope D_u(z) slope u’(z) slope Dou(®) u(x) Figure 1.1. Various approximations to u' (<) interpreted as the slope of secant lines. Each of these formulas gives a first order accurate approximation to w/(z), meaning that the size of the error is roughly proportional to h itself. Another possibility is to use the centered approximation seth ue - 0) = 5D) + D_u()), (13) This is the slope of the line interpolating w at % —h and % + h and is simply the average of the wo one-sided approximations defined above, From Figure 1.1 it should be clear that we would expect Dou() to give a better approximation than either of the one-sided approximations, In fact this gives a second onder accurate approximation—the error is proportional to h? and hence is much smaller than the error in a first order approximation when A is small. Other approximations are also possible, for example, Dou(z) = Dau(k) = Gp l2ul& + h) + Bul) — 6u(% —h) + u( 2A) (14) Itmay not be clear where this came from or why it should approximate w' at all, but in fact it turns out to be a third order accurate approximation—the error is proportional to h? when his small, Our first goal is to develop systematic ways to derive such formulas and to analyze their accuracy and relative worth. First we will look at a typical example of how the errors in these formulas compare. Example 1.1, Let u(x) = sin(x) and ¥ = 1; thus we are trying to approximate w'(1) = cos(1) = 0.5403023. Table 1.1 shows the error Du(z) — w’(2) for various values of h for each of the formulas above, We see that Du and D_u behave similarly although one exhibits an error that is roughly the negative of the other, This is reasonable from Figure 1.1 and explains why Dou, the average of the two, has an error that is much smaller than both.1.1, ‘Truncation errors 5 ‘able 1.1. Errors in various finite difference approximations to u'(Z). h Dy-u(X) D-u(X) Dou(®) Dau(®) 1.0e—01 | —4.2939e-02 4.1138e-02 —9.0005e-04 6.8207e-05 5.0e—-02 | -2.1257e-02 2,0807e-02 —2.2510e-04 8.649 1e—06 1.0e-02 | —4.2163e-03 4.1983e-03 -9.0050e-06 6.9941e—08 5.0e—03 | —2.1059e-03 2.1014e-03 —2.2513e-06 8.75402—09 Oe—03 2083e—04 4.2065e-04 —9.0050e-08 _6.9979e-11 ‘We see that D4.u(3) — ul) © -0.42h, y Dou(%) — u'(%) ¥ —0.09h?, Dyu() — u!(&) © 0.007h3, confirming that these methods are first order, second order, and third order accurate, respectively. Figure 1.2 shows these errors plotted against h on a log-log scale. This is a good way to plot errors when we expect them to behave like some power of h, since if the error E(h) behaves like E(h) » Ch?, then log|E(hi)| ~ log|C| + plogh. So on a log-log scale the error behaves linearly with a slope that is equal to p, the order of accuracy. 1.1 Truncation errors ‘The standard approach to analyzing the error in a finite difference approximation is to expand each of the function values of w in a Taylor series about the point X, e.g., 1 1 (E+ h) = WG) + hul B+ 5hPu") + Ghul) 4 oth), (Sa) i i u(& — A) = u(&) — hu! (&) + piu") - gee’ + O(n). (1.5b) ‘These expansions are valid provided that u is sufficiently smooth. Readers unfamiliar with the “big-oh” notation O(h*) are advised to read Section A.2 of Appendix A at this point since this notation will be heavily used and a proper understanding of its use is critical. Using (1.5a) allows us to compute that hy—u(®) _ oan) = Dau = 4&2 w'(®)+ shu") + zune) + O(h*).6 Chapter 1, Finite Difference Approximations | a | | Dy + —* 7 10} ° . “| Do 10°} ao Dy 10°" « Figure 1.2. The errors in Du(X) from Table 1.1 plotted against h on a log-log scale. Recall that % is a fixed point so that u"(z), w’”(#), etc., are fixed constants independent of ‘h. They depend on u of course, but the function is also fixed as we vary h. For A sufficiently small, the error will be dominated by the first term 4Au”(&) and all the other terms will be negligible compared to this term, so we expect the error to behave roughly like a constant times A, where the constant has the value 41”(Z). Note that in Example 1.1, where u(x) = sinx, we have 4u’(1) = —0.4207355, which agrees with the behavior seen in Table 1.1. Similarly, from (1.5b) we can compute that the error in D_u(3) is D_u(®) —ul(@) = — Siu" (@) + zur) + 00°), which also agrees with our expectations, Combining (1.5a) and (1.5b) shows that u(X +h) — u(% — A) = hu! (%) + Fea) + O(n) so that 7 Dou(&) — u'(%) = gihu®) + O(h4). (1.6) This confirms the second order accuracy of this approximation and again agrees with what is seen in Table 1.1, since in the context of Example 1.1 we have 1 du"@) = —gc08(1) = —0.09005038. Note that all the odd order terms drop out of the Taylor series expansion (1.6) for Dou(). This is typical with centered approximations and typically leads to a higher order approxi- mation,1.2. Deriving finite difference approximations 7 To analyze D3u we need to also expand u(% — 2h) as u(% — 2h) = u(&) — 2hu'(R) + Fe2h)?u") - geonyru"@) + O(n). (1.7) Combining this with (1.5a) and (1.5b) shows that Dsu(%) = u'() + fru@) + O(h*), (1.8) where u(4) is the fourth derivative of u. 1.2 Deriving finite difference approximations Suppose we want to derive a finite difference approximationgo u’() based on some given set of points. We can use Taylor series to derive an appropriate formula, using the method of undetermined coefficients. Example 1.2. Suppose we want a one-sided approximation to u’(%) based on u(%), u(% — A), and u(% — 2h) of the form Dyu(X) = au(%) + bu(% — h) + cu( — 2h). (1.9) We can determine the coefficients a, 6, and c to give the best possible accuracy by expand- ing in ‘Taylor series and collecting terms. Using (1.5b) and (1.7) in (1.9) gives Du = a+b + chu) — (b + 2e)hu'(%) + se + 4e)h?u" (x) - 70 + 8c)hPu() +o If this is going to agree with u’(%) to high order, then we need atb+c=0, b+2c¢=-1/h, (1.10) b+4ce= ‘We might like to require that higher order coefficients be zero as well, but since there are only three unknowns a, b, and c, we cannot in general hope to satisfy more than three such conditions. Solving the linear system (1.10) gives a=3/2h, b=-2h, c= 1/2h so that the formula is Dru ou) — 4u(%— h) + u@& — 2h). (ua) ‘This approximation is used, for example, in the system of equations (2.57) for a 2-point boundary value problem with a Neumann boundary condition at the left boundary.8 Chapter 1. Finite Difference Approximations ‘The error in this approximation is Dyu(%)—w'(&) = -70 + 8c)ABu!(E) po Lance 3 72) = ah w"@)+ 01H). There are other ways to derive the same finite difference approximations. One way is to approximate the function u(x) by some polynomial p(x) and then use p’(#) as an approximation to w'(Z). If we determine the polynomial by interpolating u at an appropriate set of points, then we obtain the same finite difference methods as above. Example 1.3. To derive the method of Example 1.2 in this way, let p(x) be the quadratic polynomial that interpolates u at 2, #—h and ¥—2h, and then compute p/(3) ‘The result is exactly (1.11). 1.3. Second order derivatives Approximations to the second derivative w(x) can be obtained in an analogous manner. The standard second order centered approximation is given by DPu(z) = Awe =A) — 2u(Z)+uE+A)] : i (1.13) =ul(a)+ peut owns). Again, since this is a symmetric centered approximation, all the odd order terms drop out, This approximation can also be obtained by the method of undetermined coefficients, or alternatively by computing the second derivative of the quadratic polynomial interpolating u(x) at £—h, %, and ¥ +h, as is done in Example 1.4 below for the more general case of unequally spaced points, Another way to derive approximations to higher order derivatives is by repeatedly applying first order differences. Just as the second derivative is the derivative of w’, we can view D?u(Z) as being a difference of first differences. In fact, D?u(&) = Dy D_u() since D4(D_u(%)) = jlD-ue-+8) ~ D_u()] a (Sn) one) kh h ( h J = D*u(a). Alternatively, D?(z) = D_D,.u(z), or we can also view it as a centered difference of cen- tered differences, if we use a step size h/2 in cach centered approximation to the first derivative. If we define but=}(e(x+8)-a(--#)),1.4, Higher order derivatives 9 then we find that BotBout) = a ((“& 0) 18) _ (sant -»)) ol Example 1.4. Suppose we want to approximate 1/"(x2) based on data values Uj, U2, and Us, at three unequally spaced points x1, x2, and x3. This approximation will be used in Section 2.18. Let hy = x2 — x1 and ha = x3 — x2. The approximation can be found by interpolating by a quadratic function arid differentiating twice. Using the Newton form of the interpolating polynomial (see Section B.2.3), P(x) = Ula] + U fen, xa] — x1) + Ofer, x2, x3) (% — x1) = X2)s we see that the second derivative is constant and equal to twice the second order divided difference, , P" (2) = 2U[1, x2, 3] __ (a7 a2 (a8 te ) [ct +h) aia) = eU + 2U) + 030s, where 2 2 2 =o, a, sep iy + ha) a Ihe > Tah + ha) This would be our approximation to u"(xz). The same result can be found by the method of undetermined coefficients. ‘To compute the error in this approximation, we can expand (x1) and u(x3) in Taylor series about x2 and find that a (Ls) cru ai) + cau (x2) + esu(xs) — u"(%2) 1 1 (Wi +h3 (1.16) = dy — aye 1( s+), cn zlliz Inu) (2) +5 es 8). (eva) In general, if hy # a, the error is proportional to max(/1, 2) and this approximation is “first order” accurate. Tn the special case ty = hp (equally spaced points), the approximation (1.14) reduces to the standard centered approximate D?u(x2) from (1.13) with the second order error shown there. 1.4 Higher order derivatives Finite difference approximations to higher order derivatives can also be obtained using any of the approaches outlined above, Repeatedly differencing approximations to lower order derivatives is a particularly simple approach. Example 1.5. As an example, here are two different approximations to w’”(%), The first is uncentered and first order accurate:10 Chapter 1. Finite Difference Approximations Dy D'u(%) = pu + 2h) — 3u(% +A) + 3u(Z) —u(% —h)) =u") + Shu") + O(n). ‘The next approximation is centered and second order accurate: Do Dy D-u(X) = pu + 2h) — 2u(% + A) + 2u(% — A) — u(¥ — 2h) sul") + yews) + O(n). Another way to derive finite difference approximations to higher order derivatives is by interpolating with a sufficiently high order polynomial based on function values at the desired stencil points and then computing the appropriate derivative of this polynomial ‘This is generally a cumbersome way to do it. A simpler approach that lends itself well to automation is to use the method of undetermined coefficients, as illustrated in Section 1.2 for an approximation to the first order derivative and explained more generally in the next section, 1.5 A general approach to deriving the coefficients ‘The method illustrated in Section 1.2 can be extended to compute the finite difference co- efficients for computing an approximation to u(x), the kth derivative of u(x) evaluated at ¥, based on an arbitrary stencil of m > k +1 points.x1, ..., ¥- Usually X is one of the stencil points, but not necessarily. We assume u(x) is sufficiently smooth, namely, at least m + 1 times continuously differentiable in the interval containing X and all the stencil points, so that the Taylor series expansions below are valid. Taylor series expansions of u at each point x; in the stencil about u() yield u(x) = u(%) + (x7 — Bu! (R) +o + hei = Xu) +o (1.17) fori = 1, ..., n. We want to find a linear combination of these values that agrees with u® (%) as well as possible. So we want cu (xy) + cou (Xe) ++ + Cntt(Xn) = uO (H) + OCH”), (1.18) where p is as large as possible, (Here h is some measure of the width of the stencil. If we are deriving approximations on stencils with equally spaced points, then / is the mesh width, but more generally it is some “average mesh width,” so that max1. k the coefficients give a suitable finite difference approximation. How accurate is the method? The right-hand side vector has a] in the i = k +1 row, which ensures that this linear combination approximates the kth derivative. The 0 in the other component of the right-hand side ensures that the terms drop out in the linear combination of Taylor series fori — 1 # k. For i —1 < k this is necessary to get even first order accuracy of the finite difference approximation. For i —1 > k (which is possible only if n > k + 1), this gives cancellation of higher order terms in the expansion and greater than first order accuracy. In general we expect the order of accuracy of the finite difference approximation to be at least p 2 n — k. It may be even higher if higher order terms happen to cancel out as well (as often happens with centered approximations, for example). In MATLAB itis very easy to set up and solve this Vandermonde system. If xbar is the point ¥ and x (1:n) are the desired stencil points, then the following function can be used to compute the coefficients: function c A = ones (n,n); xrow = (x(:)-xbar)’; % displacements as a row vector. coef £V(k, xbar, x) for i=2:n A(i,:) = (xrow .* (i-1)) ./ factorial (4-1); end b = zeros(n,1); % b is right hand side, b(k+1) = 1; % so k’th derivative term remains ¢ = A\b; % solve system for coefficients cacy % row vector If wis a column vector of 7 values u(x), then in MATLAB the resulting approximation to u)(%) can be computed by c*u. ‘This function is implemented in the MATLAB function £dcoef £V.m available on the Web page for this book, which contains more documentation and data checking but is essentially the same as the above code. A row vector is returned since in applications we will often use the output of this routine as the row of a matrix approximating a differential operator (see Section 2.18, for example). Unfortunately, for a large number of points this Vandermonde procedure is numeri- cally unstable because the resulting linear system can be very poorly conditioned. A more stable procedure for calculating the weights is given by Fornberg [30], who also gives a FORTRAN implementation. This modified procedure is implemented in the MATLAB function £dcoeé £F .mon the Web page. Finite difference approximations of the sort derived in this chapter form the basis for finite difference algorithms for solving differential equations. In the next chapter we begin the study of this topic.