1

Numerical method

 Dr. Ali A. F. Al-Hamadani

 [email protected]
References 1. ERWIN KREYSZIG, ‘’ ADVANCED ENGINEERING MATHEMATICS’’, 2011, 10 ed, John Wiley & Sons Inc.

Tuesday, April 21,

2020
2
Numeric in General
 Numerics analysis or briefly numerics has a distinct flavor that is different from basic calculus, from solving ODEs
algebraically, or from other (nonnumeric) areas. Whereas in calculus and in ODEs there were very few choices on how to
solve the problem and your answer was an algebraic answer, in numerics you have many more choices and your answers
are given as tables of values (numbers) or graphs.

 You have to make judicious choices as to what numeric method or algorithm you want to use, how accurate you need
your result to be, with what value (starting value) do you want to begin your computation, and others. This chapter is
designed to provide a good transition from the algebraic type of mathematics to the numeric type of mathematics.

 We begin with the general concepts such as floating point, roundoff errors, and general numeric errors and their
propagation. This is followed in Sec. 19.2 by the important topic of solving equations of the type by various numeric
methods, including the famous Newton method.

Tuesday, April 21,

2020
 Numeric in General continue
Section 19.3 introduces interpolation methods. These are methods that construct new (unknown) function
values from known function values. The knowledge gained in Sec. 19.3 is applied to spline interpolation
(Sec. 19.4) and is useful for understanding numeric integration and differentiation covered in the last
section. Numerics provides an invaluable extension to the knowledge base of the problem solving
engineer. Many problems have no solution formula (think of a complicated integral or a polynomial of
high degree or the interpolation of values obtained by measurements).
In other cases a complicated solution formula may exist but may be practically useless. It is for these
kinds of problems that a numerical method may generate a good answer. Thus, it is very important that
the applied mathematician, engineer, physicist, or scientist becomes familiar with the essentials of

3 numerics and its ideas, such as estimation of errors, order of convergence, numerical methods expressed
in algorithms, and is also informed about the important numeric methods.

Prerequisite: Elementary calculus.

References and Answers to Problems: App. 1 Part E, App. 2.
Tuesday, April 21,
2020
 4 19.1 Introduction
 As an engineer or physicist you may deal with problems in elasticity and need to solve an

equation such as or a more difficult problem of finding the roots of a higher order polynomial.

Or you encounter an integral such as

1
 ‫׬‬0 exp(−𝑥 2 ) dx

 That you cannot solve by elementary calculus. Such problems, which are difficult or impossible
to solve algebraically, arise frequently in applications. They call for numeric methods, that is,
systematic methods that are suitable for solving, numerically, the problems on computers or
calculators. Such solutions result in tables of numbers, graphical representation (figures), or
both.

Tuesday, April 21,

2020
 Introduction
 Typical numeric methods are iterative in nature and, for a well-chosen problem and a good starting
5 value, will frequently converge to a desired answer. The evolution from a given problem that you
observed in an experimental lab or in an industrial setting (in engineering, physics, biology, chemistry,
economics, etc.) to an approximation suitable for numerics to a final answer usually requires the
following steps.
 1. Modeling. We set up a mathematical model of our problem, such as an integral, a system of
equations, or a differential equation.
 2. Choosing a numeric method and parameters (e.g., step size), perhaps with a preliminary error
estimation.
 3. Programming. We use the algorithm to write a corresponding program in a CAS, such as Maple,
Mathematica, Matlab, or Mathcad, or, say, in Java, C or or FORTRAN, selecting suitable routines from
a software system as needed.
 4. Doing the computation.
 5. Interpreting the results in physical or other terms, also deciding to rerun if further results are needed.
 Steps 1 and 2 are related. A slight change of the model may often admit of a more efficient method.
To choose methods, we must first get to know them. Chapters 19–21 contain efficient algorithms for
the most important classes of problems occurring frequently in practice.
 In Step 3 the program consists of the given data and a sequence of instructions to be executed by
the computer in a certain order for producing the answer in numeric or graphic form. To create a
good understanding of the nature of numeric work, we continue in this section with some simple
Tuesday, April 21,
general remarks. 2020
 Floating-Point Form of Numbers
6
 We know that in decimal notation, every real number is represented by a finite or an infinite
sequence of decimal digits. Now most computers have two ways of representing numbers, called
fixed point and floating point. In a fixed-point system all numbers are given with a fixed number of
decimals after the decimal point; for example, numbers given with 3 decimals are 62.358, 0.014,
1.000.
 In a text we would write, say, 3 decimals as 3D. Fixed-point representations are impractical in most
scientific computations because of their limited range (explain!) and will not concern us.
 In a floating-point system we write, for instance,
 0.6247. 10 3 , 0.1735. 10−13 , −0.2000. 10−1
 Or some times also
 6.247. 10 2, 1.735. 10−14 , −2.000. 10−2
 We see that in this system the number of significant digits is kept fixed, whereas the decimal point is
“floating.” Here, a significant digit of a number c is any given digit of c, except possibly for zeros to
the left of the first nonzero digit; these zeros serve only to fix the position of the decimal point. (Thus
any other zero is a significant digit of c.) For instance,
 13600, 1.3600, 0.0013600
 all have 5 significant digits. In a text we indicate, say, 5 significant digits, by 5S.
Tuesday, April 21,
2020
 The use of exponents permits us to represent very large and very small numbers. Indeed, theoretically any
nonzero number a can be written as
7
 1 𝑎 = ±𝑚 . 10𝑛 , 0.1 ≤ 𝑚 < 1, 𝑛 𝑖𝑛𝑡𝑒𝑔𝑒𝑟

 On modern computers, which use binary (base 2) numbers, m is limited to k binary digits (e.g., k=8) and n is
limited (see below), giving representations (for finitely many numbers only!)
 2 ഥ . 2𝑛 ,
𝑎ത = ±𝑚 𝑚
ഥ = 0. 𝑑1 𝑑2 … 𝑑𝑘 ,

 These numbers 𝑎ത are called k-digit binary machine numbers. Their fractional part m (or 𝑚 ഥ ) is called the
mantissa. This is not identical with “mantissa” as used for logarithms. n is called the exponent of 𝑎.
ത

 It is important to realize that there are only finitely many machine numbers and that they become less and less
“dense” with increasing a. For instance, there are as many numbers between 2 and 4 as there are between 1024
and 2048. Why?
 The smallest positive machine number 1+eps>1 with is called the machine accuracy. It is important to realize
that there are no numbers in the intervals [1,1+eps],[2,2+2.eps],…,[1024,1024+1024.eps],… . This means that,
if the mathematical answer to a computation would be 1024+1024.eps/2, the computer result will be either
1024 or 1024.eps so it is impossible to achieve greater accuracy.
Tuesday, April 21, 2020
 Underflow and Overflow
8
 The range of exponents that a typical computer can handle is very large. The IEEE (Institute
of Electrical and Electronic Engineers) floating-point standard for single precision is from 2-
126 to 2128 (1.175*10-23 to 3.403*1038 ) and for double precision it is from 2-1022 to 21024
(2.225*10-308 to 1.798*10308 ).

 If, in a computation a number outside that range occurs, this is called underflow when the
number is smaller and overflow when it is larger. In the case of underflow, the result is
usually set to zero and computation continues. Overflow might cause the computer to halt.

Tuesday, April 21,

2020
 Roundoff
9

 An error is caused by chopping (= discarding all digits from some decimal on) or
rounding. This error is called roundoff error, regardless of whether we chop or
round. The rule for rounding off a number to k decimals is as follows. (The rule
for rounding off to k significant digits is the same, with “decimal” replaced by
“significant digit.”).

 Roundoff Rule. To round a number x to k decimals, and 5 * 10-(k+1) to x and

chop the digits after the (k + 1)st digit.

Tuesday, April 21,

2020
 E X A M P L E 1 Roundoff Rule
10
 Round the number 1.23454621 to (a) 2 decimals, (b) 3 decimals, (c) 4 decimals,
(d) 5 decimals, and (e) 6 decimals.

 Solution. (a) For 2 decimals we add 5 * 10-(k+1) = 5 * 10-3 =0.005 to the given
number, that is, 1.2345621+0.005=1.23 954621. then we chop off the digits
‘’954621’’ after the space or equivalently 1.23954621-0.00954621=1.23.
 (b) 1.2345621+0.0005=1.235 04621, so that for 3 decimals we get 1.234.
 (c) 1.23459621 after chopping give us 1.2345 (4 decimals).
 (d) 1.23455121 yield 1.23455 (5 decimals)
 (e) 1.23454671 yield 1.234546 (6 decimals).
 Can you round the number to 7 decimals? (HW1)
Tuesday, April 21,
2020
11

Rounding errors may ruin a computation completely, even a small computation. In general, these
errors become the more dangerous the more arithmetic operations (perhaps several millions!) we
have to perform. It is therefore important to analyze computational programs for expected
rounding errors and to find an arrangement of the computations such that the effect of rounding
errors is as small as possible.
As mentioned, the arithmetic in a computer is not exact and causes further errors; however,
these will not be relevant to our discussion.
Tuesday, April 21,
2020
12
 Loss of Significant Digits

 This means that a result of a calculation has fewer correct digits

than the numbers from which it was obtained. This happens if we
subtract two numbers of about the same size,
 for example,
 0.1439-0.1426 (“subtractive cancellation”).
 It may occur in simple problems, but it can be avoided in most
cases by simple changes of the algorithm—if one is aware of it! Let
us illustrate this with the following basic problem.

Tuesday, April 21,

2020
 EXAMPLE2 Quadratic Equation. Loss of Significant Digits
13  Find the roots of the equation
𝑥 2 + 40 𝑥 + 2 = 0,
 using 4 significant digits (abbreviated 4S) in the computation.
 Solution. A formula for the roots x1,x2 of a quadratic equation 𝑎𝑥 2 + 𝑏 𝑥 + 𝑐 = 0 is
1 1
 (4) 𝑥1 = (−b + 𝑏 2 − 4𝑎𝑐), 𝑥2 = (−b − 𝑏 2 − 4𝑎𝑐).
2𝑎 2𝑎

 Furthermore, since another formula for those roots

𝑐
 (5) 𝑥1 = , 𝑥2 𝑎𝑠 𝑖𝑛 4 .
𝑎𝑥2

 We see that this avoids cancellation in for positive b

 If b<0, calculate x1 from (4) and then x2=c/(ax1).

 For 𝑥 2 + 40 𝑥 + 2 = 0 we obtain from (4) 𝑥 = −20 ± 398 = −20 ± 19.95, hence x2=-20.00-19.95, involving
no difficulty, and x1=-20.00+19.95 = -0.05, poor value involving loss of digits by subtractive cancellation.
 In contrast, (5) gives x1=20.00/(-39.95)=-0.050066, the absolute value of the error being less than one
 unit of the last digit, as a computation with more digits shows. The 10S-value is -0.05006265674

Tuesday, April 21, 2020