MIDDLE EAST TECHNICAL UNIVERSITY

FACULTY OF ENGINEERING
Department of Engineering Sciences

Course Code and Title:

ES 510 Numerical Solution of Ordinary Differential Equations (3-0) 3

Numerical Methods in Engineering

Numerical techniques of solving scientific and engineering problems has become an essential part
of training of applied mathematicians, engineers and scientists. One reason for this is that
numerical methods can give the solution when ordinary analytical methods fail, as for example in
finding the roots of transcendental equations, or in solving nonlinear differential equations.
Numerical methods have almost unlimited breadth of applications.

A second and perhaps more important aspect is their interrelation with digital computers. The price
one pays for the general applicability of numerical scheme is arithmetic complexity. Since digital
computers provide a nearly effortless way to perform the simple but long and tedious computations
involved in problem solving by numerical methods, the advent of these marvelous servants has
revived the interest in numerical methods.

The numbers we use today called “Arabic numbers” though Arabs call them “Indian numbers”.
These numbers are introduced to Europe by Al-Harezmi. In 773 Al-Me’mun The Khalaf of Abbasi
had a visitor named “Siddhanta” in his palace who brought a book which was having the decimal
system by which it was possible to express any number. This system was taught in University of
Kurtaba which is taken by Pope II Sylvester after two hundred years and spread in whole Europe.
In 1180, Fibonacci introduced this system to Sicily. Hence the Indian numbers are 9,8,7,6,5,4,3,2,1
and zero is introduced to this system by Arabs. Zero “0” means emptiness, “sunya” in Indian. With
these numbers arithmetic had a revolution in Europe. With this system, it became possible to
express 444 with three characters instead of MMMMCCCCXLIV. It is not difficult to think the
difficulty of carrying out arithmetic with Roman numbers.

Engineering problem solving process consists of the following steps:

1. Problem definition: selection of decision variables, specifying constraints
2. Mathematical model: Using physical laws or experimental data, mathematical
relationship among the decision variables are obtained.
3. Use of an algorithm to solve the mathematical model: Analytical, graphical or numerical
techniques are used to obtain the values of decision variables.
4. Validation of results.
Solutions of mathematical models may be obtained by standard mathematical techniques, if
models are simple enough. Even in such models, however, determination of solutions may require
a large amount of calculations. Computers are ideal tools for such tasks because they perform
calculations very quickly without numerical mistakes.

If solutions of mathematical models can be obtained by analytical methods, then the solution will
be exact. Unfortunately, very few practical systems lead to analytical solutions, hence analytical
solutions are of limited use. In certain special types of problem, graphical solutions can be found
1
useful to study the behavior of the system. However graphical solutions are less accurate, awkward
to use and can only be used if dimensionality of the problem is less than or equal to three and
require more time.

Numerical solutions are those that cannot be expressed in form of mathematical expressions. They
can be found using a suitable type of calculation intensive process known as numerical methods.

What is numerical analysis?

Outside the realm of pure mathematics, most practicing scientists and engineers are not concerned
with finding exact answers to problems. Indeed, living in a finite universe, we have no way of
exactly measuring physical quantities and even if we did’ the exact answer would not be of much
use. Just a single irrational number such as

𝜋 = 3.14159265358979384626433832795028841971693993751058209749 … ….

where the digits keep going on forever without repetition or any known pattern, has more
information in its digits than all computers in the world could possibly ever store.

Example 1: Suppose one of the municipalities is interested in finding out the amount of water
contained in one of its reserve drinking water reservoirs. It hires a contracting firm to measure this
amount. The firm begins with a large-scale pumping device to take care of most of the water,
leaving only few liters. After this, they bring out a more precise device to measure the remainder
and come out with a volume of 12 564 832.42 liters. To get a second opinion, municipality hires
a more sophisticated engineering firm (that charges 10 times as much) and that uses more advanced
measuring devices. Suppose this latter firm came up with the figure 12 564 832.3182 liters. Was
the first estimate incorrect? Maybe not, perhaps some evaporation or spilling took place-so there
cannot really be an exact answer to this problem. Was it really worth the extra cost to get this more
accurate estimate? Most likely not- even an estimate to nearest liters would have served equally
well for just about any practical purposes.

Example 2: Suppose next that the Boeing Corporation, in the design and construction of a new
767 model jumbo jet, needs some wing rivets. The engineers have determined the rivets should
have diameter of 2.75mm with a tolerance (for error) of 0.000025mm. Boeing owns a precise
machine that will cut such rivets to be of diameter 2.75±0.000006mm. But they can purchase a
much more expensive machine that will produce rivets of diameters 2.75±0.000001mm (60 times
as accurate). Is it worth it for Boeing to purchase and use this more expensive machine? The
aeronautical engineers have determined that such an improvement in rivets would not result in any
significant difference in the safety and reliability of the wing and plane; however, if the error
exceeds the given tolerance, the wings may become unstable and a safety hazard.

In mathematics, there are many problems and equations (algebraic, differential and partial
differential) whose exact solutions are known to exist but they are difficult, very time consuming,
or impossible to solve exactly. But for many practical purposes, as evidenced by above examples,
an estimate to exact answer will do just fine, provided that we have a guarantee that error is not
too large. So here is the basic problem in numerical analysis: We are interested in a solution x
(exact answer) to a problem or equation. We would like to find an estimate x * (approximation)

2
so that |𝒙 − 𝒙∗ | (the actual error) is no more than the maximum tolerance ( ε ). i.e, |𝒙 − 𝒙∗ | ≤
𝜺.
WHY DO WE NEED NUMERICAL METHODS?

Mathematical formulation of engineering we end up single or set of linear or nonlinear algebraic

equations, a set of transcendental equations, a set of ordinary or partial differential equations, a
set of homogenous equations leading to an eigenvalue problem, or an equation involving integrals
or derivatives. We may or may not be able to find the solution of governing equation analytically.
If a solution can be represented in a form of mathematical expression, it is called analytical
solution. Analytical solutions represent exact solutions that can be used to study the behavior of
the system with varying parameters. Unfortunately, very few practical systems lead to analytical
solutions. Therefore, analytical solutions have limited use. Let us see some examples related with
this matter.

1) Solution of single variable nonlinear equations with fourth degree and above cannot be
obtained analytically. There are no methods available.

Analytical solution

No analytical solution

2) Simultaneous liner equations. There are several methods to find the solution of such systems.
They are feasible to use when the number of equations is small. It is possible to work out
manually with these methods. But when the number of equations become more than ten, it
becomes difficult to apply the methods manually. Imagine a case where there are 100000
equations to be solved simultaneously. In such cases it becomes impossible to use these
methods due to large number of arithmetic calculations.

Analytic solution

n=100, analytical methods

become impractical to apply.

3
 3) Evaluation of integrals. Let us consider the following integration.

The value of this integral can be expressed analytically as

On the other hand, the integral

Does not have a closed -form (analytical) solution. This integral can only be evaluated numerically.
Since the integral is the same as the area under the curve 𝑓(𝑥), its value can be estimated by
breaking the area under the curve into small rectangles and adding the areas of rectangles as shown
in the figure below.

4
 Examples of some engineering problems

1. The following equation pertains to the concentration of a chemical in completely mixed

reactor.
 
c  cin 1  e  0.04t  c0 e  0.04t

If the initial concentration c0  4 and inflow concentration cin  10 , compute the time
required for c to be 93% of cin .

Solution: 𝑐𝑖𝑛 = 10 , 𝑐 = 0.93𝑐𝑖𝑛 = 9.3

9.3 = 10(1 − 𝑒 −0.04𝑡 ) − 4𝑒 −0.04𝑡

9.3 = 10 − 10𝑒 −0.04𝑡 − 4𝑒 −0.04𝑡

which can be simplifies to 0.7 − 14𝑒 −0.04𝑡 = 0

𝑒 −0.04𝑡 = 0.05
Taking natural logarithm of both sides -0.04tln(e)=ℓn (0.05) from which t=74.89

Solution of the problem is obtained by analytical method.

2. The volume of liquid in a hollow horizontal cylinder of radius r and length ℓ is related to
the depth of the liquid h by

 r h  2 
V  r 2 cos 1     r  h 2rh  h  
  r   

Determine h for given r=2m, ℓ=5m and V=8m3.

𝑟 = 2𝑚 , ℓ = 5𝑚 , 𝑉 = 8𝑚3
5
 2−ℎ
8 = [22 𝑐𝑜𝑠 −1 ( ) − (2 − ℎ√2 × 2 − ℎ2 )] × 5
2

which can be simplified to 4𝑐𝑜𝑠 −1 (1 − 0.5ℎ) + ℎ√4ℎ − ℎ2 −3.6=0

we ended up nonlinear single variable equation solution of which cannot be obtained by

analytical method.

3. The concentration of pollutant bacteria c in a lake decreases according to

𝑐 = 70𝑒 −1.5𝑡 + 25𝑒 −0.075𝑡

Determine the time required for the bacteria concentration to be reduced to 9.

9 = 70𝑒 −1.5𝑡 + 25𝑒 −0.075𝑡 which can be simplified to the following nonlinear equation

𝑒 −1.5𝑡 + 0.3571𝑒 −0.075𝑡 − 0.12857 = 0 solution of which cannot be determined by

analytical methods.

4. In ocean engineering, the equation for a reflected standing wave in a harbor is given by

2𝜋𝑥 2𝜋𝑡 𝜐
ℎ = ℎ0 [𝑠𝑖𝑛 ( ) 𝑐𝑜𝑠 ( ) + 𝑒 −𝑥 ]
𝜆 𝜆

Solve for the lowest positive value of x if h  0.4h0 . Take λ=16, t=12, v =48.

2𝜋 2𝜋 × 12 × 48
0.4ℎ0 = ℎ0 [𝑠𝑖𝑛 ( ) 𝑥 𝑐𝑜𝑠 ( ) 𝑥] + 𝑒 −𝑥
16 16

𝑠𝑖𝑛(0.3927𝑥) cos(96𝜋) + 𝑒 −𝑥 − 0.4 = 0

cos(96𝜋) = 1 , 𝑠𝑖𝑛(0.3927𝑥) + 𝑒 −𝑥 − 0.4 = 0,

Once again, we ended up nonlinear single variable equation solution of which cannot be
determined by analytical methods.

Numerical methods are techniques to approximate mathematical procedures (an example of a

mathematical procedure is an integral). Approximations are needed because we either cannot solve
the procedure analytically or because the analytical method is intractable. Numerical methods

6
involve a large number of arithmetic calculations, they require use of computers and their
application is closely related with the power of these instruments.

 Numerical methods can be used to find the solutions of even complex engineering
problems while the analytical solutions usually require several simplifying assumptions
of the physical system. Numerical solutions do not require such assumptions.
 Although numerical solutions cannot provide an immediate insight into the behavior of
the simplified physical system, they can be used to study the behavior of the true physical
system.

Approximations and Errors

If x is an approximation to x , the difference between the true value and the approximate value is
called true error E x is defined as E x  x  x .

xx
The relative error, R x is defined as R x  , x  0.
x

A number x is considered to be approximation to the true value x to d significant digits, if d is the

xx 1
largest positive integer for which   10  d
x 2

Example: The number π=3.1415927 is approximated as 3.1416. Find the following: (a) true error,
(b) relative error (c) number of significant digits of approximation.

(a) The true error E x  x  x  3.1415927  3.1416  0.0000073

xx 0.0000073
(b) The relative error R x    0.0000023237
x 3.1415927
(c) The relative error can be expressed as

10 5
R x  0.000002327   0.00005 hence x approximates x to five significant digits.
2

The relative error is usually more meaningful measure. Consider the following table.

True error Relative Error

x x
Ex Rx
1 0.99 0.01 0.01
1 1.01 0.01 0.01
-1.5 -1.20 0.3 0.2
100 99.99 0.01 0.0001
100 99 1 0.01

7
Thus when x  1 there is not much difference between absolute (true) and relative errors, but
when x  1 the relative error is more meaningful.

TYPES OF ERRORS

Numerical techniques yield estimates that are close to exact solutions. The difference between
the result obtained using numerical technique and the exact analytical solution is called ERROR.
There are two types of errors. These are truncation errors and round-off errors.

TRUNCATION ERRORS: This is the error which occurs due to the approximation carried out
in the numerical method.

Example: Consider the computation of the derivative of y  x 3 at x0  1.2 .

Analytical (exact) solution:

dy  dy 
 3x 2 at x0  1.2     3  1.2 2  4.32
dx  dx  x 0 1.2

Numerical (approximate) solution:

dy dy CD dy y1  y 0
 tan  , approximate  tan   .  select  x  0.01
dx dx BD dx x
8
  
x0  1.2  y 0  1.2 3  1.728

x1  1.2  0.01  1.21  y1  1.213  1.7716

dy 1.7716  1.728
  4.356 (Approximate)
dx 0.01

Error= 4.32-4.356= -0.036

This error is called truncation error. Because the derivative is calculated using an approximate
relationship.

Truncation error also takes place when approximate expression is used in the place of an exact
mathematical expression. Let us consider Taylor’s series expansion of a function.

Taylor series provide a means to predict a function value at one point in terms of the function
values at another point. In particular the theorem states that any smooth function can be
approximated as a polynomial.

Taylor’s theorem: If a function f and its first n+1 derivative are continuous on an interval
containing a and x, then the value of function at x is given by

f (a) f (a) f n (a)

f ( x)  f (a)  f (a)( x  a)  ( x  a) 2  ( x  a) 3    ( x  a) n  Rn
2! 3! n!

where Rn is remainder. Depending upon number of terms considered in the series an

approximate value can be computed for f (x) .

Let us expand ln(1  x ) at a=0 using Taylor’s series.

f (0)  ln(1)  0

1
f ( x)  , f (0)  1
1 x

1
f ( x)   , f (0)  1
1  x 2
2
f ( x)  , f (0)  2
1  x 3
6
f ( x)   , f (0)  6
1  x 4
9
 24
f ( x)  , f (0)  24
1  x 5
120
f ( x)   , f (0)  120
1  x 6
Hence

1 2 2 3 6 4 24 5 120 6
f ( x)  ln(1  x)  0  1  x  x  x  x  x  x 
2 3! 4! 5! 6!

ln(1  x)  x 
1 2 1 3 1 4 1 5 1 6
x  x  x  x  x  or

ln(1  x)  
 1i 1 x i
2 3 4 5 6 i 1
i

For simplicity, let f (x) be approximated by the first four terms of Taylor’s series approximation.
The resulting discrepancy between the exact function f (x) and the approximate function
1 1 1
f * ( x)  x  x 2  x 3  x 4 is called truncation error.
2 3 4

Let us consider a representative value of x  0.5 and find the truncation error for taking 1, 2, 3
and 4 terms of Taylor’s series expansion for the approximate function.

The exact value= f ( x)  ln(1  x)  ln(1.5)  0.405465

a) Taking 1 term:
0.405465  0.5
f * ( x)  x , f * (o.5)  0.5 , E x  0.455465  0.5  0.09435 , R x   0.23315 ,  23.3%
0.405465
b) Taking 2 terms:

x2 0.030465
f * ( x)  x  , f * (o.5)  0.375 , E x  0.455465  0.375  0.030465 , R x   0.075136 , 7.5%
2 0.405465

c) Taking 3 terms:

x 2 x3 *  0.011202
f * ( x)  x   , f (o.5)  0.416667 , E x  0.455465  0.416667  0.011202 , R x   0.027627 ,  2.8%
2 3 0.405465
d) Taking 4 terms:

x 2 x3 x 4 0.004423
f * ( x)  x    , f * (o.5)  0.401042 , E x  0.455465  0.401042  0.004423 , R x   0.010908 , 1.1%
2 3 4 0.405465

10
 APPROXIMATING e x

The value of ex can be determined as

x3 x3 x 4 x5 x6
ex  1 x       
2! 3! 4! 5! 6!

By considering a different number of terms ( one term at a time) ranging from 1 to 5 on the right
hand side of the above expansion determine the value of e. Compare the results with the exact
value of e=2.71828183, and find the absolute and relative error in each case. Write a MATLAB
program to do the same.

The following MATLAB program is prepared to carry out what is asked in the question.

n=input('enter the number of terms=');

x=input('enter the value of x=');
expx=0;
ex=exp(x);
disp([' # of terms expx error'])
for i=1:n;
k=i-1;
expx=expx+(x^k)/factorial(k);
error=abs((ex-expx)/ex);
disp([i,expx,error])
end

OUTPUT:

enter the number of terms=5

enter the value of x=0.5
# of terms expx error
1.0000 1.0000 0.3935
2.0000 1.5000 0.0902
3.0000 1.6250 0.0144
4.0000 1.6458 0.0018
5.0000 1.6484 0.0002

11
ROUND-OFF ERRORS: These errors take places due to the finite representation of real (and
complex) numbers in digital computers. Since only a finite number of digits are stored in computer,
the actual numbers may undergo chopping or rounding of the last digit.

A finite approximation to the real number system in which numbers are represented in a form

𝑥 = ±0. 𝑑1 𝑑2 𝑑3 ⋯ 𝑑𝑘 × 𝑏 𝑒
where
𝑑1 ≠ 0 floating-point numbers are normalized.
k is called the precision (number of significant digits).
b is the base.
E is the exponent and lies in some range 𝑚 ≤ 𝑒 ≤ 𝑀.

For example

8 2 6 6 6 6
= 2.66666 ⋯ = ( 1 + 2 + 3 + 4 + 5 + ⋯ ) × 101
3 10 10 10 10 10

Where the base is ten. As seen, this is an infinite series, but computer use a finite amount of
memory to represent real numbers. Thus, only a finite number of digits may be used to represent
any number, no matter by what representation method. For instance we can chop the infinite
decimal representation of 8/3 after k=4 digits,

8 2 6 6 6
≅ ( 1 + 2 + 3 + 4 ) × 101 ≅ 0.2666 × 101
3 10 10 10 10

If the maximum number of decimal digits used in the floating-point computation is k=5, then the
chopped floating-point representation of x, xchop is given by

𝑥𝑐ℎ𝑜𝑝 = ±0. 𝑑1 𝑑2 𝑑3 𝑑4 × 𝑏 𝑒

Where the kth digit if 𝑥𝑐ℎ𝑜𝑝 is identical to the kth digit of x. on the other hand, the rounded floating-
point representation of x, 𝑥𝑟𝑜𝑢𝑛𝑑 is given by

𝑥𝑟𝑜𝑢𝑛𝑑 = ±0. 𝑑1 𝑑2 𝑑3 ⋯ 𝑑𝑘−1 𝑐𝑘 × 𝑏 𝑒

Where 𝑐𝑘 , (1 ≤ 𝑐𝑘 ≤ 9) is obtained by rounding the number 𝑑𝑘 𝑑𝑘+1 𝑑𝑘+2 ⋯ to the nearest

integer. For example

𝑥 = 𝜋 = 3.14159265 ⋯

Let k=4 and b=10.

With rounding 𝑥𝑟𝑜𝑢𝑛𝑑 = +0.3142 × 101

With chopping 𝑥𝑐ℎ𝑜𝑝 = +0.3141 × 101

12
 𝑥 = 𝑒 = 2.718281828459045 ⋯

Considering k=7 (seven significant digits)

𝑥𝑟𝑜𝑢𝑛𝑑 = +0.2718282 × 101

𝑥𝑐ℎ𝑜𝑝 = +0.2718281 × 101

The numerical solution of an engineering problem is found using a suitable algorithm. An

algorithm is a finite set of precise instructions to be carried out using the given initial data in a
specified sequence in order to find the desired output. All local computational errors involved in
the various steps of algorithm will accumulate to a computational error in the output. The local
computational errors arise due to errors involved during arithmetic operations such as subtraction
of numbers of near equal magnitude.

ERROR ASSOCIATED WITH ARITHMETIC OPERATIONS

When two numbers are used in arithmetic operation, the numbers cannot be stored exactly by the
floating-point representation. Let x and y be exact numbers and 𝑥̅ and 𝑦̅ their approximate values.
Then 𝑥 = 𝑥̅ + 𝜀𝑥 and 𝑦 = 𝑦̅ + 𝜀𝑦 where 𝜀𝑥 and 𝜀𝑦 denotes the errors in x and y respectively. For
example, when a multiplication operation is used, the associated error E is given by

𝐸 = 𝑥𝑦 − 𝑥̅ 𝑦̅ = 𝑥𝑦 − (𝑥 − 𝜀𝑥 )(𝑦 − 𝜀𝑦 ) = 𝑥𝜀𝑦 + 𝑦𝜀𝑥 − 𝜀𝑥 𝜀𝑦

The relative error R is given

𝐸 𝜀𝑥 𝜀𝑦 𝜀𝑥 𝜀𝑦
𝑅= = + − = 𝑅𝑥 + 𝑅𝑦 − 𝑅𝑥 𝑅𝑦 ≈ 𝑅𝑥 + 𝑅𝑦
𝑥𝑦 𝑥 𝑦 𝑥 𝑦

Where |𝑅𝑥 | ≪ 1 and |𝑅𝑦 | ≪ 1 with 𝑅𝑥 𝑎𝑛𝑑 𝑅𝑦 denoting the relative errors in x and y respectively
and the symbol ≪ representing “much less than”.

Example 1: Perform each of following arithmetic operations. a) Exact computation, b) four-digit

approximation (chopping) and c) four digit approximation (rounding off). Determine
the absolute and relative errors involved in parts (b) and (c).

1 1 1 1 1 1 1 1
 ,  ,  , 
3 7 3 7 3 7 3 7

13
Solution:

14
Example 2: Consider equations 31.69 x  14.31y  45.00
13.11x  5.89 y  19.00 . The unique solution to this set of
equations is x=7.2 and y=-12.8. Using simple elimination method, one would obtain
the following expressions.
5.89  31.69  14.31 13.11 x  5.89  45.00  14.31 19.00
13.11 14.31  31.69  5.89 y  13.11  45.00  31.69  19.00
a) Perform these operations using four-digit (chopping) arithmetic and use your answer
to find four-digit values of x and y.
b) Explain why the answers obtained in (a) differ considerably from actual values for
x and y. Compute the relative true error for each unknown.

Solution (a):

[5.89 × 31.69 − 14.31 × 13.11]𝑥 = 5.89 × 45.00 − 14.31 × 19.00

[186.6 − 187.6]𝑥 = 265.0 − 271.8

−𝑥 = −6.800 𝑥 = 6.800

[13.11 × 14.31 − 31.69 × 5.89]𝑦 = 13.11 × 45.00 − 31.69 × 19.00

[187.6 − 186.6]𝑥 = 589.9 − 602.1

𝑦 = −12.20 𝑦 = −12.20

(b): The comparison between the exact results and the 4-digits chopping results reveals the
following fact illustrated in the table.

Computations Exact 4-digit True error Relative error.

result chopping
results
5.89 × 31.69 −0.95 + 1
-0.95 -1.000 |−0.95 + 1| = 0.05 | | = 0.053
− 14.31 × 13.11 0.95
5.89 × 45.00
−6.84 + 6.8
− 14.31 × 19.00 -6.84 -6.800 |−6.84 + 6.8| = 0.04 | | = 0.0058
6.84

13.11 × 14.31 0.95 − 1

0.95 1 |0.95 − 1| = 0.05 | | = 0.053
− 31.69 × 5.89 0.95
13.11 × 45.00 −12.16 + 12.2
-12.16 -12.20 |−12.16 + 12.2| = 0.05 | | = 0.0033
− 31.69 × 19.00 12.165

15