Numerical Methods

Introduction
 Discussion
• What are Numerical Methods?
• Why do we need them?
• Numerical Methods: Algorithms that are used
to obtain numerical solutions of a
mathematical problem.
• Why do we need them?
– No analytical solution exists,
– An analytical solution is difficult to obtain or not
practical.
 Common Engineering Problems
• Some engineering problems are very difficult
to solve for complex cases.
• Modern engineering processes involve
complex cases
– Fluid mechanics,
– Circuit design
– Structural design
 Benefits of Numerical Methods
• Cost savings by minimizing material usage.
• Increased speed to market through reduced
product development time.
• Optimized structural performance with
thorough analysis.
• Eliminate expensive trial-and-error.
Features of Good Numerical Methods
• FAST
– Problems should be reasonable amount of time
• ACCURATE
– Solutions should be representative enough to the
Physical Problem.
 What Can’t computers do?
• Solve (by reasoning) general mathematical
problems
– they can only repetitively apply arithmetic
primitives to input.
• Solve problems exactly.
• Represent all numbers.
– Only a finite subset of the numbers between 0
and 1 can be represented.
 Solution Approach for Engineering
Problems
• How do you solve an
engineering problem
with computers on a
real engineering
cycle?
 A sample engineering problem
• Newton’s 2nd law of Motion
– “The time rate change of momentum of a
body is equal to the resulting force acting
on it.”
• Formulated as F = m.a
– F = net force acting on the body
– m = mass of the object (kg)
– a = its acceleration (m/s2)
• Some complex models may require more
sophisticated mathematical techniques
than simple algebra
– Example, modeling of a falling parachutist:
• 𝐹𝑈 = Force due to air resistance = -cv (c = drag
coefficient)
• 𝐹𝐷 = Force due to gravity = mg
 Write the governing equations
• Newton’s 2nd law:

• Substituting:

• or simply:

• This is a differential equation and the solution

is given by:
 Numerical Problem Statement:
• Now consider a parachutist of mass 68.1 kg.
Compute the velocity of the parachutist for
the first 15 seconds Using MATLAB. The drag
coefficient is equal to 12.5 kg/s.
• Plot v vs.. t.

• Solve the problem with MATLAB as a in-class

activity
 Exact vs. approximate
• The equation we have developed is called an
exact solution of the problem.
• Numerical methods are approximate
solutions.
– They require a reformulation of the problem.
 Reformulate the problem
• Use approximate slope

• This is approximation
is true when:

• This approach is called

finite difference
approximation
 Finite difference formulation
• Use finite difference formulation into out
problem.

• If we rearrange the equation:

• v(t) is velocity at time t and v(t+1) is the

velocıty at time t+Δt
• This method is called Euler Method and can
be defined as:
– New value = old value + slope X step size
 In-class activity
• Problem Statement. : Perform the same
computation with the previous example but
using Euler formulation that we have
developed to compute the velocity with
MATLAB. Employ a step size of 2 s for the
calculation.

• Plot exact and the numerical solution on the

same plot.
Exact vs. Numerical solutions.
 Conservation Laws:
• Most Engineering problems abide a
conservation law.
• Newton’s laws are just parts of these
problems
• Solution of these problems often require
numerical solutions.
 Problem sets
• Nonlinear versus linear. Much of classical engineering depends on linearization to
permit analytical solutions. Although this is often appropriate, expanded insight
can often be gained if nonlinear problems are examined.
• Large versus small systems. Without a computer, it is often not feasible to
examine systems with over three interacting components. With computers and
numerical methods, more realistic multicomponent systems can be examined.
• Non-ideal versus ideal. Idealized laws abound in engineering. Often there are non-
idealized alternatives that are more realistic but more computationally demanding.
Approximate numerical approaches can facilitate the application of these non-
ideal relationships.
• Sensitivity analysis. Because they are so involved, many manual calculations
require a great deal of time and effort for successful implementation. This
sometimes discourages the analyst from implementing the multiple computations
that are necessary to examine how a system responds under different conditions.
Such sensitivity analyses are facilitated when numerical methods allow the
computer to assume the computational burden.
• Design. It is often a straightforward proposition to determine the performance of a
system as a function of its parameters. It is usually more difficult to solve the
inverse problem—that is, determining the parameters when the required
performance is specified. Numerical methods and computers often permit this
task to be implemented in an efficient manner.
 Chapter 3: Approximation and errors
• All numerical methods deals with
approximations.
• In this chapter we will discuss approximation
accuracy and errors.
• This is by far most important chapter of all!
 Significant figures:
• Whenever we employ a number in a
computation, we must have assurance that it
can be used with confidence.
• What speed is the can in the figure travelling?
 Significant figures
• The speedometer says that speed of the car is between
48 to 49 km/h
• It is not possible to say that speed of the car is
48.8642138 km/h
• We can say that the car has travelled slightly less than
87,324.5 km during its lifetime.
• The last digits of the two mesurements are not certain.
• Every measument is valid for a certain number of
significant digits.
• So do the computers!
– They don’t provide infinite number of significant numbers.
 Significant digits
• Following numbers has four significant digits:
– 0,0185
– 0,000000185

• Sometimes number of significant digits are

confusing.
– 45600 may have between 3 to 5 significant digits.
• Use scientific notation to remove confusion
– 4.56e4, 4.5600e4
 Accuracy and precision
• The errors associated with both calculations and
measurements can be characterized with regard
to their accuracy and precision.

• Accuracy refers to how closely a computed or

measured value agrees with the true value.

• Precision refers to how closely individual

computed or measured values agree with each
other.
 Accuracy vs precision
• the concepts of
accuracy and
precision.
(a) Inaccurate and
imprecise;
(b) accurate and
imprecise;
(c) inaccurate and
precise;
(d) accurate and
precise.
 Error definitions
• The error of an approximation can be described as:

• 𝐸𝑡 is the exact value of error.

• Sometimes relative error is more useful

• where 𝜀𝑡 designates the true percent relative error.

• If we don’t know the exact solution

• where 𝜀𝑎 designates approximation error. Also defined as:

 In-class activity
• Maclaurin series of the exponential function is
given as:

• Starting from 1st order approximation calculate

𝑒 0.5 and calculate the first 6 approximations.
• Calculate true percent relative error for each
approximation
• Calculate approximation error
Solution
 Round off errors
• As mentioned before computers has limited
capacity for significant figures.
– Base-2 representation of fixed length
• Errors originated from this fixed length is
called round-off error.
How binary base work?
 Signed int representation
• First bit is sign,
• The rest is number body
 Floating point representation
• Signed exponent part
• Body part
 Conclusions
1. There Is a Limited Range of Quantities That
May Be Represented.
– The largest unsigned short integer is 512
2. There Are Only a Finite Number of Quantities
That Can Be Represented within the Range
– The degree of precision is limited. You cannot
store all digits of π
Chapter 4: Truncation Errors and Taylor
Series
• In the finite difference approximation example we have assumed
that for the small setup the function changes linearly.
– Called a first order approximation
– Is true for very small steps.
• If we extend our approximation method, we can find the value of
any function ay any point with Taylor series approximation.

– If the value of the function at xi is known

– n known as the order of approximation
– Rn is the remainder. Defined as

– Rn can be interpreted as error of approximation

 Taylor Series Approximation of a
Polynomial
• Compute the first four Taylor
series approximation of

Order xi+1 Error

0 1.2 -1.0

1 0.95 -0.75

2 0.45 -0.25

4 0.2 0
 Taylor series of polynomials
• nth order Taylor series gives the exact solution
of nth order polynomial.
• In real world problems, however, it is not
possible to determine the error. (Rn)
– If we knew the error we would fix it!!!
• But remainder is still useful. From the Rn-1 we
know how accurate our approximation is.
– Provided that Rn < Rn-1
 Remainder explained:
• Here is the graphical representation of the
zero order approximations.
 Approximation order vs. step size
• It looks like when we increase the
order of approximation, we will have
better results.
• The other method can be defined as
decreasing step size dx = (xi+1 –xi).
• Then slope approximations become
closer to the limit case.
• Consider the function f(x)=x4.
• It is obvious that if we calculate fourth
order Taylor series, we get the exact
result.
• Instead, write Taylor series of first
order, and calculate for different
number of steps.
 First order approximation
• Since slope variation is small, error of first order
approximation is also small.

• That yields for the first order approximation:

• And the remainder is

 Numerical stability and the error
propagation.
• Each calculation at the previous example
propagates its error to the next point.
– When you have calculations involving many steps,
the last result will consist of the cumulative error
of the previous steps.
• Sometimes, Taylor Series expansion produces
too big residuals and residuals keeps growing
during the iterations.
– We call this instability!
 Total Numerical Error
• We have learned round-off errors before –they
are hardware generated
• Truncation error due to the Taylor series
remainder- they are numerical method
generated.
• Total error is the sum of these two.
• If you make too little calculations with a your
error becomes large: truncation error becomes
large
• If you make too much calculations, round-off
errors becomes significant.