Numerical

Analysis

1
 Rules in class

• Stop me when you do not understand

• Ask me when you have questions
• Class interactivies – be active: speak and discuss.
• Drop the class if your mid-exam result is E.
• No more than 3 times of absent.
• No taking photographs – make notes!

Reference book:
• Steven C. Chapra, Raymond P. Canale,
“Numerical Methods for Engineers, 6th edition”,
McGraw-Hill.
2
 Learning outcomes

• Able to understand and use numerical methods to

solve equations and engineering problems
• Able to use software packages in numerical methods.

3
 Introduction
Why study Numerical Analysis?

4
 Introduction

Why are Numerical Methods so widely used in Engineering?

 Engineers use mathematical modeling (equations and data)

to describe and predict the behavior of systems.
 Closed-form (analytical) solutions are only possible and
complete for simple problems (geometry, properties, etc.).
 Computers are widely available, powerful, and (relatively)
cheap.
 Powerful software packages are available (special or
general purpose).

5
 Introduction
A few applications of Numerical Methods in Engineering:

• Structural/mechanical analysis, design, and behavior.

• Communication/power
Network simulation
Train and traffic networks

• Computational Fluid Dynamics (CFD):

Weather prediction
Groundwater & pollutant movement

6
Introduction

7
Introduction

8
My research: CFD

9
My research: CFD

10
Next: What is yours?

11
 Part 1

Introduction

12
 Introduction
Why study Numerical Methods?
Numerical Analysis is a Discipline:
 Need to understand concepts and theory
- Know what problems can be solved.
- Know what problems cannot be solved, or when
problems will be troublesome.
 Need to understand methods and techniques
- Know why methods work, or judge when they are
working.
- Be able to create or modify tools (software) as
needed.
- Evaluate errors, convergence, and stability of
arithmetic approximations.
13
 Introduction

Why study Numerical Methods? (continued)

Use of Numerical Methods is an Art:

• Numerical methods are approximate.
• The most appropriate method(s) is not always
obvious.
• Evaluating precision and accuracy is an essential
part of the process.

14
 Introduction

• Numerical analysis is a part of mathematics, but it

works on questions that are strongly related to the use
of computers and to applications from Science and
Engineering.

• Using numerical analysis we will be able, for instance,

to handle large systems of equations, non-linearities,
complicated geometries and solving engineering
problems which have no analytical solution.

15
 Outlines of the Course

• Taylor Theorem • Solution of linear

• Number Representation Equations
• Solution of nonlinear • Least Squares curve
Equations (roots of eqs) fitting
• Interpolation • Solution of ordinary
• Numerical Differentiation differential equations
• Numerical Integration • Introduction of Partial
differential equations

16
 Introduction
• Roots of equations :
– We will be interested in
x 0 \ f ( x 0 )  0.
methods for solving

– These methods are very

useful in engineering
projects, because in many
occasions it is not possible
to solve the design
equations analytically.

17
 Introduction

• Systems of linear equations:

a11x1  a12x 2  b1
a 21x1  a 22x 2  b 2
– We will study methods for
computing the set of values
that simultaneously satisfy
a system of algebraic
equations.

– Applications: calculus of
structures, electric circuits,
supply networks, fit of
curves, etc.

18
 Introduction
• Optimization:

– Determine the value x0

leading to the optimal
value of f(x).

– These problems can be

subject to constraints.

19
 Introduction
• Fitting curves. Fitting techniques can be divided into two groups:

– Regression. It is used when one has errors in the experimental

data. One looks for the curve showing the trend of the data.

– Interpolation. It is used to fit tabulated data and predict

intermediate values or extrapolated data.

20
 Introduction
• Integration:
b
I   f ( x )dx
a
– Determine the area
f(x)
below a given curve.

– It has many applications Integral

in engineering. Calculus
of centers of gravity.
Calculus of areas,
volumes, etc.

a b x
– It can also be used to
solve differential
equations.

21
 Introduction
• Ordinary differential
equations :

dy
= f ( t; y)
dt

– They are very important

because many problems
can be stated in terms of
variations and not in terms
of magnitudes.

– There are two types of

problems: Initial value
problems, and boundary
value problems.

22
 Introduction
• Partial differential
equations:

 2u  2u
  f ( x, y )
2 2
x y
y
– Used for characterizing
engineering problems
where the behavior of the
physical magnitude can
be expressed in terms of
speed change with
respect to two or more
variables.

– Approximation by finite x
differences or the finite
element method.

23
 Chapter 1

Mathematical Modeling and

Engineering Problem Solving

24
Engineering Problem Solving

25
 Mathematical Model

• A mathematical model is represented as a functional

relationship of the form

Dependent independent forcing

Variable =f variables, parameters, functions

• Dependent variable: Characteristic that usually reflects the

state of the system
• Independent variables: Dimensions such as time and space
along which the systems behavior is being determined
• Parameters: reflect the system’s properties or composition
• Forcing functions: external influences acting upon the system
26
 Mathematical Models
• A mathematical expression of a given model can be
F  ma; F(kgm / s 2 ) m(kg ) a(m / s2 )
FU
F dv F dv FD  FU
a ;  ;  ;
m dt m dt m

dv mg  cv c(Kg / s )
 ;
dt m
• Analytic solution (t=0, v=0):

• Approximate solution (“Euler method (forward)”):

 c 
v ( t i 1 )  v ( t i )   g  v ( t i ) ( t i 1  t i )
 m  FD

27
 Analytical solution to Newton's Second Law

28
Analytical solution to Newton's Second Law

29
Numerical Solution to Newton's Second Law

 Numerical solution: approximates the exact solution

by arithmetic operations.

 Suppose

30
 Mathematical Models
• To solve the problem numerically, one replaces the derivative by
a finite difference, thus transforming the problem into a very
simple one containing only simple algebraic operations:

31
 Numerical Solution to Newton's Second Law

32
Comparison between Analytical vs. Numerical
Solution

33
 Mathematical Models
• To solve the problem numerically, one replaces the derivative by
a finite difference, thus transforming the problem into a very
simple one containing only simple algebraic operations:

34
 Numerical Differentiation
f  x i1   f  x i 
Forward: f   xi  
h

f  x i1   f  x i1 
Centered: f   xi  
2h
 How big a step size should we select?
 One- or two-sided formula:
What are the advantages of each?
 How is optimal step size affected by:
- precision of numerical calculations?
- precision with which f is computed?
- choice of formula?

35
 Chapter 2

Programming and Software

36
• Structured Programming
• Modular Programming (module, function)
• Flowchart
• Algorithm

37
 Chapter 3

Approximations and Round-Off

Errors

38
 APPROXIMATION AND ERRORS
Numerical Methods
Instead of solving for the exact solution we solve
math problems with a series of arithmetic
operations.
b
1
Example:  x dx
a
analytical solution: ln(b) – ln(a)
numerical solution e. g., Trapezoidal Rule
Error Analysis
(a) identify the possible sources of error
(b) estimate the magnitude of the error
(c) determine how to minimize and control error
39
 Mathematical Models
• Comparing solutions:
Numerical solution, Dt=2sec
60

50

40
Exact solution
V (m/sec)

30

Numerical solution, Dt=1sec

20

10

0
0 2 4 6 8 10 12 14 16 18 20

T (sec)
40
 Approximations and Rounding Errors
• Unfortunately, computers introduce errors in the calculations.
However, since many engineering problems have not analytical
solution, we are forced to use numerical methods
(approximations). The only option we have is to accept the error
and try to reduce it up to a tolerable level.
• The only way of minimizing the errors is by knowing and
understanding why they occur and how we can diminish them.
• The most frequent errors are:
– Rounding errors, due to the fact that computers can work
only with a finite representation of numbers.
– Truncation errors, due to differences between the exact
and the approximate (numeric) formulations of the
mathematical problem being dealt with. penyederhanaan dari persamaan
yang kompleks
• Before analyzing each one of them, we will see two important
concepts on the computer representation of numbers.

41
 Approximations and Rounding Errors
• Significant figures of a number:

– The significant figures of a number are those which can be

used with confidence.
– This concept has two important implications:

1. An approximation is acceptable when it is exact for a given

number of significant figures.
2. There are magnitudes or constants that cannot be represented
exactly:
  3.14159265...
17  4.123105...

42
 Approximations and Rounding Errors

Accuracy closeness of measured/computed values to the

"true" value (vs. inaccuracy or bias)

Bias systematic deviation from truth, "general trend"

Precision closeness of measured/computed values with

each other (spread or scatter), relates to the
number of significant figures (vs. imprecision or
uncertainty)

43
 Approximations and Rounding Errors
• Accuracy and precision:
– The errors associated with
measurements can be
characterized observing
their accuracy and
precision.

– Accuracy refers to how

close the value is to the true
value.

– Precision refers to how

close are different
measured values using the
same method.
Numerical methods must be sufficiently exact (without bias) and
precise to satisfy the requirements of engineering problems. From
now on we will refer to error to refer to the inaccuracy and lack of
precision of our predictions.
44
Approximations and Rounding Errors

(a) inaccurate imprecise

(b) accurate imprecise

(c) Inaccurate precise

(d) Accurate precise

45
 Approximations and Rounding Errors
• Error definitions:
– True value = approximation + absolute error.
– Absolute error = true value - approximation .
– Relative error = absolute error / true value .
absolute error
t  100%
true value

– In real cases not always one can know the true value, thus:
aproximate error
a  100%
approximate value

– In many occasions, the error is calculated as the difference

between the previous and the actual approximations.
actual approximation  previous approximation
a  100%
actual approximation

46
Approximations and Rounding Errors
– Thus, the stopping criterium of a numerical method can be:

a  s
 s  prefixed percent tolerance

– It is convenient to relate the errors with the number of

significant figures.If the following relation holds, one can be
sure that at least n significant figures are correct.

s  (0.5 * 102n )%

47
 Approximations and Rounding Errors
• Numerical systems:
– A numerical system is a
104 103 102 101 100
convention to represent
8 6 4 0 9
quantities. Since we have
10 fingers in our hands, the 9x
0x
1=
10 =
9
0
4 x 100 = 400
most popular numerical 6 x 1000 = 6000
a) 8 x 10000 = 80000
system has basis 10. It uses 86409
10 different digits. 27 26 25 24 23 22 21 20

1 0 1 0 1 1 0 1

– However, computers, due to 1x 1= 1

0x 2= 0
the memory structure, can 1x 4= 4
1x 8= 8
only store two digits: 0 and 0 x 16 = 16
1 x 32 = 32
1. Thus, they use the binary b)
0 x 64 = 64
1 x 128 = 128
system of numeric 173

representation.

48
 Round-off Errors

Background: How are numbers stored in a computer?

• The fundamental unit, a "word," consists of a string of "bits" (binary digits).

• Because computers are made up of gates or switches which are either
closed or open, we work in binary or base 2 system.
• A number in base q will be denoted by

(anan-1...a1a0.b1b2..bk..)q

The conversion to base 10 is, by definition

(anan-1...a1a0.b1b2..bk..)q =anqn+an-1qn-1+...+a1q+a0q0+b1q-1+b2q-2+...

Example:

(1011.01)2=1x23+0x22+1x2+1x20+0x2-1+1x2-2=11.25

49
 Round-off Errors

Conversion from base 10 to base q.

This is the recipe for conversion:

Integer part: we have to divide the integer part by 2 (successively) and

to retain the fractional part in each step.

Fractional part: we have to multiply by 2 and to retain the integer part

in each step.

Example:

(26.1)10=(11010.00011)2

50
 Round-off Errors

Example: An 8 bit word representation of the integer "35" is 00100011

or
0 0 1 0 0 0 1 1
± 26 25 24 23 22 21 20
+ 0x26 1x25 0x24 0x23 0x22 1x21 1x20 = 35
32 2 1
Note: We can only represent a finite # of numbers; for our case:
–127 to +127 (127 = 27 – 1)
or a total of 255 numbers (including 0)

51
 Approximations and Rounding Errors

• Representation of integer numbers:

– To represent base 10 numbers in binary form the signed

magnitude method is used. The first digit stores the sign (0,
positive and 1, negative). The remaining bits are used to
store the number.

– A computer working with words of 16 bits can store integer

numbers in the range -32768 to 32767.

52
 Approximations and Rounding Errors
• Floating point representation:
– This representation is used for fractional quantities. It has
the fraction part, called mantissa, and an integer part, called
exponent or characteristic.
m * be

– The mantissa is usually normalized, so that the value of m is

limited (b=2 in binary):
1
m1
b

53
 Approximations and Rounding Errors
• IEEE-floating point formats: there are two types of
“precision” (simple and double). They differ in the
number of digits available for storing the numbers:
• Simple precision (32 bits): 1 bit for the sign, 8 bits
for the exponent, 23 bits for the mantissa.
• Double precision (64 bits, two words of 32 bits): 1
bit for the sign, 11 bits for the exponent, 52 bits for
the mantissa.

The number of bits for the exponent and the mantissa

determine the “underflow” and “overflow” numbers.

54
 IEEE 754 Floating-Point Standard

• Single Precision (32-bit representation)

– 1-bit Sign + 8-bit Exponent + 23-bit Fraction

S Exponent8 Fraction23

• Double Precision (64-bit representation)

– 1-bit Sign + 11-bit Exponent + 52-bit Fraction

S Exponent11 Fraction52
(continued)

55
 Significant Digits

 Significant digits are those digits that can be

used with confidence.

 Single-Precision: 7 Significant Digits

1.175494… × 10-38 to 3.402823… × 1038

 Double-Precision: 15 Significant Digits

2.2250738… × 10-308 to 1.7976931… × 10308

56
 Chopping
Example:
=3.14159265358 to be stored on a base-10 system
carrying 7 significant digits.
=3.141592 chopping error t=0.00000065
If rounded
=3.141593 t=0.00000035
• Some machines use chopping, because rounding
adds to the computational overhead. Since number
of significant figures is large enough, resulting
chopping error is negligible.

57
 Rounding and Chopping
• Machine epsilon:

58
Round-off error