Introduction to Numerical Analysis

And
Error Analysis
Dr. Umer Farooq Ahmed

04/05/2025 Capital University of Science and Technology 1 1

 Goals and Objectives

• Apply numerical methods/algorithms to obtain approximate solutions to

complex mathematical problems.

• These complex mathematical problems arises in all fields of:

• Engineering
• The physical sciences
• Life sciences
• Scientific computations

04/05/2025 Capital University of Science and Technology 2

 Logistics

Evaluation methods Theory Weight (%)[T]

Quizzes 30
Assignments 10
Mid Term Exam 20
Final Exam 40
Total 100

Assignments will be 5 marks for written part

and 5 from the quiz

04/05/2025 Capital University of Science and Technology 3

 Text Books

04/05/2025 Capital University of Science and Technology 4

 Math and crime: Deblurring a number Plate

 A short crime story

 Burglar robs a bank

 Escapes in a getaway car

 Pursued by police

04/05/2025 Capital University of Science and Technology 5

 Math and crime: Deblurring a number Plate

 GOOD NEWS Police take a photo

 BAD NEWS Photo is blurred

 How do we deblur number plate?

04/05/2025 Capital University of Science and Technology 6

 Math and crime: Deblurring a number Plate

 SOLUTION:
 Find a model of the blurring process
 need to know the blurring function g
 Blurring formula/Process:
 Inverting the formula we can get rid the blur
 Above formula for f is very complicated, and can not be
solved by analytical method.
 Then how we solve?
 We have to design method/algorithm that solve above model
numerically.

04/05/2025 Capital University of Science and Technology 7

 Math and Data Science: Correcting a value

 Suppose you are working on a data science project and you

acquire the following data:

 It is suspected that one of the values may have been recorded

in error.
 How we locate the error and estimate the correct value for the
above data?
 Solution: By inspecting the difference table, we develop a
method that can solve the above problem.

04/05/2025 Capital University of Science and Technology 8

 Math and Data Science: Finding missing value

 Suppose you are working on a data science project. But

unfortunately, the data you collected were missing some
values

 How will you find the missing values.

 Solution: We will develop a method using finite difference
theory that can solve the above problem.

04/05/2025 Capital University of Science and Technology 9

 Math and Machine Learning: Prediction

 The owner of a retailing organization is interested in the

relationship between price at which a commodity is offered
for sale and the quantity sold.
 The following sample data have been collected:

 How we predict the quantity sold when price is 55?

Price 25 45 30 50 35 40 65 75 70
Quantity 118 105 112 100 111 108 95 88 91
sold

04/05/2025 Capital University of Science and Technology 10

 Math and Scientific Computing: Numerical Integration

 SUPARCO (the national space agency) launch a satellite

(Badr) on space. They recorded its velocity (V) for time (T)
as follows:
Time 0 10 30 40 50
Velocity 0 20 35 42 45
 Can we find the distance travelled in 50 seconds?. If yes, then
how?

04/05/2025 Capital University of Science and Technology 11

 Why Numerical methods for differentiations

 Consider a simple continuous function of single variable

y = f (x).
 If the function is known and simple, we can easily obtain its
derivative (s) analytically using calculus.
 How we compute derivative if:
– We do not know the function as such
– The function is complicated
– Function is given in a tabular form at a set of points x0,x1,
…,xn,
– In all these situation, numerical methods helps us for
computing derivatives.

04/05/2025 Capital University of Science and Technology 12

 Math and Scientific Computing: Prediction

 In short, to solve complex computing problems, we design a

numerical method.
 So What is Numerical Method?
– Numerical Computation
– Numerical Methods
– Numerical Analysis

04/05/2025 Capital University of Science and Technology 13

 What is Numerical Computing?
 “It is the study of algorithms that use numerical approximation for the
problem of mathematical analysis” ---- Wikipedia.
 Study of Algorithms:
– Create, analyze, and implement algorithms
 Numerical Approximation:
– Solving problems numerically (using arithmetic and logic operation)
and approximately (not exact)
 Mathematical Analysis:
– a set of techniques use to prove that a numerical method
approximately solves a problem you're interested in
– Problems of continuous mathematics.
– Such problems originate generally from real-world applications of
algebra, geometry and calculus, and they involve variables which vary
continuously.

04/05/2025 Capital University of Science and Technology 14

 UP-SHOT

 “Numerical Methods are techniques by which complex

mathematical problems are formulated so that they can be
solved with arithmetic and logical operations”.
 Because digital computers excel at performing such
operations, numerical methods are sometimes referred as
“computer mathematics”.
 The approach involves, in most of the cases, formulation of
mathematical models of physical situations that can be solved
with arithmetic operations
 It requires development, analysis and use of algorithms

04/05/2025 Capital University of Science and Technology 15

 Steps of Solving a Practical Problem

 Step 1: State the problem clearly, including any simplifying

assumptions.
 Step2: Develop a mathematical statement/model of the
problem in a form that can be solved numerically
(approximate answer)
– This process may involve the use of calculus.
– In some situations, other mathematical procedures may be
employed.
– When this statement is a differential equation, appropriate
initial conditions and/or boundary conditions must be
specified

04/05/2025 Capital University of Science and Technology 16

 Steps of Solving a Practical Problem

 Step 3: Solve the mathematical model/equations that are

obtained from step 2
– Sometimes the method will be algebraic,
– But frequently more advanced methods will be needed
– The result of this step is a numerical answer or set of
answers
 Step 4: Interpret the numerical result to arrive at a decision
– This will require experience and understanding of the
situation in which the problem is embedded

04/05/2025 Capital University of Science and Technology 17

 How do we solve an engineering problem?

Problem Description

Mathematical Model

Solution of Mathematical
Model
Using the Solution

04/05/2025 Capital University of Science and Technology 18

 Different forms of mathematical equations/Model

04/05/2025 Capital University of Science and Technology 19

 How to obtained solution of Mathematical Model?

 To solve mathematical equations analytically, you may use

your experiences in the courses you have studied so far such
as:
– Calculus
– Linear Algebra
– Differential equations
– Statistics
 But, in most cases, the equations/model cannot be solved
analytically with simple formulas.
 Then what is solution?
– Need to be solved approximately using numerical
methods.

04/05/2025 Capital University of Science and Technology 20

 Scope of Numerical Analysis

 Where Numerical methods applied?

– Finding roots of equations
– Solving systems of linear algebraic equations
– Interpolation and regression analysis
– Numerical differentiation
– Numerical Integration
– Solution of ordinary differential equations
– Boundary value problems
– Solution of matrix problem

04/05/2025 Capital University of Science and Technology 21

 Accuracy in Numerical Analysis
 As discussed earlier Numerical methods gives approximate solution (not
exact) of a problem.
 Numerical analysis is an approximation, but results can be made as
accurately as desired.
 Errors come in a
– variety forms and sizes;
– some are avoidable and some are not
• For example, data conversion and roundoff errors can not be
avoided, but human errors can be eliminated
 It is therefore essential to know that
– how errors arise? (source of errors)
– how they grow during numerical process (error propagation)
– how they affect the accuracy of a solution

04/05/2025 Capital University of Science and Technology 29

 Why measure errors?

 To determine
– The accuracy of numerical results.
 To develop
– Stopping criteria for iterative algorithms.
 What are the sources of numerical error?
– Numerical errors (also known as procedural error) are
introduced during the process of implementation of a
numerical method.

04/05/2025 Capital University of Science and Technology 30

 Taxonomy of errors

04/05/2025 Capital University of Science and Technology 31

 Modelling errors

 In many situations it is impractical to model each of the

components accurately and so certain simplifying
assumptions are made
– For example, while developing a model for calculating the
force acting on a falling body, we may not be able to
estimate the air resistance coefficient (drag coefficient)
properly or determine the direction and magnitude of wind
force acting on the body and so on.
 Since the model is the basic input to the numerical process,
no numerical method will provide adequate results if the
model is erroneously conceived and formulated

04/05/2025 Capital University of Science and Technology 32

 Inherent errors

 Inherent error (also known as input error) contain two

components, namely,
– data errors and conversion errors
 Data error: Data error (also known as empirical error) arises
when data for a problem are obtained by some experimental
means and are, therefore, of limited accuracy and precision.
 Conversion error: Conversion error (also known as
representational error) arise due to the limitations of the
computer to store the data exactly.

04/05/2025 Capital University of Science and Technology 33

 Numerical Errors

 Numerical errors (also known as procedural error) are

introduced during the process of implementation of a
numerical method.
 These are
– Roundoff error
– Truncation error
 Roundoff error: Roundoff error occur when a fixed number of
digits are used to represent exact numbers
 Examples:
1
0.333333 2 1.4142...
3

04/05/2025 Capital University of Science and Technology 34

 Numerical Errors

 Truncation error: Truncation error arise from using an

approximation in place of an exact mathematical procedure.
– Typically, it is the error resulting from the truncation of
numerical process
– We often use finite number of terms to estimate the sum of
infinite series

04/05/2025 Capital University of Science and Technology 35

 Example of Truncation error

Taking only a few terms of a Maclaurin series

to
approximat e x
e 2 3
x x x
e 1  x    ....................
2! 3!
If only 3 terms are used,
 x 2

Truncation Error e   1  x  
x

 2! 

04/05/2025 Capital University of Science and Technology 36

 Blunders

 Blunders are the errors that are caused due to human

imperfection
 Some common type of this error are:
– Lack of understanding of the problem
– Wrong assumption
– Overlooking of some basic assumptions required for
formulating the model
– Error in deriving the mathematical equation or using a
model that does not describe adequately the physical
system under study
– Selecting a wrong numerical method for solving the
mathematical model

04/05/2025 Capital University of Science and Technology 37

 Blunders

 Some common type of this error are:

– Selecting a wrong algorithm for implementing the
numerical method
– Making mistakes in the computer program
– Mistake in data input
– Wrong guessing the initial value

04/05/2025 Capital University of Science and Technology 38

ROUND OFF ERRORS due to Number Representation

 Most computer languages use floating-point arithmetic

 Every number is represented using a (fixed, finite) number of
binary digits, called bits. Each binary digit is referred to as a
bit.

39
 Example of Addition of Base 8

 Perform the following operations:

 (7)8 + (6)8
The sum of 7 and 6 in base-10 is 13.
This is greater than 8.
Using the remainder method, we have 13/8 = 1 remainder 5
1/8 = 0 remainder 1
The answer is (15)8.

40
 Conversion to Base 8

 Convert (0.14)10 to base-8

 0.14 × 8 = 1.12
 0.12 × 8 = 0.96
 0.96 × 8 = 7.68
 0.68 × 8 = 5.44
 0.44 × 8 = etc.
 The answer is (0.1075….)8 which is obtained from the integer
parts of the products above.

41
Machine Representation and Arithmetic Errors:

 These errors are inevitable when using floating-point

arithmetic when using computers or calculators. Examples are
rounding and chopping errors.

42
 Significant Digits

 The following statement describe the notion of significant

digits
 All non-zero digits are significant
 All zeros occurring between non-zero digits are significant
digits
 Trailing zeros following a decimal point are significant.
 For example, 3.50, 65.0 and 0.230 have three significant
digits

04/05/2025 Capital University of Science and Technology 43

 Significant Digits
 All non-zero digits are significant. 198745 contains six significant digits.
 All zeros that occur between any two non zero digits are significant. For
example, 108.0097 contains seven significant digits.
 All zeros that are on the right of a decimal point and also to the left of a
non-zero digit is never significant. For example, 0.00798 contained three
significant digits.
 All zeros that are on the right of a decimal point are significant, only if, a
non-zero digit does not follow them. For example, 20.00 contains four
significant digits.
 All the zeros that are on the right of the last non-zero digit, after the
decimal point, are significant. For example, 0.0079800 contains five
significant digits.
 All the zeros that are on the right of the last non-zero digit are significant
if they come from a measurement. For example, 1090 m contains four
significant digits.

04/05/2025 Capital University of Science and Technology 44

 Significant Digits

Q.1: Identify the number of significant digits/figures in the

following given numbers.

 45,
 0.046,
 7.4220,
 5002,
 3800

45
 Significant Digits

Answer:
 45, 2
 0.046, 2
 7.4220, 5
 5002, 4
 3800 2

46
 Examples of showing the number of significant digits

 0.0495 has three significant digits.

 4.590 has four significant digits.
 4008 has four significant digits
 4008.0 has five significant digits.
 1.079x103 has four significant digits.
 1.0790x103 has five significant digits.
 1.07900x103 has six significant digits
 So, how do we differentiate the number of digits correct in 1,000,000 and
1,079,587?
– Well for that, one may use scientific notation.
– 1,000,000= 1x106 ; 1 significant digit
– 1,079,587 = 1.079587x106 ; 7 significant digits

04/05/2025 Capital University of Science and Technology 47

 Problems created by Round off error

 28 Americans were killed on February 25, 1991 by an Iraqi

Scud missile in Dhahran, Saudi Arabia.
 The patriot defense system failed to track and intercept the
Scud. Why?
– The Patriot defense system consists of an electronic
detection device called the range gate.
– It calculates the area in the air space where it should look
for a Scud.
– To find out where it should aim next, it calculates:
• the velocity of the Scud
• the last time the radar detected the

04/05/2025 Capital University of Science and Technology 48

 Problems with Patriot missile

 Clock cycle of 1/10 seconds was represented in 24-bit fixed

point register created an error of 9.5 x 10-8 seconds.
 The battery was on for 100 consecutive hours, thus causing an
inaccuracy of
s
8 3600s
9.5 10 100hr 
0.1s 1hr
0.342s

 The shift calculated in the ranging system of the missile was

687 meters.
 The target was considered to be out of range at a distance
greater than 137 meters.

04/05/2025 Capital University of Science and Technology 49

 Types of error and how we compute them?

 There are many kinds of error:

– True Error
– Relative True Error
– Approximate Error
– Relative Approximate Error

04/05/2025 Capital University of Science and Technology 50

 True Error

 Defined as the difference between the true value in a

calculation and the approximate value found using a
numerical method etc.
 Mathematically:
True Error = True Value – Approximate Value

04/05/2025 Capital University of Science and Technology 51

 Example: True Error
The f (x) of a f (x) can be
derivative, function
approximated by the equation,
f ( x  h)  f ( x)
f ' ( x) 
h
If an h 0.3
f ( x) 7e 0.5 x
d
a) Find the approximate f ' ( 2)
value of
b) True value f ' ( 2)
of True error for part
c)
(a)

04/05/2025 Capital University of Science and Technology 52

 Example: True Error

Solution:
a) For x 2 and h 0.3
f ( 2  0.3)  f ( 2)
f ' ( 2) 
0.3
f (2.3)  f (2)

0.3
7e 0.5( 2.3)  7e 0.5( 2 )

0.3
22.107  19.028
 10.263
0.3

04/05/2025 Capital University of Science and Technology 53

 Example: True Error

Solution
:
b) The exact value f ' ( 2) can be found by using our
of
knowledge of differential calculus.
f ( x )  7 e 0 .5 x
f ' ( x ) 7 0.5 e 0.5 x
3.5e 0.5 x
So the true value f ' ( 2) is
of f ' ( 2) 3.5e 0.5( 2 )
9.5140
True error is calculated as
Et  True Value – Approximate
9.Value
5140  10.263  0.722

04/05/2025 Capital University of Science and Technology 54

 Relative True Error

 Defined as the ratio between the true error, and the true value.
 Mathematically
True Error
Relative True Error (t ) =
True Value

04/05/2025 Capital University of Science and Technology 55

 Example: Relative True Error
Find the relative true error f ( x) 7e 0.5 x at f ' ( 2) wit
for
h 0.3 h
Solution
From the previous example,
Et  0.722
Relative True Error is defined
as True Error
t 
True Value
 0.722
  0.075888
9.5140

as a percentage,
t  0.075888 100%  7.5888%

04/05/2025 Capital University of Science and Technology 56

 Relative Error Significant Digits

 If XE is the exact or true value and XA is an approximation to

XE,
 then XA is said to approximate XE to t significant digits if t is
the largest non-negative integer for which

57
 Example

 Let the exact or true value = 20/3 and,

 Approximate value =6.666
 Absolute true error is 0.000666…= 2/3000.
 The relative error is (2/3000)/(20/3)=1/10000

 So the number of significant digits is 4.

58
 Approximate Error

 What can be done if true values are not known or are very
difficult to obtain?
 Approximate error is defined as the difference between the
present approximation and the previous approximation.

Approximate ErrorE a ) = Present Approximation – Previous

( Approximation

04/05/2025 Capital University of Science and Technology 60

 Example-Approximate Error

For f ( x) 7e 0.5 x at x 2 find the

a) f (2) usin h 0.3 following,
b) f (2) gusin h 0.15
c) approximateg error for the f (2) for part
value of
Solution: b)
a) x 2 and h 0.3
For f ' ( x )  f ( x  h )  f ( x )
h
f ( 2  0.3)  f ( 2)
f ' ( 2) 
0.3

04/05/2025 Capital University of Science and Technology 61

 Example-Approximate Error

Solution: (cont.)
f (2.3)  f (2)

0.3
7e 0.5( 2.3)  7e 0.5( 2 )

0.3
22.107  19.028
 10.263
0 .3
b) For x 2 and h 0.15
f (2  0.15)  f (2)
f ' ( 2) 
0.15
f (2.15)  f (2)

0.15

04/05/2025 Capital University of Science and Technology 62

 Example-Approximate Error

Solution: (cont.)
7e 0.5( 2.15)  7e 0.5( 2 )

0.15
20.50  19.028
 9.8800
0.15

c) So the approximate E a is
error,
Ea Present Approximation – Previous
9Approximation
.8800  10.263
 0.38300

04/05/2025 Capital University of Science and Technology 63

 Relative Approximate Error

 Defined as the ratio between the approximate error and the

present approximation.
 Mathematically:
Approximate Error
Relative Approximate a) =
Error ( Present Approximation

04/05/2025 Capital University of Science and Technology 64

 Example: Relative Approximate Error
0.5 x
For f ( x) 7e at x 2 , find the relative
error using values approximate
h 0.3 and h 0.15
from
Solution:
From Example 3, the approximate f (2) 10.263
value hof
usin 0.3 and f (2) 9.8800 usin h 0.15
g Ea  Present Approximationg– Previous
9Approximation
.8800  10.263
 0.38300

04/05/2025 Capital University of Science and Technology 65

 Example: Relative Approximate Error

Solution:
(cont.) Approximate Error
a 
Present Approximation
 0.38300
  0.038765
9.8800
as a
percentage,
a  0.038765 100%  3.8765%

Absolute relative approximate errors may also

need to be calculated,
a |  0.038765 | 0.038765 or 3.8765 %

04/05/2025 Capital University of Science and Technology 66

 How is absolute Relative error used as a stopping criterion?

If |a | s wher s is a pre-specified tolerance, then no

further e iterations are necessary, and the process is
stopped.

If at least m significant digits are required to be

correct in the final answer, then
|a |0.5 10 2 m %

If I want 2 significant digits to be correct in answer so

|∈a | ≤ 0.5 x 102-2 % = 0.5%
If I want 1 significant digits to be correct in answer so
|∈a | ≤ 0.5 x 102-1 % = 5%

04/05/2025 Capital University of Science and Technology 67

 Propagation of error

 In numerical methods, the calculations are not made with

exact numbers. How do these inaccuracies propagate through
the calculations?
 Example: Find the bounds for the propagation error in adding
two numbers. For example, if one is calculating X +Y where
X = 1.5 ± 0.05
Y = 3.4 ± 0.04

04/05/2025 Capital University of Science and Technology 68

 Propagation of error

 Example: Find the bounds for the propagation error in adding

two numbers. For example, if one is calculating X +Y where
X = 1.5 ± 0.05
Y = 3.4 ± 0.04
 Solution:
Maximum possible value of X = 1.55 and Y = 3.44
Maximum possible value of X + Y = 1.55 + 3.44 = 4.99
Minimum possible value of X = 1.45 and Y = 3.36.
Minimum possible value of X + Y = 1.45 + 3.36 = 4.81
Hence
4.81 ≤ X + Y ≤ 4.99.

04/05/2025 Capital University of Science and Technology 69

 Importance of Error Propagation

Understanding error propagation is crucial for many reasons:

 Reliable Results: It allows us to assess the reliability of our
calculations and identify potential sources of error.
 Informed Decisions: Knowing the uncertainty in our
measurements helps us make more informed decisions based
on our data.
 Scientific Rigor: Error propagation is a fundamental aspect of
scientific research, ensuring that our results are presented
with appropriate levels of confidence.

04/05/2025 Capital University of Science and Technology 70

 How to Handle Error Propagation

 The good news is that there are mathematical methods to

estimate how errors propagate through calculations. These
methods depend on the specific type of calculation you're
performing.

04/05/2025 Capital University of Science and Technology 71

 Addition and Subtraction

Let's say you want to calculate the total length of a piece of

string by adding the lengths of two segments:
 Segment 1: 10 cm ± 0.1 cm (meaning the length is between
9.9 cm and 10.1 cm)
 Segment 2: 5 cm ± 0.2 cm
To find the total length, we simply add the lengths: 10 cm + 5
cm = 15 cm.
But what about the errors? The maximum error in the total
length is the sum of the individual errors: 0.1 cm + 0.2 cm = 0.3
cm.
So, the total length of the string is 15 cm ± 0.3 cm.

04/05/2025 Capital University of Science and Technology 72

 Multiplication and Division

Let's say you want to calculate the area of a rectangle:

 Length: 5 cm ± 0.1 cm
 Width: 3 cm ± 0.2 cm

The area is calculated by multiplying length and width: 5 cm x 3

cm = 15 cm².
To find the error in the area, we need to use a slightly more
complex formula:
 Relative error in area = relative error in length + relative error
in width

04/05/2025 Capital University of Science and Technology 73

 Multiplication and Division

The relative error is calculated by dividing the error by the

original value. So, the relative error in length is 0.1 cm / 5 cm =
0.02 and the relative error in width is 0.2 cm / 3 cm = 0.067.

The relative error in the area is then 0.02 + 0.067 = 0.087.

To get the absolute error in the area, we multiply the relative

error by the area: 0.087 x 15 cm² = 1.3 cm².

Therefore, the area of the rectangle is 15 cm² ± 1.3 cm².

04/05/2025 Capital University of Science and Technology 74