CS 351 – NUMERICAL ANALYSIS

HANDOUT 1

1. INTRODUCTION TO NUMERICAL MATHEMATHEMATICS

General Introduction
Numerical analysis is the area of mathematics and computer science that creates, analyzes,
and implements algorithms for solving numerically the problems of continuous mathematics.
Such problems originate generally from real-world applications of algebra, geometry and
calculus, and they involve variables which vary continuously; these problems occur
throughout the natural sciences, social sciences, engineering, medicine, and business. During
the past half-century, the growth in power and availability of digital computers has led to an
increasing use of realistic mathematical models in science and engineering, and numerical
analysis of increasing sophistication has been needed to solve these more detailed
mathematical models of the world. The formal academic area of numerical analysis varies
from quite theoretical mathematical studies to computer science issues.
With the growth in importance of using computers to carry out numerical procedures in
solving mathematical models of the world, an area known as scientific computing or
computational science has taken shape during the 1980s and 1990s.
This area looks at the use of numerical analysis from a computer science perspective. It is
concerned with using the most powerful tools of numerical analysis, computer graphics,
symbolic mathematical computations, and graphical user interfaces to make it easier for a
user to set up, solve, and interpret complicated mathematical models of the real world.
Historical background
Numerical algorithms are almost as old as human 1650
BC) of ancient Egypt describes a root finding method for solving a simple equation.
Archimedes of Syracuse (287- 212 BC) created much new mathematics, including the
“m f x ” f calculating lengths, areas, and volumes of geometric figures.
When used as a method to find approximations, it is in much the spirit of modern numerical
integration; and it was an important precursor to the development of the calculus by
Isaac Newton and Gottfried Leibnitz.
A major impetus to developing numerical procedures was the invention of the calculus by
Newton and Leibnitz, as this led to accurate mathematical models for physical reality, first in
the physical sciences and eventually in the other sciences, engineering, medicine, and
business. These mathematical models cannot usually be solved explicitly, and numerical
methods to obtain approximate solutions are needed.
Another important aspect of the development of numerical methods was the creation of
logarithms by Napier (1614) and others, giving a much simpler manner of carrying
out the arithmetic operations of multiplication, division, and exponentiation.
Newton created a number of numerical methods for solving a variety of problems, and his
name is attached today to generalizations of his original ideas. Of special note is his work on
Page 1 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017
 root finding and polynomial interpolation. Following Newton, many of the giants of
mathematics of the 18th and 19th centuries made major contributions to the numerical
solution of mathematical problems. Foremost among these are Leonhard Euler (1707-1783),
Joseph-Louis Lagrange (1736-1813), and Karl Friedrich Gauss (1777-1855). Up to the late
1800’ , m m m w q broad in their interests, and many of
them were interested in and contributed to numerical analysis.

1.1 Traditional Mathematics and Numerical Mathematics

There is much in common between traditional mathematics and numerical mathematics, and
the real differences are probably more a question of emphasis and priority than of actual
content. Both disciplines concern themselves with solving mathematical problems, but they
do so for somewhat different reasons and different aims in mind.
In both traditional mathematics and numerical mathematics, a mathematical problem is
studied from a number of points of view, and various questions are asked about the
solution(s) such as the following:
 Does the problem have a solution?
 Is there just one solution?
 What is the nature of the solution?

However, in considering the last question the traditional mathematician is likely to be more
concerned in establishing that the solution can be found than in actually finding it. Moreover,
any constructions that he gives for the solution are likely to be designed primarily to establish
the structure and properties of the solution rather than the to determine all of its values
rapidly. In contrast, numerical mathematics has developed with the specific aim of actually
solving problems and printing out the answers.
The two main differences of emphasis concern:
(i) Exactness of the solution, and
(ii) Effectiveness of the methods
The traditional mathematician (TM) studies the exact solution of a problem, the
determination of which often involves a infinite F x m , π = 3 141592….
However, no computer can perform an infinite number of calculations, so this exact solution
π) cannot ever be calculated. Similarly, in constructing a method of solution, the TM is often
concerned with structure and elegance than with efficiency and reliability, although
constraints of computer time and cost may in practice dictate that efficiency should be given
higher priority. The aims of the numerical mathematician (NM), in contrast to those of
traditional mathematician(TM) are:
 To solve a problem approximately in a finite number of steps and
 To obtain the solution by a method (numerical method) which is efficient and reliable

Page 2 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017
 Example:
The problem of determining the solution x (in radians) of the equation
tan x = 2x (0<x<π/2)
which occurs in connection with conduction of heat in a solid cannot be solved exactly in a
finite number of steps. Indeed the true solution
x = s = 1.1656…
is an irrational number with a never ending decimal representation. By sketching the graph of
y = tan x and y = 2x and considering their interceptions, or alternatively by using calculus,
TM immediately deduces that equation tan x = 2x q b w π/4
π/2. He may then subsequently simply refer to this solution by the letter s and produce a
formulae for the temperature in a solid at a given position and time in terms of s. However,
the NM is left with the task of calculating the actual value of s, and he must do so in a finite
number of steps to a required accuracy and efficiently as possible.

Many of the methods involved in numerical mathematics have only become possible to
implement as a result of the development of electronic computers and conversely the
development of better computers has led to a demand for more and better numerical
mathematics. However, it would be wrong to see numerical mathematics as a replacement for
traditional mathematics. Engineers and scientists need the complementary aspect of both
traditional and numerical mathematics. Moreover, numerical mathematics itself has its roots
in traditional mathematics, and many of the ‘new’ methods it uses are based on ideas
developed within traditional mathematics.

1.2 Numerical methods and Numerical analysis

There are a variety of textbooks on numerical mathematics. Books intended for engineers,
scientists and other practical people tend to be given a title like ‘numerical methods’, while
books intended for mathematicians, graduate students and other ‘highbrow’ people tend to be
given a title like ‘numerical analysis’. Numerical methods and numerical analysis are indeed
the two main aspects of numerical mathematics, and their tasks are essentially as follows:

Numerical methods: Numerical analysis:

find a method for obtaining a numerical analyze the properties of the methods and the
solution of a problem errors in the computed solution

Obviously both these topics are relevant in this course but greater emphasis may be given to
numerical analysis. When faced with a variety of methods which purport to solve a certain
problem, it is better to make one’s choice based on analysis (someone else’s if necessary)
than to base one’s choice on a random guess.

Page 3 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017
 How then do you analyse numerical methods? Ideally we estimate or calculate the errors in a
numerical solutions obtained by a variety of methods. We then choose the method which
achieves a required accuracy by the shortest calculation. Clearly a key factor in this is to be
able to understand and recognise all types of errors which may occur during a calculation.

1.3 Errors in numerical calculations

• The most fundamental feature of numerical computing is the inevitable presence of

errors.

• The result of any interesting computation (and of many uninteresting ones) is typically
only approximate, and our goal is to ensure that the resulting error is tolerably small.

Errors in numerical methods:

• Absolute Error: Absolute Error is the magnitude of the difference between the true
value x and the approximate value xa, Therefore absolute error between two values is
defined as

∥ x − xa ∥

x denotes the exact value and xa denotes the approximation.

• Relative Error

The relative error of x is the absolute error relative to the exact value. Look at it this way:
if your measurement has an error of ± 1 inch, this seems to be a huge error when you try
to measure something which is 3 in. long. However, when measuring distances on the
order of miles, this error is mostly negligible. The definition of the relative error is

∥ x − xa ∥/ ∥ x ∥

• The percentage error, or percentage accuracy, is the relative error scaled so as to lie
between 0 and 100:

If a floating number is rounded as follows:

0.100005E7 = 0.10000E7 (to 5 significant figures)

The absolute, relative and percentage errors are respectively

0.50000E2, 0.49997E-4 and 0.49997E-2 percent.

Page 4 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017
 Verify that:

(i) The absolute accuracy is most easily determined if the results are expressed to a
certain number of decimal places – a number rounded to p decimal places has
absolute error at most 5 in the (p+1)th decimal place
(ii) The relative accuracy is most easily determined if the results are expressed to a
certain number of significant figures – a number rounded to q significant figures
has relative error at most 5 in the qth significant figure.

That is, the absolute error is comparable with the number of decimal places while the
relative error is comparable with the number of significant figures.

In a numerical computation, error may arise because of the following reasons:

• human errors,
• Errors in the data,
• Truncation Error: error refers to the error in a method, which occurs because
some series (finite or infinite) is truncated to a fewer number of terms. Such errors
are essentially algorithmic errors and we can predict the extent of the error that will
occur in the method.
• Rounding Errors: occurs because of the computing device's inability to deal with
certain numbers. Such numbers need to be rounded off to some near approximation
which is dependent on the word size used to represent numbers of the device.
1.3.1 Human errors
This is blunder, namely slip made during the course of the calculation. It
could be an error in copying down or typing out the data, an error in desk
calculations, an error in computer program or even an error in a
mathematical formula. If a numerical fails unexpectedly, then human error
is the most likely cause. Never blame a method until you have made quite
certain that you have eliminated all human errors, or you may make
yourself unpopular.
1.3.2 Errors in the data (and conditioning)
Often the data of a problem are inexact, either because they are physical
measurements or because they are simply given to a limited accuracy.
Errors are then introduced into the data, producing consequently errors in
the solution.
If a small error in the data leads to a large error in the solution then the
problem is said to be ill-conditioned, otherwise it is said to be well-
conditioned. When a problem is ill-conditioned, a much lower accuracy
must be expected in the solution than is present in the data.
Example: (ill-conditioned problem)
The system of equations

Page 5 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017
 100x1 + 99x2 = 398

99x1 + 98x2 = 394

has solution x1 = 2, x2 = 2

while the system of equations

100x1 + 99x2 = 398

99x1 + 98x2 = 393.98

has solution x1 = -0.02, x2 = 4

1.3.3 Truncation errors (and convergence)

A numerical method involves by necessity only a finite number, n say, of
steps, while the true solution may involve an infinite process in the
calculation. If it does, an error is bound to be introduced in the numerical
method itself by ‘truncating’ the infinite process into a finite one, and this
error is called truncation error. If the true solution is y, and the numerical
solution obtained by a calculation with only n steps is yn , then the
truncation error is
y-yn
in practice n is increased until y is obtained to an acceptable accuracy. For
this to be possible the truncation error must approach zero as n becomes
larger. In other words yn must converge to y as n ‘tends to infinite’. To
consolidate this principle, try to calculate the approximate value of e using
a finite series expansion to n number of steps.
Example:
Solve the equation x5 + 3x2 – 10 = 0
x0 = 1; using the recurrent formulas where
a)
b)
solution: (exercise)
x0 = 1
x1 = 1.47577
x2 = 1.28225
x3 = 1.38344
x4 = 1.33613
x5 = 1.35951
x6 = 1.34826
x7 = 1.35375

Page 6 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017
 1.3.4 Rounding errors (and stability)
If a desk calculator or computer is used for numerical calculation, then
floating point arithmetic is normally used. This means that the result of
every arithmetic operation has to be rounded to the machine accuracy, and
the error so produced is termed rounding error. On desk calculator
numbers are rounded to the length of the visual display, which is typically
about 10 significant figures. On a computer, the result may be rounded to
6, 8 or 16 significant figures depending on the word length of the
computer used. The net effect of all the rounding performed during the
course of the calculation is called accumulated rounding error and may
have noticeable effect on the final result
A method is said to be unstable if the accumulated rounding error grows
relative to the values of the solution as we increase number m of values
calculated. If rounding error does not grow, the method is stable.
Determine whether the calculation of x1, x2, x3, x4, … defined by the
recurrence
xi = 10.1xi-1 – xi-2 , i = 2, 3, 4, …
x0 = 1.00, x1 = 0.1
is stable or unstable.
The dangerous phenomenon of significance error occurs when two nearly
equal numbers are subtracted with consequent ‘loss of significance’.
Consider the following examples where the ‘true’ calculations might be
6.5432149 – 6.5431950 = 0.0000199
But
6.54321 – 6.54320 = 0.0001
Where 6.54321 and 6.54320 already contain rounding errors, rounded to 6
significant figures, gives the result with only one significant figure – error
of about 100 per cent.

Consider also determining the solution of the quadratic equation

x2 – 1.0640x + 0.2830 = 0
where numbers are rounded to
(i) 3 significant figures
(ii) 4 significant figures

at various stages of calculation.

Page 7 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017
Exercise
Show that the calculation of xn from xi = 49xi-1 + 50xi-2 from where x0 = 1.001, x1 = -1 is
unstable (hint: calculate x4 exactly and then one 3 significant figures)
.
1.4 Steps in a Numerical Method
Steps in an ideal numerical method are given below
1. Given Mathematical problem 1. Is the problem well
posed?
2. Is the problem well-
conditioned?
3. How accurate are the
data?
2. Choose Numerical method with n 1. Does the method
steps (solution) converge?
2. Is the method efficient?
3. Is the method stable?
3. Study Initial error analysis 1. Can you predict the
truncation errors after n
steps?
2. Can you predict the
rounding errors after n
steps?

4. Write Computer program 1. Is the program correct?

2. What accuracy do you
aim for in an answer?
3. How many steps n do you
use?
5. Study Final error analysis 1. Can you estimate the
truncation error that has
actually occurred?
2. Can you estimate the
rounding error that has
actually occurred?

Exercises:

1. Write a computer program to establish that the probability of obtaining a head when a
coin is tossed is ½
2. Use a computer program to find approximate solution of radioactive substance with
following decay formula
Where m (0) = m

Page 8 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017
 3. Test the conditioning of the following problems by making 1% changes in the data on the
hand sides and observe the consequent percentage changes in the solution

a) 1.0x1 + 0.50x2 = 1.5

0.50x1 + 0.33x2 = 0.83

b) 1.7 = x0.01

Page 9 of 9
CS 351 – NUMERICAL ANALYSIS PREPARED BY O. NDHLOVU 2017