Introduction to Engineering

Computational Mathematics (IECM)

DR. M. SAEED UZ ZAMAN

Introduction to Engineering Computational Mathematics

Concept of Error

The arithmetic performed by a calculator or computer is different from the

arithmetic in algebra and calculus courses. (For example, )

The arithmetic we use in this world defines √3 as that unique positive

number that when multiplied by itself produces the integer 3. In the
computational world, however, each representable number has only a
fixed and finite number of digits. This means, for example, that only
rational numbers—and not even all of these—can be represented exactly.

IECM Lecture: Concept of Error 2

Introduction to Engineering Computational Mathematics

Concept of Error

The error that is produced when a calculator or computer is used to perform realnumber
calculations is called round-off error. It occurs because the arithmetic performed in a
machine involves numbers with only a finite number of digits, with the result that
calculations are performed with only approximate representations of the actual numbers. In
a computer, only a relatively small subset of the real number system is used for the
representation of all the real numbers. This subset contains only rational numbers, both
positive and negative, and stores the fractional part, together with an exponential part.

IECM Lecture: Concept of Error 3

Introduction to Engineering Computational Mathematics

Concept of Error

• Error due to rounding should be expected whenever computations are performed using numbers
that are not powers of 2. Keeping this error under control is extremely important when the number
of calculations is large.

• IEEE standard: A 64-bit (binary digit) representation is used for a real number. The first bit is a sign
indicator, denoted s. This is followed by an 11-bit exponent, c, called the characteristic, and a 52-
bit binary fraction, f , called the mantissa. The base for the exponent is 2.

• 0 10000000011 1011100100010000000000000000000000000000000000000000 = 27.56640625

IECM Lecture: Concept of Error 4

Introduction to Engineering Computational Mathematics

Concept of Error

• Underflow: Numbers occurring in calculations that have a magnitude less than

result in underflow and are generally set to zero.

• Overflow: Numbers greater than result in overflow and typically cause the computations to stop (unless the
program has been designed to detect this occurrence).

• Note that there are two representations for the number zero; a positive 0 when s = 0, c = 0 and f = 0, and a
negative 0 when s = 1, c = 0 and f = 0. For 64-bit integers:

• Largest unsigned: 18446744073709551615 = FFF FFFF FFFF FFFFF

• Largest signed: 9,223,372,036,854,775,807= 0x7FFFFFFFFFFFFFFF

• Smallest signed: -9,223,372,036,854,775,808 = 0x8000 0000 0000 0000

IECM Lecture: Concept of Error 5
Introduction to Engineering Computational Mathematics

Concept of Error

• Interesting reads:

• https://www.mathworks.com/help/matlab/matlab_prog/floating-point-numbers.html

• Numerical Computing with MATLAB by Cleve Moler

IECM Lecture: Concept of Error 6

Introduction to Engineering Computational Mathematics

Concept of Error

The number π has an infinite decimal expansion of the form π = 3.14159265…

Its normalized decimal form is π = 0.314159265 . . . × 10^1.

We can determine its five-digit value by two ways:

1. Chopping: The floating-point form of π using five-digit chopping is:

fl(π) = 0.31415 × 10^1 = 3.1415.

2. Rounding: The sixth digit of the decimal expansion of π is a 9, so the floating-point form of π
using five-digit rounding is fl(π) = (0.31415 + 0.00001) × 101 = 3.1416

IECM Lecture: Concept of Error 7

Introduction to Engineering Computational Mathematics

Concept of Error

Suppose that p∗ is an approximation to p. The absolute error is |p − p∗|, and the relative error is |
p − p∗| / |p|, provided that p ≠ 0.

Question: Determine the absolute and relative errors when approximating p by p ∗ when

(a) p = 0.3000 × 10^1 and p∗ = 0.3100 × 10^1;

(b) p = 0.3000 × 10^−3 and p∗ = 0.3100 × 10^−3;

(c) p = 0.3000 × 10^4 and p∗ = 0.3100 × 10^4.

IECM Lecture: Concept of Error 8

Introduction to Engineering Computational Mathematics

Concept of Error

Example:

Polynomials should always be expressed in nested form before performing an evaluation, because
this form minimizes the number of arithmetic calculations.

IECM Lecture: Concept of Error 9

Introduction to Engineering Computational Mathematics

Concept of Error

Examples from MATLAB:

Roundoff Error Roundoff Error Roundoff Error

IECM Lecture: Concept of Error 10

Introduction to Engineering Computational Mathematics

Concept of Error

Examples from MATLAB:

Cancellation can occur when you subtract a number from another

number of roughly the same magnitude, as measured by eps. For
example, eps(2^53) is 2, so the numbers 2^53 + 1 and 2^53 have the
same floating-point representation.
Cancellation
When possible, try rewriting computations in an equivalent form that
avoids cancellations.

IECM Lecture: Concept of Error 11

Introduction to Engineering Computational Mathematics

Concept of Error

Examples from MATLAB:

Swamping can occur when you perform operations on floating-point

numbers that differ by many orders of magnitude. For example, this
calculation shows a loss of precision that makes the addition
insignificant.
Swamping

IECM Lecture: Concept of Error 12

Introduction to Engineering Computational Mathematics

Concept of Error

Examples from MATLAB:

Intermediate Conversions

When you perform arithmetic with different data types, intermediate

calculations and conversions can yield unexpected results. For
example, although x and y are both 0.2, subtracting them yields a
nonzero result. The reason is that y is first converted to double before
the subtraction is performed. This subtraction result is then converted
to single, z.

IECM Lecture: Concept of Error 13

Introduction to Engineering Computational Mathematics

Concept of Error

Examples from MATLAB:

Linear Algebra: Common issues in floating-point arithmetic, such as the ones described above, can
compound when applied to linear algebra problems because the related calculations typically consist
of multiple steps. For example, when solving the system of linear equations Ax = b, MATLAB warns
that the results may be inaccurate because operand matrix A is ill conditioned.

IECM Lecture: Concept of Error 14

Introduction to Engineering Computational Mathematics

Concept of Error

The errors can be reduced by:

• Restructuring the problem / solution (See Illustration on Book. 1 pg. 26)

• Nesting the arithmetic

• Other techniques

IECM Lecture: Concept of Error 15

Total Probability Theorem and Bayes’ Rule
 Before next lecture:

1. Exercise Set 1.2 (first 10 questions)

2. Work on the optimization algorithm which will be implemented in your major

assignment

IECM Lecture: Concept of Error 16