LESSON 1

WHY STUDY NUMERICAL METHODS?

 These are techniques by which Math problems are formulated in order to be solved
using arithmetic and logical operations
Reduces higher Math to basic arithmetic operations; reinforces understanding of
Math
 Greatly expands the types of problems to be solved
Large system of equations, nonlinear systems’ complicated geometries, etc.
 Efficient way to learn how to use computers
Numerical methods are designed for computer implementation
Allows the designing of programs/scripts to solve problems involving Math
models

NUMERICAL METHODS
PREVIEW OF TOPICS
 Root-finding (Nonlinear systems)
Solve for x so that f(x) = 0.

 Linear Algebraic Equations

Given the a’s and the b’s, solve for the x’s.

 Curve fitting

 Integration
 Differentiation

 Differential Equations
 Given

Solve for y as a function of t.

PROGRAMMING
 Process of creating a set of instructions that tell a computer how to perform a task
 Done for automation purposes
 “Garbage in, garbage out” / GIGO
PROGRAMMING LANGUAGES/SOFTWARES
 C/C++, Java, SQL, VBA, Python
 Mathcad, MATLAB, Octave, Maple, Mathematica

TERMINOLOGIES
 Algorithm
o Sequence of logical steps required to perform a specific task
 Pseudocode
o English description of a program
 Flowchart
o Visual/graphical representation of a program
 Pseudocode

PROGRAMMING ERRORS (BUGS)

 Syntax error
o Will not compile. Compiler will help you to find it.
o (e.g. Writing primtf instead of printf)
 Run-time error
o Will compile but may stop running at some point.
o (e.g. Division by zero)
 Logical error
o Will compile and run, but the result is wrong.
o (e.g. From the previous example, all terms of sin(x) are positive)

*Run-time and logical error

need careful debugging of the program.

NOTES:
 For clarity:
o Put comments to explain variables, purpose of code segments, etc.
 For testing:
o Run with typical data.
o Run with unusual but valid data.
o Run with invalid data to check error handling.
 Refresh your programming knowledge.
  Find an introductory level programming book in the library/internet; check for tutorials.
 Approach the problem through the algorithm-pseudocode-computer code link.
 Practice at home (or during your free time).
 Do not be afraid of programming; try to feel the power of it. =)

ERRORS
 SIGNIFICANT FIGURES
o Digits of the number which are known to be correct
o Designate the reliability of a number
o *The number of significant figures to be used in a number depends on the origin
of the number.
o EXAMPLE: Calculation of the area of a circle A= πD2 / 4
 ACCURACY and PRECISION
o Computers used to obtain numerical solution are imperfect tools; limited to
represent the magnitudes and precision of numbers.
o Errors from calculations (and measurements) can be characterized with regard
to their accuracy AND precision.
 Precision
o How closely a computed (individual) or measured values agree with each other
 Accuracy
o How closely a computed or measured value agrees with the true value
 Imprecision
o Also called uncertainty
o Magnitude of the scatter
 Inaccuracy
Also called bias
Systematic deviation from the truth
*ERROR = Imprecision + Inaccuracy

NOTES:
 Precision is governed by the number of digits being carried in the numerical
calculations.
 Accuracy is governed by the errors in the numerical approximation.

ERROR DEFINITIONS
 TRUE ERROR (ET)
o Exact/absolute value of the error
o Doesn’t consider order of magnitude of the value under examination
ET = True Value- Approximation
o The error can be normalized to the true value

 APPROXIMATE ERROR (Ea)

o In actual situations, true value is rarely available
o Error estimates (or approximations) are determined in absence of Knowledge
regarding the true value
 o Utilize the best available estimate of the true value, then normalize

NOTES:
 Many numerical methods work in an iterative* fashion, thus there should be a stopping
criterion for these methods.
 The process stops when the error level drops below a certain specified tolerance value
(εs).

 SCARBOROUGH CRITERION (1966)

o If the tolerance is selected to be

othen the approximation is guaranteed to be correct to at least n significant

figures.
 ROUND-OFF ERROR
o Computers cannot
 Represent some quantities exactly
 Conversion from base-10 to base-2 number system may create
problems
 Numbers like π or 1/3
 Use infinitely many digits to store numbers
 Single precision (7-8 digits) vs. Double precision (15-16 digits)
o Sources of Round-off Errors
 Subtractive cancellation
 Subtracting two nearly equal numbers
 Large (arithmetic) computations
 Computations are often interdependent; consider cumulative effect
 Adding a large and a small number
 Usually occur in infinite series calculation
 Sum the series in reverse order
o Sources of Round-off Errors
 Smearing
 Individual terms in a summation are larger than the summation
itself
 Example: Series of mixed signs
 Calculation of inner products

o These errors cannot be totally eliminated, but clever algorithms may help to
minimize them.
  TOTAL NUMERICAL ERROR
o Summation of the truncation and round-off errors

ERROR TYPES
OTHER SOURCES OF ERRORS
 Gross Errors/Blunders
 Model Errors
 Data Uncertaint

Python as a CALCULATOR
“+” for addition
“-” for subtraction
“/” for division
“*” for multiplication
“**” for exponentiation

BASIC DATA TYPES

 Integers (int)
 Float (floating numbers or decimal values)
 Complex (complex numbers)
  String (character or text)
 Boolean (True or False values)

ASSIGNING VARIABLES
x=1
x = 1.2
x = ‘one’
x = (1, 2, 3)
x = [1,2,3]

COMPARISON OPERATORS

DATA STRUCTURE

LIST FORWARD INDEXING

** List start at 0
x = [ 1,2,3,4]
x[0] = 1
x[1] = 2
x[2] = 3
x[3] = 4

LIST BACKWARD INDEXING

** Last index item is -1
x = [ 1,2,3,4]
x[-1] = 4
x[-2] = 3
x[-3] = 2
x[-4] = 1

LIST SLICING
** List slicing is [start point: end point]
x = [ 1,2,3,4]
x[0:2]
x[1:4]
x[2:4]
x[1:3]

LIST SLICING WITH STEP SIZE

** [start point: end point: step size
x = [ 0,1,2,3,4,5,6,7,8,9,10]
x[0:11:2]

REVERSE ORDER OF THE LIST

x = [ 0,1,2,3,4,5,6,7,8,9,10]
x[::-1]

TUPLES
 Tuples are just like lists, except you define a tuple with parentheses instead of square
brackets.
 Tuples are immutable which means that once created they cannot be modified in any
way.
 x = (1,2,3)

SETS
 Sets are like lists and tuples but are defined by curly brackets. It obeys mathematical
set definitions.
 primes = {2,3,5,7} ; odds = {1,3,5,7,9}
 primes.union(odds) #combine primes and odds
 primes.intersect(odds)
 primes.difference(odds)

LESSON 2

OUTLINE
 Review of Matrix Concepts
 System of Linear Equations
 Types of Solution
 Introduction to Methods
  Direct Methods
o Gauss Elimination
o Gauss-Jordan Elimination
o LU Decomposition
o Inverse Method
o Cramer’s Rule

REVIEW OF MATRIX CONCEPTS

MATRIX
 A rectangular array of numbers or functions arranged in rows and columns
 Designated by a capital letter
 Enclosed by brackets, parentheses or double bars
SIZE OF MATRIX
 mxn
where:
m = number of rows
n = number of columns
ELEMENTS OF MATRIX
 Aij
where
i = particular row
j = particular column
SPECIAL TYPES OF MATRICES
 Row Matrix (Row Vector)
 Column Matrix (Column Vector)

 Square Matrix (m = n)
o Order of Matrix
o Principal (or Main) Diagonal

 Upper Triangular Matrix

o Square matrix with all elements below the principal diagonal equal to 0
o (aij = 0 for i > j)

 Lower Triangular Matrix

o Square matrix with all elements above the principal diagonal equal to 0
o (aij = 0 for i < j)

 Diagonal Matrix
o Square matrix that is both an upper triangular and lower triangular matrix
o (aij = 0 for i ≠ j)
 Identity Matrix
o Diagonal matrix where all elements along the main diagonal are equal to 1
o Represented by In

 Null Matrix
o Matrix where all elements are zero
o Represented by O

 Symmetric Matrix
o gij = gji

 Skew Symmetric Matrix

o hij = - hji

 Banded (or Band) Matrix

o Matrix whose non-zero elements are confined to particular diagonals

 Tridiagonal Matrix
o All elements not on the principal diagonal or on the two diagonals surrounding
the principal diagonal are zero

 Sparse Matrix
o Most of the elements are zero
o Usually seen in engineering and scientific applications
MATRIX PROPERTIES
Let O = null matrix; I = identity matrix
 Matrix Addition
o A + O = A [Additive Identity]
o A + (-A) = O [Additive Inverse]
o A + B = B + A [Commutative Property]
o (A + B) + C = A + (B + C)
[Associative Property]
 Matrix Multiplication
o A(BC) = (AB)C [Associative Property]
o A(B + C) = AB + AC [Left Distributive]
≠ BA + CA
o (A + B) C = AC + BC [Right Distributive]
o AI = IA = A [Multiplicative Identity]
 Scalar Multiplication
o ФxA=O
o 1xA=A
o kI(A) = k(IA) = I(kA)
o (k + I)A = kA + IA
o k(A + B) = kA + kB
*where Ф = zero (scalar)