NMO Topic1
NMO Topic1
Optimization
Topic 1:
Introduction to Numerical Methods
Lectures 1-4:
1
Lecture 1
Introduction to Numerical Methods
2
Numerical Methods
Numerical Methods:
Algorithms that are used to obtain numerical
solutions of a mathematical problem.
Why do we need them?
1. No analytical solution exists,
2. An analytical solution is difficult to obtain
or not practical.
3
What do we need?
Basic Needs in the Numerical Methods:
Practical:
Can be computed in a reasonable amount of time.
Accurate:
Good approximate to the true value,
Information about the approximation error
(Bounds, error order,… ).
4
Outlines of the Course
Number Representation
Approximate solution of nonlinear Equations
Solution of linear Equations (Direct methods)
Solution of linear Equations (Iterative methods)
Polynomial Interpolation
Least Squares approximation
Numerical Integration
5
Solution of Nonlinear Equations
Some simple equations can be solved analytically:
x2 4x 3 0
4 4 2 4(1)(3)
Analytic solution roots
2(1)
x 1 and x 3
x 9 2 x 2 5 0
x No analytic solution
xe
6
Methods for Solving Nonlinear Equations
o Bisection Method
o Newton-Raphson Method
o Secant Method
7
Solution of Systems of Linear Equations
x1 x2 3
x1 2 x2 5
We can solve it as :
x1 3 x2 , 3 x2 2 x2 5
x2 2, x1 3 2 1
What to do if we have
1000 equations in 1000 unknowns.
8
Cramer’s Rule is Not Practical
Cramer' s Rule can be used to solve the system :
3 1 1 3
5 2 1 5
x1 1, x2 2
1 1 1 1
1 2 1 2
10
Curve Fitting
Given a set of data:
x 0 1 2
y 0.5 10.3 21.3
11
Polynomial Interpolation
Given a set of data:
xi 0 1 2
yi 0.5 10.3 15.3
yi P( xi ) if xi is in the table
12
Methods for Curve Fitting
o Least Squares
o Linear Regression
o Nonlinear Least Squares Problems
o Interpolation
o Newton Polynomial Interpolation
o Lagrange Interpolation
13
Integration
Some functions can be integrated
analytically:
3 3
1 2 9 1
1 xdx 2 x 1 2 2 4
But many functions have no analytical solutions :
a
e
x2
dx ?
0
14
Methods for Numerical Integration
o Upper and Lower Sums
o Trapezoid Method
o Romberg Method
o Gauss Quadrature
15
Summary
Topics Covered in the Course
Numerical Methods:
Algorithms that are
Solution of Nonlinear Equations
used to obtain Solution of Linear Equations
numerical solution of a
Curve Fitting
mathematical problem.
We need them when Least Squares
No analytical solution Interpolation
exists or it is difficult Numerical Integration
to obtain it.
16
Lecture 2
Number Representation and Accuracy
Number Representation
Normalized Floating Point Representation
Significant Digits
Accuracy and Precision
Rounding and Chopping
17
Representing Real Numbers
You are familiar with the decimal system:
Standard Representations:
3 1 2 . 4 5
sign integral fraction
part part
18
Normalized Floating Point Representation
Normalized Floating Point Representation:
d . f1 f 2 f 3 f 4 10 n
sign mantissa exponent
d 0, n : signed exponent
Scientific Notation: Exactly one non-zero digit appears
before decimal point.
Advantage: Efficient in representing very small or very
large numbers.
19
Binary System
1. f1 f 2 f 3 f 4 2 n
sign mantissa signed exponent
20
Fact
Numbers that have a finite expansion in one numbering
system may have an infinite expansion in another
numbering system:
(1.1)10 (1.000110011001100...) 2
You can never represent 1.1 exactly in binary system.
21
IEEE 754 Floating-Point Standard
Single Precision (32-bit representation)
1-bit Sign + 8-bit Exponent + 23-bit Fraction
S Exponent8 Fraction23
S Exponent11 Fraction52
(continued)
22
Significant Digits
23
Remarks
24
Calculator Example
Suppose you want to compute:
3.578 * 2.139
using a calculator with two-digit fractions
25
Significant Digits - Example
48.9
26
Accuracy and Precision
27
28
Rounding and Chopping
29
Rounding and Chopping
30
Error Definitions – True Error
Can be computed if the true value is known:
31
Error Definitions – Estimated Error
When the true value is not known:
32
Notation
We say that the estimate is correct to n
decimal digits if:
n
Error 10
33
Summary
Number Representation
Numbers that have a finite expansion in one numbering system
may have an infinite expansion in another numbering system.
34