NMO Topic2 1
NMO Topic2 1
Optimization
Topic 2:
Solution of Nonlinear Equations
Lectures 5-11:
1
Lecture 5
Solution of Nonlinear Equations
( Root Finding Problems )
Definitions
Classification of Methods
Analytical Solutions
Graphical Methods
Numerical Methods
Bracketing Methods
Open Methods
Convergence Notations
3
Roots of Equations
A number r that satisfies an equation is called a
root of the equation.
The equation : x 4 3x 3 7 x 2 15 x 18
has four roots : 2, 3, 3 , and 1 .
i.e., x 4 3x 3 7 x 2 15 x 18 ( x 2)( x 3) 2 ( x 1)
The equation has two simple roots (1 and 2)
and a repeated root (3) with multiplici ty 2.
4
Zeros of a Function
Let f(x) be a real-valued function of a real
variable. Any number r for which f(r)=0 is
called a zero of the function.
Examples:
2 and 3 are zeros of the function f(x) = (x-2)(x-3).
5
Graphical Interpretation of Zeros
6
Simple Zeros
f ( x) x 1( x 2)
f ( x) ( x 1)x 2 x x 2
2
f ( x) x 1
2
f ( x) x 1 x 2 x 1
2 2
f ( x) x
3
10
Roots of Equations & Zeros of Function
Given the equation :
x 4 3 x 3 7 x 2 15 x 18
Move all terms to one side of the equation :
x 4 3 x 3 7 x 2 15 x 18 0
Define f ( x) as :
f ( x) x 4 3 x 3 7 x 2 15 x 18
11
Solution Methods
Several ways to solve nonlinear equations are
possible:
Analytical Solutions
Possible for special equations only
Graphical Solutions
Useful for providing initial guesses for other
methods
Numerical Solutions
Open methods
Bracketing methods
12
Analytical Methods
Analytical Solutions are available for special
equations only.
Analytical solution of : a x 2 b x c 0
b b 2 4ac
roots
2a
13
Graphical Methods
Graphical methods are useful to provide an
initial guess to be used by other methods.
x
Solve e
2 Root
x
xe x
The root [0,1] 1
root 0.6
1 2
14
Numerical Methods
Many methods are available to solve nonlinear
equations:
Bisection Method
Newton’s Method These will be
Secant Method covered in CISE301
False position Method
Muller’s Method
Bairstow’s Method
Fixed point iterations
……….
15
Bracketing Methods
In bracketing methods, the method starts
with an interval that contains the root and
a procedure is used to obtain a smaller
interval containing the root.
16
Open Methods
In the open methods, the method starts
with one or more initial guess points. In
each iteration, a new guess of the root is
obtained.
Open methods are usually more efficient
than bracketing methods.
They may not converge to a root.
17
Convergence Notation
A sequence x1 , x2 ,..., xn ,... is said to converge to x if
to every 0 there exists N such that :
xn x n N
18
Convergence Notation
Let x1 , x2 ,...., converge to x.
xn 1 x
Linear Convergenc e : C
xn x
xn 1 x
Quadratic Convergenc e : C
xn x
2
xn 1 x
Convergenc e of order P : C
xn x
p
19
Speed of Convergence
We can compare different methods in
terms of their convergence rate.
Quadratic convergence is faster than
linear convergence.
A method with convergence order q
converges faster than a method with
convergence order p if q>p.
Methods of convergence order p>1 are
said to have super linear convergence.
20
Lectures 6-7
Bisection Method
The Bisection Algorithm
Convergence Analysis of Bisection Method
Examples
21
Introduction
The Bisection method is one of the simplest
methods to find a zero of a nonlinear function.
It is also called interval halving method.
To use the Bisection method, one needs an initial
interval that is known to contain a zero of the
function.
The method systematically reduces the interval.
It does this by dividing the interval into two equal
parts, performs a simple test and based on the
result of the test, half of the interval is thrown
away.
The procedure is repeated until the desired
interval size is obtained.
22
Intermediate Value Theorem
Let f(x) be defined on the
interval [a,b].
f(a)
Intermediate value theorem:
if a function is continuous
and f(a) and f(b) have a b
different signs then the
function has at least one zero f(b)
in the interval [a,b].
23
Examples
If f(a) and f(b) have the
same sign, the function
may have an even
number of real zeros or a b
no real zeros in the
interval [a, b].
The function has four real zeros
Bisection method can
not be used in these
cases.
a b
The function has no real zeros
24
Two More Examples
If f(a) and f(b) have
different signs, the
function has at least
one real zero.
a b
25
Bisection Method
If the function is continuous on [a,b] and
f(a) and f(b) have different signs,
Bisection method obtains a new interval
that is half of the current interval and the
sign of the function at the end points of
the interval are different.
26
Bisection Method
Assumptions:
Given an interval [a,b]
f(x) is continuous on [a,b]
f(a) and f(b) have opposite signs.
27
Bisection Algorithm
Assumptions:
f(x) is continuous on [a,b]
f(a) f(b) < 0
f(a)
Algorithm:
Loop
1. Compute the mid point c=(a+b)/2
c b
2. Evaluate f(c) a
3. If f(a) f(c) < 0 then new interval [a, c]
If f(a) f(c) > 0 then new interval [c, b] f(b)
End loop
28
Bisection Method
b0
a0 a1 a2
29
Example
+ + -
+ - -
+ + -
30
Flow Chart of Bisection Method
Start: Given a,b and ε
u = f(a) ; v = f(b)
c = (a+b) /2 ; w = f(c) no
yes
is no is
Stop
yes (b-a) /2<ε
u w <0
b=c; v= w a=c; u= w
31
Example
Can you use Bisection method to find a zero of :
f ( x) x 3 3x 1 in the interval [0,2]?
Answer:
f ( x) is continuous on [0,2]
and f(0) * f(2) (1)(3) 3 0
Assumption s are not satisfied
Bisection method can not be used
32
Example
Can you use Bisection method to find a zero of :
f ( x) x 3 3x 1 in the interval [0,1]?
Answer:
f ( x) is continuous on [0,1]
and f(0) * f(1) (1)(-1) 1 0
Assumption s are satisfied
Bisection method can be used
33
Best Estimate and Error Level
Bisection method obtains an interval that
is guaranteed to contain a zero of the
function.
Questions:
What is the best estimate of the zero of f(x)?
What is the error level in the obtained estimate?
34
Best Estimate and Error Level
The best estimate of the zero of the
function f(x) after the first iteration of the
Bisection method is the mid point of the
initial interval:
ba
Estimate of the zero : r
2
ba
Error
2
35
Stopping Criteria
Two common stopping criteria
36
Stopping Criteria
cn : is the midpoint of the interval at the n th iteration
( cn is usually used as the estimate of the root).
r: is the zero of the function.
After n iterations :
b a x 0
error r - cn Ean n n
2 2
37
Convergence Analysis
Given f ( x), a, b, and
How many iterations are needed such that : x - r
where r is the zero of f(x) and x is the
bisection estimate (i.e., x ck ) ?
log( b a ) log( )
n
log( 2)
38
Convergence Analysis – Alternative Form
log( b a ) log( )
n
log( 2)
39
Example
a 6, b 7, 0.0005
How many iterations are needed such that : x - r ?
n 11
40
Example
Use Bisection method to find a root of the
equation x = cos (x) with absolute error <0.02
(assume the initial interval [0.5, 0.9])
41
CISE301_Topic2 42
Bisection Method
Initial Interval
43
Bisection Method
44
Bisection Method
45
Summary
Initial interval containing the root:
[0.5,0.9]
After 5 iterations:
Interval containing the root: [0.725, 0.75]
Best estimate of the root is 0.7375
| Error | < 0.0125
46
A Matlab Program of Bisection Method
a=.5; b=.9; c=
u=a-cos(a); 0.7000
fc =
v=b-cos(b);
-0.0648
for i=1:5
c=
c=(a+b)/2 0.8000
fc=c-cos(c) fc =
if u*fc<0 0.1033
b=c ; v=fc; c=
else 0.7500
a=c; u=fc; fc =
end 0.0183
end c=
0.7250
fc =
-0.0235
47
Example
Find the root of:
* f(x) is continuous
* f( 0 ) 1, f (1) 1 f (a ) f (b) 0
Bisection method can be used to find the root
48
Example
c= (a+b) (b-a)
Iteration a b f(c)
2 2
49
Bisection Method
Advantages
Simple and easy to implement
One function evaluation per iteration
The size of the interval containing the zero is reduced
by 50% after each iteration
The number of iterations can be determined a priori
No knowledge of the derivative is needed
The function does not have to be differentiable
Disadvantage
Slow to converge
Good intermediate approximations may be discarded
50