Scribd
Upload
51 views23 pages

Declectures Non Linear Equations

This document describes Newton's method for solving nonlinear algebraic equations and systems of equations. It begins with an example of using Newton's method to find the roots of a single equation. It then extends the method to systems of equations by taking the Jacobian matrix of partial derivatives. The document proves that Newton's method converges quadratically close to an isolated root. It also gives an example of applying Newton's method to a simple system. Finally, it discusses techniques such as line search that can improve the reliability of Newton's method.

Uploaded by

Adeniji Olusegun
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
51 views23 pages

Declectures Non Linear Equations

This document describes Newton's method for solving nonlinear algebraic equations and systems of equations. It begins with an example of using Newton's method to find the roots of a single equation. It then extends the method to systems of equations by taking the Jacobian matrix of partial derivatives. The document proves that Newton's method converges quadratically close to an isolated root. It also gives an example of applying Newton's method to a simple system. Finally, it discusses techniques such as line search that can improve the reliability of Newton's method.

Uploaded by

Adeniji Olusegun
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 23

November 2001 10.

001 Introduction to Computer


Methods
Nonlinear Algebraic Equations Example
(in)
s i
(in) (in)
p,i
r
Continuous Stirred Tank Reactor (CSTR).
Look for steady state concentrations & temperature.
: N spieces with concentrations c ,
heat capacities c and temperature T
: N reactions
In
Inside
,i
(in)
s i
p,i
with stoichiometric coefficients
and reaction constants r .
: N spieces with concentrations c
(c-s may be equal to zero),
heat capacities c and temperature T.

Out
November 2001 10.001 Introduction to Computer
Methods
Nonlinear Algebraic Equations Example
r
s r
s
N
(in)
i i ,i s
1
s
N N
(in) (in) (in)
i p,i i p,i
i 1 1
T
1 2 N
(F/ V) c c r 0 i 1, 2,..., N
Massbalancefor spieces1, 2,..., N
(F/ V) c c T c c T H r 0
Energybalance
Column of unknownvariables :x [c , c ,..., c , T]


1
+
]
1

]


November 2001 10.001 Introduction to Computer
Methods
Nonlinear Algebraic Equations Example
i 1 2 N
1 1 2 N
2 1 2 N
N 1 2 N
Each of the above equations may be written in the general form:
f (x , x ,..., x ) 0:
f (x , x ,..., x ) 0
f (x , x ,..., x ) 0
...
f (x , x ,..., x ) 0
Let x be the solution staisfying f(x)=0.
We do not know x and

[0]
take x as initial guess.
1
2
N
In vector form:
f(x)=0
f (x)
f (x)
f(x)=
...
f (x)
1
1
1
1
1
1
]
November 2001 10.001 Introduction to Computer
Methods
Nonlinear Algebraic Equations
[1] [2] [3]
[m]
m
[m]
m
We need to form a sequence of estimates to the solution:
x , x , x ,... that will hopehully converge to x.
Thus we want: lim x x
lim x x 0
Unlike with linear equations, we cant say much


about
existence or uniqueness of solutions, even for
a single equation.
November 2001 10.001 Introduction to Computer
Methods
Single Nonlinear Equation
2 3
[0]
We assume f(x) is infinitely differentiable at the solution x.
Then f(x) may be Taylor expanded around x :
1 1
f (x) f (x) (x x)f (x) (x x) f (x) (x x) f (x) ...
2 3!
Assume x being close to x so t

+ + + +
[0] [0] [0]
[0]
[0] [0] [1] [0] [1] [0]
[0]
3 2
hat the series may be truncated:
f (x ) (x x)f (x ), or
f (x )
f (x ) (x x )f (x ) x x
f (x )
Newton s method
Example : f (x) (x 3)(x 2)(x 1) x 6x 11x 6

+
November 2001 10.001 Introduction to Computer
Methods
Single Nonlinear Equation
November 2001 10.001 Introduction to Computer
Methods
Single Nonlinear Equation
November 2001 10.001 Introduction to Computer
Methods
Single Nonlinear Equation
For f(x)=(x-3)(x-2)(x-1)=x
3
-6x
2
+11x+6=0, we se that
Newtons method converges to the root at x=2 only if
1.6<x
[0]
<2.4.
We may look at the direction of the first step and see
why: x
[1]
- x
[0]
=-f(x
[0]
)/ f

(x
[0]
).
So, we have 2 easy roots: x=1, x=3 and a more
difficult one.
We may factorize out the roots we already know.
November 2001 10.001 Introduction to Computer
Methods
Single Nonlinear Equation

[i+1] [i]

f (x)
Introduce g(x)
(x 3)(x 1)
f (x) f (x) 1 1
g (x)
(x 3)(x 1) (x 3)(x 1) (x 3) (x 1)
We now use Newtons method to find the roots of g(x):
g(x)
x x get x 2.
g (x)


_
+


,

November 2001 10.001 Introduction to Computer
Methods
Systems of Nonlinear Equations
Lets extend the method to multiple equations:
f
1
(x
1
,x
2
,x,,x
N
)=0
f
2
(x
1
,x
2
,x,,x
N
)=0 => f(x)=0
.. Start from initial guess x
[0]
f
N
(x
1
,x
2
,x,,x
N
)=0
2
N N N
i i
j j k
i i j j k
j 1 j 1 k 1
j j k
[0]
As before expand each equation at the solution x with f( x )=0:
f (x) 1 f (x)
f (x) f (x) (x x ) (x x ) (x x ) ...
x 2 x x
Assume x is close to x and discard quadratic terms




+ + +


N
i
j
i j
j 1
j
:
f (x)
f (x) (x x )
x

November 2001 10.001 Introduction to Computer


Methods
Systems of Nonlinear Equations
i
ij
j
N
j
i ij j
j 1 i
Let s define the Jacobian matrix J( x ) with the elements:
f (x)
J (x)
x
Then our approximate expansion may be written as:
f (x) J (x)(x x ) J(x)(x x)
This gives us the linear system

_


,

:
f (x) J(x)(x x)


November 2001 10.001 Introduction to Computer
Methods
Systems of Nonlinear Equations
[i ] [i ] [i ] [i 1]
i
[i ] [i 1] [i ]
[i ]
f (x ) J(x )(x x ) J(x)(x x)
Note, Jacobian is evaluated at the position of
an old iteration, not at an unknown solution
Defining x x x , we rewrite the equation as
J(x )

+
+
_


,

[i ] [i ] [i ] [i ] [i ]
[i ] [0]
rel
[i ]
abs
x f (x ) or just J x f
The iterations are continued untill some convergence
criteria are met:
relative error f f
absolute error f

November 2001 10.001 Introduction to Computer


Methods
Systems of Nonlinear Equations
Newtons method does not always converge. If it does,
can we estimate the error?
Let our function f(x) be continuously differentiable in the
vicinity of x and the vector connecting x and x +p all lies
inside this vicinity.
x
x+p
x+sp
T
x sp
1
0
f
J(x sp)
x
f (x p) f (x) J(x sp) pds
use path integral along the line x - x+p, parametrized by s.
+

+ + +

November 2001 10.001 Introduction to Computer


Methods
Systems of Nonlinear Equations
[ ]
[ ]
1
0
1
0
Add and subtract J(x)p to RHS:
f (x p) f (x) J(x) p J(x sp) J(x) pds
In Newtons method we ignore the integral term and
choose p to estimate f(x+p). Thus the error in this case is:
J(x sp) J(x) pds f (x
+ + + +
+ +

p)
What is the upper bound on this error?
f(x) and J(x) are continuous:
J(x+sp)-J(x) 0 as p 0 for all 0 s 1.
November 2001 10.001 Introduction to Computer
Methods
Systems of Nonlinear Equations
[ ] [ ]
v 0
1 1
0 0
Av
The norm of the matrix is defined as: A max ,
v
Ay
so for any y A , or Ay A y , therefore
y
J(x sp) J(x) pds J(x sp) J(x) ds p
The error goes down at least as fast as p because for
a continuous Jaco


+ +

bian J(x+sp)-J(x) 0 as p 0.
November 2001 10.001 Introduction to Computer
Methods
Systems of Nonlinear Equations
[ ] [ ]
[ ]
1 1
0 0
1
0
If we suggest that there exist some L>0 such that
J(y)-J(z) L y z
or there is some upper bound on the "stretching" effect of J.
f (x sp) J(x sp) J(x) pds J(x sp) J(x) ds p
and J(x sp) J(x) ds L s p , so in t

+ + +
+

[ ] ( )
( )
( )
1
0
2 2
his case
f (x sp) J(x sp) J(x) pds L s p p
L s p O p
+ +

November 2001 10.001 Introduction to Computer


Methods
Systems of Nonlinear Equations
Thus if we are at distance p from the solution, our error
scales as ||p||
2
.
What about convergence? How the error scales with
the number of iterations?
The answer is:
2
[i 1] [i]
x x O x x - local quadratic convergence,
a very fast one! Works when you are close enough to
the solution.

+
_



,
November 2001 10.001 Introduction to Computer
Methods
Systems of Nonlinear Equations
[i] 1
[i 1] 2
[i 2] 4
[i 3] 8
x x 0.1 10
x x 0.01 10
x x 0.00001 10
x x 0.000000001 10
Works if we are close enough to an isolated
(non-singular) solution: det J( x ) 0.

1
]
:
:
:
:
November 2001 10.001 Introduction to Computer
Methods
Systems of Nonlinear Equations, Example
1
]
1

1
1
1
1
]
1

1
]
1

+
+

2
2 1
2
2
1
2
2
1
2
2
1
1
1
3
2
2
1 2
2
2
3
1 1
x 3 x 8
x 8 x 9
x
f
x
f
x
f
x
f
J
: is Jacobian The
4
3
x
0 28 x x 4 f
0 145 x 4 x 3 f
: system simple a for works
method s Newton how examine s Let
November 2001 10.001 Introduction to Computer
Methods
Systems of Nonlinear Equations, Example
( )
( )
( ) ( )
( ) ( )
] i [ ] i [ ] 1 i [ ] i [ ] i [ ] i [
3
] i [
2
2
] i [
1
2
] i [
2
3
] i [
1
] i [
2
] i [
2
] i [
1
] i [
2
2
] i [
1
] i [
x x x f x J
28 x x 4
145 x 4 x 3
f
x 3 x 8
x 8 x 9
J
: solve to
system following the have we step each At
+
1
1
]
1

+
+

1
1
]
1

+
November 2001 10.001 Introduction to Computer
Methods
Systems of Nonlinear Equations, Example
{ }
10
bs
abs 2 1
10
f , f max f criterion
e convergenc with the method s Newton
of e performanc examine us Let

<

November 2001 10.001 Introduction to Computer


Methods
Newtons Method
Newtons method works well close to the solution,
but otherwise takes large erratic steps, shows poor
performance and reliability.
Lets try to avoid such large steps: employ reduced
step Newtons algorithm.
Full Newtons step gives x
[i+1]
=x
[i]
+p
[i]
. Well use
only fraction of of the step: x
[i+1]
=x
[i]
+
i
p
[i]
, 0<
i
<1.
November 2001 10.001 Introduction to Computer
Methods
Newtons Method
How do we choose
i
?
Simplest way - weak line search:
- start with
i
=2
-m
for m=0,1,2,
As we reduce the value of
i
bya a factor of 2 at
each step, we accept the first one that satisfies a
descent criterion:
( ) ( )
] i [ ] i [
i
] i [
x f p x f < +
It can be proven that if the Jacobian is not singular,
the correct solution will be found by the reduced
step Newtons algorithm.

You might also like