NUMERICAL METHODS

Bahar Ayhan, Ph.D.

Civil Engineering Department
Office: 352 (1st floor)
E-mail : [email protected]
Phone: +90 212 285 7410
Numerical Methods
• CONTENTS
IV. Solving Nonlinear Equations ( Finding Roots)
• Background
• Estimation of Errors in Numerical Solutions
• Bracketing Methods
❑Bisection Method

❑Regula Falsi Method

• Open Methods
❑Newton’s Method

❑Secant Method

❑Fixed-Point Method

Dr. Bahar Ayhan – [email protected]

Civil Eng. Dept. Office:352
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

A scheme for finding a numerical solution of an equation of the form f ( x) = 0

f ( x ) is continuous, differentiable and has a solution near a given point.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

• The solution starts by choosing point

x1 as the first estimate of the soluiton.
• The second estimate x2 is obtained by
taking the tangent line to f(x) at the
point (x1,f(x1)) and finding the
intersection point of the tangent line
with x-axis.
• The third estimate x3 is the
intersection of the tangent line to f(x)
at the point (x2,f(x2)) with the x-axis.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

 Mathematically, for the first iteration, the slope of the

tangent at point (x1,f(x1))
f ( x1 ) − 0
f ( x1 ) =
x1 − x2

f ( x1 )
Solving the equation above for x2 gives: x2 = x1 −
f ( x1 )

General form of the equation for determining the next solution xi+1 from the present
solution xi :
f ( xi )
x =x − i +1
f ( xi )
i
(General iteration formula for Newton’s method)
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

Newton’s method can also be derived by using Taylor series.

Taylor series expansion of f(x) about x1 is given by:

f ( x ) = f ( x1 ) + ( x − x1 ) f ( x1 ) + 2!1 ( x − x1 ) 2 f ( x1 ) + ...

If x2 is a solution of the equation f(x)=0 and x1 is a point near x2, then :

f ( x2 ) = f ( x1 ) + ( x2 − x1 ) f ( x1 ) + 2! ( x2 − x1 ) f ( x1 ) + ...
 1 2

By considering only the first two terms of the series, an f ( x1 )

approximate solution can be determined by solving the x2 = x1 −
equation above f ( x1 ) Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

Algorithm for Newton’s method

1. Choose a point x1 as an initial guess of the solution.
2. For i=1,2... Until the error is smaller than a specified value, calculate xi+1 by using
equation:

f ( xi )
xi +1 = xi −
f ( xi )

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

When are the iterations stopped ?

 The iterations should be stopped when an exact solution is obtained, f(x)=0
>>> IDEAL but not possible (exact solution cannot be found computationally)
 The iterations are stopped when an estimated error is smaller than some
predetermined value.
 Two error estimates that are typically used with Newton’s method are:
1. Estimated relative error
2. Tolerance in f(x)

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

1. Estimated relative error

The iterations are stopped when the estimated relative error is smaller than a specified value ε

xi +1 − xi

xi
2. Tolerance in f(x)
The iterations are stopped when the absolute value of f(xi) is smaller than a some number δ

f ( xi )  
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

Notes on Newton’s Method

When it does not converge, it is usually because the starting point is not close
enough to the solution.
Convergence problems occur when the value of 𝑓′ 𝑥 is close to zero in the vicinity
of the solution (where f(x)=0)
The method converges
▪ ifthe function 𝑓 𝑥 and its first 𝑓′ 𝑥 and second 𝑓"(𝑥) derivatives are
continuous.
▪ if 𝑓 ′ 𝑥 is not zero at the solution and
▪ if the starting value x1 is near the actual solution.
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

The method converges

▪ if the function 𝑓 𝑥 and its first 𝑓′ 𝑥 and second 𝑓"(𝑥) derivatives are
continuous.
▪ if 𝑓 ′ 𝑥 is not zero at the solution and
▪ if the starting value x1 is near the actual solution.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Newton’s Method (Newton-Raphson Method)

A function 𝑓 ′ 𝑥 which is derivative of the function 𝑓 𝑥 has to be substituted in the

iteration formula.
In many cases,
▪ It is simple to write the derivative, but it can be difficult to determine.
▪ When an expression for the derivative is not available, it might be possible to determine
the slope numerically or to find a solution by using the Secant method.
▪ Secant method is similar to Newton’s method but does not require an expression for the
derivative.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Secant Method

 The Secant Method is a scheme for finding a numerical solution of an equation of the form f(x) =0.

The method uses two points in the neighborhood of the solution to determine a new estimate for the solution.

The two points (marked as x1 and x2 in the figure) are used to define a straight line (secant line) and the point
where the line intersects the x-axis (marked as x3 in the figure) is the new estimate for the solution.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Secant Method

 The two points can be on one side

of the solution or the solution can
be between the two points.

 The slope of the secant line is given by:

f ( x1 ) − f ( x2 ) f ( x2 ) − 0 f ( x2 ) ( x1 − x2 )
= x3 = x2 −
x1 − x2 x2 − x3 f ( x1 ) − f ( x2 )

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Secant Method

 Once point x3 is determined, it is used together with

point x2 to calculate the next estimate of the solution,
x4 .

 Equation can be generalized to an iteration

formula in which a new estimate of the solution
xi+1 is determined from the previous two
solutions xi and xi-1

f ( x2 ) ( x1 − x2 ) f ( xi ) ( xi −1 − xi )
x3 = x2 − xi +1 = xi −
f ( x1 ) − f ( x2 ) f ( xi −1 ) − f ( xi )

Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Secant Method

 Relationship to Newton’s Method

In secant method, when two points that define the secant line are close to each other, the
method is an approximated form of Newton’s method.

f ( xi )
f ( xi ) ( xi −1 − xi ) xi +1 = xi −
xi +1 = xi − f ( xi −1 ) − f ( xi )
f ( xi −1 ) − f ( xi ) It is not necessary to know the
( xi −1 − xi ) analytical form of f`(x) in Secant
method (unlike Newton’s method)

f ( xi )
xi +1 = xi −
f ( xi ) Newton’s Method equation

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

 The method is a scheme for solving an equation of the form f(x) =0.
 The method is carried out by rewriting the equation: x = g ( x )

 When x is the solution of f(x)=0, the

left side and the right side of the
equation 𝑥 = 𝑔 𝑥 are equal.

 The point of intersection of the two

plots, called the FIXED POINT, is
the solution.

Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

 The numerical value of the solution is determined by an iterative process.

 It starts by taking a value of x near the fixed point as the first guess for the solution and
substituting it in g(x).
 The value of g(x) that is obtained is the new(second) estimate for the solution.
 The second value is then substituted back in g(x), which then gives the third estimate of
the solution.
 The iteration formula is given by:

xi +1 = g ( xi )
The function g(x) is called the iteration function.
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

 The values of x that are obtained are successive iterations that progressively converge
toward the solution.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

The process of finding a solution

 Choose point 𝑥1 on the x-axis and draw a
vertical line that intersects the curve 𝑦 = 𝑔 𝑥
at point 𝑔(𝑥1 )
 Since 𝑥2 = 𝑔(𝑥1 ), a horizontal line is drawn
from point 𝑥1 , 𝑔 𝑥1 toward line y=x. The
intersection point gives the location of 𝑥2 .

 From 𝑥2 a vertical line is drawn toward the curve 𝑦 = 𝑔 𝑥 . The intersection point is
now 𝑥2 , 𝑔 𝑥2 and the value 𝑔 𝑥2 = 𝑥3 .From point 𝑥2 , 𝑔 𝑥2 a horizontal line is
drawn again toward y=x and the intersection point gives the location of 𝑥3
 As the process continues the intersection points converge toward the fixed point > the true solution xTS.
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

 It is possible that the iterations will not converge toward the fixed point >> diverge
away.
 Even though the starting point is close to the solution, subsequent points are moving
away from the solution.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

 The form f(x)=0 does NOT lend itself to deriving an iteration formula of the
form 𝑥 = 𝑔(𝑥).
 In such a case, one can always add and substract 𝑥 to 𝑓 𝑥 to obtain
𝑥 + 𝑓 𝑥 − 𝑥 = 0.
 The equation can be rewritten in the form that can be used in the fixed-point
iteration method.

x = x + f ( x) = g ( x)

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

Choosing the appropriate iteration function g(x)

Several iteration functions g(x) can be written for the same equation.
 A g(x) in the equation xi +1 = g ( xi ) for the iteration process is one for which
the iterations converge toward the solution.

 There might be more than one form that can be used, or it may be that none of the
forms are appropriate so that the fixed-point iteration method cannot be used to solve
the equation.
 In cases where there are multiple solutions, one iteration function may yield one root,
while a different function yields other roots.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

Choosing the appropriate iteration function g(x)

 The fixed-point iteration method converges if, in the neighborhood of the fixed point,
the derivatives of g(x) has an absolute value that is smaller than 1 (Lipschitz continuous)

g ( x )  1

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

Example Consider the equation given below.

xe 0.5 x
+ 1.2 x − 5 = 0
A plot of the function shows that the equation has a
solution between x=1 and x=2.

 Equation can be written in the form x=g(x) in different

ways. Three possibilities are discussed.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

xe0.5 x + 1.2 x − 5 = 0

Case a: 5 − xe 0.5 x 5 − xe 0.5 x

x= g ( x) =
1.2 1.2
e 0.5 x + 0.5 xe 0.5 x
g ( x ) = −
1.2
e (0.5)(1) + (0.5)(1)(e (0.5)(1) )
g (1) = − = −2.0609
1.2
e (0.5)(2) + (0.5)(2)(e (0.5)(2) )
g (2) = − = −4.5305
1.2
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

xe0.5 x + 1.2 x − 5 = 0

Case b : 5 5
x = 0.5 x g ( x ) = 0.5 x
e + 1.2 e + 1.2
−5e0.5 x
g ( x ) =
2 (e + 1.2 )
2
0.5 x

−5e(0.5)(1)
g (1) = = −0.5079
2 (e + 1.2 )
2
(0.5)(1)

−5e(0.5)(2)
g (2) = = −0.4426
2 (e + 1.2 )
2
(0.5)(2)
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

xe0.5 x + 1.2 x − 5 = 0
Case c: 5 − 1.2 x
5 − 1.2 x
x= g ( x) =
e 0.5 x e0.5 x
−3.7 + 0.6 x
g ( x) =
e0.5 x
−3.7 + (0.6)(1)
g (1) = (0.5)(1)
= −1,8802
e
−3.7 + (0.6)(2)
g (2) = (0.5)(2)
= −0.9197
e
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

g ( x )  1
 These results show that the iteration function g(x) from Case b is the one should be used !

5
xi +1 = 0.5 xi
e + 1.2

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

5
xi +1 = 0.5 xi Starting with x=1...
e + 1.2

5 5
x2 = = 1.7552 x5 = = 1.4776
e (0.5)(1)
+ 1.2 e (0.5)(1.5622)
+ 1.2
5 5
x3 = = 1.3869 x6 = = 1.5182
e (0.5)(1.7552)
+ 1.2 e (0.5)(1.4776)
+ 1.2
5 5
x4 = = 1.5622 x7 = = 1.4986
e (0.5)(1.3869)
+ 1.2 e (0.5)(1.5182)
+ 1.2
The values calculated in the iterations are converging toward the actual solution x=1.5050
Bahar
Ayhan
Numerical Methods
Civil Engineering Department
IV. Solving Nonlinear Equations: Open Methods >> Fixed-Point Iteration Method

When should the iterations be stopped ?

 As with Newton’s method, the iterations should be stopped either when
the relative error or the tolerance in f(x) is smaller than some
predetermined value.

Bahar
Ayhan
Numerical Methods
Civil Engineering Department
 Numerical Methods

Thank you for your attention !!

Bahar
Ayhan

Numerical Methods
Civil Engineering Department