Scribd
Upload
103 views2 pages

Broyden

Broyden's method is an iterative method for finding the root of a function f(x) = 0 in multiple dimensions. It approximates the Jacobian matrix J(x) at each iteration when the true Jacobian is unavailable. The method chooses the approximate Jacobian Ak by minimizing the change from the previous iteration, subject to satisfying the secant equation. This results in an update formula for Ak that modifies Ak-1 by a rank-one matrix involving the residual yk-1 and the step sk-1.

Uploaded by

Matthieu Grigoresco
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
103 views2 pages

Broyden

Broyden's method is an iterative method for finding the root of a function f(x) = 0 in multiple dimensions. It approximates the Jacobian matrix J(x) at each iteration when the true Jacobian is unavailable. The method chooses the approximate Jacobian Ak by minimizing the change from the previous iteration, subject to satisfying the secant equation. This results in an update formula for Ak that modifies Ak-1 by a rank-one matrix involving the residual yk-1 and the step sk-1.

Uploaded by

Matthieu Grigoresco
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

MATHS 770: Advanced Numerical Analysis

Broyden’s Method
Lecture notes
Exercise
Show a rank one matrix has the form
uv T

where u, v ∈ Rn .

Introduction to Broyden’s Method


Suppose you are solving
f (x) = 0

where f : Rn → Rn (n > 1) using an iterative scheme.


Most schemes for this problem approximate f (x) about xk (the approximation to the solution on
the k-th iteration) by
Mk (x) = fk + Ak (x − xk )

where fk = f (xk ).
Question: what do we use for Ak ?
The obvious answer is J(xk ) (the Jacobian of f (x) evaluated at x = xk ).
If the Jacobian isn’t available, we could at least require M (xk−1 ) = fk−1 i.e. fk + Ak (xk−1 − xk ) =
fk−1 . Re-arranging gives
Ak (xk − xk−1 ) = fk − fk−1

This is called the Secant equation and is analogous to the equation for the Secant method in one
dimension.
Write as
Ak sk−1 = yk−1

This is a system of n equations for the elements of Ak . Since Ak has n2 elements in general, this
system of equations does not uniquely specify Ak .
One way of imposing uniqueness is to choose Ak by minimising as much as possible the change
in M (x) from one iteration to the next, subject to satisfying the Secant equation.
We have

Mk (x) − Mk−1 (x) = fk + Ak (x − xk ) − (fk−1 + Ak−1 (x − xk−1 ))

1
= fk − fk−1 − Ak (xk − xk−1 ) + (Ak − Ak−1 )(x − xk−1 )
= (Ak − Ak−1 )(x − xk−1 )

For any x ∈ Rn , we have


x − xk−1 = αsk−1 + t

where tT sk−1 = 0. Then (Ak − Ak−1 )(x − xk−1 ) becomes

α(Ak − Ak−1 )sk−1 + (Ak − Ak−1 )t

We have no control over the first term since (Ak − Ak−1 )sk−1 = yk−1 − Ak−1 sk−1 . We can make the
second term zero for all x ∈ Rn by choosing Ak such that (Ak − Ak−1 )t = 0 for all t orthogonal to
sk−1 .
This requires
Ak − Ak−1 = usTk−1

where u ∈ Rn .
Choose u so the Secant equation is satisfied. Get

(yk−1 − Ak−1 sk−1 )sTk−1


Ak = Ak−1 +
sTk−1 sk−1

Exercise
Explain in words what the above update formula means.

You might also like