AMSC/CMSC 466 Tobias von Petersdorff

1 Fixed Point Iteration and Contraction Mapping Theorem

Notation: For two sets A, B we write A ⊂ B iff x ∈ A =⇒ x ∈ B. So A ⊂ A is true. Some people use the notation “⊆” instead.

1.1 Introduction

Consider a function y = g(x) where x, y ∈ Rn :

   
y1 g1 (x1 , . . . , xn )
 ..   ..
 . =

. 
yn gn (x1 , . . . , xn )

We assume that g(x) is defined for x ∈ D where D is a subset of Rn .

The goal is to find a solution x∗ of the fixed point equation

g(x) = x.

A method to find x∗ is the fixed point iteration: Pick an initial guess x(0) ∈ D and define for k = 0, 1, 2, . . .

x(k+1) := g(x(k) )

Note that this may not converge. But if the sequence x(k) converges, and the function g is continuous, the limit x∗ must be a
solution of the fixed point equation.

1.2 Contraction Mapping Theorem

The following theorem is called Contraction Mapping Theorem or Banach Fixed Point Theorem.

Theorem 1. Consider a set D ⊂ Rn and a function g : D → Rn . Assume

1. D is closed (i.e., it contains all limit points of sequences in D)
2. x ∈ D =⇒ g(x) ∈ D
3. The mapping g is a contraction on D: There exists q < 1 such that

∀x, y ∈ D : kg(x) − g(y)k ≤ q kx − yk (1)

Then
1. there exists a unique x∗ ∈ D with g(x∗ ) = x∗
2. for any x(0) ∈ D the fixed point iterates given by x(k+1) := g(x(k) ) converge to x∗ as k → ∞
3. x(k) satisfies the a-priori error estimate

qk
x(k) − x∗ ≤ x(1) − x(0) (2)
1−q
and the a-posteriori error estimate
q
x(k) − x∗ ≤ x(k) − x(k−1) (3)
1−q
AMSC/CMSC 466 Tobias von Petersdorff

Proof. Pick x(0) ∈ D and define x(k) for k = 1, 2, . . . by x(k) := g(x(k−1) ). We have from the contraction property (1)
x(k+1) − x(k) = g(x(k) ) − g(x(k−1) ) ≤ q x(k) − x(k−1) (4)
and hence
x(k+1) − x(k) ≤ qk x(1) − x(0) (5)
Let d := x(1) − x(0) . We have from the triangle inequality and (5)
x(k) − x(k+`) ≤ x(k) − x(k+1) + · · · + x(k+`−1) − x(k+`)
≤ qk d + · · · + qk+`−1 d = qk d (1 + q + · · · + q`−1 )
1
x(k) − x(k+`) ≤ qk d (6)
1−q
using the sum of the geometric series ∑`−1 j ∞ j
j=0 q ≤ ∑ j=0 q = 1/(1 − q). Note that (6) shows that the sequence x
(k) is a Cauchy

sequence. Therefore it must converge to a limit x∗ ∈ Rn (since the space Rn is complete). As D is closed, we must have
x∗ ∈ D.
We need to show that x∗ = g(x∗ ): We have x(k+1) = g(x(k) ), hence
lim x(k+1) = lim g(x(k) )
k→∞ k→∞
The limit of the left hand side is x∗ . Note that because of (1) the function g must be continuous. Therefore
lim g(x(k) ) = g( lim x(k) ) = g(x∗ ).
k→∞ k→∞
Next we need to show that the fixed point is unique. Assume that we have fixed points x∗ = g(x∗ ) and y∗ = g(y∗ ). Then
x∗
we obtain using the contraction property (1)
x∗ − y∗ = g(x∗ ) − g(y∗ ) ≤ q x∗ − y∗
implying (1 − q) x∗ − y∗ ≤ 0 and therefore x∗ − y∗ = 0, i.e., x∗ = y∗ .
The a-priori estimate (2) follows from (6) by letting ` tend to infinity. For the a-posteriori estimate use (2) with k = 1 for
x̃(0) := x(k) , x̃(1) = x(k+1) .

1.3 Proving the Contraction Property

The contraction property is related to the Jacobian g0 (x) which is an n × n matrix for each point x ∈ D. If the matrix norm
satisfies kg0 (x)k ≤ q < 1 then the mapping g must be a contraction:
∂gj
Theorem 2. Assume the set D ⊂ Rn is convex and the function g : D → Rn has continuous partial derivatives in D. If
∂k
for q < 1 the matrix norm of the Jacobian satisfies
∀x ∈ D : g0 (x) ≤ q (7)
the mapping g is a contraction in D and satisfies (1).

Proof. Let x, y ∈ D. Then the points on the straight line from

x to y are given by x + t(y − x) for t ∈ [0, 1]. As D is convex all
these points are contained in D. Let G(t) := g x + t(y − x) , then by the chain rule we have G0 (t) = g0 x + t(y − x) (y − x)

and
Z 1 Z 1
G0 (t)dt = g0 x + t(y − x) (y − x) dt


g(y) − g(x) = G(1) − G(0) =
0 0
Rb Rb
As an integral of a continuous function is a limit of Riemann sums the triangle inequality implies a F(t)dt ≤ a kF(t)k dt:
Z 1 Z 1
0
g0 x + t(y − x) ky − xk dt ≤ q ky − xk



kg(y) − g(x)k ≤ g x + t(y − x) (y − x)dt ≤
0 0 | {z }
≤q

This is usually the easiest method to prove that a given mapping g is a contraction, see the examples in sections 1.5, 1.6.
AMSC/CMSC 466 Tobias von Petersdorff

1.4 A-priori and a-posteriori error estimates

The error estimates (2), (3) are useful for figuring out how many iterations we need. For this we need to know the contraction
constant q (typically we get this from (7)).

A-priori estimate: For an initial guess x(0) we can find x(1) . Without computing anything else we then have the error
qk
bound x(k) − x∗ ≤ 1−q x(1) − x(0) for all future iterates x(k) , before (“a-priori”) we actually compute them. We can e.g.
use this to find a value k such that x(k) − x∗ is below a given tolerance.

A-posteriori estimate: After we have actually computed x(k) (“a-posteriori”) we would like to know where the true
solution x∗ is located. Let
q
δk := x(k) − x(k−1) , Dk := {x | x − x(k) ≤ δk }
1−q
The a-posteriori estimate states that x∗ is contained in the set Dk . Note:
• the “radius” δk of Dk decreases at least by a factor of q with each iteration: δk+1 ≤ qδk
• the sets Dk are nested: D1 ⊃ D2 ⊃ D3 ⊃ · · ·
To show Dk+1 ⊂ Dk assume x ∈ Dk+1 . Then

q
 (4) 1
(k) (k+1) (k+1) (k)
x−x ≤ x−x + x −x ≤ +1 x(k+1) − x(k) ≤ q x(k) − x(k−1) = δk (8)
| {z } 1−q 1−q
≤δk+1

If we use the ∞-norm: x(k) − x∗ ∞

≤ δk means that for each component x∗j we have a bracket
(k) (k)
x∗j ∈ [x j − δk , x j + δk ],

i.e., the set Dk is a square/cube/hypercube with side length 2δk centered in x(k) .

1.5 Example

We want to solve the nonlinear system

1
x1 = 10 [1 − x2 − sin(x1 + x2 )]
1
x2 = [2 + x1 + cos(x1 − x2 )]
10
 
1 1 − x2 − sin(x1 + x2 )
where we have g(x) = 10 .
2 + x1 + cos(x1 − x2 )
First we want to show that g is a contraction using Theorem 2. Therefore we first have to find the Jacobian g0 (x):
 
0 1 − cos(x1 + x2 ) −1 − cos(x1 + x2 )
g (x) = 10
1 − sin(x1 − x2 ) sin(x1 − x2 )
Let A := g0 (x). Let us use the ∞-norm. We need to find an upper bound for kAk∞ = max{|a11 | + |a12 | , |a21 | + |a22 |}. We
obtain for any x1 , x2 ∈ R
1 1 1 1
|a11 | = 10 |− cos(x1 + x2 )| ≤ 10 , |a12 | = 10 |−1 − cos(x1 + x2 )| ≤ 10 (1 + 1)
1 1 1 1
|a21 | = 10 |1 − sin(x1 − x2 )| ≤ 10 (1 + 1), |a22 | ≤ 10 |sin(x1 − x2 )| ≤ 10
Therefore for any x ∈ R2 we have
3
g0 (x)
= q < 1. ≤
10 ∞

By Theorem 2 we therefore obtain that g is a contraction for all of R2 .

We now want to use Theorem 1. We need to pick a set D such that the three assumptions of the theorem are satisfied. We
consider two choices:
AMSC/CMSC 466 Tobias von Petersdorff

First choice D = R2 : We can use the set D = R2 . This set is closed. For any x ∈ R2 we certainly have that g(x) ∈ R2 . We
have also shown that g is a contraction for all of R2 . Therefore we obtain from Theorem 1 that the nonlinear system g(x) = x
has exactly one solution x∗ in all of R2 .

Second choice D = [−1, 1] × [−1, 1]: We can use for D the square with −1 ≤ x1 ≤ 1 and −1 ≤ x2 ≤ 1. This is a closed
set (the boundary of the square is included). We now have to check that for x ∈ D we have that y = g(x) ∈ D: We have using
−1 ≤ sin α ≤ 1, −1 ≤ cos α ≤ 1
2 1 1 1 3
− 10 = 10 (1 − 1 − 1) ≤ y1 = 10 [1 − x2 − sin(x1 + x2 )] ≤ 10 (1 + 1 + 1) = 10
1 1 1 4
0= 10 (2 − 1 − 1) ≤ y2 = 10 [2 + x1 + cos(x1 − x2 )] ≤ 10 (2 + 1 + 1) = 10

therefore y ∈ D and the second assumption of the theorem is satisfied. We already showed that g is a contraction for all of
R2 , so the third assumption definitely holds for x, y ∈ D. We can now apply Theorem 1 and obtain that the nonlinear system
has exactly one solution x∗ which is located in the square D = [−1, 1] × [−1, 1].

Numerical Computation: We start with the initial guess x(0) = (0, 0)> . After each iteration we find δk and the square Dk
containing x∗ :
k x(k) δk Dk
1 (.1, .3)> 1.3 · 10−1 [−.02857, .2286] × [.1714, .4286]
2 (.03106, .3080)> 3.0 · 10−2 [.00151, .06060] × [.2785, .3376]
3 (.03594, .2993)> 3.7 · 10−3 [.03221, .03967] × [.2956, .3030]
4 (.03717, .3001)> 5.3 · 10−4 [.03664, .03770] × [.2996, .3007]
5 (.03689, .3003)> 1.2 · 10−4 [.03677, .03701] × [.3001, .3004]
Note: (i) δk decreases at least by a factor of q = 0.3 with each iteration.
(ii) The sets Dk are nested: D1 ⊃ D2 ⊃ D3 ⊃ · · ·

1.6 Using the Fixed Point Theorem without the Assumption g(D) ⊂ D

The tricky part in using the contraction mapping theorem is to find a set D for which both the 2nd and 3rd assumption of the
fixed point theorem hold:
• x ∈ D =⇒ g(x) ∈ D
• g is a contraction on D
Typically we can prove that kg0 (x)k ≤ q < 1 for x in some convex region D̃. We suspect that there is a solution x∗ of the fixed
point equation in D̃. But it may not be true that g(x) ∈ D̃ for all x ∈ D̃.
In this case we may be able to prove a result by computing a few iterates x(k) : Start with k = 0 and an initial guess x(0) ∈ D̃.
Then repeat
• let k := k + 1 and compute x(k) := g(x(k−1) )
q
• compute δk := x(k) − x(k−1) , let Dk := {x | x − x(k) ≤ δk }
1−q
until either Dk ⊂ D̃ or x(k) ∈
/ D̃.
If the iterates exit from the set D̃ we cannot conclude anything. But as long as the points x(k) stay inside D̃ we have δk+1 ≤ qδk
and Dk+1 ⊂ Dk . So we expect that for some k the condition Dk ⊂ D̃ will be satisfied (if x(k) converges to a limit in the interior
of D̃ the loop must terminate with Dk ⊂ D̃; but in general it is possible that the loop never terminates). If the loop does
terminate with Dk ⊂ D̃ for k = K we have the following result:
AMSC/CMSC 466 Tobias von Petersdorff

Theorem 3. Let D̃ ⊂ Rn and assume that the function g : D̃ → Rn satisfies for q < 1
∀x, y ∈ D̃ : kg(x) − g(y)k ≤ q kx − yk
Let x(0) ∈ D̃ and define for k = 0, 1, 2, . . .
q
x(k+1) := g(x(k) ), δk := x(k) − x(k−1) , Dk := {x | x − x(k) ≤ δk }
1−q
If for some K we have x(K−1) ∈ D̃ and DK ⊂ D̃ there holds
• the equation g(x) = x has a unique solution x∗ in D̃
• this solution satisfies x∗ ∈ Dk for all k ≥ K

Proof. Let x ∈ DK . We want to show that g(x) ∈ DK : As DK ⊂ D̃ the contraction property gives using the definition of Dk
and δk
g(x) − x(K) ≤ q x − x(K−1) ≤ q x − x(K) + q x(K) − x(K−1) ≤ qδK + (1 − q)δK = δK
As DK is closed and DK ⊂ D̃ the set D := DK satisfies all three assumptions of the fixed point theorem Theorem 1. Hence
there is a unique solution x∗ ∈ D. The a-posteriori estimate (3) states that x∗ ∈ Dk for all iterates x(k) with k ≥ K. Assume
that there is another fixed point y∗ ∈ D̃ with g(y∗ ) = y∗ . Then
ky∗ − x∗ k = kg(y∗ ) − g(x∗ )k ≤ q ky∗ − x∗ k
As q < 1 we must have ky∗ − x∗ k = 0.

Summary:
• Find a convex set D̃ for which you suspect x∗ ∈ D̃ and where you can show kg0 (x)k ≤ q < 1
• Pick x(0) ∈ D̃ and perform the fixed point iteration:
for each iteration:
– find x(k) and Dk
– if x(k) ∈
/ D̃: stop (we can’t conclude anything)
– if Dk ⊂ D̃: success: there is a unique solution x∗ ∈ D̃, and there holds x∗ ∈ Dk for this and all following iterations
   
1 x1 − x1 x2 + 1 0 1 1 − x2 −x1
Example: Let g(x) := 3 . Then the Jacobian is g (x) = 3 .
x2 + x1 x22 + 1 x22 1 + 2x1 x2
Let us try to use D̃ = [0, a] × [0.a] with a ≤ 1 and the ∞-norm. We then obtain for x ∈ D̃ that
g0 (x) ∞
≤ 31 max{1 + a, a2 + 1 + 2a2 }
For a = 1 we get kg0 (x)k∞ ≤ which is too large. So we try a = 0.6 which gives kg0 (x)k∞ ≤ 2.08
4
3 3 =: q < 1. Therefore g is a
   
0.6 0.41333
contraction on D̃ = [0, .6] × [0, .6]. Note that g( )= ∈
/ D̃, so D̃ does not satisfy all three assumptions of
0.6 0.60533
Theorem 1.
 
(0) 0
For x = we obtain
0
x(1) = (.33333, .33333)> ∈ D̃, D1 = [−0.42029, 1.08696] × [−0.42029, 1.08696] 6⊂ D̃
(2) >
x = (.40741, .45679) ∈ D̃, D2 = [0.12829, 0.68653] × [0.17767, 0.73591] 6⊂ D̃
x(3) = (.40710, .51393)> ∈ D̃, D3 = [0.27791, 0.53629] × [0.38474, 0.64313] 6⊂ D̃
x(4) = (.39929, .54049)> ∈ D̃, D4 = [0.33926, 0.45933] × [0.48045, 0.60052] 6⊂ D̃
(5) >
x = (.39449, .55238) ∈ D̃, D5 = [0.36761, 0.42138] × [0.52549, 0.57926] ⊂ D̃

Therefore we can conclude from Theorem 3 that there exists a unique solution x∗ ∈ D̃ = [0, 0.6] × [0, 0.6]. This solution x∗
is located in the smaller square D5 . For k = 5, 6, 7, . . . we obtain x∗ ∈ Dk where Dk is a square with side length 2δk . As

k−5
δk ≤ qk−5 δ5 ≤ 2.083 0.027 we can obtain arbitrarily small squares containing the solution if we choose k sufficiently
large.