Scribd
Upload
24 views

MAT 461/561: 5.1 Stationary Iterative Methods

This document discusses stationary iterative methods for solving linear systems Ax = b. [1] Stationary iterative methods repeatedly apply a fixed function g(x) = M^-1(Nx + b) to generate approximations x(k+1) from x(k). [2] The Jacobi and Gauss-Seidel methods are examples of stationary iterative methods that use diagonal splitting of A. [3] Successive overrelaxation (SOR) is an acceleration of Gauss-Seidel that introduces a relaxation parameter ω to take steps of different lengths toward the solution.

Uploaded by

Debisa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
24 views

MAT 461/561: 5.1 Stationary Iterative Methods

This document discusses stationary iterative methods for solving linear systems Ax = b. [1] Stationary iterative methods repeatedly apply a fixed function g(x) = M^-1(Nx + b) to generate approximations x(k+1) from x(k). [2] The Jacobi and Gauss-Seidel methods are examples of stationary iterative methods that use diagonal splitting of A. [3] Successive overrelaxation (SOR) is an acceleration of Gauss-Seidel that introduces a relaxation parameter ω to take steps of different lengths toward the solution.

Uploaded by

Debisa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

MAT 461/561: 5.

1 Stationary Iterative Methods

James V. Lambers

March 2, 2020

Announcements

Iterative Methods for Linear Systems


Why elimination (direct) methods for Ax = b are bad:

• What if we already have an approximation of x? Example: implicit time-stepping methods


for systems of ODEs. Gaussian elimination has NO way of exploiting it

• What if we don’t really need THAT much accuracy? Elimination has no way of “quitting
early” in exchange for less accuracy

• What if A is sparse (mostly zero entries)? Elimination (and pivoting) can lead to “fill-in”

• What if we don’t even have A? Example: instead we have a function A(x) that returns Ax

Stationary Iterative Methods


A stationary iterative method for Ax = b is a method of the form

x(k+1) = g(x(k) )

where x(0) is an initial guess. This is based on fixed-point iteration for solving x = g(x).
Let A = M − N be a splitting of A. Then from Ax = (M − N )x = b, we have:

Mx = Nx + b

which leads to the iteration


M x(k+1) = N x(k) + b.
This is a fixed-point iteration with

g(x) = M −1 (N x + b).

To be practical, systems involving M must be easy to solve (e.g. M diagonal or triangular)

1
Convergence Analysis
How do we know whether this iteration will converge? From equations above:

x(k+1) − x = M −1 N (x(k) − x)

Over k iterations:
x(k) − x = (M −1 N )k (x(0) − x)
Then T = M −1 N is the iteration matrix for this method. Take the norm of both sides:

kx(k) − xk ≤ kT kk kx(0) − xk

To ensure error → 0 as k → ∞, we can require kT k < 1 in some norm. This is sufficient but not
necessary.
A condition that is necessary and sufficient is ρ(T ), the spectral radius of T , is less than 1.
The spectral radius is:
ρ(T ) = max |λ|,
λ∈λ(T )

the magnitude of the largest eigenvalue of T . Recall λ is an eigenvalue of T if T x = λx for some


x 6= 0. Note that ρ(T ) ≤ kT k for any natural norm k · k. This is because if T x = λx, where x 6= 0
and |λ| = ρ(T ), then
ρ(T )kxk ≤ kT kkxk

The Jacobi Method


To describe specfic iterative methods, we write

A=L+D+U

where D is the diagonal part, L is the strictly lower triangular part, and U is the strictly upper
triangular part.
The Jacobi Method comes from M = D, N = −(L + U ). Then we have

x(k+1) = D−1 [b − (L + U )x(k) ].

Each component:  
n
(k+1) 1  X (k)
xi = bi − aij xj  , i = 1, 2, . . . , n
aii
j=1,j6=i

Implementation: given x(0) (can use x(0) = 0), have an outer loop for computing each new x(k+1)
from x(k) , k = 0, 1, 2, . . . until some stopping criterion is met (example: kx(k+1) − x(k) k < ) but
should also set a maximum number of iterations.

2
The Gauss-Seidel Method
If we examine Jacobi more closely:
 
(k+1) 1  X (k)
X (k)
xi = bi − aij xj − aij xj  , i = 1, 2, . . . , n
aii
j<i j>i

(k)
This formula uses old information xj , j < i. If we update:
 
(k+1) 1  X (k+1)
X (k)
xi = bi − aij xj − aij xj  , i = 1, 2, . . . , n
aii
j<i j>i

This is the Gauss-Seidel method:


h i
x(k+1) = D−1 b − Lx(k+1) − U x(k)

or h i
x(k+1) = (D + L)−1 b − U x(k) .

That is, M = D + L and N = −U . In some cases, it can be proven that Gauss-Seidel converges
more rapidly than Jacobi. Disadvantage: Jacobi can be parallelized, Gauss-Seidel can’t.

Successive Overrelaxation
We rewrite Gauss-Seidel as follows:
 
(k+1) (k) 1  X (k+1)
X (k) (k)
xi = xi + bi − aij xj − aij xj − aii xi  , i = 1, 2, . . . , n
aii
j<i j>i

(k+1)
To take a step of different length in the direction of xGS − x(k) , we introduce a relaxation
parameter ω:
 
(k+1) (k) ω bi −
X (k+1)
X (k) (k)
xi = xi + aij xj − aij xj − aii xi  , i = 1, 2, . . . , n
aii
j<i j>i

This leads to:


 
(k+1) (k) (k+1) (k)
X X
aii xi = (1 − ω)aii xi + ω bi − aij xj − aij xj  , i = 1, 2, . . . , n
j<i j>i

or, in matrix form:


h i
Dx(k+1) = (1 − ω)Dx(k) + ω b − Lx(k+1) − U x(k)

or:
(D + ωL)x(k+1) = [(1 − ω)D − U ] x(k) + ωb.
This is called Successive Overrelaxation (SOR). Convergence requires 0 < ω < 2. Why?

3
The iteration matrix is
Tω = (D + ωL)−1 [(1 − ω)D − U ] .
But
n
Y n
Y
det(Tω ) = a−1
ii (1 − ω)aii = (1 − ω)n
i=1 i=1

and we have
n
Y
det(Tω ) = λi
i=1

and therefore
ρ(Tω ) ≥ |1 − ω|.
SOR was introduced by David Young in his PhD dissertation, 1950.

You might also like