APPM 7400

Topics in Applied Math

Section 003
Multigrid Methods
Steve McCormick
ECOT 233 492-0662/442-0724
stevem@colorado.edu
http://amath.colorado.edu/faculty/stevem
Office hours: M@11, W@9, others TBA
U-Boulder 1 of 312
 APPM 7400-003
Syllabus
• Coursework
– No exams
– Homework exercises & computing assignments
– Project & presentation--joint is OK
– Can tailor some or all work & later lectures to your interests
• Philosophy
– Big on fundamentals, principles, & simple ideas
– Think about the scientific method in general
– Understand by concrete examples & experience
– Know the whole picture
– Ask questions & interject comments
– Make sure I explain what you need
• Text
– A Multigrid Tutorial, 2nd edition, 2nd printing
– Supplemental material as needed

U-Boulder 2 of 312
 Sources
MGNet http://www.mgnet.org/
Newsletter
Software repository
Copper Mountain Conference
March 30--April 4, 2003
We hope to support your attendance.

Course web site:

http://amath.colorado.edu/courses/7400/2003Spr/003/

Tech support: John <jruge@colorado.edu>

Homework: Ollie <Oliver.Roehrle@colorado.edu>
U-Boulder 3 of 312
 Homework exercises

Due 1 week after the relevant chapter is covered.

1: 3,4 6: 1, 2, 6

2: 1-3, 8, 10, 13, 16 7: 1, 2, 10, 12, 15, 21

3: 2-6 8: 2, 6, 8

4: 6, 7, 11 9: 1, 3

5: 1-3, 12, 13 10: 1, 5, 8, 9

U-Boulder 4 of 312
 Computing assignments
Due 2 weeks after the relevant chapter is covered.
Start assignments very early: computers are merciless!!!
Use whatever computing language/platform you like.

2: 20 4: 13, 15 6: 6 by computer!!!

7: Convert your 2-D code developed for Chapter 4

(standard coarsening & point relaxation) to solve (7.14).
Verify the code using ε = 1. Test its performance using
ε = 1/2,1/5, 1/10, 2, 5, 10. Interpret your results.

U-Boulder 5 of 312
 Project & presentation
Due at the end of the semester.

• I am VERY flexible as to what constitutes a project.

• Think about what has captured your attention.
• See me, but only after you’ve thought a lot about it.
• Computing &/or theory &/or application &/or exploration.
• Presentation = in-class short talk &/or hand-in.
• In-class talk: Prepare! Speak to your peers, not me.
• This is meant to be instructive & FUN!
• Check course web site for a bit more detail.

U-Boulder 6 of 312
 My thoughts on what’s good science
in order of importance to me
• Attitude
– You can do it! Be positive.
– But is it really right? Be critical.
– Think, think, think! Don’t just hope or guess.
– Control your emotions! Expect ups & downs in order to control them.
• Method
– Start simply. Reduce issue to the simplest possible case.
– Think with tiny steps, but keep the big picture in mind.
– Study concrete examples.
– Look for analogies. Can A be done in any way like how B was done?
• Creativity
– What do you really want? What end are you really aiming for?
– What do you really need? What you’re trying may be sufficient to do
what you want, but would an easier-to-prove weaker result do instead?
• Intelligence
– OK, so it doesn’t hurt to try to be “smart” too.

U-Boulder 7 of 312
 A Multigrid Tutorial
2nd Edition, 2nd Printing

By
William L. Briggs
CU-Denver
Van Emden Henson
LLNL ← THANKS!

Steve McCormick
CU-Boulder

U-Boulder 8 of 312
 Outline
by chapter

1. Model Problems 6. Nonlinear Problems

2. Basic Iterative Methods Full approximation scheme
Convergence tests 7. Selected Applications
Neumann boundaries
Analysis
Anisotropic problems
3. Elements of Multigrid
Variable meshes
Relaxation
Variable coefficients
Coarsening 8. Algebraic Multigrid (AMG)
4. Implementation Matrix coarsening
Complexity 9. Multilevel Adaptive Methods
Diagnostics FAC
5. Some Theory 10. Finite Elements
Spectral vs. algebraic Variational methodology

U-Boulder 9 of 312
 Suggested reading
CHECK MY MG LIBRARY & MGNET REPOSITORY

• A. Brandt, “Multi-level Adaptive Solutions to Boundary Value Problems,”

Math Comp., 31, 1977, pp 333-390.
• A. Brandt, “Multigrid techniques: 1984 guide with applications to
computational fluid dynamics,” GMD, 1984.
• W. Hackbusch, “Multi-Grid Methods & Applications,” Springer, 1985.
• W. Hackbusch & U. Trottenberg, “Multigrid Methods”, Springer-Verlag,
1982.
• S. McCormick, ed., “Multigrid Methods,” SIAM Frontiers in Applied
Math. III, 1987.
• U. Trottenberg, C. Oosterlee, & A. Schüller, “Multigrid,” Academic
Press, 2000.
• P. Wesseling, “An Introduction to Multigrid Methods,” Wylie, 1992.

U-Boulder 10 of 312
 Multilevel methods have been
developed for...
• PDEs, CFD, porous media, elasticity, electromagnetics.
• Purely algebraic problems, with no physical grid; for
example, network & geodetic survey problems.
• Image reconstruction & tomography.
• Optimization (e.g., the traveling salesman & long
transportation problems).
• Statistical mechanics, Ising spin models.
• Quantum chromo dynamics.
• Quadrature & generalized FFTs.
• Integral equations.

U-Boulder 11 of 312
 Everyone uses multilevel methods
• Multigrid, multilevel, multiscale, multiphysics, …
Use local “governing rules” on the finest level to
resolve the state of the system at these detailed
scales, but--recognizing that these “rules” have
broader implications that are hard to determine
there--use coarser levels to resolve larger scales.
Continual feedback is essential because improving
one scale impacts other scales.
• Common uses
Sight, art, politics, thinking (scientific research),
cooking, team sports, …

U-Boulder 12 of 312
 1. Model problems
• 1-D boundary value problem:

− u ″( x ) + σ u( x ) = f ( x ) 0 < x < 1, σ≥0

u( 0) = u( 1 ) = 0
• Grid:
1
h =
N
, xi = ih , i = 0, 1 , . . . N
x =0 x =1
x0 x1 x2 xi xN
• Let & for .

v i ≈ u( x i ) f i ≈ f ( xi ) i = 0, 1, . . . N

This discretizes the variables, but what about the equations?

U-Boulder 13 of 312
 Approximate u’’(x) via Taylor series

• Approximate 2nd derivative using Taylor series:

0 h2 h3
0
u ( x i + 1 ) = u( x i ) + h u ′ ( x i ) + u″ ( x i ) + u′′′ ( x i ) + O( h 4 )
2! 3!
h2 h3
u( x i − 1 ) = u( x i ) − h u′ ( x i ) + u″ ( x i ) − u′′′ ( x i ) + O( h 4 )
2! 3!

• Summing & solving:

u( x i + 1 ) − 2 u( x i ) + u( x i − 1 )
u″ ( x i ) = + O( h 2)
h2
U-Boulder 14 of 312
 Approximate equation via
finite differences
• Approximate the BVP

− u ″( x ) + σ u( x ) = f ( x ) 0 < x < 1, σ≥0

u( 0) = u( 1 ) = 0
by a finite difference scheme:

− vi − 1 + 2 vi − vi + 1
+ σ vi = f i i = 1, 2, . . . N − 1
2
h
v0 = vN = 0

U-Boulder 15 of 312
 Discrete model problem

T&
Letting v = ( v 1, v 2, . . . , v N − 1 )
T
f = ( f 1, f 2, . . . , f N − 1 )
we obtain the matrix equation Av = f, where A
is (N-1) x (N-1), symmetric, positive definite, &

 2 + σh2 −1   f1 
   v1 
 − 1 2 + σh 2   v2   f 
−1  2 
   
1  2
− 1 2 + σh − 1   v3   f 
A= , v=  , f = 3 
2 
h      
 2   N −2 
v  f N −2
 − 1 2 + σ h − 1  v   
   N − 1  f
 − 1 2 + σh2   N −1

U-Boulder 16 of 312
 Stencil notation

A = [-1 2 -1]
dropping h -2 & σ for convenience

-1

-1

U-Boulder 17 of 312
 Basic solution methods
• Direct
– Gaussian elimination
– Factorization
– Fast Poisson solvers (FFT-based, reduction-based, …)
• Iterative
– Richardson, Jacobi, Gauss-Seidel, …
– Steepest Descent, Conjugate Gradients, …
– Incomplete Factorization, ...
• Notes:
– This simple 1-D problem can be solved efficiently in many ways.
Pretend it can’t & that it’s very hard, because it shares many
characteristics with some very hard problems. If we keep things
as simple as possible by studying this model, we’ve got a chance
to really understand what’s going on.
– But, to keep our feet on the ground, let’s go to 2-D anyway…

U-Boulder 18 of 312
 2-D model problem
• Consider the problem

− ux x − uyy + σ u = f ( x , y ) , 0 < x < 1, 0<y<1

u = 0 , x = 0, x = 1, y = 0, y = 1; σ≥0
• Consider the grid
z
1 1
hx = , hy = ,
M N
( x i , yj ) = ( i hx , j hy )
y
0 ≤ i ≤ M
0≤ j ≤ N
U-Boulder x 19 of 312
 Discretizing the 2-D problem
• Let v i j ≈ u( x i , yj ) & f i j ≈ f ( xi , yj ) . Again, using 2nd-
order finite differences to approximate u x x & uy y
we arrive at the approximate equation for the
unknown , for i =1,2, … M-1 & j =1,2, …, N-1:
u( x i , yj )
− v i − 1, j + 2v i j − v i + 1, j − v i , j − 1 + 2v i j − v i , j + 1
+ + σ vi j = f i j
hx2 hy2
v i j = 0 , the
• Ordering i = 0unknowns
, i = M , j (&
= 0,also
j =M
the vector f )
lexicographically by y-lines:

T
v = (v1,1 , v1,2 ,..., v1,N −1 , v2 ,1 , v2 , 2 ,..., v2 ,N−1 ,..., vM−1,1 ,vM−1,2 ,..., vM−1,N−1 )
U-Boulder 20 of 312
 Resulting linear system
• We obtain a block-tridiagonal system Av = f :
 A1 − Iy  v1  f1 
 −I A − I    
 y 2 y  2 
v f
 2 
 − Iy A3 − Iy   v3   f 
   =  3 
    
 − Iy AN − 2 − Iy     
   vN − 1  f 
 Iy is a diagonal
where − Imatrix
y A N − 1 with on
 Nthe
−1 
diagonal & 1
hy2

 2 + 2 +σ −
1 
 hx2 hy2 hx2 
 1 2 2 1 
 − 2 + 2 +σ − 
 hx2 hx hy hx2 
Ai =  1 2 2 1 
− 2 + +σ −
 hx hx2 hy2 hx2 
 
 
1 2 2
 − 2 2 + 2 +σ

U-Boulder  hx hx hy  21 of 312
 Stencils
preferred for grid issues

Stencils are much better for showing the grid picture:

again dropping h -2 & σ
0 −1
0  • -1• •
 h 2y 
 −12 2 2
2 + 2 +σ
−1 
• • •
-1 4 -1
 hx h x hy hx2

−1
0
 h 2y
0
 • -1• •
Stencils show local relationships--grid point interactions.

U-Boulder 22 of 312
 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems • Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
– Convergence tests • Selected Applications
– Analysis – Neumann boundaries
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
– Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Matrix coarsening
– Complexity • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology

U-Boulder 23 of 312
 2. Basic iterative methods

• Consider Au = f where A is NxN & let v be an

approximation to u. (Generic A uses N not N-1.)

• Two important measures:

– The Error: with norms
e = u −v
N
|| e ||∞ = max | ei | & || e ||2 = ∑ e
i =1 i
2

– The Residual: with

What does
r = f − Av ||r|| measure???

|| r ||∞ & || r ||2 Why have both

r & e ???
U-Boulder 24 of 312
 Residual correction

• Since e = u - v, we can write Au = f as A(v + e) = f

which means that Ae = f - Au ≡ r. r = f - Av

= Au - Av
• Residual Equation: = A(u - v)
= Ae
Ae = r
• Residual Correction:
What does this
do for us???
u = v +e

U-Boulder 25 of 312
 Relaxation

• Consider the 1-D model problem

− ui − 1 +2 ui − ui + 1 = h 2 f i 1 ≤ i ≤ N −1 u0 = uN = 0

• Jacobi (simultaneous displacement): Solve the i th

equation for holding all other variables fixed:
vi

( ) 1 ( o ld)
vi n e w = ( v i − 1 + v i( +ol1d) + h 2 f i ) 1 ≤ i ≤ N −1
2

U-Boulder 26 of 312
 Jacobi in matrix form
• Let A = D - L - U, where D is diagonal & L & U are
the strictly lower & upper triangular parts of A.
• Then becomes
Au = f
(D −L −U) u = f
D u = ( L + U) u + f
• Let u = D − 1.( L + U) u + D − 1 f
– RJR is = − 1the error propagation =orDiteration
-1(D-A) matrix.
I D-1A
J
called
D ( L + U) J = -
• Then the iteration is

v ( n ew) = R J v ( o ld ) + D − 1 f
U-Boulder 27 of 312
 Error propagation matrix & the error
• From the derivation,
u = D − 1 ( L + U) u + D − 1 f
u = RJ u + D − 1 f
• the iteration is

v ( n ew) = R v ( o ld ) + D − 1 f
• subtracting, J

• or
u − v ( n ew) = R J u + D − 1 f − ( R J v ( o ld) + D − 1 f )
• hence
u − v ( n ew) = R J u − R J v ( o ld)

Error propagation! e ( n ew) = R J e ( o ld) J = I-D-1A

U-Boulder 28 of 312
 A picture
RJ = D-1 (L + U) = [ 12 0 1
2 ]
so Jacobi is an error averaging process:
ei(new) ← (ei-1(old) + ei+1(old))/2

U-Boulder 29 of 312
 But…

U-Boulder 30 of 312
 Another matrix look at Jacobi
v (new) ← D-1 (L + U) v (old) + D-1 f (L + U = D-A)
= (I - D-1 A) v (old) + D-1 f
v (new) = v (old) - D-1 (Av (old) - f) = v (old) - D-1 r

• Exact: u =u - D-1 (Au - f)

• Subtracting: e (new) = e (old) - D-1 Ae (old)
• General form: u = u - B (Au - f) with B ~ A-1
• Damped Jacobi: u = u - ωD-1 (Au - f) with 0<ω<2
• Gauss-Seidel: u = u - (D - L)-1 (Au - f)
• Exact: u = u - A-1 (Au - f) = A-1f

U-Boulder 31 of 312
 Weighted Jacobi • • •
safer (0<ω<1) changes; politics in the ‘hood • o •
• • •
• Consider the iteration
( ) ( ) ω ( o ld )
vi n e w ← ( 1 − ω) v i o l d + ( v i − 1 + v i( +o l1d) + h 2f i )
2

• Letting A = D-L-U, the matrix form is

v ( n ew) = ( 1 − ω) I + ω D − 1( L + U) v ( o ld) + ω h 2D − 1f

= R ω v ( old) + ω h 2D − 1f
• Note that
R ω = [ ( 1 − ω) I +ω R J ]
( a ppro x )
• It is easy to see that if e ≡ u ( ex a c t ) − u , then

e ( n ew) = R ω e ( o ld)
U-Boulder 32 of 312
 Gauss-Seidel (1-D)
• Solve equation i for ui & update immediately.
• Equivalently: set each component of r to zero in turn.
• Component form: for i = 1, 2, . . . N − 1, set
1
vi ← ( vi − 1 + vi + 1 + h 2 f i )
• Matrix form: 2

A = (D −L −U)
( D − L) u = U u + f
• Let u = ( D − L) − 1 U u + ( D − L) − 1 f
• ThenRiterate:
= ( D − L ) − 1U
G

v ( n ew) ← R G v ( old) + ( D − L ) − 1f
• Error propagation:

e ( n ew) ← R G e ( o ld)
U-Boulder 33 of 312
 Red-black Gauss-Seidel
• Update the EVEN points:
1
v 2i ← ( v 2i − 1 + v 2i + 1 + h 2 f 2i )
2
• Update the ODD points:

1
v 2i + 1 ← ( v 2i + v 2i + 2 + h 2 f 2i + 1 )
2

x0 xN

• 2-D:

U-Boulder 34 of 312
 Numerical experiments
• Solve A u = 0 , − ui − 1 + 2 u i − u i + 1 = 0
• Use Fourier modes as initial iterates, with N = 64:
 ikπ  1 ≤ i ≤ N − 1, 1 ≤ k ≤ N −1
vk = ( v i ) k = sin  
 N  component mode

sin(kπx), x =i/N k =1
1
0.8
0.6
k =3
0.4
0.2
0
-0.2
-0.4
k =6
-0.6
-0.8
-1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
U-Boulder 35 of 312
 Convergence factors differ for
different error components
Error, ||e||∞ , in weighted (ω=2/3) Jacobi on Au = 0
for 100 iterations using initial guesses v1, v3, & v6

1
k =1
0.9

0.8

0.7

0.6

0.5

0.4
k =3
0.3

0.2
k =6
0.1

0
0 20 40 60 80 100 120

U-Boulder 36 of 312
 Stalling convergence
relaxation shoots itself in the foot

• Weighted (ω=2/3) Jacobi on 1-D problem.

• Initial guess:
1  j π  6 jπ   32 j π  
v 0 =  s in   + s in  N  + s in  N  
3  N      
• Error plotted against iteration number:
|| e ||∞
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 10 20 30 40 50 60 70 80 90 100
U-Boulder 37 of 312
 Analysis of stationary linear iteration
• Let v(new) = Rv(old) + g . The exact solution is
unchanged by the iteration: u = Ru + g .
• Subtracting: e(new) = Re(old) .

• Let e(0) be the initial error & e(i) be the error

after the i th iteration. After n iterations, we have
e(n) = Rne(0) .

a b c 4
 
I can deal with 24, but d e f ???
 
g h i

a b c 4
 
What if d e f
e = 24 e ???
 
g h i
U-Boulder 38 of 312
 Quick review of
eigenvectors & eigenvalues
• The number λ is an eigenvalue of a matrix B & w ≠ 0
its associated eigenvector if Bw = λw.
• The eigenvalues & eigenvectors are characteristics
of a given matrix.
• Eigenvectors are linearly independent, & if there is a
complete set of N distinct eigenvectors for an NxN
matrix, then they form a basis: for any v, there exist
unique scalars vk such that

N
• Propagation: v= Σ k
vk
k=1
w Why is an
eigenvector
N useful???
n
Σ
n
B v= λ vk wk
U-Boulder k=1 39 of 312
 “Fundamental Theorem of Iteration”
• R is convergent (Rn → 0 as n → ∞) iff
.
ρ(R) = max1≤k≤N λk < 1
Thus, e(n) = Rne(0) → 0 for any initial vector v (0)
iff ρ(R)<1.

• ρ(R)<1 assures convergence of the iteration for R.

• ρ(R) is the spectral convergence factor.
– But ρ is doesn’t tell you much by itself--it’s generally valid
only asymptotically. It’s useful for the symmetric case in
particular, so we’ll accept it for now, but first a little
background & then a warning…

U-Boulder 40 of 312
 Rayleigh quotient vs. spectral radius
assume A is symmetric (wk orthogonal) & nonnegative definite (λ ≥ 0)

• RQ(v) ≤ ρ(A): v = Σvkwk

RQ(v) < Av, v > < A Σ vk wk , Σ vk wk >

= =
< v,v > < Σ vk wk , Σ vk wk >
< Σ λk vk wk , Σ vk wk > Σ λ
2
k vk ≤ λ1N = ρ(A)
≥
= =
< Σ vk wk , Σ vk wk > Σ vk2
• supv ≠ 0 RQ(v) = ρ(A):

< AwN ,wN > < λN wN ,wN >

RQ(wN ) = = = λN = ρ(A)
< wN ,wN > < wN , wN >
U-Boulder 41 of 312
 Euclidean norm vs. spectral radius
use RQ

• ||R||2 = ρ1/2(RTR):
||R||22 = supe≠0 ||Re||22 /||e||22
= supe≠0 <Re,Re >/<e,e>
= supe≠0 <RTRe,e >/<e,e> = ρ(RTR)
note: ||Re||2 ≤ ||R||2·||e||2

• ||A||2 = ρ1/2(A2) = ρ(A) for symmetric A!!!

U-Boulder 42 of 312
 ρ(R) vs. ||R||2
ρ(R) = sup |λ(R)|. ||R||2=supe≠0 ||Re||2 /||e||2= ρ1/2(RTR).

ρ(R) is norm independent. ||R||2 depends on ||· ||2.

ρ(R) is asymptotic: ||R||2 bounds ||e(n + 1)||2 /||e(n)||2:

could get large ||R||2 but ρ(R)<1 worst case! Probably pessimistic
⇒ “convergence”. Next slide. initially, but sharp sooner or later.

ρ(R) = inf ||R||

taken over all matrix norms.

ρ(R) = ||R||2
for symmetric R.
U-Boulder 43 of 312
 Example: R = ( ), K large
ρ(R) = 0 but ||R||2 = K !
(1) (0 )
 0 e 2 Re
=   ⇒ ( 0) = (0 )
( 0)
e 2
=K
 1 e 2 e 2

Thus, 1 iteration with e(0) shows dramatic L2 divergence!

But R2 = 0, so e(2) = Re(1) = R2e(0) = 0!
Thus, 2 iterations with e(0) show complete convergence!
On one hand, this is special (λ=0, large K, 2x2), so this
behavior would be more subtle & persistent in general.
On the other, this behavior would vanish for symmetric R.

U-Boulder 44 of 312
 Convergence factor & rate
• How many iterations are needed to reduce the initial error
by 10-d ?
e (n ) n n −d
(0 ) ≤ R ≤ R ~ 10
e
• So, we have
d
n~
− log10 ρ( R)
• Convergence factor = ||R|| or ρ(R).

• Convergence rate = or -log10(ρ(R)).

digits
− log10 R
iteration
U-Boulder 45 of 312
Convergence analysis: Weighted Jacobi

= I − ωD − 1 A
 2 −1 
 −1 2 −1 
ω 
Rω = I −  −1 2 −1  1-D
2 ⋅⋅ ⋅⋅ ⋅⋅ 
 ⋅ ⋅ ⋅
 −1 2 

For our 1-D model, the eigenvectors of weighted

Jacobi Rω & the eigenvectors of A are the same! Why???

The eigenvalues are related as well.

ω
λ ( Rω ) = 1 − λ ( A)
U-Boulder 2 46 of 312
 Eigenpairs of A
The eigenvectors of A are Fourier modes!
 kπ   j kπ  [-1 2 -1]
λk ( A ) = 4 sin 
2
 , wk , j = sin   λN-1 ≅ 4
 2N   N  λ1 ≅ πh2
1 1
0.8 0.8
0.6 0.6
0.4 0.4

N = 64 0.2
0
-0.2
0.2
0
-0.2
-0.4 -0.4
-0.6 -0.6
-0.8 -0.8
-1 -1

0 10 20 30 40 50 60 0 10 20 30 40 50 60

k = 1 k = 2
1 1 1
0.8 0.8 0.8
0.6 0.6 0.6
0.4 0.4 0.4
0.2 0.2 0.2
0 0 0
-0.2 -0.2 -0.2
-0.4 -0.4 -0.4
-0.6 -0.6 -0.6
-0.8 -0.8 -0.8
-1 -1 -1
0 10 20 30 40 50 60 0 10 20 30 40 50 60 0 10 20 30 40 50 60

k = 4 k = 8 k = 16
U-Boulder 47 of 312
Eigenvectors of Rω = eigenvectors of A
 kπ 
λk ( R ω ) = 1 − 2 ω sin2  2N 
 

• Expand the initial error in terms of the eigenvectors:

N−1
e ( 0)
= ∑ ck w k
k =1
• After n iterations:
N −1
Re n (0 )
= ∑ ck λ w k
n
k
k=1

• The kth error mode is reduced by λk (Rω) each iteration.

U-Boulder 48 of 312
 Relaxation suppresses eigenmodes unevenly

• Look carefully at λk ( R ω ) = 1 − 2 ω sin2  k π  .

 2N 
1

0.8
Note that if 0 < ω ≤ 1,
0.6 ω = 1/3
then | λ ( R ω ) | < 1 for
k
.
0.4 k = 1, 2, . . . , N − 1
0.2 ω = 1/ 2 For 0 < ω ≤ 1,
λ 0
N
π 
-0.2
2 λ1 = 1 − 2 ω sin 2
 2N 
-0.4
ω=1
 
ω = 2/ 3
-0.6  πh 
= 1 − 2 ω s in 
2

-0.8
 2 
-1
= 1 − O ( h2) ≈ 1
0 k N
U-Boulder 49 of 312
 Low frequencies are “undamped”
Notice that no value of ω will efficiently damp out
long waves or low frequencies.
1
What value of ω gives
0.8
the best damping of
0.6 ω = 1/3
short waves or high
0.4
frequencies N/2≤k≤N-1?
λ
0.2
ω = 1/ 2
axis 0

-0.2
N Choose such that
-0.4
2
ω
ω=1
-0.6
ω = 2/ 3 λN ( R ω ) = − λN ( R ω )
-0.8 2
2
-1 ⇒ ω=
0 N 3
U-Boulder
k axis 50 of 312
 Smoothing factor

• The smoothing factor is the largest magnitude of

the iteration matrix eigenvalues corresponding to
the oscillatory Fourier modes:
smoothing factor = max |λk(R)| for N/2≤k≤N-1.

• Why only the upper spectrum? “MG” spectral radius?

• For Rω with ω=2/3, the smoothing factor is 1/3:

|λN/2|=|λN|=1/3 & |λk|<1/3 for N/2<k<N.

• But |λk| ≈ 1 - ωk2π2h2 for long waves (k << N/2).

U-Boulder 51 of 312
 Convergence of Jacobi on Au = 0
100 100
Unweighted Jacobi Weighted Jacobi
90 90

80 80

70 70

60 60

50 50

40 40

30 30

20 20

10 10

0
0 0 10 20 30 40 50 60
0 10 20 30 40 50 60
Wavenumber, k Wavenumber, k
• Jacobi on Au = 0 with N = 64. Number of iterations
needed to reduce initial error ||e||∞ by 0.01.

• Initial guess :
 jkπ 
vkj = sin  
 N 
U-Boulder 52 of 312
 Weighted Jacobi = smoother (error)
 2j π  1  16j π  1  32j π 
• Initial error: v k j = sin 
N  + 2 sin  N  + 2 sin  N 
     
2

-1
Many relaxation schemes
-2
are smoothers:
0 0 .5 1 oscillatory error modes
are quickly eliminated,
• Error after 35 iteration sweeps: but
smooth modes are slowly
2
damped.
1

-1

-2

0 0 .5 1
U-Boulder 53 of 312
 Similar analysis for other smoothers
• Gauss-Seidel relaxation applied to the 3-point
difference matrix A (1-D model problem):
−1
RG = ( D − L ) U
• A little algebra & trigonometry shows that

kπ  kπ jkπ 
λk (RG ) = cos2  wk , j = cos j   sin 
 N  N  N 
1
1
0.8

0.8
0.6 π xj )
sin(3π
0.4

0.2
0.6
0
j
0.4 λk -0.2

-0.4
λ3 sin(3π
πxj )
-0.6
0.2
-0.8

-1
0
0 10 20 30 40 50 60
k 0 10 20 30
j 40 50 60

U-Boulder
What’s wk look like for large k ? 54 of 312
 Gauss-Seidel eigenvectors
1 0.1

0.8 0.08

0.06
0.6
k=1,n=64 0.04
0.4
0.02
k=30,n=64
0.2 0

0 −0.02
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

0.2 1

0.1 k=60,n=64
0.5
0

−0.1
0
−0.2 k=48,n=64

−0.3
−0.5
−0.4

−0.5 −1
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

U-Boulder 55 of 312
 Gauss-Seidel convergence
Au = 0
Eigenvectors of RG are not the same as those of A.
Gauss-Seidel mixes the modes of A.

100 Gauss-Seidel on Au = 0, with

90

80
N = 64. Number of iterations
70 needed to reduce initial error
60

50
||e||∞ by 0.01.
40

30 Initial guess (modes of A):

20

10

0
 jkπ 
vkj = sin  
0 10 20 30 40 50 60

 N 
U-Boulder 56 of 312
 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems • Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
– Analysis – Neumann boundaries
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
– Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Matrix coarsening
– Complexity • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology

U-Boulder 57 of 312
 3. Elements of multigrid
1st observation toward multigrid
• Many relaxation schemes have the smoothing
property: oscillatory error modes are quickly
eliminated, while smooth modes are often very
slow to disappear.
• We’ll turn this adversity around: the idea is to
use coarse grids to take advantage of smoothing.

x0 xN
Ωh
2h
x0 xN Ω
2
How?
U-Boulder 58 of 312
 Reason #1 for coarse grids:
Nested iteration
• Coarse grids can be used to compute an improved
initial guess for the fine-grid relaxation. This is
advantageous because:

– Relaxation on the coarse-grid is much cheaper: half as

many points in 1-D, one-fourth in 2-D, one-eighth in 3-D,…

– Relaxation on the coarse grid has a marginally faster

(|λ1(R)| ≈ 1 - ωπ2h2 ):

instead of

1 − O( 4h 2 ) 1 − O ( h 2)

U-Boulder 59 of 312
 Idea! Nested iteration
•
•
•
• Relax on Au = f on Ω4h to obtain initial guess v2h.
• Relax on Au = f on Ω2h to obtain initial guess vh.
• Relax on Au = f on Ωh to obtain … final solution???

• What is A2hu2h = f2h? Analogous to Ahuh = fh for now.

• How do we migrate between grids? Hang on…
• What if the error still has large smooth components
when we get to the fine grid Ωh ? Stay tuned for 2nd
observation toward multigrid…

U-Boulder 60 of 312
 Reason #2 for coarse grids:
Smooth error becomes more oscillatory
• A smooth function:
1
On the coarse grid, the
smooth error appears to
0 . 5 be relatively higher in
0
frequency: in this example
it is the 4-mode out of
- 0 . 5 a possible 15 on the fine
grid, ~1/4 the way up the
- 1
0 0 . 5 1 spectrum. On the coarse
grid, it is the 4-mode out
of a possible 7, ~1/2 the
can be represented by linear way up the spectrum.
interpolation from a coarser grid:
Relaxation on 2h is
(cheaper &) faster
on this mode!!!

U-Boulder 61 of 312
 For k=1,2,…N/2, the kth mode is
preserved on the coarse grid.

 2j k π   j kπ 
1
0.8
wk , 2j = sin 
h
 = sin   = wk2,hj 0.6

 N   N /2 
0.4
0.2
0
-0.2

Also, note that

-0.4
-0.6
-0.8
-1
h →0
wN/2 0 2 4 6 8 10 12

on the coarse grid. k=4 mode, N=12 grid

1
0.8
0.6
0.4

What happens 0.2

0
-0.2
to the modes -0.4
-0.6

between N/2 & N ?

-0.8
-1
0 1 2 3 4 5 6

k=4 mode, N=6 grid

U-Boulder 62 of 312
 For k > N/2, wkh is disguised on
the coarse grid: aliasing!!!
• For k > N/2, the kth mode on 1

the fine grid is aliased & 0.8

0.6

appears as the (N - k)th

0.4
0.2
0

mode on the coarse grid: -0.2

-0.4
-0.6
-0.8
-1
 ( 2j ) πk  0 2 4 6 8 10 12
( wkh ) = s in 
N 
k=9 mode, N=12 grid
2j 
 2πj ( N − k ) 
1
0.8
= − sin   0.6

 N 
0.4
0.2
0
-0.2

 π ( N − k) j 
-0.4
-0.6
= − sin   -0.8

 N / 2 
-1
0 1 2 3 4 5 6
k=3 mode, N=12 grid
= − ( wN
2h )
−k j
U-Boulder 63 of 312
 1-D interpolation (prolongation)
to migrate from coarse to fine grids

• Mapping from the coarse grid to the fine grid:

I 2hh : Ω 2h → Ω h
• Let , be defined on h, . Then
2h
vh v 2h Ω Ω
where I 2hh v 2h = v h

v 2hi = v i2h

v 2hi + 1
1 2h
= ( v i + v i2h+ 1 )
2
} for 0≤i ≤
N
2
− 1.

U-Boulder 64 of 312
 1-D interpolation (prolongation)
• Values at points on the coarse grid map unchanged
to the fine grid.
• Values at fine-grid points NOT on the coarse grid
are the averages of their coarse-grid neighbors.

U-Boulder 65 of 312
 1-D prolongation operator P

h
• P = I 2h is a linear operator: ℜN/2-1 ℜN-1 .
 v 1h 
• N = 8:  1/2   
 1   v 2h 
   
 1/2 1/2   v 2h 
1  v 3h 
   2h   h 
 1   2  v =  v4 
 1/2 1/2   v 2h   h 
  3  3x 1  v5 
 1   h 
   v6 
 1/2  7x 3
 vh 
• has full rank, so η(P ) = {0} .  7  7x 1

I 2hh
When is I 2hh v2h = 0?
U-Boulder 66 of 312
 “Give To” stencil for P

] 1/2 1 1/2 [

o x o x o x o
1/2 1 1/2

U-Boulder 67 of 312
 How well could v 2h approximate u?

• Imagine that a coarse-grid approximation v2h has

been found. How well could it approximate the
exact solution u ?

• If u is smooth, a coarse-grid interpolant v2h might

do very well.

U-Boulder 68 of 312
 How well could v 2h approximate u?

• Imagine that a coarse-grid approximation v2h has

been found. How well could it approximate the
exact solution u ?

• If u is oscillatory, a coarse-grid interpolant v2h

cannot work well.

U-Boulder 69 of 312
 Moral
• If what we want to compute is smooth, a
coarse-grid interpolant could do very well.
• If what we want to compute is oscillatory, a
coarse-grid interpolant cannot do very well.
• What if u is not smooth? Can we make it so?
• Can we make something smooth?

• How about e ? Can we smooth it? How would we

get e & use it to get u ? Ae = r & u ← v + e !
• Thus, use nested iteration on residual equation
to approximate the error after smoothing!!!
• Just because the coarse grid can approximate e well
doesn’t mean we know how to do it! But we will soon!
U-Boulder 70 of 312
 2nd observation toward multigrid
• The residual equation: Let v be an approximation to
the solution of Au = f, where the residual r = f -Av.
Then the error e = u - v satisfies Ae = r.

• After relaxing on Au = f on the fine grid, e will be

smooth, so the coarse grid can approximate e well.
This will be cheaper & e should be more oscillatory
there, so relaxation will be more effective.

• Therefore, we go to a coarse grid & relax on the

residual equation Ae = r.

What’s a good initial guess on grid 2h? e = 0 !

How do we get back to grid 2h? Stay tuned…

U-Boulder 71 of 312
 A way to coarsen Ae = r
• Assume we’ve relaxed so much that e is smooth.
• Ansatz: e = Pv2h for some coarse-grid v2h.
• How do we characterize e so we can hope to compute it?
Ae = r ⇒ A P v2 h = r
7x7 7x3 3x1 = 7x1

• Too many equations now & too few unknowns!

• Multiply both sides by some 3x7 matrix?
PT
P T A P v2h = P Tr
A2h
)
3x3
)
3x7 7x7 7x3 3x1 = 3x1

U-Boulder 72 of 312
 Idea! Coarse-grid correction
2-grid
h to get an approximation
• Relax on Au = f on Ω .
• Compute r = f − A v h .
• Relax on Ae = r on 2h to obtain an approximation to
Ω
the error, .
e 2h
• Correct the approximation v h ← v h + e2h.
I 2hh

This is the essence of multigrid.

• Clearly, we need a mapping for Ωh → Ω2h.

U-Boulder 73 of 312
 1-D restriction by injection
• Mapping from the fine grid to the coarse grid:

I h2h : Ω h → Ω 2h

• Let , be defined on ,
h 2h
. Then
vh v 2h Ω Ω
2h h
where I
. h v = v 2h

v i2h = v 2hi

Ωh

Ω 2h
U-Boulder 74 of 312
 1-D restriction by full weighting

• Let v h , v 2h be defined on Ω h, Ω 2h. Then

I h2h v h = v 2h
where
1 h .
v i = ( v 2i − 1 + 2 v 2hi + v 2hi + 1 )
2h
4

Ωh

Ω 2h

U-Boulder 75 of 312
 1-D restriction (full-weighting)

2h
• R = I h is a linear operator: ℜN-1 ℜN/2-1 .
 v 1h 
• N = 8:  
 v 2h 
 
1/4 1/2 1/4  v 3h   v 12h 
  h   2h 
 1/4 1/2 1/4   v4  =  v2 
 1/4 1/2 1/4   v h   2h 
 5   v3 
 h 
 v6 
 vh 
• has rank , so dim(η(R))  7  .
2h N N
Ih ∼ ∼
2 2
Look at the columns of R associated with grid 2h.
U-Boulder 76 of 312
 Prolongation & restriction are
often nicely related
• For the 1-D examples, linear interpolation & full
weighting are
1 
2 
 
1 1  1 2 1
1   2h 1 
I 2hh =  2  Ih =  1 2 1 
2   4
 1 1   1 2 1
 2
 
 1 
• So they’re related by the variational condition

for c in ℜ.
T
I 2hh = c ( I h2h )
U-Boulder 77 of 312
 2-D prolongation
1 1 1
4 2 4
1 2h 1 1
v 2i + 1, 2j = ( v i j + v ih+ 1, j )
h
1
2 2 2
1 2h
v 2hi , 2j + 1 = ( v i j + v ih, j + 1 ) 1 1 1
2 4 2 4
1 2h
v 2i + 1, 2j + 1 = ( v i j + v ih+ 1, j + v ih, j + 1 + v ih+ 1, j + 1 )
h
4

We denote the operator by

1 1 1
using a “give to” stencil ] [.
4 2 4
Centered over a C-point ,
1 1
it shows what fraction of 1
2 2
the C-point’s value 1 1 1
contributes to a neighboring 4 2 4
F-point .
U-Boulder 78 of 312
 “Get From” interpolation
1/4 1/4

[1/2 1/2]

1/4 1/4

Centered over a coarse grid point .

1/4 1/4

1/2 1/2 1/4 1/4

U-Boulder 79 of 312
 2-D restriction (full weighting)
1 1 1
16 8 16
1 1 1
8 4 8
1 1 1
16 8 16

We denote the operator by

1 1 1
using a “get from” stencil [ ].
16 8 16
Centered over a C-point , it
1 1 1
shows what fraction of the 8 4 8
value of the neighboring 1 1 1
F-point contributes to the 16 8 16
value at the C-point.
U-Boulder 80 of 312
 Now we put all these ideas together

• Nested Iteration
– effective on smooth solution (components).

• Relaxation
– effective on oscillatory error (components).

• Residual Equation
– characterizes the error.
– enables nested iteration for smooth error (components)!!!

• Prolongation (variables) & Restriction (equations)

– provides pathways between coarse & fine grids.

U-Boulder 81 of 312
 2-grid coarse-grid correction
v h ← CG ( v h , f h ) , α1 , α2

1) Relax α times on A h u h = f h on Ω h with arbitrary

1
initial guess h.
v
2) Compute h h h.
r =f −A v
h
3) Compute .
r 2h = I 2h r h
4) “Solve” h on .
2h 2h 2h
=
5) Correct fine-grid solution
A e r 2h Ω .
6) Relax times on v hon I 2hh initial
← v h +with e 2h
guess .
α2 A h uh = f h Ωh
vh

U-Boulder 82 of 312
 2-grid coarse-grid correction

Correct
Relax on Ah v h = f h
Compute r h = f h− Ahv h vh ← vh + eh

Restrict Interpolate
r 2h = I h2h r h eh = I2hh e2h

2h 2h
Solve A e = r2h
e 2h = ( A 2h) − 1 r 2h

U-Boulder 83 of 312
 What is A2h?

• For this scheme to work, we must have 2h, a coarse-

A
grid operator. For the moment, we will simply assume
that is “the coarse-grid version ” of the fine-grid
2h
A
operator .
Ah
• Later we’ll return to the question of constructing .
2h
A

U-Boulder 84 of 312
 How do we “solve” the coarse-
grid residual equation? Recursion!
α1 h h α2 = 0
vh ← G (v , f ) v h ← v h + eh

h
f
2h 2h
← Ih ( f
h h
−A v )
h
e h ← I 2h v 2h

v 2h ← Gα1( 0 , f
2h 2h
) v 2h ← v 2h + e 2h

4h 4h 2h 2h
f ← I2h ( f − A v 2h ) 2h
e 2h ← I 4h v 4h

v 4h ← Gα1( 0
4h
,f
4h
) v 4h ← v 4h + e 4h
8h 8h 4h 4h
4h 4h
f ← I4h ( f −A v ) e 4h ← I 8h v 8h
v 8h ← Gα1( 0 , f
8h 8h
) v 8h ← v 8h + e 8h

1 H
vH = eH = ( A ) − f
H
U-Boulder 85 of 312
 V-cycle (recursive form)
v h ← M V h ( v h , f h ) , α 1, α2

1) Relax α1 times on A h u h = f h with initial v h given.

2) If h is the coarsest grid, go to 4;

Ω
else:
f 2h ← I2h
h ( f h
− A h h
v )
v 2h ← 0
v 2h ← M V 2h ( v 2h , f 2h )
3) Correct: .
v h ← v h + I 2hh v 2h
4) Relax times on with initial guess .

α2 A h uh = f h vh

U-Boulder 86 of 312
 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems • Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
– Analysis – Neumann boundaries
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
√ – Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Matrix coarsening
– Complexity • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology

U-Boulder 87 of 312
 4. Implementation
Storage cost: vh & fh on each level.

In 1-D, a coarse grid has about half as many
points as the fine grid.

In 2-D, a coarse grid has about one-fourth

as many points as the fine grid.

In d-dimensions, a coarse grid has about 2-d

as many points as the fine grid.

−d − 2d − 3d − Md 2N d
d

Total storage cost: 2N ( 1 + 2 +2 +2 + …+2 ) <
1 − 2− d
less than 2, 4/3, & 8/7 the cost of storage on the fine
grid for 1-D, 2-D, & 3-D problems, respectively.
U-Boulder 88 of 312
 Computational cost
• Let one Work Unit (WU) be the cost of one
relaxation sweep on the fine grid.
• Ignore the cost of restriction & interpolation
(typically about 20% of the total cost).
• Consider a V-cycle with 1 pre-Coarse-Grid (α1 = 1)
correction sweep & 1 post-Coarse-Grid (α2 = 1)
correction sweep.
• Cost of a V-cycle (in WUs):

2− d 2 − 2d 2 − 3d 2− Md
2
2 ( 1+ + + + ... + ) <
1 − 2− d
• Cost is about 4, 8/3, & 16/7 WUs per
V-cycle in 1, 2, & 3 dimensions.
U-Boulder 89 of 312
 Convergence analysis
• First, a heuristic argument:
– The convergence factor for the oscillatory error modes
(smoothing factor) is small & bounded uniformly in h.

smoothing factor = max |λk(R)| for N/2≤k≤N-1.

– Multigrid focuses the relaxation process on attenuating the
oscillatory components on each level.

Relax 1st coarse grid Relax on fine grid

smooth oscillatory
k=1 k=N/2 k=N

⇒ The overall multigrid convergence factor is

small & bounded uniformly in h !

Bounded uniformly in h ≠ independent of h.

U-Boulder 90 of 312
 Convergence analysis
a little more precisely...
• Continuous problem: Au = f, u i = u(x i)
• Discrete problem: Ah uh = fh, vh ≈ uh

• Global/discretization error: Ei = u(x i) - uih

||E || ≤ Kh p
(p = 2 for model problem & proper norm)

• Algebraic error: eih = uih - vih

• For tolerance ε, assume the aim is to find vh so that the

total error ||e || = ||u(h) - vh || ≤ ε , where u(h) = (u (x i)).
• Then this objective is assured as follows:
||e || ≤ ||u (h) - u h || + ||u h - v h || = ||E|| + ||e h|| ≤ ε .

U-Boulder 91 of 312
 We can satisfy the convergence
objective by imposing two conditions

1) ||E|| ≤ ε/2. Achieve this condition by choosing an

appropriately small grid spacing h*:
K(h*)p = ε/2
2) ||e h|| ≤ ε/2. Achieve this condition by iterating until
||E|| ≤ ε/2 = K(h*)p on grid h; then we’ve
converged to the level of discretization error.

→ Once discretization error & algebraic error

are in balance, then it would be better to
go to grid h/2 than to iterate more!
U-Boulder 92 of 312
 Convergence to the level
of discretization error
• Use an MV scheme with convergence factor γ < 1
bounded uniformly in h (fixed α1 & α2).

• Assume a d-dimensional problem on an NxNx…xN

grid with h = N -1.

• Initial error is ||e h || = ||u h - 0 || = ||u h || = O(1) .

• Must reduce this to ||e h || = O(h p) = O(N -p) .

• We can determine θ, the number of V-cycles needed

to do this, if we can bound its convergence factor γ.

U-Boulder 93 of 312
 Work to converge to the level
of discretization error
• Using θ V-cycles with convergence factor γ gives an
overall convergence factor of γ θ.
• We must therefore have γθ ∼ O( N − p),
or .
θ ∼ O( l og N )
• Since 1 V-cycle costs O(1) WUs & 1 WUs is O(N d ),
then converging to the level of discretization error
using the MV method costs

d
O( N direct
• This compares to fast )
l og Nmethods (fast Poisson
solvers). But we’ll see that MV can do even better.

U-Boulder 94 of 312
 Numerical example

• Consider the 2-D model problem (with σ = 0):

− ux x − uy y = 2 ( 1 − 6x 2) y 2 ( 1 − y 2) + ( 1 − 6y 2) x 2 ( 1 − x 2)
in the unit square, with u = 0 Dirichlet boundary.

• The solution to this problem is

u( x , y ) = ( x 2 − x 4 ) ( y 4 − y 2)

• We examine the effectiveness of MV cycling to

solve this problem on NxN grids [(N-1) x (N-1)
interior points] for N = 16, 32, 64, 128.

U-Boulder 95 of 312
 Numerical results
MV cycling

Shown are the results of

16 V(2,1)-cycles. We
display, after each cycle,
residual norms, total
error norms, & ratios of
these norms to their
values after the previous
cycle.
N = 16, 32, 64, 128.

||r h||h = h ||r h||2

scaled residual error
||e ||h = h ||u (h) - v h||2
scaled discrete total error
U-Boulder 96 of 312
 A warning about bounds
• Bounds like ||en + 1|| ≤ γ ||en || & ||u (h) - u h|| = O(h)
are only just that--bounds!
• If you see behavior that suggests that these
bounds are sharp (e.g., halving h halves the
discretization error), then great. If you don’t see
this behavior, don’t assume things are wrong.

• Think about this:

O(h2 ) = O(h) but generally O(h) ≠ O(h2 ) !!!
(Any process that is O(h2) is also O(h),
but the converse isn’t necessarily true.)

U-Boulder 97 of 312
 Reconsideration
You want to approximate uh.
A good iteration is the V-cycle.
What’s a good way to start it?
Can you do better than vh ← 0?
Start on the coarse grid!
Use nested iteration for the V-cycle!

U-Boulder 98 of 312
 Look again at nested iteration
• Idea: It’s cheaper to solve a problem (fewer iterations)
if the initial guess is good.

• How to get a good initial guess:

– “Solve” the problem on the coarse grid first.
– Interpolate the coarse solution to the fine grid.

• Now, let’s use the V-cycle as the solver on each grid

level! This defines the Full Multigrid (FMG) cycle.

U-Boulder 99 of 312
 Full multigrid (FMG)
v h ← FM G ( f h )
• Initialize f h , f 2h , f 4h , . . . , f H

• Solve on coarsest grid −1 H

v H = ( A H) f

• Interpolate initial guess

• Perform V-cycle v 2h ← I 42hh v 4h
• Interpolate initial guess v 2h ← MV 2h ( v 2h , f 2h )

• Perform V-cycle v h ← I 2hh v 2h

v h ← MV h ( v h , f h )

U-Boulder 100 of 312

 FMG-cycle
• Restriction
• Interpolation
• High-Order Interpolation?

U-Boulder 101 of 312

 FMG-cycle (recursive form)
v h ← FM G ( f h ) , η

1) Initialize f h , f 2h , f 4h , . . . , f H

2) If h is the coarsest grid, then solve exactly;

Ω
else:
v 2h ← FM G ( f 2h )

3) Set initial guess: .

v h ← I 2hh v 2h
4) Perform η times.
v h ← M V( v h , f h )

U-Boulder 102 of 312

 FMG cycle cost

One2 V-cycle is performed per level, at a cost of

1 − 2 − d WUs per grid (where the WU is for the
size of the finest grid involved). − jd
The size of the WU for coarse-grid j is 2 times
the size of the WU for the fine grid (grid 0).
Hence, the cost of the FMG(1,1) cycle in WUs is
less than
 2  −d − 2d 2
  ( 1 + 2 + 2 + . . . ) =
 1− 2−d  −d 2
( 1−2 )
d = 1: 8 WUs; d = 2: 32/9 WUs; d = 3: 128/49 WUs.

U-Boulder 103 of 312

 Has discretization error
been reached by FMG?

• If discretization error is achieved, then ||eh|| = O(h2)

& the error norms at the “solution” points in the cycle
should form a Cauchy sequence:

||eh|| ≈ 0.25 ||e2h||

Let’s see this algebraically…

U-Boulder 104 of 312
 Interpolation stability
how interpolation affects error

• Property: || P e 2h || ≤ β|| e 2h ||
• Reasoning:
|| P e 2h || ≤ ||P|| || e 2h ||
= ||PT P||1/2 || e 2h ||

⇒ ||PT P||1/2 ≤ √2

U-Boulder 105 of 312

 Approximation property
how the discrete solutions approximate each other

Property: ||u h - P u 2h || ≤ αKh 2

Reasoning:
|(u (h) - P u (2h))i| = |u(xi) - (u(xi - 1 ) + u(xi + 1))/2| ≤ |u”(ξ)|h 2/2
⇒ ||u (h) - P u (2h) || ≤ cKh 2
⇒ ||u h - P u 2h ||
≤ ||u h - u (h) || + ||u (h) - P u (2h) || + || P u (2h) - P u 2h ||
≤ Kh 2 + cKh 2 + || P ||·||u (2h) - u 2h ||
≤ Kh 2 + cKh 2 + βK(2h)2
≤ (1 + c + 4 β)Kh 2

U-Boulder 106 of 312

 FMG accuracy
||e 2h || ≤ K(2h)2
Assume:
||e 2h || ≤ K(2h)2 induction hypothesis
||u h - P u 2h || ≤ αKh 2 approximation property (α≈5)
|| P w 2h || ≤ β|| w 2h || interpolation stability (β≈1)
Triangle inequality:
||e h || = ||u h - P v 2h ||
≤ ||u h - P u 2h || + || P (u 2h - v 2h)||
≤ αKh 2 + β K(2h)2
= (α + 4β)Kh 2
⇒ ||e h || ≤ “9”Kh 2
So we need only reduce ||eh|| by “0.1”!!!

U-Boulder 107 of 312

 Numerical example

• Consider again the 2-D model problem (with σ = 0):

− ux x − uy y = 2 ( 1 − 6x 2) y 2 ( 1 − y 2) + ( 1 − 6y 2) x 2 ( 1 − x 2)
inside the unit square, with u = 0 on the boundary.

• We examine the effectiveness of FMG cycling to

solve this problem on NxN grids [(N-1) x (N-1)
interior] for N = 16, 32, 64, 128.

U-Boulder 108 of 312

 FMG results
3 FMG cycles & comparison with MV cycle results

1.03e-04
2.58e-05
6.44e-06
1.61e-06

||e ||h = h ||u (h) - v h||2

scaled discrete total error
U-Boulder 109 of 312
 Diagnostic tools
for debugging the code, the method, the problem

• Finding mistakes in codes, algorithms, concepts, & the

problem itself challenges our scientific abilities.
• This challenge is especially tough for multigrid:
– Interactions between multilevel processes can be very subtle.
– It’s often not easy to know how well multigrid should perform.
• Achi Brandt:
– “The amount of computational work should be proportional to
the amount of real physical changes in the computed solution.”
– “Stalling numerical processes must be wrong.”
• The “computational culture” is best learned by lots of
experience & interaction, but some discussion helps.

U-Boulder 110 of 312

 Tool # 1: Be methodical

• Modularize your code.

• Test the algebraic solver first.
• Test the discretization next.
• Test the FMG solver last.
• Beware of boundaries, scales, & concepts.
• Ask whether the problem itself is well posed.

U-Boulder 111 of 312

 Tool # 2: Start simply

• Start from something that already works if you can.

• Introduce complexities slowly & methodically, testing

thoroughly along the way.

• Start with a very coarse fine grid (no oxymoron intended).

• Start with two levels if you can, using a direct solver or lots

of cycles on coarse grids if nothing else.

U-Boulder 112 of 312

 Tool # 3: Expose trouble

Start simply, but don’t let niceties mask trouble:

• Set reaction/Helmholtz terms to zero.

• Take infinite or very big time steps.

• Don’t take 1-D too seriously, not even 2-D.

U-Boulder 113 of 312

 Tool # 4: Test fixed point property
Relaxation shouldn’t alter the exact solution of
the linear system (up to machine precision).
• Create a right side: f = Au* with u* given.
• Make sure u* satisfies the right boundary conditions.
• Test relaxation starting with u* : Is r = 0, is it zero
after relaxation, does u* change?
• Test coarse-grid correction starting with u* : Is the
correction zero?

U-Boulder 114 of 312

 Tool # 5: Test on Au = 0

• The exact solution u* = 0 is known!

• Residual norm ||Au|| & error norm ||u|| are computable.
• Errors ||Au|| & ||u|| should eventually decrease steadily
with a rate that might be predicted by mode analysis.
• Multigrid can converge so fast that early stalling suggests
trouble when it’s just that all machine-representable
numbers in a nonzero u* have already been computed!
Computing r = f - Au & updating u shouldn’t have trouble
with machine precision if you have u* = 0 & thus f = 0.

U-Boulder 115 of 312

 Tool # 6: Zero out residual

• Using a normal test, try multiplying the residual by 0

before you go to the coarse grid.

• Check to see that the coarse-grid corrections are 0.

• Compare this test with a relaxation-only test--the

results should be identical.

U-Boulder 116 of 312

 Tool # 7: Print out residual norms

• Use the discrete L 2 norm:

||r||h = (hd Σ ri2)1/2 = hd/2 ||r ||2

• Output ||r||h after each pre- & post-relaxation sweep.
• These norms should decline to zero steadily for each h.
• The norm after post-relaxation should be consistently
smaller than after pre-relaxation--by the predictable
convergence factor at least.

U-Boulder 117 of 312

 A warning about residuals
• Residuals could mask large smooth errors:
– If ||e||=O(h2), then ||r||=O(h2) for smooth e, but
||r||=O(1) for oscillatory e. (λN /λ1 = O(h-2)!!!)
– This means that if we are controlling the error by
monitoring ||r||, if we don’t know the nature of e,
& if we don’t want to risk large e by requiring only
||r||=O(1), then we may have to work very hard to
make ||r||=O(h2). (We could in effect be trying to
make ||e||=O(h4) !!!)
• Multigrid tends to balance the errors, so ||r||=O(h2)
tends to mean ||e||=O(h2).

U-Boulder 118 of 312

 Tool # 8: Graph the error

• Run a test on a problem with known solution (Au = 0).

• Plot algebraic error before & after fine-grid relaxation.

• Is the error oscillatory after coarse-grid correction?

• Is the error much smoother after fine-grid relaxation?

• Are there any strange characteristics near boundaries,

interfaces, or other special phenomena?

U-Boulder 119 of 312

 Tool # 9: Test two-level cycling

• Replace the coarse-grid V-cycle recursive call with a direct

solver if possible, or iterate many times with some method
known to “work” (test ||r || to be sure it’s very small), or use
many recursive V-cycle calls.

• This can be used to test performance between two coarser

levels, especially if residual norm behavior identifies trouble
on a particular level.

U-Boulder 120 of 312

 Tool # 10: Beware of boundaries
• Boundaries usually require special treatment of the
stencils, intergrid transfers, & sometimes relaxation.
• Special treatment often means special trouble, typically
exposed in later cycles as it begins to infect the interior.
• Replace the boundary by periodic or Dirichlet conditions.
• Relax more at the boundary, perhaps using direct solvers.
• Make sure your coarse-grid approximation at the boundary
is guided by good discretization at the fine-grid boundary.

U-Boulder 121 of 312

 Tool # 11: Test for symmetry

• If your problem is symmetric or includes a symmetric

case, test for it.

• Check symmetry of the fine-grid & coarse-grid matrices:

are aij & aji relatively equal (to machine precision).

• Be especially watchful for asymmetries near boundaries.

U-Boulder 122 of 312

 Tool # 12: Check for compatibility
• This is a bit ahead of schedule, but consider the problem
-u ” = f with u ’(0) = u ’(1) = 0.

• It’s singular: If u = 1, then -u ” = 0 & u ’(0) = u ’(1) = 0.

• It’s is solvable iff f ∈ Range(∂xx) = η⊥(∂xx ) = {1}⊥ or f ⊥ 1.

• First fix the grid h right side: f h ← f h - (<f h, 1>/<1,1>)1.

• Do this on coarse grids too: f 2h ← f 2h - (<f 2h, 1>/<1,1>)1.

• Uniqueness is also a worry: u h ← u h - (<u h, 1>/<1,1>)1.

U-Boulder 123 of 312

 Tool # 13: Test for linearity

• Again, this is a bit ahead of schedule, but if you’re writing

a nonlinear FAS code, it should agree with the linear code
when you test it on a linear problem. Try it.

• Move gradually to the target nonlinear test problem by

putting a parameter in front of the nonlinear term, then
running tests as the parameter changes slowly from 0 to 1.

U-Boulder 124 of 312

 Tool # 14: Use a known PDE solution
• Set up the source term (f = Au* in Ω) & data (g = u* on Γ).
• Do multigrid results compare qualitatively with sampled u* ?
• Monitor ||u* - u h||h .
• Test a case with no discretization error (let u* = 2nd degree
polynomial). The algebraic error should tend steadily to 0.
• Test a case with discretization error (let u* have a nonzero
3rd derivative). The algebraic error should decrease rapidly
at first, then stall at discretization error level. Check error
behavior as you decrease h. Does it behave like O(h2) (h
halved ⇒ error halved) or whatever it should behave like?

U-Boulder 125 of 312

 Tool # 15: Test FMG accuracy

• Make sure first that the algebraic solver converges as

predicted, with uniformly bounded convergence factors.

• Test the discretization using Tool # 14.

• Compare FMG total error to discretization error for

various h. You might need to tune the FMG process here
(play with the number of cycles & relaxation sweeps).

U-Boulder 126 of 312

 Computing assignments
• Document:
norms/weights, V(ν1, ν2), errors, labels (table, graph)
• Use various scenarios:
Ax = 0, Ax = f, varying N & νi & ω, Jacobi/Gauss-Seidel
• Thoroughly test:
don’t stop until you get what you expect.
compare with known solution, text, others.
study discretization & algebraic errors.
report on “asymptotic” factors.
• Be kind to the reader:
code = zzz… tables = + tables ok = ++
& graphs
by hand
↓
tables & graphs & discussion (clear, ↓ = +++
concise)
• Discuss, discuss, discuss:
what do you see & think? what did you learn?

U-Boulder 127 of 312

 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems • Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
– Analysis – Neumann boundaries
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
√ – Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Matrix coarsening
– Complexity
√ • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology

U-Boulder 128 of 312

 5. Some theory
What is A2h?
• Recall the 2-grid coarse-grid correction scheme:
– 1) Relax on Ah u h = f h on Ωh to get v h.
– 2) Compute f2h = Ih2h (f h - Ah v h) .
– 4) Solve A2h u 2h = f 2h on Ω2h .
– 5) Correct fine-grid solution vh ← vh + I2h u2h.
h
• Assume that eh ∈ Range(I2h ). Then the residual
equation can be written h
rh = Aheh = Ah I2h u2h for some u2h ∈ Ω2h.
h 2h
This characterizes u , but with too many equations.
• How does Ah act on Range(I2h )?
h
U-Boulder 129 of 312
 h
How does A act on R a n ge ( I 2hh ) ?

u 2h

I 2hh u 2h

h h 2h
A I 2h u

• Therefore, the odd rows of Ah I2hh are zero (for 1-D

only) & r2i + 1 = 0. We therefore keep the even rows of
h the residual equations on Ω2h. These rows
Ah I2h for
can be picked out by applying restriction!

I h2h A h I 2hh u 2h = I h2h r h

U-Boulder 130 of 312
 Building A2h: The Galerkin condition

• The residual equation on the coarse grid is

I h2h A h I 2hh u 2h = I h2h r h

• We thus identify the coarse-grid operator as

2h 2h h h
A = I h A I 2h A2h = RAP

• How do we know RAP is symmetric?

If PT = R & RT = P, then
(RAP)T = PTATRT = RAP.
U-Boulder 131 of 312
 Computing the i th row of A 2h.
• Compute A 2h e 2i h where e 2i h = ( 0, 0, . . . , 0, 1, 0, . . . , 0) T
↑ Why is this the i th row of A2h ?

i −1 i i +1
2h
0 1 0 ei
1 1 h 2h
0 2 1 2 0 I 2h e i
1 1 1 h h 2h
− 0 0 − A I 2h e i
2 2
2h h 2h 2
1 1 1 2h h h 2h
− − I h A I 2h e i
2 2
4h 2h 4h 2
U-Boulder 132 of 312
 The i th row of A 2h looks a
lot like the i th row of A h !
• The i th row of 2h is 1
A −1 2 −1 ,
( 2h) 2
which is the 2h version of h.
Ω A
• Note that IF relaxation on h leaves only error
in the range of interpolation,Ωthen solving
A 2h u 2h = f 2h
determines the error exactly!
• This is generally not feasible, but this logic
leads to a very plausible representation for .

A 2h
U-Boulder 133 of 312
 Variational properties of coarsening
• The definition for A 2h that resulted from the
foregoing line of reasoning is useful for both
theoretical & practical reasons. Together with
the commonly used relationship between
restriction & prolongation, we have the
variational properties:

A 2h = I h2h A h I 2hh Galerkin Condition

h 2h T
I 2h = c ( I h ) for c in ℜ.
U-Boulder 134 of 312
 Properties of restriction
in a little more detail…

2h h 2h or
• Full Weighting: Ih :Ω → Ω
2h ℜN-1 ℜN/2-1
Ih :
• N = 8:
1 2 1
2h 1
Ih = 1 2 1
4
1 2 1 3× 7

• has rank & null space η( ) with dim .

2h N N
Ih 2
−1 I 2h
h 2
U-Boulder 135 of 312
 Spectral properties of restriction

• How does I 2h act on the eigenvectors of h?

h A
• Consider  j kπ  , 1 ≤ k ≤ N - 1, 0 ≤ j ≤ N.
h = s in
wk, j  N 
 
• A little algebra & trigonometry shows that

kπ  2h
( I h2h wkh ) = co s 2 2N  wk , j
j  
for 1 ≤ k ≤ N/2. ≡ c k wk2,hj

U-Boulder 136 of 312

 Spectral properties (cont’d)
• i.e., I 2h [kth mode on h] = c [ kth mode on 2h ]
h Ω k Ω

• • •
• h : N = 8, k = 2
• • Ω
• • •

I h2h
• : N = 4, k = 2
• • • Ω 2h
•
U-Boulder 137 of 312
 Spectral properties (cont’d)
N N
• Let k ′ = N − k for 1 ≤ k < , so that < k′ < N .
2 2

• A little algebra & trigonometry shows that

kπ  2h
( I h2h wkh′ ) = − sin 2
 2N  wk , j
j  

≡ s k wk2,hj

U-Boulder 138 of 312

 Spectral properties (cont’d)
h] = -sk [ kth mode on Ω 2h]
I h2h [(N-k) mode on
i.e., th
Ω
•
• •
h : N = 8, k = 6
• • Ω
• •
•
2h
Ih
• : N = 4, k = 2

•
• 2h
• Ω
•
U-Boulder 139 of 312
 Spectral properties (cont’d)

• Summarizing:
I h2h wkh = c k wk2h
}{
N
1<k <
2
I h2h wkh′ = − s k wk2h k′ = N − k

I h2h wN
h
/2 = 0
• Complementary modes:

W k = span { wkh , wkh′ }

I h2h W k → {wk2h}∞

U-Boulder 140 of 312

 Null space of restriction

• Observe that η(Ih2h) = span(Ahêih), where i is odd &

êih is the i th unit vector.

• Let ηi = Ahêih.

• While the ηi looks oscillatory, 0 − 1 contains

0 0it 0generally 2 −1 0 0
all Fourier modes of :
h
A
N −1
ηi = ∑ a k wk ak ≠ 0
• All the Fourierkmodes
=1 of are needed to
represent the null space of restriction!
h
A

U-Boulder 141 of 312

 Properties of interpolation

• Interpolation: I h : Ω 2h → Ω h or
2h
h : ℜN/2-1 ℜN-1
I 2h
• N = 8: 1
2
1 1
1
I 2hh = 2
2
1 1
2
• has full rank & null space
1 {0}.

I 2hh
U-Boulder 142 of 312
 Spectral properties of interpolation

• How does I h act on the eigenvectors of 2h ?

2h A

• Consider  j kπ  , 1 ≤ k ≤ N/2-1,
( wk ) = sin 
2h

j  N / 2  0 ≤ j ≤ N/2.

• A bit of work shows that the modes of are

NOT “preserved” by A 2h
, but that the space is
“preserved”: I 2hh Wk

kπ  h  kπ  h
I 2hh wk2h = co s 2
 2N  wk − sin  2N
2
 wk ′
   
= c k wkh − s k wkh′
U-Boulder 143 of 312
 Spectral properties of interpolation
I2hh wk2h = ck wkh − sk wkh′

• Interpolation of smooth Ω 2h modes excites

oscillatory modes on h.
Ω
• Note that if N , then
k «
2
  2   h  k2  h
k
I 2hh wk2h =  1 − O   wk + O  2  wk ′
2
 N   N 

• is 2nd-order
≈w interpolation.
h
k
h
I 2h
U-Boulder 144 of 312
 Range of interpolation

• The range of h is the span of the columns of h .

I 2h I 2h
• Let be the i th column of h .
ξi I 2h
ξi :

N −1
h
bk wkh ,
ξi = ∑ bk ≠ 0
=
k 1
• All the Fourier modes of are needed to represent
Range( ).
Ah
h
I 2h
U-Boulder 145 of 312
 Use all the facts to analyze the
coarse-grid correction scheme
relax only before correction

1) Relax once on Ωh : vh ← R vh + B f h
2) Compute & restrict residual f 2h ← I h2h ( f h − A h v h )
3) Solve residual equation
2h − 1 2h
v =(A )
2h f
4) Correct fine-grid solution .
v h ← v h + I h v 2h
• The entire process appears as 2h

v hexactα vh h 2h − 1 2h h h
• The ← R solution
I satisfies
+ 2h ( A ) Ih ( f − A ( R v h + B f h ))

v h = R uh + I2hh ( A 2h ) − 1 Ih2h ( f h − A h ( R uh + B f h ))
U-Boulder 146 of 312
 CG error propagation

• Subtracting the previous two expressions, we get

h 2h − 1 2h h
eh ← I − I2h ( A ) Ih A R e h

e h ← CG e h
• How does CG act on the modes of ? Assume e h
consists of the modes & Ah
for
& . wkh h
wk′ 1 ≤ k ≤ N /2
k′ = N − k
• We know how
act on & .α h 2h 2h − 1
R , A , I , (A )h , Ih 2h
wkh h
wk′

U-Boulder 147 of 312

 CG error propagation
• For now, assume no relaxation. Then
W k = spa n { wkh , wkh′ }
is invariant under CG:

CG wkh = s k wkh + s k wkh′

CG wkh′ = c k wkh + c k wkh′
where

 kπ   kπ 
c k = co s 2   s k = sin2  
 2N   2N 

U-Boulder 148 of 312

 CG error propagation for k <<N

• Consider the case k << N (extremely smooth &

oscillatory modes):

 k2   k2 
wk → O  + O
2  k 2  k′
w w
N  N 

  k2     k2  
wk ′ →  1 − O  +  1 − O
2  k 2   k′
w w
 N   N 
• Hence, CG eliminates the smooth modes but does
not damp the oscillatory modes of the error!

U-Boulder 149 of 312

 CG with relaxation

• Next, include one relaxation sweep. Assume that

the relaxation R preserves the modes of A h
(although this is often unnecessary). Let
λk
denote the eigenvalue of associated with .
For k < < N/2: R wk

wk → λk sk wk + λk sk wk ′ Small!

wk ′ → λk ′ ck wk + λk ′ ck wk ′ Small!

U-Boulder 150 of 312

 Crucial observation

• Between relaxation & coarse-grid correction,

both smooth & oscillatory components of the
error are effectively damped.

• This is the “spectral” picture of how multigrid

works. We examine now another viewpoint,
the “algebraic” picture of multigrid.

U-Boulder 151 of 312

 Recall the variational properties

• All the analysis that follows assumes that the

variational properties hold:

2h 2h h h
A = I h A I 2h Galerkin Condition

h 2h T
I 2h = c ( Ih ) for c in ℜ.

U-Boulder 152 of 312

 Algebraic interpretation of CG
h& 2h .
• Consider the subspaces that make up Ω Ω
From now on, ‘R( )’ refers
to the Range of a linear
h
Ω R ( I 2hh ) N ( I h2h ) operator & ‘N( )’ to its
Null Space.

h From the Fundamental Theorem

I 2h of Linear Algebra:
I h2h
2h 2h T
N( Ih ) ⊥ R( ( Ih ) )
or
Ω 2h R ( I h2h )
N ( I h2h ) ⊥ R ( I 2hh )

But we really care about N (Ih2h Ah ) !?!?

U-Boulder 153 of 312
 Subspace decomposition of Ω h
• If uh ∈ N (Ih2h Ah ), then for any u 2h we have
0 = 〈 Ih2h A hu h, u 2h 〉 = 〈 Ahu h, I2hhu 2h 〉 ,
so
R ( I h ) ⊥ h N ( I 2h A h ) ,
2h h
A
where means .
“energy”
x ⊥any
• Moreover, h y can be written
h
A x , y as= 0 inner product,
A
h h
where e& . e h = s h + t
h
• Hence, s h ∈ R ( I 2h ) t h ∈ N ( I h2h A h )
.

Ω h = R ( I 2hh ) ⊕ N ( I h2h A h )

U-Boulder 154 of 312

 Characteristics of the subspaces
• Since s h = I 2hh q2h for some q 2h ∈ Ω 2h, we
associate s h with the smooth components of e h .
But, s h generally has all Fourier modes in it.
Recall the basis vectors for h :
I 2h

• Similarly, we associate with oscillatory

components of , although t h h generally has all
Fourier modes ine hit as well. Recall
t that is
spanned by , so is spanned 2h by
the unit vectors h ( I h odd
Nfor ) i,
2h h
which “look”ηoscillatory.
i = A ei N ( Ih A )
h T
e i = ( 0, 0, . . . , 0, 1, 0, . . . , 0)

U-Boulder 155 of 312

 Algebraic analysis CG
• Recall that (without relaxation)
h 2h − 1 2h h
CG = I − I 2h ( A ) Ih A
• First note that if h , then .
This follows since s ∈ R ( I 2h )
h G sh = 0
for Csome
& therefore s h = I 2hh q 2h q 2h ∈ Ω 2h

2h − 1 2h h
CGsh = I h
− I2h ( A ) Ih A h 2h
I2h q =0
• It follows that N(CG)=R( ), that is, 2htheby null
A Galerkin property
space of CG is the range ofh interpolation.
I2h

U-Boulder 156 of 312

 More algebraic analysis of CG

• Next, note that if t h ∈ N ( I h2h A h ) , then

h h 2h − 1 2h h h
CG t = I − I2h ( A ) Ih A t

0
h h
⇒ CG t = t
• Thus, CG is the identity on .
2h h
N ( Ih A )

U-Boulder 157 of 312

 How does the algebraic picture
fit with the spectral view?
h in two ways:
• We may view Ω

h = Low frequency modes

Ω 1 ≤ k ≤ N/2
High frequency modes
N/2 < k < N

that is,

h
Ω = L ⊕ H
or

h h 2h h
Ω = R ( I 2h ) ⊕ N( Ih A )
U-Boulder 158 of 312
 Actually, each view is just part
of the picture
• The operations we have examined work on different
spaces!

• While 2h h is mostly oscillatory, it isn’t ,

N ( I h A ) is mostly smooth, it isn’t H
& while .
h
R ( I 2h ) L
• Relaxation eliminates error from .
H
• Coarse-grid correction eliminates error from .
R ( I 2hh )

U-Boulder 159 of 312

 How it actually works (cartoon)
H N ( I h2h A h )

eh
th

R ( I 2hh )
sh

U-Boulder 160 of 312

 Relaxation eliminates H, but
increases the error in R ( I 2hh ).
H N ( I h2h A h )

h
t L
eh
R ( I 2hh )
sh

U-Boulder 161 of 312

 h
CG eliminates error in R ( I 2h ),
but increases error in H
H N ( I h2h A h )

h
t L
eh
R ( I 2hh )
sh

U-Boulder 162 of 312

 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems • Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
– Analysis – Neumann boundaries
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
√ – Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Matrix coarsening
– Complexity
√ • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology
√

U-Boulder 163 of 312

 Some reflection

• What do you want out of this course?

• Do you seem to be getting there?

• Can we get there more directly?

• Start thinking about your project!

• Think about doing it in two parts.

U-Boulder 164 of 312

 6. Nonlinear problems

• How should we approach the nonlinear system

A ( u) = f
& can we use MG to solve it?

• A fundamental relation we’ve relied on is the linear

residual equation:

We A u −rely
can’t A v on=this
f −now ⇒ Aa enonlinear
A v since = r A(u)
generally means

A ( u ) − A ( v ) ≠ A ( e)
U-Boulder 165 of 312
 The nonlinear residual equation

• We still base our development around the residual

equation, now the nonlinear residual equation:

A ( u) = f
A ( u) − A ( v ) = f − A ( v )

A ( u) − A ( v ) = r
• How can we use this equation as the basis for a
solution method?

U-Boulder 166 of 312

 Let’s consider Newton’s method

• Best known & most important nonlinear problem solver!

• We wish to solve F(x) = 0 .
• Suppose for the moment that F is scalar F : ℜ→ ℜ.
• Expand F in a Taylor series about x :

F( x + s) = F( x) + sF ′( x) + s 2 F ″ ( ξ)
• Dropping higher order terms (hot), if x + s is a solution,

= F(arrive
• We0thus x) + sat
F ′Newton’s
( x) ∴ s = − F( x ) / F ′ ( x )
method:

F( x )
x← x −
F ′( x)
U-Boulder 167 of 312
 Newton’s method for systems
• The system A(u) = f in vector form is

 a1 (u1 ,u2 ,...,u N )   f1 

   
 a2 (u1 ,u2 ,...,u N )  =  f2 
 M  M
   
a N (u1 ,u2 ,...,u N )  f N 
• Expanding A(v + e) in a Taylor series about v :

A ( v + e) = A ( v ) + J( v ) e + hot
 ∂a 
J(u) =  i 
U-Boulder
 ∂u j  168 of 312
 Newton for systems (cont’d)
• J(v) is the Jacobian
 ∂a1 ∂a1 ∂a1 
 ∂u L
∂u2 ∂u N 
 1 
 ∂a2 ∂a2
L
∂a2 
J(v) =  ∂u1 ∂u2 ∂u N 
 M M O M 
 ∂a N ∂a N ∂a N 
 L 
 ∂ ∂
• If u = v + e is a solution,
u u ∂ 
2 f = A(v)N + J(v)e + hot, so
u
1
u=v
e ≈ [J(v)]-1(f - A(v))
• This leads to the iteration

v ← v + [J(v)]-1(f - A(v))
U-Boulder 169 of 312
 Newton’s via the residual equation
• The nonlinear residual equation is

A ( v + e) − A ( v ) = r
• Expanding A(v + e) in a two-term Taylor series about v
& ignoring hot:

A ( v ) +J( v ) e − A ( v ) = r
• Newton’s method is J( v)
thus: e =r

r = f − A( v)
−1
v ← v + J( v) r
U-Boulder 170 of 312
 How does multigrid fit in?

• One obvious method is to use multigrid to solve

J(v)e = r at each iteration step. This method is
called Newton-MG & can be very effective.

• However, we would like to use multigrid ideas to

treat the nonlinearity directly.

• Hence, we need to specialize the multigrid

components (relaxation, grid transfers,
coarsening) for the nonlinear case.

U-Boulder 171 of 312

 What is nonlinear relaxation?
• Several of the common relaxation schemes have
nonlinear counterparts. For A(u) = f, we describe
nonlinear Gauss-Seidel:

For each j = 1, 2, …, N :
Set the jth component of the residual to 0
& solve for vj . That is, solve (A(v))j = fj .

• Equivalently,
For each j = 1, 2, …, N :
Find s ∈ ℜ such that ,
where εj is the canonical jth unit basis vector.
( A ( v + s εj ) ) j = f j

U-Boulder 172 of 312

 How is nonlinear Gauss-Seidel done?
• Each ( A ( v ) ) j = f j is a nonlinear scalar equation for
vj . We can use the scalar Newton’s method to solve!

• Example: ( ) may be
− uso″ (that
discretized x ) + u( x ) e u x =isf given
, by
( A( v) ) j = f j
− v j − 1 + 2v j − v j + 1
vj
2
1 ≤ j ≤ N −1
+ vj e = fj
h iteration for v is given by
• Newton j

− vj − 1 + 2 vj − vj + 1 vj
2
+ vj e − fj
h
vj ← vj −
2 vj
2
+ e ( 1 + vj)
U-Boulder
h 173 of 312
 How do we do coarsening for
nonlinear multigrid?
• Recall the nonlinear residual equation

A ( v + e) − A ( v ) = r
• In multigrid, we obtain an approximate solution vh
on the fine grid, then solve the residual equation
on the coarse grid.

• The residual equation on appears as

Ω 2h

A 2h ( v 2h + e 2h ) − A 2h ( v 2h ) = r 2h

U-Boulder 174 of 312

 Look at the coarse residual equation
• We must evaluate the quantities on Ω 2h in
2h 2h 2h 2h
A (v
2h +e ) −A 2h (v ) =r
• Given vh, a fine-grid approximation, we restrict
the residual to the coarse grid:

r 2h = I h2h ( f h − A h ( v h ) )
• For v2h we restrict vh by
v 2h = I h2h v h
Thus,

A 2h ( Ih2h v h + e2h ) = A 2h ( Ih2h v h ) + Ih2h ( f h − A h( v h) )

U-Boulder 175 of 312

 We’ve obtained a coarse-grid
equation of the form A2h(u2h) = f2h
• Consider the coarse-grid residual equation:

A 2h ( Ih2h v h + e2h ) = A 2h ( Ih2h v h ) + Ih2h ( f h − A h( v h) )

u 2h f 2h
coarse-grid unknown all quantities are known
• We solve for &
obtain A 2h ( u2h) = f 2h u 2h = I h2h v h + e 2h

e 2the
• We then apply u 2h − I h2h v h
h = correction:

h 2h
vh = vh + I 2h e
U-Boulder 176 of 312
 Full approximation scheme (FAS)
2-grid form
• Perform nonlinear relaxation on A h( u h) = f h to
obtain an approximation v h .
• Restrict the approximation & its residual

v 2h = I 2h v h r 2h = I 2h ( f h − A ( v h ) )
• Solve the coarse-grid
h equation h

• Extract 2Ah2happroximation
( u 2h) = A 2h( vto2hh) error
+ r 2h
• Interpolate & correct
e 2h = u 2h − v 2h

v h = v h + I 2hh e 2h
U-Boulder 177 of 312
 A few observations about FAS
A 2h ( Ih2h v h + e2h ) = A 2h ( Ih2h v h ) + Ih2h ( f h − A h( v h) )

• If A is a linear operator, then FAS reduces directly to

the linear two-grid correction scheme.

• An exact solution to the fine-grid problem is a fixed

point of the FAS iteration.

U-Boulder 178 of 312

 A few more observations about FAS
• The FAS coarse-grid equation can be written as
2h 2h
A ( u 2h) =f + τ2hh
where 2h h is the so-called tau
τh =term
correction A(Ih &v ) − Iish A(v
2h h
2h
2h
the )original 2h source term,
f
provided you choose it that way: .
2h 2h h
f =I h f
• In general, since , the solution to the FAS
coarse-grid equationτ2hh is
≠ not
0 the same asuthe 2h solution
to the original coarse-grid problem

A 2is
• The tau correction ( u ) = f 2halter the coarse-
h as 2ah way to
grid equation to enhance its approximation properties.

U-Boulder 179 of 312

 Still more observations about FAS

• FAS may be viewed as an inner & outer iteration:

the outer iteration is the coarse-grid correction,
the inner iteration the relaxation method.

• A true multilevel FAS process is recursive, using

FAS to solve the nonlinear Ω2h problem using Ω4h.
Hence, FAS is generally employed in a V- or W-
cycling scheme.

U-Boulder 180 of 312

 Even more observations about FAS
• For linear problems, we use FMG to obtain a good
initial guess on the fine grid. Convergence of
nonlinear iterations depends critically on having a
good initial guess.
• When FMG is used for nonlinear problems, the
interpolant is generally accurate enough to
I 2hh of
be in the basin u 22hhattraction of the fine-grid
solver.
• Thus, whether FAS, Newton, or Newton-multigrid
is used on each level, one FMG cycle should provide
a solution accurate to the level of discretization,
unless the nonlinearity is extremely strong.

U-Boulder 181 of 312

 Intergrid transfers for FAS
• Generally speaking, the standard operators (linear
interpolation, full weighting) work effectively in
FAS schemes.

• For strongly nonlinear problems, higher-order

interpolation (e.g., cubic interpolation) may be
beneficial.

• For an FMG scheme, where is the

interpolation of a coarse-grid hsolution
2h to become a
I 2h u interpolation
fine-grid initial guess, higher-order
is often advised.

U-Boulder 182 of 312

 What is A2h(u2h) in FAS?
As in the linear case, there are two basic possibilities:

1. A2h(u2h) is determined by discretizing the nonlinear operator

A(u) in the same fashion as was employed to obtain Ah(uh) ,
except that the coarser mesh spacing is used.

2. A2h(u2h) is determined from the Galerkin condition

where the action2h

of the2hGalerkin
2hproduct
h h can2hbe captured in an
A ( u
implementable formula. ) = hI A ( I2h u )

The first method is usually easier & more common.

U-Boulder 183 of 312

 Nonlinear problems: An example

• Consider
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
on the unit square [0,1] x [0,1] with homogeneous
Dirichlet boundary conditions & a regular h = 1/128
Cartesian grid.

• Suppose the exact solution is

u( x, y) = ( x 2 − x 3 ) sin ( 3 π y)

U-Boulder 184 of 312

 Discretization of nonlinear example
• The operator can be written (sloppily) as
−1
1   h u i,h j
− 1 4 − 1  ui, j + γ ui, j e
h = f i, j
2
h  −1 

h
A ( u h)
• Relaxation is given by

( A h( u h) ) − f i, j
i, j
uih, j ← uih, j −
4 uhi, j
2
+ γ ( 1 + ui , j ) e
h
h

U-Boulder 185 of 312

 FAS & Newton’s method on
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
N = 128
• FAS V(2,1)-cycles until ||r|| < 10-10
γ
1 10 100 1000
convergence factor 0.135 0.124 0.098 0.072
number of FAS cycles 12 11 11 10
• Newton’s Method with exact inner solves until ||r|| < 10-10

γ
1 10 100 1000
convergence factor 4.00E-05 7.00E-05 3.00E-04 2.00E-04
number of Newton iterations 3 3 3 4

U-Boulder 186 of 312

 Newton, Newton-MG, & FAS on
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
N = 128
• Newton uses exact solves, Newton-MG is inexact Newton
with a fixed number of inner V(2,1)-cycles for the Jacobian
problem, overall stopping criterion ||r|| < 10-10.

Outer Inner
Method iterations iterations Megaflops
Newton 3 1660.6
Newton-MG 3 20 56.4
Newton-MG 4 10 38.5
Newton-MG 5 5 25.1
Newton-MG 10 2 22.3
Newton-MG 19 1 24.6
FAS 11 27.1
U-Boulder 187 of 312
 Comparing FMG-FAS & FMG-Newton
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)

• We will do one FMG cycle using a single FAS V(2,1)

-cycle as the “solver” at each new level. We then
follow that with sufficiently many FAS V(2,1)-cycles
as is necessary to obtain ||r|| < 10-10.

• Next, we will do one FMG cycle using a Newton-

multigrid step at each new level (with a single linear
V(2,1)-cycle as the Jacobian “solver.”) We then
follow that with sufficiently many Newton-multigrid
steps as is necessary to obtain ||r|| < 10-10.

U-Boulder 188 of 312

 Comparing FMG-FAS & FMG-Newton
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
N = 128
Cycle || r h || || e h || Mflops || r h || || e h || Mflops Cycle
FMG-FAS 1.10E-02 2.00E-05 3.1 1.06E-02 2.50E-05 2.4 FMG-Newton
FAS V 6.80E-04 2.40E-05 5.4 6.70E-04 2.49E-05 4.1 Newton-MG
FAS V 5.00E-05 2.49E-05 7.6 5.10E-05 2.49E-05 5.8 Newton-MG
FAS V 3.90E-06 2.49E-05 9.9 6.30E-06 2.49E-05 7.5 Newton-MG
FAS V 3.20E-07 2.49E-05 12.2 1.70E-06 2.49E-05 9.2 Newton-MG
FAS V 3.00E-08 2.49E-05 14.4 5.30E-07 2.49E-05 10.9 Newton-MG
FAS V 2.90E-09 2.49E-05 16.7 1.70E-07 2.49E-05 12.6 Newton-MG
FAS V 3.00E-10 2.49E-05 18.9 5.40E-08 2.49E-05 14.3 Newton-MG
FAS V 3.20E-11 2.49E-05 21.2 1.70E-08 2.49E-05 16.0 Newton-MG
5.50E-09 2.49E-05 17.7 Newton-MG
1.80E-09 2.49E-05 19.4 Newton-MG
5.60E-10 2.49E-05 21.1 Newton-MG
1.80E-10 2.49E-05 22.8 Newton-MG
5.70E-11 2.49E-05 24.5 Newton-MG

U-Boulder 189 of 312

 Remembering coarse-grid correction
Au = f
• Relax (Jacobi) to smooth error e = u - v :
v ← v - D-1(Av - f) ↑
I2hh

• Form the residual equation Ae = r :

Ae = A(u - v) = f - Av = r
• Use premise that smooth error ⇒ e = I2he2h :
h
h
AI2he = r
2
h T
h fewer unknowns
• Use transpose I = (I2h) to reduce equations:
2

h 2h
Ih2hAI2he = I2hr
h
h h fewer equations

A2h
U-Boulder 190 of 312
 Motivating FAS for nonlinear A
A(v + e) = f → A2h(v2h + e2h) = f2h
2h 2h h 2h 2h h 2h h h
A ( Ih v + e 2h ) = A ( Ih v ) + Ih ( f − A ( v h) )

• Analogous to asking how to discretize a PDE of this form:

A(v + e) = f
with v known.
• Example:
γuu’ - u’’ = f → γ (v + e) (v + e)’ - (v + e)’’ = f
• Expand to get an equation in e of form g + ae + be’ + …:
γvv’ - v’’ + γ (v’e + ve’ + ee’ ) - e’’ = f

A(v)
U-Boulder 191 of 312
 Differential residual equation
γ (v’e + ve’ + ee’ ) - e’’ = f - A(v) ≡ r(v)

• Right side (analogous to injection):

r(v) → rjh = r(xj)
• Coefficients:
v → vjh = v(xj), v’ → (v’)jh = (v(xj+1) - v(xj-1))/(2h)
• Unknowns:
e → e j h, e’ → (e’)jh = (ej+1 - ej-1)/(2h),
e’’ → (e’’)jh = (ej+1 - 2ej h+ ej-1)/
h 2
h
h h h

U-Boulder 192 of 312

 Leading to FAS…
γuu’-u’’ = f → A(v+e) = f → γ (v’e+ve’+ee’ )-e’’ = f-A(v)

• Fine-grid residual equation (at h point 2j):

h h h h h h h h h
v 2 j +1 −v
h e2 j +1 − e2 j −1
2 j −1 h h e2 j +1 − e2 j −1 −e2 j +1 + 2e2 j − e2 j −1 h
γ e + γv2 j 2j + γe2 j + 2 = r2 j
2h 2h 2h h
• Coarse-grid residual equation (at 2h point j):

h h 2h 2h 2h 2h 2h 2h 2h
v −v
2j+2 2j−2 2h h e
j +1 −e
j −1 2h e j +1 −e j −1 −e
j +1 + 2e − e
j j −1 2h h
γ e + γv
j 2j + γe
j + 2 = (I r ) j
h
4h 4h 4h (2h)
• FAS

2h 2h h 2h 2h h2h 2h h h
h
A ( Ih v + e ) − A ( Ih v ) = Ih ( f − A (v ) )

U-Boulder 193 of 312

 Example: Newton-MG vs. FAS

• PDE (er, ODE):

− u ″ ( x ) + u( x ) e u( x ) = f ,

• Discretization:

− v j − 1 + 2v j − v j + 1
vj
+ vj e = fj
2
h

U-Boulder 194 of 312

 One Newton-MG step
• Step 0. Given v, form the grid h linear correction equation:
−e j −1 + 2e j − e j +1 −v j −1 + 2v j − v j +1
( )
vj vj
2 + + j
(1 v )e e j = rj ≡ f j − 2 + j
v e
h
Initialize the Newton correction approximation: e = 0. h
• Step 1: Relax on the grid h linear equation.
• Step 2: Solve the grid 2h error correction equation:

2h 2h 2h
−e j −1 + 2e j − e j +1 −e2 j −1 + 2e2 j − e2 j +1
• Step
+ + 2j
(1 v )e
v 2j 2h
e j = 2j −
r (
3: Correct the grid h Newton correction:
(2h)
2
h
2 + + 2j
(1 v ) e
v2j
e2 j )
• Step 4: Stop if you’vee “solved” hthe2 hlinear equation well
← + 2h e
e I
enough for Newton correction e & set v ← v + e. Else,
leave v alone & return to Step 1.

U-Boulder 195 of 312

 One FAS step
• Step 0. Given v, form the grid h nonlinear equation:
− vj − 1 + 2vj − vj + 1 vj
2 + vj e = fj
h
• Step 1: Relax on the grid h nonlinear equation to improve v.
• Step 2: Solve the grid 2h FAS correction equation:

2h 2h 2h
−e j −1 + 2e j − e j +1 h 2h v2h j + e 2h h
h v2 j 2h h
+ ( + ) − = ( h r )j
j
2 v2j e j e v2j e I
(2h) the grid h approximation v :
• Step 3: Correct

• Step 4: Stop if you’vev “solved”

← + 2h
v I the
h
e
2 hnonlinear problem well

enough for the solution u. Else, return to Step 1.

U-Boulder 196 of 312

 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems •√ Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
– Analysis – Neumann boundaries
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
√ – Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Matrix coarsening
– Complexity
√ • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology
√

U-Boulder 197 of 312

 7. Selected applications
7a. Neumann boundary conditions

• Consider the 1-D problem

− u ″ ( x) = f ( x) 0<x <1
u ′ (this
• We discretize 0) =onu ′the
( 1) interval
=0 [0,1] with 1
h=
grid spacing & nodes , j=1,2, …, N+1 . N +1
xj = j h
• We extend the interval with two ghost points

0 x 1

-1 0 1 2 3 … j ... N-1 N N+1 N+2

U-Boulder 198 of 312
 Central differences at boundary
• We use differences as before, but now also for
the derivative in the Neumann condition:

-1 0 1 j-1 j j+1 N N+1 N+2

u1 − u − 1 − uj − 1 + 2 uj − uj + 1 uN + 2 − uN
u′ ( 0) ≈ u″ ( x ) ≈ u′ ( 1) ≈
• This yields
2h the system
j
h2 2h

− uj − 1 + 2 uj − uj + 1
= fj 0 ≤ j ≤ N +1
h2

u1 − u − 1 uN + 2 − uN
=0 =0
2h 2h

U-Boulder 199 of 312

 Eliminating the ghost points

• Use the boundary conditions to eliminate u − 1 , uN + 2

u1 − u − 1 uN + 2 − uN
u − 1 = u1 =0 uN + 2 = uN
=0 2h
2h
• Eliminating the ghost points in the j = 0 & j = N + 1
equations gives the (N+2)x(N+2) system of equations:

− uj − 1 + 2 uj − uj + 1
= fj 0 ≤ j ≤ N +1
h2

2u0 − 2u1 − 2uN + 2uN + 1

= f0 = f N +1
2 2
h h
U-Boulder 200 of 312
 Write the system in matrix form
h h h , where
• We can write A u =f
 2 −2 
 −1 2 −1 
 
h 1 −1 2 −1 
A =
2 ⋅⋅ ⋅⋅
⋅ ⋅

h  
 −1 2 −1
 
 − 21 2 
• Note that is (N+2)x(N+2) & nonsymmetric, &
the system A h
involves unknowns & at the
boundaries. uh uh
0 N +1

U-Boulder 201 of 312

 We must consider compatibility

• The problem − u ″ ( x ) = f ( x ), for 0 < x < 1 & with

u ′ ( 0) = u ′ ( 1) = 0 is not well-posed!
• If u(x) is a solution, so is u(x) + constant.
• We cannot be certain a solution exists. If one does,
it must satisfy
1 1 1
− ⌠ u″( x) dx = ⌠ f ( x) dx − [ u′( 1) − u′( 0) ] = ⌠ f ( x) dx
⌡ ⌡ ⌡
0 0 0
1
0 = ⌠ f ( x) dx
⌡
• This integral compatibility condition is necessary!
0
If f(x) doesn’t satisfy it, there is no solution!

U-Boulder 202 of 312

 The well-posed system
• The compatibility condition is necessary for a
solution to exist. In general, it is also sufficient:
∂2 is a well-behaved operator on the space of
−
∂x 2
functions u(x) that have zero mean.
• Thus we may conclude that if f(x) satisfies the
compatibility condition, this problem is well-posed:

− u ″ ( x) = f ( x) 0<x <1
u ′ ( 0) = u ′ ( 1) = 0
1
⌠ u( x ) dx = 0
• The last says:
⌡ of all possible solutions u(x) + constant,
we choose0 the one with zero mean.

U-Boulder 203 of 312

The discrete problem is not well posed
• Since all row sums of A h are zero, then 1h ∈ N S ( A h )
• Putting A h into row-echelon form shows that

h hence h h
dim(N S( A ) ) = 1 N S( A ) = spa n ( 1 )
• By the Fundamental Theorem of Linear Algebra,
has a solution if & only if Ah
f h ⊥ N S ( ( A h) T )
• It is easy to show that
h T T
• Thus, N S ( ( A ) ) = c (
has a solution if & only if 1/ 2, 1, 1, . . . , 1, 1/ 2)
h h h
A
• That is, u = f
h
f ⊥ c ( 1/2, 1, 1, . . . , 1, 1/2) T
1 h N 1 h
f 0 + ∑ f j + f N +1 = 0
h
2 j =1
2
U-Boulder 204 of 312
 We have two issues to consider

• Solvability. A solution exists iff f h ⊥ N S ( ( A h) T )

• Uniqueness. If h solves h h h, then so does u h + c1h

u A u =f
• Note that if h , then
h T h h T
A = ( A )
& solvability & uniqueness can be handled together) )
NS ( A ) = NS ( ( A

• This is easily done. Multiply 1st & last equations by 1/2,

giving
 1 −1 
 −1 2 −1 
 
h 1 −1 2 −1 
A =  ⋅⋅ ⋅⋅ 
h2 ⋅ ⋅ 
 −1 2 −1
 
 − 1 1 
U-Boulder 205 of 312
 The new system is symmetric
h h
• We have the symmetric system A uh =f :
 f h/2 
 
u0h  0 
 1 − 1    h 
 −1 2 −1  
u1h  f1 
    
1 − 1 2 − 1  
u2h  f2 h
 ⋅⋅ ⋅⋅    =  
2 ⋅ ⋅
h   M   M 
 − 1 2 − 1   uN h   h 
    fN 
 − 1 1   uh   
 N +1 f h
/2 
• Solvability is guaranteed by ensuring that
 N + 1 is
orthogonal to the constant vector : h
f
h
1
h N +1 h
f , 1h = ∑ fj =0
j =0

U-Boulder 206 of 312

 The well-posed discrete system
• The (N+3)x(N+2) system is:
− uj − 1 + 2 uj − uj + 1
= fj 0 ≤ j ≤ N +1
2
h
u0 − u1 f 0
=
2 2
h
− uN + uN + 1 f N + 1
=
h 2 2
N +1
∑ uih = 0 (choose the zero mean solution)
i =0
or, more simply
h h
A uh =f
u h, 1h =0
U-Boulder 207 of 312
 Multigrid for the Neumann problem
• We must have the interval endpoints on all grids
x 0h x 1h h
xN /2
h
xN

x 02h 2h
xN 2h
xN
/4 /2
• Relaxation is performed at all points, including endpoints:

h
v jh − 1 + + v jh+ 1 + h 2 f j
2 h 2 h
1 ← v N + h on
v 0 ←add
h v 1 + ah global
h h h
• We f0 Gram-Schmidt
v jh ←
2step after
v N +relaxation f N +1
each level to enforce the zero-mean condition

v h, 1h
vh ← vh − 1h
1h, 1h
U-Boulder 208 of 312
 Interpolation must include the
endpoints
• We use linear interpolation:

 1 
 1/2 1/2 
 
 1 
 1/2 1/2 
 
I 2hh =  1 
 1/2 1/2 
 
 1 
 1/2 1/2 
 
 1 
U-Boulder 209 of 312
 Restriction also treats the
endpoints
1 h T
• For restriction, we use I 2h = ( I 2h) , yielding the
h 2
values
2h 1 h 1 h
f 0 = f0 + f1
2 4
2h 1 h 1 h 1 h
f j = f 2j − 1 + f 2j + f 2j + 1
4 2 4

2h 1 h 1 h
f N +1 = f N + f N +1
4 2

U-Boulder 210 of 312

 The coarse-grid operator

• We compute the coarse-grid operator using the

Galerkin condition 2h 2h h h
A = I h A I 2h
j2h = 0 1
ε20h 1 0  1 −1 
 −1 2 −1 
1  
I 2hhε20h 1 0 2h 1  −1 2 −1 
2 A =
2 ⋅⋅ ⋅⋅
⋅ ⋅

4h  
2h h 2h 1 1 
A h I 2hε0 0 − −1 2 −1
2  
2h 2h 2  − 1 1 
2h h h 2h 1 1
I h A I 2hε0 −
4h 2 4h 2
U-Boulder 211 of 312
 Coarse-grid solvability
h h
• Assuming f satisfies f , 1h = 0 , the solvability
condition, we can show that theoretically the coarse-
grid problem A 2h u 2h = I 2h ( f h − A hv h ) is also solvable.
h

• To be certain numerical round-off does not perturb

solvability, we incorporate a Gram-Schmidt-like step
each time a new right-hand side is generated for
2h
the coarse grid: f
2h
f , 12h
2h 2h
f ← f − 12h
12h , 12h

U-Boulder 212 of 312

 Neumann problem: An example
• Consider the problem − u″ ( x ) = 2 x − 1 ,
0 <x <1 u′ ( 0 ) = u′ ( 1 ) = 0
which has x2 x3 as a solution for any c
u( xthe
(c = -1/12 gives ) =zero− mean
+ csolution).
2 3

grid size average number

|| r h || || e h ||
N conv. factor of cycles
31 6.30E-11 0.079 9.70E-05 9
63 1.90E-11 0.089 2.40E-05 10
V(2,1)
127 2.60E-11 0.093 5.90E-06 10 cycles
255 3.70E-11 0.096 1.50E-06 10
511 5.70E-11 0.100 3.70E-07 10
1027 8.60E-11 0.104 9.20E-08 10
2047 2.10E-11 0.112 2.30E-08 10
4095 5.20E-11 0.122 5.70E-09 10
U-Boulder 213 of 312
 7. Selected applications
7b. Anisotropic problems
1
• All problems considered thus far have had −
2
as
h
the off-diagonal entries.

• We consider two situations when the matrix has

two different constants on the off-diagonals.
These situations arise when
– the (2-d) differential equation has constant but
different coefficients for the derivatives in the
coordinate directions
– the discretization has constant but different mesh
spacing in the different coordinate directions

U-Boulder 214 of 312

 We consider two types of anisotropy
• Different coefficients on the derivatives

− ux x − α uyy = f
discretized on a uniform grid with spacing h .

• Different mesh spacings:

1
hx = h =
N
hy hx
hy =
α
U-Boulder hx 215 of 312
 Both problems lead to the same
stencil
− u j − 1, k +2 u j , k − u j + 1, k − uj , k − 1 + 2 uj , k − uj , k + 1
+α
2
h h2

−α
h 1  
A = − 1 2 + 2α − 1 
2
h  −α 

− u j − 1, k +2 u j , k − u j + 1, k − uj , k − 1 + 2 uj , k − uj , k + 1
+
2 2
h  h 
 
 α 
U-Boulder 216 of 312
 Why standard multigrid fails
−α
h 1  
A =  − 1 2 + 2α − 1 
h2 −α 
• Note that Ah has weak connections in the y-direction.
MG convergence factors degrade as α gets small, with
poor performance already at α = 0.1 .
• Consider the limiting case α ⇒ 0: 0
h 1  
A ⇒  − 1 2 +22α − 1 
2
• Collection of disconnected 1-D problems!h  0 
• Point relaxation will smooth oscillatory errors in the
x-direction (strong connections), but with no
connections in the y-direction, the errors in that
direction will generally be random; point relaxation
provides no smoothing in the y-direction.

U-Boulder 217 of 312

 We analyze weighted Jacobi
• The eigenvalues of the weighted Jacobi iteration
matrix for this problem are
2ω   iπ   lπ  
λi, l = 1 − 
2
sin   + α sin 
2

1+α   2N   2N 

1 1

1 0.9 0.9

0.8 0.8 0.8

0.7 0.7
0.6

0.6 0.6
0.4
0.5 0.5
0.2
0.4 0.4

0 15
0 0.3 0.3
10 0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
5
10
15
5

l i l
i
0 l
k

U-Boulder 218 of 312

 Two strategies for anisotropy

• Semicoarsening: Because we expect MG-like

convergence for the 1-D problems along lines of
constant y, we could coarsen the grid in the
x-direction, but not in the y-direction.

• Line relaxation: Because the the equations are

strongly coupled in the x-direction, we could solve
simultaneously for entire lines of unknowns in the
x-direction (along lines of constant y).

U-Boulder 219 of 312

 Semicoarsening with point relaxation
−α
• Point relaxation on Ah 1 
smooths in the
=  − 1 2 + 2α − 1 
h2 −α 
x-direction. Coarsen by removing every other y-line.

h
Ω Ω 2h

• We do not coarsen along the remaining y-lines.

• Semicoarsening is not as “fast” as full coarsening. The

number of points on is about half the number of
points on , instead of the usual one-fourth.
Ω 2h
Ωh
U-Boulder 220 of 312
 Interpolation with semicoarsening

• We interpolate in the 1-D way along each line of

constant y.

• The coarse-grid correction equations are

v 2h j , k = v 2h j , k + v j2,hk

v j2,hk + v j2h+ 1, k
v 2h j + 1, k = v 2h j + 1, k +
2

U-Boulder 221 of 312

 Line relaxation with full coarsening

• The other approach to this problem is to do full

coarsening, but to relax entire lines (constant y)
of variables simultaneously.
• Write Ah in block form as

 D − cI 
 − cI D − cI 
 
Ah =  − cI D − cI 
 ⋅⋅ ⋅⋅ 
 ⋅ ⋅ − cI 
where  − cI D 
&
 2 + 2α − 1 
α 1  − 1 2 + 2α − 1 
c = D =  ⋅⋅ 
h2 h2 ⋅ 
 − 1 2 + 2α 
U-Boulder 222 of 312
 Line relaxation
• One sweep of line relaxation consists of solving a
tridiagonal system for each line of constant y.

• The kth such system has the form , where h

is the kth subvector of D v k = gk
with entries
h h v&k
h
the kth right-hand side vsubvector is ( v kh ) j = v jh, k

h α
( gk ) j = f j , k +
h ( v jh, k − 1 + v jh, k + 1 )
2
h
• Because D is tridiagonal, the kth system can be solved
very efficiently.

U-Boulder 223 of 312

 Why line relaxation works
• The eigenvalues of the weighted block Jacobi
iteration matrix are
2ω  2  i π  + α s in2  l π  
λi , l = 1 −  s i n  2N   2N  
 iπ      
2 s in2   + α
 2N 

0.8 0.8
0.8

0.6 0.6 0.6

0.4
0.4
0.2 0.4

0
0.2
0.2
-0.2

0
15 0
15
10
10 -0.2
5 -0.2
5
0 2 4 6 8 10 12 14 16
0 0

l
0 2 4 6 8 10 12 14 16

i i l

U-Boulder 224 of 312

 Semicoarsening & line relaxation
• We might not know the direction of weak coupling
or it might vary.

• Suppose we want a method that can handle either

−α or −1
1   h 1  
A1 h = − 1 2 + 2α − 1  A2 = − α 2 + 2α − α 
2 2
h 
h  −α  −1 

• We could use semicoarsening in the x-direction to

handle & line relaxation in the y-direction to
take care hof .
A1
A 2h

U-Boulder 225 of 312

 Semicoarsening & line relaxation

• The original grid. • Original grid • Coarsening is

viewed as a done by deleting
stack of every other
“pencils.” Line pencil.
relaxation is
used to solve
problem along
each “pencil”.
U-Boulder 226 of 312
 An anisotropic example
• Consider − ux x − α uyy = f with u = 0 on the boundaries
of the unit square, & stencil given by

−α
1  
Ah = − 1 2 + 2α − 1 
2
h  −α 
• Suppose so the exact
solution isfgiven
( x, y) by
= 2 ( y − y 2) + 2 α ( x − x 2)
u( x, y) = ( y − y 2) ( x − x 2)
• Observe that if α is small, the x-direction dominates,
while if α is large, the y-direction dominates

U-Boulder 227 of 312

 What is smooth error?
• Consider α = 0.001 & suppose point Gauss-Seidel is
applied to a random initial guess. The error after
50 sweeps appears as:
0.12

0.1

0.08

0.06

0.1 0.04

0.02
0.08
00
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.06
Error along line of constant x
0.04

0.02
0.12

0 0.1
1 0.08
1
0.8 0.06
0.5 0.6 0.04
0.4

y 0 0
0.2

x
0.02

00 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Error along line of constant y

U-Boulder 228 of 312

 We experiment with 3 methods
• Standard V(2,1)-cycling, with point Gauss-Seidel
relaxation, full coarsening, & linear interpolation

• Semicoarsening in the x-direction. Coarse & fine

grids have the same number of points in the y-
direction. 1-D full weighting & linear interpolation
are used in the x-direction, with no y-coupling in
the intergrid transfers

• Semicoarsening in the x-direction combined with

line relaxation in the y-direction. 1-D full weighting
& interpolation.

U-Boulder 229 of 312

 With semicoarsening, the
operator must change
• To account for unequal mesh spacing, the residual
& relaxation operators must use a modified stencil
 α 
−
 hy2 
 
 
 1  2 2α  1 
A = −  2 + −
2 2 2
 h x  hx hy  hx 
 
 1 
−
 2 
• Note that as grids become coarser,
h y  grows
while remains constant.
hx
hy
U-Boulder 230 of 312
 How do the 3 methods work for
various values of α ?
N = 16

• Asymptotic convergence factors:

α
scheme 1000 100 10 1 0.1 0.01 0.001 1E-04
V(2,1)-cycles 0.95 0.94 0.58 0.13 0.58 0.90 0.95 0.95
semiC in x 0.94 0.99 0.98 0.93 0.71 0.28 0.07 0.07
semiC / line relax 0.04 0.08 0.08 0.08 0.07 0.07 0.08 0.08

y-direction strong x-direction strong

• Note: semicoarsening in x works well for α < .001
but degrades noticeably even at α = .1.

U-Boulder 231 of 312

 A semicoarsening subtlety
• Suppose α is small, so that semicoarsening in x is
used. As we progress to coarser grids, hx− 2 gets
small but − 2 remains constant.
hy

• If, on some coarse grid, − 2 becomes comparable

to hx effectively becomes
, then the problem
α hy− 2 in the y-direction. Continued
recoupled
semicoarsening can produce artificial anisotropy,
strong in the y-direction.

• When this occurs, it is best to stop semicoarsening &

use full coarsening on any further coarse grids.

U-Boulder 232 of 312

 7. Selected applications
7c. Variable meshes
• Non-uniform grids are commonly used for domain or
data irregularities or emerging solution features
• Consider how we might approach the 1-D problem

− u″( x) = f ( x) 0<x <1

posed onu(
the = u( 1 ) = grid:
0) following 0

x =0

x0 xj − 1 xj xj + 1 xN

U-Boulder 233 of 312

 We need some notation for the
mesh spacing
• Let N be a positive integer. We define the
spacing interval between x & x :
j j +1

h j + 1/2 ≡ x j + 1 − x j j = 0, 1, ... , N − 1

h j + 1/2

x0 xj − 1 xj xj + 1 xN

U-Boulder 234 of 312

 Second divided differences

’’ = (uj+1/2
uj-1/2 ’ - uj-1/2
’ )/hj

uj'-1/2 = (uj - uj-1)/hj-1/2 Uj'+1/2 = (uj+1 - uj)/hj+1/2

uj-1 ↓ uj ↓
uj+1
| |

hj-1/2 hj+1/2

hj = (hj-1/2 + hj+1/2)/2

U-Boulder 235 of 312

 We define the discrete
differential operator
• Using second-order finite differences (& wading
through a mess of algebra!), we obtain the
discrete representation

− αhj u jh− 1 + ( αhj + βhj ) u jh − βhj u jh+ 1 = f jh 1 ≤ j ≤ N −1

u0h = uN
h = 0
where

h 2 h 2
=hαj βj = h
j − 1/ 2 ( hj − 1/ 2 + hj + 1/2 ) j + 1/2 ( hj − 1/2 + hj + 1/2 )
• Multiply by (hj-1/2 + hj+1/2)/2 yields an SPD matrix.

U-Boulder 236 of 312

 We modify standard multigrid to
accommodate variable spacing
• We choose every second fine-grid point as a
coarse-grid point
x 0h x 2h j h
xN

2h
xN
x 02h x 2h
j /2
• In [x0 , x2 ], linear means
h h

v (x) = v0 + (v1 - v0 ) (x - x0 )/(x2 - x0 )

2h 2h 2h h h h
• Now just plug in x = x1 to get…
h

U-Boulder 237 of 312

 Interpolation formula

• If v h = I 2hh v 2h , then for 1 ≤ j ≤ N/2 − 1 we have

h2 j + 3/2 v j2h + h2 j + 1/2 v j2h+ 1

v 2h j = v j2h v 2h j + 1 =
h 2 j + 1/2 + h 2 j + 3/2

U-Boulder 238 of 312

 We use the variational properties
to derive restriction & A2h
2h 2h h h 2h 1 h T
A = I h A I 2h Ih = ( I 2h )
2
• This produces a stencil on Ω 2hthat is similar, but
not identical, to the fine-grid stencil. If the
resulting system is scaled by ( h + h ) , then
the Galerkin product is the same j −as + 1/2
1/2 thej fine-grid
stencil.
• For 2-d problems this approach can be generalized
readily to logically rectangular grids. However, for
irregular grids that are not logically rectangular,
AMG is a better choice.

U-Boulder 239 of 312

 7. Selected applications
7d. Variable coefficients
• A common difficulty is variable coefficients,
exemplified in 1-D by
− ( a( x) u′( x) ) ′ = f ( x) 0<x <1

u( 0 ) = u( 1 ) = 0
where a(x) is a positive function on [0,1]
• We seek to develop a conservative, or self-adjoint,
method for discretizing this problem.
• Assume we have available to us the values of a(x)
at the midpoints of the grid
a j + 1/2 ≡ a( x j + 1/2 )

x 0h x jh− 1 x jh h
xN
x jh + 1
U-Boulder 240 of 312
We discretize using central differences

• We can use second-order differences to

approximate the derivatives. To use a grid spacing
of h, we evaluate a(x)u’(x) at points midway
between the gridpoints:

( au′) |x j + 1/2 − ( a u′) |x j − 1/2

( a( x) u′( x) ) ′ ≈
= + O( h 2 )
xj
h
Points used to evaluate (au’ )’ at xj

x 0h x jh − 1 h
x jh x jh + 1 xN
U-Boulder 241 of 312
We discretize using central differences
(cont’d)

• To evaluate ( au′) |x j + 1/2 , we must sample a(x) at the

point x & use second order differences:
j + 1/2
uj + 1 − uj uj − uj − 1
( a u′ ) |x j + 1/2 ≈ a j + 1/2 ( a u′ ) |x j − 1/2 ≈ a j − 1/2
h h
where
a j + 1/2 ≡ a( x j + 1/2 )

Points used to
evaluate u’ at Points used to evaluate (au’ )’ at xj
x j + 1/2

x 0h x jh − 1 x jh h
xN
x jh + 1
U-Boulder 242 of 312
 The basic stencil

• We combine the differences for u’ & for (au’ )’ to

obtain the operator
uj + 1 − uj uj − uj − 1
aj + 1/2
h − aj − 1/2 h
− ( a( x j ) u′ ( x j ) ) ′ ( x j ) ≈ −
h

& the problem becomes, for

1 ≤ j ≤ N −1
1
2( − aj − 1/2 uj − 1 + ( a j − 1/2 + a j + 1/2 ) uj − a j + 1/2 uj + 1 ) = f j
h

u0 = uN = 0

U-Boulder 243 of 312

 Coarsening the variable
coefficient problem
• A reasonable approach is to use a standard multigrid
algorithm with linear interpolation, full weighting, &
the stencil
1
A 2h = 2
( − a 2h
j − 1/2 a j2h− 1/2 + a j2h+ 1/2 − a j2h+ 1/2 )
( 2h)
where a 2h j + 1/2 + a 2h j + 3/2
a j2h+ 1/2 =
2

• The same stencil is obtained by the Galerkin relation

U-Boulder 244 of 312

Variable mesh vs. variable coefficients
after scaling by ηj = (hj-1/2 + hj+1/2)/2 & h

• Variable mesh
1 h 1 1 h 1 h h
− uj −1 + ( + )uj + uj +1 = ηj f j
hj −1/ 2 h j −1/ 2 hj +1/ 2 h j +1/ 2
• Variable coefficients

1 h
( − aj − 1/2 uj − 1 + ( a j − 1/2 + aj + 1/2 ) uj − a j + 1/2 uj + 1 ) = h f j
h
• Correspondence

1 aj −1/ 2 1 aj +1/ 2
⇔ ⇔
hj −1/ 2 h h j +1/ 2 h

U-Boulder 245 of 312

 Standard multigrid degrades if
a(x) is highly variable

• So MG for variable coefficients is equivalent to MG (with

simple averaging) for Poisson’s equation on a variable mesh.

-(au’ )’ = f

-u’’ = f
• But simple averaging won’t accurately represent smooth
components if is close to but far from .
h
x 2j +1 x 2h j x 2h j + 2

x 2h j h
x 2j +1 x 2h j + 2

x j2h x j2h+ 1
U-Boulder 246 of 312
 One remedy is to apply
operator interpolation
• Assume that relaxation does not change smooth
error, so the residual is approximately zero.
Applying at x h yields
2j + 1
− a 2h j + 1/2 e2h j + ( a 2h j + 1/2 + a 2h j + 3/2 ) e2h j + 1 − a 2h j + 3/2 e2h j + 2
2
=0
h
• Solving for :
h
e2j +1

a 2h j + 1/2 ej2h + a 2h j + 3/2 ej2h+ 1

+1 =
h
e2j
+ 1/2 + a 2j + 3/2
h
a 2j h

U-Boulder 247 of 312

Thus, operator induced interpolation is

v 2h j = v j2h

a 2h j + 1/2 v j2h + a 2h j + 3/2 v j2h+ 1

+1 =
h
v 2j h
a 2j + 1/2 + a h
2j + 3/2

• As usual, the restriction & coarse-grid operators

are defined by the Galerkin relations

2h 2h h h 2h h T
A = I h A I 2h I h = c( I 2h )
U-Boulder 248 of 312
 A variable coefficient example
• We use V(2,1) cycle, full weighting, linear interpolation.
• We use a(x) = 1 + ρ sin(kπx) & a(x) = 1 + ρ rand(kπx)

N = 128 a(x) = 1 + ρ sin(kπx)

1 + ρ rand(kπx)

ρ k=3 k=25 k=50 k=100 k=200 k=400

0 0.085 0.085 0.085 0.085 0.085 0.085 0.085
0.25 0.084 0.098 0.098 0.094 0.093 0.083 0.083
0.5 0.093 0.185 0.194 0.196 0.195 0.187 0.173
0.75 0.119 0.374 0.387 0.391 0.39 0.388 0.394
0.85 0.142 0.497 0.511 0.514 0.514 0.526 0.472
0.95 0.191 0.681 0.69 0.694 0.699 0.745 0.672

U-Boulder 249 of 312

 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems •√ Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
√ – Neumann boundaries
– Analysis
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
√ – Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Matrix coarsening
– Complexity
√ • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology
√

U-Boulder 250 of 312

 8. Algebraic multigrid (AMG)
unstructured grids, variable coefficients,…

Automatically defines coarse “grid”

AMG has two distinct phases:

— setup phase: define MG components
— solution phase: perform MG cycles

AMG approach is opposite of geometric MG

— fix relaxation (point Gauss-Seidel)
— choose coarse “grids” & prolongation, P, so that
error not reduced by relaxation is in range(P)

(in contrast, geometric MG fixes coarse grids,

then defines suitable operators & smoothers)

— use Galerkin principle so that coarse-grid

correction eliminates error in range(P)

U-Boulder 251 of 312

 AMG has two phases
• Setup Phase
– Select coarse “grids,” Ω m + 1, m = 1, 2, . . .
m
+ 1 , m = 1, 2, . . .
– Define interpolation, Im

– Define restriction & coarse-grid operators

m+1 m T m+1 m +1 m m
Im = ( Im +1 ) A = I m A Im + 1

Solve Phase
Standard MG processes: V-cycle, W-cycle, FMG, FAS, …

Only the selection of coarse grids does not parallelize

well using existing techniques!
U-Boulder 252 of 312
 AMG fundamental concept:
Smooth error = “small” residuals
• Consider the iterative method error recurrence
ek + 1 = ( I − Q − 1 A ) ek
• Error that is slow to converge satisfies
−1 −1
( I− Q A ) e ≈ e ⇒ Q Ae≈ 0
⇒ r ≈ 0
• More precisely, it can be shown that smooth error
satisfies
r −1 « e A (1)
D

U-Boulder 253 of 312

 AMG uses strong connection to
determine MG components
• It is easy to show from (1) that smooth error
satisfies
A e , e « De , e (2)

• Define i is strongly connected to j by

− aij ≥ θ max { − aik } , 0 <θ ≤1
k ≠i
• For M-matrices, we have from (2)
1  − a i j   ei − e j  2
2 ∑  2 a ii   ei  « 1
i ≠j

implying that smooth error varies slowly in the

direction of strong connections
U-Boulder 254 of 312
 Some useful set definitions
• The set of strong connections of a variable u i ,
that is, the variables upon whose values the value
of ui depends, is defined as
 
S i =  j : − a i j > θ max − aij 
 j ≠i 
• The set of points strongly connected to a variable ui
is denoted S iT = { j : j ∈ S i } .
• The set of coarse-grid variables is denoted C.
• The set of fine-grid variables is denoted F.
• The set of interpolatory coarse-grid variables
used to interpolate the value of the fine-grid
variable ui is denoted C i .
U-Boulder 255 of 312
 Choosing the coarse grid
• Two Criteria

– (C1) For each i ∈ F, each point j ∈ S should either be in C

i
or should be strongly connected to at least one point in C .
i

– (C2) C should be a maximal subset with the property that no

two C-points are strongly connected to each other.

• Satisfying (C1) & (C2) is sometimes impossible. We use

(C2) as a guide while enforcing (C1).

U-Boulder 256 of 312

 Selecting the coarse-grid points

C-point selected
(point with
largest “value”)
Neighbors of
C-point
become F-
points
Next C-point
selected (after
updating “values”)

F-points
selected, etc.
U-Boulder 257 of 312
 Examples: Laplacian operator
5-pt FD, 9-pt FE (quads), & 9-pt FE (stretched quads)

9-pt FE
(stretched quads)

−1 −1 − 1 −1
   
 −1 4 −1  −1 8 −1 
 −1   −1 − 1 −1 
−1 −4 −1
5-pt FD   9-pt FE (quads)
 2 8 2 
 −1 −4 −1 
U-Boulder 258 of 312
 Prolongation is based on smooth
error, strong connections (from
M-matrices)
Smooth error is given by:

∑
C
F
F
ri = a i i ei + a i j ej ≈ 0
j ∈N i

i Prolongation :
C
C
ei , i∈C

( P e) = 
i  ∑ ωik ek , i∈F
F

k∈Ci

U-Boulder 259 of 312

 Prolongation is based on smooth
error, strong connections (from
M-matrices)
Sets:
Ci Strongly connected C-pts.
F
C
Dis Strongly connected F -pts.
F
Diw Weakly connected points.

i The definition of smooth error,

C
C a i i ei ≈ − ∑ a i j ej
j ≠i
F gives

a i i ei ≈ − ∑ aij ej − ∑s aij ej − ∑waij ej

j ∈Ci j ∈Di j ∈Di
Strong C Strong F Weak pts.
U-Boulder 260 of 312
 Finally, the prolongation
weights are defined
• In the smooth-error relation, use e j = ei for weak
connections. For the strong F-points use :
   
e j =  ∑ a j k ek  ⁄  ∑ a j k 
 k ∈Ci   k ∈Ci 

yielding the prolongation weights:

a ika k j
aij + ∑ s ∑ akm
j ∈Di m ∈C
i
wi j = −
aii + ∑ w a in
n ∈Di

U-Boulder 261 of 312

 AMG setup costs:
A bad rap

• Many geometric MG methods need to compute

prolongation & coarse-grid operators

• The only additional expense in the AMG setup phase is

the coarse-grid selection algorithm

• AMG setup phase is only 10-25% more

expensive than in geometric MG & may be
considerably less than that!

U-Boulder 262 of 312

 AMG performance:
Sometimes a success story
• AMG performs extremely well on the model problem
(Poisson’s equation, regular grid)- optimal convergence
factor (e.g., 0.14) & scalability w.r.t. problem size.

• AMG appears to be both scalable & efficient on

diffusion problems on unstructured grids (e.g., 0.1-0.3).

• AMG handles anisotropic diffusion coefficients on

irregular grids reasonably well.

• AMG handles anisotropic operators on structured &

unstructured grids relatively well (e.g., 0.35).
U-Boulder 263 of 312
 So, what could go wrong?
Strong F-F connections: weights depend on each other

• For point i the value ej is interpolated from k 1 , k 2 , &

is needed to make the interpolation weights for
approximating ei .
• For point j the value ei is interpolated from k 1, k 2 , &
is needed to make the interpolation weights for
approximating ej .
• It’s an implicit system!
k 1 ∈ Ci
k 1 ∈ Cj

k4 ∈ Cj
k 3 ∈ Ci i j
k 2 ∈ Ci
U-Boulder k 2 ∈ Cj 264 of 312
 Is there a fix?
A Gauss-Seidel-like iterative approach to weight
definition is implemented. Usually, two passes
suffice. But does it work? Yes, frequently:
convergence factors
2D Laplacian, N = 10,000,
stretched quadrilaterals θ standard iterative
0.25 0.47 0.14
∆ x = 10 ∆ y 0.5 0.24 0.14

0.25 0.93 0.93

∆ x = 100 ∆ y 0.5 0.55 0.28

V(1,1) cycles/Gauss-Seidel (used throughout chapter 8).

U-Boulder 265 of 312
 How’s it perform (vol I)?
regular grids, plain, old, vanilla problems,
unit square, N = 64, Dirichlet boundaries
• The Laplace operator:
Convergence Time Setup
Stencil per cycle Complexity per Cycle Times
5-pt 0.054 2.21 0.29 1.63
5-pt skew 0.067 2.12 0.27 1.52
9-pt (-1,8) 0.078 1.30 0.26 1.83
9-pt (-1,-4,20) 0.109 1.30 0.26 1.83

• Anisotropic 5-Point Laplacian: − ε Ux x − Uyy

Epsilon 0.001 0.01 0.1 0.5 1 2 10 100 1000
Convergence/cycle 0.084 0.093 0.058 0.069 0.054 0.079 0.087 0.093 0.083

U-Boulder 266 of 312

 How’s it perform (vol II)?
structured meshes, rectangular domains

• 5-point Laplacian on regular rectangular grids

Convergence factor (y-axis) plotted against number of nodes (x-axis)
0 .1 6
0 .1 4 6
0 .1 4 0 .1 3 5
0 .1 4 2
0 .1 3 3
0 .1 2
0 .1
0 .0 8 0 .0 8 2

0 .0 6 0 .0 5 5
0 .0 4
0 .0 2
0
0 500000 1000000

U-Boulder 267 of 312

 How’s it perform (vol III)?
unstructured meshes, rectangular domains

• Laplacian on random unstructured grids (regular

triangulations, 15-20% nodes randomly collapsed into neighboring nodes)

Convergence factor (y-axis) plotted against number of nodes (x-axis)

0 .3

0 .2 5 0 .2 5 3
0 .2 2 3
0 .2
0 .1 7 5
0 .1 5
0 .1 1 1
0 .1 0 .0 9 7

0 .0 5

0
0 20000 40000 60000

U-Boulder 268 of 312

 How’s it perform (vol IV)?
isotropic diffusion, structured/unstructured grids
∇ • ( d ( x , y ) ∇ u ) on structured, unstructured grids
0 . 4
0 . 3 5
6 a
0 . 3 6 b
0 . 2 5 7 a
0 . 2 7 b
0 . 1 5 8 a
8 b
0 . 1
9 a
0 . 0 5
9 b
0

N=16642 N=66049 N=13755 N=54518

Structured Structured Unstruct. Unstruct.
Problems used: “a” means parameter c = 10, “b” means c = 1000

1. 0 0. 125 ≤ max { x − 0. 5 , y − 0. 5 } ≤ 0. 25

6: d ( x , y ) = 1. 0 + c x − y 8: d( x, y) = 
c otherwise
1. 0 x ≤ 0. 5 1. 0 0. 125 ≤ ( x − 0. 5) 2 + ( y − 0. 5) 2 ≤ 0. 25
7: d ( x , y ) =  9: d( x, y) = 
 c x > 0. 5 c otherwise
U-Boulder 269 of 312
 How’s it perform (vol V)?
Laplacian operator, unstructured grids

Convergence factor

0.3

0.2754

0.25

0.2198 0.2237
0.2
0.1888
0.1715
0.15

0.1 0.1002

0.05

0
0 50000 100000 150000 200000 250000 300000 350000

Gridpoints

U-Boulder 270 of 312

 AMG for systems
• How can we do AMG on systems?
 A 11 A 12   u   f 
A A  v  = g
 21 22     
• Naïve approach: “Block” AMG (block Gauss-Seidel,
using scalar AMG to “solve” at each cycle)
−1
u ← ( A 11 ) ( f − A 12 v )

v ← ( A 22 ) − 1 ( g − A 21 u )

Great Idea! Except that it often doesn’t work!

Block AMG doesn’t account for strong inter-variable coupling.

U-Boulder 271 of 312

 AMG for systems: A solution
• To solve the system problem, allow interaction between
the unknowns at all levels:
k
k
 11 12 
A A k
 ( Ik + 1) 0 
k u
A = Ikk + 1 = 
k 
& k 
k ( Ik +1
 A 21 A 22   0 + 1) v 

• This is called the “unknown” approach.

• 2-D biharmonic, Dirichlet boundaries, unit square,
uniform quadrilateral mesh:

Mesh spacing 0.125 0.0625 0.03135 0.015625

Convergence factor 0.22 0.35 0.42 0.44

U-Boulder 272 of 312

 αAMG)
Adaptive AMG (α
to broaden applicability

adaptive interpolation
to determine sense of smoothness
+
adaptive C-point choice
to determine strength of connection

U-Boulder 273 of 312

 Adaptive interpolation

Standard AMG collapses stencils by assuming smooth

error is locally constant (Poisson “sense of smoothness”):

   
e j =  ∑ a j k ek  ⁄  ∑ a j k 
*** smooth ≈ 1 ***  k ∈Ci   k ∈Ci 

ej = ei
a i i ei ≈ − ∑ aij ej − ∑s aij ej − ∑waij ej
j ∈Ci j ∈Di j ∈Di
Strong C Strong F Weak pts.

U-Boulder 274 of 312

 Pre-relaxation
• What if we found a smooth error x that’s far from 1 ?
• If we found that xj = 1.6xi for j∈Diw, say, then we could
just make the replacement ej → 1.6ei .
• If, say, xj = 0.2xk for j∈Dis & all k∈Ci, then we could set

ej → (Σ ajk 0.2ek )/( Σ ajk ).

j∈Ci j∈Ci

• How do we obtain a representative smooth error x ?

Relax on Ax = 0 !!!
U-Boulder 275 of 312
 Adaptive C-point choice
• AMG uses the ansatz that residuals are 0 at F-points.
• What if we had rf = 0 exactly?
 A ff A fc e f   0  e f   −A−1
ff A fc

   =   ⇒ A ff e f + A fcec = 0 ⇒  = ec
 Acf Acc  ec   rc   ec   I 

• Choosing −A−1  (ideal interpolation) makes MG exact:

A
P =  ff fc 
 I 

MGe = [CG]Pec = [I − P[P AP] P A]Pec T −1 T

−1
= Pec − P[P AP] [P AP]ec = 0.
T T

U-Boulder 276 of 312

 Compatible relaxation
• How do we make the residuals zero at F-points?

Relax on Affxf = -Afcxc !!!

• This is the same as relaxing on the F-points while
maintaining 0 at the C-points.
• If Aff is well-conditioned, then this converges quickly.
• Quick “CR” convergence means that the F-point residuals
will be small after relaxation & that there is thus a P
(the ideal one) that gives good MG convergence.
• Is there a good local P ? Maybe, since CR is local & fast.

U-Boulder 277 of 312

 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems •√ Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
√ – Neumann boundaries
– Analysis
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
√ – Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Complexity √ – Matrix coarsening
√ • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology
√

U-Boulder 278 of 312

 9. Multilevel adaptive methods
fast adaptive composite grid method (FAC)

two-spike problem

U-Boulder 279 of 312

 Model 1-D problem

- u”(x) = f(x), 0<x<1

u(0) = u(1) = 0

0 1

h-2 (- uhi-1 + 2uhi - uhi-1 ) = fhi ,1 ≤ i ≤ 7,

uh0 = uh8 = 0, h = 1/8

U-Boulder 280 of 312

 Local refinement
suppose f(x) has a spike at x = 3/4
but is smooth elsewhere

y= f(x)

O
fine grid (
= interface)

coarse grid

U-Boulder 281 of 312

 Strategy

• Recognize that there’s little value of having the fine grid in

the smooth region, [0, 1/2].

• Start with standard MG

– eliminate relaxation
– eliminate intergrid transfers & residual calculations

• Interpret this process via the composite grid

(= 2h-points in [0, 1/2] + h-points in [1/2, 1]).

• We’ll try absurdly hard to eliminate work @ x = 1/2,

but we have in mind multi-dimensions & smaller patches.

U-Boulder 282 of 312

 Eliminate relaxation
notation
vh = (vih )

• Initialize vh = 0 & f12h  (f1h + 2f2h + f3h )/4 .

• Relax on vh on the local fine grid.
• Compute rh = fh - Ah vh & transfer to 2h:
f22h  (r3h + 2r4h + r5h )/4 & f32h  (r5h + 2r6h + r7h )/4 .
• Compute an approximation, v2h, to the solution of the 2h
residual equation, A2h u2h = f2h .
• Update the residual at x12h for later cycles:
f12h  f12h - (- v02h + 2 v12h - v22h )/(2h)2.
• Correct: vh  vh + Ih2hv2h.

U-Boulder 283 of 312

 Eliminate more
f12h involves
rh at x3h & x4h!
Store vh only @ x3h - x7h & save v2h  w12h !

• Initialize vh = 0, w12h = 0, & f12h  (f1h + 2f2h + f3h )/4 .

• Relax on vh on the local fine grid.
• Compute rh = fh - Ah vh (not at x1h or x2h) & transfer to 2h:
f22h  (r3h + 2r4h + r5h )/4 & f32h  (r5h + 2r6h + r7h )/4 .
• Compute an approximation, v2h, to the solution of the 2h
residual equation, A2h u2h = f2h .
• Update the residual at x12h for later cycles:
f12h  f12h - (- v02h + 2 v12h - v22h )/(2h)2.
• Accumulate the 2h approximation: w12h  w12h + v12h.
• Correct: vh  vh + Ih2hv2h (not @ x1h or x2h).

U-Boulder 284 of 312

 Eliminate the rest
r3h doesn’t change on h.
Compute change in r3h.
2h residual @ x1h = g12h - (- w12h + 2 w22h - w32h )/(2h)2 !

• Initialize vh = 0, w12h = 0, & f12h  (f1h + 2f2h + f3h )/4 .

• Relax on vh on the local fine grid.
• Compute the right sides for 2h:
f22h  g12h - (- w12h + 2 w22h - w32h )/(2h)2 &
f32h  (r5h + 2r6h + r7h )/4 .
• Compute an approximation, v2h, to the solution of the 2h
residual equation, A2h u2h = f2h .
• Update the residual at x12h for later cycles:
f12h  f12h - (- v02h + 2 v12h - v22h )/(2h)2.
• Accumulate the 2h approximation: w12h  w12h + v12h.
• Correct: vh  vh + Ih2hv2h (not @ x1h or x2h or x3h).

U-Boulder 285 of 312

 Interpretation
composite grid

(- v0c + 2 v1c - v2c )/(2h)2 = f1c ≡ (f1h + 2 f2h + f3h )/4

(- v1c + 2 v2c - v3c )/(2h)2 = f2c ≡ (f3h + 2 f4h)/4

(- vci-1 + 2 vic - vci+1 )/h = fic ≡ fhi+2

U-Boulder 286 of 312

 Issues

• Adaptivity
• Error estimates
• Norms (proper scaling)
• Multiple dimensions
– Slave points
– More complicated stencils
• Data structures
• Parallel algorithms (AFAC)
• Time-space

U-Boulder 287 of 312

 Two-spike example (Laplacian)

• Global grid h & one two-patch refined level h/2.

• V(1,0)-cycles, Gauss-Seidel
• Asymptotic convergence of the solver
• Scaled L2 discretization error estimate

U-Boulder 288 of 312

 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems •√ Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
√ – Neumann boundaries
– Analysis
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
√ – Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Complexity √ – Matrix coarsening
√ • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory •√ Finite Elements
– Spectral vs. algebraic – Variational methodology
√

U-Boulder 289 of 312

 10. Finite elements
FE, a variational methodology

FD methodology
FE methodology

method
method method

U-Boulder 290 of 312

 Discretization

What do you see?

•
• • ••
• •
• •
U-Boulder 291 of 312
 Localize

•
What choice
do you have?

U-Boulder 292 of 312

 Continuous piecewise linear functions

•
• • ••
• •
• •

U-Boulder 293 of 312

 Representation
a basis of “hat” functions
h
ε(i)
is piecewise linear &

• 0 at all nodes except

h
ε(i)
node i where it’s 1

••• •• •••
a general piecewise linear
function can be represented by

uh = Σi uihε(hi)

U-Boulder 294 of 312

 How do we use piecewise linears?

How do you take

2nd derivatives of this ???

•
• • ••
• •
• •
U-Boulder 295 of 312
 Weak form!

−u"= f
⇒ − u"v = fv ∀v
⇒ ∫ (−u")vdx = ∫ fvdx ∀v
⇒ ∫ u'v' dx − u'v | = ∫ fvdx
1
0 ∀v
⇒ ∫ u'v' dx = ∫ fvdx ∀v ∋ v(0) = v(1) = 0
⇒ (u',v') = ( f ,v ) ∀v ∋ v(0) = v(1) = 0
↓ ↓
h
Σjuj ε(j)
h h
ε(i)
U-Boulder 296 of 312
 FE basics
• Model problem Ω ⊂ ℜ2
Lu ≡ - uxx - uyy = f in Ω • Duality: Solving Lu = f is
u = 0 on ∂Ω equivalent to minimizing
the functional
• Sobolev spaces
F(u) = “(Lu, u)”/2 - (f, u)
L2(Ω) = {u : ∫Ωu2dΩ < ∞}
( ) = {u : u, ux , uy ε L2(Ω),
H01(Ω) ( ), u|∂Ω = 0} • Short story:

(u, v) = ∫Ωu v dΩ 1st derivative test

∇F(u) = Lu - f = 0
• L is self-adjoint positive definite: 2nd derivative test
(Lu, v) = (∇u, ∇v) = ∫Ω(uxvx+uyvy)dΩ = (u, Lv) F”(u) = L > 0
(Lu, u) > 0 if u ≠ 0

 ux 
∇u =  
U-Boulder
 uy  297 of 312
 Long story
Using symmetry & linearity of L & bilinearity of the inner product:
F(u + v) = (L(u + v), u + v)/2 - (f, u + v)
= [(Lu, u) + 2(Lu, v) + (Lv, v)]/2
- (f, u) - (f, v)
= F(u) + (Lu, v) - (f, v) + (Lv, v)/2
= F(u) + (Lu - f, v) + (Lv, v)/2

The last term is positive for v ≠ 0. Thus,

F(u + v) ≥ F(u) ⇔ (Lu, v) = (f, v) ∀ v ε H01(Ω)
⇔ Lu = f

U-Boulder 298 of 312

 FE constructs
• Assume that Ω is the unit square.
• Consider an nxn grid of square “cells”.
• Continuous piecewise bilinear elements:
Hh ⊂ H01(Ω).
• Each uh in Hh is determined by its node
values. This is how we’ll represent them!
• Within each square:
uh = axy + bx + cy + d
uh is linear in a coordinate direction
• If uh on one side of an element matches uh
on the other at the nodes, then it matches
on the common edge. So uh is continuous.

U-Boulder 299 of 312

 Continuity

•
•

U-Boulder 300 of 312

 FE discretization

• Minimize F(u) = (Luh, uh)/2 - (f, uh)

over uh ε Hh.
• Equilavent to solving
(Luh, vh) = (f, vh) ∀ vh ε Hh.
• Basis: ε(ij) is the element of Hh that
h
equals 1 @ node ij & 0 elsewhere.
• Expansion: uh(x,y) = Σij uij ε(ij) (x,y).
• Problem: What is Luh ≡ - huxxh - uyy ???
h h

U-Boulder 301 of 312

 Weak form
• The Gauss Divergence Theorem & homogeneous
boundary conditions yield
(Lu, v) = (- uxx - uyy, v) = (-∇•∇u, v) = (∇u, ∇v)

• Note:
(∇u, ∇v) = ∫Ω(uxvx + uyvy)dΩ.

• So the problem becomes

(∇uh, ∇vh) = (f, vh) ∀ vh ε Hh
or
∫Ω(uxvx + uyvy)dΩ = ∫ΩfvhdΩ ∀ vh ε Hh.
h h h h

U-Boulder 302 of 312

 Towards the matrix equation
for nodal values

(∇uh, ∇vh) = (f, vh) ∀ vh ε H h

• Using uh(x,y) = Σij uijh ε(hij) (x,y) & choosing vh = ε(hkl) :

Σij uijh (∇ε(hij), ∇ε(hkl)) = (f, ε(hkl)) ∀ k, l
• Matrix terms (∇ε(ij), ∇ε(kl)) are 0 when
h h
|i - k| or |j - l| > 1.
• We compute
(∇ε(ij), ∇ε(ij)) = 8/3 & (∇ε(ij), ∇ε(i±1j ±1)) = -1/3.
• Assumeh f is bilinear:
h h h
(f, ε(kl)) = ∫Ωf ε(kl)dΩ ≈ h2f(xkl).
h h

actually,
Bh fh
uh = (uijh) & fh = (h2f(xkl)) (mass matrix)
U-Boulder 303 of 312
 The matrix equation
Ahuh = fh
where
uh = (uij) & fh = (h2f(xkl))
h

& the matrix is given by the stencil

−1 −1 −1
1 
Aij = −1 8 −1.
h
stiffness matrix
3 
−1 −1 −1
U-Boulder 304 of 312
 Some matrix properties
−1 −1 −1
1 
Aij = −1 8 −1
h

3 
−1 −1 −1
• Symmetric! ij “reaches” to i±1j±1 as i±1j±1 to ij.
• Singular? Ah 1 = 0 ?! Not next to boundaries!

@ west boundary: −1 −1

1 
Aij =  8 −1
h

3 
• Positive definite! −1 −1
Diagonally dominant (strictly so @ boundaries).

U-Boulder 305 of 312

 Abstract FE relaxation
focus on functions, not nodal vectors

• Relaxation involves “local” changes: uh ← uh - s εh for some

scalar s & εh = ε(hij).
But how do we pick s ???
• Use FE principle of minimizing F(uh - sεh) over s :
F(uh - sεh) = (L(uh - sεh), uh - sεh)/2 - (f, uh - sεh)
= F(uh) - s (Luh - f, εh) + (s2/2) (Lεh, εh).
• So just pick s by taking the derivative of this quadratic
polynomial & setting it to 0: s = (Luh - f, εh)/(Lεh, εh).

This is just Gauss-Seidel!

U-Boulder 306 of 312
 Abstract FE coarsening
again with focus on functions

• Coarsening involves a “global” change: uh ← uh + w2h for

some coarse-grid function w2h.
But how do we pick w2h ???

• Use FE principle of minimizing F(uh + w2h) over w2h:

F(uh + w2h) = (L(uh + w2h), uh + w2h)/2 - (f, uh + w2h)
= F(uh) + (Luh - f, w2h) + (Lw2h, w2h)/2.
• So just pick w2h by taking the gradient of this quadratic
functional & setting it to 0. This is tricky because you need
to write the gradient as a function in the subspace H2h.
We go instead from abstract functions to nodal vectors…

U-Boulder 307 of 312

 h
Concrete FE coarsening: I2h
now with focus on nodal vectors

• Adding nodal representations of vh & v2h (ε = ε(ij)) :

v2h(x,y) = Σij v2hij ε2h (x,y) (sum over 2h indices
2h nodal values ↔ even h indices)

= Σij vhij εh (x,y) ???

h nodal values

• We should be able to do this because v2h ε H2h ⊂ Hh.

• Row 2j:
vh2i 2j = v2hij
vh2i+1 2j = (v2hij + v2hi+1 j)/2
vh2i 2j+1 = (v2hij + v2hij+1)/2
vh2i+1 2j+1 = (v2hij + v2hi+1 j +
v2hij+1 + v2hi+1 j+1)/4
U-Boulder 308 of 312
 Concrete FE coarsening (cont’d)
Ahuh = fh
• Solving this matrix equation is equivalent to minimizing
Fh(vh) ≡ (Ah vh, vh)/2 - (fh, vh) (parens here mean Euclidean norm)
over vh ε Hh. So how do we now correct vh ???
• We minimize Fh(vh + I2h v2h) over v2h ε H2h :
h
Fh(vh + I2h v2h)
h
= (Ah (vh + I2h v2h), vh + I2h v2h)/2 - (fh, vh + I2h v2h)
h h h
= Fh(vh) + (A2h v2h, v2h)/2 - (f2h, v2h)
variational conditions
h T
f2h = I2h(fh- Ah vh)
A2h = I2hhTAhI2hh
≡ Fh(vh) + F2h (v2h)
U-Boulder 309 of 312
 Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems •√ Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
√ • Selected Applications
– Convergence tests
√ – Neumann boundaries
– Analysis
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
√ – Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Complexity √ – Matrix coarsening
√ • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory •√ Finite Elements
– Spectral vs. algebraic – Variational methodology
√ √

U-Boulder 310 of 312

 Multigrid rules!
We conclude with a few observations:

o We have barely scratched the surface of the myriad ways

that multigrid has been, & can be, employed.

o With diligence & care, multigrid can be made to handle many

types of complications in a robust, efficient manner.

o Further extensions to multigrid methodology are being sought

by many people working on many different problems.

U-Boulder 311 of 312

 Multigrid/multilevel/multiscale
an important methodology

• Multigrid has proved successful on a wide variety of

problems, especially elliptic PDEs, but has also found
application in parabolic & hyperbolic PDEs, integral
equations, evolution problems, geodesic problems, …

• Multigrid can be optimal, often O(N ).

• Multigrid can be robust in a practical sense.

• Multigrid is of great interest because it is one of the

very few scalable algorithms, & it can be parallelized
readily & efficiently!
U-Boulder 312 of 312