Lecture notes in theoretical-computational chemistry

Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

1
The numerical solution of Poisson's
equation using multigrid methods
I. Poissons equation
The electrostatic potential once the density is given is:
=

(1.1)
However, it is not easy to perform this numerically. As you can see, for each point , one
has to do an integral on . Is there a better way? One approach is to represent the
potential as a solution to a differential equation. Indeed, we will show that the required
equation is Poisson's equation:

= 4 (1.2)
Where is the electrostatic potential and is the charge density. This equation
must be solved over all space. It is a way to obtain the potential due to a distribution
of charges in space. If the potential is assumed to decay to zero when far from the origin
( 0, ) then the solution of this equation is just Eq. (1.1). To see this, let us
apply the Laplace operator to this equation:

1
|

(1.3)
We will show now that the function

||
is very special: it is zero at all and its
integral over space is finite. Thus it is proportional to a delta-function. Lets prove it.
Assume first that . Then:
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

2

1
||
=

2
+
2
+
2
=

2
+
2
+
2

3/2

=
1

2
+
2
+
2

3/2
+3

2

2
+
2
+
2

5/2

=
1

2
+
2
+
2

3/2
1 3

2

2
+
2
+
2

with obvious terms for

and

1
||
=
1

2
+
2
+
2

3/2
1 3

2

2
+
2
+
2

1
||
=
1

2
+
2
+
2

3/2
1 3

2

2
+
2
+
2

Therefore, we have, for the sum over the x, y, and z derivatives, i.e. for the Laplacian:

1
||
=
1

/
3 3 = 0 (1.4)
This shows the function is zero everywhere except at the origin where it is divergent.
Next, we show that the integral of this divergent function gives a finite value.
We first make an infinitesimal change in the denominator, rendering the integral finite:

1
||

= lim

1
||

(1.5)
Next, notice that the integrand depends only on and not on the angles and so that
these angles can be integrated to 4 and we are left with the 1-dimensional integral. In
radial coordinates the Laplacian is replaced by

, so:

1
||

= lim

(1.6)
Next, we evaluate the second derivative:
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

3

1
||

= lim

= lim

+
3

(1.7)
Pulling out of the parenthesis the factor

we obtain after some elementary manipulation:

1
||

= lim

(1.8)
In this last integral, it is amazing that we can scale out by a mere change of variable
=

. This gives:

1
||

= 12 lim

+ 1

(1.9)
It is a kind of a miracle that the integral is over the variable is independent of so we
can just forget the limit process. This technique, where a variable is entered into a
seemingly divergent integral only to be eliminated when a convergent form is obtained is
called renormalization technique. The renormalization thus eliminated the divergence
completely:

1
||

= 12

+ 1

= 4
(1.10)
Here we used the fact that:

2
+1
5

=
tan
2

tan
2
+1
5
tan

= tan
2
cos
5

cos
2

= sin
2
cos

= sin
2
sin

=
sin

=
1
3

Thus:
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

4

1
||

= 4 (1.11)
From the fact that

1
||
= 0 for all 0 and its integral is 4 we conclude:

1
||
= 4 (1.12)
We see that the differential equation is equivalent to the integral. However, the integral
can be pretty difficult to perform numerically while the differential equation is perhaps
more efficiently handled. In addition, with a differential equation one can generalize the
solution by imposing boundary conditions. For example, when there are metal objects
around the potential is constant inside the areas where the metals lie.
So, we will learn how to numerically solve Poissons equation as a means of computing
the Hartree potential in DFT. This can be an efficient alternative to doing 2-electron
integrals in quantum chemical codes. Indeed, when = the electrostatic
potential is nothing else than the Hartree potential

. Note, that once the

Hartree potential is obtained the Hartree energy

is simply a
single integral:

=
1
2

(1.13)
II. Finite difference formulae
In this section we discuss a technique for producing finite difference formulae to
approximate the operation of derivatives on functions on a grid.
Suppose we have a grid

= . Any function is represented on the grid using its

values on the grid points:

. How do we represent derivatives? Since

derivatives are linear operators and any function can be written as a linear superposition
of plane waves, a very general technique is to consider just a general single plane wave
on the grid: =

.
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

5
First, let us state that not all plane waves can be reasonable represented on a grid. For
example, the wave sin =

with =

will be represented on the grid

by:

= sin

= 0. So it is indistinguishable from the function = 0. It is also

indistinguishable from =

with =

etc. Thus, the fact that we have a finitie

sampling restricts our description in a fundamental way: the highest frequency
representable on a grid is:

(2.1)
Functions with higher frequencies are aliased to lower ones. The error incurred by
descretization is called the descretization error. We call frequencies with || >

high frequency and those with || <

low frequency:
low freqency: || <

2

high freqency: || >

2

(2.2)
Of course, high and low are relative superlatives, they are related to the grid spacing .
Clearly, the lower the frequency the better it is described by the grid, since there are
many sampling points in each wave length. High frequencies sampled by few points per
wave length well.
Now, let us come back to the question of derivatives. We were considering the function
=

. We want to find a local approximation for the derivative

in terms of

values at consecutive (nearby) grid-points. For example, with 3 consecutive

values:

(2.3)
For our plane wave we have:

(2.4)
Thus:
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

6

(2.5)
In order to solve this, denote: =

then = log and obtain the polynomial

approximation:
log

(2.6)
Now, how do we find the best coefficients

? We want the formula to best reproduce

the derivative for small frequencies, since then we know that the original function is well
represented on the grid (there is no point at demanding high accuracy of the derivative for
functions which are themselves not well represented). So, we want high accuracy when
is close to zero (low frequency). Thus we are looking at close to 1. So we Taylor-
expand = log around = 1 and use the expansion coefficients for determining

. We have:

= log + 1 ,

so: 1 = 0,

1 = 1,

1 = 1 thus,
we have:
log = 1 +
1
2
1

+ 1

=
1
2
1 +1 + 1

=
1
2

1 + 1

keeping only terms to second order. Thus:
1
2

1 +

(2.7)
We now immediately read off the coefficients:

=
1
2

= 0

=
1
2

(2.8)
The finite difference formula is:

2
+

(2.9)
Let us go for higher order accuracy. We include more points:

(2.10)
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

7
Once again we want to get the best coefficients:

log

(2.11)
So, we Taylor-expand =

log around = 1. We have:

= 2 log +,

= 2 log +3,

and

. Thus:

1 =1,

1 =3,

1 = 2,

1 = 2 ,

log

=
1

1 +
3
2
1

+
2
6
1

2
24
1

=
1
12
1 8 +8

+1

(2.12)
Thus:

=
1
12

=
8
12

= 0

=
8
12
,

=
1
12

(2.13)
The derivative is (VWM):

+8

12
+

(2.14)
The same technique can be applied to the second derivative. We know that

. Thus:

(2.15)
Put =

and so: log =

. Thus
log

(2.16)
We Taylor-expand:
log

= 1

(2.17)
Thus:

2 + 1

(2.18)
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

8
And so the coefficients can be read off:

=
1

=
2

=
1

(2.19)
And:

=
2 + +

(2.20)
The next order:

(2.21)
Or:

log

(2.22)
Now:

log

= 1

+ 1

1
12
1

=
1
12
+
4
3

5
2

+
4
3

1
12

(2.23)
Thus:

=
1
12

=
4
3

=
5
2

=
4
3

=
1
12

(2.24)
And so (VWM):

=
2 +16 30 + 16 + +2
12

This is a useful formula, since it gives high accuracy for only 5 function evaluations at
each grid point.
Suppose we are next to the left grid boundary. Then we cannot go to the left. Still we can
use our method. For first derivative:

(2.25)

Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

9
Then:

(2.26)
Thus:
log

(2.27)
Then:
log =
1
2
3 + 4

(2.28)
So:
3 + 4

(2.29)
And we have:

=
3
2

=
2

=
1
2

(2.30)
And:

=
3

+4

2
+

(2.31)
The second derivative will be:
log

(2.32)
So:
1 2 +

(2.33)
So:

=
1

=
2

=
1

(2.34)
And:
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

10

+
(2.35)
This formula is of low quality. Next:
log

(2.36)
The expansion is log

13

. Thus:

=
45

154

+214

156

+ 61

10

12

(2.37)
III. Discretization
In order to solve numerically Poissons equation we need to represent the functions
and in some numerical way. One approach is to use a grid. We use a grid to
describe functions. The grid spans a portion of 1D, 2D or 3D space. For example, we
will work with 2D. The physical space is a 2D box of lengths

and

. In x dimension
we have 1
x
N points and in y dimension 1
y
N thus the pairs of integers ( ) , i j ,
0...
x
i N = and 0...
y
j N = designate ( )( ) 1 1
x y
N N points in the plane. The grid
spacing in each direction is assumed, for simplicity equal: =

. We will suppose
that the function is strictly zero when 0 i = or
x
i N = or 0 j = or
y
j N = .
The charge density is given on the grid as:
( )
,
ij i j
x y = .
ij
must be obtained by
solving Poisson's equation. To do this, we must define the Laplacian on the grid. We use
the finite difference formula derived in the previous section
Thus:

( )
, 1 , , 1 2 2
2
2
i j i j i j
y
ij
O h
h

l
=
l
(2.38)
And similar expression for
x

Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

11
To study the approximation, let us use a 1D example. For a grid with grid-spacing h , the
maximal frequency represented on the grid is given by:

min max
min
2
2h k
h

= = = (2.39)
Let us then look at the following analysis. Consider again the Laplacian operation on a
wave
n
ik
n
e =
x
:

2
2 2
2 cos 1
2
n n
ikh ikh
ikx ikx ikx
n
e e kh
e e e
h h

l
=
l
(2.40)
Compare this with the exact solution:
2
n
ikx
k e , we see that the approximation implies:
2
2
cos 1
2
kh
k
h

. For 1 kh we have ( ) ( )
( )
2 4
1
cos 1
2
kh kh O kh = , so the
approximation is reasonable. But once the frequency k is high, so kh is considerably
larger than 1, this approximation is very bad (see figure below). As mentioned above, yhe
error incurred by descretization is called the descretization error.

-12
-10
-8
-6
-4
-2
0
2
4
6
0 1 2 3 4
kh

"
exact
disrete
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

12
IV. Solving Poisson's equation by iterative smoothing
How do we solve? We denote the exact solution of the descretized (algebraic) equation
by
n
:

1 1
2
2
n n n
n
f
h



= (3.1)
or:

2
1 1
2
n n n n
h f

= (3.2)
How close is the exact solution

of the algebraic equation to the exact analytical

solution of the differential equation

= is a matter of how well the problem is

represented on the grid. Since we know descretization errors are proportional to

we know how to control this error.
Suppose now we have an approximate solution
n
to the descritized, algebraic, equation.
It differs from the exact solution by the algebraic error vector:

n n n
= (3.3)
A measure of this kind of error is =

( is the total number of grid points).

Another measure is the residual vector:

(3.4)
Again, the norm of this vector is a global indicator called the residual: =

.
How can we reduce these indicators once we have a guess to the solution

?
We can try to improve it by iteration. In Eq. (3.2) we solve for

and make it an
iteration:

( )
2
1 1
1
2
better
n n n n
h f

= (3.5)
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

13
We are assuming that the bopundary conditions are

= 0 so we do not change
these values: only the inner gridpoint values of

are changed). Let us take an example.

The function we take is a spike

=
,

where +1 is the number of grid points.

The interval is 1,1. Thus =

. For = 20, we have = 0.1 and the iterations

show the following convergence:

Figure 1: The residual vs num of iterations for the h=1/16 grid (left) and h=1/64 grid (right).
We find that the error indicators can be reduced by this method but the process is very
slow. Initially it looks promising but after a few iterations it slows down. When is
larger the situation is worse: after 80 ietrations the residual was reduced by just a little
more than a factor 2.
In what way
better
n
this really better? Let's check. The difference between and , i.e.
the error is a combination of waves:

n
ikx
n n k
k
e =

(3.6)
Because of linearity of the problem, we can treat each wave separately, so suppose just:

n
n
ikx
n n k
ikx better better
n n k
e
e


=
=
(3.7)
Where k can be any wave-vector. Plugging this in (3.5), using (3.2), we have:
0 20 40 60 80
0.010
0.020
0.030
0.015
20 40 60 80
0.0050
0.0020
0.0030
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

14

( ) ( )
2
1 1
1 1
2 2
cos
n
n
ikx better ikh ikh
n n n n k
ikx
n k
h f e e e
khe

=
=
(3.8)
Thus cos
better
k k
kh =
and
:
cos
better
k
k
k
s kh

= = (3.9)
k
s is the reduction factor of the error at frequency k . For medium frequencies,
2
kh


we see that the iteration effectively reduces the error (
k
s small). But for low frequency, at
1 kh or high, at kh , the iteration is very inefficient (
k
s close to 1).
Thus we see that the first few iterations may be efficient since they reduce efficiently all
the medium components. So the total error drops dramatically in the first iterations. But
after that there are no more medium frequencies. Only high or low frequency errors
remain. For these the iterations have virtually no effect! In the example we studied, the
inefficiency is seen. We needed hundreds of iterations to get reasonable convergence in
the coarse grid when = 0.1. In a finer grid, when = 0.01 the solution, since it has a
cusp, has a much higher content of high frequencies and these are not well suppressed by
the iterations.
Lets try another type of iteration. Adding 2

where is arbitrary to both sides of

Eq. (3.2), we obtain:
21 +

+2

(3.10)
And so, instead of Eq. (3.5), we have now the iteration:

=
1
21 +

+2

(3.11)
This iteration has the arbitrary parameter which gives it an additional degree of freedom.
When we take = 1 we have for our problem:
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

15

Figure 2: Same as Figure 1 but for = .
It seems nothing substantial has been gained: convergence is slow. But let's wave analyze
it:

2 cos
2 2 1
n n
ikh ikh
ikx ikx better
n n k n k
e e kh
e e

= =

(3.12)
And the reduction factor is:

cos
1
k
kh
s

(3.13)
The effect of the additional degree of freedom is shown in the graph, where we take 4
cases: 0, 0.5,1,2 = . The first of these is our previous iteration:
0 20 40 60 80
0.020
0.030
0.015
20 40 60 80
0.0050
0.0030
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

16

Figure 3: reduction factor vs frequency for 3 values of .
We see that the choice = 1 is special since it yields an iteration that acts as a
"smoother": the higher the frequency the more efficient is the iteration at reducing the
error. With these iterations the spike charge problem above will be much better behaved.
Yet, even then one needs a huge number of iterations because the low frequency
components decay very slowly: After several iterations of the smoother, any initial error
is smoothed and has only long wavelengths: the high frequencies were killed. These
low frequency errors are not efficiently damped and the iterations become inefficient.
Another smoother can be obtained by taking = 0 iteration "in-place", i.e. iterate on:

( )
2
1 1
1
2
better better
n n n n
h f

= (3.14)
This is called the Gauss Seidel iteration. We can analyze it as:

( )
1 1
1
2
n n n
ikx better ikx better ikx
k k k
e e e

= (3.15)
Thus:
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
kh
p
s
H
k
L
w=2
w=1
w=
1
2
w=0
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

17

( )
2
1
2
5 4cos
better ikh ikh
k k
better
ikh
k
ikh
k
e e
e
e
kh

=
= =

(3.16)
The successive over-relaxation Gauss-Seidel (SOR-GS) iteration is obtained by working
in place and in additional averaging the old iterant and the new one:

( ) ( )
2
1 1
2 1 2
better better
n n n n n
h f

= (3.17)
It was found that the GS method over shoots the solution and so the averaging procedure
introduces some relaxation of the overshooting.
The analysis gives:

( ) ( )
( ) ( ) ( )
2 1 2
2 2 1
better better ikh ikh
k k k k
better ikh ikh
k k
e e
e e

=
=


(3.18)
Thus the relaxation factor becomes:

( )
2 1
2
ikh
better
k
k ikh
k
e
s
e

= =

(3.19)
The performance as smoothers is given in the graph:
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

18

Figure 4: The Gauss-Seidl with various successive relaxation coefficients.
The = 1 and =

behave similarly. However the = 1 method damps higher

frequencies somewhat better. The benefit of the GS-SOR method is that it needs less
memory since everything is done in place.
V. The multigrid method
We select a smoother. It efficiently reduces the error at high frequencies, but it is very
inefficient for low ones. Can we turn this deficiency into a winning advantage? The
answer, pioneered by Brandt[1] is yes. Since the error is smooth after iterations, we
should move the problem to a coarse grid and try to determine the error there. By doing
this we should not lose much, because we are only looking for the long wavelengths, and
on a coarse grid all operations are twice as fast (in 1D, and 4 or 8 times as fast in 2D or
3D). Some of the low frequency components in the fine grid are now high frequencies in
the coarse grid. So they can be effectively reduced by additional smoothing iterations in
that grid. If necessary we can then go to even coarser grid.
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
h k
p
s
H
k
L
w

= 1
w

= 23
w

= 12
w

= 13
w = 1
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

19
To explain how this idea is implemented, let us write the Poisson equation on the grid
with spacing h (called henceforth h grid) as:

h h h
n
n
L f
l
=
l
(4.1)
L is the h-grid discretized Laplacian operator. The current approximation, after 2-3
smoothing iterations is
h
n
, the exact solution is
h
n
and the error:

h h h
n n n
= (4.2)
We dont know what the error is, but since we used a smoother, we know it is composed
of mostly low frequency components: it is smooth. While the error is unknown, the
residual can be calculated: it is the deviance of the charge density from the Laplacian of
the potential:

h h h h
n n
n
r f L
l
=
l
(4.3)
By plugging (4.2) into (4.3), we get:

h h h h h h h h
n n
n n n
r f L L L
l l l
= =
l l l
l l l
(4.4)
Thus, that the unknown error and known residual obey a Poisson equation themselves!
The new charge density is the residual and the new potential sought for is the error. Since
the error is known to be dominated by low frequency components, we can solve this
new Poisson equation on a coarser grid a grid with spacing 2h .
We use the fine-to coarse operator
2h
h
I to designate a method of transforming the residual
(new "charge density") to the coarse grid. We also designate
2h
L the discretization of the
Laplacian operator on the coarse 2h grid:

2 2 2
2 2
h h h
n
n
h h h
h
L f
f I r

l
=
l
l
=
(4.5)
Once
2h
is solved, we transform it back to the fine grid using a coarse-to-fine
transform:
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

20

2
2
h h h
h
I = (4.6)
We then add
k
to
h
and iterate a few more times to smooth again on the fine grid.
This additional iteration is usually necessary because some high frequencies which were
aliased into the coarse grid now reappear in the fine one.
The process can be done several times. Furthermore, the passage to coarser and coarser
grids is advantageous, until a coarse enough grid is reached where matrix inversion is
beneficial. Suppose problem can is solved with 10 smoother iterations in each level so
smoother work on the level is 10

where

= 2

is the number of grid points in the

th level and where is the dimension of the problem ( = 1,2,3). The total amount of
work is 10

= 10

= 10

1. Thus, in general, the amount of

work to solve the problem is of the order 10

.
There are many methods to accomplish
2h
h
I . The most straightforward is the injection,
which in 1D is:

2 2
2
h h h h
n h n
n
f I r r
l
= =
l
(4.7)
This is directly generalized to 2 and 3D.
The transformation back can be done by interpolation:

( )
2
2
2 2 2
1
2
2 2 1
h
k h h h h h
h h n n h n
n
k k
n k
I
n k

'
1
=
1
1 l
= =
!
l
l
1
=
1
1+
(4.8)
A convenient description is:

2
1 1
1
2 2
h
h
I
1



( )
(4.9)
Meaning that the point in the coarse grid donates a full weight to itself in the fine grid and
half to each neighbor point. Overall it has a weight of 2. This is fine since the long wave
length shape is preserved this way and short wavelength distortions will be easily fixed
by smoothing iterations. In 2D we have
Lecture notes in theoretical-computational chemistry
Professor Roi Baer

All rights reserved to Roi Baer. Email: [email protected]

21

2
1 1 1
4 2 4
1 1
1
2 2
1 1 1
4 2 4
h
h
I
1

( )
(4.10)
You can read more about multigrid in the book A Multigrid Tutorial by Briggs et al
[2].


[1] A. Brandt, Multi-Level Adaptive Solutions to Boundary-Value Problems, Math.
Comp 31, 333 (1977).
[2] W. L. Briggs, V. E. Henson, and S. F. McCormick, A Multigrid Tutorial (SIAM:
Society for Industrial and Applied Mathematics, Philadelphia, 2000).