Sparse Grid Tutorial

Jochen Garcke
Centre for Mathematics and its Applications
Mathematical Sciences Institute
Australian National University
Canberra ACT 0200, Australia
[email protected]
August 12, 2006

1 Introduction
The sparse grid method is a special discretization technique, which allows to
cope with the curse of dimensionality of grid based approaches to some extent.
It is based on a hierarchical basis [Fab09, Yse86, Yse92], a representation of a
discrete function space which is equivalent to the conventional nodal basis, and
a sparse tensor product construction.
The method was originally developed for the solution of partial differen-
tial equations [Zen91, Gri91, Bun92, Bal94, Ach03] and is now also successfully
used for integral equations [FHP96, GOS99], interpolation and approximation
[Bas85, Tem89, SS99, GK00, Kna00, KW05b]. Furthermore there is work on
stochastic differential equations [ST03a, ST03b], differential forms in the context
of the Maxwell-equation [GH03] and with a wavelet-based sparse grid discretiza-
tion parabolic problems are treated in [vPS04]. Besides Galerkin finite element
approaches there are also finite differences on sparse grids [Gri98, Sch99, Kos02,
GK03] and finite volume approaches [Hem95].
Besides working directly in the hierarchical basis a sparse grid represen-
tation of a function can also be computed using the combination technique
[GSZ92], where a certain sequence of partial functions is linearly combined.
Applications include eigenvalue problems [GG00], numerical integration [GG98,
BD03, GG03], parabolic equations for options pricing [Rei04], machine learn-
ing [GGT01, GG02, Gar04, GG05] and data analysis [LNSH05] and it is used
for solving the stochastic master equation applied to gene regulatory networks
[HBS+ 06].
The underlying idea of sparse grids can be traced back to the Russian math-
ematician Smolyak [Smo63], who used it for numerical integration. The concept
is also closely related to hyperbolic crosses [Bab60, Tem89, Tem93a, Tem93b],
boolean methods [Del82, DS89], discrete blending methods [BDJ92] and split-
ting extrapolation methods [LLS95].

1
 For the representation of a function f defined over a d-dimensional domain
the sparse grid approach employs O(h1 1 d1
n log(hn ) ) grid points in the dis-
n
cretization process, where hn := 2 denotes the mesh size. It can be shown
that the order of approximation to describe a function f , under certain smooth-
ness conditions, is O(h2n log(h1
n )
d1
). This is in contrast to conventional grid
methods, which need O(hn ) for an accuracy of O(h2n ). Since the curse of di-
d

mensionality of full grid method arises for sparse grids at this much smaller
extent they can be used for higher dimensional problems.
For ease of presentation we will consider the domain = [0, 1]d here and in
the following. This situation can be achieved for bounded rectangular domains
by a proper rescaling.

2 Sparse grids
We introduce some notation while describing the conventional case of a piecewise
linear finite element basis. Let l = (l1 , . . . , ld ) d denote a multi-index.
We define the anisotropic grid l on with mesh size hl := (hl1 , . . . , hl ) :=
d
(2l1 , . . . , 2ld ), it has different, but equidistant mesh sizes in each coordinate
direction t. This way the grid l consists of the points

xl,j := (xl1 ,j1 , . . . , xld ,jd ), (1)

with xlt ,jt := jt hlt = jt 2lt and jt = 0, . . . , 2lt . For a grid l we define an
associated space Vl of piecewise d-linear functions

Vl := span{l,j | jt = 0, . . . , 2lt , t = 1, . . . , d}, (2)

which is spanned by the usual basis of d-dimensional piecewise d-linear hat

functions
Yd
l,j (x) := lt ,jt (xt ). (3)
t=1

The one-dimensional functions l,j (x) with support [xl,j hl , xl,j + hl ] [0, 1] =
[(j 1)hl , (j + 1)hl ] [0, 1] are defined by:

1 |x/hl j|, x [(j 1)hl , (j + 1)hl ] [0, 1];
l,j (x) = (4)
0, otherwise.

See Figure 1(a) for a one-dimensional example and Figure 2 for a two-dimensional
basis function.

2.1 Hierarchical subspace-splitting

Till now and in the following the multi-index l d denotes the level, i.e. the
discretization resolution, of a grid l , a space Vl or a function fl , whereas the
multi-index j d gives the position of a grid point xl,j or the corresponding
basis function l,j ().

2
(a) Nodal basis f
ur V3 (b) Hierarchical basis f
ur V3

Figure 1: Nodal and hierarchical basis of level 3

Figure 2: Basis function 1,1 on grid 2,1 .

3
 We now define a hierarchical difference space Wl via
d
M
Wl := Vl \ Vlet , (5)
t=1

where et is the t-th unit vector. In other words, Wl consists of all k,j Vl
which are not included in any of the spaces Vk smaller1 than Vl . To complete
the definition, we formally set Vl := 0, if lt = 1 for at least one t {1, . . . , d}.
As can easily be seen from (2) and (5), the definition of the index set
  lt

d  jt = 1, . . . , 2 1, jt odd, t = 1, . . . , d, if lt > 0,

Bl := j  jt = 0, 1, (6)
t = 1, . . . , d, if lt = 0

leads to
Wl = span{l,j |j Bl }. (7)
These hierarchical difference spaces now allow us the definition of a multilevel
subspace decomposition. We can write Vn := Vn as a direct sum of subspaces
n
M n
M M
Vn := Wl = Wl . (8)
l1 =0 ld =0 |l| n

Here and in the following refers to the element-wise relation. |l| :=

Pd
max1td lt and |l|1 := t=1 lt are the discrete L - and the discrete L1 -norm
of l, respectively.
The family of functions
n
{l,j |j Bl }l=0 (9)

is just the hierarchical basis [Fab09, Yse86, Yse92] of Vn , which generalizes the
one-dimensional hierarchical basis [Fab09], see Figure 1(b), to the d-dimensional
case with a tensor product ansatz. Observe that the supports of the basis
functions l,j (x), which span Wl , are disjunct. See Figure 3 for a representation
of the supports of the basis functions of the difference spaces Wl1 ,l2 forming
V3,3 .
Now each function f Vn can be represented as
X X
f (x) = l,j l,j (x), (10)
|l| n jBl

where l,j are the coefficients of the representation in the hierarchical

tensor product basis. The number of basis functions, which describe a f Vn
in nodal or hierarchical basis is (2n + 1)d . For example a resolution of 17 points
in each dimensions, i.e. n = 4, for a ten-dimensional problem therefore needs
2 1012 coefficients, we encounter the curse of dimensionality.
1 We call a discrete space V smaller than a space V if k l and t : k < l . In the
k l t t t t t
same way a grid k is smaller than a grid l .

4
 W0,0 W1,0 W2,0 W3,0

W0,1 W1,1 W2,1 W3,1

W0,2 W1,2 W2,2 W3,2

W0,3 W1,3 W2,3 W3,3

Figure 3: Supports of the basisfunctions of the hierarchical subspaces Wl of the

space V3

Note, that for the spaces Vl the following decomposition holds

l1
M ld
M M
Vl := Wk = Wk .
k1 =0 kd =0 kl

2.2 Properties of the hierarchical subspaces

Now consider the d-linear interpolation of a function f V by a fn Vn , i.e.
a representation as in (10). First we look at the linear interpolation in one
dimension, for the hierarchical coefficients l,j , l 1, holds
f (xl,j h) + f (xl,j + h) f (xl,j1 ) + f (xl,j+1 )
l,j = f (xl,j ) = f (xl,j ) .
2 2

5
 2,1 1,1 2,3
3,1 3,3 3,5 3,7

Figure 4: Interpolation with the hierarchical basis

This and Figure 4 illustrate why the l,j are also called hierarchical surplus,
they specify what has to be added to the hierarchical representation from level
l 1 to obtain the one of level l. We can rewrite this in the following operator
form  
1 1
l,j = 1 f
2 2 l,j
and with that we generalize to the d-dimensional hierarchization operator as
follows
d   !
Y 1 1
l,j = 1 f (11)
t=1
2 2 lt ,jt
Note that the coefficients for the basis functions associated to the boundary are
just 0,j = f (x0,j ), j = 0, 1.
Now let us define the so-called Sobolev-space with dominating mixed deriva-
2
tive Hmix in which we then will show approximation properties of the hierar-
chical basis. We define the norm as
X  |k|1 2
 
kf k2Hmix
s =  xk  ,
 f 
0ks 2

2
and the space Hmix in the usual way:
s
Hmix := {f : : kf k2Hmix
s < }

Furthermore we define the semi-norm |f |Hmix

2 := |f |H 2
mix

 |k| 2
 1 
|f |H k :=  k f  ,
mix x 2

6
 s
Note that the continuous function spaces Hmix , like the discrete spaces Vl , have a
tensor product structure [Wah90, Hoc99, GK00, HKZ00] and can be represented
as a tensor product of one dimensional spaces:
s
Hmix = H s H s.

We now look at the properties of the hierarchical representation of a function

f , especially at the size of the hierarchical surplusses. We recite the following
proofs from [Bun92, Bun98, BG99, BG04], see these references for more details
on the following and results in other norms like k k or k kE . For ease of
presentation we assume f H0,mix 2 i.e. zero boundary values, and l > 0
(),
to avoid the special treatment of level 0, i.e. the boundary functions in the
hierarchical representation.
Straightforward calculation shows
Lemma 1. For any piecewise d-linear basis function l,j holds

kl,j k2 C(d) 2|l|1 /2 .

2
Lemma 2. For any hierarchical coefficient l,j of f H0,mix it holds
()

d
ht
Y Z
l,j = l,j D2 f (x)dx. (12)
t=1
2

Proof. In one dimension partial integration provides

xl,j +h
2 f (x)dx 2 f (x)dx
Z Z
l,j = l,j
x2 xl,j h x2
 x +h Z xl,j +h
f (x)dx l,j l,j f (x)dx
= l,j
x xl,j h xl,j h x x
Z xl,j Z xl,j +h
1 f (x)dx 1 f (x)dx
= +
xl,j h h x xl,j h x
1
= (f (xl,j h) 2f (xl,j ) + f (xl,j + h))
h
2
= l,j
h
The d-dimensional result is achieved via the tensor product formulation (11).
Lemma 3. Let f H0,mix2 be as in
() hierarchical representation above, it
holds  
|l,j | C(d) 2(3/2)|l|1 f |supp(l,j )  2 .
 
Hmix

7
Proof.
 
d d
ht ht
Y Z  Y
2
|l,j | =  l,j D f (x)dx kl,j k2 D2 f |supp(l,j )
 

t=1
2 
t=1
2 2
 
C(d) 2(3/2)|l|1 f |supp(l,j )  2
 
Hmix

2
Lemma 4. f H0,mix is as above in hierarchical representation. For its
()
components fl Wl holds

kfl k2 C(d) 22|l|1 |f |Hmix

2 . (13)

Proof. Since the supports of all l,j are mutually disjoint we can write
2
X X
2
|l,j |2 kl,j k22

kfl k2 =

=
l,j l,j (x)
jBl jBl
2

With Lemma 3 and 1 it now follows

X  2
kfl k22 C(d) 23|l|1 f |supp(l,j )  C(d) 2|l|1
 
2
Hmix
jBl
2
C(d) 24|l|1 |f |H 2
mix

which completes the proof.

2.3 Sparse grids

Motivated by the relation (13) of the importance of the hierachical compo-
nents fl Zenger [Zen91] and Griebel [Gri91] introduce the so-called sparse grids,
where hierarchical basis functions with a small support, and therefore a small
part in the function representation, are not included in the discrete space of
level n anymore.
Formally we define the sparse grid function space Vns Vn as
M
Vns := Wl . (14)
|l|1 n

We replace in the definition (8) of Vn in terms of hierarchical subspaces the

condition |l| n with |l|1 n. In Figure 3 the used subspaces are given in
black, the difference spaces Wl which are not used anymore, in comparison to
(8), are given in grey. Every f Vns can now be represented, analogue to (10),
as X X
fns (x) = l,j l,j (x). (15)
|l|1 n jBl

8
Figure 5: Two-dimensional sparse grid (left) and three-dimensional sparse grid
(right) of level n = 5 each.

The resulting grids corresponding to the approximation space Vns are called
sparse grids. Note that sparse grids were introduced in [Zen91, Gri91] using the
definition M
s
V0,n := Wl . (16)
|l|1 n+d1

This definition is useful for the numerical treatment of partial differential equa-
tions with Dirichlet boundary conditions where no degrees of freedom exist on
s
the boundary. Using V0,n the level of refinement n in the sparse grid corresponds
to the full grid case again. Examples in two and three dimensions are given in
Figure 5. Sparse grids are studied in detail in [Bun92, Bun98, BG99, Kna00,
BG04] besides others.
The following results hold for both definitions Vns and V0,n s
. The proofs are
s
somewhat easier without basis functions on the boundary, so we use Vn,0 in the
following, like it is done in the referenced source publications.
First we look at the size of the sparse grid space.
s
Lemma 5. The dimension of the sparse grid space V0,n , i.e. the number of
inner grid points, is given by
s
|V0,n | = O(h1 1 d1
n log(hn ) ) (17)

Proof. We follow [BG04] and use in first part the definition
(16), the size of
i1
a hierarchical subspace |Wl | = 2|l1|1 and that there are d1 possibilities to

9
represent i as a sum of d natural numbers.
 

M

X n+d1
X X
s
2|l1|1 2id
 
|V0,n | = 
 Wl  = = 1
|l|1 n+d1  |l|1 n+d1 i=d |l|1 =i
n+d1 
X i1
= 2id
d1
i=d
n1  
X i+d1
= 2i
i=0
d1

We now represent the summand as the (d1)-derivation of a function evaluated

at x = 2
n1  
X i+d1
2i
i=0
d1
n1
1 X
(d1) 
= xi+d1
(d 1)! i=0

x=2

 n
 (d1)
1 d1 1 x

= x

(d 1)! 1x


x=2
d1    (d1i) 
1 X d1 d1 n+d1 (i)

1
= x x

(d 1)! i=0 i 1x


x=2
d1 
X n+d1 
= (1)d + 2n (2)d1i .
i=0
i

The summand for i = d 1 is the largest one and it holds

 d1 
n (n + d 1)! n n d2
2 =2 + O(n )
(d + 1)!n! (d 1)!

which gives a total order of O(2n nd1 ) or in other notation, with hn = 2n ,

of O(h1 1 d1
n log(hn ) ).
This is far less than the size of the corresponding full grid space |Vn | =
O(hd
n ) = O(2
dn
) and allows the treatment of higher dimensional problems.
We now look at approximation properties of sparse grids. For the proof we
again follow [BG04] and first look at the error for the interpolation of a function
2 s
f H0,mix by f0,n V0,n which can be written in regard to the partial functions
from the hierarchical subspaces as
X X X
s
f f0,n = fl fl = fl .
l |l|1 n+d1 |l|1 >n+d1

10
For any norm now holds
X
s
kf f0,n k kfl k. (18)
|l|1 >n+d1

We need the following lemma to estimate the interpolation error

Lemma 6. For s it holds
 
X
s|l|1 sn sd
X
si i+n+d1
2 = 2 2 2
i=0
d1
|l|1 >n+d1

nd1
 
2sn 2sd 2 + O(nd2 ) ,
(d 1)!

Proof. The steps of the proof are similar to the ones in the previous lemma,
first we get
X
X X
2s|l|1 = 2si 1
|l|1 >n+d1 i=n+d |l|1 =i
 
X i1
= 2si
d1
i=n+d
 
sn sd
X
si i+n+d1
= 2 2 2 .
i=0
d1

Since

!(d1)
xn
 
X
i i+n+d1 X
i+n+d1
x = x
i=0
d1 (d 1)! i=0
(d1)
xn

n+d1 1
= x
(d 1)! 1x
n d1    (d1k)
x X d1 n+d1 (k)

1
= x
(d 1)! k 1x
k=0
d1    d1k
X n+d1 x 1
= ,
k 1x 1x
k=0

it follows with x = 2s the relation

  d1    d1 
X
si i+n+d1 X n+d1 n d2
2 2 =2 + O(n )
i=0
d1 k (d 1)!
k=0

which finishes the proof.

11
 2
Theorem 1. For the interpolation error of a function f H0,mix in the sparse
s
grid space V0,n holds

||f fns ||2 = O(h2n log(h1

n )
d1
). (19)
Proof.
X
||f fns ||2 kfl k2 C(d) 22|l|1 |f |Hmix
2

|l|1 >n+d1

nd1
 
C(d) 22n |f |Hmix
2 + O(nd2 ) ,
(d 1)!
which gives the wanted relation.
Note that same results holds in the maximum-norm as well:
||f fns || = O(h2n log(h1
n )
d1
)
2
for f H0,mix .

2.4 Sparse grid combination technique

The so-called combination technique [GSZ92], which is based on multi-variate
extrapolation [BGR94], is another method to achieve a function representation
on a sparse grid. The function is discretized on a certain sequence of grids using
a nodal discretization.A linear combination of these partial functions then gives
the sparse grid representation. This approach can have numerical advantages
over working directly in the hierarchical basis, where e.g. the stiffness matrix is
not sparse and the on-the-fly computation of the matrix-vector-product, which
complexity scales better, is challenging in the implementation [Ach03, Bal94,
Bun98].
In particular, we discretize the function f on a certain sequence of anisotropic
grids l = l1 ,...,ld with uniform mesh sizes ht = 2lt in the t-th coordinate
direction. These grids possess in general different mesh sizes for the different
coordinate directions. To be precise, we consider all grids l with
|l|1 := l1 + ... + ld = n q, q = 0, .., d 1, lt 0.
Note that in the original [GSZ92] and other papers as well, a slightly different
definition was used, again due to Dirichlet boundary conditions,
|l|1 := l1 + ... + ld = n + (d 1) q, q = 0, .., d 1, lt > 0.
The grids employed by the combination technique of level 4 in two dimensions
are shown in Figure 6.
A finite element approach with piecewise d-linear functions l,j (x) on each
grid l now gives the representation in the nodal basis
l l
21
X 2d
X
fl (x) = ... l,j l,j (x).
j1 =0 jd =0

12


4,0 3,1 2,2 1,3 0,4

3,0 2,1 1,2 0,3

=
X X
fnc = fl1 ,l2 fl1 ,l2
l1 +l2 =n l1 +l2 =n1 s4

Figure 6: Combination technique with level n = 4 in two dimensions

Finally, we linearly combine the discrete partial functions fl (x) from the different
grids l according to the combination formula
d1  
q d1
X X
fnc (x) := (1) fl (x). (20)
q=0
q
|l|1 =nq

The resulting function fnc lives in the sparse grid space Vns , the combined in-
terpolant is identically with the hierarchical sparse grid interpolant fns [GSZ92].
This can be seen by rewriting each fl in their hierarchical representation (10)
and some straightforward calculation using the telescope sum property, i.e. the
hierarchical functions get added and subtracted. We write it exemplary in the
two dimensional case, using fl1 +l2 Wl1 ,l2 instead of all the basis functions of
Wl1 ,l2 for ease of presentation
X X
fnc = fl1 ,l2 fl1 ,l2
l1 +l2 =n l1 +l2 =n1

fk1 ,k2 fk1 ,k2

X X X X X X
=
l1 n k1 l1 k2 nl1 l1 n1 k1 l1 k2 n1l1

fk1 ,1 + fk1 ,k2 fk1 ,k2

X X X X X
=
k1 l1 l1 n1 k1 l1 k2 nl1 k2 n1l1

fk1 ,1 + fk1 ,nl1

X X X
=
k1 l1 l1 n1 k1 l1

fk1 ,t
X
=
k1 +tn

with t := n + 1 l1 . This is exactly (15).

13
 Note that the solution obtained with the combination technique fnc for the
numerical treatment of partial differential equations is in general not the sparse
grid solution fns . However, the approximation property is of the same order as
long a series expansion of the error
d
cj1 ,...,jm (hj1 , . . . , hjm ) hpj1 . . . hpjm ,
X X
f fl = (21)
i=1 j1 ,...,jm 1,...,d

with bounded cj1 ,...,jm (hj1 , . . . , hjm ) exists, see [GSZ92]. Its existence was
shown for model-problems in [BGRZ94].
Let us again consider the two dimensional case and consider the error of the
combined solution f fnc following [GSZ92]. We have
X X
f fnc = f fl1 ,l2 + fl1 ,l2
l1 +l2 =n l1 +l2 =n1
X X
= (f fl1 ,l2 ) (f fl1 ,l2 ) .
l1 +l2 =n l1 +l2 =n1

Plugging in the error expansion (21) leads to

X
f fnc = c1 (hl1 ) h2l1 + c2 (hl2 ) h2l2 + c1,2 (hl1 , hl2 ) h2l1 h2l2

l1 +l2 =n
X
c1 (hl1 ) h2l1 + c2 (hl2 ) h2l2 + c1,2 (hl1 , hl2 ) h2l1 h2l2



l1 +l2 =n1
!
X X
= c1 (hn ) + c2 (hn ) + c1,2 (hl1 , hl2 ) 4 c1,2 (hl1 , hl2 ) h2n
l1 +l2 =n l1 +l2 =n1

And we get the estimation, using ci ,

 
 X X 
c 2
|f fn | 2 hn +  c1,2 (hl1 , hl2 ) 4 c1,2 (hl1 , hl2 ) h2n
 
 
l1 +l2 =n l1 +l2 =n1
X X
2 h2n + |c1,2 (hl1 , hl2 )| h2n + 4 |c1,2 (hl1 , hl2 )| h2n
l1 +l2 =n l1 +l2 =n1

2 hn + nh2n + 4(n 1)h2n

2

= h2n (5n 2) = h2n (5 log(h1

n ) 2)
= O(h2n log(h1 n ))

Observe that terms are canceled for hli with li 6= n and the accumulated h2l1 h2l2
result in log(h1 2 1
n )-term. The approximation order O(hn log(hn )) is just as in
Theorem 1. See [GSZ92, PZ99, Rei04] for results in higher dimension.
The combination technique was used in fluid dynamics [GHSZ93, GH95,
GT95, Hub96, Kra02], for the Lame-equation [Heu97], for eigenvalue-problems
[Gar98, GG00], singular perturbation problems [NH00], machine learning and

14
data analysis [GGT01, GG02, Gar04, GG05, LNSH05] and parabolic equations
for option pricing [Rei04].
There are close connections to so-called boolean [Del82, DS89] and dis-
crete blending methods [BDJ92], as well as the splitting extrapolation-method
[LLS95]. Furthermore, there are relations between the sparse grid combination
technique and additive models like e.g. ANOVA [Wah90] or MARS [Fri91].

A Sparse grids in python

We give the listing of some python code for a sparse grid without functions on
the boundary, i.e. according to formula (16). If you are interested in the source
files, write to [email protected].
A MATLAB implementation of sparse grids [KW05a, Kli06] can be found
here http://www.ians.uni-stuttgart.de/spinterp/.

Listing 1: sparse grid representation

import math

class gridPoint :
def i n i t ( s e l f , i n d e x=None , domain=None ) :
s e l f . hv = [ ] # h i e r a r c h i c a l v a l u e
s e l f . fv = [ ] # function value
i f i n d e x i s None :
s e l f . pos = [ ] # p o s i t i o n o f g r i d p o i n t
else :
s e l f . pos = s e l f . p o i n t P o s i t i o n ( index , domain )

def p o i n t P o s i t i o n ( s e l f , index , domain=None ) :

coord = l i s t ( )
i f domain i s None :
for i in range ( l e n ( i n d e x ) ) :
coord . append ( i n d e x [ i ] [ 1 ] / 2 . i n d e x [ i ] [ 0 ] )
else :
for i in range ( l e n ( i n d e x ) ) :
coord . append ( ( domain [ i ] [ 1 ] domain [ i ] [ 0 ] ) \
i n d e x [ i ] [ 1 ] / 2 . i n d e x [ i ] [ 0 ] + domain [ i ] [ 0 ] )
return coord

def p r i n t P o i n t ( s e l f ) :
i f s e l f . pos i s [ ] :
pass
else :
out =
for i in range ( l e n ( s e l f . pos ) ) :
out += s t r ( s e l f . pos [ i ] ) + \ t

15
 print out

class sparseGrid :
def i n i t ( s e l f , dim=1, l e v e l =1):
s e l f . dim = dim
self . level = level
s e l f . gP = {}
se lf . indices = [ ]
s e l f . domain = ( ( 0 . 0 , 1 . 0 ) , ) dim
s e l f . action = ()

def p r i n t G r i d ( s e l f ) :
print s e l f . hSpace

def e v a l A c t i o n ( s e l f ) :
b a s i s = ( s e l f . evalPerDim [ 0 ] [ s e l f . hSpace [ 0 ] 1 ] [ 0 ] , )
v a l u e = s e l f . evalPerDim [ 0 ] [ s e l f . hSpace [ 0 ] 1 ] [ 1 ]
# compute i n d e x o f nonz e r o b a s i s f u n c t i o n i n hSpace and i t s v a l u e on
for i in range ( 1 , s e l f . dim ) :
v a l u e = s e l f . evalPerDim [ i ] [ s e l f . hSpace [ i ] 1 ] [ 1 ]
b a s i s += ( s e l f . evalPerDim [ i ] [ s e l f . hSpace [ i ] 1 ] [ 0 ] , )
s e l f . v a l u e += s e l f . gP [ b a s i s ] . hv v a l u e

def e v a l F u n c t ( s e l f , x ) :
s e l f . value = 0.0
s e l f . evalPerDim = [ ]
for i in range ( s e l f . dim ) :
s e l f . evalPerDim . append ( [ ] )
for j in range ( 1 , s e l f . l e v e l +1):
# which b a s i s i s unzero on x f o r dim i and l e v e l j
pos = ( x [ i ] s e l f . domain [ i ] [ 0 ] ) / ( s e l f . domain [ i ] [ 1 ] \
s e l f . domain [ i ] [ 0 ] )
b a s i s = i n t ( math . c e i l ( pos 2( j 1))2 1)
i f b a s i s == 1:
basis = 1
s e l f . evalPerDim [ i ] . append ( [ ( j , b a s i s ) ] )
else :
s e l f . evalPerDim [ i ] . append ( [ ( j , b a s i s ) ] )
# v a l u e o f t h i s b a s i s f u n c t i o n on x [ i ]
s e l f . evalPerDim [ i ] [ j 1 ] . append ( e v a l B a s i s 1 D ( x [ i ] , \
s e l f . evalPerDim [ i ] [ j 1 ] [ 0 ] , s e l f . domain [ i ] ) )
s e l f . action = s e l f . evalAction
s e l f . loopHierSpaces ()
return s e l f . v a l u e

# go t h r o u g h t h e h i e r a r c h i c a l s u b s p a c e s o f s e l f . l e v e l

16
def l o o p H i e r S p a c e s ( s e l f ) :
for i in range ( 1 , s e l f . l e v e l +1):
s e l f . hSpace = [ i ]
s e l f . l o o p H i e r S p a c e s R e c ( s e l f . dim 1, s e l f . l e v e l ( i 1))

def l o o p H i e r S p a c e s R e c ( s e l f , dim , l e v e l ) :
i f dim > 1 :
for i in range ( 1 , l e v e l +1):
s e l f . hSpace . append ( i )
s e l f . l o o p H i e r S p a c e s R e c ( dim 1, l e v e l ( i 1))
s e l f . hSpace . pop ( )
else :
for i in range ( 1 , l e v e l +1):
s e l f . hSpace . append ( i )
s e l f . action ()
s e l f . hSpace . pop ( )

# f i l l s e l f . gP w i t h t h e p o i n t s f o r t h e i n d i c e s g e n e r a t e d b e f o r e h a n d
def g e n e r a t e P o i n t s ( s e l f ) :
s e l f . i n d i c e s = s e l f . g e n e r a t e P o i n t s R e c ( s e l f . dim , s e l f . l e v e l )
for i in range ( l e n ( s e l f . i n d i c e s ) ) :
s e l f . gP [ s e l f . i n d i c e s [ i ] ] = g r i d P o i n t ( s e l f . i n d i c e s [ i ] , s e l f . domain )

def g e n e r a t e P o i n t s R e c ( s e l f , dim , l e v e l , a k t l e v e l=None ) :

basis akt = l i s t ()
i f a k t l e v e l == None :
akt level = 1
for i in range ( 1 , 2 ( a k t l e v e l ) + 1 , 2 ) :
b a s i s a k t . append ( ( a k t l e v e l , i ) )
i f dim == 1 and a k t l e v e l == l e v e l :
return b a s i s a k t
e l i f dim == 1 :
b a s i s a k t += s e l f . g e n e r a t e P o i n t s R e c ( dim , l e v e l , a k t l e v e l +1)
return b a s i s a k t
e l i f a k t l e v e l == l e v e l :
return c r o s s ( b a s i s a k t , \
s e l f . g e n e r a t e P o i n t s R e c ( dim 1, l e v e l a k t l e v e l +1))
else :
return c r o s s ( b a s i s a k t , s e l f . g e n e r a t e P o i n t s R e c ( dim 1 ,\
l e v e l a k t l e v e l +1)) \
+ s e l f . g e n e r a t e P o i n t s R e c ( dim , l e v e l , a k t l e v e l +1)

# c o n v e r s i o n from n o d a l t o h i e r a r c h i c a l b a s i s i n one dimension

def nodal2Hier1D ( s e l f , node , i , j , dim ) :
l e f t = ( i 1, j / 2 )
r i g h t = ( i 1, j /2+1)

17
 while l e f t [1]%2 == 0 and l e f t [ 0 ] > 0 :
l e f t = ( l e f t [0] 1 , l e f t [ 1 ] / 2 )
while r i g h t [1]%2 == 0 and r i g h t [ 0 ] > 0 :
r i g h t = ( r i g h t [0] 1 , r i g h t [ 1 ] / 2 )
i f i s i n s t a n c e ( node [ 0 ] , t u p l e ) :
preCurDim = node [ 0 : dim ]
postCurDim = node [ dim : l e n ( node )+1]
i n d e x = preCurDim + ( ( i , j ) , ) + postCurDim
l e f t = preCurDim + ( l e f t , ) + postCurDim
r i g h t = preCurDim + ( r i g h t , ) + postCurDim
else :
i f dim == 0 :
i n d e x = ( ( i , j ) , ) + ( node , )
l e f t = ( l e f t , ) + ( node , )
r i g h t = ( r i g h t , ) + ( node , )
e l s e : #dim w i l l be 1 o t h e r w i s e i n t h i s c a s e
i n d e x = ( node , ) + ( ( i , j ) , )
l e f t = ( node , ) + ( l e f t , )
r i g h t = ( node , ) + ( r i g h t , )
i f l e f t [ dim ] [ 0 ] == 0 :
i f r i g h t [ dim ] [ 0 ] != 0 :
s e l f . gP [ i n d e x ] . hv = 0 . 5 s e l f . gP [ r i g h t ] . hv
e l i f r i g h t [ dim ] [ 0 ] == 0 :
s e l f . gP [ i n d e x ] . hv = 0 . 5 s e l f . gP [ l e f t ] . hv
else :
s e l f . gP [ i n d e x ] . hv = 0 . 5 ( s e l f . gP [ l e f t ] . hv + s e l f . gP [ r i g h t ] . hv )

# c o n v e r s i o n from n o d a l t o h i e r a r c h i c a l b a s i s
def n o d a l 2 H i e r ( s e l f ) :
for i in range ( l e n ( s e l f . i n d i c e s ) ) :
s e l f . gP [ s e l f . i n d i c e s [ i ] ] . hv = s e l f . gP [ s e l f . i n d i c e s [ i ] ] . f v
for d in range ( 0 , s e l f . dim ) :
for i in range ( s e l f . l e v e l , 0 , 1 ) :
i n d i c e s = s e l f . g e n e r a t e P o i n t s R e c ( s e l f . dim 1, s e l f . l e v e l i +1)
for j in range ( 1 , 2 i + 1 , 2 ) :
for k in range ( l e n ( i n d i c e s ) ) :
s e l f . nodal2Hier1D ( i n d i c e s [ k ] , i , j , d )

# compute c r o s s p r o d u c t o f a r g s
def c r o s s ( a r g s ) :
ans = [ [ ] ]
for arg in a r g s :
ans = [ x+[y ] for x in ans for y in arg ]
ans2 = [ ]
for i in range ( l e n ( ans ) ) :
i f i s i n s t a n c e ( ans [ i ] [ 1 ] [ 0 ] , t u p l e ) :

18
 dummy = ( ans [ i ] [ 1 ] [ 0 ] , )
for j in range ( 1 , l e n ( ans [ i ] [ 1 ] ) ) :
dummy = (dummy [ 0 : l e n (dummy) ] ) + ( ans [ i ] [ 1 ] [ j ] , )
ans2 . append ( ( ans [ i ] [ 0 ] , ) + dummy)
else :
ans2 . append ( ( ans [ i ] [ 0 ] , ans [ i ] [ 1 ] ) )
return ans2

# e v a l u a t i o n o f t h e b a s i s f u n c t i o n s i n one dimension
def e v a l B a s i s 1 D ( x , b a s i s , i n t e r v a l=None ) :
i f i n t e r v a l i s None :
return 1 . abs ( x 2 b a s i s [0] b a s i s [ 1 ] )
else :
pos = ( xi n t e r v a l [ 0 ] ) / ( i n t e r v a l [1] i n t e r v a l [ 0 ] )
return 1 . abs ( pos 2 b a s i s [0] b a s i s [ 1 ] )

Listing 2: unit test for listing 1

# define the unit t e s t s
import pysg
import u n i t t e s t
import math
c l a s s t e s t F u n c t e s t ( u n i t t e s t . TestCase ) :
def testSGNoBound ( s e l f ) :
sg = pysg . s p a r s e G r i d ( 3 , 3 )
sg . g e n e r a t e P o i n t s ( )
s e l f . a s s e r t E q u a l ( l e n ( sg . i n d i c e s ) , 3 1 )
for i in range ( l e n ( sg . i n d i c e s ) ) :
sum = 1 . 0
pos = sg . gP [ sg . i n d i c e s [ i ] ] . pos
for j in range ( l e n ( pos ) ) :
sum = 4 . pos [ j ] ( 1 . 0 pos [ j ] )
sg . gP [ sg . i n d i c e s [ i ] ] . f v = sum
sg . n o d a l 2 H i e r ( )
for i in range ( l e n ( sg . i n d i c e s ) ) :
s e l f . a s s e r t E q u a l ( sg . gP [ sg . i n d i c e s [ i ] ] . fv , \
sg . e v a l F u n c t ( sg . gP [ sg . i n d i c e s [ i ] ] . pos ) )

# testing
if n a m e == m a i n :
u n i t t e s t . main ( )

References
[Ach03] Achatz, S. Higher Order Sparse Grid Methods for Elliptic Par-
tial Differential Equations with Variable Coefficients. Computing,

19
 71(1):115, 2003.
[Bab60] Babenko, KI. Approximation of periodic functions of many vari-
ables by trigonometric polynomials. Dokl. Akad. Nauk SSSR,
132:247250, 1960. Russian, Engl. Transl.: Soviet Math. Dokl. 1:513
516, 1960.
[Bal94] Balder, R. Adaptive Verfahren fur elliptische und parabolische Dif-
ferentialgleichungen auf d
unnen Gittern. Dissertation, Technische
Universitat Munchen, 1994.
[Bas85] Baszenski, G. N th order polynomial spline blending. In Schempp,
W and Zeller, K, editors, Multivariate Approximation Theory III,
ISNM 75, pages 3546. Birkhauser, Basel, 1985.
[BD03] Bungartz, HJ and Dirnstorfer, S. Multivariate Quadrature on
Adaptive Sparse Grids. Computing, 71:89114, August 2003.
[BDJ92] Baszenski, G, Delvos, FJ, and Jester, S. Blending approxima-
tions with sine functions. In Braess, D, editor, Numerical Methods
in Approximation Theory IX, ISNM 105, pages 119. Birkh auser,
Basel, 1992.
[BG99] Bungartz, HJ and Griebel, M. A Note on the Complexity of
Solving Poissons Equation for Spaces of Bounded Mixed Derivatives.
J. Complexity, 15:167199, 1999. Also as Report No 524, SFB 256,
Univ. Bonn, 1997.
[BG04] Bungartz, HJ and Griebel, M. Sparse Grids. Acta Numerica,
13:147269, 2004.
[BGR94] Bungartz, HJ, Griebel, M, and Ru de, U. Extrapolation, com-
bination, and sparse grid techniques for elliptic boundary value prob-
lems. Comput. Methods Appl. Mech. Eng., 116:243252, 1994. Also
in C. Bernardi and Y. Maday, Editoren, International conference on
spectral and high order methods, ICOSAHOM 92. Elsevier, 1992.
[BGRZ94] Bungartz, HJ, Griebel, M, Ro schke, D, and Zenger, C.
Pointwise convergence of the combination technique for the Laplace
equation. East-West J. Numer. Math., 2:2145, 1994.
[Bun92] Bungartz, HJ. D unne Gitter und deren Anwendung bei der adap-
tiven Losung der dreidimensionalen Poisson-Gleichung. Disserta-
tion, Institut f
ur Informatik, Technische Universit
at M
unchen, 1992.
[Bun98] Bungartz, HJ. Finite Elements of Higher Order on Sparse
Grids. Habilitation, Institut f
ur Informatik, Technische Universit
at
M
unchen and Shaker Verlag, Aachen, 1998.

20
[Del82] Delvos, FJ. d-Variate Boolean Interpolation. J. Approx. Theory,
34:99114, 1982.
[DS89] Delvos, FJ and Schempp, W. Boolean Methods in Interpolation
and Approximation. Pitman Research Notes in Mathematics Series
230. Longman Scientific & Technical, Harlow, 1989.
[Fab09]
Faber, G. Uber stetige Funktionen. Mathematische Annalen, 66:81
94, 1909.
[FHP96] Frank, K, Heinrich, S, and Pereverzev, S. Information
Complexity of Multivariate Fredholm Integral Equations in Sobolev
Classes. J. of Complexity, 12:1734, 1996.
[Fri91] Friedman, JH. Multivariate adaptive regression splines. Ann.
Statist., 19(1):1141, 1991.
[Gar98] Garcke, J. Berechnung von Eigenwerten der station aren
Schrodingergleichung mit der Kombinationstechnik. Diplomarbeit,
Institut f
ur Angewandte Mathematik, Universitat Bonn, 1998.
[Gar04] Garcke, J. Maschinelles Lernen durch Funktionsrekonstruktion mit
verallgemeinerten d
unnen Gittern. Doktorarbeit, Institut f
ur Nu-
merische Simulation, Universit
at Bonn, 2004.
[GG98] Gerstner, T and Griebel, M. Numerical Integration using
Sparse Grids. Numer. Algorithms, 18:209232, 1998.
[GG00] Garcke, J and Griebel, M. On the computation of the eigenprob-
lems of hydrogen and helium in strong magnetic and electric fields
with the sparse grid combination technique. Journal of Computa-
tional Physics, 165(2):694716, 2000.
[GG02] Garcke, J and Griebel, M. Classification with sparse grids using
simplicial basis functions. Intelligent Data Analysis, 6(6):483502,
2002. (shortened version appeared in KDD 2001, Proc. of the Seventh
ACM SIGKDD, F. Provost and R. Srikant (eds.), pages 87-96, ACM,
2001).
[GG03] Gerstner, T and Griebel, M. DimensionAdaptive Tensor
Product Quadrature. Computing, 71(1):6587, 2003.
[GG05] Garcke, J and Griebel, M. Semi-supervised learning with sparse
grids. In Amini, MR, Chapelle, O, and Ghani, R, editors, Pro-
ceedings of ICML, Workshop on Learning with Partially Classified
Training Data, pages 1928. 2005.
[GGT01] Garcke, J, Griebel, M, and Thess, M. Data mining with sparse
grids. Computing, 67(3):225253, 2001.

21
[GH95] Griebel, M and Huber, W. Turbulence simulation on sparse
grids using the combination method. In Satofuka, N, Periaux, J,
and Ecer, A, editors, Parallel Computational Fluid Dynamics, New
Algorithms and Applications, pages 7584. North-Holland, Elsevier,
1995.
[GH03] Gradinaru, V and Hiptmair, R. Multigrid for discrete differen-
tial forms on sparse grids. Computing, 71(1):1742, 2003.
[GHSZ93] Griebel, M, Huber, W, Sto rtkuhl, T, and Zenger, C. On
the parallel solution of 3D PDEs on a network of workstations and
on vector computers. In Bode, A and Cin, MD, editors, Parallel
Computer Architectures: Theory, Hardware, Software, Applications,
volume 732 of Lecture Notes in Computer Science, pages 276291.
Springer Verlag, 1993.
[GK00] Griebel, M and Knapek, S. Optimized tensor-product approxi-
mation spaces. Constructive Approximation, 16(4):525540, 2000.
[GK03] Griebel, M and Koster, F. Multiscale Methods for the Simula-
tion of Turbulent Flows. In Hirschel, E, editor, Numerical Flow
Simulation III, volume 82 of Notes on Numerical Fluid Mechanics
and Multidisciplinary Design, pages 203214. Springer-Verlag, 2003.
[GOS99] Griebel, M, Oswald, P, and Schiekofer, T. Sparse Grids for
Boundary Integral Equations. Numer. Mathematik, 83(2):279312,
1999.
[Gri91] Griebel, M. A parallelizable and vectorizable multi-level algorithm
on sparse grids. In Hackbusch, W, editor, Parallel Algorithms for
partial differential equations, Notes on Numerical Fluid Mechanics,
volume 31, pages 94100. Vieweg Verlag, Braunschweig, 1991. Also
as SFB Bericht, 342/20/90 A, Institut f ur Informatik, TU M
unchen,
1990.

[Gri98] Griebel, M. Adaptive Sparse Grid Multilevel Methods for elliptic

PDEs based on finite differences. Computing, 61(2):151179, 1998.
[GSZ92] Griebel, M, Schneider, M, and Zenger, C. A combination
technique for the solution of sparse grid problems. In de Groen,
P and Beauwens, R, editors, Iterative Methods in Linear Algebra,
pages 263281. IMACS, Elsevier, North Holland, 1992.
[GT95] Griebel, M and Thurner, V. The efficient solution of fluid dy-
namics problems by the combination technique. Int. J. Num. Meth.
for Heat and Fluid Flow, 5(3):251269, 1995. Also as SFB Bericht
342/1/93 A, Institut f
ur Informatik. TU M
unchen, 1993.

22
[HBS+ 06] Hegland, M, Burden, C, Santoso, L, MacNamara, S, and
Booth, H. A solver for the stochastic master equation applied to
gene regulatory networks. Journal of Computational and Applied
Mathematics, In Press, Corrected Proof, 2006.
[Hem95] Hemker, P. Sparse-Grid finite-volume multigrid for 3D-problems.
Advances in Computational Mathematics, 4:83110, 1995.
[Heu97] Heuer, N. Berechnung einer D unngitterl
osung der dreidimen-
sionalen Lame-Gleichung mit der Kombinationsmethode. Diplomar-
beit, Institut f
ur Angewandte Mathematik, Rheinische Friedrich-
Wilhelms-Universitat Bonn, 1997.
[HKZ00] Hochmuth, R, Knapek, S, and Zumbusch, G. Tensor prod-
ucts of Sobolev spaces and applications. submitted, 2000. Also as
Technical Report 685, SFB 256, Univ. Bonn.
[Hoc99] Hochmuth, R. Wavelet Bases in numerical Analysis and Restric-
tal Nonlinear Approximation. Habilitation, Freie Universit
at Berlin,
1999.
[Hub96] Huber, W. Turbelenzsimulation mit der Kombinationsmethode auf
Workstation-Netzen und Parallelrechnern. Dissertation, Institut f
ur
Informatik, Technische Universit
at M
unchen, 1996.
[Kli06] Klimke, A. Sparse Grid Interpolation Toolbox Users Guide. Tech-
nical Report IANS report 2006/001, University of Stuttgart, 2006.
[Kna00] Knapek, S. Approximation und Kompression mit Tensorprodukt-
Multiskalenr
aumen. Doktorarbeit, Universit
at Bonn, April 2000.
[Kos02] Koster, F. Multiskalen-basierte Finite Differenzen Verfahren auf
adaptiven dnnen Gittern. Doktorarbeit, Universit
at Bonn, January
2002.
[Kra02] Kranz, CJ. Untersuchungen zur Kombinationstechnik bei der nu-
merischen Str omungssimulation auf versetzten Gittern. Dissertation,
Institut f
ur Informatik, Technische Universit
at Munchen, 2002.
[KW05a] Klimke, A and Wohlmuth, B. Algorithm 847: Spinterp: piece-
wise multilinear hierarchical sparse grid interpolation in MATLAB.
ACM Trans. Math. Softw., 31(4):561579, 2005.
[KW05b] Klimke, A and Wohlmuth, B. Computing expensive multivari-
ate functions of fuzzy numbers using sparse grids. Fuzzy Sets and
Systems, 154(3):432453, 2005.
[LLS95] Liem, CB, Lu, T, and Shih, TM. The Splitting Extrapolation
Method. World Scientific, Singapore, 1995.

23
[LNSH05] Laffan, SW, Nielsen, OM, Silcock, H, and Hegland, M.
Sparse Grids: a new predictive modelling method for the analysis of
geographic data. International Journal of Geographical Information
Science, 19(3):267292, 2005.
[NH00] Noordmans, J and Hemker, P. Application of an Adaptive Sparse
Grid Technique to a Model Singular Perturbation Problem. Comput-
ing, 65:357378, 2000.
[PZ99] Pflaum, C and Zhou, A. Error analysis of the combination tech-
nique. Numer. Math., 84(2):327350, 1999.
[Rei04] Reisinger, C. Numerische Methoden f ur hochdimensionale
parabolische Gleichungen am Beispiel von Optionspreisaufgaben.
Ph.D. thesis, Ruprecht-Karls-Universit
at Heidelberg, 2004. In Vor-
bereitung.
[Sch99] Schiekofer, T. Die Methode der Finiten Differenzen auf d unnen
Gittern zur Losung elliptischer und parabolischer partieller Differen-
tialgleichungen. Doktorarbeit, Universitat Bonn, 1999.
[Smo63] Smolyak, SA. Quadrature and interpolation formulas for Tensor
Products of Certain Classes of Functions. Dokl. Akad. Nauk SSSR,
148:10421043, 1963. Russian, Engl. Transl.: Soviet Math. Dokl.
4:240243, 1963.
[SS99] Sickel, W and Sprengel, F. Interpolation on Sparse Grids and
NikolskijBesov spaces of dominating mixed smoothness. J. Comput.
Anal. Appl., 1:263288, 1999.
[ST03a] Schwab, C and Todor, R. Sparse finite elements for stochastic
elliptic problems higher order moments. Computing, 71(1):4363,
2003.
[ST03b] Schwab, C and Todor, RA. Sparse finite elements for ellip-
tic problems with stochastic loading. Numer. Math., 95(4):707734,
2003.
[Tem89] Temlyakov, VN. Approximation of functions with bounded mixed
derivative. Proc. Steklov Inst. Math., 1, 1989.
[Tem93a] Temlyakov, VN. Approximation of Periodic Functions. Nova Sci-
ence, New York, 1993.
[Tem93b] Temlyakov, VN. On approximate recovery of functions with
bounded mixed derivative. J. Complexity, 9:4159, 1993.
[vPS04] von Petersdorff, T and Schwab, C. Numerical Solution of
Parabolic Equations in High Dimensions. Mathematical Modeling
and Numerical Analysis, 2004. To appear.

24
[Wah90] Wahba, G. Spline models for observational data, volume 59 of Series
in Applied Mathematics. SIAM, Philadelphia, 1990.
[Yse86] Yserentant, H. On the Multi-Level Splitting of Finite Element
Spaces. Numerische Mathematik, 49:379412, 1986.
[Yse92] Yserentant, H. Hierarchical bases. In R. E. OMalley, J et al.,
editors, Proc. ICIAM91. SIAM, Philadelphia, 1992.
[Zen91] Zenger, C. Sparse Grids. In Hackbusch, W, editor, Parallel Al-
gorithms for Partial Differential Equations, Proceedings of the Sixth
GAMM-Seminar, Kiel, 1990, volume 31 of Notes on Num. Fluid
Mech., pages 241251. Vieweg-Verlag, 1991.

25