Automatic code generator for higher order integrators

Asif Mushtaqa , Kåre Olaussenb

a Institutt for matematiske fag, NTNU, N-7491 TRONDHEIM, Norway
b Institutt for fysikk, NTNU, N-7491 TRONDHEIM, Norway

Abstract
Some explicit algorithms for higher order symplectic integration of a large class of
Hamilton’s equations have recently been discussed by Mushtaq et. al. Here we present
a Python program for automatic numerical implementation of these algorithms for a
given Hamiltonian, both for double precision and multiprecision computations. We
provide examples of how to use this program, and illustrate behaviour of both the code
generator and the generated solver module(s).
Keywords: Splitting methods, Modified integrators, Higher order methods,
Automatic code generation
2010 MSC: 33F05, 37M15, 46N40, 65P10, 74S30

PROGRAM SUMMARY
Manuscript Title: Automatic code generator for higher order integrators.
Authors: Asif Mushtaq, Kåre Olaussen.
Program Title: HOMsPy: Higher Order (Symplectic) Methods in Python
Journal Reference:
Catalogue identifier:
Licensing provisions: None.
Programming language: Python 2.7.
Computer: PC’s or higher performance computers.
Operating system: Linux, MacOS, MSWindows.
RAM: Kilobytes to a several gigabytes (problem dependent).
Number of processors used: 1
Keywords: Splitting methods, Modified integrators, Higher order methods, Automatic code
generation.
Classification: 4.3 Differential equations, 5 Computer Algebra.
External routines/libraries: SymPy library [1] for generating the code. NumPy library [2], and
optionally mpmath [3] library for running the generated code. The matplotlib [4] library for
plotting results.
Nature of problem: We have developed algorithms [5] for numerical solution of Hamilton’s

Email addresses: [email protected] (Asif Mushtaq), [email protected] (Kåre

Olaussen)

Preprint submitted to Computer Physics Communications October 4, 2013

equations,

∂ H(q, p) ∂ H(q, p)
q̇a = , ṗa = − , a = 1, . . . , N (1)
∂ pa ∂ qa
for Hamiltonians of the form
1
H(q, p) = T (p) +V (q) = pT Mp +V (q), (2)
2
with M a symmetric positive definite matrix. The algorithms preserve the symplectic property
of the time evolution exactly, and are of orders τ N (for 2 ≤ N ≤ 8) in the timestep τ. Although
explict, the algorithms are time-consuming and error-prone to implement numerically by hand,
in particular for larger N.
Solution method: We use computer algebra to perform all analytic calculations required for a
specific model, and to generate the Python code for numerical solution of this model, including
example programs using that code.
Restrictions: In our implementation the mass matrix is assumed to be equal to the unit matrix,
and V (q) must be sufficiently differentiable.
Running time: Subseconds to eons (problem dependent). See discussion in the main article.

References
[1] SymPy Developement Team, http://sympy.org/

[2] NumPy Developers, http://numpy.org/

[3] Fredrik Johansson et. al., Python library for arbitrary–precision floating-point arithmetic,
http://code.google.code/p/mpmath/ (2010)

[4] J.D. Hunter, Matplotlib: A 2D graphics environment, Computing in Science & Engineering
9, 90–95 (2007)

[5] A. Mushtaq, A. Kværnø, K. Olaussen, Higher order Geometric Integrators for a

class of Hamiltonian systems, International Journal of Geometric Methods in Modern
Physics, vol 11, no. 1 (2014), 1450009-1–1450009-20. DOI: 10.1142/S0219887814500091.
arXiv.org:1301.7736

1. Introduction

The Hamilton equations of motion (1) play an important role in physics and math-
ematics. They often require numerical methods for solution [1, 2, 3]. A well-behaved
class of such methods are the symplectic solvers, which preserve symplecticity of the
time evolution exactly. One simple way to construct a symplectic solver is to split the
time evolutions into kicks,
∂V (q)
q̇a = 0, ṗa = − , (3)
∂ qa

2
which is straightforward to integrate to give

qa (t + τ) = qa (t), (4)
∂V (q(t), p(t))
pa (t + τ) = pa (t) − τ , (5)
∂ qa
followed by moves,
∂ T (p)
q̇a = = ∑ M ab pb , ṗa = 0, (6)
∂ pa b

which integrates to

qa (t + τ) = qa (t) + τ ∑ M ab pb (t + τ),
b
∂V (q(t), p(t))
pa (t + τ) = pa (t) − τ .
∂ qa
This scheme was already introduced by Newton [4] (as more accessible explained by
Feynman [5]). A symmetric scheme is to make a kick of size 12 τ, a move of size τ,
and a kick of size 21 τ (and repeating). This is often referred to as the Störmer-Verlet
method [6, 7]; it has a local error of order τ 3 . The solution provided by this method
can be viewed as the exact solution of a slightly different Hamiltonian system, with
a Hamiltonian HSV which differ from (2) by an amount proportional to τ 2 . For this
reason the scheme respects long-time conservation of energy to order τ 2 . It will also
exactly preserve conservation laws due to Nöther symmetries which are common to
T (p) = 12 pT M p and V (q), like momentum and angular momentum which are often
preserved in physical models [8].
Recently Mushtaq et. al. [9, 10] proposed some higher order extensions of the Stör-
mer-Verlet scheme. These extensions are also based on the kick-move-kick idea, only
with modified Hamiltonians,
1
H1 ≡ Teff = pT Mp + ∑ T2k (q, p), (7a)
2 k≥1
H2 ≡ Veff = V (q) + ∑ V2k (q), (7b)
k≥1

where T2k and V2k are proportional to τ 2k . I.e., the proposal is to replace V (q) in
equation (3) by Veff (q), and T (p) in equation (6) by Teff (q, p). The goal is to construct
Veff and Teff such that the combined kick-move-kick process corresponds to an evolution
by a Hamiltonian Heff which lies closer to the Hamiltonian H of equation (2). The
difference being of order τ 2N+2 when summing terms to k = N in equations (7).
One problem with this approach is that Teff in general will depend on both q and
p; hence the move-steps of equation (6) can no longer be integrated explicitly. To
overcome this problem we introduce a generating function [3]

G(q, P ; τ) = ∑ Gk (q, P ) τ k (8)

0≤k≤N

3
such that the transformation (q, p) → (Q, P ) defined by
∂G
pa = , (9a)
∂ qa
∂G
Qa = , (9b)
∂ Pa
preserves the symplectic structure exactly, and reproduce the time evolution generated
by Teff to order τ N . Here Qa is shorthand for qa (t + τ), and Pa shorthand for pa (t + τ).
Equation (9a) is implicit and in general nonlinear, but the nonlinearity is of order τ 3
(hence small for practical values of τ). In the numerical code we solve (9a) by straight-
forward iteration (typically two to four iterations in the cases we have investigated).
The rest of this paper is organized as follows: In section 2 we introduce compact
notation in which we present the general explicit expressions for Teff (q, p), Veff (q),
and G(q, Q). Because of their compactness these expressions are straightforward to
implement in SymPy.
In section 3 we provide examples of how to use the code generator on specific
problems. This process proceeds through two stages: (i) By providing the potential
V (q) (possibly depending on extra parameters) code for solving the resulting Hamil-
ton’s equations (the solver module) is generated, and (ii) this solver module is used to
analyse the model. The last stage must of course be implemented by the user, but an
example program which explicitly demonstrates how the solver module can be used is
also generated during the first stage.
The examples given in subsections 3.1 (Vibrating beam) and 3.2 (One-parameter
family of quartic anharmonic oscillators) have known exact solutions; this makes it
easy to check whether the algorithms behave like expected with respect to accuracy.
The example in subsection 3.3 (two-dimensional pendulum) demonstrates that the pro-
gram can handle nonpolynomial functions, and that it generates code which preserves
angular momentum exactly.
In section 4 we consider a collection of many coupled quartic oscillators. This
is intended as a stress-test of the code, investigating how it behaves with respect to
precision as well as time and memory use for larger and more complex models. Test
of both of the code-generating process, and of the solver modules generated. We find
that the latter continue to behave as expected with respect to accuracy, but that there
is room for improvement in the area of time and memory efficiency, in particular for
large structured systems.
In section 5 we present the organization of our code generating program itself,
including diagrammatic representations of its structure. We finally summarize our ex-
periences in section 6.

2. Explicit expressions
Compact explicit expressions for the terms of order τ N for N ∈ {2, 4, 6} in equations
(7) were given in [9, 10]. With the notation
∂
∂a ≡ , ∂ a ≡ M ab ∂b , pa ≡ M ab pb , D ≡ pa ∂ a , D̄ ≡ (∂aV )∂ a , (10)
∂ qa

4
where the Einstein summation convention1 is employed, they are
1 2 2
T2 = − D Vτ , (11a)
12
1
D4 − 9D̄D2 + 3DD̄ V τ 4 ,


T4 = (11b)
720
1
T6 = − 2 D6 − 40 D̄D4 + 46 DD̄D3 − 15 D2 D̄D2
60480
+ 54 D̄2 D2 − 9 D̄DD̄D − 42 DD̄2 D + 12 D2 D̄2 V τ 6


(11c)
1
V2 = D̄V τ 2 , (11d)
24
1 2 4
V4 = D̄ V τ , (11e)
480
1
17 D̄3 − 10 D̄3 V τ 6 .


V6 = (11f)
161280
In the last line we have introduced

D̄3 ≡ (∂aV )(∂bV )(∂cV )∂ a ∂ b ∂ c . (12)

By introducing the operator D = Pa ∂a the explicit expressions for the coefficients Gk

can be written

G0 = qa Pa , (13a)
1 a
G1 = 2 P Pa , (13b)
G2 = 0, (13c)
1
G3 = − 12 D 2V, (13d)
1
G4 = − 24 D 3V, (13e)
1
3 D 4 + 3 D̄D 2 − D D̄D V,


G5 = − 240 (13f)
 
1
G6 = − 720 2 D 5 + 8 D̄D 3 − 5 D D̄D 2 V, (13g)

1
G7 = − 20160 10 D 6 + 10 D̄D 4 + 90 D D̄D 3 − 75 D 2 D̄D 2

+ 18 D̄2 D 2 − 3 D̄D D̄D − 14 D D̄2 D + 4 D 2 D̄2 V, (13h)

1
G8 = − 40320 3 D 7 − 87 D̄D 5 + 231 D D̄D 4 − 133 D 2 D̄D 3 + 63 D̄2 D 3
− 3 D D̄2 D 2 − 21 D 2 D̄2 D + 4 D 3 D̄2 − 63 D̄D D̄D 2 (13i)

+ 25 D D̄D D̄D V.

1 An index which occur twice, once in lower position and once in upper position, are implicitly summed

over all available values. I.e, M ab ∂b ≡ ∑b M ab ∂b (we generally use the matrix M to rise an index from lower
to upper position).

5
 The equations (11,13), when used in equations (3,9), define the kick-move-kick scheme
for a general potential V . If one uses all the listed terms the local error becomes of order
τ 9 , and the scheme will respect long-time conservation of energy to order τ 8 .
However, explicit implementation of the numerical code for a specific potential is
rather laborious and error-prone to do by hand, since the repeated differentiations (with
respect to many variables) and multiplications by lengthy expressions are usually in-
volved. We have therefore written a code-generating program in SymPy, wich takes a
given potential V as input, perform all the necessary algebra symbolically, and auto-
matically constructs a Python module for solving one full kick-move-kick timestep. It
also writes a Python program example using the module; this example may serve as a
template for larger applications.

3. Examples of code generation

The submitted code includes a file makeExamples.py, with various examples

which demonstrate how the code generator can be used. We go through these examples
in this section; they also provide some illustrations of how the integrators perform in
practical use.

3.1. Vibrating beam

A simple model for a vibrating beam is defined by the Hamiltonian
1 2 1 2 1 4
H= p − q + q . (14)
2 2 4
A Python code snippet which generates a numerical solver for this problem is the fol-
lowing:
Creating a module for solving a vibrating beam
1 def makeVibratingBeam():
2 # Choose names for coordinate and momentum
3 q, p = sympy.symbols([’q’, ’p’])
4 qvars = [q]; pvars = [p]
5 # Define the potential in terms of the coordinate
6 V = -q*q/2 + q**4/4
7 # Create code for a double-precision solver
8 kimoki.makeModules(’VibratingBeam’, V, qvars, pvars)
Line 8 generates the files VibratingBeam.py and runVibratingBeam.py. The file
VibratingBeam.py contains the general double precision solver module for this prob-
lem. A simple use of it in an interactive session is illustrated below:
Interactive use of the solver module
1 >>> from __future__ import division
2 >>> from VibratingBeam import *
3 >>> z = numpy.array([1/2, 5/4])
4 >>> kiMoKi(z); print z

6
5 [ 0.62690658 1.28822851]
6 >>> kiMoKi(z); print z
7 [ 0.75756578 1.32399846]
Here z = [q, p] is a NumPy array containing the current state of the solution. Each
call of kiMoKi updates this state (data from previous timesteps are not kept).
The file runVibratingBeam.py is a small example program demonstrating basic
use of VibratingBeam.py. A code snippet illustrating some essential steps is:
Time evolution of a vibrating beam
1 # Select your preferred number of timesteps, order and timestep
2 nMax = 161
3 VibratingBeam.order = 8; VibratingBeam.tau = 1/10
4 z = numpy.array([1/2, 5/4]) # Initial condition
5 for n in xrange(nMax):
6 VibratingBeam.kiMoKi(z) # Integrate one full timestep
The equation is integrated in line 6. The complete code in runVibratingBeam.py is
an extension of this snippet. The initial condition is generated at random, and saved
for possible reuse. Also the complete solution is saved to a (temporary) file for further
processing, together with the parameters tau, order, and nMax. By running the file
runVibratingBeam.py a single solution is first generated and afterwards displayed
in a plot. The plot is also saved in the file VibratingBeam_Soln.png. This plot will
look similar to Figure 1.

Numerical solution for timestep 1

10 and method of order 8
2.0
z0
1.5 z1

1.0

0.5
Solution

0.0

−0.5

−1.0

−1.5

−2.0
0 2 4 6 8 10 12 14 16
Time

Figure 1: Example solution of a vibrating beam with random initial conditions. Here
z0 denotes the position q, and z1 the momentum p.

To give some impression of the quality of the generated solution, and how this
depends on the timestep and order of the integrator, the example runfile also make a set

7
of runs with the same initial condition, but various values of tau, order, and nMax. A
simple quality measure, which is straightforward to implement in general, is how well
the initial energy is preserved as time increases. This quantity is plotted, with the plot
first displayed and next saved in the file VibratingBeam_EgyErr.png. The plot will
look similar to Figure 2. If one prefers to save the plots as .pdf-files the code
#import mathplotlib; matplotlib.use(’PDF’) # Uncomment to ...
on line 17 of the example runfile must be uncommented (then the plot will most likely
not be displayed on screen). During the run process solution data is saved to several
.pkl-files; these are normally removed after the data has been plotted. To keep this
data the code
os.remove(filename) # Comment out to keep datafile
on lines 108 and/or 202 of the example runfile must be commented out.

Approximate energy conservation

2nd order 4th order
0.6 0.8
1 1
0.4 τ= 5 τ= 5
0.6
(E(t) − E(0))/τ 2

(E(t) − E(0))/τ 4
0.2 1 1
τ= 10 τ= 10
0.0 0.4
1 1
τ= 20 τ= 20
−0.2 0.2
−0.4
0.0
−0.6
−0.8 −0.2
0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
6th order 8th order
1.4 0.4
1 1
1.2 τ= 5 0.2 τ= 5
1.0
(E(t) − E(0))/τ 6

(E(t) − E(0))/τ 8

1 0.0 1
τ= 10 τ= 10
0.8
1 −0.2 1
0.6 τ= 20 τ= 20
−0.4
0.4
−0.6
0.2
0.0 −0.8
−0.2 −1.0
0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
Time t Time t

Figure 2: Example of how well energy is conserved for integrators of various orders N,
and how the energy errors scale as expected with the timestep τ. The error decreases
like τ N when τ decreases. In this case, with a periodic solution, there is also a periodic
variation in the energy error.

As illustrated by Figure 2, and verified by all other cases we have investigated, the
energy error scales like τ N , where τ is the timestep and N is the order of the integrator.

8
 N τ = 0.2 τ = 0.1 τ = 0.05
2 0.26 0.26 0.26
4 1.03 0.84 0.73
6 1.23 1.01 0.88
8 1.54 1.25 1.09

Table 1: Evaluation time per integration step in milliseconds, dependent on timestep τ and method order N.
The higher order methods (N > 2) run faster for smaller τ because the iterative solution of (9a) requires fewer
iterations, hence becomes faster. This example, like all others, have been run on a workstation equipped with
two four-core Intel Xeon E5520 processors. The ratio between numbers like those above are probably more
relevant that their absolute values.

The energy error does not grow with time, but varies in a periodic manner — following
the periodicity of the generated solution. Note that the integration module, in this case
VibratingBeam, contains a parameter epsilon which governs how accurate (9a) is
solved. We have observed a systematic growth in the energy error when this parameter
is chosen too large, thereby violating symplecticity (too much).
Another property of interest and importance is how the average time per integration
step varies with the order of the method. The run example prints a measure of the
computer time used. The results of this, for a longer run than the unmodified run
example, are shown in Table 1. As can be seen, the penalty of using a higher order
method is quite modest for a simple model like the vibrating beam, in particular when
the timestep τ is small.
The exact solution of this problem can be expressed in terms of Jacobi elliptic
functions, cf. equations (16) below. This allows direct comparison between the exact
and the numerical solutions, as shown in Figure 3. The initial condition is chosen such
that the energy is close to the critical energy, E = 0, where the solutions bifurcates from
motion over the potential hill at q = 0 to motion in only one of the two potential wells.
The exact solution moves over the potential hill. As can be seen, this is respected by
the solutions of order N = 6 and 8, but not by the solutions of order N = 2 (Störmer-
Verlet) and 4. This demonstrates that there may be cases where a higher order method,
or an impractically small stepsize τ, is required to obtain even the qualitatively correct
solution.
Up to the time t we have computed and plotted the solutions, the order N = 8
numerical solution cannot be visually distinguished from the exact one in this plot. A
difference would become visible for sufficiently large t, because the two solutions have
slighly different periods. The difference in periods is of order τ 8 for N = 8.

3.2. One-parameter family of quartic anharmonic oscillators

A generalization of the previous problem is the class of non-linear oscillators de-
fined by the Hamiltonian
1 1 1
H = p2 + α q2 + q4 . (15)
2 2 4

9
 Comparison of exact and numerical solutions
2nd order 4th order
1.5 1.5

1.0 1.0

0.5 0.5

0.0 0.0

−0.5 −0.5

−1.0 −1.0

−1.5 −1.5
1465 1470 1475 1480 1485 1490 1495 1500 1465 1470 1475 1480 1485 1490 1495 1500
6th order 8th order
1.5 1.5

1.0
qe (t) 1.0
qnum(t)
pe (t) pnum(t)
0.5 0.5

0.0 0.0

−0.5 −0.5

−1.0 −1.0
Time t
−1.5 −1.5
1465 1470 1475 1480 1485 1490 1495 1500 1465 1470 1475 1480 1485 1490 1495 1500

Figure 3: Comparison of the exact elliptic solution (16) with those found √ by our au-
tomatically generated numerical code, for the initial condition q0 = 2 + 10−6 and
p0 = 0, with timestep τ = 10−1 . The thick blue [red] lines show the numerical solu-
tions for q(t) [p(t)], the thin blue [red] lines show the exact solution for q(t) [p(t)].

A one-parameter class of exact solutions to this problem can be expressed in terms of

Jacobi elliptic functions [11],

q(t) = q0 cn(νt|k), (16a)

p(t) = −q0 ν sn(νt|k) dn(νt|k). (16b)

Here the initial conditions are q(0) = q0 , and p(0) = 0. The vibrating beam discussed
in the previous subsection corresponds to the case of α = −1. Equations (16) exhaust
the set of solutions which have a maximum at q0 > 0 (or a minimum at q0 < 0). This
condition imposes the restriction that α + q20 ≥ 0. The parameters and energy of the
solution are

1/2 1 1
ν = α + q20 , k = 2−1/2 q0 /ν, E = α q20 + q40 . (17)
2 4
For α + 21 q20 > 0 the energy is positive, and q(t) oscillates symmetrically around q = 0;
for α + 12 q20 < 0 the energy is negative, and q(t) oscillates in one of the two possible po-
tential wells (depending on the sign of q0 ). For α + q20 < 0 the solution has a minimum

10
 at q0 > 0 (or maximum at q0 < 0).2
A code snippet for generating numerical solvers for this problem is the following
Solving a one-parameter class of anharmonic oscillators
1 def makeAnharmonicOscillator():
2 # Choose names for coordinate, momentum and parameter
3 q, p, alpha = sympy.symbols([’q’, ’p’,’alpha’])
4 qvars = [q]; pvars = [p]; params = [alpha]
5 # Define potential in terms of coordinate and parameters
6 V = alpha*q**2/2 + q**4/4
7 # Code for double-precision and multiprecision computations
8 kimoki.makeModules(’AnharmonicOscillator’, V, qvars, pvars,
9 PARAMS=params, MP=True, VERBOSE=True)
The code in line 8-9 shows that the makeModules function may take optional argu-
ments: If the Hamiltonian depends on a list of parameters, this list must be assigned to
the keyword PARAMS. If the MP keyword is set to True then two additional files are gen-
erated: In this case the files AnharmonicOscillatorMP.py, which is a solver module
using multiprecision arithemetic, and runAnharmonicOscillatorMP.py, which is a
runfile example using this multiprecision solver. When the VERBOSE keyword is set
to True some information from the code generating process will be written to screen,
mainly information about the time used to process the various stages. This may be
of use when the code generation process takes a very long time, as will happen with
complicated models.

2 In which case the solution can be written

q(t) = q0 nd(νt, k), p(t) = νq0 k2 sd(νt, k) cd(νt, k), (18)

1/2 1/2
with ν = −α − q20 /2 and k = ν −1 −α − q20 . In (16) the modulus k > 1 when α + q20 /2 < 0. An


alternative expression, with 0 < k < 1, is
q(t) = q0 dn(νt, k), p(t) = −νq0 sn(νt, k) cn(νt, k), (19)
−1
1/2
= 2−1/2 q0 2

where ν and k = q0 2(α + q0 ) .

11
 Global error (L2 -norm) for integrators of order N
10−2
N =2
10−3 N = 2 (fit)
N =4
10−4 N = 4 (fit)
N =6
10−5
N = 6 (fit)
N =8
10−6
N = 8 (fit)

10−7

10−8

10−9

10−10

10−11
0 5 10 15 20 25 30
Time t

Figure 4: The global error (20) computed for integrators of various orders N, and
1
stepsizes τ = 15 , 10 1
, 20 1
, 40 . To check the τ N -scaling the errors for the last three τ’s
are multiplied by respectively 2N , 4N , 8N . The dashed lines are crude fits to equation
(21) with CN as the fitting parameters. The plot is not untypical, with the N = 8 result
deviating from (21) for the smallest values of τ. This is a consequence of the finite
accuracy of double precision calculations, not a failure of the N = 8 integrator.

To check the accuracy of the numerical solution in more detail, we have modified
runAnharmonicOscillator.py to analyseAnharmonicOscillator.py, where the
global error
ε(t) ≡ k(q(e) (tm ), p(e) (tm )) − (qm (n)
(n) , pm )k (20)
is computed (for random values of α and q0 ), and plotted. Here the sub|super-script (e)
labels the exact solution, and (n) the numerical one. One resulting plot, for parameters
α = 0.13 and q0 = 0.54, is shown in Figure 4. In general, the global error fits well to
the formula, cf. theorem 3.1 in the book [3],

ε(t) ≡ CN τ N t, (21)

where N is the order of the integrator, and CN is independent of τ but depends on the
parameters of the model, the initial conditions, and the norm k · k used in (20). We have
used the L2 norm in Figures 4 and 5.
It can be deduced from Figure 4 that to fully exploit the power of the higher order
integrators one must go beyond double precision accuracy. We have therefore imple-
mented an option (MP=True) for automatically generating multiprecision versions of
the integrators and run examples. The file analyseAnharmonicOscillatorMP.py is
an adaption of runAnharmonicOscillatorMP.py which computes the global error

12
to multiprecision accuracy. The result for N = 8 and various small values of τ (and the
same parameters as in Figure 4) is shown in Figure 5.

Multiprecision calculation of global error (N = 8)

10−11
1
10−12 τ= 20

10−13 τ= 1
20 (fit)
10−14 1
τ= 40
10−15 τ= 1
(fit)
40
−16
10 τ= 1
80
10−17
τ= 1
80 (fit)
10−18 1
τ= 160
10−19
τ= 1
160 (fit)
10−20
1
τ= 320
10−21

10−22 τ= 1
320 (fit)

10−23

10−24

10−25
0 50 100 150 200 250 300
Time t

Figure 5: The global error (20) computed to 35 decimals precision for the N = 8 in-
1 1 1 1 1
tegrator, with stepsizes τ = 20 , 40 , 80 , 160 , and 320 . The dashed lines correspond to
1
equation (21), with C8 fitted crudely to the τ = 20 data.

3.3. Two-dimensional pendulum

As a final example of this section we want to demonstrate that our program can
handle non-polynomial potentials as well. Hence we consider the Hamiltonian for a
slightly distorted3 version of a two-dimensional pendulum
q 
1 2 2

2 2
H= p + p1 − cos q0 + q1 . (22)
2 0

Here both the kinetic and potential energy is invariant under rotations; hence we expect
the generated code to preserve angular momentum,

L(t) = [q0 (t)p1 (t) − q1 (t)p0 (t)] , (23)

exactly.

3 The motion of a real pendulum is constrained to the surface of a sphere, which cannot be described by a

constant mass matrix.

13
 Solving a two-dimensional pendulum
1 def makeTwoDPendulum():
2 # Choose names for coordinate, momentum and parameter
3 q0, q1, p0, p1 = sympy.symbols([’q0’, ’q1’, ’p0’, ’p1’])
4 qvars = [q0, q1]; pvars = [p0, p1]
5 # Define potential in terms of coordinate and parameters
6 V = -cos(sqrt(q0**2+q1**2))
7 # Code for multiprecision computation only
8 kimoki.makeModules(’TwoDPendulum’, V, qvars, pvars, DP=False,
9 MP=True, MAXORDER=6, VERBOSE=True)
Here we demonstrated one additional optional argument of makeModules, MAXORDER,
which can be used the restrict the maximum order of solvers being generated (6 in this
example).

×10−34 Conservation of angular momentum

0.0

−0.2

−0.4

−0.6
L(t) − L(0)

−0.8

−1.0

−1.2

−1.4

−1.6

−1.8
0 5 10 15 20
Time t

Figure 6: This figure demonstrates that our integration algorithm respects conservation
of a angular momentum exactly (within numerical accuracy). Here we have integrated
Hamilton’s equations with a timestep tau = 1/mpf(10) with an order = 6 multi-
precision solver, using 35 decimal digits accuracy. Obviously, this figure only displays
how roundoff errors are accumulated with time.

14
4. Analysis of many anharmonic oscillators
Consider a sum of Hamiltonians like (15),
N  
1 2 1 a 2 1 a 4
H=∑ Pa + αa (Q ) + (Q ) . (24)
a=1 2 2 4
Since the corresponding Hamiltonian equations of motion decouple, the solution for
each pair (Qa , Pa ) is given by expressions like (16). A direct numerical solution of this
model would not provide any additional test of the integrators. However, if we make
an orthogonal coordinate transformation,
N N
Qa = ∑ Ra j q j , Pa = ∑ Ra j p j , (25)
j=1 j=1

we obtain an expression
N
1
H=
2 ∑ p2j +V (q), (26)
j=1
where V (q) looks like a general polynomial potential in N variables with quadratic
and quartic terms. We expect the numerical algoritms to behave like the general case
for this model, while the exact solution is known in the form
j
q(e) = ∑ Ra j Qa(e) , p(e) a (e)
j = ∑ R j Pa . (27)
a a

Time used for code generation

105

Time [secs]
104

103

Total generation time

102 Time to make V 2, . . . , V 6
Time to make G3, . . . , G8

101

100

N
10−1
5 10 15 20

Figure 7: This figure illustrates how the time used for code generation scale with sys-
tem size N for the Hamiltonian (26). When N is large the most time-consuming
individual task seems to be the computation of D̄3V .

15
 We have generated such Hamiltonians, using a random orthogonal matrix with ra-
tional coefficents Ra j = Ra j , for a range of N -values (the way we construct Ra j only
nearest- and next-nearest-neighbor couplings are generated between the variables q j ).
This allows us to investigate how the code generator behave for models of increasingly
size and complexity. As illustrated in Figure 7 the time used to generated the solver
module increases quite rapidly with the number N of variables.
Global L2 error (short time behaviour) Global L2 error
10−32 10−30

10−31
10−33

10−32

10−34
kz (e) (t) − z (n) (t)k2

kz (e) (t) − z (n) (t)k2

10−33

10−35 10−34

10−35
10−36
N =6 N =6
10−36
N =5 N =5
10 −37 N =4 N =4
N =3 10−37 N =3
N =2 N =2
−38
10 10−38
0.00 0.05 0.10 0.15 0.20 0 5 10 15 20
Time t Time t

Figure 8: These figures illustrates how the global error varies with time, system size
N , and initial conditions. The short-time behavior is shown to the left; the long-time
behaviour, for the same model and initial conditions, to the right. We have used a
multiprecision integrator of order 8, with τ = 10−3 and computations to 35 decimals
precision.

In Figure 8 we illustrate how the global error in these models behave. Although
there is a general trend that the accuracy detoriates with system size, this trend is not
strictly followed (as can be seen by the case of N = 3). This is a reflection of the fact
that both the models and their initial conditions are generated with a certain degree of
randomness.

5. Structure of the programs

From a functional point of view our code generating program can be characterized
as a module, hence it should (to our understanding) be organized into a single file.
However, with 1700+ lines of code and comments this would make code development
and maintainance impractical. Hence we have organized it as a package. I.e., as a set
of .py-files, including a file named init .py, in a directory (folder) with the same
name as the module (kimoki). A graphical overview of the main components of this
structure is illustrated in Figure 9.
For a given Hamiltonian this module generates one or two solver modules with
routines for numerical solution of the model, each with a runfile example using the
solver. Each runfile example is intended to demonstrate and check basic properties of
its solver module, and to be modified into more useful programs by the user.
In a separate examples directory there is a file named makeExamples.py, contain-
ing the examples discussed in section 3. By running makeExamples.py the examples

16
directory will, after some time, be populated with several solver modules and runfile
examples. Running the runfile examples will in turn generate many .pkl-files with
numerical data (which are normally deleted after use), and some .png-files with plots
of the solution, and how well the solution respects energy conservation.

5.1. Diagramatic overview of the code generating module

makeModules __init__

writeHeaderDP _MAXORDER

writeHeaderMP _USEHORNER

makeTk _MULTIPREC

writeEnergyDefinition _VERBOSE

writeTkDefinitions _ALLTK

makeVk _START

writeVkDefinitions _COUNTITRS

writeKickDefinitions
helperFunctions
makeGk
pvars2Pvars
writePushDefinitions
diffA
writeMoveDefinitions
diffB
writeVectorizedFunctions
diffC
writeKiMoKi
convert2MP
writeRunExampleDP
writeExpression
writeRunExampleMP

Figure 9: Components of the code generating program. The main process is run in
the makeModules(...) routine; this calls a sequence of subroutines as illustrated in
the left column. This routine takes a number of optional keyword arguments, the val-
ues of some of them are stored in global variables (as listed in the __init__ box).
Some functions, most of them required at several places in the program, are collected
in helperFunctions. All routines listed in the left column, plus __init__ and
helperFunctions are defined in files with the same name plus the extension .py.
These files are in turn collected in the directory (folder) kimoki, the name of the code-
generating module.

5.2. Brief description of the routines

◦ makeModules(modname, V, qvars, pvars[, kwargs])
This is the main subroutine, and the only one intended to by called by the user.
Here modname is a “basename” of the generated files, V is a symbolic expression

17
 for the potential V , qvars is a list of symbolic positions variables (generalized
coordinates), and pvars is a list of symbolic momentum variables (canonically
conjugate momenta). This routine takes a number of optional keyword argu-
ments (kwargs) with defaults:

PARAMS=None is a list of symbolic parameters used in V .

MAXORDER=8 is the maximum order of the generated solvers.
DP=True is a switch which determines if code for normal (double) preci-
sion solvers should be generated.
MP=False is a switch which determines if code for multiprecision solvers
should be generated.
VERBOSE=False is a switch which determines if messages from the code
generation process should be written.
ALLTK=False is a switch wich determines if code for calculating T2 , T4 ,
and T6 is generated.
COUNTITRS=False is a switch which determines if code for monitoring
the iterative solution of (9a) is generated. This code makes a histogram of
the number of iterations used for solutions.

◦ diffA(V, qvars, pvars), diffB(V, V0, qvars), diffC(V, qvars)

These three functions implement respectively the operator D defined in equa-
tion (10), the operator D̄ defined in expressions (10), and the operator D̄3 de-
fined in equation (12). Here V0 is a symbolic expression of the potential defin-
ing the model, while V can be any symbolic expression depending on qvars,
pvars, and potential parameters params. The function diffA is used by the rou-
tines makeTk and makeGk; the function diffB is used by the routines makeTk,
makeVk and makeGk; the function diffC is used by the routine makeVk.

◦ writeExpression(outfile, expression), convert2MP(match)

The function writeExpression writes expression to outfile in python for-
mat. If a multiprecision version of the solver module is generated, the function
convert2MP is used to assure that fractions are converted to multiprecision for-
mat. Optionally a polynomial expression can be converted to Horner form
first (this is not recommended if expression depends on many variables, due
to time and memory use).
◦ writeHeaderDP(outfile, pvars, params),
writeHeaderMP(outfile, pvars, params)
These routines write the header part of respectively the double precision and mul-
tiprecision solver modules. Some important solver module variables are defined
here, with defaults:

tau = 1/10 (DP), tau = 1/mpf(1000) (MP).

The timestep τ used by the solvers.

18
 epsilon = 1/10**12 (DP), epsilon = 1/mpf(10**20) (MP).
The accuracy to which (9a) must be solved. We have observed that the
symplectic preserving property of the solvers is lost when epsilon is too
large, but it must be somewhat larger than the numerical precision used.
order = MAXORDER. Which order of solver to use, setting order larger
than maxorder (see below) has no effect.
params. A list of the symbolic potential parameters; these parameters must
be set before starting a solution.
maxorder = MAXORDER. The maximum order of generated solvers. Must
not be changed by the user.
dim. The number of phase space variables. Must not be changed.
itrs[20]. A histogram of how many iterations are used to solve (9a).
Exists only if COUNTITRS is set to True. Must not be changed by the user.
◦ makeTk(V0, qvars, pvars)
Calculates the contributions T0 , T2 , T4 , and T6 to Teff , cf. (7a), using explicit
expressions in (11). T0 is used by the routine writeEnergyDefinition; T2 , T4 ,
and T6 by the routine writeTkDefinitions. T2 , T4 , and T6 are computed only
when the optiononal parameter ALLTK=True, otherwise they are set to 0.
◦ writeEnergyDefinition(outfile, qvars, pvars, T0, V0)
Writes the definition of the function energy(z), which evaluates the energy
T0 (p) +V0 (q), cf. equation (2), at the phase space point z ≡ (q, p).
◦ writeTkDefinitions(outfile, qvars, pvars, Tk)
Writes the definitions of the functions T2(z), T4(z) and T6(z), using the sym-
bolic expressions in the list Tk calculated by makeTk.
◦ makeVk(V0, qvars)
Calculates the contributions V2 , V4 , and V6 to Veff , cf. equation (7b), using ex-
plicit expressions in equation (11). V2 , V4 , and V6 are used by the routines
writeVkDefinitions and writeKickDefinitions.
◦ writeVkDefinitions(outfile, qvars, Vk)
Writes the definitions of the functions V2(q), V4(q) and V6(q), using the sym-
bolic expressions in the list Vk calculated by makeVk.
◦ writeKickDefinitions(outfile, qvars, Vk)
Calculates the symbolic expressions −∂Veff /∂ qa , cf. equation (3), using sym-
bolic expressions in the list Vk calculated by makeVk. These expressions are
used to define the functions kicka(z) used in the kick-steps of the solvers.
◦ makeGk(V0, qvars, Pvars)
Calculates the contributions G3 , G4 , . . . , G8 to the generation function G(q, P ; τ),
cf. equation (8), using the explicit expressions in equation (13). G3 , G4 , . . . , G8
are used by writePushDefinitions and writeMoveDefinitions.

19
 ◦ writePushDefinitions(outfile, qvars, Pvars, Gk)
Calculates the symbolic expressions ∂ G/∂ qa , cf. equation (9a), using symbolic
expressions in the list Gk calculated by makeGk. These expressions are used to
define the functions pusha(z) used in the push-steps of the solvers.

◦ writeMoveDefinitions(outfile, qvars, Pvars, Gk)

Calculates the symbolic expressions ∂ G/∂ Pa , cf. equation (9b), using symbolic
expressions in the list Gk calculated by makeGk. These expressions are used to
define the functions movea(z) used in the move-steps of the solver(s).
◦ writeVectorizedDefinitions(outfile, qvars, Pvars)
Writes definitions of functions vecKicks(idx, z), vecPushes(idx, z), and
vecMoves(idx, z), simulating parallel evaluation of kicka(z), pusha(z),
and movea(z), for all a in the list idx. We don’t think parallel evaluation is
actually achieved4 ; hence presently this is only syntactic sugar simplifying the
main solver routine, kiMoKi(z).

◦ writeKiMoKi(outfile)
Writes the definition of the main algorithm of the solver module, kiMoKi(z).
The routine kiMoKi(z) processes the kick-push-move-kick substeps of a full
timestep, including the iterative solution of equation (9a).
◦ writeRunExampleDP(outfile, modname, qvars, pvars, params),
writeRunExampleMP(outfile, modname, qvars, pvars, params)
Writes a simple example program illustrating how to use the solver module.
Some routines of this program solves the Hamilton’s equation over a time in-
terval, with random initial conditions and parameters (which most likely must
first be manually changed to sensible values), and writes a plot of the solution to
a pdf file. Other routines check how well energy is conserved by the solver, for
a set of timesteps, and writes a plot of the energy errors to another pdf file.

4 This depends on how the numpy.vectorize function is implemented.

20
 makeExamples

makeVibratingBeam
kimoki DP = True, MP = False, Verbose = False

makeExamples makeAnharmonicOscillator
DP = True, MP = True, Verbose = True,
make<Example> PARAMS = params

makeTwoDPendulum
DP = False, MP = True, Verbose = True,
MAXORDER = 6
DP = True MP = True

Functions & Variables

<Example>.py <Example>MP.py energy(z) kiMoKi(z)

Functions & Variables Functions & Variables tau epsilon

order maxorder
run<Example>.py run<Example>MP.py
dim params

Figure 10: This figure illustrates use of the code generator. The call
of kimoki.makeModules(’<Example>’,...) will generate a solver module,
<Example>.py, and a demonstration runfile run<Example>.py. When the optional
parameter MP=True a multiprecision version of the solver module and runfile is gener-
ated. The solver module consists of various functions and variables. Its most important
function is kiMoKi(z), which updates the solution z through one full timestep. The
function energy(z) evaluates the Hamiltonian at the phase space point z. Many other
functions are also defined. F.i., T2(z), T4(z), T6(z) (these return 0 if ALLTK=False),
and V2(z), V4(z), V6(z). Plus additional functions which are not intended to be
called directly by the user. Several parameters, some which can be changed by the
user, are also defined: The timestep tau, the accuracy epsilon to which equation
(9a) must be solved, which order of the integrator to use when running kiMoKi(z).
The maximum order maxorder of solvers available (must not be changed by the user),
the phase space dimension dim of the model being solved (must not be changed by
the user), and a (possibly empty) list of parameters params on which the Hamiltonian
depends (must not be changed by the user).

21
 <Example> <Example>MP runExamples

runVibratingBeam
runExamples runExamplesMP
DP = True, MP = False, Verbose = False

run<Example> run<Example>MP
runAnharmonicOscillator
DP = True, MP = True, Verbose = True,
PARAMS = params

<Example>_Soln <Example>_SolnMP runTwoDPendulum

DP = False, MP = True, Verbose = True,
<Example>_EgyErr <Example>_EgyErrMP MAXORDER = 6

Figure 11: This figure illustrates the runfile example. Normally the runfile generates
two figure files in .png format (this can easily be changed to .pdf format by the user).
The runfile also generate several intermediate files. Their names begin with the symbol
#. These files are normally deleted after use (this can easily be changed by the user).

6. Concluding remarks

In this paper we have demonstrated that the proposed extensions of the standard
Störmer-Verlet symplectic integration scheme can be implemented numerically, and
that the implemented code behave as expected with respect to accuracy. Here we have
not focused on time or memory efficiency of the generated code, which may be viewed
as a reference implementation known to work correctly. We have experienced this to be
a good starting point for manual implementation of more efficient code for large, struc-
tured systems, f.i. the Fermi-Pasta-Ulam-Tsingou type lattice models studied in [9],
and molecular dynamics type simulations studied in [12]. Code for the latter systems
are quite straightforward to implement using NumPy arrays [13], which also leads to
efficient working code.
It is also straightforward to modify our program to generate code in other computer
languages.

Acknowledgements

We thank professor Anne Kværnø for useful discussions, helpful feedbacks, and
careful proofreading. We also acknowledge support provided by Statoil via Roger
Sollie, through a professor II grant in Applied mathematical physics.

References

[1] J. M. Sanz-Serna, M. P. Calvo, Numerical Hamiltonian Problems, Chapman and

Hall, (1994)
[2] R. I. McLachlan, G. R. Quispel, Splitting methods, Acta Numerica, 341–434
(2002)

22
 [3] E. Hairer, Ch. Lubich, G. Wanner, Geometric Numerical Integrators. Structure-
Preserving Algorithms for Ordinary Differential Equations, Springer-Verlag, 2nd
edition (2006).
[4] I. Newton, Philosophiæ Naturalis Principia Mathematica (1687).

[5] Richard Feynman, The Character of Physical Law, p. 43, Penguin Science series,
Penguin Books, London (1992).
[6] C. Störmer, Méthode d’intégration numérique des équations différentielles or-
dinares, C.R. Congress Internat. Stassbourg 1920, 243–257 (1921).
[7] Loup Verlet, Computer “Experiments” on Classical Fluids. I. Thermodynamical
Properties of Lennard-Jones Molecules, Physical Review 159, 98-103 (1967).
[8] H. Goldstein, Classical Mechanics (3rd ed.), Addison-Wesley, 589–598 (2001).
[9] A. Mushtaq, A. Kværnø, K. Olaussen, Higher order Geometric Integra-
tors for a class of Hamiltonian systems, International Journal of Geomet-
ric Methods in Modern Physics, 11, 1450009-1–1450009-20 (2014). DOI:
10.1142/S0219887814500091, arXiv.org:1301.7736
[10] A. Mushtaq, A. Kværnø, K. Olaussen, Systematic Improvement of Splitting Meth-
ods for the Hamilton Equations, Proceedings for the World Congress on Engi-
neering, London July 4–6, Vol I, 247-251, (2012). arXiv.org:1204.4117v1.

[11] M. Abramowitz and I.S. Segun, Handbook of Mathematical Functions, Ch. 16,
Dover Publications (1968)
[12] A. Mushtaq, A. Noreen, K. Olaussen, and R. Sollie, Ensemble and Occupation
Time Probabilities, and the Rôle of Identical Particles, in preparation (2013).

[13] S. van der Walt, S.C. Colbert, G. Varoquaux, The NumPy Array: A Structure
for Efficient Numerical Computation, Computing in Science & Engineering 13,
22–30 (2011).

23