Applications of the Parareal Algorithm

Ma. Cristina Bargo

Laboratoire Jacques-Louis Lions

Université Pierre et Marie Curie
University of the Philippines Diliman

Groupe de Travail des Thésards

June 2, 2009

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 1 / 21
Outline

1 Overview
History
The Algorithm
Some Remarks
Convergence Results
Stability Results
Simple Implementation
Complexity

2 Applications
Published Works on Parareal
Current Work
Parareal with GPUs
Parareal on Reactive Flows
Parareal Applied to the KdVB equation

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 2 / 21
History

Lions, Maday and Turinici [5] (2001)

Bal and Maday [2] (2002) - equivalent to [5] for linear problems, but
with better results for nonlinear problems
Baffico et al [1] (2002)

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 3 / 21
The Algorithm
The Problem
Find u such that
(
∂t u + A (t, u) = 0, t > t0
(1)
u = u0 , t = t0

where A : R × V → V 0 (V a Hilbert space) and t0 ≥ 0.

Remarks :
If solution to (1) exists : Solution can be written as
u(t + τ ) = E(t + τ, t, v), where v = u(t) and τ > 0, with

∀µ > 0, τ > 0, E (t + τ + µ, t + τ, E(t + τ, t, v)) = E(t + τ + µ, t, v)

Let t0 = T0 < T1 < · · · < TN = T , and ∆Tn = Tn − Tn−1 . Then

∀n > 0, u(Tn ) = E(Tn , T0 , u0 ) = E(Tn , Tn−1 , u(Tn−1 ))

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 4 / 21
The Algorithm

Notations :
Fine solver F (t2 , t1 , u1 ) - approximation of the solution u(t2 ) to
problem (1) with initial condition u(t1 ) = u1
Coarse solver G(t2 , t1 , u1 ) - another approximation to u(t2 ), less
accurate than F (t2 , t1 , u1 ) but cheaper to solve

Parareal Algorithm
U00 = u0 (the initial condition in problem (1))
U0n+1 = G(tn+1 , tn , U0n ) (the coarse solver)
For k = 1, 2, · · · , Uk0 = u0 and
k+1
(2) Un+1 = G(tn+1 , tn , Uk+1 k k
n ) + F (tn+1 , tn , Un ) − G(tn+1 , tn , Un )

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 5 / 21
Some Remarks

F can be a classical discretization scheme with small timestep δt

G can be another discretization scheme with a larger timestep δT
(δT >> δt)
At iteration k + 1 : F (tn+1 , tn , Ukn ) and G(tn+1 , tn , Ukn ) dependent
only on values obtained from iteration k → can be done in parallel
with N processors
For k → ∞, then Ukn → Un where

Un+1 = F (tn+1 , t0 , U0 ) = F (tn+1 , tn , Un )

∀n = 0, 1, 2, · · · , N we can show that Unn = Un

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 6 / 21
Convergence Results
Maday, Rønquist and Staff, 2006 [6] :
Let δF = E − F and δG = E − G. If ∀τ > 0,

|δF (t + τ, t, x)| ≤ Cτ η(1 + |x|), |δG(t + τ, t, x)| ≤ Cτ (1 + |x|)

and in addition, if ∀τ ,

|δF (t + τ, t, x) − δF (t + τ, t, y)| ≤ Cτ η|x − y|,

and
|δG(t + τ, t, x) − δG(t + τ, t, y)| ≤ Cτ |x − y|.
Then the error between the exact solution and the parareal solution satisfies

(3) |Ukn − u(Tn )| ≤ C(k + η), ∀Tn < T

where k ≤ K with some fixed K < N/2.

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 7 / 21
Stability Results
Staff and Rønquist, 2005 [10] :
Assume that we want to solve the autonomous differential equation

y 0 = µy, y(0) = y0 , 0>µ∈R

using the parareal algorithm. Assume also that the system is stiff, meaning
that z = µ∆T << −1, and that the fine propagator is close to exact.
Then the stability function can be written as
 
k n−1
H(n, k, R) = (−1) Rn ,
k

and stability is guaranteed if the following property is fulfilled :

1
(4) R∞ = lim |R(z)| <
z→−∞ 2

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 8 / 21
Simple Implementation

 T
Sample Problem : Find u = u1 u2 so that

 ∂u = Au, t ∈ (0, 100]
∂t h iT
u = u0 = 1 0 , t=0

where A is the 2 × 2 matrix given by

 
−0.02 0.2
A=
−0.2 −0.02

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 9 / 21
Simple Implementation

t0 = 0 and T = 100
N = 100 and ∆Tn = ∆T = 1
Coarse scheme : Implicit Euler with δT = ∆T = 1
Fine scheme : Implicit Euler with δt = 0.1
Horizontal axis is u1 and the vertical axis is u2
fine scheme : 0.62 s
parareal scheme : 0.29496 s

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 10 / 21
Simple Implementation
Iteration 0

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 11 / 21
Simple Implementation
Iteration 1

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 12 / 21
Simple Implementation
Iteration 2

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 13 / 21
Simple Implementation
Iteration 3

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 14 / 21
Simple Implementation
Iteration 4

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 15 / 21
Complexity

complexity α number of time steps

N ∆T
fine scheme : δt CF
parareal scheme with N processors : KN CG + (K − 1)N ∆T
δt CF
local complexity for parareal : KN CG + (K − 1) ∆T
δt CF
Remarks :
if CG is negligible compared with CF → efficiency : N/(K − 1)
K should be small (ideally K = 2)
choice of G : close enough to F but cheap to solve

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 16 / 21
Published Works on Parareal

pricing of an American put (Bal and Maday, 2002 [2])

molecular dynamics simulations (Baffico et al, 2002 [1]) - fine scheme
uses the full model, coarse scheme is based on a simpler model of the
original
control problems (Maday and Turinici, 2002 [7])
combined with domain-decomposition methods (Maday and Turinici,
2005 [8])
Navier-Stokes (Fischer, Hecht and Maday, 2005 [4]) - using fine and
coarse mesh in space

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 17 / 21
Parareal with GPUs

NVIDIA R CUDATM (compute unified device architecture) -

computations with GPUs
CPU : double precision ; GPU : single precision
applications
I Accelerating numerical solution of Stochastic Differential Equations
with CUDA (Januszewski and Kostur, 2009) : up to 675x
I GPU accelerated Monte Carlo simulation of the 2D and 3D Ising model
(Preisa et al, 2009) : up to 60x
I Many-Core Algorithms for Statistical Phylogenetics (Suchard and
Rambaut, 2009) : up to 90x
I Solving Kinetic Equations on GPUs I : Model Kinetic Equations
(Frezzotti et al, 2009) : up to 500x
Idea :
I Solve the fine scheme using CPUs (in double precision)
I Solve the coarse scheme using GPUs (in single precision)

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 18 / 21
Parareal on Reactive Flows

uses the idea in Baffico et al [1] of using the simpler version of the
original model
adaptive chemistry approach by Schwer, Lu and Green (2003) [9] -
“active species” vary with spatial location
Idea :
I Use the adaptive chemistry for the coarse scheme
I Use the full chemistry for the fine scheme

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 19 / 21
Parareal Applied to the KdVB equation

KdVB Equation

1
(5) ut +uux = uxx +uxxx , ux (−∞) = 0, u(−∞) = 1, u(∞) = 0
R
Idea :
uses the idea in Baffico et al [1]
uses the idea in Fischer et al [4] - solve the problem in a coarse spatial
mesh for the coarse scheme, and use a finer mesh in space for the fine
scheme

Modified Parareal Algorithm

(6)
Uk+1 F C k+1 k F G k
n+1 = πC G(tn+1 , tn , πF Un )+F (tn+1 , tn , Un )−πG G(tn+1 , tn , πF Un )

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 20 / 21
Coarse Scheme : Average Equation
Based on the work by Chorin [3] :
1 x+`
Z
Let u(x, t) = u(y, t)dy (local average of the solution u in (5))
2` x−`

Effective Equation

(7) vt + vvx = νef f vxx + vxxx

with the same initial and boundary conditions in (5)

Note : νef f is the constant that minimizes

Z +∞
I= |u(x, t) − v(x, t)|2 dx
−∞

and νef f = R3/4 Φ(`)

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 21 / 21
 L. Baffico, S. Bernard, Y. Maday, G. Turinici, and G. Zérah.
Parallel-in-time molecular-dynamics simulations.
Phys. Rev. E, 66(5) :057701, Nov 2002.
Guillaume Bal and Yvon Maday.
A “parareal” time discretization for nonlinear PDE’s with application to
the pricing of an american put.
In L.F. Pavarino and A. Toselli, editors, Recent Developments in
Domain Decomposition Methods, volume 23 of Lecture Notes in
Computational Science and Engineering, pages 189–202.
Springer-Verlag, Berlin, 2002.
Alexandre J. Chorin.
Averaging and renormalization for the Korteveg-deVries-Burgers
equation.
Proc. Natl. Acad. Sci. USA, 100(17) :9674–9679, 2003.
Paul F. Fischer, Frédéric Hecht, and Yvon Maday.
A parareal in time semi-implicit approximation of the Navier-Stokes
equations.
MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 21 / 21
 In Domain Decomposition Methods in Science and Engineering,
volume 40 of Lecture Notes in Computational Science and Engineering,
pages 433–440. Springer, Berlin, 2005.
Jacques-Louis Lions, Yvon Maday, and Gabriel Turinici.
Résolution d’EDP par un schéma en temps “pararéel”.
C. R. Acad. Sci. Paris Sér. I Math., 332(7) :661–668, 2001.
Y. Maday, E.M. Rø nquist, and G.A. Staff.
The parareal-in-time algorithm : basics, stability and more.
2006.
Yvon Maday and Gabriel Turinici.
A parareal in time procedure for the control of partial differential
equations.
C. R. Math. Acad. Sci. Paris, 335(4) :387–392, 2002.
Yvon Maday and Gabriel Turinici.
The parareal in time iterative solver : a further direction to parallel
implementation.

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 21 / 21
 In Domain decomposition methods in science and engineering,
volume 40 of Lecture Notes in Computational Science and Engineering,
pages 441–448. Springer, Berlin, 2005.
Douglas A Schwer, Pisi Lu, and William Jr. H. Green.
An adaptive chemistry approach to modeling complex kinetics in
reacting flows.
Combustion and Flame, 133(4) :451–465, 2003.
Gunnar Andreas Staff and Einar M. Rønquist.
Stability of the parareal algorithm.
In Domain Decomposition Methods in Science and Engineering,
volume 40 of Lecture Notes in Computational Science and Engineering,
pages 449–456. Springer, Berlin, 2005.

MC Bargo (UPMC + UPD) Applications of the Parareal Algorithm GTT June 2, 2009 21 / 21