1.1.1 Solving ODEs

To define a problem, a user must call the constructor for the appropriate problem object. Since ordinary differential equations (ODEs) are represented in the general form as

the ODEProblem is defined by specifying a function f and an initial condition u₀. For example, we can define the linear ODE using the commands:

using DifferentialEquations
f(t,y) = 0.5y
u0 = 1.5
timespan = (0.0,1.0) # Solve from time = 0 to
time = 1
prob = ODEProblem(f,u0,timespan)

Many other examples are provided in the documentation¹ and the Jupyter notebook tutorials in DiffEqTutorials.jl² (for use with Julia, see IJulia.jl³). To solve the ODE, the user can simply call the solve command on the problem:

sol = solve(prob) # Solves the ODE

By using a dispatch architecture on AbstractArrays and using the array-defined indexing functionality provided by Julia (such as eachindex(A)), DifferentialEquations.jl accepts problems defined on arrays of any size. For example, one can define and solve a system of equations where the dependent variable u is a matrix as follows:

A = [1. 0 0 –5
     4 –2 4 –3
    –4  0 0  1
     5 –2 2  3]
u0 = rand (4,2)
f(t,u) = A*u
prob = ODEProblem(f,u0,timespan)
sol = solve(prob)

For most other packages, one would normally have to define u as a vector and rewrite the system of equations in the vector form. However, by allowing arbitrary problem sizes, DifferentialEquations.jl allows the user to specify problems in the natural format and solve directly on any array of numbers. This can be helpful for problems like discretizations of partial differential equations (PDEs) where the matrix format matches some underlying structure, and could result in a denser formulation.

The solver returns a solution object which holds all of the information about the solution. Dispatches to array functions are provided on the sol object, allowing for the solution object act like a timeseries array. In addition, high-order efficient interpolations are lazily constructed throughout the solution (by default, a feature which can be turned off) and the sol object’s call is overloaded with the interpolating function. Thus the solution object can both be used as an array of the solution values, and as a continuous approximation given by the numerical solution. The syntax is as follows:

sol[i] # ith solution value
sol.t[i] # ith timepoint
sol(t) # Interpolated solution at time t

The solution can be plotted using the provided plot recipes through Plots.jl⁴. The plot recipes use the solver object to build a default plot which is customizable using any of the commands from the Plots.jl package, and can be plotted to any plotting backend provided by Plots.jl. For example, we can by default plot to the PyPlot.jl⁵ backend (a Julia wrapper for matplotlib⁶) via the command:

plot(sol)

These defaults are deliberately made so that a standard user does not need to dig further into the manual and understand the differences between all of the algorithms. However, an extensive set of functionality is available if the user wishes. All of these functions can be modified via additional arguments. For example, to change the solver algorithm to a highly efficient Order 7 method due to Verner [29], set the line width in the plot to 3 pixels, and add some labels to the plot, one could instead use the commands:

sol = solve(prob, Vern7()) # Unrolled Verner 7th
Order Method
plot(sol,linewidth=3,xlabel=“t”,ylabel=“u(t)”)

The output of this command is shown in Figure 1. Note that the output is automatically smoothed using 10*length(sol) equally spaced interpolated values through the timespan.

Lastly, these solvers tie into Julia integrated development environments (IDEs) to further enhance the ease of use. Users of the Juno IDE [16] are equipped with a progressbar and time estimates to monitor the progress of the solver. Additionally, all of the DifferentialEquations.jl functions are thoroughly tested and documented with the Jupyter notebook system [19], allowing for reproducible exploration.

1.1.2 Solving SDEs

By using multiple-dispatch, the same user API is offered for other types of equations. For example, if one wishes to solve a stochastic differential equation (SDE):

then one builds an SDEProblem object by specifying the initial condition and now the two functions, f and g. However, the rest of the usage is the same: simply use the solve and plot functions. To extend the previous example to have multiplicative noise, the code would be:

g(t,u) = 0.3u
prob = SDEProblem(f,g,u0,timespan)
sol = solve(prob)
plot(sol)

While this user interface is simple, the default methods these algorithms can call are efficient high-order solvers with adaptive timestepping [21]. These methods tie into the plotting functionality and IDEs in the same manner as the ODE solvers, making it easy for users to explore stochastic modeling without having to learn learn a new interface.

1.1.3 Solving (Stochastic) PDEs

Again, the same user API is offered for the available stochastic PDE solvers. Instead, one builds a HeatProblem object which dispatches to algorithms for solving (Stochastic) PDEs. An example using the previously defined functions is:

T = 5
dx = 1/2^(1)
dt = 1/2^(7)
fem_mesh = parabolic_squaremesh([0 1 0 1],dx,
dt,T,:neumann)
prob = HeatProblem(f,mesh,σ=σ)
sol = solve(prob)

Additional keyword arguments can be supplied to HeatProblem to specify boundary data and initial condtions. Notice that the main difference is now we must specify a space-time mesh (and boundary conditions as optional keyword arguments). Again, the same plotting and analysis commands apply to the solution object sol (where now the plot dispatch is to a trisurf plot).

1.2.1 A Macro-Based Interface

Most differential equations packages require that the user understands some details about the implementation of the library. However, the DifferentialEquations.jl ecosystem implements various Domain-Specific Languages (DSLs) via macros in order to give more natural options for defining mathematical constructs. In this section we will demonstrate the DSL for defining ODEs. For demonstrations related to other types of equations, please see the documentation.

The famous Lorenz system is mathematically defined as

A user must re-write this function in a “computer friendly format”, defining u=[x;y;z] as a vector and writing the equation in terms of this vector. The format for ODE.jl, which is similar to other scripting languages like SciPy or MATLAB, is as follows:

f = (t,u,du) -> begin
du[1] = 10*(u[2]-u[1])
du[2] = u[1]*(28-u[3]) - u[2]
du[3] = u[1]*u[2] - 8/3*u[3]
end

While this format is accepted by DifferentialEquations.jl, additional usability macros are provided which will automatically translate user input from a more mathematical format. For ODEs, @ode_def is provided which allows the user to define the same ODE as follows:

f = @ode_def Lorenz begin
dx = σ*(y-x)
dy = x*(ρ-z) - y
dz = x*y - β*z
end σ=>10. ρ=>28. β=(8/3)

Since Julia allows for the use of Unicode within code, this format matches the style one would expect to see in a TeX’d publication. The macro takes in this definition, finds the values for the left-hand side of the form “d__”, and uses a dictionary in order to find/replace these values to write a function which is in the format of the other scripting language libraries. Thus the translation to a vector system can be done by DifferentialEquations.jl, allowing the users to have more readable scripts while not sacrificing performance. In addition, the macro produces a function which updates an input du in-place as the output. This detail can be hard for non-programmers to understand but is required for achieving fast solutions since otherwise every function call requires an array allocation.

1.2.2 Explicit Parameters

A unique feature from this form of function definition is that the parameters are built into the function type itself. The actual implementation involves creating a type Lorenz with fields for the parameters (inlining parameters defined with = instead of => during compilation). Then the type is set to have its call overloaded by the standard f(t,u,du) function signature, effectively acting like the appropriate function. However, the parameters are still accessible via the type fields, for example f.a or the overloaded f[:a]. This allows for sensitivity analysis, bifurcation diagrams, and parameter estimations to be computed using the same function, allowing for this infrastructure to extend far beyond the domain of differential equations solvers.

1.2.3 Enhanced Performance Through Symbolic Calculations

Also, since the code is analyzed by the program at the expression level, silent optimizations are able to be performed. For example, during the construction of the function, the code is transformed into a symbolic form for use in the high-performance CAS SymEngine.jl⁷ [28], where the Jacobian is calculated and an in-place function for its computation is created. In addition, the symbolic expression is inverted, allowing for stiff solvers which require inverting Jacobians to be written as directly computed matrices and matrix multiplications.

1.3.1 Building the solve function from DiffEqBase.jl and Component Solvers

DifferentialEquations.jl is designed to use multiple-dispatch in order to allow for the different solver methods to be defined in separate packages. In Julia, the command:

solve(prob,Algorithm())

calls a different “method” depending on the type of Algorithm. This function is called the “common solve interface”. The actual method that it calls does not need to be in the same package where the original function is defined. Using this design, the central package to the DifferentialEquations.jl ecosystem is DiffEqBase.jl⁹. DiffEqBase.jl defines the abstract type hierarchy, along with the problem and solution types, and the shared components. All of the component solver packages have a dependency on DiffEqBase.jl and add a method to this solve function. For example, for ODEs, the current list of component solvers is:

• OrdinaryDiffEq.jl¹⁰
• Sundials.jl¹¹
• ODE.jl¹²
• ODEInterface.jl¹³
• LSODA.jl¹⁴

Some of the packages, like OrdinaryDiffEq.jl and ODE.jl, are native Julia libraries, whereas Sundials.jl, ODEInterface.jl, and LSODA.jl are all interfaces to popular C and Fortran codes. Through this interface, users can switch between libraries by switching only the algorithm choice, and new packages can extend what’s available as needed. Note that the information in this structure is unidirectional: anyone can add a new package of solvers to the solve function by adding a dependency on DiffEqBase.jl without DiffEqBase.jl needing to be changed. This means that private projects, such as those which contain private research or proprietary methods, can extend this interface without being forced to edit the public DiffEqBase.jl repository. This same setup is then applied to each of the other types of equations. For full details on the current list of solvers methods that are available, along with the available methods for the other types of equations, please see the documentation. Note that from this modular structure, developers of other packages can use the functionality of DifferentialEquations.jl without having to depend on the entirety of the differential equations suite. For example, one can build an add-on package which uses the native ODE solvers and only include DiffEqBase.jl and OrdinaryDiffEq.jl as dependencies. This reduces the complexity associated with using the functionality, and helps developers keep dependencies as lean as possible.

1.3.2 The Add-on Packages

The add-on packages are a set of functionality which use the common solve interface. For example, parameter estimation functionality is provided by defining algorithms which only use the abstraction of the solve function, and allows the user to pass in the algorithm. Therefore these algorithms are generic, supporting many different equation types and internally able to use many different solver packages. Other such add-on functionality includes parameter sensitivity analysis, Monte Carlo parallelism functions, and uncertainty quantification. For more information on the current add-on packages, please consult the documentation.

1.4.1 Case 1: Arbitrary Precision Numerics

One advantage of this design beyond speed is that it allows the user of DifferentialEquations.jl to use any type which is a subclass of Number as the number type for the equations. This includes not only the basic types like Float64 and Int64, but also Rational and arbitrary precision BigFloats (based off of GNU MPFR). However, even numeric types defined in packages (which implement +,–,/, and for optionally for adaptive timestepping, sqrt) can be used within DifferentialEquations.jl. Some examples which have been shown to work are ArbFloats.jl¹⁵ (a library for faster high-precision numbers than MPFR floats between 64 and 512 bits based on the Arb library of Fredrik Johansson [14]) and DecFP.jl¹⁶ (an implementation of IEEE 754–2008 Decimal Floating-Point Arithmetic).

The combination of high-performance number systems with high order Runge-Kutta methods such as the Order 14 methods due to Feagin allows for fast solving with high accuracy. For an example showing this combination, see the “Feagin’s Order 10, 12, and 14 methods” notebook in the examples folder¹⁷.

1.4.2 Case 2: Unitful Numbers

This design also allows DifferentialEquations.jl to be compatible with number systems which have physical units. SIUnits.jl¹⁸ and Unitful.jl¹⁹ are packages which have developed number implementations which have units. Numbers defined by these packages automatically constrain the equations to satisfy dimensional constraints. For example, if one tries to add a quantity with units of seconds with a quantity with units of Newtons, it will throw an error. This is useful in fields like physics where these dimensional analysis tools are used to check for correctness in equations. DifferentialEquations.jl was developed such that the internal solvers satisfy dimensional constraints. Thus one can use unitful numbers like other arbitrary number systems. The “Unit Checked Arithmetic via Unitful” notebook in the examples folder²⁰ describes the usage of this feature. For example, we can solve an ODE where the dependent variable is in terms of seconds and the independent variable is in terms of Newtons via the following equation:

using DifferentialEquations, Unitful
f = (t,y) -> 0.5*y
u = 1.5u“N”
prob = ODEProblem(f,u,(0.0u“s”,1.0u“s”))
sol = solve(prob,dt=(1/2^4)u“s”)

The attentive reader should realize that this will correctly throw an error: the output of the function in an ODE must be a rate, and therefore must have units of N/s in this example. Unitful.jl will thus return an error notifying the user that the dimensions are off by a unit of seconds. Instead, the pleased physicists would modify the previous code by a rate constant and use the following code instead:

f = (t,y) -> 0.5*y/3.0u“s”
u = 1.5u“N”
prob = ODEProblem(f,u,(0.0u“s”,1.0u“s”))
sol = solve(prob,dt=(1/2^4)u“s”)

This will produce an output whose units are in terms of Newtons, and with time in terms of seconds.

1.5.1 Within-Method Multithreading

DifferentialEquations.jl also includes parallelism whenever possible. One area where parallelism is currently being employed is via “within-method” parallelism for Runge-Kutta methods. Using Julia’s experimental multithreading, DifferentialEquations.jl provides a multithreaded version of the DP5 solver. Benchmarks using the tools from Section 1.6 show that this can give a 30% non-multithreaded algorithm for problem sizes ranging from 75 × 75 matrices to 200 × 200 matrices. For larger problems this trails off as more time is spent within the function evaluations, thus reducing the difference between the methods. See the “Multithreaded Runge-Kutta Methods” notebook²¹ in DiffEqBenchmarks.jl for the most up-to-date results as this may change rapidly along with Julia’s threading implementation.

1.5.2 Multi-Node Monte Carlo Simulations

Also, DifferentialEquations.jl provides methods for performing parallel Monte Carlo simulations. Using Julia’s pmap construct, one is able to specify for a problem to be solved N times, and DifferentialEquations.jl will distribute this automatically across multiple nodes of a cluster. A vector of results along with summary statistics is returned for the solution. This functionality has been tested on the local UC Irvine cluster (using SGE) and the XSEDE Comet cluster (using Slurm).

1.6.1 Development

The design of DifferentialEquations.jl allows for users to add new integration methods by adding new dispatches. One way to add new methods is to simply create a new package which extends the solve function of DiffEqBase.jl as described in 1.3. Another way is to extend the current open-source native Julia solvers on the common interface. Let’s take as an example the ODE solver suite OrdinaryDiffEq.jl²², which contains the native Julia methods developed in tandem with DifferentialEquations.jl. The ODE solver works by setting up options and fixing types, and then builds the integrator type and enters the internal loop. All dispatchs on loop functions are done by the algorithm/cache type. Thus to define a new algorithm, one defines a new algorithm type which subtypes the OrdinaryDiffEqAlgorithm type. This will make solve plug into OrdinaryDiffEq.jl’s version of solve. From there, a new dispatch for perform_step needs to be added which performs the algorithm’s update from u_n to u_n+1. For example, the Midpoint Method steps as follows:

@inline function perform_step!(integrator,
                  cache::MidpointConstantCache,
                  f=integrator.f)
   @unpack t,dt,uprev,u,k = integrator
   halfdt = dt/2
   k = integrator.fsalfirst
   k = f(t+halfdt,uprev+halfdt*k)
   u = uprev + dt*k
   integrator.fsallast = f(t+dt,u) # For
              interpolation, then FSAL’d
   integrator.k[1] = integrator.fsalfirst
   integrator.k[2] = integrator.fsallast
   @pack integrator = t,dt,u
end

Additionally a new cache type must be developed for holding the cache variables. By doing this, all of the functionality of OrdinaryDiffEq.jl will be automatically applied to the algorithm. Thus interpolated (dense) output, FSAL optimizations (first-same-as-last, skipping an extra function evaluation), progress monitoring, event handling, and integrator interfaces will be available. Additionally, if an error estimator is given, the PI-contolled adaptive timestepping will be enabled for the algorithm. For more details, see the Developer Documentation²³.

1.6.2 Testing

The DifferentialEquations.jl suite includes a large number of testing functions to ensure correctness of all of the algorithms. Many premade problems with analytical solutions are provided and convergence testing functionality is included to be able to test the order of accuracy and plot the results. All of the DifferentialEquations.jl algorithms are tested using the Travis and AppVoyer Continuous Integration (CI) testing services to ensure correctness.

1.6.3 Benchmarking

Lastly, a benchmarking suite is included to test the efficiency of different algorithms. Two forms of benchmarking are included: the Shootout and the WorkPrecision. A Shootout solves using all of the algorithms in a given setup and calculates an average time (over a user-chosen number of runs) and error for each algorithm. The WorkPrecision and WorkPrecisionSet additionally take in vectors of tolerances and draw work-precision diagrams to compare algorithms. Up to date benchmarks can be found in the repository’s benchmarks folder²⁴. These notebooks can be opened to be run locally via the commands

using IJulia
notebook(dir=Pkg.dir(“DiffEqBenchmarks”)*“/
benchmarks”)

As of this publication, the benchmarks show that native methods from OrdinaryDiffEq.jl (which were developed in tandem with DifferentialEquations.jl) achieve an order of magnitude speedup on nonstiff problems when achieving the same error over the classic Hairer Runge-Kutta implementations and the ODE.jl implementations.