Noname manuscript No.

(will be inserted by the editor)

pyomo.dae: A Modeling and Automatic Discretization Framework

for Optimization with Differential and Algebraic Equations
Bethany Nicholson John D. Siirola Jean-Paul Watson Victor
M. Zavala Lorenz T. Biegler

Received: date / Accepted: date

Abstract We describe pyomo.dae, an open source Python-based modeling framework that enables high-level
abstract specification of optimization problems with differential and algebraic equations. The pyomo.dae
framework is integrated with the Pyomo open source algebraic modeling language, and is available at http:
//www.pyomo.org. One key feature of pyomo.dae is that it does not restrict users to standard, predefined
forms of differential equations, providing a high degree of modeling flexibility and the ability to express constraints that cannot be easily specified in other modeling frameworks. Other key features of pyomo.dae are the
ability to specify optimization problems with high-order differential equations and partial differential equations, defined on restricted domain types, and the ability to automatically transform high-level abstract models
into finite-dimensional algebraic problems that can be solved with off-the-shelf solvers. Moreover, pyomo.dae
users can leverage existing capabilities of Pyomo to embed differential equation models within stochastic and
integer programming models and mathematical programs with equilibrium constraint formulations. Collectively, these features enable the exploration of new modeling concepts, discretization schemes, and the benchmarking of state-of-the-art optimization solvers.
Keywords Dynamic optimization Mathematical modeling Algebraic modeling language DAE constrained
optimization PDE constrained optimization

Bethany Nicholson Lorenz T. Biegler

Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh PA 15213
John D. Siirola Jean-Paul Watson
Center for Computing Research, Sandia National Laboratories, Albuquerque NM 87185
Victor M. Zavala
Department of Chemical and Biological Engineering, University of Wisconsin-Madison, Madison, WI, 53706

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler
Tool

Types

ACADO [17]

ODE, DAE

APMonitor [15]

ODE, DAE

GPOPSII [25]

ODE, DAE

gPROMS [26]

ODE, DAE, PDAE

Optimica and JModelica.org [1]

ODE, DAE

SOCS [6] and SOS [5]

ODE,DAE

TACO [21]

Non-Canonical Forms

Open Source

Open Language

Solution Methods

C++

Shooting Methods, Collocation

Collocation

Shooting Methods, Collocation

Collocation
X

ODE, DAE

Shooting, Collocation
Fortran

Finite Difference, Collocation

Python

Finite Difference, Collocation

Tomlab PROPT [27]

ODE, DAE

pyomo.dae

ODE, DAE, PDAE

Shooting
Collocation

Table 1 Capabilities of existing tools for specifying and solving optimization problems with differential equations.

1 Introduction and Motivation

Optimization problems constrained by differential and algebraic equations (DAEs) or partial differential and
algebraic equations (PDAEs) are ubiquitous in engineering and science. Application domains include aerospace
systems, chemical reactor systems, infrastructure (water, gas, electricity, and transportation) networks, economics and finance, robotics, and environmental engineering. Differential constraints arising in these domains
are often expressed in specific canonical forms, but more complex and exotic (non-canonical) forms such as
multi-point boundary conditions and PDAEs spanning multiple domains are not uncommon. The latter forms
cannot be expressed easily in existing modeling packages or by variational methods. Moreover, the solution
of problems with differential constraints requires sophisticated transcription (discretization) schemes that may
require the combination of different techniques and associated experimentation. These schemes are often implemented manually and are thus tedious, time-consuming, and error-prone. To avoid such complications, it
is desirable to separate model abstractions from discretization schemes and to automate such schemes so that
users can experiment with alternative approaches and assess performance in a more systematic manner.
There are many approaches available to users interested in incorporating differential equations into optimization models. One widely used alternative is to simply apply manual discretization, expressing the resulting finite-dimensional representation in an algebraic modeling language. Algebraic modeling languages
typically allow a user to represent optimization problems using a concise and natural syntax, perform limited
model checking and automatic differentiation, and provide interfaces for communicating with optimization
solvers. Well-known algebraic modeling languages include GAMS [11] and AMPL [12]. A limitation of these
tools is that they define their own proprietary syntax for representing optimization problems and are not open
source. Consequently, they have limited extensibility.
In contrast to commercial alternatives, algebraic modeling tools that are embedded in high-level programming languages such as Python, Matlab, and Julia provide more flexibility to incorporate new syntax, components, and processing capabilities. Examples of such tools include FlopC++ [18], PuLP [24], Pyomo [14], and
more recently JuMP [22]. While these algebraic modeling languages allow the user to formulate problems in
a high-level programming language, they are currently restricted in the classes of optimization problems they
can represent. Specifically, only a small subset of these tools provide syntax extensibility and processing (discretization) capabilities for differential equations. An overview of such tools and their capabilities is provided
in Table 1. This table summarizes the types of differential equations each tool can express and manipulate,
indicates whether the tool is open source, and indicates whether the tool represents models using a high-level
programming language (as opposed to a proprietary syntax). The last column of the table lists the solution
methods each tool provides to the user. As the table indicates, the new package described in this paper
pyomo.dae allows for representation of a broad range of differential and algebraic equation components in
optimization models, is open source and based on a widely used high-level programming language (Python),
and provides a range of discretization capabilities for solution of specified models.
Most of the existing tools that can express and solve optimization models with differential equations require users to formulate models using a proprietary language. Notable exceptions are ACADO and SOCS/SOS, which can be respectively used directly from C++ and Fortran. ACADO is open source and provides both
shooting and direct collocation methods for solving specified problems. However, ACADO cannot represent

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

optimal control problems in non-canonical forms. While these tools provide tailored solution algorithms for
models with specific classes of differential equations, they are unable to represent and solve models which
deviate even slightly from those specific classes. APMonitor offers additional flexibility over ACADO in expressing differential equations in non-canonical forms and high-index DAE systems, and is freely accessible
via the web. However, APMonitor is not open source and uses a proprietary syntax.
Of the tools surveyed here, gPROMS is the only package that provides complete flexibility in the types of
differential equations that can be modeled, including support for PDAEs and non-canonical forms. However,
gPROMS is a commercial software. Tools such as Optimica, TACO, and Tomlabs PROPT are extensions for the
existing programming languages and software packages Modelica, AMPL, and Matlab, respectively. None of
these tools include support for differential equations in non-canonical forms and the extensions are all based
on a proprietary syntax.
In contrast to the tools described above, pyomo.dae is a flexible and extensible tool for representing
systems of ODEs, DAEs, and PDAEs on certain domains and provides capabilities to automatically applying numerical discretization techniques for converting differential equations to algebraic equations. Using
pyomo.dae, a user may represent ordinary and partial differential equations of arbitrary order, of arbitrary
dimension, and in any form including complex mixed partial derivatives. The pyomo.dae package is also capable of expressing and solving optimal control problems in non-canonical forms. We have based pyomo.dae
on a widely used open source algebraic modeling language, Pyomo, which in turn is written in the high-level
programming language Python.
The remainder of this paper is organized as follows. In Section 2, we provide a brief overview of Pyomo and
describe the new modeling constructs available in pyomo.dae. Then, in Section 3, we review the automatic
discretization schemes that are provided in pyomo.dae to transform dynamic models to algebraic approximations, and describe their use within Pyomo and Python. In Section 4, we detail how pyomo.dae can be
extended to support custom discretization schemes. We discuss the expression and solution of several illustrative optimization problems with dynamics using pyomo.dae in Section 5. Finally, we conclude in Section 6
with a summary of pyomo.dae capabilities, our contributions to algebraic modeling languages for optimization, and directions for future research.

2 Modeling Differential Equations Using pyomo.dae

In Section 2.1 we provide a brief overview of the Pyomo algebraic modeling language, surveying its core capabilities and justifying our choice of Pyomo as the basis for our DAE and PDAE modeling and transformation
capabilities. In Section 2.2, we then describe our new modeling components, which integrate directly with and
extend the Pyomo library.

2.1 Pyomo: Background

Pyomo is an open source algebraic modeling language written in Python [14, 13]. Since its introduction, Pyomo
has undergone major restructuring and extension, such that Pyomo is now stable, flexible, and widely used.
An overview of the current features of Pyomo is shown in Figure 1. Pyomo supports a wide range of problem types including Linear Programming (LP), Mixed-Integer Programming (MIP), Nonlinear Programming
(NLP), and Mixed-Integer Nonlinear Programming (MINLP). Pyomo also provides interfaces to a variety of
optimization solvers and provides automatic differentiation (AD) for NLP problems via the open source AMPL
Solver Library (ASL).
One of Pyomos main advantages over other algebraic modeling languages is that it is written in a highlevel programming language, Python. Consequently, a user does not have to learn a specialized modeling
language in order to formulate and solve optimization problems; a basic understanding of Python is all that

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

Fig. 1 Summary of Pyomo features

is required. Models are represented using Python objects and can be formulated and manipulated in sophisticated ways using simple scripts. Furthermore, Pyomo users have access to a large collection of other Python
packages which include tools for plotting, numerical and statistical analysis, and input/output. These capabilities enable the development of novel algorithms, complicated model formulations, and general model transformations. All of these features make Pyomo a promising platform for implementing extensions for problem
classes such as dynamic optimization.
2.2 Modeling Components for Expressing DAEs and PDAEs
One of the main constructs in any algebraic modeling language is the notion of an indexing set. Indexing
sets allow users to compactly specify related collections of parameters, variables, and constraints. In standard
algebraic modeling languages, indexing sets are assumed to be discrete. This assumption makes it challenging
to represent optimization problems with continuous domains and dynamics without first converting the model
to a discretized approximation, causing loss of information and transparency with respect to the precise nature
of the model. To address these shortcomings, the pyomo.dae library allows direct specification of continuous
domains and arbitrary derivatives, through the introduction of two new Pyomo modeling components. We
now describe these components and provide a brief overview of how they can be used to express a diverse
range of algebraic optimization models with differential equations. More details about these components can
be found in the online documentation available at http://www.pyomo.org.
2.2.1 The ContinuousSet Component
Analogous to the discrete set construct available in most algebraic modeling languages, we now introduce
the notion of a continuous set. The ContinuousSet construct allows users to represent continuous bounded
domains in an algebraic optimization model. Such domains can be interpreted as spatial or time domains,
but such interpretations are not strictly necessary. Rather, a continuous domain can represent any domain of a
function, e.g., a parameter domain.
The ContinuousSet component is similar in structure to that of Pyomos Set component and can be
used to index variables and constraints. Variables and constraints can be defined over an arbitrary number of
continuous domains. For instance, the expression
w(`, x, t) : ` L := {0, 1, ..., L}, t T := [0, T ], x X := [0, X]

(2.1)

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

defines a variable w with indices ` spanning the discrete set L, such that each w(`, , ) is defined over the
continuous rectangular domain T X = [0, T ] [0, L]. By default, a ContinuousSet will be treated as a
closed set. The code to declare variable (2.1) using pyomo.dae is shown below (assuming L, T , and X are
previously defined constants).
Listing 1 Declaring a variable over discrete and continuous domains
1
2
3
4
5

m =
m. l
m. x
m. t
m.w

ConcreteModel ( )
= RangeSet ( 0 , L )
= ContinuousSet ( bounds = ( 0 , X ) )
= ContinuousSet ( bounds = ( 0 , T ) )
= Var (m. l , m. x , m. t )

A ContinuousSet must be initialized with two numeric values that specify the lower and upper bounds
of the continuous domain being represented. A user may also specify additional points in the domain that may
be communicated to a transformation method and/or solver. Such data can be used, for instance, to ensure that
a discretization mesh includes a subset of points, which in turn may be required to specify path constraints or
ensure consistency with experimental data. The following code fragment illustrates this capability:
Listing 2 Declaring continuous domains using ContinuousSet components
1
2
3
4

# declare
model . t =
# declare
model . x =

by providing bounds
ContinuousSet ( bounds = ( 0 , 5 ) )
s p e c i f i c p o i n t s i n domain
ContinuousSet ( i n i t i a l i z e = [ 0 , 1 , 2 . 5 , 3 . 3 , 5 ] )

A user may also declare a ContinuousSet as part of a purely abstract Pyomo model, mirroring AMPLs
paradigm of explicitly separating model from data. Generally, most valid methods to declare and initialize a
Pyomo Set can be used to declare and initialize a ContinuousSet.
When an algebraic optimization model with DAEs and/or PDAEs is manually discretized, the indexing set
for a continuous variable typically takes the form of an arbitrary set of integers and the user is responsible for
converting each index to the desired temporal or spatial quantity. This conversion process is one of the most
common places where modeling errors occur, particularly when a more complex discretization scheme is used,
e.g., collocation on finite elements. The conversion can be particularly challenging in the case of unequally
spaced finite elements. Beyond discretization for a solver, accuracy of this conversion is crucial in visualization
and analysis of results. One of the main design strengths of the ContinuousSet component in pyomo.dae
is that this manual conversion step has been eliminated and domain scaling is done directly within the set. In
other words, the discrete points in a ContinuousSet directly capture actual floating point values from the
bounded continuous domain being represented.
The ContinuousSet component can also be used to store information from a transformation and/or
solver. For instance, if the set is discretized using a collocation method over finite elements, one can obtain
a list of the discretization points used so that the user can interrogate and manipulate the ContinuousSet
component and implement customized initializations or constraints. Examples of these capabilities are shown
below:
1
2
3

m = ConcreteModel ( )
m. t = ContinuousSet ( bounds = ( 0 , 1 0 ) )
m. v = Var (m. t )

4
5
6
7

# D i s c r e t i z e model using Radau C o l l o c a t i o n over f i n i t e elements

d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . c o l l o c a t i o n )
d i s c r e t i z e r . a p p l y t o (m, wrt=m. t , n f e =100 , ncp =3)

8
9
10
11

# Obtain a Python d i c t i o n a r y c o n t a i n i n g d e t a i l s about t h e d i s c r e t i z a t i o n

# scheme t h a t has been a p p l i e d t o t h e Continuous S e t
d i s c i n f o = m. t . g e t d i s c r e t i z a t i o n i n f o

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

12
13
14
15
16

# Apply c o n s t r a i n t a t f i n i t e element boundary p o i n t s only

def con (m, i ) :
r e t u r n m. v [ i ] >= 0
m. con = C o n s t r a i n t (m. t . g e t f i n i t e e l e m e n t s ( ) , r u l e = con )

17
18
19
20
21
22

# Obtain t h e upper and lower f i n i t e element boundaries f o r

# internal collocation points
f o r i i n m. t :
upper = m. t . get upper element boundary ( i )
lower = m. t . g e t l o w e r e l e m e n t b o u n d a r y ( i )

The ContinousSet component provides a user with a highly flexible mechanism to impose constraints
and objectives in non-standard forms. In particular, we do not impose restrictions on the number of independent dimensions of a variable or constraint, nor do we constrain the order of appearance of continuous and
regular sets in an indexed variable or constraint. For example, we enable specifications of the following form:
w(`, x, y, z, t) : ` L, x X , y Y, z Z, t T .

(2.2)

Analogous to the definition of a subset of a discrete set L L is the definition of a subset of a continuous set
T T . Here, the subset T can be either discrete (with elements in [0, T ]) or continuous. Constraints involving
variables defined over a continuous set may be defined over a subset of a continuous set. For instance, if we
define X = [0, 2] and T = [1, 10], the following constraints are conceptually valid:
h(w(`, x, t)) = 0, x [0, 1], t [5, 10]
h(w(`, x, t)) = 0, x {0, 0.1, 0.5, 1}, t T
h(w(`, x, t)) = 0, x {0, 0.1, 0.5, 1}, t {2, 3, 4}.
Currently, pyomo.dae does not allow a ContinuousSet component to be defined as a subset of another
ContinuousSet. This extension is on-going work. Instead, constraints over continuous subsets can currently
be implemented using constraint skipping. Examples of declaring constraints over continuous and discrete
subsets are shown below:
Listing 3 Declaring constraints over a subset of a continuous set
1
2

m. x s u b s e t = S e t ( i n i t i a l i z e = [ 0 , 0 . 1 , 0 . 5 , 1 ] )
m. t s u b s e t = S e t ( i n i t i a l i z e = [ 2 , 3 , 4 ] )

3
4
5

m. x = ContinuousSet ( bounds = ( 0 , 2 ) , i n i t i a l i z e =m. x s u b s e t )

m. t = ContinuousSet ( bounds = ( 1 , 1 0 ) , i n i t i a l i z e =m. t s u b s e t )

6
7
8
9
10
11
12

def

con1 (m, l , x , t ) :
i f x <= 1 and t >= 5 :
r e t u r n m. h [ l , x , t ] == 0
else :
r e t u r n C o n s t r a i n t . Skip
m. con1 = C o n s t r a i n t (m. l , m. x , m. t , r u l e = con1 )

13
14
15
16

def

con2 (m, l , x , t ) :
r e t u r n m. h [ l , x , t ] == 0
m. con2 = C o n s t r a i n t (m. l , m. xsubset , m. t , r u l e = con2 )

17
18
19
20

def

con3 (m, l , x , t ) :
r e t u r n m. h [ l , x , t ] == 0
m. con3 = C o n s t r a i n t (m. l , m. xsubset , m. t s u b s e t , r u l e = con3 )

The ability to impose constraints at discrete points allows the user to specify path and point constraints. This
in turn allows the imposition of multi-point boundary conditions including periodicity and delay constraints.

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

Recirculation (cyclic) constraints are typical in chemical reactors, periodicity constraints are typical in model
predictive control, and delay constraints are typical in signal and transportation networks. Specifications of
these types of constraints take the following form:
h(w1 (x, 0)) = f (w1 (x, T )),

xX

h(w1 (x, T )) = f (w1 (x, T )),

xX

h(w1 (X, 0)) = f (w1 (X/2, T /2))

where h() and f () are arbitrary functions. The constraints expressed above can be implemented as follows
assuming X and T are previously defined constants:
Listing 4 Declaring multi-point boundary conditions
1

m. d e l t a = Param ( i n i t i a l i z e =5)

2
3
4

m. x = ContinuousSet ( bounds = ( 0 , X ) , i n i t i a l i z e =[X/ 2 ] )

m. t = ContinuousSet ( bounds = ( 0 , T ) , i n i t i a l i z e =[T/2 , Tm. d e l t a ] )

5
6
7

m. h = E x p r e s s i o n (m. x , m. t , r u l e = a r b i t r a r y h e x p r e s s i o n )
m. f = E x p r e s s i o n (m. x , m. t , r u l e = a r b i t r a r y f e x p r e s s i o n )

8
9
10
11

def

con1 (m, x ) :
r e t u r n m. h [ x , 0 ] == m. f [ x , T ]
m. con1 = C o n s t r a i n t (m. x , r u l e = con1 )

12
13
14
15

def

con2 (m, xidx ) :

r e t u r n m. h [ x , T ] == m. f [ x , Td e l t a ]
m. con2 = C o n s t r a i n t (m. x , r u l e = con2 )

16
17
18
19

def

con3 (m) :
r e t u r n m. h [ X , 0 ] == m. f [ X/2 , T/2]
m. con3 = C o n s t r a i n t ( r u l e = con3 )

We also allow the user to impose constraints between variables and across domains. For instance, the constraint
h(w(`, x, T )) = f (w(` + 1, x, 0)),

xX

can be implemented as:

Listing 5 Declaring constraints across domains
1
2
3

m. l = RangeSet ( 1 , 1 0 )
m. x = ContinuousSet ( bounds = ( 0 , X ) )
m. t = ContinuousSet ( bounds = ( 0 , T ) )

4
5
6
7
8
9
10

def

con (m, l , x ) :
i f l == 1 0 :
r e t u r n C o n s t r a i n t . Skip
else :
r e t u r n m. h [ l , x , T ] == m. f [ l +1 , x , 0 ]
m. con = C o n s t r a i n t (m. l , m. x , r u l e = con )

This feature enables the user to couple physical elements described by different sets of differential equations. For instance, multiple domains arise in the modeling and control of lithium-ion batteries, fuel cells,
buildings, and gas networks. Coupling across domains is also common in multi-stage optimal control where
the horizon is lifted into stages.
The above example illustrates that pyomo.dae provides an intuitive, straightforward syntax for formulating and linking dynamic algebraic optimization models over several domains. However, we note that we

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

do not make any claims concerning our ability to solve arbitrary dynamic models over continuous domains.
Rather, our tool simply allows the user to formulate their problem in a direct, high-level manner.
2.2.2 The DerivativeVar Component
The DerivativeVar component is used to declare a derivative of a standard Pyomo Var. A variable may only
be differentiated with respect to a ContinuousSet. Furthermore, the ContinuousSet must be included as
an explicit indexing set of the Var. The indexing sets of a DerivativeVar are obtained directly from the Var
it is differentiating. We also allow for specification of derivatives of arbitrary order. In the case of the variable
w(`, x, t) defined in (2.1), the derivatives
w(`, x, t)
(2.3a)
t
w(`, x, t)
(2.3b)
x
w(`, x, t)
(2.3c)
t x
can be imposed for , {1, 2, . . .}. The following code fragment illustrates the declaration of first-order,
second-order, and mixed-order derivatives. The variable being differentiated is supplied as the only positional
argument to the DerivativeVar constructor and the type of derivative is specified using the wrt keyword
argument (the more verbose withrespectto can also be used).
Listing 6 Declaring derivatives
1
2
3

m. dwdt = D e r i v a t i v e V a r (m.w, wrt=m. t )

m. dwdx2 = D e r i v a t i v e V a r (m.w, wrt =(m. x , m. x ) )
m. dwdxt = D e r i v a t i v e V a r (m.w, wrt =(m. x , m. t ) )

We assume that a variable defined over a continuous set is sufficiently smooth, such that any defined
derivatives exist. The user is responsible for ensuring that these requirements are fulfilled. As is the case with
the ContinuousSet component, the specification syntax is agnostic to the specifics of any transformation
and/or solution method that may be employed.
Derivatives can be referenced in the body of any constraint. This is an important feature, as we do not
impose any predefined structure on differential models. This enables the expression of DAEs with complex
mass matrices or boundary conditions. For instance, consider the following constraints:
w(`, x, t)
= f (w(`, x, t)), ` L, x X , t T
x
w(`, x, t)
0=
g (w(`, x, t)) , ` L, x X , t T .
x
The code implementing such constraints has the form:
Listing 7 Declaring constraints with derivatives
1
2

m. f = E x p r e s s i o n (m. l , m. x , m. t , r u l e = a r b i t r a r y f e x p r e s s i o n )
m. g = E x p r e s s i o n (m. l , m. x , m. t , r u l e = a r b i t r a r y g e x p r e s s i o n )

3
4
5
6

def

con1 (m, l , x , t ) :
r e t u r n m. dwdx[ l , x , t ] == m. f [ l , x , t ]
m. con1 = C o n s t r a i n t (m. l , m. x , m. t , r u l e = con1 )

7
8
9
10

def

con2 (m, l , x , t ) :
r e t u r n 0 == m. dwdx [ l , x , t ] m. g [ l , x , t ]
m. con2 = C o n s t r a i n t (m. l , m. x , m. t , r u l e = con2 )

The code fragment below provides a more explicit and complete example of how to specify DerivativeVar
components:

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

Listing 8 Declaring derivatives using DerivativeVar components

1
2
3
4
5
6
7

model =
model . s
model . t
model . l
model . x
model . y
model . z

ConcreteModel ( )
= S e t ( i n i t i a l i z e =[ a , b ] )
= ContinuousSet ( bounds = ( 0 , 5 ) )
= ContinuousSet ( bounds =( 10 , 1 0 ) )
= Var ( model . t )
= Var ( model . s , model . t )
= Var ( model . t , model . l )

8
9
10

# D e c l a r e f i r s t d e r i v a t i v e o f model . x with r e s p e c t t o model . t

model . dxdt = D e r i v a t i v e V a r ( model . x , w i t h r e s p e c t t o =model . t )

11
12
13
14

# D e c l a r e second d e r i v a t i v e o f model . y with r e s p e c t t o model . t

# Note t h a t t h i s D e r i v a t i v e V a r w i l l be indexed by both model . s and model . t
model . dydt2 = D e r i v a t i v e V a r ( model . y , wrt =( model . t , model . t ) )

15
16
17
18

# D e c l a r e p a r t i a l d e r i v a t i v e o f model . z with r e s p e c t t o model . l

# Note t h a t t h i s D e r i v a t i v e V a r w i l l be indexed by both model . t and model . l
model . dzdl = D e r i v a t i v e V a r ( model . z , wrt=model . l , i n i t i a l i z e =0)

19
20
21
22

# D e c l a r e mixed second order p a r t i a l d e r i v a t i v e o f model . z with r e s p e c t

# t o model . t and model . l and s e t bounds
model . dz2 = D e r i v a t i v e V a r ( model . z , wrt =( model . t , model . l ) , bounds = ( 1 0 , 1 0 ) )

The design of the DerivativeVar component diverges from that of core Pyomo components in its use of
positional arguments. Typically, positional arguments in Pyomo are used to specify indexing sets of a particular
component. However, for DerivativeVar components, a variable is supplied as a positional argument. This
design choice was intended to codify the notion of a derivative being an operation on a variable rather than
a distinct object. We note that the initialize keyword argument shown above will initialize the value of a
derivative, rather than specifying an initial condition or boundary constraint.
After the derivatives in a model have been declared using DerivativeVar components, differential equations are declared as standard Pyomo constraints and are not required to have any particular form. The following code fragment illustrates how one might declare an ordinary or partial differential equation using one
or more of the derivatives defined in the previous code fragment:
Listing 9 Declaring differential equations
1
2
3
4
5
6

# An o r d i n a r y d i f f e r e n t i a l e q u a t i o n
def o d e r u l e (m, t ) :
i f t == m. t . f i r s t ( ) :
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. dxdt [ t ] == m. x [ t ] 2
model . ode = C o n s t r a i n t ( model . t , r u l e = o d e r u l e )

7
8
9
10
11
12
13

# A p a r t i a l d i f f e r e n t i a l equation
def p d e r u l e (m, t , l ) :
i f t == m. t . f i r s t ( ) or l == m. l . f i r s t ( ) or l == m. l . l a s t ( ) :
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. dzdl [ t , l ] == m. dz2 [ t , l ]
model . pde = C o n s t r a i n t ( model . t , model . l , r u l e = p d e r u l e )

A modeler may not want to define a differential equation at one or both boundaries of a continuous domain.
This choice can be specified explicitly in the Constraint declaration using the Constraint.Skip return value,
as shown above. By default, a constraint declared over a ContinuousSet will be applied at every discretization point contained in the set. One can also think of this as the distinction between applying a constraint over
an open or closed set. Finally, for the case of boundary conditions of PDAEs, we note that the user is respon-

10

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

sible for providing consistent expressions and pyomo.dae provides syntax to communicate such constraints
(e.g., derivative objects and access to specific points).

3 Discretization Transformations
Before a Pyomo model with ContinuousSet and DerivativeVar components can be processed by a finite
dimensional solver it must first be discretized. This transformation process converts a dynamic optimization
model to a purely algebraic optimization model using a simultaneous discretization approach. Specifically,
the continuous domains in the dynamic optimization model are discretized and any derivatives are approximated using algebraic equations defined at the discretization points. Two families of discretization schemes
are currently implemented in pyomo.dae: finite differences and collocation.
By separating models from discretization schemes we enable users to easily experiment with alternative
discretization schemes, to identify the one that works best with their particular problem. This separation also
provides users the freedom to combine discretization schemes in non-standard ways, e.g., using collocation to
discretize a spatial domain and finite difference method to discretize in time or vice versa.
One of the key differences between pyomo.dae and similar tools is that the discretized model is returned to
the user after each transformation is applied. This design feature allows the user to interrogate the discretized
model and examine the discretization equations that were added to the model. Additionally, users can further
modify the model following discretization.
Applying one of the discretization schemes available in pyomo.dae to a differential equation is analogous
to approximating the solution of that differential equation using a numerical method. Numerical methods
vary in terms of accuracy and the type of problems to which they can be applied. While we provide a brief
overview of the methods implemented in pyomo.dae below, a detailed description of these methods is outside
the present scope. For more information on these techniques and details of their applicability, we defer to [3, 4,
9, 10].

3.1 Finite Difference Transformation

Finite difference methods are the simplest discretization schemes to apply manually, and approximate the
derivative at a particular point using a difference equation. The dae.finite difference transformation
in pyomo.dae includes implementations of several finite difference methods. The most commonly-used finite
difference method is the backward difference method, also referred to as implicit or backward Euler. In our implementation, a discretization is applied to a particular continuous domain and propagated to each derivative
and constraint over that domain. For instance, consider the following derivative and associated constraint:


dx(t)
, f (x(t), u(t)) = 0, t [0, T ].
(3.4)
dt
After applying the backward difference method to domain t, the resulting derivative and constraint pair is

dx 
xk+1 xk
=
, k = 0, ..., N 1
(3.5)

dt tk+1
h
!

dx 
g
, f (xk+1 , uk+1 ) = 0, k = 0, ..., N 1
(3.6)
dt tk+1
where xk = x(tk ), tk = kh, and h is the step size between discretization points or the size of each finite element. We note that the discretization scheme is applied to all constraints and variables of the model in a
given continuous domain. Higher order derivative terms are approximated using recursive schemes. When a
dae.finite difference transformation is applied to a Pyomo model, equations such as (3.5) are automatically generated and added to the resulting discretized Pyomo model as equality constraints. The following

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

11

Python script applies the backward difference method to a Pyomo model. The code also illustrates how to add
a constraint to a model after discretization.
Listing 10 Applying a finite difference discretization to a Pyomo model
1
2

from pyomo . environ import

from pyomo . dae import

3
4
5

# Import c o n c r e t e Pyomo model

from pyomoExample import model

6
7
8
9

# D i s c r e t i z e model using Backward D i f f e r e n c e method

d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . f i n i t e d i f f e r e n c e )
d i s c r e t i z e r . a p p l y t o ( model , n f e =20 , wrt=model . time , scheme= BACKWARD )

10
11
12
13
14

# Add a c o n s t r a i n t t o t h e d i s c r e t i z e d model
def s u m l i m i t (m) :
r e t u r n sum (m. x1 [ i ] f o r i i n m. time ) <= 50
model . c o n s u m l i m i t = C o n s t r a i n t ( r u l e = s u m l i m i t )

15
16
17
18

# Solve d i s c r e t i z e d model
solver = SolverFactory ( ipopt )
r e s u l t s = s o l v e r . s o l v e ( model )

There are several options available to a dae.finite difference transformation, which can be specified
as keyword arguments to the apply function of the transformation object. These keywords are as follows:

nfe: Specifies the desired number of finite element points to be included in the discretization. The default
value is 10.
wrt: Specifies the ContinuousSet for which the transformation should be applied. If this keyword argument is not specified then the same scheme will be applied to all continuous sets in the model.
scheme: Specifies which finite difference method to apply. Alternatives are BACKWARD, CENTRAL, or FORWARD.
The default scheme is the backward difference method.

If the existing number of finite element points in a ContinuousSet is less than the desired number, new
discretization points will be automatically added to the set. If a user specifies a desired number of finite element
points which is less than the number of points already included in the ContinuousSet then the transformation will ignore the desired number and proceed with the larger set of points. Discretization points will never
be removed from a ContinuousSet during the discretization transformation.

3.2 Collocation Transformation

Orthogonal collocation (or simply collocation) over finite elements is another popular numerical method for
discretization of differential equations. Collocation works by first breaking a continuous domain into N 1
finite elements. Then over each finite element i the profile of the differential variable x(t) is approximated using
a polynomial of order K + 1. This polynomial is defined using K collocation points, which act as additional
discretization points within each finite element i. Continuity is enforced at the finite element boundaries for
the differential variables x. For example, the discretization equations for a constraint of the form (3.4) are given

12

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

by

K
1 X
dx 
d`j (k )
=
, k = 1, . . . , K, i = 1, . . . , N 1
xij
dt tij
hi j=0
d
!

dx 
, f (xik , uik ) , k = 1, . . . , K, i = 1, . . . , N 1
0=g
dt tij
xi+1,0 =

K
X

i = 1, . . . , N 1

`j (1)xij ,

(3.7a)

(3.7b)
(3.7c)

j=0

where tij = ti1 + j hi , x(tij ) = xij . Further, we note that the solution x(t) is interpolated as follows:
x(t) =

K
X

`j ( )xij ,

t [ti1 , ti ],

[0, 1]

(3.8a)

j=0

`j ( ) =

K
Y
k=0,6=j

( k )
.
(j k )

(3.8b)

The advantage of using a collocation method over a finite difference method is that collocation results in significantly more accurate algebraic approximations. The drawback of collocation is that it is much harder to
implement and debug manually, relative to finite difference methods. Furthermore, implementing the collocation discretization equations on higher-order derivatives, partial derivatives, or differential equations that are
not in a standard form is non-trivial.
There are many variations of collocation methods, differing primarily in the functional representation of
the state profile over each finite element and how the collocation points are defined. The current version of
pyomo.dae includes two types of collocation methods, both using Lagrange polynomials to represent the
state profiles but differing in the choice of collocation points k . One variant uses shifted Gauss-Radau roots
and the other one uses shifted Gauss-Legendre roots. The term shifted refers to the fact that the collocation
points are defined in the domain [0, 1] rather than [1, 1]. For more information on orthogonal
collocation we refer the reader to Chapter 10 of [7].
The following Python script applies the collocation method with Lagrange polynomials and Gauss-Radau
roots to a Pyomo model, and additionally illustrates how to add an objective function to the resulting discretized model.
Listing 11 Applying a collocation discretization to a Pyomo model
1
2

from pyomo . environ import

from pyomo . dae import

3
4
5

# Import c o n c r e t e Pyomo model

from pyomoExample2 import model

6
7
8
9

# D i s c r e t i z e model using Radau C o l l o c a t i o n

d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . c o l l o c a t i o n )
d i s c r e t i z e r . a p p l y t o ( model , n f e =20 , ncp =6 , scheme= LAGRANGERADAU )

10
11
12
13
14

# Add o b j e c t i v e f u n c t i o n a f t e r model has been d i s c r e t i z e d

def o b j r u l e (m) :
r e t u r n sum ( (m. x [ i ]m. x r e f ) 2 f o r i i n m. time )
model . o b j = O b j e c t i v e ( r u l e = o b j r u l e )

15
16
17
18

# Solve d i s c r e t i z e d model
solver = SolverFactory ( ipopt )
r e s u l t s = s o l v e r . s o l v e ( model )

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

13

The collocation discretization transformation in pyomo.dae allows the user to specify the number of finite
elements, nfe, independently of the number of collocation points per finite element, ncp. The discretization options available to a dae.collocation transformation are the same as those described above for the
dae.finite difference transformation, with the following additions:
scheme: Specifies the desired collocation scheme, either LAGRANGE-RADAU or LAGRANGE-LEGENDRE.
The default is LAGRANGE-RADAU.
ncp: Specifies the number of collocation points within each finite element. The default value is 3.
We note that any points that exist in a ContinuousSet before a transformation is applied will be used
as finite element boundaries and not as collocation points. The locations of the collocation points cannot be
specified by the user. Rather, they must be generated by the transformation.
3.3 Applying Multiple Transformations
Discretization transformations can be applied independently to each ContinuousSet in a Pyomo model,
providing great flexibility to users. For example, the same scheme can be applied with different resolutions in
different domains, as follows:
Listing 12 Applying different discretization resolutions to different continuous domains
1
2
3

d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . f i n i t e d i f f e r e n c e )
d i s c r e t i z e . a p p l y t o ( model , wrt=model . t1 , n f e =10)
d i s c r e t i z e . a p p l y t o ( model , wrt=model . t2 , n f e =100)

This feature can be used, for instance, when discretizing different pipelines in a network that exhibit different dynamic behavior or discretizing different stages in an optimal control model with different resolutions.
Different schemes can also be applied to different domains. For example, we can apply a forward difference
method to one ContinuousSet and the central finite difference method to another ContinuousSet:
Listing 13 Applying different finite difference transformations to different continuous domains
1
2
3

d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . f i n i t e d i f f e r e n c e )
d i s c r e t i z e r . a p p l y t o ( model , wrt=model . t1 , scheme= FORWARD )
d i s c r e t i z e r . a p p l y t o ( model , wrt=model . t2 , scheme= CENTRAL )

Further, users may combine finite difference and collocation discretizations, e.g., as follows:
Listing 14 Combining finite difference and collocation schemes
1
2
3
4

discretizer
discretizer
discretizer
discretizer

f e = T r a n s f o r m a t i o n F a c t o r y ( dae . f i n i t e d i f f e r e n c e )
f e . a p p l y t o ( model , wrt=model . t1 , n f e =10)
c o l = T r a n s f o r m a t i o n F a c t o r y ( dae . c o l l o c a t i o n )
c o l . a p p l y t o ( model , wrt=model . t2 , n f e =10 , ncp =5)

To apply the same discretization to all ContinuousSet components in a model, users simply specify a single discretization transformation without the wrt keyword argument. This approach will apply the selected
scheme to all ContinuousSet components in the model that have not already been discretized.
4 Package Implementation and Extensibility
One of the main implementation goals for automatic discretization in pyomo.dae is extensibility, as there
are a plethora of discretization schemes documented in the literature and some specialized schemes might be
required in certain applications. As part of our implementation we have developed a general transformation
framework along with certain utility functions so that advanced users may easily implement their own custom
discretization schemes while reusing the syntax of the modeling language. The transformation framework
consists of the following steps:

14

1.
2.
3.
4.
5.

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

Specify Discretization Options

Discretize the Continuous Sets
Update Model Components
Add Discretization Equations
Return Discretized Model

If a user would like to create a custom finite difference scheme then they only have to re-implement step (4)
in the framework. The discretization equations for a particular scheme have been isolated from of the rest of the
code for implementing the transformation. For example, the specific function for the forward finite difference
method is:
Listing 15 pyomo.dae implementation of the forward finite difference scheme
1

def

2
3
4
5
6
7
8
9

forward transform ( v , s ) :

Applies t h e Forward D i f f e r e n c e formula o f order O( h ) f o r f i r s t d e r i v a t i v e s

def fwd fun ( i ) :

tmp = s o r t e d ( s )
idx = tmp . index ( i )
r e t u r n 1/(tmp [ idx +1]tmp [ idx ] ) ( v ( tmp [ idx +1]) v ( tmp [ idx ] ) )
r e t u r n fwd fun

In this function, v represents the continuous variable or function that the method is being applied to
while s represents the set of discrete points in the continuous domain. In order to implement a custom finite
difference method, a user would have to duplicate the above function and just replace the equation next to the
first return statement with their method. After implementing a custom finite difference method using the above
function template, the only other change that must be made is to add the custom method to the all schemes
dictionary in the Finite Difference Transformation class.
In the case of a custom collocation method, changes will be made in steps (2) and (4) of the transformation
framework. The code below shows the function specific to a collocation scheme with Lagrange interpolation
polynomials and Radau collocation points.
Listing 16 pyomo.dae implementation of collocation using Lagrange polynomials and Radau roots
1
2
3
4
5
6
7
8
9
10
11
12
13

def

lagrange radau transform (v , s ) :

ncp = s . g e t d i s c r e t i z a t i o n i n f o ( ) [ ncp ]
adot = s . g e t d i s c r e t i z a t i o n i n f o ( ) [ adot ]
def f u n ( i ) :
tmp = s o r t e d ( s )
idx = tmp . index ( i )
i f idx == 0 : # Don t apply t h i s e q u a t i o n a t i n i t i a l p o i n t
r a i s e I n d e x E r r o r ( l i s t index out o f range )
low = s . g e t l o w e r e l e m e n t b o u n d a r y ( i )
lowidx = tmp . index ( low )
r e t u r n sum ( v ( tmp [ lowidx+ j ] ) adot [ j ] [ idxlowidx ] \
( 1 . 0 / ( tmp [ lowidx+ncp]tmp [ lowidx ] ) ) f o r j i n range ( ncp + 1 ) )
return fun

In addition to implementing the discretization equations, the user would have to ensure that the desired
collocation points are added to the ContinuousSet being discretized. The collocation transformation in
pyomo.dae calculates Radau or Legendre collocation points following chapter 10 in [7] if the user has the
Python package Numpy installed, otherwise it uses precomputed values (for up to 10th -order polynomials).
So far we have shown that pyomo.dae is extensible in terms of implementing custom discretization schemes
but it is also extensible in a more general sense. The modeling abstraction introduced here allows for general
implementations of any model transformation or operation typically applied to optimal control problems. For
instance, the control profiles can be constrained to follow predefined functions. To illustrate this, we extended

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

15

the dae.collocation transformation with a function that forces a control variable to have a certain profile;
for example, piecewise constant or piecewise linear. This is useful, for instance, in optimal control problems
where the control variable is often restricted to be constant over each finite element while the other state variables are not. This function can be implemented by reducing the number of free collocation points for a particular variable. The reduce collocation points() function is specified using the following keywords:
var: The variable being restricted to fewer collocation points
contset: The continuous set indexing the specified var that was previously discretized.
ncp: The new number of collocation points. Must be at least 1 and less than the number of collocation points
used to discretize the contset.
The function may only be applied to a model after a collocation discretization transformation has been
called to discretize the specified ContinousSet. The function works by adding constraints to the discretized
model which force any extra, undesired collocation points to be interpolated from the others. These constraints
have the following form over each finite element:
ui =

K
X

+ 1, ..., K
ij uj , i = K

(4.9)

j=1

is the reduced number of points. A sample implewhere K is the original number of collocation points and K
mentation is shown below.
Listing 17 Enforce path constraints on continuous variables
1
2
3

# D i s c r e t i z e model using Radau C o l l o c a t i o n

d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . c o l l o c a t i o n )
d i s c r e t i z e r . a p p l y t o ( model , wrt=model . time , n f e =20 , ncp =6)

4
5
6

# C o n t r o l v a r i a b l e u made c o n s t a n t over each f i n i t e element

d i s c r e t i z e r . r e d u c e c o l l o c a t i o n p o i n t s ( var=model . u , c o n t s e t =model . time , ncp =1)

We also note that pyomo.dae does not distinguish between state and control variables. This means that
the reduce collocation points() function can be applied just as easily to impose a path constraint on a
continuous state variable.
The modular nature of Pyomo also allows for extensions in terms of solvers for dynamic optimization.
For example, a typical workflow with the current implementation of pyomo.dae is shown in Figure 2. A
straightforward extension would be to replace the Write .nl File and AMPL Solver Library blocks with
another tool for automatic differentiation such as CasADi [2]. Or the workflow could be extended to output
the dynamic model, before applying a discretization scheme, and then solving the problem using a shooting
method. Finally, there are several parallel solvers available for dynamic optimization problems that rely on
the discretized model being separated at the finite element boundaries. A general implementation of this step
could easily be added to pyomo.dae.

5 Illustrative Applications
We now provide application examples that illustrate the use and flexibility of the modeling and transformation
capabilities provided by pyomo.dae. We begin with a detailed presentation of a simple heat transfer problem.
We then consider a small optimal control example, which is illustrative of common application for dynamic
optimization. Next, we examine a medium-scale parameter estimation problem for a dynamic disease transmission model, which demonstrates how to integrate time-dependent data into a optimization model using
pyomo.dae. We conclude with a more complex example involving stochastic optimal control of natural gas
networks, which (1) includes all key modeling and transformation elements found in pyomo.dae and (2)
demonstrates the scalability of pyomo.dae. All examples use Pyomo version 4.3, Python version 2.7.6, and

16

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

Fig. 2 Pyomo workflow including discretization transformations

Ipopt version 3.10.1 on a desktop computer running Ubuntu 14.04. Code for these examples can be accessed at
https://software.sandia.gov/svn/pyomo/pyomo/trunk/examples/dae/.

5.1 Heat Transfer

Our first application example is taken from [23] and demonstrates the solution of a one-dimensional heat
equation. The equation, initial condition, and boundary conditions are as follows:
u
2u
=
t
x2
u(x, 0) = sin(x)

(5.10b)

u(0, t) = 0

(5.10c)

u
(1, t) = 0
x
x [0, 1], t [0, T ].
et +

(5.10a)

(5.10d)
(5.10e)

This model has a number of interesting features, including first and second order partial derivatives, boundary conditions involving both the variable and its partial derivative, and nonlinearity induced by a trigonometric function. In Listing 18, we present a Pyomo model that implements a manual discretization, without
pyomo.dae. We discretized this model in space using the backward difference method and in time using orthogonal collocation with Radau points. The first line in the model imports the required Pyomo packages.
Lines 46 declare the number of discretization points and lines 911 declare sets specifying indices for all the
discretization points. Lines 1420 define the collocation matrix for 4th order Lagrange polynomials with Radau
roots. Computing this matrix by hand is often one of the more tedious parts of implementing a dynamic algebraic optimization model. Lines 2223 define the spatial and time scaling for each finite element and lines
25-30 define the model variables, including the differential variables.
Models that are discretized manually typically involve discretization points that lack any physical meaning,
i.e., time/distance scaling is handled separately from the definition of the discretization points. This approach
often leads to modeling errors and confusion especially when more complicated discretization schemes such
as collocation are considered. The model below tries to overcome this difficulty by introducing an additional
differential variable m.t that is used to calculate the properly scaled time points. Lines 3254 define constraints
representing the heat transfer model (5.10a). The discretization schemes are then applied in lines 5693. Finally,
a dummy objective function (because the example involves no optimization objective) is declared in line
95 and the model is solved using Ipopt in lines 9798. The entire implementation requires fewer than 100
lines of code. While changing the number of finite element points is straightforward, changing the number of
collocation points or either of the discretization schemes would require significant changes.
Listing 18 Manually discretized heat transfer model in Pyomo

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations
1

from pyomo . environ import

2
3
4
5
6
7

m = ConcreteModel ( )
m. n f e t = Param ( i n i t i a l i z e =20)
m. ncp = Param ( i n i t i a l i z e =4)
m. nfex = Param ( i n i t i a l i z e =25)
m. p i = Param ( i n i t i a l i z e = 3 . 1 4 1 6 )

#
#
#
#

Number o f f i n i t e elements i n time

Number o f c o l l o c a t i o n p o i n t s per f i n i t e element
Number o f f i n i t e elements i n space
Pi

8
9
10
11

m. t f e = RangeSet ( 0 , value (m. n f e t ) 1) # S e t o f f i n i t e element p o i n t s i n time

m. t c p = RangeSet ( 1 , value (m. ncp ) )
# S e t o f c o l l o c a t i o n p o i n t s i n time
m. x = RangeSet ( 0 , value (m. nfex ) )
# S e t o f f i n i t e element p o i n t s i n space

12
13
14
15
16
17
18
19
20

# C o l l o c a t i o n Matrix
radau adot = { ( 1 , 1 ) : 9 . 0 0 0 0 0 0 0 0 0 0 0 0 0 , ( 1 , 2 ) : 4 . 1 3 9 3 8 7 6 9 1 3 3 9 8 , ( 1 , 3 ) : 1 . 7 3 9 3 8 7 6 9 1 3 3 9 8 , \
(1 ,4): 2.9999999999999 ,(2 ,1):10.0488093998274 ,(2 ,2):3.2247448713915 ,\
(2 ,3): 3.5678400846904 ,(2 ,4):5.5319726474218 ,(3 ,1): 1.3821427331607 ,\
(3 ,2):1.1678400846904 ,(3 ,3):0.7752551286084 ,(3 ,4): 7.5319726474218 ,\
(4 ,1):0.3333333333333 ,(4 ,2): 0.2531972647421 ,(4 ,3):1.0531972647421 ,\
(4 ,4):5.0000000000000}
m. adot = Param (m. tcp , m. tcp , i n i t i a l i z e =radau adot )

21
22
23

m. ht = Param ( i n i t i a l i z e =2.0/m. n f e t ) # Length o f time f i n i t e element

m. hx = Param ( i n i t i a l i z e =1.0/m. nfex ) # Length o f s p a t i a l f i n i t e element

24
25
26

m. u = Var (m. x , m. t f e , m. tcp , i n i t i a l i z e = 0 . 2 )

m. t = Var (m. t f e , m. t c p )

27
28
29
30

m. dudx = Var (m. x , m. t f e , m. t c p )

m. dudx2 = Var (m. x , m. t f e , m. t c p )
m. dudt = Var (m. x , m. t f e , m. t c p )

31
32
33
34
35
36

def

pde (m, i , j , k ) :
i f i == 0 or i == value (m. nfex ) or k == 0 :
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. p i 2m. dudt [ i , j , k ] == m. dudx2 [ i , j , k ]
m. pde = C o n s t r a i n t (m. x , m. t f e , m. tcp , r u l e = pde )

37
38
39
40
41
42

def

i n i t c o n (m, i ) :
i f i == 0 or i == value (m. nfex ) :
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. u [ i , 0 , 1 ] == s i n (m. p i i m. hx )
m. i n i t c o n = C o n s t r a i n t (m. x , r u l e = i n i t c o n )

43
44
45
46

def

lowerbound (m, j , k ) :
r e t u r n m. u [ 0 , j , k ] == 0
m. lowerbound = C o n s t r a i n t (m. t f e , m. tcp , r u l e = lowerbound )

47
48
49
50

def

upperbound (m, j , k ) :
r e t u r n m. p i exp(m. t [ j , k ] ) +m. dudx [ value (m. nfex ) , j , k ] == 0
m. upperbound = C o n s t r a i n t (m. t f e , m. tcp , r u l e = upperbound )

51
52
53
54

def

i n i t t (m) :
r e t u r n m. t [ 0 , 1 ] == 0
m. i n i t t = C o n s t r a i n t ( r u l e = i n i t t )

55
56
57
58
59
60

# Apply f i n i t e d i f f e r e n c e d i s c r e t i z a t i o n e q u a t i o n s
def dudx backwardDifference (m, i , j , k ) :
i f i == 0 :
r e t u r n C o n s t r a i n t . Skip
r e t u r n (m. u [ i , j , k]m. u [ i 1, j , k ] ) /m. hx == m. dudx [ i , j , k ]

17

18
61

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

m. dudx backwardDifference = C o n s t r a i n t (m. x , m. t f e , m. tcp , r u l e = dudx backwardDifference )

62
63
64
65
66
67

def

dudx2 backwardDifference (m, i , j , k ) :

i f i == 0 or i == 1 :
r e t u r n C o n s t r a i n t . Skip
r e t u r n (m. u [ i 2, j , k]2m. u [ i 1, j , k ]+m. u [ i , j , k ] ) /m. hx2 == m. dudx2 [ i , j , k ]
m. dudx2 backwardDifference = C o n s t r a i n t (m. x , m. t f e , m. tcp , r u l e = dudx2 backwardDifference )

68
69
70
71
72
73
74

# Apply c o l l o c a t i o n d i s c r e t i z a t i o n e q u a t i o n s
def d u d t c o l l o c a t i o n (m, i , j , k ) :
i f k == 1 :
r e t u r n C o n s t r a i n t . Skip
r e t u r n sum (m. u [ i , j , s ] m. adot [ s , k ] f o r s i n m. t c p ) == m. ht m. dudt [ i , j , k ]
m. d u d t c o l l o c a t i o n = C o n s t r a i n t (m. x , m. t f e , m. tcp , r u l e = d u d t c o l l o c a t i o n )

75
76
77
78
79
80

def

d t c o l l o c a t i o n (m, j , k ) :
i f k == 1 :
r e t u r n C o n s t r a i n t . Skip
r e t u r n sum (m. t [ j , s ] m. adot [ s , k ] f o r s i n m. t c p ) == m. ht
m. d t c o l l o c a t i o n = C o n s t r a i n t (m. t f e , m. tcp , r u l e = d t c o l l o c a t i o n )

81
82
83
84
85
86
87

# Apply c o l l o c a t i o n c o n t i n u i t y e q u a t i o n s
def u c o n t i n u i t y (m, i , j ) :
i f j == value (m. n f e t ) 1:
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. u [ i , j + 1 , 1 ] == m. u [ i , j , 4 ]
m. u c o n t i n u i t y = C o n s t r a i n t (m. x , m. t f e , r u l e = u c o n t i n u i t y )

88
89
90
91
92
93

def

t c o n t i n u i t y (m, j ) :
i f j == value (m. n f e t ) 1:
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. t [ j + 1 , 1 ] == m. t [ j , 4 ]
m. t c o n t i n u i t y = C o n s t r a i n t (m. t f e , r u l e = t c o n t i n u i t y )

94
95

m. o b j = O b j e c t i v e ( expr =1)

96
97
98

solver = SolverFactory ( ipopt )

r e s u l t s = s o l v e r . s o l v e (m, t e e =True )

We now contrast the Pyomo model with manual discretization to one that is discretized using pyomo.dae
as shown in Listing 19 below. The model requires one additional package import in order to access the modeling components and transformations available in pyomo.dae. Lines 4-8 define the model sets and variables.
Note that temporal and spatial scaling is handled explicitly via the defined ContinuousSet components. Specifically, after applying a discretization scheme, the components m.t and m.x will contain actual points in time
and space, respectively. This approach eliminates the need for an additional differential variable to represent
time, as was used in the manually discretized model. Lines 10-13 define the derivatives that appear in the
model, lines 15-34 define the equations associated with the heat transfer model, and line 36 defines the objective function. Lines 38-42 apply discretization schemes defined in pyomo.dae to the model. Line 41 applies
the backward difference method to the spatial ContinuousSet m.x and line 42 applies Radau collocation to
the temporal ContinuousSet m.t.
Listing 19 Automatically discretized heat transfer model in Pyomo using pyomo.dae
1
2

from pyomo . environ import

from pyomo . dae import

3
4
5
6

m = ConcreteModel ( )
m. p i = Param ( i n i t i a l i z e = 3 . 1 4 1 6 )
m. t = ContinuousSet ( bounds = ( 0 , 2 ) )

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations
7
8

19

m. x = ContinuousSet ( bounds = ( 0 , 1 ) )
m. u = Var (m. x ,m. t )

9
10
11
12
13

# D e c l a r e d e r i v a t i v e s i n t h e model
m. dudx = D e r i v a t i v e V a r (m. u , wrt=m. x )
m. dudx2 = D e r i v a t i v e V a r (m. u , wrt =(m. x , m. x ) )
m. dudt = D e r i v a t i v e V a r (m. u , wrt=m. t )

14
15
16
17
18
19
20

# D e c l a r e PDE
def pde (m, i , j ) :
i f i == 0 or i == 1 or j == 0 :
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. p i 2m. dudt [ i , j ] == m. dudx2 [ i , j ]
m. pde = C o n s t r a i n t (m. x , m. t , r u l e = pde )

21
22
23
24
25
26

def

i n i t c o n (m, i ) :
i f i == 0 or i == 1 :
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. u [ i , 0 ] == s i n (m. p i i )
m. i n i t c o n = C o n s t r a i n t (m. x , r u l e = i n i t c o n )

27
28
29
30

def

lowerbound (m, j ) :
r e t u r n m. u [ 0 , j ] == 0
m. lowerbound = C o n s t r a i n t (m. t , r u l e = lowerbound )

31
32
33
34

def

upperbound (m, j ) :
r e t u r n m. p i exp( j )+m. dudx [ 1 , j ] == 0
m. upperbound = C o n s t r a i n t (m. t , r u l e = upperbound )

35
36

m. o b j = O b j e c t i v e ( expr =1)

37
38
39
40
41
42

# D i s c r e t i z e using F i n i t e D i f f e r e n c e and C o l l o c a t i o n
d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . f i n i t e d i f f e r e n c e )
d i s c r e t i z e r 2 = T r a n s f o r m a t i o n F a c t o r y ( dae . c o l l o c a t i o n )
d i s c r e t i z e r . a p p l y t o (m, n f e =25 , wrt=m. x , scheme= BACKWARD )
d i s c r e t i z e r 2 . a p p l y t o (m, n f e =20 , ncp =3 , wrt=m. t )

43
44
45

solver = SolverFactory ( ipopt )

r e s u l t s = s o l v e r . s o l v e (m, t e e =True )

The implementation using pyomo.dae requires only half the number of lines of the manually discretized
model, leading to a much more concise and consequently comprehensible model. The applied discretization
schemes are completely independent of the model, and as a result, the correspondence between the algebraic model and the mathematical model is significantly more clear. The automatically discretized model does
not have any of the discretization-specific parameters used in the manually discretized model. Furthermore,
changing the number of finite elements, the number of collocation points, or the entire discretization scheme
is simply accomplished by modifying one of the 4 lines of code defining the discretization. To illustrate this
point, some alternative approaches to discretize the heat transfer model are as follows:
Listing 20 Alternate discretizations for heat the transfer model in Pyomo using pyomo.dae
1
2
3
4

# D i s c r e t i z e e n t i r e model using F i n i t e D i f f e r e n c e Method

d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . f i n i t e d i f f e r e n c e )
d i s c r e t i z e r . a p p l y t o (m, n f e =25 , wrt=m. x , scheme= BACKWARD )
d i s c r e t i z e r . a p p l y t o (m, n f e =20 , wrt=m. t , scheme= FORWARD )

5
6
7
8
9

# D i s c r e t i z e e n t i r e model using C o l l o c a t i o n
d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . c o l l o c a t i o n )
d i s c r e t i z e r . a p p l y t o (m, n f e =10 , ncp =3 , wrt=m. x , scheme= LAGRANGELEGENDRE )
d i s c r e t i z e r . a p p l y t o (m, n f e =20 , ncp =3 , wrt=m. t , scheme= LAGRANGERADAU )

20

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

With pyomo.dae, experimenting with the type of discretization scheme in addition to the resolution becomes accessible and systematic. However, we observe that not every discretization scheme is appropriate
for every model. Different boundary and initial conditions may need to be applied, depending on the model
and the discretization scheme used. The pyomo.dae package does not perform any model checking to ensure
appropriateness of the model to which a discretization scheme is applied. Rather, such checking is the responsibility of the user. The pyomo.dae package, however, provides the necessary constructs to enable the user to
easily tailor the specification of boundary conditions.

5.2 Optimal Control

The purpose of optimal control problems is to find a sequence of inputs to a dynamic system that optimize
some system performance metric. For example, a typical optimization objective involves minimizing or maximizing the value of a state variable at the end of a fixed time horizon. Consider the following small example,
taken from [19]:
min

(5.11a)

x3 (tf )

s.t. x 1 = x2

(5.11b)

x 2 = x2 + u

(5.11c)

x 3 = x21 + x22 + 0.005 u2

2

x2 8 (t 0.5) + 0.5 0
x1 (0) = 0, x2 (0) = 1, x3 (0) = 0, tf = 1

(5.11d)
(5.11e)
(5.11f)

This example consists of three state variables x1 , x2 , x3 and one control variable u. The following code fragment
illustrates the implementation of some of the more interesting features of this model, including the objective
function (5.11a) and the complex path constraint (5.11e).
Listing 21 Partial implementation of problem (5.11) in Pyomo using pyomo.dae
1
2

# S e t t i n g the o b j e c t i v e function
m. o b j = O b j e c t i v e ( expr=m. x3 [ 1 ] )

3
4
5
6
7

# D e c l a r i n g t h e path c o n s t r a i n t
def con (m, t ) :
r e t u r n m. x2 [ t ] 8( t 0 . 5 ) 2 + 0 . 5 <= 0
m. con = C o n s t r a i n t (m. t , r u l e = con )

Discretizing problem (5.11) using orthogonal collocation with 7 finite elements and 6 collocation points
per element will result in the optimal profiles shown in the top portion of Figure 3. In this solution, the control variable is allowed to vary at every time point. In many applications this sort of continuous change in
the control variable is undesirable or impossible to implement. Therefore, in practice, control variables are
often restricted to be piecewise constant over a certain time interval. Such restrictions can be easily implemented using pyomo.dae, using the reduce collocation points() method of the dae.collocation
discretization transformation. We illustrate use of this method in the code fragment presented below. The last
line of the fragment restricts the control variable u to only 1 free collocation point per finite element, rendering
it piecewise constant. The resulting state and control profiles are shown in the bottom of Figure 3. If instead
we wanted the control variable to be piecewise linear we would only have to modify the ncp keyword argument in the reduce collocation points() method call to ncp=2, which would yield two free collocation
points (degrees of freedom) for the control variable per finite element.

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

21

Listing 22 Code for discretizing the optimal control model and reducing the degrees of freedom
1
2
3

# D i s c r e t i z e model using radau c o l l o c a t i o n

d i s c r e t i z e r = T r a n s f o r m a t i o n F a c t o r y ( dae . c o l l o c a t i o n )
d i s c r e t i z e r . a p p l y t o ( model , wrt=model . t , n f e =7 , ncp =6)

4
5
6

# R e s t r i c t c o n t r o l t o be p i e c e w i s e c o n s t a n t
d i s c r e t i z e r . r e d u c e c o l l o c a t i o n p o i n t s ( model , var=model . u , ncp =1 , c o n t s e t =model . t )

Fig. 3 Solution to the optimal control problem (top) with no restrictions on the control variable and (bottom) restricting the control variable
to be piecewise constant. Black dotted line shows the inequality path constraint.

22

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

5.3 Parameter Estimation for Disease Transmission

Our next application considers a non-linear parameter estimation problem taken from [30]. The objective of
this problem is to estimate parameters for a model of childhood infectious disease, using real-world disease
case data. The mathematical model for this problem is as follows:
min

(Ii )2 +

iF

s.t.

dS
dt
dI
dt
d
dt
Rk?

(k )2

(5.12a)

kT

(y(t))S(t)I(t)
I (t) + B(t)
N
(y(t))S(t)I(t)
=
+ I (t) I(t)
N
(y(t))S(t)I(t)
=
+ I (t)
N
= k (i,k i,k1 ) + k , k T
=

k =
1.0 =

patt

k ,
P
patt
kT k
mag

len(T )
P
Si
S = iF ,
len(F)

0 I(t),

S(t) N,

kT

(5.12b)
(5.12c)
(5.12d)
(5.12e)
(5.12f)
(5.12g)

kT k
len(T )

0 (y(t)),

(5.12h)
0 (t)

(5.12i)

In this model, S represents the number of people susceptible to the disease, I denotes the number of people
with the disease who are infectious, and denotes the cumulative incidence of the disease. The R? quantity
corresponds to the reported incidence of the disease and is a known input at a discrete set of reporting times.
For a more detailed description of this model, we refer the reader to [30].
The purpose of this model is to estimate the time-varying disease transmission parameter under the
assumption that varies seasonally but is constant from year to year (controlling for season). This hypothesis
induces the following structural elements to the temporal data: T , the set of reporting times over a single
year, and F, the set of all reporting times over a 20 year period. The set F is represented by the pyomo.dae
continuous set model.TIME. The following code fragment below shows how equation (5.12d) is implemented
accounting for these two time sets:
Listing 23 Code for declaring a subset of the components and implementing the differential equation (5.12d) in the disease transmission
model
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

model . TIME = ContinuousSet ( i n i t i a l i z e = T I M E i n i t , bounds = ( 0 , None ) )

model . b e t a = Var ( model . S BETA , i n i t i a l i z e = i n i t b e t a , bounds = ( 0 . 0 1 , 5 ) )
model . S = Var ( model . TIME , i n i t i a l i z e = i n i t S , bounds= people bounds )
model . I = Var ( model . TIME , i n i t i a l i z e = i n i t I , bounds= people bounds )
model . phi = Var ( model . TIME , i n i t i a l i z e = i n i t p h i , bounds = ( 0 , None ) )
model . Sdot = D e r i v a t i v e V a r ( model . S , i n i t i a l i z e = i n i t S d o t )
model . I d o t = D e r i v a t i v e V a r ( model . I , i n i t i a l i z e = i n i t I d o t )
model . phidot = D e r i v a t i v e V a r ( model . phi , i n i t i a l i z e = i n i t p h i d o t , bounds =( 10 ,None ) )
def p h i d o t e q ( model , i ) :
i f i == 0 :
r e t u r n C o n s t r a i n t . Skip
f e = model . TIME . get upper element boundary ( i )
j = model . TIME . f e . index ( f e )
r e t u r n model . phidot [ i ] == model . e p s I [ j ] \
+ ( model . b e t a [ model . P BETA NDX [ j ] ] model . I [ i ] model . S [ i ] ) / model . P POP
model . p h i d o t e q = C o n s t r a i n t ( model . TIME , r u l e = p h i d o t e q )

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

Creation Time (sec)

Solve Time (CPU sec)
Ipopt Iterations
Objective (105 )

Manual Discretization
1.38
2.65
27
1.4716

23

Using pyomo.dae
2.20
2.57
27
1.4716

Table 2 Comparison of two implementations of the disease transmission parameter estimation problem

In this example, lines 12 and 13 take a time point i F and determine the corresponding upper finite element
boundary fe. The parameter model.P BETA NDX then maps this finite element value to the correct index in
T for model.beta.
Our infectious disease transmission model has three differential equations and was discretized with 520
finite elements and 3 collocation points per finite element, using orthogonal collocation with Lagrange polynomials and Radau roots. This leads to an non-linear program with 10,458 variables and 9,910 equality constraints. The model was discretized such that all finite element points correspond to a reporting time for R? .
As with our previous applications example, the model was implemented twice: manually, as reported in [30],
and using pyomo.dae, by the authors. Table 2 compares timing and solution statistics for the two implementations. The creation time refers to the amount of time required to execute the python script that constructs the
model. For the implementation using pyomo.dae, the creation time also includes the time for automatic discretization. If we compare the creation time for the two implementations, we observe no significant differences
implying that there is relatively little overhead associated with applying the discretization automatically.
Comparing the solver results for the two implementations, we see analogous results. In particular, the solve
time for the two models is practically identical, and Ipopt requires the same number of iterations to locate an
optimal solution. While the model creation time results reported in Table 2 show minimal differences between
the two implementations, the statistics do not account for one crucial component of the applications development process: the time required to design, implement, debug, and refine the manual discretization. Initial
manual implementation and debugging of a discretization scheme like collocation can easily take over an hour
for novice modelers, for a model with a single differential equation. In contrast, incorporating a differential
equation using pyomo.dae into an optimization model is as simple as writing a regular model constraint and
selecting a particular discretization scheme to apply can be accomplished in a fraction of the time.
Finally, we consider the issue of model initialization, which is a critical step in solving optimization models
with differential equations. The following code fragment shows how the variable S is initialized in our disease
transmission model:
Listing 24 Code for initializing S in the disease transmission model
1
2
3
4
5
6
7
8
9

def

i n i t S ( model , i ) :
i f i ==0:
r e t u r n model . i n i t S i n i t
f e = model . TIME . get upper element boundary ( i )
j = model . TIME . f e . index ( f e )
r e t u r n model . i n i t S [ j ]
def people bounds ( model ) :
r e t u r n ( 0 . 0 , model . P POP )
model . S = Var ( model . TIME , i n i t i a l i z e = i n i t S , bounds= people bounds )

This example is only one illustration of how a pyomo.dae model can be initialized. However, other alternatives are available. In particular, the approach presented above initializes the finite element points using
a profile defined by the time-varying parameter model.init S and initializes the intermediate collocation
points by equating them to the value at the upper finite element boundary. We chose this initialization approach because it is sufficiently flexible to handle discretizations with any number of collocation points. The
pyomo.dae package does not currently provide any automatic frameworks for model initialization. However,
using existing Pyomo and Python constructs, a user can create any requisite initializations.

24

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

5.4 Stochastic Optimal Control of Natural Gas Networks

In our last application example, we consider the problem of optimizing natural gas network inventories while
accounting for uncertainties in the system. Specifically, the objective is to satisfy uncertain gas demands in the
network by building up inventory in the pipelines in a way that minimizes the required compression power.
A model of this problem was proposed in [31], which is as follows:
min

XX

cs si,t +

tT iSn

XX

XX

ce P`,t +

tT `L

s.t.

cd (dj,t dj,t )2

(5.13a)

tT jD

cT (p`,T,k p`,0,k )2 +

kX `L

XX

XX

cT (f`,T,k f`,0,k )2

(5.13b)

kX `L

p` (x, )
f` (x, )
= c1,`
, ` L, x [0, L` ], [0, T ]

x
p` (x, )
f` (x, )|f` (x, )|
f` (x, )
= c2,`
c3,`
, ` L, x [0, L` ], [0, T ]
t
x
p` (x, )

(5.13c)
(5.13d)

p` (L` , ) = rec(`) ( ), ` L, [0, T ]

(5.13e)

p` (0, ) = snd(`) ( ), ` Lp , [0, T ]

(5.13f)

p` (0, ) = snd(`) ( ) + ` ( ), ` La , [0, T ]

X
X
X
X
f` (L` , )
f` (0, ) +
si ( )
dj ( ) = 0, n N [0, T ]

(5.13g)

`Lrec
n

iSn

`Lsnd
n


P` ( ) = cp T f` (0, )

snd(`) ( ) + ` ( )
snd(`) ( )

(5.13h)

jDn

 1

!
1 , ` L, [0, T ]

sup(i), = isup , i S, [0, T ]

(5.13j)

f` (x, 0)
, ` L, x [0, L` ]
x
p` (x, 0)
f` (x, 0)|f` (x, 0)|
0 = c2,`
c3,`
, ` L, x [0, L` ]
x
p` (x, 0)
0 = c1,`

P`L P` ( ) P`U , ` La , [0, T ]

`suc,L snd(`) ( ) `suc,U , ` La , [0, T ]
`dis,L snd(`) ( ) + ` ( ) `dis,U , ` La ,

(5.13i)

(5.13k)
(5.13l)
(5.13m)
(5.13n)

[0, T ]

(5.13o)

This model includes detailed network dynamics captured by the PDEs (5.13c) and (5.13d). Note that these
PDEs are indexed by the number of links L in the network, indicating that optimization of even moderatelysized networks will require the simultaneous solution of many PDEs. Uncertainty is captured by considering
multiple scenarios for natural gas demand, and the resulting formulation is a two-stage stochastic optimization
problem. Because they describe scenario-specific dynamics, model PDEs are replicated for each scenario. The
resulting optimization model is very large-scale, scaling as a function of the number of network links, scenarios, and discretization points. The implementations in pyomo.dae for the PDEs and the pressure boundary
conditions for this model are given in Listings 25 and 26.
Listing 25 Code for implementing the PDE equations (5.13c) and (5.13d)
1
2
3
4
5
6

# F i r s t PDE f o r gas network model

def f l o w r u l e (m, j , i , t , k ) :
i f t == m. TIME . f i r s t ( ) or k == m. DIS . l a s t ( ) :
r e t u r n C o n s t r a i n t . Skip # Do not apply pde a t i n i t i a l time or f i n a l l o c a t i o n
r e t u r n m. dpxdt [ j , i , t , k ]/3600 + m. c1 [ i ]/m. l l e n g t h [ i ] m. dfxdx [ j , i , t , k ] == 0
model . flow = C o n s t r a i n t ( model . SCEN, model . LINK , model . TIME , model . DIS , r u l e = f l o w r u l e )

7
8
9

# Second PDE f o r gas network model

def p r e s s r u l e (m, j , i , t , k ) :

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations
10
11
12
13

25

i f t == m. TIME . f i r s t ( ) or k == m. DIS . l a s t ( ) :
r e t u r n C o n s t r a i n t . Skip # Do not apply pde a t i n i t i a l time or f i n a l l o c a t i o n
r e t u r n m. d f x d t [ j , i , t , k ]/3600 == m. c2 [ i ]/m. l l e n g t h [ i ] m. dpxdx [ j , i , t , k ] m. s l a c k [ j , i , t , k ]
model . p r e s s = C o n s t r a i n t ( model . SCEN, model . LINK , model . TIME , model . DIS , r u l e = p r e s s r u l e )

14
15
16
17
18
19

def s l a c k e q r u l e (m, j , i , t , k ) :
i f t == m. TIME . l a s t ( ) :
r e t u r n C o n s t r a i n t . Skip
r e t u r n m. s l a c k [ j , i , t , k ] m. px [ j , i , t , k ] == m. c3 [ i ] m. f x [ j , i , t , k ] m. f x [ j , i , t , k ]
model . s l a c k e q = C o n s t r a i n t ( model . SCEN, model . LINK , model . TIME , model . DIS , r u l e = s l a c k e q r u l e )
Listing 26 Code for implementing the pressure boundary conditions (5.13e), (5.13f), and (5.13g)

1
2
3
4

# boundary c o n d i t i o n s p r e s s u r e , p a s s i v e l i n k s
def p r e s s p a s s t a r t r u l e (m, j , i , t ) :
r e t u r n m. px [ j , i , t ,m. DIS . f i r s t ( ) ] == m. p [ j ,m. l s t a r t l o c [ i ] , t ]
model . p r e s s p a s s t a r t = C o n s t r a i n t ( model . SCEN, model . LINK P , model . TIME , r u l e = p r e s s p a s s t a r t r u l e )

5
6
7
8

def p r e s s p a s e n d r u l e (m, j , i , t ) :
r e t u r n m. px [ j , i , t ,m. DIS . l a s t ( ) ] == m. p [ j ,m. l e n d l o c [ i ] , t ]
model . p r e s s p a s e n d = C o n s t r a i n t ( model . SCEN, model . LINK P , model . TIME , r u l e = p r e s s p a s e n d r u l e )

9
10
11
12
13

# boundary c o n d i t i o n s p r e s s u r e , a c t i v e l i n k s
def p r e s s a c t s t a r t r u l e (m, j , i , t ) :
r e t u r n m. px [ j , i , t ,m. DIS . f i r s t ( ) ] == m. p [ j ,m. l s t a r t l o c [ i ] , t ]+m. dp [ j , i , t ]
model . p r e s s a c t s t a r t = C o n s t r a i n t ( model . SCEN, model . LINK A , model . TIME , r u l e = p r e s s a c t s t a r t r u l e )

14
15
16
17

def p r e s s a c t e n d r u l e (m, j , i , t ) :
r e t u r n m. px [ j , i , t ,m. DIS . l a s t ( ) ] == m. p [ j ,m. l e n d l o c [ i ] , t ]
model . p r e s s a c t e n d = C o n s t r a i n t ( model . SCEN, model . LINK A , model . TIME , r u l e = p r e s s a c t e n d r u l e )

The model includes two continuous domains: temporal and spatial. The model was discretized in time using a backward finite difference scheme and was discretized in space using a forward finite difference scheme.
Discretization involved 47 time intervals and multiple numbers of spatial points. Table 3 reports the computational effort required for different spatial discretization resolutions. Figure 4 illustrates how the flow profile
converges as the number of discretization points (Nx ) increases. The largest problem solved had over 400,000
variables and constraints. Note that the creation time for the model is only a small fraction of the overall solution time. Again, the creation time includes the time to apply a discretization scheme automatically. We also
see that the creation time does not increase as dramatically as the solve time as the number of discretization
points is increases. This result suggests that the automatic discretization routines in pyomo.dae are scalable.
Table 3 Effect of spatial discretization resolution on computational performance
Scenarios
1
1
1
1
1
1
3
3
3

Nx
2
6
10
20
60
100
2
10
20

Variables
9686
25718
41750
81830
242150
402470
29056
125248
245488

Constraints
9144
25128
41112
81072
240912
400752
27872
123776
243656

Iterations
35
45
43
50
56
67
37
54
64

Creation (sec)
0.64
1.71
2.92
5.87
17.72
31.54
1.84
8.96
18.07

Solve (sec)
2.27
11.85
19.26
40.84
147.72
412.18
17.50
124.44
276.18

We note that the model reported in [31] was first implemented in Pyomo using a manual discretization
scheme. During the process of converting the manually discretized model to pyomo.dae we actually discovered an off-by-one error in the manual discretization. These sort of errors are easy to make and often difficult

26

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

Fig. 4 Optimal axial flow profile in a single-scenario gas network for different resolutions. Darker colors indicate more discretization
points.

to find, given the complexity of such models. In contrast, pyomo.dae provides a concise and automatic way
of discretizing the variables and constraints in a model.
We again highlight that pyomo.dae does not perform any model checking. Therefore, users must specify
a well-posed model and select discretization schemes that are appropriate for their model. In this particular
example, we highlight the need for the user to supress the enforcement of the PDE at certain domain boundaries (as shown in Listing 25) in order to ensure consistency between the specified boundary conditions and
the applied discretization scheme.

6 Conclusions and Future Directions

The level of modeling abstraction provided by pyomo.dae enables the development of novel formulations
and the implementation of generalized analysis frameworks. For example, consider common applications of
dynamic optimization, such as model predictive control (MPC). These applications require solution of a sequence of related optimization problems with differential equation components. Using pyomo.dae, users can
implement such strategies as general frameworks, independent of the system model to which they are applied.
In fact, a general implementation of MPC has already been implemented using this framework [28].
Furthermore, there is the strong potential for pyomo.dae to be used in multigrid / multiscale applications.
By separating the model from the discretization scheme, we enable general implementations of algorithms
requiring either (1) the sequential solution of problems with different discretization resolutions or (2) the ability
to communicate discretization information to specialized solvers.
In addition to the core modeling components available in any algebraic modeling language, Pyomo includes specialized packages for expressing generalized disjunctive programming (GDP) and stochastic programming (SP) problems. These extensions leverage the modular design of Pyomo and can be combined with
pyomo.dae. For example, Pyomo users can now easily represent GDP problems with differential equations
or SP problems incorporating a dynamic model in each scenario. By providing users a straightforward way
to represent such complex models, our goal is to ultimately enable the development of new algorithms for
solving them.
In the course of developing pyomo.dae, our design emphasis was on syntactic flexibility. This does not
imply that models with every conceivable differential equation can be solved using this package. Also, because

pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations

27

we are not enforcing specific structure(s) on the differential equations or restricting the order of the derivatives,
we do not check for modeling errors or consistency. Rather, the responsibility falls on the user not to implement
an inconsistent model. A similar observation also applies to general algebraic modeling languages in which
the user is responsible for not implementing models with incompatible constraints or empty feasible regions,
for instance. However, with the availability of pyomo.dae, it is now possible to develop templates that create
models with certain canonical structures, and as a result prevent users from specifying incompatible models.
Further, such templates would allow implementation of solution algorithms tailored for specific structures, to
check for modeling errors, and to more easily interface with other existing DAE solvers.
Currently pyomo.dae can only be used to express models with bounded rectangular continuous domains.
In the future we plan on extending the package to represent unbounded domains, making it easier to represent
control problems over infinite horizons. We also plan on adding a modeling component to represent integrals.
The pyomo.dae package includes a variety of discretization transformations as well as a framework for
users to implement their own custom discretization strategies. However, simultaneous discretization is just one
of several approaches available. Single and multiple shooting methods are other common solution strategies
[8, 29, 20]. We plan to eventually link pyomo.dae to existing integrators, such as Sundials [16], and then add
implementations of shooting methods. An integrator would also allow us to simulate dynamic models in order
to initialize our discretized optimization problem or develop hybrid discretization / shooting algorithms.
Another useful extension of pyomo.dae would involve the development of more sophisticated frameworks for model initialization and specification of time dependent data. In the current implementation a user
can provide an initialization for a model after it has been discretized. However, this is not entirely straightforward, and the initialization is tied to the discretization applied. We would also like to implement several
data interpolation schemes in order to make this initialization process easier and consistent across any choice
of discretization. An interpolation scheme would also be useful for estimating differential variable values at
any point in the continuous domain after the model has been solved, especially in the case of a collocation
discretization where a high order functional form of the state profile already exists. An automatic, element-byelement initialization would be another possible extension.
Acknowledgements We thank Carl D. Laird for useful technical discussions and for providing the disease transmission model. Victor
M. Zavala acknowledges funding from the U.S. Department of Energy Early Career program. The research was supported in part by the
U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program under
contract number KJ0401000 through the Project Multifaceted Mathematics for Complex Energy Systems. Sandia is a multi-program
laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energys National Nuclear
Security Administration under Contract DE-AC04-94-AL85000.

References

en, K.E., Gafvert, M., Bergdahl, T., Tummescheit, H.: Modeling and optimization with Optimica and JModelica.org
1. Akesson,
J., Arz
languages and tools for solving large-scale dynamic optimization problems. Computers & Chemical Engineering 34(11), 17371749
(2010)
2. Andersson, J.: A general-purpose software framework for dynamic optimization. PhD thesis, Arenberg Doctoral School, KU Leuven,
Department of Electrical Engineering (ESAT/SCD) and Optimization in Engineering Center, Kasteelpark Arenberg 10, 3001-Heverlee,
Belgium (2013)
3. Ascher, U.M., Mattheij, R.M., Russell, R.D.: Numerical solution of boundary value problems for ordinary differential equations, vol. 13.
SIAM (1994)
4. Ascher, U.M., Petzold, L.R.: Computer methods for ordinary differential equations and differential-algebraic equations, vol. 61. SIAM
(1998)
5. Betts, J.T.: Sparse Optimization Suite (SOS). Applied Mathematical Analysis, LLC (2013)
6. Betts, J.T., Huffman, W.P.: Sparse Optimal Control Software SOCS. Mathematics and Engineering Analysis Technical Document MEALR-085, Boeing Information and Support Services, The Boeing Company, PO Box 3707, 98,1242207 (1997)
7. Biegler, L.T.: Nonlinear Programming: Concepts, Algorithms, and Applications to Chemical Processes. SIAM (2010)
8. Bock, H.G., Plitt, K.J.: A multiple shooting algorithm for direct solution of optimal control problems. Proceedings of the IFAC World
Congress (1984)

28

Bethany Nicholson, John D. Siirola, Jean-Paul Watson, Victor M. Zavala, Lorenz T. Biegler

9. Brenan, K.E., Campbell, S.L., Petzold, L.R.: Numerical solution of initial-value problems in differential-algebraic equations, vol. 14.
SIAM (1996)
10. Butcher, J.: Numerical Methods for Ordinary Differential Equations. Wiley (2003). URL https://books.google.com/books?id=
nYuDWkxhDGUC
11. Corporation, G.D.: General Algebraic Modeling System (GAMS) Release 24.2.1. Washington, DC, USA (2013). URL http://www.
gams.com/
12. Fourer, R., Gay, D., Kernighan, B.: AMPL: A Modeling Language for Mathematical Programming. Scientific Press (1993). URL https:
//books.google.com/books?id=8vJQAAAAMAAJ
13. Hart, W., Laird, C., Watson, J., Woodruff, D.: Pyomooptimization modeling in Python, vol. 67. Springer Science & Business Media
(2012)
14. Hart, W., Watson, J., Woodruff, D.: Pyomo: modeling and solving mathematical programs in Python. Mathematical Programming
Computation 3(3), 219260 (2011)
15. Hendengren, J.: APMonitor modeling language (2014). URL http://APMonitor.com
16. Hindmarsh, A.C., Brown, P.N., Grant, K.E., Lee, S.L., Serban, R., Shumaker, D.E., Woodward, C.S.: SUNDIALS: Suite of nonlinear
and differential/algebraic equation solvers. ACM Trans. Math. Softw. 31(3), 363396 (2005). DOI 10.1145/1089014.1089020. URL
http://doi.acm.org/10.1145/1089014.1089020
17. Houska, B., Ferreau, H., Diehl, M.: ACADO Toolkit an open source framework for automatic control and dynamic optimization.
Optimal Control Applications and Methods 32(3), 298312 (2011)
18. Hultberg, T.: FlopC++ an algebraic modeling language embedded in C. In: in Operations Research Proceedings 2006, ser. Operations
Research Proceedings, K.-H. Waldmann and, pp. 187190. Springer (2006)
19. Jacobson, D., Lele, M.: A transformation technique for optimal control problems with a state variable inequality constraint. Automatic
Control, IEEE Transactions on 14(5), 457464 (1969). DOI 10.1109/TAC.1969.1099283
20. Kraft, D.: On converting optimal control problems into nonlinear programming problems. In: Computational mathematical programming, pp. 261280. Springer (1985)
21. Leyffer, S., Kirches, C.: TACO - a toolkit for AMPL control optimization. Mathematical Programming Computation pp. 139 (2013)
22. Lubin, M., Dunning, I.: Computing in operations research using julia. INFORMS Journal on Computing 27(2), 238248 (2015). DOI
10.1287/ijoc.2014.0623. URL http://dx.doi.org/10.1287/ijoc.2014.0623
23. MATLAB: MATLAB function reference, chap. pdepe, pp. 63866397. The MathWorks Incorporated (2015)
24. Mitchell, S., OSullivan, M., Dunning, I.: PuLP: A linear programming toolkit for Python (2011)
25. Patterson, M.A., Rao, A.V.: GPOPS-II: A MATLAB software for solving multiple-phase optimal control problems using hp-adaptive
gaussian quadrature collocation methods and sparse nonlinear programming. ACM Trans. Math. Softw. 41(1), 1:11:37 (2014). DOI
10.1145/2558904. URL http://doi.acm.org/10.1145/2558904
26. Process Systems Enterprise: gPROMS (1997-2014). URL www.psenterprise.com/gproms
27. Rutquist, P., Edvall, M.: PROPT Matlab optimal control software. Tomlab Optimization Inc (2010)
28. Santamara, F.L., Gmez, J.M.: Framework in {PYOMO} for the assessment and implementation of (as)nmpc controllers. Computers & Chemical Engineering 92, 93 111 (2016). DOI http://dx.doi.org/10.1016/j.compchemeng.2016.05.005. URL http:
//www.sciencedirect.com/science/article/pii/S0098135416301533
29. Sargent, R., Sullivan, G.: The development of an efficient optimal control package. In: Optimization Techniques, pp. 158168. Springer
(1978)
30. Word, D.P.: Nonlinear programming approaches for efficient large-scale parameter estimation with applications in epidemiology. PhD
thesis, Texas A&M University, Department of Chemical Engineering, College Station, TX (2013)
31. Zavala, V.M.: Stochastic optimal control model for natural gas networks. Computers & Chemical Engineering 64, 103
113 (2014). DOI http://dx.doi.org/10.1016/j.compchemeng.2014.02.002. URL http://www.sciencedirect.com/science/
article/pii/S0098135414000349