This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAC.2020.2976042, IEEE
Transactions on Automatic Control

An Augmented Lagrangian for Optimal Control

of DAE Systems: Algorithm and Properties
Marco Aurelio Aguiar, Eduardo Camponogara, and Bjarne Foss

Abstract—This work proposes a relax-and-discretize ap- a distributed solution scheme, such a decoupling allows
proach for optimal control of continuous-time differential subsystem problems to be solved with distinct approaches,
algebraic systems (DAE). It works by relaxing the algebraic unlike discretize-and-relax approaches. The paper illus-
equations and penalizing the violation into the objective
function using the augmented Lagrangian, which converts the trates the computational and implementation aspects of
original problem into a sequence of optimal control problems the algorithm in a simple, but representative example.
(OCPs) of ordinary differential equations (ODEs). The relax-
and-discretize approach brings about flexibility, by allowing II. OPTIMAL CONTROL PROBLEM
the OCPs of ODEs to be solved by the method of choice, This work is concerned with the optimal control problem
such as direct or indirect methods. Conditions are developed
for global, local, and sub-optimal convergence in terms of for a system of differential-algebraic equations of the form:
the solution of the underlying OCPs. The method is applied
to an illustrative example.
Z tf
P: min J(x, y, u) = L(x, y, u, t) dt (1a)
Index Terms—Optimal control, Differential-Algebraic Sys- t0
tems, Nonlinear systems, Optimization algorithms.
s.t.: ẋ = f (x, y, u, t) (1b)
g(x, y, u, t) = 0 (1c)
I. INTRODUCTION
x(t0 ) = x0 (1d)
The augmented Lagrangian method is well established in u(t) ∈ UB , t ∈ [t0 , tf ] (1e)
constrained optimization, arguably because of its efficient
algorithms and strong theory [1]. It has enjoyed diverse with UB = {u ∈ U | uL ≤ u ≤ uU } and where
Nx
applications that include model predictive control [2], x(t) ∈ X = R is the state variable, y(t) ∈ Y = RNy
distributed and parallel optimization [3], to name a few. is the algebraic variable, u(t) ∈ UB ⊂ U = RNu is the
For optimal control, an equivalent method has been control variable, and t is the time variable. The function of
proposed and applied to some academic systems [4], [5]. dynamics f , the function of algebraic relations g, and the
The algorithms thereof relax the algebraic equations of an function of dynamic cost L are assumed to be continuously
optimal control problem (OCP) with a system of a differ- differentiable with respect to their arguments.
ential algebraic equations (DAE). However, little effort has The DAE system formed by (1b) and (1c) is assumed
been put into developing a theoretical endorsement for such to be in the semi-explicit index-1 form, which means that
∂g
algorithms. An exception is [6], which provides conditions it is solvable for y and the Jacobian ∂y is invertible. The
that ensure converge for problems with a convex objective algebraic equation can also be used to model equality
and a linear system. constraints, e.g. u 1 + u 2 = 0, where either u1 or u2 can
The contributions are as follows. This paper proposes be represented as an algebraic variable y. Problems of the
an algorithm for solving optimal control problems of form P with a final cost function can be framed to this
continuous-time DAE systems, providing conditions for approach by transforming the objective [7].
global and local convergence, and convergence with sub- The Hamiltonian function of the OCP (1) is
optimal iterations. This algorithm facilitates the solution
H(x, λ, y, ν, u, t) = L(x, y, u, t) + λT f (x, y, u, t)
of OCPs of DAEs in embedded hardware with limited
computational power by eliminating the need of a DAE + ν T g(x, y, u, t) (2)
solver. For a network system with subsystems coupled by where λ : [t0 , tf ] → RNx is the multiplier associated
algebraic input-output equations, the proposed algorithm with the state equations, and ν(t) ∈ RNy is the multiplier
enables the decoupling by relaxing these equations. In associated with the algebraic equations.
Using this Hamiltonian, the necessary conditions for
Funded by an INTPart grant from the Research Council of Norway (x∗ , λ∗ , y ∗ , ν ∗ , u∗ ) to be optimal are given by [8]:
and a doctoral scholarship from CAPES/Brasil.
M. de Aguiar and E. Camponogara are with the Department of Au- ∂H T ∂L T ∂f T ∗ ∂g T ∗
tomation and Systems Engineering, Federal University of Santa Catarina, = −λ̇∗ = + λ + ν (3a)
∂x ∂x ∂x ∂x
Florianópolis, Brazil, e-mail: marcoaaguiar@gmail T T T T
∂H ∂L ∂f ∗ ∂g ∗
B. Foss is with the Engineering Cybernetics Department, Norwegian = + λ + ν =0 (3b)
University of Science and Technology, Trondheim, Norway ∂y ∂y ∂y ∂y

0018-9286 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Otago. Downloaded on May 30,2020 at 23:54:30 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAC.2020.2976042, IEEE
Transactions on Automatic Control

u∗ (t) = arg min H(x∗ (t), λ∗ (t), y ∗ (t), ν ∗ (t), u, t) Algorithm 1 Augmented Lagrangian for Optimal Control
(3c)
u∈UB
Require: µ0 , ν0 , and εg :
∂H T ∗ ∗ ∗ ∗ ∗ 1: for k = 1, 2, . . . do
= ẋ = f (x , y , u ), λ (tf ) = 0 (3d)
∂λ 2: (Jk , xk , yk , uk ) ← solve{PL (µk , νk )}
∂H T ∗ ∗ ∗ ∗ 3: νk+1 ← νk + µk g(xk , yk , uk )
= g(x , y , u ) = 0, x (t0 ) = x0 (3e)
∂ν 4: µk+1 ← update mu{µk }
for all t ∈ [t0 , tf ]. More general necessary conditions are 5: if kg(xk , yk , uk )k < εg then
found in [9] which concerns OCPs with mixed constraints. 6: return uk
Less restrictive conditions were recently developed for pure 7: end if
and mixed constraints in [10]. 8: end for
Methods that solve the boundary value problem (BVP)
resulting of the necessary optimality conditions (3) are
known as indirect methods [11]. The proposed relax-and- the violation of the algebraic constraint. Starting with
discretize algorithm is presented and its properties are these parameters, at each iteration k, the problem (6) is
demonstrated, both of which make use of the Hamiltonian. solved, the multiplier estimate and penalty are updated,
and the process is repeated until an acceptable tolerance
I I I . AU G M E N T E D L AG R A N G I A N A L G O R I T H M is achieved, as detailed in Algorithm 1.
The pseudo-function solve yields a solution for the sub-
The algorithm proposed in this work solves the OCP in problem P and returns the functional values J and the
L k
the form P by first relaxing the algebraic constraint (1c), trajectories for the states, algebraic and control variables.
and then introducing a new objective functional, The pseudo-function update mu represents the use of an
Z tf
update rule for the penalization µk . For the convergence
Jµ (x, y, u, ν) = Lµ (x, y, u, ν, t) dt (4) analysis it is assumed that µ
k+1 = βµk with a β > 1
t0
to ensure that µk → ∞. In practice, however, a µk →
where the function Lµ is defined by ∞ will cause ill-conditioning on the Hessian of the sub-
problem PL , therefore when performing a computational
Lµ (x, y, u, ν, t) = L(x, y, u, t) + ν(t)T g(x, y, u, t)
implementation, it is recommended to use an upper bound
µ 2
+ kg(x, y, u, t)k , (5) µmax for the penalization.
2
where µ > 0 is a scalar, and the function ν : [t0 , tf ] → B. Mathematical Properties
RNy is an estimate of the multiplier function ν ∗ , which will
Conditions are now established for the solution sequence
be driven by the algorithm towards satisfying the optimality
produced by the algorithm to arrive at a global solution
conditions (3) of problem P.
of the OCP of DAE. Less restrictive conditions are then
The functional (4) is the objective of the auxiliary OCP
presented for convergence to local solutions and conver-
solved by the algorithm at each iteration k, given by
Z tf gence under a suboptimal solution sequence, which reflect
situations typically found in practice.
PL (µk , νk ) : min Jµk = Lµk (x, y, u, νk , t) dt (6a)
y,u t0
Before presenting the convergence theorems, some defi-
s.t.: ẋ = f (x, y, u, t) (6b) nitions are in order.
x(t0 ) = x0 (6c) Assumption 1 (Regularity). For problem P (1) and
u ∈ UB , t ∈ [t0 , tf ] (6d) PL (µk , νk ) (6) to be well-conditioned, we assume that
1) x : [t0 , tf ] → RNx is continuously differentiable;
Notice that without an algebraic equation, the variable y is y : [t0 , tf ] → RNy , u : [t0 , tf ] → UB , and νk :
free to be optimized. In this sense, the algebraic variable [t0 , tf ] → RNy are continuous,
plays the same role as the control variable u. Therefore, 2) L, g, and f are continuously differentiable with
an extended control variable u b = [u, y] can be defined, respect to all the arguments,
where ub(t) ∈ Ub = UB × Y . Using u b, problem PL meets 3) The space of feasible functions for problems P and
the standard form of an OCP of ODE, whose optimality PL are compact,
conditions are well established [7]. ∂g
4) the Jacobian ∂y (x(t), y(t), u(t), t) has full rank for
all x(t) ∈ X, y(t) ∈ Y , u(t) ∈ UB , and t ∈ [t0 , tf ],
A. Algorithm 5) the sequence {µk } has the property that 0 < µk <
µk+1 for all k, and µk → ∞ as k → ∞,
The proposed algorithm follows the same structure
6) problem P and PL (µk , νk ) are solvable.
of the augmented Lagrangian for standard constrained
optimization [1]. Let µ0 be an initial value for the sequence From condition 4) of the Assumption 1, the algorithm
of penalty values {µk }, ν0 be an initial estimate for the is not applicable to OCP with DAE of index greater than
sequence of multipliers {νk }, and εg be a tolerance on one.

0018-9286 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Otago. Downloaded on May 30,2020 at 23:54:30 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAC.2020.2976042, IEEE
Transactions on Automatic Control

The following theorems will make use of uniform con- Any solution to problem PL satisfies all of the con-
vergence and uniform norm for functions, their definitions straints of P except the relaxed algebraic equations. How-
are given in Appendix A. ever (11) ensures that the limiting functions x∗ , y ∗ , and
u∗ do satisfy the algebraic equation. By definition, J ∗ is
Theorem 1. Let the functions hxk , yk , uk i be global
less or equal to the objective of any feasible functions for
minima of the problem PL (µk , νk ) (Eq. 6) at each iteration
problem P, therefore we have
k. In addition, assume that {hxk , yk , uk i}K and {νk }K
are uniformly convergent subsequences. Then, under As-
sumption 1, the limiting functions of every subsequences J ∗ ≤ J(x∗ , y ∗ , u∗ ) (13)
{hxk , yk , uk i}K are a global minimizer of problem P and
the subsequence {Jµk (xk , yk , uk , νk )}K converges to the Using (12) and (13), we conclude that
optimum objective of P.
Proof: Let hx∗ , y ∗ , u∗ i be limiting functions of the J ∗ ≤ J(x∗ , y ∗ , u∗ ) ≤ J ∗ =⇒ J ∗ = J(x∗ , y ∗ , u∗ ) (14)
subsequence {hxk , yk , uk i}K . By definition of xk , yk , and
uk , for a given k which proves that the limiting functions x∗ , y ∗ , and
u∗ are global minimizers for problem P and that
Jµk (xk , yk , uk , νk ) ≤ Jµk (x, y, u, νk ) (7)
{Jµk (xk , yk , uk , νk )}K → J ∗ .
for all feasible x, y, and u.
Let J ∗ denote the optimal value of P. We have that
Definition 1. Let V be a function space, then a nonempty
J∗ = min
u
J= min
y,u
Jµk (µk , νk ) (8) set V ∗ ⊂ V is said to be an isolated set of local minima of
s.t. (1b)-(1e) s.t. (6b)-(6d) problem P if each function v ∗ ∈ V ∗ is a local minimum
g(x,y,u,t)=0
of problem P and, for some ε > 0, the set
the last term implies the minimization of the problem PL
over y and u with the additional equation g(x, y, u, t) = 0. Vε∗ = {v ∈ V : kv − v ∗ k ≤ ε for some v ∗ ∈ V ∗ } (15)
The first equality holds by definition. The second equality
holds because P and PL are equivalent when the equation contains no local minima of problem P other than the
g(x, y, u, t) = 0 is included in PL . functions of V ∗ .
The inequality (7) holds for any x, y, and u, including a
minimizer of (8). Therefore, we can substitute the optimum An isolated set of local minima consisting of a single
value J ∗ on the right-hand side of (7), and on the left-hand function is a strict local minimum.
side we substitute Jµk (xk , yk , uk , νk ) with its definition to
obtain Theorem 2. Suppose that the regularity Assumption 1
Z tf holds, and that V ∗ is a compact and isolated set of
local minima of problem P. If hxk , yk , uk i is a local
L(xk , yk , uk , t) + νkT g(xk , yk , uk , t)
t0 minimizer for problem PL for each k, then there exists
µk 2 a subsequence {hxk , yk , uk i}K converging to a limiting
+ kg(xk , yk , uk , t)k dt ≤ J ∗ (9)
2 function hx∗ , y ∗ , u∗ i ∈ V ∗ . Furthermore, if V ∗ consists of
∗ ∗ ∗
Given that the subsequence {νk }K is uniformly conver- a single function hx , y , u i, then there exists∗ a ∗sequence ∗
gent, it has a limiting function ν ∗ . By taking the limit with {hxk , yk , uk i} such that {hxk , yk , uk i} → hx , y , u i.
k → ∞ in the inequality (9) we obtain Proof: Consider the set
Z tf
L(x∗ , y ∗ , u∗ , t) + ν ∗T g(x∗ , y ∗ , u∗ , t) dt
 
Vε̃∗ = {v ∈ V : kv − v ∗ k ≤ ε̃ for some v ∗ ∈ V ∗ } (16)
t0
µk tf
Z
+ lim kg(xk , yk , uk , t)k2 dt ≤ J ∗ (10) where V is the set of feasible functions of PL , with some
k→∞ 2 t0 0 < ε̃ < ε, and ε is as in (15). From (16) and because
Since kg(xk , yk , uk , t)k2 ≥ 0 and µk → ∞, it follows that V is compact by Assumption 1, it follows that Vε̃∗ is also
we must have g(xk , yk , uk , t) → 0 and compact, and hence the problem

g(x∗ , y ∗ , u∗ , t) = 0 ∀t ∈ [t0 , tf ] (11) Z tf

min Jµk = Lµk (x, y, u, νk , t) dt (17a)
otherwise the limit on the left-hand side of (10) would go x,y,u t0
to +∞ which does not hold since J ∗ is finite. Therefore, s.t.: ẋ = f (x, y, u, t) ∀t ∈ [t0 , tf ] (17b)
Z tf
∗ ∗ ∗
u(t) ∈ UB ∀t ∈ [t0 , tf ] (17c)
J(x , y , u ) = L(x∗ , y ∗ , u∗ , t) dt ≤ J ∗ (12)
t0 hx, y, ui ∈ Vε̃∗ , x(t0 ) = x0 (17d)

0018-9286 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Otago. Downloaded on May 30,2020 at 23:54:30 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAC.2020.2976042, IEEE
Transactions on Automatic Control

has a global minimum hxk , yk , uk i ∈ Vε̃∗ . By Theorem 1, and with respect to λ, u, and x are
every limiting function hx∗ , y ∗ , u∗ i of {hxk , yk , uk i}K is
∂L ∗ ∗ ∗ T ∂f ∗ ∗ ∗ T ∗
a global minimum of the problem −λ̇∗ = (x , y , u , t) + (x , y , u , t) λ
∂x ∂x
Z tf ∂g ∗ ∗ ∗ T ∗
min J = L(x, y, u, t) dt (18a) + (x , y , u , t) νe (20b)
x,y,u ∂x
t0
u∗ (t) = arg inf H(x∗ (t), λ∗ (t), y ∗ , ν ∗ , u, t) (20c)
s.t.: ẋ = f (x, y, u, t) ∀t ∈ [t0 , tf ] (18b) u∈UB
∗ ∗ ∗ ∗
g(x, y, u, t) = 0 ∀t ∈ [t0 , tf ] (18c) ẋ = f (x , y , u , t). (20d)
u(t) ∈ UB ∀t ∈ [t0 , tf ] (18d) Proof: The derivative of Lµk w.r.t. y results in
hx, y, ui ∈ Vε̃∗ , x(t0 ) = x0 (18e)
∂Lµk ∂L
Furthermore, each global minimum of the problem above (xk , yk , uk , νk , t) = (xk , yk , uk , t)
∂y ∂y
must belong to V ∗ by the definition of Vε̃∗ . Thus there is a
T ∂g
subsequence {hxk , yk , uk i}K converging to hx∗ , y ∗ , u∗ i ∈ + [νk + µk g(xk , yk , uk , t)] (xk , yk , uk , t) (21)
∂y
V ∗ . If V ∗ contains only one local optimum, then all the
subsequences will lead to this local optimum, therefore Then, by defining for all k
{hxk , yk , uk i} → hx∗ , y ∗ , u∗ i ∈ V ∗ .
From a practical point of view, numerical methods are νek = νk + µk g(xk , yk , uk , t) (22)
expected to terminate when the optimality conditions of
PL are almost satisfied, meaning that for a small scalar replacing νek into (21) results in
εk > 0 the necessary optimality conditions [7] are
∂Lµk ∂L
kf (xk , yk , uk , t) − ẋk ≤ εk , (19a)
(xk , yk , uk , νk , t) = (xk , yk , uk , t)
∂y ∂y
∂g
+ νekT (xk , yk , uk , t). (23)
∂Lµk ∂y
(xk , yk , uk , νk , t)T +
∂x ∂g
Since ∂y is invertible, we can derive the following expres-
∂f sion for νek ,
+ (xk , yk , uk , t)T λk + λ̇k ≤ εk , (19b)
∂x
 −1 
∂g ∂Lµk
νek = (xk , yk , uk , t)T (xk , yk , uk , νk , t)T
uk (t) − arg inf H(xk (t), λk (t), yk , νk , u, t) ≤ εk (19c) ∂y ∂y
u∈UB 
∂L
− (xk , yk , uk , t)T (24)
∂Lµk ∂f ∂y
(xk , yk , uk , νk , t)T + (xk , yk , uk , t)T λk ≤ εk .
∂y ∂y From (24) we can say that there exists an F such that
(19d)
νek = F (xk , yk , uk , νk ) (25)
The following theorem shows that if εk → 0, the
algorithm still converges. which is continuous since all the functions in (24) are
continuous. Given that a subsequence {hxk , yk , uk i}K con-
Theorem 3. Suppose that Assumption 1 holds and verges to hx∗ , y ∗ , u∗ i and {νk } converges to ν ∗ , Theorem
let hxk , yk , uk i be a suboptimal solution obtained for 4 (from Appendix A) is invoked to conclude that
PL (µk , νk ) such that the violation of the optimality con-
ditions are given by (19), for which inequality (19d) is {e νk = F (xk , yk , uk , νk )}K → νe∗ = F (x∗ , y ∗ , u∗ , ν ∗ )
fundamental, where 0 ≤ εk , and εk → 0 as k → ∞, {νk } (26)
is a uniform convergent sequence, and λk is the costate at ∗
the k-th algorithm iteration. Assume that a subsequence which shows ∗that {νk + µk g(xk , yk , uk , t)}K → νe uni-
{hxk , yk , uk i}K converges uniformly to hx∗ , y ∗ , u∗ i such formly, and νe is given by
∂g
that ∂y (x∗ , y ∗ , u∗ , t) has full rank and is bounded for all  −1 
∗ ∂g ∗ ∗ ∗ T ∂Lµ∗ ∗ ∗ ∗ ∗ T
t ∈ [t0 , tf ]. νe = (x , y , u , t) (x , y , u , ν , t)
Then the subsequence {νk + µk g(xk , yk , uk , t)}K con- ∂y ∂y

verges uniformly to νe∗ , such that the following relations ∂L ∗ ∗ ∗ T
− (x , y , u , t) . (27)
are obtained, with respect to y ∂y

∂L ∗ ∗ ∗ T ∂f ∗ ∗ ∗ T ∗ Considering the optimality conditions for y, given in

(x , y , u , t) + (x , y , u , t) λ (19d), and taking the limit k → ∞, we obtain
∂y ∂y
∂g ∗ ∗ ∗ T ∗ ∂Lµ∗ ∗ ∗ ∗ ∗ ∂f ∗ ∗ ∗
+ (x , y , u , t) νe = 0 (20a) (x , y , u , ν , t) = −λ∗T (x , y , u , t) (28)
∂y ∂y ∂y

0018-9286 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Otago. Downloaded on May 30,2020 at 23:54:30 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAC.2020.2976042, IEEE
Transactions on Automatic Control

which can be substituted into (27) to obtain The objective of the controller is to stabilize the tanks
 −1  1 and 2, while reducing the variation in the pump flows,
∂g ∗ ∗ ∗ T ∂L ∗ ∗ ∗ T which is expressed by the following objective
νe∗ = (x , y , u , t) − (x , y , u , t)
∂y ∂y Z tf
∆xT ∆x + uT u dt

∂f ∗ ∗ ∗ T ∗ min J = (34)
− (x , y , u , t) λ (29) u t0
∂y
with ∆x = x − xref , x = [h1 , h2 , h3 , h4 , qp,1 , qp,2 ], and
which can be rearranged into
u = [δ1 , δ2 ].
∂L ∗ ∗ ∗ ∂f ∗ ∗ ∗
(x , y , u , t) + λ∗T (x , y , u , t)
∂y ∂y B. Applying the Augmented Lagrangian
∂g ∗ ∗ ∗
+ νe∗T (x , y , u , t) = 0 (30) By using the algorithm to relax the algebraic equation
∂y (31), the following relaxed problem is obtained
and related to the necessary conditions (3d) of the original
min Jµk , s.t.: eq. (32) and (33) (35)
OCP P. Similar approach can be used to obtain the u,y
conditions for x, u, and λ.
where:
Since the sequence {νk } is bounded and {νk +
µk g(xk , yk , uk , t)}K → νe∗ from (26), it follows that Z tf X4   p 
T T
{µk g(xk , yk , uk , t)}K is bounded. Given that µk → ∞ we Jµk = ∆x ∆x+u u+ νi,k qt,i − ai 2ghi
t0
must have g(xk , yk , uk , t) → 0 with g(x∗ , y ∗ , u∗ , t) = 0 i=1

µk 2
for all t.
p
+ qt,i − ai 2ghi dt (36)
Notice that the sequence {νk } was never specified, other 2
than it is a uniformly convergent sequence. From Theorem At each algorithm iteration, the relaxed problem (35)
3, an update rule can be derived such that {νk } → νe∗ . is solved and the solution is used to compute the new
Corollary 1. By defining νk+1 = νk + µk g(xk , yk , uk , t) multiplier estimates νi,k+1 according to the rule
we have that {νk } → νe∗ and {µk g(xk , yk , uk , t)} → 0. h p i
νi,k+1 = νi,k + µk qt,i − ai 2ghi (37)
Proof: For any uniformly convergent sequence {νk },
Theorem 3 ensures that {νk + µk g(xk , yk , uk , t)} → νe∗ . As discussed in [4], since νi,k is a function that can
Therefore, we can define νk+1 = νk + µk g(xk , yk , uk , t), assume any shape, a piecewise polynomial approximation
which makes the sequence become {νk+1 } → νe∗ . with a finite number of terms is used instead. For this
application, the Lagrangian polynomial was chosen as it
facilitates the computation of updates.
I V. C O M P U TAT I O N A L D E TA I L S A N D
EXPERIMENTS
C. Computational Experiments
To illustrate the algorithm behavior and to remark some
implementation details, the algorithm is applied to the To solve the relaxed subproblem, an indirect collocation
optimal control problem of stabilizing a four-tank system. method with polynomials of order 3 was used, discretized
in 30 finite elements and implemented using YAOCP-
Tool and CasADi [12]. The same settings were used to
A. Problem Modeling
solve the original problem (Eqs. (31)-(34)) for comparison
Instead of modeling the four tank as an ODE system as purpose. The resulting nonlinear programming problems
it is commonly done, this paper represents the process as a were solved with the IPOPT solver. By using indirect
DAE system by using an algebraic variable for the outflow methods, the multipliers are easily obtained which allow us
of each tank. For every tank i, the outflow is given by to compare with the estimate νi yielded by our algorithm.
p The multiplier estimates νi are also approximated with a
qt,i = ai 2ghi . (31)
piecewise polynomial of degree 3 with 30 finite elements.
where g is the gravity constant, ai is the cross section area Since no information on the multipliers is available the
of the orifice and hi is the fluid level of tank i, given by algorithm is initialized with the multiplier estimates as zero
q3 + γ1 qp,1 − qt,1 q4 + γ2 qp,2 − qt,2 (ν0 = 0 for all t ∈ [t0 , tf ]). The penalization term starts
ḣ1 = , ḣ2 = (32a)
A1 A2 with µ0 = 0.1 and increases at a rate β = 4.
(1 − γ2 )qp,2 − qt,3 (1 − γ1 )qp,1 − qt,4 Although not shown here, the trajectories of the proposed
ḣ3 = , ḣ4 = (32b)
A3 A4 algorithm coincide with the optimal trajectories obtained
where Ai is the cross section area of the tank i, and the by the indirect method. To evaluate if the algorithm is
flow on each pump j is given by the differential equation converging to the optimal solution of the original problem,
in Fig. 1, the relaxed objective (Jµk ) and the evaluation
q̇p,j = δj . (33) of the solution iteration on the original objective (J ) are
k

0018-9286 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Otago. Downloaded on May 30,2020 at 23:54:30 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAC.2020.2976042, IEEE
Transactions on Automatic Control

The proposed algorithm can be deployed for optimal

control in applications where DAE solvers are too costly,
which are supported by the convergence conditions estab-
lished heretofore. As future work, the structure of certain
problems could be exploited in the augmented Lagrangian
to enable distributed and parallel computations.
APPENDIX A
Definition 2. Let f : [t0 , tf ] → RN be a continuous
Figure 1. Comparison of the objective functions. function then kf k is given by kf k = maxt∈[t0 ,tf ] kf (t)k∞ .
Definition 3. Let fk : [t0 , tf ] → RN be a function for
every k ∈ N. The sequence of functions {fk } converges
uniformly to the limiting function f ∗ : [t0 , tf ] → RN if, for
every ε > 0, there exists a number K ∈ N such that for all
t ∈ [t0 , tf ] and all k ≥ K, we have kfk (t) − f ∗ (t)k < ε.
Theorem 4. Let g : Rd1 → Rd2 be a continuous function,
and the sequence of functions {fn } to converge uniformly
Figure 2. Convergence of the algebraic function to zero (red) and the to f , where fn : [0, 1] → Rd1 . Let the function norm k·k
multiplier estimate converging to the original problem multiplier (blue). be given by kgk = maxx∈[0,1] kg(x)k∞ . Then {g(fn )}
converges uniformly to g(f ).

compared to the optimal cost obtained with the indirect Proof: If fn converges uniformly to f , then for all
method (J ∗ ). It can be seen that the objectives converge εf exists N , such that kfn − f k < εf for all n > N , and
to the same objective value J ∗ as the indirect method. As exists an upper bound M s.t. kfn k ≤ M for all n ∈ N.
for the violation of algebraic equations, the line in blue Then, consider g : [−M, M ]d1 → Rd2 . As g is continu-
of Fig. 2 shows the violation rapidly converging to zero; ous in a compact set, for all εg > 0, there exists a δg > 0
the line in red shows the norm of the difference between such that kg(z1 ) − g(z2 )k < εg for all kz1 − z2 k < δg .
the multiplier obtained with the indirect method and the Using εf = δg , kfn − f k < εf = δg for all n > N .
multiplier estimate computed by the proposed algorithm, Therefore, kg(fn ) − g(f )k < εg for all n > N .
which decreases as the algorithm iterates. REFERENCES
An experiment was performed using the proposed algo-
[1] D. P. Bertsekas, Constrained Optimization and Lagrange Multiplier
rithm with direct multiple shooting to solve the subprob- Methods. Athena Scientific, 1996.
lems (AL-DMS). To solve the ODE of the suproblems, [2] R. R. Negenborn, B. De Schutter, and J. Hellendoorn, “Multi-agent
we apply Sundails CVODES and a 4th order Runge-Kutta model predictive control for transportation networks: Serial versus
parallel schemes,” Eng. Appl. Artif. Intell., vol. 21, no. 3, pp. 353–
method (RK4). The proposed algorithm was compared 366, Apr. 2008.
against DMS applied directly to the original OCP, whereby [3] N. Chatzipanagiotis, D. Dentcheva, and M. M. Zavlanos, “An
Sundails IDAS was used to solve the DAE. Table I presents augmented Lagrangian method for distributed optimization,” Math-
ematical Programming, vol. 152, no. 1, pp. 405–434, Aug 2015.
the results that indicate faster convergence of proposed [4] M. A. S. de Aguiar, E. Camponogara, and B. Foss, “An augmented
algorithm with RK4, which might be more suitable for em- Lagrangian method for optimal control of continuous time DAE
bedded applications with restricted computational power. systems,” in IEEE CCA, 2006, pp. 1185–1190.
[5] S. Hentzelt and K. Graichen, “An augmented Lagrangian method
in distributed dynamic optimization based on approximate neighbor
Table I dynamics,” in IEEE SMC, 2013, pp. 571–576.
A L G O R I T H M S O LV I N G T I M E ( S ) [6] H. Attouch and M. Soueycatt, “Augmented Lagrangian and proximal
alternating direction methods of multipliers in Hilbert spaces.
AL-DMS (CVODES) AL-DMS (RK4) DMS (IDAS) applications to games, PDE’s and control,” Pacific Journal of
24.15 0.11 2.42 Optimization, vol. 5, no. 1, pp. 17–37, Nov. 2008.
[7] D. Kirk, Optimal Control Theory: An Introduction, 4th ed. Dover
Publications, 2004.
[8] L. T. Biegler, Nonlinear Programming. SIAM, jan 2010.
[9] M. R. de Pinho, R. B. Vinter, and H. Zheng, “A maximum principle
V. C O N C L U S I O N for optimal control problems with mixed constraints,” IMA Journal
An algorithm based on the augmented Lagrangian for of Math. Control and Information, vol. 18, pp. 189–205, 2001.
[10] A. Boccia, M. D. R. de Pinho, and R. B. Vinter, “Optimal control
solving OCPs of DAEs by relaxing the algebraic equations problems with mixed and pure state constraints,” SIAM Journal on
was presented. Properties for global and local convergence, Control and Optimization, vol. 54, no. 6, pp. 3061–3083, 2016.
and convergence under sub-optimal iterations are shown. [11] J. T. Betts, Practical Methods for Optimal Control and Estimation
Using Nonlinear Programming. Cambridge Univ. Press, 2009.
The algorithm properties are verified with an illustrative [12] J. Andersson, J. Gillis, G. Horn, J. Rawlings, and M. Diehl, “CasADi
example, showing the convergence of the objective value, – A software framework for nonlinear optimization and optimal
multiplier estimate, and optimal trajectory. control,” Math. Progr. Comp., vol. 11, no. 1, pp. 1–36, 2019.

0018-9286 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Otago. Downloaded on May 30,2020 at 23:54:30 UTC from IEEE Xplore. Restrictions apply.