Learning Region of Attraction for Nonlinear Systems

∗
Shaoru Chen, Mahyar Fazlyab, Manfred Morari, George J. Pappas, Victor M. Preciado

Abstract
Estimating the region of attraction (ROA) of general nonlinear autonomous systems remains
arXiv:2110.00731v1 [eess.SY] 2 Oct 2021

a challenging problem and requires a case-by-case analysis. Leveraging the universal approxi-
mation property of neural networks, in this paper, we propose a counterexample-guided method
to estimate the ROA of general nonlinear dynamical systems provided that they can be approx-
imated by piecewise linear neural networks and that the approximation error can be bounded.
Specifically, our method searches for robust Lyapunov functions using counterexamples, i.e., the
states at which the Lyapunov conditions fail. We generate the counterexamples using Mixed-
Integer Quadratic Programming. Our method is guaranteed to find a robust Lyapunov function
in the parameterized function class, if exists, after collecting a finite number of counterexamples.
We illustrate our method through numerical examples.

1 Introduction
Certifying the stability of nonlinear dynamical systems described by differential or difference equa-
tions is a long-standing fundamental problem in control theory. A general method for proving
stability is the method of Lyapunov whereby one attempts to find, from a hypothesis class, a pos-
itive definite function that decreases along the trajectories of the dynamical system [1, 2]. Once a
Lyapunov function is found, its level sets can be used to estimate the region of attraction (ROA)
of the equilibrium, which is a useful tool in constructing safety certificates [3, 4].
In general, in order to find Lyapunov functions for nonlinear systems, one needs to solve a non-
convex optimization problem. To avoid intractable non-convex optimization problems, there has
been an increasing interest in developing data-driven and machine-learning inspired methods for
stability analysis. For example, the authors in [5] parameterize the Lyapunov function as a neural
network and train it through stochastic gradient descent to learn the largest ROA of a nonlinear
system. Giesl et al. [6] learn a Lyapunov function as a reproducing kernel Hilbert space predictor
from noisy trajectories of a nonlinear system. Boffi et al. [7] treat Lyapunov function synthesis
as a machine learning problem and provide generalization error bounds for the learned Lyapunov
function.
Although success examples have been demonstrated for the learning-based methods, the guar-
antees on the stability certificates are often of probabilistic nature since only sampled trajectory
data are used. On the other hand, feedforward neural network or recurrent neural network models
obtained from learning-based system identification are shown to have a strong predictive power [8]
and can approximate complex nonlinear dynamics with small errors. This offers an opportunity to
∗
Shaoru Chen, Manfred Morari, George J. Pappas, and Victor M. Preciado are with the Department of Electrical
and Systems Engineering, University of Pennsylvania. Email: srchen, morari, pappasg, [email protected].
Mahyar Fazlyab is with the Mathematical Institute for Data Science, Johns Hopkins University. Email: mahyarfa-
[email protected]. This work was partially supported by the AFOSR grant FA9550-19-1-0265 (Assured Autonomy in
Contested Environments).

1
obtain deterministic stability guarantees on nonlinear systems by analyzing their neural network
approximations with the model errors taken into account. However, how to analyze the stability
of uncertain neural network dynamics remains a challenge with only limited work available [9, 10].
In this paper, we propose a sampling-based method for constructing robust Lyapunov functions
(see Theorem 1 for definition) for a class of uncertain dynamical systems with feedforward ReLU
networks modeling the nominal dynamics over a compact region of interest (ROI). Our method
consists of a learner, which is responsible for generating Lyapunov function candidates, and a
verifier, which validates or rejects the proposed candidates with counterexamples to guide the
search. More importantly, if a Lyapunov function exists in the parameterized Lyapunov function
class, our method is guaranteed to find one in a finite number of steps. These properties are
particularly desirable for analysis of uncertain neural network dynamical systems. For comparison,
the convex relaxation based methods [10] easily become excessively conservative as the size of the
neural network or the model uncertainty increases. The counterexample-guided Lyapunov function
synthesis methods in [11–13] do not have finite-step termination guarantees which means that
the number of calls to the computationally expensive verifier cannot be bounded. Our method is
inspired by [9] which proposes a sampling-based method for stability analysis of autonomous hybrid
systems. We extend the method in [9] to handle uncertain dynamical systems and demonstrate the
potential of our proposed method to automate Lyapunov function synthesis for nonlinear systems
through their neural network dynamics approximations.
We state the assumption on the uncertain neural network dynamical system and the problem
formulation in Section 2. In Section 3, the parameterization of robust Lyapunov function is in-
troduced and a sampling-based method to synthesize a robust Lyapunov function is presented in
Section 4. The practical use of the proposed framework is discussed in Section 5 with numerical
examples given in Section 6 to demonstrate our method. Section 7 concludes the paper.

2 Problem Statement
Consider a discrete-time nonlinear dynamical system

x+ = f (x) (1)

where x ∈ Rnx is the state, x+ denotes the state at the next time instance, and f : Rnx 7→ Rnx
is a locally Lipschitz continuous function. We assume that system (1) has an equilibrium x = 0,
i.e., 0 = f (0), and is asymptotically stable at the origin. Verifying that system (1) is locally stable
around the origin can be done by analyzing the linearization of the system at x = 0 if it exists. Of
more practical use is estimating the region of attraction of the origin.

Definition 1 (Region of attraction). The region of attraction for system (1) at the origin is
O = {x0 | limk→∞ xk = 0} where xk denotes the state of the system at time k = 0, 1, 2, · · · .

The ROA characterizes how much the system can be perturbed from x = 0 without diverging.
Therefore, the ROA is often used to describe the safety of nonlinear systems. In this work, we are
also interested in the region where the convergence of system trajectories to a small neighborhood
of the origin can be verified since this relaxed notion of ROA also suffices to show safety of nonlinear
systems in practical applications.
Computing the exact ROA O for general nonlinear systems is infeasible and hence, one seeks
to find an estimate Õ of ROA for specific classes of dynamical systems. In this work, we assume
that the map f can be approximated by a piecewise linear multi-layer neural network fN N and

2
that the approximation error f − fN N can be bounded. Specifically, consider a neural network with
ReLU1 activation:
z0 = x
z`+1 = max(W` z` + b` , 0), ` = 0, · · · , L − 1 (2)
fN N (x) = WL zL + bL
where z0 = x ∈ Rn0 (n0 = nx ) is the input to the neural network, z`+1 ∈ Rn`+1 is the the output
vector of the (` + 1)-th hidden layer with n`+1 neurons, fN N (x) ∈ RnL+1 (nL+1 = nx ) is the output
of the neural network, and W` ∈ Rn`+1 ×n` , b` ∈ Rn`+1 are the weight matrix and the bias vector of
the (` + 1)-th hidden layer.
Assumption 1. For a given Schur stable matrix A ∈ Rnx ×nx and a compact region of interest
(ROI) X ⊂ Rnx with 0 ∈ int(X ), where int(X ) denotes the interior of X , the map f (x) can be
decomposed as
f (x) = Ax + fN N (x) + w(x), ∀x ∈ X (3)
where the approximation error w(x) ∈ Rnx satisfies
kw(x)k∞ ≤ γkxk∞ + δ, ∀x ∈ X , (4)
for some known positive constants γ, δ.
Now consider the uncertain dynamical system
x+ = Ax + fN N (x) +w, w ∈ W(x) (5)
| {z }
:=fˆ(x)

where W(x) = {w ∈ Rnx | kwk∞ ≤ γkxk∞ + δ} is a state dependent set that contains the
“disturbance” w. Under Assumption 1, the nonlinear dynamics f (x) is contained in the uncertain
dynamics class described by (5) which enjoys a specific structure that can be exploited for stability
analysis. The main problem considered in this paper is stated as follows.
Definition 2 (Robust ROA). Let xk , wk denote the state and disturbance of the uncertain sys-
tem (5) at time k = 0, 1, · · · , respectively, and B ∈ X be a small neighborhood of the origin with
0 ∈ int(B). The robust ROA of the uncertain system (5) is defined as O = {x0 | limk→∞ xk ∈
B, ∀wk ∈ W(xk )}.
Problem 1. Find an estimate of the robust ROA of the uncertain neural network dynamical sys-
tem (5).
In the Section 3 and 4, we propose a sampling-based method to synthesize a robust Lyapunov
function which identifies an estimate of the robust ROA Õ for the uncertain system (5). By virtue
of Assumption 1, Õ would be a valid inner estimate of ROA for system (1). We postpone the
discussion on the applicability of Assumption 1 to Section 5.

3 Parameterization of Robust Lyapunov functions

For the uncertain system (5), we aim to find a robust Lyapunov function that monotonically
decreases for all possible realization of uncertainty. Once a robust Lyapunov function is found, its
sublevel sets encode an inner estimate of the robust ROA for the uncertain system (5), and hence
the original system (1).
1
ReLU (rectified linear unit) activation function is defined by φ(x) = max(0, x) for x ∈ Rn where the maximum
is applied entry-wise.

3
3.1 Robust Lyapunov function
For the uncertain system (5) with the uncertainty set W(x), we aim to show that the system
trajectories converge to a neighborhood B of the origin. The neighborhood B is chosen according
to the practical need of verification. Let Br = {x ∈ Rnx |kxk∞ ≤ r} ⊂ X denote an `∞ norm
ball with radius r > 0 contained inside the ROI. The robust Lyapunov function is defined in the
following theorem.
Definition 3 (Successor set). For the uncertain dynamics (5), we denote the successor set from a
set B as succ(B) = {y ∈ Rnx |∃x ∈ B, w ∈ W s.t. y = fˆ(x) + w}.
Theorem 1. Consider the discrete-time uncertain system (5) and assume there exists a continuous
function V (x) : X 7→ R satisfying

V (x) > 0, ∀x ∈ X \ Br , (6a)

V (fˆ(x) + w) − V (x) < 0, ∀x ∈ X \ Br , ∀w ∈ W(x). (6b)

Denote Ωsucc(Br ) (V ) the smallest sublevel set of V (x) that contains the successor set of Br and
Ω̄X (V ) the largest sublevel set of V (x) that is contained in X . Then for all initial states x0 ∈ Ω̄X (V )
of the uncertain system (5), we have limk→∞ xk ∈ Ωsucc(Br ) (V ).

Proof. All trajectories starting from x0 ∈ Ω̄X (V ) reach Br in a finite number of steps since V (x) is
upper bounded in X and its decrease in each step is lower bounded by a positive number as long
as x ∈
/ Br . Once the state xk reaches Br , it is mapped to xk+1 ∈ succ(Br ) ⊂ Ωsucc(Br ) (V ) in the
next step. Whether xk+1 ∈ / Br or xk+1 ∈ Br , the trajectory starting from xk+1 will always stay in
Ωsucc(Br ) (V ).

We call any V (x) satisfying constraints (6a) and (6b) a robust Lyapunov function and those
satisfying constraint (6a) a robust Lyapunov function candidate. Theorem 1 states that a robust
Lyapunov function V (x) certifies the convergence of system trajectories under uncertainty to a
neighborhood of the origin whose size and shape is jointly decided by V (x) and Br . The largest
sublevel set of V (x) contained in the ROI X gives an estimate of robust ROA for the uncertain
system, and Õ = Ω̄X (V ) is also an inner-approximation of ROA of the original nonlinear system (1).
In the rest of the paper, we refer to robust Lyapunov functions simply as Lyapunov functions.

3.2 Lyapunov function parameterization

To make the search for a Lyapunov function V (x) tractable, we parameterize the Lyapunov function
candidate as a quadratic function using predicted nominal trajectories of the uncertain system (5)
for a finite horizon as the basis. For a horizon of k ≥ 0, the basis is given by
> >
zk (x) = [x> fˆ(1) (x)> fˆ(2) (x) · · · fˆ(k) (x) ]>

where fˆ(i+1) (x) = fˆ(fˆ(i) (x)) for i ≥ 0 and fˆ(0) (x) = x. By denoting Sn the set of n × n-dimensional
symmetric matrices, and Sn+ (Sn++ ) the set of n × n-dimensional positive semidefinite (definite)
matrices, the Lyapunov function candidate of order k is defined as

Vk (x; P ) = zk> (x)P zk (x), P ∈ Sn++

x
. (7)

This parameterization is inspired by the non-monotonic Lyapunov function [14] and finite-step Lya-
punov function [15] methods which construct Lyapunov function candidates using system states sev-
eral steps ahead. By construction, Vk (x; P ) is positive definite and readily satisfies constraint (6a).

4
In addition, the parameterization of Vk (x; P ) has two desirable properties: (i) Vk (x; P ) is a linear
function in P when the state x is fixed, and (ii) for k ≥ 1, Vk (x; P ) is a piecewise quadratic function
in x and its complexity can be easily tuned by the order k.
The first property indicates that the set of valid Lyapunov function candidates is convex. To
show this, define the Lyapunov difference function as

∆Vk (x, w, P ) = Vk (fˆ(x) + w; P ) − Vk (x; P ). (8)

From the first property of Vk (x; P ), we know that ∆Vk (x, w, P ) is also a linear function in P when
the state x and uncertainty w are fixed. We describe the set of valid Lyapunov functions in the
parameter space as

F = {P ∈ Snx |0 ≺ P  I, ∆Vk (x, w, P ) < 0, ∀x ∈ X \ Br , ∀w ∈ W(x)} (9)

and denote F the target set. It follows that F is a convex set since it is defined by two linear matrix
inequalities and infinitely many linear inequalities in P . The convexity of F is essential for the
design of our main algorithm to search for Lyapunov functions.
The second property follows from the fact that the ReLU network fˆ(x) is a piecewise affine
function and so are fˆ(i) (x). By tuning the order k, we can parameterize functions with high
complexity, i.e., piecewise quadratic functions with a large amount of partitions, by using a relatively
small number of parameters.
After identifying the set of Lyapunov functions F, our goal becomes finding a feasible point in
F or proves that F is empty with the given parameters. Although the set F is convex, finding a
feasible point in it is challenging due to the infinitely many constraints in (9) and the complexity
of the underlying uncertain neural network dynamics (5).

4 Sampling-based Lyapunov function synthesis

In this section, we extend the sampling-based method in [9] to synthesize a Lyapunov function for
the uncertain system (5) with neural network nominal dynamics. Specifically, the proposed method
is guaranteed to find a Lyapunov function in the target set F in a finite number of iterations when
F is full-dimensional, and is capable of providing an infeasibility certificate of F when F is empty.
Similar to [9], we obtain this guarantee by sampling according to the analytic center cutting-plane
method [16, 17] from convex optimization. A step-by-step explanation is given below.

4.1 Localization of the target set

To handle the infinitely many linear constraints in (9), we first observe that an over-approximation
F̃ of the target set F can be obtained by relaxing the Lyapunov difference constraint in (9) to hold
only on a sample set with a finite number of state-disturbance pairs S = {(xSi , wiS )}N
i=1 , i.e.,

F̃ = {P ∈ Snx |0 ≺ P  I, ∆Vk (x, w, P ) < 0, ∀(x, w) ∈ S}

with samples xSi ∈ X \ Br , wiS ∈ W(xSi ) for i = 1, · · · , N . We call F̃ a localization set of F and
F ⊆ F̃ holds by definition. Since the localization set F̃ is given by a finite number of convex
inequalities, finding a feasible point in F̃ or showing infeasibility of F̃ can be done by solving a
convex program efficiently. If for a sample set S, the localization set F̃ is empty, then we have a
certificate that the target set F is empty. When the target set is nonempty, we can find a feasible
point in F by iteratively expanding the sample set S and refining the localization set F̃. However,

5
expanding the sample set in an arbitrary or random way is not efficient or may not improve the
over-approximation of F; in order to obtain performance guarantees, we need to add samples that
are informative at each iteration. This is done by alternating between a learner and a verifier to
select the samples.

4.2 Construction of the learner

Given a sample set S and the corresponding localization set F̃, the learner proposes a Lyapunov
function candidate Vk (x; Pac ) with Pac as the analytic center of F̃.
Definition 4 (Analytic center [18]). The analytic center xac of a set of convex inequalities and
linear equalities hi (x) ≤ 0, i = 1,· · ·, m, F x = g, is defined as the solution of the convex problem
m
X
minimize − log(−hi (x)) subject to F x = g.
x
i=1

By the definition of analytic center, Pac is given by

X
Pac := argmin − log(−∆Vk (x, w, P )) − log det(I − P ) − log det(P ). (10)
P
(x,w)∈S

where − log det(I −P )−log det(P ) is the barrier function of the set {P |0 ≺ P ≺ I}. If problem (10)
is infeasible, the localization set F̃ is empty and so is the target set F. If problem (10) is feasible,
Vk (x; Pac ) is proposed by the learner as the Lyapunov function candidate based on the localization
set F̃ and is passed to the verifier.

4.3 Construction of the verifier

For the given Lyapunov function candidate Vk (x; Pac ) proposed by the learner, the verifier either
validates Vk (x; Pac ) or rejects it by giving a counterexample (x∗ , w∗ ) at which Vk (x; Pac ) violates
the Lyapunov difference condition (6b), i.e., ∆Vk (x∗ , w∗ , Pac ) ≥ 0. This is done by solving the
global optimization problem:

p∗ = maximize ∆Vk (x, w, Pac ) (11)

x∈X \Br ,w∈W(x)

which is nonconvex. Denote p∗ the optimal value of (11) and (x∗ , w∗ ) the optimal solution. The
verifier validates Vk (x; Pac ) if p∗ < 0 and rejects Vk (x; Pac ) if p∗ ≥ 0 with (x∗ , w∗ ) as the coun-
terexample. In the former case, we already have a valid Lyapunov function Vk (x; Pac ) for the
uncertain system (5); in the latter case, the counterexample (x∗ , w∗ ) is added to the sample set
S and the localization set F̃ is updated. Then the learner generates a new Lyapunov function
candidate based on the updated localization set. The alternation between the learner and the
verifier is repeated until either a Lyapunov function in F is found or F is certified to be empty.
We summarize our method in Algorithm 1. Solving a nonconvex optimization problem to global
optimality is in general intractable, but for the uncertain neural network dynamical system (5), we
are able to formulate problem (11) as a nonconvex mixed-integer quadratic program (MIQP) for
which global optimization algorithms exists [19, 20]. In practice, Algorithm 1 is easy to implement
since solving nonconvex MIQPs to global optimality can be done in off-the-shell solvers such as
Gurobi v9.0 [21]. Interested readers are referred to [9] for more discussions on the complexity and
solvability of MIQPs. Next, we show how to formulate problem (11) as an MIQP by exploiting the
mixed-integer linear formulation of ReLU networks.

6
 Algorithm 1: Learning robust Lyapunov function
Data: Initial sample set S
Result: Infeasibility certificate or Lyapunov function
while True do
Update localization set F̃
Call the learner
if (10) is feasible then
Propose candidate Vk (x; Pac )
else
Return infeasibility certificate
Call the verifier
if max (11) < 0 then
Return Lyapunov function
else
Generate counterexample (x∗ , w∗ )
Update sample set S = S ∪ {(x∗ , w∗ )}

The formulation of the MIQP is based on the fact that the feedforward ReLU network fN N (x)
can be described by a set of mixed-integer linear (MIL) constraints. For the (` + 1)-th activation
layer in the ReLU network described in (2), let m` and m̄` be the element-wise lower and upper
bounds on the input, i.e., m` ≤ W` z` + b` ≤ m̄` . Then the ReLU activation function is equivalent
to the following set of mixed-integer linear constraints [22]:

z`+1 = max(W` z` + b` , 0) ⇐⇒



 z`+1 ≥ W` z` + b`
(12)

`+1 ≤ W` z` + b` − diag(m` )(1 − t` )
z
z`+1 ≥ 0



`+1 ≤ diag(m̄` )t` ,
z

where t` ∈ {0, 1}n`+1 is a vector of binary variables for the (` + 1)-th activation layer. Since the
input to the ReLU network is bounded in our considered problem, various methods are available to
find the element-wise pre-activation bounds {m` , m̄` } such as interval bound propagation [23] and
linear programming [24]. By assembling the mixed-integer linear constraints (12) for each layer, we
obtain the mixed-integer linear representation of the ReLU network and also the nominal dynamics
x+ = fˆ(x) in (5). Consequently, the composition fˆ(i) (x) for i = 1, · · · , k can be described by
mixed-integer linear constraints as well.
With the given Lyapunov function candidate Vk (x; Pac ), problem (11) can be rewritten as

maximize zk> (y)Pac zk> (y) − zk> (x)Pac zk> (x) (13a)
x,y,w

subject to y = Ax + fN N (x) + w (13b)

kwk∞ ≤ γkxk∞ + δ (13c)
x ∈ X \ Br (13d)

where the MIL representation of constraint (13b) and basis zk (x), zk (y) follows from that of the
ReLU network, and the MIL representation of constraints (13c) and (13d) follow from that of the

7
`∞ norm. Since the objective function (13a) is quadratic and indefinite, problem (13) is a nonconvex
MIQP which can be solved by Gurobi to global optimality.

4.4 Termination guarantee

Algorithm 1 is guaranteed to find a Lyapunov function in the target set under the following as-
sumption:

Assumption 2. The target set F defined in (9) is full-dimensional, i.e., there exists Pc ∈ S(k+1)nx
and  > 0 such that {P ∈ S(k+1)nx |kP − Pc kF ≤ } ⊂ F where k·kF is the Frobenius norm.

Theorem 2. Under Assumption 2, Algorithm 1 finds a Lyapunov function in the target set F in
at most O(((k + 1)nx )3 /2 ) iterations.

Theorem 2 states that when the target set is full-dimensional, Algorithm 1 is guaranteed to find
a Lyapunov function in a finite number of steps. Such a guarantee follows from the termination
guarantee of the analytic center cutting-plane method considered in [25] and this is the main
reason we require the learner to propose the analytic center of the localization sets as Lyapunov
function candidates. From an optimization point of view, the sampling-based method described in
Algorithm 1 is equivalent to finding a feasible solution in the target set through the analytic center
cutting-plane method with the verifier serving as the cutting-plane oracle. The proof of Theorem 2
follows from that of [9, Theorem 2] since our extension of the Lyapunov function synthesis method
in [9] to the uncertain dynamical system (5) does not change the convexity of the target set.

Remark 1. When the target set is empty, we do not have a finite-step termination guarantee
for the proposed algorithm. Therefore, Algorithm 1 is semi-complete and in practice, we can add
additional stopping criteria, e.g., maximum number of iterations, to Algorithm 1.

To summarize, the sampling-based Lyapunov function synthesis method in [9] looks for Lya-
punov functions simply based on the mixed-integer representation of hybrid systems. This makes
it suitable to verify the stability of neural network dynamical systems since modeling the behavior
of ReLU network over a compact set through MIL constraints is straightforward. In this paper
we extend this method to handle uncertain dynamical systems and apply it in a novel setting to
estimate the ROA of nonlinear dynamical systems through their neural network approximations.
The non-conservativeness and the finite-step termination guarantee provided by Theorem 1 makes
this method favorable to analyze uncertain dynamical systems. However, this is obtained at the
cost of the worst-case exponential complexity of the MIQP (13).

5 Approximation of dynamical systems by neural networks

In this section, we investigate the applicability of Assumption 1 and show how to estimate the error
bound parameters (γ, δ).

5.1 Neural network training

The decomposition of the nonlinear map f (x) = Ax + fN N (x) + w(x) in (3) is motivated by two
facts: first, stability verification tools [9, 10] for neural network dynamical systems have been
developed; second, powerful and easy-to-use machine learning tools are available to train fN N (x)
to high accuracy and keep the approximation error w(x) small. In training fN N (x), we want to
obtain a neural network model that not only achieves a small approximation error but also is easy

8
to verify. In light of the latter objective, it is desirable to increase the number of stable ReLUs in
fN N (x), i.e., ReLUs that always output zero or one. For these ReLUs, we do not need to introduce
binary variables in the MIQP (13). As a result, any training method that can increase the number
of stable ReLUs can significantly improve the complexity of the verification. As shown in [26],
applying `1 -regularization to the training objective generally works well in reducing the complexity
of verification. Other training methods with the same goal are presented in [26] as well.

5.2 Identification of error bound

For a given neural network fN N (x) which approximates the nonlinear map f (x), we estimate an
error bound in the form of (4) based on dense sampling of the ROI X . This method is not meant
to compete with the modeling error quantification methods in the literature. Instead, we use it to
provide a reasonable error bound to set up the simulation.
By assumption f (x) is locally Lipschitz continuous. Let Lf (LN ) be an upper bound on the
Lipschitz constant of f (x) (Ax + fN N (x)) over the ROI X . Since w(x) = f (x) − Ax − fN N (x), we
can upper bound the Lipschitz constant of w(x) as Lw = Lf + LN . Then we sample densely from
the ROI X and in particular the samples SX = {xSi }N i=0 ⊂ X form a -net of X , i.e., for all x ∈ X ,
there exists a sample y ∈ SX such that kx − yk ≤ . We define the admissible set of (γ, δ) as

Γ = {(γ, δ)|kw(x)k∞ ≤ γkxk∞ + δ, ∀x ∈ SX }

and any pair (γ 0 , δ 0 ) ∈ Γ gives an estimate of the upper bound in (4). Since we want to keep (γ 0 , δ 0 )
small, we let δ 0 = min δ s.t. (γ 0 , δ) ∈ Γ. Note that there are infinitely many pairs in the admissible
set and decreasing one parameter would increase the other one.
Given any admissible pair (γ 0 , δ 0 ), for all x ∈ X we have

kw(x)k∞ ≤ kw(x) − w(y)k∞ + kw(y)k∞

≤ Lw kx − yk∞ + γ 0 kyk∞ + δ 0
(14)
≤ Lw kx − yk∞ + γ 0 ky − xk∞ + γ 0 kxk∞ + δ 0
≤ γ 0 kxk∞ + δ 0 + (Lw + γ 0 )

where y ∈ SX is chosen such that kx − yk ≤ . Therefore, γ = γ 0 , δ = δ 0 + (Lw + γ 0 ) gives a valid

upper bound for kw(x)k∞ over the ROI X if we assume Lf is known. Otherwise, inequality (14)
indicates that by choosing  small enough, the admissible pair (γ 0 , δ 0 ) obtained from sampling gives
a good estimate of a deterministic error bound.
Since every pair from the admissible set Γ generates an error bound estimate, we can combine
K admissible (γ 0 , δ 0 ) pairs to obtain a tighter error bound as

kwk∞ ≤ γi0 kxk∞ + δi0 , i = 1, · · · , K (15)

from the sampling. One special, interesting choice is setting γ 0 = 0 since in this case constraint (13c)
in the MIQP becomes kwk∞ ≤ δ 0 and is convex.

6 Numerical examples
We consider a 2- and 3-dimensional nonlinear system from the literature and train ReLU networks
to approximate their dynamics. The approximation errors are estimated from sampling to generate
the uncertain neural network dynamical systems, for which our proposed algorithm is called to
synthesize robust Lyapunov functions. All experiments are run on an Intel Core i7-6700K CPU.

9
Figure 1: The neural network approximation errors are marked by the blue dots for a dense sample set over
the ROI. The affine error bounds are denoted by the blue, orange, green, and red lines corresponding to
(γ, δ) = (0, 0.0288), (1.167, 0.0141), (0.0233, 4.767 × 10−3 ), (0.0350, 4.026 × 10−4 ), respectively.

6.1 2-D rational system

Consider the 2 dimensional discrete-time rational system adapted from [27, Example 2]:
x1 + x32 x31 − 0.25x2
x+
1 = x1 − , x+
2 = x2 + . (16)
1 + x21 1 + x22
and denote the dynamics in (16) as x+ = f (x). For the uncertain dynamics decomposition (5), we
choose the Jacobian of f (x) at the origin as matrix A = [0 0; 0 0.75]. Then we sample 22500 states
uniformly from the box R = {x ∈ R2 |kxk∞ ≤ 1.5} and train a fully-connected ReLU network with
the structure 2 − 100 − 100 − 100 − 2 (three hidden layers with 100 neurons each) with Keras [28]
and the Adam optimizer [29] in 100 epochs to approximate the error dynamics f (x) − Ax. Since
the approximation error bound (4) is written in terms of the `∞ norm, the training loss function is
set to the mean absolute error. Furthermore, in order to alleviate the practical computational cost
of the MIQP (13) in the analysis phase, we add `1 regularization with weight 10−4 during training.
We construct the ROI X by uniformly sampling initial states from the region R and simulate
the trajectories of the system (16) for 50 steps. Then, the convex hull of the initial condition
samples from which the trajectories converge to the origin is chosen as the reference ROI Xref :=
{x|Aref x ≤ bref }. We tune the ROI by X = τ Xref := {x|Aref ≤ τ bref } with 0 < τ ≤ 1.0. In this
example, we set the scaling parameter τ = 0.9. The sampled convergent initial states are marked
as the black dots in Fig. 2 where the ROI is also plotted.
After training the neural network and fixing the ROI, we estimate the neural network approxi-
mation error by sampling a uniformly spaced grid from the ROI. The grid consists of 72306 state
samples which form an -net of the ROI with  ≈ 0.005. We plot the (kxk∞ , kwk∞ ) pairs for all the
samples in Fig. 1 and our estimated error bound with (γ, δ) gives an affine upper bound over all the
sampled graph of (kxk∞ , kwk∞ ). Selected values of (γ, δ) and their corresponding affine bounds are
plotted in Fig. 1. Although each pair (γ, δ) gives an estimate on the bound of the neural network
approximation error, formulating a concave piecewise affine upper bound by combining different
values of (γ, δ) is tighter than using a single one as discussed in Section 5. As shown in (14), if the

10
sampling is dense enough, the estimated error bounds from samples is close to the true bounds. In
this section, we apply estimated error bounds from sampling for simplicity.
Finally, we run Algorithm 1 to find a robust Lyapunov function for the considered neural network
approximation system with the error bound given by the concave bound shown in Fig. 1 using four
pairs of (γ, δ) values. Let B denote the neighborhood that is excluded from the search space in place
of the `∞ norm ball Br in (6). For this example, we choose B = {x ∈ R2 |−0.05 ≤ x1 ≤ 0.05, −0.2 ≤
x2 ≤ 0.2} as a rectangle according to the simulated uncertain dynamical system (5). Then we set
the order of the Lyapunov function candidates to be k = 1 in (7) and initialize Algorithm 1 with an
empty sample set. Algorithm 1 terminates in 6 iterations with a total solver time of 748 seconds.
It successfully finds a robust Lyapunov function V1 (x; P ) with P ∈ S4++ . The resulting estimate of
the ROA Õ which is the largest sublevel set of V1 (x; P ) is plotted in Fig. 2 together with several
simulated trajectories of the uncertain system (5). We observe that the estimate ROA provided by
V1 (x; P ) is close to the true ROA estimated by the samples.
As a comparison, we run Algorithm 1 with error bound (γ, δ) = (0, 0.0288) which corresponds
to the blue line in Fig 1. Since γ = 0, the constraint (13c) becomes convex at the cost of looser
characterization of the approximation errors. With the same setup, Algorithm 1 finds a robust
Lyapunov function in 17 iterations with the total solver time 3162 seconds. The synthesized robust
Lyapunov function generates an estimate ROA similar to that shown in Fig 2 with about 4 times
the running time.

Figure 2: Estimate ROA of system (16) through neural network approximation and synthesizing robust
Lyapunov function. The estimate ROA Õ is marked by the blue curve and the ROI is given by the red
polytope with dashed lines. The set B is a rectangle around the origin which is excluded from the search
space of the verifier.

6.2 3-D polynomial system

Consider the 3-dimensional continuous-time polynomial system in [30, Example B]:

x1 (x21 + x22 − 1) − x2 (x23 + 1)

   
ẋ1
ẋ2  = x2 (x21 + x22 − 1) + x1 (x23 + 1) . (17)
ẋ3 10x3 (x23 − 1)

11
 (a) Slice at x1 = 0 (b) Slice at x3 = 0

Figure 3: Estimated ROA Õ (blue curve) given by the robust Lyapunov function and the ROI (red polytope
with dashed lines) on slices x1 = 0 and x3 = 0. Simulated trajectories of the uncertain neural network
dynamical system and their initial states (white dots) are projected to the corresponding slices for plotting.

We apply the Euler discretization method with sampling time dt = 0.1 seconds to discretize the
dynamics and choose the Jacobian of the discretized dynamics at the origin as the A matrix. Then
we apply a 3 − 100 − 100 − 100 − 3 fully-connected ReLU network to approximate the discrete-time
dynamics over the box R = {x ∈ R3 |kxk∞ ≤ 1}. The neural network is trained with 125000 samples
uniformly spaced in R by the same training method described in the 2-dimensional example. The
ROI is obtained similarly through simulation and we estimate the neural network approximation
error bound from sampling. The samples form an -net in the ROI with  ≈ 0.01 and the error
bound (γ, δ) = (0, 0.0165) is estimated from sampling.
We run Algorithm 1 for the uncertain neural network dynamical system with the exclusion of
B = {x ∈ R3 |kxk∞ ≤ 0.05} from the ROI. The sample set is initialized as an empty set. With
order k = 2, the algorithm terminates in one iteration with a total solver time of 10 seconds. Upon
termination, a robust Lyapunov function V2 (x; P ) is found whose largest sublevel set is shown in
Fig 3 together with the ROI at slices x1 = 0 and x3 = 0. Simulated trajectories of the uncertain
neural network dynamical system starting from uniformly spaced initial conditions inside the ROI
are plotted in Fig. 3 to illustrate the system dynamics.

7 Conclusion
Finding Lyapunov functions for nonlinear dynamical systems and estimating their region of attrac-
tion relies on deep insights from experts and requires a case-by-case analysis. In this paper, we move
a step towards automating Lyapunov function synthesis for nonlinear systems. Our method starts
with approximating the map of a dynamical system by a piecewise linear neural network. Assuming
that the resulting approximation error can be bounded, we then develop a counterexample-guided
method to synthesize a Lyapunov function that is robust to the approximation error.
While in practice the approximation error can be made arbitrarily small, computing determin-
istic bounds on it can be challenging, especially in high dimensions. In future work, we will explore
efficient ways to bound the approximation error.

12
References
[1] W. M. Haddad and V. Chellaboina, Nonlinear dynamical systems and control: a Lyapunov-based ap-
proach. Princeton university press, 2011.

[2] P. Giesl and S. Hafstein, “Review on computational methods for Lyapunov functions,” Discrete &
Continuous Dynamical Systems-B, vol. 20, no. 8, p. 2291, 2015.

[3] F. Berkenkamp, R. Moriconi, A. P. Schoellig, and A. Krause, “Safe learning of regions of attraction for
uncertain, nonlinear systems with gaussian processes,” in 2016 IEEE 55th Conference on Decision and
Control (CDC). IEEE, 2016, pp. 4661–4666.

[4] L. Wang, D. Han, and M. Egerstedt, “Permissive barrier certificates for safe stabilization using sum-of-
squares,” in 2018 Annual American Control Conference (ACC). IEEE, 2018, pp. 585–590.

[5] S. M. Richards, F. Berkenkamp, and A. Krause, “The Lyapunov neural network: Adaptive stability
certification for safe learning of dynamical systems,” in Conference on Robot Learning. PMLR, 2018,
pp. 466–476.

[6] P. Giesl, B. Hamzi, M. Rasmussen, and K. N. Webster, “Approximation of Lyapunov functions from
noisy data,” arXiv preprint arXiv:1601.01568, 2016.

[7] N. M. Boffi, S. Tu, N. Matni, J.-J. E. Slotine, and V. Sindhwani, “Learning stability certificates from
data,” arXiv preprint arXiv:2008.05952, 2020.

[8] O. Ogunmolu, X. Gu, S. Jiang, and N. Gans, “Nonlinear systems identification using deep dynamic
neural networks,” arXiv preprint arXiv:1610.01439, 2016.

[9] S. Chen, M. Fazlyab, M. Morari, G. J. Pappas, and V. M. Preciado, “Learning Lyapunov functions
for hybrid systems,” in Proceedings of the 24nd ACM International Conference on Hybrid Systems:
Computation and Control, 2021.

[10] H. Yin, P. Seiler, and M. Arcak, “Stability analysis using quadratic constraints for systems with neural
network controllers,” arXiv preprint arXiv:2006.07579, 2020.

[11] D. Ahmed, A. Peruffo, and A. Abate, “Automated and sound synthesis of Lyapunov functions with
smt solvers,” in International Conference on Tools and Algorithms for the Construction and Analysis
of Systems. Springer, 2020, pp. 97–114.

[12] A. Abate, D. Ahmed, M. Giacobbe, and A. Peruffo, “Formal synthesis of Lyapunov neural networks,”
IEEE Control Systems Letters, vol. 5, no. 3, pp. 773–778, 2020.

[13] Y.-C. Chang, N. Roohi, and S. Gao, “Neural Lyapunov control,” arXiv preprint arXiv:2005.00611,
2020.

[14] A. A. Ahmadi and P. A. Parrilo, “Non-monotonic Lyapunov functions for stability of discrete time
nonlinear and switched systems,” in 2008 47th IEEE Conference on Decision and Control. IEEE,
2008, pp. 614–621.

[15] R. Bobiti and M. Lazar, “A sampling approach to finding Lyapunov functions for nonlinear discrete-time
systems,” in 2016 European Control Conference (ECC). IEEE, 2016, pp. 561–566.

[16] Y. Nesterov, “Cutting plane algorithms from analytic centers: efficiency estimates,” Mathematical Pro-
gramming, vol. 69, no. 1, pp. 149–176, 1995.

[17] D. S. Atkinson and P. M. Vaidya, “A cutting plane algorithm for convex programming that uses analytic
centers,” Mathematical Programming, vol. 69, no. 1-3, pp. 1–43, 1995.

[18] S. Boyd and L. Vandenberghe, Convex optimization. Cambridge university press, 2004.

13
[19] P. Belotti, C. Kirches, S. Leyffer, J. Linderoth, J. Luedtke, and A. Mahajan, “Mixed-integer nonlinear
optimization,” Acta Numerica, vol. 22, p. 1, 2013.

[20] P. Belotti, J. Lee, L. Liberti, F. Margot, and A. Wächter, “Branching and bounds tightening techniques
for non-convex minlp,” Optimization Methods & Software, vol. 24, no. 4-5, pp. 597–634, 2009.

[21] Gurobi Optimization, LLC, “Gurobi Optimizer Reference Manual,” 2021. [Online]. Available:
https://www.gurobi.com

[22] V. Tjeng, K. Xiao, and R. Tedrake, “Evaluating robustness of neural networks with mixed integer
programming,” arXiv preprint arXiv:1711.07356, 2017.

[23] C.-H. Cheng, G. Nührenberg, and H. Ruess, “Maximum resilience of artificial neural networks,” in
International Symposium on Automated Technology for Verification and Analysis. Springer, 2017, pp.
251–268.

[24] E. Wong and Z. Kolter, “Provable defenses against adversarial examples via the convex outer adversarial
polytope,” in International Conference on Machine Learning, 2018, pp. 5286–5295.

[25] J. Sun, K.-C. Toh, and G. Zhao, “An analytic center cutting plane method for semidefinite feasibility
problems,” Mathematics of Operations Research, vol. 27, no. 2, pp. 332–346, 2002.

[26] K. Y. Xiao, V. Tjeng, N. M. Shafiullah, and A. Madry, “Training for faster adversarial robustness
verification via inducing relu stability,” arXiv preprint arXiv:1809.03008, 2018.

[27] D. Coutinho and C. E. de Souza, “Local stability analysis and domain of attraction estimation for
a class of uncertain nonlinear discrete-time systems,” International Journal of Robust and Nonlinear
Control, vol. 23, no. 13, pp. 1456–1471, 2013.

[28] F. Chollet et al. (2015) Keras. [Online]. Available: https://github.com/fchollet/keras

[29] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint
arXiv:1412.6980, 2014.

[30] A. I. Doban and M. Lazar, “Computation of Lyapunov functions for nonlinear differential equations via
a massera-type construction,” IEEE Transactions on Automatic Control, vol. 63, no. 5, pp. 1259–1272,
2017.

14