Learning and Verifying Maximal Taylor-Neural Lyapunov functions
Abstract
We introduce a novel neural network architecture, termed Taylor-neural Lyapunov functions, designed to approximate Lyapunov functions with formal certification. This architecture innovatively encodes local approximations and extends them globally by leveraging neural networks to approximate the residuals. Our method recasts the problem of estimating the largest region of attraction—specifically for maximal Lyapunov functions—into a learning problem, ensuring convergence around the origin through robust control theory. Physics-informed machine learning techniques further refine the estimation of the largest region of attraction. Remarkably, this method is versatile, operating effectively even without simulated data points. We validate the efficacy of our approach by providing numerical certificates of convergence across multiple examples. Our proposed methodology not only competes closely with state-of-the-art approaches, such as sum-of-squares and LyZNet, but also achieves comparable results even in the absence of simulated data. This work represents a significant advancement in control theory, with broad potential applications in the design of stable control systems and beyond.
Stability of nonlinear systems, Neural networks, Robust control, Machine learning, Region of attraction
1 Introduction
Dynamical systems apply to many engineering technologies and natural phenomena [1], and thus the analysis of their properties provides key insights. The most fundamental of these properties is stability, which ensures the evolution of a dynamic system towards an equilibrium state. The predominant paradigm in stability analysis is the Lyapunov approach, which seeks to identify an energy function for the system [2]. This kind of stability certificate has been demonstrated to be a valuable tool due to its versatility, as it can be readily applied to a range of contexts, including controlled systems [3], performance certification [4], high- or infinite-dimensional systems [5], and discrete-time systems [2].
Nevertheless, discovering a Lyapunov function for a general system represents a significant challenge, as evidenced by decades of literature on the subject. In the context of linear, time-invariant systems, it is well established that the existence of a quadratic Lyapunov function is equivalent to global exponential stability [6]. Furthermore, the determination of a quadratic Lyapunov function is equivalent to the resolution of a linear matrix inequality, for which there exist efficient numerical solvers [7]. In general, for non-linear systems, the Lyapunov function is not quadratic, and there is then no general procedure [2].
Furthermore, the stability of a dynamical system may be constrained to a limited region around an equilibrium, called a region of attraction. This region includes all initial states that will evolve towards the given equilibrium and may not coincide with the entire state space. Consequently, an additional challenge is to compute a Lyapunov function that leads to the largest region of attraction. Such functions are known as a maximal Lyapunov function [8].
For polynomial systems, sum-of-squares techniques have been investigated for estimating a maximal Lyapunov function in [9, 10, 11]. However, this approach suffers from numerical errors when dealing with high-dimensional systems and does not accurately approximate the region of attraction for stiff systems [12]. For a more general class of systems, rational Lyapunov functions have been considered in [8, 13, 14] together with an algorithm to find a maximal Lyapunov function. However, rational Lyapunov functions suffer from the lack of efficient numerical tools. Considering quadratic functions, robust theory encapsulates non-linearities in a cone to compute an inner estimate of the region of attraction [15]. This approach is quite conservative and leads to a poor estimate of the maximal region of attraction for complex systems.
In this work, we plan to take advantage of the physics-informed machine learning paradigm [16, 17]. The idea is to approximate a solution to a differential equation by expressing it as a dynamical constraint in the learning problem. This approach has been proven to be successful in many applications [18, 19, 20], and it has been recently applied to Lyapunov functions in [21, 22] for instance. As investigated in [12], the largest region of attraction will be estimated using Zubov’s theory [23]. More specifically, we will translate part of the methodology described in [23, p91] with series expansion to the neural network case.
1.1 Main contributions
The current literature on neural Lyapunov functions does not rely on physics biases to improve the convergence properties. In this paper, we propose the following contributions:
-
•
A Taylor-based neural network as a universal approximation for Lyapunov functions (discrepancy bias);
-
•
A new loss function to apply constraints on a null set (learning bias);
-
•
A new training algorithm using the Taylor decomposition to enforce local stability (inductive bias);
-
•
A new sampling methodology to certify that the neural approximation of the Lyapunov function is a Lyapunov function.
We claim that these modifications enable us to discard the use of simulated data (observation bias) and improve the robustness of the algorithm, which means that the training algorithm more often converges to the optimal solution, independently of the initialization.
1.2 Background and related work
As first noted by [24] and enlightened more recently by [25] and in the recent survey by [26], it is possible to construct Lyapunov functions that are neural networks. The seminal work by [27] led to the non-convex optimization problem that the neural network Lyapunov function must satisfy. It also showed the approximation capabilities of neural networks but nothing was conducted regarding the region of attraction or the robustness of the training. In 2019, [28] focused on learning Lyapunov functions with a specific architecture to enforce some properties of the Lyapunov function. However, they used this knowledge for an enhanced learning of stable dynamical systems.
The key breakthrough was with the Physics-Informed Machine Learning framework [16] which incorporates physical priors in the form of a dynamical constraint. This was later discussed and enlarged by [17]. This new framework fits perfectly with the Lyapunov methodology. In fact, a Lyapunov function must satisfy constraints expressed in terms of differential inequalities. Such an approach has been investigated in many papers in the last three years.
The authors of [29] proposed to use a neural Lyapunov function to derive a control law for the system with a provable guarantee of stability. The control was obtained as the solution to the constrained optimization problem. The safety is ensured by, first, estimating the region of attraction and, secondly, by using a falsifier which penalizes the outer estimate of the region of attraction. The region of attraction is, however, computed using a regularization agent, leading to a result that is highly sensitive to the hyper-parameter, not guaranteed to converge, and often conservative.
The lack of formal guarantees has been investigated more thoroughly by [30]. They generate Lyapunov neural networks using symbolic computations which offer a trade-off between analytical and numerical methods. The method also relies on training using a verifier which is building counter-examples to get a more robust learning. However, the method works only for global asymptotically stable systems, which means that the region of attraction is . This is a very restrictive assumption since many nonlinear systems have several equilibrium points and thus are not globally asymptotically stable.
Control has been investigated by [31] and [21]. They both use neural networks to estimate a control Lyapunov function, which leads to a stable system under the designed control law. The first one focuses on safety in robotic applications while the second one provides a better neural network architecture that enforces positive definiteness. Finally, [22, 32] worked on a similar topic, trying to combine all the ideas previously cited into one. They estimated the region of attraction of an equilibrium of a partially unknown nonlinear autonomous system using satisfiability modulo theories as a verifier. All these works obtain a rather conservative estimate of the region of attraction, and the obtained estimate is not robust across several trainings. Similar topics with the same conclusions have been investigated in a discrete-time context by [33, 34, 35, 36] to cite a few.
Recent work by [12] aims at learning a neural Lyapunov function using Zubov’s theorem to maximize the region of attraction [23]. Consequently, the learning is more robust and almost always estimates the true region of attraction. However, as done by [33], a simulator is needed to compute if some initial states are leading to an unstable equilibrium point. This knowledge is used as data to enhance learning. In the case of controller synthesis, for example, we often can’t simulate the system. This highlights the need for a pure learning procedure for the maximum region of attraction of a general nonlinear dynamical system.
From the previous papers, it appears that very little work was done on the training algorithm. The introduction of a falsifier or verifier was the only addition to certify the training a posteriori. Consequently, current works are very sensitive to initialization. Our main contribution is to improve the training algorithm designed by [21] to introduce a robust estimation of the region of attraction. Moreover, the neural architecture has been changed such that we can derive a universal Lyapunov function approximation theorem. Similarly to the work by [12], we use Zubov’s theorem to maximize the region of attraction but we do not use an external simulator to get some additional data.
1.3 Organization
The organization of the paper is as follows. In Section 2, some preliminaries are given that lead to the formulation of the problem. Section 3 focuses on the construction of Taylor-Neural Lyapunov functions. Section 4 focuses on the efficient learning of such functions. Section 5 explores the certification aspect. Section 6 is devoted to simulations and discussion. Section 7 concludes the article.
1.4 Notation
Throughout the paper, refers to the set of real numbers, is the set of three-times differentiable functions from to . For and , we use the notation , referring to the element-wise operation. For , we define the rectified linear function as . For , the euclidean scalar product is written as , the norm is defined as and the infinity norm is . For two squared symmetric matrices and of the same size, means that has strictly negative eigenvalues. For a discrete set , refers to its cardinal.
2 Preliminaries and Problem statement
This section formalizes the problem and introduces the working assumptions.
2.1 Problem formulation
We consider the following dynamical system:
(1) |
where is the dimension of the system, is Lipschitz continuous and possibly non-linear. Under these conditions, there exists a unique solution to the previous problem which is forward complete [37].
We assume without loss of generality that the origin is an equilibrium point of , i.e. . We are interested in showing the asymptotic stability of the origin as defined in [38, Definition 1.3] and reminded below.
Definition 1
The origin of (1) is said to be locally asymptotically stable in the open and connected set containing the origin if for each there exists such that
and .
is called a region of attraction of (1) around the origin.
First, let us define a Lyapunov function similar to that proposed in [2, Theorem 3.1].
Definition 2
A continuously differentiable function is said to be a local Lyapunov function for if
(2a) | ||||
(2b) | ||||
(2c) |
The Lyapunov direct method [2, Theorem 3.1] provides a way to demonstrate asymptotic stability in a region of attraction.
Theorem 1
If there exists a local Lyapunov function then the origin is asymptotically stable and a region of attraction is .
The existence of a region of attraction is guaranteed by the following assumption on .
Assumption 1
Assume that in (1) can be written as
(3) |
such that has all eigenvalues with strictly negative real parts.
The Lyapunov indirect method [38, Theorem 12.6] then concludes that there exists a local Lyapunov function for (1). Consequently, the set
is not empty.
Let the following application be such that
The function is the volume of the region of attraction related to a Lyapunov function . This defines a relation of order between the Lyapunov function such as is equivalent to for any . Since for any we get , then is upper-bounded by . The following optimization problem is well-defined:
From the original work of Zubov [23] and later about maximum Lyapunov functions in [8], we define
(4) |
and then get the following set inclusion:
We are now able to state the problem statement of this article.
Problem 2.2.
We want to find a approximation of a Lyapunov function leading to the largest region of attraction for the dynamical system (1) in terms of volume, i.e.
2.2 General remarks
Note that the assumption done when defining the system in (1) is not restrictive, as any open interval can be rescaled and shifted to .
In the case of , one can consider the non-linear transformation to map to . However, global asymptotic stability () requires radial unboundedness of the Lyapunov function ( when ) [2]. As in [8], let a maximal Lyapunov function as
. If is a local Lyapunov function as defined previously then is a Lyapunov function and . Consequently, the radial unboundedness of is equivalent to
This condition can be added to ensure global stability, but this point will not be discussed further in this article.
Concerning the definition of the region of attraction, it differs slightly from the classical ones in [38, Section 12.2] or [2, Section 3.1], where . The proposed version decreased the number of parameters by scaling the local Lyapunov function such that on the boundary of the region of attraction.
Regarding Assumption 1, it is not very restrictive. Indeed, any analytic function will admit such a decomposition [39]. The constant term can be removed by an appropriate change of variable such that the origin becomes an equilibrium point for (1). If at least one eigenvalue of has a strictly positive real part, then the equilibrium point is not asymptotically stable [38, Theorem 12.2]. However, if there is an eigenvalue on the imaginary axis, the equilibrium point might still be asymptotically stable [38, Example 12.1]. We do not deal with these corner cases in this article.
3 Taylor-Neural Lyapunov functions
Finding a Lyapunov function in the set is challenging, it boils down to solving equations (2) together with . This system of equations is generally numerically intractable. Similar to robust control theory where the problem is conservatively relaxed by considering quadratic Lyapunov functions, we introduce Taylor-neural Lyapunov functions as universal approximations of maximal Lyapunov functions.
First, let us pick a Lyapunov function in . Using Taylor expansion in several variables [40, Theorem 5.4], we get that for any :
(5) |
where is the Hessian of evaluated at , is the gradient of and .
The following results are classical and related to the indirect Lyapunov method.
Lemma 3.3.
-
1.
and is a symmetric definite positive matrix.
-
2.
Proof 3.4.
- 1.
- 2.
Using the previous lemma, we get:
(6) |
We now introduce the notion of neural network residual.
Definition 3.5.
Let the neural network residual where is the number of neurons per layer be
(7) |
where is the number of hidden layers, the weights , , , biases and
The parameters of the neural network are packed into the tensor .
The activation function is of class and is bounded up to the order .
A Taylor-neural Lyapunov function can then be proposed as
(8) |
where such that , and is a given basis of .
Proposition 3.6.
Under Assumption 1, for any , there exist and such that is a Lyapunov function and
(9) |
Proof 3.7.
We provide a formal proof in Appendix .1 and briefly outline it here. Select one in and write it as in (5). For , the proposed Taylor-neural Lyapunov function in (8) is quadratic and as close to as desired in a sufficiently small neighborhood around the origin. Outside of , one can choose to be as close to as desired provided a sufficient number of neurons [41, Theorem 4], so that the region of attraction is approximated as well as desired and that is a Lyapunov function.
Remark 3.8.
Note that compared to all other papers on the subject so far [29, 30, 27, 31, 21, 22, 12], none could argue that the proposed neural network approximation was a Lyapunov function. In [21], the authors pointed out this fact by noticing that a neural network approximation of a Lyapunov function usually does not have a negative time derivative everywhere around the origin. Using a third-order Taylor expansion prevents this phenomenon, leading to the previous proposition.
We have used here a physics-informed machine learning approach since we introduce a neural network approximation of a Lyapunov function and the constraints written in (2) translate into partial differential equations. Next section will focus on the optimization problem formulation.
4 Learning a Taylor-Neural Lyapunov function
In this section, we discuss how to learn a Taylor-Neural Lyapunov function. We formulate the training problem as a suitable constrained optimization problem, encoding the properties of Lyapunov functions, and we propose an efficient solver for the resulting problem.
4.1 Optimization problem formulation
The Taylor-neural Lyapunov function proposed in (8) does not enforce any properties of a Lyapunov function presented in (2a)-(2c). We want to encode in the neural network architecture as many constraints as possible to minimize the work of the learning procedure. To that extent, we introduce the following slightly modified function for :
(10) |
where is a smooth approximation of the absolute value such that for any , . Note that can be chosen as close to as desired.
Remark 4.9.
From now on, to ease the reading, we will make the following abuse of notation .
For , is positive definite by construction as in (2b) and (2a). For to be a Lyapunov function, i.e. , it remains to satisfy (2c). Since we also approximate the largest region of attraction, we want . Combining these two facts leads to the formulation:
(11) |
where is definite positive.
The positive definiteness of in is equivalent to the existence of such that for all . Equality (11) then translates into
(12) |
where, to ease the writing, we use .
Remark 4.10.
In [12], they have a similar equality. The authors state that equality constraints are much better handled in training algorithms. However, in their case, they must pick as a neural network which prevents the equality from being strictly enforced for all since the left-hand side can never equal the right-hand one.
Integrating (12) leads to
(13) |
Any Lyapunov function in will satisfy (13). To ensure that , that is, to maximize the region of attraction, one needs to add the objective that for . This leads to the following optimization problem:
(14) |
4.2 Numerical solution to the constrained optimization problem
The constrained optimization problem (14) is numerically intractable because of the integrals and the dynamical constraint. Therefore, we propose the training method depicted in Algorithm 1, which consists of the following routines:
-
1.
sampling the integral in the loss and constraints,
-
2.
formulate the problem as a Lagrangian optimization problem defined on sampled points,
-
3.
apply a primal-dual strategy.
These steps will lead to a practical algorithm for solving (14) which is numerically efficient.
4.2.1 Sampling of the integrals
The constraint is an integral over part of the domain . It is classical [17] to use a uniform sampling over the whole domain using, for instance, latin-hyperspace sampling. Considering that we draw points from the uniform distribution, we get the discrete set . The novelty here comes to consider only a part of this domain, leading to the following approximation:
(15) |
where and is the cardinal of .
Concerning the objective, one can do something similar by sampling points on the curve . We first need to draw points from to obtain the discrete set . Then we scale these points so that they fall on the boundary of the set , i.e., for each point in , we create the variable such that . We then get
With the previous definitions, and we get the following approximation of the integral:
(16) |
Remark 4.11.
The previous approximation is correct if the set is a star domain111A set is a star domain at if for all the line-segment lies in . at the origin. Otherwise, there might not be a unique for each point, and consequently, the sampling will not be uniform at the boundary.
4.2.2 Lagrangian formulation
The optimization problem in (17) is a learning problem under constraints. Some techniques to solve them are explored in [42] and we will follow the strategy mentioned in [43, 44]. Let first define the following costs:
(18) |
We use the Lagrange multipliers . The extended cost is expressed as:
(19) |
Problem (14) is then equivalent to solving
(20) |
Since , a solution to the previous problem would be with , ensuring then for all . As discussed in the next section, this rephrased problem can be solved using primal-dual optimization.
4.2.3 Training algorithm
The training algorithm is divided into several parts, each contributing to the overall robustness.
The primal-dual algorithm
The training algorithm is based on primal-dual optimization [45]. A similar training scheme has been used and investigated in [43, 46] for physics-informed machine learning problems and has shown great potential to improve the robustness of the training algorithm (i.e. decrease the sensitivity to the initialization). The main idea of this algorithm is to alternate between solving the and problems. The primal problem is expressed in terms of the primal variables , and a first-order algorithm is generally written as:
(21) |
where are initial random values and is the primal learning rate.
Remark 4.12.
The dual problem aims at approximating the solution to the problem. Let the dual variables be , that leads to the following first-order optimization scheme:
(22) |
with being the dual learning rate.
Remark 4.13.
In light of curriculum learning [48], it has been shown that the constraint containing derivatives usually brings complexity into the original optimization problem [49]. Consequently, a solution to get more robust training is to start with and increase their values. Thus, the objective will be taken into account at a later stage, putting the focus on the constraint first. Note that is increasing since both and are positive.
Projection subroutine
The advantage of the Taylor-neural Lyapunov functions lies in its explainability locally around the origin. Since , using Lemma 3.3, the following equation should hold:
The previous inequality implies that the matrix must belong to the positive cone
Checking if a matrix belongs to this positive cone is a semi-definite program. However, after one initial step, the obtained does not necessarily belong to . To enforce this property, a projection onto the positive cone is needed, defined as
(23) |
where the norm on the symmetric definite matrix cone is defined as the spectral radius. However, this problem is not a semidefinite program. Using Schur’s complement [7, Section 2.1] leads to the following equivalent formulation for :
Matrix is linear in each of its variables. The projection can be rewritten into a linear matrix inequality problem and thus solved efficiently using e.g. cvxpy [50] since it is a semi-definite program (SDP):
(24) |
Boundary estimate subroutine
To estimate correctly the boundary cost (16) which is related to the largest region of attraction, one must choose the correct parameters such that, for a given sampling , we get . We use a first-order optimization scheme, defined as follows for :
(25) |
(26) |
Lemma 4.14.
Let , such that .
If there exists a unique such that , then the following holds:
Proof 4.15.
Let . If there exists a unique such that , then the function can be equivalently written as:
Since and , the sequence is decreasing to . The smallest attainable value in that case is . Since , is positive when . Consequently, at any .
Since is a geometric sequence with a common ratio , there exists , such that and . After this , the largest attainable value is since .
Combining these two facts leads to
(27) |
which concludes the proof.
The previous lemma ensures that we can approximate the the boundary of arbitrarily close provided that the learning rate is sufficiently small. In some specific cases, we can even prove that .
Proposition 4.16.
Under the same conditions as in Lemma 4.14 together with
-
1.
;
-
2.
is strictly decreasing with and is diverging,
then the following holds:
Proof 4.17.
A sketch of the proof is that Lemma 4.14 applies but because of the divergence of the series together with 1), there will be such that . We can apply Lemma 4.14 again but has decreased and consequently the convergence interval (27) is tighter around . Since strictly decreases to , then can be as close as desired to .
Resampling subroutine
Inequality (15) holds if the law of large number is verified, and consequently, is large. This might impact the training time and the efficiency of the solver [51]. That is why we can consider resampling regularly during the training with fewer points [52]. This has two advantages:
-
1.
it keeps the computational burden low,
- 2.
Remark 4.18.
Note that we do not resample because each original is associated with a parameter .
Stopping criteria
Since is an universal approximation of the optimal Lyapunov function, that means the optimal solution has for any sampling and . We will stop the training when for multiple different samplings and where is the machine precision.
We might want to stop the training early if the algorithm has converged to a suboptimal solution. This will be indicated by a slow variation of both and . Then a possibility is to add a refine step which consists of freezing some variables and updating the others for and . This will force us to find a valid Lyapunov function and forget about the optimality.
Once stopped, the Lyapunov function can be verified on multiple different sampling. If there is one point such that , then one can rescale the Lyapunov function to exclude this point, i.e.
5 Verifiying a Taylor-Neural Lyapunov function
The previous algorithm is not guaranteed to converge to a valid Lyapunov function, which means that there might exist a point for which the constraint . Existing works such as [12, 33, 31] are using a verifier to ensure that the optimized neural Lyapunov function is indeed a Lyapunov function. This does not extend straightforwardly to our case. In this section, we will instead derive conditions on the sampling , which guarantees that in a compact set is strictly included in .
5.1 Lipschitz continuity of
We first show that has bounded variations in the following two lemmas.
Lemma 5.19.
The function is Lipschitz continuous on , i.e. for any
Proof 5.20.
Since is continuous and almost everywhere differentiable for any , its derivative is bounded. Therefore, is Lipschitz continuous and is the maximum over all of the previous derivative on .
There have been some works focusing on computing the Lipschitz constant of neural networks [54, 55]. However, in our case, is the Lipschitz constant of the derivative of a neural network. Considering hyperbolic tangent as the activation function, a symbolic upper bound of can be derived manually but this goes beyond the scope of this paper.
Lemma 5.21.
Let be Lipschitz continuous on with Lipschitz constant . The function is Lipschitz continuous on with Lipschitz constant
Proof 5.22.
Since is Lipschitz continuous on , it is also bounded, and the same holds for . The result of this proposition comes from the following inequality:
Using the definition of Lipschitz continuity leads to the expression of given above.
One can note that depends both on and the maximum of but will also increase with dimension.
5.2 Local and global certifications
Based on the work in [56], one can estimate the size of the balls around each sampling point in which is negative. The following lemma provides such an estimate.
Lemma 5.23.
Assume and let . Then, for any , we have .
Proof 5.24.
Since, for a given sampling, and , then . Let , then the following holds:
This leads to the main statement of this lemma.
The previous lemma shows a local result. We would like instead to find a sampling that guarantees that the union of all balls is a cover of the region of attraction. To that extent, we introduce the following definition.
Definition 5.25.
A sampling is said to be uniform with parameters when
for with .
Note that is a cover of . To find a region of attraction, we must find a level set of the Lyapunov function . Let, for any , the following subset of the region of attraction:
The largest certified region of attraction based on a uniform sampling is estimated in the following proposition.
Proposition 5.26.
Let such that is a region of attraction. Consider a uniform sampling of parameters such that
-
•
;
-
•
,
where
If on , then the set is a certified region of attraction with
(28) |
where
Proof 5.27.
First of all, the approximation properties of Taylor-neural Lyapunov functions together with projection (24) ensures the existence of as introduced in the proposition. Consequently, for any , is definite negative.
For any such that , we get that there exists such that
Consequently, we get which in turn results in
Since on , from Lemma 5.23, we get on .
The largest region of attraction in is then obtained by solving the optimization problem (28).
Since is an open set, we must have and . Therefore, it is impossible to confirm that the entire set is indeed a region of attraction. Moreover, since on , we get the following:
Consequently, the sampling must have a finer grain if
-
1.
is closer to : the certified region of attraction is larger;
-
2.
is smaller: the constraint (12) is close to violation;
-
3.
is large: the Lyapunov function has very fast variations or the system is stiff ( and is large);
-
4.
is small: we cannot find a large region of attraction around the origin;
-
5.
is large.
The previous proposition also helps us understand how fine the sampling should be during training to obtain a region of attraction that is meaningful. The previous proposition practically highlights that we need to have a finer sampling when we are close to the boundary of the region of attraction and around the origin.
There are still some computational concerns regarding optimization problem (28):
-
1.
Finding the largest level set included in might be a challenge in high dimension and one can consider a greedy algorithm to estimate it.
- 2.
6 Simulations and discussion
In this section, we present and discuss the results of applying our proposed method to different systems, both globally and locally stable. In addition, we compare our solution with state-of-the-art alternatives. We conclude by discussing the robustness of the training algorithm to different initializations to evaluate the consistency of the solutions it provides. To highlight the robustness of the method, we chose the same hyperparameters for Algorithm 1 throughout this section. These parameters are listed in Table 1.
Parameter | |||||||
Value |
6.1 Simulations
6.1.1 Globally stable system
We propose to consider first the following system:
(29) |
We can rewrite it as where belongs to the polytope with . This system is globally stable because there exists a common quadratic Lyapunov function to all :
We use the method described in this paper with the hyperparameters in Table 1 and a maximum number of epochs of . The result of one training is displayed in Figure 1.
We notice that the early-stopping conditions are always reached. The difficulty with global systems is that the condition on cannot be enforced. Compared to other works [12, 21], our methodology is capable of finding regions of attraction that are the whole domain .
![Refer to caption](x1.png)
6.1.2 Locally stable equilibrium point
The second system considered is the model of a generator [38, Example 11.2], described as follows:
(30) |
Since there are multiple equilibrium points, it is well known that this system is not globally stable. The system is locally stable because it satisfies Assumption 1. In [38, Example 12.6], they provide the following Lyapunov function
which gives a rather conservative region of attraction.
We run the algorithm with the same hyperparameters as previously and a maximum number of epochs of . The result is displayed in Figure 2. This time, the stopping conditions were never reached. We can see that the blue dots, which correspond to the boundary estimate are very close to the real boundary, which is a success. From the flow arrows, it seems that the region of attraction is very close to the real one (and much larger than the one obtained using ). We can see that the level lines are not elliptical, which implies that the Lyapunov function is not quadratic. This indicates that higher-order terms in the Taylor decomposition have been learned successfully.
![Refer to caption](x2.png)
6.1.3 Van der Pol oscillator
The last example considers a special case of Van der Pole oscillator [57]:
(31) |
We investigate the case . This system has a polynomial structure which makes the use of SOS Lyapunov functions possible [11]. The region of attraction has a nonconvex shape which becomes stiffer as increases.
We trained the Taylor-neural Lyapunov function using the same set of hyper-parameters with a maximum of epochs. The result is shown in Figure 3. Similarly to the previous example, the optimal region of attraction is well-estimated. The obtained result is very close to the SOS result (dashed line in the figure). The evolution of the training loss is shown in Figure 4. One can see that without early stop around epochs, the loss will have spikes which indicates bad fitting. This shows that it is important to monitor the loss during training and that it might be necessary to enforce early stopping for better convergence.
![Refer to caption](x3.png)
![Refer to caption](x4.png)
LyzNet [12] | SOS [11] | This method | |
RoA coverage | |||
System dimensions | Potentially high | Low | Intermediate |
System characteristic | Strictly stable | Polynomial | Strictly stable |
Data | Simulated | None | None |
We compare our results with SOS [11] and the LyzNet method developed in [12] and the results are presented in Table 2. A high-resolution estimation of the largest region of attraction is estimated using numerical integration. If the trajectory is close to the origin after some time, we consider that the initial point is part of the region of attraction. We computed points uniformly spread in the region of attraction based on this criteria. We then checked if these points were part of the region of attraction for the SOS formulation, LyzNet, and our method. We found that the best-performing method in this example is LyzNet, closely followed by SOS, and then our method is behind by approximately . Our method still gives a region of attraction very close to the optimal one (). However, Lyznet is much slower and uses external data generated by the simulator, which significantly restricts the interest in the method. Indeed, using an external simulator brings no guarantee if the point is stable or not. Moreover, it involves spending time simulating points.
Note that we also tried the algorithm for larger values of , but the stiffness of the system makes the algorithm diverge.
6.2 Robustness analysis
Investigating how the training algorithm differs when initialized differently is of tremendous importance. A robust training algorithm will almost always produce the same region of attraction, regardless of the initialization.
To evaluate robustness, the training algorithm is run times with different initial states, and the obtained regions of attraction are denoted for . Results are reported in Table 3. The numbers in the shared volume column indicate the probability that a point that is part of one learned region of attraction belongs to another learned region of attraction. The IoU column refers to the intersection over the union (also called the Jaccard similarity index). This is the percentage of points in the largest region of attraction which belongs to the smallest region of attraction .
Globally stable system (29) shows high percentages for both, indicating a very robust algorithm. This can be explained easily since there exist many Lyapunov functions that will return the optimal region of attraction in a bounded domain. Another explanation comes from the verification section, since variations in the region of attraction are close to the boundary of .
The locally stable system (30) has a more challenging region of attraction. However, the percentages are still relatively high, which indicates a good convergence of the algorithm. Looking more carefully at the plots shows convergence to suboptimal regions of attraction in some rare cases. This significantly decreases the shared and common volumes.
The last example has a more complex region of attraction. Without any surprise, it is harder to learn, but the region of attraction (even if often suboptimal) is relatively consistent over the tries. One region of attraction was relatively small, significantly impacting the percentage of shared volumes. To mitigate this issue, the solution might be to look at the training losses and stop the training when the loss is relatively low, and this time might be reached at different epochs for different initializations. Another issue comes from some non-connected points which are also identified as stable. A mitigation strategy could be to consider “ensemble learning”, where the final region of attraction is the intersection of multiple regions of attraction, making the process more stable but also more conservative. However, considering the large common volume in all cases, this technique would probably only remove corner-case points and keep a good estimate of the region of attraction.
7 Conclusion and perspectives
This paper proposes a new class of neural networks which rely on Taylor expansion. The approximation capabilities of such neural networks have been proven for candidate Lyapunov functions. The classical training algorithm based on gradient descent had been adapted to this case to provide certification that the obtained candidate Lyapunov function is indeed valid around the origin. The paper also addresses the issue of estimating the largest region of attraction, leveraging Zubov’s theorem. The efficiency of the approach has been demonstrated in the estimation of the largest region of attraction, where results comparable to the state of the art have been obtained on some examples. An extension to the numerical certification of the obtained Lyapunov function has also been discussed.
However, numerical experiments have shown that some improvement is possible in estimating the region of attraction. In some cases, the very poor convergence of the algorithms suggests that a better initialization procedure should be investigated. One solution would be to consider state-of-the-art algorithms in robust control, for example. In the same vein, alternative solutions for maximizing the region of attraction should be considered to mitigate the convergence to a bad local minimum. Research from the machine learning side can provide some insights.
The proposed method opens the way for new research directions. The capacity to estimate the largest region of attraction without any data introduces a fundamental change compared to other methodologies. The extension to controller synthesis is one of the most promising research directions, but the consideration of much more complex systems (of infinite dimension or uncertain, for instance) is another avenue. The investigation of other sampling strategies to ensure a better convergence is left for future research.
.1 Proof of Proposition 3.6
Approximation of the region of attraction
Positive-definiteness
Since the activation function , then is bounded. Consequently, we get .
Note that and , consequently, there is such that
(34) |
Let and . The universal approximation theorem [41, Theorem 4] states that for any , there exist , weights and biases such that
(35) |
Then, the inequality (2b) holds.
Negative time-derivative
Note first that
Consequently, similarly to the proof of Lemma 3.3, we get:
Note that the universal approximation [41, Theorem 4] also holds for the first derivative, we get that for any , there exist , weights, and biases such that
Using Lemma 3.3.2., a similar reasoning as in the previous subsection implies that on .
Conclusion
For , there exist , weights and biases such that the universal approximation theorem holds for then is a Lyapunov function. By optimality of the region of attraction, the right part of the inclusion (9) holds.
References
- [1] K. J. Åström and R. Murray, Feedback systems: an introduction for scientists and engineers. Princeton university press, 2021.
- [2] H. K. Khalil, Nonlinear Systems, ser. Pearson Education. Prentice Hall, 2002.
- [3] E. D. Sontag, Mathematical control theory: deterministic finite dimensional systems. Springer Science & Business Media, 2013, vol. 6.
- [4] J. Veenman, C. W. Scherer, and H. Köroğlu, “Robust stability and performance analysis based on integral quadratic constraints,” European Journal of Control, vol. 31, pp. 1–32, 2016.
- [5] R. F. Curtain and H. Zwart, An introduction to infinite-dimensional linear systems theory. Springer Science & Business Media, 2012, vol. 21.
- [6] A. M. Lyapunov, “The general problem of the stability of motion,” International journal of control, vol. 55, no. 3, pp. 531–534, 1992.
- [7] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear matrix inequalities in system and control theory. SIAM, 1994.
- [8] A. Vannelli and M. Vidyasagar, “Maximal lyapunov functions and domains of attraction for autonomous nonlinear systems,” Automatica, vol. 21, no. 1, pp. 69–80, 1985.
- [9] W. Tan and A. Packard, “Stability region analysis using polynomial and composite polynomial lyapunov functions and sum-of-squares programming,” IEEE Transactions on Automatic Control, vol. 53, no. 2, pp. 565–571, 2008.
- [10] M. Jones and M. M. Peet, “Converse lyapunov functions and converging inner approximations to maximal regions of attraction of nonlinear systems,” in 2021 60th IEEE Conference on Decision and Control (CDC). IEEE, 2021, pp. 5312–5319.
- [11] D. Henrion and M. Korda, “Convex computation of the region of attraction of polynomial control systems,” IEEE Transactions on Automatic Control, vol. 59, no. 2, pp. 297–312, 2013.
- [12] J. Liu, Y. Meng, M. Fitzsimmons, and R. Zhou, “Physics-Informed Neural Network Lyapunov Functions: PDE Characterization, Learning, and Verification,” arXiv preprint arXiv:2312.09131, 2023.
- [13] G. Chesi, “Rational lyapunov functions for estimating and controlling the robust domain of attraction,” Automatica, vol. 49, no. 4, pp. 1051–1057, 2013.
- [14] G. Valmorbida and J. Anderson, “Region of attraction estimation using invariant sets and rational lyapunov functions,” Automatica, vol. 75, pp. 37–45, 2017.
- [15] S. Tarbouriech, G. Garcia, J. M. G. da Silva Jr, and I. Queinnec, Stability and stabilization of linear systems with saturating actuators. Springer Science & Business Media, 2011.
- [16] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational physics, 2019.
- [17] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,” Nature Reviews Physics, 2021.
- [18] S. Cai, Z. Mao, Z. Wang, M. Yin, and G. E. Karniadakis, “Physics-informed neural networks (pinns) for fluid mechanics: A review,” Acta Mechanica Sinica, vol. 37, no. 12, pp. 1727–1738, 2021.
- [19] G. Kissas, Y. Yang, E. Hwuang, W. R. Witschey, J. A. Detre, and P. Perdikaris, “Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4d flow mri data using physics-informed neural networks,” Computer Methods in Applied Mechanics and Engineering, vol. 358, p. 112623, 2020.
- [20] Y. Bai, T. Chaolu, and S. Bilige, “The application of improved physics-informed neural network (ipinn) method in finance,” Nonlinear Dynamics, vol. 107, no. 4, pp. 3655–3667, 2022.
- [21] N. Gaby, F. Zhang, and X. Ye, “Lyapunov-net: A deep neural network architecture for lyapunov function approximation,” in 2022 IEEE 61st Conference on Decision and Control (CDC), 2022.
- [22] R. Zhou, T. Quartz, H. De Sterck, and J. Liu, “Neural lyapunov control of unknown nonlinear systems with stability guarantees,” Advances in Neural Information Processing Systems, vol. 35, 2022.
- [23] V. I. Zubov, Methods of AM Lyapunov and their application. US Atomic Energy Commission, 1961, vol. 4439.
- [24] Y. Long and M. M. Bayoumi, “Feedback stabilization: control lyapunov functions modelled by neural networks,” in Proceedings of 32nd IEEE Conference on Decision and Control, vol. 3, 1993.
- [25] T. X. Nghiem, J. Drgoňa, C. Jones, and al., “Physics-informed machine learning for modeling and control of dynamical systems,” in American Control Conference (ACC), 2023.
- [26] C. Dawson, S. Gao, and C. Fan, “Safe control with learned certificates: A survey of neural Lyapunov, barrier, and contraction methods for robotics and control,” IEEE Transactions on Robotics, 2023.
- [27] L. Grüne, “Computing lyapunov functions using deep neural networks,” Journal of Computational Dynamics, vol. 8, no. 2, pp. 131–152, 2021.
- [28] Z. J. Kolter and G. Manek, “Learning stable deep dynamics models,” Advances in neural information processing systems, vol. 32, 2019.
- [29] Y.-C. Chang, N. Roohi, and S. Gao, “Neural Lyapunov control,” Advances in neural information processing systems, vol. 32, 2019.
- [30] A. Abate, D. Ahmed, M. Giacobbe, and A. Peruffo, “Formal synthesis of lyapunov neural networks,” IEEE Control Systems Letters, 2020.
- [31] C. Dawson, Z. Qin, S. Gao, and C. Fan, “Safe nonlinear control using robust neural Lyapunov-barrier functions,” in Conference on Robot Learning. PMLR, 2022, pp. 1724–1735.
- [32] A. Abate, D. Ahmed, A. Edwards, M. Giacobbe, and A. Peruffo, “FOSSIL: a software tool for the formal synthesis of lyapunov functions and barrier certificates using neural networks,” in Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control, 2021, pp. 1–11.
- [33] S. M. Richards, F. Berkenkamp, and A. Krause, “The Lyapunov neural network: Adaptive stability certification for safe learning of dynamical systems,” in Conference on Robot Learning. PMLR, 2018, pp. 466–476.
- [34] A. Mehrjou, M. Ghavamzadeh, and B. Schölkopf, “Neural lyapunov redesign,” arXiv preprint arXiv:2006.03947, 2020.
- [35] H. Dai, B. Landry, L. Yang, M. Pavone, and R. Tedrake, “Lyapunov-stable neural-network control,” arXiv preprint arXiv:2109.14152, 2021.
- [36] S. Mukherjee, J. Drgoňa, A. Tuor, M. Halappanavar, and D. Vrabie, “Neural Lyapunov differentiable predictive control,” in 2022 IEEE 61st Conference on Decision and Control (CDC). IEEE, 2022, pp. 2097–2104.
- [37] D. Angeli and E. D. Sontag, “Forward completeness, unboundedness observability, and their lyapunov characterizations,” Systems & Control Letters, vol. 38, no. 4-5, pp. 209–217, 1999.
- [38] T. Glad and L. Ljung, Control theory. CRC press, 2000.
- [39] S. Krantz and H. Parks, A Primer of Real Analytic Functions, ser. A Primer of Real Analytic Functions. Birkhäuser Boston, 2002.
- [40] R. Coleman, Calculus on normed vector spaces. Springer Science & Business Media, 2012.
- [41] K. Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural networks, vol. 4, no. 2, pp. 251–257, 1991.
- [42] L. Lu, R. Pestourie, W. Yao, Z. Wang, F. Verdugo, and S. G. Johnson, “Physics-informed neural networks with hard constraints for inverse design,” SIAM Journal on Scientific Computing, vol. 43, no. 6, pp. B1105–B1132, 2021.
- [43] M. Barreau, M. Aguiar, J. Liu, and K. H. Johansson, “Physics-informed learning for identification and state reconstruction of traffic density,” in 2021 60th IEEE Conference on Decision and Control (CDC), 2021, pp. 2653–2658.
- [44] M. L. Delle Monache, C. Pasquale, M. Barreau, and R. Stern, “New frontiers of freeway traffic control and estimation,” in 2022 IEEE 61st Conference on Decision and Control (CDC). IEEE, 2022, pp. 6910–6925.
- [45] M. X. Goemans and D. P. Williamson, “The primal-dual method for approximation algorithms and its application to network design problems,” Approximation algorithms for NP-hard problems, pp. 144–191, 1997.
- [46] M. L. Delle Monache, C. Pasquale, M. Barreau, and R. Stern, “New frontiers of freeway traffic control and estimation,” in 2022 IEEE 61st Conference on Decision and Control (CDC), 2022, pp. 6910–6925.
- [47] D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR), San Diego, CA, USA, 2015.
- [48] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in Proceedings of the 26th annual international conference on machine learning, 2009, pp. 41–48.
- [49] P. Rathore, W. Lei, Z. Frangella, L. Lu, and M. Udell, “Challenges in training pinns: A loss landscape perspective,” arXiv preprint arXiv:2402.01868, 2024.
- [50] S. Diamond and S. Boyd, “CVXPY: A Python-embedded modeling language for convex optimization,” Journal of Machine Learning Research, vol. 17, no. 83, pp. 1–5, 2016.
- [51] M. Münzer and C. Bard, “A curriculum-training-based strategy for distributing collocation points during physics-informed neural network training,” arXiv preprint arXiv:2211.11396, 2022.
- [52] A. Daw, J. Bu, S. Wang, P. Perdikaris, and A. Karpatne, “Mitigating propagation failures in physics-informed neural networks using retain-resample-release (r3) sampling,” arXiv preprint arXiv:2207.02338, 2022.
- [53] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
- [54] M. Fazlyab, A. Robey, H. Hassani, M. Morari, and G. Pappas, “Efficient and accurate estimation of Lipschitz constants for deep neural networks,” Advances in neural information processing systems, vol. 32, 2019.
- [55] Y. Ebihara, X. Dai, V. Magron, D. Peaucelle, and S. Tarbouriech, “Local Lipschitz constant computation of relu-fnns: Upper bound computation with exactness verification,” arXiv preprint arXiv:2310.11104, 2023.
- [56] M. Mohri, A. Rostamizadeh, and A. Talwalkar, Foundations of Machine Learning. MIT press, 2018.
- [57] B. Van der Pol, “A theory of the amplitude of free and forced triode vibrations,” Radio Review, vol. 1, no. 701–710, 1920.