Chapter 2

Background on Nonlinear Systems and Control

In this chapter, we review some basic results on the analysis and control of nonlinear
systems. This review is not intended to be exhaustive but to provide the reader with
the necessary background for the results presented in the subsequent chapters. The
results presented in this chapter are standard in the nonlinear systems and control
literature. For detailed discussion and proofs of the results, the reader may refer to
the classic books [72, 76].

2.1 Notation

Throughout this book, the operator | · | is used to denote the absolute value of a scalar
and the operator  ·  is used to denote Euclidean norm of a vector, while we use
 · Q to denote the square of a weighted Euclidean norm, i.e., xQ = x T Qx for all
x ∈ Rn . The symbol Ωr is used to denote the set Ωr := {x ∈ Rn : V (x) ≤ r} where
V is a scalar positive definite, continuous differentiable function and V (0) = 0, and
the operator ‘/’ denotes set subtraction, that is, A/B := {x ∈ Rn : x ∈ A, x ∈ / B}.
The notation R = [r1 r2 ] is used to denote the augmented vector R ∈ Rm+n com-
prising the vectors r1 ∈ Rm and r2 ∈ Rn . The notation x(T + ) denotes the limit of
the trajectory x(t) as T is approached from the right, i.e., x(T + ) = limt→T + x(t).
The notation Lf h denotes the standard Lie derivative of a scalar function h(·) with
respect to the vector function f (·), i.e., Lf h(x) = ∂x
∂h
f (x).

2.2 Nonlinear Systems

In this book, we deal with a class of time invariant nonlinear systems that can be
described by the following state-space model:

ẋ = f (x, u), (2.1)

P. Mhaskar et al., Fault-Tolerant Process Control, DOI 10.1007/978-1-4471-4808-1_2, 9

© Springer-Verlag London 2013
10 2 Background on Nonlinear Systems and Control

where x ∈ Rn denotes the vector of state variables, u ∈ Rm denotes the vector of

control (manipulated) input variables, and f is a locally Lipschitz vector function
on Rn × Rm such that f (0, 0) = 0. This implies that the origin is an equilibrium
point for the unforced system. The input vector is restricted to be in a nonempty
convex set U ⊆ Rm which is defined as follows:
 
U := u ∈ Rm : u ≤ umax , (2.2)

where umax is the magnitude of the input constraint. Another version of the set that
we will use is
 
Ucon := u ∈ Rm : umin
i ≤ ui ≤ umax
i , i = 1, . . . , m , (2.3)

where umin
i and umax
i denote the constraints on the minimum and maximum value
of the ith input.
In many chapters, we will restrict our analysis to a special case of the system of
Eq. (2.1) where the input vector u enters the dynamics of the state x in an affine
fashion as follows:
ẋ = f (x) + G(x)u, (2.4)
where f is a locally Lipschitz vector function on Rn such that f (0) = 0 and G is an
n × m matrix of locally Lipschitz vector functions on Rn .

2.3 Stability of Nonlinear Systems

For all control systems, stability is the primary requirement. One of the most widely
used stability concepts in control theory is that of Lyapunov stability, which we
employ throughout the book. In this section, we briefly review basic facts from
Lyapunov’s stability theory. To begin with, we note that Lyapunov stability and
asymptotic stability are properties not of a dynamical system as a whole, but rather
of its individual solutions. We restrict our attention to the class of time-invariant
nonlinear systems:
ẋ = f (x), (2.5)
where the control input u does not appear explicitly. This does not necessarily mean
that the input to the system is zero. It could be that the input u has been specified as
a given function of the state x, u = u(x), and could be considered as a special case
of the system of Eq. (2.1).
The solution of Eq. (2.5), starting from x0 at time t0 ∈ R, is denoted as
x(t; x0 , t0 ), so that x(t0 ; x0 , t0 ) = x0 . Because the solutions of Eq. (2.5) are invariant
under a translation of t0 , that is, x(t + T ; x0 , t0 + T ) = x(t; x0 , t0 ), the stability prop-
erties of x(t; x0 , t0 ) are uniform, i.e., they do not depend on t0 . Therefore, without
loss of generality, we assume t0 = 0 and write x(t; x0 ) instead of x(t; x0 , 0).
2.3 Stability of Nonlinear Systems 11

Lyapunov stability concepts describe continuity properties of x(t; x0 , t0 ) with

respect to x0 . If the initial state x0 is perturbed to x̃0 , then, for stability, the perturbed
solution x̃(t; x0 ) is required to stay close to x(t; x0 ) for all t ≥ 0. In addition, for
asymptotic stability, the error x̃(t; x0 ) − x(t; x0 ) is required to vanish as t → ∞.
Some solutions of Eq. (2.5) may be stable and some unstable. We are particularly
interested in studying and characterizing the stability properties of equilibria, that
is, constant solutions x(t; xe ) ≡ xe satisfying f (xe ) = 0.
For convenience, we state all definitions and theorems for the case when the
equilibrium point is at the origin of Rn ; that is, xe = 0. There is no loss of generality
in doing so since any equilibrium point under investigation can be translated to
the origin via a change of variables. Suppose xe = 0, and consider the change of
variables, z = x − xe . The derivative of z is given by:

ż = ẋ = f (x) = f (z + xe ) := g(z),

where g(0) = 0. In the new variable z, the system has an equilibrium point at the ori-
gin. Therefore, for simplicity and without loss of generality, we will always assume
that f (x) satisfies f (0) = 0 and confine our attention to the stability properties of
the origin xe = 0.

2.3.1 Stability Definitions

The origin is said to be a stable equilibrium point of the system of Eq. (2.5), in the
sense of Lyapunov, if for every ε > 0 there exists a δ > 0 such that we have:
   
x(0) ≤ δ =⇒ x(t) ≤ ε, ∀t ≥ 0. (2.6)

In this case, we will also simply say that the system of Eq. (2.5) is stable. A similar
convention will apply to other stability concepts introduced below. The origin is said
to be unstable if it is not stable. The ε–δ requirement for stability takes a challenge–
answer form. To demonstrate that the origin is stable, for every value of ε that a
challenger may care to design, we must produce a value of δ, possibly dependent on
ε, such that a trajectory starting in a δ neighborhood of the origin will never leave
the ε neighborhood.
The origin of the system of Eq. (2.5) is said to be asymptotically stable if it is
stable and δ in Eq. (2.6) can be chosen so that (attractivity property of the origin):
 
x(0) ≤ δ =⇒ x(t) → 0 as t → ∞. (2.7)

When the origin is asymptotically stable, we are often interested in determining

how far from the origin the trajectory can be and still converge to the origin as
t approaches ∞. This gives rise to the definition of the region of attraction (also
called region of asymptotic stability, domain of attraction, and basin). Let φ(t; x)
be the solution of Eq. (2.5) that starts at initial state x at time t = 0. Then the region
12 2 Background on Nonlinear Systems and Control

of attraction is defined as the set of all points x such that limt→∞ φ(t; x) = 0. If the
origin is a stable equilibrium and its domain of attraction is the entire state-space,
then the origin is called globally asymptotically stable.
If the system is not necessarily stable but has the property that all solutions with
initial conditions in some neighborhood of the origin converge to the origin, then
it is called (locally) attractive. We say that the system is globally attractive if its
solutions converge to the origin from all initial conditions.
The system of Eq. (2.5) is called exponentially stable if there exist positive real
constants δ, c, and λ such that all solutions of Eq. (2.5) with x(0) ≤ δ satisfy the
inequality:
   
x(t) ≤ cx(0)e−λt , ∀t ≥ 0. (2.8)
If this exponential decay estimate holds for any x(0) ∈ Rn , the system is said to be
globally exponentially stable.

2.3.2 Stability Characterizations Using Function Classes K, K∞ ,

and KL

Scalar comparison functions, known as class K, K∞ , and KL, are important stabil-
ity analysis tools that are frequently used to characterize the stability properties of a
nonlinear system.

Definition 2.1 A function α : [0, a) → [0, ∞) is said to be of class K if it is con-

tinuous, strictly increasing, and α(0) = 0. It is said to belong to class K∞ if a = ∞
and α(r) → ∞ as r → ∞.

Definition 2.2 A function β : [0, a) × [0, ∞) → [0, ∞) is said to be of class KL

if, for each fixed t ≥ 0, the mapping β(r, t) is of class K with respect to r and, for
each fixed r, the mapping β(r, t) is decreasing with respect to t and β(r, t) → 0 as
t → ∞.

We will write α ∈ K and β ∈ KL to indicate that α is a class K function and β

is a class KL function, respectively. As an immediate application of these function
classes, we can rewrite the stability definitions of the previous section in a more
compact way. For example, stability of the system of Eq. (2.5) is equivalent to the
property that there exist a δ > 0 and a class K function, α, such that all solutions
with x(0) ≤ δ satisfy:
   
x(t) ≤ α x(0) , ∀t ≥ 0. (2.9)

Asymptotic stability is equivalent to the existence of a δ > 0 and a class KL func-

tion, β, such that all solutions with x(0) ≤ δ satisfy:
    
x(t) ≤ β x(0), t , ∀t ≥ 0. (2.10)
2.3 Stability of Nonlinear Systems 13

Global asymptotic stability amounts to the existence of a class KL function, β, such

that the inequality of Eq. (2.10) holds for all initial conditions. Exponential stability
means that the function β takes the form β(r, s) = cre−λs for some c, λ > 0.

2.3.3 Lyapunov’s Direct (Second) Method

Having defined stability and asymptotic stability of equilibrium points, the next task
is to find ways to determine stability. To be of practical interest, stability conditions
must not require that we explicitly solve Eq. (2.5). The direct method of Lyapunov
aims at determining the stability properties of an equilibrium point from the proper-
ties of f (x) and its relationship with a positive-definite function V (x).

Definition 2.3 Consider a C 1 (i.e., continuously differentiable) function V :

Rn → R. It is called positive-definite if V (0) = 0 and V (x) > 0 for all x = 0. If
V (x) → ∞ as x → ∞, then V is said to be radially unbounded.

If V is both positive-definite and radially unbounded, then there exist two class
K∞ functions α1 , α2 such that V satisfies:
   
α1 x ≤ V (x) ≤ α2 x (2.11)

for all x. We write V̇ for the derivative of V along the solutions of the system of
Eq. (2.5), i.e.:
∂V
V̇ (x) =
f (x). (2.12)
∂x
The main result of Lyapunov’s stability theory is expressed by the following state-
ment.

Theorem 2.1 (Lyapunov) Let x = 0 be an equilibrium point for the system of

Eq. (2.5) and D ⊂ Rn be a domain containing x = 0 in its interior. Suppose that
there exists a positive-definite C 1 function V : Rn → R whose derivative along the
solutions of the system of Eq. (2.5) satisfies:

V̇ (x) ≤ 0, ∀x ∈ D (2.13)

then x = 0 of the system of Eq. (2.5) is stable. If the derivative of V satisfies:

V̇ (x) < 0, ∀x ∈ D/{0} (2.14)

then x = 0 of the system of Eq. (2.5) is asymptotically stable. If in the latter case,
V is also radially unbounded, then x = 0 of the system of Eq. (2.5) is globally
asymptotically stable.
14 2 Background on Nonlinear Systems and Control

A continuously differentiable positive-definite function V (x) satisfying

Eq. (2.13) is called a Lyapunov function. The surface V (x) = c, for some c > 0,
is called a Lyapunov surface or a level surface. The condition V̇ ≤ 0 implies that
when a trajectory crosses a Lyapunov surface V (x) = c, it moves inside the set
Ωc = {x ∈ Rn : V (x) ≤ c} and can never come out again. When V̇ < 0, the trajec-
tory moves from one Lyapunov surface to an inner Lyapunov surface with smaller
c. As c decreases, the Lyapunov surface V (x) = c shrinks to the origin, showing
that the trajectory approaches the origin as time progresses. If we only know that
V̇ (x) ≤ 0, we cannot be sure that the trajectory will approach the origin, but we can
conclude that the origin is stable since the trajectory can be contained inside any
ball, Bε , by requiring that the initial state x0 lie inside a Lyapunov surface contained
in that ball.
The utility of a Lyapunov function arises from the need (or difficulty) of specify-
ing a unique (necessary and sufficient) direction of movement of states for stability.
To understand this, consider any scalar system (whether linear or nonlinear). The
necessary and sufficient condition for stability is that, for any value of the state x,
the value of ẋ should be opposite in sign to x, and greater than zero in magnitude
(unless x = 0). A Lyapunov function that allows readily capturing this requirement
2
is V (x) = x2 , resulting in V̇ (x) = x ẋ. If and only if the origin of the systems is
stable (i.e., x is opposite in sign to ẋ), it will result in V̇ (x) < 0.
For non-scalar systems, this ‘unique’ direction of movement of states, while pos-
sible for linear systems (see Remark 2.1), is in general difficult to identify for non-
linear systems. For instance, if one considers a simple two state system, and restricts
the choice of the Lyapunov function to quadratic forms, it is clear that the square
of the distance to the origin (resulting in ‘circles’ as level curves) is not necessarily
the only choice of the Lyapunov-function, and there is no unique way to find a nec-
essary and sufficient direction of the movement of states to achieve stability. This
is the problem that lies at the core of the Lyapunov-stability theory—the inability
to define (and/or construct) a unique Lyapunov function for a given system that is
necessary and sufficient to establish stability. Having recognized this limitation, it
is important to note that the Lyapunov-based analysis at least provides sufficient
conditions to ascertain stability.
In this direction, various converse Lyapunov theorems show that the conditions
of Theorem 2.1 are also necessary. For example, if the system is asymptotically
stable, then there exists a positive-definite C 1 function V that satisfies the inequality
of Eq. (2.14). The theorems, however, do not provide a way of constructing this
Lyapunov function.

Remark 2.1 It is well-known that for the linear time-invariant system

ẋ = Ax (2.15)

asymptotic stability, exponential stability, and their global versions are all equivalent
and amount to the property that A is a Hurwitz matrix, i.e., all eigenvalues of A have
2.3 Stability of Nonlinear Systems 15

negative real parts. Fixing an arbitrary positive-definite symmetric matrix Q and

finding the unique positive-definite symmetric matrix P that satisfies the Lyapunov
equation
AT P + P A = −Q,
one obtains a quadratic Lyapunov function V (x) = x T P x whose time derivative
along the solutions of the system of Eq. (2.15) is V̇ = −x T Qx. The explicit formula
for P is
 ∞
T
P= eA t QeAt dt.
0
Indeed, we have
 ∞ d  AT t At 
AT P + P A = e Qe dt = −Q,
0 dt
because A is Hurwitz.

2.3.4 LaSalle’s Invariance Principle

With some additional knowledge about the behavior of solutions, it is possi-

ble to prove asymptotic stability using a Lyapunov function which satisfies the
nonstrict inequality of Eq. (2.13). This is facilitated by LaSalle’s invariance
principle. To state this principle, we first recall the definition of an invariant
set.

Definition 2.4 A set M is called (positively) invariant with respect to the given
system if all solutions starting in M remain in M for all future times.

We now state a version of LaSalle’s theorem.

Theorem 2.2 (LaSalle) Suppose that there exists a positive-definite C 1 function V :

Rn → R whose derivative along the solutions of the system of Eq. (2.5) satisfies
the inequality of Eq. (2.13). Let M be the largest invariant set contained in the
set {x : V̇ (x) = 0}. Then the system of Eq. (2.5) is stable and every solution that
remains bounded for t ≥ 0 approaches M as t → ∞. In particular, if all solutions
remain bounded and M = {0}, then the system of Eq. (2.5) is globally asymptotically
stable.

To deduce global asymptotic stability with the help of this result, one needs
to check two conditions. First, all solutions of the system must be bounded.
This property follows automatically from the inequality of Eq. (2.13) if V is
chosen to be radially unbounded; however, radial boundedness of V is not nec-
essary when boundedness of solutions can be established by other means. The
16 2 Background on Nonlinear Systems and Control

second condition is that V be not identically zero along any nonzero solution.
We also remark that if one only wants to prove asymptotic convergence of
bounded solutions to zero and is not concerned with Lyapunov stability of the
origin, then positive-definiteness of V is not needed (this is in contrast to Theo-
rem 2.1).
While Lyapunov’s stability theorem readily generalizes to time-varying systems,
for LaSalle’s invariance principle this is not the case. Instead, one usually works
with the weaker property that all solutions approach the set {x : V̇ (x) = 0}.

2.3.5 Lyapunov’s Indirect (First) Method

Lyapunov’s indirect method allows one to deduce stability properties of the nonlin-
ear system of Eq. (2.5), where f is C 1 , from stability properties of its linearization,
which is the linear system of Eq. (2.15) with

∂f
A := (0). (2.16)
∂x
By the mean value theorem, we can write

f (x) = Ax + g(x)x,

where g is given componentwise by gi (x) := ∂f ∂fi

∂x (zi ) − ∂x (0) for some point, zi ,
i

on the line segment connecting x to the origin, i = 1, . . . , n. Since ∂f ∂x is contin-

uous, we have g(x) → 0 as x → 0. From this it follows that if the matrix A
is Hurwitz (i.e., all its eigenvalues lie in the open left half of the complex plane),
then a quadratic Lyapunov function for the linearization serves—locally—as a Lya-
punov function for the original nonlinear system. Moreover, its rate of decay in a
neighborhood of the origin can be bounded below by a quadratic function, which
implies that stability is, in fact, exponential. This is summarized by the following
result.

Theorem 2.3 If f is C 1 and the matrix of Eq. (2.16) is Hurwitz, then the system of
Eq. (2.5) is locally exponentially stable.

It is also known that if the matrix A has at least one eigenvalue with a posi-
tive real part, the origin of the nonlinear system of Eq. (2.5) is not stable. If A has
eigenvalues on the imaginary axis but no eigenvalues in the open right half-plane,
the linearization test is inconclusive. However, in this critical case, the system of
Eq. (2.5) cannot be exponentially stable since exponential stability of the lineariza-
tion is not only a sufficient but also a necessary condition for (local) exponential
stability of the nonlinear system.
2.3 Stability of Nonlinear Systems 17

2.3.6 Input-to-State Stability

It is of interest to extend stability concepts to systems with disturbance inputs. In

the linear case represented by the system

ẋ = Ax + Bθ,

it is well known that if the matrix A is Hurwitz, i.e., if the unforced system, ẋ = Ax,
is asymptotically stable, then bounded inputs θ lead to bounded states while inputs
converging to zero produce states converging to zero. Now, consider a nonlinear
system of the form
ẋ = f (x, θ ), (2.17)
where θ is a measurable bounded disturbance input. In general, global asymptotic
stability of the unforced system ẋ = f (x, 0) does not guarantee input-to-state sta-
bility with respect to θ of the kind mentioned above. For example, the scalar system

ẋ = −x + xθ (2.18)

has unbounded trajectories under the bounded input θ ≡ 2. This motivates the fol-
lowing important concept, introduced by Sontag [151].

Definition 2.5 The system of Eq. (2.17) is called input-to-state stable (ISS) with
respect to θ if for some functions γ ∈ K∞ and β ∈ KL, for every initial state x(0),
and every input θ , the corresponding solution of the system of Eq. (2.17) satisfies
the inequality
      
x(t) ≤ β x(0), t + γ θ s (2.19)
[0,t] ,
where θs[0,t] := ess.sup{θ (s) : s ∈ [0, t]} (supremum norm on [0, t] except for a
set of measure zero).

Since the system of Eq. (2.17) is time-invariant, the same property results if we
write
      
x(t) ≤ β x(t0 ), t − t0 + γ θ s
[t0 ,t] , ∀t ≥ t0 ≥ 0. (2.20)
The ISS property admits the following Lyapunov-like equivalent characterization:
The system of Eq. (2.17) is ISS if and only if there exists a positive-definite radially
unbounded C 1 function V : Rn → R such that for some class K∞ functions α and
χ we have
∂V    
f (x, θ ) ≤ −α x + χ θ  , ∀x, θ. (2.21)
∂x
This is, in turn, equivalent to the following “gain margin” condition:
  ∂V  
x ≥ ρ θ  =⇒ f (x, θ ) ≤ −α x , (2.22)
∂x
18 2 Background on Nonlinear Systems and Control

where α, ρ ∈ K∞ . Such functions V are called ISS-Lyapunov functions. If the sys-

tem of Eq. (2.17) is ISS, then θ (t) → 0 implies x(t) → 0.
The system of Eq. (2.17) is said to be locally input-to-state stable (locally ISS)
if the bound of Eq. (2.19) is valid for solutions with sufficiently small initial condi-
tions and inputs, i.e., if there exists a δ > 0 such that Eq. (2.19) is satisfied whenever
x(0) ≤ δ and θ s[0,t] ≤ δ. It turns out that (local) asymptotic stability of the un-
forced system ẋ = f (x, 0) implies local ISS.

2.4 Stabilization of Nonlinear Systems

This book is primarily about control design. Our objective is to create closed-loop
systems with desirable stability and performance properties, rather than analyze the
properties of a given system. For this reason, we are interested in an extension of
the Lyapunov function concept, called a control Lyapunov function (CLF).
Suppose that our problem for the time-invariant system

ẋ = f (x, u), (2.23)

where x ∈ Rn , u ∈ R (i.e., we consider the unconstrained problem), f (0, 0) = 0,

is to design a feedback control law α(x) for the control variable u such that the
equilibrium x = 0 of the closed-loop system
 
ẋ = f x, α(x) (2.24)

is globally asymptotically stable. We can pick a function V (x) as a Lyapunov func-

tion candidate, and require that its derivative along the solutions of the system of
Eq. (2.24) satisfies V̇ ≤ −W (x), where W (x) is a positive-definite function. We
therefore need to find α(x) to guarantee that for all x ∈ Rn
∂V  
(x)f x, α(x) ≤ −W (x). (2.25)
∂x
This is a difficult task. A stabilizing control law for the system of Eq. (2.23) may
exist, but it may fail to satisfy Eq. (2.25) because of a poor choice of V (x) and
W (x). A system for which a good choice of V (x) and W (x) exists is said to possess
a CLF. This notion is made more precise below.

Definition 2.6 A smooth positive-definite radially unbounded function V : Rn → R

is called a control Lyapunov function (CLF) for the system of Eq. (2.23) if

∂V
inf (x)f (x, u) < 0, ∀x = 0. (2.26)
u∈R ∂x

The CLF concept of Artstein [9] is a generalization of Lyapunov design results

by Jacobson and Judjevic and Quinn. Artstein showed that Eq. (2.26) is not only
2.4 Stabilization of Nonlinear Systems 19

necessary, but also sufficient for the existence of a control law satisfying Eq. (2.25),
that is, the existence of a CLF is equivalent to global asymptotic stabilizability.
For systems affine in the control, namely,

ẋ = f (x) + g(x)u, f (0) = 0, (2.27)

the CLF inequality of Eq. (2.25) becomes

Lf V (x) + Lg V (x)u ≤ −W (x). (2.28)

If V is a CLF for the system of Eq. (2.27), then a particular stabilizing control law
α(x), smooth for all x = 0, is given by Sontag’s formula [150]:
⎧ √
⎨ Lf V (x)+ (Lf V )2 (x)+(Lg V )4 (x)
− Lg V (x), Lg V (x) = 0,
u = αs (x) = (Lg V )2 (x) (2.29)
⎩0, L V (x) = 0.
g

It should be noted that Eq. (2.28) can be satisfied only if

Lg V (x) = 0 =⇒ Lf V (x) < 0, ∀x = 0. (2.30)

The intuitive interpretation of the existence of a CLF is as follows: For any x such
that Lg V (x) = 0, since there are no constraints on the input, V̇ can be made negative
by picking a ‘large enough’ control action, with an appropriate sign, to counter the
effect of possibly positive Lf V (x) term. For all x such that Lg V (x) = 0, the control
action has no effect on the Lyapunov-function derivative. For it to be possible to
show stability using the CLF V , it should therefore be true that whenever Lg V (x) =
0, we also have that Lf V (x) < 0. This is the requirement that is formalized in
Eq. (2.30). With such a CLF, Eq. (2.29) results in

W (x) = (Lf V )2 (x) + (Lg V )4 (x) > 0, ∀x = 0. (2.31)

A further characterization of a stabilizing control law α(x) for the system of

Eq. (2.27) with a given CLF V is that α(x) is continuous at x = 0 if and only if
the CLF satisfies the small control property: For each ε > 0 there is a δ(ε) > 0 such
that, if x = 0 satisfies |x| ≤ δ, then there is some u with |u| < ε such that

Lf V (x) + Lg V (x)u < 0. (2.32)

The main deficiency of the CLF concept as a design tool is that for most nonlinear
systems a CLF is not known. The task of finding an appropriate CLF maybe as com-
plex as that of designing a stabilizing feedback law. In the next section, we review
one commonly used tool for designing a Lyapunov-based control law that utilizes
coordinate transformations. We also note that in the presence of input constraints,
the concept of a CLF needs to be revisited, and this issue is discussed in Sect. 2.6.
20 2 Background on Nonlinear Systems and Control

2.5 Feedback Linearization and Zero Dynamics

One of the popular methods for nonlinear control design (or alternatively, one way
to construct a Lyapunov-function for the purpose of control design) is feedback
linearization, which employs a change of coordinates and feedback control to trans-
form a nonlinear system into a system whose dynamics are linear (at least partially).
This transformation allows the construction and use of a Lyapunov function for the
control design utilizing results from linear systems analysis. A great deal of re-
search has been devoted to this subject over the last four decades, as evidenced by
the comprehensive books [72, 126] and the references therein. In this section, we
briefly review some of the basic geometric concepts that will be used in subsequent
chapters. While this book does not require the formalism of differential geometry,
we will employ Lie derivatives only for notational convenience. If f : Rn → Rn
is a vector field and h : Rn → R is a scalar function, the notation Lf h is used for
∂h
∂x f (x). It is recursively extended to

  ∂  k−1 
Lkf h(x) = Lf Lk−1
f h(x) = Lf h(x) f (x).
∂x
Let us consider the following nonlinear system:

ẋ = f (x) + g(x)u,
(2.33)
y = h(x),

where x ∈ Rn , u ∈ R, y ∈ R, f , g, h are analytic (i.e., infinitely differentiable)

vector functions. The derivative of the output y = h(x) is given by

∂h ∂h
ẏ = (x)f (x) + (x)g(x)u
∂x ∂x
= Lf h(x) + Lg h(x)u. (2.34)

If Lg h(x0 ) = 0, then the system of Eq. (2.33) is said to have relative degree one at
x0 (note that since the functions are smooth Lg h(x0 ) = 0 implies that there exists a
neighborhood of x0 on which Lg h(x) = 0). In our terminology, this implies that the
output y is separated form the input u by one integration only. If Lg h(x0 ) = 0, there
are two cases:
(i) If there exist points arbitrarily close to x0 such that Lg h(x) = 0, then the system
of Eq. (2.33) does not have a well-defined relative degree at x0 .
(ii) If there exists a neighborhood B0 of x0 such that Lg h(x) = 0 for all x ∈ B0 ,
then the relative degree of the system of Eq. (2.33) may be well-defined.
In case (ii), we define

ψ1 (x) = h(x), ψ2 (x) = Lf h(x) (2.35)

2.5 Feedback Linearization and Zero Dynamics 21

and compute the second derivative of y

∂ψ2 ∂ψ2
ÿ = (x)f (x) + (x)g(x)u
∂x ∂x
= L2f h(x) + Lg Lf h(x)u. (2.36)

If Lg Lf h(x0 ) = 0, then the system of Eq. (2.33) is said to have relative degree two
at x0 . If Lg Lf h(x) = 0 in a neighborhood of x0 , then we continue the differentiation
procedure.

Definition 2.7 The system of Eq. (2.33) is said to have relative degree r at the point
x0 if there exists a neighborhood B0 of x0 on which

Lg h(x) = Lg Lf h(x) = · · · = Lg Lr−2

f h(x) = 0, (2.37)

f h(x) = 0.
Lg Lr−1 (2.38)

If Eq. (2.37)–(2.38) are valid for all x ∈ Rn , then the relative degree of the system
of Eq. (2.33) is said to be globally defined.

Suppose now that the system of Eq. (2.33) has relative degree r at x0 . Then
we can use a change of coordinates and feedback control to locally transform this
system into the cascade interconnection of an r-dimensional linear system and an
(n − r)-dimensional nonlinear system. In particular, after differentiating r times the
output y = h(x), the control appears:

y (r) = Lrf h(x) + Lg Lr−1

f h(x)u. (2.39)

f h(x) = 0 in a neighborhood of x0 , we can linearize the input–output

Since Lg Lr−1
dynamics of the system of Eq. (2.33) using feedback to cancel the nonlinearities in
Eq. (2.39):
1  r 
u= −Lf h(x) + v . (2.40)
Lg Lr−1
f h(x)

Then the dynamics of y and its derivatives are governed by a chain of r integra-
tors: y (r) = v. Since our original system of Eq. (2.33) has dimension n, we need
to account for the remaining n − r states. Using differential geometry tools, it can
be shown that it is always possible to find n − r functions ψr+1 , . . . , ψn (x) with
∂ψi
∂x (x)g(x) = 0, for i = r + 1, . . . , n such that the change of coordinates

ζ1 = y = h(x), ζ2 = ẏ = Lf h(x), . . . , ζr = y (r−1) = Lr−1

f h(x),
(2.41)
η1 = ψr+1 , . . . , ηn−r = ψn (x)
22 2 Background on Nonlinear Systems and Control

is locally invertible and transforms, along with the feedback law of Eq. (2.40), the
system of Eq. (2.33) into
ζ̇1 = ζ2 ,
..
.
ζ̇r = v,
η̇1 = Ψ1 (ζ, η), (2.42)
..
.
η̇n−r = Ψn−r (ζ, η),
y = ζ1 ,

where Ψ1 (ζ, η) = Lr+1 f h(x), Ψn−r (ζ, η) = Lf h(x).

n

The states η1 , . . . , ηn−r have been rendered unobservable from the output y by
the control of Eq. (2.40). Hence, feedback linearization in this case is the nonlin-
ear equivalent of placing n − r poles of a linear system at the origin and canceling
the r zeros with the remaining poles. Of course, to guarantee stability, the canceled
zeros must be stable. In the nonlinear case, using the new control input v to sta-
bilize the linear subsystem of Eq. (2.42) does not guarantee stability of the whole
system, unless the stability of the nonlinear part of the system of Eq. (2.42) has been
established separately.
When v is used to keep the output y equal to zero for all t > 0, that is, when
ζ1 ≡ · · · ≡ ζr ≡ 0, the dynamics of η1 , . . . , ηn−r are described by

η̇1 = Ψ1 (0, η),

.. (2.43)
.
η̇n−r = Ψn−r (0, η).

They are called the zero dynamics of the system of Eq. (2.33) because they evolve
on the subset of the state-space on which the output of the system is identically
zero. If the equilibrium at η1 = · · · = ηn−r = 0 of the zero dynamics of Eq. (2.43) is
asymptotically stable, the system of Eq. (2.33) is said to be minimum phase.

Remark 2.2 Most nonlinear analytical controllers emanating from the area of ge-
ometric control are input–output linearizing and induce a linear input–output re-
sponse in the absence of constraints [72, 81]. For the class of processes modeled by
equations of the form of Eq. (2.33) with relative order r and under the minimum
phase assumption, the appropriate linearizing state feedback controller is given by
1  
u= v − Lrf h(x) − β1 Lr−1
f h(x) − · · · − βr−1 Lf h(x) − βr h(x)
Lg Lr−1
f h(x)
(2.44)
2.6 Input Constraints 23

and induces the linear rth order response

dr y d r−1 y dy
r
+ β1 r−1
+ · · · + βr−1 + βr y = v, (2.45)
dt dt dt
where the tunable parameters, β1 , . . . , βr , are essentially closed-loop time constants
that influence and shape the output response. The nominal stability of the process is
guaranteed by placing the roots of the polynomial s r + β1 s r−1 + · · · + βr−1 s + βr
in the open left-half of the complex plane.

2.6 Input Constraints

The presence of input constraints requires revisiting the concept of the CLF for both
linear and nonlinear systems. To understand this, consider a scalar linear system of
the form ẋ = αx + βu, with umin ≤ u ≤ umax . For the sake of simplicity and without
loss of generality, let us assume umin < 0 < umax and β > 0. For the case of scalar
systems, it is possible to determine the entire set of initial conditions from where
the system can be driven to the origin subject to input constraints (regardless of the
choice of the control law). This set is generally referred to as the null controllable
region (NCR). An explicit computation of the NCR is possible in this case because
for scalar systems (as discussed earlier) there exists a unique direction in which the
system states needs to move to achieve stability.
To determine this set, one can simply analyze the system trajectory to the left
and right of zero. Consider first x > 0, and the requirement that for x > 0, ẋ < 0.
If α < 0, ẋ < 0 ∀x > 0 (and also ẋ > 0 ∀x < 0). On the other hand, if α > 0,
ẋ < 0 can only be achieved for x < −uα β . Similarly, ẋ > 0 can only be achieved
min

for x > −u α β . The analysis reveals what was perhaps intuitive to begin with: For
max

linear systems, if the steady state is open-loop stable, the NCR is the entire state
space, while if the steady state is open-loop unstable, it has a finite NCR, which
in this case is {x : −u α β < x < −uα β }. The same result for the NCR can also
max min

2
be obtained using a CLF V (x) = x2 and determining the states for which V̇ < 0 is
achievable using the available control action. Furthermore, it points to the require-
ment of additional considerations when defining CLFs for systems with constrained
inputs. In particular, requiring that V̇ (x) < 0 ∀x is simply not achievable for certain
cases, at best what is achievable is that V̇ (x) < 0 ∀x ∈ NCR − {0}. The definition of
a CLF (or more appropriately, a constrained CLF) then becomes intricately linked
with the characterization of the NCR. The characterization of the NCR, however,
is an increasingly difficult (although possible, see [71]) problem when considering
non-scalar linear systems, and currently an open problem for nonlinear systems.
To understand the impact of the lack of availability of constrained CLFs
(CCLFs), let us first consider again the linear scalar system under a feedback law
of the form uc (x) = −kx, with k > 0 such that (α − kβ) < 0 under two possible
scenarios: (i) α < 0 (i.e., for the unforced system, there is an isolated equilibrium
24 2 Background on Nonlinear Systems and Control

point at the origin and the system is stable at that operating point) and (ii) α > 0 (i.e.,
for the unforced system, there is an isolated equilibrium point at the origin and the
system is unstable at that operating point). Due to the presence of input constraints,
the closed-loop system is no longer a linear system, but operates in three ‘modes’,
depending on the state, described by the following set of equations:
dx
= αx + βuc , umin ≤ uc ≤ umax ,
dt
dx
= αx + βumax , uc > umax , (2.46)
dt
dx
= αx + βumin , umin > uc .
dt
Let us analyze the three possible modes of operation of the closed-loop system
for scenario (i). For − |u k | ≤ x ≤ |u k | , we have that dx
max min
dt = αx + βuc = (α − kβ)x,
which establishes that for all initial conditions x0 such that − |u k | ≤ x0 ≤ |u k | ,
max min

the prescribed control action uc is within the constraints and the system state will
be driven to the origin. For |u k | < x ≤ −uα β , uc > umax resulting in u = umax , in
min min

turn resulting in ẋ < 0. A similar result is obtained for −u α β < x < − |u k | . The
max max

analysis shows that for scalar systems, while the region of unconstrained operation
for a particular control law might depend on the specific control law chosen, the
stability region under the control law might still possibly be the entire NCR.
The issue of directionality again crops up when considering non-scalar systems.
While it is relatively easy to determine the region of unconstrained operation for a
particular control law, and, in certain cases, the region of attraction for the closed-
loop system, it is not necessary that the region of attraction for the closed-loop
system match the NCR. This happens due to the fact that it is in general difficult
to determine, for a particular value of the state, the unique direction in which the
inputs should saturate to achieve closed-loop stability. To achieve this objective, re-
cent control designs have utilized the explicit characterization of the NCR [71] in
designing CCLF based control laws that ensure stabilization from all initial con-
ditions in the NCR [93, 94]. For nonlinear systems, where the characterization of
the NCR is still an open problem, a meaningful control objective is to be able to
explicitly account for the constraints in the control design and provide an explicit
characterization of the closed-loop stability region.

2.7 Model Predictive Control

One of the control methods useful for accounting for constraints and optimality si-
multaneously is that of model predictive control (MPC). MPC is an approach which
accounts for optimality considerations explicitly and is widely adopted in industry
as an effective approach to deal with large multivariable constrained optimal con-
trol problems. The main idea of MPC is to choose control actions by repeatedly
2.7 Model Predictive Control 25

solving an online a constrained optimization problem, which aims at minimizing a

performance index over a finite prediction horizon based on predictions obtained by
a system model. In general, an MPC design is composed of three components:
1. A model of the system. This model is used to predict the future evolution of the
system in open-loop and the efficiency of the calculated control actions of an
MPC depends highly on the accuracy of the model.
2. A performance index over a finite horizon. This index is minimized subject to
constraints imposed by the system model, restrictions on control inputs and sys-
tem state, and other considerations at each sampling time to obtain a trajectory
of future control inputs.
3. A receding horizon scheme. This scheme introduces the notion of feedback into
the control law to compensate for disturbances and modeling errors, whereby
only the first piece of the future input trajectory is implemented and the con-
strained optimization problem is resolved at the next sampling instance.
Consider the control of the system of Eq. (2.1) and assume that the state mea-
surements of the system of Eq. (2.1) are available at synchronous sampling time
instants {tk≥0 }, a standard MPC is formulated as follows [60]:
 tk+N
      
min x̃(τ ) + u(τ ) dτ + F x(tk+N ) (2.47)
u∈S(Δ) tk Q
c R c

 
˙ = f x̃(t), u(t) ,
s.t. x̃(t) (2.48)
u(t) ∈ U, (2.49)
x̃(tk ) = x(tk ), (2.50)

where S(Δ) is the family of piece-wise constant functions with sampling period
Δ, N is the prediction horizon, Qc and Rc are strictly positive definite symmetric
weighting matrices, x̃ is the predicted trajectory of the system due to control input
u with initial state x(tk ) at time tk , and F (·) denotes the terminal penalty.
The optimal solution to the MPC optimization problem defined by Eq. (2.47)–
(2.50) is denoted as u∗ (t|tk ) which is defined for t ∈ [tk , tk+N ). The first step value
of u∗ (t|tk ) is applied to the closed-loop system for t ∈ [tk , tk+1 ). At the next sam-
pling time tk+1 , when a new measurement of the system state x(tk+1 ) is available,
and the control evaluation and implementation procedure is repeated. The manipu-
lated input of the system of Eq. (2.1) under the control of the MPC of Eq. (2.47)–
(2.50) is defined as follows:

u(t) = u∗ (t|tk ), ∀t ∈ [tk , tk+1 ), (2.51)

which is the standard receding horizon scheme.

In the MPC formulation of Eq. (2.47)–(2.50), Eq. (2.47) defines a performance
index or cost index that should be minimized. In addition to penalties on the state
and control actions, the index may also include penalties on other considerations;
for example, the rate of change of the inputs. Equation (2.48) is the model of the
26 2 Background on Nonlinear Systems and Control

system of Eq. (2.1) which is used in the MPC to predict the future evolution of the
system. Equation (2.49) takes into account the constraint on the control input, and
Eq. (2.50) provides the initial state for the MPC which is a measurement of the
actual system state. Note that in the above MPC formulation, state constraints are
not considered but can be readily taken into account.
It is well known that the MPC of Eq. (2.47)–(2.50) is not necessarily stabilizing.
To understand this, let us consider a discrete time version of the MPC implementa-
tion, for a scalar system described by x(k + 1) = αx(k) + u(k), in the absence of
input constraints. Also, let N = 1, q and r denote the horizon, penalty on the state
deviation and input deviation, respectively. The objective function then simplifies to
q(α 2 x(k)2 + u(k)2 + 2αx(k)u(k)) + ru(k)2 , and the minimizing control action is
u(k) = −qαx(k) rαx(k)
q+r , resulting in the closed-loop system x(k +1) = q+r . The minimiz-
ing solution will result in stabilizing control action only if q > r(α − 1). Note that
for α < 1, this trivially holds (i.e., the result trivially holds for stabilization around
an open-loop stable steady state). For α > 1, the result establishes how large the
penalty on the set point deviation should be compared to the penalty on the control
action for the controller to be stabilizing. The analysis is meant to bring out the fact
that generally speaking, the stability of the closed-loop system in the MPC depends
on the MPC parameters (penalties and the control horizon) as well as the system
dynamics. Note also that even though we have analyzed an unconstrained system,
the prediction horizon we used was finite (in comparison to linear quadratic regula-
tor designs, where the infinite horizon cost is essentially captured in computing the
control action, and therefore results in stabilizing controller in the absence of con-
straints). Finally, also note that for the case of infinite horizon, the optimum solution
is also the stabilizing one, and it can be shown that such an MPC will stabilize the
system with the NCR as the stability region (albeit at an impractical computational
burden).
To achieve closed-loop stability without relying on the objective function pa-
rameters, different approaches have been proposed in the literature. One class of
approaches is to use well-designed terminal penalty terms that capture infinite hori-
zon costs; please, see [16, 100] for surveys of these approaches. Another class
of approaches is to impose stability constraints in the MPC optimization problem
[3, 14, 100]. There are also efforts focusing on getting explicit stabilizing MPC laws
using offline computations [92]. However, the implicit nature of MPC control law
makes it very difficult to explicitly characterize, a priori, the admissible initial con-
ditions starting from where the MPC is guaranteed to be feasible and stabilizing.
In practice, the initial conditions are usually chosen in an ad hoc fashion and tested
through extensive closed-loop simulations.

2.8 Lyapunov-Based MPC

In this section, we introduce Lyapunov-based MPC (LMPC) designs proposed in
[93, 108, 110] which allow for an explicit characterization of the stability region
and guarantee controller feasibility and closed-loop stability.
2.8 Lyapunov-Based MPC 27

For the predictive control of the system of Eq. (2.1), the key idea in LMPC-based
designs is to utilize a Lyapunov-function based constraint and achieve immediate
decay of the Lyapunov function. The set of initial conditions for which it is possible
to achieve an instantaneous decay in the Lyapunov function value can be computed
explicitly, and picking the (preferably largest) level curve contained in this set can
provide the explicitly characterized feasibility and stability region for the LMPC.
The following example of the LMPC design is based on an existing explicit con-
trol law h(x) which is able to stabilize the closed-loop system [108, 110]. The for-
mulation of the LMPC is as follows:
 tk+N
    
min x̃(τ ) + u(τ ) dτ (2.52)
u∈S(Δ) tk Q c R c

 
s.t. ˙ = f x̃(t), u(t) ,
x̃(t) (2.53)
u(t) ∈ U, (2.54)
x̃(tk ) = x(tk ), (2.55)
∂V (x(tk ))   ∂V (x(tk ))   
f x(tk ), u(tk ) ≤ f x(tk ), h x(tk ) , (2.56)
∂x ∂x
where V (x) is a Lyapunov function associated with the nonlinear control law h(x).
The optimal solution to this LMPC optimization problem is denoted as u∗l (t|tk )
which is defined for t ∈ [tk , tk+N ). The manipulated input of the system of Eq. (2.1)
under the control of the LMPC of Eq. (2.52)–(2.56) is defined as follows:

u(t) = u∗l (t|tk ), ∀t ∈ [tk , tk+1 ), (2.57)

which implies that this LMPC also adopts a standard receding horizon strategy.
In the LMPC defined by Eq. (2.52)–(2.56), the constraint of Eq. (2.56) guarantees
that the value of the time derivative of the Lyapunov function, V (x), at time tk is
smaller than or equal to the value obtained if the nonlinear control law u = h(x)
is implemented in the closed-loop system in a sample-and-hold fashion. This is a
constraint that allows one to prove (when state measurements are available every
synchronous sampling time) that the LMPC inherits the stability and robustness
properties of the nonlinear control law h(x) when it is applied in a sample-and-hold
fashion; please, see [30, 125] for results on sampled-data systems.
Let us denote the stability region of h(x) as Ωρ . The stability properties of the
LMPC implies that the origin of the closed-loop system is guaranteed to be stable
and the LMPC is guaranteed to be feasible for any initial state inside Ωρ when the
sampling time Δ is sufficiently small. Note that the region Ωρ can be explicitly
characterized; please, refer to [110] for more discussion on this issue. The main
advantage of the LMPC approach with respect to the nonlinear control law h(x)
is that optimality considerations can be taken explicitly into account (as well as
constraints on the inputs and the states [110]) in the computation of the control
actions within an online optimization framework while improving the closed-loop
performance of the system. Since the closed-loop stability and feasibility of the
28 2 Background on Nonlinear Systems and Control

LMPC of Eq. (2.52)–(2.56) are guaranteed by the nonlinear control law h(x), it is
unnecessary to use a terminal penalty term in the cost index (see Eq. (2.52) and
compare it with Eq. (2.47)) and the length of the horizon N does not affect the
stability of the closed-loop system but it affects the closed-loop performance.

2.9 Hybrid Systems

Hybrid systems are characterized by the co-existence of continuous modes of op-

eration along with discrete switches between the distinct modes of operation and
arise frequently in the design and analysis of fault-tolerant control systems. The
class of hybrid systems of interest to the focus of this book–switched systems–can
be described by
ẋ = fi(x,t) (x) + gi(x,t) (x)ui(x,t) , (2.58)
where x ∈ Rn , u ∈ Rn are the continuous variables and i ∈ N are the discrete vari-
ables indexing the mode of operation. The nature of the function i(x, t) and, in par-
ticular, its two specific forms i(x) and i(t) result in the so-called state-dependent and
time-dependent switching. What is of more interest from a stability analysis and de-
sign point of view (both when considering the design of control laws and, in the case
of time-dependent switching, the switching signal) is the possibility of infinitely
many switches where it becomes crucial to explicitly consider the switched nature
of the system in the stability analysis. In particular, when the possibility of infinitely
many switches exists, establishing stability in the individual modes of operation
is not sufficient [19], and additional conditions on the behavior of the Lyapunov-
functions (used to establish stability in the individual modes of operation) during
the switching (as well as of sufficient dwell-time [68]) need to be satisfied for the
stability of the switched system. For the case of finite switches, the considerations
include ensuring stability requirements at the onset of a particular mode are satisfied
and, in particular, satisfied for the terminal (last) mode of operation.

2.10 Conclusions
In this chapter, some fundamental results on nonlinear systems analysis and control
were briefly reviewed. First, the class of nonlinear systems that will be considered
in this book was presented; then the definitions of stability of nonlinear systems
were introduced; and following that, techniques for stabilizing nonlinear systems,
for example, Lyapunov-based control, feedback linearization, handling constraints,
model predictive control and Lyapunov-based model predictive control and stability
of hybrid (switched) systems were discussed.