Lecture 4

Modeling Physics
 Plan
• Dynamical system models
– notions of solutions
– Linear dynamical systems
– Connection to automata
– Stability
– Lyapunov method
 Map of CPS models
Dynamical systems
Discrete transition Differential Markov chains
systems, automata inclusions
Probabilistic automata,
Markov decision
processes (MDP)

Continuous time,
Hybrid systems continuous state MDPs

Stochastic Hybrid systems

Lecture Slides by Sayan Mitra [email protected]

All this was in the two plague years 1665 and 1666, for in those
days I was in my prime of age for invention, and minded
mathematics and philosophy more than at any time since.
---Isaac Newton

From: Wilczek, Frank. A Beautiful Question: Finding Nature's Deep Design (p. 87).
 Introduction to dynamical systems
Behaviors of physical processes are described in terms of instantaneous laws

Common notation:
where time ; state

Example. ;

Initial value problem: Given system (1) and initial state , and input find a state
trajectory or solution of (1).
 Notions of solution
What is a solution? Many different notions.

Definition 1. (First attempt) Given and is a solution or trajectory iff

(1) and
(2) .

Mathematically makes sense, but too restrictive. Assumes that is not

only continuous, but also differentiable. This disallows to be
discontinuous, which is often required for optimal control.
Getting from point a to point b
 Modified notion
Definition. is a piece-wise continuous with set of discontinuity
points if 𝑢 (𝑡)
(1) ;
(2) Continuous from right
(3) is finite 𝜏1 𝜏2

) is the set of all piece-wise continuous functions over the

domain

Define for a given Since is PC in so is in the second argument.

Definition 2. Given and is a solution or trajectory iff (1) and

Lecture Slides by Sayan Mitra [email protected]
(2) .
 Is PC input adequate for guaranteeing existence of
solutions?
Example.
Solution: for ; check
Problem: discontinuous is

Example.
Solution: works for ; check
Problem: As then ; grows too fast
 Lipschitz continuity
A function is Lipschitz continuous if there exist such that for any
pair ,

Examples: all differentiable functions with bounded derivatives

Non-examples: (locally Lipschitz)

 Existence and uniqueness of solutions
Theorem. If is Lipschitz continuous in the first argument then (1)
has unique solutions.

Transition system model

 Linear time-varying systems
In general, for nonlinear dynamical systems we do not have closed form solutions for
but there are numerical solvers like CAPD, VNODE

--- (2)

continuous everywhere except

Theorem. Let be the solution for (2) with points of discontinuity

1. is continuous and differentiable
 Example 1: Simple model of an economy

• : national income
• rate of consumer spending
• : rate government expenditure
 Linear system and solutions

For a given initial state the solution is a function

We studied several properties of in the last lecture: continuity

with respect to first and third argument, linearity, decomposition
 Linear system and solutions
• Since is a linear function of the initial state and input,
• = +
• Let us focus on the linear function
• Define
• This : is called the state transition matrix
 Properties of

• is the unique solution of (2) and is defined by a (Peano-Baker) infinite

sequence of integrals

• with
– Continuous everywhere

– Differentiable everywhere except ( isn’t)

• is invertible
 Solution of linear systems in
Theorem.
 Linear time invariant system

Matrix exponential:

Theorem. , that is
 Discrete time models / discrete transition systems

• autonomous
• Execution:

–;
• Deterministic
 Discretized or sampled-time model

• Assume: is a finite set

• Fix a sampling period

– ; iff
 Properties for dynamical systems
What type of properties are we interested in?
• Invariance (as in the case of automata)
• State remains bounded
• Converges to target
• Bounded input gives bounded output (BIBO)
 Requirements: Stability
• We will focus on time invariant autonomous systems (closed

• –(1)
systems, systems without inputs)

• is the solution
• norm
• is an equilibrium point if
• For analysis we will assume 0 to be an equilibrium point of (1)
with out loss of generality
 Example: Pendulum

upright
Pendulum equation

down
=

friction coefficient

Two equilibrium points:

CW

sta
ble
speed=0
un
sta
ble

upright
CCW down
 Aleksandr M. Lyapunov
Aleksandr Mikhailovich Lyapunov (June 6 1857–
November 3, 1918) was a Russian
mathematician and physicist.

His methods make it possible to define the

stability of ordinary differential equations. In the
theory of probability, he generalized the works
of Chebyshev and Markov, and proved
the Central Limit Theorem under more general
conditions than his predecessors.
 Lyapunov stability
Lyapunov stability: The system (1) is said to be Lyapunov stable
(at the origin) if
such that

How is this related to

invariants and 𝛿𝜀
𝑥0
reachable states ? 𝜀
 Asymptotically stability
The system (1) is said to be Asymptotically stable (at the origin) if
it is Lyapunov stable and
such that as
If the property holds for any then Globally Asymptotically Stable

𝛿2
𝑥0
Phase portrait of pendulum with friction
 Butterfly*
=

All solutions converge to 0

but the equilibrium point
(0,0) is not Lyapunov stable
 Van der pol oscillator
Van der pol oscillator

coupling coefficient

stable ?
 Stability of solutions* (instead of points)
• For any define the s-norm

• A dynamical system can be seen as an operator that maps initial states to signals

• Lyapunov stability required that this operator is continuous

• The solution is Lyapunov stable if is continuous as for every there exists such that
for every if then
 Verifying Stability for Linear Systems
Consider the linear system

Theorem.
1. It is asymptotically stable iff all the eigenvalues of A have strictly negative real
parts (Hurwitz).

2. It is Lyapunov stable iff all the eigenvalues of A have real parts that are either
zero or negative and the Jordan blocks corresponding to the eigenvalues with
zero real parts are of size 1.
 Jordan decomposition
For every n x n matrix A, there exists a nonsingular n x n matrix P such that

where each is a upper triangular matrix called a Jordan block

 Example 1: Simple model of an economy

• : national income rate of consumer spending; : rate

government expenditure

• are positive constants

• What is the equilibrium?
•
• Dynamics:
 Example: Simple linear model of an economy

• Negative real parts, therefore,

asymptotically stable and the
national income and consumer
spending rate converge to
 Stability of nonlinear systems
• For any positive definite function of state

• Sublevel sets of |
– and iff

V differentiable with continuous first derivative

• is also continuous
• is radially unbounded if
 Verifying Stability
Theorem. (Lyapunov) Consider the system (1) with state space
and suppose there exists a positive definite, continuously
differentiable function . The system is:
1. Lyapunov stable if for all
2. Asymptotically stable if for all
3. It is globally AS if V is also radially unbounded.
 Proof sketch: Lyapunov stable if

• Assume
• Consider a ball around the origin of
radius
B𝜀
• Pick a positive number 𝐿𝑏
𝐵𝛿
• Let be a radius of ball around origin
which is inside
• Since along all trajectories V is non-
increasing, starting from each solution
satisfies and therefore remains in
 Proof sketch: Asymptotically stable if
• Assume
• Take arbitrary initial state where this comes from some for Lyapunov
stability
• Since and decreasing along it has a limit at
• It suffices to show that this limit is actually 0
• Suppose not, c > 0 then the solution evolves in the compact set for
some sufficiently small
• Let [slowest rate]
B𝜀
• This number is well-defined and negative
• for all t
𝑟
𝑆
• But then eventually
 Example 2: Reasoning about stability without solving ODEs

Given that
𝑥˙ 1 =−𝑥 1+𝑔(𝑥 2)
• Use
𝑥˙ 2=−𝑥 2 +h(𝑥 1)
=

We conclude global asymptotic stability (in fact global

Proposition. If V is a Lyapunov function then every sublevel set of
V is an invariant

Proof.
 An aside: Checking inductive invariants

– set of variables

– written as a program
• How do we check that is an inductive invariant?

• Implies that without computing the executions or reachable states of A

• The key is to find such
 Finding Lyapunov Functions
• The key to using Lyapunov theory is to find a Lyapunov
function and verify that it has the properties
• In general, for nonlinear systems this is hard
• There are several approaches
– Quadratic Lyapunov functions for linear systems
– Decide the form/template of the function (e.g., quadratic,
polynomial), parameterized by some parameters and find values of
the parameters so that the conditions hold (Chapter 3 last section)
 Linear autonomous systems

• The Lyapunov equation:

where are symmetric

• Interpretation: then

[using chain rule

=

• If is the generalized energy then is the associated dissipation

 Quadratic Lyapunov Functions
• If (positive definite)

• The sub-level sets are ellipsoids

• If then the system is globally asymptotically stable
 Same example
Lyapunov equations are solved as a set of
equations in variables. Cost O(

Choose solving Lyapunov equations we get

and we get the quadratic Lyapunov function
an a sequence of invariants
 Converse Lyapunov
Converse Lyapunov theorems show that conditions of the previous
theorem are also necessary. For example, if the system is asymptotically
stable then there exists a positive definite, continuously differentiable
function V, that satisfies the inequalities.

For example if the LTI system is globally asymptotically stable then there is
a quadratic Lyapunov function that proves it.