ME5701: Mathematics for Engineering Research

Professor Gregory CHIRIKJIAN

Assistant Professor Gianmarco MENGALDO

 Week 4: Dynamical Systems II

Assistant Professor Gianmarco MENGALDO

Dynamical Systems - Week 4 Outline

Outline for Week 4

Recap on linear dynamical system

Types of control, stability, controllability and observability, and optimal control
Laplace transform and impulse response

Part I: Nonlinear dynamical systems

Introduction
Intuitive examples
Solution methods
Linearization around equilibrium points
Koopman operator theory [optional]
Control

Part II: Optimization in the context of control

Introduction
Gradient descent
Newton method

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 3

Dynamical Systems - Module I

Recap on linear dynamical systems

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research

Dynamical Systems - Week 4 Recap on linear dynamical systems

General picture

dx dx
= Φ(x, u) = Ax + Bu
dt dt

y = Ψ(x, u) y = Cx + Du
Time-invariant Linear dynamical
dynamical system system

c
mq̈ + cq̇ + kq = 0
  
d x1 c/m k/m x1
=
dt x2 1 0 x2

Connections to
Practical examples
ODEs

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 5

Dynamical Systems - Week 4 Recap on linear dynamical systems

Analytical solution

xic = expA(t t0 )
x0
Initial condition response | {z }
matrix exponential

+ Z t
xu = eA(t ⌧)
Bu(⌧ ) d⌧
Forced response t0
| {z }
convolution between input and solution
=
Z t
A(t t0 )
Total solution xic + xu = e x0 + eA(t ⌧)
Bu(⌧ ) d⌧
t0

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 6

Dynamical Systems - Week 4 Recap on linear dynamical systems

Stability

1
xic = Te[diag( i )](t t0 )
T x0

1
Initial conditions in transformed variables z0 = T x0

1
Solution for transformed variables e[diag( i )](t t0 )
T x0 = z

1
Solution for original variables Te[diag( i )](t t0 )
T x0 = xic

2 3
1 (t t0 )
e
6 .. 7
e[diag( i )](t t0 )
=4 . 5 Easy to compute
n (t t0 )
e

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 7

Dynamical Systems - Week 4 Recap on linear dynamical systems

Stability

xic = expA(t t0 )
x0
| {z }
matrix exponential

1
1
xic = eA(t t0 )
x0 = eTÃT (t t0 )
= Te[diag( i )](t t0 )
T x0

The eigenvalues i are generally complex numbers;

we define ↵i as the real part and i as the imaginary part of i such that:

i (t t0 )
e = e↵i (t t0 ) j
e i (t t0 )

= e|↵i (t t0 )
{z } {cos [ i (t t0 )] + j sin [ i (t t0 )]}
| {z }
monotonic part oscillatory part

Stability depends on monotonic part, or, in other words, on the real

part of the eigenvalues of A

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 8

Dynamical Systems - Week 4 Recap on linear dynamical systems

Stability

i (t t0 )
e = e↵i (t t0 ) j
e i (t t0 )

= e|↵i (t t0 )
{z } {cos [ i (t t0 )] + j sin [ i (t t0 )]}
| {z }
monotonic part oscillatory part

Im

Stable region α < 0 Unstable region α > 0

Re

Stability depends on monotonic part, or, in other words, on the real

part of the eigenvalues of A
Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 9
Dynamical Systems - Week 4 Recap on linear dynamical systems

Phase portraits

dx
= Ax + Bu,
dt

x(0) = x0

Asymptotically Asymptotically Asymptotically Stable

stable stable stable

Unstable Unstable Unstable Unstable

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 10

Dynamical Systems - Week 4 Recap on linear dynamical systems

Role of control u

i (t t0 )
e = e↵i (t t0 ) j
e i (t t0 )

= e|↵i (t t0 )
{z } {cos [ i (t t0 )] + j sin [ i (t t0 )]}
| {z }
monotonic part oscillatory part

Im dx
= Ax − BKx
× u = -Kx dt
= (A − BK)x
Task of control is
Re to push unstable
× eigenvalues to
Unstable eigenvalues stable region

Stability depends on monotonic part, or, in other words, on the real

part of the eigenvalues of A

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 11

Dynamical Systems - Week 4 Recap on linear dynamical systems

Controllability and observability

Theorem: given the controllability matrix

C = [B AB A2 B . . . An 1
B]

the system is controllable iff rank(C) = n

x ∈ ℝn
dx
= Ax + Bu A ∈ ℝn × n
dt B ∈ ℝn × p
y = Cx C ∈ ℝq × n
y ∈ ℝq Theorem: given the observability matrix
2 3
C
6 CA 7
6 7
6 CA2 7
O=6 7
6 .. 7
4 . 5
CAn 1

the system is observable iff rank(O) = n

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 12

Dynamical Systems - Week 4 Recap on linear dynamical systems

Optimal control
x ∈ ℝn
dx
= Ax + Bu A ∈ ℝn × n
dt B ∈ ℝn × p
y = Cx C ∈ ℝq × n u y
y ∈ ℝq

For linear dynamical systems there are two

results that provide optimal control and state
estimation

Control: Linear Quadratic Regulator (LQR)

Control Estimate
Estimation: Kalman filter (full state estimator)
B = − K x̃ x̃ ≈ x

The two together provide an optimal control LQR Kalman filter

problem for linear systems that goes under the
name of LQG (Linear Quadratic Gaussian) Optimal control
LGQ

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 13

Dynamical Systems - Week 4 Recap on linear dynamical systems

Laplace transform

dx L [x(t)] = x̂(s)
= Ax + Bu s x̂ (s) − x(0) = Ax̂ (s) + Bû (s)
dt
L [u(t)] = û(s)
y = Cx + Du ŷ (s) = Cx̂ (s) + Dû (s)
L [y(t)] = ŷ(s)

Differential system Laplace Transform Algebraic system

Solution is through a
x̂ (s) = (sI − A)−1Bû (s) + (sI − A)−1x(0)
set of algebraic equations
for s different from
ŷ (s) = [C(sI − A)−1B + D]û (s) + C(sI − A)−1x(0)
eigenvalues of A

An important relationship is the one between the input and the output,
of our linear system: ŷ (s) = [C(sI − A)−1B + D]û (s) = G(s)û (s)

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 14

 Dynamical Systems - Week 4 Recap on linear dynamical systems

Impulse response

ion L(f )(s)

f (t) f (t) L(f=)(s)
F (s)
= F (s) Z 1
Impulse Laplace transform y= g(t ⌧ ) u(⌧ )d⌧
0 | {z }
se (t) (t) 1 1 impulse response
1 1
g(t) g(t) s s

1
The output1of a linear dynamical system is completely
t · g(t)t · g(t) s2 s2
characterized by its impulse response, why?
n n n! n!
power t · g(t)
t · g(t) sn+1 sn+1
Zat 1 at 1 1 time
ential
decay decay
e e
Weighted integral of δ functions
s+a s+a
u= u(⌧ ) ⌧ d⌧ invariant
0 sin(!t) ! !
sin(!t) s2 +! 2 s2 +! 2
Z 1 s s
ycos(!t)
= cos(!t)
G[ ⌧ (t)]u(⌧
s2 +! 2 s)2 +!
d⌧2 Response given system G
0
Z 1
Impulse response y= g⌧ (t)u(⌧ ) d⌧
g⌧ (t) = G[ ⌧ (t)]
for system G 0

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 15

Dynamical Systems - Week 4 Recap on linear dynamical systems

Take-home message 1

Given a transfer function G(s):

if the system is stable, then G(s) can provide information about the stability of a
feedback control system (no need to enter into details about gain and phase
margins)
If the system is unstable, then G(s) does NOT provide any information
regarding the stability of a feedback control system

u y
G(s)

Gc(s)

The impulse response of a system can be used to determine its stability:

If it diverges the system is unstable
If it oscillates forever without diverging or converging the system is stable
If it has converging oscillations the system is asymptotically stable

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 16

Dynamical Systems - Week 4 Recap on linear dynamical systems

Take-home message 2

State space Impulse response

dx Z 1
= Ax + Bu y= u g(t ⌧ ) d⌧
dt | {z }
0
impulse response
y = Cx + Du
Time
domain

Transfer function

s x̂ (s) − x(0) = Ax̂ (s) + Bû (s)

ŷ (s) = Cx̂ (s) + Dû (s)

Frequency
domain

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 17

Dynamical Systems - Week 4 Recap on linear dynamical systems

Final recap notes

Time-invariant and continuous Linear Dynamical Systems

dx Strong theoretical results

= Ax + Bu
dt Excellent knowledge of their behavior
y = Cx + Du
However: Many problems are nonlinear!

Time-invariant and continuous Nonlinear Dynamical Systems

dx dx dx
= Φ(x, u) = L + N(x, u, κ) = L + N(x, u, κ)
dt dt dt

y = Ψ(x, u) y = Ψ(x, u)

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 18

Dynamical Systems - Module I

Part I
Nonlinear dynamical systems

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Real-world problems are tough!

Nonlinearity: we have a model

High-dimensionality: hidden model

Unknown dynamics: no model

In addition, lack or inaccurate

measurements

Courtesy of Nils Wedi (Head of Earth System Modeling at ECMWF)

Wedi et al., (2020) A baseline for global weather and climate simulations at 1 km resolution, to appear in JAMES.
Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 20
Dynamical Systems - Module I Part I: Introduction to dynamical systems

… so, the big question here is: what can we do about real-world problems?

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Real-world problems are tough!

- Solve the nonlinear equations

- Linearize the system
- Koopman operator theory

Nonlinearity: we have a model

Pattern identification via e.g.

High-dimensionality: hidden model - SVD,
- Tensor Analysis

Unknown dynamics: no model

Data-driven model discovery

via e.g. machine learning

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 22

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Real-world problems are tough!

- Solve the nonlinear equations

Nonlinearity: we have a model

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 23

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Single pendulum

Let’s solve the nonlinear system directly

mL2 ✓¨ + c✓˙ + mgL sin(✓) = 0

  L
x1 ✓
x= =
x2 ✓˙
   ✓
dx d x1 ✓˙ x2
= = = g m
dt dt x2 ✓¨ L sin(x1 ) c
mL2 x2

Check pendulum_single.m file g

available as part of this module

I have added damping, but we will set it to zero and then increase its value

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 24

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Single pendulum


✓=⇡ Pendulum up
x̄b =
✓˙ = 0
g

30
sol1
sol2
20
x1

10

0
0 10 20 30 40 50 60 70 80
t
2
sol1
1 sol2
x2

-1

-2
0 10 20 30 40 50 60 70 80
t

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 25

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Single pendulum


✓=⇡ Pendulum up
x̄b =
✓˙ = 0
g
x 2 = t [deg/s]

100
0
-100
-90 0 90 180 270 360
x1 = [deg]

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 26

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Single pendulum


✓=0 Pendulum down
x̄a =
✓˙ = 0
g

0.2
sol1
0.1 sol2
x1

-0.1

-0.2
0 10 20 30 40 50 60 70 80
t
0.2
sol1
0.1 sol2
x2

-0.1

-0.2
0 10 20 30 40 50 60 70 80
t

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 27

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Single pendulum


✓=0 Pendulum down
x̄a =
✓˙ = 0
g

100
x 2 = t [deg/s]

50

-50

-100

-90 0 90 180 270 360

x1 = [deg]

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 28

 remain within Part
theI: Nonlinear
Dynamical Systems - Week 4
region dynamical systems δ
defined
Connections by stability?
with

Solution xa is stable
iff ∀ϵ > 0, there exist
Dr Thulasi Mylvaganam

n the sense of Lyapunov) δϵ > 0 such that ∀δx0

that satisfies

∥δx0∥ ≤ δϵ
Note: Stability is a property of the equilibrium
re exists a such ε
art within the we have that
d by , you
n the region δ ∥δx∥ ≤ ϵ, ∀ t

In nonlinear systems, it is a
property of the equilibrium point
(in other words, of where our
system starts from)

a property of the equilibrium

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 29

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Double pendulum
Let’s solve the nonlinear system directly

(m1 + m2 )L1 ✓¨1 + m2 L2 ✓¨2 cos(✓1 ✓2 ) + . . . L1

+ m2 L2 ✓˙2 sin(✓1 ✓2 ) + g(m1 + m2 ) sin(✓1 ) = 0
2

m2 L2 ✓¨2 + m2 L1 ✓¨1 cos(✓1 ✓2 ) + . . . ✓1 m1

m2 L1 ✓˙2 sin(✓1 ✓2 ) + m2 g sin(✓2 ) = 0
1
L2
2 3 2 3 2 3 2 3 2 3
x1 ✓1 x1 ✓˙1 x2
6 x2 7 6 ✓˙1 7 6 7 6 7 6 7 ✓2
6 7 6 7 d 6 x2 7 6 ✓1 7
6 ¨
7 6
ed bf
7
4 x3 5 = 4 ✓2 5 6
6
7 6
7=6 ˙ 7 =
6
6
ad cb 7
7
x4 ✓˙2 dt 6 x3 7 6 ✓2 77 6 x4 7 m2
4 5 4 5 4 5
af ce
x4 ✓¨2 ad cb
g
Equivalent dynamical system

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 30

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Double pendulum
2 3 2 3
x1 ✓ 1
6 x2 7 6 ✓˙1 7 Check pendulum_double.m file
6 7 6 7
4 x3 5 = 4 ✓2 5 available as part of this module L1
x4 ✓˙2
2 3 2 ˙ 3 2 3
x1 ✓1 x2 ✓1
6 7 6 7 6 ed bf 7 m1
6 7 6 ¨ 7 6
d 6 x 2 6
7 6
✓ 1 7 6 ad cb 7 7
6 7=6 ˙ 7 = 6 7
dt 6 x3 7 6 ✓2 7 7 4 6 x 4 7 L2
4 5 4 5 5
af ce
x4 ✓¨2 ad cb
✓2
a = (m1 + m2 )L1
b = m2 L2 cos(x1 x3 ) m2
c = m2 L1 cos(x1 x3 )
d = m2 L2
e= m2 L2 x24 sin(x1 x3 ) g(m1 + m2 ) sin(x1 ) g
f = m2 L1 x22 sin(x1 x3 ) m2 g sin(x3 )

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 31

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Double pendulum
ODE45 = Runge-Kutta 4

Local error ∼ O(Δt5) Taylor expansion up to fourth term

Global error ∼ O(Δt4) Accumulation of local errors over integration period

x
At each time-step, a small perturbation of
the solution is amplified by the nonlinear
dynamics: trajectories diverge!

dt
Integrating numerically nonlinear
systems is dangerous

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 32

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Double pendulum
ODE45 = Runge-Kutta 4

Local error ∼ O(Δt5) Taylor expansion up to fourth term

Global error ∼ O(Δt4) Accumulation of local errors over integration period

Integrating numerically nonlinear

systems is dangerous

Recipes to avoid disasters:

Sequentially halve time-step and check solution agrees with the one obtained
using larger time-step
Symplectic integrators —> Hamiltonian system conservation
Variational integrators —> Lagrangian system conservation

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 33

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Real-world problems are tough!

- Solve the nonlinear equations

- Linearize the system

Nonlinearity: we have a model

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 34

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 35

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

Linearization

 
dx
q·· = q − q 3 = χ
x1 q
x= = x̄ : =0
x2 q̇ dt
q 2(q 2 − 2)   
χ= dx x2 x2 0
= =
4 dt x1 x31 x1 (1 x21 ) 0

Nonlinear ODE Nonlinear dynamical Fixed (equilibrium) points

system

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 36

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A


1
dx x̄a =
0
x̄ : =0
dt 
0
  x̄b =
x2 0 0
= 
x1 (1 x21 ) 0
1
x̄c =
Fixed (equilibrium) points 0 x̄a x̄b x̄c

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 37

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

0.5 x̄a x̄b x̄c

x2

Stable Unstable Stable

-0.5

x̄a x̄b x̄c

-1
-1.5 -1 -0.5 0 0.5 1 1.5
x1

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 38

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

Nonlinear ODE
··
mL 2θ + mgL sin(θ) = 0 L

✓
m

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 39

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

Nonlinear ODE
··
mL 2θ + mgL sin(θ) = 0 L

✓
m

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 40

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

Nonlinear ODE
··
mL 2θ + mgL sin(θ) = 0 Linearization

 
x1 ✓ dx
x= = x̄ : =0
x2 ✓˙ dt x̄
  
dx x2 x2 0
= g g =
dt L sin(x1 ) L sin(x1 ) 0

Nonlinear dynamical system Fixed (equilibrium) points

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 41

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

Pendulum down
dx 
x̄ : =0 ✓=0
dt x̄a = g
x̄ ✓˙ = 0
  
x2 0 ✓=⇡
x̄b =
g = ✓˙ = 0
L sin(x1 ) 0 Pendulum up

Fixed (equilibrium) points

g

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 42

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

 
✓=0 ✓=⇡
Stable x̄a = x̄b = Unstable
✓˙ = 0 ✓˙ = 0
100
x 2 = t [deg/s]

50

-50

-100

-90 0 90 180 270 360

x1 = [deg]

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 43

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

 
✓=0 ✓=⇡
Stable x̄a = x̄b = Unstable
✓˙ = 0 ✓˙ = 0
100
x 2 = t [deg/s]

50

-50

-100

-90 0 90 180 270 360

x1 = [deg]

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 44

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

Pendulum down
 
✓=0 ✓=⇡ Unstable
Stable x̄a = x̄b =
✓˙ = 0 ✓˙ = 0
100 g Small
perturbations
x 2 = t [deg/s]

50

-50
Pendulum up
-100

-90 0 90 180 270 360

x1 = [deg] g

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 45

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx xic = expA(t t0 )
x0
= Φ(x) = Ax | {z }
dt dt matrix exponential

Stability depends on
the eigenvalues of A

Taylor expansion Pendulum down

d(x̄ + x′)
= Φ(x̄ + x′)
dt g Small
DΦ D2 Φ (x′)2 perturbations
≈ Φ(x̄) + x′ + +…
Dx Dx2 2!
x̄ x̄
Pendulum up

dx′ DΦ DΦ
= x′ Jacobian J = g
dt Dx Dx
x̄ x̄

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 46

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Jacobian

dx′ DΦ dx xic = expA(t t0 )

x0
= x′ = Jx = Ax | {z }
dt Dx dt matrix exponential
x̄
Stability depends on
the eigenvalues of A

DΦ
Jacobian J =
Dx
x̄
2 3
@ 1 /@x1 @ 1 /@x2 ... @ 1 /@xn
D 6 @ 2 /@x1 @ 2 /@x2 ... @ 2 /@xn
7
6 7
J= =6 .. .. .. .. 7
Dx x̄ 4 . . . . 5
@ n /@x2 @ n /@x2 ... @ n /@xn x̄

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 47

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Recap linearization of a nonlinear system

dx dx
= Φ(x) = Ax
dt dt

1. Identify fixed points x̄ : Φ(x̄) = 0

2. Compute Jacobian of the system
2 3
@ 1 /@x1 @ 1 /@x2 ... @ 1 /@xn
D 6 @ 2 /@x1 @ 2 /@x2 ... @ 2 /@xn
7
6 7
J= =6 .. .. .. .. 7
Dx x̄ 4 . . . . 5
@ n /@x2 @ n /@x2 ... @ n /@xn x̄

d(x̄ + x′)
= Φ(x̄ + x′)
dt dx′ DΦ
DΦ D2 Φ (x′)2 = x′
≈ Φ(x̄) + x′ + +… dt Dx
Dx Dx 2 2! x̄
x̄ x̄

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 48

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Bifurcations

q̈ = q q3 q̈ = q q3 = 0 q( q2 ) = 0

Fixed points

q=0p q
q = ± , for  > 0
Stable
Jacobian for stability of fixed points

D(q q 3 )
J= = 3q 2 Stable Unstable
Dq 

J|q=0 = 
Stable
J|q=±p = 2, for  > 0

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 49

 Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Lyapunov stability

x x movimento
Solution for xb
perturbato ϵ Solution xa is stable
iff ∀ϵ > 0, there exist
δϵ > 0 such that ∀δx0
that satisfies
Solution for xa
movimento ∥δx0∥ ≤ δϵ
x 0b nominale
we have that
xx0a ∥δx∥ ≤ ϵ, ∀ t
0n t
t

Fig. 3 : Movimento stabile

i dirà instabile se non è stabile, ossia se non soddisfa la condizione d

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 50
Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx 1. Identify fixed points x̄ : Φ(x̄) = 0

= Φ(x) = Ax
dt dt 2. Compute Jacobian of the system
2 3
@ 1 /@x1 @ 1 /@x2 ... @ 1 /@xn
D 6 @ 2 /@x1 @ 2 /@x2 ... @ 7
2 /@xn
6 7
J= =6 .. .. .. .. 7
Dx x̄ 4 . . . . 5
@ n /@x2 @ n /@x2 ... @ n /@xn


Example: pendulum ✓=0
x̄a =
✓˙ = 0 Fixed
Equations of motions (dynamics)  points
✓=⇡
·· x̄b =
✓˙ = 0
mL 2θ + mgL sin(θ) = 0
L
 
x1 ✓
✓ x= =
x2 ✓˙
m   
dx d x1 ✓˙ x2
= = = g
g dt dt x2 ✓¨ L sin(x1 )

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 51

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx 1. Identify fixed points x̄ : Φ(x̄) = 0

= Φ(x) = Ax
dt dt 2. Compute Jacobian of the system
2 3
@ 1 /@x1 @ 1 /@x2 ... @ 1 /@xn
D 6 @ 2 /@x1 @ 2 /@x2 ... @ 7
2 /@xn
6 7
J= =6 .. .. .. .. 7
Dx x̄ 4 . . . . 5
@ n /@x2 @ n /@x2 ... @ n /@xn


Example: pendulum ✓=0
x̄a =
✓˙ = 0 Fixed
 points
✓=⇡
x̄b =
Jacobian (linearized dynamics) ✓˙ = 0
L 
0 1
J= g
✓ L cos(x1 ) 0

m   Jacobians
0 1 0 1
J|x̄a = g J|x̄b = g evaluated at
L 0 L 0
g fixed points

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 52

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx 1. Identify fixed points x̄ : Φ(x̄) = 0

= Φ(x) = Ax
dt dt 2. Compute Jacobian of the system
2 3
@ 1 /@x1 @ 1 /@x2 ... @ 1 /@xn
D 6 @ 2 /@x1 @ 2 /@x2 ... @ 7
2 /@xn
6 7
J= =6 .. .. .. .. 7
Dx x̄ 4 . . . . 5
@ n /@x2 @ n /@x2 ... @ n /@xn


Example: pendulum ✓=0
x̄a =
✓˙ = 0 Fixed
 points
✓=⇡
x̄b =
✓˙ = 0
L Eigenvalues (linearized dynamics)
 
0 1 0 1
✓ J|x̄a = g J|x̄b = g
L 0 L 0
m

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 53

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx 1. Identify fixed points x̄ : Φ(x̄) = 0

= Φ(x) = Ax
dt dt 2. Compute Jacobian of the system
2 3
@ 1 /@x1 @ 1 /@x2 ... @ 1 /@xn
D 6 @ 2 /@x1 @ 2 /@x2 ... @ 7
2 /@xn
6 7
J= =6 .. .. .. .. 7
Dx x̄ 4 . . . . 5
@ n /@x2 @ n /@x2 ... @ n /@xn

Example: pendulum

 
✓=0 0 1
x̄a = J|x̄a = g
✓˙ = 0 L 0

r
g
( I J|x̄a ) ) 1,2 = ±j Stable
L

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 54

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Linearization of a nonlinear system

dx dx 1. Identify fixed points x̄ : Φ(x̄) = 0

= Φ(x) = Ax
dt dt 2. Compute Jacobian of the system
2 3
@ 1 /@x1 @ 1 /@x2 ... @ 1 /@xn
D 6 @ 2 /@x1 @ 2 /@x2 ... @ 7
2 /@xn
6 7
J= =6 .. .. .. .. 7
Dx x̄ 4 . . . . 5
@ n /@x2 @ n /@x2 ... @ n /@xn

Example: pendulum

 
✓=⇡ 0 1
x̄b = J|x̄b = g
✓˙ = 0 L 0

r
g
( I J|x̄b ) ) 1,2 =± Unstable
L

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 55

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Pendulum

Equations of motions (dynamics)

mL2 ✓¨ + c✓˙ + mgL sin(✓) = 0

L
 
x1 ✓ 
✓ x= = ✓=0
x2 ✓˙ x̄a =
✓˙ = 0 Fixed
m   
dx ✓˙ x2 ✓=⇡ points
= = x̄b =
g dt ✓¨
g
L sin(x1 )
c
mL2 x2
✓˙ = 0

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 56

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Pendulum with control

Equations of motions (dynamics)

mL2 ✓¨ + c✓˙ + mgL sin(✓) = u

L
 
x1 ✓
✓ x= =
x2 ✓˙
m   
dx ✓˙ x2 0
= ¨ = g c + 1 u
g dt ✓ L sin(x1 ) mL2 x2 mL2

 
✓=⇡ 0 1
x̄b = J|x̄b = g
✓˙ = 0 L 0

r
g
( I J|x̄b ) ) 1,2 =± Unstable
L

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 57

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Recap so far

- Solve the nonlinear equations

Nonlinearity: we have a model
- Linearize the system

Integrating numerically nonlinear systems is dangerous

Check time-step convergence
Use appropriate time-integrators (symplectic, variational)

Linearize the system

Identify fixed points,
Linearize system around fixed points,
Study stability of fixed points,
Control linearized system around fixed points

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 58

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Last note

When a linearized system is an acceptable representation of its corresponding

nonlinear system?

Hartman-Grobman theorem (also known as linearization theorem).

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 59

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Real-world problems are tough!

- Solve the nonlinear equations

- Linearize the system
- Koopman operator theory

Nonlinearity: we have a model

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 60

Dynamical Systems - Week 4 Part I: Nonlinear dynamical systems

Koopman operator theory at glance [Optional]

Nonlinear finite representation Linear infinite representation

q̇1 = µq1
Kt g = g
q̇2 = ⌫(q1 q22 )

2 3 2 3 In practice, seek
y1 q1 finite representation
6 7 6 7
4 y2 5 = 4 q 2 5
y3 q12
2 3 2 32 3 Ergodic Theory, Dynamic Mode Decomposition,
y1 µ 0 0 y1 and Computation of Spectral Properties of the
d 6 7 6 76 7 Koopman Operator
4 y2 5 = 4 0 ⌫ ⌫ 5 4 y2 5 H. Arbabi and I. Mezić
dt SIAM Journal on Applied Dynamical Systems (2017)
y3 0 0 2µ y3

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 61

Dynamical Systems - Module I

Part II
Optimization in the context of control

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Reminder of optimal control for LINEAR systems

dx x ∈ ℝn
= Ax + Bu Control
dt A ∈ ℝn × n Optimal K matrix
B ∈ ℝn × p B = -Kx
y = Cx + Du C ∈ ℝq × n
>> K = lqr(A,B,Q,R)

Z 1
F= yT Qy + uT Ru dt Functional to be minimized
0 for optimal controller

If not in desired position

xT Qx = state penalization
xT Qx 0

Minimal energy to reach desired

xT Rx = control penalization state otherwise
xT Rx 0

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 63

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Some more …
dx Z
= Ax + Bu 1 Functional to be
dt F= yT Qy + uT Ru dt minimized for optimal
controller subject to
0
y = Cx + Du linear dynamics

By substituting y = Cx + Du into the functional F and by defining:

Q̃ = CT QC Z 1  T   !
x Q̃ S x
S = CT QD F= dt
0 u ST R̃ u
T
R̃ = D QD + R
dx
Subject to = Ax + Bu
dt

Constrained (to the dynamics) optimization

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 64

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Constrained and unconstrained optimization

Constrained optimization
Z    !
1 T
x Q̃ S x
F= dt
0 u ST R̃ u

dx
Subject to = Ax + Bu
dt

Constraint
Unconstrained optimization
Z (   ✓ ◆)
1 T
x Q̃ S x T dx
F= + Ax + Bu dt
0 u ST R̃ u `
dt

Lagrange
additional degrees of freedom multipliers

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 65

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Generalization constrained optimization

f = F(x, u) Functional
(x, u) = 0 Constraint (equality)
Must hold ∀δx
!
@
@F @F @F @F @x
f= x+ u=0 f= @
x=0
@x @u @x @u @u

@ @
x+ u=0
@x @u @F @F @
@x
@
=0
@x @u @u
@
@x x
u= @
@u
(x, u) = 0
Constrained optimization
problem

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 66

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Constrained to unconstrained (Lagrange multipliers)

f = F(x, u) Functional
(x, u) = 0 Constraint (equality)

✓ ◆
@F @F @F @F @ @
f= x+ u=0 x+ u+ ` x+ u =0
@x @u @x @u @x @u
✓ ◆ ✓ ◆
@ @ @F @ @F @
x+ u=0 + ` x+ + ` u=0
@x @u @x @x @u @u
If valid, their linear
@F @
combination is also valid + ` =0
@x @x Equivalent
@F @ unconstrained
+ ` =0 optimization
f (x, u, `) = F(x, u) + ` (x, u) @u @u problem
Equivalent unconstrained functional (x, u) = 0

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 67

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Remember …

Constrained optimization
Z    !
1 T
x Q̃ S x
F= dt
0 u ST R̃ u

dx
Subject to = Ax + Bu
dt

Constraint
Unconstrained optimization
Z (   ✓ ◆)
1 T
x Q̃ S x T dx
F= + Ax + Bu dt
0 u ST R̃ u `
dt

Lagrange
additional degrees of freedom multipliers

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 68

Dynamical Systems - Week 4 Part II: Optimization in the context of control

For inequality constraints (KKT conditions) [Optional]

y= (x, u) Functional
Optimization problem
(x, u) = 0 Constraint (equality) with inequality constraints
⇠(x, u) > 0 Constraint (inequality)

Equivalent unconstrained
f (x, u, ` , µ` ) = F(x, u) + ` (x, u) + µ` ⇠(x, u)
functional

Karush-Kuhn-Tucker (KKT) theorem. If (x*, μ* ℓ

) is a saddle point of f (x, u, λℓ, μℓ),
in a convex subset x ∈ X ⊂ ℝn, and μℓ ≥ 0, then x* is an optimal solution for the above
optimization problem above.

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 69

 Dynamical Systems - Week 4 Part II: Optimization in the context of control

Optimization
In general, we have a functional to optimize (aka, minimize or maximize),
that can have various shapes
f (x, u, `) = F(x, u) + ` (x, u)

u u

x
x
Convex functional Sort of complicated Quite complicated
functional functional

Complexity
Pictures credit: O’Reilly Media [2 left], https://www.cs.umd.edu/~tomg/projects/landscapes/ [right]
Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 70
Dynamical Systems - Week 4 Part II: Optimization in the context of control

Optimization areas

Convex optimization

Quadratic optimization

Fractional optimization
We will focus on gradient
descent and Newton’s method
Nonlinear optimization
broadly applicable to a relatively
large set of problems
Stochastic optimization

Robust optimization

etc …

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 71

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Gradient descent

min F(z)
z
Iterate zk+1 ( ) = zk rF(zk )

Direction of steepest descent

… however, local minima
Local minimum

Note, many machine learning algorithms face

the same challenge of minimizing complicated
functionals (or loss functions)
Global minimum

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 72

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Gradient descent

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 73

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Newton’s method

Second-order Taylor expansion min F(z)

z
0 00
2
F(zk + ) ⇡ F(zk ) + F (zk ) + F (zk )
2!
 2
d d 0 00
F(zk + ) ⇡ F(zk ) + F (zk ) + F (zk ) =0
d d 2!

d 0 00
F(zk + ) = F (zk ) + F (zk ) Local minimum
d
0
F (zk )
=
F 00 (zk )

0
F (zk )
Iterate zk+1 = zk + = zk
F 00 (zk ) Global minimum

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 74

 Consider the unconstrained optimization problem:
Dynamical Systems - Week 4 Part II: Optimization in the context of control
the gradient of f (x) and H(x) is the Hessian of f (x) at x.
(P): min f (x)
t Newton’s method
h(·) is a quadratic function. We can attempt to minimize
ting a point x for which ∇h(x) x=∈0.Rn Since
. the gradient of

At x = x̄, f (·) can be approximated by its second-order Taylor expansion

Second-order
∇h(x) = ∇fTaylor
(x̄) +expansion
H(x̄)(x − x̄) , min F(z)
at x̄, which we denote by h(·): z
re motivated to solve:
f (x) ≈ h(x) := f (x̄) + ∇f (x̄)T (x − x̄) + 1 (x − x̄)T H(x̄)(x − x̄) .
2

Here ∇f (x) is the gradient of f (x) and H(x) is the Hessian of f (x) at x.
0 = ∇h(x) = ∇f (x̄) + H(x̄)(x − x̄) ,
Notice that h(·) is a quadratic function. We can attempt to minimize
h(·)
whichby is given bya point x for which ∇h(x) = 0. Since the gradient of
computing
h(x) is:
−1 Local minimum
Iterate x = x̄ −∇h(x)
zk+1 H(x̄)
= zk += ∇f
∇f=(x̄)
(x̄) H 1−
zk+. H(x̄)(x rF
x̄) ,
we therefore
−H(x̄) −1 ∇fare
(x̄)motivated
is called to
thesolve:
Newton direction, or the Newton
we compute the next iterate as
0 = ∇h(x) = ∇f (x̄) + H(x̄)(x − x̄) ,
x̄N := x̄ − H(x̄)−1 ∇f (x̄) ,
the solution of which is given by
x̄N the Newton iterate xfrom the
= x̄ − point
H(x̄) −1
∇fx̄.
(x̄) If
. we compute all Global minimum
way, we call the algorithm Newton’s Method, whose formal
The direction −H(x̄)−1 ∇f (x̄) is called the Newton direction, or the Newton
presented
step, at x̄. in Algorithm
If we 1. next iterate as
compute the
t Dr
Newton’s Method presumes that H(x k ) is nonsingular at
Gianmarco Mengaldo −1 75
x̄N := x̄ − H(x̄) ∇f (x̄) ,for Engineering Research
ME5701: Mathematics
Dynamical Systems - Week 4 Part II: Optimization in the context of control

Newton’s method

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 76

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Optimal control for nonlinear systems

min F(x, u) Functional for nonlinear problems

x,u

dx
= Φ(x, u)
dt Subject to the nonlinear dynamics

y = Ψ(x, u)

Generally, nonlinear control is difficult — typically one wishes

to linearize such that linear optimal control can be applied

Z 1 ⇣ ⌘
F= xT Qx + uT Ru dt Functional for linear problems
0 Subject to linear dynamics

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 77

Dynamical Systems - Week 4 Part II: Optimization in the context of control

Pendulum with control

Equations of motions (dynamics)

mL2 ✓¨ + c✓˙ + mgL sin(✓) = u

L
 
x1 ✓
✓ x= =
x2 ✓˙
m   
dx ✓˙ x2 0
= ¨ = g c + 1 u
g dt ✓ L sin(x1 ) mL2 x2 mL2

 
✓=⇡ 0 1
x̄b = J|x̄b = g
✓˙ = 0 L 0

r
g
( I J|x̄b ) ) 1,2 =± Unstable
L

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 78

Dynamical Systems - Module I

Final notes on these two weeks

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research

Dynamical Systems - Week 4

What I did not talk about in these two weeks?

dx
= Φ(x, u, t; κ) + d
dt Dynamical systems subject to stochastic d
disturbances and measurement noise w
y = Ψ(x, t) + w

Kalman filter
Robust control
Control with delays

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 80

Dynamical Systems - Module I Part I: Introduction to dynamical systems

This is an very broad course covering several complex subjects

You are not meant to know in depth every single topic introduced, but you
should rather know in depth the topics that interest you the most or that
you might need in your current work and future career!

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research

Dynamical Systems - Week 4

Learning outcomes

Part I: Nonlinear dynamical systems

LO1.) Definition and issues of nonlinear dynamical systems

LO2.) Fixed (or equilibrium) points of a nonlinear dynamical system

LO3.) Linearization around fixed points and why it is useful

LO4.) Control for nonlinear dynamical systems around fixed points

Part II: Optimization in the context of control

LO5.) What optimization is in the context of control

LO6.) Gradient descent method

LO7.) Newton’s method

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 82

Dynamical Systems - Week 4

Exercises
m

g
L

✓
M u

Given the above simplified model of e.g. a Segway, along with the equations of
motions and parameters provided in the next slide:
1. Linearize the equations around the pendulum in the upward position (θ = π)
2. Write the corresponding linear dynamical system
3. Write the transfer function
4. Solve the linear dynamical system via
1. Impulse response in Matlab
2. ODE45 in Matlab
5. Discuss the stability of the system
6. Design an optimal control (LQR) for the system in Matlab

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 83

Dynamical Systems - Week 4

Exercises

mL2 (M + m(1 cos(✓)2 ))q̈ + m2 L2 g cos(✓) sin(✓) + . . .

Cart
mL2 (mL✓˙2 sin(✓) cq̇) = mL2 u

mL2 (M + m(1 cos(✓)2 ))✓¨ + mL cos(✓)(mL✓˙2 sin(✓) cq̇) + . . .

Pendulum
(m + M )mgL sin(✓) = mL cos(✓)u

m
q = position of the cart
g θ = angle of the pendulum
L M = mass of the cart = 5
m = mass at the pendulum tip = 2
✓ g = gravity = -9.81
M u
L = length of the pendulum = 2
c = damping on the cart = 0.1
u = input
q

Dr Gianmarco Mengaldo ME5701: Mathematics for Engineering Research 84