Computational Control

Identification

Saverio Bolognani

Automatic Control Laboratory (IfA)

ETH Zurich
Model requirements

The methods that we have seen so far, i.e.,

LQR
Model predictive control
require a state space model of the system.

xt+1 = Axt + But

yt = Cxt

Knowing a state-space representation of a system implies a very good

knowledge of the plant.
What is the Markovian state?
What are the parameters of the update equation?
What are the parameters of the input (and output) maps?

1 / 21
No first principles to select the state
No idea of the dimension of the state
Accurate modeling sometimes suggests a huge state dimensions
Even if we can postulate a Markovian state, we cannot measure it

2 / 21
In fact, state-space representations are not even unique.

xt+1 = Axt + But

yt = Cxt

Consider an invertible square matrix T and the system

wt+1 = T −1 ATwt + T −1 But

yt = CTwt

The two systems have exactly the same input-output behavior.

Proof: Because of linearity, we only need to check that the response to each
input ut = ei δt is the same, where
(
1 if t = 0
δt =
0 otherwise

In the first case xt = 0, bi , Abi , A2 bi , . . . and yk = 0, Cbi , CAbi , CA2 bi , . . ..

In the second case wt = 0, T −1 bi , T −1 Abi , T −1 A2 bi , . . . and
yk = 0, Cbi , CAbi , CA2 bi , . . ..

3 / 21
Impulse response

t−1
X
yt = CAt x + CAt−1−j Buj
| {z }0
j=0
initial response | {z }
convolution

SISO system
Scalar input ut ∈ R, scalar output yt ∈ R.
Impulse response
yt = CAt−1 B t>0

Are you convinced that the impulse-response is a complete representation of

the input-output relation of the system?
How would you estimate is on a real plant?

4 / 21
Markov parameters

Generalization to MIMO systems

m inputs (u ∈ Rm ) and p outputs (y ∈ Rp )

CAt−1 B ∈ Rp×m , t>0

The elements in position i, j represent the output i to an unit impulse on the input j.
This sequence of matrices are the Markov parameters of the system.

How can you estimate these parameters? With how many experiments?

ut yt
?

5 / 21
Controllability and Observability
Controllability

Consider the problem of driving a system from x0 = 0 to a desired x̄ ∈ Rn .

In one step, we can drive the system to

x1 = Bu0

therefore to any state in the image of B.

In two steps, we can drive the system to

x2 = ABu0 + Bu1
 
therefore to any state in the image of B AB .
...
In n steps, we can drive the system to any state in the image of

C = B AB A2 B . . . An−1 B
 

which we call the controllability matrix of the system.

6 / 21
 A2 B An−1 B
 
C= B AB ...
If the rank of C is n (full row rank) then any state x̄ ∈ Rn can be reached in n steps.

Remarks
(obvious) You may be able to control the system to any state in less than n
steps
▶ Also with a scalar input?
(non obvious) If you don’t have controllability in n steps, adding additional
steps will not help.

Given a target state x̄, how can you compute the input sequence that drives a
controllable system to x̄?
un−1
 

If C is an invertible matrix (which means a scalar input u) then xn = C  ...  and

 

u0
this relation can be inverted to compute the input sequence.
In case of non-invertible C, the problem is underdetermined (there are more valid
input sequences). We can find one (the one with smallest norm) by using the
pseudoinverse C ⊤ (CC ⊤ )−1 of the matrix C.
7 / 21
Observability
Consider the problem of detecting the effect of an initial condition x0 in the output
of the system.
At t = 0, the output of the system is

y0 = Cx0

therefore any x0 in the kernel of C is unobservable.

In one step, the output is
y1 = CAx0
 
C
therefore any x0 in the kernel of is unobservable.
CA
. . . After n steps, any initial state in the kernel of the observability matrix

C
 
 CA 
 2 
O =  CA 
 
 . 
.
 . 
CAn−1

is unobservable.

8 / 21
 C
 
 CA  If the rank of O is n (full column rank)
 2 
O =  CA  then any initial state x0 ∈ Rn is
 
 . 
 ..  observable in the output.
CAn−1

Remarks
(obvious) You may be able to see the effect of x0 on the output in less than n
steps
▶ Also with a scalar output?
(non obvious) If you don’t have observability in n steps, adding additional
steps will not help.

Given a desired output sequence y0 , y1 , . . . , yn , what is the initial state x0 that

produces that sequence?
y0
 

If O is an invertible matrix (which means a scalar input u) then  ...  = Ox0 and
 

yn−1
this relation can be inverted to compute the initial state.
In case of non-invertible ′ we can employ the pseudoinverse as before.

9 / 21
Controllable and observable systems

Remember the goal

We want derive a state-space representation (realization) when the only thing we
know are input and output data.

→ We restrict our attention on systems that are controllable and observable.

“well-posed” relation between state and input/output signals
exclude the pathological case of extra states that have no effect on the I/O
aim at minimal realizations

Example: Consider a plant whose output is simply a moving average of the input

yt = a1 ut−1 + a2 ut−2 + a3 ut−3

What is a minimal state-space representation? Is it observable and controllable?

10 / 21
We use the state to store a delayed copy of the input:
 
ut−1
xt = ut−2 
ut−3

This is easily accomplished via the state-space model

   
0 0 0 1
xt+1 = 1 0 0 xt + 0 ut
0 1 0 0
| {z } |{z}
A B

which we then combine with the output equation

 
yt = a1 a2 a3 xt
| {z }
C

The controllability
 matrix
 is C = I and the observability matrix is
a1 a2 a3
O = a2 a3 0 .
a3 0 0

11 / 21
Identification from data

ut yt
?

Suppose that you only have the Markov parameters of the system.
For example: sequence of impulse-response experiments on the real plant.
How to compute A, B, C (realization)?

More general system ID

Given generic input-output data, construct a realization A, B, C.

12 / 21
Hankel matrix

C
 
 CA 
 2 
Hankel matrix H := OC =  CA  B AB A2 B . . . An−1 B
  
 . 
 .. 
CAn−1
CAB CA2 B . . . CAn−1 B
 
CB
 CAB
 CA2 B CA3 B . . . CAn B 

2 3
=  CA B
 CA B ... ... 

 ... ... 
CAn−1 B ... . . . CA2n−2 B

Remember the Markov parameters

CB, CAB, CA2 B, CAt−1 B,


..., ...

13 / 21
Ho-Kalman algorithm
Not the Kalman filter!

1 Construct the Hankel matrix H from the Markov parameters (experiments)

2 Factorize H as OC
3 Derive realization A, B, C from O and C

Step 1: Construct H

Problem: how many Markov parameters do we need? We don’t know n!

We are looking for a realization that is minimal (controllable and observable).

→ Both O and C will have rank n, so the Hankel matrix has rank n.
When do you stop adding Markov parameters? Try!

Low-pass filter Frictionless cart

xt+1 = axt + ut vt+1 = vt + ut

yt = xt zt+1 = zt + vt
yt = zt

14 / 21
Step 2: How do you factorize H in OC?
Interestingly, it does not matter!
How is that possible? Different O and C must return different A, B, C!
Consider the change of basis: Ã = T −1 AT , B̃ = T −1 B, C̃ = CT .
  
C̃ CT

  CTT −1 AT 
Õ =  C̃Ã  = OT
 . = ..
.. .

Similarly one can show that C˜ = T −1 C, so that ÕC˜ = OTT −1 C = OC.

→ different factorizations return you the same system in different coordinates!

You can take O = I and C = H

You can take O = H and C = I (do you remember?)
There are preferrable ways to do it (for numerical reasons)

15 / 21
Singular value decomposition
Let H be a m × n matrix of rank r.

H = UΣV ⊤
U is a m × m matrix
V is a n × n matrix
Σ is a “diagonal” m × n matrix
 
Σ1 0r×n−r ⊤
= U1 Σ1 V1⊤
  
H= U1 U2 V1 V2
0m−r×r 0m−r×n−r

Your linear algebra workhorse!

computationally very fast and numerically robust
U1 columns span the image of H
the dimension of Σ1 is the rank of H
factorizes a matrix is a numerically “balanced” way (orthonormal U, V )

Do you see how we could use this?

16 / 21
Extended Hankel matrix

Let us build a Hankel matrix of size k × ℓ, where k and ℓ are larger than n.
 
G1 G2 ... Gℓ
 G2 G3 ... Gℓ+1 
 
Hk,ℓ =  G 3 . . . 

. . . ... 
Gk Gk+1 . . . Gk+ℓ−1

How do you pick k, ℓ larger than n?

If these are the Markov parameters of an A, B, C system, then Hk,ℓ is the product
between
 
C
 CA 
2 
   2 ℓ−1

Ok = 
 CA  and Cℓ = B AB A B . . . A B
 ... 
CAk−1

called extended observability/controllability matrices (what is their rank?).

17 / 21
SVD of the Hankel matrix

Let’s compute the SVD of Hk,ℓ .

 
Σ1 0r×n−r ⊤
= U1 Σ1 V1⊤
  
Hk,ℓ = U1 U2 V1 V2
0m−r×r 0m−r×n−r

→ dimension of the system n from Σ1 .

Decompose Hk,ℓ as
1/2 1/2
Hk,ℓ = U1 Σ1 Σ1 V1⊤
| {z } | {z }
Ok Cℓ

to obtain C and B
 
C
 CA 
CA2
 
A2 B Aℓ−1 B
 
Ok = 


 and Cℓ = B AB ...
 ... 
CAk−1

18 / 21
How to compute A?
Consider the shifted Hankel matrix

   
G1 G2 ... Gℓ G2 G3 ... Gℓ+1
G2 G3 ... Gℓ+1   G3 G4 ... Gℓ+2 
↑
   
Hk,ℓ G3
= ... 
 Hk,ℓ  G4
= ... 

. . . ...   ... ... 
Gk Gk+1 ... Gk+ℓ−1 Gk+1 Gk+2 ... Gk+ℓ

And notice that

  C 
CA2 B CA3 B

CAB ...
CA2 B  CA  
↑ CA3 B CA4 B . . .
A2 B

Hk,ℓ = = 2 A B AB ...

CA3 B CA4 B ... CA  |
 
..
{z }
... ... . Cℓ
| {z }
Ok

→ A = Ok† Hk,ℓ
↑
Cℓ†

19 / 21
Kalman-Ho algorithm

1 Collect impulse reponses of the system as Markov parameters

2 Estimate k, ℓ > n
↑
3 Build Hankel matrix Hk,ℓ and shifted Hankel matrix Hk,ℓ
4 Compute the SVD of Hk,ℓ to
▶ determine n
▶ compute extended observability and controllability matrices
5 Derive A, B, C.

Take home message

When a state-space system model is not available, it can be identified.
Data contains all the model information (data = model?)
The control design methods that we have seen can be applied to the identified
model.

20 / 21
Underlying assumptions today

A real LTI finite-dimensional system exists

There is no noise

This was just a sample of system identification (nonlinear systems, model

reduction, stochastic data, missing data, priors, parametric/non-parametric,
and much more!)

21 / 21
 This work is licensed under a
Creative Commons Attribution-ShareAlike 4.0 International License

https://bsaver.io/COCO