University of Groningen

On Persistency of Excitation and Formulas for Data-Driven Control

De Persis, Claudio; Tesi, Pietro

Published in:
Proceedings of the 58th IEEE Conference on Decision and Control

DOI:
10.1109/CDC40024.2019.9029185

IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from
it. Please check the document version below.

Document Version
Publisher's PDF, also known as Version of record

Publication date:
2019

Link to publication in University of Groningen/UMCG research database

Citation for published version (APA):

De Persis, C., & Tesi, P. (2019). On Persistency of Excitation and Formulas for Data-Driven Control. In
Proceedings of the 58th IEEE Conference on Decision and Control (pp. 873-878). [9029185] (Proceedings
of the IEEE Conference on Decision and Control; Vol. 2019-December). IEEE.
https://doi.org/10.1109/CDC40024.2019.9029185

Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the
author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).

The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.
More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-
amendment.

Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.

Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the
number of authors shown on this cover page is limited to 10 maximum.

Download date: 29-03-2023

2019 IEEE 58th Conference on Decision and Control (CDC)
Palais des Congrès et des Expositions Nice Acropolis
Nice, France, December 11-13, 2019

On Persistency of Excitation and Formulas for

Data-driven Control
C. De Persis and P. Tesi

Abstract— In a paper by Willems and coworkers it was problems range from PID-like control design [9] to model
shown that persistently exciting data could be used to represent reference [10], [11], [12], predictive [13], [14], robust and
the input-output trajectory of a linear system. Inspired by optimal control design [15], [16], [17], [18], [19], [20], the
this fundamental result, we derive a parametrization of linear
feedback systems that paves the way to solve important control latter being one of the most frequently considered problems.
problems using data-dependent Linear Matrix Inequalities only. The corresponding techniques are also quite varied, ranging
The result is remarkable in that no explicit system’s matrices from dynamics programming to optimization techniques and
identification is required. The examples of control problems algebraic methods. We refer the interested reader to the sur-
we solve include the state feedback stabilization and the linear vey [21] for more references on data-driven control methods.
quadratic regulation problems. We also extend the stabilization
problem to the case of output feedback control design. Paper contribution. Despite the differences, all the afore-
I. I NTRODUCTION mentioned contributions are centred around the question of
how one can replace process models with process data. For
Learning from data is essential to every area of science. linear systems, there is actually a fundamental result which
It is the core of statistics and artificial intelligence, and answers this question, proposed by Willems et al. [22].
is becoming ever more prevalent also in the engineering Roughly, this result stipulates that the whole set of trajecto-
domain. Control engineering is one of the fileds where ries that a linear system can generate can be represented by a
learning from data is now considered as a prime issue. finite set of system trajectories provided that such trajectories
Learning from data is actually not novel in control theory. come from sufficiently excited dynamics. While this result
System identification [1] is one of the major developments has been (more or less explicitly) used in the context of data-
of this paradigm, where modeling based on first principles is driven control [14], [15], [23], [24], [25], we feel that certain
replaced by data-driven learning algorithms. Prediction error, implications of the so-called Willems et al.’s fundamental
maximum likelihood as well as subspace methods [2] are lemma have not yet been fully captured.
all data-driven techniques which can be now regarded as
In this paper, we show that Willems et al.’s fundamen-
standard for what concerns modeling. The learning-from-
tal lemma, can be used to obtain a data-based represen-
data paradigm has been widely pursued also for control
tation of the closed-loop system transition matrix, where
design purposes. A main question is how to design control
the controller is itself parametrized through the sample
systems directly from process data with no intermediate
trajectories (Theorem 1). Theorem 1 turns out to have
system identification step. Besides their theoretical value,
surprisingly straightforward, yet profound, implications for
answers to this question could have a major practical impact
control design. We discuss this fact in Section IV. The main
especially in those situations where identifying a process
point is that the parametrization provided in Theorem 1
model can be difficult and time consuming, for instance when
can be naturally related to the classic Lyapunov stability
data are affected by noise. Despite many developments in this
inequalities. This makes it possible to cast the problem of
area, data-driven control is not yet well understood even if
designing a stabilizing controller in terms of a simple Linear
we restrict the attention to linear dynamics, which contrasts
Matrix Inequality (LMI) [26] (Theorem 2). In Theorem 3, the
the achievements obtained in system identification. A major
very same arguments are used to solve a Linear Quadratic
challenge is how to incorporate data-dependent stability and
Regulation problem through a convex optimization program.
performance requirements in the control design procedure.
The main derivations are given for state feedback control.
Contributions to data-driven control can be traced back
The case of output feedback control (Theorem 4) is discussed
to the pioneering work by Ziegler and Nichols [3], direct
in Section V. Discussion and some research directions are
adaptive control [4] and neural networks [5] theories. Since
given in Section VI. Throughout the paper we report several
then, many remarkable techniques have been developed. We
numerical examples to substantiate the analysis.
mention unfalsified control [6], iterative feedback tuning [7],
and virtual reference feedback tuning [8]. This topic is now The proofs as well as further results such as the data-
attracting more and more researchers, and the considered driven stabilization of continuous-time systems, the design
of stabilizing controllers when data are corrupted by noise,
C. De Persis and P. Tesi are with ENTEG and the J.C. Willems Center for the stabilization of unstable equilibria of nonlinear systems
Systems and Control, University of Groningen, 9747 AG Groningen, The and the output feedback stabilization problem for MIMO
Netherlands. Email: {c.de.persis,p.tesi}@rug.nl. P. Tesi
is also with DINFO, University of Florence, 50139 Firenze, Italy E-mail: systems, are omitted due to space limitations and can be
[email protected]. found in [27].

978-1-7281-1398-2/19/$31.00 ©2019 IEEE 873

Authorized licensed use limited to: University of Groningen. Downloaded on May 12,2020 at 10:55:04 UTC from IEEE Xplore. Restrictions apply.
 Notation. Given a signal z : Z → Rσ , we denote by Definition 1: [22, p. 327] The signal z[0,T −1] ∈ Rσ is
z[k,k+T ] , where k ∈ Z, T ∈ N, the restriction in vectorized L if the matrix
persistently exciting of order
form of the signal z to the interval [k, k + T ] ∩ Z, namely  
  z(0) z(1) . . . z(T − L)
z(k)  z(1) z(2) . . . z(T − L + 1)
Z0,L,T −L+1 = 
 
z[k,k+T ] =  .. .. .. .. ..
.
  
.  . . . . 
z(k + T ) z(L − 1) z(L) . . . z(T − 1)
When the signal is not restricted to an interval then it has full rank σL. 

is simply denoted by its symbol, say z. To avoid nota- For a signal z to be persistently exciting of order L, it
tional burden, we use z[k,k+T ] to denote also the sequence must be sufficiently long, namely T ≥ (m + 1)L − 1. We
{z(k), . . . , z(k + T )}. For the same reason, we simply write can now recall two results by Willems et al. [22] which are
[k, k + T ] to denote the discrete interval [k, k + T ] ∩ Z. key for the developments of the paper.
We denote the Hankel matrix associated to z as
  Lemma 1: [22, Corollary 2] Consider system (1a). If the
z(i) z(i + 1) . . . z(i + N − 1) input ud,[0,T −1] is persistently exciting of order n + t, then
 z(i + 1) z(i + 2) . . . z(i + N ) condition (3) holds.


Zi,t,N = 
 
.. .. . . .
. 
 . . . .  Lemma 2: [22, Theorem 1] Consider system (1). Then the
z(i + t − 1) z(i + t) . . . z(i + t + N − 2) following holds:
where i ∈ Z and t, N ∈ N. The first subscript denotes the (i) If ud,[0,T −1] is persistently exciting of order n + t, then
time at which the first sample of the signal is taken, the any t-long input/output trajectory of system (1) can be
second one the number of samples per each column, and expressed as
the last one the number of signal samples per each row.
   
u[0,t−1] U
Sometimes – but not always – if t = 1, noting that Zi,t,N = 0,t,T −t+1 g
y[0,t−1] Y0,t,T −t+1
has only one block row, then for compactness we  simply
write Zi,N = z(i) z(i + 1) . . . z(i + N − 1) . where g ∈ RT −t+1 .
(ii) Given a T -long input/output trajectory of system (1),
II. P RELIMINARIES AND THE W ILLEMS et al.’ S any linear combination of the columns of the matrix in
FUNDAMENTAL LEMMA (2), that is  
In this section, we revisit the main result in [22] and state a U0,t,T −t+1
g,
few auxiliary results inspired by subspace identification [2], Y0,t,T −t+1
which will be useful throughout the paper. is a t-long input/output trajectory of (1). 
Consider x ∈ Rn , u ∈ Rm and y ∈ Rp and a controllable
and observable discrete-time linear system Lemma 1 shows that, for a fixed t, if T is taken sufficiently
large, then the rank condition (3) turns out to be satisfied.
x(k + 1) =Ax(k) + Bu(k) (1a) In turn, condition (3) makes it possible to represent any
y(k) =Cx(k) + Du(k). (1b) input/output trajectory of the system as a linear combination
of previously collected input/output data. Lemma 2 has been
Let (ud,[0,T −1] , yd,[0,T −1] ) be the input-output data of the originally proven in [22, Theorem 1] using the behavioral
system collected during an experiment, and let language, and it was later referred to as the fundamental
lemma [28] to describe a linear system through a finite
 
U0,t,T −t+1
(2) collection of its input/output data.
Y0,t,T −t+1
be the corresponding Hankel matrix. We also introduce III. DATA - BASED SYSTEM REPRESENTATION
the matrix X0,T −t+1 = xd (0) xd (1) . . . xd (T − t) Theorem 1 below shows how one can parametrize feed-
where xd (i) are the state samples produced by the system back interconnections just by using data. This result will be
(1a) under the input ud,[0,T −1] . For ud , yd , and xd , we use key later on for deriving control design methods that skip
the subscript d so as to emphasize that these are the sample any system identification step.
data collected from the system during some experiment. Consider a persistently exciting input sequence ud,[0,T −1]
Throughout the paper, having the rank condition of order t + n with t = 1, that is such that U0,1,T is full-
  row rank. Notice that the only requirement on T is that T ≥
U0,t,T −t+1
rank = n + tm (3) (n + 1)m + n = (m + 1)n + m, which is necessary for the
X0,T −t+1
persistence of excitation condition to hold. By Lemma 1,
satisfied plays an important role. Even though it is in general  
U0,1,T
difficult to check this condition when only input-output data rank =n+m (4)
X0,T
are accessible, it is possible to have it satisfied when the input
is persistently exciting of sufficient order. We first recall the From now on, we will directly refer to condition (4), bearing
notion of persistency of excitation. in mind that this condition requires persistently exciting

874

Authorized licensed use limited to: University of Groningen. Downloaded on May 12,2020 at 10:55:04 UTC from IEEE Xplore. Restrictions apply.
inputs of order n + 1. In this respect, we point out that problem can be actually cast in terms of a simple Linear
condition (4) can always be tested if the state of the system is Matrix Inequality (LMI).
accessible. This is the standing assumption we will make in Theorem 2: Let condition (4) be satisfied. Then, any ma-
Section IV when considering state-feedback control design trix Q satisfying
problems. Later in Section V we will provide a condition  
alternative to (4) for the case where only input/output data X0,T Q X1,T Q
0 (6)
are accessible. Q> X1,T
>
X0,T Q
is such that
A. Data-based closed-loop representation
We now exploit Lemma 2 to derive a parametrization of K = U0,1,T Q(X0,T Q)−1 (7)
system (1a) in closed-loop with a feedback law u = Kx. is a stabilizing feedback gain for system (1a). Conversely, if
Theorem 1: Let condition (4) be satisfied. Then system K is a stabilizing state-feedback gain for system (1a) then
(1a) in closed-loop with a state feedback u = Kx has the it can be written as in (7), with Q solution of (6). 

following equivalent representation x(k+1) = X1,T GK x(k) Note that in the formulation (6) the parametrization of the
where GK is a T × n matrix satisfying closed-loop matrix A + BK is given by X1,T Q(X0,T Q)−1 ,
  
K U
 that is GK = Q(X0,T Q)−1 which satisfies X0,T GK = In
= 0,1,T GK . (5) corresponding to the second identity in (5). On the other
In X0,T
hand, the constraint corresponding to the first identity in (5)
In particular u(k) = U0,1,T GK x(k).  is guaranteed by the choice K = U0,1,T Q(X0,T Q)−1 . This
is the reason why (6) is representative of closed-loop stability
B. From indirect to direct data-driven control even if no constraint like (5) appears in the formulation
A crucial observation that emerges from Theorem 1 is (6). We also stress that this result characterizes the whole
that also the controller K can be parametrized through data set of stabilizing state-feedback gains in the sense that any
via (5). Thus for design purposes one can regard GK as a stabilizing feedback gain K can be expressed as in (7) for
decision variable, and search for the matrix GK that guar- some matrix Q satisfying (6).
antees prescribed stability and performance specifications. Illustrative example. As an illustrative example, consider
In fact, as long as GK satisfies the condition X0,T GK = In the discretized version of a batch reactor [29] using a
in (5) we are ensured that X1,T GK provides an equivalent sampling time of 0.1s (up to the third digit),
representation of the closed-loop matrix A + BK with 1.178 0.001 0.511 −0.403 0.004 −0.087
 
feedback matrix K = U0,1,T GK . As shown in the next   −0.051 0.661 −0.011 0.061 0.467 0.001 
A B = .
section, this corresponds to a design procedure that avoids 0.076 0.335 0.560 0.382 0.213 −0.235 
0 0.335 0.089 0.849 0.213 −0.016
any identification step.
Before proceeding, we note that Theorem 1 already gives The system to be controlled is open-loop unstable. The
an identification-free method for checking whether a can- control design procedure is implemented in MATLAB. We
didate controller K is stabilizing or not. In fact, given K, generate the data with random initial conditions and by
any solution GK to (5) is such that X1,T GK = A + BK. applying to each input channel a random input sequence of
Hence, one can compute the eigenvalues of X1,T GK to length T = 15 by using the MATLAB command rand. To
check whether K is stabilizing or not. Notice that this solve (6) we used CVX [30], obtaining
method does not require to place K into feedback, in the 
0.7610 −1.1363 1.6945 −1.8123

same spirit of unfalsified control theory [6]. K= ,
3.5351 0.4827 3.3014 −2.6215
IV. E XAMPLES OF DIRECT DATA - DRIVEN DESIGN which stabilizes the closed-loop dynamics in agreement with
In this section, we discuss how Theorem 1 can be used Theorem 1. 

to get identification-free design algorithms. Although the

B. Linear quadratic regulation
problems considered hereafter are all of practical relevance,
we would like to regard them as application examples of Matrix (in)equalities similar to the one in (6) are recurrent
Theorem 1. In fact, we are confident that Theorem 1 can in control design, with the major difference that in (6) only
be used to approach other, more complex, design problems information collected from data appears, rather than the
such as H∞ control and quadratic stabilization [26]. system matrices. Yet, these matrix inequalities can inspire
the data-driven solution of other control problems. Important
A. Stabilizing state feedback examples are quadratic regulation problems. Consider the
By Theorem 1, the closed-loop system under state- discrete-time system
feedback u = Kx is such that A + BK = X1,T GK where x(k + 1) = Ax(k) + Bu(k) + ξ(k)
GK satisfies (5). One can therefore search for a matrix GK " 1/2 #" #
such that X1,T GK satisfies the classic Lyapunov stability Qx 0 x(k) (8)
z(k) =
condition. As the next result shows, it turns out that this 0 R1/2 u(k)

875

Authorized licensed use limited to: University of Groningen. Downloaded on May 12,2020 at 10:55:04 UTC from IEEE Xplore. Restrictions apply.
where ξ is an external input to the system, and where z Consider a SISO systems as in (1) in left difference
is a performance signal of interest; Qx  0, R  0 are operator representation [34, Section 2.3.3],
weighting matrices with (Qx , A) observable. The objective
y(k) + an y(k − 1) + . . . + a2 y(k − n + 1) + a1 y(k − n)
is to design a state-feedback law u = Kx which renders
A + BK stable and minimizes the H2 norm of the transfer = bn u(k − 1) + . . . + b2 u(k − n + 1) + b1 u(k − n)
function g : ξ → z [31, Section 4], (12)
This representation allows us to reduce the output measure-
Z 2π  21
ment case to the state measurement case with minor effort.

1 
jθ >

jθ


kgk2 := trace g e g e dθ . (9) Define the state vector
2π 0
This problem can be equivalently formulated as a convex χ(k) := col(y(k − n), y(k − n + 1), . . . , y(k − 1),
(13)
program [32], [33]. In fact, as a natural counterpart of the u(k − n), u(k − n + 1), . . . , u(k − 1)),
continuous-time formulation in [32], the optimal controller
from (12) we obtain the state space system (14) on the next
K can be found by solving
page. Note that we turned our attention to a system of order
minK,W,X trace (Qx W ) + trace (X) 2n, which is not minimal.

>
Consider now the matrix in (4) written for the system
 (A + BK)W (A + BK) − W + In  0 χ(k + 1) = Aχ(k) + Bu(k) in (14), with T ≥ 2n + 1. If

(10)

s.t. W  In this matrix is full-row rank, then the analysis in the previous

 X − R1/2 KW K > R1/2  0
 sections can be repeated also for system (14). We observe
that written for system (14), the matrix in question is
This can be cast as a convex optimization problem by    
means of suitable change of variables [32]. Based on this U0,1,T u (0) ud (1) . . . ud (T − 1)
= d , (15)
formulation, it is straightforward to derive a data-dependent X̂0,T χd (0) χd (1) . . . χd (T − 1)
formulation of this optimization problem. where χd (i + 1) = Aχd (i) + Bud (i) for i ≥ 0 and where
Theorem 3: Let condition (4) be satisfied. The optimal H2 χd (0) is the initial condition in the experiment,
state-feedback controller K for system (8) can be computed
as K = U0,1,T Q(X0,T Q)−1 where Q optimizes χd (0) = col(yd (−n), yd (−n + 1), . . . , yd (−1),
ud (−n), ud (−n + 1), . . . , ud (−1)).
minQ,X trace (Qx X0,T Q) + trace (X)
" # We have the following result.
1/2
 X R U0,1,T Q


 0 Lemma 3: The identity
 Q> U0,1,T
>
R1/2 X0 Q

(11)
  
s.t.
  U0,1,T
U0,1,T
=  Y−n,n,T  (16)
" #
X0,T Q − In X1,T Q

X̂0,T



 0 U−n,n,T
Q> X >

 X Q
1,T 0,T
holds. Moreover, if ud,[0,T −1] is persistently exciting of order

2n + 1 then
Illustrative example. We consider the batch reactor system
 
U0,1,T
of the previous subsection. As before, we generate the data rank = 2n + 1. (17)
X̂0,T
with random initial conditions and by applying to each input

channel a random input sequence of length T = 15 by using
the MATLAB command rand. We let Qx = In and R = A. Direct data-driven design of output feedback controllers
Im . To solve (11) we used CVX, obtaining
  Consider the left difference operator representation (12),
0.0639 −0.7069 −0.1572 −0.6710 its realization (14) and the input/state pair (u, χ). We intro-
K=
2.1481 0.0875 1.4899 −0.9805 duce a controller of the form
This controller coincides with the controller K obtained y c (k) + cn y c (k − 1) + . . . + c1 y c (k − n)
(18)
with the MATLAB command dare which solves the classic = dn uc (k − 1) + . . . + d1 uc (k − n)
DARE equation. In particular, kK − Kk ≈ 10−7 . 

Its state space representation, with state χc defined similar

V. I NPUT- OUTPUT DATA : THE CASE OF SISO SYSTEMS to (13), input uc and output y c , can be given analogously
In Section IV-A, the measured data are the inputs and the to (14). We omit it due to lack of space. In the closed-loop
full state, and the starting point is to express the trajectories system, we enforce the following interconnection conditions
of the system and the control gain in terms of the Hankel relating the process and the controller uc (k) = y(k), y c (k) =
matrix of input-state data. This section considers the case u(k), k ≥ 0. Note in particular the identity, for k ≥ n,
where only input-output data of the system are accessible.
The main derivations are given for single-input single-output
   c   
y[k−n,k−1] u 0 In
χ(k) = = c[k−n,k−1] = n×n χc (k).
(SISO) systems. u[k−n,k−1] y[k−n,k−1] In 0n×n

876

Authorized licensed use limited to: University of Groningen. Downloaded on May 12,2020 at 10:55:04 UTC from IEEE Xplore. Restrictions apply.
    
0 1 0 ...
0 ... 0 0 0 0 0
 0 0 1 ...
0 . . . 00  0 00
.. .. .. .. . . .. 
.. .. ..
 .. 
   
 ..

 . . . .. . ..  . .
.
 
 0 0 0 ... 1 0 0 0 ... 0   0
  
−a1 −a2 −a3 . . . −an b1 b2 b3 . . . bn 
 χ(k) + 0 u(k)
 
χ(k + 1) = 
 0
 0 0 ... 0 0 1 0 ... 0   0
 
 0 0 0 ... 0 0 0 1 ... 0  0
(14)

  
 . . . . . . . . .
 .. .. .. . . . .. .. .. .. . . . ..   .. 

   
 0 0 0 ... 0 0 0 0 ... 1  0
0 0 0 ... 0 0 0 0 ... 0 1
| {z } |{z}
 A  B
y(k) = −a1 −a2 −a3 . . . −an b1 b2 b3 . . . bn χ(k)
| {z }
C

Hence, for k ≥ n, there is no loss of generality in considering in (20) is stabilizing for system (12). As a final point, we
the system note that given a solution K as in (23) the resulting entries

0 1 0 ... 0 0 0 0 ... 0
 ordered as in (21) lead to the following state-space realization
 0 0 1 ... 0 0 0 0 ... 0  of order n for the controller
 .. .. .. .. .. .. .. . 
 
.. ..
. .. 
   
 .
 . . . . . . .  −cn 1 0 . . . 0 dn
 0 0 0 ... 1 0 0 0 ... 0 
  −cn−1 0 1 . . . 0 dn−1 
−a1 −a2 −a3 . . . −an b1 b2 b3 . . . bn     
χ(k + 1) =   χ(k).  .. . . . .
.. .. . . ..  ξ(k) + . 
 0 0 0 ... 0 0 1 0 ... 0  ξ(k + 1) =  .  ..  y(k)

 
 0 0 0 ... 0 0 0 1 ... 0  
 0
   (24)

 . .. .. .. .. .. .. .. .. . 
 0 0 . . . 1  d2 
 . . . .. 
 .
 0
. . . . . .  −c1 0 0 . . . 0 d1
0 0 ... 0 0 0 0 ... 1   
d1 d2 d3 . . . dn −c1 −c2 −c3 . . . −cn u(k) = 1 0 0 . . . 0 0 ξ(k).
(19)
as the closed-loop system. In the following statement we say
Illustrative example. Consider a system [35] made up by
that (18) stabilizes system (12), meaning that the closed-loop
two carts. The two carts are mechanically coupled by a spring
system (19) is asymptotically stable.
with unknown stiffness γ ∈ [0.25, 1.5]. The aim is to control
Theorem 4: Let condition (17) be satisfied. Then: the position of one cart by applying a force to the other cart.
(i) The closed-loop system (19) has the equivalent repre- The system state-space description is given by
sentation χ(k + 1) = X̂1,T GK χ(k), where GK is a    
T × 2n matrix such that 0 1 0 0 0
        −γ 0 γ 0   1 
K U0,1,T A B 
 0 0
  
= GK , (20) = 0 1   0  . (25)
I2n X̂0,T C D 

 γ 0 −γ 0  0 
and 0 0 1 0 0
 
K := d1 . . . dn − c1 . . . −cn (21)
Assume that γ = 1 (unknown). The system is controllable
is the vector of coefficients of the controller (18). and observable. All the open-loop eigenvalues are on the
(ii) Any matrix Q satisfying imaginary axis. The input-output discretized version using a
" # sampling time of 1s is as in (12) with coefficients (up to the
X̂0,T Q X̂1,T Q
 0, (22) fourth digit)
Q> X̂1,T
>
X̂0,T Q
   
a1 a2 a3 a4 = 1 −2.311 2.623 −2.311
is such that controller (18) with coefficients given by    
b1 b2 b3 b4 = 0.039 0.383 0.383 0.039 .
K = U0,1,T Q(X̂0,T Q)−1 (23)
We design a controller following Theorem 4. We generate
stabilizes system (12). Conversely, any controller (18)
the data with random initial conditions and by applying a
that stabilizes system (12) must have coefficients K
random input sequence of length T = 9. To solve (22) we
given by (23), with Q a solution of (22). 
used CVX, obtaining from (23)
As for the case of full-state measurements, the result above 
unveils the identity A + BK = X̂1,T GK , which gives a 1.1837 −1.5214 1.3408 −1.4770
K= 
method to check whether a controller (18) with coefficients 0.0005 −0.5035 −0.9589 −0.9620 ,

877

Authorized licensed use limited to: University of Groningen. Downloaded on May 12,2020 at 10:55:04 UTC from IEEE Xplore. Restrictions apply.
which stabilizes the closed-loop dynamics in agreement with [12] L. Campestrini, D. Eckhard, A. Bazanella, and M. Gevers, “Data-
Theorem 4. In particular, a minimal state-space representa- driven model reference control design by prediction error identifica-
tion,” Journal of the Franklin Institute, vol. 354, no. 6, pp. 2828–2647,
tion (Ac , Bc , Dc , Dc ) of this controller is given by (see (24)) 2017.
   [13] J. Salvador, D. Muñoz de la Peña, T. Alamo, and A. Bemporad, “Data-
−0.9620 1 0 0 −1.4770 based predictive control via direct weight optimization,” in 6th IFAC
    −0.9589 0 1 0  1.3408  Conference on Nonlinear Model Predictive Control, 2018, pp. 356–
Ac B c    361.
 −0.5035
= 0 0 1   −1.5214
 .
Cc Dc  [14] J. Coulson, J. Lygeros, and F. Dörfler, “Data-enabled predictive con-
 0.0005 0 0 0 1.1837 trol: In the shallows of the DeePC,” https://arxiv.org/abs/1811.05890,
 
1 0 0 0 0 2018.

[15] I. Markovsky and P. Rapisarda, “On the linear quadratic data-driven
control,” in European Control Conference, 2007.
VI. C ONCLUSIONS
[16] K. Vamvoudakis and F. Lewis, “Online actor-critic algorithm to
We have shown the existence of a parametrization of solve the continuous-time infinite horizon optimal control problem,”
feedback control systems that allows us to reduce the sta- Automatica, vol. 46, no. 5, pp. 878–888, 2010.
bilization problem to an equivalent data-dependent linear [17] B. Pang, T. Bian, and Z.-P. Jiang, “Data-driven finite-horizon optimal
control for linear time-varying discrete-time systems,” in 57th IEEE
matrix inequality. Since LMIs are ubiquitous in systems Conference on Decision and Control (CDC). IEEE, 2018.
and control we expect that our approach will lead to data- [18] S. Mukherjee, H. Bai, and A. Chakrabortty, “On model-free rein-
driven solutions to many other control problems, such as H∞ forcement learning of reduced-order optimal control for singularly
perturbed systems,” in 57th IEEE Conference on Decision and Control
control and quadratic stabilization [26]. As first examples, we (CDC). IEEE, 2018.
have considered the LQR problem and the case of output [19] R. Gonçalves da Silva, A. Bazanella, C. Lorenzini, and L. Campestrini,
feedback controllers. The important extension to the case “Data-driven LQR control design,” IEEE Control Systems Letters,
when data are corrupted by noise is tackled in [27]. vol. 3, no. 1, pp. 180–185, 2019.
A great leap forward will come from extending the meth- [20] G. Baggio, V. Katewa, and F. Pasqualetti, “Data-driven minimum-
energy controls for linear systems,” IEEE Control Systems Letters,
ods of this paper to systems where identification is hard, vol. 3, no. 3, pp. 589–594, 2019.
such as nonlinear systems. Our results in [27, Section V.B] [21] Z.-S. Hou and Z. Wang, “From model-based control to data-driven
show that this approach is concretely promising for nonlinear control: Survey, classification and perspective,” Information Sciences,
vol. 235, pp. 3–35, 2013.
systems, but we have only scratched the surface of this
[22] J. C. Willems, P. Rapisarda, I. Markovsky, and B. L. De Moor, “A
research area. Recent results have reignited the interest of the note on persistency of excitation,” Systems & Control Letters, vol. 54,
community on system identification for nonlinear systems, no. 4, pp. 325–329, 2005.
interestingly pointing out the importance of the concept of [23] I. Markovsky and P. Rapisarda, “Data-driven simulation and control,”
persistently exciting signals [36]. We are confident that our International Journal of Control, vol. 81, no. 12, pp. 1946–1959, 2008.
[24] U. Park and M. Ikeda, “Stability analysis and control design of
approach will also play a fundamental role in developing a lti discrete-time systems by the direct use of time series data,”
systematic methodology for the data-driven design of control Automatica, vol. 45, pp. 1265–1271, 2009.
laws for nonlinear systems. [25] T. Maupong and P. Rapisarda, “Data-driven control: A behavioral
approach,” Systems & Control Letters, vol. 101, pp. 37–43, 2009.
R EFERENCES [26] C. Scherer and S. Weiland, “Linear matrix inequalities in control,”
Notes for a course of the Dutch Institute of Systems and Control,
[1] L. Ljung, System Identification: Theory for the User. Prentice Hall,
2004.
1987.
[2] M. Verhaegen and V. Verdult, Filtering and system identification: a [27] C. De Persis and P. Tesi, “Formulas for data-driven control: Stabiliza-
least squares approach. Cambridge university press, 2007. tion, optimality and robustness,” under review for journal publication,
[3] J. Ziegler and N. Nichols, “Optimum settings for automatic con- https://arxiv.org/abs/1903.06842, 2019.
trollers,” Transactions of the ASME, vol. 64, pp. 759–768, 1942. [28] I. Markovsky, J. Willems, and B. De Moor, “State representations from
[4] K. Åström and B. Wittenmark, Adaptive Control. Addison-Wesley, finite time series,” in 44th IEEE Conference on Decision and Control,
1989. 2005.
[5] P. Werbos, “Neural networks for control and system identification,” in [29] G. Walsh and H. Ye, “Scheduling of networked control systems,” IEEE
28th IEEE Conference on Decision and Control (CDC). IEEE, 1989. Control Systems Magazine, vol. 21, no. 1, pp. 57–65, 2001.
[6] M. Safonov and T. Tsao, “The unfalsified control concept: a direct [30] M. Grant and S. Boyd, “CVX: Matlab software for disciplined convex
path from experiment to controller,” in Feedback control, nonlinear programming, version 2.1,” http://cvxr.com/cvx, Mar. 2014.
systems and complexity, ser. Lecture Notes in Control and Information
Sciences, B. Francis and A. Tannenbaum, Eds. Berlin: Springer- [31] T. Chen and B. Francis, Optimal Sampled-Data Control Systems.
Verlag, 1995, pp. 196–214. London: Springer, 1995.
[7] H. Hjalmarsson and M. Gevers, “Iterative feedback tuning: Theory [32] E. Feron, V. Balakrishnan, S. Boyd, and L. El Ghaoui, “Numerical
and applications,” IEEE Control Systems Magazine, vol. 18, no. 4, pp. methods for H2 related problems,” in 1992 American Control Con-
26–41, 1998. ference. IEEE, 1992, pp. 2921–2922.
[8] M. Campi, A. Lecchini, and S. Savaresi, “Virtual reference feedback [33] V. Balakrishnan and L. Vandenberghe, “Semidefinite programming
tuning: a direct method for the design of feedback controllers,” duality and linear time-invariant systems,” IEEE Transactions on
Automatica, vol. 38, pp. 337–1346, 2002. Automatic Control, vol. 48, no. 1, pp. 30–41, 2003.
[9] M. Fliess and J. C, “Model-free control,” International Journal of [34] G. C. Goodwin and K. S. Sin, Adaptive filtering prediction and control.
Control, vol. 86, no. 12, pp. 2228–2252, 2013. Courier Corporation, 2014.
[11] K. van Heusden, A. Karimi, and D. Bonvin, “Data-driven model ref-
[35] P. Gahinet, A. Nemirovski, and A. J. Laub, LMI control toolbox user’s
erence control with asymptotically guaranteed stability,” International
guide. MathWorks, 1995, pp. 3–22.
Journal of Adaptive Control and Signal Processing, vol. 25, pp. 331–
351, 2011. [36] A. Padoan, G. Scarciotti, and A. Astolfi, “A geometric characterization
[10] S. Formentin, A. Karimi, and S. Savaresi, “Optimal input design for of the persistence of excitation condition for the solutions of au-
direct data-driven tuning of model-reference controllers,” Automatica, tonomous systems,” IEEE Transactions on Automatic Control, vol. 62,
vol. 49, pp. 1874–1882, 2013. no. 11, pp. 5666–5677, 2017.

878

Authorized licensed use limited to: University of Groningen. Downloaded on May 12,2020 at 10:55:04 UTC from IEEE Xplore. Restrictions apply.