International Journal of Control

ISSN: 0020-7179 (Print) 1366-5820 (Online) Journal homepage: www.tandfonline.com/journals/tcon20

Data-driven simulation and control

Ivan Markovsky & Paolo Rapisarda

To cite this article: Ivan Markovsky & Paolo Rapisarda (2008) Data-driven simulation and
control, International Journal of Control, 81:12, 1946-1959, DOI: 10.1080/00207170801942170
To link to this article: https://doi.org/10.1080/00207170801942170

Published online: 08 Oct 2008.

Submit your article to this journal

Article views: 2661

View related articles

Citing articles: 87 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=tcon20
International Journal of Control
Vol. 81, No. 12, December 2008, 1946–1959

Data-driven simulation and control

Ivan Markovsky* and Paolo Rapisarda
School of Electronics and Computer Science, University of Southampton, Southampton, UK
(Received 21 June 2007; final version received 24 January 2008)

Classical linear time-invariant system simulation methods are based on a transfer function, impulse response, or
input/state/output representation. We present a method for computing the response of a system to a given input
and initial conditions directly from a trajectory of the system, without explicitly identifying the system from the
data. Similar to the classical approach for simulation, the classical approach for control is model-based: first a
model representation is derived from given data of the plant and then a control law is synthesised using the model
and the control specifications. We present an approach for computing a linear quadratic tracking control signal
that circumvents the identification step. The results are derived assuming exact data and the simulated response
or control input is constructed off-line.
Keywords: simulation; data-driven control; output matching; linear quadratic tracking; system identification

1. Introduction that avoid the explicit derivation of a model

The usual starting point of systems and control representation.
problems is a given representation of the plant. As a Data-driven algorithms for systems and control
consequence, the developed solution methods and problems are presently less developed than their
algorithms are based on input/state/output, transfer model-based counterparts. Only a few control
function, matrix fraction, impulse response, etc. problems have data-driven solutions. Our purpose
representations. We call this research paradigm in this paper is to outline a generic approach to
‘model-based’, and refer to problem formulations data-driven simulation and show its fundamental role
based on the input/state/output representation for deriving data-driven control algorithms. We
(perhaps the most often used representation) as impose the strong assumption that the data is
classical problem formulations. generated by a linear time-invariant system of an
In this paper, we consider alternative problem a priori bounded order and all the data is available
formulations, which we call ‘data-driven’, that start off-line. Thus, the algorithms presented in this paper
from a given observed trajectory of the to-be-simulated are non-recursive and apply to exact data. In a future
or to-be-controlled system (the plant). From the work, we will address the approximation issue, the
applications point of view, data-driven formulations recursive and real-time operation of the algorithms,
are closer than their classical counterparts to real-life and data-driven feedback control.
problems: indeed in practise one rarely has a given
input/state/output representation but often may
observe a trajectory of the plant. From the theoretical 1.1 Overview of the literature, outline of the paper,
point of view, data-driven formulations allow for and summary of contributions
a new class of solution methods and algorithms Data-driven simulation of free response and impulse
that are not based on any particular representation response is used in Markovsky, Willems, Rapisarda,
of the plant. and De Moor (2005a) for deriving balanced input/
Of course, a data-driven problem could be solved state/output representations from data. In its full
by a combination of system identification and a generality the data-driven simulation problem is first
model-based method. System identification derives formulated and solved in Markovsky, Willems, and De
from the given data a model representation of the Moor (2005b). In Section 4, we review the results of
plant, and a model-based simulation or control Markovsky et al. (2005b), because they are funda-
design method could then be applied. We are looking mental for the solution of the data-driven control
for alternatives to this obvious two-stage solution problems, considered in Sections 5 and 6.

*Corresponding author. Email: [email protected]

ISSN 0020–7179 print/ISSN 1366–5820 online

 2008 Taylor & Francis
DOI: 10.1080/00207170801942170
http://www.informaworld.com
 International Journal of Control 1947

Perhaps the first data-driven control method is the then synthesises the controller by solving the
Ziegler-Nichols procedure for tuning PID controllers. corresponding Riccati equation. The second solution
That method is based, however, on the plant step is to compute an impulse response representation of
response, which assumes zero initial conditions and the plant and then find the optimal trajectory by
step input. In addition, the method is graphical and solving a weighted least squares problem. The third
does not generalise to other control problems. solution computes the optimal trajectory directly from
Procedures for deriving multivariable linear quadratic the given data by projecting the reference trajectory on
Gaussian controller, using the plant impulse response, the set B0 of all zero initial conditions trajectories
are proposed in Furuta and Wongsaisuwan (1995) and of the plant. Under certain conditions, which will be
Shi and Skelton (2000). specified, a basis for B0 can be computed from the
Data-driven synthesis methods using an arbitrary given trajectory wd of the plant, which makes the
response are developed under different names: unfalsi- procedure implementable. We illustrate the equiva-
fied control (Safonov 1996; Safonov and Tsao 1997), lence of the three approaches by simulation examples.
iterative feedback tuning (Hjalmarsson, Gevers, The third approach is similar to that used in Favoreel
Gunnarsson and Lequin 1998; Hildebrand, Lecchini, (1999); Ikeda et al. (2001) and Fujisaki et al. (2004).
Solari and Gevers 2004), virtual reference feedback However, the approach presented here has the follow-
tuning (Campi, Lecchini and Savaresi 2002), and model ing novel features compared to the cited sources:
free control (Favoreel 1999; Woodley 2001; Ikeda,
Fujisaki and Hayashi 2001; Fujisaki, Duan and Ikeda 1. sufficient conditions are given under which the
2004; Park and Ikeda 2006). The model free methods as data-driven simulation and control problems
well as the methods proposed in this paper can be are solvable;
classified as subspace-type methods, i.e., similarly to the 2. a new algorithm is derived for data-driven
subspace system identification methods (Van Overschee control (see Algorithm 8); and
and De Moor 1996; Verhaegen and Dewilde 1992), they 3. the solution of the data-driven control problem
are based on linear algebra operations, such as is related to the basic idea of computing special
projections and solution of linear systems of equations. responses of the unknown plant (i.e., data-
The model less methods of Favoreel (1999), Ikeda et al. driven simulation).
(2001) and Fujisaki et al. (2004) and the data-driven The control criterion, considered in Section 6 of the
control method, presented in Section 6 of this paper, paper, is over a finite horizon, so that the initial
minimise a finite horizon 2-norm cost function. The conditions play an important role. In iterative feedback
solution to this problem is given in closed form by a tuning (Hjalmarsson et al. 1998; Hildebrand et al. 2004)
projection and consequently the solution methods are and virtual reference feedback tuning (Campi et al.
related. The proposed algorithms for solving the 2002) approaches, instead, the objective is to achieve a
problem, however, are different. desired operating regime of the plant. In this case, an
In Section 5, we consider a control problem, called infinite horizon cost is used and therefore the initial
output matching, which is in the following sense the conditions are not relevant. This difference is also
opposite to simulation: the output matching controller reflected in the fact that in the approach of this paper
aims to determine an input that achieves exactly a one designs a control input while in iterative feedback
desired output, while a simulation algorithm aims to tuning and virtual reference feedback tuning the aim is
determine an output that corresponds to a given input. to tune the parameters of a feedback controller.
(In both problems the initial conditions are assumed to
be given.) We present an algorithm for output match-
ing, which is a trivial modification of the data-driven 2. Preliminaries and notation
simulation algorithm. As will be shown, a shortcoming We use the behavioural language. A discrete-time
of the output matching problem formulation is that it dynamical system B with w external variables (inputs
does not impose restrictions on the control signal and and outputs) is a subset of the signal space (Rw)N. The set
therefore the input can grow unbounded. of natural numbers N is the time-axis, so that a trajectory
In Section 6, we consider a data-driven finite w of B is a vector time series w ¼ (w(1), w(2), . . . ), where
horizon linear quadratic tracking problem, which w(t) 2 Rw, for all t 2 N. The notation Bt is used for the
implicitly constrains the input signal and thus avoids restriction of the behaviour on the interval [1, t], i.e.,
the main shortcoming of the output matching control. Bt :¼ fwp 2 ðRw Þt j there is wf , such that wp ^ wf 2 Bg,
We present three solutions to the linear quadratic
tracking problem. The first one is the classical ð1Þ
model-based solution that first computes an where wp ^ wf denotes the concatenation of the
input/state/output representation of the plant and trajectories wp and wf. Thus, w 2 Bt is a finite trajectory
1948 I. Markovsky and P. Rapisarda

(w(1), . . . w(t)) of the system B.  denotes the back- persistently exciting of order L if the Hankel matrix
wards shift operator w(t) :¼ w(t þ 1). HL(u) is of full row rank.
The number of inputs m and the number of The block lower-triangular Toeplitz matrix with t
outputs p of a system B 2 (Rm þ p)N are invariant of block rows, composed of the signal H ¼ (H(0),
the representation. Modulo a permutation of the H(1), . . . , H(t  1)) is denoted by
variables, any trajectory w 2 B has an input/output 2 3
Hð0Þ 0 0 ... 0
partition 6
6 .. .. 7
7
  6 Hð1Þ Hð0Þ 0 . . 7
u 6 7
w ¼ colðu, yÞ :¼ , 6 7
y Tt ðHÞ :¼ 6 Hð2Þ Hð1Þ Hð0Þ 0 0 7:
6 7
6 .. .. .. .. 7
where u is an input, i.e., it is free, and y is an output, 6 . . . . 0 7
4 5
i.e., it is determined by the input, the system, and the
Hðt  1Þ  Hð2Þ Hð1Þ Hð0Þ
initial condition.
Denote by Lw the class of linear, time-invariant, ð4Þ
and finite dimensional systems with w variables. There Let H be the impulse response of B, i.e., with B ¼ B(A,
are a number of equivalent representations of a system B, C, D) of order n, we have
B 2 Lw . In this paper, we use the input/state/output
representation Hð0Þ ¼ D, HðÞ ¼ CA1 B, for  ¼ 1, 2, . . . , t  1:
ð5Þ
BðA, B, C, DÞ :¼ fcolðu, yÞ 2 ðRmþp ÞN j 9 x 2 ðRn ÞN ,
Using the notation (2) for the observability matrix and
such that x ¼ Ax þ Bu
the notation (4) for the lower triangular Toeplitz
and y ¼ Cx þ Dug, matrix, we can express the condition that w is a
trajectory of B as a linear system of equations
parametrised by the matrices A 2 Rnn, B 2 Rnm,
C 2 Rpn, and D 2 Rpm. The state dimension n is w ¼ colðu, yÞ 2 Bt () 9 xini 2 Rn , such that
called the order of the representation. An input/state/ y ¼ Ot ðA, CÞxini þ Tt ðHÞu:
output representation B(A, B, C, D) is minimal if its
ð6Þ
order is as small as possible. This smallest possible
order n(B) is invariant and is called the order of the We use the same letter y for the signal (y(1), . . . , y(t)) as
system. Another invariant that is used in the paper well as for the vector col(y(1), . . . , y(t)), i.e., we identify
is the lag l(B) of B. It is defined as the observabi- the spaces (Rp)t and Rpt. Corresponding to a
lity index of an input/state/output representation given input u 2 (Rm)t, the system B 2 Lmþp has an
B(A, B, C, D) of B, i.e., the smallest integer ‘, for n(B)-dimensional space of outputs y, such that
which the observability matrix col(u, y) 2 Bt. This space is parametrised by the initial
condition.
O‘ ðA, CÞ :¼ colðC, CA, . . . , CA‘1 Þ ð2Þ The Moore-Penrose pseudoinverse of the matrix A
is denoted by Aþ. col span(A) denotes the span of the
has rank n(B).
columns of A. Im is the m  m identity matrix and 0mn
The Hankel matrix with t1 block rows, composed
is the m  n zero matrix.
of the signal w 2 (Rw)T is denoted by
2 3
wð1Þ wð2Þ  wðt2 Þ
6 7 3. Fundamental lemmas
6 wð2Þ wð3Þ  wðt2 þ 1Þ 7
6 7 3.1 Specification of initial condition
6 7
6
Ht1 , t2 ðwÞ :¼ 6 wð3Þ wð4Þ    wðt2 þ 2Þ 7 In a representation free setting, we specify initial
7
6 . .. .. 7 condition for a trajectory wr 2 BTr by considering the
6 .. . . 7
4 5 extended trajectory wini ^ wr, where wini 2 BTini is a
wðt1 Þ wðt1 þ 1Þ  wðt1 þ t2  1Þ suitably chosen initial trajectory. Assuming that the
ð3Þ length Tini of wini is greater than or equal to the lag l(B)
of the system, the state x(Tini þ 1) of a minimal input/
If the index t2 is skipped, the matrix Ht1(w) is assumed state/output representation of B is uniquely deter-
to have the maximal possible number of columns mined at time Tini þ 1. This state serves as an initial
t2 ¼ T  t1 þ 1. The signal u ¼ (u(1), . . . , u(T)) is called condition for the trajectory wr.
 International Journal of Control 1949

Lemma 1 (initial condition): Let B(A, B, C, D) be a distinguish the particular observed trajectory wd
minimal input/state/output representation and let H be from a general one. Due to the linearity and
the impulse response of B 2 Lw . Then for all time-invariance of B, the T  t þ 1 columns of the
wini 2 (Rw)Tini, with Tini  l(B), Hankel matrix Ht(wd), where t  T, are t samples
long trajectories of B. Therefore, any linear combi-
wini ^ colður , yr Þ 2 BTini þTr ) there is unique
nation of the columns of Ht(wd) is also a t-samples
nðBÞ
xini 2 R , such that yr ¼ OTr ðA, CÞxini þ TTr ðHÞur : long trajectory of B, i.e.,
ð7Þ
Ht ðwd Þg 2 Bt , for all g 2 RTtþ1 :
Proof: Given an initial trajectory wini, we need to
The following lemma shows that under additional
prove the existence of a unique initial vector
assumptions on wd and B, for every trajectory w 2 Bt,
xini 2 Rn(B), such that (7) holds, for all ur 2 (Rm)Tr. We
there is a vector g, such that w ¼ Ht (wd)g.
do this constructively. Let col(uini, yini) be an input/
output partitioning of wini. Note that wini is not free: Lemma 2 (Willems et al. 2005): Let
. w ¼ col(ud, yd) be a T samples long trajectory of
wini ^ colður , yr Þ 2 BTini þTr ¼) wini 2 BTini :
a linear time-invariant system B, i.e., wd 2 BT;
Since wini is a trajectory of BTini, it follows from (6), . the system B be controllable; and
that there is an initial state x(1), such that . the input ud be persistently exciting of order
t þ n(B).
yini ¼ OTini ðC, AÞxð1Þ þ TTini ðHÞuini : ð8Þ
Then any t samples long trajectory w ¼ col(u, y) of B can
Moreover, the assumptions that B(A, B, C, D) is a be written as a linear combination of the columns
minimal representation and Tini  l(B) imply that the of Ht(wd) and any linear combination Ht(wd)g, where
extended observability matrix OTini ðC, AÞ is full column g 2 RT  t þ 1, is a trajectory of B, i.e.,
rank. Therefore, the system of equations (8) has a  
unique solution x(1). The initial condition xini is equal col span Ht ðwd Þ ¼ Bt :
to the state x(Tini þ 1), i.e., Under the assumptions of Lemma 2, called the
xini ¼ xðTini þ 1Þ ¼ A xð1ÞTini fundamental lemma, computing a trajectory w of Bt
  with certain specified properties, e.g., w corresponds
þ ATini 1 B ATini 2 B    AB B uini : ð9Þ to a given input u and initial trajectory wini,
The uniqueness of xini follows from the uniqueness amounts to choosing a vector g that imposes
of x(1). those properties through w ¼ Hl (wd)g. For example,
choosing g, such that wini ^ col(u, y) 2 BTini þ t
Lemma 1 shows that assuming wini 2 BTini and requires solving a linear system of equations for g
Tini  l(B), a trajectory wr 2 BTr, such that with right-hand-side wini and u.
wini ^ wr 2 BTini þ Tr, has a uniquely specified initial
condition and is therefore unique, given its input
component ur. Moreover, the proof gives an explicit
expression (8–9) for an initial state xini in terms of the 4. Data-driven simulation
initial trajectory wini, the input/state/output represen- In this section, we consider the simulation problem:
tation parameters A, B, C, D, and the impulse response given a system B with an input/output partition
H. This expression is used in Section 6.1 for the w ¼ col(u, y) of the variables, an input signal ur, and
solution of the linear quadratic tracking problem by an initial condition; find the response yr of Bt to ur and
input/state/output representation. the given initial condition.

3.2 Construction of trajectories 4.1 Problem formulation

Next, we consider an important result for the Classically, the system B and implicitly the input/
solution of the data-driven simulation and control output partition are given by an input/state/output,
problems, related to the construction of new transfer function, or impulse response representation.
trajectories of the system from a known one. The input/state/output representation is used most
Assume that wd 2 (Rw)T is a trajectory of B 2 Lw . often, because it allows a convenient specification of
The subscript d stands for ‘data’ and aims to the initial condition by an initial state vector xini and
1950 I. Markovsky and P. Rapisarda

because of the good numerical linear algebra Assuming that B is identifiable from wd, standard
methods developed in the state space setting. identification algorithms can be used to find an explicit
Consequently, in what follows, we refer to the representation of B. This suggests the following
simulation problem with a given input/state/output solution of the data-driven simulation problem that
representation of the system as the ‘classical simula- reduces it to the classical simulation problem:
tion problem’.
1. identification wd 7 ! (A, B, C, D);
Problem 1 (classical simulation): Given 2. observer design (wini, (A, B, C, D)) 7 ! xini;
3. classical simulation (ur, xini, (A, B, C, D)) 7 ! y.
. an input/state/output representation B(A, B,
C, D) of a linear time-invariant system B, The question occurs:
. an input signal ur ¼ (ur(1), . . . , ur(Tr)) 2 (Rm)Tr, Can we find a response y such that (10) holds without
and deriving an explicit representation of B?
. an initial condition xini 2 Rn,
As shown in the following section, the answer is
find the response yr of B(A, B, C, D) to the input ur positive. Moreover, the main idea for doing data-
and the initial condition xini. driven simulation can be used as well for data-driven
The data-driven simulation problem is defined control. This is explained in Sections 5 and 6.
similarly to the classical simulation problem with
the differences that the system B and the initial
conditions are specified implicitly by given 4.2 Data-driven simulation algorithm
trajectories. Consider given wd, ur, and wini. Under the assumptions
of the fundamental lemma,
Problem 2 (data-driven simulation): Given
. a trajectory wd ¼ (wd(1), . . . , wd(T)) 2 (Rw)T of wini ^ colður , yÞ 2 BTini þTr () 9 g,
 
a linear time-invariant system BT, such that HTini þTr ðwd Þg ¼ col wini , colður , yÞ :
. an input signal ur ¼ (ur(1), . . . , ur (Tr)) 2 (Rm)Tr,
The equation
and
. an initial trajectory wini ¼ (wini(1), . . . , wini  
HTini þTr ðwd Þg ¼ col wini , colður , yÞ ð11Þ
(Tini)) 2 (Rw)Tini, where wini 2 BTini,
is a linear system for the unknowns g and y. The
find a response y of B to the input ur, such that
equations with right-hand-side y can be used to define
wini ^ colður , yÞ 2 BTini þTr : ð10Þ y for given g. The remaining equations impose
restrictions on g, determined by wini and ur. This
Note 1 (uniqueness of the response y): Equation 10 suggests the following generic data-driven simulation
does not necessarily define a unique y, i.e., in algorithm:
Problem 2 we do not impose that the response y is
unique. In most cases, however, uniqueness is 1. compute any solution g of the equations in (11)
desirable. As proven in Lemma 1, the additional with right-hand-side wini and ur;
assumption Tini  l(B) imposes uniqueness. 2. define y from the remaining equations in (11)
by substituting the computed g on step 1.
It is assumed, as part of the problem formula-
tion, that wd and wini are trajectories of a linear In order to make the algorithm more concrete,
time-invariant system B. Since wd and wini are of define
finite length and the order of B is not a priori
U :¼ HTini þTr ðud Þ and Y :¼ HTini þTr ðyd Þ
restricted, the assumptions wd 2 BT and wini 2 BTini
can be made without loss of generality. We do need and the partitionings
restrictive assumptions, however, in order to guar-
antee that wd uniquely specifies B (identifiability). " # " #
Up mTini Yp pTini
If, in addition, we insist on a unique solution to U ¼: and Y ¼: :
Problem 2, wini should specify a unique initial Uf mTr Yf pTr
condition. Sufficient conditions for identifiability
can be derived from Lemma 1, see Markovsky, The blocks Up and Yp are referred to as the ‘past’
Willems, Van Huffel and De Moor (2006a, Theorem (of the computed response) and are used to set up the
8.16), and a uniqueness condition is given in initial conditions. The blocks Uf and Yf are referred to
Lemma 2. as the ‘future’ and are used for computing the desired
 International Journal of Control 1951

response y. With this notation steps 1 and 2 above can not satisfied, then (12) may have no solution.
be formulated as the following Algorithm 1. In this case, one can still execute Algorithm 1 by
computing an approximate solution for g, aiming at an
approximate response of the system. The question of
which notion of approximate solution one should use
Algorithm 1 Data-driven simulation.
depends on additional assumptions (or prior knowl-
Input: wd 2 (Rw)T, ur 2 (Rm)Tr, and wini 2 (Rw)Tini. edge) about the data generating mechanism. Results
1: Compute a solution of for the case when ud is exact and yd is noisy (output
2 3 2 3 error setup) are given in Markovsky, Vaccaro and
Up uini Van Huffel (2006b).
6 7 6 7
4 Yp 5g ¼ 4 yini 5: ð12Þ Note 3 (recursive implementation): The persistency of
Uf ur excitation for ud, required by Algorithm 1 is of order
2: Compute Tr þ Tini þ n(B). Typically Tini and n(B) are small
compared to the length T of the given trajectory,
Yf g ¼: y: ð13Þ however, Tr might be big. In Markovsky et al. (2005a)
Output: y. a refinement of the data-driven simulation algorithm is
presented (in the special cases of impulse response and
free response computation) that requires persistency of
Proposition 1: Assume that excitation of order 1 þ Tini þ n(B). The idea is to
construct the response y iteratively by weaving
. the data generating system B is controllable,
together consecutive patches of y. Due to lack of
and
space, we do not present iterative versions of the
. the input component ud of the trajectory wd is
algorithms in this paper.
persistently exciting of order Tini þ Tr þ n(B).
Next, we describe three special cases of the generic
Then for any ur 2 (Rm)Tr, the system of equations (12) is
data-driven simulation algorithm that are of indepen-
compatible and for any particular solution g, the output
dent interest and are used in the solution of data-driven
y:¼ Yfg of Algorithm 1 is such that (10) holds. If in
control problems. The observations made in Notes 2
addition, Tini  l(B), then y ¼ yr is the unique signal that
and 3 can be repeated verbatim for the special cases of
satisfies (10).
Algorithm 1, presented below.
Proof: Under the assumptions of the proposition, we
can apply Lemma 2. It follows that
4.3 Special case u ¼ 0: zero input response
colspan ðHTini þTr ðwd ÞÞ ¼ BTini þTr :
The columns of an extended observability matrix
Since by assumption wini 2 BTini, for any ur 2 (Rm)Tr, OTr ðA, CÞ of B are Tr samples long free responses of
there is y 2 (Rp)Tr, such that (10) holds. Then there is in B. Therefore, by using a data-driven simulation
general a nonunique vector g, such that (11) holds. The algorithm we can construct an observability matrix
system of equations (12) on step 1 of Algorithm 1 directly from data. In Algorithm 2 we compute
consists of a subset of the equations of (11), which j :¼ T  Tr  Tini þ 1 free responses f1, . . . , fj and do a
proves that (12) is compatible. Assume now that g is a rank revealing factorisation of the matrix F :¼ [f1    fj],
particular solution of (12) and y is defined as in (13). i.e., factor F as OXini , where O is full column rank and
Then by Lemma 2, Xini is full row rank. Due to the persistency of
excitation assumption of Proposition 1, the number
HTini þTr ðwd Þg ¼ colðwini , colður , yÞÞ 2 BTini þTr ,
of linearly independent free responses computed in this
so that (10) holds. way is n(B).
Under the additional assumption that Tini  l(B),
we can apply Lemma 1. It follows that wini and ur
Algorithm 2 Compute an observability matrix.
determine a unique output yr, such that (10) holds.
Then the signal y, computed by Algorithm 1 is unique Input: wd 2 (Rw)T, Tini 2 N, and Tr 2 N.
and is equal to yr. œ 1: Compute a solution of
Note 2 (‘noisy’ data): The choice of the particular 2 3 2 3
Up Up
solution g on step 1 of Algorithm 1 is irrelevant as long 6 7 6 7
4 Yp 5G ¼ 4 Yp 5
as the assumptions of Proposition 1 are satisfied.
However, if one or more of the assumptions are Uf 0mTr j
1952 I. Markovsky and P. Rapisarda
2: Compute Yf G ¼: F: 2: Compute

3: Compute a rank revealing factorisation Yf G ¼: Y0 :

F ¼ OXini : 3: Define W0 :¼ [col(u0,1, y0,1)    col(u0,j, y0,j)], where u0,i is
the ith column of HTr(ud) and y0,i is the ith column of
Output: O. Y0.
4: Compute a rank revealing factorisation W0 ¼ B0L
Output: B0.
Proposition 2: If the assumptions of Proposition 1 are
satisfied and Tr  l(B), the matrix O, computed by
Algorithm 2, is an observability matrix of B with Tr
block rows, corresponding to a minimal input/state/ Proposition 3: If the assumptions of Proposition 1 are
output representation B (A, B, C, D) of B, i.e., satisfied, the columns of the matrix B0, computed by
Algorithm 3, form a basis for the zero initial conditions
O ¼ OTr ðA, CÞ:
subbehaviour B0,Tr of BTr.
Proof: From Proposition 1 it follows that every
Proof: From Proposition 1, it follows that the
column of the matrix F, computed on step 2 of
columns of Y0 are zero initial condition responses of
Algorithm 2, is a free response of B. Moreover,
B. By construction the corresponding inputs are the
from Willems, Rapisarda, Markovsky and De Moor
columns of HTr(ud), so that col span(W0)  B0,Tr. The
(2005, Corollary 2) and the assumption Tr  l(B) it
persistency of excitation assumption implies
follows that rank(F) ¼ n(B). The statement of the
rank(HTr(ud)) ¼ mTr. Therefore rank(W0) ¼ mTr. But
proposition follows from rankðOÞ ¼ rankðFÞ ¼ nðBÞ.œ
dim(B0,Tr) ¼ mTr, and consequently

4.4 Special case wini ¼ 0: zero initial col spanðB0 Þ ¼ col spanðW0 Þ ¼ B0, Tr : œ
condition response
Define the zero initial conditions subbehaviour B0 of
B as follows 4.5 Special case wini ¼ 0, u ¼ Id: impulse response
 Let ei be the ith column of the m  m identity matrix Im.
B0, Tr :¼ wf 2 ðRw ÞTr
k ð0, . . . , 0Þ ^wf 2 BTini þTr , for all Tini 2 N : ð14Þ
Choosing zero initial conditions wini ¼ 0 and pulse
|fflfflfflfflffl{zfflfflfflfflffl} input ui(1) ¼ ei, ui(t) ¼ 0, for t > 1, we find the impulse
Tini
response of B. Note that this procedure constitutes a
The set B0 consists of all trajectories of B that when system identification method, because a representation
preceded with an arbitrary number of zero samples are of B is derived from data.
still trajectories of B  l(B) or more leading zero samples
specify zero initial conditions, so B0 is indeed the
Algorithm 4 Compute the impulse response.
subspace of B consisting of all zero initial conditions
trajectories. Note that dim(B0,Tr) ¼ mTr, so we can Input: wd 2 (Rw)T, Tini 2 N, and Tr 2 N.
compute a basis for B0,Tr directly from data, by
1: Compute a solution of
computing mTr linearly independent zero initial condi-
2 3 2 3
tion responses. In Algorithm 3, we compute Up 0mTini m
j :¼ T  Tr  Tini þ 1 zero initial condition responses 6 7 6 0pTini m 7
4 Yp 5G ¼ 4 5: ð15Þ
w0,1, . . . , w0,j, because the persistency of excitation of ud
guarantees that among them there are exactly mTr Uf colðIm , 0mðTr 1Þm Þ
linearly independent one. Then a basis for B0,Tr can be 2: Compute
computed from the matrix W0 :¼ [w0,1    w0,j] by doing
Yf G ¼: H:
a rank revealing factorisation.
Output: H.
Algorithm 3 Compute a basis for the zero initial conditions
subbehaviour.
Input: wd 2 (Rw)T, Tini 2 N, and Tr 2 N. Proposition 4: If the assumptions of Proposition 1 are
satisfied, the matrix H, computed by Algorithm 4,
1: Compute a solution of
2 3 2 3 contains the first Tr samples of the impulse
Up 0mTini j response of B.
6 7 6 7
4 Yp 5G ¼ 4 0pTini j 5:
Proof: Follows trivially from Proposition 1 and the
Uf HTr ðud Þ structure of the right-hand-side of (15). œ
 International Journal of Control 1953

5. Output matching control

Simulation can be viewed as ‘input matching’: deter-
mine the output component of a system trajectory,
corresponding to a given input component and initial
condition. By reversing the roles of the input and the
output, we have an ‘output matching’ problem:
determine the input component of a system trajectory, Figure 1. Structure of the output matching controller.
given an output component and initial condition. This
is a control problem, which we call ‘output matching
control’. outputs, such that the series connection of B1 and B
The output matching problem is a special (singular) is Ip. Note that the existence of a right inverse requires
case of the output tracking problem, when the input that the given system B has more inputs than outputs.
does not enter into the cost function and the desired Proposition 5 (model-based output matching): If a
output can be achieved exactly. Alternatively, the causal right inverse system B1 of B exists, then
output matching problem can be viewed as a special for any reference signal yr 2 (Rp)Tr and initial
dead-beat control problem, when the initial condition condition xini 2 Rn, there is a control signal u, such
of the system is given. In Section 5.2, we present a that (16) holds.
model-based solution and in Section 5.3 we present a
Proof: If a causal right inverse B1 exists, the
data-driven solution to this problem.
feedforward controller, given in Figure 1 solves the
output matching problem for any yr 2 (Rp)Tr and
xini 2 Rn.
5.1 Problem formulation
We call the output matching problem ‘classical’, if the We do not consider further conditions for existence of
to-be-controlled system is given by an input/state/ a causal right inverse system B1 nor algorithms for
output representation. computing its representation from a given representa-
tion of B. The reader is referred to Sain and Massey
Problem 3 (classical output matching): Given (1969) for related results. Our concern next is an
. an input/state/output representation B(A, B, algorithm for data-driven output matching control.
C, D) of a linear time-invariant system B,
. a reference response yr ¼ (yr(1), . . . , yr
(Tr)) 2 (Rp)Tr, and 5.3 Data-driven output matching algorithm
. an initial condition xini 2 Rn, Consider given wd, yr, and wini. Under the assumptions
of the fundamental lemma,
find an input u 2 (Rm)Tr, such that the response of B to
u and initial condition xini is yr. wini ^ colðu, yr Þ 2 BTini þTr () 9 g, such that
 
In the data-driven output matching control problem, HTini þTr ðwd Þg ¼ col wini , colðu, yr Þ :
the to-be-controlled system and the initial condition are The equation
implicitly specified by given trajectories of the system.  
HTini þTr ðwd Þg ¼ col wini , colðu, yr Þ ð17Þ
Problem 4 (data-driven output matching): Given
. a trajectory wd ¼ (wd(1), . . . , wd(T)) 2 (Rw)T now has as unknowns g and u. Therefore, we determine
of B, g from the equations in (17) with right-hand-side
. a reference response yr ¼ (yr(1), . . . , yr(Tr)) 2 wini and yr, and define u from the remaining equations.
Using the notation Up, Uf, Yp, Yf, defined in
(Rp)Tr, and
Section 4.2, we obtain Algorithm 5.
. an initial trajectory wini ¼ (wini(1), . . . , wini
(Tini)) 2 (Rw)Tini, where wini 2 BTini,
find an input u 2 (Rm)Tr, such that Algorithm 5 Data-driven output matching.
Input: wd 2 (Rw)T, yr 2 (Rp)Tr, and wini 2 (Rw)Tini.
wini ^ ðu, yr Þ 2 BTini þTr : ð16Þ
1: Compute a solution of
5.2 Model-based output matching 2 3 2 3
Up uini
Figure 1 shows the structure of a feedforward output 6 7 6 7
4 Yp 5g ¼ 4 yini 5: ð18Þ
matching controller. B1 denotes a right inverse of B,
i.e., a linear time-invariant system with p inputs and m Yf yr
1954 I. Markovsky and P. Rapisarda

2: Compute the input u, computed by Algorithm 5. The assumptions

Uf g ¼: u: of Proposition 6 are satisfied and we experimentally
verify that the input u achieves the reference output yr.
Output: u.
In the particular experiment, however, u grows
unbounded as t increases. The reason for this is that
the plant B is non-minimum phase (has zero at 1.17),
Proposition 6: If the assumptions of Proposition 1 are and the output matching controller internally computes
satisfied and the zero initial conditions subbehaviour B0 the response of the inverse system B1.
of B, see (14), satisfies the property B0,1 ¼ Rm, the
In the next section, we consider a more general
vector u computed by Algorithm 5 is such that (16) holds.
tracking problem – follow a reference trajectory wr by
Proof: Consider an input/state/output representation trading-off errors in both the input and the output. In
B(A, B, C, D) of B and let H ¼ (H(0), H(1), . . . ) be the the infinite horizon case, the latter problem avoids the
impulse response of B. Under the assumptions of instability issue, pointed out in Example 1.
Proposition 1, wini defines a unique initial state xini.
Then (18) is compatible if and only if
6. Linear quadratic tracking
yr  OTr ðA, CÞxini ¼ TTr ðHÞu ð19Þ
In a linear quadratic tracking problem the objective is
is compatible. A necessary and sufficient condition for to choose a trajectory w ¼ (u, y) of a linear time-
compatibility of (19) is that the matrix TTr(H) is full invariant system B that is as close as possible to a given
row rank. Due to the lower triangular Toeplitz reference trajectory wr 2 (Rw)Tr. More precisely, in a
structure of TTr(H), it is easy to see that the full row finite-horizon case, the objective is to minimise over
rank condition for TTr(H) is equivalent to a full row w 2 BTr the cost function
rank condition for H(0) ¼ D. The statement of the X
Tr
 >  
proposition follows from kwr  wk2 :¼ wr ðtÞ  wðtÞ  wr ðtÞ  wðtÞ ,
t¼1
D is full row rank () B0, 1 ¼ Rm :
ð20Þ
œ
where  2 Rww is a given positive definite weight
Example 1 (unbounded input): Consider the linear
matrix. In the special case when the reference trajectory
time-invariant system B of order n ¼ 2, with m ¼ 1
is the zero trajectory, the tracking problem is referred
input and p ¼ 1 output, induced by the transfer
to as the regulation problem. In this case, the optimal
function
tracking, aiming solely at minimising the criterion (20)
ðz  0:7847Þðz þ 1:17Þ over all trajectories w of B, has a trivial solution—the
HðzÞ ¼ :
z2  1:615z þ 0:6972 zero trajectory. The regulation problem is meaningful,
when a non-zero initial condition is specified.
The trajectory wd is a random one with T ¼ 200 samples, Therefore, we introduce initial conditions specification
the reference signal is yr ¼ 0 with horizon Tr ¼ 30, in the general tracking problem.
and the initial trajectory is wini ¼ (1, 1). Figure 2 shows The classical formulation of the tracking problem
assumes that an input/state/output representation of
80
the system B is given. In the context of data-driven
60 tracking, we start instead from a given trajectory
40 wd 2 (Rw)T of B, and aim at finding an optimal
20 trajectory without explicitly computing a representa-
tion (in particular an input/state/output representa-
0
tion) of B.
u

−20
Problem 5 (linear quadratic tracking): Given
−40
−60 . a trajectory wd ¼ (wd(1), . . . , wd(T)) 2 (Rw)T of
a linear time-invariant system B,
−80
. a reference trajectory wr ¼ (wr(1), . . . , wr
−100 (Tr)) 2 (Rw)Tr,
5 10 15 20 25 30
t . an initial trajectory wini ¼ (wini(1), . . . , wini
Figure 2. An input u that achieves the reference output yr in
(Tini)) 2 (Rw)Tini, where wini 2 BTini, and
the output matching example. . a positive definite matrix  2 Rww,
 International Journal of Control 1955

find a trajectory w* 2 B that minimises the cost in time recursion that depends on the reference signal wr
function kwr  wk2 and has as a prefix the initial and the initial condition xini. The formulae for the
trajectory wini, i.e., solve the problem continuous-time case are given in Markovsky, Willems
and De Moor (2002, Theorem 1). For simplicity, next
min kwr  wk2 subject to wini ^ w 2 BTini þTr :
w we give the solution for the finite-horizon
linear quadratic regulation problem, i.e., for the special
In the following we propose three solutions to
case of wr ¼ 0.
Problem 5. The first one computes parameters A, B,
We can eliminate the variable y in (21) by
C, D of an input/state/output representation of the
substitution. This gives the following linear quadratic
data generating system and then proceeds by solving
regulation problem with complete state information
a classical tracking problem, using standard linear
quadratic control techniques. The second solution XTr       9
uðtÞ > I 0 > I 0 uðtÞ > >
computes the impulse response of the system and min  >
>
x, u xðtÞ D C D C xðtÞ =
then solves the linear quadratic tracking problem as t¼1
ð22Þ
a weighted least squares problem. The third solution subject to xðt þ 1Þ ¼ AxðtÞ þ BuðtÞ, >
>
>
>
for solving Problem 5 is completely data-driven. It is ;
xð1Þ ¼ xini , for t ¼ 1, . . . , Tr :
not based on any particular representation of the plant,
however, as in the solution using the impulse response, Define the partitioning
one of the steps is a solution of a large weighted least  
u uy
squares problem.  ¼: :
yu y
The solution of (22) is (see, eg, Åström and
6.1 Algorithm using input/state/output Wittenmark (1997, Theorem 11.1))
representation x ðt þ 1Þ ¼ ðA  BLt Þx ðtÞ, xð1Þ ¼ xini ,
The classical but indirect solution of Problem 5 is to  
 Lt ð23Þ
compute first an input/state/output representation w ðtÞ ¼ x ðtÞ,
C  DLt
B(A, B, C, D) of B from the data wd and then using
the parameters (A, B, C, D) to compute the optimal where
trajectory w*. This is the well known model-based  1
approach that we summarise for completeness. Lt :¼ B> Stþ1 B þ u þ uy D þ D> > >
uy þ D y D
By assumption the data w is an exact trajectory of  
 B> Stþ1 A þ uy C þ D> y C ð24Þ
the unknown system B. Therefore, we are dealing
with an exact (deterministic) identification problem and
wd 7 ! B. Under the assumptions of Markovsky et al.  >
(2006a, Theorem 8.16), the system is identifiable and St ¼ A> Stþ1 A þ C> y C  B> Stþ1 A þ uy C þ D> y C
 1
an input/state/output representation of B can be  B> Stþ1 B þ u þ uy D þ D> > >
uy þ D y D
computed. Algorithms for computing an input/state/  
 B> Stþ1 A þ uy C þ D> y C , STr þ1 ¼ 0: ð25Þ
output representation from exact data are developed in
the (deterministic) subspace identification setting, see The resulting algorithm for solving Problem 5, in the
Van Overschee and De Moor (1996, Chapter 2) and special case wr ¼ 0, using an input/state/output
Markovsky et al. (2006a, Chapter 8). representation of the plant is Algorithm 6.
A procedure for deriving the initial state from the
initial trajectory is outlined in the proof of Lemma 1. Algorithm 6 Data-driven linear quadratic regulation, using
Once the state space representation and the initial state an input/state/output representation.
are available, Problem 17 becomes Input: wd 2 (Rw)T, wr 2 (Rp)Tr, wini 2 (Rw)Tini, and  2 Rww,
 > 0.
min kwr  colðu, yÞk2
x, u, y Identification, e:g:, ð Algorithm 8:5Þ
xðt þ 1Þ ¼ AxðtÞ þ BuðtÞ, 1 : ðwd , Tini Þ !
subject to xð1Þ ¼ xini , ðA, B, C, DÞ
yðtÞ ¼ CxðtÞ þ DuðtÞ, Observer ð5:8:9Þ
for t ¼ 1, . . . , Tr : 2: ðwini , A, B, C, DÞ ! xini
ð21Þ Synthesis ð23, 24:25Þ
3: ! w
ð, wr , xini , A, B, C, DÞ 
The solution leads to a difference Riccati equation that Output: w*.
depends only on the A, B, C, D matrices and a backward
1956 I. Markovsky and P. Rapisarda

6.2 Algorithm using impulse response representation which is a standard weighted least squares problem. Its
Another approach for solving Problem 5 is to compute solution is
the impulse response H of B from the data wd, using  
u ¼ T> ee e 1 T> ðHÞ
e ðw
e r  wfree Þ, ð29Þ
Algorithm 4, and then using H to compute w*. Tr ðHÞTTr ðHÞ Tr
Let B(A, B, C, D) be an input/state/output representa- where
tion of B and let
2 3 e ¼ diagð, . . . , Þ 2 RTr wTr w ,

colð0mn , CÞ
6 colð0mn , CAÞ 7 so that
6 7
e
O :¼ 6 .. 7
6 7
4 . 5 e  þ wfree :
w ¼ TTr ðHÞu ð30Þ
Tini þTr 1
colð0mn , CA Þ
The resulting algorithm for solving Problem 5, using
be an extended observability matrix of B with Tini þ Tr an impulse response representation of the plant, is
block rows. Any zero-input trajectory wfree ¼ col(0, f) Algorithm 7.
of B can be written as wfree ¼ e
Oxini , for some xini 2 Rn.
Define also the augmented Markov parameters
 
e ¼ col Im , Hð0Þ ,
Hð0Þ Algorithm 7 Data-driven linear quadratic tracking, using an
impulse response representation.
and
Input: wd 2 (Rw)T, wr 2 (Rp)Tr, wini 2 (Rw)Tini, and  2 Rww,
   > 0.
e ¼ col 0m , HðtÞ ,
HðtÞ for t ¼ 1, 2, . . . : Algorithm 4
1: ðwd , Tini , Tr Þ ! H
Then for any trajectory wini ^ w 2 BTiniþTr there is a
Algorithm 2
corresponding initial condition xini and an input signal 2: ðwd , Tini , Tini þ Tr Þ ! O
uini ^ u, such that
ð27, 28Þ
    3: ðwini , H, OÞ ! wfree
wini u
¼e e ini :
Oxð1Þ þ TTini þTr ðHÞ ð26Þ ð29, 30Þ
w u 4: ð, wr , wfree , HÞ ! w
Define the partitionings Output: w*.
" # " #
e e 0
e
O ¼:
O 1 e ¼: T11
and TTini þTr ðHÞ ,
e
O2 e21
T e
TTr ðHÞ

that are conformable with the partitionings of

6.3 Algorithm for data-driven linear quadratic
col(wini, w) and col(uini, u). The equations in (26) with
tracking
left-hand-side wini
In this section, we present a complete data-driven
wini ¼ e e11 uini
O1 xð1Þ þ T ð27Þ solution to Problem 5, i.e., one that does not rely on
any specific system representation. The basic idea
are decoupled from the other equations and can be underlying the data-driven linear quadratic tracking
solved independently in terms of the initial state x(1) algorithm is to project the trajectory wrwfree on the
and initial input signal uini. The remaining equations zero initial conditions subbehaviour B0,Tr (see (14)) of
B. The data-driven simulation algorithms, presented in
w¼e e21 uini þ TTr ðHÞu
O2 xð1Þ þ T e
Section 4 can be used to determine the free trajectory
specify w in terms of the input u. Define wfree and a basis for B0,Tr.
Theorem 1: Let W0 2 RTrw. be a matrix, such that
wfree :¼ e e21 uini :
O2 xð1Þ þ T ð28Þ col span (W0) ¼ B0,Tr. Then the solution of Problem 17
Note that wfree is the free trajectory of B, with prefix is given by
wini (or equivalently by the initial condition xini).  þ >
w ¼ W0 W> e e r  wfree Þ þ wfree , ð31Þ
The optimal tracking problem becomes 0 W0 W0 ðw

e 2, where wfree is the free trajectory of B, caused by

min kwr  wfree  TTr ðHÞuk 
u the initial trajectory wini. If W0 defines a basis for
 International Journal of Control 1957

0.7 0.6
0.6 0.5
0.5 0.4

0.4 0.3

u*

y*
0.2
0.3
0.1
0.2
0
0.1 −0.1
0 −0.2
5 10 15 20 25 30 5 10 15 20 25 30
t t
Figure 3. First experiment: wr dotted line, w* solid line.

0.6 1.2
0.5 1
0.4 0.8
0.3 0.6
u*

y*
0.2 0.4
0.1 0.2
0 0
−0.1 −0.2
10 20 30 40 50 60 10 20 30 40 50 60
t t

Figure 4. Second experiment: wr dotted line, w* solid line.

B0,Tr, then the pseudoinverse in (31) can be replaced directly from the data wd, using Algorithm 1. Then by
by inverse. Theorem 1, the optimal trajectory is (31), and we
completely circumvent the need to compute the
Proof: Any zero initial conditions trajectory
e 0. impulse response H.
w0 ¼ col(u0, y0) 2 (Rw)Tr is of the form w0 ¼ TTr(H)u
The resulting algorithm for solving Problem 5 is
Therefore,
Algorithm 8
   
e ¼ col span W0 :
B0,Tr ¼ col span TTr ðHÞ
Algorithm 8 Data-driven linear quadratic tracking.
Consider the space W ¼ ðRw ÞTr with inner product
defined by hw1 , w2 i ¼ w> e Input: wd 2 (Rw)T, wr 2 (Rp)Tr, wini 2 (Rw)Tini, and ( 2 Rww,
1 w2 . The projector on B0,Tr
in W is ( > 0.
Algorithm 3
  1: ðwd , Tini , Tr Þ ! W0
e T> ðHÞ
TTr ðHÞ e T e 1 T> ðHÞ
e Tr ðHÞ ee
Tr Tr
 > þ > Algorithm 1
¼ W0 W0 W e 0 W : e 2: ðwd , 0mTr 1 , Wini Þ 
! wfree
0

Then (31) follows from (29, 30). œ ð31Þ

3: ð, wr , wfree , W0 Þ ! w
Theorem 1 is based on the fact that the optimal Output: w*.
solution w* depends only on the subspace B0,Tr, the
metric, given by the weight matrix , and the free
trajectory wfree, initiated by wini, and not on the
particular basis of B0,Tr. In (29, 30), we used as a basis
for B0,Tr the columns of the Toeplitz matrix TTr(H) e
constructed from the impulse response of B. However, 6.4 Simulation examples
using Algorithm 3, we can find another basis for B0,Tr. We show simulation examples that illustrate
In addition, the free trajectory wfree can be computed numerically the equivalence of the three methods for
1958 I. Markovsky and P. Rapisarda

data-driven control linear quadratic tracking. The References

to-be-controlled plant B is the same as that in
Example 1. The data wd, used by the algorithms, Åström, K., and Wittenmark, B. (1997), Computer Control
is a random trajectory of B with T ¼ 200 samples. It is Systems: Theory and Design, Upper Saddle River, NJ:
the same in both experiments. A reference trajectory wr Prentice Hall.
Campi, M., Lecchini, A., and Savaresi, S. (2002), ‘Virtual
with Tr samples and an initial trajectory wini with
Reference Feedback Tuning: A Direct Method for the
Tini ¼ l(B) ¼ 2 samples are chosen as follows: Design of Feedback Controllers’, Automatica, 38,
. Experiment 1: data-driven regulation Tr ¼ 30, 1337–1346.
    Favoreel, W. (1999). ‘Subspace methods for identification
1 1
wr ¼ 0 and wini ¼ , , and control of linear and bilinear systems’, PhD thesis,
1 1 Faculty of Engineering, K.U.Leuven.
. Experiment 2: data-driven step tracking Fujisaki, Y., Duan, Y., and Ikeda, M. (2004). ‘System
Tr ¼ 60, Representation and Optimal Control in Input–Output
Data Space’, in Proceedings of the 10th IFAC Symposium
0, for t ¼ 1, . . . , 30 on Large Scale Systems, Osaka, Japan, pp. 197–202.
ur ¼ 0, yr ðtÞ ¼
1, for t ¼ 31, . . . , 60 Furuta, K., and Wongsaisuwan, M. (1995), ‘Discrete-time
    LQG Dynamic Controller Design using Plant Markov
1 1
and wini ¼ , : Parameters’, Automatica, 31, 1317–1324.
1 1 Hildebrand, R., Lecchini, A., Solari, G., and Gevers, M.
The weight matrix  is chosen as the 2  2 identity (2004), ‘Prefiltering in Iterative Feedback Tuning:
matrix. Optimization of the Prefilter for Accuracy’, IEEE
In both experiments the compared methods com- Transactions on Automatic Control, 49, 1801–1806.
pute the same optimal trajectory, see Figures 3 and 4. Hjalmarsson, H., Gevers, M., Gunnarsson, S., and Lequin, O.
The optimal values of the cost function in experiment 1 (1998), ‘Iterative Feedback Tuning: Theory and
and 2 are kw k2 ¼ 1:1139 and kwr  w k2 ¼ 2:1034, Applications’, IEEE Control Systems Magazine, 18, 26–41.
Ikeda, M., Fujisaki, Y., and Hayashi, N. (2001),
respectively.
‘A Model-Less Algorithm for Tracking Control Based on
Input-Ouput Data’, Nonlinear Analysis, 47, 1953–1960.
Markovsky, I., Willems, J.C., and De Moor, B.
7. Conclusions (2002). ‘Continuous-Time Errors-in-Variables Filtering’,
We presented a generic approach for construction of in Proceedings of the 41st Conference on Decision and
linear time-invariant system responses directly from Control, Las Vegas, NV, pp. 2576–2581.
data and showed its fundamental role for solving Markovsky, I., Willems, J.C., Rapisarda, P., and De Moor,
output matching and finite horizon linear quadratic B. (2005a), ‘Algorithms for Deterministic Balanced
tracking data-driven control problems. Three solutions Subspace Identification’, Automatica, 41, 755–766.
Markovsky, I., Willems, J.C., and De Moor, B. (2005b).
to the linear quadratic tracking problem are: (1)
‘Data Driven Simulation with Applications to System
identify an input/state/output representation of the
Identification’, in Proceedings of the 16th IFAC World
plant and use classical state space synthesis, (2) identify Congress, Prague, Czech Republic.
an impulse response representation of the plant and Markovsky, I., Willems, J.C., Van Huffel, S., and De
solve a weighted least squares problem, and (3) do Moor, B. (2006a), Exact and Approximate Modeling of
a projection on the subbehaviour B0 of zero-initial Linear Systems: A Behavioral Approach, Philadelphia:
conditions responses of the plant. The third solution is SIAM.
based on simulation of zero input and zero initial Markovsky, I., Vaccaro, R.J., and Van Huffel, S.
conditions responses of the plant. Using the data- (2006b), ‘System Identification by Optimal Subspace
driven simulation algorithm such responses can be Estimation’, Technical Report 06–162. Dept. EE,
computed without deriving a representation of the K.U.Leuven.
plant. All solutions need the same basic assumptions: Park, U., and Ikeda, M. (2006). ‘Control Design of Linear
the plant B is controllable, and an input component of Time-Invariant Discrete-Time Systems with Direct use of
Time Series Data’, in Proceedings of the Conference on
the given trajectory is persistently exciting of order
Decision and Control, pp. 4454–4459.
n(B) þ l(B) þ 1, which are the identifiability conditions Safonov, M. (1996), ‘Focusing on the Knowable:
of Markovsky et al. (2006a, Chapter 8). The solution Controller Invalidation and Learning’, in Control
given by the input/state/output approach, however, is Using Logic-based Switching, ed. A. Morse, Berlin:
in the form of a feedback, while the other solutions Springer-Verlag, pp. 224–233.
compute off-line the optimal trajectory. This poses the Safonov, M., and Tsao, T. (1997), ‘The Unfalsified Control
question of finding an optimal feedback control law Concept and Learning’, IEEE Transactions on Automatic
directly from data. Control, 42, 843–847.
 International Journal of Control 1959

Sain, M., and Massey, J. (1969), ‘Invertibility of Linear Verhaegen, M., and Dewilde, P. (1992), ‘Subspace model
Time-invariant Dynamical Systems’, IEEE Transactions on identification, Part 1: The Output-error State-space Model
Automatic Control, 14, 141–149. Identification Class of Algorithms’, International Journal
Shi, G., and Skelton, R. (2000), ‘Markov Data-based LQG of Control, 56, 1187–1210.
Control’, Journal of Dynamic Systems, Measurement, and Willems, J.C., Rapisarda, P., Markovsky, I., and De Moor,
Control, 122, 551–559. B. (2005), ‘A Note on Persistency of Excitation’, Control
Van Overschee, P., and De Moor, B. (1996), Subspace Letters, 54, 325–329.
Identification for Linear Systems: Theory, Implementation, Woodley, B. (2001) ‘Model Free Subspace Based H1
Applications, Boston: Kluwer. Control.’ PhD thesis, Stanford University.