Department of Mechanical Engineering, MIT H.

Harry Asada

2.160 I DENTIFICATION , ESTIMATION , AND LEARNING

LECTURE NOTES NO. 14

15. Time-Series Data Compression

15.1 FIR Model

b3
b2
u(t) y(t) b1
FIR

Finite Impulse Response Model t

Consider a FIR Model with colored noise v(t)

y ( t )  b1 u ( t  1)  b 2 u ( t  2 )         b m u ( t  m ) + v(t)
(1)
Suppose that we want to identify the parameters involved in G(q)  B(q) only,

   b1 b2 bm 
T

(2)
 (t )   u (t  1) u (t  2) u (t  m) 
T

Consider the least square estimate for a given set of data

Z N  (u(t ), y(t )) | t  1 N

ˆNLS  arg min   y(t )   T (t ) 

1 N 1
(3)
 N t 1 2

Let  0 be the vector of true parameter values. The parameter estimation error is given

by

ˆNLS  0   R( N )  f * ( N )
1

(4)
where
1 N
R( N )   (t ) (t )T   T 1 N
N t 1 f *( N )   (t ) v(t )
N t 1

Pro’s and Con’s of FIR Modeling

Pros.
LSE gives a consistent estimate limˆNLS  0 as long as the input sequence u (t ) is
N 

uncorrelated with the noise term v(t )  H (q)e(t ) , which may be colored and even
correlated with { y(t )} .

1
 Department of Mechanical Engineering, MIT H. Harry Asada

Cons
The number of parameters, m  nb ,  m , may be too large to estimate. This may
occur if
 The impulse response has a slow decaying mode, or
 The sampling rate is high.
The persistently exciting condition rank   full rank can hardly be satisfied.
Check the eigenvalues of   (or the singular values of  )
T

1  2    m

It is likely 1  2    n  n1  0    m .

Often m becomes more than 50 and it is difficult to obtain such an input series having
50 non-zero singular values.

Time series data compression is an effective method for coping with this
difficulty. Before formulating the above least square estimate problem, data are
processed so that the information contained in the series of regressor may be
represented in compact, low-dimensional form.

.
. . .
. . Filter Design Lk(q)

.
Rm Rn n<m

Coordinate Transformation (Data Compression)

L1(q)

u(t) y(t)
L2(q)

Ln(q)

2
 Department of Mechanical Engineering, MIT H. Harry Asada

15.2 Continuous-Time Laguerre Series Expansion

Let us begin with continuous-time Laguerre expansion.

[Theorem 15.1] If a transfer function G(s) is

 Strictly proper lim G( s)  0 G()  0 (5)
s 

 Analytic in Re( s)  0 No pole on RHP

 Continuous in Re( s)  0

then, there exists a sequence g k  such that

k 1
2a  s  a 

G( s)   g k   (6)
k 1 sa sa

where a  0 .

Proof:
Consider the transformation given by
sa
z (7)
sa
sz  a z  s  a s( z  1)  a ( z  1)
z 1
The inverse transform is given by  s  a (8)
z 1
This called “bilinear transformation”.

Im
sa Im
sa
sa
z  1 for s  j
sa
z

s
sa
sa

z-plane
Fast z  (s  a )  (s  a ) 1
sa
z
sa

Slow
Fast Re
Slow -1 1 Re
s
sa sa
z
sa
sa z
sa
sa The imaginary axis is
mapped to the unit circle.
sa The right half plane is
z  1
sa mapped to the outside of
the unit circle.

3
Department of Mechanical Engineering, MIT H. Harry Asada

Im Im

G(s) is analytic in
1

-1 1
Re Re

 
Consider  z 1  (44) is analytic outside
G( z)  G  a 
 z 1 
  the unit circle.
 s 

Therefore, there exists a Laurent expansion for G (z ) :


G ( z )   g k z k z 1 (9)
k 1

Example 1
1  z 1  1
G( s)  , Ga  G( z )
( s  p1 )( s  p2 )  z  1   a z  1  p  a z  1  p 
 1  2
 z 1  z  1 
sa
Considering that G()  0 lim  1  G ( z )  0 at z = 1
s  s  a

This implies that G( z ) has a factor of ( z  1) or (1  z 1 ) z


1
G ( z)  (1  z 1 ) g k z ( k 1) (10)
2a k 1

Substituting (7) into (10)

 ( k 1)
sa 1 sa  sa
G   G ( s )  (1  ) g k  
sa 2a s  a k 1  s  a 
( k 1)

2a  s  a 
 G(s)   g k  
k 1 sa sa
( k 1)
2a  s  a 
Lk ( s)    (11)
sa sa

All-pass Filter
Low Pass Filter

The Laplace transform of the Laguerre functions

4
Department of Mechanical Engineering, MIT H. Harry Asada

The Main Point

The above Laguerre series expansion can be used for data compression if the
parameter a , called a Laguerre pole, is chosen such that slow poles, i.e. dominant
poles, of the original system are close to the Laguerre pole. Let pi be a slow real pole
of the original transfer function G(s). If the Laguerre pole a is chosen such that
a  pi , then a truncated Laguerre expansion:

k 1
n
2a  s  a 
Gn ( s )   g k    G( s ) (12)
k 1 sa sa
converges to the original G(s) quickly for the following reason.

- Recall -
For a continuous-time system, a pole close to the imaginary axis is slow to converge,
while a pole far from the imaginary axis converges quickly. Likewise, in discrete time,
a pole close to the origin of a z-plane quickly converges, while the ones near the unit
circle are slow.

s-plane Im z-plane

Fast Slow Re Fast Slow

Choose a such that p1  a , where p1 is a slow, stable pole. Using the bilinear
transform, this slow pole in the s-plane can be transferred to a fast pole in the z-plane,
as shown below. Representing in the z-plane, the transfer function can be truncated;
just a few terms can approximate the impulse response since it converges quickly.

p1 Im

Re converges

Transformation
Im
1
Truncated

-1
time
1 Re
Fast Convergence

-1
Fast pole

When the original transfer function has many poles, the Laguerre pole is placed near

5
Department of Mechanical Engineering, MIT H. Harry Asada

the dominant pole so that most of the slow poles may be approximated by the
Laguerre pole.

Im Im

Dominant pole
1

-1 1
Re
Re

In the transformed plane, all

the poles are confined within
a circle of radius r<1.

With the bilinear transform, these slow poles are transferred to the ones near the
origin of the z-plane. They are fast, hence the transfer function can be approximated
to a low order model. The following example demonstrates these unique features of
the Laguerre Series Expansion.

Example 2
Consider the continuous-time, Laguerre series expansion of the following two
transfer functions:
1 1
G1 ( s)  , G2 ( s) 
( s  1) 2
( s  1)(s  2)

a). The Laguerre series expansion is associated with the transformation of variables
given by (7). Change the variable of the above transfer function G1 ( s) from s to z, and

obtain the new transfer function G1 ( z ) . Find the poles of G1 ( z ) , when the parameter a
is 1; a  1.

Solution
sa
Solving z for s,
sa

sz  az  s  a s( z  1)  a( z  1)

z 1
 sa
z 1

6
Department of Mechanical Engineering, MIT H. Harry Asada

Replacing s by this in G1 ( s) yields:

 z  1
G1 ( z )  G1  a
1 z  1 2

 
 z 1  z  1 
2
a z  1  z  12
 a  1
 z 1 

 G1 ( z ) 
 z  1
2
Poles z
a 1 a 1
, , Repeated poles
a  1z  a  12 a 1 a 1

b). Setting the parameter a to a  1 , obtain the coefficients of the Laguerre series
expansion of G1 ( s) :

k 1

2a  s  a 
G1 ( s)   g k  
k 1 sasa

Is the coefficient series g k  a finite series or an infinite series? What if the parameter

a is not 1: a  1 ? Explain why.

Solution
Substituting a = 1 into the above transfer function G1 ( z ) in part a) yields:

1 1 1 1
G1 ( z )  (1  z 1 )2   z 1  z 2
4 4 2 4
This shows that the Laguerre series expansion converges in the second term. Therefore,
it must be expressed as:
1 2 2 s 1
G1 ( s)   g1  g2
( s  1) 2
s 1 s 1 s 1
2 g1  g 2 s  2 g1  g 2 

( s  1) 2

Multiplying ( s  1)2 to both sides, we obtain 1  2 g1  g 2 s  2 g1  g 2  .

Comparing the terms,
g1  g 2  0  g 2   g1

1
2 g1  g 2   1  2 2 g1  1  g1    g2
2 2

1 1
 g1  g2  
2 2 2 2

7
Department of Mechanical Engineering, MIT H. Harry Asada

c). Change the variable of G2 ( s) from s to z in the same way as part a), and obtain the
new transfer function G2 ( z ) . Find all the poles of the new transfer function G2 ( z ) .

Solution
 z  1 1
G 2 ( z )  G2  a 
 z  1   a z  1  1 a z  1  2 
  
 z  1  z  1 


 z  1
2

a z  1  z  1a z  1  2z  1


z  12
a  1z  a  1a  2z  a  2
a 1 a2
Poles z1   , z2  
a 1 a2

d). Set a  1.5 , and plot the poles of G2 ( z ) on a complex plane. Next, set a  5 , and
plot the poles once again. Which parameter value gives faster convergence in the
Laguerre series expansion, a  1.5 or a  5 ? Explain mathematically why it
converges more quickly than the other.

Solution
a  1.5 ,
0.5 1
z1    ,
2.5 5
 0.5 1 Re
z2   
3.5 7
a 5,
4 2
z1    ,
6 3
3
z2  
7

The unit circle in the z - plane corresponds to the imaginary axis in the s - plane. If poles
exist near the unit circle in the z - domain, convergence is slow. On the other hand, if
poles are close to the origin in the z - domain, convergence is fast. Namely, the impulse
response doesn’t need many terms. An extreme case is that all the poles are at the origin
of the z - domain. As shown in Part b), only two terms are needed when both poles are at
the origin.

8
Department of Mechanical Engineering, MIT H. Harry Asada

1
When a  1.5 , both poles are within a radius of from the origin in the complex
5
plane, while those poles for a  5 are outside of this radius. Therefore, the impulse
response of G2 ( z ) with a  1.5 converges more quickly than G2 ( z ) of a = 5.

9
Department of Mechanical Engineering, MIT H. Harry Asada

15.3 Discrete-Time Laguerre Series Expansion

[Theorem 15.2]
Assume that a Z-transform G(z) is
 Strictly proper G()  0

 Analytic in z  1 RHP

 Continuous in z  1

Then
k 1

K  1  az 
G( z )   gk   (13)
k 1 za za 
where -1 < a < 1 and

K  (1  a 2 )T T = sampling Interval (14)

Proof
Consider the bilinear transformation:
za
w (15)
1  az
wa
w  azw  z  a w  a  z(aw  1) therefore, z  (16)
aw  1

The relationship between z-plane and w-plane is more complex, but we can show the
following correspondence between the two:

Transformation

z-plane w-plane

 wa 
G ( w)  G  is analytic in w  1 , and is proper G()  0
 aw  1 

10
Department of Mechanical Engineering, MIT H. Harry Asada

1 1
lim w   G ( )  0 (17)
z  a a

Therefore

T
G ( w)  (a  w 1 ) g k w ( k 1) (18)
K k 1

 ( k 1)
 za  T 1  az   za 
G( z )  G    (a  ) g k   (19)
 1  az  K z  a k 1  1  az 
k 1

K  1  az 
 G( z )   g k  
k 1 za za 
Q.E.D.

Now we can write

y ( t )  G ( q )u ( t )
k 1
n
K  1  aq  n
  gk   u(t )   g k Lk ( q)u(t )
k 1 q  a  q  a  k 1

where Lk (q), k  1,, n is a series of filters. Once the original input data are filtered

with Lk (q), k  1,, n ,

xk  Lk (q)u(t ), k  1,, n (20)

The output y(t ) is represented as a moving average of the transformed input xk ,

that is, a FIR model.

L1(q)

u(t) y(t)
L2(q)

11
Department of Mechanical Engineering, MIT H. Harry Asada

Furthermore, xk (t ) can be computed recursively.

K Kq 1
x1 (t )  L1 ( q)u(t )  u(t )  u(t )
qa 1  aq 1

x1 (t )  ax1 (t  1)  Ku(t  1)

x1 (t )  ax1 (t  1)  Ku(t  1)

1  aq q 1  a
x2 (t )  x1 (t )  x1 (t )
qa 1  aq 1

x2 (t )  ax2 (t  1)  x1 (t  1)  ax1 (t )

x2 (t )  ax2 (t  1)  x1 (t  1)  ax1 (t )

xk (t )  axk (t  1)  xk 1 (t  1)  axk 1 (t ) (21)

Recursive Filters

12