Scribd
Upload
61 views

2.160 Identification, Estimation, and Learning Lecture Notes No. 1

Uploaded by

Cristóbal Eduardo Carreño Mosqueira
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
61 views

2.160 Identification, Estimation, and Learning Lecture Notes No. 1

Uploaded by

Cristóbal Eduardo Carreño Mosqueira
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

2.

160 Identification, Estimation, and Learning


Lecture Notes No. 1
February 8, 2006
Mathematical models of real-world systems are often
alone.
ifi
Figure by MIT OCW.
Figure by MIT OCW.
too difficult to build based on first principles
System Ident cation;
Let the data speak about the system.
Image removed for copyright reasons.
HVAC
Courtesy of Prof. Asada. Used with permission.
Physical Modeling
1. Passive elements: mass, damper, spring
2. Sources
3. Transducers
4. Junction structure
Physically meaningful parameters
(
(
m m 1
s G ) =
s Y ) s b + s b +L+b
=
0 1 m
s
n
(s U ) + s a
n 1
+L+a
1 n
, a
i
=a
i
(M , K B )
( , b
i
= M b , K B )
i
1
System Identification
Input u( t) Output y( t)
Black Box
m
Y (s ) s b + s b
m 1
+L+b
=
0 1 m
G (s ) =
U (s ) s
n
+ s a
n 1
+L+a
1 n
Physical
modeling
Comparison
Pros
1. Physical insight and knowledge
2. Modeling a conceived system
before hardware is built
Cons
1. Often leads to high system order
with too many parameters
2. Input-output model has a
complex parameter structure
3. Not convenient for parameter
tuning
4. Complex system; too difficult to
analyze
Black Box
Pros
1. Close to the actual input-output
behavior
2. Convenient structure for
parameter tuning
3. Useful for complex systems; too
difficult to build physical model
Cons
1. No direct connection to physical
parameters
2. No solid ground to support a
model structure
3. Not available until an actual
system has been built
2
Introduction: System Identification in a Nutshell
y(t )
FIR
u(t )
b
3
b
2
b
1
Finite Impulse Response Model
t
( ( ( ( t y ) = t u b 1) + t u b 2) + +b t u m )
1 2 m
]
T m
Define :=[b , b , , b R unknown
1 2 m
(
T m
(t ) :=[ t u ), 1 t u ), 2 , t u m )] R known ( (
Vector collectively represents model parameters to be identified based on observed data
y(t) and (t ) for a time interval of 1 t N .
Observed data: y ( ), 1 , y (N )
T
(
This predicted output may be different from the actual y(t) .
Find that minimize V
N
()
Estimate Estimation t y ) = (t )
N
1
V
N
() =

( t y ) t y ))
2
( (
N
t = 1

=avg minV ()
N

dV
N
()
=0
d
N
}
1
(
T 2
V
N
() =

( t y ) (t ))
N
t = 1
N
2
(
T

( t y ) (t ))() =0
N
t = 1
N N
(
T

t y )(t ) =

( (t ))(t )
t = 1 t = 1
3
N N

T
(
(

((t ) (t )
(
=

t y )(t )
t =1 t =1
=

R
N
N

1
(
N
=R
N
t y )(t )
t =1
Question1 What will happen if we repeat the experiment and obtain

again?
N

Consider the expectation of


N
when the experiment is repeated many times?

Average of
N
Would that be the same as the true parameter ?
0
Lets assume that the actual output data are generated from
(
T
( t y ) = (t ) + t e )
0

0
is considered to be the true value.
Assume that the noise sequence {e(t)} has a zero mean value, i.e. E[e(t)]=0, and has no
correlation with input sequence {u(t)}.

N N
1
(
1 T
(
N
=R
N
t y )(t ) =R
N
|( (t ) + t e ))(t )|
t =1
N
1
|
T
0
t =1
N
1
) ( =R
N

(t ) (t ) |
|
+R
N
( t e t )
0
\t =1 . t =1
R
N
N

N

0
=R
N
( t e t )

1
) (
t =1
Taking expectation
N N

1
) (
1
E [ ] =E

R
N
( t e t )
(
(
=R
N
(t ) E [ t e )] =0
N 0
(
t =1 t =1
Question2 Since the true parameter is unknown, how do we know how close
0

0

N
will be to ? How many data points, N , do we need to reduce the error to a
N 0
certain level?
4
Consider the variance (the covariance matrix) of the parameter estimation error.
T

N
)(

P
N
=E [(

N
) ]
0 0
N N
T
1
) (
1
) (
|
(
=E

R
N
( t e t )
|
R

( s e s ) | (
N

t = 1 \ s = 1 . (

N N
1
) ( ) (
T
=E

R
N
( s e t e t ) (s )R
1
(
N
(
t = 1 s = 1
N N
1

) ( ) (
T
=R
N

( E t | s e t e )| (s )
(
(
R
1
N
t = 1 s = 1
Assume that {e(t)} is stochastically independent
| ( ) (
E [ s e t e )] =0 t s
s e t e E )| =

( ) (
2

E [e (t )] = t =s
N
1

T
(
1 1
Then
P
N
=R
N

(t ) (t )
(
R
N
= R
N
t = 1
As N increases, R
N
tends to blow out, but R
N
/N converges under mild assumptions.
N
1
T
lim

(t ) (t ) = lim
1
R
N
=R
N
N
t = 1
N
N
1

1
For large N , R R N , R
N
R
N
N
0


N
P
N
=

R
1
for large N .
N
5
I. The covariance P
N
decays at the rate 1/N.
1
Parameters approach he limiting value at the rate of
II. The covariance is inversely proportional to
N

P
N

R magnitude
(
r
11
K
m
r
1
M O M
m
r
1
K
mm
r

(
(
(

R =
N
N

t u ( t u i ) ( j) r
ij
=
t 1 =

large.
IV. The covariance does not depend on the average of the input signal. Only the
second moment

N
What will be addressed in 2.160?
A) How to best estimate the parameters
What type of input is maximally informative?

0
may be accelerated if we design inputs such that R is III. The convergence of to
Informative data sets
Persistent excitation
Experiment design
Pseudo Random Binary signals, Chirp sine waves, etc.
How to best tune the model / best estimate parameters
How to best use each data point
Covariance analysis
Recursive Least Squares
Kalman filters
Unbiased estimate
Maximum Likelihood
6
B). How to best determine a model structure
How do we represent system behavior? How do we parameterize the model?
i. Linear systems
FIR, ARX, ARMA, BJ,..
Data compression: Laguerre series expansion
ii. Nonlinear systems
Neural nets
Radial basis functions
iii. Time-Frequency representation
Wavelets
Model order: Trade-off between accuracy/performance and reliability/robustness
Akaikes Information Criterion
MDL
7

You might also like