1

C
h
a
p
t
e
r

7
Development of Empirical Models
From Process Data
In some situations it is not feasible to develop a theoretical
(physically-based model) due to:
1. Lack of information
2. Model complexity
3. Engineering effort required.
An attractive alternative: Develop an empirical dynamic
model from input-output data.
Advantage: less effort is required
Disadvantage: the model is only valid (at best) for the
range of data used in its development.
i.e., empirical models usually dont extrapolate very
well.
2
Simple Linear Regression: Steady-State Model
As an illustrative example, consider a simple linear model
between an output variable y and input variable u,
where and are the unknown model parameters to be
estimated and is a random error.
Predictions of y can be made from the regression model,
C
h
a
p
t
e
r

7

y
1 2
y u = + +
1

1 2

(7-3) y u = +
where and denote the estimated values of
1
and
2
,
and denotes the predicted value of y.
1

1 2
(7-1)
i i i
Y u = + +
Let Y denote the measured value of y. Each pair of (u
i
, Y
i
)
observations satisfies:
3
( )
2
2
1 1 2
1 1
(7-2)
N N
i i
i i
S Y u
= =
= =

The least squares method is widely used to calculate the
values of
1
and
2
that minimize the sum of the squares of
the errors S for an arbitrary number of data points, N:
Replace the unknown values of
1
and
2
in (7-2) by their
estimates. Then using (7-3), S can be written as:
2
1
where the -th residual, , is defined as,

(7 4)
N
i
i
i
i i i
S e
i e
e Y y
=
=

The Least Squares Approach

C
h
a
p
t
e
r

7
4
The least squares solution that minimizes the sum of
squared errors, S, is given by:
( )
1
2

(7-5)
uu y uy u
uu u
S S S S
NS S

( )
2
2

(7-6)
uy u y
uu u
NS S S
NS S

where:
2
1 1
N N
u i uu i
i i
S u S u
= =


1 1
N N
y i uy i i
i i
S Y S u Y
= =


The Least Squares Approach (continued)
C
h
a
p
t
e
r

7
5
C
h
a
p
t
e
r

7
Least squares estimation can be extended to more general
models with:
1. More than one input or output variable.
2. Functionals of the input variables u, such as poly-
nomials and exponentials, as long as the unknown
parameters appear linearly.
A general nonlinear steady-state model which is linear in the
parameters has the form,
1
(7-7)
p
j j
j
y X
=
= +

Extensions of the Least Squares Approach

where each X
j
is a nonlinear function of u.
6
C
h
a
p
t
e
r

7
The sum of the squares function analogous to (7-2) is
2
1 1
(7-8)
p
N
i j ij
i j
S Y X
= =
| |
= |
|
\ .

which can be written as,
( ) ( )
(7-9)
T
S = Y - X Y X
where the superscript T denotes the matrix transpose and:
1 1

n p
Y
Y
(
(
(
(
= =
(
(
(
(


# # Y
7
11 12 1
21 22 2
1 2
p
p
n n np
X X X
X X X
X X X
(
(
(
=
(
(
(

"
"
# # #
"
X
C
h
a
p
t
e
r

7
The least squares estimates is given by,

( )
1

(7-10)

=
T T
X X X Y
providing that matrix X
T
X is nonsingular so that its inverse exists.
Note that the matrix X is comprised of functions of u
j
; for
example, if:
2
1 2 3
y u u = + + +
This model is in the form of (7-7) if X
1
= 1, X
2
= u, and
X
3
= u
2
.
8
C
h
a
p
t
e
r

7
Simple transfer function models can be obtained graphically
from step response data.
A plot of the output response of a process to a step change in
input is sometimes referred to as a process reaction curve.
If the process of interest can be approximated by a first- or
second-order linear model, the model parameters can be
obtained by inspection of the process reaction curve.
The response of a first-order model, Y(s)/U(s)=K/(s+1), to
a step change of magnitude Mis:
( )
/
(1 ) (5-18)
t
y t KM e

=
Fitting First and Second-Order Models
Using Step Tests
9
C
h
a
p
t
e
r

7
0
1
(7-15)

t
d y
dt KM
=
| |
=
|
\ .
The initial slope is given by:
The gain can be calculated from the steady-state changes
in u and y:
where is the steady-state change in .
y y
K
u M
y y

= =

10
C
h
a
p
t
e
r

7
Figure 7.3 Step response of a first-order system and
graphical constructions used to estimate the time constant, .
11
First-Order Plus Time Delay Model
-
( )
1
Ke s
G s
s
=
+
For this FOPTD model, we note the following charac-
teristics of its step response:
C
h
a
p
t
e
r

7
1. The response attains 63.2% of its final response
at time, t = +.
2. The line drawn tangent to the response at
maximum slope (t = ) intersects the y/KM=1
line at (t = + ).
3. The step response is essentially complete at t=5.
In other words, the settling time is t
s
=5.
12
C
h
a
p
t
e
r

7
Figure 7.5 Graphical analysis of the process reaction curve
to obtain parameters of a first-order plus time delay model.
13
There are two generally accepted graphical techniques for
determining model parameters , , and K.
Method 1: Slope-intercept method
First, a slope is drawn through the inflection point of the
process reaction curve in Fig. 7.5. Then and are
determined by inspection.
Alternatively, can be found from the time that the
normalized response is 63.2% complete or from
determination of the settling time, t
s
. Then set =t
s
/5.
C
h
a
p
t
e
r

7
Method 2. Sundaresan and Krishnaswamys Method
This method avoids use of the point of inflection
construction entirely to estimate the time delay.
14
C
h
a
p
t
e
r

7
They proposed that two times, t
1
and t
2
, be estimated from a
step response curve, corresponding to the 35.3% and 85.3%
response times, respectively.
The time delay and time constant are then estimated from the
following equations:
( )
1 2
2 1
1.3 0.29
(7-19)
0.67
t t
t t
=
=
These values of and approximately minimize the
difference between the measured response and the model,
based on a correlation for many data sets.
Sundaresan and Krishnaswamys Method
15
C
h
a
p
t
e
r

7
In general, a better approximation to an experimental step
response can be obtained by fitting a second-order model to
the data.
Figure 7.6 shows the range of shapes that can occur for the
step response model,
( )
( )( )
1 2
(5-39)
1 1
K
G s
s s
=
+ +
Figure 7.6 includes two limiting cases: , where the
system becomes first order, and , the critically
damped case.
The larger of the two time constants, , is called the
dominant time constant.
2 1
/ 0 =
2 1
/ 1 =
1

Estimating Second-order Model Parameters

Using Graphical Analysis
16
C
h
a
p
t
e
r

7
Figure 7.6 Step response for several overdamped second-
order systems.
17
1. Determine t
20
and t
60
from the step response.
2. Find and t
60
/ from Fig. 7.7.
3. Find t
60
/ from Fig. 7.7 and then
t
60
is known).
C
h
a
p
t
e
r

7
Assumed model:
( )

2 2
2 1
s
Ke
G s
s s

=
+ +
Procedure:
Smiths Method
calculate (since
18
C
h
a
p
t
e
r

7
19
C
h
a
p
t
e
r

7
Fitting an Integrator Model
to Step Response Data
In Chapter 5 we considered the response of a first-order process
to a step change in input of magnitude M:
( )
( )
/
1
M 1 (5-18)
t
y t K e

=
For short times, t < , the exponential term can be approximated
by
/
1

t
t
e

so that the approximate response is:
( )
1
M
M 1 1 (7-22)

t K
y t K t
(
| |
=
|
(
\ .

20
C
h
a
p
t
e
r

7
is virtually indistinguishable from the step response of the
integrating element
( )
2
2
(7-23)
K
G s
s
=
In the time domain, the step response of an integrator is
( )
2 2
(7-24) y t K Mt =
Hence an approximate way of modeling a first-order process is
to find the single parameter
2
(7-25)

K
K =
that matches the early ramp-like response to a step change in
input.
21
C
h
a
p
t
e
r

7
If the original process transfer function contains a time delay
(cf. Eq. 7-16), the approximate short-term response to a step
input of magnitude M would be
( ) ( ) ( )

KM
y t t S t
t
=
where S(t-) denotes a delayed unit step function that starts at
t=.
22
C
h
a
p
t
e
r

7
Figure 7.10. Comparison of step responses for a FOPTD
model (solid line) and the approximate integrator plus time
delay model (dashed line).
23
C
h
a
p
t
e
r

7
Development of Discrete-Time
Dynamic Models
A digital computer by its very nature deals internally with
discrete-time data or numerical values of functions at equally
spaced intervals determined by the sampling period.
Thus, discrete-time models such as difference equations are
widely used in computer control applications.
One way a continuous-time dynamic model can be converted to
discrete-time form is by employing a finite difference
approximation.
Consider a nonlinear differential equation,
( )
( )
, (7-26)
dy t
f y u
dt
=
where y is the output variable and u is the input variable.
24
C
h
a
p
t
e
r

7
This equation can be numerically integrated (though with some
error) by introducing a finite difference approximation for the
derivative.
For example, the first-order, backward difference
approximation to the derivative at is
where is the integration interval specified by the user and
y(k) denotes the value of y(t) at . Substituting Eq. 7-26
into (7-27) and evaluating f (y, u) at the previous values of y and
u (i.e., y(k 1) and u(k 1)) gives:
( ) ( )
1
(7-27)
y k y k
dy
dt t

t k t =
t
t k t =
( ) ( )
( ) ( )
( )
( ) ( ) ( ) ( )
( )
1
1 , 1 (7-28)
1 1 , 1 (7-29)
y k y k
f y k u k
t
y k y k tf y k u k

= +
25
C
h
a
p
t
e
r

7
Second-Order Difference
Equation Models
( ) ( ) ( ) ( ) ( )
1 2 1 2
1 2 1 2 (7-36) y k a y k a y k bu k b u k = + + +
Parameters in a discrete-time model can be estimated directly
from input-output data based on linear regression.
This approach is an example of system identification (Ljung,
1999).
As a specific example, consider the second-order difference
equation in (7-36). It can be used to predict y(k) from data
available at time (k 1) and (k 2) .
In developing a discrete-time model, model parameters a
1
, a
2
,
b
1
, and b
2
are considered to be unknown.
t t
26
( ) ( )
( ) ( )
1 1 2 2 3 1 4 2
1 2
3 4
, , ,
1 , 2 ,
1 , 2
a a b b
X y k X y k
X u k X u k





2
1 1
(7-8)
p
N
i j ij
i j
S Y X
= =
| |
= |
|
\ .

This model can be expressed in the standard form of Eq. 7-7,
1
(7-7)
p
j j
j
y X
=
= +

The parameters are estimated by minimizing a least squares

error criterion:
by defining:
C
h
a
p
t
e
r

7
27
Equivalently, S can be expressed as,
( ) ( )
(7-9)
T
S = Y - X Y X
where the superscript T denotes the matrix transpose and:
C
h
a
p
t
e
r

7
1 1

n p
Y
Y
(
(
(
(
= =
(
(
(
(


# # Y
The least squares solution of (7-9) is:
( )
1

(7-10)

=
T T
X X X Y