Nonlinear Parameter Estimation: a Case Study Comparison

The literature abounds with the application of optimization methods for estimating model parameters in equation systems. The utility of these methods is frequently demonstrated on pathological examples using simulated data generated from a known model with a random error component and a known statistical distribution. Unfortunately, parameter estimation problems encountered in practice do not have ti advantage. The true model is frequently not known. In fact, one is faced with hs choosing among various candidate models, all of which may be wrong. Moreover, the error structure is generally unknown and must be estimated from the data. Finally, a great deal of mathematical expertise is required to transform the model and select meaningful starting guesses before parameter estimation can be successful. In order to demonstrate the difficulties of parameter estimation in the industrial environment and the limitations of existing methods, a parameter estimation problem formulated by the Dow Chemical Company is presented and solved. This test problem consists of a stiff differentiallalgebraic (DAE) model that describes complex kineticsand requires the estimation of nine parameters from batch reactor data. Here the model was inadequate to describe the data, the error structure was not specified and the starting guesses led to a nontrivial optimization problem. The Dow parameter estimation problem was distributed in 1981to 165 researchers as a followup to the 1980IWCAPD conference. Of those researchers, eleven agreed to apply their methodologies and expertise to this problem. However, only five acceptable solutions were finally submitted. Here we present and compare these results. Each solution was obtained using different strategies. In most cases the form of the model was also changed to accommodate the algorithms used and to ease the solution procedure. Therefore, while this case study does not present a direct numerical comparison of algorithms, it does offer guidelines and insight towards the solution of difficult parameter estimation problems.

L. T. BIEGLER and J. J. DAMIANO

Chemical Engineering Department Carnegie-Mellon University Pittsburgh, PA and

G. E. BLAU
Dow Chemical Company Midland, MI

SCOPE
Parameter estimation arises in fitting models containing several unknown parameters to experimental data through adjustment of these parameters. Model formulation is not a unique process; many different formulations may be used to fit the data and optimize model parameters. Of particular concern are formulations that are sufficiently accurate to represent physical or chemical phenomena and also are amenable to reasonably efficient computer solutions. Here we consider a parameter-estimation problem formulated by the Dow Chemical Company in order that current parameter estimation methods can be compared on an industrial kinetic parameter estimation problem with real data. The model consists of nonlinear differential and algebraic equations and is stiff over a wide range of parameter values. Thus one must choose a reliable model solver before parameter estimation can begin. Several methods for solving stiff ODE and DAE systems are available (see, e.g., Carnahan and Wakes, 1981). Most of those used in this case study are based on Gears method. After a model is proposed and solution techniques are chosen, an objective function that determines the goodness of fit must be selected. From maximum likelihood analysis, several alternative
J J Damfano
IF

currently with Standard 011of Ohlo, Cleveland, Ohlo

objective function forms can be chosen, depending on our assumptions of the error structure of the data. These can range from simple least-squares functions to fairly complex nonlinear functions that incorporate general unknown covariance of the measurements and heteroscedastic ermrs (see e.g., Bard, 1974). The final step in parameter estimation is to optimize the model parameters and any unknown statistical parameters in the objective function. The optimization problem formed by the model and the objective function generally requires repeated and expensive solution of the ODE or DAE system. An efficient algorithm should minimize the number of model and function evaluations and still converge easily to the solution. Generally, these algorithms require gradient information and some,approximation to the second derivative matrix. For simple and weighted leastsquares objective functions, the Gauss-Newton and LevenbergMarquardt algorithms provide the second derivativeinformation quite easily. Otherwise, quasi-Newton updating algorithms such as BFGS (Gill et al., 1981)can also be applied. Five researchers have submitted solutions to the parameterestimation problem described below. After the strategies that were used are outlined, the solutions are compared in terms of accuracy and efficiency.

AlChE Journal (Vol. 32, No. 1)

January, 1986

Page 29

CONCLUSIONS AND SIGNIFICANCE

A parameter-estimation problem formulated by the Dow Chemical Co. to study a batch reactor system was solved by five independent mearchers. After pmenting the differentiallalgebraic equation (DAE)model and a suggested form for the objective function, six solution strategies are described and presented. Al of these solutions used implicit ODE or DAE solvers; four inl vestigators used variationsof Gear's method. Except for an unsuccessful solution presented for illustration, only Gauss-Newtontype methods and successive quadratic minimization were used for optimization. Based on the results, it is clear that the most important consideration for parameter estimation is formulation of a simple, yet realistic process model. Guidelines for this should include the following points for more efficient solution: 1. Eliminate all dependent equations in the proposed model. T i leads to a model that is more efficiently handled by the ODE hs solver. 2. Eliminate as many unn-ry model parameters as p&ble. This leads to a smaller and better behaved optimization problem.
3. If the error structure is not known, use an objective function

based on maximum likelihood that allows direct application of

Gauss-Newton-type methods. 4 Determine an initial set of parameter estimates through . some physical insight into the model parameters. This not only leads to a closer starting point but also avoids any u n n v stiffness problems due to starting p i t far from the optimum. ons All of the solutions required careful attention by the researchers to several computational difficulties. None of the investigators was able to solve the problem automatically in one go, regardless of the sophistication of the software. Judging from the very wide exposure the problem received, it appears that discovery and improvement of parameter-estimation algorithms remains a very necessary and fruitfularea for research and development. Finally, the solutions presented here do not offer a direct a m parison among model solving and optimization algorithms, behs cause they relied on different problem formulations. Instead, ti study illustratesthe importance of model formulation and insight in tackling difficult parameter-estimation problems as well as the limitations of current methods.

PROBLEM DESCRIPTION

The parameter estimation problem is based on a kinetic model of an isothermal batch reactor system. The reactions occur in an anhydrous, homogeneous, liquid phase catalyzedby a completely dissociated species. The reacting species have been disguised for proprietary reasons, The desired reaction is given by: HA 2BM AB MBMH where AB is the desired product. The reaction is initiated by adding the catalyst QM to a batch reactor containing the two miscible reactants with reactant BM in excess. The catalyst QM is initially assumed to be 100% dissociated to Q+ and M - ions. The following mechanism is proposed to describe the reaction: Slow Kinetic Reactions

[ABM- ][If'] [(HABM)NI The anionic species may then be represented by:

K,

[MBM-]

(KI

KJMBMH] + [H'I)

M-

+
A-

BM-k_Ikl-7MBM-

+ BM ks=%4BMMaterial balance equations for the three reactants in the slow kinetic reactions yield:

M-

+ A B C k - - 3)3=ABM-

Rapid Acid-Base Reactions

MBMHeKi*MBM-

+ H'

H A ~ K ~ = A + H' HABMeKseABM- + Hi
In order to devise a model to account for these reactions, it is first necessary to distinguishbetween the overall concentration of a species and the concentration of its neutral form. Overall concentrations are defined for three components based on neutral and ionic species.

-drBM1 dt

-kJM-][AB] -k,[M-][BM]

kkJABM-I

(d)

k-,[MBM-l

- k,[A-l[BMl

(e)

d[ABl = at

-k,[M-J[AB]

+ k-JABM-]

[MBMH]

[(MBMH),]

[MBM-I

[HA1 = [(HA),] + [A-I [HABMI = [(HABM),] + [ABM-I

where [ ] denote concentration of the species in gmol/kg. By assuming the rapid acid-base reactions are at equilibrium, the equilibrium constants K,,K2,K3can be defined a follows: s

From stoichiometry, rate expressions can also be written for the total species:

d[MBMH1 dt

k J M - ] [ B M ] - k-,[MBM-]
= k,[A-][BM]

(g)

'IHA1

dt

Page 30

January, 1986

AlChE Journal (Vol. 32, No. 1)

+ k,[M-][AB] -

k-JABM-1

(i)

An electroneutrality constraint gives the hydrogen ion concentration [ H + ] as:

[H'] + [Q']= [ M - ] + [ M B M - ] + [A-] + [ A B M - ] (j) The measured values came from actual kinetic data obtained from three isothermal runs in a batch reactor. The temperatures were set at 40,67, and 100O C . The data p m n t e d for parameter estimation represent a pretreatment of the actual data. Four component concentration vs. time profiles are presented, but only three species HA, HABM, and AB were actually measured. These three measurements were normalized so that: [HA]

[HABM]

[AB]

initial [ H A ]

Normalization adjustments resulted in concentration changes of less than ten percent, A fourth species, [BM],was derived from:

The nine parameters form the vector, 0, given by: 0 = [al, E l , ( w 2 , E 2 , a - 1 , E - l , K ~ , ~ 2and the predicted concentrationsform ,~~] the vector, y , given by: y = [ H A , B M , H A B M , A B , M B M H , M - , H + ,A-,ABM-,MBM-1. The model initial conditions and the initial parameter estimates are given in Appendix A. As a follow-up to the 1980 FOCAPD conference, the above problem was distributed to its 165 participants in 1981. Eleven of these researchers agreed to tackle this problem and five groups submitted acceptable solutions. Before describing these solutions and the methodologies behind them, we briefly summarize the algorithms that were used. We classify these in terms of solving the DAE model, choosing an appropriate objectivefunction from maximum likelihood, and optimizing for the parameters.
SOLVING THE DAE MODEL

[BW

initial [HABMJ - 2[AB]

However, this relationship is only true if [AB] [MBMH], an unmeasured species. From the model, this relation holds at long times. In all of the runs, the initial catalyst concentration was 0.0131 gmollkg. The data sets are given in Appendix A. Although many data are present, they are insufficient to estimate all of the parameters present in the proposed model and a few assumptionswere made. First, it was determined from other data that the equilibrium constants did not vary with temperature over the interval 40-100C. Also, based on the similarities of the reacting species, we assume:

k,
k-,
=

k, 112 k-,

Based on these last two assumptions, three rate constants, kl, kz, k - I must be estimated. Each of these can be fitted with adjustable model parameters, assuming an Arrhenius temperature dependence. That is:

Numerous methods have been developed for the solution of initial value ordinary differentialequations. Also, thorough analyses of these methods have led to a fairly complete classification according to stability, accuracy and performance. As a result of these comparisons, the most commonly used ODE IV methods are:

kl =

kL= aIexp(-E,/RT) exp( - E J R T )

a2exp( - E,/RT)

1) Explicit Runge-Kutta and linear multistep methods for nonstiff systems, and 2) Semi-implicit Runge-Kutta and the Gear methods for stiff problems.
Runge-Kutta Methods

. = h

where R is the gas constant, T is the reaction temperature in Kelvins, and the parameters, a,E , represent the preexponential factor and activation energy, respectively, for the appropriate rate constant. The model can therefore be expressed mathematically as six differentialequations and four algebraic equations. The letter labels for the following equations refer to the corresponding kinetic and equilibrium expressions derived above.

Given a differential model of the form:

the Runge-Kutta (R-K) techniques of integration use formulas defined by:

i- 1

with

wheren = step, counter h = stepsize T order of Runge-Kutta method

are The coefficientsaii,bi,ci determined by matching the above series with a Taylor series expansion of desired order. When ail = 0 for j , 1 a the qi can be calculated in order and the R-K method is called explicit. When ai,= 0 for j > i, and aij # 0

AlChE Journal (Vol. 32, No. 1)

January, 1986

Page 31

for j = i, method is called semi-implicit. Finally, when aij# 0 the for j > i, the method is termed implicit. While implicit R-K methods have a drawback in that the qi must be.solved iteratively, semi-implicit methods can be linearized and sohed with only one iteration per step. Numerous methods exploit this strategy (see, e.g., Carnahan and Wilkes, 1981).Chan et al., 1978).Prokopakis and Seider (1981), and Michelsen (1976) give accounts of semi-implicit Runge-Kutta schemes.
Linear Multistep Methods

e = vector of adjustable model parameters E = matrix partitioned as [I/OIT so that Eqs. 14 and 16 are equivalent
For DAE systems the analog of Eq. 15has the Jacobian: E

aF

aY

Linear multistep methods for numerical integration of differential equations have the following form:

j-1

j-0

with coefficients, a,, and bni, determined by postulating the solution as an interpolating polynomial. Gear (1971)proposea a family of formulas that are "nearly" Astable. Hence, formulas of order Y are obtained with 11 = r and l2 = 0, i.e.:
yn = Canj~n--r
j= 1

+ hbnofn

(13)

Petzold (1982) has shown that solving DAE systems may lead to far more difficulties than solvinga standard ODE model. From a practical standpoint, enforcing convergence by reducing the step size leads to an ill-conditionedJacobian because E is singular for DAE systems. Moreover, using linear, nonhomogeneous systems as an example, Petzold describesa number of problems with error estimates, termination criteria, and convergence failures when discontinuities in the forcing function are present. She points out that DAE systems can have many similaritiesto stiff systems and, in certain cases, have errors that do not vanish as the step size goes to zero. While the example problem described in this paper does not fall into this problem class, Petzold advises extreme caution in solving nonlinear DAE systems. Many of her suggestionsfor dealing with the above problems have been incorporated into her code, DASSI,, which is described later.
MAXIMUM LIKELIHOOD METHOD

These formulas, called backward difference formulas, are implemented in the popular Gear, Episode, and LSODE (Byrne, 1981) software packages. The desirable stability property they have, which has been termed stiffly stable, allows the numerical algorithm to track the model components accurately with larger step sizes after the transient or stiff region has passed. A discussion of backward difference formulas can be found in Carnahan and Wilkes (1981).
Systems of DifferentiallAlgebraic Equations (DAE)

The choice of the objective function used in parameter estimation should incorporate the specific error structure of the experimental data if the best fit of the data is to be achieved. The likelihood function provides a general formulation for the oojective function by means of which many typesof error relationships can be represented. To apply the likelihood function assume a relationship of the
form:
Z", =

Models composed of nonlinear algebraic and stiff ordinary differential equations of the form:

"I

d Y 4 =0 where y = vector of state variables yn = subvector of state variables to be solved from differential equations. 0 = vector of adjustable parameters
often occur in chemically reacting systems as well as in other fields such as circuit analysis or transient flowsheet simulation. Note that the system is coupled, with variables from the algebraic equations needed to compute the state variables yoand vice versa. Gear (1971) stated that for stiff equations, a backward difference formula should be used to discretize the differential equations. The algebraic equations could then be combined with the discretized equations and the resulting algebraic system could be solved by Newton's method. The discretized ODEShave the form of Eq. 13:
4

where z,,, = measured value for component 1 Y,,~(B) = computed value of comoonent j from model in uth experiment 0 = vector of adjustable model parameters Euj = residual error, assuming the model is correct Here, the measured variables, z, have experimental errors associated with their values. The form of these experimental errors is given by a covariance matrix; the diagonal elements of the matrix represent the independent variances of the measured variables and off-diagonal terms represent estimates of correlated or dependent error relationships. If it is assumed that the measured variables, z , have a Gaussian probability distribution, p ( z ) , with mean q , and covariance V , exp d asN(v,V), then:
p ( z ) = ( ~ T ) - " / ~ ( V det-li2(V) exp[-l/2(z )

- t~)~V-'(- q)] z
(17)

y. - Can,Yn-i - hb*,d"= 0
j- I

(15)

The Jacobian of this equation, needed in the Newton-Raphson where J = afn/aYn. iteration, is given by: I - hbnoJ(yn), The combined system of algebraic and differential equations can be written in the general form:
EY' = F(Y,B,t)

for a single experiment in which m variables are measured with a covariance V between these measured variables. The experimental error vector, eu. defined as the difference, E ( z - t ~ )thus has , the normal distribution, N ( 0 , Vu). If n experiments are carried out, in each of which m variables are measured with covariance V, between the variables in experiment u , but with no correlation between measured variables in the different experiments, then:

(16)

where F = combined f and g equations y' = derivatives of y

Page 32

January, 1986

AlChE Journal (Vol. 32, No. 1)

If one assumes that the proposed model is correct and that the parameters e are not far from the optimal parameters, the residual vector, t,, can be substituted into Eq. 18as an adequate representation of the measurement errors. This leads to the likelihood function:

log L

-nm/2[log(27r)

1 1

u=lj-l
n
/

The maximum likelihood method seeksthose values of the adjustable model parameters 6 for which the probability of obtaining ' values is maximized. To solve our parameter estimationproblem, the following error structure for the measured values was assumed in the Dow problem statement. Observation errors at different time points t,, are uncorrelated The errors at t, are normally distributed with zero mean and a covariance matrix, V, It is assumed the measured variables are independent so that the covariancematrix is diagonal It is assumed the error is heteroscedastic and the diagonal ele ment of V, corresponding to component j , uui, is a power transformation depending on the magnitude of the expected value of zuj. Here, the model prediction, yu,,for time t, and component 1 1s substituted for the expected value.

This objective function was suggested by Reilly et al. (1977).In this case study, several participants modified this objective function even further. If we assume that the errors in each experiment are Gaussian and distributed with the same diagonal covariance matrix but are not heteroscedastic, then all yj's are set to zero and Eq. 23 simply becomes: log L(0)
=

mn/2[log(nl27r)

-1 1
m n

a-1

u-1

Thus,one must maximize Eq. 24 for parameters 8 and estimate

v, from Eq. 22 using the optimal parameters, 0 (Note that by max-

imizing log L the residuals should be smaller than the actual errors, since the parameters 0 were chosen to make the residuals as small as possible. Because of this, some bias is introduced. Assuming a general unknown covariance matrix leads to an expression smlr to Eq. 24 (Bard, 1974, p. 66). iia Finally, if we assume that each measured value has Gaussian errors with the same known variance, then the likelihood function reduces to a simple residual sum of squares which can be minimized to fit the observed data, i.e.:

where u ;

yi = yi =

variance coefficient for component j 0, absolute error constant throughout experiment 2, relative error constant throughout experiment

u=l

where E,

residual vector for uth experiment

Substitutingthis diagonal covariancematrix into the likelihood function yields:

It should be noted that the above derivations apply only to experimental runs with no missing data. This is not true in the example considered here and different approaches for missing entries were used in the solutions described below. Analyses of this problem can also be found in Bard (1974) and Stewart and Sorensen (1981).
OPTIMIZATION ALGORITHMS FOR PARAMETER ESTIMATION

u-lj-1

where n = total number of measurements for a component m = number of measured components yt,j = model prediction which is a functin of 0 Note that the knowledge of the error structure of the data is instrumental in the derivation of the final model and objective function equations. For some functions the unknown statistical parameters are not present, and in other instances they can be removed by simple transformation at the optimum. In this particular problem, the likelihood function is to be maximized for the 0, o,and y parameters. At this maximum,
-=

A number of gradient-based algorithms have been developed which exploit least-squares structure and significantly reduce the computation necessary to find the optimal parameters. The important stepsof these algorithms are the calculation of a search direction and determination of the step length to take along this direction. The iteration step for the optimization variables, 0, can be expressed as:
where 6 H

aiog L

Using this relation results in solving explicitly for Wi:

g = One can approximate Hessian information by exploiting the form of the objective function. In particular, Gauss-Newton and Marquardt methods take advantage of the structure of several functions derived from maximum likelihood. For example, the simple least-squares function is given by:

step length Hessian matrix of the objective function or its approximation gradient of objective function

u=l

yUI..'
U=l

where cuj is the model error, zUj- yuj. SubstitutingEq. 22 into Eq. 21 yields:

or

AlChE Journal (Vol. 32, No. 1)

January, 1986

Page 33

where yu

= =

z,

vector of model predictions, yai, at time u vector of measured values at time u

with the gradient represented by:

or
g(9)
=

Here {is a small positive tolerance. Other methods for handling nondescent directions after spectral decomposition are given in Bard (1974). Finally, several quasi-Newton methods are availablefor unconstrained minimization (see Gill et al., 1981). These methods assume nothing about the structure of the objective function, but instead approximate second derivatives from past changes to the gradient vector. While these methods ensure positive definiteness of H, they may become unstable if the problem is ill-conditioned. These methods were only tried in the attempted solution and yielded unfavorable results.
Computing the Gradient

-&.'$
U=l

and the Hessian obtained by:

with

Note that the first term in the Hessian formulation contains the residual vector, e,. Assuming that the residual is small, the Hessian can be approximated only by the first derivatives contained in the second term of the Hessian. Using this Hessian to find a , search direction, s, leads to the Gauss-Newton metbod:

Z J ~ J- - CJ'e
n
n
S,
u- 1

u-1

However, problems can occur when solving the least squares problem if the matrix, J , does not have full rank. The Levenberg-Marquardt method proposes a nonnegative addition to the Gauss-NewtonHessian approximation to insure a positive definite Hessian and a descent search direction. The alternative Hessian is proposed as:

and the search direction is chosen to satisfy:

/ n
\

In parameter estimation, gradient calculations needed in the optimization algorithm usually make up the most expensive step. Here, the adjustable model parameters, 8, do not appear explicitly in the least-squaresor likelihood function. Thus, the calculation of the gradients of the objective function with respect to the model parameters cannot be calculated analytically if the model predictions, y, are not solved explicitlyfrom the model. Two alternative approaches are available to calculate the gradients. The first alternative is to use either forward or central difference formulas. In either case, the choice of the finite-difference perturbation size is important. Its value must be chosen to minimize the effects of truncation error and computation error. Truncation error is due to the neglected higher-order terms in the Taylor series expansion and is proportional to the perturbation size for the forward difference formula, and perturbation size squared for the central difference formula. The computation error is due to errors in computing the function values used in the gradient approximation. This type of error is proportional to the inverse of the perturbation size for both finite-differenceformulas. Therefore, it is important that the finite-differenceinterval be kept small to control the truncation errors, but not too small to lead to roundoff errors. Since forward differencesrequire half the number of function evaluations needed for central difference approximations, the former are more efficient but less accurate. For instance, forward differences should not be used if the change in perturbed function values is small for a given step size, since computation errors will overshadow the gradient calculation. Thus, as the magnitude of the gradient approaches zero at the solution of an unconstrained problem, a switch to central differences may be required. The second alternative for gradient calculation is through sensitivity analysis. Here, one needs to obtain dy/d8, where the values, y, are the model predictions used in the objective function. We differentiate both sides of Eq. 16 with respect to 8, and using the chain rule we have:

The choice of p is crucial in order to insure a descent direction. Bard (1974) summarizes Marquardt's original algorithm for the selection of p. Another way of insuring descent directions with Gauss-Newton type methods is through rotational discrimination (Fariss and Law, 1979), Here the H matrix undergoes a spectral decomposition to:

Interchanging the order of differentiation yields:

H = RAR' where A is a diagonal matrix. Defining a new coordinate transformation:

s - RTs

= Rrg
- y.g I !

leads to each element of s given by:

Si =

In order to obtain c3yyl8 one must solve the DAE model together with the set of simultaneous differential and algebraic equations given by Eq. 25, with the quantities aFylaB and d F / h determined by simple differentiation. Equations 25 are called sensitivity equations and the values %/a% are called sensitivity coefficients. Bard (1974) illustrates the use of these equations to calculate the gradient for a nonlinear differential equation model.
SOLUTIONS OF PARAMETER-ESTIMATION PROBLEM

u, =

A,-' if X

> <, or

v i = 0, otherwise

Five research groups provided solutions to the parameter-estimation problem formulated in the first section. All of the solu-

Page 34

January, 1986

AlChE Journal (Vol. 32, No. 1)

tions were obtained by reformulating the model or the parameters. In three cases, different initial parameter estimates were used for the optimization. Needless to say, these modifications preclude a straightforward comparison of the model solution and optimization algorithms. However, some conclusions can be drawn from this study and much can be gained from the problem-solvingtechniques used for this parameter estimation problem. The investigators are listed with their affiliationsin Table 1. A summary of results is given in Table 2. However, since only three solutions r e g r d on all four measurements, only these could be compared with a common objectivefunction. Instead, a transformation of the final parameter vectors for each solution is given in Table 3. The explanation for this transformation will be d i s c d after describing the solutions. Due to the different nature of each solution, the most straightforward comparison can be made simply by plotting the model, solved with each set of final parameter values, against the three data sets. However, for the sake of brevity we have chosen to present only the first (and largest) data set in Figures 2 through 8, with a plot of the model with initial parameter estimates in Figure 1. It should be cautioned that systematic errors may be observed in the fits of this data set due to the need to compensate for opposing errors in the other sets. Hence, while Figures 2 through 8 give a qualitative comparison of the five solutions, they may not be totally objective. Finally, the optimal parameter values and their confidence intervals are listed in Appendix B. Due to the different nature of the solutions and for the sake of brevity, we refrain from reporting and interpreting final covariance matrices on a consistent basis. Instead we describe the characteristics of the solutionsand model more qualitatively. More specific details about each solution are available on request from the first author.
Attempted Solution

To provide a measure of the difficulty of this problem, a solution was attempted by the second author using the model, objective function, and starting point stated above. The intention of this exercise was to see how easily the problem could be solved with availablesoftware and no coaxingon the researcherspart. A brief account of the computational difficulties encountered is given below.

The original model consists of a DAE system with six ODEs and four algebraic equations and is stiff for the initial set of parameters. Very few numerical integration packages have the built-in capability of solving DAEs simultaneously. One of these is the DASSL software package developed at the Sandia National Laboratory in Livermore, California. This code uses the stable Gear backward difference corrector formulas to convert the differential equations to algebraic equations. The algebraic and differential equation variables are then solved using a modified Newton method. With this approach, convergence problems can occur in solvingthe model for certain choices of the model parameters picked by the optimization scheme. It was observed, however, that convergence failures occurred less often if tighter tolerances are imposed for each step of the DAE solver. Two optimization strategies, a modified Complex (MC) routine (Box, 1965) and a quasi-Newton (QN) gradient-based strategy were chosen for parameter adjustment. Both algorithms handle parameter bounds easily but do not take advantage of the structure of the objective function. The two different optimization techniques required different model tolerances to handle the limitations of the model solving routine. With the MC routine, convergence problems could be handled by defining a constraint function that was violated whenever the model failed to converge. This prevented the Complex algorithm from using an incorrect value of the objective function but required the algorithm to choose more points. Because the model convergence problems could not be eliminated completely by tightening the error tolerance, a loose error tolerance was used to evaluate more points for a fixed CPU time. For the gradient-based algorithm, however, it was essential for the model to converge as often as possible. Thus, tight relative and absolute error toleranceswere used in order to compute gradients and search directions. If model convergence failure occurred during a forward difference perturbation then the gradient with respect to the perturbed variable was set to zero. This heuristic helped to avoid grossly inaccurate search directions as a result of failed gradient calculations. Before beginning, a few minor changes were made in the formulation of the objectivefunction. To prevent the second term in Eq. 21 from getting too large, the lower bound for the value of the model prediction, yur, was set to 1.0 x Also, since the experimental values of component BM are derived and the other

TABLE SUMMARY 1. OF SUBMITTED SOLUTIONS-

Na of Components

No. of

Investigator (1) M. Caracotsios, W. E. Stewart & J. Sorensen, University of Wisconsin, Madison, WI (2) R. H. Fariss, Monsanto Plastics and Resins, Indian Orchard,
MA

Objective Function Unknown General V, Unknown 7,

Regressed
2

Parameters Optimized
12

Final Model Formulation 6 ODEs

4 Algebraic Eqs.

Unknown Diagonal Vii Unknown yi

17

3 ODEs 4 Algebraic Eqs. 3 ODEs

(3) B. S. Ahn, Korean Advanced Institute of Science and Technology, Seoul, Korea
(4) R. Klaus, T. Rimensberger & D. W. T. Rippin,

Unknown Diagonal V,l,rt = 1

Unknown Diagonal Vtt,r, = 0

4 ODEs

Eidgenoessische Technixhe Hochwhule, Zurich, Switzerland (5) B. Sarup, M. Michelsen & J. Villadsen, Danmarks Tekniske Hojskole, Lyngby, Denmark

Unit Matrix V, (LeastSquares)

3 ODEs

See Amendix B for initial and final Darameter estimates of each investieator
_________

AlChE Journal (Vol. 32, No. 1)

January, 1986

Page 35

TABLE 2.

SUMMARY O~IM~ZATION OF RESULTS

Participants (Refer to Table 1)

(3)
~

(4) Marq. 362.80 656.94 47 144

(5)

(*)

Optimization algorithm

Quadratic
Min. - 1,009.9

Final objective
function value

Rot. Discrim. 753.84 753.84

Marq. 561.72 561.72 17 104 14.30

Marq.
0.233

QN
- 199.9 ( - 295.0) - 199.9 (- 295.0) 55 (115) 637 (290) 459.0 (55.27)

(MC)

Equivalent value, Eq. 23

No. of iterations No. of function evaluations CPU time, min

20 20**

19 38 21.53

24 577 38.35

.**

0.17

Computer
system

VAX
11/780

IBM
3081

Cyber
174/18

CDC
6500

IBM
3033

DEC-20

Attempted solution (Damiano). **Sensitivityequations evaluated once per iteration. *** Not reported due to excessive I /Oat execution.

three measurements were normalized, only two of the four experimental components, H A and HABM, were used. Logarithmic transformations of the kinetic parameters were made for manipulation by the optimization algorithms. Finally, the heteroscedasticity parameters, yi, were held constant at 1.O throughout the optimization runs. The results of the optimizations using M C and QN are displayed in Figures 2 and 3, respectively. As mentioned above, much tighter model tolerances were chosen for QN, with the result that it required much more computational effort than MC. Both algorithms terminated far from the optimum after exhausting too much CPU time. The DAE solver had failure rates of 28 and 12% for MC and QN, respectively. A summary of the results is given in Table 2.
First Solution

for deletion when [HA] was missing. Secondly, the diagonal variance in Eq. 20 was replaced by a full covariance matrix Yi/e with elements: Vuij= ~ , , ~ ~ y , ~ Y i / z y ~ ~Here .courtis normally equal to the parameter wij, but is replaced by 6 if either yui or yuj is , missing. This generalizes the objective function to:

u-1;-lj-I
n m

u-1

u-li=l

where 8{ = eU{/yui%/z. The model parameters were transformed , as follows: log k1 = p , - (1IT - 1/Tb)p, log k2 = p2 - (1/T - 1/Tb)ps log(kl/k-l) = - p 3 / T b - (1/T - 1/Tb)Po lo@,) -p4ITb log(&) = -p51Tb log(K3) = -pelT, Tb 342.15 K T i transformation was chosen to prevent the parameters hs from varying over large orders of magnitude and to reduce the intercorrelation o the parameters in the rate constants. Finally, the f elementsof the measurement covariancematrix were added as regression parameters. The successive quadratic minimization algorithm described in Stewart and Sorensen (1981) was applied here. This approach is a generalization of the Gauss-Newton method that allows for missing data, linear constraints on the parameters, and general unknown covariance. Solution of this problem pnxeeded in two stages. After determining that the HABM and AB measurements were probably not independent, the problem was formulated without heteroscedasticity. After 15 iterations with Marquardts method, the problem converged but the normal equations were ill-conditioned. Closer inspection of the Hessian showed that parameters p , and p , could not be estimated simultaneously. The second stage proceeded from this solution with the heteroscedastic objective function given above and p , fixed. Convergence with successive quadratic minimization was obtained after four additional iterations and the objective was found to be

The first group of investigators also used DASSL to solve the DAE model. To avoid the convergenceproblems discussed above, after some trial runsthe error tolerances were adjusted automatically over the course of the optimization. Also, sensitivity equationsgiven by Eq. 25 were used to calculate the gradient of the objective function with respect to all of the model and statistical parameters. Several modifications were made to the maximum likelihood objective function. First, because of the data dependencies mentioned above, at most two observations were used (HABM and AB) from each time point t,. This number was reduced to one whenever a concentration was missing; [AB] was chosen

TABLE SUMMARY OPTIMAL 3. OF SOLUTION VECTORS

Participants Transformed
Parameters k, (342) kz (342).
E,(X

10-7

E,(X lop3)
E ~ ( low3) X
K1/K3

k-,(342)K31Kz

(1) 1.88 2.73 18.74 18.88 25.67 1.44 9.84

(2) 1.84 2.89 18.48 19.07 26.05 1.44 10.92

(3) 1.94 2.38 18.84 17.87 25.15 1.42 8.52

(4) 2.04 2.68 18.26 18.41 21.90 1.44 9.98

(5) 2.21 2.78 17.84 18.85 25.20 1.43 10.14

Page 36

January, 1986

AlChE Journal (Vol. 32, No. 1)

MOOEL FIT

"/ I
f a
1 . 1 -

A
1 . IY

50

160

150

200

250

SO0

350

400

450

500

.-

.
65:

550

rh=1
Figure 2. Model with optimal parameter estimates from MC attempted solution.

AlChE Journal (Vol. 32, No. 1)

January, 1986

Page 37

independent of pg as well. The parameter Y H A B M converged to its lower bound of zero, while the final YAB was 0.19 f 0.12. Figure 4 shows how this solution fits the experimental data. Second Solution Results from the second solution are shown in Figure 5 . Here the DAE model was transformed by noting that three of the six ODEs are dependent and can be reduced to algebraic equations. The revised model consisted of Eqs. 1 , 4 , and 6, with predictions for yz, y3, and yscalculated by material balances using yl, y4,and ye. Problems were observed in the numerical integration scheme o with ye becoming negative. T eliminate these problems, Eqs. 1 and 6 were divided by y1 and y, respectively. This left the left , hand sides of Eqs. 1 and 6 as d(1og yl)/dtand d(1og y,)/dt, respectively. To avoid additional problems with log yl, Eq. 1was further modified to force the derivative d(1og yl)/dtto zero smoothly for small values of yl. The differential equations were first integrated using the Harwell stiff integrating package, DCOlAD, which uses Gear's backward difference formulas. Then for fixed values , of y1 to ye, the original algebraic equations were solved using y and Eq. 7 as a tear set and subsequently solvingfor yw,y9,and yl0. Here, a tight tolerance was required for the algebraic system to avoid numerical problems with the integration scheme. Also, the model parameters were transformed to: The log ko parameters were calculated relative to a base temperature of 67OC, according to the transformation: log k
=

This participant opted for the above transformation because a values tend to vary over orders of magnitude and their interaction with the activation energy leads to an ill-conditioned problem. In addition, the objective function, Eq. 21 was modified by replacing the model prediction yui, by 112 (zUi+ yJ. Since a diagonal covariance matrix was assumed, missing data were handled simply by using partial summationsin the objectivefunction. The rotational discrimination technique of Faris and Law (1979) was used for optimization. Gradients were calculated using a central difference formula requiring twice as many function evaluations as gradient values. Finally, this investigator also optimized the statistical parameters y and o and noted that the solution is not unique because two linear combinations of the model parameters can be identified. More will be said later about dependency in the model parameters. The solution was found by executingsix passes of the optimization algorithm. In each pass a subset of the kinetic and statistical parameten were allowed to vary, with all parameters varying in the last pass as the optimum is approached. The solution was thus found by careful monitoring and appropriate intervention by the investigator over the course of the convergencehistory. No confidence intervals were reported by this investigator because he felt the assumed error structure was inappropriate for this problem. This is confirmed by his results in Appendix B. Here V,, is about the same value as the other variances but the parameters y, and up that determine it are drastically different.
Third Solution

log k o

( E I0.67558)-

T - 67
T

+ 273

The third investigator reduced the DAE model to three independent ODEs that provide values for yl, y3, and y5. The IMSL routine DGEAR was used to integrate the ODE'S and Eq. 23 was

Page 38

January, 1986

AlChE Journal (Vol. 32,No. 1)

Figure 4. Model wlth optimal parameter estimates from solution (1).

AlChE Journal (Vd. 32, No. 1)

January, 1986

Page 39

changed so that the variance was proportional to the measured value raised to a power, rather than the value of the model prediction. T i substitution set the last term and the denominator of hs the second term in Eq. 23 equal to a constant, which allowed him to optimize a simpler objectivefunction, and still provide a value for the original objective function. The heteroscedasticityparameters, y, were set to unity for the optimization runs and the Levenberg-Marquardt routine from the IMSL library was used to optimize the objective function. Gradients were calculated by finite difference as described in the IMSL manual. A partial summation was used in the objective function to account for the missing data. Moreover, the starting point was altered by changing the initial valueof &from lo- to 10-l5.Evenso, theproblem required 104 function evaluationsbefore it converged. Figure 6 displays the solution found by this investigator.
Fourth Solution

by changing the initial estimate of K2 from 1.0 x lo- to 1.0 x 10-l7.An optimal solution could not be reported with the original objective function. However, ti investigator reported hs good results using an unknown diagonal covariance matrix without heteroscedasticity. The objective function takes the general form given by the second term in Eq. 24. The best results reprted by this investigator are shown in Table 2 and Figure 7.
FifthSolution

This participant reduced the six ODEs and four algebraic equations to four ODES by substituting for the equilibrium relationships and dependent state variables. The model was solved using a semiimplicitthird-order Runge-Kutta method by Michelsen (1976). The optimization routine was developed by Klaus and Rippin (1979). It contains a modified Marquardt algorithm with revisions by Fletcher (1971), a gradient projection technique for handling bounds, and several strategies for treating numerical difficulties. All gradients were calculated by finite difference. Zeroes were substituted for missing data in calculating the objective function. Some problems were encountered in the integration step with initial values of the parameter estimates, but these were resolved

These participants elected to reduce the system of ODEs to only threeby writing total mass balances. Further, using the three equilibrium relations and assuming almost undissociated species and weak acids, they reduced the DAE model to three coupled ODES. These researchers refrained from using the likelihood function based on Eq. 20, stating that the dependencies in the data do not make thischoice suitable. Instead, they used a simple least-squares function and excluded the derived BM measurements from the analysis; a partial summation of this function was used to account for the missing data. Sensitivity equations were formulated and solved simultaneously with the model equations to compute the gradients of the residuals with respect to the model parameters. The IMSL library subroutine, DGEAR, was used for this task, after modification to handle the sensitivity equations. The Harwell library routine (VBOIAD), based on the Levenberg-Marquardt method, was used to minimize the least-squaresproblem. Also, the rate constant parameters were reformulated using a transformation similar to that used in the second solution. Log k , O and (E,/RT,,) (T,&?= 340 K) were the adjustable kinetic parameters and the parameters log (K,IK3)and were given starting guesses of zero. More will be said log (K3/K2)

Figure 6. Model with optimal parameter estimates from solution (3).

Page 40

January, 1986

AlChE Journal (Vol. 32, No. 1)

about this transformation in the following sections. Results for this solution are presented in Figure 8.
SUITABILITY OF THE KINETIC MODEL

From the solutions described above, a number of observations can be made about the kinetic model and how it should be treated for parameter estimation. With their solutions, several investigators submitted very detailed analyses about the suitability of the model. A summary of their comments along with a few other points is given below. First, it is readily seen from the kinetic mechanism that [HA]is predicted by the model to vanish at steady state. The [HA]measurements, however, level off at about 3 - 4 x gmollkg. Thus the errors at long times will be biased by the model and render the error assumptions invalid. The second investigator attempted to remedy this situation by making the second reaction reversible and introducing new parameters for k - z . His solution to the modified model yielded an improved maximum likelihood gmolikg. He also optimum and led to final [HA]of about found the modified model much easier to solve. Clearly the addition of the reverse step helps to remedy the model deficiency, although this step is not apparent from prior knowledge of this reaction. Second, severalinvestigatorsnoted dependenciesamong the kinetic parameters. This not only leads to nonunique optimal solutions but also yields an H matrix that becomes singular. Consequently, one also encounters convergence problems. Through analysis of the Hessian matrix after spectral decomposition, the second investigator noted a two-dimensional dependency among the parameters k-,, K , , Kz, and K3. In Table 3 we present

values for K1/K3,ko-1K3/K2 and the remaining parameters for the five solutions. Although the original optimal parameter values vary a great deal for these solutions (seeAppendix B), the transformed parameters in Table 3 are very similar. The model can also be reduced to deal with a smaller parameter set through physical arguments. The fifth investigator performed part of this reduction by assuming that the small equilibrium constants would lead to [ H + ] = 0. This immediately eliminates the equilibrium relations and reduces the model to eight parameters and three ODES. After solving the regression hs lo problem, ti assumption was a s consistent with the results. Finally, we note that three species were measured as experimental data and normalized to meet a mass balance constraint. The fourth component was derived from a relation on [BW that generally does not hold. Consequently, these data dependencies and the model deficiency mentioned above clearly violate the assumed-error structure for which the maximum likelihood function was derived. The first and fourth investigators observed that these measurements were not independent even if only two components were used in the objective function. Thus the unknown diagonal covariance assumption covariance should be replaced with an unknown general covariance, as was done by the first investigator. A l q while the justification for the heteroscedasticity assumption is weak, the complexity of the objective function greatly inhs creases. T i becomes especially serious when [HA] goes to zero and the objective function becomes unbounded. Thus, during the optimization, y1 is forced to zero regardless of the actual error structure. To avoid this problem, the second and third investiga/(, yuj)yi for yujyi in the tors substituted either zuIyj or 1 2 2 , objective function. The first investigators used [max( yuj)]7j in their calculations.

0 5

3
J

Figure 7. Model with optimal parameter estimates from solution (4).

AlChE Journal (Vol. 32, No. 1)

January, 1986

Page 41

Figure 8. Model with optimal parameter estimates from solution (5).

GUIDELINES FOR PARAMETER ESTIMATION

The following conclusions serve as guidelines for tackling dynamic parameter-estimation problems. First, we need to consider transformation of kinetic parameters. Here, the parameters should be relatively uncorrelated and scaled so that they do not vary over many orders of magnitude. All but the third and fourth investigators used log transformations of the equilibrium constants and kinetic parameters centered about a mean base temperature. This helps to improve the conditioning, and hence the convergenceof the problem. Second, a number of problems can also be avoided by transforming the model. For the original model, DASSL was used to solve the DAE system. Here, numerous difficulties were encountered especially for stiff sets of parameter values. For the attempted solution these led to frequent convergence failures and poor performance. Most investigators reduced the model to a system of ODES.This led to fewer convergence problems and more efficient solutions. Finally, the last three investigators used different starting guesses in solvingthis problem. The only explanation for this was offered by the fifth investigators who noted that, with the initial guesses in the Dow problem statement, complete conversion of H A to HABM occurs before any A B is formed. This is easily seen in Figure 1. To obtain lower, more reasonable peak values of HABM, a starting guess of K,IK, = 1 and K J K , = 1 was selected. The use of different starting points suggests a more general approach that appears in all of the solutions. Investigators one and two used multiple restarts to converge from the original starting point. Moreover, their preliminary runs dealt with simpler optimization problems which may have given better progress than Eq. 23. Although it was not necessary for obtaining a solution,

the first investigators began with a likelihood function without heteroscedasticity, while the second kept the statistical parameters constant until he was close to the solution. This suggests that an interactive approach and a great deal of experiencemay still be required to solve difficult parameter-estimation problems. Also, from the above solutions it appears that efficient and userfriendly software is not yet available that automatically handles all of the difficulties encountered.
ACKNOWLEDGMENT

The authors wish to express their gratitude to G. Agin, L. Kirkby, and M. Marks of the Dow Chemical Company who assisted in the problem formulation. Special thanks go to the five participants whose submitted solutions, helpful comments, and interest made this paper possible. Finally, acknowledgment is made to the Donors of the Petroleum Research Fund, administered by the American Chemical Society, for partial support of this work.
NOTATION

a,,,, b,,, b,, c,

E E, e,,

= = =
=

=
=

h H

coefficientsin ODE solver partitioned matrix in DAE system activation energies experimental error vector for experiment u righthand side of ODE system righthand side of DAE system algebraic equations; also gradient vector step size for ODE solver Hessian matrix

Page 42

January, 1986

AlChE Journal (Vol. 32,No. 1)

iteration counter equilibrium constants rate constants maximum likelihood function number of measured variables number of experiments transformed parameter vector function values in ODE solver eigenvector matrix for rotational discrimination

search directionfor 0
temperature time at which experiment u was taken covariance matrix model prediction measured variables

Gill, I? E., W. Murray, and M. H . Wright, Practical OptirnizaCion, Academic Pres, London (1981). Klaus, R., and D. W. T. Rippin A New Flexible and Easy-to-UseGeneralPurpose Regression Program, Comp. and C h a . Eng., 105 (1979). Michelsen, M. L., Efficient General-Purpose Method for the IntegraAn tion of Stiff Ordinary Differential Equations, AIChE l.,22, 594 \ (1976). petzold, L., Differential/AlgebraicEquations Are Not ODES,Siam 1. Sci. Stat. Comput., 3 (3), 367 (1982). Prokopakis, G. J., and W. D. Seider, AdaptiveSemi-ImplicitRunge-Kutta tf Method for Solution of S i f Ordinary Differential Equations, Ind. Eng. Chem. Fund., 2.0, W (1981). Reilly, l? M., et.al., Guidelinesfor the Optimal Design of Experimentr to oes Estimate Parameters in First-Order Kinetic M d l , Can. 1. C h . Eng., 55,614, (1977). Stewart, W. E., and J. Soremen, Bayesian Estimation of Common Parameters from Multiresponse Data with Missing Observations,Technometrics, 23,131 (1981).
Manusrript received June5,1084, and seoision received Ian. 22,1985.

preexponential factor in rate constants heteroscedasticity parameters step length residual vector for experiment u tolerance for rotational discrimination parameter vector mean of measured variables eigenvalues of H Levenberg-Marquardt parameter transformed search direction variance coefficient for heteroscedasticity
LITERATURE CITED

APPENDIX A EXPERIMENTALDATA AND INITIAL CONDITIONS

The initial parameter estimates are:

( Y ~

=
3

El
(Y2

Ez
Bard, Y A., Nonlinear Pammeter Estimation, Academic Press. New York . (1974). Box, M. J., A New Method of Constrained Optimizationand a Comparison with Other MethodsComputerJ., 8 (l), 42 (1965). Byme, G . D., Some Software for Solving Ordinary Differential Equations. Foundations of Computw-Aided Process Design, Engineering Foundation, New York, 1,403 (1981). Carnahan, B., and J. 0. Wilkes. Numerical Solution of Diffenmtial Equations-An Overview, Foundations of Computer-Aided Chemical Process Design,Engineering Foundation, New York, 1,225 (1981). Chan, Y. N., I. Bimbaum, and L. Lapidus, Solution of Stiff Differential s Equations and the U e of Embedding Techniques I rL EC Fund. 17, 133, (1978). Fariss, R. H., and V. H. Law, AnEfficientComputationalTechnique for Generalized Application of Maximum Likelihd.to Improve Correlation of Experimental Data, Comp. and Chem. Eng., 3,95 (1979). Fletcher, R., A Modified Marquardt Subroutine for Nonlinear Least Squares,AAEREReport. No. 6799, H m e l l , England (1971). Gear, C. W., SimultaneousNumerical Solution of Differential-Algebraic Equations,IEEE Tmns. on Circuit Theory, 18 (l),89 (Jan. 1971).

E_, K, Kz
y] yz
73
y 4

= =
=

=
= =

2.0 2.0 2.0 2.0 4.3 2.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

x 1013 x 104 x 1013

x 104 x 11 05 x 104 x 10-17 x lo- x 10-17

The initial model conditions in addition to those given in the

data sets are:
Y5

ye = 0.0131

- 0
=

y7
Ys
Y9
YlO

1/21 - Kz + [K,2 + 4K2y1(0)]0,5}

= Y7 = 0

AlChE Journal (Vol. 32,No. 1)

January, 1986

Page 43

TABLE RUN1 CONCENTRATION w DATA 40OC Al. vs. T AT

Time h

TABLE RUN 2 CONCENTRATIONTIME A2. VS. DATA 67OC (cont'd) at

AB
Timeh HA Concentration (gmol/kg) EM HABM AB

HA
1.7066 1.6960 1.6826 1.6596 1.6305 1.6143 1.5892 1.5673 1.5133 1.4075 1.2308 1.0931 0.7268 0.5773 0.2065 0.0650 0.0391 0.0244 0.0145 0.0083 0.0074 0.0050 0.0047 0.0042 0.0015 0.0017 -

Concentration (gmollkg) BM HABM

0.00 0.08 0.58 1.58 2.75 3.75 4.75 5.75 8.75 13.05 21.75 28.75 46.25 52.25 76.25 106.25 124.25 147.25 172.25 196.25 219.75 240.25 274.25 292.25 316.25 340.75 364.25 386.75 412.25 442.75 460.75 483.75 507.25 553.75 580.75 651.25 673.25 842.75

8.3200 8.3065 8.2954 8.2730 8.2437 8.2277 8.2026 8.1781 8.1265 8.0167 7.8440 7.6977 7.3134 7.1495 6.6123 6.2309 6.1220 6.0084 5.9193 5.8556 5.8037 5.7680 5.7222 5.7021 5.6722 5.6593 5.6351 5.6176 5.6131 5.5991 5.5939 5.5905 5.5736 5.5558 5.5631 5.5472 5.5516 5.5465

O.oo00
0.0077 0.0234 0.0470 0.0763 0.0923 0.1174 0.1371 0.1935 0.2949 0.4760 0.6047 0.9530 1.0881 1,2929 1.1941 1.1370 1.0528 0.9835 0.9326 0.8821 0.8492 0.8064 0.7869 0.7628 0.7495 0.7263 0.7112 0.7063 0.6927 0.6871 0.6837 0.6672 0.6494 0.6467 0.6408 0.6452 0.6397

0.0000 0.0029 0.0oO6

-

93.33 102.42 124.83 148.08 171.83 197.33 228.33 270.33 293.33

0.0022 0.0022 0.0026 0.0012 0.0036 0.0027 0.0024 0.0026 0.0032

5.4796 5.4799 5.4792 5.4790 5.4857 5.4843 5.4862 5.4858 5.5069

0.5580 0.5587 0.5572 0.5602 0.5617 0.5617 0.5646 0.5638 0.5441

1.1012 1.1007 1.1018 1.1004 1.0963 1.0970 1.0946 1.0952 1.0945

0.0024
-

0.0042
-

0.0088 0.0268 0.0412 0.2074 0.4475 0.5305 0.6294 0.7086 0.7659 0.8171 0.8514 0.8957 0.9155 0.9425 0.9556 0.9793 0.9956 1.0003 1.0141 1.0195 1.0229 1.0396 1.0574 1.0551 1.0660 1.0616 1.0669
TABLE RUN3 CONCENTFIATION DAIX lOOOC A3. vs. TIME AT Concentration (gmollkg) Time h HA BM HABM AB

0.0046

0.00 0.08 0.42 0.75 1.17 1.50 2.00 2.50 3.00 3.50
4.00 4.50 5.00 5.50 6.50

1.5608 1.5316 0.7016 0.3763 0.2229 0.1793 0.1336 0.0894 0.0752 0.0626 0.0518 0.0598 0.0320 0.0228 0.0225 0.0180 0.0155 0.0135 0.0118 0.0110 0.0096 0.0085 0.0081 0.0200 0.0183 0.0060 0.0192 0.0059 0.0115 0.0059 0.0049 0.0039

8.3546 8.3325 7.3364 6.7519 6.3849 6.2899 6.1287 5.9985 5.9541 5.9186 5.8818 5.8878 5.8285 5.8086 5.8048 5.7944 5.7870 5.7833 5.7775 5.7774 5.7735 5.7706 5.7704 5.7912 5.7875 5.7644 5.7874 5.7641 5.7680 5.7596 5.7515 5.7468

0.0082 0.0445 0.7088 0.7745 0.7147 0.7065 0.6363 0.5953 0.5785 0.5686 0.5542 0.5434 0.5397 0.5382 0.5354 0.5336 0.5320 0.5315 0.5295 0.5302 0.5303 0.5288 0.5294 0.5268 0.5261 0.5276 0.5246 0.5271 0.5202 0.5234 0.5173 0.5138

0.0086 0.0015 0.1674 0.4268 0.6402 0.6918 0.8075 0.8931 0.9237 0.9464 0.9720 0.9744 1.0059 1.0166 1.0199 1.0260 1.0305 1.0326 1.0365 1.0362 1.0381 1.0403 1.0401 1.0310 1.0332 1.0440 1.0340 1.0444 1.0459 1.0485 1.0556 1.0597

TABLE RUN2 CONCENTRATION MDATA 67OC A2. vs. T ~ E AT

Concentration (gmollkg) Time h 0.00 0.08 1.08 2.33 3.33 HA BM HABM AB

7.00 7.50 S.00 8.50 9.00 9.50 10.00 10.50 11.00 11.50 12.50 13.50 14.50 16.50 21.75 29.50 53.00

1.6497 1.6400 1.4068 1.0895 0.8389 0.6485 0.4604 0.1068 0.0322 0.0171 0.0298 0.0049 0.0030

8.2262 8.2158 7.9826 7.6584 7.3861 7.1231 6.8643 6.0636 5.7212 5.6208 5.6680 5.5139 5.4859

0.0104 0.0186 0.2522 0.5656 0.7915 0.9097 1.0267 0.9332 0.7396 0.6690 0.6916 0.5877 0.5627

0.0017 0.0028 0.0026 0.0095 0.0312 0.1036 0.1745 0.6216 0.8896 0.9751 0.9402 1.0692 1.0957

4.33 5.33 12.83 23.33 27.83 30.83 51.67 83.33

Page 44

January, 1986

AlChE Journal (Vol. 32, No. 1)

APPENDIX B: INITIAL AND FINAL PARAMETER ESTIMATES WITH 95% CONFIDENCE INTERVALS

TABLE B4.
Parameter
ffl

INVESTIGATOR (4), RIPPIN, ESTIMATES

Final Estimate
~~

Initial Estimate

El

TABLE B1.
_____

INVESTIGATOR (l), CARACOTSIOS, AND SORENSEN, STEWART, &mm

Initial Estimate Final Estimate

ffZ

E Z
ff-1

Parameter

Pl Pz P3 P 4 P5
Pe P 7 P H P9 PO l

1.1938 1.1938 1,837.0 13,393.0 8,666.0 13,393.0 10,070.0 10,070.0 0

0.7965 f 1.1658 f 12,392 f 19,885 f 8,200'*

0.0615 0.0558 50 10

E-1 Kl

K Z
K3

2.0 x 2.0 x 2.0 x 2.0 x 4.3 x 2.0 x 1.0 x LO x 1.0 x

1013 104 1013 104 1015 104 10-17 10-17 10-17

(1.0955 (1.8255 k (1.8165 (1.8412 (2.8397 & (2.1901 (1.2263 f (2.0404 (8.4829 k

* 0.00186)*1012 0.00036)*104 * 0.00515)*101* * 0.00024)*104 0.00897)*1016 * 0.00076)*104 0.00266)*10-18 * 0.0132)*10-17

0.0196)*10-19

20,000** 9,493 f 211 9,457 f 233 -3,230 f 579 (1.08 f 0.32)010-~ (-2.39 f 1.95)*10-' (5.72 f 2.06)*10-4

TABLE B5.
Parameter In l q El ln k ? E Z In b-, E-I 1n(K1/K3)
1n(K3/KZ)

INVESTIGATOR (5), SARUP, MICHELSEN, VILLADSEN, AND ESTIMATES

Initial Estimate Final Estimate

Pll PlZ PI3

P14

*
t

Om*
0.190 f 0.118

1.022 20,000 1.022 20,000 6.393 20,000 0 0

0.78 f 17,837 f 1.01 f 18,850 f

0.164 1000 0.033 222.3

'Not specified. **95% confidenceintervals arenotestimated.

5.0 f 0.82 25,210 f 553.9 0.3570f 0.0181 -2.700 f 0.5

To

TABLE INVESTIGATOR B2. (2), FARISS, Esnmm

Parameter
ff1

- 340K

Initial Estimate

Final Estimate

El
ffZ

Ez
ff-1

2.0 2.0 2.0 2.0 4.3

x x x x x

1013 104 1013 104 1015

1.3708 x 1.8476 x 5.2282 x 1.9075 x 1.6215 x

1012 lo4 1012 104 1W

TABLE B6. THIS STUDY, DAMIANO, ESTIMATES MC

Parameter
ffl

Initial Estimate

Final Estimate

E-1 Kl K Z K3 Y1
YZ
Y3
Y4

2.0 x 104 1.0 x 10-17 1.0 x 10-11 1.0 x 10-17 1.0 1.0 1.0 1.0 0.065 0.143 0.057 0.0525

2.6046 x 104 2.575 x 10-16 4.876 x 10-14 1.7884 x 10-18 0.8149 7.2271 -0.2972 0.1352 0.06393 4.792.10-3 0.03105 0.02473

El
ff2

Ez
ff-I

2.0 2.0 2.0 2.0 4.3

x x x x x

1013 104 1013 104 1015 104 10-17 10-11 10-17

1.8745 x 1.960 x 1.8911 x 2.0046 x 3.8003 x 2.0110 x 1.0296 x 1.3969 x 6.3845 x

1013 104 1013 14 0 1015 104 10-17 10-11 IO-IW

W1

E-I Kl K Z K 3

2.0 x 1.0 x 1.0 x 1.0 x

WZ

03
w4

'95% confidence intervals not estimated.

'95% confidence intervals not reported.

TABLE B7. TABLE B3. INVESTIGATOR (3). AHN.ESTIMATES

~ ~ ~~~

THIS STUDY, DAMUNO, ESTIMATES ON

Initial Estimate Final Estimate

Parameter

Parameter lm l El lna, Ez l m-

Initial Estimate

Final Estimate

30.627 2.0.104 30.627 2.0.104 35.997 2.0.104 -39.144 -34.388 - 39.144

28.533 f (1.8835 f 27.313 f (1.7875 f 44.594 f (2.5150 -38.667 -33.779 -39.019

1.264 0.0943)*104 0.391 0.0254).104 10.173

2.0 2.0 2.0 2.0 4.3

x x x x x

1013 104 1013 104 10'5

7.8961 1.7486 7.1852 1.7958 5.1796 2.2052 1.1944 1.9016 7.8221

x 10'3

x x x x

14 0 1011 104 1015

E-1 lnK, lnK, lnK3

f 0.0473)*104 f 5.822 f 9.166 f 5.821

2.0 x 104 1.0 x 10-17 1.0 x 10-11 1.0 x 10-17

'95% confidence intervals not estimated.

x 104 x 10-17 x 10-11 x 10-18

AlChE Journal (Vd. 32, No. 1)

January, 1986

Page 45