Curve Fitting

What is regression?
Regression analysis gives information on the relationship between a response variable
and one or more independent variables to the extent that information is contained in the
data. The goal of regression analysis is to express the response variable as a function of
the predictor variables.
Once regression analysis relationship is obtained, it can be used to predict values of the
response variable, identify variables that most affect response, or verify hypothesized
casual models of the response.

3.1 Linear regression

Linear regression is the most popular regression model. In this model we wish to
predict response to n data points (x1 ,y 1 ), (x 2 ,y 2 ), ....., (x n , y n ) data by a regression model
given by.
y a0 a1 x

Where a 0 and a 1 are the constants of the regression model.

A measure of goodness of fit, that is, how a0 a1 x predicts the response variable y is

the magnitude of the residual, i at each of the n data points.

i yi (a 0 a1 xi )

Ideally, if all the residuals i are zero, one may have found an equation in which all

the points lie on the model. Thus, minimization of the residual is an objective of
obtaining regression coefficients.
The most popular method to minimize the residual is the least squares method, where
the estimates of the constants of the models are chosen such that the sum of the squared
n
residuals is minimized, that is minimize i
2
.
i 1

Let us use the least squares criterion where we minimize

n n 2
2
Sr i yi a0 a1 xi
i 1 i 1

S r is called the sum of the square of the residuals.

Numerical Analysis /Lec. 2 - 11 -
Fourth Stage
 y

xi , yi

i yi a0 a1 xi xn , y n

x2 , y 2
x3 , y3

x1 , y1 y a0 a1 x

Figure 3.1 Linear regression of y vs. x data showing residuals at a typical point, x i .

To find a 0 and a 1 , we minimize S r with respect to a 0 and a 1 :

n
Sr
2 yi a0 a1 xi 1 0
a0 i 1
n
Sr
2 yi a0 a1xi xi 0
a1 i 1

Giving
n n n
yi a0 a1 xi 0
i 1 i 1 i 1
n n n
y i xi a0 xi a1 x i2 0
i 1 i 1 i 1
n
Noting that a0 a0 a0 . . . a0 na 0
i 1
n n
na 0 a1 xi yi (3.1)
i 1 i 1
n n n
a0 xi a1 x i2 xi y i (3.2)
i 1 i 1 i 1

Solving the above equations gives:

n n n
n xi y i xi yi
i 1 i 1 i 1
a1 2
n n
n x i2 xi
i 1 i 1
n n n n
x i2 yi xi xi yi
i 1 i 1 i 1 i 1
a0 2
n n
n x i2 xi
i 1 i 1

Or from equation (3.2)

Numerical Analysis /Lec. 2 - 12 -
Fourth Stage
 n n
yi a1 xi
i 1 i 1
ao y a1 x
n n

Example 3.1
The following y vs. x data is given
x 1 7 13 19 25
y 1 49 169 361 625

y vs x

800
600
y 400
200
0
0 5 10 15 20 25 30
x

Figure 3.1 Data points of the y vs x data

Although y x 2 is an exact fit to the data, a scientist thinks that y a0 a1 x can

explain the data. Find constants of the model, a0 , and a1 ,

Solution
First find the constants of the assumed model
y a0 a1 x

a0 y a1 x

n 5
n 5
xi y i xi y i 1 1 7 49 13 169 19 361 25 625 25025
i 1 i 1
n 5
xi2 xi2 12 7 2 13 2 19 2 25 2 1205
i 1 i 1
n 5
yi yi 1 49 169 361 625 1205
i 1 i 1

Numerical Analysis /Lec. 2 - 13 -

Fourth Stage
 n 5
xi xi 1 7 13 19 25 65
i 1 i 1
n n n
n xi y i xi yi
i 1 i 1 i 1
a1 2
n n
2
n x i xi
i 1 i 1

_ _
a0 y a1 x
5 25025 65 1205
a1 2
26
5 1205 65
1205 65
a0 y a1 x 26 241 26 13 97
5 5
This gives
y a0 a1 x
y 97 26 x

Example 3.2
The following table gives the value of density of saturated water for various
temperatures of saturated stream.
TempoC ( = T) 100 150 200 250 300
Density kg/m3 (= D) 958 917 865 799 712

a) Use curve fitting to fit the results to a first-order polynomial D A BT .

b) Find the densities when the temperatures are 130oC and 275oC respectively.
Solution:
a 0 and a 1 can be computed by constructing the following table:
Ti Di Ti 2 T i Di
100 958 10000 95800
150 917 22500 137550
200 865 40000 173000
250 799 62500 199750
300 712 90000 213600
1000 4251 225000 819700
5 819700 1000 4251
a1 - 1.22
5 225000 (1000) 2
4251 1000
a0 a1 1094.2
5 5
D=1094.2-1.22×T
Numerical Analysis /Lec. 2 - 14 -
Fourth Stage
To compare the predicted values to the experimental values:

Ti D i (estimated) Di
D=1094.2-1.22×T
100 958 972.2
150 917 911.2
200 865 850.2
250 799 789.2
300 712 728.2
D(130)= 1094.2-1.22×130=935.6
D(175)= 1094.2-1.22×175=880.7

3.2 Polynomial Models

Given N data points (x 1 , y 1 ), (x 2 , y 2 ). . , (x N , y N ) use least squares method to regress
the data to an nth order polynomial.
In the development, we use n as the degree of the polynomial and N as the number of
data pairs ( xi , yi ). We will always have N n 1 in the following.
Assume the functional relationship for fitting
Y ( x) a0 a1 x a2 x 2 an x n
with errors defined by
ei yi Y ( xi ) yi a0 a1 xi a 2 xi 2 an xi n ,
in which i 1, 2, 3,…, N.
We minimize the sum of error squares,
N N
S ei 2 ( yi a0 a1 xi a2 x i 2 an xi n ) 2 .
i 1 i 1

At the minimum, all the first partial derivatives with respect to ai ’s vanish. We have
N
S
0 2 ( yi a0 a1 xi a2 xi 2 an xi n )( 1) ,
a0 i 1
N
S
0 2 ( yi a0 a1 xi a2 xi 2 an xi n )( xi ) ,
a1 i 1
N
S
0 2 ( yi a0 a1 xi a2 xi 2 an xi n )( xi 2 ) ,
a2 i 1

N
S
0 2 ( yi a0 a1 xi a2 xi 2 an xi n )( xi n ) ,
an i 1

Rearrange them to get

N N N N
a0 N a1 xi a2 xi 2 an xi n yi ,
i 1 i 1 i 1 i 1

Numerical Analysis /Lec. 2 - 15 -

Fourth Stage
To compare the predicted values to the experimental values:

Ti D i (estimated) Di
D=1094.2-1.22×T
100 958 972.2
150 917 911.2
200 865 850.2
250 799 789.2
300 712 728.2
D(130)= 1094.2-1.22×130=935.6
D(175)= 1094.2-1.22×175=880.7

3.2 Polynomial Models

Given N data points (x 1 , y 1 ), (x 2 , y 2 ). . , (x N , y N ) use least squares method to regress
the data to an nth order polynomial.
In the development, we use n as the degree of the polynomial and N as the number of
data pairs ( xi , yi ). We will always have N n 1 in the following.
Assume the functional relationship for fitting
Y ( x) a0 a1 x a2 x 2 an x n
with errors defined by
ei yi Y ( xi ) yi a0 a1 xi a2 xi 2 an xi n ,
in which i 1, 2, 3,…, N.
We minimize the sum of error squares,
N N
S ei 2
( yi a0 a1 xi a 2 xi 2 a n xi n ) 2 .
i 1 i 1

At the minimum, all the first partial derivatives with respect to ai ’s vanish. We have
N
S
0 2 ( yi a0 a1 xi a2 xi 2 an xi n )( 1) ,
a0 i 1
N
S
0 2 ( yi a0 a1 xi a2 xi 2 an xi n )( xi ) ,
a1 i 1
N
S
0 2 ( yi a0 a1 xi a2 xi 2 an xi n )( xi 2 ) ,
a2 i 1

N
S
0 2 ( yi a0 a1 xi a2 xi 2 an xi n )( xi n ) ,
an i 1

Rearrange them to get

N N N N
a0 N a1 xi a2 xi 2 an xi n yi ,
i 1 i 1 i 1 i 1

Numerical Analysis /Lec. 2 - 16 -

Fourth Stage
 N N N N N
a0 xi a1 xi 2 a2 xi 3 an xi n 1
xi yi ,
i 1 i 1 i 1 i 1 i 1
N N N N N
a0 xi 2
a1 xi 3
a2 xi 4
an xi n 2
xi 2 yi ,
i 1 i 1 i 1 i 1 i 1

N N N N N
a0 xi n a1 xi n 1
a2 xi n 2
an xi 2 n xi n yi ,
i 1 i 1 i 1 i 1 i 1

or, in matrix form,

N N N N
N xi xi 2 xi n yi
i 1 i 1 i 1 i 1
N N N N a0 N
xi xi 2 xi 3 xi n 1
xi yi
i 1 i 1 i 1 i 1 a1 i 1
N N N N
a2 N
. (3.3)
xi 2 xi 3 xi 4 xi n 2
xi 2 yi
i 1 i 1 i 1 i 1 i 1

an
N N N N N
xi n xi n 1
xi n 2
xi 2 n xi n yi
i 1 i 1 i 1 i 1 i 1

Equations (3.3) represent a linear system. However, this system is usually

ill-conditioned and round-off errors can distort the solution of ai ’s. Up to degree-3 or
4, the problem is not too great. It is very infrequent to use a degree higher than 4.

Example 3.3
Rotameter calibration data (flow rate versus Rotameter reading) are as follows:
Rotameter Reading R 10 30 50 70 90
Flow rate V(L/min) 20 52.1 84.6 118.3 151
a) Using curve fitting to fit the calibration data to second order polynomial.
b) Calculate the flowrate (V) at rotameter reading R=73.
Solution:
a) 2nd order polynomial
n n
2
Sr i ( yi ao a1 xi a2 xi ) 2
i 1 i 1
n
dS r
2 ( yi ao a1 xi a2 xi2 ) ( 1) 0
dao i 1
n
dS r
2 ( yi ao a1 xi a2 xi2 ) ( xi ) 0 (1)
da1 i 1
n
dS r
2 ( yi ao a1 xi a2 xi2 ) ( xi2 ) 0
da2 i 1

Re arranging above equations

Numerical Analysis /Lec. 2 - 17-

Fourth Stage
 n n n
ao n a1 xi a2 xi2 yi
i 1 i 1 i 1
n n n n
ao xi a1 xi2 a2 xi3 xi yi (2)
i 1 i 1 i 1 i 1
n n n n
ao xi2 a1 xi3 a2 xi4 xi2 yi
i 1 i 1 i 1 i 1

Making required table

R V R2 R3 R4 RV R2 y
10 20 100 1000 10000 200 2000
30 52.1 900 27000 810000 1563 46890
50 84.6 2500 125000 6250000 4230 211500
70 118.3 4900 343000 2401000 8281 579670
90 151 8100 729000 6561000 13590 1223100
250 426 16500 1225000 9669000 27864 2063160
By substitution in equation 2
5a o 250a 1 16500a 2 426
250a o 16500a 1 1225000a 2 27864
16500a o 1225000a 1 96690000a 2 2063160
Solving above equation simultaneously gives;
ao =3.8786 , a1 =1.5981 , a2 = 4.2857×10-4
then
V 3.8786 1.5981 R 4.2857 × 10-4 R 2

B)
V (73) 3.8786 1.5981 73 4.2857 ×10-4 732 122.83

Numerical Analysis /Lec. 2 - 18 -

Fourth Stage
3.3 Nonlinear Data
Whenever data from experimental tests are not linear, we need to fit to them some
function other than a first-degree polynomial. Popular forms that are tried are the
power form
y ax b
or the exponential form
y aebx .
Since such nonlinear equations are much more difficult to solve than linear equations,
they are usually linearized by taking logarithms before determining the parameters:
ln y ln a b ln x ,
or
ln y ln a bx .

linear non-linear
Figure 3.2 Linear vs non-linear data

In cases when such linearization of the function is not desirable, or when no method
of linearization can be discovered, graphical methods are frequently used; one merely
plots the experimental values and sketches in a curve that seems to fit well.

Example 3.4
The progress of a homogeneous chemical reaction is followed and it is desired to
evaluate the rate constant and the order of the reaction. The rate law expression for
the reaction is known to follow the power function form r kC n
Use the data provided in the table to obtain n and k .
C A (gmol/l) 4 2.25 1.45 1.0 0.65 0.25 0.006
rA ( gmol/l s) 0.398 0.298 0.238 0.198 0.158 0.098 0.048

Solution
Taking the natural log of both sides of Equation, we obtain
ln r ln k n ln C
Numerical Analysis /Lec. 2 - 19 -
Fourth Stage
Let
z ln r
w ln C
a0 ln(k ) implying that k e a0
a1 n
We get
z a0 a1 w
This is a linear relation between z and w , where
n n n
n wi z i wi zi
i 1 i 1 i 1
a1 2
n n
2
n w i wi
i 1 i 1

n n
zi wi
i 1 i 1
a0 a1
n n

Table: Kinetics rate law using power function

i C r w z w z w2
1 4 0.398 1.3863 -0.92130 -1.2772 1.9218
2 2.25 0.298 0.8109 -1.2107 -0.9818 0.65761
3 1.45 0.238 0.3716 -1.4355 -0.5334 0.13806
4 1 0.198 0.0000 -1.6195 0.0000 0.00000
5 0.65 0.158 -0.4308 -1.8452 0.7949 0.18557
6 0.25 0.098 -1.3863 -2.3228 3.2201 1.9218
7 0.006 0.048 -5.1160 -3.0366 15.535 26.173
7
-4.3643 -12.391 16.758 30.998
i 1

n 7
7
wi 4.3643
i 1
7
zi 12.391
i 1
7
wi z i 16.758
i 1
7
wi2 30.998
i 1

From above equations

Numerical Analysis /Lec. 2 - 20 -

Fourth Stage
 7 16.758 4.3643 12.391
a1 2
7 30.998 4.3643
0.31943
12.391 4.3643
a0 .31943
7 7
1.5711
Then
k e 1.5711
0.20782
n a1
0.31941
Finally, the model of progress of that chemical reaction is
r 0.20782 C 0.31941

Example 3.5
It is suspected from theoretical considerations that the rate of water flow from a
firehouse is proportional to some power of the nozzle pressure. Assume pressure data
is more accurate. You are transforming the data.
Flow rate, F (gallons/min) 96 129 135 145 168 235
Pressure, p (psi) 11 17 20 25 40 55
b
What is the exponent b of the nozzle pressure in the regression model F ap

Numerical Analysis /Lec. 2 - 21 -

Fourth Stage
Solution
The linearization of the above data is done as follows.
F ap b
ln( F ) ln(a ) b ln( p )
z a 0 bx
Where
z ln F
x ln( p)
a0 ln a
Implying
a ea0
There is a linear relationship between z and x.
Linear regression constants are given by
n n n
n xi zi xi zi
b i 1 i 1 i 1
2
n n
2
n x i xi
i 1 i 1
n n n n
xi2 zi xi xi zi
a0 i 1 i 1 i 1 i 1
2
n n
n xi2 xi
i 1 i 1

Since
n 6
6
x izi ln(11) ln(96) ln(17) ln(129) ln(20) ln(135) ln(25) ln(145) ln(40) ln(168)
i 1

ln(55) ln(235) 96.208

6
xi ln(11) ln(17) ln(20) ln(25) ln(40) ln(55) 19.142
i 1
6
zi ln(96) ln(129) ln(135) ln(145) ln(168) ln(235) 29.890
i 1
6
xi2 (ln(11)) 2 (ln(17)) 2 (ln(20)) 2 (ln(25)) 2 (ln(40)) 2 (ln(55)) 2 62.779
i 1

then
6 96.208 19.142 29.890
b
6 62.779 19.1422
577.25 572.15
376.67 366.41
0.49721

Numerical Analysis /Lec. 2 - 22 -

Fourth Stage
Example 3.6
The following data have been obtained for the decomposition of benzene diazonium
chloride to chlorobenzene:
T (K) 313 319 323 328 333
-1
k (s ) 0.0043 0.0103 0.018 0.0355 0.0717
From this data, determine the pre-exponential factor A and activation energy E,
assuming that the rate constant follows an Arrhenius form.
E
k A exp
RT
Solution:
E
ln k ln A
RT
y ln k
x 1/ T
ao ln A
E
a1
R
We get
y ao a1 x
T (K) k (s-1) x=1/T y=ln k x2 xy
313 0.0043 0.00319 -5.44914 1.02073e-05 -0.01741
319 0.0103 0.00313 -4.57561 9.82695e-06 -0.01434
323 0.018 0.00310 -4.01738 9.58506e-06 -0.01244
328 0.0355 0.00305 -3.33822 9.29506e-06 -0.01018
333 0.0717 0.00300 -2.63526 9.01803e-06 -0.00791
0.01548 -20.0156 4.79324e-05 -0.06228
n n n
n xi yi xi yi
i 1 i 1 i 1
0.08
a1 2
- 14612 k Exp.
n n
2 0.07 k Pred.
n xi xi
i 1 i 1
0.06
ao y a1 x 41.2272 0.05

0.04
ao ln A 41.2272
A exp(40.2272) 8.0303 1017 0.03

a1 E/R 0.02
E a1 R ( 14612) 8.314 121480 0.01

0
310 315 320 325 330 335
Temperature (K)

Numerical Analysis /Lec. 2 - 23 -

Fourth Stage