APPLIED NUMERICAL METHODS

SEMM/SKMM3023

Curve Fitting

Department of Aeronautics, Automotive, & Ocean Engineering

School of Mechanical Engineering, Faculty of Engineering,
Universiti Teknologi Malaysia

1
 Why curve fitting?

■ Field data is often accompanied by noise.

■ Even though all independent variables remain constant, the resultant

outcomes (dependent variables) vary.

■ A process of quantitatively estimating the trend of the outcomes, also

known as regression or curve fitting, therefore becomes necessary.

2
Application of Curve Fitting

■ Data fitting
■ Optimisation

3
 Curve Fitting

■ In this chapter we will learn three types of curve fitting:

1. Collocation-Polynomial
2. Least square regression (Statistical method) – mth order polynomial
regression (linear, quadratic, cubic, 4th ,5th and so on) and multiple
regression (more than one independent variable)
3. Interpolation (Calculus method) – Lagrange and Divided difference

4
 Collocation Polynomial

■ The collocation method fits a curve that passes through all the data
points. It is also referred to an exact fit or a direct fit
■ It is usually used when the data is known to be accurate, and a specific
trend is clear
■ Not suited for scattered data

5
 Collocation Polynomial

■ For a collocation-polynomial fit, the polynomial curve fit depends on the

number of data points. If there are n data points, then a n-1 order
polynomial fit can be used.
■ For example, if we have 4 points (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), (𝑥3 , 𝑦3 ) and (𝑥4 , 𝑦4 ),
the curve to fit would be a 3rd order polynomial:
𝑦 = 𝑎3 𝑥 3 + 𝑎2 𝑥 2 + 𝑎1 𝑥 + 𝑎0
■ If we substitute the data points into the equation:
𝑦1 = 𝑎3 𝑥13 + 𝑎2 𝑥12 + 𝑎1 𝑥1 + 𝑎0
𝑦2 = 𝑎3 𝑥23 + 𝑎2 𝑥22 + 𝑎1 𝑥2 + 𝑎0
𝑦3 = 𝑎3 𝑥33 + 𝑎2 𝑥32 + 𝑎1 𝑥3 + 𝑎0
𝑦4 = 𝑎3 𝑥43 + 𝑎2 𝑥42 + 𝑎1 𝑥4 + 𝑎0
■ We now have 4 equations and 4 unknowns 𝑎3 , 𝑎2 , 𝑎1 and 𝑎0

6
 Collocation Polynomial

■ In matrix form:

𝑥13 𝑥12 𝑥1 1 𝑎3 𝑦1
𝑥23 𝑥22 𝑥2 1 𝑎2 𝑦2
𝑎1 = 𝑦3
𝑥33 𝑥32 𝑥3 1
𝑎0 𝑦4
𝑥43 𝑥42 𝑥4 1

■ The matrix above can now be solved using any of the methods you have
learnt previously

7
 Collocation Polynomial

■ Example: Use the collocation method to fit a polynomial to the following

data points
i 1 2 3 4 5
x 4 8 12 16 18
y 486 12138 68750 228690 372368

■ Since there are 5 data points, a 4th order polynomial can be used

8
 Collocation Polynomial
i 1 2 3 4 5
x 4 8 12 16 18
■ In matrix form y 486 12138 68750 228690 372368

■ Solving it results in the following equation

𝑦 = 4𝑥 4 − 8𝑥 3 − 3𝑥 2 + 3𝑥 + 2

9
Collocation Polynomial
i 1 2 3 4 5
x 4 8 12 16 18
y 486 12138 68750 228690 372368

10
 Collocation Polynomial

■ Steps for collocation-polynomial fit:

1. Determine if the data points are somewhat accurate and strongly

represent a polynomial curve
2. Determine the number of data points, n
3. Use a n-1 order polynomial or lower for the curve-fit
4. Construct the linear equations and matrix to solve for the unknown
coefficients in the polynomial

■ Sometimes, although you have many data points, you may want to
ignore some points since the trend is a lower order polynomial

11
Collocation Polynomial

x y
1 1
3 3
7 7
8 8
10 10

12
 Collocation Polynomial

x y
1 1
3 3
7 7
8 8
10 10

■ Although there are 5 data points, the trend is obviously linear. Hence,
we only need to choose two data points to fit. However, if you try to fit a
4th order polynomial, you may get coefficients a4, a3 and a2 = 0.

13
 Least Square Regression

■ The method to obtain best-fit curve of a given type is the curve that has
the minimal sum of the deviations squared (least square error) from a
given set of data. It represents the general trend of the data and the
curve may not pass through all the data points.
■ Usually applied for scattered data

14
 Least Square Regression

■ Suppose you have the following data points for x and y, x y

and you want to fit a curve through it 10 100
11.9 180
14.2 230
15.2 330
16.3 320
17.2 405
18.1 410
18.3 401
19.5 405
22.1 520
22.4 450
23.5 540
25.2 610

15
 Least Square Regression

■ Now suppose you want to fit a curve, f(x) to the data i x y

1 10 100
points. The least square regression method states that
2 11.9 180
the best fitting curve has a property of: 3 14.2 230
4 15.2 330
5 16.3 320
6 17.2 405
7 18.1 410
8 18.3 401
9 19.5 405
10 22.1 520
11 22.4 450
12 23.5 540
13 25.2 610

16
 Least Square Regression

■ The methods of least square regression fitting includes:

1. Linear regression method (1st degree polynomial)

2. Quadratic/Parabola regression method (2nd degree polynomial)
3. mth order regression method
4. Multiple regression

17
 Linear Regression Method

■ To approximate a set of data that has more or equal to 2 data points, a

linear line can be used to fit the data.
𝑓(𝑥) = 𝑎1 𝑥 + 𝑎0
■ Again, the best fitting curve f(x) would have the least square error,
meaning

■ Where 𝑎1 and 𝑎0 are unknown coefficients and 𝑦𝑖 and 𝑥𝑖 are known

18
 Linear Regression Method

■ To obtain the least square error, the unknown coefficients 𝑎0 and 𝑎1

must yield zero first derivatives

■ Expanding the above equations, we have

19
 Linear Regression Method

■ 𝑎1 and 𝑎0 can now be solved using the methods of simultaneous

equation, resulting in the following:

■ Note that 𝑎1 is the gradient and 𝑎0 is the y-intercept of the linear line

20
 Linear Regression Method

■ Example: Use linear

regression to fit the i xi xi2 yi xiyi
following data points: 1 1 1 0.7 0.7
2 2 4 2.2 4.4
i 1 2 3 4 5 3 3 9 2.8 8.4
4 4 16 4.4 18
xi 1 2 3 4 5
5 5 25 4.9 25
yi 0.7 2.2 2.8 4.4 4.9 Σ 15 55 15 56

n=5

21
 Linear Regression Method

■ 𝑎1 and 𝑎0 can now be solved using the methods of simultaneous

equation, resulting in the following: i xi xi2 yi xiyi
1 1 1 0.7 0.7
2 2 4 2.2 4.4
3 3 9 2.8 8.4
4 4 16 4.4 18
5 5 25 4.9 25
Σ 15 55 15 56
5 56 − 15 15
𝑎1 = = 1.1
5 55 − 15 2

22
 Linear Regression Method
i xi xi2 yi xiyi
1 1 1 0.7 0.7
2 2 4 2.2 4.4
3 3 9 2.8 8.4
4 4 16 4.4 18
5 5 25 4.9 25
1 Σ 15 55 15 56
𝑎0 = 15 − 1.1 15 = −0.3
5

■ Hence the linear fit equation f(x) becomes:

𝑓 𝑥 = 1.1𝑥 − 0.3

23
Linear Regression Method

𝑓 𝑥 = 1.1𝑥 − 0.3

24
 Quadratic Regression Method

■ To approximate a set of data that has more or equal to 3 data points, a

quadratic line can be used to fit the data.
𝑓(𝑥) = 𝑎2 𝑥 2 + 𝑎1 𝑥 + 𝑎0
■ Again, the best fitting curve f(x) would have the least square error,
meaning

■ Where 𝑎2 , 𝑎1 and 𝑎0 are unknown coefficients and 𝑦𝑖 and 𝑥𝑖 are known

25
 Quadratic Regression Method

■ To obtain the least square error, the unknown coefficients 𝑎0 , 𝑎1 and 𝑎2

must yield zero first derivatives

26
 Quadratic Regression Method

■ Expanding the equations, we get

■ Again, 𝑎0 , 𝑎1 and 𝑎2 can now be solved using methods for simultaneous

equations

27
 Quadratic Regression Method

■ Arranging the simultaneous linear equations into matrix form:

 n n
  n

 n x i  2
xi    yi 
 n i =0 i =0
  a 0   i =0

3    
n n n
 x
 x 
2
xi  a1  =   xi yi 
i =0
i
i =0
i
i =0
    i =0 
n n n  a2   n 
 xi2  i
x 3
 x 4
  x 2
y
i i
 i =0 
i
i =0 i =0  i =0 

28
 Quadratic Regression Method

■ Example: Use parabolic regression to fit the following data points:

xi yi i xi xi2 xi3 xi4 yi xiyi xi2yi
1 8.79 1 1 1 1 1 8.79 8.79 8.79
2 7.76 2 2 4 8 16 7.76 15.52 31.04
3 6.25 3 3 9 27 81 6.25 18.75 56.25
4 5.39 4 4 16 64 256 5.39 21.56 86.24
5 6.38 5 5 25 125 625 6.38 31.9 159.5
6 6.51 6 6 36 216 1296 6.51 39.06 234.36
7 8.78 7 7 49 343 2401 8.78 61.46 430.22
8 10.96 8 8 64 512 4096 10.96 87.68 701.44
Σ 36 204 1296 8772 60.82 284.72 1707.84
n=8

29
 Quadratic Regression Method

■ Filling in the matrix:

 n n
  n 
 n x i  x2
i    i  y
 n i =0 i =0
 a0   ni =0 
 x
n n
3    
 x   i i 
2
x i  1a = x y
i =0
i
i =0
i
i =0
    i =0 
n n n  2   n
a

 xi2  i
x 3
 x 4
  x 2
y
i i
 i =0 
i
i =0 i =0   i =0 
8 36 204 𝑎0 61
36 204 1296 𝑎1 = 284.72
204 1296 8772 𝑎2 1707.84

30
 Quadratic Regression Method

■ Solution:
𝑎0 11.5693
𝑎1 = −2.8263
𝑎2 0.3432

𝑓 𝑥
= 0.3432𝑥 2 − 2.8263𝑥 + 11.5693

31
 mth Order Regression Method

■ To approximate a set of data that has more or equal to m+1 data points,
the least square regression method uses a mth degree polynomial,
𝑓(𝑥) = 𝑎𝑚 𝑥 𝑚 + 𝑎𝑚−1 𝑥 𝑚−1 +. . . 𝑎2 𝑥 2 + 𝑎1 𝑥 + 𝑎0
■ The best fitting curve f(x) would have the least square error, meaning

■ Where 𝑎𝑚 , 𝑎𝑚−1 ,…. 𝑎2 , 𝑎1 and 𝑎0 are unknown coefficients and 𝑦𝑖 and 𝑥𝑖

are known

32
 mth Order Regression Method

■ Again, to obtain the least square error, the unknown coefficients must
yield zero first derivatives

𝑎0

𝑎1

𝑎2

𝑎𝑚

33
 mth Order Regression Method

■ And hence form the following set of linear equations which can be
solved

34
 mth Order Regression Method

■ Solution for a 4 x 4 matrix

35
 Multiple Regression Method

■ This method is used for equations with more than one independent
variable. To approximate a set of data that has more or equal to 3 data
points in a linear case, the equation is as follows
𝑓(𝑥, 𝑦) = 𝑎 + 𝑏𝑥 + 𝑐𝑦
■ The best fitting curve f (x) has the least square error

■ Where 𝑎, 𝑏 and 𝑐 are unknown coefficients while all 𝑥𝑖 and 𝑦𝑖 are given.

36
 Multiple Regression Method

■ The unknown coefficients a, b and c must yield zero first derivatives

37
 Multiple Regression Method

■ Expanding the above equations

■ The unknown coefficients a, b, and c can be obtained by solving the

above simultaneous linear system

38
 Multiple Regression Method

■ In matrix form:

39
 Least Square Regression
How to calculate errors?
■ Tips for least square
regression: i xi xi2 yi xiyi f(xi) [yi-f(xi)]2
1 1 1 0.7 0.7 0.8
1. Select method to solve curve 2 2 4 2.2 4.4 1.9
fitting, 1st, 2nd m- order or
3 3 9 2.8 8.4 3.0
multiple regression.
4 4 16 4.4 18 4.1
2. Calculate error for each 5 5 25 4.9 25 5.2
method. Σ 15 55 15 56 Error
3. The smaller the error the
better the solution

40
 Interpolation

■ Interpolation is a method of predicting un-tested data points within a

set range of known data points.
■ A curve is fitted to a certain number of data points and the curve is used
to predicted the other values between the maximum and minimum
values of the data points
■ It is the opposite of extrapolation

41
 Interpolation

■ In interpolation, we will focus on two methods involving polynomial

equations:

1. Lagrange interpolation for polynomials

2. Divided difference polynomials

■ In interpolation, the curve will fit through all the data points.

45
 Lagrange Polynomials

■ Can be used for both unequally spaced data and equally spaced data.
■ However, for values of n > 3, this solution needs a computer program
and may not be very accurate.
■ It is based on fitting second-degree Lagrange polynomial, to a (sub)set
of three discrete data pairs (quadratic function through 3 points) such
as (𝑥0 , 𝑦0 ), (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 )
■ The second-degree Lagrange polynomial is:

■ Similar to the collocation-polynomial method, the order of polynomial is

equal to n-1, where n is the number of data points

46
 Lagrange Polynomials

■ Differentiating the second-order Lagrange polynomial results in

47
48
 Since for n>3 it becomes
Lagrange Polynomials less accurate, it may be
better to divide data
points into set of 3 data
and fit multiple 2nd order
■ Third-order Lagrange polynomial: Lagrange

■ General equation:

49
 Divided Difference Polynomials

■ Can be used for both unequally spaced data and equally spaced data.
■ Given the points (x0, y0), (x1, y1), (x2, y2), (x3, y3), the function f(x) can be
expressed in a 3rd order polynomial

■ Similar to the collocation-polynomial method, the order of polynomial is

equal to n-1, where n is the number of data points

50
 Divided Difference Polynomials

■ The unknown coefficients a0, a1, a2 and a3 can be found using the divided
difference table:

51
52
 Divided Difference Polynomials

■ Differentiating the equation results in:

53
 Divided Difference Polynomials

■ For a second-order polynomial, the table reduces to:

54
 Example 3 data point hence must use
second-order polynomial!

■ Evaluate the derivatives by numerical differentiation formulae

developed by fitting
1. Collocation-polynomial Let:
𝑥0 , 𝑦0 = (3.4, 0.294118)
2. Lagrange polynomial 𝑥1 , 𝑦1 = (3.5, 0.285714)
3. Divided difference polynomial 𝑥2 , 𝑦2 = (3.6, 0.277778)
For to the following set of discrete data:

■ Exact derivatives at x = 3.5 are f ′(3.5) = −0.081633 and f ′′(3.5) =

0.046647

55
 Example

1. Collocation-polynomial: 𝑦 = 𝑎2 𝑥 2 + 𝑎1 𝑥 + 𝑎0

𝑦0 = 𝑎2 𝑥02 + 𝑎1 𝑥0 + 𝑎0
𝑦1 = 𝑎2 𝑥12 + 𝑎1 𝑥1 + 𝑎0
𝑦2 = 𝑎2 𝑥22 + 𝑎1 𝑥2 + 𝑎0 Largest can fit would
be a 2nd order
polynomial!

2
0.294118 = 𝑎2 3.4 + 𝑎1 3.4 + 𝑎0
2
0.285714 = 𝑎2 3.5 + 𝑎1 3.5 + 𝑎0
2
0.277778 = 𝑎2 3.6 + 𝑎1 3.6 + 𝑎0

56
 Example

In matrix form

2
0.294118 = 𝑎2 3.4 + 𝑎1 3.4 + 𝑎0
0.285714 = 𝑎2 3.5 2 + 𝑎 3.5 + 𝑎
1 0
0.277778 = 𝑎2 3.6 2 + 𝑎 3.6 + 𝑎
1 0

11.56 3.4 1 𝑎2 0.294118

12.25 3.5 1 𝑎1 = 0.285714
12.96 3.6 1 𝑎0 0.277778

Can be solved using either of the methods you have previously learnt

57
 Example

Hence

𝑎2 0.023400
𝑎1 = −0.245500
𝑎0 0.858314

𝑓 𝑥 = 0.023400𝑥 2 − 0.245500𝑥 + 0.858314

𝑓 ′ (𝑥) = 0.046800𝑥 − 0.245500 → 𝑓 ′ 3.5 = −0.081700
𝑓′′ 𝑥 = 0.046800

58
 Example

1. Lagrange polynomial

59
 Example

Differentiation:

f’(3.5)

f’’(3.5)

60
 Example

1. Divided difference polynomial

61
 Example

Forming the equation:

62
 Example

Differentiating

f’(3.5)
f’’(3.5)

63
 Example

■ Note that all 3 methods here will give the exact same answer if the same
points are used!
■ The error in f ′(3.5) is
Error = 100% x |f ′(3.5) − (-0.081633)|/-0.081633
= 100% x |−0.081700 + 0.081633|/-0.081633
= 0.082%

■ The error in f ′′(3.5) is

Error = 100% x |f ′′(3.5) − 0.046647|/ 0.046647
= 100% x |0.046800 − 0.046647|/ 0.046647
= 0.328%

64
 Curve Fitting in MATLAB

■ In MATLAB, there are two functions to fit a polynomial curve to a set of

data points:

1. polyfit
2. polyval

Polyfits takes in at least 3 inputs: x, y and n where x and y are the data
points and n is the order/degree of the polynomial. The function then
outputs the coefficients for the specified polynomial
Polyval needs at least two inputs, p and x where p is a vector of coefficients
obtained from polyfit and x is a vector of the variable input. It outputs the y
values of the polynomial

65
Thank you

66