Applications in Industrial

Engineering
Method of Least Squares
• When performing experiments, we often tabulate data in the form
of ordered pairs (x1, y1), (x2, y2),...,(xn,yn) with each xi distinct. We
want to find a function f(x) such that

f ( x1 )  y1 , f ( x2 )  y2 ,..., f ( xn )  yn

• We shall confine our attention to the problem of finding a linear

polynomial f(x) = ax + b or straight line that best fits the data (x1, y1),
(x2, y2),...,(xn,yn) . The procedure for finding linear equations is
known as method of least squares.
Example
• Consider the data (1,1), (2,3),
(3,4), (4,6), (5,5). The line y = x + y 1  ax 1  b

1 passes through the data points, y

y
2 

ax
ax
2 

b
b
we might take this line as the
n n

best fitting data. Y  AX

• If we try to match the data
points with the function f(x) = ax  y1   x1 1
  a
+ b, then we wish to find a and b Y   y2 
 
A   x 2 1 X   
that satisfy the system of y   x 1 b
 n  n 
equations.
Least Squares Line
• If the data points are (x1, y1), (x2, y2),...,(xn,yn), then one way to
determine how well the linear function f(x) = ax+ b fits the data is to
measure the vertical distances between the points and the graph of
f:
ei  yi  f (xi ),i 1,2,...,n
• We can think of ei as the error in approximating the data value yi by
the functional value f(xi). A more convenient approach to the
problem is to find a linear function f so that the sum of squares of
all ei values is a minimum.
• We shall define the solution of the system to be those coefficients a
and b that minimize the expression 2 2 2
e1  e2  ...  en
n 2

E    yi  axi  b
i 1
• The expression E is called the sum of the square errors. The line y =
ax+b that minimizes the sum of the square errors is defined to be the
line of best fit and is called the least sqaures line for the data (x1, y1),
(x2, y2),...,(xn,yn).
cont.
• The unique solution X = (ATA)-1ATY
• We say that X is the least squares solution of the overdetermined
system.
Example. Least Squares Line
• Find the least square line for the • Now by identifying
data (1,1), (2,3), (3,4), (4,6), (5,5).
Calculate the sum of square 1
   1 1
errors E for this line and the line  3

2 1

y = x + 1.



 T  55 15 
Y   4
A 3 1
 
A A   
Solution    4 1  15 5 
6  5 1
a+b=1 5
 
 
2a + b = 3
3a + b = 4
4a + b = 6
5a + b = 5
Solution
1

1

• Thus the least squares solution is
1  2
 55 15  
X    3
1
1
a = 1.1 and b = 0.5, and the least
 15 5  
4 1

squares line is y = 1.1x +0.5.
5 1

• For this line the sum of the
1  5
1
 
 3
 15 1 2 3 4 5  
square errors is:
    4
50   15 55 1 1 1 1 1  
6
5
• E = [1-f(1)]2 + [3-f(2)]2+ [4-f(3)]2 +
 
[6-f(4)]2+ [5-f(5)]2
1  5  15  68   1.1 
      
50   15 55  19   0.5  • = [1-1.6]2+ [3-2.7]2+ [4-3.8]2+
[6-4.9]2+[5-6]2 = 2.7
Example: Least squares parabola
• Find the least squares parabola
for the data (1,1),(2,4),
(3,7),(4,5) 1
 
1

1 1
  354 100 30 
4
  4 2 1  
a b+ c = 1 Y   , A  
7 9 3 1
andAT
A   100 30 10
 30 10 4 

     
5 16 
4a + 2b + c = 4    4 1

9a + 3b + c = 7
16a + 4b + c = 5
Solution

T
1 1 1 1 1
 354 100 30       5  25 25 160    1.25 
  4 2 1  4 1     
X   100 30 10  9
   25 129  135  50    7.75 
 30 10 4  3 1 7 20    
 25  135 155  17    5.75 

     
16 4 1 5
  

• The equation of the least

1 
 5  25
1
25 1 4 9 16  
  4  squares parabola is
   25 129  135 1 2 3 4  
20   7

 25  135 155 1 1 1 1  5 
  • f(x) = -1.25x2 + 7.75x - 5.75
Exercise
• Find the least squares line for
the given data
• (2,1),(3,2),(4,3),(5,2)
Polynomial Approximation by
Finite Difference
Definition
• Finite Difference Methods replace derivatives in a differential
equation with approximation.
• Solve the resulting algebraic equations to get the approximate
solution
Example
• Given the points (1,0), (2,-2), (3,-2), (4,0), (5,4) and (6,10) are a set of
points on the graph of a polynomial function, determine the equation
of the polynomial function satisfying these coordinates.
Example 2
• The following represent a set of x y
points that lie on the graph of a 1 0
function. Determine the
equation of the polynomial using 2 -12
an algebraic method. 3 -10
4 18
5 84
6 200
Exercise
• The following represent a set of x y
points that lie on the graph of a -3 -45
function. Determine the
equation of the polynomial using -2 -16
an algebraic method. -1 -3
0 0
1 -1
2 0
3 9
4 32