Least Square Estimation

-1-
-2-
  
n n 2

RSS= ( yi  yˆi )2   yi  ( ˆ0  ˆ1 xi ) .

i 1 i 1

The Quantity RSS is also called the sum of squares for error (SSE). If SSE possesses a
minimum, it will occur for values of  0 and 1 that satisfies the equations,

  SSE    SSE 
 0 and  0 . Taking the partial derivatives of SSE with respect to ˆ0
ˆ
0 ˆ1
and ̂1 , setting them equal to zero, we obtain

 
n 2

 yi  ( ˆ0  ˆ1 xi )
  SSE  n
 n n

 i 1
  2[ yi  ( 0  1 xi ]  2   yi  n0  1  xi   0
ˆ ˆ ˆ ˆ
ˆ0 ˆ0 i 1  i i 1 

and

 
n 2

 yi  ( ˆ0  ˆ1 xi )
  SSE  n
 n n n

 i 1
  2[ yi  ( ˆ0  ˆ1 xi ] xi  2   xi yi  ˆ0  xi  ˆ1  xi 2   0
ˆ1 ˆ1 i 1  i i 1 i 1 

-3-
   SSE    SSE 
The equations  0 and  0 are called the least squares equations for
ˆ
0 ˆ1

estimating the parameters of a line.

The least squares equations are linear in ˆ0 and ̂1 and hence can be solved
simultaneously. We can verify that the solutions are
n n
1 n
 n

 ( xi  x )( yi  y )  x y  n   x 
i i

 y  i i
ˆ1  i 1
n
 i 1 i 1 i 1
2
1 
 (x  x )
n n

x    xi 
2 2
i i
i 1 i 1 n  i 1 
 
0  y  1 x .

Summary
Least Square Estimators for the Simple Linear Regression Model
 S xy n n
1. 1  where S xy   ( xi  x )( yi  y ) and S xx   ( xi  x )2 .
S xx i 1 i 1
 
2. 0  y  1 x .

Example 1:
Use the method of least squares to fit a straight line to the n  5 data points
given in the Table.

Table: Data, Example 1

x y
-2 0
-1 0
0 1
1 1
2 3
Calculations for finding the coefficients

-4-
 xi yi xi yi xi 2
-2 0 0 4
-1 0 0 1
0 1 0 0
1 1 1 1
2 3 6 4
n n n n

 xi  0
i 1
 yi  5
i 1
 xi yi  7
i 1
x i 1
i
2
 10

Plot of data points and least squares line for the example 1

n
1 n n

 S xy  ( xi yi )   i  yi
x
n i 1 i 1
1
7  (0)(5)
5
1   1
2
  0.7
S xx n
1 n  1 2
10  (0)

i 1
xi  2
 xi
n  i 1  5

  5
 0  y  1 x   (.7)(0)  1
5

And the fitted line is y  1  0.7 x

-5-
Exercise: (HW)
Auditors are often required to compare the audited (or current) value of an inventory
item with the book (or listed) value. If a company is keeping its inventory and books
up to date, there should be a strong linear relationship between the audited and book
value. A sample of ten inventory items from a certain company gave the data in the
accompanying table on audited and book values. Fit the model Y  0  1 x   to these
data.
What is your estimate for the expected change in audited value for a one-unit change
in book value? If the book value is x=100, what would you use to estimate the audited
value?

Item Audit Value ( yi ) Book Value ( xi )

1 9 10
2 14 12
3 7 9
4 29 27
5 45 47
6 109 112
7 40 36
8 238 241
9 60 59
10 170 167

-6-
Answer:

Scatterplot of y vs x
250

200
Audit Value(y)

150

100

50

0
0 50 100 150 200 250
Book Value(x)

n n n n

 xi  720
i 1
 yi  691
i 1
 xi yi  104745
i 1
x
i 1
i
2
 106554

n
1 n n

 S xy  ( xi yi )   i  yi
x
n i 1 i 1
104745 
1
10
(720)(691)
54993
1   1
2
   1.0051
S xx n
1 n  1 54714
106554  (720)
  xi
2
2

n  i 1 
xi 10
i 1

  691 720
 0  y  1 x   (1.0051)  69.1  72.367  3.2672 3.27 .
10 10

fitted line is y  3.27  1.0051x

when x  100 , y  3.27  1.0051 100  100.51 3.27  97.24.

-7-