Scribd
Upload
138 views

Formulas Linear Regression PDF

This document provides a summary of formulas for simple linear regression including: - The slope formula which calculates the slope (b) as a function of the differences between the mean of the x and y values. - Formulas for calculating the variance, intercept, estimated mean and variance at different x values, total sum of squares, regression sum of squares, error sum of squares, R-squared, mean square error, and degrees of freedom. - An example of applying the formulas to a dataset with 5 data points to calculate the slope, intercept, variance, residuals and more.

Uploaded by

NatRib
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
138 views

Formulas Linear Regression PDF

This document provides a summary of formulas for simple linear regression including: - The slope formula which calculates the slope (b) as a function of the differences between the mean of the x and y values. - Formulas for calculating the variance, intercept, estimated mean and variance at different x values, total sum of squares, regression sum of squares, error sum of squares, R-squared, mean square error, and degrees of freedom. - An example of applying the formulas to a dataset with 5 data points to calculate the slope, intercept, variance, residuals and more.

Uploaded by

NatRib
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Summary formula sheet for simple linear

regression

Slope b = ! (Yi -Y)(Xi -X) / ! (Xi -X)2


_ _ _

Variance 5 2 / ! (Xi -X)2


_
_ _
Intercept a= Y - b X _
2
Variance of a [ n + !
1 X _
2
] 5 2
(Xi -X)

Estimated mean at X_ 0 a + b X0
(X0 -X)_2
Variance [ n + !
1
2
] 5 2
(Xi -X)

Estimated individual at _X0 a + b X0


(X0 -X)_2
Variance [1 + n + !
1
] 5 2
(Xi -X)2

Total SS = ! (Yi -Y)2


_

[ ! (Yi -Y)(Xi -X)] / ! (Xi -X)2


Regression
_ SS =_ _
2

Error SS = Total SS - Regression SS


R2 = Regression SS/ Total SS = "proportion
explained"

MSE = error mean square = estimate of 5 2


= Error SS/ df
df= degrees of freedom = n-2 for simple
linear.
Example

Data points (x1 ,y1 ), (x2 ,y2 ), ...., (xn ,yn )


_ (1,5), (2,7),_ (3,9), (4,6), (5,8)
x = 15/5=3, y = 7
Corrected sum of squares for x:
!(xi - x)2 = Sxx = (1-3)2 +...+(5-3)2 = 10
n _
i=1
Corrected sum of squares for y:
!(yi -y)2 = Syy = (5-7)2 +...+(8-7)2 = 10
n _
i=1
Corrected sum of cross products = Sxy =
!(xi - x)(yi -y) =
n _ _
i=1
(-2)(-2)+(-1)(0)+...+(2)(1) = 5 =
!xi yi -n x y = 110-5(3)(7)
n __
i=1

Slope: b = Sxy /Sxx = 5/10 = 0.5


Intercept:
_ _
y - b x = 7-0.5(3) = 5.5
^
y=5.5 + 0.5x

y 5 7 9 6 8
y^ 6 6.5 7 7.5 8
r=y- y^ -1 0.5 2 -1.5 0

!ri2 = "Error sum of squares" =


n

i=1
SSE = 1+0.25+4+2.25=7.5
SSE is also Syy - S2xy /Sxx = Syy - b2 Sxx =
10-52 /10

Variance of b:

ÈMSE/Sxx is called "standard error" of b.


MSE/Sxx œ 2.5/10 = 0.25.
t = b/È0.25 = 1 which is not an
Task: test H0 : true slope is 0

unusual t.
data a; input x y @@; cards;
15 27 39 46 58
;
proc reg; model Y =X / p;
run;
Dependent Variable: y

Analysis of Variance

Sum of Mean
Source DF Squares Square F Value Pr > F
Model 1 2.50000 2.50000 1.00 0.3910
Error 3 7.50000 2.50000
Corr Total 4 10.00000

Root MSE 1.58114 R-Square 0.2500


Dependent Mean 7.00000 Adj R-Sq 0.0000
Coeff Var 22.58770

Parameter Estimates

Parameter Standard
Variable DF Estimate Error t Value Pr > |t|
Intercept 1 5.50000 1.65831 3.32 0.0452
x 1 0.50000 0.50000 1.00 0.3910

Output Statistics
Dep Var Predicted
Obs y Value Residual

1 5.0000 6.0000 -1.0000


2 7.0000 6.5000 0.5000
3 9.0000 7.0000 2.0000
4 6.0000 7.5000 -1.5000
5 8.0000 8.0000 0

You might also like