0% found this document useful (0 votes)
15 views5 pages

Stats135 Reviewer

The document provides an overview of the simple linear regression model, detailing its assumptions, parameter estimation, and the least squares method used to minimize errors. It explains the significance of coefficients, the goodness of fit measures such as R-squared, and the importance of residual analysis in evaluating model accuracy. Additionally, it discusses the relationship between predictor and response variables, emphasizing the linearity and statistical significance required for effective predictions.

Uploaded by

feliz
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
15 views5 pages

Stats135 Reviewer

The document provides an overview of the simple linear regression model, detailing its assumptions, parameter estimation, and the least squares method used to minimize errors. It explains the significance of coefficients, the goodness of fit measures such as R-squared, and the importance of residual analysis in evaluating model accuracy. Additionally, it discusses the relationship between predictor and response variables, emphasizing the linearity and statistical significance required for effective predictions.

Uploaded by

feliz
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

THE SIMPLE LINEAR REGRESSION ★​ where NID stands for normally

MODEL AND ITS ASSUMPTIONS independently distributed


○​ the quantities in the ()
★​ the relationship between a response var Y denote the mean nd the
and a predictor var X is postulated as a variance, respectively, of the
linear model: norm distribution

parameter estimation
○​ where β0 and β1 are ★​ we estimate the para using the least
constants called the model squares method
regression coefficients or ○​ gives the line that minimizes the
parameters (also regre beta sum of squares of the vertical
coef) distances from each point to the
○​ B1 is the slope of the regre line
line ○​ these errors can be obtained as
○​ ε is a random disturbance or
error (also known as residual
errors) ★​ the sum of the squares of these distances
★​ the obsv on the dependent var Y can then be written as
○​ assumed to be random obsv
from populations of random
vars wt the mean of each
popu given by the expected
★​ the values of β0 and β1 that minimize the
value of E[Y]
equ above r given by
○​ the deviation of an
observation Y from its
population mean is taken
into acc by adding a random
and
error ε
★​ the random errors ε𝑖 have zero
mean ★​ the estimates 𝐵0 and 𝐵1 r called least
★​ the random errors are assumed to squares estimates of β0 and β1 bc they r
have a common variance or the solution to the least squares method
homoscedastic and to be pairwise
○​ the intercept nd the slope of the line
independent
that has the smallest possible sum
★​ the random errors are assumed to
of squares of the vertical distances
be normally distributed
from each point to the line
○​ implies that the Y are also
○​ for this reason, the line is called the
normally distributed
least squares regression line which
○​ the random error is given by
assumptions are frequently
stated as
★​ fitted values
○​ for each obsv in our data, we can ➔​ after we compute the least squares
compute estimates of the parameters of a linear
model, let us compute the following
quantities

○​ thus, the 𝑖th fitted value, 𝑦𝑖, is the


point on the least squares regre
line of equation corresponding to
𝑥𝑖
○​ the vertical distance corresponding
to the 𝑖th obsv is

★​ these vertical distances r called ordinary ➔​ a fundamental equality, in both slr n mlr is
least squares residuals given by
○​ property: their sum is equal to zero SST = SSR + SSE
○​ means that the sum of the dist ➔​ this arises from the descrip of an
above the line is = to the sum of the obsv as
dist below d line
★​ ex.
➔​ subtracting y bar from both sides we
obtain

2
➔​ the ratio 𝑅 = SSR/SST can be interpreted
as the proportion of the total variation in
Y that is accounted for by the predictor
test of hypotheses var X
➔​ hypotheses for the constant β0 2
◆​ 𝑅 is called as the goodness of fit
index
2
◆​ we can rewrite 𝑅 as
➔​ hypotheses for the slope β1

◆​ additionally, it can b shown that


measuring the quality of fit
➔​ altho scatter plots of Y vs 𝑌 and corr(Y,𝑌) r
2
redundant in slr, they give us indication of ➔​ note that 𝑅 is greater than or equal to 0
the qual of fit in both simple n multi but less than or equal to 1 bc SSE is less
regression than or equal to SST
2
➔​ in both slr n mlr, corr(Y,𝑌) is related to ◆​ 𝑅 is near 1, then X accounts for a
another useful measure of the quality of fit large part of the variation of Y
of the linear model to the obsvd data
2
◆​ 𝑅 is known ad the coefficient of residual standard error (rse)
determination bc it gives us an idea ➔​ ave variation of points around the fitted
of how the predictor var X regre line
accounts for the response var Y ➔​ one of the metrics used to evaluate the
2 overall quality of the fitted regre model
◆​ the same interpretation of 𝑅 will
➔​ the lower the rse, the better it is !!!
carry over to the case of multiple
regression
since the mean error term is zero, the outcome var
y can b approximately estimated as follow
SIMPLE LINEAR REGRESSION ANALYSIS
simple linear regression
➔​ used to predict a quantitative outcome y on ➔​ mathematically, the beta coef r determined
the basis of one single predictor var x so that the ssr is as minimal as possible
➔​ goal is to build a mathematical model or ➔​ this method of determining the beta coef is
formu that defines y as a function of the x technically called least squares or ordinary
var least squares (ols) regre.
➔​ once we built a statistically signi model, its ➔​ once the beta coef r calculated, a t-test is
possible to use it for predicting future performed to check whether or not these
outcome on the basis of new x values coef r significantly diff from zero
➔​ assumption ◆​ a non-zero beta coef means that
◆​ rel between the outcome n predictor there is sig rel between the
var is linear nd additive predictors x nd the outcome var y

scatter plot correlation coefficient


➔​ measures the level of the association
between two var x n y
➔​ value ranges between -1 and +1
◆​ -1 (neg corr. when x inc, y dec)
◆​ +1 (pos corr. when x inc, y dec)
➔​ a value closer to zero suggests a weak rel
between vars
◆​ low corr (-0.2 < x < 0.2) prolly
suggests that much of the variation
of the y is not explain by the x
➔​ b4 using the formula to predict future data,
obsv: u shud make sure that the model is
➔​ not all data points fall exactly on the fitted statistically significant
regression line ◆​ there is stat sig rel between
➔​ some r above and some r below predictor nd outcome vars
➔​ overall, the residual errors have ◆​ the model built fits v well the data in
approximately mean zero our hand

residual sum of squares (ssr) model summary


➔​ sum of the squares of the residual errors ➔​ call
◆​ shows the function call used to ●​ the more stars, the more
compute the regre model significant
➔​ residuals ➔​ standard errors and ci
◆​ quick view of the distribution of the ◆​ measures the variability/accu of
residuals the beta coef
◆​ have a mean zero ◆​ used to compute the ci of the coef
◆​ median shudnt be far from zero
◆​ min and max shud be roughly equal ​ ​
in abso value note: for 95% ci for B1
➔​ coefficients
◆​ shows the regre beta coef nd their
stat sig
◆​ predictor var: significantly
➔​ model accuracy
associated to the outcome var,
◆​ also referred to as the goodness of
marked by stars *
fit
➔​ t-stats nd p-values
◆​ if atleast 1 var is significantly
◆​ tests whether or not there is a stat
associated the outcome, then
signi rel between x and y
continue the diagnostic by checking
●​ that is whether ot not the
how well the model fits the data
beta coef of the x is signi diff
◆​ rse, r squared, f-stats
from zero
◆​ statistical hypotheses:
additionals: le rev info
●​ null hypo 𝐻0: the coeffs r
★​ in slr, the corr coef (r) and the slope regre
equal to zero line β1 always have the same sign
●​ alt hypo 𝐻𝑎: the coeffs r not ○​ bc both describe the direc of the rs
equal to zero between x n y
◆​ mathematically, for a given beta ★​ the sum of the residuals in slr is always
coeff (β), the t-test is computed as zero if the model includes an intercept term
○​ 𝐵0 !
2
●​ where SE(b) is the standard ★​ coeff of deter (𝑅 ) is can be negative
error of the coef B ★​ multicollinearity is not a concern for slr (its
◆​ t-stats measures the number of for mlr)
standard dev that β is away from 0 ★​ the standard error of the regre slope 𝐵1
●​ thus, a large t-stats wud prod decreases as the variability of the predictor
a small p-value var X increases
●​ the higher t-stats, the more ★​ if the corr between x nd y is zero, then the
sig the predictor slope of the regre line is also zero
●​ the symbols to the right ★​ outliers in the X can signi influence the
visually specifies the lev of slope of the regre line
significance ○​ since it can skew the regre line
●​ one star means 0.01 < p < ★​ residuals in slr shud be normally distributed
0.05
★​ least squares method minimizes the sum
of the squared deviations between
observed and predicted values
★​ sst
○​ sum of squared diff between indiv
data pts and the mean of the
response var Y
○​ estimated the error obtained when
using the target’s mean value to
predict the target’s value
★​ ssr
○​ sum of squared diff between
predicted data pts and the mean of
the resp var Y
○​ measures the deviation of the
target’s predicted values from the
target’s mean
★​ sse
○​ sum of squared diff between
predicted data pts and obsvd data
pts
○​ error sum of sqrs, which is the sum
of squared residuals
★​ rse
○​ provides absolute measure of
patterns in the data that cannot b
explain by the model
○​ closer to 0 the better
★​ r squared & adjusted r squared
○​ ranges from 0 to 1
○​ represents the proportion of info in
the data that can be explained by
the model
○​ adj r squared adjusts for the
degrees of freedom
○​ higher the better
★​ f-statistics
○​ gives the overall significance of the
model
○​ assess whether at least one
predictor var has a non-zero coef
○​ identical to the square of the t-test
(1 df)
○​ a large f-stats correspond to a
statistically significant p-value (p
< 0.05)
○​ higher the better

You might also like