THE SIMPLE LINEAR REGRESSION ★​ where NID stands for normally

MODEL AND ITS ASSUMPTIONS independently distributed

○​ the quantities in the ()
★​ the relationship between a response var Y denote the mean nd the
and a predictor var X is postulated as a variance, respectively, of the
linear model: norm distribution

parameter estimation
○​ where β0 and β1 are ★​ we estimate the para using the least
constants called the model squares method
regression coefficients or ○​ gives the line that minimizes the
parameters (also regre beta sum of squares of the vertical
coef) distances from each point to the
○​ B1 is the slope of the regre line
line ○​ these errors can be obtained as
○​ ε is a random disturbance or
error (also known as residual
errors) ★​ the sum of the squares of these distances
★​ the obsv on the dependent var Y can then be written as
○​ assumed to be random obsv
from populations of random
vars wt the mean of each
popu given by the expected
★​ the values of β0 and β1 that minimize the
value of E[Y]
equ above r given by
○​ the deviation of an
observation Y from its
population mean is taken
into acc by adding a random
and
error ε
★​ the random errors ε𝑖 have zero
mean ★​ the estimates 𝐵0 and 𝐵1 r called least
★​ the random errors are assumed to squares estimates of β0 and β1 bc they r
have a common variance or the solution to the least squares method
homoscedastic and to be pairwise
○​ the intercept nd the slope of the line
independent
that has the smallest possible sum
★​ the random errors are assumed to
of squares of the vertical distances
be normally distributed
from each point to the line
○​ implies that the Y are also
○​ for this reason, the line is called the
normally distributed
least squares regression line which
○​ the random error is given by
assumptions are frequently
stated as
★​ fitted values
 ○​ for each obsv in our data, we can ➔​ after we compute the least squares
compute estimates of the parameters of a linear
model, let us compute the following
quantities

○​ thus, the 𝑖th fitted value, 𝑦𝑖, is the

point on the least squares regre
line of equation corresponding to
𝑥𝑖
○​ the vertical distance corresponding
to the 𝑖th obsv is

★​ these vertical distances r called ordinary ➔​ a fundamental equality, in both slr n mlr is
least squares residuals given by
○​ property: their sum is equal to zero SST = SSR + SSE
○​ means that the sum of the dist ➔​ this arises from the descrip of an
above the line is = to the sum of the obsv as
dist below d line
★​ ex.
➔​ subtracting y bar from both sides we
obtain

2
➔​ the ratio 𝑅 = SSR/SST can be interpreted
as the proportion of the total variation in
Y that is accounted for by the predictor
test of hypotheses var X
➔​ hypotheses for the constant β0 2
◆​ 𝑅 is called as the goodness of fit
index
2
◆​ we can rewrite 𝑅 as
➔​ hypotheses for the slope β1

◆​ additionally, it can b shown that

measuring the quality of fit
➔​ altho scatter plots of Y vs 𝑌 and corr(Y,𝑌) r
2
redundant in slr, they give us indication of ➔​ note that 𝑅 is greater than or equal to 0
the qual of fit in both simple n multi but less than or equal to 1 bc SSE is less
regression than or equal to SST
2
➔​ in both slr n mlr, corr(Y,𝑌) is related to ◆​ 𝑅 is near 1, then X accounts for a
another useful measure of the quality of fit large part of the variation of Y
of the linear model to the obsvd data
 2
◆​ 𝑅 is known ad the coefficient of residual standard error (rse)
determination bc it gives us an idea ➔​ ave variation of points around the fitted
of how the predictor var X regre line
accounts for the response var Y ➔​ one of the metrics used to evaluate the
2 overall quality of the fitted regre model
◆​ the same interpretation of 𝑅 will
➔​ the lower the rse, the better it is !!!
carry over to the case of multiple
regression
since the mean error term is zero, the outcome var
y can b approximately estimated as follow
SIMPLE LINEAR REGRESSION ANALYSIS
simple linear regression
➔​ used to predict a quantitative outcome y on ➔​ mathematically, the beta coef r determined
the basis of one single predictor var x so that the ssr is as minimal as possible
➔​ goal is to build a mathematical model or ➔​ this method of determining the beta coef is
formu that defines y as a function of the x technically called least squares or ordinary
var least squares (ols) regre.
➔​ once we built a statistically signi model, its ➔​ once the beta coef r calculated, a t-test is
possible to use it for predicting future performed to check whether or not these
outcome on the basis of new x values coef r significantly diff from zero
➔​ assumption ◆​ a non-zero beta coef means that
◆​ rel between the outcome n predictor there is sig rel between the
var is linear nd additive predictors x nd the outcome var y

scatter plot correlation coefficient

➔​ measures the level of the association
between two var x n y
➔​ value ranges between -1 and +1
◆​ -1 (neg corr. when x inc, y dec)
◆​ +1 (pos corr. when x inc, y dec)
➔​ a value closer to zero suggests a weak rel
between vars
◆​ low corr (-0.2 < x < 0.2) prolly
suggests that much of the variation
of the y is not explain by the x
➔​ b4 using the formula to predict future data,
obsv: u shud make sure that the model is
➔​ not all data points fall exactly on the fitted statistically significant
regression line ◆​ there is stat sig rel between
➔​ some r above and some r below predictor nd outcome vars
➔​ overall, the residual errors have ◆​ the model built fits v well the data in
approximately mean zero our hand

residual sum of squares (ssr) model summary

➔​ sum of the squares of the residual errors ➔​ call
 ◆​ shows the function call used to ●​ the more stars, the more
compute the regre model significant
➔​ residuals ➔​ standard errors and ci
◆​ quick view of the distribution of the ◆​ measures the variability/accu of
residuals the beta coef
◆​ have a mean zero ◆​ used to compute the ci of the coef
◆​ median shudnt be far from zero
◆​ min and max shud be roughly equal ​ ​
in abso value note: for 95% ci for B1
➔​ coefficients
◆​ shows the regre beta coef nd their
stat sig
◆​ predictor var: significantly
➔​ model accuracy
associated to the outcome var,
◆​ also referred to as the goodness of
marked by stars *
fit
➔​ t-stats nd p-values
◆​ if atleast 1 var is significantly
◆​ tests whether or not there is a stat
associated the outcome, then
signi rel between x and y
continue the diagnostic by checking
●​ that is whether ot not the
how well the model fits the data
beta coef of the x is signi diff
◆​ rse, r squared, f-stats
from zero
◆​ statistical hypotheses:
additionals: le rev info
●​ null hypo 𝐻0: the coeffs r
★​ in slr, the corr coef (r) and the slope regre
equal to zero line β1 always have the same sign
●​ alt hypo 𝐻𝑎: the coeffs r not ○​ bc both describe the direc of the rs
equal to zero between x n y
◆​ mathematically, for a given beta ★​ the sum of the residuals in slr is always
coeff (β), the t-test is computed as zero if the model includes an intercept term
○​ 𝐵0 !
2
●​ where SE(b) is the standard ★​ coeff of deter (𝑅 ) is can be negative
error of the coef B ★​ multicollinearity is not a concern for slr (its
◆​ t-stats measures the number of for mlr)
standard dev that β is away from 0 ★​ the standard error of the regre slope 𝐵1
●​ thus, a large t-stats wud prod decreases as the variability of the predictor
a small p-value var X increases
●​ the higher t-stats, the more ★​ if the corr between x nd y is zero, then the
sig the predictor slope of the regre line is also zero
●​ the symbols to the right ★​ outliers in the X can signi influence the
visually specifies the lev of slope of the regre line
significance ○​ since it can skew the regre line
●​ one star means 0.01 < p < ★​ residuals in slr shud be normally distributed
0.05
★​ least squares method minimizes the sum
of the squared deviations between
observed and predicted values
★​ sst
○​ sum of squared diff between indiv
data pts and the mean of the
response var Y
○​ estimated the error obtained when
using the target’s mean value to
predict the target’s value
★​ ssr
○​ sum of squared diff between
predicted data pts and the mean of
the resp var Y
○​ measures the deviation of the
target’s predicted values from the
target’s mean
★​ sse
○​ sum of squared diff between
predicted data pts and obsvd data
pts
○​ error sum of sqrs, which is the sum
of squared residuals
★​ rse
○​ provides absolute measure of
patterns in the data that cannot b
explain by the model
○​ closer to 0 the better
★​ r squared & adjusted r squared
○​ ranges from 0 to 1
○​ represents the proportion of info in
the data that can be explained by
the model
○​ adj r squared adjusts for the
degrees of freedom
○​ higher the better
★​ f-statistics
○​ gives the overall significance of the
model
○​ assess whether at least one
predictor var has a non-zero coef
○​ identical to the square of the t-test
(1 df)
○​ a large f-stats correspond to a
statistically significant p-value (p
< 0.05)
○​ higher the better