Scribd
Upload
12 views9 pages

CE463 Fall 2024 Regression Lecture 13

Uploaded by

agrawaladi09
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
12 views9 pages

CE463 Fall 2024 Regression Lecture 13

Uploaded by

agrawaladi09
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

MLE Estimator for 𝝈 𝟐

𝑛
𝜕𝐿 𝜃 𝑛 1 2
=0 − 2 + 4 ෍ 𝑦𝑖 − 𝛽0 − 𝛽1 𝑥𝑖 =0
𝜕𝜎 2 2𝜎 2𝜎
𝑖=1
𝑛
1 2
−𝑛 + 2
෍ 𝑦𝑖 − 𝛽0 − 𝛽1 𝑥𝑖 =0
𝜎
𝑖=1
𝑛
෢ 1 2 𝑅𝑆𝑆
2 ෢0 − 𝛽
𝜎 = ෍ 𝑦𝑖 − 𝛽 ෢1 𝑥𝑖 =
𝑛 𝑛
𝑖=1

𝑅𝑆𝑆
Correcting for error d.f., 𝜎෢2 =
𝑛−2

2024 21
Standard error of MLE estimators
 Following properties of MLE estimators,
𝑛
1
෢0
𝛽 𝛽0 𝜎2 ෍ 𝑥𝑖2 −𝑥ҧ
~MVN2 , 𝑛
෢
𝛽1 𝛽1 𝑆𝑥𝑥 𝑖=1 Same as
−𝑥ҧ 1 MLS
estimators

𝑅𝑆𝑆 2
 The distribution of 𝜎 2 is given by ~χ𝑛−2 and is independent of
𝜎2
෢0
𝛽
the regression coefficients .
෢
𝛽1

 The true value of 𝜎 2 is not known.

2024 22
Confidence intervals of MLE estimators

෢0 and 𝛽
100(1-α)% confidence intervals for 𝛽 ෢1 given by

𝛽መ0 ± 𝑡𝑛−2,1−α/2 ∗ SE (𝛽
෢0)
𝛽መ1 ± 𝑡𝑛−2,1−α/2 ∗ SE (𝛽
෢1)

2024 23
Hypothesis testing on regression
parameters
 One is typically interested in making inferences on 𝛽1 .
 E.g. Is there association between advertisement and sales?

 Null hypothesis H0: 𝛽1 = 0


 Alternate hypothesis H1: 𝛽1 ≠ 0

 Define a test-statistic t.

 Reject H0 based on either fixed confidence or p-value.

 One can also carry out hypothesis testing on 𝛽0 .

2024 24
Hypothesis Testing on Regression Coefficients

Hypothesis Fixed significance testing p-value

H0 : 𝛽 ≥ 𝛽 ∗ ; H1 : 𝛽 < 𝛽 ∗ Reject H0 if 𝑡𝑜𝑏𝑠 < 𝑡𝑛−2,𝛼 Prob (𝑡𝑛−2 < 𝑡𝑜𝑏𝑠 )

H0 : 𝛽 ≤ 𝛽 ∗ ; H1 : 𝛽 > 𝛽 ∗ Reject H0 if 𝑡𝑜𝑏𝑠 > 𝑡𝑛−2,𝛼 Prob (𝑡𝑛−2 > 𝑡𝑜𝑏𝑠 )

H0 : 𝛽 = 𝛽 ∗ ; H1 : 𝛽 ≠ 𝛽 ∗ Reject H0 if 𝑡𝑜𝑏𝑠 > 𝑡𝑛−2,1−𝛼 2*Prob (𝑡𝑛−2 > 𝑡𝑜𝑏𝑠 )


2

2024 25
Assessing accuracy/ goodness of fit
 Important because ε always results in ‘misfit ‘.

 Standard deviation of error(σ):


 Average amount of deviation from the regression line
 Takes into account the variability that can not be explained by the model

 The lack of fit is therefore


𝑅𝑆𝑆 σ𝑛 𝑦𝑖 )2
𝑖=1(𝑦𝑖 −ෞ
σ = RSE=
ෝ =
𝑛−2 𝑛−2

 But RSE depends on the unit of Y!

2024 26
Assessing accuracy/ goodness of fit
 To avoid scale dependency, coefficient of determination (R2 ) is
used.
𝑇𝑆𝑆 − 𝑅𝑆𝑆
𝑅2 =
𝑇𝑆𝑆
TSS = σ𝑛𝑖=1(𝑦𝑖 − 𝑦)
ത 2 [Total sum of squares]

 0 ≪ 𝑅2 ≪ 1.

 R2 measures the proportion of variability in Y that can be


explained by ……………….. effects of X.

 For simple linear regression, 𝑅 2 = 𝑟𝑥𝑦


2

2024 27
Example#1
Data of discharge versus rainfall is provided to you. Find out the
regression coefficients and test the hypotheses that rainfall has no linear
effect on discharge.

2024 28
T-table

2024 29

You might also like