Weighted Least Sq (15)

weighted Least Square Afra Burran
WEIGHTED LEAST SQUARE REGRESSION

Student Afra Burran
Supervisor Dr.Suboohi Safdar
What is Weighted Least Squares (WLS)?

WLS is a regression technique that adjusts for differences in variance or importance across data points by
assigning weights to them.
PURPOSE
The purpose of WLS is to improve model accuracy when the assumptions of ordinary least squares (OLS)
regression are violated, specifically the assumption of constant variance in residuals.
HOW IT WORKS
• A variation of the ordinary least squares (OLS) method, WLS adds a weight variable W, to the model.
• The weight are proportional to the inverse of the response variable. The method of ordinary least squares
assumes that there is constant variance in the errors (which is called homoscedasticity).
• The method of weighted least squares can be used when the ordinary least squares assumption of constant
variance in the errors is violated (which is called heteroscedasticity). The model under consideration is
y = xβ + ϵ∗ (1)
• where is assumed to be normally distributed with mean 0 and non-constant variance.
1 WHEN
• Handle non-constant variance of the error terms in linear regression.
• Handle non-constant variance of the error terms in linear regression.
• Account for sample misrepresentation.

• Account for duplicate observations.
2 WHY
• WLS regression is used to correct heteroscedasticity which is when the error variance is non constant.
• In WLS regression more weight is given to observations with smaller variance which are more reliable.
Descember 2024
3 Scatter Plot
x vs y
From this scatter plot, the overall relationship between X and Y appears to be positive linear relationship
but there may be some heteroscedasticity (non-constant variance of Y at different levels of X) and clustering in
the data.
4 SUMMARY OF SIMPE LINEAR REGRESSION

4.1 SIMPE LINEAR REGRESSION (Model Summary):
First, an ordinary least squares line is fit to this data. Below is the summary of the simple linear regression fit
for this data
Multiple Regression 0.916278

R Square 0.839565
Adjusted R Square 0.834703
Standard Error 1.460197
Observations 35
4.2 SIMPE LINEAR REGRESSION(Coefficients):
Coefficients Standard Error t Stat p-value Lower 95% Upper95%

constant -0.58502 0.681146 -0.85888 0.396603 -1.97082 0.80078
X 1.136701 0.086499 13.14117 1.15E-14 0.960717 1.312685
4.3 SIMPE LINEAR REGRESSION(ANOVA):
Source of Variation d.f SS MS F Signi F

Regression 1 368.2064 368.2064 172.6904 1.15E-14
Resudual 33 70.36182 2.132176
Total 34 438.5682
4.4 OLS Equation:

Y = −0.58502 + 1.136701X
2
5 OLS RESIDUALS PLOT
OLS res plot
We checked the assumption of homoscedasticity by plotting the residuals of the

model versus the fitted values. The plot shows a funnel shape. We conclude that
the variance is not constant and the assumption of homoscedasticity is violated.
(Which will bias the standard errors of the regression coefficients, the confidence
intervals, and the p-value.
We will turn to the weighted least squares to address this problem
6 SUMMARY OF WEIGHTED LEAST SQUARES:

6.1 The Weighted Least Squares(Model Summary):
Multiple Regression 0.959

R Square 0.920
Adjusted R Square 0.918
Standard Error 1.143
Log-likelihood Function Value -51.764
6.2 The Weighted Least Squares (Coefficients):
Term Unstandardized Coefficients Standardized Coefficients t Sig

Beta Std. Error Beta Std.Error
Constant -.892 .302 -2.958
X 1.165 .060 .959 .049 19.534 .000
6.3 The Weighted Least Squares (ANOVA):
Source of Variation d.f SS MS F Signi F

Regression 1 498.463 498.463 381.583 .000
Resudual 33 43.108 1.306
Total 34 541.572
6.4 WLS Equation:

Y = −0.892 + 1.165X
3
7 WLS RESIDUALS PLOT
WLS res plot
The WLS residuals appear more evenly spread compared to the OLS residuals,
it indicates that the WLS model effectively handled the heteroscedasticity. OR
it is say that the spread of residuals is more consistent and centered around zero
compared to the OLS residual plot, it demonstrates the effectiveness of WLS.
8 SUMMARY
• The WLS β0 (0.88929) is lower than the OLS β0 (0.58052).
• The difference arises because WLS adjusts for heteroscedasticity, reducing the influence of
data points with high variance, which may pull the OLS β0 upward.
• The β1 in WLS is slightly larger than in OLS .The higher β1 in WLS suggests that ob-
servations with lower variance, which are weighted more heavily, have a stronger positive
relationship between x and y.
• The standard errors of coefficients are significantly smaller in the WLS model. This reduc-
tion indicates that WLS provides more precise estimates due to its ability to account for
heteroscedasticity.
• R2 for WLS (0.92054) is higher than OLS (0.84011) indicating that the WLS model explains
a larger proportion of the variance in Y.This suggests that WLS provides a better fit to the
data.
• The standard error of the estimate of WLS is lower than OLS further indicating that the
WLS model produces more accurate predictions with less error.OR it is more efficient when
heteroscedasticity is present.
9 NPP
NPP
4
Residuals generally fall on a straight line, implying that the errors are normally
distributed. This implies that the proposed model is adequate and excludes any
possibility of independence violation or a constant variance assumption.

Weighted Least Sq (15)

Uploaded by

Copyright:

Available Formats

Weighted Least Sq (15)

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Weighted Least Sq (15)

Uploaded by

Copyright:

Available Formats

weighted Least Square Afra Burran

WEIGHTED LEAST SQUARE REGRESSION

What is Weighted Least Squares (WLS)?

• where is assumed to be normally distributed with mean 0 and non-constant variance.

• Account for sample misrepresentation.

4 SUMMARY OF SIMPE LINEAR REGRESSION

Multiple Regression 0.916278

4.2 SIMPE LINEAR REGRESSION(Coefficients):

Coefficients Standard Error t Stat p-value Lower 95% Upper95%

4.3 SIMPE LINEAR REGRESSION(ANOVA):

Source of Variation d.f SS MS F Signi F

4.4 OLS Equation:

OLS res plot

We checked the assumption of homoscedasticity by plotting the residuals of the

6 SUMMARY OF WEIGHTED LEAST SQUARES:

Multiple Regression 0.959

6.2 The Weighted Least Squares (Coefficients):

Term Unstandardized Coefficients Standardized Coefficients t Sig

6.3 The Weighted Least Squares (ANOVA):

Source of Variation d.f SS MS F Signi F

6.4 WLS Equation:

WLS res plot

You might also like