Scribd Logo
Upload

Welcome to Scribd!

  • 20 views

    Lecture-2 Least Squares Regression

    Uploaded by

    Dr. Sarfaraz Bhutto
    The document discusses least squares regression and the Gauss-Markov theorem. The Gauss-Markov theorem states that if its assumptions are met, ordinary least squares gives the best linear unbiased estimate of regression coefficients. The assumptions are linearity, random sampling, non-collinearity of regressors, exogeneity, and homoscedasticity of error terms. Ordinary least squares finds the line of best fit by minimizing the sum of squared vertical distances between the data points and regression line.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd

    Lecture-2 Least Squares Regression

    Uploaded by

    Dr. Sarfaraz Bhutto
    20 views18 pages
    The document discusses least squares regression and the Gauss-Markov theorem. The Gauss-Markov theorem states that if its assumptions are met, ordinary least squares gives the best linear unbiased estimate of regression coefficients. The assumptions are linearity, random sampling, non-collinearity of regressors, exogeneity, and homoscedasticity of error terms. Ordinary least squares finds the line of best fit by minimizing the sum of squared vertical distances between the data points and regression line.

    Copyright

    © © All Rights Reserved

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses least squares regression and the Gauss-Markov theorem. The Gauss-Markov theorem states that if its assumptions are met, ordinary least squares gives the best linear unbiased estimate of regression coefficients. The assumptions are linearity, random sampling, non-collinearity of regressors, exogeneity, and homoscedasticity of error terms. Ordinary least squares finds the line of best fit by minimizing the sum of squared vertical distances between the data points and regression line.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    20 views18 pages

    Lecture-2 Least Squares Regression

    Uploaded by

    Dr. Sarfaraz Bhutto
    The document discusses least squares regression and the Gauss-Markov theorem. The Gauss-Markov theorem states that if its assumptions are met, ordinary least squares gives the best linear unbiased estimate of regression coefficients. The assumptions are linearity, random sampling, non-collinearity of regressors, exogeneity, and homoscedasticity of error terms. Ordinary least squares finds the line of best fit by minimizing the sum of squared vertical distances between the data points and regression line.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    of 18

    Least Squares Regression

    Gauss Markov Theorem

    The Gauss Markov theorem tells us that if a certain set


    of assumptions are met, the ordinary least squares
    estimate for regression coefficients gives you the best
    linear unbiased estimate (BLUE) possible.
    Gauss Markov Assumptions

    • Linearity: the parameters we are estimating using the OLS method


    must be themselves linear.
    • Random: our data must have been randomly sampled from the
    population.
    • Non-Collinearity: the regressors being calculated aren’t perfectly
    correlated with each other.
    • Exogeneity: the regressors aren’t correlated with the error term.
    • Homoscedasticity: no matter what the values of our regressors might
    be, the error of the variance is constant.
    Linearity

    The parameters we are estimating using the OLS method must be


    themselves linear.
    Random

    Random samples are used to avoid bias and other unwanted


    effects.
    Non-Collinearity

    A set of points is collinear if you can draw one line through


    them all. The word literally means “together on a line.”
    Exogeneity

    Exogeneity is a standard assumption made in regression


    analysis, and when used in reference to a regression equation
    tells us that the independent variables X are not dependent on
    the dependent variable (Y)
    Homoscedasticity
    What is a Least Squares Regression Line?

    • If your data shows a linear relationship between the X and Y variables,


    you will want to find the line that best fits that relationship.
    • That line is called a Regression Line and has the equation ŷ= a + b x.
    • The Least Squares Regression Line is the line that makes the vertical
    distance from the data points to the regression line as small as
    possible.
    • It’s called a “least squares” because the best line of fit is one that
    minimizes the variance (the sum of squares of the errors).
    • This can be a bit hard to visualize but the main point is you are aiming
    to find the equation that fits the points as closely as possible.
    What is Least Squares Fitting?

    • Least squares fitting (also called least squares estimation) is a way to


    find the best fit curve or line for a set of points.
    • In this technique, the sum of the squares of the offsets (residuals) are
    used to estimate the best fit curve or line instead of the absolute
    values of the offsets.
    • The resulting equation gives you a y-value for any x-value, not just
    those x and y values plotted with points.
    Advantages of least squares fitting

    • Least squares allows the residuals to be treated as a continuous


    quantity where derivatives (measures of how much a function’s
    output changes when an input changes) can be found. This is
    invaluable, as the point of finding an equation in the first place is to
    be able to predict where other points on the line (even points that are
    way beyond the original points) might lie.
    Disadvantages of Least Squares Fitting

    • Outliers can have a disproportionate effect if you use the least


    squares fitting method of finding an equation for a curve. This is
    because the squares of the offsets are used instead of the absolute
    value of the offsets; outliers naturally have larger offsets and will
    affect the line more than points closer to the line. These
    disproportionate values may be beneficial in some cases.
    Ordinary Least Squares

    • Ordinary least squares regression is a way to find the line of best fit
    for a set of data. It does this by creating a model that minimizes the
    sum of the squared vertical distances (residuals).
    • The distances are squared to avoid the problem of distances with a
    negative sign. Then the problem just becomes figuring out where you
    should place the line so that the distances from the points to the line
    are minimized. In the following image, the best fit line A has smaller
    distances from the points to the line than the randomly placed line B.
    The Residuals
    • The difference between the observed and predicted values of y is the
    error, or residual.

    You might also like