Scribd
Upload
33 views5 pages

3 Linear Regression 1

1) Linear regression is a statistical approach for modeling the relationship between a dependent variable and one or more independent variables. It can be used to predict a quantitative outcome based on values of the predictor variables. 2) The document discusses linear regression in the context of an advertising dataset containing sales data and advertising expenditure data across different media for 200 markets. 3) Key questions addressed by linear regression for this dataset include determining the strength and nature of the relationship between advertising expenditure and sales, and which media contribute most to sales. Linear regression can estimate the effect of each medium and accurately predict future sales.

Uploaded by

neuro.ultragod
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
33 views5 pages

3 Linear Regression 1

1) Linear regression is a statistical approach for modeling the relationship between a dependent variable and one or more independent variables. It can be used to predict a quantitative outcome based on values of the predictor variables. 2) The document discusses linear regression in the context of an advertising dataset containing sales data and advertising expenditure data across different media for 200 markets. 3) Key questions addressed by linear regression for this dataset include determining the strength and nature of the relationship between advertising expenditure and sales, and which media contribute most to sales. Linear regression can estimate the effect of each medium and accurately predict future sales.

Uploaded by

neuro.ultragod
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

9/22/2021

Linear Regression

Introduction

• Linear regression, a very simple Statistical approach for


supervised learning.

• Used for modelling relationship between a dependent variable


with a given set of independent variables

• Useful tool for predicting a quantitative response.

• Many fancy statistical learning approaches can be seen as


generalizations or extensions of linear regression.

1
9/22/2021

Dataset
The Advertising dataset. The figure displays sales (in thousands of
units) for a particular product as a function of advertising budgets
(in thousands of dollars) for TV, radio, and newspaper media in
200 different markets.

Important Questions

1. Is there a relationship between advertising budget and sales?


2. Is the relationship linear?
3. How strong is the relationship between advertising budget
and sales?
4. Which media contribute to sales?
5. How accurately can we estimate the effect of each medium on
sales?
6. How accurately can we predict future sales?
7. Is there synergy among the advertising media?

2
9/22/2021

Simple Linear Regression


• Approach for predicting a quantitative response Y based on a
single predictor variable X.

• Assumes a linear relationship between X and Y.

• Use the training data to produce estimates 𝛽መ0 and 𝛽መ1 for the
model coefficients; then predict future sales based on a
particular value of TV advertising

Estimating the coefficients


• Goal: obtain coefficient estimates such that the linear model fits
the best the available data.
• the resulting line is as close as possible to the training observations.

• The most common approach involves minimizing the least


squares criterion.

2
1 n
MSE = å Yi - Yˆi
n i=1
( )
1 n
(
= å Yi - b̂0 - b̂1 X1 - )
2
- b̂ p X p
n i=1

3
9/22/2021

Least Squares Fit

The least squares approach chooses


the coefficient estimates to minimize
the RSS

Some Relevant Questions


Estimated (least squares) line.
• The regression line from the
sample is not the regression line
from the population.

• What we want to do:


• Assess how well the estimated line
describes the true population line.
• Guess the slope of the population
line.
• Guess what value Y would take for
a given X value

True (population) line. Unobserved

4
9/22/2021

Hypothesis Testing
• Is Xj an important variable?
• We use a hypothesis test to answer this question

• H0 :There is no relationship between X and Y (b j=0)


• Ha :There is some relationship between X and Y (b j0)

• To test the null hypothesis, we need to determine whether 𝛽መ1 ,


our estimate for 𝛽1 , is sufficiently far from zero; be confident
that 𝛽1 is non-zero.

Number of standard deviations


• Calculate t-statistic away from zero.

• If t is large (equivalently p-value is small) we can be sure that


b j0 and that there is a relationship.

• P-value: a small p-value indicates that it is unlikely to observe


such a substantial association between the predictor and the
response due to chance, in the absence of any real association.
• Small p-value => there is an association between predictor and response
• reject the null hypothesis => relationship exists between X and Y.

bˆ1 SE ( bˆ1 )
ˆ
b 1 is 17.67 SE’s from 0
P-value
10

10

You might also like