Scribd
Upload
12 views14 pages

Week 5 Lecture Q A

The document discusses quantitative analysis and multiple linear regression models. It defines linearity as a direct proportional relationship between predictor and response variables that can be represented by a straight line. A multiple linear regression model extends this relationship to account for multiple independent variables influencing a dependent variable. The key parameters of a multiple linear regression model (intercept and coefficients) can be estimated using formulas involving sums of the variables. Matrix notation provides a compact way to represent the model, data, and results.

Uploaded by

Huma Rehman
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
12 views14 pages

Week 5 Lecture Q A

The document discusses quantitative analysis and multiple linear regression models. It defines linearity as a direct proportional relationship between predictor and response variables that can be represented by a straight line. A multiple linear regression model extends this relationship to account for multiple independent variables influencing a dependent variable. The key parameters of a multiple linear regression model (intercept and coefficients) can be estimated using formulas involving sums of the variables. Matrix notation provides a compact way to represent the model, data, and results.

Uploaded by

Huma Rehman
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

QUANTITATIVE ANALYSIS (FOMS)

Week 5 – Lecture

Shafiq Ur Rehman
Lecturer (Statistics)
Department of Mathematics & Statistics
University of Central Punjab, Lahore.
WHAT IS “LINEARITY”?
The term "linear" refers to the relationship between the predictor variable(s) and the response variable.
A linear relationship means that the change in the response variable is directly proportional to the change in
the predictor variable. Mathematically, this can be represented by a straight line when the predictor variable
is plotted on the x-axis and the response variable is plotted on the y-axis.
For instance, if we are trying to predict the price of a house based on its size, a linear relationship would
mean that as the size of the house increases, the price of the house also increases at a constant rate. In other
words, a linear regression model would assume that the relationship between the size of the house and its
price can be expressed by a straight line.

MULTIPLE LINEAR REGRESSION MODEL


In multiple regression analysis, the model for simple linear regression is extended to account for the
relationship between the dependent variable y and p independent variables i.e., x1, x2, . . ., xp. The general
form of the multiple regression model is
y = 𝛼 + β1 x1 + β2 x2 + . . . + βk xk + ε
Where,
𝛼 is the intercept term; 𝛽𝑖 is coefficient of the ith explanatory variable
𝜀 is the random factor or error term
MULTIPLE LINEAR REGRESSION MODEL
ERROR AND RESIDUALS:
Error: Used for the population data.

The difference between the actual value of the Y and the conditional/ expected value of the Y on the given
X-variable. i.e., Error (for ith observation) 𝜇𝑖 = 𝑌𝑖 − 𝐸(𝑌 / 𝑋𝑖 ) = 𝑌𝑖 − 𝐸(𝑌𝑖 )

Residual: Used for sample data.

The difference between the actual observed value of Y and “estimate” of the expected value of the response
ε𝑖 = 𝑌𝑖 − 𝐸(𝑌෠ / 𝑋𝑖 ) = 𝑌𝑖 − 𝑌෠𝑖

Suppose that the yield in pounds of conversion in a chemical process depends on temperature and the
catalyst concentration. A multiple regression model that might describe this relationship is
𝑦 = 𝛽0 + 𝛽1 𝑥1 + 𝛽2 𝑥2
where y denotes the yield, x 1 denotes the temperature, and x 2 denotes the catalyst concentration. This is a
multiple linear regression model with two regressor variables.
(Cont.) MULTIPLE LINEAR REGRESSION MODEL
Multiple linear regression models are often used as empirical models or approximating functions. That is,
the true functional relationship between y and x1 , x2 , . . . , xk is unknown, but over certain ranges of the
regressor variables the linear regression model is an adequate approximation to the true unknown function.
SOME IMPORTANT POINTS
1. Models that are more Complex In Structure than previous example may often still be analyzed by
multiple linear regression techniques. For example, consider the cubic polynomial model

If we let x1 = x , x2 = x2 , and x3 = x3 , then above equation can be written as

which is a multiple linear regression model with three regressor variables.


2. Models that include Interaction Effects may also be analyzed by multiple linear regression methods.
For example, suppose that the model is

If we let x3 = x1x2 and β3 = β12 , then above equation can be written as

which is a multiple linear regression model.


MULTIPLE LINEAR REGRESSION MODEL
For multiple linear regression model with two independent variable, the parameters 𝛼, 𝛽1 , 𝛽2 can be
estimated by the following formulas.

∑𝑥1 𝑦 ∑𝑥22 − (∑𝑦𝑥2 )(∑𝑥1 𝑥2 )


𝛽෠1 = 𝑏1 =
∑ 𝑥12 ∑ 𝑥22 − ∑𝑥1 𝑥2 2

∑𝑥2 𝑦 ∑𝑥12 − (∑𝑦𝑥1 )(∑𝑥1 𝑥2 )


𝛽෠2 = 𝑏2 =
∑ 𝑥12 ∑ 𝑥22 − ∑𝑥1 𝑥2 2
𝛼ො = 𝑎 = 𝑌ത − 𝑏1 𝑋1 − 𝑏2 𝑋2
Where,
∑𝑋1 2
∑𝑥12 = ∑𝑋12 −
𝑛

∑𝑋2 2
∑𝑥22 = ∑𝑋22 −
𝑛

∑𝑋1 ∑𝑋2
∑𝑥1 𝑥2 = ∑𝑋1 𝑋2 −
𝑛

∑𝑌 ∑𝑋1
∑𝑦𝑥1 = ∑𝑌𝑋1 −
𝑛

∑𝑌 ∑𝑋2
∑𝑦𝑥2 = ∑𝑌𝑋2 −
𝑛
EXAMPLE
The following data relate to radio advertising expenditures, newspaper advertising expenditures and sales.
Fit a regression 𝑌෠ = 𝑎 + 𝑏1 𝑋1 + 𝑏2 𝑋2.

Radio ad. exp. (’000 Rs) (X1) 4 7 9 12


Newspaper ad. exp. (’000 Rs) (X2) 1 2 5 8
Sales (Rs lakh) (Y ) 7 12 17 20
Solution
Calculating all the required values in the form of a table for convenience

Table
EXAMPLE
Putting the values into the formulas we get
∑𝑋1 2 322
∑𝑥12 = ∑𝑋12 − = 290 − = 34
𝑛 4

∑𝑋2 2 162
∑𝑥22 = ∑𝑋22 − = 94 − = 30
𝑛 4

∑𝑋1 ∑𝑋2 32∗16


∑𝑥1 𝑥2 = ∑𝑋1 𝑋2 − = 159 − = 31
𝑛 4

∑𝑌 ∑𝑋1 56∗32
∑𝑦𝑥1 = ∑𝑌𝑋1 − = 505 − = 57
𝑛 4

∑𝑌 ∑𝑋2 56∗16
∑𝑦𝑥2 = ∑𝑌𝑋2 − = 276 − = 52
𝑛 4

Table
EXAMPLE
Putting the values into the formulas we get
∑𝑥12 = 34; ∑𝑥22 =30; ∑𝑥1 𝑥2 =31; ∑𝑦𝑥1 = 57; ∑𝑦𝑥2 = 52

∑𝑥1 𝑦 ∑𝑥22 − (∑𝑦𝑥2 )(∑𝑥1 𝑥2 ) 57 ∗ 30 − (52 ∗ 31)


𝛽෠1 = 𝑏1 = = = 1.661
∑ 𝑥12 ∑ 𝑥22 − ∑𝑥1 𝑥2 2 34 ∗ 30 − 31 2

∑𝑥2 𝑦 ∑𝑥12 − (∑𝑦𝑥1 )(∑𝑥1 𝑥2 ) 52 ∗ 34 − (57 ∗ 31)


𝛽෠2 = 𝑏2 = = = 0.0169
∑ 𝑥12 ∑ 𝑥22 − ∑𝑥1 𝑥2 2 34 ∗ 30 − 31 2

𝛼ො = 𝑎 = 𝑌ത − 𝑏1 𝑋1 − 𝑏2 𝑋2 = 14 − 1.661 ∗ 8 − 0.0169 ∗ 4 = 4.6036

So the estimated model for the given data set is


𝑌෠ = 𝑎 + 𝑏1 𝑋1 + 𝑏2 𝑋2 = 4.6036 + 1.661 𝑋1 + 0.0169 𝑋2
MATRIX FORM OF THE MULITPLE LINEAR REGRESSION
The method of least squares can be used to estimate the regression coefficients in
y = 𝛼 + β1 x1 + β2 x2 + . . . + βk xk + ε
Suppose that n > k observations are available, and let yi denote the ith observed response and xij denote the ith
observation or level of regressor xj.

Note that there are p = k + 1 normal equations, one for each of the unknown regression coefficients. The
solution to the normal equations will be the least – squares estimators 𝛽෠0 , 𝛽෠1 , … , 𝛽෠𝑘 .
It is more convenient to deal with multiple regression models if they are expressed in matrix notation. This
allows a very compact display of the model, data, and results.
𝒚 = 𝑿𝜷 + 𝜺
Where
ESTIMATION OF THE MODEL PARAMETERS
𝒚 = 𝑿𝜷 + 𝜺
Where,

In general, y is an “n × 1” vector of the observations, X is an “n × p” matrix of the levels of the regressor


variables, β is a “p × 1” vector of the regression coefficients, and ε is an “n × 1” vector of random errors.
The least - squares estimator of β is
෡ = 𝑿𝑿`
𝜷 −𝟏
𝑿`𝒀
provided that the inverse matrix ( X ′ X ) − 1 exists. The ( X ′ X ) − 1 matrix will always exist if the
regressors are linearly independent , that is, if no column of the X matrix is a linear combination of the other
columns.
TITLE
Text
TITLE
Text
TITLE
Text
TITLE
Text

You might also like