MFIN 305: Quantitative

Methods of Finance
Further Development of the Classical Linear Regression
Model
1. Generalizing the Simple Model to Multiple
Linear Regression
𝑦𝑡 = 𝛼 + 𝛽𝑥𝑡 + 𝑢𝑡 𝑡 = 1,2, … , 𝑇
• What if the dependent variable is influenced by more than one x?
• Arbitrage pricing theory, for example, is not a single factor model like the CAPM

• 𝑅𝑖𝑡 = 𝐸 𝑅𝑖𝑡 + σ𝑘𝑗=1 𝛽𝑖𝑗 𝐹𝑖𝑗 − 𝐸 𝐹𝑖𝑗 + 𝑢𝑖𝑡

• Stock returns might depend on unexpected changes in:
(1) Inflation
(2) Differences in returns on short- and long-dated bonds (i.e., slope of the term
structure of interest rates)
(3) Industrial production
(4) Default risk measured as the difference between BAA and AAA rated corporate
bonds
1. The Yield Curve or the Terms Structure of
Interest Rates
1. The US Yield Curve
2019-10-25
3

2.5

1.5

0.5

0
1. The US Yield Curve: Movie
2008-07-22
6

0
1. Generalizing the Simple Model to Multiple
Linear Regression
Generalize the simple model to one with 𝑘 regressors (interdependent
variables)

𝑦𝑡 = 𝛽1 + 𝛽2 𝑥2𝑡 + 𝛽3 𝑥3𝑡 + ⋯ + 𝛽𝑘 𝑥𝑘𝑡 + 𝑢𝑡 , 𝑡 = 1,2, … , 𝑇

• 𝑥2t , 𝑥3t , … , 𝑥𝑘t are a set of 𝑘 − 1 explanatory variables thought to

influence 𝑦
• 𝛽1 , 𝛽2 , … , 𝛽𝑘 are parameters that quantify the effect of these
explanatory variables on 𝑦
1. Generalizing the Simple Model to Multiple
Linear Regression
• Each coefficient is a partial regression coefficient, interpreted as
representing the partial effect of a given explanatory variable on the
explained variable, after holding constant, all other explanatory
variables
• The partial regression coefficient can also be viewed as a net effect:
∆𝑦ො𝑡 ∆𝑦ො𝑡
, …
∆𝑥2𝑡 ∆𝑥3𝑡

• i.e., each coefficient measures the average change in 𝑦 per unit change
in one of the 𝑥s holding all other independent variables constant at
their average values
2. The Constant Term
The constant term is represented by a column of ones of length T
1
1
𝑥1 =
⋮
1
• 𝛽1 is still referred to as the intercept
• 𝑘 = number of explanatory variables including the constant term
= number of estimated parameters in the regression equation
2. The Constant Term
Matrix form of 𝑦𝑡 = 𝛽1 + 𝛽2 𝑥2𝑡 + 𝛽3 𝑥3𝑡 + ⋯ + 𝛽𝑘 𝑥𝑘𝑡 + 𝑢𝑡

𝑦 = 𝑋𝛽 + 𝑢

Column for each x in the 𝑋 matrix

• 𝑦 is a (𝑇 ⨯ 1) vector
• 𝑋 is a 𝑇 ⨯ 𝐾 matrix
• 𝛽 is a (𝐾 ⨯ 1) vector
• 𝑢 is a (𝑇 ⨯ 1) vector
• All time observations are stacked up in a vector
2. The Constant Term
We could write a separate equation for every value of t:
𝑦1 = 𝛽1 + 𝛽2 𝑥21 + 𝛽3 𝑥31 + ⋯ + 𝛽𝑘 𝑥𝑘1 + 𝑢1
𝑦2 = 𝛽1 + 𝛽2 𝑥22 + 𝛽3 𝑥32 + ⋯ + 𝛽𝑘 𝑥𝑘2 + 𝑢2
⋮ ⋮ ⋮
𝑦𝑇 = 𝛽1 + 𝛽2 𝑥2𝑇 + 𝛽3 𝑥3𝑇 + ⋯ + 𝛽𝐾 𝑥𝑘𝑇 + 𝑢 𝑇
2. The Constant Term
We could write 𝑦𝑡 = 𝛽1 + 𝛽2 𝑥2𝑡 + 𝛽3 𝑥3𝑡 + ⋯ + 𝛽𝑘 𝑥𝑘𝑡 + 𝑢𝑡 in the
matrix form as:
𝑦1 1 𝑥21 𝑥31 … 𝑥𝑘1 𝛽1 𝑢1
𝑦2 1 𝑥22 𝑥32 ⋯ 𝑥𝑘2 ⋮ ⋮
⋮ = ⋮ ⋮ ⋮ ⋯ ⋮ ⋮
+ ⋮
𝑦𝑇 1 𝑥2𝑇 𝑥3𝑇 … 𝑥𝑘𝑇 𝛽𝑘 𝑢𝑇

𝑇⨯1 𝑇⨯𝑘 𝑘⨯1 𝑇⨯1

𝑦 = 𝑋𝛽 + 𝑢
2. The Constant Term
• Matrices written in the previous form are conformable (i.e., matrix
multiplication and addition on RHS are valid)
3. How are the Parameters Calculated?
• Previously, the residual sum of squares, σ𝑇𝑡=1 𝑢ො 𝑡2 was minimized with
respect to 𝛼ො and 𝛽መ
• Now, in the general case, to obtain estimates of the parameters
𝛽1 , 𝛽2 , … , 𝛽𝑘 , the RSS would be minimized with respect to all the
elements of 𝛽መ
• The optimization problem is therefore: 𝑚𝑖𝑛{𝛽෡1,…, 𝛽෡𝑘} σ𝑇𝑡=1 𝑢ො 𝑡2
3. How are the Parameters Calculated?
𝑢ො 1
𝑢ො 2
𝑢ො =
⋮
𝑢ො 𝑇
• Loss function in a matrix notation:
𝑢ො 1
𝑢ො 2
𝐿 = 𝑢ො ′ 𝑢ො = 𝑢ො 1 𝑢ො 2 … 𝑢ො 𝑇
⋮
𝑢ො 𝑇
= 𝑢ො 12 + 𝑢ො 22 + ⋯ + 𝑢ො 2𝑇 = σ𝑇𝑡=1 𝑢ො 𝑡2
4. Derivation of OLS estimator in Matrix
Form (Appendix)
• RSS is still equivalent to the loss function
𝑢ො 1
𝑢ො 2
𝐿 = 𝑢ො ′ 𝑢ො = 𝑢ො 1 𝑢ො 2 … 𝑢ො 𝑇 = 𝑢ො 12 + 𝑢ො 22 + ⋯ + 𝑢ො 2𝑇 = σ𝑇𝑡=1 𝑢ො 𝑡2
⋮
𝑢ො 𝑇
መ we have:
• Denoting the vector of estimated parameters as 𝛽,
′
𝑦ො = 𝑋𝛽መ and 𝐿= 𝑢ො ′ 𝑢ො = 𝑦 − 𝑋𝛽መ 𝑦 − 𝑋𝛽መ
4. Derivation of OLS estimator in Matrix
Form (Appendix)
′
𝐿 = 𝑦 − 𝑋𝛽መ 𝑦 − 𝑋𝛽መ
= 𝑦 ′ − 𝛽መ ′ 𝑋 ′ 𝑦 − 𝑋𝛽መ
= 𝑦 ′ 𝑦 − 𝑦 ′ 𝑋𝛽መ − 𝛽መ ′ 𝑋 ′ 𝑦 + 𝛽መ ′ 𝑋 ′ 𝑋𝛽መ
= 𝑦 ′ 𝑦 − 𝛽መ ′ 𝑋 ′ 𝑦 − 𝑦 ′ 𝑋𝛽መ + 𝛽መ ′ 𝑋 ′ 𝑋𝛽መ

1⨯𝑘 𝑘⨯𝑇 𝑇⨯1 1⨯𝑇 𝑇⨯𝑘 𝑘⨯1

(1 ⨯ 1) (1 ⨯ 1)
4. Derivation of OLS estimator in Matrix
Form (Appendix)
• So, 𝛽መ ′ 𝑋 ′ 𝑦 = 𝑦 ′ 𝑋𝛽መ
• We can write:
𝐿 = 𝑦 ′ 𝑦 − 2𝛽መ ′ 𝑋 ′ 𝑦 + 𝛽መ ′ 𝑋 ′ 𝑋𝛽መ

• Differentiating with respect to 𝛽መ and setting the derivative to zero

gives the parameter values that minimize RSS
𝜕𝐿
= −2𝑋 ′ 𝑦 + 2𝑋 ′ 𝑋𝛽መ = 0
𝜕 𝛽መ
4. Derivation of OLS estimator in Matrix
Form (Appendix)
𝜕𝑦 ′ 𝑦 ෡ ′ 𝑋 ′ 𝑋𝛽
𝜕𝛽 ෡
• ෡ = 0 and = 2𝑋 ′ 𝑋𝛽መ since 𝛽መ ′ 𝑋 ′ 𝑋𝛽መ is like the square
𝜕𝛽 𝜕𝛽
of 𝑋𝛽መ which is differentiated to 2𝑋 ′ 𝑋𝛽መ

• Thus, we can write:

2𝑋 ′ 𝑋𝛽መ = 2𝑋 ′ 𝑦 ⇒ 𝑋 ′ 𝑋𝛽መ = 𝑋 ′ 𝑦

• Pre-multiplying both sides by (𝑋 ′ 𝑋)−1 gives:

𝛽መ = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝑦
4. Derivation of OLS estimator in Matrix
Form (Appendix)
• Thus, the vector of OLS coefficient estimates for a set of k parameters
is given by:
𝛽መ1
መ 𝛽መ2
𝛽= = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝑦
⋮
𝛽መ𝑘 (𝑘 ⨯ 1) vector
4. Derivation of OLS estimator in Matrix
Form (Appendix)
• In the previous chapter, simple linear regression; bivariate:
2 ෝ𝑡2
σ𝑢 degrees of freedom
𝑦 = 𝛼 + 𝛽𝑥𝑡 + 𝑢𝑡 ⇒ 𝑠 =
𝑇−2

• In the case of more than one explanatory variable and a constant

(𝑦𝑡 = 𝛽1 + 𝛽2 𝑥2𝑡 + 𝛽3 𝑥3𝑡 + ⋯ + 𝛽𝑘 𝑥𝑘𝑡 + 𝑢𝑡 ), using matrix notation:
ෝ′ 𝑢
𝑢 ෝ ෝ𝑡2
σ𝑢
𝑠2 = = 𝑇 − 𝑘 degrees of
𝑇−𝑘
𝑇−𝑘
freedom
Where 𝑘 = number of regressors including a constant
5. Derivation of the OLS Standard Error in
the Multiple Regression Context
v𝑎𝑟 𝛽መ = 𝐸 (𝛽መ − 𝛽)(𝛽መ − 𝛽)′

• In the scalar world: v𝑎𝑟 𝛽መ = 𝐸 (𝛽መ − 𝐸 𝛽መ )2 = 𝐸 (𝛽መ − 𝛽)2

𝛽መ is unbiased
• Since 𝑦 = 𝑋𝛽 + 𝑢:
𝛽መ = 𝑋 ′ 𝑋 −1
𝑋 ′ (𝑋𝛽 + 𝑢)
𝛽መ = 𝛽 + (𝑋 ′ 𝑋)−1 𝑋 ′ 𝑢
5. Derivation of the OLS Standard Error in
the Multiple Regression Context
Thus, it is possible to express the variance of β෠ as:
𝐸 (𝛽መ − 𝛽)(𝛽መ − 𝛽)′
= 𝐸 (𝛽 + 𝑋 ′ 𝑋 −1 𝑋 ′ 𝑢 − 𝛽)(𝛽 + 𝑋 ′ 𝑋 −1 𝑋 ′ 𝑢 − 𝛽)′

⇒ 𝐸 (𝛽መ − 𝛽)(𝛽መ − 𝛽)′ = 𝐸 ( 𝑋 ′ 𝑋 −1 𝑋 ′ 𝑢)( 𝑋′𝑋 −1 𝑋 ′ 𝑢)′

= 𝐸 (𝑋 ′ 𝑋)−1 𝑋 ′ 𝑢𝑢′ 𝑋(𝑋 ′ 𝑋)−1

= (𝑋 ′ 𝑋)−1 𝑋 ′ 𝐸 𝑢𝑢′ 𝑋(𝑋 ′ 𝑋)−1

5. Derivation of the OLS Standard Error in
the Multiple Regression Context
• Now, 𝐸 𝑢𝑢′ = 𝑠 2 𝐼, so that
𝐸 (𝛽መ − 𝛽)(𝛽መ − 𝛽)′ = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝑠 2 𝐼𝑋(𝑋 ′ 𝑋)−1
= 𝑠 2 (𝑋 ′ 𝑋)−1 𝑋 ′ 𝑋(𝑋 ′ 𝑋)−1
= 𝑠 2 (𝑋 ′ 𝑋)−1

Estimated variance-covariance matrix (𝑘 ⨯ 𝑘) of the coefficients

5. Derivation of the OLS Standard Error in
the Multiple Regression Context
• The estimated variance-covariance matrix of the coefficients can be written as:
v𝑎𝑟 𝛽መ1 𝑐𝑜𝑣 𝛽መ1 , 𝛽መ2 ⋯ 𝑐𝑜𝑣 𝛽መ1 , 𝛽መ𝑘
𝑐𝑜𝑣 𝛽መ2 , 𝛽መ1 v𝑎𝑟 𝛽መ2 ⋯ 𝑐𝑜𝑣 𝛽መ2 , 𝛽መ𝑘
⋮ ⋮ ⋱ ⋮
𝑐𝑜𝑣 𝛽መ𝑘 , 𝛽መ1 𝑐𝑜𝑣 𝛽መ𝑘 , 𝛽መ2 ⋯ v𝑎𝑟 𝛽መ𝑘
• Diagonal terms = estimated variances
• Off-diagonal terms = estimated covariances between the parameter estimates
• The coefficient standard errors are given by simply taking the square roots of each
of the terms on the leading diagonal
5. Derivation of the OLS Standard Error in
the Multiple Regression Context
Example
The following model is estimated with 3 regressors and 15 observations:
𝑦 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 + 𝑢
And the following data have been calculated:
2.0 3.5 −1.0 −3
𝑋′𝑋 −1 = 3.5 1.0 6.5 , 𝑋 ′ 𝑦 = 2.2 , 𝑢ො ′ 𝑢ො = 10.96
−1.0 6.5 4.3 0.6
5. Derivation of the OLS Standard Error in
the Multiple Regression Context
• The dimension of 𝑋 is 𝑇 ⨯ 𝑘 = (15 ⨯ 3)
• The dimension of 𝑋 ′ 𝑋 = 3 ⨯ 15 15 ⨯ 3 = 3 ⨯ 3
• The dimension of 𝑋 ′ 𝑦 = 3 ⨯ 15 15 ⨯ 1 = (3 ⨯ 1)
𝛽መ1 2.0 3.5 −1.0 −3.0
• 𝛽መ = 𝛽መ2 = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝑦 = 3.5 1.0 6.5 2.2
𝛽መ3 −1.0 6.5 4.3 0.6
2.0 ⨯ −3 + 3.5 ⨯ 2.2 + −1.0 ⨯ 0.6 1.10
= 3.5 ⨯ −3 + 1.0 ⨯ 2.2 + 6.5 ⨯ 0.6 = −4.40
−1.0 ⨯ −3 + 6.5 ⨯ 2.2 + 4.3 ⨯ 0.6 19.88
5. Derivation of the OLS Standard Error in
the Multiple Regression Context
𝑅𝑆𝑆 10.96
• 𝑠2 = = = 0.91
𝑇−𝑘 15−3
• The variance covariance matrix of 𝛽መ is given by:
2.0 3.5 −1.0 1.82 3.19 −0.91
𝑠 2 𝑋 ′ 𝑋 −1 = 0.91 3.5 1.0 6.5 = 3.19 0.91 5.92
−1.0 6.5 4.3 −0.91 5.92 3.91
• v𝑎𝑟 𝛽መ1 = 1.82 ⇒ 𝑠𝑒 𝛽መ1 = 1.35
• v𝑎𝑟 𝛽መ2 = 0.91 ⇒ 𝑠𝑒 𝛽መ2 = 0.95
• v𝑎𝑟 𝛽መ3 = 3.91 ⇒ 𝑠𝑒 𝛽መ3 = 1.98
6. Testing Multiple Hypotheses: the 𝐹-test
We used the t-test to test single hypotheses (involving only one
coefficient)
But, what if the interest involves more than one coefficient
simultaneously?
We do this by using the 𝐹-test, which requires 2 regressions:
(1) The unrestricted regression is the one in which the coefficients are
freely determined by the data
(2) The restricted regression is the one in which the coefficients are
restricted
6. Testing Multiple Hypotheses: the 𝐹-test
• The residual sum of squares from each regression are determined, the
two residual sums of squares are then compared in the test statistic
𝑅𝑅𝑆𝑆−𝑈𝑅𝑆𝑆 𝑇−𝑘
𝑡𝑒𝑠𝑡 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 = ⨯
𝑈𝑅𝑆𝑆 𝑚

(𝑅𝑅𝑆𝑆−𝑈𝑅𝑆𝑆)ൗ
𝑚
= 𝑈𝑅𝑆𝑆 ~𝐹(𝑚, 𝑇 − 𝑘)
ൗ 𝑇−𝑘

• 𝑚 = df of the numerator
• 𝑇 − 𝑘 = df of the denominator
6. Testing Multiple Hypotheses: the 𝐹-test
𝑅𝑅𝑆𝑆 − 𝑈𝑅𝑆𝑆 𝑇 − 𝑘
𝑡𝑒𝑠𝑡 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 = ⨯
𝑈𝑅𝑆𝑆 𝑚
Where:
• RRSS: RSS from restricted regression
• URSS: RSS from unrestricted regression
• 𝑚: number of restrictions
• 𝑇: number of observations
• 𝑘: number of regressors in unrestricted regression including a constant in
the unrestricted regression (or the total number of parameters to be
estimated)
6. Testing Multiple Hypotheses: the 𝐹-test
• Generally 𝑅𝑅𝑆𝑆 ≥ 𝑈𝑅𝑆𝑆
• Intuition: if imposing the restrictions does not result in a much larger
RRSS, then the restrictions are supported by the data
Example
• 𝑦 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 + 𝛽4 𝑥4 + 𝑢 (unrestricted regression)

𝐻0 : 𝛽3 + 𝛽4 = 1
• The restriction 𝛽3 + 𝛽4 = 1 is under test ⇒ ቊ
𝐻1 : 𝛽3 + 𝛽4 ≠ 1
6. Testing Multiple Hypotheses: the 𝐹-test
• Restricted regression:
𝑦 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 + 𝛽4 𝑥4 + 𝑢 subject to 𝛽3 + 𝛽4 = 1
• Substitute the restriction back into the regression:
𝛽3 + 𝛽4 = 1 ⇒ 𝛽4 = 1 − 𝛽3
• The restricted regression is given by:
𝑦 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 + 1 − 𝛽3 𝑥4 + 𝑢
• After some algebra:
𝑦 − 𝑥4 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 − 𝑥4 + 𝑢
𝑃 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑄 + 𝑢
7. The Relationship Between the t and the 𝐹-
Distributions
• Any hypothesis that could be tested with a t-test could also have been
tested using an 𝐹-test, but not the other way around
• Single hypothesis: t-test or 𝐹-test
• Multiple hypothesis: 𝐹-test
7. The Relationship Between the t and the 𝐹-
Distributions
Example:
H0 : 𝛽2 = 0.5
H1 : 𝛽2 ≠ 0.5

• The hypothesis could be tested by a t-test:

𝛽መ2 − 0.5
𝑡𝑒𝑠𝑡 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 =
𝑠𝑒(𝛽መ2 )
• Or it could be tested by using an 𝐹-test
7. The Relationship Between the t and the 𝐹-
Distributions
• The square of the t-distribution is equivalent to a particular form of the
𝐹-distribution
𝑍 2 ~𝑡 2 (𝑇 − 𝑘) then also 𝑍 2 ~𝐹(1, 𝑇 − 𝑘)

• 𝐹-statistic > 𝐹-critical

⇓
Reject 𝐻0
Rejection Region

𝐹-critical
8. Determining the Number of Restrictions,
𝑚
• The number of restrictions is the number of = under the null
hypothesis

H0 (Null hypothesis) No. of restrictions, 𝑚

𝛽1 + 𝛽2 = 2 1 (𝑡 or 𝐹-test)
𝛽1 = 1 and 𝛽3 = −1 2 (𝐹-test)
𝛽2 = 0, 𝛽3 = 0 𝑎𝑛𝑑 𝛽4 = 0 3 (𝐹-test)
8. Determining the Number of Restrictions,
𝑚
• Consider:
H0 : 𝛽2 = 0 𝑎𝑛𝑑 𝛽3 = 0 𝑎𝑛𝑑 𝛽4 = 0
ൠ THE regression 𝐹- statistic
H1 : 𝛽2 ≠ 0 𝑜𝑟 𝛽3 ≠ 0 𝑜𝑟 𝛽4 ≠ 0
• This test is sometimes called test for junk regression
• if 𝐻0 is not rejected, none of the independent variables was able to
explain variations in 𝑦
Note: Nonlinear hypothesis such as H0 : 𝛽2 𝛽3 = 2 or H0 : 𝛽22 = 1 cannot
be tested
8. Determining the Number of Restrictions,
𝑚
Example
A researcher wants to test whether the returns on a company stock (𝑦)
show unit sensitivity to two factors (factor 𝑥2 and factor 𝑥3 ) among 3
considered
𝑦 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 + 𝛽4 𝑥4 + 𝑢
Here, 𝑘 = 4 and 𝑇 = 144
8. Determining the Number of Restrictions,
𝑚
(1) What are the restricted and unrestricted regressions?
Restricted regression: impose 𝛽2 = 1 and 𝛽3 = 1
𝑦 = 𝛽1 + 𝑥2 + 𝑥3 + 𝛽4 𝑥4 + 𝑢
𝑦 − 𝑥2 − 𝑥3 = 𝛽1 + 𝛽4 𝑥4 + 𝑢
𝑍 = 𝛽1 + 𝛽4 𝑥4 + 𝑢
8. Determining the Number of Restrictions,
𝑚
(2) If the 𝑅𝑅𝑆𝑆 = 436.1 and 𝑈𝑅𝑆𝑆 = 397.2, 𝑇 = 144, 𝑚 = 2, 𝑘 = 4,
perform the test
𝑅𝑅𝑆𝑆 − 𝑈𝑅𝑆𝑆 (𝑇 − 𝑘)
𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 = ⨯
𝑈𝑅𝑆𝑆 𝑚
436.1−397.2 144−4
= ⨯ = 6.86
397.2 2

• Critical value 𝐹 2,140 = 3 at 5% level

• Critical value 𝐹 2,140 =4.61 at the 1% level
• 𝐹-statistic > critical value ⇒ 𝐻0 is rejected ⇒ restrictions are not
supported by the data
9. Multiple Regression in Eviews Using APT-
style Model
• Multiple (instead of single) factor model
• APT tries to explain stock returns by unexpected changes in financial
and macroeconomic variables
• The APT posits that stock returns can be explained by unexpected
changes in macroeconomic variables rather than their levels
• The unexpected value of a variable can be defined as the difference
between the actual (realized) value of the variable and its expected
value
9. Multiple Regression in Eviews Using APT-
style Model
How do we believe that investors might have formed their
expectations?
• Easiest: investors have made expectations that the next period value of
the variable is equal to the current value
⇒ the change in the value of the variable from one period to the next is
the unexpected change (because investors are assumed to expect no
change)
 2
10. Goodness of Fit Statistics: 𝑅
• 𝑅2 relates to 𝑅𝑆𝑆: ↗ 𝑅𝑆𝑆 ⇒ ↘ 𝑅2 (Lower fit)
But, 𝑹𝑺𝑺 alone doesn’t mean much
• A scaled version of the 𝑅𝑆𝑆 is usually employed:
• 𝜌ො𝑦2𝑦ො = 𝑅2
• The 𝑅2 is the square of the correlation coefficient between 𝑦 and 𝑦ො
(correlation between fitted values and dependent variable)
• Since −1 ≤ 𝜌ො𝑦𝑦ො ≤ +1 ⇒ 0 ≤ 𝑅2 ≤ 1
• The regression line attempts to explain variability in 𝑦 around its mean
 2
10. Goodness of Fit Statistics: 𝑅
• 𝑦 acts like a benchmark
• Benchmark of 𝑦𝑡 = 𝛽1 + 𝑢𝑡 => 𝛽መ1 = 𝑦
• The total variation across all observations of the dependent variable
about its mean value is known as the total sum of square (𝑇𝑆𝑆)
𝑇

𝑇𝑆𝑆 = ෍(𝑦𝑡 − 𝑦)2

𝑡=1
• 𝑇𝑆𝑆 can be split into two parts: the part that has been explained by the
model (𝐸𝑆𝑆) and the part that the model was not able to explain (𝑅𝑆𝑆)
𝑇𝑆𝑆 = 𝐸𝑆𝑆 + 𝑅𝑆𝑆
 2
10. Goodness of Fit Statistics: 𝑅
𝑇𝑆𝑆 = 𝐸𝑆𝑆 + 𝑅𝑆𝑆
𝑇 𝑇 𝑇

෍ 𝑦𝑡 − 𝑦ത 2
= ෍ 𝑦ො𝑡 − 𝑦ത 2
+ ෍ 𝑢ො 𝑡2
𝑡=1 𝑡=1 𝑡=1

𝑇 𝑇 𝑇
2 2 2
෍ 𝑦𝑡 − 𝑦ത = ෍ 𝑦ො𝑡 − 𝑦ത + ෍ 𝑦𝑡 − 𝑦ො𝑡
𝑡=1 𝑡=1 𝑡=1
• The goodness of fit statistic is given by:
𝐸𝑆𝑆 𝑇𝑆𝑆 − 𝑅𝑆𝑆
2
𝑅𝑆𝑆
𝑅 = = =1−
𝑇𝑆𝑆 𝑇𝑆𝑆 𝑇𝑆𝑆
 2
10. Goodness of Fit Statistics: 𝑅
Extreme cases
𝑦𝑡 𝑦𝑡

𝑥𝑡 𝑥𝑡
𝑅𝑆𝑆 = 𝑇𝑆𝑆 ⇒ 𝑅2 =0
E𝑆𝑆 = 𝑇𝑆𝑆 ⇒ 𝑅2 = 1 (𝑅𝑆𝑆 = 0)
11. Problems with 𝑅 2 as a Goodness of Fit
Measure
(1) 𝑅2 cannot be compared across models with different 𝑦
(2) 𝑅2 never falls if more variables are added to the regression
Regression 1: 𝑦𝑡 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 + 𝑢
Regression 2: 𝑦𝑡 = 𝛽1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 + 𝛽4 𝑥4 + 𝑢
𝑅2 regression 2 ≥ 𝑅2 regression 1
(3) 𝑅2 can be as high as 0.9 for times-series regressions
 2
12. Adjusted 𝑅
2 𝑇−1
𝑅 =1− (1 − 𝑅2 )
𝑇−𝑘
• 𝑘 ↗ as more regressors are added => T-k decreases
• The net effect depends on whether the increase in R2 resulting from the addition of new
variables outweighs the increase in R2
𝑇−1
• In other words, the term increases with the addition of variables while the term (1-R2)
𝑇−𝑘
decreases
• The net effect on R2 depends on which of the two effects dominates
2
• 𝑅2 has to increase by more than off-setting amount for 𝑅 to increase
2
• If 𝑅 rises, one should include the variable or the group of variables in the regression
 2
13. Relationship between 𝐹-statistic and 𝑅
• Unrestricted: 𝑦𝑡 = 𝛽1 + 𝛽2 𝑥2𝑡 + ⋯ + 𝛽𝑘 𝑥𝑘𝑡 + 𝑢 ⇒
𝑈𝑅𝑆𝑆 = 𝑅𝑆𝑆
• Restricted: 𝑦𝑡 = 𝛽1 + 𝑢𝑡
1. Number of restrictions is 𝑚 = 𝑘 − 1
2. For the restricted regression, there is no ESS ⇒ 𝑇𝑆𝑆 = 𝑅𝑅𝑆𝑆

𝑇𝑆𝑆−𝑅𝑆𝑆 𝑇−𝑘 𝐸𝑆𝑆ൗ 𝑇−𝑘

𝑇𝑆𝑆
𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 = ⨯ = 𝑅𝑆𝑆ൗ ⨯
𝑅𝑆𝑆 𝑘−1 𝑇𝑆𝑆 𝑘−1
 2
13. Relationship between 𝐹-statistic and 𝑅
Thus, we can write:
𝑅2 (𝑇 − 𝑘)
𝑇𝐻𝐸 𝑟𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 =
1 − 𝑅2 (𝑘 − 1)