CFA®

Preparation
QUANTITATIVE METHODS www.dbf-finance.com

Reading
Reading Title Study Session
Number

4 Introduction to Linear Regression

5 Multiple Regression 2

6 Time-Series Analysis

7 Machine Learning

8 Big Data Projects 3

Probabilistic Approaches: Scenario Analysis, Decision

9
Trees and Simulations

Luis M. de Alfonso
 CFA® Preparation
QUANTITATIVE METHODS www.dbf-finance.com

Multiple Regression
Study Session 2

Reading Number 5

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.a: Formulate a multiple regression equation to describe the relation between a dependent variable and several independent variables and determine
the statistical significance of each independent variable

Multiple regression Regression analysis with more than one independent variable

Multiple linear regression model:

Regression equation:

Residuals:

Multiple regression methodology estimates the intercept term and slope coefficients such that the sum of
the squared error terms is minimized

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.b: Interpret estimated regression coefficients and their p-values

MULTIPLE REGRESSION MODEL

Intercept Term Is the value of the dependent variable when the independent variables are all equal to zero

Estimated Slope Coefficient Each slope coefficient is the estimated change in the dependent variable for a one-unit change in
the independent variable, holding the other independent variables constant (that is why in
multiple regression, they are called partial slope coefficients)

When a new independent variable is added in a regression equation the slope coefficient of the previous variables
normally chage (unless the new variable is uncorrelated with the previous ones)

The regression equation captures the relationship between variables

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.c Formulate a null and an alternative hypothesis about the population value of a regression coefficient, calculate the value of the test statistic, and
determine whether to reject the null hypothesis at a given level of significance
LOS 5.d: Interpret the results of hypothesis test of regression coefficients

t-test
Process:

1.- Calculate t statistic

2.- Introduce in the t statistic equation b" which is the

hypothesized value of the slope coefficient that we want to
test ( Null hypothesis: the slope coefficient is equal to the b"
value introduced in the equation)
Rejection of the null means that the slope coefficient is different than the 𝐛𝐣 value
3.- Compare t statistic with the critical t-value with n-k-1 we are testing
degrees of freedom and the desired level of significance
To test whether an independent variable explain the variation of a dependent
(use t distribution table) variable (is statistically significant) we test the hypothesis that the slope
Two tailed test !!! coefficient is zero
n-k-1 degrees of freedom
“teting statistical significance”

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.c Formulate a null and an alternative hypothesis about the population value of a regression coefficient, calculate the value of the test statistic, and
determine whether to reject the null hypothesis at a given level of significance
LOS 5.d: Interpret the results of hypothesis test of regression coefficients

EG10: 10 years growth in the S&P 500

PR: payout ratio of the stocks in the index
YCS: yield curve slope (10-year T-bond yield minus 3-month T-bill yield)

(all variables measures in %)

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.c Formulate a null and an alternative hypothesis about the population value of a regression coefficient, calculate the value of the test statistic, and
determine whether to reject the null hypothesis at a given level of significance
LOS 5.d: Interpret the results of hypothesis test of regression coefficients

Interpreting p-values

p-value Smallest level of significance for which the null hypothesis can be rejected

Ø If the p- value < than the significance level, the null hypothesis can be rejected

Ø If the p- value > than the significance level, the null hypothesis cannot be rejected

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.c Formulate a null and an alternative hypothesis about the population value of a regression coefficient, calculate the value of the test statistic, and
determine whether to reject the null hypothesis at a given level of significance
LOS 5.d: Interpret the results of hypothesis test of regression coefficients

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.c Formulate a null and an alternative hypothesis about the population value of a regression coefficient, calculate the value of the test statistic, and
determine whether to reject the null hypothesis at a given level of significance
LOS 5.d: Interpret the results of hypothesis test of regression coefficients

Other test of the regression coefficients

Enter the value in the equation replacing b"

1.- Null Hypothesis is that the coefficient is equal to some value

Two tail test !!! Same process

Enter the value in the equation replacing b"

2.- Null Hypothesis is that the coefficient is greater than or less
than some value
Same process but using “one-tailed” table
One tail test !!!

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.e: Calculate and interpret 1) a confidence interval for the population value of a regression coefficient and 2) a predicted value for the dependent
variable, given an estimated regression model and assumed values for the independent variables

Ø Confidence Interval for the regression coefficient b" is calculated as:

t& Critical two-tailed t-value for the selected confidence

level an n-k-1 degrees of freedom

s*
() Standard error of the regression coefficient

Ø Predicting the dependent variable:

Just introduce in the equation the forecasted values of the

independent variables

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.g: Calculate and interpret the F-statistic, and describe how it is used in regression analysis

Ø An F-test assesses how well the set of independet variables, as a group, explains the variation in the dependent variable
F - Test
Ø F-test is used to test whether at least one of the individual variables explains a significant portion of the variation of the
dependent variable

Hypothesis tested Ho: b1=b2=….=bn = 0; versus Ha: at least one bj different than zero
F - Statistic

Process: If F (test statistic) > Fc (critical value) then Ho hypothesis is rejected, that means
that at least one of the slope coefficients is significantly different that zero
1.- Calculate F-Statistic (using ANOVA table data)

2.- Calculate critical F-value using the F table and: At least one of the independent variables in the regression model makes a
significant contribution to the explanation of the dependent variable
df numerator = k (number of variables)
df denominator = n – k – 1

2.- Compare F-statistic with the critical F-value If you are asked to test all the coefficients simultaneously, use the F-test
(do not test each coefficient with a t-test for each one)
Always one tailed test !!!

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.g: Calculate and interpret the F-statistic, and describe how it is used in regression analysis

SST = RSS + SSE

Total sum of squares = Regression sum of squares +

Sum of squared errors

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.h: Distinguish between and interpret the 𝑅, and adjusted 𝑅, in multiple regression

Percentage of the variation in the dependent variable (Y) that is explained by the set of
Coefficient of determination 𝑹𝟐
independent variables

Remember: SST = RSS + SSE

𝑅, increases as the number of independent variables increases even though the marginal contribution of the new variables is not statistically significant,
so it may not be a reliable measure of the explanatory power of the multiple regression model
This problem is referred as “overestimating the regression”

Adjusted 𝑹𝟐 ( 𝑹𝟐𝒂 ) It is used to avoid “overestimating the regression”

𝑅,0 <= 𝑅,

While adding a new independent variable to the model will increase 𝑅, , it

may either increase or decrease 𝑅,0

𝑅,0 may be less than zero if 𝑅, is low enough

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.i: Evaluate how well a regression model explains the dependent variable by analyzing the output of the regression equation and an ANOVA table

ANOVA Table Analysis of variance ANOVA is a statistical procedure that provides information on the explanatory power of a regression

From the ANOVA table we can calculate 𝑹𝟐 , the F-statistic, and the estándar error of estimate (SEE)

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.i: Evaluate how well a regression model explains the dependent variable by analyzing the output of the regression equation and an ANOVA table
Remember:

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.i: Evaluate how well a regression model explains the dependent variable by analyzing the output of the regression equation and an ANOVA table

Step 2: Determine the significance of the individual independent variables

Step 3: Determine the utility of the model as a whole

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.j: Formulate a multiple regression equation by using dummy variables to represent qualitative factors and interpret the coefficients and regressions results

DUMMY Variable Binary variables (“1” or “0”) which are often used to quantify the impact of qualitative events also known
as “qualitative independent variables”

Interpreting the coefficients in a Dummy Variable Regression

Consider the following regression equation for explaining quarterly EPS (earnings per share) in terms of the quarter of their ocurrence:

EPS4 = quarterly observation of EPS

Q64 = 1 if period t is the first quarter; Q64 = 0 otherwise
Q,4 = 1 if period t is the first quarter; Q,4 = 0 otherwise
Q74 = 1 if period t is the first quarter; Q74 = 0 otherwise

When performing multiple regression with dummy variables, whenever we want to distinguish between n classes, we must use n – 1 dummy
variables. Otherwise the regression assumption of no exact linear relationship between independent variables would be violated

e.g. If we are studying quarters (there are 4 quarters) we use 3 dummy variables

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.j: Formulate a multiple regression equation by using dummy variables to represent qualitative factors and interpret the coefficients and regressions results

Interpreting the coefficients in a Dummy Variable Regression

Example:
Average fourth quarter EPS = 1,25

EPS4 = b 8 + b 6Q64 + b ,Q,4 + b 7 Q74 Average first quarter EPS = 1,25 + 0,75 = 2,00
Average second quarter EPS = 1,25 – 0,20 = 1,05

EPS4 = 1,25 + 0,75Q64 - 0,20 Q,4 + 0,10Q74 Average third quarter EPS = 1,25 + 0,1 = 1,35

b 8 : average value of the dependent variable (EPS) for the fourth quarter (dummy variable omitted)
b 6 : difference in earnings per share between quarter 1 and quarter 4 (omitted) (0,75 = 2,00 – 1,25)
b , : difference in earnings per share between quarter 2 and quarter 4 (omitted) (-0,2 = 1,05 – 1,25)
b 7 : difference in earnings per share between quarter 3 and quarter 4 (omitted) (0,1 = 1,35 – 1,25)

“Slope coefficient is interpreted as the change in the dependent variable for the case when the dummy variable is one”

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.j: Formulate a multiple regression equation by using dummy variables to represent qualitative factors and interpret the coefficients and regressions results

Dummy Variable Regression

Ø As with all multiple regression results, the F-statistic for the set of coefficients and the R , should be evaluated to determine if the dummy
variables, individually or collectively, contribute to the explanation of the dependent variable

Ø We can also perform the t-test for each slope coefficient using the following null hypotheses:

H8 : b 6 = 0 test whether fourth quarter EPS = first quarter EPS (remind that b 6 is the difference in earnings per share between
quarter 1 and quarter 4 (omitted))

H8 : b , = 0 test whether fourth quarter EPS = second quarter EPS

H8 : b 7 = 0 test whether fourth quarter EPS = third quarter EPS

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.j: Formulate a multiple regression equation by using dummy variables to represent qualitative factors and interpret the coefficients and regressions results

EPS4 = 1,25 + 0,75Q64 - 0,20 Q,4 + 0,10Q74

40 quarterly observations (n = 40)

Remind again: H8 : b 6 = 0 test whether fourth quarter EPS = first quarter EPS

b 6 is the difference in earnings per share between quarter 1 and quarter 4 (omitted)

If b 6 = 0 then EPS 1Q = EPS 4Q

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.j: Formulate a multiple regression equation by using dummy variables to represent qualitative factors and interpret the coefficients and regressions results

Regression application with quantitative variables and dummy variables

Y = b8 + b6X6 + b, X, + b7 D7 D7 is a Dummy variable

Interpretation:

b 7 > 0 dependent variable Y is higher (b 7 units higher) when Dummy variable D7 takes value “1” than when Dummy variable D7 takes value “0”

e.g.

Y : loan spread (basis points over LIBOR) on private debt contracts

X6 and X, : two any independent quantitative variables (e.g., standard deviation of daily stock return, market to book ratio,…)
D7 : Dummy variable ( D7 = 1 if the loan was a result of corporate restructuring, and D7 = 0 otherwise)

If 𝑏7 > 0 and is statistically significant, means that loan spread (Y) is higher when loan is used for corporate restructuring than for loans used for
other porpuses

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.f: Explain the assumptions of a multiple regression model

Assumptions of multiple regression mostly pertain to the error term, 𝜺𝒊

1. A linear relationship exist between the dependent variable and the independet variables

2. The independent variables are not random, and there is not exact linear relation between any two ore more
independent variables

3. The expected value of the error term is zero

4. The variance of the error term is constant for all observations

5. The error for one observation is not correlated with that of another observation

6. The error term is normally distributed

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.f: Explain the assumptions of a multiple regression model

Recall:

Ø The estimated t-statistic is calculated as :

Ø s*
() is the standard error for coefficient j and is calculated using the standard error of estimate (SEE) which is the standard deviation
of the error term.

Any violation of an assumption that affect the error term will affect the coefficient standard error

Consequently, this will affect the t-statistic and F- Statistic and any conclusions drawn from hypothesis test involving these statistics

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Ø There are three primary assumptions violations that we may encounter:

Heteroskedasticity
Serial Correlation (Autocorrelation)
Multicollinearity

Ø We need to know:

What it is?
Effects
Detection
Correction

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Heteroskedasticity

1.- What is Heteroskedasticity When the variance of the residuals is not the same across all observations of the residuals

Ø Unconditional heteroskedasticity: when heteroskedasticity is not related to the level of the independent
variables (causes no major problems with the regression)
Ø Contitional heteroskedasticity: when heteroskedasticity is related to the level of the independent
variables (it creates significant problems for statistical inference)

Contitional heteroskedasticity

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Heteroskedasticity

2.- Effects of Heteroskedasticity Four main effects:

a. Standard errors are usually unreliable estimates

b. The coefficient estimates (b") are not affected (coefficients are consistent)

c. If stardard error is too small (standard error understimated) and the coefficient b" is not affected, t-statistic
will be too large and null hypothesis will be rejected too often (Type I error - rejection of the null hypothesis
when it is actually true)

d. The F-test is also unreliable

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Heteroskedasticity

3.- Detecting Heteroskedasticity Two methods

a. Examining scatter plots of the residuals

b. Breusch-Pagan chi-square test (BP 𝜒 , test = n x 𝑅IJKLM

, with k degrees of freedom)

,
𝑅IJKLM = 𝑅 , from a second regression (squared residuals vs independent variables)

BP chi-square test method:

Ø Formulate a regression where the dependent variable are the squared errors and de independent variables are the original ones (we try to see if
there is relationship between the independent variables and the squared residuals)

Ø If BP 𝜒 , calculated is greater than critical BP 𝜒 , value (obtained from the table) then null hypothesis is rejected THERE IS CONDITIONAL
HETEROSKEDASTICITY BECAUSE THE INDEPENDENT VARIABLES SIGNIFICANTLY CONTRIBUTE TO THE EXPLANATION OF THE SQUARED RESIDUALS

One tailed test !!!

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

( n = 60 )

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Heteroskedasticity

4.- Correcting Heteroskedasticity Using “white corrected” standard errors (also called robust standard errors)

Robust standard errors are then used to recalculate the t-statistic using the original regression coefficients

With the new t-statistic we do again the t-test

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Observe how the value of the t statistic

changes when using the new white
corrected standard errors

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Serial Correlation (Autocorrelation)

1.- What is Serial Correlation Residuals terms are correlated with one another (common problem with time series data)

Ø Positive serial correlation: when a positive regression error in one time period increases the probability of
observing a positive regression error for the next time period

Ø Negative serial correlation: when a positive regression error in one time period increases the probability of
observing a negative regression error for the next time period

Residuals Residuals

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Serial Correlation (Autocorrelation)

2.- Effects of Serial Correlation

a. The coefficient estimates (b") are not affected (coefficients are consistent)

b. When positive serial correlation, standard errors are often understimated (too many Type I errors – rejection of the null
hypothesis when it is actually true)

s*
()

c. The F-test is also unreliable (because MSE is understimated)

SSE

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Serial Correlation (Autocorrelation)

3.- Detecting Serial Correlation Two methods

a. Examining scatter plots of the residuals versus time

b. Durbin – Watson statistic (DW) DW ≈ 2 (1 – r) r = correlation coefficient between residuals from one
period and those from the previous period

r > 0 positive correlation DW < 2 Positive serial correlation

r < 0 negative correlation DW > 2 Negative serial correlation
r = 0 no correlation DW = 2 Homoskedastic: no serial correlation (r = 0)

DW test method: When we have positive

serial correlation (DW<2),
then DW test method is
Ø Enter the table and obtain 𝐷P and 𝐷Q values used to analize if it is
statistically significant
Ø Calculate DW = 2 (1 - r)

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.k: Explain the types of heteroskedasticity and how heteroskedasticity and serial correlation affect statistical inference

Serial Correlation (Autocorrelation)

Use the Hansen method to adjust standard errors (also called “Hansen – white
4.- Correcting Serial Correlation
standard errors” or “serial correlation consistent standard errors”)

Hansen - white standard errors are then used in hypothesis testing of the regression coefficients

Ø If there is only Heteroskedasticity use White – corrected standard errors

Ø If there is only Serial Correlation use Hansen - white standard errors

Ø If both conditions are present use the Hansen method

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.l: Describe multicollinearity and explain its causes and effects in regression analysis

Multicollinearity

Two or more independent variables (or linear combinations of the independent variables)
1.- What is Multicollinearity
are highly correlated with each other

2.- Effects of Multicollinearity

a. Coefficients are consistent but unreliable

b. Standard errors are overestimated (too many Type II errors – greater probability that we will incorrectly conclude
that a variable is not statistically significant – no rejection of the null hypothesis when it is actually false)

s(R
)

c. The F-test is also unreliable (because MSE is overstimated)

SSE

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.l: Describe multicollinearity and explain its causes and effects in regression analysis

Multicollinearity

3.- Detecting Multicollinearity

Conflict between t-tests and F-test.

(while t-test indicates that non of the independent coefficients are significantly ≠ 0 and F-test says that all
together are significantly ≠ 0 )

It could be that there is no direct correlation between independent variables but there is correlation
between linear combinations of the independent variables, and therefore there is Multicollinearity

4.- Correcting Multicollinearity

Omit one or more of the correlated independent variables

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.l: Describe multicollinearity and explain its causes and effects in regression analysis

Summary of what to know regarding violations the of assumptions of multiple regression

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.m: Describe how model misspecification affects the results of a regression analysis and describe how to avoid common forms os misspecification

MODEL SPECIFICATION

Regression model specification is the selection of the explanatory (independent) variables to be included in
the regression and the transformations (if any) of those explanatory variables

MODEL MISSPECIFICATION

ü Important variables are omitted

ü Variables should be transformed
ü Incorrectly pooling data
ü Using lagged dependent variable as independent variable
ü Forecasting the past (e.g. using variables measured at the end of a period to predict a variable mesaured
during that period)
ü Measuring independent variables with error

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.m: Describe how model misspecification affects the results of a regression analysis and describe how to avoid common forms os misspecification

MODEL MISSPECIFICATION

Effects of Model Misspecification:

Model Misspecification

Biased and inconsistent

regression coefficients

Unreliable hypothesis testing

and inaccurate predictions

Luis M. de Alfonso
 CFA® Preparation
QM – Multiple Regression www.dbf-finance.com

LOS 5.n: Describe models with qualitative dependent variables

LOS 5.o: Evaluate and interpret a multiple regression model and its results

Ø Models with qualitative (dummy) “dependent” variables

Models with qualitative dependent variables (e.g. bankrupt versus non bankrupt) requires methods other than ordinary
least squares like probit, logit and discriminant analysis
The analysis of regressions models with qualitative dependent variables is the same as we have been discussing (examine individual coefficients
using t-test, determine the validity of the model with the F-test and 𝑅, , and look out for heteroskedasticity, serial correlation and
multicollinearity)

Ø Evaluate and interpret a multiple regression model and its results

The values of the slope coefficients suggest the economic meaning of the relationship between the independent and the
dependent variables.
(Each slope coefficient is the estimated change in the dependent variable for a one-unit change in the independent variable, holding the other
independent variables constant)

But it is important for the analyst to keep in mind that a regression may have statistical significance even when there is no
practical economic significance in the relationship
(e.g. A study of dividend announcements may identify a statistically significant abnormal return following the announcement, but these
returns may not be sufficient to cover transactions costs)

Luis M. de Alfonso