Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (2 of 37)

• Assumption MLR.6 (Normality of error terms)

Chapter 4
Multiple Regression Analysis:
Inference

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 1 password-protected website or school-approved learning management system for classroom use. 3

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (1 of 37) Multiple Regression Analysis: Inference (3 of 37)
• Statistical inference in the regression model • Discussion of the normality assumption
• Hypothesis tests about population parameters • The error term is the sum of “many” different unobserved factors.
• Construction of confidence intervals • Sums of independent factors are normally distributed (CLT).
• Problems:
• Sampling distributions of the OLS estimators • How many different factors? Number large enough?
• The OLS estimators are random variables • Possibly very heterogenuous distributions of individual factors
• We already know their expected values and their variances • How independent are the different factors?
• However, for hypothesis tests we need to know their distribution
• In order to derive their distribution we need additional assumptions • The normality of the error term is an empirical question.
• Assumption about distribution of errors: normal distribution • At least, the error distribution should be “close” to normal.
• In many cases, normality is questionable or impossible by definition.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 2 password-protected website or school-approved learning management system for classroom use. 4
 Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (4 of 37) Multiple Regression Analysis: Inference (6 of 37)
• Discussion of the normality assumption (cont.) • Testing hypotheses about a single population parameter
• Examples where normality cannot hold: • Theorem 4.2 (t-distribution for the standardized estimators)
• Wages (nonnegative; also: minimum wage) • Under assumptions MLR.1 – MLR.6
• Number of arrests (takes on a small number of integer values)
• Unemployment (indicator variable, takes on only 1 or 0)

• In some cases, normality can be achieved through transformations

of the dependent variable (e.g. use log(wage) instead of wage).
• Under normality, OLS is the best (even nonlinear) unbiased estimator • Null hypothesis (for more general hypotheses, see below)
• Important: For the purposes of statistical inference, the assumption of
normality can be replaced by a large sample size.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 5 password-protected website or school-approved learning management system for classroom use. 7

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (5 of 37) Multiple Regression Analysis: Inference (7 of 37)
• Terminology • t-statistic (or t-ratio)

• Theorem 4.1 (Normal sampling distributions)

• Under assumptions MLR.1 – MLR.6:
• Distribution of the t-statistic if the null hypothesis is true

• Goal: Define a rejection rule so that, if it is true, H0 is rejected only

with a small probability (= significance level, e.g. 5%)
© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 6 password-protected website or school-approved learning management system for classroom use. 8
 Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (8 of 37) Multiple Regression Analysis: Inference (10 of 37)
• Testing against one-sided alternatives (greater than zero) • Example: Wage equation (cont.)
• Reject the null hypothesis in favour of the
alternative hypothesis if the estimated
coefficient is “too large” (i.e. larger than a
critical value).

• Construct the critical value so that, if the null

hypothesis is true, it is rejected in, for example,
5% of the cases.

• In the given example, this is the point of the t-

distribution with 28 degrees of freedom that is
exceeded in 5% of the cases.
• The effect of experience on hourly wage is statistically greater than
• Reject if t-statistic is greater than 1.701 zero at the 5% (and even at the 1%) significance level.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 9 password-protected website or school-approved learning management system for classroom use. 11

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (9 of 37) Multiple Regression Analysis: Inference (11 of 37)
• Example: Wage equation • Testing against one-sided alternatives (less than zero)
• Test whether, after controlling for education and tenure, higher work
• Reject the null hypothesis in favour of the
experience leads to higher hourly wages. alternative hypothesis if the estimated
coefficient is “too small” (i.e. smaller than a
critical value).

• Construct the critical value so that, if the null

hypothesis is true, it is rejected in, for example,
5% of the cases.

• In the given example, this is the point of the t-

distribution with 18 degrees of freedom so that
5% of the cases are below the point.

• Reject if t-statistic is less than -1.734

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 10 password-protected website or school-approved learning management system for classroom use. 12
 Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (12 of 37) Multiple Regression Analysis: Inference (14 of 37)
• Example: Student performance and school size • Example: Student performance and school size (cont.)
• Test whether smaller school size leads to better student performance • Alternative specification of functional form:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 13 password-protected website or school-approved learning management system for classroom use. 15

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (13 of 37) Multiple Regression Analysis: Inference (15 of 37)
• Example: Student performance and school size (cont.) • Example: Student performance and school size (cont.)

• The hypothesis that there is no effect of school size on student performance

can be rejected in favor of the hypothesis that the effect is negative.
• How large is the effect?

• One cannot reject the hypothesis that there is no effect of school size
on student performance (not even for a lax significance level of 15%).

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 14 password-protected website or school-approved learning management system for classroom use. 16
 Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (16 of 37) Multiple Regression Analysis: Inference (18 of 37)
• Testing against two-sided alternatives • “Statistically significant” variables in a regression
• If a regression coefficient is different from zero in a two-sided test, the
• Reject the null hypothesis in favour of the
alternative hypothesis if the absolute value of corresponding variable is said to be “statistically significant”.
the estimated coefficient is too large. • If the number of degrees of freedom is large enough so that the normal
approximation applies, the following rules of thumb apply:
• Construct the critical value so that, if the null
hypothesis is true, it is rejected in, for example,
5% of the cases.

• In the given example, these are the points of

the t-distribution so that 5% of the cases lie in
the two tails.

• Reject if absolute value of t-statistic is less than

-2.06 or greater than 2.06

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 17 password-protected website or school-approved learning management system for classroom use. 19

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (17 of 37) Multiple Regression Analysis: Inference (19 of 37)
• Example: Determinants of college GPA • Testing more general hypotheses about a regression coefficient
• Null hypothesis

• t-statistic

• The test works exactly as before, except that the hypothesized value is
substracted from the estimate when forming the statistic.
© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 18 password-protected website or school-approved learning management system for classroom use. 20
 Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (20 of 37) Multiple Regression Analysis: Inference (22 of 37)
• Example: Campus crime and enrollment • How the p-value is computed (here: two-sided test)?
• An interesting hypothesis is whether crime increases by one percent if • The p-value is the significance level at which one is
enrollment is increased by one percent. indifferent between rejecting and not rejecting the null
hypothesis.

• In the two-sided case, the p-value is thus the

probability that the t-distributed variable takes on a
larger absolute value than the realized value of the test
statistic, e.g.
P(|t|>1.85) = 2*(0.0359) = .0718

• From this, it is clear that a null hypothesis is rejected if

and only if the corresponding p-value is smaller than
the significance level.

• For example, for a significance level of 5% the t-statistic

would not lie in the rejection region.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 21 password-protected website or school-approved learning management system for classroom use. 23

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (21 of 37) Multiple Regression Analysis: Inference (23 of 37)
• Computing p-values for t-tests • Guidelines for discussing economic and statistical significance
• If the significance level is made smaller and smaller, there will be a point where • If a variable is statistically significant, discuss the magnitude of the coefficient to
the null hypothesis cannot be rejected anymore. get an idea of its economic or practical importance.
• The reason is that, by lowering the significance level, one wants to avoid more • The fact that a coefficient is statistically significant does not necessarily mean it
and more to make the error of rejecting a correct H0. is economically or practically significant!
• The smallest significance level at which the null hypothesis is still rejected, is • If a variable is statistically and economically important but has the “wrong”
called the p-value of the hypothesis test. sign, the regression model might be misspecified.
• A small p-value is evidence against the null hypothesis because one would • If a variable is statistically insignificant at the usual levels (10%, 5%, or 1%), one
reject the null hypothesis even at small significance levels. may think of dropping it from the regression.
• A large p-value is evidence in favor of the null hypothesis. • If the sample size is small, effects might be imprecisely estimated so that the
• P-values are more informative than tests at fixed significance levels. case for dropping insignificant variables is less strong.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 22 password-protected website or school-approved learning management system for classroom use. 24
 Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (24 of 37) Multiple Regression Analysis: Inference (26 of 37)
• Confidence intervals • Example: Model of firms‘ R&D expenditures
• Simple manipulation of the result in Theorem 4.2 implies that

• Interpretation of the confidence interval

• The bounds of the interval are random.
• In repeated samples, the interval that is constructed in the above way will cover
the population regression coefficient in 95% of the cases.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 25 password-protected website or school-approved learning management system for classroom use. 27

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (25 of 37) Multiple Regression Analysis: Inference (27 of 37)
• Confidence intervals for typical confidence levels • Testing hypotheses about a linear combination of the parameters
• Example: Return to education at two-year vs. at four-year colleges

• Relationship between confidence intervals and hypotheses tests • A possible test statistic would be:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 26 password-protected website or school-approved learning management system for classroom use. 28
 Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (28 of 37) Multiple Regression Analysis: Inference (30 of 37)
• The standard error of the difference in parameters is impossible to • Testing multiple linear restrictions: The F-test
with standard regression output • Testing exclusion restrictions

• An alternative method is to make a substitution in variables.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 29 password-protected website or school-approved learning management system for classroom use. 31

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (29 of 37) Multiple Regression Analysis: Inference (31 of 37)
• Estimation results • Estimation of the unrestricted model

• This method works always for single linear hypotheses

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 30 password-protected website or school-approved learning management system for classroom use. 32
 Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (32 of 37) Multiple Regression Analysis: Inference (34 of 37)
• Estimation of the restricted model • Test decision in example

• Test statistic: • Discussion

• The three variables are “jointly significant”
• They were not significant when tested individually
• The likely reason is multicollinearity between them

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 33 password-protected website or school-approved learning management system for classroom use. 35

Introductory Econometrics: A Modern Approach (7e) Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (33 of 37) Multiple Regression Analysis: Inference (35 of 37)
• Rejection rule • Test of overall significance of a regression
• A F-distributed variable only takes
on positive values. This corresponds
to the fact that the sum of squared
residuals can only increase if one
moves from H1 to H0.

• Choose the critical value so that we

incorrectly reject the null hypothesis
in, for example, only 5% of the • Discussion
cases. • The three variables are “jointly significant”
• They were not significant when tested individually
• The likely reason is multicollinearity between them

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a © 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 34 password-protected website or school-approved learning management system for classroom use. 36
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (36 of 37)

• Testing general linear restrictions with the F-test
• Example: Test whether house price assessments are rational

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 37

Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Inference (37 of 37)

• Unrestricted regression

• Restricted regression

• Test statistic

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 38