Econometric S

Econometrics
Streamlined, Applied and e-Aware
Francis X. Diebold
University of Pennsylvania
Edition 2016
Version Thursday 17th March, 2016
Econometrics
Econometrics
Streamlined, Applied and e-Aware
Francis X. Diebold
c 2013-2016
Copyright
by Francis X. Diebold.
This work is freely available for your use, but be warned: it is preliminary, incomplete, and evolving. It is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
(Briefly: I retain copyright, but you can use, copy and distribute non-commercially,
so long as you give me attribution and do not modify. To view a copy of the license, go to http://creativecommons.org/licenses/by-nc-nd/4.0/.) In
return I ask that you please cite the book whenever appropriate, as: Diebold,
F.X. (2016), Econometrics, Department of Economics, University of Pennsylvania, http://www.ssc.upenn.edu/~fdiebold/Textbooks.html.
To my undergraduates,
who continually surprise and inspire me
Brief Table of Contents

About the Author
xix
About the Cover
xxi
Guide to e-Features
xxiii
Preface
xxxi
Beginnings
1 Introduction to Econometrics
2 Graphics and Graphical Style
13
II
29
Cross Sections
3 Regression Under the Full Ideal Conditions
31
4 Non-Normal Disturbances
67
5 Stochastic Regressors I: Measurement Error, Omitted Variables, and Multicollinearity

77
6 Indicator Variables in Cross Sections: Group Heterogeneity 87
7 Non-Linear Functional Form in Cross Sections
8 Heteroskedasticity and Clustering in Cross Sections
ix
95
113
III
BRIEF TABLE OF CONTENTS
Time Series
121
9 Indicator Variables in Time Series: Trend and Seasonality 123

10 Non-Linearity and Structural Change in Time Series
137
11 Serial Correlation in Observed Time Series
155
12 Serial Correlation in Time Series Regression
201
13 Heteroskedasticity in Time Series
221
IV
263
More
14 Binary Regression and Classification
265
15 Panel Data
275
16 Stochastic Regressors II: Endogeneity
279
283
Appendices
A Probability and Statistics Review
285
B Construction of the Wage Datasets
297
C Some Popular Books Worth Encountering
303
Detailed Table of Contents

About the Author
xix
About the Cover
xxi
Guide to e-Features
xxiii
Preface
xxxi
Beginnings
1 Introduction to Econometrics
1.1 Welcome . . . . . . . . . . . . . . . . . . .
1.1.1 Who Uses Econometrics? . . . . . .
1.1.2 What Distinguishes Econometrics?
1.2 Types of Recorded Economic Data . . . .
1.3 Online Information and Data . . . . . . .
1.4 Software . . . . . . . . . . . . . . . . . . .
1.5 Tips on How to use this book . . . . . . .
1.6 Exercises, Problems and Complements . .
1.7 Notes . . . . . . . . . . . . . . . . . . . .
2 Graphics and Graphical Style
2.1 Simple Techniques of Graphical Analysis
2.1.1 Univariate Graphics . . . . . . .
2.1.2 Multivariate Graphics . . . . . .
2.1.3 Summary and Extension . . . . .
2.2 Elements of Graphical Style . . . . . . .
2.3 U.S. Hourly Wages . . . . . . . . . . . .
xi
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
3
3
5
5
6
7
9
10
11
.
.
.
.
.
.
13
13
14
14
17
19
21
xii
DETAILED TABLE OF CONTENTS
2.4
2.5
2.6
2.7
II
Concluding Remarks . . . . . . . . . .
Exercises, Problems and Complements
Notes . . . . . . . . . . . . . . . . . . .
Graphics Legend: Edward Tufte . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Cross Sections
3 Regression Under the Full Ideal Conditions

3.1 Preliminary Graphics . . . . . . . . . . . . . . . . . . .
3.2 Regression as Curve Fitting . . . . . . . . . . . . . . .
3.2.1 Bivariate, or Simple, Linear Regression . . . . .
3.2.2 Multiple Linear Regression . . . . . . . . . . . .
3.2.3 Onward . . . . . . . . . . . . . . . . . . . . . .
3.3 Regression as a Probability Model . . . . . . . . . . . .
3.3.1 A Population Model and a Sample Estimator .
3.3.2 Notation, Assumptions and Results: The Full
Conditions . . . . . . . . . . . . . . . . . . . . .
A Bit of Matrix Notation . . . . . . . . . . . . .
Assumptions: The Full Ideal Conditions (FIC) .
Results . . . . . . . . . . . . . . . . . . . . . . .
3.4 A Wage Equation . . . . . . . . . . . . . . . . . . . . .
3.4.1 Mean dependent var 2.342 . . . . . . . . . . . .
3.4.2 S.D. dependent var .561 . . . . . . . . . . . . .
3.4.3 Sum squared resid 319.938 . . . . . . . . . . . .
3.4.4 Log likelihood -938.236 . . . . . . . . . . . . . .
3.4.5 F statistic 199.626 . . . . . . . . . . . . . . . .
3.4.6 Prob(F statistic) 0.000000 . . . . . . . . . . .
3.4.7 S.E. of regression .492 . . . . . . . . . . . . . .
3.4.8 R-squared .232 . . . . . . . . . . . . . . . . . .
3.4.9 Adjusted R-squared .231 . . . . . . . . . . .
3.4.10 Akaike info criterion 1.423 . . . . . . . . . . . .
3.4.11 Schwarz criterion 1.435 . . . . . . . . . . . . . .
3.4.12 Hannan-Quinn criter. 1.427 . . . . . . . . . . .
3.4.13 Durbin-Watson stat. 1.926 . . . . . . . . . . . .
3.4.14 The Residual Scatter . . . . . . . . . . . . . . .
3.4.15 The Residual Plot . . . . . . . . . . . . . . . . .
21
22
26
27
29
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
Ideal
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
31
31
33
33
36
37
38
38
39
39
40
41
42
45
45
46
46
47
47
47
49
49
50
50
51
51
52
53
3.5
3.6
3.7
3.8
3.4.16 A Bit More on AIC and SIC . . . . .

Quantile Regression . . . . . . . . . . . . . .
Exercises, Problems and Complements . . .
Notes . . . . . . . . . . . . . . . . . . . . . .
Regressions Inventor: Carl Friedrich Gauss .
xiii
.
.
.
.
.
.
.
.
.
.
4 Non-Normal Disturbances
4.0.1 Results . . . . . . . . . . . . . . . . . . .
4.1 Assessing Normality . . . . . . . . . . . . . . .
4.1.1 QQ Plots . . . . . . . . . . . . . . . . .
4.1.2 Residual Sample Skewness and Kurtosis
4.1.3 The Jarque-Bera Test . . . . . . . . . .
4.2 Outliers . . . . . . . . . . . . . . . . . . . . . .
4.2.1 Outlier Detection . . . . . . . . . . . . .
Graphics . . . . . . . . . . . . . . . . . .
Leave-One-Out and Leverage . . . . . .
4.3 Robust Estimation . . . . . . . . . . . . . . . .
4.3.1 Robustness Iteration . . . . . . . . . . .
4.3.2 Least Absolute Deviations . . . . . . . .
4.4 Wage Determination . . . . . . . . . . . . . . .
4.4.1 W AGE . . . . . . . . . . . . . . . . . .
4.4.2 LW AGE . . . . . . . . . . . . . . . . .
4.5 Exercises, Problems and Complements . . . . .
4.6 Notes . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
54
55
57
65
66
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
67
67
68
69
69
69
70
71
71
71
71
72
73
74
74
74
75
76
5 Stochastic Regressors I: Measurement Error, Omitted Variables, and Multicollinearity

5.1 Dropping the Fixed X Assumption . . . . . . . . . . . . . .
5.2 Omitted Variables . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 More on Causal vs. Non-Causal Regression . . . . . . . . . . .
5.3.1 Causal Predictive Modeling and T-Consistency . . . .
5.3.2 Non-Causal Predictive Modeling and P-Consistency . .
5.3.3 Correlation vs. Causality, and P-Consistency vs. TConsistency . . . . . . . . . . . . . . . . . . . . . . . .
5.3.4 An Example of Correlation Without Causality . . . .
5.4 Measurement Error . . . . . . . . . . . . . . . . . . . . . . . .
5.5 Multicollinearity . . . . . . . . . . . . . . . . . . . . . . . . .
77
77
78
79
80
80
81
81
82
82
xiv
.
.
.
.
83
83
84
84
6 Indicator Variables in Cross Sections: Group Heterogeneity

6.1 0-1 Dummy Variables . . . . . . . . . . . . . . . . . . . . . . .
6.2 Group Dummies in the Wage Regression . . . . . . . . . . . .
6.3 Exercises, Problems and Complements . . . . . . . . . . . . .
6.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.5 Dummy Variables, ANOVA, and Sir Ronald Fischer . . . . . .
87
87
89
91
93
94
7 Non-Linear Functional Form in Cross Sections

7.1 Models Linear in Transformed Variables . . . . . . . . . . . .
7.1.1 Logarithms . . . . . . . . . . . . . . . . . . . . . . . .
7.1.2 Box-Cox and GLM . . . . . . . . . . . . . . . . . . . .
7.2 Intrinsically Non-Linear Models . . . . . . . . . . . . . . . . .
7.2.1 Nonlinear Least Squares . . . . . . . . . . . . . . . . .
7.3 Series Expansions . . . . . . . . . . . . . . . . . . . . . . . . .
7.4 A Final Word on Nonlinearity and the FIC . . . . . . . . . . .
7.5 Selecting a Non-Linear Model . . . . . . . . . . . . . . . . . .
7.5.1 t and F Tests, and Information Criteria . . . . . . . .
7.5.2 The RESET Test . . . . . . . . . . . . . . . . . . . .
7.6 Non-Linearity in Wage Determination . . . . . . . . . . . . . .
7.6.1 Non-Linearity in Continuous and Discrete Variables Simultaneously . . . . . . . . . . . . . . . . . . . . . . .
7.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
95
96
96
98
99
99
100
102
102
102
103
103
5.6
5.5.1 Perfect Multicollinearity . . . .

5.5.2 Imperfect Multicollinearity . . .
5.5.3 A Bit More . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
105
107
112
8 Heteroskedasticity and Clustering in Cross Sections

113
8.1 Covariance Matrix Clustering . . . . . . . . . . . . . . . . . 120
8.2 Exercises, Problems and Complements . . . . . . . . . . . . . 120
8.3 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
III
xv
Time Series
121
9 Indicator Variables in Time Series: Trend and Seasonality

9.1 Linear Trend . . . . . . . . . . . . . . . . . . . . . . . . . . .
9.2 Seasonality . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9.2.1 Seasonal Dummies . . . . . . . . . . . . . . . . . . . .
9.2.2 More General Calendar Effects . . . . . . . . . . . . .
9.3 Trend and Seasonality in Liquor Sales . . . . . . . . . . . . . .
9.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
123
123
125
126
128
128
130
135
10 Non-Linearity and Structural Change in Time Series

10.1 Exponential Trend . . . . . . . . . . . . . . . . . . . . . . .
10.2 Quadratic Trend . . . . . . . . . . . . . . . . . . . . . . . .
10.3 More on Non-Linear Trend . . . . . . . . . . . . . . . . . . .
10.3.1 Moving-Average Trend and De-Trending . . . . . . .
10.3.2 Hodrick-Prescott Trend and De-Trending . . . . . . .
10.4 Structural Change . . . . . . . . . . . . . . . . . . . . . . .
10.4.1 Gradual Parameter Evolution . . . . . . . . . . . . .
10.4.2 Abrupt Parameter Breaks . . . . . . . . . . . . . . .
Exogenously-Specified Breaks . . . . . . . . . . . . .
The Chow test with Endogenous Break Selection . .
10.5 Dummy Variables and Omitted Variables, Again and Again
10.5.1 Dummy Variables . . . . . . . . . . . . . . . . . . . .
10.5.2 Omitted Variables . . . . . . . . . . . . . . . . . . .
10.6 Non-Linearity in Liquor Sales Trend . . . . . . . . . . . . . .
10.7 Exercises, Problems and Complements . . . . . . . . . . . .
10.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
137
137
139
140
141
142
142
143
143
143
144
145
145
145
146
148
153
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
11 Serial Correlation in Observed Time Series

11.1 Characterizing Time-Series Dynamics . . . . . . . . . . . . . .
11.1.1 Covariance Stationary Time Series . . . . . . . . . . .
11.2 White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.3 Estimation and Inference for the Mean, Autocorrelation and
Partial Autocorrelation Functions . . . . . . . . . . . . . . . .
11.3.1 Sample Mean . . . . . . . . . . . . . . . . . . . . . . .
11.3.2 Sample Autocorrelations . . . . . . . . . . . . . . . . .
155
155
156
161
166
166
167
xvi
11.3.3 Sample Partial Autocorrelations . . . . . . . . . . . . .

11.4 Autoregressive Models for Serially-Correlated Time Series . . .
11.4.1 Some Preliminary Notation: The Lag Operator . . . .
11.4.2 Autoregressions . . . . . . . . . . . . . . . . . . . . . .
The AR(1) Process . . . . . . . . . . . . . . . . . . . .
11.4.3 The AR(p) Process . . . . . . . . . . . . . . . . . . .
11.4.4 Alternative Approaches to Estimating Autoregressions
11.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
170
171
171
173
173
179
181
183
185
12 Serial Correlation in Time Series Regression

12.1 Testing for Serial Correlation . . . . . . . . . . . . . . . . . .
12.1.1 The Durbin-Watson Test . . . . . . . . . . . . . . . . .
12.1.2 The Breusch-Godfrey Test . . . . . . . . . . . . . . . .
12.1.3 The Residual Correlogram . . . . . . . . . . . . . . . .
12.2 Estimation with Serial Correlation . . . . . . . . . . . . . . .
12.2.1 Regression with Serially-Correlated Disturbances . . .
12.2.2 Serially-Correlated Disturbances vs. Lagged Dependent Variables . . . . . . . . . . . . . . . . . . . . . . .
12.2.3 A Full Model of Liquor Sales . . . . . . . . . . . . . . .
12.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
201
203
203
206
207
209
209
13 Heteroskedasticity in Time Series

13.1 The Basic ARCH Process . . . . . . . . . . . . . . . . . . . .
13.2 The GARCH Process . . . . . . . . . . . . . . . . . . . . . . .
13.3 Extensions of ARCH and GARCH Models . . . . . . . . . . .
13.3.1 Asymmetric Response . . . . . . . . . . . . . . . . . .
13.3.2 Exogenous Variables in the Volatility Function . . . . .
13.3.3 Regression with GARCH disturbances and GARCH-M
13.3.4 Component GARCH . . . . . . . . . . . . . . . . . . .
13.3.5 Mixing and Matching . . . . . . . . . . . . . . . . . . .
13.4 Estimating, Forecasting and Diagnosing GARCH Models . . .
13.5 Stock Market Volatility . . . . . . . . . . . . . . . . . . . . . .
13.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
221
222
227
234
235
236
236
237
238
238
240
255
261
211
215
218
220
IV
xvii
More
263
14 Binary Regression and Classification

14.1 Binary Regression . . . . . . . . . . . . . .
14.1.1 Binary Response . . . . . . . . . .
14.1.2 The Logit Model . . . . . . . . . .
Logit . . . . . . . . . . . . . . . . .
Ordered Logit . . . . . . . . . . . .
Dynamic Logit . . . . . . . . . . .
Complications . . . . . . . . . . . .
14.1.3 Classification and 0-1 Forecasting
14.2 Exercises, Problems and Complements . .
14.3 Notes . . . . . . . . . . . . . . . . . . . . .
15 Panel Data
15.1 Panel Data . . . . . . . . . . . . . . .
15.2 Individual and Time Effects . . . . . .
15.2.1 Individual Effects . . . . . . . .
15.2.2 Individual and Time Effects . .
15.3 Exercises, Problems and Complements
15.4 Notes . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
265
265
265
267
267
268
269
269
270
271
274
.
.
.
.
.
.
275
275
276
276
277
277
277
16 Stochastic Regressors II: Endogeneity

279
0
16.1 Causes of E(X ) 6= 0 . . . . . . . . . . . . . . . . . . . . . . . 279
16.1.1 Omitted Variables . . . . . . . . . . . . . . . . . . . . 279
16.1.2 Simultaneity . . . . . . . . . . . . . . . . . . . . . . . . 279
16.1.3 Measurement Error . . . . . . . . . . . . . . . . . . . . 280
16.1.4 Sample Selection . . . . . . . . . . . . . . . . . . . . . 280
16.2 Instrumental Variables . . . . . . . . . . . . . . . . . . . . . . 280
16.2.1 The Basic Idea and the IV Estimator . . . . . . . . . . 280
16.2.2 Instrument Strength and Exogeneity . . . . . . . . . . 280
Exogenous but Weakly-Relevant Instruments . . . . 280
Relevant but Slightly-Endogenous Instruments . . . 280
16.2.3 Sources of Instruments . . . . . . . . . . . . . . . . . . 280
Randomized Experiments . . . . . . . . . . . . . . . . 280
Natural Experiments . . . . . . . . . . . . . . . . . . . 281
Thought Experiments (Structural Econometric Models) 281
xviii
16.3
16.4
16.5
16.6
16.7
Time . . . . . . . . . . . . . . . .
Additional Useful Strategies . . . . . . .
16.3.1 Regression Discontinuity Designs
16.3.2 Differences of Differences . . . . .
16.3.3 Matching . . . . . . . . . . . . .
Graphical Models . . . . . . . . . . . . .
Internal and External Validity . . . . . .
Exercises, Problems and Complements .
Notes . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Appendices
281
281
281
281
281
281
281
282
282
283
A Probability and Statistics Review

A.1 Populations: Random Variables, Distributions and Moments .
A.1.1 Univariate . . . . . . . . . . . . . . . . . . . . . . . . .
A.1.2 Multivariate . . . . . . . . . . . . . . . . . . . . . . . .
A.2 Samples: Sample Moments . . . . . . . . . . . . . . . . . . . .
A.2.1 Univariate . . . . . . . . . . . . . . . . . . . . . . . . .
A.2.2 Multivariate . . . . . . . . . . . . . . . . . . . . . . . .
A.3 Finite-Sample and Asymptotic Sampling Distributions of the
Sample Mean . . . . . . . . . . . . . . . . . . . . . . . . . . .
A.3.1 Exact Finite-Sample Results . . . . . . . . . . . . . . .
A.3.2 Approximate Asymptotic Results (Under Weaker Assumptions) . . . . . . . . . . . . . . . . . . . . . . . .
A.4 Exercises, Problems and Complements . . . . . . . . . . . . .
A.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
285
285
285
288
289
289
291
B Construction of the Wage Datasets
297
C Some Popular Books Worth Encountering
303
292
292
293
294
296
About the Author
Francis X. Diebold is Professor of Economics, Finance and Statistics at the

University of Pennsylvania. He has won both undergraduate and graduate
economics teacher of the year awards, and his academic family includes
thousands of undergraduate students and nearly 75 Ph.D. students. Diebold
has published widely in econometrics, forecasting, finance, and macroeconomics. He is an NBER Faculty Research Associate, as well as an elected
Fellow of the Econometric Society, the American Statistical Association, and
the International Institute of Forecasters. He has also been the recipient of
Sloan, Guggenheim, and Humboldt fellowships, Co-Director of the Wharton Financial Institutions Center, and President of the Society for Financial
Econometrics. His academic research is firmly linked to practical matters:
During 1986-1989 he served as an economist under both Paul Volcker and
Alan Greenspan at the Board of Governors of the Federal Reserve System,
during 2007-2008 he served as Executive Director of Morgan Stanley Investment Management, and during 2012-2013 he served as Chairman of the Federal Reserve Systems Model Validation Council.
xix
About the Cover
The colorful painting is Enigma, by Glen Josselsohn, from Wikimedia Commons. As noted there:
Glen Josselsohn was born in Johannesburg in 1971. His art has
been exhibited in several art galleries around the country, with a
number of sell-out exhibitions on the South African art scene ...
Glens fascination with abstract art comes from the likes of Picasso,
Pollock, Miro, and local African art.
I used the painting mostly just because I like it. But econometrics is indeed
something of an enigma, part economics and part statistics, part science and
part art, hunting faint and fleeting signals buried in massive noise. Yet,
perhaps somewhat miraculously, it often succeeds.
xxi
Guide to e-Features
Hyperlinks to internal items (table of contents, index, footnotes, etc.)

appear in red.
Hyperlinks to bibliographic references appear in green.
Hyperlinks to the web appear in cyan.
Hyperlinks to external files (e.g., video) appear in blue.
Many images are clickable to reach related material.
Key concepts appear in bold, and they also appear in the books (hyperlinked) index.
Additional related materials appear on the books web page. These may
include book updates, presentation slides, datasets, and computer code.
Facebook group: Diebold Econometrics.
Additional relevant material sometimes appears on Facebook groups
Diebold Forecasting and Diebold Time Series Econometrics, on Twitter @FrancisDiebold, and on the No Hesitations blog.
xxiii
xxiv
Guide
List of Figures
1.1
1.2
1.3
1.4
Resources for Economists Web

Eviews Homepage . . . . . . .
Stata Homepage . . . . . . . .
R Homepage . . . . . . . . . .
.
.
.
.
7
8
9
10
2.1
2.2
2.3
1-Year Goverment Bond Yield, Levels and Changes . . . . . .

Histogram of 1-Year Government Bond Yield . . . . . . . . . .
Bivariate Scatterplot, 1-Year and 10-Year Government Bond
Yields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Scatterplot Matrix, 1-, 10-, 20- and 30-Year Government Bond
Yields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Distributions of Wages and Log Wages . . . . . . . . . . . . .
Tufte Teaching, with a First Edition Book by Galileo . . . . .
15
16
2.4
2.5
2.6
3.1
3.2
3.3
3.4
3.5
3.6
3.7
6.1
6.2
6.3
6.4
6.5
Page
. . .
. . .
. . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Distributions of Log Wage, Education and Experience . .

(Log Wage, Education) Scatterplot . . . . . . . . . . . .
(Log Wage, Education) Scatterplot with Superimposed
gression Line . . . . . . . . . . . . . . . . . . . . . . . .
Regression Output . . . . . . . . . . . . . . . . . . . . .
Wage Regression Residual Scatter . . . . . . . . . . . . .
Wage Regression Residual Plot . . . . . . . . . . . . . .
Carl Friedrich Gauss . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
. . .
. . .
Re. . .
. . .
. . .
. . .
. . .
Histograms for Wage Covariates . . . . . . . . . . . . . . . . .

Wage Regression on Education and Experience . . . . . . . . .
Wage Regression on Education, Experience and Group Dummies
Residual Scatter from Wage Regression on Education, Experience and Group Dummies . . . . . . . . . . . . . . . . . . .
Sir Ronald Fischer . . . . . . . . . . . . . . . . . . . . . . . .
xxv
17
18
21
27
32
34
35
42
52
54
66
88
90
90
91
94
xxvi
7.1
7.2
7.3
LIST OF FIGURES
106
7.5
Basic Linear Wage Regression . . . . . . . . . . . . . . . . . .

Quadratic Wage Regression . . . . . . . . . . . . . . . . . . .
Wage Regression on Education, Experience, Group Dummies,
and Interactions . . . . . . . . . . . . . . . . . . . . . . . . . .
Wage Regression with Continuous Non-Linearities and Interactions, and Discrete Interactions . . . . . . . . . . . . . . . .
Regression Output . . . . . . . . . . . . . . . . . . . . . . . .
9.1
9.2
9.3
9.4
9.5
9.6
9.7
9.8
Various Linear Trends . . . . . . . . . . . . . . . . . . . .

Liquor Sales . . . . . . . . . . . . . . . . . . . . . . . . . .
Log Liquor Sales . . . . . . . . . . . . . . . . . . . . . . .
Linear Trend Estimation . . . . . . . . . . . . . . . . . . .
Residual Plot, Linear Trend Estimation . . . . . . . . . . .
Estimation Results, Linear Trend with Seasonal Dummies
Residual Plot, Linear Trend with Seasonal Dummies . . .
Seasonal Pattern . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
124
129
130
131
132
133
134
135
Various Exponential Trends . . . . . . . . . . . . . . . . . . .

Various Quadratic Trends . . . . . . . . . . . . . . . . . . . .
Log-Quadratic Trend Estimation . . . . . . . . . . . . . . . .
Residual Plot, Log-Quadratic Trend Estimation . . . . . . . .
Liquor Sales Log-Quadratic Trend Estimation with Seasonal
Dummies . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.6 Residual Plot, Liquor Sales Log-Quadratic Trend Estimation
With Seasonal Dummies . . . . . . . . . . . . . . . . . . . . .
138
140
147
148
12.1
12.2
12.3
12.4
***.
***.
***.
***.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
212
213
213
215
13.1
13.2
13.3
13.4
13.5
13.6
Time Series of Daily NYSE Returns. . . . . . . . .

Correlogram of Daily Stock Market Returns. . . . .
Histogram and Statistics for Daily NYSE Returns. .
Time Series of Daily Squared NYSE Returns . . . .
Correlogram of Daily Squared NYSE Returns. . . .
GARCH(1,1) Estimation, Daily NYSE Returns. . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
243
243
244
245
245
253
7.4
.
.
.
.
.
.
.
.
10.1
10.2
10.3
10.4
10.5
***.
***.
***.
***.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
104
105
107
112
149
150
LIST OF FIGURES
13.7 Estimated Conditional Standard Deviation, Daily NYSE Returns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13.8 Conditional Standard Deviation, History and Forecast, Daily
NYSE Returns. . . . . . . . . . . . . . . . . . . . . . . . . . .
13.9 Correlogram of Squared Standardized GARCH(1,1) Residuals,
Daily NYSE Returns. . . . . . . . . . . . . . . . . . . . . . . .
13.10AR(1) Returns with Threshold t-GARCH(1,1)-in Mean. . . .
xxvii
254
254
255
257
xxviii
LIST OF FIGURES
List of Tables
2.1
Yield Statistics . . . . . . . . . . . . . . . . . . . . . . . . . .
xxix
25
xxx
LIST OF TABLES
Preface
Most good texts arise from the desire to leave ones stamp on a discipline
by training future generations of students, driven by the recognition that
existing texts are deficient in various respects. My motivation is no different,
but it is more intense: In recent years I have come to see most existing texts
as highly deficient, in four ways.
First, many existing texts attempt exhaustive coverage, resulting in large
tomes impossible to cover in a single course (or even two, or three). Econometrics, in contrast, does not attempt exhaustive coverage. Indeed the coverage
is intentionally selective and streamlined, focusing on the core methods with
the widest applicability. Put differently, Econometrics is not designed to impress people with the breadth of my knowledge; rather, its designed to teach
real students, and it can be realistically covered in a one-semester course.
Core material appears in the main text, and additional material appears in
the end-of-chapter Exercises, Problems and Complements.
Second, many existing texts emphasize theory at the expense of serious
applications. Econometrics, in contrast, is applications-oriented throughout,
using detailed real-world applications not simply to illustrate theory, but to
teach it (in truly realistic situations in which not everything works perfectly!).
Econometrics uses modern software throughout, but the discussion is not
wed to any particular software students and instructors can use whatever
computing environment they like best.
xxxi
xxxii
PREFACE
Third, although Econometrics is designed for a first course in econometrics

and uses only elementary mathematics, it nevertheless conveys a strong feel
for the important advances made in recent years. It touches sometimes
extensively upon topics such as:
statistical graphics and exploratory data analysis
nonlinear and non-Gaussian environments
simulation methods
unit roots and stochastic trends
volatility modeling
Big Data: selection, shrinkage, etc.
Much such material appears in the already-mentioned Exercises, Problems
and Complements, which form an integral part of the book. They are organized so that instructors and students can pick and choose, according to their
backgrounds and interests. Coverage of such topics helps prepare students
for more advanced or specialized books, such as my Forecasting.
Finally, almost all existing texts remain shackled by Middle-Ages paper
technology. Econometrics, in contrast, is e-aware. Its colorful, hyperlinked
internally and externally, and tied to a variety of media effectively a blend
of a traditional book, a DVD, a web page, a Facebook group, a blog, and
whatever else I can find thats useful. Its continually evolving and improving
on the web, and its freely available to all, as opposed to the obscene but nowstandard $250 for a pile of paper. It wont make me any new friends among
the traditional publishers, but thats not my goal.
Econometrics should be useful to students in a variety of fields in economics, of course, but also business, finance, public policy, statistics, and even
engineering. It is directly accessible at the undergraduate and masters levels,
PREFACE
xxxiii
as the only prerequisite is an introductory probability and statistics course.

I have used the material successfully for many years in my undergraduate
econometrics course at Penn, as background for various other undergraduate
courses, and in masters-level executive education courses given to professionals in economics, business, finance and government.
Many people have contributed to the development of this book. One way
or another, all of the following deserve thanks: Xu Cheng, University of Pennsylvania; Barbara Chizzolini, Bocconi University; Frank Di Traglia, University of Pennsylvania; Carlo Favero, Bocconi University; Bruce Hansen, University of Wisconsin; Frank Schorfheide, University of Pennsylvania; James
H. Stock, Harvard University; Mark W. Watson, Princeton University.
I am especially grateful to the University of Pennsylvania, which for many
years has provided an unparalleled intellectual home, the perfect incubator
for the ideas that have congealed here. Related, I am grateful to an army of
energetic and enthusiastic Penn graduate and undergraduate students, who
read and improved much of the manuscript and code. Graduate students
include Ross Askanazi, Lorenzo Braccini, Laura Liu, Minchul Shin, and Molin
Zhong.
Finally, I apologize and accept full responsibility for the many errors and
shortcomings that undoubtedly remain minor and major despite ongoing
efforts to eliminate them.
Francis X. Diebold
Philadelphia
Thursday 17th March, 2016
Econometrics
xxxvi
PREFACE
Part I
Beginnings
Chapter 1
Introduction to Econometrics
1.1
1.1.1
Welcome
Who Uses Econometrics?
Econometric modeling is important it is used constantly in business,

finance, economics, government, consulting and many other fields. Econometric models are used routinely for tasks ranging from data description to
policy analysis, and ultimately they guide many important decisions.
To develop a feel for the tremendous diversity of econometrics applications, lets sketch some of the areas where they feature prominently, and the
corresponding diversity of decisions supported.
One key field is economics (of course), broadly defined. Governments,
businesses, policy organizations, central banks, financial services firms, and
economic consulting firms around the world routinely use econometrics.
Governments, central banks and policy organizations use econometric models to guide monetary policy, fiscal policy, as well as education and training,
health, and transfer policies.
Businesses use econometrics for strategic planning tasks. These include
management strategy of all types including operations management and control (hiring, production, inventory, investment, ...), marketing (pricing, distributing, advertising, ...), accounting (budgeting revenues and expenditures),
3
CHAPTER 1. INTRODUCTION TO ECONOMETRICS
and so on.
Sales modeling is a good example. Firms routinely use econometric models
of sales to help guide management decisions in inventory management, sales
force management, production planning, new market entry, and so on.
More generally, firms use econometric models to help decide what to produce (What product or mix of products should be produced?), when to produce (Should we build up inventories now in anticipation of high future demand? How many shifts should be run?), how much to produce and how much
capacity to build (What are the trends in market size and market share? Are
there cyclical or seasonal effects? How quickly and with what pattern will
a newly-built plant or a newly-installed technology depreciate?), and where
to produce (Should we have one plant or many? If many, where should we
locate them?). Firms also use forecasts of future prices and availability of
inputs to guide production decisions.
Econometric models are also crucial in financial services, including asset
management, asset pricing, mergers and acquisitions, investment banking,
and insurance. Portfolio managers, for example, have been interested in
empirical modeling and understanding of asset returns such as stock returns,
interest rates, exchange rates, and commodity prices.
Econometrics is similarly central to financial risk management. In recent
decades, econoemtric methods for volatility modeling have been developed
and widely applied to evaluate and insure risks associated with asset portfolios, and to price assets such as options and other derivatives.
Finally, econometrics is central to the work of a wide variety of consulting
firms, many of which support the business functions already mentioned. Litigation support is also a very active area, in which econometric models are
routinely used for damage assessment (e.g., lost earnings), but for analyses,
and so on.
Indeed these examples are just the tip of the iceberg. Surely you can think
1.2. TYPES OF RECORDED ECONOMIC DATA
of many more situations in which econometrics is used.

1.1.2
What Distinguishes Econometrics?
Econometrics is much more than just statistics using economic data, although it is of course very closely related to statistics.
Econometrics must confront the fact that economic data is not generated
from well-designed experiments. On the contrary, econometricians must
generally take whatever so-called observational data theyre given.
Econometrics must confront the special issues and features that arise
routinely in economic data, such as trends, seasonality and cycles.
Econometricians are sometimes interested in non-causal predictive modeling, which requires understanding only correlations (or, more precisely,
conditional expectations), and sometimes interested in evaluating treatment effects, which involve deeper issues of causation.
With so many applications and issues in econometrics, you might fear
that a huge variety of econometric techniques exists, and that youll have to
master all of them. Fortunately, thats not the case. Instead, a relatively
small number of tools form the common core of much econometric modeling.
We will focus on those underlying core principles.
1.2
Types of Recorded Economic Data
Several aspects of economic data will concern us frequently.

One issue is whether the data are continuous or binary. Continuous
data take values on a continuum, as for example with GDP growth, which in
principle can take any value in the real numbers. Binary data, in contrast,
take just two values, as with a 0-1 indicator for whether or not someone
purchased a particular product during the last month.
Another issue is whether the data are recorded over time, over space, or
some combination of the two. Time series data are recorded over time, as
for example with U.S. GDP, which is measured once per quarter. A GDP
dataset might contain data for, say, 1960.I to the present. Cross sectional
data, in contrast, are recorded over space (at a point in time), as with yesterdays closing stock price for each of the U.S. S&P 500 firms. The data
structures can be blended, as for example with a time series of cross sections. If, moreover, the cross-sectional units are identical over time, we speak
of panel data, or longitudinal data. An example would be the daily closing stock price for each of the U.S. S&P 500 firms, recorded over each of the
last 30 days.
1.3
Online Information and Data
Much useful information is available on the web. The best way to learn about
whats out there is to spend a few hours searching the web for whatever interests you. Here we mention just a few key must-know sites. Resources for
Economists, maintained by the American Economic Association, is a fine portal to almost anything of interest to economists. (See Figure 1.1.) It contains
hundreds of links to data sources, journals, professional organizations, and so
on. FRED (Federal Reserve Economic Data) is a tremendously convenient
source for economic data. The National Bureau of Economic Research site
has data on U.S. business cycles, and the Real-Time Data Research Center
at the Federal Reserve Bank of Philadelphia has real-time vintage macroeconomic data. Finally, check out Quandl, which provides access to millions of
data series on the web.
1.4. SOFTWARE
Figure 1.1: Resources for Economists Web Page
1.4
Software
Econometric software tools are widely available. One of the best high-level
environments is Eviews, a modern object-oriented environment with extensive time series, modeling and forecasting capabilities. (See Figure 1.2.) It
implements almost all of the methods described in this book, and many more.
Eviews reflects a balance of generality and specialization that makes it ideal
for the sorts of tasks that will concern us, and most of the examples in this
book are done using it. If you feel more comfortable with another package,
however, thats fine none of our discussion is wed to Eviews in any way,
and most of our techniques can be implemented in a variety of packages.
Eviews has particular strength in time series environments. Stata is
Figure 1.2: Eviews Homepage
a similarly good packaged with particular strength in cross sections and

panels. (See Figure 1.3.)
Eviews and Stata are examples of very high-level modeling environments.
If you go on to more advanced econometrics, youll probably want also to
have available slightly lower-level (mid-level) environments in which you
can quickly program, evaluate and apply new tools and techniques. R is
one very powerful and popular such environment, with special strengths in
modern statistical methods and graphical data analysis. (See Figure 1.4.) In
this authors humble opinion, R is the key mid-level econometric environment
for the foreseeable future. Other notable such environments include Python
and the rapidly emerging Julia.
1.5. TIPS ON HOW TO USE THIS BOOK
Figure 1.3: Stata Homepage
1.5
Tips on How to use this book
As you navigate through the book, keep the following in mind.

Hyperlinks to internal items (table of contents, index, footnotes, etc.)
appear in red.
Hyperlinks to references appear in green.
Hyperlinks to external items (web pages, video, etc.) appear in cyan.
Key concepts appear in bold, and they also appear in the (hyperlinked)
index.
Many figures are clickable to reach related material, as are, for example,
all figures in this chapter.
Most chapters contain at least one extensive empirical example in the
Econometrics in Action section.
10
Figure 1.4: R Homepage
The end-of-chapter Exercises, Problems and Complements sections

are of central importance and should be studied carefully. Exercises
are generally straightforward checks of your understanding. Problems,
in contrast, are generally significantly more involved, whether analytically or computationally. Complements generally introduce important
auxiliary material not covered in the main text.
1.6
1. (No example is definitive!)

Recall that, as mentioned in the text, most chapters contain at least one
extensive empirical example. At the same time, those examples should
not be taken as definitive or complete treatments there is no such
1.7.
NOTES
11
thing! A good idea is to think of the implicit Problem 0 at the end

of each chapter as Critique the modeling in this chapters Econometrics in Action section, obtain the relevant data, and produce a superior
analysis.
2. (Nominal, ordinal, interval and ratio data)
We emphasized time series, cross-section and panel data, whether continuous or discrete, but there are other complementary categorizations.
In particular, distinctions are often made among nominal data, ordinal data, interval data, and ratio data. Which are most common
and useful in economics and related fields, and why?
3. (Software differences and bugs: caveat emptor)
Be warned: no software is perfect. In fact, all software is highly imperfect! The results obtained when modeling in different software environments may differ sometimes a little and sometimes a lot for
a variety of reasons. The details of implementation may differ across
packages, for example, and small differences in details can sometimes
produce large differences in results. Hence, it is important that you
understand precisely what your software is doing (insofar as possible,
as some software documentation is more complete than others). And
of course, quite apart from correctly-implemented differences in details,
deficient implementations can and do occur: there is no such thing as
bug-free software.
1.7
Notes
For a compendium of econometric and statistical software, see the software

links site, maintained by Marius Ooms at the Econometrics Journal.
12
R is available for free as part of a massive and highly-successful opensource project. RStudio provides a fine R working environment, and, like
R, its free. A good R tutorial, first given on Coursera and then moved to
YouTube, is here. R-bloggers is a massive blog with all sorts of information
about all things R. Finally, Quandl has a nice R interface.
Chapter 2
Graphics and Graphical Style
Its almost always a good idea to begin an econometric analysis with graphical
data analysis. When compared to the modern array of econometric methods,
graphical analysis might seem trivially simple, perhaps even so simple as to
be incapable of delivering serious insights. Such is not the case: in many
respects the human eye is a far more sophisticated tool for data analysis
and modeling than even the most sophisticated statistical techniques. Put
differently, graphics is a sophisticated technique. Thats certainly not to
say that graphical analysis alone will get the job done certainly, graphical
analysis has its limitations of its own but its usually the best place to
start. With that in mind, we introduce in this chapter some simple graphical
techniques, and we consider some basic elements of graphical style.
2.1
Simple Techniques of Graphical Analysis
We will segment our discussion into two parts: univariate (one variable) and
multivariate (more than one variable). Because graphical analysis lets the
data speak for themselves, it is most useful when the dimensionality of
the data is low; that is, when dealing with univariate or low-dimensional
multivariate data.
13
14
CHAPTER 2. GRAPHICS AND GRAPHICAL STYLE
2.1.1
Univariate Graphics
First consider time series data. Graphics is used to reveal patterns in time
series data. The great workhorse of univariate time series graphics is the
simple time series plot, in which the series of interest is graphed against
time.
In the top panel of Figure 2.1, for example, we present a time series plot
of a 1-year Government bond yield over approximately 500 months. A number of important features of the series are apparent. Among other things,
its movements appear sluggish and persistent, it appears to trend gently upward until roughly the middle of the sample, and it appears to trend gently
downward thereafter.
The bottom panel of Figure 2.1 provides a different perspective; we plot
the change in the 1-year bond yield, which highlights volatility fluctuations.
Interest rate volatility is very high in mid-sample.
Univariate graphical techniques are also routinely used to assess distributional shape, whether in time series or cross sections. A histogram, for
example, provides a simple estimate of the probability density of a random
variable. The observed range of variation of the series is split into a number
of segments of equal length, and the height of the bar placed at a segment
is the percentage of observations falling in that segment.1 In Figure 2.2 we
show a histogram for the 1-year bond yield.
2.1.2
Multivariate Graphics
When two or more variables are available, the possibility of relations between the variables becomes important, and we use graphics to uncover the
existence and nature of such relationships. We use relational graphics to
1
In some software packages (e.g., Eviews), the height of the bar placed at a segment is simply the
number, not the percentage, of observations falling in that segment. Strictly speaking, such histograms are
not density estimators, because the area under the curve doesnt add to one, but they are equally useful
for summarizing the shape of the density.
2.1. SIMPLE TECHNIQUES OF GRAPHICAL ANALYSIS
Figure 2.1: 1-Year Goverment Bond Yield, Levels and Changes
15
16
Figure 2.2: Histogram of 1-Year Government Bond Yield
display relationships and flag anomalous observations. You already understand the idea of a bivariate scatterplot.2 In Figure 2.3, for example, we show
a bivariate scatterplot of the 1-year U.S. Treasury bond rate vs. the 10-year
U.S. Treasury bond rate, 1960.01-2005.03. The scatterplot indicates that the
two move closely together; in particular, they are positively correlated.
Thus far all our discussion of multivariate graphics has been bivariate.
Thats because graphical techniques are best-suited to low-dimensional data.
Much recent research has been devoted to graphical techniques for highdimensional data, but all such high-dimensional graphical analysis is subject
to certain inherent limitations.
One simple and popular scatterplot technique for high-dimensional data
and one thats been around for a long time is the scatterplot matrix, or
multiway scatterplot. The scatterplot matrix is just the set of all possible
bivariate scatterplots, arranged in the upper right or lower left part of a
matrix to facilitate comparisons. If we have data on N variables, there are
2
Note that connecting the dots is generally not useful in scatterplots. This contrasts to time series
plots, for which connecting the dots is fine and is typically done.
2.1. SIMPLE TECHNIQUES OF GRAPHICAL ANALYSIS
17
Figure 2.3: Bivariate Scatterplot, 1-Year and 10-Year Government Bond Yields
N 2 N
2
such pairwise scatterplots. In Figure 2.4, for example, we show a
scatterplot matrix for the 1-year, 10-year, 20-year, and 30-year U.S. Treasury
Bond rates, 1960.01-2005.03. There are a total of six pairwise scatterplots,
and the multiple comparison makes clear that although the interest rates are
closely related in each case, with a regression slope of approximately one, the
relationship is more precise in some cases (e.g., 20- and 30-year rates) than
in others (e.g., 1- and 30-year rates).
2.1.3
Summary and Extension
Lets summarize and extend what weve learned about the power of graphics:
a. Graphics helps us summarize and reveal patterns in univariate time-series
data. Time-series plots are helpful for learning about many features of
time-series data, including trends, seasonality, cycles, the nature and location of any aberrant observations (outliers), structural breaks, etc.
b. Graphics helps us summarize and reveal patterns in univariate cross-section
data. Histograms are helpful for learning about distributional shape.
18
Figure 2.4: Scatterplot Matrix, 1-, 10-, 20- and 30-Year Government Bond Yields
2.2. ELEMENTS OF GRAPHICAL STYLE
19
c. Graphics helps us identify relationships and understand their nature, in

both multivariate time-series and multivariate cross-section environments.
The key graphical device is the scatterplot, which can help us to begin
answering many questions, including: Does a relationship exist? Is it
linear or nonlinear? Are there outliers?
d. Graphics helps us identify relationships and understand their nature in
panel data. One can, for example, examine cross-sectional histograms
across time periods, or time series plots across cross-sectional units.
e. Graphics facilitates and encourages comparison of different pieces of data
via multiple comparisons. The scatterplot matrix is a classic example
of a multiple comparison graphic.
We might add to this list another item of tremendous relevance in our age
of big data: Graphics enables us to summarize and learn from huge datasets.
We will study aspects of big-data econometrics in Chapter ??.
2.2
Elements of Graphical Style
In the preceding sections we emphasized the power of graphics and introduced

various graphical tools. As with all tools, however, graphical tools can be
used effectively or ineffectively, and bad graphics can be far worse than no
graphics. In this section youll learn what makes good graphics good and bad
graphics bad. In doing so youll learn to use graphical tools effectively.
Bad graphics is like obscenity: its hard to define, but you know it when
you see it. Conversely, producing good graphics is like good writing: its
an iterative, trial-and-error procedure, and very much an art rather than a
science. But thats not to say that anything goes; as with good writing, good
graphics requires discipline. There are at least three keys to good graphics:
a. Know your audience, and know your goals.
20
b. Show the data, and only the data, withing the bounds of reason.
c. Revise and edit, again and again (and again). Graphics produced using
software defaults are almost never satisfactory.
We can use a number of devices to show the data. First, avoid distorting
the data or misleading the viewer, in order to reveal true data variation rather
than spurious impressions created by design variation. Thus, for example,
avoid changing scales in midstream, use common scales when performing
multiple comparisons, and so on. The sizes of effects in graphics should match
their size in the data.
Second, minimize, within reason, non-data ink (ink used to depict anything other than data points). Avoid chartjunk (elaborate shadings and
grids that are hard to decode, superfluous decoration including spurious 3-D
perspective, garish colors, etc.)
Third, choose a graphs aspect ratio (the ratio of the graphs height, h,
to its width, w) to maximize pattern revelation. A good aspect ratio often
makes the average absolute slope of line segments connecting the data points
approximately equal 45 degrees. This procedure is called banking to 45
degrees.
Fourth, maximize graphical data density. Good graphs often display lots
of data, indeed so much data that it would be impossible to learn from them
in tabular form.3 Good graphics can present a huge amount of data in a
concise and digestible form, revealing facts and prompting new questions, at
both micro and macro levels.4
Graphs can often be shrunken greatly with no loss, as with sparklines
(tiny graphics, typically time-series plots, meant to flow with text) and the
3
Conversely, for small amounts of data, a good table may be much more appropriate and informative
than a graphic.
4
Note how maximization of graphical data density complements our earlier prescription to maximize the
ratio of data ink to non-data ink, which deals with maximizing the relative amount of data ink. High data
density involves maximizing as well the absolute amount of data ink.
2.3. U.S. HOURLY WAGES
21
Figure 2.5: Distributions of Wages and Log Wages
sub-plots in multiple comparison graphs, increasing the amount of data ink

per unit area.
2.3
U.S. Hourly Wages
We use CPS hourly wage data; for a detailed description see Appendix B.
wage histogram skewed
wage kernel density estimate with normal superimposed skewed
log wage histogram more symmetric
log wage kernel density estimate with normal superimposed not too ba
d a fit
2.4
Concluding Remarks
Ultimately good graphics proceeds just like good writing, and if good writing
is good thinking, then so too is good graphics good thinking. And good
writing is just good thinking. So the next time you hear someone pronounce
22
ignorantly along the lines of I dont like to write; I like to think, rest
assured, both his writing and his thinking are likely poor. Indeed many of
the classic prose style references contain many insights that can be adapted
to improve graphics (even if Strunk and White would view as worthless filler
my use of indeed earlier in this sentence (non-thought ink?)).
So when doing graphics, just as when writing, think. Then revise and edit,
revise and edit, ...
2.5
1. (NBER Recession Bars: A Useful Graphical Device)

In U.S. time-series situations its often useful to superimpose NBER
Recession Bars on time-series plots, to help put things in context. You
can find the dates of NBER expansions and contractions at http://
www.nber.org/cycles.html.
2. (Empirical Warm-Up)
(a) Obtain time series of quarterly real GDP and quarterly real consumption for a country of your choice. Provide details.
(b) Display time-series plots and a scatterplot (put consumption on the
vertical axis).
(c) Convert your series to growth rates in percent, and again display
time series plots.
(d) From now on use the growth rate series only.
(e) For each series, provide summary statistics (e.g., mean, standard
deviation, range, skewness, kurtosis, ...).
(f) For each series, perform t-tests of the null hypothesis that the population mean growth rate is 2 percent.
2.5. EXERCISES, PROBLEMS AND COMPLEMENTS
23
(g) For each series, calulate 90 and 95 percent confidence intervals for
the population mean growth rate. For each series, which interval is
wider, and why?
(h) Regress consumption on GDP. Discuss.
3. (Simple vs. partial correlation)
The set of pairwise scatterplots that comprises a multiway scatterplot
provides useful information about the joint distribution of the set of variables, but its incomplete information and should be interpreted with
care. A pairwise scatterplot summarizes information regarding the simple correlation between, say, x and y. But x and y may appear highly
related in a pairwise scatterplot even if they are in fact unrelated, if
each depends on a third variable, say z. The crux of the problem is
that theres no way in a pairwise scatterplot to examine the correlation
between x and y controlling for z, which we call partial correlation.
When interpreting a scatterplot matrix, keep in mind that the pairwise
scatterplots provide information only on simple correlation.
4. (Graphics and Big Data)
Another aspect of the power of statistical graphics comes into play in
the analysis of large datasets, so its increasingly more important in our
era of Big Data: Graphics enables us to present a huge amount of
data in a small space, and hence helps to make huge datasets coherent.
We might, for example, have supermarket-scanner data, recorded in fiveminute intervals for a year, on the quantities of goods sold in each of
four food categories dairy, meat, grains, and vegetables. Tabular or
similar analysis of such data is simply out of the question, but graphics
is still straightforward and can reveal important patterns.
5. (Color)
24
There is a temptation to believe that color graphics is always better

than grayscale. Thats often far from the truth, and in any event, color
is typically best used sparingly.
a. Color can be (and often is) chartjunk. How and why?
b. Color has no natural ordering, despite the evident belief in some quarters that it does. What are the implications for heat map graphics?
Might shades of a single color (e.g., from white or light gray through
black) be better?
c. On occasion, however, color can aid graphics both in showing the data
and in appealing to the viewer. One key show the data use is in
annotation. Can you think of others? What about uses in appealing
to the viewer?
d. Keeping in mind the principles of graphical style, formulate as many
guidelines for color graphics as you can.
6. (Principles of Tabular Style)
The power of tables for displaying data and revealing patterns is very
limited compared to that of graphics, especially in this age of Big Data.
Nevertheless, tables are of course sometimes helpful, and there are principles of tabular style, just as there are principles of graphical style.
Compare, for example, the nicely-formatted Table 2.1 (no need to worry
about what it is or from where it comes...) to what would be produced
by a spreadsheet such as Excel.
Try to formulate a set of principles of tabular style. (Hint: One principle
is that vertical lines should almost never appear in tables, as in the table
above.)
7. (More on Graphical Style: Appeal to the Viewer)
25
Table 2.1: Yield Statistics

Maturity
(Months)
y (1) y (12)
6
12
24
36
60
120
4.9
5.1
5.3
5.6
5.9
6.5
2.1
2.1
2.1
2.0
1.9
1.8
0.98
0.98
0.97
0.97
0.97
0.97
0.64
0.65
0.65
0.65
0.66
0.68
Notes: We present descriptive statistics for end-of-month yields at various maturities. We show sample
mean, sample standard deviation, and first- and twelfth-order sample autocorrelations. Data are from the
Board of Governors of the Federal Reserve System. The sample period is January 1985 through December
2008.
Other graphical guidelines help us appeal to the viewer. First, use clear
and modest type, avoid mnemonics and abbreviations, and use labels
rather then legends when possible. Second, make graphics self-contained;
a knowledgeable reader should be able to understand your graphics without reading pages of accompanying text. Third, as with our prescriptions
for showing the data, avoid chartjunk.
8. (The Golden Aspect Ratio, Visual Appeal, and Showing the Data)
A time-honored approach to visual graphical appeal is use of an aspect
ratio such that height is to width as width is to the sum of height and
width. This turns out to correspond to height approximately sixty percent of width, the so-called golden ratio. Graphics that conform to
the golden ratio, with height a bit less than two thirds of width, are
visually appealing. Other things the same, its a good idea to keep the
golden ratio in mind when producing graphics. Other things are not
always the same, however. In particular, the golden aspect ratio may
not be the one that maximizes pattern revelation (e.g., by banking to
45 degrees).
26
9. Graphics, non-profit and for-profit.

Check out the non-profit community of creative people at www.visualizing.
org.
Check out Google Charts at https://developers.google.com/chart/.
Poke around.
Whats good?
Whats bad?
Can you use it to do
sparklines?
Check out www.zevross.com.
2.6
Notes
R implements a variety of sophisticated graphical techniques and in many

respects represents the cutting edge of statistical graphics software.
ggplot2 is a key R package that provides a broad catalog of graphics capabilities; see www.ggplot2.org. It implements the grammar of graphics developed by Leland Wilkenson, which allows you to produce highly customized
graphics in a modular fashion. This grammar leads to a slightly unusual syntax, which must be learned, but once learned you can do almost anything.
(The simple plot commands in R allow for some customization and have a
shorter learning curve, but theyre not as powerful.) ggplot2 documentation
is at www.cran.r-project.org/web/packages/ggplot2/ggplot2.pdf. A
helpful cheatsheet is at www.zevross.com/blog/2014/08/04/beautiful-plotting-in

#change-the-grid-lines-panel.grid.major.
2.7. GRAPHICS LEGEND: EDWARD TUFTE
2.7
27
Graphics Legend: Edward Tufte
Figure 2.6: Tufte Teaching, with a First Edition Book by Galileo
This chapter has been heavily influenced by Tufte (1983), as are all modern
discussions of statistical graphics.5 Tuftes book is an insightful and entertaining masterpiece on graphical style, and I recommend enthusiastically. Be
sure to check out his web page and other books, which go far beyond his 1983
work.
Photo details follow.

Date: 7 February 2011.
Source: http://www.flickr.com/photos/roebot/5429634725/in/set-72157625883623225.
Author: Aaron Fulkerson.
Originally posted to Flickr by Roebot at http://flickr.com/photos/40814689@N00/5429634725. Reviewed
on 24 May 2011 by the FlickreviewR robot and confirmed to be licensed under the terms of the cc-by-sa-2.0.
Licensed under the Creative Commons Attribution-Share Alike 2.0 Generic license.
28
Part II
Cross Sections
29
Chapter 3
Regression Under the Full Ideal
Conditions
You have already been introduced to probability and statistics, but chances
are that you could use a bit of review before plunging into regression, so
begin by studying Appendix A. Be warned, however: it is no substitute
for a full-course introduction to probability and statistics, which you should
have had already. Instead it is intentionally much more narrow, reviewing
some material related to moments of random variables, which we will use
repeatedly. It also introduces notation, and foreshadows certain ideas, that
we develop subsequently in greater detail.
3.1
Preliminary Graphics
In this chapter well be working with cross-sectional data on log wages, education and experience. We already examined the distribution of log wages.
For convenience we reproduce it in Figure 3.1, together with the distributions
of the new data on education and experience.
31
32
CHAPTER 3. REGRESSION UNDER THE FULL IDEAL CONDITIONS
Figure 3.1: Distributions of Log Wage, Education and Experience
3.2. REGRESSION AS CURVE FITTING
3.2
3.2.1
33
Regression as Curve Fitting

Bivariate, or Simple, Linear Regression
Suppose that we have data on two variables, y and x, as in Figure 3.2, and
suppose that we want to find the linear function of x that best fits y, where
best fits means that the sum of squared (vertical) deviations of the data
points from the fitted line is as small as possible. When we run a regression,
or fit a regression line, thats what we do. The estimation strategy is called
least squares, or sometimes ordinary least squares to distinguish it from
fancier versions that well introduce later.
The specific data that we show in Figure 3.2 are log wages (LWAGE, y)
and education (EDUC, x) for a random sample of nearly 1500 people, as
described in Appendix B.
Let us elaborate on the fitting of regression lines, and the reason for the
name least squares. When we run the regression, we use a computer to fit
the line by solving the problem
min
T
X
(yt 1 2 xt )2 ,
t=1
where is shorthand notation for the set of two parameters, 1 and 2 . We

and its elements by 1 and 2 .
denote the set of fitted parameters by ,
It turns out that the 1 and 2 values that solve the least squares problem
have well-known mathematical formulas. (More on that later.) We can use
a computer to evaluate the formulas, simply, stably and instantaneously.
The fitted values are
yt = 1 + 2 xt ,
t = 1, ..., T . The residuals are the difference between actual and fitted
values,
et = yt yt ,
34
Figure 3.2: (Log Wage, Education) Scatterplot
35
Figure 3.3: (Log Wage, Education) Scatterplot with Superimposed Regression Line
t = 1, ..., T .
In Figure 3.3, we illustrate graphically the results of regressing LWAGE on
EDUC. The best-fitting line slopes upward, reflecting the positive correlation
between LWAGE and EDUC.1 Note that the data points dont satisfy the
fitted linear relationship exactly; rather, they satisfy it on average. To predict
LWAGE for any given value of EDUC, we use the fitted line to find the value
of LWAGE that corresponds to the given value of EDUC.
1
Note that use of log wage promostes several desiderata. First, it promotes normality, as we discussed
\
in Chapter 2. Second, it enforces positivity of the fitted wage, because W\
AGE = exp(LW
AGE), and
exp(x) > 0 for any x.
36
Numerically, the fitted line is

\
LW
AGE = 1.273 + .081EDU C.
We conclude with a brief comment on notation. A standard cross-section
notation for indexing the cross-sectional units is i = 1, ..., N . A standard
time-series notation for indexing time periods is t = 1, ..., T . Much of our
discussion will be valid in both cross-section and time-series environments, but
still we have to pick a notation. Without loss of generality, we will typically
use t = 1, ..., T regardless of whether were in a cross section or time series
environment.
3.2.2
Multiple Linear Regression
Everything generalizes to allow for more than one RHS variable. This is
called multiple linear regression.
Suppose, for example, that we have two RHS variables, x2 and x3 . Before
we fit a least-squares line to a two-dimensional data cloud; now we fit a leastsquares plane to a three-dimensional data cloud. We use the computer to
find the values of 1 , 2 , and 3 that solve the problem
min
T
X
(yt 1 2 x2t 3 x3t )2 ,
t=1
where denotes the set of three model parameters. We denote the set of
with elements 1 , 2 , and 3 . The fitted values
estimated parameters by ,
are
yt = 1 + 2 x2t + 3 x3t ,
and the residuals are
et = yt yt ,
t = 1, ..., T .
37
For our wage data, the fitted model is

\
LW
AGE = .867 + .093EDU C + .013EXP ER.
Extension to the general multiple linear regression model, with an arbitrary number of right-hand-side (RHS) variables (K, including the constant),
is immediate. The computer again does all the work. The fitted line is
yt = 1 + 2 x2t + 3 x3t + ... + K xKt ,
which we sometimes write more compactly as
yt =
K
X
i xit ,
k=1
where x1t = 1 for all t.

3.2.3
Onward
Before proceeding, two aspects of what weve done so far are worth noting.
First, we now have two ways to analyze data and reveal its patterns. One is
the graphical scatterplot of Figure 3.2, with which we started, which provides
a visual view of the data. The other is the fitted regression line of Figure 3.3,
which summarizes the data through the lens of a linear fit. Each approach
has its merit, and the two are complements, not substitutes, but note that
linear regression generalizes more easily to high dimensions.
Second, least squares as introduced thus far has little to do with statistics
or econometrics. Rather, it is simply a way of instructing a computer to
fit a line to a scatterplot in a way thats rigorous, replicable and arguably
reasonable. We now turn to a probabilistic interpretation.
38
3.3
Regression as a Probability Model
We work with the full multiple regression model (simple regression is of course
a special case). Collect the RHS variables into the vector x, where x0t =
(1, x1t , x2t , ..., xKt ).
3.3.1
A Population Model and a Sample Estimator
Thus far we have not postulated a probabilistic model that relates yt and xt ;
instead, we simply ran a mechanical regression of yt on xt to find the best
fit to yt formed as a linear function of xt . Its easy, however, to construct
a probabilistic framework that lets us make statistical assessments about
the properties of the fitted line. We assume that yt is linearly related to
an exogenously-determined xt , and we add an independent and identically
distributed zero-mean (iid) Gaussian disturbance:
yt = 1 + 2 x2t + ... + K xKt + t
t iidN (0, 2 ),
t = 1, ..., T . The intercept of the line is 1 , the slope parameters are the
i s, and the variance of the disturbance is 2 .2 Collectively, we call the s
the models parameters. The index t keeps track of time; the data sample
begins at some time weve called 1 and ends at some time weve called T ,
so we write t = 1, ..., T . (Or, in cross sections, we index cross-section units
by i and write i = 1, ..., N .)
Note that in the linear regression model the expected value of yt conditional upon xt taking a particular value, say xt , is
E(yt |xt = xt ) = 1 + 2 x2t + ... + K xKt .
2
We speak of the regression intercept and the regression slope.
3.3. REGRESSION AS A PROBABILITY MODEL
39
That is, the regression function is the conditional expectation of yt .

We assume that the the linear model sketched is true in population; that
is, it is the data-generating process (DGP). But in practice, of course, we
dont know the values of the models parameters, 1 , 2 , ..., K and 2 . Our
job is to estimate them using a sample of data from the population. We estiP
mate the s precesiely as before, using the computer to solve min Tt=1 2t .
3.3.2
Notation, Assumptions and Results: The Full Ideal Conditions
The discussion thus far was intentionally a bit loose, focusing on motivation
and intuition. Let us now be more precise about what we assume and what
results obtain.
A Bit of Matrix Notation
It will be useful to arrange all RHS variables into a matrix X. X has K

columns, one for each regressor. Inclusion of a constant in a regression
amounts to including a special RHS variable that is always 1.
We put
that in the leftmost column of the X matrix, which is just ones. The other
columns contain the data on the other RHS variables, over the cross section
in the cross-sectional case, i = 1, ..., N , or over time in the time-series case,
t = 1, ..., T . With no loss of generality, suppose that were in a time-series
situation; then notationally X is a T K matrix.
1 x21 x31 . . . xK1
1 x22 x32 . . . xK2

.
X=
...
1 x2T x3T . . . xKT

One reason that the X matrix is useful is because the regression model
40
can be written very compactly using it. We have written the model as
yt = 1 + 2 x2t + ... + K xKt + t , t = 1, ..., T.
Alternatively, stack yt , t = 1, ..., T into the vector y, where y 0 = (y1 , y2 , ..., yT ),
and stack j , j = 1, ..., K into the vector , where 0 = (1 , 2 , ..., K ), and
stack t , t = 1, ..., T , into the vector , where 0 = (1 , 2 , ..., T ).
Then we can write the complete model over all observations as
y = X + .
(3.1)
Our requirement that

t iid N (0, 2 )
becomes
N (0, 2 I)
(3.2)
This concise representation is very convenient.

Indeed representation (3.1)-(3.2) is crucially important, not simply because it is concise, but because the various assumptions that we need to
make to get various statistical results are most naturally and simply stated
on X and in equation (3.1). We now proceed to discuss such assumptions.
Assumptions: The Full Ideal Conditions (FIC)
1. The DGP is (3.1)-(3.2), and the fitted model matches the DGP exactly.
2. X is fixed in repeated samples.

3. X is of full column rank (K)
FIC 1 has many important sub-conditions embedded. For example:
1. Linear relationship, E(y) = X
3.3. REGRESSION AS A PROBABILITY MODEL
41
2. Fixed coefficients,
3. N
4. has constant variance 2
5. The s are uncorrelated.
FIC 2 says that re-running the world to generate a new sample y would
entail simply generating new shocks and running them through equation
(3.1):
y = X + .
That is, X would stay fixed across replications. This fixed X condition
looks innocuous, but its not, and we will have much more to say about it.
FIC 3 is more of a technical detail. It effectively says that there is no
redundancy among the variables contained in X (more precisely, no regressor
is a perfect linear combination of the others). Full column rank of X guarantees that X 0 X is non-singular and hence invertible, which is necessary for
calculation of the OLS estimator, (X 0 X)1 X 0 y. The condition is largely technical insofar as X will always be of full column rank unless you do something
really silly, like entering the same regressor twice. However, situations where
X 0 X is almost singular can occur and raise issues, if not cause disasters. We
will discuss such situations later.
The FIC are surely heroic in many contexts, and much of econometrics is
devoted to detecting and dealing with various FIC failures. But before we
worry about FIC failures, its invaluable first to understand what happens
when they hold.
Results
The least squares estimator is

LS = (X 0 X)1 X 0 y,
42
Figure 3.4: Regression Output
and under the FIC it is MVUE and normally distributed with covariance
matrix 2 (X 0 X)1 . We write

LS N , 2 (X 0 X)1 .
We estimate the covariance matrix 2 (X 0 X)1 using s2 (X 0 X)1 , where s2 =
PT 2
t=1 et /(T K).
3.4
A Wage Equation
Now lets do more than a simple graphical analysis of the regression fit.
Instead, lets look in detail at the computer output, which we show in Figure
6.2 for a regression of LW AGE on an intercept, EDU C and EXP ER. We
run regressions dozens of times in this book, and the output format and
interpretation are always the same, so its important to get comfortable with
it quickly. The output is in Eviews format. Other software will produce
more-or-less the same information, which is fundamental and standard.
3.4. A WAGE EQUATION
43
Before proceeding, note well that the FIC may not be satisfied for this
dataset, yet we will proceed assuming that they are satisfied. As we proceed
through this book, we will confront violations of the various assumptions
indeed thats what econometrics is largely about and well return repeatedly
to this dataset and others. But we must begin at the beginning.
The printout begins by reminding us that were running a least-squares
(LS) regression, and that the left-hand-side (LHS) variable is the log wage
(LWAGE), using a total of 1323 observations.
Next comes a table listing each RHS variable together with four statistics.
The RHS variables EDUC and EXPER are education and experience, and the
C variable refers to the earlier-mentioned intercept. The C variable always
equals one, so the estimated coefficient on C is the estimated intercept of the
regression line.3
The four statistics associated with each RHS variable are the estimated
coefficient (Coefficient), its standard error (Std. Error), a t statistic,
and a corresponding probability value (Prob.). The standard errors of
the estimated coefficients indicate their likely sampling variability, and hence
their reliability. The estimated coefficient plus or minus one standard error is
approximately a 68% confidence interval for the true but unknown population
parameter, and the estimated coefficient plus or minus two standard errors
is approximately a 95% confidence interval, assuming that the estimated
coefficient is approximately normally distributed, which will be true if the
regression disturbance is normally distributed or if the sample size is large.
Thus large coefficient standard errors translate into wide confidence intervals.
Each t statistic provides a test of the hypothesis of variable irrelevance:
that the true but unknown population parameter is zero, so that the corresponding variable contributes nothing to the forecasting regression and can
therefore be dropped. One way to test variable irrelevance, with, say, a 5%
3
Sometimes the population coefficient on C is called the constant term, and the regression estimate is
called the estimated constant term.
44
probability of incorrect rejection, is to check whether zero is outside the 95%

confidence interval for the parameter. If so, we reject irrelevance. The t
statistic is just the ratio of the estimated coefficient to its standard error,
so if zero is outside the 95% confidence interval, then the t statistic must be
bigger than two in absolute value. Thus we can quickly test irrelevance at the
5% level by checking whether the t statistic is greater than two in absolute
value.4
Finally, associated with each t statistic is a probability value, which is
the probability of getting a value of the t statistic at least as large in absolute
value as the one actually obtained, assuming that the irrelevance hypothesis
is true. Hence if a t statistic were two, the corresponding probability value
would be approximately .05. The smaller the probability value, the stronger
the evidence against irrelevance. Theres no magic cutoff, but typically probability values less than 0.1 are viewed as strong evidence against irrelevance,
and probability values below 0.05 are viewed as very strong evidence against
irrelevance. Probability values are useful because they eliminate the need for
consulting tables of the t distribution. Effectively the computer does it for us
and tells us the significance level at which the irrelevance hypothesis is just
rejected.
Now lets interpret the actual estimated coefficients, standard errors, t
statistics, and probability values. The estimated intercept is approximately
.867, so that conditional on zero education and experience, our best forecast
of the log wage would be 86.7 cents. Moreover, the intercept is very precisely
estimated, as evidenced by the small standard error of .08 relative to the
estimated coefficient. An approximate 95% confidence interval for the true
but unknown population intercept is .867 2(.08), or [.71, 1.03]. Zero is
far outside that interval, so the corresponding t statistic is huge, with a
4
If the sample size is small, or if we want a significance level other than 5%, we must refer to a table of
critical values of the t distribution. We also note that use of the t distribution in small samples also requires
an assumption of normally distributed disturbances.
45
probability value thats zero to four decimal places.

The estimated coefficient on EDUC is .093, and the standard error is again
small in relation to the size of the estimated coefficient, so the t statistic is
large and its probability value small. The coefficient is positive, so that
LWAGE tends to rise when EDUC rises. In fact, the interpretation of the
estimated coefficient of .09 is that, holding everything else constant, a oneyear increase in EDUC will produce a .093 increase in LWAGE.
The estimated coefficient on EXPER is .013. Its standard error is is also
small, and hence its t statistic is large, with a very small probability value.
Hence we reject the hypothesis that EXPER contributes nothing to the forecasting regression. A one-year increase in EXP ER tends to produce a .013
increase in LWAGE.
A variety of diagnostic statistics follow; they help us to evaluate the adequacy of the regression. We provide detailed discussions of many of them
elsewhere. Here we introduce them very briefly:
3.4.1
Mean dependent var 2.342
The sample mean of the dependent variable is

T
1X
yt .
y =
T t=1
It measures the central tendency, or location, of y.

3.4.2
S.D. dependent var .561
The sample standard deviation of the dependent variable is

s
PT
)2
t=1 (yt y
SD =
.
T 1
It measures the dispersion, or scale, of y.
46
3.4.3
Sum squared resid 319.938
Minimizing the sum of squared residuals is the objective of least squares

estimation. Its natural, then, to record the minimized value of the sum of
squared residuals. In isolation its not of much value, but it serves as an
input to other diagnostics that well discuss shortly. Moreover, its useful for
comparing models and testing hypotheses. The formula is
SSR =
T
X
e2t .
t=1
3.4.4
Log likelihood -938.236
The likelihood function is the joint density function of the data, viewed
as a function of the model parameters. Hence a natural estimation strategy, called maximum likelihood estimation, is to find (and use as estimates) the parameter values that maximize the likelihood function. After all,
by construction, those parameter values maximize the likelihood of obtaining the data that were actually obtained. In the leading case of normallydistributed regression disturbances, maximizing the likelihood function (or
equivalently, the log likelihood function, because the log is a monotonic transformation) turns out to be equivalent to minimizing the sum of squared residuals, hence the maximum-likelihood parameter estimates are identical to the
least-squares parameter estimates. The number reported is the maximized
value of the log of the likelihood function.5 Like the sum of squared residuals, its not of direct use, but its useful for comparing models and testing
hypotheses. We will rarely use the log likelihood function directly; instead,
well focus for the most part on the sum of squared residuals.
5
Throughout this book, log refers to a natural (base e) logarithm.
3.4.5
47
F statistic 199.626
We use the F statistic to test the hypothesis that the coefficients of all variables in the regression except the intercept are jointly zero.6 That is, we
test whether, taken jointly as a set, the variables included in the forecasting
model have any predictive value. This contrasts with the t statistics, which
we use to examine the predictive worth of the variables one at a time.7 If no
variable has predictive value, the F statistic follows an F distribution with
k 1 and T k degrees of freedom. The formula is
F =
(SSRres SSR)/(K 1)
,
SSR/(T K)
where SSRres is the sum of squared residuals from a restricted regression that
contains only an intercept. Thus the test proceeds by examining how much
the SSR increases when all the variables except the constant are dropped. If
it increases by a great deal, theres evidence that at least one of the variables
has predictive content.
3.4.6
Prob(F statistic) 0.000000
The probability value for the F statistic gives the significance level at which
we can just reject the hypothesis that the set of RHS variables has no predictive value. Here, the value is indistinguishable from zero, so we reject the
hypothesis overwhelmingly.
3.4.7
S.E. of regression .492
If we knew the elements of and forecasted yt using x0t , then our forecast
errors would be the t s, with variance 2 . Wed like an estimate of 2 ,
6
We dont want to restrict the intercept to be zero, because under the hypothesis that all the other
coefficients are zero, the intercept would equal the mean of y, which in general is not zero. See Problem 6.
7
In the degenerate case of only one RHS variable, the t and F statistics contain exactly the same information, and F = t2 . When there are two or more RHS variables, however, the hypotheses tested differ, and
F 6= t2 .
48
because it tells us whether our forecast errors are likely to be large or small.
The observed residuals, the et s, are effectively estimates of the unobserved
population disturbances, the t s . Thus the sample variance of the es, which
we denote s2 (read s-squared), is a natural estimator of 2 :
PT
s =
2
t=1 et
T K
s2 is an estimate of the dispersion of the regression disturbance and hence is

used to assess goodness of fit of the model, as well as the magnitude of forecast
errors that were likely to make. The larger is s2 , the worse the models fit,
and the larger the forecast errors were likely to make. s2 involves a degreesof-freedom correction (division by T K rather than by T 1, reflecting the
fact that K regression coefficients have been estimated), which is an attempt
to get a good estimate of the out-of-sample forecast error variance on the
basis of the in-sample residuals.
The standard error of the regression (SER) conveys the same information; its an estimator of rather than 2 , so we simply use s rather than
s2 . The formula is
SER =
s
s2
PT
2
t=1 et
T K
The standard error of the regression is easier to interpret than s2 , because

its units are the same as those of the es, whereas the units of s2 are not. If
the es are in dollars, then the squared es are in dollars squared, so s2 is in
dollars squared. By taking the square root at the end of it all, SER converts
the units back to dollars.
Sometimes its informative to compare the standard error of the regression (or a close relative) to the standard deviation of the dependent variable
(or a close relative). The standard error of the regression is an estimate of
the standard deviation of forecast errors from the regression model, and the
standard deviation of the dependent variable is an estimate of the standard
49
deviation of the forecast errors from a simpler forecasting model, in which

the forecast each period is simply y . If the ratio is small, the variables in the
model appear very helpful in forecasting y. R-squared measures, to which we
now turn, are based on precisely that idea.
3.4.8
R-squared .232
If an intercept is included in the regression, as is almost always the case,

R-squared must be between zero and one. In that case, R-squared, usually
written R2 , is the percent of the variance of y explained by the variables
included in the regression. R2 measures the in-sample success of the regression equation in forecasting y; hence it is widely used as a quick check of
goodness of fit, or forecastability of y based on the variables included in
the regression. Here the R2 is about 23% well above zero but not great.
The formula is
PT
2
t=1 et
R = 1 PT
)2
t=1 (yt y
We can write R2 in a more roundabout way as

1
T
R =1
1
T
PT
2
t=1 et
PT
)2
t=1 (yt y
which makes clear that the numerator in the large fraction is very close to
s2 , and the denominator is very close to the sample variance of y.
3.4.9
Adjusted R-squared .231
The interpretation is the same as that of R2 , but the formula is a bit different.
Adjusted R2 incorporates adjustments for degrees of freedom used in fitting
the model, in an attempt to offset the inflated appearance of good fit if many
RHS variables are tried and the best model selected. Hence adjusted R2
is a more trustworthy goodness-of-fit measure than R2 . As long as there is
50
more than one RHS variable in the model fitted, adjusted R2 is smaller than
R2 ; here, however, the two are extremely close (23.1% vs. 23.2%). Adjusted
2 ; the formula is
R2 is often denoted R
2 = 1
R
PT 2
1
t=1 et
T K
,
P
T
1
2
(y
)
t
t=1
T 1
where K is the number of RHS variables, including the constant term. Here
the numerator in the large fraction is precisely s2 , and the denominator is
precisely the sample variance of y.
3.4.10
Akaike info criterion 1.423
The Akaike information criterion, or AIC, is effectively an estimate of

the out-of-sample forecast error variance, as is s2 , but it penalizes degrees
of freedom more harshly. It is used to select among competing forecasting
models. The formula is:
AIC = e( )
2K
T
3.4.11
PT
2
t=1 et
Schwarz criterion 1.435
The Schwarz information criterion, or SIC, is an alternative to the AIC

with the same interpretation, but a still harsher degrees-of-freedom penalty.
The formula is:
SIC = T ( )
K
T
PT
2
t=1 et
.
T
The AIC and SIC are tremendously important for guiding model selection
in a ways that avoid data mining and in-sample overfitting.
3.4.12
51
Hannan-Quinn criter. 1.427
Hannan-Quinn is yet another information criterion for use in model selection.

We will not use it in this book.
3.4.13
Durbin-Watson stat. 1.926
The Durbin-Watson statistic is useful in time series environments for assessing

whether the t s are correlated over time; that is, whether the iid assumption
(part of the full ideal conditions) is violated. It is irrelevant in the present application to wages, which uses cross-section data. We nevertheless introduce
it briefly here.
The Durbin-Watson statistic tests for correlation over time, called serial correlation, in regression disturbances. It works within the context of
a regression model with disturbances
t = t1 + vt
vt iidN (0, 2 ).
The regression disturbance is serially correlated when 6= 0 . The hypothesis
of interest is that = 0 . When = 0, the ideal conditions hold, but when
6= 0, the disturbance is serially correlated. More specifically, when 6= 0,
we say that t follows an autoregressive process of order one, or AR(1) for
short.8 If > 0 the disturbance is positively serially correlated, and if < 0
the disturbance is negatively serially correlated. Positive serial correlation
is typically the relevant alternative in the applications that will concern us.
The formula for the Durbin-Watson (DW ) statistic is
PT
DW =
8
2
t=2 (et et1 )
.
PT 2
e
t=1 t
Although the Durbin-Watson test is designed to be very good at detecting serial correlation of the AR(1)
type. Many other types of serial correlation are possible; well discuss them extensively in Chapter 11.1.
52
Figure 3.5: Wage Regression Residual Scatter
DW takes values in the interval [0, 4], and if all is well, DW should be
around 2. If DW is substantially less than 2, there is evidence of positive
serial correlation. As a rough rule of thumb, if DW is less than 1.5, there
may be cause for alarm, and we should consult the tables of the DW statistic,
available in many statistics and econometrics texts.
3.4.14
The Residual Scatter
The residual scatter is often useful in both cross-section and time-series

situations. It is a plot of y vs y. A perfect fit (R2 = 1) corresponds to all
points on the 45 degree line, and no fit (R2 = 0) corresponds to all points on
a vertical line corresponding to y = y.
In Figure 3.5 we show the residual scatter for the wage regression. It is
not a vertical line, but certainly also not the 45 degree line, corresponding to
the positive but relatively low R2 of .23.
3.4.15
53
The Residual Plot
In time-series settings, its always a good idea to assess visually the adequacy of the model via time series plots of the actual data (yt s), the fitted
values (
yt s), and the residuals (et s). Often well refer to such plots, shown
together in a single graph, as a residual plot.9 Well make use of residual
plots throughout this book. Note that even with many RHS variables in
the regression model, both the actual and fitted values of y, and hence the
residuals, are simple univariate series that can be plotted easily.
The reason we examine the residual plot is that patterns would indicate
violation of our iid assumption. In time series situations, we are particularly
interested in inspecting the residual plot for evidence of serial correlation
in the et s, which would indicate failure of the assumption of iid regression
disturbances. More generally, residual plots can also help assess the overall
performance of a model by flagging anomalous residuals, due for example to
outliers, neglected variables, or structural breaks.
Our wage regression is cross-sectional, so there is no natural ordering of
the observations, and the residual plot is of limited value. But we can still
use it, for example, to check for outliers.
In Figure 3.6, we show the residual plot for the regression of LWAGE on
EDUC and EXPER. The actual and fitted values appear at the top of the
graph; their scale is on the right. The fitted values track the actual values
fairly well. The residuals appear at the bottom of the graph; their scale is
on the left. Its important to note that the scales differ; the et s are in fact
substantially smaller and less variable than either the yt s or the yt s. We
draw the zero line through the residuals for visual comparison. No outliers
are apparent.
9
Sometimes, however, well use residual plot to refer to a plot of the residuals alone. The intended
meaning should be clear from context.
54
Figure 3.6: Wage Regression Residual Plot
3.4.16
A Bit More on AIC and SIC
You will want to start using AIC and SIC immediately, so we provide a
bit more information here. Model selection by maximizing R2 , or equivalently minimizing residual SSR, is ill-advised, because they dont penalize
for degrees of freedom and therefore tend to prefer models that are too big.
2 , or equivalently minimizing residual s2 ,
Model selection by maximizing R
2 and s2 penalize somewhat for degrees of
is still ill-advised, even though R
freedom, because they dont penalize harshly enough and therefore still tend
to prefer models that are too big. In contrast, AIC and SIC get things just
right. SIC has a wonderful asymptotic optimality property when the set of
candidate models is viewed as fixed: Basically SIC gets it right asymptotically, selecting either the DGP (if the DGP is among the models considered)
or the best predictive approximation to the DGP (if the DGP is not among
the models considered). AIC has a different and also-wonderful asymptotic
3.5. QUANTILE REGRESSION
55
optimality property, known as efficiency, when the set of candidate models

is viewed as expanding as the sample size grows. In practice, the models
selected by AIC and SIC rarely disagree.
3.5
Quantile Regression
Beyond OLS: Non-Quadratic Objectives

Ordinary Least Squares (OLS)
Recall that the OLS estimator, OLS , solves:
min
T
X
(yt 1 2 x2t ... K xKt ) = min
t=1
Simple
(analytic closed-form expression, (X 0 X)1 X 0 y)
Wonderful properties under the FIC
(Unbiased, consistent, Gaussian, MVUE)
But other approaches are possible and sometimes useful.
T
X
t=1
2t
56
Quantile Regression (QR)

Objective function linear on each side of 0 but with (generally) unequal
slopes.
QR estimator QR minimizes linlin loss, or check function loss:
min
T
X
linlin(t ),
t=1
where:
linlin(e) =
a|e|, if e 0
b|e|, if e > 0
= a|e| I(e 0) + b|e| I(e > 0).

I(x) = 1 if x is true, and I(x) = 0 otherwise.
I() stands for indicator variable.
linlin refers to linearity on each side of the origin.
Not as simple as OLS, but still simple
(solves a linear programming problem)
What Does Quantile Regression Fit?

QR fits the d 100% quantile:
quantiled (y|X) = x
where
d=
b
1
=
a + b 1 + a/b
Important generalization of regression

(e.g., How do the wages of people in the far left tail of the wage distribution
vary with education and experience, and how does that compare to those in
57
the center of the wage distribution?)
3.6
1. (Regression with and without a constant term) Consider Figure 3.3, in

which we showed a scatterplot of y vs. x with a fitted regression line
superimposed.
a. In fitting that regression line, we included a constant term. How can
you tell?
b. Suppose that we had not included a constant term. How would the
figure look?
c. We almost always include a constant term when estimating regressions. Why?
d. When, if ever, might you explicitly want to exclude the constant term?
2. (Interpreting coefficients and variables) Let yt = 1 + 2 xt + 3 zt + t ,
where yt is the number of hot dogs sold at an amusement park on a
given day, xt is the number of admission tickets sold that day, zt is the
daily maximum temperature, and t is a random error. Assume the FIC.
a. State whether each of yt , xt , zt , 1 , 2 and 3 is a coefficient or a
variable.
b. Determine the units of 1 , 2 and 3 , and describe the physical meaning of each.
c. What do the signs of the a coefficients tell you about how the various variables affects the number of hot dogs sold? What are your
expectations for the signs of the various coefficients (negative, zero,
positive or unsure)?
58
d. Is it sensible to entertain the possibility of a non-zero intercept (i.e.,

1 6= 0)? 2 > 0? 3 < 0?
3. (Scatter plots and regression lines)
Draw qualitative scatter plots and regression lines for each of the following two-variable datasets, and state the R2 in each case:
a. Data set 1: y and x have correlation 1
b. Data set 2: y and x have correlation -1
c. Data set 3: y and x have correlation 0.
4. (Desired values of regression diagnostic statistics)
For each of the diagnostic statistics listed below, indicate whether, other
things the same, bigger is better, smaller is better, or neither. Explain your reasoning. (Hint: Be careful, think before you answer, and
be sure to qualify your answers as appropriate.)
a. Coefficient
b. Standard error
c. t statistic
d. Probability value of the t statistic
e. R-squared
f. Adjusted R-squared
g. Standard error of the regression
h. Sum of squared residuals
i. Log likelihood
j. Durbin-Watson statistic
k. Mean of the dependent variable
59
l. Standard deviation of the dependent variable

m. Akaike information criterion
n. Schwarz information criterion
o. F statistic
p. Probability-value of the F statistic
5. (Regression semantics)
Regression analysis is so important, and used so often by so many people,
that a variety of associated terms have evolved over the years, all of which
are the same for our purposes. You may encounter them in your reading,
so its important to be aware of them. Some examples:
a. Ordinary least squares, least squares, OLS, LS.
b. y, LHS variable, regressand, dependent variable, endogenous variable
c. xs, RHS variables, regressors, independent variables, exogenous variables, predictors, covariates
d. probability value, prob-value, p-value, marginal significance level
e. Schwarz criterion, Schwarz information criterion, SIC, Bayes information criterion, BIC
6. (Regression when X Contains Only an Intercept)
Consider the regression model (3.1)-(3.2), but where X contains only an
intercept.
a. What is the OLS estimator of the intercept?
b. What is the distribution of the OLS estimator under the full ideal
conditions?
c. Does the variance-covariance matrix of the OLS estimator under the
ideal conditions depend on any unknown parameters, and if so, how
would you estimate them?
60
7. (Dimensionality)
We have emphasized, particularly in Chapter 2, that graphics is a powerful tool with a variety of uses in the construction and evaluation of
econometric models. We hasten to add, however, that graphics has its
limitations. In particular, graphics loses much of its power as the dimension of the data grows. If we have data in ten dimensions, and we try to
squash it into two or three dimensions to make graphs, theres bound to
be some information loss.
Thus, in contrast to the analysis of data in two or three dimensions,
in which case learning about data by fitting models involves a loss of
information whereas graphical analysis does not, graphical methods lose
their comparative advantage in higher dimensions. In higher dimensions, graphical analysis can become comparatively laborious and less
insightful.
8. (Wage regressions)
The relationship among wages and their determinants is one of the most
important in all of economics. In the text we have examined, and will
continue to examine, the relationship for 1995 using a CPS subsample.
Here you will thoroughly analyze the relationship for 2004 and 2012,
compare your results to those for 1995, and think hard about the meaning and legitimacy of your results.
(a) Obtain the relevant 1995, 2004 and 2012 CPS subsamples.
(b) Discuss any differences in the datasets. Are the same people in each
dataset?
(c) For now, assume the validity of the ideal conditions. Using each
dataset, run the OLS regression W AGE c, EDU C, EXP ER.
(Note that the LHS variable is W AGE, not LW AGE.) Discuss and
compare the results in detail.
61
(d) Now think of as many reasons as possible to be skeptical of your

results. (This largely means think of as many reasons as possible
why the IC might fail.) Which of the IC might fail? One? A few?
All? Why? Insofar as possible, discuss the IC, one-by-one, how/why
failure could happen here, the implications of failure, how you might
detect failure, what you might do if failure is detected, etc.
(e) Repeat all of the above using LW AGE as the LHS variable.
9. (Quantile Regression)
For the 1995 CPS subsample (see EPC 8) re-do the regression LW AGE
c, EDU C, EXP ER using 20%, 50% and 80% quantile regression instead
of OLS regression.
10. (The variety of information criteria reported across software packages)
Some authors, and software packages, examine and report the logarithms
of the AIC and SIC,
PT
2
t=1 et
ln(AIC) = ln
T
PT
ln(SIC) = ln
2
t=1 et

+
!
+
2K
T
K ln(T )
.
T
The practice is so common that log(AIC) and log(SIC) are often simply called the AIC and SIC. AIC and SIC must be greater than
zero, so log(AIC) and log(SIC) are always well-defined and can take on
any real value. The important insight, however, is that although these
variations will of course change the numerical values of AIC and SIC
produced by your computer, they will not change the rankings of models under the various criteria. Consider, for example, selecting among
three models. If AIC1 < AIC2 < AIC3 , then it must be true as well
62
that ln(AIC1 ) < ln(AIC2 ) < ln(AIC3 ) , so we would select model 1 regardless of the definition of the information criterion used.
11. (Parallels between the sampling distribution of the sample mean under simple random sampling, and the sampling distribution of the OLS
estimator under the FIC)
Consider first the sample mean under Gaussian simple random sampling.
(a) What is a Gaussian simple random sample?
(b) What is the sample mean, and what finite-sample properties does
it have under Gaussian simple random sampling?
(c) Display and discuss the exact distribution of the sample mean.
(d) How would you estimate and plot the exact distribution of the sample mean?
Now consider the OLS regression estimator under the FIC.
(a) What are the FIC?
(b) What is the OLS estimator, and what finite-sample properties does
it enjoy?
(c) Display and discuss the exact distribution of the OLS estimator.
Under what conditions, if any, do your sample mean answers and
OLS answers precisely coincide?
12. (The sum of squared residuals, SSR)
(a) What is SSR and why is it reported?
(b) Do you agree with bigger is better, smaller is better, or neither?
Be careful.
63
2,
(c) Describe in detail and discuss the use of regression statistics R2 , R
F , SER, and SIC. What role does SSR play in each of the test
statistics?
(d) Under the FIC, is the maximized log likelihood related to the SSR?
If so, how? Would your answer change if we dropped normality?
13. (OLS regression residuals sum to zero)
Assertion: As long as an intercept is included in a linear regression, the
OLS residuals must sum to precisely zero. The intuition is simply that
non-zero residual mean (residual constant term) would automatically
be pulled into the residual constant term, hence guaranteeing a zero
residual mean.
(a) Prove the assertion precisely.
(b) Evaluate the claim that the assertion implies the regression fits perfectly on average, despite the fact that it fits imperfectly pointby-point.
14. (Maximum-likelihood estimation and likelihood-ratio tests)
A natural estimation strategy with wonderful asymptotic properties,
called maximum likelihood estimation, is to find (and use as estimates) the parameter values that maximize the likelihood function.
After all, by construction, those parameter values maximize the likelihood of obtaining the data that were actually obtained.
In the leading case of normally-distributed regression disturbances, maximizing the likelihood function turns out to be equivalent to minimizing
the sum of squared residuals, hence the maximum-likelihood parameter
estimates are identical to the least-squares parameter estimates.
To see why maximizing the Gaussian log likelihood gives the same parameter estimate as minimizing the sum of squared residuals, let us
64
derive the likelihood for the Gaussian linear regression model with nonstochastic regressors,
yt = x0t +
t iidN (0, 2 ).
The model implies that
yt iidN (x0t , 2 ),
so that
1
f (yt ) = (2 2 ) 2 e 22 (yt xt ) .
Hence f (y1 , ..., yT ) = f (y1 )f (y2 ) f (yT ) (by independence of the yt s).
In particular,
L=
T
Y
(2 2 ) 2 e 22 (yt xt )
t=1
so
ln L = (2 )
T
2
1 X
(yt x0t )2 .
2
2 t=1
Note in particular that the vector that maximizes the likelihood (or
log likelihood the optimizers must be identical because the log is a
positive monotonic transformation) is the vector that minimizes the
sum of squared residuals.
The log likelihood is also useful for hypothesis testing via likelihoodratio tests. Under very general conditions we have asymptotically that:
2(ln L0 ln L1 ) 2d ,
where ln L0 is the maximized log likelihood under the restrictions implied
by the null hypothesis, ln L1 is the unrestricted log likelihood, and d is
the number of restrictions imposed under the null hypothesis.
3.7. NOTES
65
t and F tests are likelihood ratio tests under a normality assumption.

Thats why they can be written in terms of minimized SSRs rather
than maximized ln Ls.
3.7
Notes
Dozens of software packages implement linear regression analysis. Most automatically include an intercept in linear regressions unless explicitly instructed
otherwise. That is, they automatically create and include a C variable.
The R command for ordinary least squares regression is lm. Its already
pre-loaded into R as the default package for estimating linear models. It
uses standard R format for such models, where you specify formula, data,
and various estimation options. It returns a model estimated by OLS including coefficients, residuals, and fitted values. You can also easily calculate
summary statistics using the summary function.
The standard R quantile regression package is quantreg, written by Roger
Koenker, the inventor of quantile regression. The command rq functions
similarly to lm. It takes as input a formula, data, the quantile to be
estimated, and various estimation options.
66
3.8
Regressions Inventor: Carl Friedrich Gauss
Figure 3.7: Carl Friedrich Gauss
This is a photographic reproduction of public domain work of art, an oil

painting of German mathematician and philosopher Carl Friedrich Gauss by
G. Biermann (1824-1908). Date: 1887 (painting). Source Gau-Gesellschaft
Gttingen e.V. (Foto: A. Wittmann). Photo by A. Wittmann.
Chapter 4
Non-Normal Disturbances
Here we consider a violation of the full ideal conditions, non-normal disturbances.
Non-normality and outliers, which we introduce in this chapter, are
closely related, because deviations from Gaussian behavior are often characterized by fatter tails than the Gaussian, which produce outliers. It is important to note that outliers are not necessarily bad, or requiring treatment.
Every data set must have some most extreme observation, by definition! Statistical estimation efficiency, moreover, increases with data variability. The
most extreme observations can be the most informative about the phenomena of interest. Bad outliers, in contrast, are those associated with things
like data recording errors (e.g., you enter .753 when you mean to enter 75.3)
or one-off events (e.g., a strike or natural disaster).
4.0.1
Results
To understand the properties of OLS without normality, it is helpful first to

consider the properties of the sample mean without normality.
As reviewed in Chapter A, for a non-Gaussian simple random sample,
yt iid(, 2 ), i = 1, ..., T,
67
68
CHAPTER 4. NON-NORMAL DISTURBANCES
we have that the sample mean is best linear unbiased (BLUE), with
a

2
y N ,
.
T
This result forms the basis for asymptotic inference. It is a Gaussian central
limit theorem. We consistently estimate the large-sample variance of the
sample mean using s2 /T .
Now consider the full linear regression model without normal disturbances.
We have that the linear regression estimator is BLUE, with
a

OLS N , 2 (X 0 X)1 .
We consistently estimate the large-sample variance of OLS using s2 (X 0 X)1 .

Clearly the linear regression results for Gaussian vs. non-Gaussian situations precisely parallel those for the sample mean in Gaussian vs. nonGaussian situations.1 In each case the original result obtains even when we
drop normality, except that it become a large-sample rather than an exact
finite-sample result. The central limit theorem is a wonderful thing!
4.1
Assessing Normality
There are many methods, ranging from graphics to formal tests.

1
3.
Indeed they must, as the sample mean corresponds to regression on an intercept. See EPC 6 of Chapter
4.1. ASSESSING NORMALITY
4.1.1
69
QQ Plots
We introduced histograms earlier in Chapter 2 as a graphical device for learning about distributional shape. If, however, interest centers on the tails of
distributions, QQ plots often provide sharper insight as to the agreement or
divergence between the actual and reference distributions.
The QQ plot is simply a plot of the quantiles of the standardized data
against the quantiles of a standardized reference distribution (e.g., normal). If
the distributions match, the QQ plot is the 45 degree line. To the extent that
the QQ plot does not match the 45 degree line, the nature of the divergence
can be very informative, as for example in indicating fat tails.
4.1.2
Residual Sample Skewness and Kurtosis
Recall skewness and kurtosis, which we reproduce here for convenience:

S=
E(y )3
3
E(y )4
K=
.
4
Obviously, each tells about a different aspect of non-normality. Kurtosis, in
particular, tells about fatness of distributional tails relative to the normal.
A simple strategy is to check various implications of residual normality,
such as S = 0 and K = 3 , via informal examination of S and K.

4.1.3
The Jarque-Bera Test
The Jarque-Bera test (JB) effectively aggregates the information in the

data about both skewness and kurtosis to produce an overall test of the joint
The test statistic
hypothesis that S = 0 and K = 3 , based upon S and K.
is
T
JB =
6

1
3)2 .
S2 + (K
4
70
Under the null hypothesis of independent normally-distributed observations

(S = 0, K = 3), JB is distributed in large samples as a 2 random variable
with two degrees of freedom.2
4.2
Outliers
Outliers refer to big disturbances (in population) or residuals (in sample).

Outliers may emerge for a variety of reasons, and they may require special
attention because they can have substantial influence on the fitted regression
line.
On the one hand, OLS retains its magic in such outlier situations it is
BLUE regardless of the disturbance distribution. On the other hand, the
fully-optimal (MVUE) estimator may be highly non-linear, so the fact that
OLS remains BLUE is less than fully comforting. Indeed OLS parameter
estimates are particularly susceptible to distortions from outliers, because the
quadratic least-squares objective really hates big errors (due to the squaring)
and so goes out of its way to tilt the fitted surface in a way that minimizes
them.
How to identify and treat outliers is a time-honored problem in data analysis, and theres no easy answer. If an outlier is simply a data-recording
mistake, then it may well be best to discard it if you cant obtain the correct data. On the other hand, every dataset, even perfectly clean datasets
have a most extreme observation, but it doesnt follow that it should be discarded. Indeed the most extreme observations are often the most informative
precise estimation requires data variation.
2
We have discussed the case of an observed time series. If the series being tested for normality is the
residual from a model, then T can be replaced with T K, where K is the number of parameters estimated,
although the distinction is inconsequential asymptotically.
4.3. ROBUST ESTIMATION
4.2.1
71
Outlier Detection
Graphics
One obvious way to identify outliers in bivariate regression situations is via

graphics: one xy scatterplot can be worth a thousand words. In higher
dimensions, residual yy scatterplots remain invaluable, as does the residual
plot of y y.
Leave-One-Out and Leverage
Another way to identify outliers is a leave-one-out coefficient plots, where

we use the computer to sweep through the sample, leaving out successive
observations, examining differences in parameter estimates with observation
(t)
t in vs. out. That is, in an obvious notation, we examine and plot OLS ,
OLS
t = 1, ..., T .
It can be shown, however, that the change in OLS is
(t)
OLS OLS =
1
(X 0 X)1 x0t et ,
1 ht
where ht is the t-th diagonal element of the hat matrix, X(X 0 X)1 X 0 .
(t)
Hence the estimated coefficient change
OLS is driven by 1 . ht is
OLS
1ht
called the time-t leverage. ht can be shown to be in [0, 1], so that the larger
(t)
is ht , the larger is
OLS . Hence one really just needs to examine the
OLS
leverage sequence, and scrutinize carefully observations with high leverage.
4.3
Robust Estimation
Robust estimation provides a useful middle ground between completely discarding allegedly-outlying observations (dummying them out) and doing
nothing. Here we introduce outlier-robust approaches to regression. The
first involves OLS regression, but on weighted data, an the second involves
72
switching from OLS to a different estimator.

4.3.1
Robustness Iteration
Fit at robustness iteration 0:

y(0) = X (0)
where
(0) = argmin
" T
X
#
(yt x0t )2 .
t=1
Robustness weight at iteration 1:

(1)
t = S
(0)
et
(0)
6 med|et |
where
(0)
(0)
et = yt yt ,
and S(z) is a function such that S(z) = 1 for z [1, 1] but downweights
outside that interval.
Fit at robustness iteration 1:
y(1) = X (1)
where
"
(1) = argmin
T
X
t=1
Continue as desired.
#
(1)
t (yt xt 0 )
4.3. ROBUST ESTIMATION
4.3.2
73
Least Absolute Deviations
Recall that the OLS estimator solves

min
T
X
(yt 1 2 x2t ... K xKt )2 .
t=1
Now we simply change the objective to

min
T
X
|yt 1 2 x2t ... K xKt |.
t=1
or
min
T
X
|t |
t=1
That is, we change from squared-error loss to absolute-error loss. We call

the new estimator least absolute deviations (LAD)and we write LAD .3
By construction, LAD is not influenced by outliers as much as OLS . Put
differently, LAD is more robust to outliers than is OLS.
Of course nothing is free, and the price of LAD is a bit of extra computational complexity relative to OLS. In particular, the LAD estimator does
not have a tidy closed-form analytical expression like OLS, so we cant just
plug into a simple formula to obtain it. Instead we need to use the computer
to find the optimal directly. If that sounds complicated, rest assured that
its largely trivial using modern numerical methods, as embedded in modern
software.4
It is important to note that whereas OLS fits the conditional mean function:
mean(y|X) = x,
3
Note that LAD regression is just quantile regression for d = .50.

Indeed computation of the LAD estimator turns out to be a linear programming problem, which is
well-studied and simple.
4
74
LAD fits the conditional median function (50% quantile):

median(y|X) = x
The conditional mean and median are equal under symmetry and hence under
normality, but not under asymmetry, in which case the median is a better
measure of central tendency. Hence LAD delivers two kinds of robustness to
non-normality: it is robust to outliers and robust to asymmetry.
4.4
4.4.1
Wage Determination
W AGE
Consider W AGE c, EDU C, EXP ER

Residual plot
Residual scatterplot
Residual histogram and stats
Residual Gaussian QQ plot
Leverages, {ht }, t = 1, ..., T
Robustness iteration estimation
LAD estimation
4.4.2
LW AGE
Consider LW AGE c, EDU C, EXP ER

Residual plot
Residual scatterplot
Residual histogram and stats
Residual Gaussian QQ plot
Leave-one-out plot
Robustness iteration
75
LAD estimation
4.5
1. (Talebs The Black Swan)

Nassim Taleb is a financial markets trader turned pop author. His book,
The Black Swan (Taleb (2007)), deals with many of the issues raised
in this chapter. Black swans are seemingly impossible or very lowprobability events after all, swans are supposed to be white that
occur with annoying regularity in reality. Read his book. Where does
your reaction fall on the spectrum from A to B below?
A. Taleb offers crucial lessons for econometricians, heightening awareness
in ways otherwise difficult to achieve. After reading Taleb, its hard to
stop worrying about non-normality, model uncertainty, etc.
B. Taleb belabors the obvious for hundreds of pages, arrogantly informing us that non-normality is prevalent, that all models are misspecified,
and so on. Moreover, it takes a model to beat a model, and Taleb offers
little.
2. Additional ways of quantifying outliers.
(a) Consider the outlier probability,
P |y | > 5
(there is of course nothing magical about our choice of 5). In practice
we use a sample version of the population object.
(b) Consider the tail index , such that
s.t. P (y > y ) = ky .
76
In practice we use a sample version of the population object.

3. (Leave-One-Out Coefficient Plots)
Leave-one-out coefficient plots are more appropriate for cross-section
data than for time-series data. Why? How might you adapt them to
handle time-series data?
4.6
Notes
The Jarque-Bera test is developed in Jarque and Bera (1987). Closely-related

(and earlier) is the Kiefer-Salmon test (Kiefer and Salmon (1983)).
Koenker (2005) provides extensive discussion of LAD and its extensions.
Chapter 5
Stochastic Regressors I: Measurement
Error, Omitted Variables, and
Multicollinearity
5.1
Dropping the Fixed X Assumption
Sometimes its logically impossible to maintain the fixed X assumption in

the FIC, as for example in a time-series environment when X contains a
lagged dependent variable.1 In any event, even if its logically possible to
maintain the fixed X assumption, it often feels unnatural, and wed like to
relax it.
Actually its related to how we conceptualize the regression. Fixed X
(fixed design) has more the feel of an experiment, where an experimenter
treats certain subjects in certain ways, and he could and would do so
identically in repeated samples. Stochastic X (random design) has more
the feel of an observational study, where neither y nor X is controlled by an
experimenter.
If we drop the fixed X assumption, then under a key condition (plus
some regularity conditions on X see EPC 1) many of our earlier results
continue to hold. In particular, OLS is still consistent and asymptotically
1
We will study such environments later in Chapter 12. For now we continue to work in cross sections.
77
78CHAPTER 5. STOCHASTIC REGRESSORS I: MEASUREMENT ERROR, OMITTED VARIABLES,
normally distributed, and its covariance matrix is still consistently estimated

by 2 (X 0 X)1 .
The key condition is that E(X 0 ) = 0. Under the rest of the FIC, the
condition says that X and are independent. We can go even farther and
relax normality and fixed-X simultaneously, and our usual estimators still
have their usual asymptotic properties so long as X and are independent.2
Notice that moving from fixed to stochastic X changes nothing in terms
of our calculation of OLS , its sample covariance matrix, and its asymptotic
normality. Thats because inference proceeds conditional on the observed
X. But the key to everything is the orthogonality condition, E(X 0 ) = 0.
Notice that in the fixed X case theres no need to worry about it X cant
be correlated with because its not random and hence cant be correlated
with anything. But in the stochastic X case, theres a lot to worry about.
Here we begin to worry.
Let us call the weaker conditions (the FIC but without normality, and
with the fixed X assumption replaced by stochastic X with and X independent) the ideal conditions (IC), in contrast to the FIC.
Now that were allowing for stochastic X, we can introduce some violations
of the IC that explicitly involve the stochastic X matrix.
5.2
Omitted Variables
Many of the diagnostic statistics that we have introduced are designed to

detect violations of the full ideal conditions (e.g., the Durbin-Watson test for
serial correlation in disturbances). Omission of relevant variables is another
violation of the full ideal conditions.
Suppose that the DGP is
yt = 1 + 2 zt + t ,
2
Actually we can significantly weaken even the requirement that X and are independent; see EPC 2.
5.3. MORE ON CAUSAL VS. NON-CAUSAL REGRESSION
79
but that we incorrectly run

y c, x
where corr(xt , zt ) > 0.
Clearly well estimate a positive effect of x on y, in large as well as small
samples, even though its completely spurious and would vanish if z had been
included in the regression. The positive bias arises because in our example
we assumed that corr(xt , zt ) > 0; in general the sign of the bias could go
either way. We speak of omitted variable bias. Note the key breakdown
in the IC: the assumption that regressors and disturbances are independent
is violated in the regression
y c, x.
It is important to note the distinction between what we will call consistency for a predictive effect and consistency for a treatment effect. The
relationship estimated by running
y c, x
is useful for predicting y given an observation on x (consistency for a predictive effect). But its not useful for determining the effect on y of an
exogenous shift in x (the true effect is 0!)(consistency for a treatment effect).
5.3
More on Causal vs. Non-Causal Regression
There are actually two flavors of consistency of concern in econometrics.

They happen to coincide under the FIC, but they generally diverge when the
the FIC fail.
5.3.1
Causal Predictive Modeling and T-Consistency
Consider a standard linear regression setting with K regressors.

We will say that an estimator is consistent for a treatment effect (Tconsistent) if
plimk = E(y|x)/xk , k = 1, ..., K; that is, if

E(y|x)
k
p 0, k = 1, ..., K.
xk
Hence in large samples k provides a good estimate of the effect on y of a
one-unit treatment or intervention performed on xk .
T-consistency is the standard econometric notion of consistency.
OLS is T-consistent under the FIC.
OLS is generally not T-consistent without the FIC.
And Remember How Stringent the FIC Are!
5.3.2
Non-Causal Predictive Modeling and P-Consistency
Again consider a standard linear regression setting with K regressors.

Assuming quadratic loss, the predictive risk of a parameter configuration
is
R() = E(y x0 )2 .
Let B be a set of s and let B minimize R().
We will say that is consistent for a predictive effect (P-consistent) if
= R( ); that is, if
plimR()

R() R( ) p 0.
Hence in large samples provides a good way to predict y for any hypo OLS is effectively always P-consistent; we require
thetical x: simply use x0 .
almost no conditions of any kind!
5.3. MORE ON CAUSAL VS. NON-CAUSAL REGRESSION
5.3.3
81
Correlation vs. Causality, and P-Consistency vs. T-Consistency
The distinction between P-consistency and T-consistency is related to the

distinction between correlation and causality. As is well known, correlation
does not imply causality! As long as x and y are correlated, we can exploit
very generally to predict y given knowledge
the correlation (as captured in )
of x. That is, there will be a nonzero predictive effect of x knowledge on
y. But nonzero correlation doesnt necessarily tell us anything about the
causal treatment effect of x treatments on y. That requires the full ideal
conditions. Even if there is a non-zero predictive effect of x on y (as captured
by LS ), there may or may not be a nonzero treatment effect of x on y, and
even if nonzero it will generally not equal the predictive effect.
So, assembling things, we have that:
1. P-consistency is consistency for a non-causal predictive effect. (Almost
trivially easy to obtain.)
2. T-consistency is consistency for a causal predictive effect. (Notoriously
difficult to obtain reliably.)
To link to our wage equation example, predicting someones wage based
on knowledge of her education is straightforward, regardless of whether the
FIC hold. Predicting the extra wage accruing to people treated with an extra
year of education (i.e., predicting the effects of such a policy) is quite another
matter, however, requiring the FIC.
5.3.4
An Example of Correlation Without Causality
To take a simple example, suppose that y and x are in fact causally unrelated,
so that the true treatment effect of x on y is 0 by construction. But suppose
that x is also highly correlated with an unobserved variable z that does cause
y. Then y and x will be correlated due to their joint dependence on z, and
that correlation can be used to predict y given x, despite the fact that, by
construction, x treatments (interventions) will have no effect on y.
Consider a thought experiment. Let the DGP be yi = zi + i , and suppose
also that there exists a variable x such that corr(x, z) > 0. Consider running
the regression y x. OLS P-consistent, as always. But its not OLS
T-consistent, because the omitted variable zi is in i , so that the regressor
and disturbance in the fitted regression are correlated. The fitted regression
coefficient on x will be non-zero and may be very large, even asymptotically,
despite that fact that the true causal impact of x on y is zero, by construction.
5.4
Measurement Error
Suppose the DGP is

yt = 1 + 2 xt + t ,
but that we cant measure xt accurately. Instead we measure
xm
t = xt + vt .
Think of vt as an iid measurement error with variance v2 . (Assume that it
is also independent of t .) Clearly, as v2 gets larger relative to x2 , the fitted
regression
y c, xm
is progressively less able to identify the true relationship. In the limit as v2
/ x2 , it is impossible. In any event, LS is biased toward zero, in small
as well as large samples. We speak of the errors in variables problem.
5.5
Multicollinearity
Collinearity refers to two x variables that are highly correlated. But all pairwise correlations could be small and yet an x variable is highly correlated
5.5. MULTICOLLINEARITY
83
with a linear combination of other x variables. That raises the idea of multicollinearity, where an x variable is highly correlated with a linear combination
of other x variables. Hence collinearity is a special case of multicollinearity,
and from now on we will simply speak of multicollinearity.
There are two types of multicollinearity, perfect and imperfect.
5.5.1
Perfect Multicollinearity
Perfect multicollinearity refers to perfect correlation among some regressors,

or linear combinations of regressors. Perfect multicollinearity is indeed a
problem; X 0 X matrix is singular, so (X 0 X)1 does not exist, and the OLS
estimator cannot even be computed! Perfect multicollinearity is disastrous,
but its unlikely to occur unless you do something really silly, like entering
the same regressor twice.3 In any event the solution is trivial: simply drop
one of the redundant variables.
5.5.2
Imperfect Multicollinearity
Imperfect multicollinearity, in contrast occurs routinely but is not necessarily problematic, although in extreme cases it may require some attention. Imperfect collinearity/multicollinearity refers to (imperfect) correlation among some regressors, or linear combinations of regressors. Imperfect
multicollinearity is not a problem in the sense that something was done
incorrectly, and it is not a violation of the FIC. Rather, it just reflects the
nature of economic and financial data. But we still need to be aware of it
and understand its effects.
Telltale symptoms are large F and R2 , yet small ts (large s.e.s), and/or
coefficients that are sensitive to small changes in sample period. That is,
3
A classic and more sophisticated example involves the dummy variable trap, in which we include as
regressors a full set of dummy variables and an intercept. We will define dummy variables and note the
dummy variable trap in Chapters 6 and 9.
OLS has trouble parsing individual influences, yet its clear that there is an
overall relationship.
OLS is in some sense just what the doctor ordered orthogonal projection.
5.5.3
A Bit More
It can be shown, and it is very intuitive, that
var(k ) = f |{z}
2 , x2k , Rk2
|{z} |{z}
+
where Rk2 is the R2 from a regression of xk on all other regressors. In the

limit, as Rk2 1, var(k ) , because xk is then perfectly explained by
the other variables and is therefore completely redundant. Rk2 is effectively
a measure of the strength of the multicollinearity affecting k . We often
measure the strength of multicollinearity by the variance inflation factor,
V IF (k ) =
1
,
1 Rk2
which is just a transformation of Rk2 .
5.6
1. Regularity conditions on X.
We have repeatedly mentioned (in section 5.1, for example) regularity
conditions on X, without saying more, required for various of the results claimed. In cross sections, sufficient regularity would obtain for
X iid. In time series, sufficient regularity would obtain for X covariance stationary. We will define and discuss covariance stationarity in
Chapter 11. It corresponds to a certain kind of stability over time.
2. Dropping the assumption that X and are independent.
85
Actually we can go even farther than we did in section 5.1, and relax
normality and fixed-X simultaneously, under even weaker conditions not
requiring full independence of X and , and our usual estimators still
have their usual asymptotic properties. We just need that be conditional mean and variance independent of X. More precisely, in the
iid X (cross-section) case we need E(t |xt ) = 0 and var(t |xt ) = 2 ,
and in the dynamic (time-series) case we need E(t |xt , xt1 , ...) = 0 and
var(t |xt , xt1 , ...) = 2 .
3. Measurement error correlated with X.
In section 5.4, we considered measurement error v that was independent of , and we argued that such measurement error biases the OLS
estimator toward 0. What happens if v and are correlated ?
4. Omitted variable bias.
How might you assess whether a regression suffers from omitted variable
bias?
5. Included Irrelevant Variables
Another violation of the full ideal conditions is inclusion of irrelevant
variables. Fortunately the effects are minor; some degrees of freedom
are wasted, but otherwise theres no problem.
How would you assess whether a variable included in a regression is
irrelevant? A set of variables?
Chapter 6
Indicator Variables in Cross Sections:
Group Heterogeneity
From one perspective we contimue working under the FIC. From another
we now begin relaxing the FIC, effectively by recognizing RHS variables that
were omitted from, but should not have been omitted from, our original wage
regression.
6.1
0-1 Dummy Variables
A dummy variable, or indicator variable, is just a 0-1 variable that

indicates something, such as whether a person is female, non-white, or a
union member. We use dummy variables to account for such group effects,
if any. We might define the dummy UNION, for example, to be 1 if a person
is a union member, and 0 otherwise. That is,
(
1, if observation t corresponds to a union member
U N IONt =
0, otherwise.
In Figure 6.1 we show histograms and statistics for all potential determinants of wages. Education (EDUC) and experience (EXPER) are standard
continuous variables, although we measure them only discretely (in years);
87
88CHAPTER 6. INDICATOR VARIABLES IN CROSS SECTIONS: GROUP HETEROGENEITY
Figure 6.1: Histograms for Wage Covariates
we have examined them before and there is nothing new to say. The new variables are 0-1 dummies, UNION (already defined) and NONWHITE, where
(
1, if observation t corresponds to a non white person
N ON W HIT Et =
0, otherwise.
Note that the sample mean of a dummy variable is the fraction of the
sample with the indicated attribute. The histograms indicate that roughly
one-fifth of people in our sample are union members, and roughly one-fifth
are non-white.
We also have a third dummy, FEMALE, where
(
1, if observation t corresponds to a female
F EM ALEt =
0, otherwise.
We dont show its histogram because its obvious that FEMALE should be
approximately 0 w.p. 1/2 and 1 w.p. 1/2, which it is.
6.2. GROUP DUMMIES IN THE WAGE REGRESSION
89
Sometimes dummies like UNION, NONWHITE and FEMALE are called

intercept dummies, because they effectively allow for a different intercept
for each group (union vs. non-union, non-white vs. white, female vs. male).
The regression intercept corresponds to the base case (zero values for all
dummies) and the dummy coefficients give the extra effects when the respective dummies equal one. For example, in a wage regression with an intercept
and a single dummy (UNION, say), the intercept corresponds to non-union
members, and the estimated coefficient on UNION is the extra effect (up or
down) on LWAGE accruing to union members.
Alternatively, we could define and use a full set of dummies for each category (e.g., include both a union dummy and a non-union dummy) and drop
the intercept, reading off the union and non-union effects directly.
In any event, never include a full set of dummies and an intercept. Doing
so would be redundant because the sum of a full set of dummies is just a unit
vector, but thats what the intercept is.1 If an intercept is included, one of
the dummy categories must be dropped.
6.2
Group Dummies in the Wage Regression
Recall our basic wage regression,

LW AGE c, EDU C, EXP ER,
shown in Figure 6.2. Both explanatory variables highly significant, with
expected signs.
Now consider the same regression, but with our three group dummies
added, as shown in Figure 6.3. All dummies are significant with the expected
signs, and R2 is higher. Both SIC and AIC favor including the group dummies. We show the residual scatter in Figure 6.4. Of course its hardly the
1
Well examine such issues in detail later when we study multicollinearity in Chapter ??.
Figure 6.2: Wage Regression on Education and Experience
Figure 6.3: Wage Regression on Education, Experience and Group Dummies
91
Figure 6.4: Residual Scatter from Wage Regression on Education, Experience and Group
Dummies
forty-five degree line (the regression R2 is higher but still only .31), but its
getting closer.
6.3
1. (Slope dummies)
Consider the regression
yt = 1 + 2 xt + t .
The dummy variable model as introduced in the text generalizes the
intercept term such that it can change across groups. Instead of writing
the intercept as 1 , we write it as 1 + Dt .
We can also allow slope coefficients to vary with groups. Instead of
writing the slope as 2 , we write it as 2 + Dt . Hence to capture slope
variation across groups we regress not only on an intercept and x, but
also on D x.
Allowing for both intercept and slope variation across groups corresponds
to regressing on an intercept, D, x, and D x.
2. (Dummies vs. separate regression)
Consider the simple regression, yt c, xt .
(a) How is inclusion of a group G intercept dummy related to the idea
of running separate regressions, one for G and one for non-G? Are
the two strategies equivalent? Why or why not?
(b) How is inclusion of group G intercept and slope dummies related
to the idea of running separate regressions, one for G and one for
non-G? Are the two strategies equivalent? Why or why not?
3. (Analysis of variance (ANOVA) and dummy variable regression)
[You should have learned about analysis of variance (ANOVA) in
your earlier statistics course. In any event theres good news: If you
understand regression on dummy variables, you understand analysis of
variance (ANOVA), as any ANOVA analysis can be done via regression
on dummies. So here we go.]
You treat each of 1000 randomly-selected farms that presently use no
fertilizer. You either do nothing, or you apply one of four experimental
fertilizers, A, B, C or D. Using a dummy variable regression setup:
(a) How would you test the hypothesis that none of the four new fertilizers is effective?
6.4. NOTES
93
(b) Assuming that you reject the null, how would you estimate the improvement (or worsening) due to using fertilizer A, B, C or D?
6.4
Notes
ANOVA traces to Sir Ronald Fischers 1918 article, The Correlation Between Relatives on the Supposition of Mendelian Inheritance, and it was
featured prominently in his classic 1925 book, Statistical Methods for Research Workers. Fischer is in many ways the father of much of modern
statistics.
6.5
Dummy Variables, ANOVA, and Sir Ronald Fischer
Figure 6.5: Sir Ronald Fischer
Photo credit: From Wikimedia commons. Source: http://www.swlearning.com/quant/kohler/stat/

biographical_sketches/Fisher_3.jpeg Rationale: Photographer died 70yrs ago = PD. Date: 2008-0530 (original upload date). Source: Transferred from en.wikipedia. Author: Original uploader was Bletchley
at en.wikipedia. Permission (Reusing this file): Released under the GNU Free Documentation License; PDOLD-70. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU
Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included
in the section entitled GNU Free Documentation License.
Chapter 7
Non-Linear Functional Form in Cross
Sections
In general there is no reason why the conditional mean function should be linear. That is, the appropriate functional form may not be linear. Whether
linearity provides an adequate approximation is an empirical matter.
non-linearity is related to non-normality, which we studied in chapter 7.
In particular, in the mutivariate normal case, the conditional mean function
is linear in the conditioning variables. But once we leave the terra firma
of multivariate normality, anything goes. The conditional mean function
and disturbances may be linear and Gaussian, non-linear and Gaussian, linear
and non-Gaussian, or non-linear and non-Gaussian.
In the Gaussian case, because the conditional mean is a linear function
of the conditioning variable(s), it coincides with the linear projection. In
non-Gaussian cases, however, linear projections are best viewed as approximations to generally non-linear conditional mean functions. That is, we can
view the linear regression model as a linear approximation to a generally nonlinear conditional mean function. Sometimes the linear approximation may
be adequate, and sometimes not.
95
96
CHAPTER 7. NON-LINEAR FUNCTIONAL FORM IN CROSS SECTIONS
7.1
Models Linear in Transformed Variables
Models can be non-linear but nevertheless linear in non-linearly-transformed

variables. A leading example involves logarithms, to which we now turn. This
can be very convenient. Moreover, coefficient interpretations are special, and
similarly convenient.
7.1.1
Logarithms
Logs turn multiplicative models additive, and they neutralize exponentials.

Logarithmic models, although non-linear, are nevertheless linear in logs.
In addition to turning certain non-linear models linear, they can be used
to enforce non-negativity of a left-hand-side variable and to stabilize a disturbance variance. (More on that later.)
Log-Log Regression
First, consider log-log regression. We write it out for the simple regression case, but of course we could have more than one regressor. We have
lnyt = 1 + 2 lnxt + t .
yt is a non-linear function of the xt , but the function is linear in logarithms,
so that ordinary least squares may be applied.
To take a simple example, consider a Cobb-Douglas production function
with output a function of labor and capital,
yt = ALt Kt exp(t ).
Direct estimation of the parameters A, , would require special techniques.
Taking logs, however, yields
lnyt = lnA + lnLt + lnKt + t .
7.1. MODELS LINEAR IN TRANSFORMED VARIABLES
97
This transformed model can be immediately estimated by ordinary least

squares. We simply regress lnyt on an intercept, lnLt and lnKt . Such log-log
regressions often capture relevant non-linearities, while nevertheless maintaining the convenience of ordinary least squares.
Note that the estimated intercept is an estimate of lnA (not A, so if you
want an estimate of A you must exponentiate the estimated intercept), and
the other estimated parameters are estimates of and , as desired.
Recall that for close yt and xt , (ln yt ln xt ) is approximately the percent
difference between yt and xt . Hence the coefficients in log-log regressions give
the expected percent change in E(yt |xt ) for a one-percent change in xt , the
so-called elasticity of yt with respect to xt .
Log-Lin Regression
Second, consider log-lin regression, in which lnyt = xt + . We have
a log on the left but not on the right. The classic example involves the
workhorse model of exponential growth:
yt = Aert
Its non-linear due to the exponential, but taking logs yields
lnyt = lnA + rt,
which is linear. The growth rate r gives the approximate percent change in
E(yt |t) for a one-unit change in time (because logs appear only on the left).
Lin-Log Regression
Finally, consider lin-log Regression:
yt = lnxt + .
Its a bit exotic but it sometimes arises. gives the effect on E(yt |xt ) of a
one-percent change in xt , because logs appear only on the right.
98
7.1.2
Box-Cox and GLM
Box-Cox
The Box-Cox transformation generalizes log-lin regression. We have
B(yt ) = 1 + 2 xt + t ,
where
yt 1
B(yt ) =
.
Hence
E(yt |xt ) = B 1 (1 + 2 xt ).
Because

lim0
y 1

= ln(yt ),
the Box-Cox model corresponds to the log-lin model in the special case of
= 0.
GLM
The so-called generalized linear model (GLM) provides an even more
flexible framework. Almost all models with left-hand-side variable transformations are special cases of those allowed in the generalized linear model
(GLM). In the GLM, we have
G(yt ) = 1 + 2 xt + t ,
so that
E(yt |xt ) = G1 (1 + 2 xt ).
Wide classes of link functions G can be entertained. Log-lin regression,
for example, emerges when G(yt ) = ln(yt ), and Box-Cox regression emerges
when G(yt ) =
yt 1
.
7.2. INTRINSICALLY NON-LINEAR MODELS
7.2
99
Intrinsically Non-Linear Models
Sometimes we encounter intrinsically non-linear models. That is, there

is no way to transform them to linearity, so that they can then be estimated
simply by least squares, as we have always done so far.
As an example, consider the logistic model,
y=
1
,
a + brx
with 0 < r < 1. The precise shape of the logistic curve of course depends on
the precise values of a, b and r, but its S-shape is often useful. The key
point for our present purposes is that there is no simple transformation of y
that produces a model linear in the transformed variables.
7.2.1
Nonlinear Least Squares
The least squares estimator is often called ordinary least squares, or OLS.
As we saw earlier, the OLS estimator has a simple closed-form analytic expression, which makes it trivial to implement on modern computers. Its
computation is fast and reliable.
The adjective ordinary distinguishes ordinary least squares from more
laborious strategies for finding the parameter configuration that minimizes
the sum of squared residuals, such as the non-linear least squares (NLS)
estimator. When we estimate by non-linear least squares, we use a computer
to find the minimum of the sum of squared residual function directly, using
numerical methods, by literally trying many (perhaps hundreds or even thousands) of different values until we find those that appear to minimize the
sum of squared residuals. This is not only more laborious (and hence slow),
but also less reliable, as, for example, one may arrive at a minimum that is
local but not global.
Why then would anyone ever use non-linear least squares as opposed to
100
OLS? Indeed, when OLS is feasible, we generally do prefer it. For example,
in all regression models discussed thus far OLS is applicable, so we prefer it.
Intrinsically non-linear models cant be estimated using OLS, however, but
they can be estimated using non-linear least squares. We resort to non-linear
least squares in such cases.
Intrinsically non-linear models obviously violate the linearity assumption
of the FIC. But the violation is not a big deal. Under the remaining FIC
(that is, dropping only linearity), N LS has a sampling distribution similar
to that under the FIC.
7.3
Series Expansions
Really no such thing as an intrinsically non-linear model...

In the bivariate case we can think of the relationship as
yt = g(xt , t )
or slightly less generally as
yt = f (xt ) + t
Taylor
First consider Taylor series expansions of f (xt ).
The linear (first-order) approximation is
f (xt ) 1 + 2 x,
and the quadratic (second-order) approximation is
f (xt ) 1 + 2 xt + 3 x2t .
7.3. SERIES EXPANSIONS
101
In the multiple regression case, Taylor approximations also involve interaction terms. Consider, for example, f (xt , zt ):
f (xt , zt ) 1 + 2 xt + 3 zt + 4 x2t + 5 zt2 + 6 xt zt + ....
Such interaction effects are also relevant in situations involving dummy
variables. There we capture interactions by including products of dummies.1
Key insight:
The ultimate point is that so-called intrinsically non-linear models are
themselves linear when viewed from the series-expansion perspective. In principle, of course, an infinite number of series terms are required, but in practice
nonlinearity is often quite gentle (e.g., quadratic) so that only a few series
terms are required.
So non-linearity is in some sense really an omitted-variables problem
Fourier
f (xt ) 1 + 2 sin(xt ) + 3 cos(xt ) + 4 sin(2xt ) + 5 cos(2xt ) + ...

One can also mix Taylor and Fourier approximations by regressing not
only on powers and cross products (Taylor terms), but also on various sines
and cosines (Fourier terms).
Mixing may facilitate parsimony.
1
Notice that a product of dummies is one if and only if both individual dummies are one.
102
7.4
A Final Word on Nonlinearity and the FIC
It is of interest to step back and ask what parts of the FIC are violated in
our various non-linear models.
Models linear in transformed variables (e.g., log-log regression) actually
dont violate the FIC, after transformation. Neither do series expansion models, if the adopted expansion order is deemed correct, because they too are
linear in transformed variables.
The series approach to handling non-linearity is actually very general and
handles intrinsically non-linear models as well, and low-ordered expansions
are often adequate in practice, even if an infinite expansion is required in
theory. If series terms are needed, a purely linear model would suffer from
misspecification of the X matrix (a violation of the FIC) due to the omitted
higher-order expansion terms. Hence the failure of the FIC discussed in this
chapter can be viewed either as:
1. The linearity assumption (E(y|X) = X 0 ) is incorrect, or
2. The linearity assumption (E(y|X) = X 0 ) is correct, but the assumption
that X is correctly specified (i.e., no omitted variables) is incorrect, due
to the omitted higher-order expansion terms.
7.5
7.5.1
Selecting a Non-Linear Model

t and F Tests, and Information Criteria
One can use the usual t and F tests for testing linear models against nonlinear alternatives in nested cases, and information criteria (AIC and SIC)
for testing against non-linear alternatives in non-nested cases. To test linearity against a quadratic alternative in a simple regression case, for example,
we can simply run y c, x, x2 and perform a t-test for the relevance of x2 .
And of course, use AIC and SIC as always.
7.6. NON-LINEARITY IN WAGE DETERMINATION
7.5.2
103
The RESET Test
Direct inclusion of powers and cross products of the various X variables in

the regression can be wasteful of degrees of freedom, however, particularly
if there are more than just one or two right-hand-side variables in the regression and/or if the non-linearity is severe, so that fairly high powers and
interactions would be necessary to capture it.
In light of this, a useful strategy is first to fit a linear regression yt c, Xt
and obtain the fitted values yt . Then, to test for non-linearity, we run the
regression again with various powers of yt included,
yt c, Xt , yt2 , ..., ytm .
Note that the powers of yt are linear combinations of powers and cross products of the X variables just what the doctor ordered. There is no need
to include the first power of yt , because that would be redundant with the
included X variables. Instead we include powers yt2 , yt3 , ... Typically a small
m is adequate. Significance of the included set of powers of yt can be checked
using an F test. This procedure is called RESET (Regression Specification
Error Test).
7.6
Non-Linearity in Wage Determination
For convenience we reproduce in Figure 7.1 the results of our current linear
wage regression,
LW AGE c, EDU C, EXP ER,
F EM ALE, U N ION, N ON W HIT E.
The RESET test from that regression suggests neglected non-linearity; the
p-value is .03 when using yt2 and yt3 in the RESET test regression.
Non-Linearity in EDU C and EXP ER: Powers and Interactions
104
Figure 7.1: Basic Linear Wage Regression
Given the results of the RESET test, we proceed to allow for non-linearity.
In Figure 7.2 we show the results of the quadratic regression
LW AGE EDU C, EXP ER
EDU C 2 , EXP ER2 , EDU C EXP ER,
F EM ALE, U N ION, N ON W HIT E
Two of the non-linear effects are significant. The impact of experience is
decreasing, and experience seems to trade off with education, insofar as the
interaction is negative.
Non-Linearity in F EM ALE, U N ION and N ON W HIT E: Interactions
Just as continuous variables like EDU C and EXP ER may interact (and
we found that they do), so too may discrete dummy variables. For example,
the wage effect of being female and non-white might not simply be the sum
of the individual effects. We would estimate it as the sum of coefficients on
the individual dummies F EM ALE and N ON W HIT E plus the coefficient
7.6. NON-LINEARITY IN WAGE DETERMINATION
105
Figure 7.2: Quadratic Wage Regression
on the interaction dummy F EM ALE*N ON W HIT E.

In Figure 7.4 we show results for
LW AGE EDU C, EXP ER,
F EM ALE, U N ION, N ON W HIT E,
F EM ALEU N ION, F EM ALEN ON W HIT E, U N ION N ON W HIT E.
The dummy interactions are insignificant.
7.6.1
Non-Linearity in Continuous and Discrete Variables Simultaneously
Now lets incorporate powers and interactions in EDU C and EXP ER, and
interactions in F EM ALE, U N ION and N ON W HIT E.
In Figure 7.4 we show results for
106
Figure 7.3: Wage Regression on Education, Experience, Group Dummies, and Interactions

F EM ALE, U N ION, N ON W HIT E,
F EM ALEU N ION, F EM ALEN ON W HIT E, U N ION N ON W HIT E.
The dummy interactions remain insignificant.
Note that we could explore additional interactions among EDU C, EXP ER
and the various dummies. We leave that to the reader.
Assembling all the results, our tentative best model thus far is the of
section 7.6,
F EM ALE, U N ION, N ON W HIT E.
The RESET statistic has a p-value of .19, so we would not reject adequacy
107
Figure 7.4: Wage Regression with Continuous Non-Linearities and Interactions, and Discrete
Interactions
of functional form at conventional levels.
7.7
1. (Non-linear vs. linear relationships)

2. (Tax revenue and the tax rate)
The U.S. Congressional Budget Office (CBO) is helping the president
to set tax policy. In particular, the president has asked for advice on
where to set the average tax rate to maximize the tax revenue collected
per taxpayer. For each of 65 countries the CBO has obtained data on
the tax revenue collected per taxpayer and the average tax rate.
(a) Is tax revenue likely related to the tax rate? (That is, do you think
that the mean of tax revenue conditional on the tax rate actually is
a function of the tax rate?)
(b) Is the relationship likely linear? (Hint: how much revenue would be
108
collected at tax rates of zero or one hundred percent?)

(c) If not, is a linear regression nevertheless likely to produce a good
approximation to the true relationship?
3. (Graphical regression diagnostic: scatterplot of et vs. xt )
This plot helps us assess whether the relationship between y and the set
of xs is truly linear, as assumed in linear regression analysis. If not,
the linear regression residuals will depend on x. In the case where there
is only one right-hand side variable, as above, we can simply make a
scatterplot of et vs. xt . When there is more than one right-hand side
variable, we can make separate plots for each, although the procedure
loses some of its simplicity and transparency.
4. (Difficulties with non-linear optimization)
Non-linear optimization can be a tricky business, fraught with problems.
Some problems are generic. Its relatively easy to find a local optimum,
for example, but much harder to be confident that the local optimum
is global. Simple checks such as trying a variety of startup values and
checking the optimum to which convergence occurs are used routinely,
but the problem nevertheless remains. Other problems may be software
specific. For example, some software may use highly accurate analytic
derivatives whereas other software uses approximate numerical derivatives. Even the same software package may change algorithms or details
of implementation across versions, leading to different results.
5. (Conditional mean functions)
Consider the regression model,
yt = 1 + 2 xt + 3 x2t + 4 zt + t
109
under the full ideal conditions. Find the mean of yt conditional upon
xt = xt and zt = zt . Is the conditional mean linear in (xt ? zt )?
6. (OLS vs. NLS)
Consider the following three regression models:
yt = 1 + 2 xt + t
yt = 1 e2 xt t
yt = 1 + e2 xt + t .
a. For each model, determine whether OLS may be used for estimation
(perhaps after transforming the data), or whether NLS is required.
b. For those models for which OLS is feasible, do you expect NLS and
OLS estimation results to agree precisely? Why or why not?
c. For those models for which NLS is required, show how to avoid it
using series expansions.
7. (Graphical regression diagnostic: scatterplot of et vs. xt )
This plot helps us assess whether the relationship between y and the set
of xs is truly linear, as assumed in linear regression analysis. If not,
the linear regression residuals will depend on x. In the case where there
is only one right-hand side variable, as above, we can simply make a
scatterplot of et vs. xt . When there is more than one right-hand side
variable, we can make separate plots for each, although the procedure
loses some of its simplicity and transparency.
8. (What is linear regression really estimating?)
It is important to note the distinction between a conditional mean and
a linear projection. The conditional mean is not necessarily a linear function of the conditioning variable(s). The linear projection is of
110
course a linear function of the conditioning variable(s), by construction.

Linear projections are best viewed as approximations to generally nonlinear conditional mean functions. That is, we can view an empirical
linear regression as estimating the population linear projection, which
in turn is an approximation to the population conditional expectation.
Sometimes the linear projection may be an adequate approximation, and
sometimes not.
9. Putting lots of things together.
Consider the cross-sectional (log) wage equation that we studied extensively, which appears again in Figure 7.5 for your reference.
(a) The model was estimated using ordinary least squares (OLS). What
loss function is optimized in calculating the OLS estimate? (Give
a formula and a graph.) What is the formula (if any) for the OLS
estimator, and what is its exact finite-sample distribution under the
full ideal conditions (FIC)?
(b) Consider instead estimating the same model numerically (i.e., by
NLS) rather than analytically (i.e., by OLS). What loss function is
optimized in calculating the NLS estimate? (Give a formula and a
graph.) What is the formula (if any) for the NLS estimator, and
what is its exact finite-sample distribution under the FIC?
(c) Does the estimated equation indicate a statistically significant effect
of union status on log wage? An economically important effect?
What is the precise interpretation of the estimated coefficient on
UNION? How would the interpretation change if the wage were not
logged?
(d) Precisely what hypothesis does the F-statistic test? What are the
restricted and unrestricted sums of squared residuals to which it
111
is related, and what are the two OLS regressions to which they
correspond?
(e) Consider an additional regressor, AGE, where AGE = 6 + EDUC
+ EXPER. (The idea is that 6 years of early childhood, followed by
EDUC years of education, followed by EXPER years of work experience should, under certain assumptions, sum to a persons age.)
Discuss the likely effects, if any, of adding AGE to the regression.
(f) The log wage may of course not be linear in EDUC and EXPER.
How would you assess the possibility of quadratic nonlinear effects
using t-tests? An F-test? The Schwarz criterion (SIC)? R2 ?
(g) Suppose you find that the log wage relationship is indeed non-linear
but still very simple, with only EXPER2 entering in addition to the
E(LWAGE | X)
in the expanded model?
EXPER
E(LWAGE | X)
in the original model of Figure
EXPER
variables in Figure 7.5. What is

How does it compare to
7.5? What are the economic interpretations of the two derivatives?

(X refers to the full set of included right-hand-side variables in a
regression.)
(h) Return to the original model of Figure 7.5. How would you assess
the overall adequacy of the fitted model using the Durbin-Watson
statistic? The standard error of the regression? The model residuals? Which is likely to be most useful/informative?
(i) Consider estimating the model not by OLS or NLS, but rather by
quantile regression (QR). What loss function is optimized in calculating the QR estimate? (Give a formula and a graph.) What is the
formula (if any) for the QR estimator? How is the least absolute
deviations (LAD) estimator related to the QR estimator? Under
the FIC, are the OLS and LAD estimates likely very close? Why or
why not?
112
Figure 7.5: Regression Output
(j) Discuss whether and how you would incorporate trend and seasonality by using a linear time trend variable and a set of seasonal dummy
variables.
7.8
Notes
Chapter 8
Heteroskedasticity and Clustering in
Cross Sections
Generalized Least Squares (GLS)
Consider the FIC except that we now let:
N (0, )
The old case is = 2 I, but things are very different when 6= 2 I:
OLS parameter estimates consistent but inefficient
(no longer MVUE or BLUE)
OLS standard errors are biased and inconsistent. Hence t ratios do not
have the t distribution in finite samples and do not have the N (0, 1)
distribution asymptotically
The GLS estimator is:
GLS = (X 0 1 X)1 X 0 1 y
Under the remaining full ideal conditions it is consistent, normally
distributed with covariance matrix (X 0 1 X)1 , and MVUE:

GLS N , (X 0 1 X)1 .
113
114CHAPTER 8. HETEROSKEDASTICITY AND CLUSTERING IN CROSS SECTIONS
Heteroskedasticity in Cross-Section Regression

Homoskedasticity: variance of i is constant across i
Heteroskedasticity: variance of i is not constant across i
Relevant cross-sectional heteroskedasticity situation
(on which we focus for now):
i independent across i but not identically distributed across i
12 0 . . .
0 22 . . . 0
=
... ... . . . ...
2
0 0 . . . N
Can arise for many reasons
Engel curve (e.g., food expenditure vs. income) is classic example
Consequences
OLS inefficient (no longer MVUE or BLUE),
in finite samples and asymptotically
Standard errors biased and inconsistent.
Hence t ratios do not have the t distribution in finite samples
and do not have the N (0, 1) distribution asymptotically
Detection
Graphical heteroskedasticity diagnostics
Formal heteroskedasticity tests
115
Graphical Diagnostics
Graph e2i against xi , for various regressors
Problem: Purely pairwise
Recall Our Final Wage Regression
Squared Residual vs. EDUC
The Breusch-Godfrey-Pagan Test (BGP)
Estimate the OLS regression, and obtain the squared residuals
Regress the squared residuals on all regressors
To test the null hypothesis of no relationship, examine N R2 from this
regression. In large samples N R2 2 under the null.
BGP Test
Whites Test
Estimate the OLS regression, and obtain the squared residuals
Regress the squared residuals on all regressors, squared regressors, and
pairwise regressor cross products
To test the null hypothesis of no relationship, examine N R2 from this
regression. In large samples N R2 2 under the null.
(Whites test is a natural and flexible
generalization of the Breusch-Godfrey-Pagan test)
White Test
GLS for Heteroskedasticity
Weighted least squares (WLS)
Take a stand on the DGP. Get consistent standard errors and efficient
parameter estimates.
117
(Infeasible) Weighted Least Squares

DGP:
yi = x0i + i
i idN (0, i2 )
Weight the data (yi , xi ) by 1/i :

yi
x0 i
= i +
i
i
i
The DGP is now:
yi = xi 0 + i
i iidN (0, 1)
OLS is MVUE!
Problem: We dont know i2
Remark on Weighted Least Squares
Weighting the data by 1/i is the same as
weighting the residuals by 1/i2 :
min
2
N
X
yi x 0
i
i=1
N
X
1
2
= min
(yi x0i )
2
i=1 i
Feasible Weighted Least Squares

Intuition: Replace the unknown i2 values with estimates
Some good ideas:
Use wi = 1/eb2i , where eb2i are from the BGP test regression
Use wi = 1/eb2i , where eb2i are from the White test regression
What about WLS directly using wi = 1/e2i ?
Not such a good idea
119
e2i is too noisy; wed like to use not e2i but rather E(e2i |xi ). So we use an
estimate of E(e2i |xi ), namely eb2i from e2 X
Regression Weighted by Fit From White Test Regression
A Different Approach
(Advanced but Very Important)
Whites Heteroskedasticity-Consistent Standard Errors
Perhaps surprisingly, we make direct use of e2i
Dont take a stand on the DGP
Give up on efficient parameter estimates, but get consistent s.e.s.
Using advanced methods, one can obtain consistent
using only e2
s.e.s (if not an efficient )
i
Standard errors are rendered consistent.

remains unchanged at its OLS value. (Is that a problem?)
Robustness to heteroskedasticity of unknown form
Regression with Whites Heteroskedasticity-Consistent Standard Errors
8.1
Covariance Matrix Clustering
8.2
1. (Robustness iteration)
Do a second-stage WLS with weights 1/|et |, or something similar. This
is not a heteroskedasticity correction, but a purely mechanical strategy
to downweight outliers.
2. (Vocabulary)
All these have the same meaning: White standard errors, White-washed
standard errors, heteroskedasticity-robust standard errors, heteroskedasticityconsistent standard errors, and robust standard errors.
8.3
Notes
Part III
Time Series
121
Chapter 9
Indicator Variables in Time Series:
Trend and Seasonality
The time series that we want to model vary over time, and we often mentally attribute that variation to unobserved underlying components related
to trend and seasonality.
9.1
Linear Trend
Trend involves slow, long-run, evolution in the variables that we want to

model and forecast. In business, finance, and economics, for example, trend
is produced by slowly evolving preferences, technologies, institutions, and
demographics. Well focus here on models of deterministic trend, in which
the trend evolves in a perfectly predictable way. Deterministic trend models
are tremendously useful in practice.1
Linear trend is a simple linear function of time,
T rendt = 1 + 2 T IM Et .
The indicator variable T IM E is constructed artificially and is called a time
trend or time dummy. T IM E equals 1 in the first period of the sample,
1
Later well broaden our discussion to allow for stochastic trend.
123
124CHAPTER 9. INDICATOR VARIABLES IN TIME SERIES: TREND AND SEASONALITY
Figure 9.1: Various Linear Trends
2 in the second period, and so on. Thus, for a sample of size T , T IM E =

(1, 2, 3, ..., T 1, T ). Put differently, T IM Et = t, so that the TIME variable
simply indicates the time. 1 is the intercept; its the value of the trend at
time t=0. 2 is the slope; its positive if the trend is increasing and negative
if the trend is decreasing. The larger the absolute value of 1 , the steeper
the trends slope. In Figure 9.1, for example, we show two linear trends,
one increasing and one decreasing. The increasing trend has an intercept
of 1 = 50 and a slope of 2 = .8, whereas the decreasing trend has an
intercept of 1 = 10 and a gentler absolute slope of 2 = .25.
In business, finance, and economics, linear trends are typically increasing,
corresponding to growth, but such need not be the case. In recent decades,
for example, male labor force participation rates have been falling, as have
the times between trades on stock exchanges. In other cases, such as records
(e.g., world records in the marathon), trends are decreasing by definition.
Estimation of a linear trend model (for a series y, say) is easy. First we
need to create and store on the computer the variable T IM E. Fortunately we
dont have to type the T IM E values (1, 2, 3, 4, ...) in by hand; in most good
software environments, a command exists to create the trend automatically.
9.2. SEASONALITY
125
Then we simply run the least squares regression y c, T IM E.
9.2
Seasonality
In the last section we focused on the trends; now well focus on seasonality.
A seasonal pattern is one that repeats itself every year.2 The annual repetition can be exact, in which case we speak of deterministic seasonality, or
approximate, in which case we speak of stochastic seasonality. Here we
focus exclusively on deterministic seasonality models.
Seasonality arises from links of technologies, preferences and institutions
to the calendar. The weather (e.g., daily high temperature) is a trivial but
very important seasonal series, as its always hotter in the summer than in
the winter. Any technology that involves the weather, such as production of
agricultural commodities, is likely to be seasonal as well.
Preferences may also be linked to the calendar. Consider, for example,
gasoline sales. People want to do more vacation travel in the summer, which
tends to increase both the price and quantity of summertime gasoline sales,
both of which feed into higher current-dollar sales.
Finally, social institutions that are linked to the calendar, such as holidays,
are responsible for seasonal variation in a variety of series. In Western countries, for example, sales of retail goods skyrocket every December, Christmas
season. In contrast, sales of durable goods fall in December, as Christmas
purchases tend to be nondurables. (You dont buy someone a refrigerator for
Christmas.)
You might imagine that, although certain series are seasonal for the reasons described above, seasonality is nevertheless uncommon. On the contrary, and perhaps surprisingly, seasonality is pervasive in business and economics. Many industrialized economies, for example, expand briskly every
2
Note therefore that seasonality is impossible, and therefore not an issue, in data recorded once per year,
or less often than once per year.
fourth quarter and contract every first quarter.

9.2.1
Seasonal Dummies
A key technique for modeling seasonality is regression on seasonal dummies. Let s be the number of seasons in a year. Normally wed think of four
seasons in a year, but that notion is too restrictive for our purposes. Instead,
think of s as the number of observations on a series in each year. Thus s = 4
if we have quarterly data, s = 12 if we have monthly data, s = 52 if we have
weekly data, and so forth.
The pure seasonal dummy model is
Seasonalt =
s
X
i SEASit
i=1
(
where SEASit =
1 if observation t falls in season i

0 otherwise
The SEASit variables are called seasonal dummy variables. They simply
indicate which season were in.
Operationalizing the model is simple. Suppose, for example, that we have
quarterly data, so that s = 4. Then we create four variables3 :
SEAS1 = (1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, ..., 0)0
SEAS2 = (0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, ..., 0)0
SEAS3 = (0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, ..., 0)0
SEAS4 = (0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, ..., 1)0 .
SEAS1 indicates whether were in the first quarter (its 1 in the first quarter
and zero otherwise), SEAS2 indicates whether were in the second quarter
(its 1 in the second quarter and zero otherwise), and so on. At any given
time, we can be in only one of the four quarters, so one seasonal dummy is
1, and all others are zero.
3
For illustrative purposes, assume that the data sample begins in Q1 and ends in Q4.
9.2. SEASONALITY
127
To estimate the model for a series y, we simply run the least squares
regression,
y SEAS1 , ..., SEASs .
Effectively, were just regressing on an intercept, but we allow for a different
intercept in each season. Those different intercepts (that is i s) are called the
seasonal factors; they summarize the seasonal pattern over the year, and we
often may want to examine them and plot them. In the absence of seasonality,
those intercepts are all the same, so we can drop all the seasonal dummies
and instead simply include an intercept in the usual way.
In time-series contexts its often most natural to include a full set of seasonal dummies, without an intercept. But of course we could instead include
any s 1 seasonal dummies and an intercept. Then the constant term is the
intercept for the omitted season, and the coefficients on the seasonal dummies give the seasonal increase or decrease relative to the omitted season. In
no case, however, should we include s seasonal dummies and an intercept.
Including an intercept is equivalent to including a variable in the regression
whose value is always one, but note that the full set of s seasonal dummies
sums to a variable whose value is always one, so it is completely redundant.
Trend may be included as well. For example, we can account for seasonality and linear trend by running4
y T IM E, SEAS1 , ..., SEASs .
In fact, you can think of what were doing in this section as a generalization
of what we did in the last, in which we focused exclusively on trend. We still
want to account for trend, if its present, but we want to expand the model
so that we can account for seasonality as well.
4
Note well that we drop the intercept! (Why?)
9.2.2
More General Calendar Effects
The idea of seasonality may be extended to allow for more general calendar
effects. Standard seasonality is just one type of calendar effect. Two
additional important calendar effects are holiday variation and tradingday variation.
Holiday variation refers to the fact that some holidays dates change over
time. That is, although they arrive at approximately the same time each year,
the exact dates differ. Easter is a common example. Because the behavior
of many series, such as sales, shipments, inventories, hours worked, and so
on, depends in part on the timing of such holidays, we may want to keep
track of them in our forecasting models. As with seasonality, holiday effects
may be handled with dummy variables. In a monthly model, for example,
in addition to a full set of seasonal dummies, we might include an Easter
dummy, which is 1 if the month contains Easter and 0 otherwise.
Trading-day variation refers to the fact that different months contain different numbers of trading days or business days, which is an important consideration when modeling and forecasting certain series. For example, in a
monthly forecasting model of volume traded on the London Stock Exchange,
in addition to a full set of seasonal dummies, we might include a trading day
variable, whose value each month is the number of trading days that month.
More generally, you can model any type of calendar effect that may arise,
by constructing and including one or more appropriate dummy variables.
9.3
Trend and Seasonality in Liquor Sales
Well illustrate trend and seasonal modeling with an application to liquor

sales. The data are measured monthly.
We show the time series of liquor sales in Figure 9.2, which displays clear
trend (sales are increasing) and seasonality (sales skyrocket during the Christ-
9.3. TREND AND SEASONALITY IN LIQUOR SALES
129
Figure 9.2: Liquor Sales
mas season, among other things).

We show log liquor sales in Figure 9.3 ; we take logs to stabilize the
variance, which grows over time.5 Log liquor sales has a more stable variance,
and its the series for which well build models.6
Linear trend estimation results appear in Table 9.4. The trend is increasing
and highly significant. The adjusted R2 is 84%, reflecting the fact that trend
is responsible for a large part of the variation in liquor sales.
The residual plot (Figure 9.5) suggests, however, that linear trend is inadequate. Instead, the trend in log liquor sales appears nonlinear, and the
neglected nonlinearity gets dumped in the residual. (Well introduce nonlinear trend later.) The residual plot also reveals obvious residual seasonality.
The Durbin-Watson statistic missed it, evidently because its not designed to
have power against seasonal dynamics.7
In Figure 9.6 we show estimation results for a model with linear trend
5
The nature of the logarithmic transformation is such that it compresses an increasing variance. Make
a graph of log(x) as a function of x, and youll see why.
6
From this point onward, for brevity well simply refer to liquor sales, but remember that weve taken
logs.
7
Recall that the Durbin-Watson test is designed to detect simple AR(1) dynamics. It also has the ability
to detect other sorts of dynamics, but evidently not those relevant to the present application, which are very
different from a simple AR(1).
Figure 9.3: Log Liquor Sales
and seasonal dummies. (Note that we dropped the intercept!) The seasonal
dummies are highly significant, and in many cases significantly different from
each other. R2 is higher.
In Figure 9.7 we show the corresponding residual plot. The model now
picks up much of the seasonality, as reflected in the seasonal fitted series and
the non-seasonal residuals.
In Figure 9.8 we plot the estimated seasonal pattern, which peaks during
the winter holidays.
All of these results are crude approximations, because the linear trend
is clearly inadequate. We will subsequently allow for more sophisticated
(nonlinear) trends.
9.4
1. (Mechanics of trend estimation and detrending)

Obtain from the web a quarterly time series of U.S. real GDP in levels,
spanning the last forty years, and ending in Q4.
a. Produce a time series plot and discuss.
131
Dependent Variable: LSALES

Method: Least Squares
Date: 08/08/13 Time: 08:53
Sample: 1987M01 2014M12
Included observations: 336
Variable
Coefficient
Std. Error
t-Statistic
Prob.
C
TIME
6.454290
0.003809
0.017468
8.98E-05
369.4834
42.39935
0.0000
0.0000
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
F-statistic
Prob(F-statistic)
0.843318
0.842849
0.159743
8.523001
140.5262
1797.705
0.000000
Mean dependent var

S.D. dependent var
Akaike info criterion
Schwarz criterion
Hannan-Quinn criter.
Durbin-Watson stat
7.096188
0.402962
-0.824561
-0.801840
-0.815504
1.078573
Figure 9.4: Linear Trend Estimation
b. Fit a linear trend. Discuss both the estimation results and the residual
plot.
c. Is there any evidence of seasonality in the residuals? Why or why
not?
d. The residuals from your fitted model are effectively a linearly detrended version of your original series. Why? Discuss.
2. (Using model selection criteria to select a trend model)
You are tracking and forecasting the earnings of a new company developing and applying proprietary nano-technology. The earnings are trending upward. You fit linear, quadratic, and exponential trend models,
yielding sums of squared residuals of 4352, 2791, and 2749, respectively.
Which trend model would you select, and why?
3. (Seasonal adjustment)
Just as we sometimes want to remove the trend from a series, sometimes
Figure 9.5: Residual Plot, Linear Trend Estimation
we want to seasonally adjust a series before modeling it. Seasonal

adjustment may be done using a variety of methods.
a. Discuss in detail how youd use a linear trend plus seasonal dummies
model to seasonally adjust a series.
b. Seasonally adjust the log liquor sales data using a linear trend plus
seasonal dummy model. Discuss the patterns present and absent from
the seasonally adjusted series.
c. Search the Web (or the library) for information on the latest U.S.
Census Bureau seasonal adjustment procedure, and report what you
learned.
4. (Handling sophisticated calendar effects)
Describe how you would construct a purely seasonal model for the following monthly series. In particular, what dummy variable(s) would you
use to capture the relevant effects?
133
Figure 9.6: Estimation Results, Linear Trend with Seasonal Dummies
a. A sporting goods store suspects that detrended monthly sales are

roughly the same for each month in a given three-month season. For
example, sales are similar in the winter months of January, February
and March, in the spring months of April, May and June, and so on.
b. A campus bookstore suspects that detrended sales are roughly the
same for all first, all second, all third, and all fourth months of
each trimester. For example, sales are similar in January, May, and
September, the first months of the first, second, and third trimesters,
respectively.
c. (Trading-day effects) A financial-markets trader suspects that de-
Figure 9.7: Residual Plot, Linear Trend with Seasonal Dummies
trended trading volume depends on the number of trading days in

the month, which differs across months.
d. (Time-varying holiday effects) A candy manufacturer suspects that
detrended candy sales tend to rise at Easter.
5. (Testing for seasonality)
Using the log liquor sales data:
a. As in the chapter, construct and estimate a model with a full set of
seasonal dummies.
b. Test the hypothesis of no seasonal variation. Discuss.
c. Test for the equality of the January through April seasonal factors.
Discuss.
d. Test for equality of the May through November seasonal factors. Discuss.
9.5. NOTES
135
Figure 9.8: Seasonal Pattern
e. Estimate a suitable pruned model with fewer than twelve seasonal

dummies that nevertheless adequately captures the seasonal pattern.
9.5
Notes
Nerlove et al. (1979) and Harvey (1991) discuss a variety of models of trend
and seasonality.
The two most common and important official seasonal adjustment methods are X-12-ARIMA from the U.S. Census Bureau, and TRAMO-SEATS
from the Bank of Spain.
Chapter 10
Non-Linearity and Structural Change
in Time Series
In time series a central issue is nonlinear trend. Here we focus on it.
10.1
Exponential Trend
The insight that exponential growth is non-linear in levels but linear in logarithms takes us to the idea of exponential trend, or log-linear trend,
which is very common in business, finance and economics.1
Exponential trend is common because economic variables often display
roughly constant real growth rates (e.g., two percent per year). If trend is
characterized by constant growth at rate 2 , then we can write
T rendt = 1 e2 T IM Et .
The trend is a non-linear (exponential) function of time in levels, but in
logarithms we have
ln(T rendt ) = ln(1 ) + 2 T IM Et .
Thus, ln(T rendt ) is a linear function of time.
1
Throughout this book, logarithms are natural (base e) logarithms.
137
(10.1)
138CHAPTER 10. NON-LINEARITY AND STRUCTURAL CHANGE IN TIME SERIES
Figure 10.1: Various Exponential Trends
In Figure 10.1 we show the variety of exponential trend shapes that can
be obtained depending on the parameters. Depending on the signs and sizes
of the parameter values, exponential trend can achieve a variety of patterns,
increasing or decreasing at increasing or decreasing rates.
Although the exponential trend model is non-linear, we can estimate it by
simple least squares regression, because it is linear in logs. We simply run the
least squares regression, ln y c, T IM E. Note that because the intercept
in equation (10.1) is not 1 , but rather ln(1 ), we need to exponentiate the
estimated intercept to get an estimate of 1 . Similarly, the fitted values from
this regression are the fitted values of lny, so they must be exponentiated to
get the fitted values of y. This is necessary, for example, for appropriately
comparing fitted values or residuals (or statistics based on residuals, like AIC
and SIC) from estimated exponential trend models to those from other trend
models.
10.2. QUADRATIC TREND
139
Its important to note that, although the same sorts of qualitative trend
shapes can be achieved with quadratic and exponential trend, there are subtle differences between them. The non-linear trends in some series are well
approximated by quadratic trend, while the trends in other series are better
approximated by exponential trend. Ultimately its an empirical matter as
to which is best in any particular application.
10.2
Quadratic Trend
Sometimes trend appears non-linear, or curved, as for example when a variable increases at an increasing or decreasing rate. Ultimately, we dont require
that trends be linear, only that they be smooth.
We can allow for gentle curvature by including not only T IM E, but also
T IM E 2 ,
T rendt = 1 + 2 T IM Et + 3 T IM Et2 .
This is called quadratic trend, because the trend is a quadratic function of
T IM E.2 Linear trend emerges as a special (and potentially restrictive) case
when 3 = 0.
A variety of different non-linear quadratic trend shapes are possible, depending on the signs and sizes of the coefficients; we show several in Figure
10.2. In particular, if 2 > 0 and 3 > 0 as in the upper-left panel, the trend is
monotonically, but non-linearly, increasing, Conversely, if 2 < 0 and 3 < 0,
the trend is monotonically decreasing. If 2 < 0 and 3 > 0 the trend has a
U shape, and if 2 > 0 and 3 < 0 the trend has an inverted U shape. Keep
in mind that quadratic trends are used to provide local approximations; one
rarely has a U-shaped trend, for example. Instead, all of the data may lie
on one or the other side of the U.
Estimating quadratic trend models is no harder than estimating linear
2
Higher-order polynomial trends are sometimes entertained, but its important to use low-order polynomials to maintain smoothness.
Figure 10.2: Various Quadratic Trends
trend models. We first create T IM E and its square; call it T IM E2, where
T IM E2t = T IM Et2 . Because T IM E = (1, 2, ..., T ), T IM E2 = (1, 4, ..., T 2 ).
Then we simply run the least squares regression y c, T IM E, T IM E2.
Note in particular that although the quadratic is a non-linear function, it is
linear in the variables T IM E and T IM E2.
10.3
More on Non-Linear Trend
The trend regression technique is one way to estimate trend. Two additional
ways involve model-free smoothing techniques. They are moving-average
smoothers and Hodrick-Prescott smoothers. We briefly introduce them here.
10.3. MORE ON NON-LINEAR TREND
10.3.1
141
Moving-Average Trend and De-Trending
Well focus on three: two-sided moving averages, one-sided moving averages,

and one-sided weighted moving averages.
Denote the original data by {yt }Tt=1 and the smoothed data by {st }Tt=1 .
Then the two-sided moving average is
st = (2m + 1)
m
X
yti ,
i=m
the one-sided moving average is

st = (m + 1)
m
X
yti ,
i=0
and the one-sided weighted moving average is

st =
m
X
wi yti ,
i=0
where the wi are weights and m is an integer chosen by the user. The standard one-sided moving average corresponds to a one-sided weighted moving
average with all weights equal to (m + 1)1 .
a. For each of the smoothing techniques, discuss the role played by m. What
happens as m gets very large? Very small? In what sense does m play a
role similar to p, the order of a polynomial trend?
b. If the original data runs from time 1 to time T , over what range can
smoothed values be produced using each of the three smoothing methods?
What are the implications for real-time smoothing or on-line
smoothing versus ex post smoothing or off-line smoothing?
10.3.2
Hodrick-Prescott Trend and De-Trending
A final approach to trend fitting and de-trending is known as HodrickPrescott filtering. The HP trend solves:
min
T
{st }t=1
T
X
(yt st ) +
t=1
T 1
X
((st+1 st ) (st st1 ))2
t=2
a. is often called the penalty parameter. What does govern?

b. What happens as 0?
c. What happens as ?
d. People routinely use bigger for higher-frequency data. Why? (Common
values are = 100, 1600 and 14,400 for annual, quarterly, and monthly
data, respectively.)
10.4
Structural Change
Recall the full ideal conditions. Here we deal with violation of the assumption
that the coefficients, , are fixed.
The cross-section dummy variables that we already studied effectively allow for structural change in the cross section (heterogeneity across groups).
But structural change is of special relevance in time series. It can be gradual (Lucas critique, learning, evolution of tastes, ...) or abrupt (e.g., new
legislation).
Structural change is related to nonlinearity, because abrupt structural
change is actually a type of nonlinearity. Structural change is also related
to outliers, because outliers can sometimes be viewed as a kind of structural
change a quick intercept break and return.
For notational simplicity we consider the case of simple regression throughout, but the ideas extend immediately to multiple regression.
10.4. STRUCTURAL CHANGE
10.4.1
143
Gradual Parameter Evolution
In many cases, parameters may evolve gradually rather than breaking abruptly.
Suppose, for example, that
yt = 1t + 2t xt + t
where
1t = 1 + 2 T IM Et
2t = 1 + 2 T IM Et .
Then we have:
yt = (1 + 2 T IM Et ) + (1 + 2 T IM Et )xt + t .
We simply run:
yt c, , T IM Et , xt , T IM Et xt .
This is yet another important use of dummies. The regression can be
used both to test for structural change (F test of 2 = 2 = 0), and to
accommodate it if present.
10.4.2
Abrupt Parameter Breaks
Exogenously-Specified Breaks
Suppose that we dont know whether a break occurred, but we know that if
it did occur, it occurred at time T .
A Dummy-Variable Approach That is, we entertain the possibility that
(
11 + 21 xt + t , t = 1, ..., T
yt =
12 + 22 xt + t , t = T + 1, ..., T
Let
0, t = 1, ..., T
Dt =
1, t = T + 1, ...T
Then we can write the model as:

yt = (11 + (12 11 )Dt ) + (21 + (22 21 )Dt )xt + t
We simply run:
yt c, Dt , xt , Dt xt
The regression can be used both to test for structural change, and to accommodate it if present. It represents yet another use of dummies. The no-break
null corresponds to the joint hypothesis of zero coefficients on Dt and Dt xt ,
for which an F test is appropriate.
The Chow Test The dummy-variable setup and associated F test above
is actually just a laborious way of calculating the so-called Chow breakpoint
test statistic,
Chow =
SSRres SSR)/K
,
SSR/(T 2K)
where SSRres is from the regression using sample t = 1, ..., T and SSR =
SSR1 + SSR2 , where SSR1 is from the regression using sample t = 1, ..., T
and SSR2 is from the regression using sample t = T + 1, ...T . Under the
FIC, Chow is distributed F , with K and T 2K degrees of freedom.
The Chow test with Endogenous Break Selection
Thus far we have (unrealistically) assumed that the potential break date is
known. In practice, potential break dates are often unknown and are identified by peeking at the data. We can capture this phenomenon in stylized
fashion by imagining splitting the sample sequentially at each possible break
date, and picking the split at which the Chow breakpoint test statistic is
maximized. Implicitly, thats what people often do in practice, even if they
10.5. DUMMY VARIABLES AND OMITTED VARIABLES, AGAIN AND AGAIN
145
dont always realize or admit it.

The distribution of such a test statistic is not F , as for the traditional
Chow breakpoint test statistic. Rather, the distribution is that of the maximum of many draws from an F , which will be pushed far to the right of the
distribution of a single F draw.
The test statistic is
M axChow = max Chow( ),
1 2
where denotes sample fraction (typically we take 1 = .15 and 2 = .85).

The distribution of M axChow has been tabulated.
10.5
Dummy Variables and Omitted Variables, Again

and Again
10.5.1
Dummy Variables
Notice that dummy (indicator) variables have arisen repeatedly in our discussions. We used 0-1 dummies to handle group heterogeneity in cross-sections.
We used time dummies to indicate the date in time series. We used 0-1
seasonal dummies to indicate the season in time series.
Now, in this chapter, we used both (1) time dummies to allow for gradual
parameter evolution, and (2) 0-1 dummies to indicate a sharp break date, in
time series.
10.5.2
Omitted Variables
Notice that omitted variables have also arisen repeatedly in our discussions.
1. If there are neglected group effects in cross-section regression, we fix the
problem (of omitted group dummies) by including the requisite group
dummies.
2. If there is neglected trend or seasonality in time-series regression, we fix

the problem (of omitted trend or seasonal dummies) by including the
requisite trend or seasonal dummies.
3. If there is neglected non-linearity, we fix the problem (effectively one
of omitted Taylor series terms) by including the requisite Taylor series
terms.
4. If there is neglected structural change in time-series regression, we fix the
problem (effectively one of omitted parameter trend dummies or break
dummies) by including the requisite trend dummies or break dummies.
You can think of the basic uber-strategy as If some systematic feature of
the DGP is missing from the model, then include it. That is, if something
is missing, then model whats missing, and then the new uber-model wont
have anything missing, and all will be well (i.e., the FIC will be satisfied).
This is an important recognition. In subsequent chapters, for example, well
study violations of the FIC known as heteroskedasticity (Chapters 8 and 13)
and serial correlation (Chapter 12). In each case the problem amounts to a
feature of the DGP neglected by the initially-fitted model, and we address
the problem by incorporating the neglected feature into the model.
10.6
Non-Linearity in Liquor Sales Trend
We already fit a non-linear (exponential) trend to liquor sales, when we fit a

linear trend to log liquor sales. But it still didnt fit so well.
We now examine quadratic trend model (again in logs). The log-quadratic
trend estimation results appear in Figure 10.3. Both T IM E and T IM E2
are highly significant. The adjusted R2 for the log-quadratic trend model
is 89%, higher than for the the log-linear trend model. As with the loglinear trend model, the Durbin-Watson statistic provides no evidence against
10.6. NON-LINEARITY IN LIQUOR SALES TREND
147

Date: 08/08/13 Time: 08:53
Sample: 1987M01 2014M12
Variable
Coefficient
Std. Error
t-Statistic
Prob.
C
TIME
TIME2
6.231269
0.007768
-1.17E-05
0.020653
0.000283
8.13E-07
301.7187
27.44987
-14.44511
0.0000
0.0000
0.0000
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
F-statistic
Prob(F-statistic)
0.903676
0.903097
0.125439
5.239733
222.2579
1562.036
0.000000
Mean dependent var

S.D. dependent var
Schwarz criterion
Durbin-Watson stat
7.096188
0.402962
-1.305106
-1.271025
-1.291521
1.754412
Figure 10.3: Log-Quadratic Trend Estimation
the hypothesis that the regression disturbance is white noise. The residual
plot (Figure 10.4) shows that the fitted quadratic trend appears adequate,
and that it increases at a decreasing rate. The residual plot also continues
to indicate obvious residual seasonality. (Why does the Durbin-Watson not
detect it?)
In Figure 10.5 we show the results of regression on quadratic trend and
a full set of seasonal dummies. The trend remains highly significant, and
the coefficients on the seasonal dummies vary significantly. The adjusted R2
rises to 99%. The Durbin-Watson statistic, moreover, has greater ability to
detect residual serial correlation now that we have accounted for seasonality, and it sounds a loud alarm. The residual plot of Figure 10.6 shows no
seasonality, as the model now accounts for seasonality, but it confirms the
Durbin-Watson statistics warning of serial correlation. The residuals appear
highly persistent.
There remains one model as yet unexplored, exponential trend fit to
LSALES. We do it by N LS (why?) and present the results in Figure ***.
Figure 10.4: Residual Plot, Log-Quadratic Trend Estimation
Among the linear, quadratic and exponential trend models for LSALES,
both SIC and AIC clearly favor the quadratic.
Exogenously-specified break in log-linear trend model
Endogenously-selected break in log-linear trend model
SIC for best broken log-linear trend model vs. log-quadratic trend model
10.7
1. Specifying and testing nonlinear trend models.

In 1965, Intel co-founder Gordon Moore predicted that the number of
transistors that one could place on a square-inch integrated circuit would
double every twelve months.
a. What sort of trend is this?
b. Given a monthly series containing the number of transistors per square
inch for the latest integrated circuit, how would you test Moores prediction? How would you test the currently accepted form of Moores
149

Date: 08/08/13 Time: 08:53
Sample: 1987M01 2014M12
Variable
Coefficient
Std. Error
t-Statistic
Prob.
TIME
TIME2
D1
D2
D3
D4
D5
D6
D7
D8
D9
D10
D11
D12
0.007739
-1.18E-05
6.138362
6.081424
6.168571
6.169584
6.238568
6.243596
6.287566
6.259257
6.199399
6.221507
6.253515
6.575648
0.000104
2.98E-07
0.011207
0.011218
0.011229
0.011240
0.011251
0.011261
0.011271
0.011281
0.011290
0.011300
0.011309
0.011317
74.49828
-39.36756
547.7315
542.1044
549.3318
548.8944
554.5117
554.4513
557.8584
554.8647
549.0938
550.5987
552.9885
581.0220
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
Durbin-Watson stat
0.987452
0.986946
0.046041
0.682555
564.6725
0.581383
Mean dependent var

S.D. dependent var
Schwarz criterion
7.096188
0.402962
-3.277812
-3.118766
-3.214412
Figure 10.5: Liquor Sales Log-Quadratic Trend Estimation with Seasonal Dummies
Law, namely that the number of transistors actually doubles every

eighteen months?
2. (Properties of polynomial trends)
Consider a sixth-order deterministic polynomial trend:
Tt = 1 + 2 T IM Et + 3 T IM Et2 + ... + 7 T IM Et6 .
a. How many local maxima or minima may such a trend display?
b. Plot the trend for various values of the parameters to reveal some of
the different possible trend shapes.
c. Is this an attractive trend model in general? Why or why not?
Figure 10.6: Residual Plot, Liquor Sales Log-Quadratic Trend Estimation With Seasonal
Dummies
d. Fit the sixth-order polynomial trend model to a trending series that

interests you, and discuss your results.
3. (Selecting non-linear trend models)
Using AIC and SIC, perform a detailed comparison of polynomial vs.
exponential trend in LSALES. Do you agree with our use of quadratic
trend in the text?
4. (Difficulties with non-linear optimization)
Non-linear optimization can be a tricky business, fraught with problems.
Some problems are generic. Its relatively easy to find a local optimum,
for example, but much harder to be confident that the local optimum
is global. Simple checks such as trying a variety of startup values and
checking the optimum to which convergence occurs are used routinely,
but the problem nevertheless remains. Other problems may be software
specific. For example, some software may use highly accurate analytic
derivatives whereas other software uses approximate numerical deriva-
151
tives. Even the same software package may change algorithms or details
of implementation across versions, leading to different results.
5. (Direct estimation of exponential trend in levels)
We can estimate an exponential trend in two ways. First, as we have
emphasized, we can take logs and then use OLS to fit a linear trend.
Alternatively we can use NLS, proceeding directly from the exponential
representation and letting the computer find
(1 , 2 ) = argmin1 ,2
T
X
yt 1 e2 T IM Et
2
t=1
a. The NLS approach is more tedious? Why?

b. The NLS approach is less thoroughly numerically trustworthy? Why?
c. Nevertheless the NLS approach can be very useful? Why? (Hint:
Consider comparing SIC values for quadratic vs. exponential trend.)
6. (Logistic trend)
In the main text we introduced the logistic functional form. A key
example is logistic trend, which is
T rendt =
1
,
a + brT IM Et
with 0<r<1.
a. Graph the trend shape for various combinations of a and b values.
When might such a trend shape be useful?
b. Can you think of other specialized situations in which other specialized trend shapes might be useful? Produce mathematical formulas
for the additional specialized trend shapes you suggest.
7. (Modeling Liquor Sales Trend and Seasonality)
Consider the liquor sales data. Never include an intercept. Discuss all
results in detail.
(a) Fit a linear trend plus seasonal dummy model to log liquor sales
(LSALES), using a full set of seasonal dummies.
(b) Find a best linear trend plus seasonal dummy LSALES model.
That is, consider tightening the seasonal specification to include
fewer than 12 seasonal dummies, and decide whats best.
(c) Keeping the same seasonality specification as in (7b), re-estimate
the model in levels (that is, the LHS variable is now SALES rather
than LSALES) using exponential trend and nonlinear least squares.
Do your coefficient estimates match those from (7b)? Does the SIC
match that from (7b)?
(d) Repeat (7c), again using SALES and again leaving intact your seasonal specification from (7b), but try linear and quadratic trend instead of the exponential trend in (7c). What is your final SALES
model?
(e) Critique your final SALES model from (7d). In what ways is it
likely still deficient? You will of course want to discuss its residual
plot (actual values, fitted values, residuals), as well as any other
diagnostic plots or statistics that you deem relevant.
(f) Take your final estimated SALES model from (7d), and include as
regressors three lags of SALES (i.e., SALESt1 , SALESt2 and
SALESt3 ). What role do the lags of SALES play? Consider this
new model to be your final, final SALES model, and repeat (7e).
8. Regime Switching I: Observed-Regime Threshold Model
10.8. NOTES
153
yt =
(u)
c(u) + (u) yt1 + t , (u) < ytd

(m)
c(m) + (m) yt1 + t , (l) < ytd < (u)
(l)
c(l) + (l) yt1 + t , (l) > ytd
9. Regime Switching II: Markov-Switching Model

Regime governed by latent 2-state Markov process:
!
p00
1 p00
M=
1 p11
p11
Switching mean:
1
f (yt |st ) =
exp
2
!
(yt st )2
.
2 2
Switching regression:
f (yt |st ) =
10.8
Notes
1
exp
2
(yt x0t st )2
2 2
!
.
Chapter 11
Serial Correlation in Observed Time
Series
Observed Time Series.
11.1
Characterizing Time-Series Dynamics
Weve already considered models with trend and seasonal components. In this
chapter we consider a crucial third component, cycles. When you think of a
cycle, you probably think of the sort of rigid up-and-down pattern depicted
in Figure ??. Such cycles can sometimes arise, but cyclical fluctuations in
business, finance, economics and government are typically much less rigid.
In fact, when we speak of cycles, we have in mind a much more general,
all-encompassing, notion of cyclicality: any sort of dynamics not captured by
trends or seasonals.
Cycles, according to our broad interpretation, may display the sort of
back-and-forth movement characterized in Figure ??, but they dont have to.
All we require is that there be some dynamics, some persistence, some way in
which the present is linked to the past, and the future to the present. Cycles
are present in most of the series that concern us, and its crucial that we know
how to model and forecast them, because their history conveys information
155
156
CHAPTER 11. SERIAL CORRELATION IN OBSERVED TIME SERIES
regarding their future.

Trend and seasonal dynamics are simple, so we can capture them with
simple models. Cyclical dynamics, however, are more complicated. Because
of the wide variety of cyclical patterns, the sorts of models we need are
substantially more involved. Thus we split the discussion into three parts.
First we develop methods for characterizing cycles, and then we introduce
models of cycles. All of the material is crucial, and its also a bit difficult the
first time around because its unavoidably rather mathematical, so careful,
systematic study is required.
11.1.1
Covariance Stationary Time Series
A realization of a time series is an ordered set,

{..., y2 , y1 , y0 , y1 , y2 , ...}.
Typically the observations are ordered in time hence the name time series
but they dont have to be. We could, for example, examine a spatial series,
such as office space rental rates as we move along a line from a point in
midtown Manhattan to a point in the New York suburbs thirty miles away.
But the most important case, by far, involves observations ordered in time,
so thats what well stress.
In theory, a time series realization begins in the infinite past and continues
into the infinite future. This perspective may seem abstract and of limited
practical applicability, but it will be useful in deriving certain very important
properties of the models well be using shortly. In practice, of course, the data
we observe is just a finite subset of a realization, {y1 , ..., yT }, called a sample
path.
Shortly well be building models for cyclical time series. If the underlying
probabilistic structure of the series were changing over time, wed be doomed
there would be no way to relate the future to the past, because the laws gov-
11.1. CHARACTERIZING TIME-SERIES DYNAMICS
157
erning the future would differ from those governing the past. At a minimum
wed like a series mean and its covariance structure (that is, the covariances
between current and past values) to be stable over time, in which case we
say that the series is covariance stationary. Lets discuss covariance stationarity in greater depth. The first requirement for a series to be covariance
stationary is that the mean of the series be stable over time. The mean of
the series at time t is Eyt = t . If the mean is stable over time, as required
by covariance stationarity, then we can write Eyt = , for all t. Because the
mean is constant over time, theres no need to put a time subscript on it.
The second requirement for a series to be covariance stationary is that
its covariance structure be stable over time. Quantifying stability of the
covariance structure is a bit tricky, but tremendously important, and we do
it using the autocovariance function. The autocovariance at displacement
is just the covariance between yt and yt . It will of course depend on ,
and it may also depend on t, so in general we write
(t, ) = cov(yt , yt ) = E(yt )(yt ).
If the covariance structure is stable over time, as required by covariance
stationarity, then the autocovariances depend only on displacement, , not
on time, t, and we write (t, ) = ( ), for all t.
The autocovariance function is important because it provides a basic summary of cyclical dynamics in a covariance stationary series. By examining
the autocovariance structure of a series, we learn about its dynamic behavior. We graph and examine the autocovariances as a function of . Note that
the autocovariance function is symmetric; that is, ( ) = ( ), for all .
Typically, well consider only non-negative values of . Symmetry reflects the
fact that the autocovariance of a covariance stationary series depends only
on displacement; it doesnt matter whether we go forward or backward. Note
also that (0) = cov(yt , yt ) = var(yt ).
158
There is one more technical requirement of covariance stationarity: we

require that the variance of the series the autocovariance at displacement
0, (0), be finite. It can be shown that no autocovariance can be larger
in absolute value than (0), so if (0) < , then so too are all the other
autocovariances.
It may seem that the requirements for covariance stationarity are quite
stringent, which would bode poorly for our models, almost all of which invoke covariance stationarity in one way or another. It is certainly true that
many economic, business, financial and government series are not covariance
stationary. An upward trend, for example, corresponds to a steadily increasing mean, and seasonality corresponds to means that vary with the season,
both of which are violations of covariance stationarity.
But appearances can be deceptive. Although many series are not covariance stationary, it is frequently possible to work with models that give special
treatment to nonstationary components such as trend and seasonality, so that
the cyclical component thats left over is likely to be covariance stationary.
Well often adopt that strategy. Alternatively, simple transformations often
appear to transform nonstationary series to covariance stationarity. For example, many series that are clearly nonstationary in levels appear covariance
stationary in growth rates.
In addition, note that although covariance stationarity requires means and
covariances to be stable and finite, it places no restrictions on other aspects
of the distribution of the series, such as skewness and kurtosis.1 The upshot
is simple: whether we work directly in levels and include special components
for the nonstationary elements of our models, or we work on transformed
data such as growth rates, the covariance stationarity assumption is not as
unrealistic as it may seem.
Recall that the correlation between two random variables x and y is defined
1
For that reason, covariance stationarity is sometimes called second-order stationarity or weak stationarity.
11.1. CHARACTERIZING TIME-SERIES DYNAMICS
159
by
corr(x, y) =
cov(x, y)
.
x y
That is, the correlation is simply the covariance, normalized, or standardized, by the product of the standard deviations of x and y. Both the
correlation and the covariance are measures of linear association between two
random variables. The correlation is often more informative and easily interpreted, however, because the construction of the correlation coefficient guarantees that corr(x, y) [1, 1], whereas the covariance between the same two
random variables may take any value. The correlation, moreover, does not
depend on the units in which x and y are measured, whereas the covariance
does. Thus, for example, if x and y have a covariance of ten million, theyre
not necessarily very strongly associated, whereas if they have a correlation of
.95, it is unambiguously clear that they are very strongly associated.
In light of the superior interpretability of correlations as compared to
covariances, we often work with the correlation, rather than the covariance,
between yt and yt . That is, we work with the autocorrelation function,
( ), rather than the autocovariance function, ( ). The autocorrelation
function is obtained by dividing the autocovariance function by the variance,
( ) =
( )
, = 0, 1, 2, ....
(0)
The formula for the autocorrelation is just the usual correlation formula,
specialized to the correlation between yt and yt . To see why, note that the
variance of yt is (0), and by covariance stationarity, the variance of y at any
other time yt is also (0). Thus,
cov(yt , yt )
( )
( )
p
p
( ) = p
=p
=
,
(0)
var(yt ) var(yt )
(0) (0)
as claimed. Note that we always have (0) =
(0)
(0)
= 1 , because any series
160
is perfectly correlated with itself. Thus the autocorrelation at displacement

0 isnt of interest; rather, only the autocorrelations beyond displacement 0
inform us about a series dynamic structure.
Finally, the partial autocorrelation function, p( ), is sometimes useful. p( ) is just the coefficient of yt in a population linear regression of
yt on yt1 , ..., yt .2 We call such regressions autoregressions, because the
variable is regressed on lagged values of itself. Its easy to see that the
autocorrelations and partial autocorrelations, although related, differ in an
important way. The autocorrelations are just the simple or regular correlations between yt and yt . The partial autocorrelations, on the other hand,
measure the association between yt and yt after controlling for the effects
of yt1 , ..., yt +1 ; that is, they measure the partial correlation between yt
and yt .
As with the autocorrelations, we often graph the partial autocorrelations
as a function of and examine their qualitative shape, which well do soon.
Like the autocorrelation function, the partial autocorrelation function provides a summary of a series dynamics, but as well see, it does so in a different
way.3
All of the covariance stationary processes that we will study subsequently
have autocorrelation and partial autocorrelation functions that approach
zero, one way or another, as the displacement gets large. In Figure ***
we show an autocorrelation function that displays gradual one-sided damping, and in Figure *** we show a constant autocorrelation function; the latter
could not be the autocorrelation function of a stationary process, whose autocorrelation function must eventually decay. The precise decay patterns
2
To get a feel for what we mean by population regression, imagine that we have an infinite sample
of data at our disposal, so that the parameter estimates in the regression are not contaminated by sampling
variation that is, theyre the true population values. The thought experiment just described is a population
regression.
3
Also in parallel to the autocorrelation function, the partial autocorrelation at displacement 0 is always
one and is therefore uninformative and uninteresting. Thus, when we graph the autocorrelation and partial
autocorrelation functions, well begin at displacement 1 rather than displacement 0.
11.2. WHITE NOISE
161
of autocorrelations and partial autocorrelations of a covariance stationary

series, however, depend on the specifics of the series. In Figure ***, for example, we show an autocorrelation function that displays damped oscillation
the autocorrelations are positive at first, then become negative for a while,
then positive again, and so on, while continuously getting smaller in absolute
value. Finally, in Figure *** we show an autocorrelation function that differs
in the way it approaches zero the autocorrelations drop abruptly to zero
beyond a certain displacement.
11.2
White Noise
In this section well study the population properties of certain important time
series models, or time series processes. Before we estimate time series
models, we need to understand their population properties, assuming that
the postulated model is true. The simplest of all such time series processes
is the fundamental building block from which all others are constructed. In
fact, its so important that we introduce it now. We use y to denote the
observed series of interest. Suppose that
yt = t
t (0, 2 ),
where the shock, t , is uncorrelated over time. We say that t , and hence yt
, is serially uncorrelated. Throughout, unless explicitly stated otherwise,
we assume that 2 < . Such a process, with zero mean, constant variance,
and no serial correlation, is called zero-mean white noise, or simply white
162
noise.4 Sometimes for short we write

t W N (0, 2 )
and hence
yt W N (0, 2 ).
Note that, although t and hence yt are serially uncorrelated, they are
not necessarily serially independent, because they are not necessarily normally distributed.5 If in addition to being serially uncorrelated, y is serially
independent, then we say that y is independent white noise.6 We write
yt iid(0, 2 ),
and we say that y is independently and identically distributed with zero
mean and constant variance. If y is serially uncorrelated and normally
distributed, then it follows that y is also serially independent, and we say
that y is normal white noise, or Gaussian white noise.7 We write
yt iidN (0, 2 ).
We read y is independently and identically distributed as normal, with zero
mean and constant variance, or simply y is Gaussian white noise. In
Figure *** we show a sample path of Gaussian white noise, of length T = 150,
simulated on a computer. There are no patterns of any kind in the series due
to the independence over time.
Youre already familiar with white noise, although you may not realize it.
4
Its called white noise by analogy with white light, which is composed of all colors of the spectrum,
in equal amounts. We can think of white noise as being composed of a wide variety of cycles of differing
periodicities, in equal amounts.
5
Recall that zero correlation implies independence only in the normal case.
6
Another name for independent white noise is strong white noise, in contrast to standard serially
uncorrelated weak white noise.
7
Carl Friedrich Gauss, one of the greatest mathematicians of all time, discovered the normal distribution
some 200 years ago; hence the adjective Gaussian.
11.2. WHITE NOISE
163
Recall that the disturbance in a regression model is typically assumed to be

white noise of one sort or another. Theres a subtle difference here, however.
Regression disturbances are not observable, whereas were working with an
observed series. Later, however, well see how all of our models for observed
series can be used to model unobserved variables such as regression disturbances. Lets characterize the dynamic stochastic structure of white noise,
yt W N (0, 2 ). By construction the unconditional mean of y is E(yt ) = 0,
and the unconditional variance of y is var(yt ) = 2 .
Note that the unconditional mean and variance are constant. In fact, the
unconditional mean and variance must be constant for any covariance stationary process. The reason is that constancy of the unconditional mean was
our first explicit requirement of covariance stationarity, and that constancy
of the unconditional variance follows implicitly from the second requirement
of covariance stationarity, that the autocovariances depend only on displacement, not on time.8
To understand fully the linear dynamic structure of a covariance stationary time series process, we need to compute and examine its mean and its
autocovariance function. For white noise, weve already computed the mean
and the variance, which is the autocovariance at displacement 0. We have yet
to compute the rest of the autocovariance function; fortunately, however, its
very simple. Because white noise is, by definition, uncorrelated over time, all
the autocovariances, and hence all the autocorrelations, are zero beyond displacement 0.9 Formally, then, the autocovariance function for a white noise
process is
( ) =
, = 0
0, 1,
Recall that 2 = (0).

If the autocovariances are all zero, so are the autocorrelations, because the autocorrelations are proportional to the autocovariances.
9
164
and the autocorrelation function for a white noise process is
1, = 0
( ) =
0, 1.
In Figure *** we plot the white noise autocorrelation function.
Finally, consider the partial autocorrelation function for a white noise
series. For the same reason that the autocorrelation at displacement 0 is
always one, so too is the partial autocorrelation at displacement 0. For a
white noise process, all partial autocorrelations beyond displacement 0 are
zero, which again follows from the fact that white noise, by construction, is
serially uncorrelated. Population regressions of yt on yt1 , or on yt1 and
yt2 , or on any other lags, produce nothing but zero coefficients, because the
process is serially uncorrelated. Formally, the partial autocorrelation function
of a white noise process is
p( ) =
1, = 0
0, 1.
We show the partial autocorrelation function of a white noise process in

Figure ***. Again, its degenerate, and exactly the same as the autocorrelation function!
White noise is very special, indeed degenerate in a sense, as what happens
to a white noise series at any time is uncorrelated with anything in the past,
and similarly, what happens in the future is uncorrelated with anything in the
present or past. But understanding white noise is tremendously important
for at least two reasons. First, as already mentioned, processes with much
richer dynamics are built up by taking simple transformations of white noise.
Second, the goal of all time series modeling (and 1-step-ahead forecasting)
11.2. WHITE NOISE
165
is to reduce the data (or 1-step-ahead forecast errors) to white noise. After
all, if such forecast errors arent white noise, then theyre serially correlated,
which means that theyre forecastable, and if forecast errors are forecastable
then the forecast cant be very good. Thus its important that we understand
and be able to recognize white noise.
Thus far weve characterized white noise in terms of its mean, variance,
autocorrelation function and partial autocorrelation function. Another characterization of dynamics involves the mean and variance of a process, conditional upon its past. In particular, we often gain insight into the dynamics in
a process by examining its conditional mean.10 In fact, throughout our study
of time series, well be interested in computing and contrasting the unconditional mean and variance and the conditional mean and variance of
various processes of interest. Means and variances, which convey information
about location and scale of random variables, are examples of what statisticians call moments. For the most part, our comparisons of the conditional
and unconditional moment structure of time series processes will focus on
means and variances (theyre the most important moments), but sometimes
well be interested in higher-order moments, which are related to properties
such as skewness and kurtosis.
For comparing conditional and unconditional means and variances, it will
simplify our story to consider independent white noise, yt iid(0, 2 ). By
the same arguments as before, the unconditional mean of y is 0 and the unconditional variance is 2 . Now consider the conditional mean and variance,
where the information set t1 upon which we condition contains either the
past history of the observed series, t1 = yt1 , yt2 , ..., or the past history of
the shocks, t1 = t1 , t2 .... (Theyre the same in the white noise case.)
In contrast to the unconditional mean and variance, which must be constant
by covariance stationarity, the conditional mean and variance need not be
10
If you need to refresh your memory on conditional means, consult any good introductory statistics book,
such as Wonnacott and Wonnacott (1990).
166
constant, and in general wed expect them not to be constant. The unconditionally expected growth of laptop computer sales next quarter may be ten
percent, but expected sales growth may be much higher, conditional upon
knowledge that sales grew this quarter by twenty percent. For the independent white noise process, the conditional mean is
E(yt |t1 ) = 0,
and the conditional variance is
var(yt |t1 ) = E[(yt E(yt |t1 ))2 |t1 ] = 2 .
Conditional and unconditional means and variances are identical for an independent white noise series; there are no dynamics in the process, and hence
no dynamics in the conditional moments.
11.3
Estimation and Inference for the Mean, Autocorrelation and Partial Autocorrelation Functions
Now suppose we have a sample of data on a time series, and we dont know
the true model that generated the data, or the mean, autocorrelation function
or partial autocorrelation function associated with that true model. Instead,
we want to use the data to estimate the mean, autocorrelation function, and
partial autocorrelation function, which we might then use to help us learn
about the underlying dynamics, and to decide upon a suitable model or set
of models to fit to the data.
11.3.1
Sample Mean
The mean of a covariance stationary series is

= Eyt .
11.3. ESTIMATION AND INFERENCE FOR THE MEAN, AUTOCORRELATION AND PARTIAL AUT
A fundamental principle of estimation, called the analog principle, suggests

that we develop estimators by replacing expectations with sample averages.
Thus our estimator for the population mean, given a sample of size T , is the
sample mean,
T
1X
yt.
y =
T t=1
Typically were not directly interested in the estimate of the mean, but its
needed for estimation of the autocorrelation function.
11.3.2
Sample Autocorrelations
The autocorrelation at displacement for the covariance stationary series y

is
( ) =
E [(yt )(yt )]
.
E[(yt )2 ]
Application of the analog principle yields a natural estimator,

( ) =
1
T
PT
)(yt
t= +1 [(yt y
P
T
1
)2
t=1 (yt y
T
y)]
PT
=
)(yt
t= +1 [(yt y
PT
)2
t=1 (yt y
y)]
This estimator, viewed as a function of , is called the sample autocorrelation function, or correlogram. Note that some of the summations begin
at t = + 1, not at t = 1; this is necessary because of the appearance of yt
in the sum. Note that we divide those same sums by T , even though only
T terms appear in the sum. When T is large relative to (which is the
relevant case), division by T or by T will yield approximately the same
result, so it wont make much difference for practical purposes, and moreover
there are good mathematical reasons for preferring division by T .
Its often of interest to assess whether a series is reasonably approximated
as white noise, which is to say whether all its autocorrelations are zero in
population. A key result, which we simply assert, is that if a series is white
noise, then the distribution of the sample autocorrelations in large samples
168
is

( ) N
1
0,
T

.
Note how simple the result is. The sample autocorrelations of a white noise
series are approximately normally distributed, and the normal is always a
convenient distribution to work with. Their mean is zero, which is to say the
sample autocorrelations are unbiased estimators of the true autocorrelations,
which are in fact zero. Finally, the variance of the sample autocorrelations
is approximately 1/T (equivalently, the standard deviation is 1/ T ), which

is easy to construct and remember. Under normality, taking plus or minus
two standard errors yields an approximate 95% confidence interval. Thus, if
the series is white noise, approximately 95% of the sample autocorrelations
should fall in the interval 0 2/ T . In practice, when we plot the sample

autocorrelations for a sample of data, we typically include the two standard
error bands, which are useful for making informal graphical assessments of
whether and how the series deviates from white noise.
The two-standard-error bands, although very useful, only provide 95%
bounds for the sample autocorrelations taken one at a time. Ultimately,
were often interested in whether a series is white noise, that is, whether all
its autocorrelations are jointly zero. A simple extension lets us test that
hypothesis. Rewrite the expression

1
( ) N 0,
T
as
T ( ) N (0, 1).
11.3. ESTIMATION AND INFERENCE FOR THE MEAN, AUTOCORRELATION AND PARTIAL AUT
Squaring both sides yields11

T 2 ( ) 21 .
It can be shown that, in addition to being approximately normally distributed, the sample autocorrelations at various displacements are approximately independent of one another. Recalling that the sum of independent 2
variables is also 2 with degrees of freedom equal to the sum of the degrees
of freedom of the variables summed, we have shown that the Box-Pierce
Q-statistic,
QBP = T
m
X
2 ( ),
=1
is approximately distributed as a 2m random variable under the null hypothesis that y is white noise.12 A slight modification of this, designed to follow
more closely the 2 distribution in small samples, is
QLB = T (T + 2)
m
X
=1
1
T
2 ( ).
Under the null hypothesis that y is white noise, QLB is approximately distributed as a 2m random variable. Note that the Ljung-Box Q-statistic is
the same as the Box-Pierce Q statistic, except that the sum of squared autocorrelations is replaced by a weighted sum of squared autocorrelations, where
the weights are (T + 2)/(T ). For moderate and large T , the weights are
approximately 1, so that the Ljung-Box statistic differs little from the BoxPierce statistic.
Selection of m is done to balance competing criteria. On one hand, we
dont want m too small, because after all, were trying to do a joint test on
11
Recall that the square of a standard normal random variable is a 2 random variable with one degree
of freedom. We square the sample autocorrelations ( ) so that positive and negative values dont cancel
when we sum across various values of , as we will soon do.
12
m is a maximum displacement selected by the user. Shortly well discuss how to choose it.
170
a large part of the autocorrelation function. On the other hand, as m grows

relative to T , the quality of the distributional approximations weve invoked
deteriorates. In practice, focusing on m in the neighborhood of T is often

reasonable.
11.3.3
Sample Partial Autocorrelations
Recall that the partial autocorrelations are obtained from population linear
regressions, which correspond to a thought experiment involving linear regression using an infinite sample of data. The sample partial autocorrelations
correspond to the same thought experiment, except that the linear regression
is now done on the (feasible) sample of size T . If the fitted regression is
yt = c + 1 yt1 + ... + yt ,
then the sample partial autocorrelation at displacement is
p( ) .
Distributional results identical to those we discussed for the sample autocorrelations hold as well for the sample partial autocorrelations. That is, if
the series is white noise, approximately 95% of the sample partial autocorre
lations should fall in the interval 2/ T . As with the sample autocorrelations, we typically plot the sample partial autocorrelations along with their
two-standard-error bands.
A correlogram analysis simply means examination of the sample autocorrelation and partial autocorrelation functions (with two standard error
bands), together with related diagnostics, such as Q statistics.
We dont show the sample autocorrelation or partial autocorrelation at
displacement 0, because as we mentioned earlier, they equal 1.0, by construction, and therefore convey no useful information. Well adopt this convention
11.4. AUTOREGRESSIVE MODELS FOR SERIALLY-CORRELATED TIME SERIES171
throughout.
Note that the sample autocorrelation and partial autocorrelation are identical at displacement 1. Thats because at displacement 1, there are no earlier
lags to control for when computing the sample partial autocorrelation, so it
equals the sample autocorrelation. At higher displacements, of course, the
two diverge.
11.4
Autoregressive Models for Serially-Correlated Time

Series
11.4.1
Some Preliminary Notation: The Lag Operator
The lag operator and related constructs are the natural language in which
time series models are expressed. If you want to understand and manipulate
time series models indeed, even if you simply want to be able to read the
software manuals you have to be comfortable with the lag operator. The
lag operator, L, is very simple: it operates on a series by lagging it. Hence
Lyt = yt1 . Similarly, L2 yt = L(L(yt )) = L(yt1 ) = yt2 , and so on. Typically
well operate on a series not with the lag operator but with a polynomial
in the lag operator. A lag operator polynomial of degree m is just a linear
function of powers of L, up through the m-th power,
B(L) = b0 + b1 L + b2 L2 + ...bm Lm .
To take a very simple example of a lag operator polynomial operating on
a series, consider the m-th order lag operator polynomial Lm , for which
Lm yt = ytm .
A well-known operator, the first-difference operator , is actually a first-order
172
polynomial in the lag operator; you can readily verify that

yt = (1 L)yt = yt yt1 .
As a final example, consider the second-order lag operator polynomial 1 +
.9L + .6L2 operating on yt . We have
(1 + .9L + .6L2 )yt = yt + .9yt1 + .6yt2 ,
which is a weighted sum, or distributed lag, of current and past values.
All time-series models, one way or another, must contain such distributed
lags, because theyve got to quantify how the past evolves into the present
and future; hence lag operator notation is a useful shorthand for stating and
manipulating time-series models.
Thus far weve considered only finite-order polynomials in the lag operator;
it turns out that infinite-order polynomials are also of great interest. We write
the infinite-order lag operator polynomial as
2
B(L) = b0 + b1 L + b2 L + ... =
bi Li .
i=0
Thus, for example, to denote an infinite distributed lag of current and past
shocks we might write
B(L)t = b0 t + b1 t1 + b2 t2 + ... =
bi ti .
i=0
At first sight, infinite distributed lags may seem esoteric and of limited practical interest, because models with infinite distributed lags have infinitely
many parameters (b0 , b1 , b2 , ...) and therefore cant be estimated with a finite
sample of data. On the contrary, and surprisingly, it turns out that models
involving infinite distributed lags are central to time series modeling. Wolds
theorem, to which we now turn, establishes that centrality.
11.4.2
Autoregressions
When building models, we dont want to pretend that the model we fit is
true. Instead, we want to be aware that were approximating a more complex
reality. Thats the modern view, and it has important implications for timeseries modeling. In particular, the key to successful time series modeling
is parsimonious, yet accurate, approximations. Here we emphasize a very
important class of approximations, the autoregressive (AR) model.
We begin by characterizing the autocorrelation function and related quantities under the assumption that the AR model is true.13 These characterizations have nothing to do with data or estimation, but theyre crucial
for developing a basic understanding of the properties of the models, which
is necessary to perform intelligent modeling. They enable us to make statements such as If the data were really generated by an autoregressive process,
then wed expect its autocorrelation function to have property x. Armed
with that knowledge, we use the sample autocorrelations and partial autocorrelations, in conjunction with the AIC and the SIC, to suggest candidate
models, which we then estimate.
The autoregressive process is a natural approximation to time-series dynamics. Its simply a stochastic difference equation, a simple mathematical
model in which the current value of a series is linearly related to its past
values, plus an additive stochastic shock. Stochastic difference equations are
a natural vehicle for discrete-time stochastic dynamic modeling.
The AR(1) Process
The first-order autoregressive process, AR(1) for short, is

yt = yt1 + t
13
Sometimes, especially when characterizing population properties under the assumption that the models
are correct, we refer to them as processes, which is short for stochastic processes.
174
t W N (0, 2 ).
In lag operator form, we write
(1 L)yt = t.
In Figure *** we show simulated realizations of length 150 of two AR(1)
processes; the first is
yt = .4yt1 + t ,
and the second is
yt = .95yt1 + t ,
where in each case
t iidN (0, 1),
and the same innovation sequence underlies each realization. The fluctuations
in the AR(1) with parameter = .95 appear much more persistent that those
of the AR(1) with parameter = .4. Thus the AR(1) model is capable of
capturing highly persistent dynamics.
Certain conditions must be satisfied for an autoregressive process to be
covariance stationary. If we begin with the AR(1) process,
yt = yt1 + t ,
and substitute backward for lagged ys on the right side, we obtain
yt = t + t1 + 2 t2 + ...
In lag operator form we write
yt =
1
t .
1 L
This moving average representation for y is convergent if and only if || < 1
; thus, || < 1 is the condition for covariance stationarity in the AR(1) case.
Equivalently, the condition for covariance stationarity is that the inverse of
the root of the autoregressive lag operator polynomial be less than one in
absolute value.
From the moving average representation of the covariance stationary AR(1)
process, we can compute the unconditional mean and variance,
E(yt ) = E(t + t1 + 2 t2 + ...)
= E(t ) + E(t1 ) + 2 E(t2 ) + ...
=0
and
var(yt ) = var(t + t1 + 2 t2 + ...)
= 2 + 2 2 + 4 2 + ...
= 2
2i
i=0
2
= 12 .
The conditional moments, in contrast, are
E(yt |yt1 ) = E(yt1 + t |yt1 )
= E(yt1 |yt1 ) + E(t |yt1 )
= yt1 + 0
= yt1
176
and
var(yt |yt1 ) = var((yt1 + t )|yt1 )
= 2 var(yt1 |yt1 ) + var(t |yt1 )
= 0 + 2
= 2.
Note in particular that the simple way that the conditional mean adapts to
the changing information set as the process evolves.
To find the autocovariances, we proceed as follows. The process is
yt = yt1 + t ,
so that multiplying both sides of the equation by yt we obtain
yt yt = yt1 yt + t yt .
For 1, taking expectations of both sides gives
( ) = ( 1).
This is called the Yule-Walker equation. It is a recursive equation; that is,
given ( ), for any , the Yule-Walker equation immediately tells us how to
get ( + 1). If we knew (0) to start things off (an initial condition), we
could use the Yule-Walker equation to determine the entire autocovariance
sequence. And we do know (0); its just the variance of the process, which
we already showed to be
2
(0) = 12 .
Thus we have
2
(0) = 12

2
(1) = 12
2
(2) = 2 12 ,
and so on. In general, then,
2
( ) = 12 , = 0, 1, 2, ....
Dividing through by (0) gives the autocorrelations,
( ) = , = 0, 1, 2, ....
Note the gradual autocorrelation decay, which is typical of autoregressive
processes. The autocorrelations approach zero, but only in the limit as the
displacement approaches infinity. In particular, they dont cut off to zero, as
is the case for moving average processes. If is positive, the autocorrelation
decay is one-sided. If is negative, the decay involves back-and-forth oscillations. The relevant case in business and economics is > 0, but either way,
the autocorrelations damp gradually, not abruptly. In Figure *** and ***
we show the autocorrelation functions for AR(1) processes with parameters
= .4 and = .95. The persistence is much stronger when = .95.
Finally, the partial autocorrelation function for the AR(1) process cuts off
abruptly; specifically,
, = 1
p( ) =
.
0, > 1.
Its easy to see why. The partial autocorrelations are just the last coefficients in a sequence of successively longer population autoregressions. If the
true process is in fact an AR(1), the first partial autocorrelation is just the
autoregressive coefficient, and coefficients on all longer lags are zero.
In Figures *** and *** we show the partial autocorrelation functions for
our two AR(1) processes. At displacement 1, the partial autocorrelations are
178
simply the parameters of the process (.4 and .95, respectively), and at longer
displacements, the partial autocorrelations are zero.
More on the Stability Condition in AR(1)

The key stability condition is || < 1
P
j
Recall yt =
j=0 tj
P
2j 2
= var(yt ) =
j=0
This is the sum of a geometric series. Hence:
2
if || < 1
var(yt ) =
1 2
var(yt ) = otherwise
A More Complete Picture of AR(1) Stability
(On Your Own)
Series yt is persistent but eventually reverts to a fixed mean
Shocks t have persistent effects but eventually die out
P
j
Hint: Consider yt = +
j=0 tj , || < 1
Autocorrelations ( ) nonzero but decay to zero
Autocorrelations ( ) depend on (of course) but not on time
Hint: Use back substitution to relate yt and yt2 . How does it compare to
the relation between yt and yt1 when || < 1?
Series yt varies but not too extremely
Hint: Consider var(yt ) =
2
12 ,
|| < 1
All of this makes for a nice, stable environment.

Covariance stationarity
11.4.3
The AR(p) Process
The general p-th order autoregressive process, or AR(p) for short, is

yt = 1 yt1 + 2 yt2 + ... + p ytp + t
t W N (0, 2 ).
In lag operator form we write
(L)yt = (1 1 L 2 L2 ... p Lp )yt = t.
In our discussion of the AR(p) process we dispense with mathematical
derivations and instead rely on parallels with the AR(1) case to establish
intuition for its key properties.
An AR(p) process is covariance stationary if and only if the inverses of all
roots of the autoregressive lag operator polynomial (L) are inside the unit
circle.14 In the covariance stationary case we can write the process in the
convergent infinite moving average form
yt =
1
t.
(L)
The autocorrelation function for the general AR(p) process, as with that of
the AR(1) process, decays gradually with displacement. Finally, the AR(p)
partial autocorrelation function has a sharp cutoff at displacement p, for
the same reason that the AR(1) partial autocorrelation function has a sharp
cutoff at displacement 1.
Lets discuss the AR(p) autocorrelation function in a bit greater depth.
The key insight is that, in spite of the fact that its qualitative behavior
(gradual damping) matches that of the AR(1) autocorrelation function, it
Pp
A necessary condition for covariance stationarity, which is often useful as a quick check, is i=1 i < 1.
If the condition is satisfied, the process may or may not be stationary, but if the condition is violated, the
process cant be stationary.
14
180
can nevertheless display a richer variety of patterns, depending on the order

and parameters of the process. It can, for example, have damped monotonic
decay, as in the AR(1) case with a positive coefficient, but it can also have
damped oscillation in ways that AR(1) cant have. In the AR(1) case, the
only possible oscillation occurs when the coefficient is negative, in which case
the autocorrelations switch signs at each successively longer displacement. In
higher-order autoregressive models, however, the autocorrelations can oscillate with much richer patterns reminiscent of cycles in the more traditional
sense. This occurs when some roots of the autoregressive lag operator polynomial are complex.15 Consider, for example, the AR(2) process,
yt = 1.5yt1 .9yt2 + t.
The corresponding lag operator polynomial is 1 1.5L + .9L2 , with two
complex conjugate roots, .83 .65i. The inverse roots are .75 .58i, both of
which are close to, but inside, the unit circle; thus the process is covariance
stationary. It can be shown that the autocorrelation function for an AR(2)
process is
(0) = 1
( ) = 1 ( 1) + 2 ( 2), = 2, 3, ...
(1) =
1
1 2
Using this formula, we can evaluate the autocorrelation function for the
process at hand; we plot it in Figure ***. Because the roots are complex,
the autocorrelation function oscillates, and because the roots are close to the
unit circle, the oscillation damps slowly.
Finally, lets step back once again to consider in greater detail the precise
way that finite-order autoregressive processes approximate the Wold repre15
Note that complex roots cant occur in the AR(1) case.
sentation. As always, the Wold representation is

yt = B(L)t ,
where B(L) is of infinite order. The moving average representation associated
with the AR(1) process is
yt =
1
t .
1 L
Thus, when we fit an AR(1) model, were using
1
1L ,
a rational polyno-
mial with degenerate numerator polynomial (degree zero) and denominator

polynomial of degree one, to approximate B(L). The moving average representation associated with the AR(1) process is of infinite order, as is the
Wold representation, but it does not have infinitely many free coefficients. In
fact, only one parameter, , underlies it.
The AR(p) is an obvious generalization of the AR(1) strategy for approximating the Wold representation. The moving average representation
associated with the AR(p) process is
yt =
1
t .
(L)
When we fit an AR(p) model to approximate the Wold representation were

still using a rational polynomial with degenerate numerator polynomial (degree zero), but the denominator polynomial is of higher degree.
11.4.4
Alternative Approaches to Estimating Autoregressions
We can estimate autoregressions directly by OLS.

Alternatively, we can write the AR model as a regression on an intercept,
with a serially correlated disturbance. We have
yt = + t
182
(L)t = vt
vt W N (0, 2 ).
We can estimate each model in identical fashion using nonlinear least
squares. Eviews and other packages proceed in precisely that way.16
This framework regression on a constant with serially correlated disturbances has a number of attractive features. First, the mean of the process
is the regression constant term.17 Second, it leads us naturally toward regression on more than just a constant, as other right-hand side variables can be
added as desired.
*************
Non-Zero Mean I (AR(1) Example): Regression on an Intercept and yt1 ,With
White Noise Disturbances
(yt ) = (yt1 ) + t
t iidN (0, 2 ), || < 1
= yt = c + yt1 + t , where c = (1 )
Back-substitution reveals that:
yt = +
j tj
j=0
= E(yt ) =
Non-Zero Mean II (AR(1) Example, Contd): Regression on an Intercept
Alone, with AR(1) Disturbances
16
Thats why, for example, information on the number of iterations required for convergence is presented
even for estimation of the autoregressive model.
17
Hence the notation for the intercept.
183
y t = + t
t = t1 + vt
vt iidN (0, 2 ), || < 1
11.5
1. (Autocorrelation functions of covariance stationary series)

While interviewing at a top investment bank, your interviewer is impressed by the fact that you have taken a course on time series. She
decides to test your knowledge of the autocovariance structure of covariance stationary series and lists five autocovariance functions:
a. (t, ) =
b. (t, ) = e
c. (t, ) =
d. (t, ) =
, where is a positive constant. Which autocovariance
function(s) are consistent with covariance stationarity, and which are

not? Why?
2. (Autocorrelation vs. partial autocorrelation)
Describe the difference between autocorrelations and partial autocorrelations. How can autocorrelations at certain displacements be positive
while the partial autocorrelations at those same displacements are negative?
3. (Simulating time series processes)
184
Many cutting-edge estimation techniques involve simulation. Moreover,

simulation is often a good way to get a feel for a model and its behavior.
White noise can be simulated on a computer using random number
generators, which are available in most statistics, econometrics and
forecasting packages.
a. Simulate a Gaussian white noise realization of length 200. Call the
white noise t . Compute the correlogram. Discuss.
b. Form the distributed lag yt = t + .9t1 , t = 2, 3, ..., 200. Compute
the sample autocorrelations and partial autocorrelations. Discuss.
c. Let y1 = 1 and yt = .9yt1 + t , t = 2, 3, ..., 200. Compute the sample
autocorrelations and partial autocorrelations. Discuss.
4. (Sample autocorrelation functions for trending series)
A tell-tale sign of the slowly-evolving nonstationarity associated with
trend is a sample autocorrelation function that damps extremely slowly.
a. Find three trending series, compute their sample autocorrelation functions, and report your results. Discuss.
b. Fit appropriate trend models, obtain the model residuals, compute
their sample autocorrelation functions, and report your results. Discuss.
5. (Sample autocorrelation functions for seasonal series)
A tell-tale sign of seasonality is a sample autocorrelation function with
sharp peaks at the seasonal displacements (4, 8, 12, etc. for quarterly
data, 12, 24, 36, etc. for monthly data, and so on).
a. Find a series with both trend and seasonal variation. Compute its
sample autocorrelation function. Discuss.
11.6.
NOTES
185
b. Detrend the series. Discuss.

c. Compute the sample autocorrelation function of the detrended series.
Discuss.
d. Seasonally adjust the detrended series. Discuss.
e. Compute the sample autocorrelation function of the detrended, seasonallyadjusted series. Discuss.
6. (Outliers in Time Series)
Outliers can arise for a number of reasons. Perhaps the outlier is simply
a mistake due to a clerical recording error, in which case youd want to
replace the incorrect data with the correct data. Well call such outliers
measurement outliers, because they simply reflect measurement errors. In a time-series context, if a particular value of a recorded series is
plagued by a measurement outlier, theres no reason why observations
at other times should necessarily be affected.
Alternatively, outliers in time series may be associated with large unanticipated shocks, the effects of which may certainly linger. If, for example, an adverse shock hits the U.S. economy this quarter (e.g., the price
of oil on the world market triples) and the U.S. plunges into a severe
depression, then its likely that the depression will persist for some time.
Such outliers are called innovation outliers, because theyre driven by
shocks, or innovations, whose effects naturally last more than one period due to the dynamics operative in business, economic, and financial
series.
11.6
Notes
186
11.6.
NOTES
187
188
11.6.
NOTES
189
190
11.6.
NOTES
191
192
11.6.
NOTES
193
194
11.6.
NOTES
195
196
11.6.
NOTES
197
198
11.6.
NOTES
199
200
Chapter 12
Serial Correlation in Time Series
Regression
Recall the full ideal conditions.
Here we deal with violation of the assumption that ***
Consider:
N (0, )
The FIC case is = 2 I. When is 6= 2 I?
Weve already seen heteroskedasticity.
Now we consider serial correlation or autocorrelation.
t is correlated with t
Can arise for many reasons, but they all boil down to:
The included X variables fail to capture all the dynamics in y.
No additional explanation needed!
On with Heteroskedasticity vs. Serial Correlation
201
202
CHAPTER 12. SERIAL CORRELATION IN TIME SERIES REGRESSION
With heteroskedasticity, i is independent across i but not identically

distributed across i (variance of i varies with i):
12 0 . . .
0 22 . . .
=
... ... . . .
0
..
.
2
0 . . . N
With serial correlation, t is correlated across t

but unconditionally identically distributed across t:
(1)
. . . (T 1)
2
(1)
.
.
.
(T
2)
=
..
..
.
.
.
.
.
.
.
.
(T 1) (T 2) . . .
2
Consequences of Serial Correlation
OLS inefficient (no longer BLUE),
in finite samples and asymptotically
Standard errors biased and inconsistent. Hence t ratios do not have the t
distribution in finite samples and do not have the N (0, 1) distribution
asymptotically
Does this sound familiar?
Detection
Graphical autocorrelation diagnostics
Residual plot
12.1. TESTING FOR SERIAL CORRELATION
203
Scatterplot of et against et
12.1
Testing for Serial Correlation
If a model has extracted all the systematic information from the data,
then whats left the residual should be iid random noise. Hence the
usefulness of various residual-based tests of the hypothesis that regression disturbances are white noise.
Formal autocorrelation tests and analyses
Durbin-Watson
Breusch-Godfrey
Residual correlogram
Liquor Sales Regression on Trend and Seasonals
Graphical Diagnostics - Residual Plot
Graphical Diagnostics - Scatterplot of et against et1
12.1.1
The Durbin-Watson Test
Formal Tests and Analyses: Durbin-Watson (0.59!)

The Durbin-Watson test (discussed in Chapter 3) is the most popular.
Simple paradigm (AR(1)):
yt = x0t + t
t = t1 + vt
204
205
vt iid N (0, 2 )
We want to test H0 : = 0 against H1 : 6= 0
Regress y X and obtain the residuals et
PT
DW =
2
t=2 (et et1 )
PT 2
t=1 et
Understanding the Durbin-Watson Statistic
PT
2
t=2 (et et1 )
=
PT 2
e
t=1 t
DW =
1
T
PT
2
t=2 et +
1
T
1
T
PT
2
t=2 (et et1 )
PT 2
1
t=1 et
T
PT
2
t=2 et1 2
PT 2
1
t=1 et
T
1
T
PT
t=2 et et1
Hence as T :
2 + 2 2cov(et , et1 )
DW
= 2(1 corr(et , et1 ))
|
{z
}
2
e (1)
206
= DW [0, 4], DW 2 as 0, and DW 0 as 1

Note that the Durbin-Watson test is effectively based only on the first
sample autocorrelation and really only tests whether the first autocorrelation
is zero. We say therefore that the Durbin-Watson is a test for first-order
serial correlation.
In addition, the Durbin-Watson test is not valid if the regressors include
lagged dependent variables.1 (See EPC ***) On both counts, wed like more
general and flexible approaches for diagnosing serial correlation.
12.1.2
The Breusch-Godfrey Test
The Breusch-Godfrey test is an alternative to the Durbin-Watson test.

Its designed to detect pth -order serial correlation, where p is selected by the
user, and is also valid in the presence of lagged dependent variables.
General AR(p) environment:
yt = x0t + t
t = 1 t1 + ... + p tp + vt
vt iidN (0, 2 )
We want to test H0 : (1 , ..., p ) = 0 against H1 : (1 , ..., p ) 6= 0
Regress yt xt and obtain the residuals et
Regress et xt , et1 , ..., etp
1
Following standard, if not strictly appropriate, practice, in this book we often report and examine the
Durbin-Watson statistic even when lagged dependent variables are included. We always supplement the
Durbin-Watson statistic, however, with other diagnostics such as the residual correlogram, which remain
valid in the presence of lagged dependent variables, and which almost always produce the same inference as
the Durbin-Watson statistic.
207
Examine T R2 . In large samples T R2 2p under the null.

Does this sound familiar?
BG for AR(1) Disturbances
(T R2 = 168.5, p = 0.0000)
(T R2 = 216.7, p = 0.0000)
(T R2 = 219.0, p = 0.0000)
12.1.3
The Residual Correlogram
When we earlier introduced the correlogram in Chapter ***, we focused on

the case of an observed time series, in which case we showed that the Q
statistics are distributed as 2m . Now, however, we want to assess whether
unobserved model disturbances are white noise. To do so, we use the model
208
12.2. ESTIMATION WITH SERIAL CORRELATION
209
residuals, which are estimates of the unobserved disturbances. Because we fit

a model to get the residuals, we need to account for the degrees of freedom
used. The upshot is that the distribution of the Q statistics under the white
noise hypothesis is better approximated by a 2mk random variable, where k
is the number of parameters estimated.
cov(e
c t , et )
e ( ) =
=
vd
ar(et )
1
T
P
t et et
P
1
2
t et
T
pe ( ) is the coefficient on et in the regression

et c, et1 , ..., et( 1) , et
Approximate 95% Bartlett bands under the iid N null: 0
QBP = T
m
X
2e ( ) 2mK under iid N
=1
QLB = T (T + 2)
m
X
=1
1
T
2e ( ) 2mK
Residual Correlogram for Trend + Seasonal Model
12.2
Estimation with Serial Correlation
Correcting for Autocorrelation

12.2.1
Regression with Serially-Correlated Disturbances
GLS quasi-differencing, Cochrane-Orcutt iteration
2
T
210
Generalized least squares

Transform the data such that the classical conditions hold
Heteroskedasticity and autocorrelation consistent (HAC) s.e.s
Use OLS, but calculate standard errors robustly
Recall Generalized Least Squares (GLS)
Consider the FIC except that we now let:
N (0, )
The GLS estimator is:

GLS = (X 0 1 X)1 X 0 1 y
Under the remaining full ideal conditions it is consistent, normally
distributed with covariance matrix (X 0 1 X)1 , and MVUE:
GLS N , (X 0 1 X)1
211
Infeasible GLS
(Illustrated in the Durbin-Watson AR(1) Environment)
yt = x0t + t (1a)
t = t1 + vt (1b)
vt iid N (0, 2 ) (1c)
Suppose that you know . Then you could form:
yt1 = x0t1 + t1 (1a)
= (yt yt1 ) = (x0t x0t1 ) + (t t1 ) (just (1a) (1a))
= yt = yt1 + x0t x0t1 () + vt
Satisfies the classical conditions! Note the restriction.
12.2.2
Serially-Correlated Disturbances vs. Lagged Dependent

Variables
Closely related. Inclusion of lagged dependent variables is the more general

(and simple!) approach. OLS estimation.
So, two key closely-related regressions:
yt xt (with AR(1) disturbances)
yt yt1 , xt , xt1 (with W N disturbances and a coef. restr.)
Feasible GLS
(1) Replace the unknown value with
212
Figure 12.1: ***. ***.
an estimate and run the OLS regression:

t1 ) (x0 x
0 )
(yt y
t
t1
Iterate if desired: 1 , 1 , 2 , 2 , ...
(2) Run the OLS Regression
yt yt1 , xt , xt1
subject to the constraint noted earlier (or not)
Generalizes trivially to AR(p): yt yt1 , ..., ytp , xt , xt1 , ..., xtp
(Select p using the usual AIC, SIC, etc.)
Trend + Seasonal Model with AR(4) Disturbances
Residual Plot
Residual Correlogram
Figure 12.2: ***. ***.
Figure 12.3: ***. ***.
213
214
Trend + Seasonal Model with Four Lags of Dep. Var.

How Did we Arrive at AR(4) Dynamics?
Everything points there:
Supported by original trend + seasonal residual correlogram
Supported by DW
Supported by BG
Supported by SIC pattern:
AR(1) = 3.797
AR(2) = 3.941
AR(3) = 4.080
AR(4) = 4.086
AR(5) = 4.071
AR(6) = 4.058
AR(7) = 4.057
AR(8) = 4.040
Heteroskedasticity-and-Autocorrelation Consistent (HAC) Standard Errors
215
Figure 12.4: ***. ***.
Using advanced methods, one can obtain consistent

under minimal assumptions
standard errors (if not an efficient ),
HAC standard errors
Robust standard errors
Newey-West standard errors
remains unchanged at its OLS value. Is that a problem?
Trend + Seasonal Model with HAC Standard Errors
12.2.3
A Full Model of Liquor Sales
Well model monthly U.S. liquor sales. We graphed a short span of the series
in Chapter *** and noted its pronounced seasonality sales skyrocket during
the Christmas season. In Figure ***, we show a longer history of liquor sales,
1968.01 - 1993.12. In Figure *** we show log liquor sales; we take logs to
216
stabilize the variance, which grows over time.2 The variance of log liquor
sales is more stable, and its the series for which well build models.3
Liquor sales dynamics also feature prominent trend and cyclical effects.
Liquor sales trend upward, and the trend appears nonlinear in spite of the
fact that were working in logs. To handle the nonlinear trend, we adopt a
quadratic trend model (in logs). The estimation results are in Table 1. The
residual plot (Figure ***) shows that the fitted trend increases at a decreasing
rate; both the linear and quadratic terms are highly significant. The adjusted
R2 is 89%, reflecting the fact that trend is responsible for a large part of the
variation in liquor sales. The standard error of the regression is .125; its
an estimate of the standard deviation of the error wed expect to make in
forecasting liquor sales if we accounted for trend but ignored seasonality and
serial correlation. The Durbin-Watson statistic provides no evidence against
the hypothesis that the regression disturbance is white noise.
The residual plot, however, shows obvious residual seasonality. The DurbinWatson statistic missed it, evidently because its not designed to have power
against seasonal dynamics.4 The residual plot also suggests that there may be
a cycle in the residual, although its hard to tell (hard for the Durbin-Watson
statistic as well), because the pervasive seasonality swamps the picture and
makes it hard to infer much of anything.
The residual correlogram (Table 2) and its graph (Figure ***) confirm the
importance of the neglected seasonality. The residual sample autocorrelation
function has large spikes, far exceeding the Bartlett bands, at the seasonal
displacements, 12, 24, and 36. It indicates some cyclical dynamics as well;
apart from the seasonal spikes, the residual sample autocorrelation and par2
The nature of the logarithmic transformation is such that it compresses an increasing variance. Make
a graph of log(x) as a function of x, and youll see why.
3
From this point onward, for brevity well simply refer to liquor sales, but remember that weve taken
logs.
4
Recall that the Durbin-Watson test is designed to detect simple AR(1) dynamics. It also has the ability
to detect other sorts of dynamics, but evidently not those relevant to the present application, which are very
different from a simple AR(1).
217
tial autocorrelation functions oscillate, and the Ljung-Box statistic rejects the
white noise null hypothesis even at very small, non-seasonal, displacements.
In Table 3 we show the results of regression on quadratic trend and a full set
of seasonal dummies. The quadratic trend remains highly significant. The
adjusted R2 rises to 99%, and the standard error of the regression falls to
.046, which is an estimate of the standard deviation of the forecast error we
expect to make if we account for trend and seasonality but ignore serial correlation. The Durbin-Watson statistic, however, has greater ability to detect
serial correlation now that the residual seasonality has been accounted for,
and it sounds a loud alarm.
The residual plot of Figure *** shows no seasonality, as thats now picked
up by the model, but it confirms the Durbin-Watsons warning of serial correlation. The residuals are highly persistent, and hence predictable. We
show the residual correlogram in tabular and graphical form in Table ***
and Figure ***. The residual sample autocorrelations oscillate and decay
slowly, and they exceed the Bartlett standard errors throughout. The LjungBox test strongly rejects the white noise null at all displacements. Finally,
the residual sample partial autocorrelations cut off at displacement 3. All
of this suggests that an AR(3) would provide a good approximation to the
disturbances Wold representation.
In Table 5, then, we report the results of estimating a liquor sales model
with quadratic trend, seasonal dummies, and AR(3) disturbances. The R2
is now 100%, and the Durbin-Watson is fine. One inverse root of the AR(3)
disturbance process is estimated to be real and close to the unit circle (.95),
and the other two inverse roots are a complex conjugate pair farther from
the unit circle. The standard error of this regression is an estimate of the
standard deviation of the forecast error wed expect to make after modeling
the residual serial correlation, as weve now done; that is, its an estimate
218
of the standard deviation of v.5 Its a very small .027, roughly half that
obtained when we ignored serial correlation.
We show the residual plot in Figure *** and the residual correlogram in
Table *** and Figure ***. The residual plot reveals no patterns; instead, the
residuals look like white noise, as they should. The residual sample autocorrelations and partial autocorrelations display no patterns and are mostly
inside the Bartlett bands. The Ljung-Box statistics also look good for small
and moderate displacements, although their p-values decrease for longer displacements.
All things considered, the quadratic trend, seasonal dummy, AR(3) specification seems tentatively adequate. We also perform a number of additional
checks. In Figure ***, we show a histogram and normality test applied to the
residuals. The histogram looks symmetric, as confirmed by the skewness near
zero. The residual kurtosis is a bit higher then three and causes Jarque-Bera
test to reject the normality hypothesis with a p-value of .02, but the residuals
nevertheless appear to be fairly well approximated by a normal distribution,
even if they may have slightly fatter tails.
12.3
1. (Serially correlated disturbances vs. lagged dependent variables)

Estimate the quadratic trend model for log liquor sales with seasonal
dummies and three lags of the dependent variable included directly.
Discuss your results and compare them to those we obtained when we
instead allowed for AR(3) disturbances in the regression.
2. (Liquor sales model selection using AIC and SIC)
Use the AIC and SIC to assess the necessity and desirability of including
trend and seasonal components in the liquor sales model.
5
Recall that v is the innovation that drives the AR process for the regression disturbance, .
219
a. Display the AIC and SIC for a variety of specifications of trend and
seasonality. Which would you select using the AIC? SIC? Do the AIC
and SIC select the same model? If not, which do you prefer?
b. Discuss the estimation results and residual plot from your preferred
model, and perform a correlogram analysis of the residuals. Discuss,
in particular, the patterns of the sample autocorrelations and partial
autocorrelations, and their statistical significance.
c. How, if at all, are your results different from those reported in the
text? Are the differences important? Why or why not?
3. (Diagnostic checking of model residuals)
The Durbin-Watson test is invalid in the presence of lagged dependent
variables. Breusch-Godfrey remains valid.
a. Durbins h test is an alternative to the Durbin-Watson test. As
with the Durbin-Watson test, its designed to detect first-order serial
correlation, but its valid in the presence of lagged dependent variables. Do some background reading as well on Durbins h test and
report what you learned.
b. Which do you think is likely to be most useful to you in assessing the
properties of residuals from time-series models: the residual correlogram, Durbins h test, or the Breusch-Godfrey test? Why?
4. (Assessing the adequacy of the liquor sales model trend specification)
Critique the liquor sales model that we adopted (log liquor sales with
quadratic trend, seasonal dummies, and AR(3) disturbances).
a. If the trend is not a good approximation to the actual trend in the
series, would it greatly affect short-run forecasts? Long-run forecasts?
b. Fit and assess the adequacy of a model with log-linear trend.
220
c. How might you fit and assess the adequacy of a broken linear trend?
How might you decide on the location of the break point?
d. Recall our assertion that best practice requires using a 2mk distribution rather than a 2m distribution to assess the significance of
Q-statistics for model residuals, where m is the number of autocorrelations included in the Box-Pierce statistic and k is the number of
parameters estimated. In several places in this chapter, we failed to
heed this advice when evaluating the liquor sales model. If we were
instead to compare the residual Q-statistic p-values to a 2mk distribution, how, if at all, would our assessment of the models adequacy
change?
e. Return to the log-quadratic trend model with seasonal dummies, allow
for AR(p) disturbances, and do a systematic selection of p and q using
the AIC and SIC. Do AIC and SIC select the same model? If not,
which do you prefer? If your preferred model differs from the AR(3)
that we used, replicate the analysis in the text using your preferred
model, and discuss your results.
5. Fixed X and lagged dependent variables.
In section 5.1 we claimed that its logically impossible to maintain the
assumption of fixed X in the presence of a lagged dependent variable.
Why?
12.4
Notes
The idea that regression models with serially correlated disturbances are more
restrictive than other sorts of transfer function models has a long history
in econometrics and engineering and is highlighted in a memorably-titled
paper, Serial Correlation as a Convenient Simplification, not a Nuisance,
by Hendry and Mizon (1978)***.
Chapter 13
Heteroskedasticity in Time Series
The celebrated Wold decomposition makes clear that every covariance
stationary series may be viewed as ultimately driven by underlying weak
white noise innovations. Hence it is no surprise that every model discussed
in this book is driven by underlying white noise. To take a simple example, if
the series yt follows an AR(1) process, then yt = yt1 + t , where t is white
noise. In some situations it is inconsequential whether t is weak or strong
white noise, that is, whether t is independent, as opposed to merely serially
uncorrelated. Hence, to simplify matters we sometimes assume strong white
noise, t iid(0, 2 ). Throughout this book, we have thus far taken that
approach, sometimes explicitly and sometimes implicitly.
When t is independent, there is no distinction between the unconditional
distribution of t and the distribution of t conditional upon its past, by definition of independence. Hence 2 is both the unconditional and conditional
variance of t . The Wold decomposition, however, does not require that t be
serially independent; rather it requires only that t be serially uncorrelated.
If t is dependent, then its unconditional and conditional distributions will
differ. We denote the unconditional innovation distribution by t (0, 2 ).
We are particularly interested in conditional dynamics characterized by heteroskedasticity, or time-varying volatility. Hence we denote the conditional
221
222
CHAPTER 13. HETEROSKEDASTICITY IN TIME SERIES
distribution by t |t1 (0, t2 ), where t1 = t1 , t2 , .... The conditional

variance t2 will in general evolve as t1 evolves, which focuses attention on
the possibility of time-varying innovation volatility.1
Allowing for time-varying volatility is crucially important in certain
economic and financial contexts. The volatility of financial asset returns, for
example, is often time-varying. That is, markets are sometimes tranquil and
sometimes turbulent, as can readily be seen by examining the time series of
stock market returns in Figure 1, to which we shall return in detail. Timevarying volatility has important implications for financial risk management,
asset allocation and asset pricing, and it has therefore become a central part
of the emerging field of financial econometrics. Quite apart from financial
applications, however, time-varying volatility also has direct implications for
interval and density forecasting in a wide variety of applications: correct confidence intervals and density forecasts in the presence of volatility fluctuations
require time-varying confidence interval widths and time-varying density forecast spreads. The models that we have considered thus far, however, do not
allow for that possibility. In this chapter we do so.
13.1
The Basic ARCH Process
Consider the general linear process,

yt = B(L)t
B(L) =
bi Li
i=0
b2i <
i=0
1
In principle, aspects of the conditional distribution other than the variance, such as conditional skewness,
could also fluctuate. Conditional variance fluctuations are by far the most important in practice, however,
so we assume that fluctuations in the conditional distribution of are due exclusively to fluctuations in t2 .
13.1. THE BASIC ARCH PROCESS
223
b0 = 1
t W N (0, 2 ).
We will work with various cases of this process.
Suppose first that t is strong white noise, t iid(0, 2 ). Let us review
some results already discussed for the general linear process, which will prove
useful in what follows. The unconditional mean and variance of y are
E(yt ) = 0
and
E(yt2 )
b2i ,
i=0
which are both time-invariant, as must be the case under covariance stationarity. However, the conditional mean of y is time-varying:
E(yt |t1 ) =
bi ti ,
i=1
where the information set is

t1 = t1 , t2 , ....
The ability of the general linear process to capture covariance stationary
conditional mean dynamics is the source of its power.
Because the volatility of many economic time series varies, one would hope
that the general linear process could capture conditional variance dynamics
as well, but such is not the case for the model as presently specified: the
conditional variance of y is constant at

E (yt E(yt |t1 ))2 |t1 = 2 .
224
This potentially unfortunate restriction manifests itself in the properties of

the h-step-ahead conditional prediction error variance. The minimum mean
squared error forecast is the conditional mean,
E(yt+h |t ) =
bh+i ti ,
i=0
and so the associated prediction error is
yt+h E(yt+h |t ) =
h1
X
bi t+hi ,
i=0
which has a conditional prediction error variance of

E (yt+h E(yt+h |t )) |t =
h1
X
b2i .
i=0
The conditional prediction error variance is different from the unconditional variance, but it is not time-varying: it depends only on h, not on the
conditioning information t . In the process as presently specified, the conditional variance is not allowed to adapt to readily available and potentially
useful conditioning information.
So much for the general linear process with iid innovations. Now we extend
it by allowing t to be weak rather than strong white noise, with a particular
nonlinear dependence structure. In particular, suppose that, as before,
yt = B(L)t
B(L) =
bi Li
i=0
b2i <
i=0
b0 = 1,
13.1. THE BASIC ARCH PROCESS
225
but now suppose as well that

t |t1 N (0, t2 )
t2 = + (L)2t
> 0(L) =
p
X
i Li i 0f oralli
i < 1.
i=1
Note that we parameterize the innovation process in terms of its conditional density,
t |t1 ,
which we assume to be normal with a zero conditional mean and a conditional variance that depends linearly on p past squared innovations. t is
serially uncorrelated but not serially independent, because the current conditional variance t2 depends on the history of t .2 The stated regularity
conditions are sufficient to ensure that the conditional and unconditional
variances are positive and finite, and that yt is covariance stationary.
The unconditional moments of t are constant and are given by
E(t ) = 0
and
E(t E(t ))2 =
The important result is not the particular formulae for the unconditional
mean and variance, but the fact that they are fixed, as required for covariance
stationarity. As for the conditional moments of t , its conditional variance
2
In particular, t2 depends on the previous p values of t via the distributed lag

(L)2t .
226
is time-varying,

E (t E(t |t1 ))2 |t1 = + (L)2t ,
and of course its conditional mean is zero by construction.
Assembling the results to move to the unconditional and conditional moments of y as opposed to t , it is easy to see that both the unconditional mean
and variance of y are constant (again, as required by covariance stationarity),
but that both the conditional mean and variance are time-varying:
E(yt |t1 ) =
bi ti
i=1

E (yt E(yt |t1 ))2 |t1 = + (L)2t .
Thus, we now treat conditional mean and variance dynamics in a symmetric fashion by allowing for movement in each, as determined by the evolving
information set t1 . In the above development, t is called an ARCH(p)
process, and the full model sketched is an infinite-ordered moving average
with ARCH(p) innovations, where ARCH stands for autoregressive conditional heteroskedasticity. Clearly t is conditionally heteroskedastic, because
its conditional variance fluctuates. There are many models of conditional
heteroskedasticity, but most are designed for cross-sectional contexts, such
as when the variance of a cross-sectional regression disturbance depends on
one or more of the regressors.3 However, heteroskedasticity is often present as
well in the time-series contexts relevant for forecasting, particularly in financial markets. The particular conditional variance function associated with
the ARCH process,
t2 = + (L)2t ,
3
The variance of the disturbance in a model of household expenditure, for example, may depend on
income.
13.2. THE GARCH PROCESS
227
is tailor-made for time-series environments, in which one often sees volatility clustering, such that large changes tend to be followed by large changes,
and small by small, of either sign. That is, one may see persistence, or serial
correlation, in volatility dynamics (conditional variance dynamics), quite
apart from persistence (or lack thereof) in conditional mean dynamics. The
ARCH process approximates volatility dynamics in an autoregressive fashion;
hence the name autoregressiveconditional heteroskedasticity. To understand
why, note that the ARCH conditional variance function links todays conditional variance positively to earlier lagged 2t s, so that large 2t s in the
recent past produce a large conditional variance today, thereby increasing
the likelihood of a large 2t today. Hence ARCH processes are to conditional
variance dynamics precisely as standard autoregressive processes are to conditional mean dynamics. The ARCH process may be viewed as a model for
the disturbance in a broader model, as was the case when we introduced it
above as a model for the innovation in a general linear process. Alternatively,
if there are no conditional mean dynamics of interest, the ARCH process may
be used for an observed series. It turns out that financial asset returns often
have negligible conditional mean dynamics but strong conditional variance
dynamics; hence in much of what follows we will view the ARCH process as
a model for an observed series, which for convenience we will sometimes call
a return.
13.2
The GARCH Process
Thus far we have used an ARCH(p) process to model conditional variance

dynamics. We now introduce the GARCH(p,q) process (GARCH stands
for generalized ARCH), which we shall subsequently use almost exclusively.
As we shall see, GARCH is to ARCH (for conditional variance dynamics) as
ARMA is to AR (for conditional mean dynamics).
228
The pure GARCH(p,q) process is given by4

yt = t
t |t1 N (0, t2 )
t2 = + (L)2t + (L)t2
(L) =
p
X
i Li , (L) =
i=1
q
X
i Li
i=1
> 0, i 0, i 0,
i +
i < 1.
The stated conditions ensure that the conditional variance is positive and
that yt is covariance stationary.
Back substitution on t2 reveals that the GARCH(p,q) process can be
represented as a restricted infinite-ordered ARCH process,
t2
X
(L) 2
P
i 2ti ,
+
t =
+
i 1 (L)
1 i i=1
which precisely parallels writing an ARMA process as a restricted infiniteordered AR. Hence the GARCH(p,q) process is a parsimonious approximation
to what may truly be infinite-ordered ARCH volatility dynamics.
It is important to note a number of special cases of the GARCH(p,q)
process. First, of course, the ARCH(p) process emerges when
(L) = 0.
Second, if both (L) and (L) are zero, then the process is simply iid Gaussian noise with variance . Hence, although ARCH and GARCH processes
may at first appear unfamiliar and potentially ad hoc, they are in fact much
more general than standard iid white noise, which emerges as a potentially
4
By pure we mean that we have allowed only for conditional variance dynamics, by setting yt = t . We
could of course also introduce conditional mean dynamics, but doing so would only clutter the discussion
while adding nothing new.
229
highly-restrictive special case.

Here we highlight some important properties of GARCH processes. All
of the discussion of course applies as well to ARCH processes, which are
special cases of GARCH processes. First, consider the second-order moment
structure of GARCH processes. The first two unconditional moments of the
pure GARCH process are constant and given by
E(t ) = 0
and
E(t E(t ))2 =
P ,
i i
while the conditional moments are

E(t |t1 ) = 0
and of course

E (t E(t |t1 ))2 |t1 = + (L)2t + (L)t2 .
In particular, the unconditional variance is fixed, as must be the case under
covariance stationarity, while the conditional variance is time-varying. It is no
surprise that the conditional variance is time-varying the GARCH process
was of course designed to allow for a time-varying conditional variance but
it is certainly worth emphasizing: the conditional variance is itself a serially
correlated time series process.
Second, consider the unconditional higher-order (third and fourth) moment structure of GARCH processes. Real-world financial asset returns,
which are often modeled as GARCH processes, are typically unconditionally symmetric but leptokurtic (that is, more peaked in the center and with
fatter tails than a normal distribution). It turns out that the implied uncondi-
230
tional distribution of the conditionally Gaussian GARCH process introduced

above is also symmetric and leptokurtic. The unconditional leptokurtosis of
GARCH processes follows from the persistence in conditional variance, which
produces clusters of low volatility and high volatility episodes associated
with observations in the center and in the tails of the unconditional distribution, respectively. Both the unconditional symmetry and unconditional
leptokurtosis agree nicely with a variety of financial market data.
Third, consider the conditional prediction error variance of a GARCH
process, and its dependence on the conditioning information set. Because
the conditional variance of a GARCH process is a serially correlated random
variable, it is of interest to examine the optimal h-step-ahead prediction,
prediction error, and conditional prediction error variance. Immediately, the
h-step-ahead prediction is
E(t+h |t ) = 0,
and the corresponding prediction error is
t+h E(t+h |t ) = t+h .
This implies that the conditional variance of the prediction error,

E (t+h E(t+h |t ))2 |t = E(2t+h |t ),
depends on both h and
t ,
because of the dynamics in the conditional variance. Simple calculations
231
reveal that the expression for the GARCH(p, q) process is given by

!
h2
X
2
E(2t+h |t ) =
((1) + (1))i + ((1) + (1))h1 t+1
.
i=0
In the limit, this conditional variance reduces to the unconditional variance

of the process,
lim E(2t+h |t ) =
.
1 (1) (1)
For finite h, the dependence of the prediction error variance on the current
information set t can be exploited to improve interval and density forecasts.
Fourth, consider the relationship between 2t and t2 . The relationship is
important: GARCH dynamics in t2 turn out to introduce ARMA dynamics
in 2t .5 More precisely, if t is a GARCH(p,q) process, then
2t
has the ARMA representation
2t = + ((L) + (L))2t (L)t + t ,
where
t = 2t t2
is the difference between the squared innovation and the conditional variance
at time t. To see this, note that if t is GARCH(p,q), then
t2 = + (L)2t + (L)t2 .
Adding and subtracting
(L)2t
5
Put differently, the GARCH process approximates conditional variance dynamics in the same way that
an ARMA process approximates conditional mean dynamics.
232
from the right side gives

t2 = + (L)2t + (L)2t (L)2t + (L)t2
= + ((L) + (L))2t (L)(2t t2 ).
Adding
2t
to each side then gives
t2 + 2t = + ((L) + (L))2t (L)(2t t2 ) + 2t ,
so that
2t = + ((L) + (L))2t (L)(2t t2 ) + (2t t2 ),
= + ((L) + (L))2t (L)t + t.
Thus,
2t
is an ARMA((max(p,q)), p) process with innovation t , where
t [t2 , ).
2t is covariance stationary if the roots of (L)+(L)=1 are outside the
unit circle.
Fifth, consider in greater depth the similarities and differences between t2
and
2t .
It is worth studying closely the key expression,
t = 2t t2 ,
233
which makes clear that

2t
is effectively a proxy for t2 , behaving similarly but not identically, with
t being the difference, or error. In particular, 2t is a noisy proxy: 2t is
an unbiased estimator of t2 , but it is more volatile. It seems reasonable,
then, that reconciling the noisy proxy 2t and the true underlying t2 should
involve some sort of smoothing of 2t . Indeed, in the GARCH(1,1) case t2
is precisely obtained by exponentially smoothing 2t . To see why, consider
the exponential smoothing recursion, which gives the current smoothed value
as a convex combination of the current unsmoothed value and the lagged
smoothed value,
2t = 2t + (1 )
2t1 .
Back substitution yields an expression for the current smoothed value as
an exponentially weighted moving average of past actual values:
2t =
wj 2tj ,
where
wj = (1 )j .
Now compare this result to the GARCH(1,1) model, which gives the current volatility as a linear combination of lagged volatility and the lagged
2
squared return, t2 = + 2t1 + t1
.
Back substitution yields t2 =
j1 2tj , so that the GARCH(1,1)
process gives current volatility as an exponentially weighted moving average

of past squared returns.
Sixth, consider the temporal aggregation of GARCH processes. By temporal aggregation we mean aggregation over time, as for example when we
convert a series of daily returns to weekly returns, and then to monthly
returns, then quarterly, and so on. It turns out that convergence toward
234
normality under temporal aggregation is a feature of real-world financial asset returns. That is, although high-frequency (e.g., daily) returns tend to
be fat-tailed relative to the normal, the fat tails tend to get thinner under
temporal aggregation, and normality is approached. Convergence to normality under temporal aggregation is also a property of covariance stationary
GARCH processes. The key insight is that a low-frequency change is simply
the sum of the corresponding high-frequency changes; for example, an annual
change is the sum of the internal quarterly changes, each of which is the sum
of its internal monthly changes, and so on. Thus, if a Gaussian central limit
theorem can be invoked for sums of GARCH processes, convergence to normality under temporal aggregation is assured. Such theorems can be invoked
if the process is covariance stationary.
In closing this section, it is worth noting that the symmetry and leptokurtosis of the unconditional distribution of the GARCH process, as well as the
disappearance of the leptokurtosis under temporal aggregation, provide nice
independent confirmation of the accuracy of GARCH approximations to asset return volatility dynamics, insofar as GARCH was certainly not invented
with the intent of explaining those features of financial asset return data.
On the contrary, the unconditional distributional results emerged as unanticipated byproducts of allowing for conditional variance dynamics, thereby
providing a unified explanation of phenomena that were previously believed
unrelated.
13.3
Extensions of ARCH and GARCH Models
There are numerous extensions of the basic GARCH model. In this section,
we highlight several of the most important. One important class of extensions
allows for asymmetric response; that is, it allows for last periods squared
13.3. EXTENSIONS OF ARCH AND GARCH MODELS
235
return to have different effects on todays volatility, depending on its sign.6

Asymmetric response is often present, for example, in stock returns.
13.3.1
Asymmetric Response
The simplest GARCH model allowing for asymmetric response is the threshold GARCH, or TGARCH, model.7 We replace the standard GARCH con-
2
ditional variance function, t2 = + 2t1 + t1
, with t2 = + 2t1 + 2t1 Dt1 +
1, if t < 0
where Dt =
.
0otherwise.
The dummy variable D keeps track of whether the lagged return is posi-
tive or negative. When the lagged return is positive (good news yesterday),
D=0, so the effect of the lagged squared return on the current conditional
variance is simply . In contrast, when the lagged return is negative (bad
news yesterday), D=1, so the effect of the lagged squared return on the current conditional variance is +. If =0, the response is symmetric and we
have a standard GARCH model, but if 6=0 we have asymmetric response
of volatility to news. Allowance for asymmetric response has proved useful for modeling leverage effects in stock returns, which occur when <0.8
Asymmetric response may also be introduced via the exponential GARCH
(EGARCH) model,
ln(t2 )

2
).
= + t1 + t1 + ln(t1
t1
t1
Note that volatility is driven by both size and sign of shocks; hence the model
allows for an asymmetric response depending on the sign of news.9 The
6
In the GARCH model studied thus far, only the square of last periods return affects the current conditional variance; hence its sign is irrelevant.
7
For expositional convenience, we will introduce all GARCH extensions in the context of GARCH(1,1),
which is by far the most important case for practical applications. Extensions to the GARCH(p,q) case are
immediate but notationally cumbersome.
8
Negative shocks appear to contribute more to stock market volatility than do positive shocks. This is
called the leverage effect, because a negative shock to the market value of equity increases the aggregate
debt/equity ratio (other things the same), thereby increasing leverage.
9
The absolute size of news is captured by |rt1 /t1 | , and the sign is captured by rt1 /t1 .
236
log specification also ensures that the conditional variance is automatically

positive, because t2 is obtained by exponentiating ln(t2 ) ; hence the name
exponential GARCH.
13.3.2
Exogenous Variables in the Volatility Function
Just as ARMA models of conditional mean dynamics can be augmented to

include the effects of exogenous variables, so too can GARCH models of
conditional variance dynamics.
We simply modify the standard GARCH volatility function in the obvious
way, writing
2
t2 = + 2t1 + t1
+ xt ,
where is a parameter and x is a positive exogenous variable.10 Allowance

for exogenous variables in the conditional variance function is sometimes
useful. Financial market volume, for example, often helps to explain market
volatility.
13.3.3
Regression with GARCH disturbances and GARCH-M
Just as ARMA models may be viewed as models for disturbances in regressions, so too may GARCH models. We write
yt = 0 + 1 xt + t
t |t1 N (0, t2 )
2
t2 = + 2t1 + t1
. Consider now a regression model with GARCH
disturbances of the usual sort, with one additional twist: the conditional
variance enters as a regressor, thereby affecting the conditional mean. We
10
Extension to allow multiple exogenous variables is straightforward.
13.3. EXTENSIONS OF ARCH AND GARCH MODELS
237
write
yt = 0 + 1 xt + t2 + t
t |t1 N (0, t2 )
2
t2 = + 2t1 + t1
. This model, which is a special case of the gen-
eral regression model with GARCH disturbances, is called GARCH-in-Mean

(GARCH-M). It is sometimes useful in modeling the relationship between
risks and returns on financial assets when risk, as measured by the conditional variance, varies.11
13.3.4
Component GARCH
) +
) = (2t1
Note that the standard GARCH(1,1) process may be written as (t2
where
=
is the unconditional variance.12 This is precisely the GARCH(1,1)
model introduced earlier, rewritten in a slightly different but equivalent form.

In this model, short-run volatility dynamics are governed by the parameters
and , and there are no long-run volatility dynamics, because
is constant.
Sometimes we might want to allow for both long-run and short-run, or persistent and transient, volatility dynamics in addition to the short-run volatility
dynamics already incorporated. To do this, we replace
with a time-varying
2
qt1 ), where the timeprocess, yielding (t2 qt ) = (2t1 qt1 ) + (t1
2
varying long-run volatility, qt , is given by qt = + (qt1 ) + (2t1 t1
).
This component GARCH model effectively lets us decompose volatility

dynamics into long-run (persistent) and short-run (transitory) components,
which sometimes yields useful insights. The persistent dynamics are governed
by , and the transitory dynamics are governed by and .13
11
One may also allow the conditional standard deviation, rather than the conditional variance, to enter
the regression.
12
is sometimes called the long-run variance, referring to the fact that the unconditional variance is
the long-run average of the conditional variance.
13
It turns out, moreover, that under suitable conditions the component GARCH model introduced here is
covariance stationary, and equivalent to a GARCH(2,2) process subject to certain nonlinear restrictions on
its parameters.
238
13.3.5
Mixing and Matching
In closing this section, we note that the different variations and extensions of
the GARCH process may of course be mixed. As an example, consider the fol-
lowing conditional variance function: (t2 qt ) = (2t1 qt1 ) + (2t1 qt1 )Dt1 + (
This is a component GARCH specification, generalized to allow for asymmetric response of volatility to news via the sign dummy D, as well as effects from
the exogenous variable x.
13.4
Estimating, Forecasting and Diagnosing GARCH

Models
Recall that the likelihood function is the joint density function of the data,
viewed as a function of the model parameters, and that maximum likelihood
estimation finds the parameter values that maximize the likelihood function.
This makes good sense: we choose those parameter values that maximize
the likelihood of obtaining the data that were actually obtained. It turns
out that construction and evaluation of the likelihood function is easily done
for GARCH models, and maximum likelihood has emerged as the estimation
method of choice.14 No closed-form expression exists for the GARCH maximum likelihood estimator, so we must maximize the likelihood numerically.15
Construction of optimal forecasts of GARCH processes is simple. In fact,
we derived the key formula earlier but did not comment extensively on it.
Recall, in particular, that
2
t+h,t
h1
X
2

= E t+h |t =
[(1) + (1)]i
!
2
+ [(1) + (1)]h1 t+1
.
i=1
14
The precise form of the likelihood is complicated, and we will not give an explicit expression here, but
it may be found in various of the surveys mentioned in the Notes at the end of the chapter.
15
Routines for maximizing the GARCH likelihood are available in a number of modern software packages
such as Eviews. As with any numerical optimization, care must be taken with startup values and convergence
criteria to help insure convergence to a global, as opposed to merely local, maximum.
13.4. ESTIMATING, FORECASTING AND DIAGNOSING GARCH MODELS
239
In words, the optimal h-step-ahead forecast is proportional to the optimal

1-step-ahead forecast. The optimal 1-step-ahead forecast, moreover, is easily
2
calculated: all of the determinants of t+1
are lagged by at least one period,
so that there is no problem of forecasting the right-hand side variables. In

practice, of course, the underlying GARCH parameters and are unknown
2
and so must be estimated, resulting in the feasible forecast
t+h,t
formed in
the obvious way. In financial applications, volatility forecasts are often of direct interest, and the GARCH model delivers the optimal h-step-ahead point
2
. Alternatively, and more generally, we might not be intrinforecast, t+h,t
sically interested in volatility; rather, we may simply want to use GARCH

volatility forecasts to improve h-step-ahead interval or density forecasts of t ,
which are crucially dependent on the h-step-ahead prediction error variance,
2
t+h,t
. Consider, for example, the case of interval forecasting. In the case
of constant volatility, we earlier worked with Gaussian ninety-five percent

interval forecasts of the form
yt+h,t 1.96h ,
where h denotes the unconditional h-step-ahead standard deviation (which
also equals the conditional h-step-ahead standard deviation in the absence of
volatility dynamics). Now, however, in the presence of volatility dynamics
we use
yt+h,t 1.96t+h,t .
The ability of the conditional prediction interval to adapt to changes in
volatility is natural and desirable: when volatility is low, the intervals are
naturally tighter, and conversely. In the presence of volatility dynamics, the
unconditional interval forecast is correct on average but likely incorrect at any
given time, whereas the conditional interval forecast is correct at all times.
The issue arises as to how to detect GARCH effects in observed returns, and
240
related, how to assess the adequacy of a fitted GARCH model. A key and
simple device is the correlogram of squared returns, 2t . As discussed earlier,
2t is a proxy for the latent conditional variance; if the conditional variance
displays persistence, so too will 2t .16 Once can of course also fit a GARCH
model, and assess significance of the GARCH coefficients in the usual way.
Note that we can write the GARCH process for returns as t = t vt ,
2
where vt iidN (0, 1), t2 = + 2t1 + t1
. Equivalently, the standard-
ized return, v, is iid, t = vt iidN (0, 1).

t
This observation suggests a way to evaluate the adequacy of a fitted

GARCH model: standardize returns by the conditional standard deviation
from the fitted GARCH model,
, and then check for volatility dynamics missed by the fitted model by examining the correlogram of the squared
standardized return, (t /
t )2 . This is routinely done in practice.
13.5
Stock Market Volatility
We model and forecast the volatility of daily returns on *** from *** through
***, excluding holidays, for a total of *** observations. We estimate using
observations ***, and then we forecast observations ***.
In Figure *** we plot the daily returns, rt . There is no visual evidence
of serial correlation in the returns, but there is evidence of serial correlation in the amplitude of the returns. That is, volatility appears to cluster:
large changes tend to be followed by large changes, and small by small, of
either sign. In Figure *** we show the histogram and related statistics for
rt . The mean daily return is slightly positive. Moreover, the returns are
approximately symmetric (only slightly left skewed) but highly leptokurtic.
16
Note well, however, that the converse is not true. That is, if 2t displays persistence, it does not necessarily
follow that the conditional variance displays persistence. In particular, neglected serial correlation associated
with conditional mean dynamics may cause serial correlation in t and hence also in 2t . Thus, before
proceeding to examine and interpret the correlogram of 2t as a check for volatility dynamics, it is important
that any conditional mean effects be appropriately modeled, in which case t should be interpreted as the
disturbance in an appropriate conditional mean model.
13.5. STOCK MARKET VOLATILITY
241
The Jarque-Bera statistic indicates decisive rejection of normality. In Figure

*** we show the correlogram for rt . The sample autocorrelations are tiny
and usually insignificant relative to the Bartlett standard errors, yet the autocorrelation function shows some evidence of a systematic cyclical pattern,
and the Q statistics (not shown), which cumulate the information across all
displacements, reject the null of weak white noise. Despite the weak serial
correlation evidently present in the returns, we will proceed for now as if returns were weak white noise, which is approximately, if not exactly, the case.17
In Figure *** we plot rt2 . The volatility clustering is even more evident than
it was in the time series plot of returns. Perhaps the strongest evidence of
all comes from the correlogram of rt2 , which we show in Figure ***: all sample autocorrelations of rt2 are positive, overwhelmingly larger than those of
the returns themselves, and statistically significant. As a crude first pass at
modeling the stock market volatility, we fit an AR(5) model directly to rt2 ;
the results appear in Table ***. It is interesting to note that the t-statistics
on the lagged squared returns are often significant, even at long lags, yet the
R2 of the regression is low, reflecting the fact that rt2 is a very noisy volatility
proxy. As a more sophisticated second pass at modeling the return volatility,
we fit an ARCH(5) model to rt ; the results appear in Table ***. The lagged
squared returns appear significant even at long lags. The correlogram of
squared standardized residuals shown in Figure ***, however, displays some
remaining systematic behavior, indicating that the ARCH(5) model fails to
capture all of the volatility dynamics, potentially because even longer lags
are needed.18 In Table *** we show the results of fitting a GARCH(1, 1)
model. All of the parameter estimates are highly statistically significant, and
the ARCH coefficient () and GARCH coefficient () sum to a value
near unity (.987), with substantially larger then , as is commonly found
17
In the Exercises, Problems and Complements at the end of this chapter we model the conditional mean,
as well as the conditional variance, of returns.
18
In the Exercises, Problems and Complements at the end of this chapter we also examine ARCH(p)
models with p > 5.
242
for financial asset returns. We show the correlogram of squared standardized

GARCH(1, 1) residuals in Figure ***. All sample autocorrelations are tiny
and inside the Bartlett bands, and they display noticeably less evidence of
any systematic pattern than for the squared standardized ARCH(5) residuals. In Figure *** we show the time series of estimated conditional standard
deviations implied by the estimated GARCH(1, 1) model. Clearly, volatility
fluctuates a great deal and is highly persistent. For comparison we show in
Figure *** the series of exponentially smoothed rt2 , computed using a standard smoothing parameter of .05.19 Clearly the GARCH and exponential
smoothing volatility estimates behave similarly, although not at all identically. The difference reflects the fact that the GARCH smoothing parameter
is effectively estimated by the method of maximum likelihood, whereas the
exponential smoothing parameter is set rather arbitrarily. Now, using the
model estimated using observations ***, we generate a forecast of the conditional standard deviation for the out-of-sample observations ***. We show
the results in Figure ***. The forecast period begins just following a volatility burst, so it is not surprising that the forecast calls for gradual volatility
reduction. For greater understanding, in Figure *** we show both a longer
history and a longer forecast. Clearly the forecast conditional standard deviation is reverting exponentially to the unconditional standard deviation (***),
per the formula discussed earlier.
Heteroskedasticity in Time Series

Key Fact 1: Stock Returns are Approximately Serially Uncorrelated
Key Fact 2: Returns are Unconditionally Non-Gaussian
Unconditional Volatility Measures
Variance: 2 = E(rt )2 (or standard deviation: )
19
For comparability with the earlier-computed GARCH estimated conditional standard deviation, we
actually show the square root of exponentially smoothed rt2 .
Figure 13.1: Time Series of Daily NYSE Returns.
Figure 13.2: Correlogram of Daily Stock Market Returns.
243
244
Figure 13.3: Histogram and Statistics for Daily NYSE Returns.
Mean Absolute Deviation: M AD = E|rt |

Interquartile Range: IQR = 75% 25%
Outlier probability: P |rt | > 5 (for example)
Tail index: s.t. P (rt > r) = k r
Kurtosis: K = E(r )4 / 4
p% Value at Risk (V aRp )): x s.t. P (rt < x) = p
Key Fact 3: Returns are Conditionally Heteroskedastic I
Key Fact 3: Returns are Conditionally Heteroskedastic II
Background: Financial Economics Changes Fundmentally
When Volatility is Dynamic
Risk management
Portfolio allocation
Asset pricing
Figure 13.4: Time Series of Daily Squared NYSE Returns
Figure 13.5: Correlogram of Daily Squared NYSE Returns.
245
246
Hedging
Trading
Asset Pricing I: Sharpe Ratios
Standard Sharpe:
E(rit rf t )
Conditional Sharpe:
E(rit rf t )
t
Asset Pricing II: CAPM Standard CAPM:
(rit rf t ) = + (rmt rf t )
=
cov((rit rf t ), (rmt rf t ))
var(rmt rf t )
Conditional CAPM:
t =
covt ((rit rf t ), (rmt rf t ))

vart (rmt rf t )
Asset Pricing III: Derivatives

Black-Scholes:
C = N (d1 )S N (d2 )Ker
d1 =
ln(S/K) + (r + 2 /2)
d2 =
ln(S/K) + (r 2 /2)
247
PC = BS(, ...)
(Standard Black-Scholes options pricing)
Completely different when varies!
Conditional Return Distributions
f (rt ) vs. f (rt |t1 )
Key 1: E(rt |t1 )
Are returns conditional mean independent? Arguably yes.
Returns are (arguably) approximately serially uncorrelated, and (arguably)
approximately free of additional non-linear conditional mean dependence.
Conditional Return Distributions, Continued
Key 2: var(rt |t1 ) = E((rt )2 |t1 )
Are returns conditional variance independent? No way!
Squared returns serially correlated, often with very slow decay.
Linear Models (e.g., AR(1))
rt = rt1 + t
t iid(0, 2 ), || < 1
Uncond. mean: E(rt ) = 0 (constant)
Uncond. variance: E(rt2 ) = 2 /(1 2 ) (constant)
Cond. mean: E(rt | t1 ) = rt1 (varies)
248
Cond. variance: E([rt E(rt | t1 )]2 | t1 ) = 2 (constant)

Conditional mean adapts, but conditional variance does not
ARCH(1) Process
rt |t1 N (0, ht )
2
ht = + rt1
E(rt ) = 0
E(rt 2 ) =
(1 )
E(rt |t1 ) = 0
2
E([rt E(rt |t1 )]2 |t1 ) = + rt1
GARCH(1,1) Process (Generalized ARCH)
rt | t1 N (0, ht )
2
ht = + rt1
+ ht1
E(rt ) = 0
E(rt 2 ) =
(1 )
E(rt |t1 ) = 0
2
E([rt E(rt | t1 )]2 | t1 ) = + rt1
+ ht1
Well-defined and covariance stationary if
249
0 < < 1, 0 < < 1, + < 1

GARCH(1,1) and Exponential Smoothing
Exponential smoothing recursion:
2
t2 =
t1
+ (1 )rt2
t2
= (1 )
2
j rtj
But in GARCH(1,1) we have:

2
ht = + rt1
+ ht1
2
+
j1 rtj
ht =
1
Unified Theoretical Framework
Volatility dynamics (of course, by construction)
Volatility clustering produces unconditional leptokurtosis
Temporal aggregation reduces the leptokurtosis
Tractable Empirical Framework
L(; r1 , . . . , rT ) = f (rT |T 1 ; )f ((rT 1 |T 2 ; ) . . . ,

where = (, , )0
If the conditional densities are Gaussian,

2
1
1
r
t
,
f (rt |t1 ; ) = ht ()1/2 exp
2 ht ()
2
250
Explanatory variables in the variance equation: GARCH-X

Fat-tailed conditional densities: t-GARCH
Asymmetric response and the leverage effect: T-GARCH
Regression with GARCH disturbances
Time-varying risk premia: GARCH-M
so
ln L = const
1 X rt2
1X
ln ht ()
2 t
2 t ht ()
Variations on the GARCH Theme

Explanatory variables in the Variance Equation: GARCH-X
2
ht = + rt1
+ ht1 + zt
where z is a positive explanatory variable

Fat-Tailed Conditional Densities: t-GARCH
If r is conditionally Gaussian, then
p
rt = ht N (0, 1)
But often with high-frequency data,
r
t leptokurtic
ht
251
So take:
p
rt = ht
td
std(td )
and treat d as another parameter to be estimated

Asymmetric Response and the Leverage Effect: T-GARCH
2
Standard GARCH: ht = + rt1
+ ht1
2
2
Dt1 + ht1
+ rt1
T-GARCH: ht = + rt1
(
1 if rt < 0
Dt =
0 otherwise
positive return (good news): effect on volatility

negative return (bad news): + effect on volatility
6= 0: Asymetric news response
> 0: Leverage effect
Regression with GARCH Disturbances
yt = x0t + t
t |t1 N (0, ht )
Time-Varying Risk Premia: GARCH-M
Standard GARCH regression model:
yt = x0t + t
252
t |t1 N (0, ht )
GARCH-M model is a special case:
yt = x0t + ht + t
t |t1 N (0, ht )
Back to Empirical Work Standard GARCH(1,1)
GARCH(1,1)
GARCH(1,1)
GARCH(1,1)
GARCH(1,1)
A Useful Specification Diagnostic
rt |t1 N (0, ht )
rt =
ht t , t iidN (0, 1)
Figure 13.6: GARCH(1,1) Estimation, Daily NYSE Returns.
253
254
Figure 13.7: Estimated Conditional Standard Deviation, Daily NYSE Returns.
Figure 13.8: Conditional Standard Deviation, History and Forecast, Daily NYSE Returns.
255
Figure 13.9: Correlogram of Squared Standardized GARCH(1,1) Residuals, Daily NYSE

Returns.
r
t = t , t iidN (0, 1)
ht
Infeasible: examine t . iid? Gaussian?
p
t . iid? Gaussian?
Feasible: examine t = rt / h
Key deviation from iid is volatility dynamics. So examine correlogram of
squared standardized returns, 2t
GARCH(1,1)
Fancy GARCH(1,1)
Fancy GARCH(1,1)
13.6
1. (Graphical regression diagnostic: time series plot of e2t or |et |)

Plots of e2t or |et | reveal patterns (most notably serial correlation) in the
squared or absolute residuals, which correspond to non-constant volatility, or heteroskedasticity, in the levels of the residuals. As with the
standard residual plot, the squared or absolute residual plot is always a
256
simple univariate plot, even when there are many right-hand side variables. Such plots feature prominently, for example, in tracking and
forecasting time-varying volatility.
2. (Removing conditional mean dynamics before modeling volatility dynamics)
In the application in the text we noted that NYSE stock returns appeared to have some weak conditional mean dynamics, yet we ignored
them and proceeded directly to model volatility.
a. Instead, first fit autoregressive models using the SIC to guide order
selection, and then fit GARCH models to the residuals. Redo the
entire empirical analysis reported in the text in this way, and discuss
any important differences in the results.
b. Consider instead the simultaneous estimation of all parameters of
AR(p)-GARCH models. That is, estimate regression models where
the regressors are lagged dependent variables and the disturbances
Dependent Variable: R
Method: ML - ARCH (Marquardt) - Student's t distribution
Date: 04/10/12 Time: 13:48
Sample (adjusted): 2 3461
Included observations: 3460 after adjustments
Convergence achieved after 19 iterations
Presample variance: backcast (parameter = 0.7)
GARCH = C(4) + C(5)*RESID(-1)^2 + C(6)*RESID(-1)^2*(RESID(-1)<0)
+ C(7)*GARCH(-1)
Variable
Coefficient
Std. Error
z-Statistic
Prob.
@SQRT(GARCH)
C
R(-1)
0.083360
1.28E-05
0.073763
0.053138
0.000372
0.017611
1.568753
0.034443
4.188535
0.1167
0.9725
0.0000
Variance Equation
C
RESID(-1)^2
RESID(-1)^2*(RESID(1)<0)
GARCH(-1)
1.03E-06
0.014945
2.23E-07
0.009765
4.628790
1.530473
0.0000
0.1259
0.094014
0.922745
0.014945
0.009129
6.290700
101.0741
0.0000
0.0000
T-DIST. DOF
5.531579
0.478432
11.56188
0.0000
Figure 13.10: AR(1) Returns with Threshold t-GARCH(1,1)-in Mean.
257
258
display GARCH. Redo the entire empirical analysis reported in the

text in this way, and discuss any important differences in the results
relative to those in the text and those obtained in part a above.
3. (Variations on the basic ARCH and GARCH models) Using the stock
return data, consider richer models than the pure ARCH and GARCH
models discussed in the text.
a. Estimate, diagnose and discuss a threshold GARCH(1,1) model.
b. Estimate, diagnose and discuss an EGARCH(1,1) model.
c. Estimate, diagnose and discuss a component GARCH(1,1) model.
d. Estimate, diagnose and discuss a GARCH-M model.
4. (Empirical performance of pure ARCH models as approximations to
volatility dynamics)
Here we will fit pure ARCH(p) models to the stock return data, including
values of p larger than p=5 as done in the text, and contrast the results
with those from fitting GARCH(p,q) models.
a. When fitting pure ARCH(p) models, what value of p seems adequate?
b. When fitting GARCH(p,q) models, what values of p and q seem adequate?
c. Which approach appears more parsimonious?
5. (Direct modeling of volatility proxies)
In the text we fit an AR(5) directly to a subset of the squared NYSE
stock returns. In this exercise, use the entire NYSE dataset.
a. Construct, display and discuss the fitted volatility series from the
AR(5) model.
259
b. Construct, display and discuss an alternative fitted volatility series

obtained by exponential smoothing, using a smoothing parameter of
.10, corresponding to a large amount of smoothing, but less than done
in the text.
c. Construct, display and discuss the volatility series obtained by fitting
an appropriate GARCH model.
d. Contrast the results of parts a, b and c above.
e. Why is fitting of a GARCH model preferable in principle to the AR(5)
or exponential smoothing approaches?
6. (Assessing volatility dynamics in observed returns and in standardized
returns)
In the text we sketched the use of correlograms of squared observed returns for the detection of GARCH, and squared standardized returns
for diagnosing the adequacy of a fitted GARCH model. Examination of
Ljung-Box statistics is an important part of a correlogram analysis. It
can be shown that the Ljung-Box statistics may be legitimately used on
squared observed returns, in which case it will have the usual 2m distribution under the null hypothesis of independence. One may also use the
Ljung-Box statistic on the squared standardized returns, but a better
distributional approximation is obtained in that case by using a 2mk
distribution, where k is the number of estimated GARCH parameters,
to account for degrees of freedom used in model fitting.
7. (Allowing for leptokurtic conditional densities)
Thus far we have worked exclusively with conditionally Gaussian GARCH
models, which correspond to t = t vt vt iidN (0, 1), or equivalently,
to normality of the standardized return, t /t .
a. The conditional normality assumption may sometimes be violated.
260
However, GARCH parameters are consistently estimated by Gaussian

maximum likelihood even when the normality assumption is incorrect.
Sketch some intuition for this result.
b. Fit an appropriate conditionally Gaussian GARCH model to the stock
return data. How might you use the histogram of the standardized
returns to assess the validity of the conditional normality assumption?
Do so and discuss your results.
c. Sometimes the conditionally Gaussian GARCH model does indeed
fail to explain all of the leptokurtosis in returns; that is, especially
with very high-frequency data, we sometimes find that the conditional
density is leptokurtic. Fortunately, leptokurtic conditional densities
are easily incorporated into the GARCH model. For example, in the
conditionally Students-t GARCH model, the conditional density
is assumed to be Students t, with the degrees-of-freedom d treated
as another parameter to be estimated. More precisely, we write
vt iid
td
.
std(td )
t = t vt
What is the reason for dividing the Students t variable, td , by its
standard deviation, std(td ) ? How might such a model be estimated?
8. (Multivariate GARCH models)
In the multivariate case, such as when modeling a set of returns rather
than a single return, we need to model not only conditional variances,
but also conditional covariances.
a. Is the GARCH conditional variance specification introduced earlier,
2
say for the i th return, it2 = + 2i,t1 + i,t1
, still appealing in
the multivariate case? Why or why not?
13.7.
NOTES
261
b. Consider the following specification for the conditional covariance between i th and j-th returns: ij,t = + i,t1 j,t1 + ij,t1 . Is it
appealing? Why or why not?
c. Consider a fully general multivariate volatility model, in which every conditional variance and covariance may depend on lags of every
conditional variance and covariance, as well as lags of every squared
return and cross product of returns. What are the strengths and
weaknesses of such a model? Would it be useful for modeling, say, a
set of five hundred returns? If not, how might you proceed?
13.7
Notes
262
Part IV
More
263
Chapter 14
Binary Regression and Classification
14.1
Binary Regression
Another appearance of dummy variables: LHS.

Here we work with limited dependent variables, meaning that they
can take only a limited number of values. The classic case is a 0-1 dummy
variable. Dummy right-hand side variables (RHS) variables create no
problem, and you already understand them. The new issue is Dummy lefthand-side variables (LHS), which do raise special issues.
14.1.1
Binary Response
Note that the basic regression model,

yt = x0t + ,
immediately implies that
E(yt |xt ) = x0t .
Here we consider left-hand-side variables yt = It (z), where the dummy variable (indicator variable) It (z) indicates whether event z occurs; that
is,
265
266
CHAPTER 14. BINARY REGRESSION AND CLASSIFICATION
(
It (z) =
1 if event z occurs
0 otherwise.
In that case we have

E(It (z)|xt ) = x0t .
A key insight, however, is that
E(It (z)|xt ) = P (It (z) = 1|xt ),
so the model is effectively
P (It (z) = 1|xt ) = x0t .
(14.1)
That is, when the LHS variable is a 0-1 indicator variable, the model is effectively a model relating a conditional probability to the conditioning variables.
There are numerous events that fit the 0-1 paradigm. Leading examples
include recessions, bankruptcies, loan or credit card defaults, financial market
crises, and consumer choices.
But how should we fit a line when the LHS variable is binary? The
linear probability model does it by brute-force OLS regression It (z) xt .
There are several econometric problems associated with such regressions, but
the one of particular relevance is simply that the linear probability model fails
to constrain the fitted values of E(It (z)|xt ) = P (It (z) = 1|xt ) to lie in the unit
interval, in which probabilities must of course lie. We now consider models
that impose that constraint by running x0t through a monotone squashing
function, F (), that keeps P (It (z) = 1|xt ) in the unit interval. That is, we
move to models with
P (It (z) = 1|xt ) = F (x0t ),
where F () is monotone increasing, with limw F (w) = 1 and limw F (w) =
14.1. BINARY REGRESSION
267
0. Many squashing functions can be entertained, and many have been entertained.
14.1.2
The Logit Model
The most popular and useful squashing function for our purposes is the logistic function, which takes us to the so-called logit model. There are several
varieties and issues, to which we now turn.
Logit
In the logit model, the squashing function F () is the logistic function,

ew
1
F (w) =
=
,
1 + ew
1 + ew
so
ext
P (It (z) = 1|xt ) =
0 .
1 + e xt
At one level, theres little more to say; it really is that simple. The likelihood
function can be derived, and the model can be immediately estimated by
numerical maximization of the likelihood function.
But an alternative latent variable formulation yields deep and useful insights. In particular, consider a latent variable, yt , where
yt = x0t + t
t logistic(0, 1),
and let It (z) be It (yt > 0), or equivalently, It ( > x0t ). Interestingly, this
is the logit model. To see this, note that
E(It (yt > 0)|xt ) = P (yt > 0)|xt ) = P (t > x0t )
= P (t < x0t ) (by symmetry of the logistic density of )
268

0
ext
=
0 ,
1 + e xt
where the last equality holds because the logistic density has cdf is ew /(1+ew ).
This way of thinking about the logit DGP a continuously-evolving latent
variable yt with an observed indicator that turns on when yt > 0 is very
useful. For example, it helps us to think about consumer choice as a function of continuous underlying utility, business cycle regime as a function of
continuous underlying macroeconomic conditions, bond ratings as a function
of continuous underlying firm health, etc.
The latent-variable approach also leads to natural generalizations like ordered logit, to which we now turn.
Ordered Logit
Here we still imagine a continuously-evolving underlying latent variable, but

we have a more-refined indicator, taking not just two values, but several
(ordered) values. Examples include financial analyst stocks ratings of buy,
hold and sell; S&P and Moodys bond ratings in several buckets; and
surveys of that ask about degree of belief in several categories categories
ranging from strongly disagree through strongly agree.
Suppose that there are N ordered outcomes. As before, we have a
continuously-evolving latent variable,
yt = x0t + t
t logistic(0, 1).
14.1. BINARY REGRESSION
269
But now we have an indicator with a finer gradation:
0 if yt < c1
1 if c1 < yt < c2
It (yt ) =
2 if c2 < yt < c3
..
N if c < y .
N
We can estimate this ordered logit model by maximum likelihood, just

as with the standard logit model. All interpretation remains the same, so
long as relationships between all pairs of outcome groups are the same, an
assumption known as proportional odds.
Dynamic Logit
Note that one or more of the x variables could be lagged dependent variables,
Iti (z), i = 1, 2, ...
Complications
In logit regression, both the marginal effects and the R2 are hard to determine
and/or interpret directly.
Marginal Effects
Logit marginal effects E(y|x)/xi are hard to determine directly; in particular, they are not simply given by the i s. Instead we have
E(y|x)
= f (x0 )i ,
xi
where f (x) = dF (x)/dx is the density corresponding the cdf f .1 So the
marginal effect is not simply i ; instead it is i weighted by f (x0 ), which
depends on all s and xs. However, signs of s are the signs of the effects,
1
In the leading logit case, f (x) would be the logistic density, given by ***.
270
because f must be positive. In addition, ratios of s do give ratios of effects,

because the f s cancel.
R2
Recall that traditional R2 for continuous LHS variables is
P
(yt yt )2
.
R2 = 1 P
(yt yt )2
Its not clear how to define or interpret R2 when the LHS variable is 0-1, but
several variants have been proposed. The two most important are Effrons
and McFaddens.
Effrons R2 is
2
R =1
(yt P (It (z) = 1|xt ))2

P
.
(yt yt )2
Effrons R2 attempts to maintain the R2 interpretation as variation explained

and as correlation between actual and fitted values.
McFaddens R2 is
1
lnL
,
R =1
0
lnL
2
0 is the maximized restricted log likelihood (only an intercept inwhere lnL

1 is the maximized unrestricted log likelihood. McFaddens
cluded) and lnL
R2 attempts to maintain the R2 interpretation as improvement from restricted
to unrestricted model.
14.1.3
Classification and 0-1 Forecasting
Examples: Make loan or not, grant credit card or not, hire a worker or not,
will consumer buy or not
Classification maps probabilities into 0-1 forecasts. Bayes classifier uses
a cutoff of .5.
Decision boundary. Suppose we use a Bayes classifier.
271
We predict 1 when logit(x0 ) > 1/2. But thats the same as predicting
1 when x0 > 0. If there are 2 x variables (potentially plus an intercept),
then the condition x0 > 0 defines a line in R2 . Points on one side will be
classified as 0, and points on the other side will be classified as 1. That line
is the decision boundary.
We can also have non-linear decision boundaries. Suppose for example
that that x vector contains not only x1 and x2 , but also x21 and x22 . Now the
condition x0 > 0 defines a circle in R2 . Points inside will be classified as 0,
and points outside will be classified as 1. The circle is the decision boundary.
** Figures illustrating linear and non-linear decision boundaries
14.2
1. (Logit and Ordered-Logit Situations)

In the chapter we gave several examples where logit or ordered-logit
modeling would be appropriate.
a. Give three additional examples where logit modeling would be appropriate. Why?
b. Give three additional examples where ordered-logit modeling would
be appropriate. Why?
2. (The Logistic Squashing Function)
We used the logistic function throughout this chapter. In particular, it
is the foundation on which the logit model is built.
a. What is the logistic function? Write it down precisely.
b. From where does the logistic function come?
272
c. Verify that the logistic function is a legitimate squashing function.

That is, verify that it is monotone increasing, with limw F (w) = 1
and and limw F (w) = 0.
3. (The Logit Likelihood Function)
Consider the logit model (14.1). It is more formally called a binomial
logit model, in reference to its two outcome categories.
a. Derive the likelihood function. (Hint: Consider the binomial structure.)
b. Must the likelihood be maximized numerically, or is an analytic formula available?
4. (Logit as a Linear Model for Log Odds)]
The odds O(It (z) = 1|xt ) of an event z are just a simple transformation
of its probability
O(It (z) = 1|xt ) =
P (It (z) = 1|xt )

.
1 P (It (z) = 1|xt )
Consider a linear model for log odds

P (It (z) = 1|xt )
ln
= x0t .
1 P (It (z) = 1|xt )
Solving the log odds for P (It (z) = 1|xt ) yields the logit model,
0
1
ext
P (It (z) = 1|xt ) =
=
0
0 .
1 + ext
1 + e xt
Hence the logit model is simply a linear regression model for log odds.
A full statement of the model is
yt Bern(pt )
pt
ln
1 pt
273
= x0t .
5. (Probit and GLM Squashing Functions)

Other squashing functions are sometimes used for binary-response regression.
a. In the probit model, we simply use a different squashing function
to keep probabilities in the unit interval. F () is the standard normal
cumulative density function (cdf), so the model is
P (It (z) = 1|xt ) = (x0t ),
where (x) = P (z x) for N (0, 1) random variable z.
b. More exotic, but equally simple, squashing functions have also been
used. Almost all (including those used with logit and probit) are
special cases of those allowed in the generalized linear model
(GLM), a flexible regression framework with uses far beyond just
binary-response regression. In the GLM,
E(yt |xt ) = G1 (x0t ),
and very wide classes of link functions G can be entertained.
6. (Multinomial Models)
In contrast to the binomial logit model, we can also have more than
two categories (e.g., what transportation method will I choose to get to
work: Private transportation, public transportation, or walking?), and
use multinomial logit.
7. (Other Situations/Mechanisms Producing Limited Dependent Variables)
Situations involving censoring or counts also produce limited dependent
variables.
274
a. Data can be censored by definition (e.g. purchases cant be negative).

For example, we might see only yt , where yt = yt if yt 0, and 0
otherwise, and where
yt = 0 + 1 xt + t .
This is the framework in which the Tobit model works.
b. Data can be censored due to sample selection, for example if income
is forecast using a model fit only to high-income people.
c. Counts (e.g., points scored in hockey games) are automatically censored, as they must be in the natural numbers, 1, 2, 3...
14.3
Notes
Chapter 15
Panel Data
We still work under the full ideal conditions, but we consider new aspects of
the regression model.
More use of dummy variables:
Cross-section dummies (individual effects)
Time-series dummies (time effects)
15.1
Panel Data
Consider the bivariate panel regression case. Extension to multiple panel

regression is immediate but notationally tedious. The data are (yit , xit ) (for
person i at time t), i = 1, ..., N (cross section dimension), t = 1, ..., T (time
series dimension).
In panels we have N T observations, or data points, which is a lot
of data. In a pure cross section we have just i = 1, ..., N observations, so
we could never allow for different intercepts (say) across all people, because
there are N people, and we have only N observations, so wed run out of
degrees of freedom. Similarly, in a pure time series we have just t = 1, ..., T
observations, so we could never allow for different intercepts (say) across
all time periods, because there are T time periods, and we have only T
observations, so wed run out of degrees of freedom. But with panel data,
275
276
CHAPTER 15. PANEL DATA
allowing for such individual and time effects is generally no problem. There
are N + T coefficients to be estimated (N individual effects and T time
effects), but we have N T observations!
In traditional panel situations, N >> T . In chapter ?? we deal with
a class of models called vector autoregressions, which also involve both
cross-section and time-series data, but for which T >> N .
15.2
Individual and Time Effects
15.2.1
Individual Effects
We write
yit = i + 2 xit + it .
Individual effects correspond to a different intercept (i ) for each person.
The estimation strategy, in principle, is to run
yit I(i = 1), I(i = 2), I(i = 3), ..., I(i = N ), xit .
Note that this regression is impossible in pure cross sections of size N , in
which case one could at best consider a middle ground in which individuals
are grouped into ui < N units. In the panel case the regression can be run
in principle, and it has the benefit of allowing one to examine the individual
effects (i s) and the common effect (2 ), but the cost of potentially significant
numerical/computational difficulty unless N is small. In panels of typical size,
the regression is infeasible.
How Really to do it
We eliminate the i by writing the model in deviations from means,
(yit yi ) = (i i ) + 2 (xit xi ) + (it i ) ,
where xi =
1
T
T
P
xit and i =
t=1
1
T
T
P
277
it . Because i is constant, i = i , so
t=1
(i i ) = 0. Hence we simply run

(yit yi ) (xit xi ) .
15.2.2
Individual and Time Effects
We write
yit = i + t + 2 xit + it .
The estimation strategy, in principle, is to run
yit I(i = 1), I(i = 2), ..., I(i = N ), I(t = 1), I(t = 2), ..., I(t = T ), xit .
How Really to do it
15.3
1. Panels vs. Time Series of Cross Sections

The two are not the same. Why?
15.4
Notes
Our panel effects are called fixed effects, for obvious reasons. We can also
entertain so-called random effects.
278
CHAPTER 15. PANEL DATA
Chapter 16
Stochastic Regressors II: Endogeneity
A host of complications can produce E(X 0 ) 6= 0.
In this chapter we confront those complications.
16.1
Causes of E(X 0) 6= 0
16.1.1
Omitted Variables
We already introduced the problem above.

If an omitted variable (confounding variable) is correlated with an
included variable, then stop omitting it! But of course thats easier said than
done. We simply may never know about various omitted variables, or we may
suspect them but be unable to measure them.
There may also be problems of over-controlling.
16.1.2
Simultaneity
Suppose that y and x are jointly determined, as for example in simultaneous

determination of price and quantity in market equilibrium. We may write
yt = x0t + , but note that the shocks affect not only y but also x. That
is, is correlated with x, violating the FIC.
279
280
16.1.3
CHAPTER 16. STOCHASTIC REGRESSORS II: ENDOGENEITY
Measurement Error
Measurement errors in x may be correlated with y and hence . For example,

those people in the extreme tails of the distribution of y (that is, those people
who got extremely large positive or negative shocks) may have their xs
measured particularly well or poorly.
16.1.4
Sample Selection
16.2
Instrumental Variables
16.2.1
The Basic Idea and the IV Estimator
16.2.2
Instrument Strength and Exogeneity
We want instruments that are both strong or relevant (highly-correlated

with x) and exogenous (uncorrelated with ). There is a tradeoff. A strong
but slightly-endogenous instrument might nevertheless be highly valuable, as
might a weak but completely-exogenous instrument.
Exogenous but Weakly-Relevant Instruments
Nelson-Startz disease
Relevant but Slightly-Endogenous Instruments
16.2.3
Sources of Instruments
Randomized Experiments
Randomized experiments, sometimes called randomized control trials (RCTs),

are the gold standard. Randomistas.
16.3. ADDITIONAL USEFUL STRATEGIES
281
Natural Experiments
Thought Experiments (Structural Econometric Models)
Require many assumptions. But if the assumptions are credible, they can be
used to assess the effects of a wide variety of treatments. Counterfactuals.
Time
In time-series contexts with xt serially correlated, an obvious instrument for

xt is its lag, xt1 . Due to the serial correlation in x, xt1 is correlated with
xt , yet xt1 cant be correlated with t , which is independent over time and
hence uncorrelated with xt1 .
16.3
Additional Useful Strategies
16.3.1
Regression Discontinuity Designs
16.3.2
Differences of Differences
16.3.3
Matching
16.4
Graphical Models
16.5
Internal and External Validity
Even randomized experiments have issues. They reveal the treatment effect only for the precise experiment performed. Put differently, if done well,
they enjoy internal validity, but there is no guarantee of external validity.
Even slight differences in experiments can produce different results. For example, there can be large differences between open RCTs and doubleblind RCTs. See http://boringdevelopment.com/2014/04/09/a-torpedoaimed-straight-at-h-m-s-randomista/
282
CHAPTER 16. STOCHASTIC REGRESSORS II: ENDOGENEITY
16.6
1. Moment conditions, GMM, and IV.

2. Big data.
(a) Big Data is typically found data basically digital exhaust.
That is, it is non-experimental, so econometric methods are necessary for estimating causal effects.
(b) Some Big Data, however, is experimental. Google, for example, does
many experiments to see how ad click-through rates depend on ad
placement.
16.7
Notes
Part V
Appendices
283
Appendix A
Probability and Statistics Review
Here we review a few aspects of probability and statistics that we will rely
upon at various times.
A.1
Populations: Random Variables, Distributions and

Moments
A.1.1
Univariate
Consider an experiment with a set O of possible outcomes.
A random
variable Y is simply a mapping from O to the real numbers. For example, the experiment might be flipping a coin twice, in which case O =
{(Heads, Heads), (T ails, T ails), (Heads, T ails), (T ails, Heads)}. We might
define a random variable Y to be the number of heads observed in the two
flips, in which case Y could assume three values, y = 0, y = 1 or y = 2.1
Discrete random variables, that is, random variables with discrete
probability distributions, can assume only a countable number of values
P
yi , i = 1, 2, ..., each with positive probability pi such that i pi = 1 . The
probability distribution f (y) assigns a probability pi to each such value yi .
In the example at hand, Y is a discrete random variable, and f (y) = 0.25 for
1
Note that, in principle, we use capitals for random variables (Y ) and small letters for their realizations
(y). We will often neglect this formalism, however, as the meaning will be clear from context.
285
286
APPENDIX A. PROBABILITY AND STATISTICS REVIEW
y = 0, f (y) = 0.50 for y = 1, f (y) = 0.25 for y = 2, and f (y) = 0 otherwise.

In contrast, continuous random variables can assume a continuous
range of values, and the probability density function f (y) is a nonnegative continuous function such that the area under f (y) between any
points a and b is the probability that Y assumes a value between a and b.2
In what follows we will simply speak of a distribution, f (y). It will
be clear from context whether we are in fact speaking of a discrete random
variable with probability distribution f (y) or a continuous random variable
with probability density f (y).
Moments provide important summaries of various aspects of distributions. Roughly speaking, moments are simply expectations of powers of random variables, and expectations of different powers convey different sorts of
information. You are already familiar with two crucially important moments,
the mean and variance. In what follows well consider the first four moments:
mean, variance, skewness and kurtosis.3
The mean, or expected value, of a discrete random variable is a probabilityweighted average of the values it can assume,4
E(y) =
pi y i .
Often we use the Greek letter to denote the mean, which measures the
location, or central tendency, of y.
The variance of y is its expected squared deviation from its mean,
var(y) = E(y )2 .
We use 2 to denote the variance, which measures the dispersion, or scale,
of y around its mean.
2
In addition, the total area under f (y) must be 1.

In principle, we could of course consider moments beyond the fourth, but in practice only the first four
are typically examined.
4
A similar formula holds in the continuous case.
3
A.1. POPULATIONS: RANDOM VARIABLES, DISTRIBUTIONS AND MOMENTS 287
Often we assess dispersion using the square root of the variance, which is
called the standard deviation,
= std(y) =
p
E(y )2 .
The standard deviation is more easily interpreted than the variance, because
it has the same units of measurement as y. That is, if y is measured in dollars
(say), then so too is std(y). V ar(y), in contrast, would be measured in rather
hard-to-grasp units of dollars squared.
The skewness of y is its expected cubed deviation from its mean (scaled
by 3 for technical reasons),
E(y )3
S=
.
3
Skewness measures the amount of asymmetry in a distribution. The larger
the absolute size of the skewness, the more asymmetric is the distribution.
A large positive value indicates a long right tail, and a large negative value
indicates a long left tail. A zero value indicates symmetry around the mean.
The kurtosis of y is the expected fourth power of the deviation of y from
its mean (scaled by 4 , again for technical reasons),
E(y )4
K=
.
4
Kurtosis measures the thickness of the tails of a distribution. A kurtosis
above three indicates fat tails or leptokurtosis, relative to the normal,
or Gaussian distribution that you studied earlier. Hence a kurtosis above
three indicates that extreme events (tail events) are more likely to occur
than would be the case under normality.
288
A.1.2
Multivariate
Suppose now that instead of a single random variable Y , we have two random
variables Y and X.5 We can examine the distributions of Y or X in isolation,
which are called marginal distributions. This is effectively what weve
already studied. But now theres more: Y and X may be related and therefore
move together in various ways, characterization of which requires a joint
distribution. In the discrete case the joint distribution f (y, x) gives the
probability associated with each possible pair of y and x values, and in the
continuous case the joint density f (y, x) is such that the area in any region
under it gives the probability of (y, x) falling in that region.
We can examine the moments of y or x in isolation, such as mean, variance,
skewness and kurtosis. But again, now theres more: to help assess the
dependence between y and x, we often examine a key moment of relevance
in multivariate environments, the covariance. The covariance between y
and x is simply the expected product of the deviations of y and x from their
respective means,
cov(y, x) = E[(yt y )(xt x )].
A positive covariance means that y and x are positively related; that is, when
y is above its mean x tends to be above its mean, and when y is below its
mean x tends to be below its mean. Conversely, a negative covariance means
that y and x are inversely related; that is, when y is below its mean x tends
to be above its mean, and vice versa. The covariance can take any value in
the real numbers.
Frequently we convert the covariance to a correlation by standardizing
5
We could of course consider more than two variables, but for pedagogical reasons we presently limit
ourselves to two.
A.2. SAMPLES: SAMPLE MOMENTS
289
by the product of y and x ,

corr(y, x) =
cov(y, x)
.
y x
The correlation takes values in [-1, 1]. Note that covariance depends on units
of measurement (e.g., dollars, cents, billions of dollars), but correlation does
not. Hence correlation is more immediately interpretable, which is the reason
for its popularity.
Note also that covariance and correlation measure only linear dependence;
in particular, a zero covariance or correlation between y and x does not necessarily imply that y and x are independent. That is, they may be non-linearly
related. If, however, two random variables are jointly normally distributed
with zero covariance, then they are independent.
Our multivariate discussion has focused on the joint distribution f (y, x).
In various chapters we will also make heavy use of the conditional distribution f (y|x), that is, the distribution of the random variable Y conditional
upon X = x. Conditional moments are similarly important. In particular, the conditional mean and conditional variance play key roles in
econometrics, in which attention often centers on the mean or variance of a
series conditional upon the past.
A.2
A.2.1
Samples: Sample Moments

Univariate
Thus far weve reviewed aspects of known distributions of random variables,

in population. Often, however, we have a sample of data drawn from an
unknown population distribution f ,
{yi }N
i=1 f (y),
290
and we want to learn from the sample about various aspects of f , such as
its moments. To do so we use various estimators.6 We can obtain estimators by replacing population expectations with sample averages, because the
arithmetic average is the sample analog of the population expectation. Such
analog estimators turn out to have good properties quite generally. The
sample mean is simply the arithmetic average,
N
1 X
yi .
y =
N i=1
It provides an empirical measure of the location of y.

The sample variance is the average squared deviation from the sample
mean,
2 =
PN
i=1 (yi
y)2
.
N
It provides an empirical measure of the dispersion of y around its mean.
We commonly use a slightly different version of
2 , which corrects for the
one degree of freedom used in the estimation of y, thereby producing an
unbiased estimator of 2 ,
s2 =
PN
y)2
.
N 1
i=1 (yi
Similarly, the sample standard deviation is defined either as

s
PN
)2
i=1 (yi y
2
=
=
N
or
s=
s
s2
PN
y)2
.
N 1
i=1 (yi
It provides an empirical measure of dispersion in the same units as y.

6
An estimator is an example of a statistic, or sample statistic, which is simply a function of the sample
observations.
A.2. SAMPLES: SAMPLE MOMENTS
291
The sample skewness is

S =
1
N
PN
i=1 (yi
y)3
It provides an empirical measure of the amount of asymmetry in the distribution of y.

The sample kurtosis is
=
K
1
N
PN
i=1 (yi
y)4
It provides an empirical measure of the fatness of the tails of the distribution

of y relative to a normal distribution.
Many of the most famous and important statistical sampling distributions
arise in the context of sample moments, and the normal distribution is the
father of them all. In particular, the celebrated central limit theorem establishes that under quite general conditions the sample mean y will have a
normal distribution as the sample size gets large. The 2 distribution arises
from squared normal random variables, the t distribution arises from ratios
of normal and 2 variables, and the F distribution arises from ratios of
2 variables. Because of the fundamental nature of the normal distribution
as established by the central limit theorem, it has been studied intensively,
a great deal is known about it, and a variety of powerful tools have been
developed for use in conjunction with it.
A.2.2
Multivariate
We also have sample versions of moments of multivariate distributions. In

particular, the sample covariance is
N
1 X
[(yi y)(xi x)],
cov(y,
c
x) =
N i=1
292
and the sample correlation is

corr(y,
d
x) =
A.3
cov(y,
c
x)
.
y
x
Finite-Sample and Asymptotic Sampling Distributions of the Sample Mean
Here we refresh your memory on the sampling distribution of the most important sample moment, the sample mean.
A.3.1
Exact Finite-Sample Results
In your earlier studies you learned about statistical inference, such as how
to form confidence intervals for the population mean based on the sample
mean, how to test hypotheses about the population mean, and so on. Here
we partially refresh your memory.
Consider the benchmark case of Gaussian simple random sampling,
yi iid N (, 2 ), i = 1, ..., N,
which corresponds to a special case of what we will later call the full ideal
conditions for regression modeling. The sample mean y is the natural estimator of the population mean . In this case, as you learned earlier, y is
unbiased, consistent, normally distributed with variance 2 /N , and indeed
the minimum variance unbiased (MVUE) estimator. We write

2
y N ,
,
N
or equivalently
N (
y ) N (0, 2 ).
A.3. FINITE-SAMPLE AND ASYMPTOTIC SAMPLING DISTRIBUTIONS OF THE SAMPLE MEAN2
We construct exact finite-sample confidence intervals for as

s
y t1 2 (N 1)
w.p. ,
N
where t1 2 (N 1) is the 1
percentile of a t distribution with N 1
degrees of freedom. Similarly, we construct exact finite-sample (likelihood

ratio) hypothesis tests of H0 : = 0 against the two-sided alternative
H0 : 6= 0 using
y 0
s
N
A.3.2
t1 2 (N 1).
Approximate Asymptotic Results (Under Weaker Assumptions)
Much of statistical inference is linked to large-sample considerations, such

as the law of large numbers and the central limit theorem, which you also
studied earlier. Here we again refresh your memory.
Consider again a simple random sample, but without the normality assumption,
yi iid(, 2 ), i = 1, ..., N.
Despite our dropping the normality assumption we still have that y is unbiased, consistent, asymptotically normally distributed with variance 2 /N ,
and best linear unbiased (BLUE). We write,
a

2
y N ,
.
N
More precisely, as T ,
N (
y ) d N (0, 2 ).
294
This result forms the basis for asymptotic inference. It is a Gaussian central
limit theorem, and it also has a law of large numbers (
y p ) imbedded
within it.
We construct asymptotically-valid confidence intervals for as

y z1 2
w.p. ,
N
where z1 2 is the 1
percentile of a N (0, 1) distribution. Similarly, we
construct asymptotically-valid hypothesis tests of H0 : = 0 against the

two-sided alternative H0 : 6= 0 using
y 0
A.4
N (0, 1).
1. (Interpreting distributions and densities)

The Sharpe Pencil Company has a strict quality control monitoring program. As part of that program, it has determined that the distribution
of the amount of graphite in each batch of one hundred pencil leads
produced is continuous and uniform between one and two grams. That
is, f (y) = 1 for y in [1, 2], and zero otherwise, where y is the graphite
content per batch of one hundred leads.
a. Is y a discrete or continuous random variable?
b. Is f (y) a probability distribution or a density?
c. What is the probability that y is between 1 and 2? Between 1 and
1.3? Exactly equal to 1.67?
d. For high-quality pencils, the desired graphite content per batch is 1.8
grams, with low variation across batches. With that in mind, discuss
the nature of the density f (y).
A.4. EXERCISES, PROBLEMS AND COMPLEMENTS
295
2. (Covariance and correlation)

Suppose that the annual revenues of worlds two top oil producers have
a covariance of 1,735,492.
a. Based on the covariance, the claim is made that the revenues are
very strongly positively related. Evaluate the claim.
b. Suppose instead that, again based on the covariance, the claim is
made that the revenues are positively related. Evaluate the claim.
c. Suppose you learn that the revenues have a correlation of 0.93. In
light of that new information, re-evaluate the claims in parts a and b
above.
3. (Simulation)
You will often need to simulate data from various models. The simplest
model is the iidN (, 2 ) (Gaussian simple random sampling) model.
a. Using a random number generator, simulate a sample of size 30 for
y, where y iidN (0, 1).
b. What is the sample mean? Sample standard deviation? Sample skewness? Sample kurtosis? Discuss.
c. Form an appropriate 95 percent confidence interval for E(y).
d. Perform a t test of the hypothesis that E(y) = 0.
e. Perform a t test of the hypothesis that E(y) = 1.
4. (Sample moments of the CPS wage data)
Use the 1995 CPS wage dataset.
a. Calculate the sample mean wage and test the hypothesis that it equals
$9/hour.
b. Calculate sample skewness.
296
c. Calculate and discuss the sample correlation between wage and years
of education.
A.5
Notes
Numerous good introductory probability and statistics books exist. Wonnacott and Wonnacott (1990) remains a time-honored classic, which you may
wish to consult to refresh your memory on statistical distributions, estimation and hypothesis testing. Anderson et al. (2008) is a well-written recent
text.
Appendix B
Construction of the Wage Datasets
We construct our datasets from randomly sampling the much-larger Current
Population Survey (CPS) datasets.1
We extract the data from the March CPS for 1995, 2004 and 2012 respectively, using the National Bureau of Economic Research (NBER) front end
(http://www.nber.org/data/cps.html) and NBER SAS, SPSS, and Stata
data definition file statements (http://www.nber.org/data/cps_progs.html).
We use both personal and family records. Here we focus our discussion on
1995.
There are many CPS observations for which earnings data are completely
missing. We drop those observations, as well as those that are not in the
universe for the eligible CPS earning items ( ERNEL=0), leaving 14363 observations. From those, we draw a random unweighted subsample with ten
percent selection probability. This results in 1348 observations.
We use seven variables. From the CPS we obtain AGE (age), FEMALE
(1 if female, 0 otherwise), NONWHITE (1 if nonwhite, 0 otherwise), and
UNION (1 if union member, 0 otherwise). We also create EDUC (years
of schooling) based on CPS variable PEEDUCA (educational attainment).
Because the CPS does not ask about years of experience, we create EXPER
1
See http://aspe.hhs.gov/hsp/06/catalog-ai-an-na/cps.htm for a brief and clear introduction to

the CPS datasets.
297
298
APPENDIX B. CONSTRUCTION OF THE WAGE DATASETS
(potential working experience) as AGE minus EDUC minus 6.

We construct the variable WAGE as follows. WAGE equals PRERNHLY
(earnings per hour) in dollars for those paid hourly. For those not paid hourly
(PRERNHLY=0), we use PRERNWA (gross earnings last week) divided by
PEHRUSL1 (usual working hours per week). That sometimes produces missing values, which we treat as missing earnings and drop from the sample.
The final dataset contains 1323 observations with AGE, FEMALE, NONWHITE, UNION, EDUC, EXPER and WAGE.
299
Variable
Age
Labor force status
Name (95) Name (04,12)

PEAGE
A AGE
A LFSR
Class of worker
A CLSWKR
Selection Criteria
18-65
1 working (we exclude armed
forces)
1,2,3,4 (we exclude selfemployed and pro bono)
CPS Personal Data Selection Criteria
300
Variable
PEAGE (A AGE)
A LFSR
A CLSWKR
PEEDUCA (A HGA)
PERACE (PRDTRACE)
PESEX (A SEX)
PEERNLAB (A UNMEM)
PRERNWA (A GRSWK)
PEHRUSL1 (A USLHRS)
PEHRACTT (A HRS1)
PRERNHLY (A HRSPAY)
Description
Age
Labor force status
Class of worker
Educational attainment
RACE
SEX
UNION
Usual earnings per week
Usual hours worked weekly
Hours worked last week
Earnings per hour
AGE
Equals PEAGE
FEMALE
Equals 1 if PESEX=2, 0 otherwise
NONWHITE
Equals 0 if PERACE=1, 0 otherwise
UNION
Equals 1 if PEERNLAB=1, 0 otherwise
EDUC
Refers to the Table
EXPER
Equals AGE-EDUC-6
WAGE
Equals PRERNHLY or PRERNWA/ PEHRUSL1
NOTE: Variable names in parentheses are for 2004 and 2012.
Variable List
301
EDUC
0
1
5
7
9
10
11
12
12
12
14
14
16
18
PEEDUCA
(A HGA)
31
32
33
34
35
36
37
38
39
40
41
42
43
44
20
45
20
46
Description
Less than first grade
Frist, second, third or four grade
Fifth or sixth grade
Seventh or eighth grade
Ninth grade
Tenth grade
Eleventh grade
Twelfth grade no diploma
High school graduate
Some college but no degree
Associate degree-occupational/vocational
Associate degree-academic program
Bachelor degree (B.A., A.B., B.S.)
Master degree (M.A., M.S., M.Eng., M.Ed., M.S.W.,
M.B.A.)
Professional school degree (M.D., D.D.S., D.V.M.,
L.L.B., J.D.)
Doctorate degree (Ph.D., Ed.D.)
Definition of EDUC
302
Appendix C
Some Popular Books Worth
Encountering
I have cited many of these books elsewhere, typically in various end-of-chapter
complements. Here I list them collectively.
Lewis (2003) [Michael Lewis, Moneyball ]. Appearances may lie, but the
numbers dont, so pay attention to the numbers.
Gladwell (2000) [Malcolm Gladwell, The Tipping Point]. Nonlinear phenomena are everywhere.
Gladwell pieces together an answer to the puzzling question of why certain
things take off whereas others languish (products, fashions, epidemics, etc.)
More generally, he provides deep insights into nonlinear environments, in
which small changes in inputs can lead to small changes in outputs under
some conditions, and to huge changes in outputs under other conditions.
Taleb (2007) [Nassim Nicholas Taleb, The Black Swan] Warnings, and
more warnings, and still more warnings, about non-normality and much else.
See Chapter 7 EPC 1.
Angrist and Pischke (2009) [Joshua Angrist and Jorn-Steffen Pischke,
Mostly Harmless Econometrics]. Natural and quasi-natural experiments
suggesting instruments.
This is a fun and insightful treatment of instrumental-variables and related
303
304
APPENDIX C. SOME POPULAR BOOKS WORTH ENCOUNTERING
methods. Just dont be fooled by the books attempted landgrab, as discussed

in a 2015 No Hesitations post.
Silver (2012) [Nate Silver, The Signal and the Noise]. Pitfalls and opportunities in predictive modeling.
Bibliography
Anderson, D.R., D.J. Sweeney, and T.A. Williams (2008), Statistics for Business and Economics, South-Western.
Angrist, J.D. and J.-S. Pischke (2009), Mostly Harmless Econometrics,
Princeton University Press.
Gladwell, M. (2000), The Tipping Point, Little, Brown and Company.
Harvey, A.C. (1991), Forecasting, Structural Time Series Models and the
Kalman Filter, Cambridge University Press.
Jarque, C.M. and A.K. Bera (1987), A Test for Normality of Observations
and Regression Residuals, International Statistical Review , 55, 163172.
Kiefer, N. and M. Salmon (1983), Testing Normality in Econometric Models, Economic Letters, 11, 123127.
Koenker, R. (2005), Quantile Regression, Econometric Society Monograph
Series, Cambridge University Press, 2005.
Lewis, M. (2003), Moneyball, Norton.
Nerlove, M., D.M. Grether, and J.L. Carvalho (1979), Analysis of Economic
Time Series: A Synthesis. New York: Academic Press. Second Edition.
Silver, N.. (2012), The Signal and the Noise, Penguin Press.
Taleb, N.N. (2007), The Black Swan, Random House.
305
306
BIBLIOGRAPHY
Tufte, E.R. (1983), The Visual Display of Quantatative Information, Chesire:

Graphics Press.
Wonnacott, T.H. and R.J. Wonnacott (1990), Introductory Statistics. New
York: John Wiley and Sons, Fifth Edition.
Index
F distribution, 34
Banking to 45 degrees, 19
F -statistic, 54
Binary data, 5
R-squared, 55
binomial logit, 336
s-squared, 55
Box-Cox transformation, 109
t distribution, 34
Box-Pierce Q-statistic, 163
t-statistic, 51
Breusch-Godfrey test, 178
2 distribution, 34
Calendar effects, 76
Fitted values, 44
Central tendency, 30
Holiday variation, 77
Chartjunk, 19
Seasonal dummy variables, 75
Cointegration, 280
Adjusted R-squared, 56
Akaike information criterion, 56
Analog principle, 161
Analysis of variance, 80
AR(p) process, 171
ARCH(p) process, 289
Aspect ratio, 19
Asymmetric response, 295
Asymmetry, 31
Asymptototic, 35
Autocorrelation function, 156
Autocovariance function, 154
Autoregressions, 156
Autoregressive (AR) model, 166
Common scales, 19
Conditional distribution, 32
Conditional expectation, 47
Conditional mean, 32
Conditional mean and variance, 160
Conditional mean function, 97
Conditional moment, 32
Conditional variance, 32
Constant term, 51
Continuous data, 5
Continuous random variable, 30
Correlation, 32
Correlogram, 162
Correlogram analysis, 164
307
308
Covariance, 31
Covariance stationary, 154
Cross correlation function, 242
Cross sectional data, 5
Cross sections, 6
Cross-variable dynamics, 235
CUSUM, 137
CUSUM plot, 137
Cycles, 153
INDEX
Durbin-Watson statistic, 57
Econometric modeling, 3
Error-correction, 281
Estimator, 33
Ex post smoothing, 119
Expected value, 30
Exploratory data analysis, 23
Exponential GARCH, 296
Exponential smoothing, 261
Data mining, 57
Exponential trend, 115
Data-generating process (DGP), 48
Exponentially weighted moving aver-
De-trending, 81
Deterministic seasonality, 74
Deterministic trend, 72, 249
Dickey-Fuller distribution, 255
Discrete probability distribution, 29
Discrete random variable, 29
age, 261
Feedback, 243
Financial econometrics, 286
First-order serial correlation, 178
Fourier series expansions, 123
Functional form, 107
Dispersion, 30
Distributed lag, 165
GARCH(p,q) process, 290
Distributed lag model, 222
Gaussian distribution, 31
Distributed lag regression model with Gaussian white noise, 158

lagged dependent variables, 233 Generalized linear model, 109, 337
Distributed-lag regression model with GLM, 109, 337
Golden ratio, 23
AR disturbances, 233
Disturbance, 47
Goodness of fit, 56
Dummy left-hand-side variable, 331
Heteroskedasticity, 285
Dummy right-hand-side variable, 331 Histogram, 15

Dummy variable, 69
Hodrick-Prescott filtering, 119
Durbins h test, 191
Holt-Winters Smoothing, 263
INDEX
309
Holt-Winters Smoothing with Season- Location, 30

ality, 264
Impulse-response function, 238
In-sample overfitting, 57
Independent white noise, 158
Indicator variable, 69, 331
Innovation outliers, 190
Instrumental variables, 349
Log-lin regression, 108, 109

Log-linear trend, 115
Log-log regression, 107
Logistic function, 332
Logistic model, 110
Logistic trend, 130
Logit model, 332
Integrated, 248
Marginal distribution, 31
Interaction effects, 111, 122
Markov-switching model, 140
Intercept, 72
Maximum likelihood estimation, 53
Intercept dummies, 70
Mean, 30
Interval data, 9
Measurement outliers, 190
Intrinsically non-linear models, 110
Model selection, 319
Jarque-Bera test, 100

Joint distribution, 31
Kurtosis, 31
Lag operator, 165
Least absolute deviations, 103
Least squares, 41
Moments, 30, 160

Multinomial logit, 338
Multiple comparisons, 18
Multiple linear regression, 44
Multivariate, 13
Multivariate GARCH, 317
Multiway scatterplot, 16
Leptokurtosis, 31
Neural networks, 126
Likelihood function, 53
Nominal data, 9
Limited dependent variable, 331
Non-data ink, 19
Linear probability model, 332
non-linear least squares (NLS), 110
Linear projection, 97, 343
Non-linearity, 97
Linear trend, 72
Non-normality, 97
Link function, 109, 338
Normal distribution, 31
Ljung-Box Q-statistic, 163
Normal white noise, 158
310
INDEX
Odds, 336
QQ plots, 99
Off-line smoothing, 119
Quadratic trend, 117
On-line smoothing, 119

One-sided moving average, 119
One-sided weighted moving average,
119
Ordered logit, 334
Ordered outcomes, 333
Ordinal data, 9
Outliers, 97
Random number generator, 189

Random walk, 137, 248
Random walk with drift, 249
Ratio data, 9
Real-time smoothing, 119
Realization, 154
Recursive residuals, 136
Regime switching, 139
Panel data, 5
Regression function, 47
Panels, 6
Regression intercept, 47
Parameter instability, 136
Regression on seasonal dummies, 75
Parameters, 47
Regression slope coefficients, 47
Partial autocorrelation function, 156
Relational graphics, 15
Partial correlation, 21
RESET test, 112
Polynomial distributed lag, 222
Residual plot, 59
Polynomial in the lag operator, 165
Residual scatter, 58
Polynomial trend, 117
Residuals, 44
Population, 32
Robustness iteration, 151
Population model, 47
Sample, 32
Population regression, 156
Sample autocorrelation function, 162
Positive serial correlation, 57
Sample correlation, 34
Predictive causality, 237
Sample covariance, 34
Prob(F -statistic), 54
Sample kurtosis, 33
Probability density function, 30
Sample mean, 33, 162
Probability value, 52
Sample mean of the dependent vari-
Probit model, 337

Proportional odds, 334
able, 53
Sample partial autocorrelation, 164
INDEX
311
Sample path, 154
Statistic, 33
Sample skewness, 33
Stochastic processes, 166
Sample standard deviation, 33
Stochastic seasonality, 74
Sample standard deviation of the de- Stochastic trend, 72, 249

pendent variable, 53
Strong white noise, 158
Sample statistic, 33
Students-t GARCH, 317
Sample variance, 33
Sum of squared residuals, 53
Scale, 30
Superconsistency, 253
Scatterplot matrix, 16
Schwarz information criterion, 57
Seasonal adjustment, 81
Seasonality, 72, 74
Second-order stationarity, 155
Serial correlation, 57
Serially uncorrelated, 158
Simple correlation, 21
Simple exponential smoothing, 261
Simple random sampling, 34
Simulating time series processes, 189
Single exponential smoothing, 261
Skewness, 31
Slope, 72
Slope dummies, 78
Taylor series expansions, 121

Threshold GARCH, 295
Threshold model, 139
Time dummy, 72
Time series, 6, 154
Time series data, 5
Time series of cross sections, 5
Time series plot, 13
Time series process, 157
Time-varying volatility, 285
Tobit model, 338
Trading-day variation, 77
Trend, 72
Two-sided moving average, 119
Smoothing, 118
Unconditional mean and variance, 160
Spurious regression, 282
Unit autoregressive root, 247
Standard deviation, 30
Unit root, 247
Standard error of the regression, 55
Univariate, 13
Standard errors, 51
Standardized recursive residuals, 137
Variance, 30
Vector autoregression of order p, 235
312
Volatility clustering, 289

Volatility dynamics , 289
Weak stationarity, 155
Weak white noise, 158
White noise, 158
Yule-Walker equation, 169
Zero-mean white noise, 158
INDEX

Econometric S

Uploaded by

Copyright:

Available Formats

Econometric S

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Econometric S

Uploaded by

Copyright:

Available Formats

Econometrics

Streamlined, Applied and e-Aware

Brief Table of Contents

About the Cover

2 Graphics and Graphical Style

3 Regression Under the Full Ideal Conditions

5 Stochastic Regressors I: Measurement Error, Omitted Variables, and Multicollinearity

BRIEF TABLE OF CONTENTS

9 Indicator Variables in Time Series: Trend and Seasonality 123

11 Serial Correlation in Observed Time Series

12 Serial Correlation in Time Series Regression

13 Heteroskedasticity in Time Series

14 Binary Regression and Classification

16 Stochastic Regressors II: Endogeneity

A Probability and Statistics Review

B Construction of the Wage Datasets

C Some Popular Books Worth Encountering

Detailed Table of Contents

About the Cover

DETAILED TABLE OF CONTENTS

3 Regression Under the Full Ideal Conditions

DETAILED TABLE OF CONTENTS

3.4.16 A Bit More on AIC and SIC . . . . .

5 Stochastic Regressors I: Measurement Error, Omitted Variables, and Multicollinearity

DETAILED TABLE OF CONTENTS

6 Indicator Variables in Cross Sections: Group Heterogeneity

7 Non-Linear Functional Form in Cross Sections

5.5.1 Perfect Multicollinearity . . . .

8 Heteroskedasticity and Clustering in Cross Sections

DETAILED TABLE OF CONTENTS

9 Indicator Variables in Time Series: Trend and Seasonality

10 Non-Linearity and Structural Change in Time Series

11 Serial Correlation in Observed Time Series

DETAILED TABLE OF CONTENTS

11.3.3 Sample Partial Autocorrelations . . . . . . . . . . . . .

12 Serial Correlation in Time Series Regression

13 Heteroskedasticity in Time Series

DETAILED TABLE OF CONTENTS

14 Binary Regression and Classification

16 Stochastic Regressors II: Endogeneity

DETAILED TABLE OF CONTENTS

A Probability and Statistics Review

B Construction of the Wage Datasets

C Some Popular Books Worth Encountering

About the Author

Francis X. Diebold is Professor of Economics, Finance and Statistics at the

About the Cover

Hyperlinks to internal items (table of contents, index, footnotes, etc.)

Resources for Economists Web

1-Year Goverment Bond Yield, Levels and Changes . . . . . .

Distributions of Log Wage, Education and Experience . .

Histograms for Wage Covariates . . . . . . . . . . . . . . . . .

Basic Linear Wage Regression . . . . . . . . . . . . . . . . . .

Various Linear Trends . . . . . . . . . . . . . . . . . . . .

Various Exponential Trends . . . . . . . . . . . . . . . . . . .

Time Series of Daily NYSE Returns. . . . . . . . .

13.7 Estimated Conditional Standard Deviation, Daily NYSE Returns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Third, although Econometrics is designed for a first course in econometrics

as the only prerequisite is an introductory probability and statistics course.