Elements of econometrics

C. Dougherty
EC2020
2016

Undergraduate study in
Economics, Management,
Finance and the Social Sciences

This is an extract from a subject guide for an undergraduate course offered as part of the
University of London International Programmes in Economics, Management, Finance and
the Social Sciences. Materials for these programmes are developed by academics at the
London School of Economics and Political Science (LSE).
For more information, see: www.londoninternational.ac.uk
This guide was prepared for the University of London International Programmes by:
Dr. C. Dougherty, Senior Lecturer, Department of Economics, London School of Economics
and Political Science.
With typesetting and proof-reading provided by:
James S. Abdey, BA (Hons), MSc, PGCertHE, PhD, Department of Statistics, London School
of Economics and Political Science.
This is one of a series of subject guides published by the University. We regret that due
to pressure of work the author is unable to enter into any correspondence relating to, or
arising from, the guide. If you have any comments on this subject guide, favourable or
unfavourable, please use the form at the back of this guide.

University of London International Programmes

Publications Office
Stewart House
32 Russell Square
London WC1B 5DN
United Kingdom
www.londoninternational.ac.uk

Published by: University of London

University of London 2011
Reprinted with minor revisions 2016
The University of London asserts copyright over all material in this subject guide except
where otherwise indicated. All rights reserved. No part of this work may be reproduced
in any form, or by any means, without permission in writing from the publisher. We make
every effort to respect copyright. If you think we have inadvertently used your copyright
material, please let us know.
 Contents

Contents

Preface 1
0.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
0.2 What is econometrics, and why study it? . . . . . . . . . . . . . . . . . . 1
0.3 Aims . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
0.4 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
0.5 How to make use of the textbook . . . . . . . . . . . . . . . . . . . . . . 3
0.6 How to make use of this subject guide . . . . . . . . . . . . . . . . . . . 3
0.7 How to make use of the website . . . . . . . . . . . . . . . . . . . . . . . 4
0.7.1 Slideshows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
0.7.2 Data sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
0.8 Online study resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
0.8.1 The VLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
0.8.2 Making use of the Online Library . . . . . . . . . . . . . . . . . . 6
0.9 Prerequisite for studying this subject . . . . . . . . . . . . . . . . . . . . 6
0.10 Application of linear algebra to econometrics . . . . . . . . . . . . . . . . 7
0.11 The examination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
0.12 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
0.13 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
0.14 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
0.15 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 11
0.16 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 22

1 Simple regression analysis 27

1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.4 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 30
1.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 35

2 Properties of the regression coefficients and hypothesis testing 41

2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

i
Contents

2.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.3 Further material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.4 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.5 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 48
2.6 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 53

3 Multiple regression analysis 59

3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.4 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 63
3.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 64

4 Transformations of variables 69
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.3 Further material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.4 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.5 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 74
4.6 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 77

5 Dummy variables 85
5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.4 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 94
5.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 100

6 Specification of regression variables 115

6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
6.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
6.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.4 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 123
6.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 129

7 Heteroskedasticity 145
7.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
7.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

ii
 Contents

7.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

7.4 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 152
7.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 159

8 Stochastic regressors and measurement errors 169

8.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
8.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
8.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
8.4 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 172
8.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 180

9 Simultaneous equations estimation 185

9.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
9.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
9.3 Further material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
9.4 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
9.5 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 194
9.6 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 199

10 Binary choice and limited dependent variable models, and maximum

likelihood estimation 213
10.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
10.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
10.3 Further material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
10.4 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
10.5 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 225
10.6 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 231

11 Models using time series data 239

11.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
11.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
11.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
11.4 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 245
11.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 250

12 Properties of regression models with time series data 261

12.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
12.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

iii
Contents

12.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

12.4 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 269
12.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 273

13 Introduction to nonstationary time series 285

13.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
13.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
13.3 Further material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
13.4 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
13.5 Answers to the starred exercises in the textbook . . . . . . . . . . . . . . 291
13.6 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 295

14 Introduction to panel data 299

14.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
14.2 Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
14.3 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
14.4 Answer to the starred exercise in the textbook . . . . . . . . . . . . . . . 304
14.5 Answers to the additional exercises . . . . . . . . . . . . . . . . . . . . . 306

15 Regression analysis with linear algebra primer 313

15.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
15.2 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
15.3 Test exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
15.4 The multiple regression model . . . . . . . . . . . . . . . . . . . . . . . . 314
15.5 The intercept in a regression model . . . . . . . . . . . . . . . . . . . . . 315
15.6 The OLS regression coefficients . . . . . . . . . . . . . . . . . . . . . . . 316
15.7 Unbiasedness of the OLS regression coefficients . . . . . . . . . . . . . . . 317
15.8 The variance-covariance matrix of the OLS regression coefficients . . . . 317
15.9 The GaussMarkov theorem . . . . . . . . . . . . . . . . . . . . . . . . . 319
15.10 Consistency of the OLS regression coefficients . . . . . . . . . . . . . . 319
15.11 FrischWaughLovell theorem . . . . . . . . . . . . . . . . . . . . . . . 320
15.12 Exact multicollinearity . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
15.13 Estimation of a linear combination of regression coefficients . . . . . . . 324
15.14 Testing linear restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . 325
15.15 Weighted least squares and heteroskedasticity . . . . . . . . . . . . . . 325
15.16 IV estimators and TSLS . . . . . . . . . . . . . . . . . . . . . . . . . . 327
15.17 Generalised least squares . . . . . . . . . . . . . . . . . . . . . . . . . . 329

iv
 Contents

15.18 Appendix A: Derivation of the normal equations . . . . . . . . . . . . . 330

15.19 Appendix B: Demonstration that u b0 u
b/(n k) is an unbiased estimator
2
of u . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
15.20 Appendix C: Answers to the exercises . . . . . . . . . . . . . . . . . . . 334

A Syllabus for the EC2020 Elements of econometrics examination 341

A.1 Review: Random variables and sampling theory . . . . . . . . . . . . . . 341
A.2 Chapter 1 Simple regression analysis . . . . . . . . . . . . . . . . . . . . 341
A.3 Chapter 2 Properties of the regression coefficients . . . . . . . . . . . . . 342
A.4 Chapter 3 Multiple regression analysis . . . . . . . . . . . . . . . . . . . 342
A.5 Chapter 4 Transformation of variables . . . . . . . . . . . . . . . . . . . 343
A.6 Chapter 5 Dummy variables . . . . . . . . . . . . . . . . . . . . . . . . . 343
A.7 Chapter 6 Specification of regression variables . . . . . . . . . . . . . . . 343
A.8 Chapter 7 Heteroskedasticity . . . . . . . . . . . . . . . . . . . . . . . . . 343
A.9 Chapter 8 Stochastic regressors and measurement errors . . . . . . . . . 344
A.10 Chapter 9 Simultaneous equations estimation . . . . . . . . . . . . . . . 344
A.11 Chapter 10 Binary choice models and maximum likelihood estimation . . 344
A.12 Chapter 11 Models using time series data . . . . . . . . . . . . . . . . . . 345
A.13 Chapter 12 Autocorrelation . . . . . . . . . . . . . . . . . . . . . . . . . 345
A.14 Chapter 13 Introduction to nonstationary processes . . . . . . . . . . . . 346

v
Contents

vi
Preface

0.1 Introduction

0.2 What is econometrics, and why study it?

Econometrics is the application of statistical methods to the quantification and critical
assessment of hypothetical economic relationships using data. It is with the aid of
econometrics that we discriminate between competing economic theories and put
numerical clothing onto the successful ones. Econometric analysis may be motivated by
a simple desire to improve our understanding of how the economy works, at either the
microeconomic or macroeconomic level, but more often it is undertaken with a specific
objective in mind. In the private sector, the financial benefits that accrue from a
sophisticated understanding of relevant markets and an ability to predict change may
be the driving factor. In the public sector, the impetus may come from an awareness
that evidence-based policy initiatives are likely to have the greatest impact.
It is now generally recognised that nearly all professional economists, not just those
actually working with data, should have a basic understanding of econometrics. There
are two major benefits. One is that it facilitates communication between
econometricians and the users of their work. The other is the development of the ability
to obtain a perspective on econometric work and to make a critical evaluation of it.
Econometric work is more robust in some contexts than in others. Experience with the
practice of econometrics and a knowledge of the potential problems that can arise are
essential for developing an instinct for judging how much confidence should be placed
on the findings of a particular study.
Such is the importance of econometrics that, in common with intermediate
macroeconomics and microeconomics, an introductory course forms part of the core of
any serious undergraduate degree in economics and is a prerequisite for admission to a
serious Masters level course in economics or finance.

0.3 Aims
The aim of EC2020 Elements of econometrics is to give you an opportunity to
develop an understanding of econometrics to a standard that will equip you to
understand and evaluate most applied analysis of cross-sectional data and to be able to
undertake such analysis yourself. The restriction to cross-sectional data (data raised at
one moment in time, often through a survey of households, individuals, or enterprises)
should be emphasised because the analysis of time series data (observations on a set of
variables over a period of time) is much more complex. Chapters 11 to 13 of the
textbook, Introduction to econometrics, and this subject guide are devoted to the

1
Preface

analysis of time series data, but, beyond very simple applications, the objectives are
confined to giving you an understanding of the problems involved and making you
aware of the need for a Masters level course if you intend to work with such data.
Specifically the aims of the course are to:

develop an understanding of the use of regression analysis and related techniques

for quantifying economic relationships and testing economic theories

equip you to read and evaluate empirical papers in professional journals

provide you with practical experience of using mainstream regression programmes

to fit economic models.

0.4 Learning outcomes

By the end of this course, and having completed the Essential reading and activities,
you should be able to:

describe and apply the classical regression model and its application to
cross-sectional data

describe and apply the:

GaussMarkov conditions and other assumptions required in the application of
the classical regression model
reasons for expecting violations of these assumptions in certain circumstances
tests for violations
potential remedial measures, including, where appropriate, the use of
instrumental variables

recognise and apply the advantages of logit, probit and similar models over
regression analysis when fitting binary choice models

competently use regression, logit and probit analysis to quantify economic

relationships using standard regression programmes (Stata and EViews) in simple
applications

describe and explain the principles underlying the use of maximum likelihood
estimation

apply regression analysis to fit time-series models using stationary time series, with
awareness of some of the econometric problems specific to time-series applications
(for example, autocorrelation) and remedial measures

recognise the difficulties that arise in the application of regression analysis to

nonstationary time series, know how to test for unit roots, and know what is meant
by cointegration.

2
 0.5. How to make use of the textbook

0.5 How to make use of the textbook

The only reading required for this course is my textbook:

C. Dougherty, Introduction to econometrics (Oxford: Oxford University Press,

2016) fifth edition [ISBN 9780199676828].
The syllabus is the same as that for EC220 Introduction to econometrics, the
corresponding internal course at the London School of Economics. The textbook has
been written to cover it with very little added and nothing subtracted.
When writing a textbook, there is a temptation to include a large amount of non-core
material that may potentially be of use or interest to students. There is much to be said
for this, since it allows the same textbook to be used to some extent for reference as
well as a vehicle for a taught course. However, my textbook is stripped down to nearly
the bare minimum for two reasons. First, the core material provides quite enough
content for an introductory year-long course and I think that students should initially
concentrate on gaining a good understanding of it. Second, if the textbook is focused
narrowly on the syllabus, students can read through it as a continuous narrative
without a need for directional guidance. Obviously, this is particularly important for
those who are studying the subject on their own, as is the case for most of those
enrolled on EC2020 Elements of econometrics.
An examination syllabus is provided as an appendix to this subject guide, but its
function is mostly to indicate the expected depth of understanding of each topic, rather
than the selection of the topics themselves.

0.6 How to make use of this subject guide

The function of this subject guide differs from that of other subject guides you may be
using. Unlike those for other courses, this subject guide acts as a supplementary
resource, with the textbook as the main resource. Each chapter forms an extension to a
corresponding chapter in the textbook with the same title. You must have a copy of the
textbook to be able to study this course. The textbook will give you the information you
need to carry out the activities and achieve the learning outcomes in the subject guide.
The main purpose of the subject guide is to provide you with opportunities to gain
experience with econometrics through practice with exercises. Each chapter of the
subject guide falls into two parts. The first part begins with an overview of the
corresponding chapter in the textbook. Then there is a checklist of learning outcomes
anticipated as a result of studying the chapter in the textbook, doing the exercises in
the subject guide, and making use of the corresponding resources on the website.
Finally, in some of the chapters, comes a section headed Further material. This
consists of new topics that may be included in the next edition of the textbook. The
second part of each chapter consists of additional exercises, followed by answers to the
starred exercises in the text and answers to the additional exercises.
You should organise your studies in the following way:

first read this introductory chapter

3
Preface

read the Overview section from the Review chapter of the subject guide

read the Review chapter of the textbook and do the starred exercises

refer to the subject guide for answers to the starred exercises in the text and for
additional exercises

check that you have covered all the items in the learning outcomes section in the
subject guide.

You should repeat this process for each of the numbered chapters. Note that the subject
guide chapters have the same titles as the chapters in the text. In those chapters where
there is a Further material section in the subject guide, this should be read after
reading the chapter in the textbook.

0.7 How to make use of the website

You should make full use of the resources available at the Online Resource Centre
maintained by the publisher, Oxford University Press (OUP):
www.oup.com/uk/orc/bin/9780199567089. Here you will find PowerPoint slideshows
that provide a graphical treatment of the topics covered in the textbook, data sets for
practical work and statistical tables.

0.7.1 Slideshows
In principle you will be able to acquire mastery of the subject by studying the contents
of the textbook with the support of this subject guide and doing the exercises
conscientiously. However, I strongly recommend that you do study all the slideshows as
well. Some do not add much to the material in the textbook, and these you can skim
through quickly. Some, however, provide a much more graphical treatment than is
possible with print and they should improve your understanding. Some present and
discuss regression results and other hands-on material that could not be included in the
text for lack of space, and they likewise should be helpful.

0.7.2 Data sets

To use the data sets, you must have access to a proper statistics application with
facilities for regression analysis, such as Stata or EViews. The student versions of such
applications are adequate for doing all, or almost all, the exercises and of course are
much cheaper than the professional ones. Product and pricing information can be
obtained from the applications websites, the URL usually being the name of the
application sandwiched between www. and .com.
If you do not have access to a commercial econometrics application, you should use
gretl. This is a sophisticated application almost as powerful as the commercial ones, and
it is free. See the gretl manual on the OUP website for further information.
Whatever you do, do not be tempted to try to get by with the regression engines built
into some spreadsheet applications, such as Microsoft Excel. They are not remotely

4
 0.8. Online study resources

adequate for your needs.

There are three major data sets on the website. The most important one, for the
purposes of this subject guide, is the Consumer Expenditure Survey (CES ) data set.
You will find on the website versions in the formats used by Stata, EViews and gretl. If
you are using some other application, you should download the text version
(comma-delimited ASCII) and import it. Answers to all of the exercises are provided in
the relevant chapters of this subject guide.
The exercises for the CES data set cover Chapters 110 of the text. For Chapters
1113, you should use the Demand Functions data set, another major data set, to do
the additional exercises in the corresponding chapters of this subject guide. Again you
should download the data set in the appropriate format. For these exercises, also,
answers are provided.
The third major data set on the website is the Educational Attainment and Earnings
Function data set, which provides practical work for the first 10 chapters of the text
and Chapter 14. No answers are provided, but many parallel examples will be found in
the text.

0.8 Online study resources

In addition to the subject guide and the Essential reading, it is crucial that you take
advantage of the study resources that are available online for this course, including the
VLE and the Online Library.
You can access the VLE, the Online Library and your University of London email
account via the Student Portal at: http://my.londoninternational.ac.uk
You should have received your login details for the Student Portal with your official
offer, which was emailed to the address that you gave on your application form. You
have probably already logged into the Student Portal in order to register! As soon as
you registered, you will automatically have been granted access to the VLE, Online
Library and your fully functional University of London email account.
If you forget your login details at any point, please email [email protected]
quoting your student number.

0.8.1 The VLE

The VLE, which complements this subject guide, has been designed to enhance your
learning experience, providing additional support and a sense of community. It forms an
important part of your study experience with the University of London and you should
access it regularly.
The VLE provides a range of resources for EMFSS courses:

Electronic study materials: All of the printed materials which you receive from
the University of London are available to download, to give you flexibility in how
and where you study.
Discussion forums: An open space for you to discuss interests and seek support

5
Preface

from your peers, working collaboratively to solve problems and discuss subject
material. Some forums are moderated by an LSE academic.

Videos: Recorded academic introductions to many subjects; interviews and

debates with academics who have designed the courses and teach similar ones at
LSE.

Recorded lectures: For a few subjects, where appropriate, various teaching

sessions of the course have been recorded and made available online via the VLE.

Audio-visual tutorials and solutions: For some of the first year and larger later
courses such as Introduction to Economics, Statistics, Mathematics and Principles
of Banking and Accounting, audio-visual tutorials are available to help you work
through key concepts and to show the standard expected in examinations.

Self-testing activities: Allowing you to test your own understanding of subject

material.

Study skills: Expert advice on getting started with your studies, preparing for
examinations and developing your digital literacy skills.

Note: Students registered for Laws courses also receive access to the dedicated Laws
VLE.
Some of these resources are available for certain courses only, but we are expanding our
provision all the time and you should check the VLE regularly for updates.

0.8.2 Making use of the Online Library

The Online Library (http://onlinelibrary.london.ac.uk) contains a huge array of journal
articles and other resources to help you read widely and extensively.
To access the majority of resources via the Online Library you will either need to use
your University of London Student Portal login details, or you will be required to
register and use an Athens login.
The easiest way to locate relevant content and journal articles in the Online Library is
to use the Summon search engine.
If you are having trouble finding an article listed in a reading list, try removing any
punctuation from the title, such as single quotation marks, question marks and colons.
For further advice, please use the online help pages
(http://onlinelibrary.london.ac.uk/resources/summon) or contact the Online Library
team: [email protected]

0.9 Prerequisite for studying this subject

The prerequisite for studying this subject is a solid background in mathematics and
elementary statistical theory. The mathematics requirement is a basic understanding of
multivariate differential calculus. With regard to statistics, you must have a clear
understanding of what is meant by the sampling distribution of an estimator, and of the

6
 0.10. Application of linear algebra to econometrics

principles of statistical inference and hypothesis testing. This is absolutely essential. I

find that most problems that students have with introductory econometrics are not
econometric problems at all but problems with statistics, or rather, a lack of
understanding of statistics. There are no short cuts. If you do not have this background
knowledge, you should put your study of econometrics on hold and study statistics first.
Otherwise there will be core parts of the econometrics syllabus that you do not begin to
understand.
In addition, it would be helpful if you have some knowledge of economics. However,
although the examples and exercises relate to economics, most of them are so
straightforward that a previous study of economics is not a requirement.

0.10 Application of linear algebra to econometrics

At the end of this subject guide you will find a primer on the application of linear
algebra (matrix algebra) to econometrics. It is not part of the syllabus for the
examination, and studying it is unlikely to confer any advantage for the examination. It
is provided for the benefit of those students who intend to take a further course in
econometrics, especially at the Masters level. The present course is ambitious, by
undergraduate standards, in terms of its coverage of concepts and, above all, its focus
on the development of an intuitive understanding. For its purposes, it has been quite
sufficient and appropriate to work with uncomplicated regression models, typically with
no more than two explanatory variables.
However, when you progress to the next level, it is necessary to generalise the theory to
cover multiple regression models with many explanatory variables, and linear algebra is
ideal for this purpose. The primer does not attempt to teach it. There are many
excellent texts and there is no point in duplicating them. The primer assumes that such
basic study has already been undertaken, probably taking about 20 to 50 hours,
depending on the individual. It is intended to show how the econometric theory in the
text can be handled with this more advanced mathematical approach, thus serving as
preparation for the higher-level course.

0.11 The examination

Important: the information and advice given here are based on the examination
structure used at the time this subject guide was written. Please note that subject
guides may be used for several years. Because of this we strongly advise you to always
check both the current Programme regulations for relevant information about the
examination, and the VLE where you should be advised of any forthcoming changes.
You should also carefully check the rubric/instructions on the paper you actually sit
and follow those instructions.
Candidates should answer eight out of 10 questions in three hours: all of the questions
in Section A (8 marks each) and three questions from Section B (20 marks each).
A calculator may be used when answering questions on this paper and it must comply
in all respects with the specification given with your Admission Notice.

7
Preface

Remember, it is important to check the VLE for:

up-to-date information on examination and assessment arrangements for this course

where available, past examination papers and Examiners commentaries for the
course which give advice on how each question might best be answered.

8
 0.12. Overview

Review: Random variables and

sampling theory

0.12 Overview
The textbook and this subject guide assume that you have previously studied basic
statistical theory and have a sound understanding of the following topics:

descriptive statistics (mean, median, quartile, variance, etc.)

random variables and probability

expectations and expected value rules

population variance, covariance, and correlation

sampling theory and estimation

unbiasedness and efficiency

loss functions and mean square error

normal distribution

hypothesis testing, including:

t tests
Type I and Type II error
the significance level and power of a t test
one-sided versus two-sided t tests

confidence intervals

convergence in probability, consistency, and plim rules

convergence in distribution and central limit theorems.

There are many excellent textbooks that offer a first course in statistics. The Review
chapter of my textbook is not a substitute. It has the much more limited objective of
providing an opportunity for revising some key statistical concepts and results that will
be used time and time again in the course. They are central to econometric analysis and
if you have not encountered them before, you should postpone your study of
econometrics and study statistics first.

9
 Preface

0.13 Learning outcomes

After working through the corresponding chapter in the textbook, studying the
corresponding slideshows, and doing the starred exercises in the textbook and the
additional exercises in this subject guide, you should be able to explain what is meant
by all of the items listed in the Overview. You should also be able to explain why they
are important. The concepts of efficiency, consistency, and power are often
misunderstood by students taking an introductory econometrics course, so make sure
that you aware of their precise meanings.

0.14 Additional exercises

[Note: Each chapter has a set of additional exercises. The answers to them are
provided at the end of the chapter after the answers to the starred exercises in the text.]

AR.1 A random variable X has a continuous uniform distribution from 0 to 2. Define its
probability density function.

probability
density

0 2 X

AR.2 Find the expected value of X in Exercise AR.1, using the expression given in Box
R.1 in the text.

AR.3 Derive E(X 2 ) for X defined in Exercise AR.1, using the expression given in Box
R.1.

AR.4 Derive the population variance and the standard deviation of X as defined in
Exercise AR.1, using the expression given in Box R.1.

AR.5 Using equation (R.9), find the variance of the random variable X defined in
Exercise AR.1 and show that the answer is the same as that obtained in Exercise
AR.4. (Note: You have already calculated E(X) in Exercise AR.2 and E(X 2 ) in
Exercise AR.3.)

AR.6 In Table R.6, 0 and 1 were three standard deviations apart. Construct a similar
( )
table for the case where they are two standard deviations apart.
rejection region acceptance region rejection region

10
 0.15. Answers to the starred exercises in the textbook

AR.7 Suppose that a random variable X has a normal distribution with unknown mean
and variance 2 . To simplify the analysis, we shall assume that 2 is known. Given
a sample of observations, an estimator of is the sample mean, X. An investigator
wishes to test H0 : = 0 and believes that the true value cannot be negative. The
appropriate alternative hypothesis is therefore H1 : > 0 and the investigator
decides to perform a one-sided test. However, the investigator is mistaken because
could in fact be negative. What are the consequences of erroneously performing a
one-sided test when a two-sided test would have been appropriate?

AR.8 Suppose that a random variable X has a normal distribution with mean and
variance 2 . Given a sample of n independent observations, it can be shown that:

2 1 X 2

b = Xi X
n1

is an unbiased estimator of 2 . Is b2 either an unbiased or a consistent estimator
of ?

0.15 Answers to the starred exercises in the textbook

R.2 A random variable X is defined to be the larger of the two values when two dice
are thrown, or the value if the values are the same. Find the probability
distribution for X.
Answer:
The table shows the 36 possible outcomes. The probability distribution is derived
by counting the number of times each outcome occurs and dividing by 36. The
probabilities have been written as fractions, but they could equally well have been
written as decimals.

red 1 2 3 4 5 6
green
1 1 2 3 4 5 6
2 2 2 3 4 5 6
3 3 3 3 4 5 6
4 4 4 4 4 5 6
5 5 5 5 5 5 6
6 6 6 6 6 6 6

Value of X 1 2 3 4 5 6
Frequency 1 3 5 7 9 11
Probability 1/36 3/36 5/36 7/36 9/36 11/36

11
 Preface

R.4 Find the expected value of X in Exercise R.2.

Answer:
The table is based on Table R.2 in the text. It is a good idea to guess the outcome
before doing the arithmetic. In this case, since the higher numbers have the largest
probabilities, the expected value should clearly lie between 4 and 5. If the
calculated value does not conform with the guess, it is possible that this is because
the guess was poor. However, it may be because there is an error in the arithmetic,
and this is one way of catching such errors.

X p Xp
1 1/36 1/36
2 3/36 6/36
3 5/36 15/36
4 7/36 28/36
5 9/36 45/36
6 11/36 66/36
Total 161/36 = 4.4722

R.7 Calculate E(X 2 ) for X defined in Exercise R.2.

Answer:
The table is based on Table R.3 in the text. Given that the largest values of X 2
have the highest probabilities, it is reasonable to suppose that the answer lies
somewhere in the range 1530. The actual figure is 21.97.

X X2 p X 2p
1 1 1/36 1/36
2 4 3/36 12/36
3 9 5/36 45/36
4 16 7/36 112/36
5 25 9/36 225/36
6 36 11/36 396/36
Total 791/36 = 21.9722

R.10 Calculate the population variance and the standard deviation of X as defined in
Exercise R.2, using the definition given by equation (R.8).
Answer:
The table is based on Table R.4 in the textbook. In this case it is not easy to make
a guess. The population variance is 1.97, and the standard deviation, its square
root, is 1.40. Note that four decimal places have been used in the working, even
though the estimate is reported to only two. This is to eliminate the possibility of
the estimate being affected by rounding error.

12
 0.15. Answers to the starred exercises in the textbook

X p X X (X X )2 (X X )2 p
1 1/36 3.4722 12.0563 0.3349
2 3/36 2.4722 6.1119 0.5093
3 5/36 1.4722 2.1674 0.3010
4 7/36 0.4722 0.2230 0.0434
5 9/36 0.5278 0.2785 0.0696
6 11/36 1.5278 2.3341 0.7132
Total 1.9715

R.12 Using equation (R.9), find the variance of the random variable X defined in
Exercise R.2 and show that the answer is the same as that obtained in Exercise
R.10. (Note: You have already calculated X in Exercise R.4 and E(X 2 ) in
Exercise R.7.)
Answer:
E(X 2 ) is 21.9722 (Exercise R.7). E(X) is 4.4722 (Exercise R.4), so 2X is 20.0006.
Thus the variance is 21.9722 20.0006 = 1.9716. The last-digit discrepancy
between this figure and that in Exercise R.10 is due to rounding error.

R.14 Suppose a variable Y is an exact linear function of X:

Y = + X

where and are constants, and suppose that Z is a third variable. Show that
XZ = Y Z
Answer:    
We start by noting that Yi Y = Xi X . Then:
h  i
E Yi Y Z i Z
Y Z = s 
  2   2 
E Yi Y E Zi Z

h   i
E Xi X Zi Z
= s 
 2    2 
2
E Xi X 2
E Zi Z

h  i
E Xi X Zi Z
= s   2   2 
2
E Xi X E Zi Z

= XZ .

R.16 Show that, when you have n observations, the condition that the generalised
estimator (1 X1 + + n Xn ) should be an unbiased estimator of X is
1 + + n = 1.

13
 Preface

Answer:
E(Z) = E(1 X1 + + n Xn )
= E(1 X1 ) + + E(n Xn )
= 1 E(X1 ) + + n E(Xn )
= 1 X + + n X
= (1 + + n )X .
Thus E(Z) = X requires 1 + + n = 1.

R.19 In general, the variance of the distribution of an estimator decreases when the
sample size is increased. Is it correct to describe the estimator as becoming more
efficient?
Answer:
No, it is incorrect. When the sample size increases, the variance of the estimator
decreases, and as a consequence it is more likely to give accurate results. Because it
is improving in this important sense, it is very tempting to describe the estimator
as becoming more efficient. But it is the wrong use of the term. Efficiency is a
comparative concept that is used when you are comparing two or more alternative
estimators, all of them being applied to the same data set with the same sample
size. The estimator with the smallest variance is said to be the most efficient. You
cannot use efficiency as suggested in the question because you are comparing the
variances of the same estimator with different sample sizes.

R.21 Suppose that you have observations on three variables X, Y , and Z, and suppose
that Y is an exact linear function of Z:
Y = + Z
where and are constants. Show that bXZ = bXY . (This is the counterpart of
Exercise R.14.)
Answer:    
We start by noting that Yi Y = Zi Z . Then:
P  
Xi X Yi Y
bXY = r  2 P  2
P
Xi X Yi Y
P   
Xi X Zi Z
= r  2 P  2
P
Xi X 2 Z i Z
P  
Xi X Zi Z
= r  2 P  2
P
Xi X Zi Z

= bXZ

14
 0.15. Answers to the starred exercises in the textbook

R.26 Show that, in Figures R.18 and R.22, the probabilities of a Type II error are 0.15
in the case of a 5 per cent significance test and 0.34 in the case of a 1 per cent test.
Note that the distance between 0 and 1 is three standard deviations. Hence the
right-hand 5 per cent rejection region begins 1.96 standard deviations to the right
of 0 . This means that it is located 1.04 standard deviations to the left of 1 .
Similarly, for a 1 per cent test, the right-hand rejection region starts 2.58 standard
deviations to the right of 0 , which is 0.42 standard deviations to the left of 1 .
Answer:
For the 5 per cent test, the rejection region starts 3 1.96 = 1.04 standard
deviations below 1 , given that the distance between 1 and 0 is 3 standard
deviations. See Figure R.18. According to the standard normal distribution table,
the cumulative probability of a random variable lying 1.04 standard deviations (or
less) above the mean is 0.8508. This implies that the probability of it lying 1.04
standard deviations below the mean is 0.1492. For the 1 per cent test, the rejection
region starts 3 2.58 = 0.42 standard deviations below the mean. See Figure R.22.
The cumulative probability for 0.42 in the standard normal distribution table is
0.6628, so the probability of a Type II error is 0.3372.

R.27 Explain why the difference in the power of a 5 per cent test and a 1 per cent test
becomes small when the distance between 0 and 1 becomes large.
Answer:
The powers of both tests tend to one as the distance between 0 and 1 becomes
large. The difference in their powers must therefore tend to zero.

R.28 A random variable X has unknown population mean . A researcher has a sample
of observations with sample mean X. He wishes to test the null hypothesis
H0 : = 0 . The figure shows the potential distribution of X conditional on H0
being true. It may be assumed that the distribution is known to have variance
equal to one.

f(X)

5% rejection region

0
0 0
X

The researcher decides to implement an unorthodox (and unwise) decision rule. He

decides to reject H0 if X lies in the central 5 per cent of the distribution (the tinted
area in the figure).
(a) Explain why his test is a 5 per cent significance test.

15
Preface

(b) Explain in intuitive terms why his test is unwise.

(c) Explain in technical terms why his test is unwise.
Answer:
The following discussion assumes that you are performing a 5 per cent significance
test, but it applies to any significance level.
If the null hypothesis is true, it does not matter how you define the 5 per cent
rejection region. By construction, the risk of making a Type I error will be 5 per
cent. Issues relating to Type II errors are irrelevant when the null hypothesis is true.
The reason that the central part of the conditional distribution is not used as a
rejection region is that it leads to problems when the null hypothesis is false. The
probability of not rejecting H0 when it is false will be lower. To use the obvious
technical term, the power of the test will be lower.
The figure shows the power functions for the test using the conventional upper and
lower 2.5 per cent tails and the test using the central region. The horizontal axis is
the difference between the true value and the hypothetical value 0 in terms of
standard deviations. The vertical axis is the power of the test. The first figure has
been drawn for the case where the true value is greater than the hypothetical value.
The second figure is for the case where the true value is lower than the hypothetical
value. It is the same, but reflected horizontally.
The greater the difference between the true value and the hypothetical mean, the
more likely is it that the sample mean will lie in the right tail of the distribution
conditional on H0 being true, and so the more likely is it that the null hypothesis
will be rejected by the conventional test. The figure shows that the power of the
test approaches one asymptotically. However, if the central region of the
distribution is used as the rejection region, the probability of the sample mean
lying in it will diminish as the difference between the true and hypothetical values
increases, and the power of the test approaches zero asymptotically. This is an
extreme example of a very bad test procedure.

1.0

0.8
conventional rejection region
(upper and lower 2.5% tails)

0.6

0.4

0.2

rejection region central 5%

0.0
0 1 2 3 4

Figure 1: Power functions of a conventional 5 per cent test and one using the central
region (true value > 0 ).

16
 0.15. Answers to the starred exercises in the textbook

1.0

0.8
conventional rejection region
(upper and lower 2.5% tails)

0.6

0.4

0.2

rejection region central 5%

0.0
-4 -3 -2 -1 0

Figure 2: Power functions of a conventional 5 per cent test and one using the central
region (true value < 0 ).

R.29 A researcher is evaluating whether an increase in the minimum hourly wage has
had an effect on employment in the manufacturing industry in the following three
months. Taking a sample of 25 firms, what should she conclude if:
(a) the mean decrease in employment is 9 per cent, and the standard error of the
mean is 5 per cent
(b) the mean decrease is 12 per cent, and the standard error is 5 per cent
(c) the mean decrease is 20 per cent, and the standard error is 5 per cent
(d) there is a mean increase of 11 per cent, and the standard error is 5 per cent?
Answer:
There are 24 degrees of freedom, and hence the critical values of t at the 5 per cent,
1 per cent, and 0.1 per cent levels are 2.06, 2.80, and 3.75, respectively.
(a) The t statistic is 1.80. Fail to reject H0 at the 5 per cent level.
(b) t = 2.40. Reject H0 at the 5 per cent level but not the 1 per cent level.
(c) t = 4.00. Reject H0 at the 1 per cent level. Better, reject at the 0.1 per cent
level.
(d) t = 2.20. This would be a surprising outcome, but if one is performing a
two-sided test, then reject H0 at the 5 per cent level but not the 1 per cent
level.

R3.33 Demonstrate that the 95 per cent confidence interval defined by equation (R.89)
has a 95 per cent probability of capturing 0 if H0 is true.
Answer:
If H0 is true, there is 95 per cent probability that:
 
X 
0
 < tcrit .


 s.e.(X) 

17
 Preface

Hence there is 95 per cent probability that |X 0 | < tcrit s.e.(X). Hence there is
95 per cent probability that (a) X 0 < tcrit s.e.(X) and (b)
0 X < tcrit s.e.(X).
(a) can be rewritten X tcrit s.e.(X) < 0 , giving the lower limit of the confidence
interval.
(b) can be rewritten X 0 > tcrit s.e.(X) and hence X + tcrit s.e.(X) > 0 ,
giving the upper limit of the confidence interval.
Hence there is 95 per cent probability that 0 will lie in the confidence interval.

R.34 In Exercise R.29, a researcher was evaluating whether an increase in the minimum
hourly wage has had an effect on employment in the manufacturing industry.
Explain whether she might have been justified in performing one-sided tests in
cases (a) (d), and determine whether her conclusions would have been different.
Answer:
First, there should be a discussion of whether the effect of an increase in the
minimum wage could have a positive effect on employment. If it is decided that it
cannot, we can use a one-sided test and the critical values of t at the 5 per cent, 1
per cent, and 0.1 per cent levels become 1.71, 2.49, and 3.47, respectively.
1. The t statistic is 1.80. We can now reject H0 at the 5 per cent level.
2. t = 2.40. No change, but much closer to rejecting at the 1 per cent level.
3. t = 4.00. No change. Reject at the 1 per cent level (and 0.1 per cent level).
4. t = 2.20. Here there is a problem because the coefficient has the unexpected
sign. In principle we should stick to our guns and fail to reject H0 . However,
we should consider two further possibilities. One is that the justification for a
one-sided test is incorrect (not very likely in this case). The other is that the
model is misspecified in some way and the misspecification is responsible for
the unexpected sign. For example, the coefficient might be distorted by
omitted variable bias, to be discussed in Chapter 6.

2
R.37 A random variable X has population mean X and population variance X .A
sample of n observations {X1 , . . . , Xn } is generated. Using the plim rules,
demonstrate that, subject to a certain condition that should be stated:
 
1 1
plim = .
X X

Answer:
plim X = X by the weak law of large numbers. Provided that X 6= 0, we are
entitled to use the plim quotient rule, so:
 
1 plim 1 1
plim = = .
X plim X X

18
 0.15. Answers to the starred exercises in the textbook

R.39 A random variable X has unknown population mean X and population variance
2
X . A sample of n observations {X1 , . . . , Xn } is generated. Show that:

1 1 1 1 1
Z = X1 + X2 + X3 + + n1 + n1 Xn
2 4 8 2 2

is an unbiased estimator of X . Show that the variance of Z does not tend to zero
as n tends to infinity and that therefore Z is an inconsistent estimator, despite
being unbiased.
Answer:
The weights sum to unity, so the estimator is unbiased. However, its variance is:
 
1 1 1 1
Z2 = + + + n1 + n1 2
X .
4 16 4 4

2
This tends to X /3 as n becomes large, not zero, so the estimator is inconsistent.
Note: the sum of a geometric progression is given by:

1 an+1
1 + a + a2 + + an = .
1a

Hence:
 
1 1 1 1 1 1 1 1 1
+ + + + n1 + n1 = 1 + + + n2 + n1
2 4 8 2 2 2 2 2 2

n1
1 1 12 1
= 1 + n1
2 1 2 2
1 1
= 1 + =1
2n1 2n1

and:
 
1 1 1 1 1 1 1 1
+ + + n1 + n1 = 1 + + + n2 + n1
4 16 4 4 4 4 4 4

n1
1 1 14 1
= 1 + n1
4 1 4 4
 n1 !
1 1 1 1
= 1 + n1
3 4 4 3

as n becomes large.

R.41 A random variable X has a continuous uniform distribution over the interval from
0 to , where is an unknown parameter.

19
Preface

f (X)

0
0 X

The following three estimators are used to estimate , given a sample of n

observations on X:
(a) twice the sample mean
(b) the largest value of X in the sample
(c) the sum of the largest and smallest values of X in the sample.
Explain verbally whether or not each estimator is (1) unbiased, and (2) consistent.
Answer:
(a) It is evident that E(X) = E(X) = /2. Hence 2X is an unbiased estimator of .
2 2
The variance of X is X /n. The variance of 2X is therefore 4X /n. This will
tend to zero as n tends to infinity. Thus the distribution of 2X will collapse to
a spike at and the estimator is consistent.
(b) The estimator will be biased downwards since the highest value of X in the
sample will always be less than . However, as n increases, the distribution of
the estimator will be increasingly concentrated in a narrow range just below .
To put it formally, the
probability of the highest value being more than 
n
below will be 1  and this will tend to zero, no matter how small  is,
as n tends to infinity. The estimator is therefore consistent. It can in fact be
n
shown that the expected value of the estimator is n+1 and this tends to as n
becomes large.
(c) The estimator will be unbiased. Call the maximum value of X in the sample
Xmax and the minimum value Xmin . Given the symmetry of the distribution of
X, the distributions of Xmax and Xmin will be identical, except that that of
Xmin will be to the right of 0 and that of Xmax will be to the left of . Hence,
for any n, E(Xmin ) 0 = E(Xmax ) and the expected value of their sum is
equal to . The estimator will be consistent for the same reason as explained in
(b).
The first figure shows the distributions of the estimators (a) and (b) for 1,000,000
samples with only four observations in each sample, with = 1. The second figure
shows the distributions when the number of observations in each sample is equal to

20
 0.15. Answers to the starred exercises in the textbook

100. The table gives the means and variances of the distributions as computed from
the results of the simulations. If the mean square error is used to compare the
estimators, which should be preferred for sample size 4? For sample size 100?

25

20

15

10

5
(b)
(a)
0
0 0.5 1 1.5 2

Sample size = 4

25

20

15 (b)

10

5
(a)

0
0 0.5 1 1.5 2

Sample size = 100

Sample size 4 Sample size 100

(a) (b) (a) (b)
Mean 1.0000 0.8001 1.0000 0.9901
Variance 0.0833 0.0267 0.0033 0.0001
Estimated bias 0.0000 0.1999 0.0000 0.0099
Estimated mean square error 0.0833 0.0667 0.0033 0.0002

It can be shown (Larsen and Marx, An Introduction to Mathematical Statistics and

Its Applications, p.382, that estimator (b) is biased downwards by an amount
/(n + 1) and that its variance is:
n2
(n + 1)2 (n + 2)

21
 Preface

while estimator (a) has variance 2 /3n. How large does n have to be for (b) to be
preferred to (a) using the mean square error criterion?
The crushing superiority of (b) over (a) may come as a surprise, so accustomed are
we to finding that the sample mean in the best estimator of a parameter. The
underlying reason in this case is that we are estimating a boundary parameter,
which, as its name implies, defines the limit of a distribution. In such a case the
optimal properties of the sample mean are no longer guaranteed and it may be
eclipsed by a score statistic such as the largest observation in the sample.
Note that
the standard deviation of the sample mean is inversely proportional to n, while
that of (b) is inversely proportional to n (disregarding the differences between n,
n + 1, and n + 2). (b) therefore approaches its limiting (asymptotically unbiased)
value much faster than (a) and is said to be superconsistent. We will encounter
superconsistent estimators again when we come to cointegration in Chapter 13.
Note that if we multiply (b) by (n + 1)/n, it is unbiased for finite samples as well
as superconsistent.

0.16 Answers to the additional exercises

AR.1 The total area under the function over the interval [0, 2] must be equal to 1. Since
the length of the rectangle is 2, its height must be 0.5. Hence f (X) = 0.5 for
0 X 2, and f (X) = 0 for X < 0 and X > 2.

AR.2 Obviously, since the distribution is uniform, the expected value of X is 1. However
we will derive this formally.
Z 2 Z 2  2 2  2   2 
X 2 0
E(X) = Xf (X) dX = 0.5X dX = = = 1.
0 0 4 0 4 4

AR.3 The expected value of X 2 is given by:

Z 2 Z 2  3 2  3   3 
2 2 2 X 2 0
E(X ) = X f (X) dX = 0.5X dX = = = 1.3333.
0 0 6 0 6 6

AR.4 The variance of X is given by:

Z 2 Z 2
2 2
0.5[X 1]2 dX


E [X X ] = [X X ] f (X) dX =
0 0
Z 2
= (0.5X 2 X + 0.5) dX
0
2
X3 X2 X

= +
6 2 2 0
 
8
= 2 + 1 [0] = 0.3333.
6
The standard deviation is equal to the square root, 0.5774.

22
 0.16. Answers to the additional exercises

AR.5 From Exercise AR.3, E(X 2 ) = 1.3333. From Exercise AR.2, the square of E(X) is
1. Hence the variance is 0.3333, as in Exercise AR.4.

AR.6 Table R.6 is reproduced for reference:

Table R.6 Trade-off between Type I and Type II errors, one-sided and two-sided tests
Probability of Type II error if = 1
One-sided test Two-sided test
5 per cent significance test 0.09 0.15
2.5 per cent significance test 0.15 (not investigated)
1 per cent significance test 0.25 0.34

Note: The distance between 1 and 0 in this example was 3 standard deviations.
Two-sided tests
Under the (false) H0 : = 0 , the right rejection region for a two-sided 5 per cent
significance test starts 1.96 standard deviations above 0 , which is 0.04 standard
deviations below 1 . A Type II error therefore occurs if X is more than 0.04
standard deviations to the left of 1 . Under H1 : = 1 , the probability is 0.48.
Under H0 , the right rejection region for a two-sided 1 per cent significance test
starts 2.58 standard deviations above 0 , which is 0.58 standard deviations above
1 . A Type II error therefore occurs if X is less than 0.58 standard deviations to
the right of 1 . Under H1 : = 1 , the probability is 0.72.
One-sided tests
Under H0 : = 0 , the right rejection region for a one-sided 5 per cent significance
test starts 1.65 standard deviations above 0 , which is 0.35 standard deviations
below 1 . A Type II error therefore occurs if X is more than 0.35 standard
deviations to the left of 1 . Under H1 : = 1 , the probability is 0.36.
Under H0 , the right rejection region for a one-sided 1 per cent significance test
starts 2.33 standard deviations above 0 , which is 0.33 standard deviations above
1 . A Type II error therefore occurs if X is less than 0.33 standard deviations to
the right of 1 . Under H1 : = 1 , the probability is 0.63.
Hence the table is:

Trade-off between Type I and Type II errors, one-sided and two-sided tests
Probability of Type II error if = 1
One-sided test Two-sided test
5 per cent significance test 0.36 0.48
1 per cent significance test 0.63 0.72

AR.7 We will assume for sake of argument that the investigator is performing a 5 per
cent significance test, but the conclusions apply to all significance levels.
If the true value is 0, the null hypothesis is true. The risk of a Type I error is, by
construction, 5 per cent for both one-sided and two-sided tests. Issues relating to
Type II error do not arise because the null hypothesis is true.

23
Preface

If the true value is positive, the investigator is lucky and makes the gain associated
with a one-sided test. Namely, the power of the test is uniformly higher than that
for a two-sided test for all positive values of . The power functions for one-sided
and two-sided tests are shown in the first figure below.
If the true value is negative, the power functions are as shown in the second figure.
That for the two-sided test is the same as that in the first figure, but reflected
horizontally. The larger (negatively) is the true value of , the greater will be the
probability of rejecting H0 and the power approaches 1 asymptotically. However,
with a one-sided test, the power function will decrease from its already very low
value. The power is not automatically zero for true values that are negative because
even for these it is possible that a sample might have a mean that lies in the right
tail of the distribution under the null hypothesis. But the probability rapidly falls
to zero as the (negative) size of grows.

1.0

0.8

one-sided 5% test
two-sided 5% test
0.6

0.4

0.2

0.0
0 1 2 3 4 5

Figure 3: Power functions of one-sided and two-sided 5 per cent tests (true value > 0).

1.0

0.8
two-sided 5% test

0.6

0.4

0.2

one-sided 5% test

0.0
-4 -3 -2 -1 0

Figure 4: Power functions of one-sided and two-sided 5 per cent tests (true value < 0).

24
 0.16. Answers to the additional exercises

AR.8 We will refute the unbiasedness proposition by considering the more general case
where Z 2 is an unbiased estimator of 2 . We know that:

E (Z )2 = E(Z 2 ) 2E(Z) + 2 = 22 2E(Z).

 

Hence:
1 
E (Z )2 .

E(Z) =
2
Z is therefore a biased estimator of except for the special case where Z is equal
to for all samples, that is, in the trivial case where there is no sampling error.
Nevertheless, since a function of a consistent estimator will, under quitegeneral
conditions, be a consistent estimator of the function of the parameter, b2 will be
a consistent estimator of .

25
Preface

26
Chapter 1
Simple regression analysis

1.1 Overview
This chapter introduces the least squares criterion of goodness of fit and demonstrates,
first through examples and then in the general case, how it may be used to develop
expressions for the coefficients that quantify the relationship when a dependent variable
is assumed to be determined by one explanatory variable. The chapter continues by
showing how the coefficients should be interpreted when the variables are measured in
natural units, and it concludes by introducing R2 , a second criterion of goodness of fit,
and showing how it is related to the least squares criterion and the correlation between
the fitted and actual values of the dependent variable.

1.2 Learning outcomes

After working through the corresponding chapter in the text, studying the
corresponding slideshows, and doing the starred exercises in the text and the additional
exercises in this subject guide, you should be able to explain what is meant by:

dependent variable

explanatory variable (independent variable, regressor)

parameter of a regression model

the nonstochastic component of a true relationship

the disturbance term

the least squares criterion of goodness of fit

ordinary least squares (OLS)

the regression line

fitted model

fitted values (of the dependent variable)

residuals

total sum of squares, explained sum of squares, residual sum of squares

R2 .

27
 1. Simple regression analysis

In addition, you should be able to explain the difference between:

the nonstochastic component of a true relationship and a fitted regression line, and

the values of the disturbance term and the residuals.

1.3 Additional exercises

A1.1 The output below gives the result of regressing FDHO, annual household
expenditure on food consumed at home, on EXP, total annual household
expenditure, both measured in dollars, using the Consumer Expenditure Survey
data set. Give an interpretation of the coefficients.

. reg FDHO EXP if FDHO>0

Source | SS df MS Number of obs = 6334

-------------+------------------------------ F( 1, 6332) = 3431.01
Model | 972602566 1 972602566 Prob > F = 0.0000
Residual | 1.7950e+09 6332 283474.003 R-squared = 0.3514
-------------+------------------------------ Adj R-squared = 0.3513
Total | 2.7676e+09 6333 437006.15 Root MSE = 532.42

------------------------------------------------------------------------------
FDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
EXP | .0627099 .0010706 58.57 0.000 .0606112 .0648086
_cons | 369.4418 10.65718 34.67 0.000 348.5501 390.3334
------------------------------------------------------------------------------

A1.2 Download the CES data set from the website (see Appendix B of the text),
perform a regression parallel to that in Exercise A1.1 for your category of
expenditure, and provide an interpretation of the regression coefficients.

A1.3 The output shows the result of regressing the weight of the respondent, in pounds,
in 2011 on the weight in 2004, using EAWE Data Set 22. Provide an interpretation
of the coefficients. Summary statistics for the data are also provided.

. reg WEIGHT11 WEIGHT04

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 1, 498) = 1207.55
Model | 769248.875 1 769248.875 Prob > F = 0.0000
Residual | 317241.693 498 637.031513 R-squared = 0.7080
-------------+------------------------------ Adj R-squared = 0.7074
Total | 1086490.57 499 2177.33581 Root MSE = 25.239
------------------------------------------------------------------------------
WEIGHT11 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
WEIGHT04 | .9739736 .0280281 34.75 0.000 .9189056 1.029042
_cons | 17.42232 4.888091 3.56 0.000 7.818493 27.02614
------------------------------------------------------------------------------

28
 1.3. Additional exercises

. sum WEIGHT04 WEIGHT11

Variable | Obs Mean Std. Dev. Min Max

-------------+--------------------------------------------------------
WEIGHT04 | 500 169.686 40.31215 95 330
WEIGHT11 | 500 182.692 46.66193 95 370

A1.4 The output shows the result of regressing the hourly earnings of the respondent, in
dollars, in 2011 on height in 2004, measured in inches, using EAWE Data Set 22.
Provide an interpretation of the coefficients, comment on the plausibility of the
interpretation, and attempt to give an explanation.
. reg EARNINGS HEIGHT

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 1, 498) = 9.23
Model | 1393.77592 1 1393.77592 Prob > F = 0.0025
Residual | 75171.3726 498 150.946531 R-squared = 0.0182
-------------+------------------------------ Adj R-squared = 0.0162
Total | 76565.1485 499 153.437171 Root MSE = 12.286

------------------------------------------------------------------------------
EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
HEIGHT | .4087231 .1345068 3.04 0.003 .1444523 .6729938
_cons | -9.26923 9.125089 -1.02 0.310 -27.19765 8.659188
------------------------------------------------------------------------------

A1.5 A researcher has data for 50 countries on N , the average number of newspapers
purchased per adult in one year, and G, GDP per capita, measured in US $, and
fits the following regression (RSS = residual sum of squares):
N
b = 25.0 + 0.020G R2 = 0.06, RSS = 4,000.0
The researcher realises that GDP has been underestimated by $100 in every
country and that N should have been regressed on G , where G = G + 100.
Explain, with mathematical proofs, how the following components of the output
would have differed:
the coefficient of GDP
the intercept
RSS
R2 .

A1.6 A researcher with the same model and data as in Exercise A1.5 believes that GDP
in each country has been underestimated by 50 per cent and that N should have
been regressed on G , where G = 2G. Explain, with mathematical proofs, how the
following components of the output would have differed:
the coefficient of GDP
the intercept
RSS
R2 .

29
 1. Simple regression analysis

A1.7 Some practitioners of econometrics advocate standardising each variable in a

regression by subtracting its sample mean and dividing by its sample standard
deviation. Thus, if the original regression specification is:

Yi = 1 + 2 Xi + ui

the revised specification is:

Yi = 1 + 2 Xi + vi

where:
Yi Y Xi X
Yi = and Xi =

bY
bX
Y and X are the sample means of Y and X, bY and bX are the estimators of the
standard deviations of Y and X, defined as the square roots of the estimated
variances:
n n
1 X 1 X
bY2 =
(Yi Y )2 and
bX2
= (Xi X)2
n 1 i=1 n 1 i=1

and n is the number of observations in the sample. We will write the fitted models
for the two specifications as:
Ybi = b1 + b2 Xi
and:
Ybi = b1 + b2 Xi .
Taking account of the definitions of Y and X , show that b1 = 0 and that
b2 = bbXY b2 . Provide an interpretation of b2 .

A1.8 For the model described in Exercise A1.7, suppose that Y is regressed on X
without an intercept:
Ybi = b2 Xi .
Determine how b2 is related to b2 .

A1.9 A variable Yi is generated as:

Y i = 1 + u i (1.1)
where 1 is a fixed parameter and ui is a disturbance term that is independently
and identically distributed with expected value 0 and population variance u2 . The
least squares estimator of 1 is Y , the sample mean of Y . Give a mathematical
demonstration that the value of R2 in such a regression is zero.

1.4 Answers to the starred exercises in the textbook

1.9 The output shows the result of regressing the weight of the respondent in 2004,
measured in pounds, on his or her height, measured in inches, using EAWE Data
Set 21. Provide an interpretation of the coefficients.

30
 1.4. Answers to the starred exercises in the textbook

. reg WEIGHT04 HEIGHT

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 1, 498) = 176.74
Model | 211309 1 211309 Prob > F = 0.0000
Residual | 595389.95 498 1195.56215 R-squared = 0.2619
-------------+------------------------------ Adj R-squared = 0.2605
Total | 806698.98 499 1616.63116 Root MSE = 34.577

------------------------------------------------------------------------------
WEIGHT04 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
HEIGHT | 5.073711 .381639 13.29 0.000 4.32389 5.823532
_cons | -177.1703 25.93501 -6.83 0.000 -228.1258 -126.2147
------------------------------------------------------------------------------
Answer:
Literally the regression implies that, for every extra inch of height, an individual
tends to weigh an extra 5.1 pounds. The intercept, which literally suggests that an
individual with no height would weigh 177 pounds, has no meaning.

1.11 A researcher has international cross-sectional data on aggregate wages, W ,

aggregate profits, P , and aggregate income, Y , for a sample of n countries. By
definition:
Yi = Wi + Pi .
The regressions:
W
ci =
b1 +
b 2 Yi
Pbi = b1 + b2 Yi
are fitted using OLS regression analysis. Show that the regression coefficients will
automatically satisfy the following equations:

b2 + b2 = 1

b1 + b1 = 0.
Explain intuitively why this should be so.
Answer:

P   P  
Yi Y W i W Yi Y Pi P

b2 + b2 = P 2 + P 2
Yi Y Yi Y
P  
Y i Y W i + Pi W P
= P 2
Yi Y
P  
Yi Y Yi Y
= P 2
Yi Y

= 1

31
 1. Simple regression analysis

     
b1 + b1 = W
b2Y + P b2Y = W + P (b
2 + b2 )Y = Y Y = 0.
The intuitive explanation is that the regressions break down income into predicted
wages and profits and one would expect the sum of the predicted components of
income to be equal to its actual level. The sum of the predicted components is
W
ci + Pbi = (b
1 +
b2 Yi ) + (b1 + b2 Yi ), and in general this will be equal to Yi only if
the two conditions are satisfied.

1.13 Suppose that the units of measurement of X are changed so that the new measure,
X , is related to the original one by Xi = 2 Xi . Show that the new estimate of the
slope coefficient is b2 /2 , where b2 is the slope coefficient in the original regression.
Answer:

P
 
Xi X Yi Y
b2 = P 2

Xi X
P  
2 Xi 2X Yi Y
= P 2
2 Xi 2X
P  
2 X i X Yi Y
= P 2
22 Xi X

b2
= .
2

1.14 Demonstrate that if X is demeaned but Y is left in its original units, the intercept
in a regression of Y on demeaned X will be equal to Y .
Answer:
Let Xi = Xi X and b1 and b2 be the intercept and slope coefficient in a

regression of Y on X . Note that X = 0. Then:

b1 = Y b2X = Y .
The slope coefficient is not affected by demeaning:
P
  P  
Xi X Yi Y [Xi X] 0 Yi Y
b2 = = = b2 .
2
P  P 2
Xi X [Xi X] 0

1.15 The regression output shows the result of regressing weight on height using the
same sample as in Exercise 1.9, but with weight and height measured in kilos and
centimetres: WMETRIC = 0.454 WEIGHT04 and HMETRIC = 2.54 HEIGHT .
Confirm that the estimates of the intercept and slope coefficient are as should be
expected from the changes in the units of measurement.

32
 1.4. Answers to the starred exercises in the textbook

. gen WTMETRIC = 0.454*WEIGHT04

. gen HMETRIC = 2.54*HEIGHT

. reg WTMETRIC HMETRIC

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 1, 498) = 176.74
Model | 43554.1641 1 43554.1641 Prob > F = 0.0000
Residual | 122719.394 498 246.424486 R-squared = 0.2619
-------------+------------------------------ Adj R-squared = 0.2605
Total | 166273.558 499 333.213544 Root MSE = 15.698

------------------------------------------------------------------------------
WMETRIC | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
HMETRIC | .9068758 .0682142 13.29 0.000 .7728527 1.040899
_cons | -80.43529 11.77449 -6.83 0.000 -103.5691 -57.30148
------------------------------------------------------------------------------

Answer:
Abbreviate WEIGHT04 to W , HEIGHT to H, WMETRIC to W M , and
HMETRIC to HM . W M = 0.454W and HM = 2.54H. The slope coefficient and
intercept for the regression in metric units, b2M and b1M , are then given by:

P  
HMi HM W Mi W M
b2M = P 2
HMi HM
P    
2.54 Hi H 0.454 Wi W
= P  2
2
2.54 Hi H
P  
Hi H Wi W
= 0.179 P 2
Hi H

= 0.179b2
= 0.179 5.074
= 0.908

b1M = W M b2MHM
 
0.454 b
= 0.454W 2 (2.54H)
2.54

= 0.454(W b2H)
= 0.454b1
= 0.454 177.2
= 80.4.

33
 1. Simple regression analysis

The regression output confirms that the calculations are correct (subject to
rounding error in the last digit).

1.16 Consider the regression model:

Yi = 1 + 2 Xi + ui .
It implies:
Y = 1 + 2X + u
and hence that:
Yi = 2 Xi + vi
where Yi = Yi Y , Xi = Xi X and vi = ui u.
Demonstrate that a regression of Y on X using (1.49) will yield the same
estimate of the slope coefficient as a regression of Y on X. Note: (1.49) should be
used instead of (1.35) because there is no intercept in this model.
Evaluate the outcome if the slope coefficient were estimated using (1.35), despite
the fact that there is no intercept in the model.
Determine the estimate of the intercept if Y were regressed on X with an
intercept included in the regression specification.
Answer:
Let b2 be the slope coefficient in a regression of Y on X using (1.49). Then:
P  
P
X Y X i X Y i Y
b2 = P i 2i = 2 = b2 .
Xi P  
Xi X

Let b2 be the slope coefficient in a regression of Y on X using (1.35). Note that

Y and X are both zero. Then:
P
 

Xi X Yi Y P
X Y

2 =
b = P i 2i = b2 .
P 2 Xi

Xi X

Let b1 be the intercept in a regression of Y on X using (1.35). Then:

b1 = Y b2X = 0.

1.18 Demonstrate that the fitted values of the dependent variable are uncorrelated with
the residuals in a simple regression model. (This result generalises to the multiple
regression case.)
Answer:
The numerator of the sample correlation coefficient for Yb and u b can be decomposed
as follows, using the fact that u
b = 0:
1 X  b b   1 X b 
Yi Y u bi u
b = [1 + b2 Xi ] [b1 + b2X] ubi
n n
1 b X 
= 2 Xi X u bi
n
= 0

34
 1.5. Answers to the additional exercises

by (1.65). Hence the correlation is zero.

1.23 Demonstrate that, in a regression with an intercept, a regression of Y on X must

have the same R2 as a regression of Y on X, where X = 2 X.
Answer:
Let the fitted regression of Y on X be written Ybi = b1 + b2 Xi . b2 = b2 /2
(Exercise 1.13).
b2
b1 = Y b2X = Y 2X = b1 .
2
Hence:
b2
Ybi = b1 + 2 Xi = Ybi .
2
The fitted and actual values of Y are not affected by the transformation and so R2
is unaffected.

1.25 The output shows the result of regressing weight in 2011 on height, using EAWE
Data Set 21. In 2011 the respondents were aged 2731. Explain why R2 is lower
than in the regression reported in Exercise 1.9.
. reg WEIGHT11 HEIGHT

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 1, 498) = 139.97
Model | 236642.736 1 236642.736 Prob > F = 0.0000
Residual | 841926.912 498 1690.61629 R-squared = 0.2194
-------------+------------------------------ Adj R-squared = 0.2178
Total | 1078569.65 499 2161.46222 Root MSE = 41.117

------------------------------------------------------------------------------
WEIGHT11 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
HEIGHT | 5.369246 .4538259 11.83 0.000 4.477597 6.260895
_cons | -184.7802 30.8406 -5.99 0.000 -245.3739 -124.1865
------------------------------------------------------------------------------
Answer:
The explained sum of squares is actually higher than that in Exercise 1.9. The
reason for the fall in R2 is the huge increase in the total sum of squares, no doubt
caused by the cumulative effect of variations in eating habits.

1.5 Answers to the additional exercises

A1.1 Expenditure on food consumed at home increases by 6.3 cents for each dollar of
total household expenditure. Literally the intercept implies that $369 would be
spent on food consumed at home if total household expenditure were zero.
Obviously, such an interpretation does not make sense. If the explanatory variable
were income, and household income were zero, positive expenditure on food at
home would still be possible if the household received food stamps or other
transfers, but here the explanatory variable is total household expenditure.

35
 1. Simple regression analysis

A1.2 For each category, the regression sample has been restricted to households with
non-zero expenditure. All the slope coefficients are highly significant. Housing has
the largest coefficient, as one should expect. Surprisingly, it is followed by
education. However, most households spent nothing at all on this category. For
those that did, it was important.

EXP
n b2 R2
ADM 2,815 0.0235 0.228
CLOT 4,500 0.0316 0.176
DOM 1,661 0.0409 0.134
EDUC 561 0.1202 0.241
ELEC 5,828 0.0131 0.180
FDAW 5,102 0.0527 0.354
FDHO 6,334 0.0627 0.351
FOOT 1,827 0.0058 0.082
FURN 487 0.0522 0.102
GASO 5,710 0.0373 0.278
HEAL 4,802 0.0574 0.174
HOUS 6,223 0.1976 0.469
LIFE 1,253 0.0193 0.101
LOCT 692 0.0068 0.059
MAPP 399 0.0329 0.102
PERS 3,817 0.0069 0.213
READ 2,287 0.0048 0.104
SAPP 1,037 0.0045 0.034
TELE 5,788 0.0160 0.268
TEXT 992 0.0040 0.051
TOB 1,155 0.0165 0.088
TOYS 2,504 0.0145 0.076
TRIP 516 0.0466 0.186

A1.3 The summary data indicate that, on average, the respondents put on 13 pounds
over the period 20042011. Was this due to the relatively heavy becoming even
heavier, or to a general increase in weight? The regression output indicates that
weight in 2011 was approximately equal to weight in 2004 plus 17 pounds, so the
second explanation appears to be the correct one. Note that this is an instance
where the constant term can be given a meaningful interpretation and where it is as
of much interest as the slope coefficient. The R2 indicates that 2004 weight accounts
for 71 per cent of the variance in 2011 weight, so other factors are important.

A1.4 The slope coefficient indicates that hourly earnings increase by 41 cents for every
extra inch of height. The negative intercept has no possible interpretation. The
interpretation of the slope coefficient is obviously highly implausible, so we know
that something must be wrong with the model. The explanation is that this is a
very poorly specified earnings function and that, in particular, we are failing to
control for the sex of the respondent. Later on, in Chapter 5, we will find that

36
 1.5. Answers to the additional exercises

males earn more than females, controlling for observable characteristics. Males also
tend to be taller. Hence we find an apparent positive association between earnings
and height in a simple regression. Note that R2 is very low.

A1.5 The coefficient of GDP: Let the revised measure of GDP be denoted G , where

G = G + 100. Since Gi = Gi + 100 for all i, G = G + 100 and so Gi G = Gi G
for all i. Hence the new slope coefficient is:
P
  P  
Gi G Ni N Gi G Ni N
b2 = = = b2 .
2
P  P 2
Gi G Gi G

The coefficient is unchanged.

The intercept: The new intercept is:

 

1 = N 2G = N 2 G + 100 = b1 100b2 = 23.0.
b b b

bi , is given by:
RSS: The residual in observation i in the new regression, u
bi = Ni b1 b2 Gi = Ni (b1 100b2 ) b2 (Gi + 100) = u
u bi
the residual in the original regression. Hence RSS is unchanged.
R2 :
RSS
R2 = 1 P  2
Ni N
P 2
and is unchanged since RSS and Ni N are unchanged.
Note that this makes sense intuitively. R2 is unit-free and so it is not possible for
the overall fit of a relationship to be affected by the units of measurement.

A1.6 The coefficient of GDP: Let the revised measure of GDP be denoted G, where


G = 2G. Since Gi = 2Gi for all i, G = 2G and so Gi G = 2 Gi G for all i.
Hence the new slope coefficient is:
P
 
Gi G Ni N
b2 = P 2

Gi G
P   
2 Gi G Ni N
= P  2
4 Gi G
P  
2 Gi G Ni N
= P 2
4 Gi G

b2
=
2
= 0.010

37
 1. Simple regression analysis

where b2 = 0.020 is the slope coefficient in the original regression.

The intercept: The new intercept is:

b2
b1 = N b2G = N 2G = N b2G = b1 = 25.0
2
the original intercept.
bi , is given by:
RSS : The residual in observation i in the new regression, u

b2
bi = Ni b1 b2 Gi = Ni b1 2Gi = u
u bi
2
the residual in the original regression. Hence RSS is unchanged.
R2 :
RSS
R2 = 1 P  2
Ni N
P 2
and is unchanged since RSS and Ni N are unchanged. As in Exercise A1.6,
this makes sense intuitively.

A1.7 By construction, Y = X = 0. So b1 = Y b2X = 0.
P
 
Xi X Yi Y
b2 = P 2

Xi X
P
X Y
= P i 2i
Xi
P Xi X   Yi Y 


bX
bY
= P 2

Xi X

bX

P  

bX Xi X Yi Y
= 2

bY P
Xi X

bX b
= 2 .

bY

b2 provides an estimate of the effect on Y , in terms of standard deviations of Y , of

a one-standard deviation change in X.

A1.8 We have:
P
 

P
X Y X i X Y i Y
b2 = P i 2i = 2 = b2 .
Xi

Xi X
P

38
 1.5. Answers to the additional exercises

A1.9 We have: 2
P b
Yi Y
R2 = P  2
Yi Y

and Ybi = Y for all i.

39
1. Simple regression analysis

40
Chapter 2
Properties of the regression
coefficients and hypothesis testing

2.1 Overview
Chapter 1 introduced least squares regression analysis, a mathematical technique for
fitting a relationship given suitable data on the variables involved. It is a fundamental
chapter because much of the rest of the text is devoted to extending the least squares
approach to handle more complex models, for example models with multiple explanatory
variables, nonlinear models, and models with qualitative explanatory variables.
However, the mechanics of fitting regression equations are only part of the story. We are
equally concerned with assessing the performance of our regression techniques and with
developing an understanding of why they work better in some circumstances than in
others. Chapter 2 is the starting point for this objective and is thus equally
fundamental. In particular, it shows how two of the three main criteria for assessing the
performance of estimators, unbiasedness and efficiency, are applied in the context of a
regression model. The third criterion, consistency, will be considered in Chapter 8.

2.2 Learning outcomes

After working through the corresponding chapter in the text, studying the
corresponding slideshows, and doing the starred exercises in the text and the additional
exercises in this subject guide, you should be able to explain what is meant by:

cross-sectional, time series, and panel data

unbiasedness of OLS regression estimators

variance and standard errors of regression coefficients and how they are determined

GaussMarkov theorem and efficiency of OLS regression estimators

two-sided t tests of hypotheses relating to regression coefficients and one-sided t

tests of hypotheses relating to regression coefficients

F tests of goodness of fit of a regression equation

in the context of a regression model. The chapter is a long one and you should take
your time over it because it is essential that you develop a perfect understanding of
every detail.

41
2. Properties of the regression coefficients and hypothesis testing

2.3 Further material

Derivation of the expression for the variance of the nave estimator in
Section 2.3.
The variance of the nave estimator in Section 2.3 and Exercise 2.9 is not of any great
interest in itself, but its derivation provides an example of how one obtains expressions
for variances of estimators in general.
In Section 2.3 we considered the nave estimator of the slope coefficient derived by
joining the first and last observations in a sample and calculating the slope of that line:

Yn Y1
b2 = .
Xn X 1

It was demonstrated that the estimator could be decomposed as:

un u1
b2 = 2 +
X n X1

and hence that E(b2 ) = 2 .

The population variance of a random variable X is defined to be E([X X ]2 ) where
X = E(X). Hence the population variance of b2 is given by:

  2 !  2 !
  un u1 un u1
2b2 = E [b2 2 ]2 = E 2 + 2 =E .
Xn X 1 Xn X1

On the assumption that X is nonstochastic, this can be written as:

 2
1
2b2 E [un u1 ]2 .


=
Xn X1

Expanding the quadratic, we have:

 2
1
2b2 E u2n + u21 2un u1


=
X n X1
 2
1
E(u2n ) + E(u21 ) 2E(un u1 ) .
 
=
X n X1

Each value of the disturbance term is drawn randomly from a distribution with mean 0
and population variance u2 , so E(u2n ) and E(u21 ) are both equal to u2 . un and u1 are
drawn independently from the distribution, so E(un u1 ) = E(un )E(u1 ) = 0. Hence:

2u2 u2
2b2 = = 1 .
(Xn X1 )2 2
(Xn X1 )2

42
 2.4. Additional exercises

Define A = 12 (X1 + Xn ), the average of X1 and Xn , and D = Xn A = A X1 . Then:

1 1
(Xn X1 )2 = (Xn A + A X1 )2
2 2
1
(Xn A)2 + (A X1 )2 + 2(Xn A)(A X1 )

=
2
1 2
D + D2 + 2(D)(D) = 2D2

=
2
= (Xn A)2 + (A X1 )2
= (Xn A)2 + (X1 A)2
= (Xn X + X A)2 + (X1 X + X A)2
= (Xn X)2 + (X A)2 + 2(Xn X)(X A)
+(X1 X)2 + (X A)2 + 2(X1 X)(X A)
= (X1 X)2 + (Xn X)2 + 2(X A)2 + 2(X1 + Xn 2X)(X A)
= (X1 X)2 + (Xn X)2 + 2(X A)2 + 2(2A 2X)(X A)
= (X1 X)2 + (Xn X)2 2(X A)2
= (X1 X)2 + (Xn X)2 2(A X)2
1
= (X1 X)2 + (Xn X)2 (X1 + Xn 2X)2 .
2
Hence we obtain the expression in Exercise 2.9. There must be a shorter proof.

2.4 Additional exercises

A2.1 A variable Y depends on a nonstochastic variable X with the relationship:

Y = 1 + 2 X + u

where u is a disturbance term that satisfies the regression model assumptions.

Given a sample of n observations, a researcher decides to estimate 2 using the
expression: P
Xi Yi
2 = P 2 .
b
Xi
(This is the OLS estimator of 2 for the model Y = 2 X + u.)
(a) Demonstrate that b2 is in general a biased estimator of 2 .
(b) Discuss whether it is possible to determine the sign of the bias.
(c) Demonstrate that b2 is unbiased if 1 = 0.
(d) Demonstrate that b2 is unbiased if X = 0.

A2.2 A variable Yi is generated as:

Y i = 1 + u i

43
 2. Properties of the regression coefficients and hypothesis testing

where 1 is a fixed parameter and ui is a disturbance term that is independently

and identically distributed with expected value 0 and population variance u2 . The
least squares estimator of 1 is Y , the sample mean of Y . However, a researcher
believes that Y is a linear function of another variable X and uses ordinary least
squares to fit the relationship:
Yb = b1 + b2 X
calculating b1 as Y b2X, where X is the sample mean of X. X may be assumed to
be a nonstochastic variable. Determine whether the researchers estimator b1 is
biased or unbiased, and if biased, determine the direction of the bias.

A2.3 With the model described in Exercise A2.2, standard theory states that the
population variance of the researchers estimator of 1 is:

2
1 X
u2 + P 2 .

n
Xi X

In general, this is larger than the population variance of Y , which is u2 /n. Explain
the implications of the difference in the variances.
In the special case where X = 0, the variances are the same. Give an intuitive
explanation.

A2.4 A variable Y depends on a nonstochastic variable X with the relationship:

Y = 1 + 2 X + u

where u is a disturbance term that satisfies the regression model assumptions.

Given a sample of n observations, a researcher decides to estimate 2 using the
expression: P
Xi Yi
2 = P 2 .
b
Xi
It can be shown that the population variance of this estimator is u2 / Xi2 .
P

We saw in Exercise A2.1 that b2 is in general a biased estimator of 2 . However, if

either 1 = 0 or X = 0, the estimator is unbiased. What can be said in this case
about the efficiency of the estimator in these two cases, comparing it with the
estimator:
P  
Xi X Yi Y
P 2 ?
Xi X

Returning to the general case where 1 6= 0 and X 6= 0, suppose that there is very
little variation in X in the sample. Is it possible that b2 might be a better
estimator than the OLS estimator?

A2.5 Using the output for the regression in Exercise A1.1, reproduced below, perform
appropriate statistical tests.

44
 2.4. Additional exercises

. reg FDHO EXP if FDHO>0

Source | SS df MS Number of obs = 6334

-------------+------------------------------ F( 1, 6332) = 3431.01
Model | 972602566 1 972602566 Prob > F = 0.0000
Residual | 1.7950e+09 6332 283474.003 R-squared = 0.3514
-------------+------------------------------ Adj R-squared = 0.3513
Total | 2.7676e+09 6333 437006.15 Root MSE = 532.42

------------------------------------------------------------------------------
FDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
EXP | .0627099 .0010706 58.57 0.000 .0606112 .0648086
_cons | 369.4418 10.65718 34.67 0.000 348.5501 390.3334
------------------------------------------------------------------------------

A2.6 Using the output for your regression in Exercise A1.2, perform appropriate
statistical tests.

A2.7 Using the output for the regression of weight in 2004 on height in Exercise 1.9,
reproduced below, perform appropriate statistical tests.

. reg WEIGHT04 HEIGHT

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 1, 498) = 176.74
Model | 211309 1 211309 Prob > F = 0.0000
Residual | 595389.95 498 1195.56215 R-squared = 0.2619
-------------+------------------------------ Adj R-squared = 0.2605
Total | 806698.95 499 1616.63116 Root MSE = 34.577

------------------------------------------------------------------------------
WEIGHT04 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
HEIGHT | 5.073711 .381639 13.29 0.000 4.32389 5.823532
_cons | -177.1703 25.93501 -6.83 0.000 -228.1258 -126.2147
------------------------------------------------------------------------------

A2.8 Using the output for the regression of earnings on height in Exercise A1.4,
reproduced below, perform appropriate statistical tests.

. reg EARNINGS HEIGHT

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 1, 498) = 9.23
Model | 1393.77592 1 1393.77592 Prob > F = 0.0025
Residual | 75171.3726 498 150.946531 R-squared = 0.0182
-------------+------------------------------ Adj R-squared = 0.0162
Total | 76565.1485 499 153.437171 Root MSE = 12.286

------------------------------------------------------------------------------
EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
HEIGHT | .4087231 .1345068 3.04 0.003 .1444523 .6729938
_cons | -9.26923 9.125089 -1.02 0.310 -27.19765 8.659188
------------------------------------------------------------------------------

45
 2. Properties of the regression coefficients and hypothesis testing

A2.9 Explain whether it would be justifiable to use a one-sided test on the slope
coefficient in the regression of the rate of growth of employment on the rate of
growth of GDP in Exercise 2.20.

A2.10 Explain whether it would be justifiable to use a one-sided test on the slope
coefficient in the regression of weight on height in Exercise 1.9.

A2.11 With the information given in Exercise A1.5, how would the change in the
measurement of GDP affect:
the standard error of the coefficient of GDP
the F statistic for the equation?

A2.12 With the information given in Exercise A1.6, how would the change in the
measurement of GDP affect:
the standard error of the coefficient of GDP
the F statistic for the equation?

A2.13 [This is a continuation of Exercise 1.16 in the text.] A sample of data consists of n
observations on two variables, Y and X. The true model is:

Yi = 1 + 2 Xi + ui

where 1 and 2 are parameters and u is a disturbance term that satisfies the usual
regression model assumptions. In view of the true model:

Y = 1 + 2X + u

where Y , X, and u are the sample means of Y , X, and u. Subtracting the second
equation from the first, one obtains:

Yi = 2 Xi + ui

where Yi = Yi Y , Xi = Xi X and ui = ui u. Note that, by construction, the

sample means of Y , X , and u are all equal to zero.
One researcher fits:
Yb = b1 + b2 X. (1)
A second researcher fits:

Yb = b1 + b2 X . (2)

[Note: The second researcher included an intercept in the specification.]

Comparing regressions (1) and (2), demonstrate that Ybi = Ybi Y .
Demonstrate that the residuals in (2) are identical to the residuals in (1).
Demonstrate that the OLS estimator of the variance of the disturbance term
in (2) is equal to that in (1).
Explain how the standard error of the slope coefficient in (2) is related to that
in (1).

46
 2.4. Additional exercises

Explain how R2 in (2) is related to R2 in (1).

Explain why, theoretically, the specification (2) of the second researcher is
incorrect and he should have fitted:
Yb = b2 X (3)
not including a constant in his specification.
If the second researcher had fitted (3) instead of (2), how would this have
affected his estimator of 2 ? Would dropping the unnecessary intercept lead to
a gain in efficiency?

A2.14 For the model described in Exercise A1.7, show that Ybi = (Ybi Y )/b
Y , and thus

bi = u
that u bi /b bi are the fitted value of Yi and the residual in the
Y , where Ybi and u
transformed model.
Hence show that:

bX
s.e.(b2 ) = s.e.(b2 ).

bY
Hence find the relationship between the t statistic for b2 and the t statistic for b2
and the relationship between R2 for the original specification and R2 for the revised
specification.

A2.15 A variable Yi is generated as:

Yi = 1 + 2 Xi + ui (1)
where 1 and 2 are fixed parameters and ui is a disturbance term that satisfies the
regression model assumptions. The values of X are fixed and are as shown in the
figure. Four of them, X1 to X4 , are close together. The fifth, X5 , is much larger.
The corresponding values that Y would take, if there were no disturbance term, are
given by the circles on the line. The presence of the disturbance term in the model
causes the actual values of Y in a sample to be different. The solid black circles
depict a typical sample of observations.

0
0 X1 X2 X3 X4 X5 X

47
 2. Properties of the regression coefficients and hypothesis testing

Discuss the advantages and disadvantages of dropping the observation

corresponding to X5 when regressing Y on X. If you keep the observation in the
sample, will this cause the regression estimates to be biased?

2.5 Answers to the starred exercises in the textbook

2.1 Derive the decomposition of b1 shown in equation (2.29):
X
b1 = 1 + ci u i

1
where ci = n
aiX and ai is defined in equation (2.23).
Answer:
   X 
b1 = Y b2X = 1 + 2X + u X 2 + ai ui

1X X
= 1 + ui X ai u i
n
X
= 1 + ci u i .

2.5 An investigator correctly believes that the relationship between two variables X
and Y is given by:
Yi = 1 + 2 Xi + ui .
Given a sample of observations on Y , X, and a third variable Z (which is not a
determinant of Y ), the investigator estimates 2 as:
P  
Zi Z Yi Y
P  .
Zi Z Xi X

Demonstrate that this estimator is unbiased.

Answer:
 
Noting that Yi Y = 2 Xi X + ui u, we have:

P  
Z i Z Yi Y
b2 = P  
Zi Z Xi X
P    P 
Zi Z 2 Xi X + Zi Z (ui u)
= P  
Z i Z Xi X
P 
Zi Z (ui u)
= 2 + P   .
Z i Z Xi X

48
 2.5. Answers to the starred exercises in the textbook

Hence: P 
Zi Z E (ui u)
E(b2 ) = 2 + P    = 2 .
Zi Z Xi X

2.8 Using the decomposition of b1 obtained in Exercise 2.1, derive the expression for
2b given in equation (2.42).
1

Answer:
b1 = 1 + ci ui , where ci = n1 aiX, and E(b1 ) = 1 . Hence:
P

 X !
2  X 1 X X 2 X
2b1 = E ci u i = u2 c2i = u2 n 2 2 ai + X a2i .
n n
P
From Box 2.2, ai = 0 and:
X 1
a2i = P  2 .
Xi X

Hence:
2
1 X
2b1 = u2 + P  2 .

n
Xi X

2.9 Given the decomposition in Exercise 2.2 of the OLS estimator of 2 in the model
Yi = 2 Xi + ui , demonstrate that the variance of the slope coefficient is given by:
2
2b2 = P u 2 .
Xj

Answer:
n
P P 2 n
b2 = 2 + di ui , where di = Xi / Xj , and E(b2 ) = 2 . Hence:
i=1 j=1

!2
n n n 2
X X X X i

2b2 = E di ui = u2 d2i 2
= u

!2
i=1 i=1 i=1 n
P
2
Xj

j=1

n
u2 X
= !2 Xi2
n i=1
Xj2
P
j=1

u2
= n .
Xj2
P
j=1

49
 2. Properties of the regression coefficients and hypothesis testing

2.12 It can be shown that the variance of the estimator of the slope coefficient in
Exercise 2.5:
P  
Zi Z Yi Y
P  
Zi Z Xi X

is given by:
u2 1
2b2 = P  2 2
rXZ
Xi X

where rXZ is the correlation between X and Z. What are the implications for the
efficiency of the estimator?
Answer:
If Z happens to be an exact linear function of X, the population variance will be
2
the same as that of the OLS estimator. Otherwise 1/rXZ will be greater than 1, the
variance will be larger, and so the estimator will be less efficient.

2.15 Suppose that the true relationship between Y and X is Yi = 1 + 2 Xi + ui and

that the fitted model is Ybi = b1 + b2 Xi . In Exercise 1.13 it was shown that if
Xi = 2 Xi , and Y is regressed on X , the slope coefficient b2 = b2 /2 . How will
the standard error of b2 be related to the standard error of b2 ?
Answer:
In Exercise 1.23 it was demonstrated that the fitted values of Y would be the same.
This means that the residuals are the same, and hence bu2 , the estimator of the
variance of the disturbance term, is the same. The standard error of b2 is then
given by:
v
u bu2

s.e.(b2 ) = u
2
tP  

Xi X
v
u bu2

= u
tP  2
2 Xi 2X
v
u bu2

= u
t P 2
22 Xi X

1
= s.e.(b2 ).
2

2.17 A researcher with a sample of 50 individuals with similar education, but differing
amounts of training, hypothesises that hourly earnings, EARNINGS, may be
related to hours of training, TRAINING, according to the relationship:

EARNINGS = 1 + 2 TRAINING + u.

50
 2.5. Answers to the starred exercises in the textbook

He is prepared to test the null hypothesis H0 : 2 = 0 against the alternative

hypothesis H1 : 2 6= 0 at the 5 per cent and 1 per cent levels. What should he
report:
(a) if b2 = 0.30, s.e.(b2 ) = 0.12?
(b) if b2 = 0.55, s.e.(b2 ) = 0.12?
(c) if b2 = 0.10, s.e.(b2 ) = 0.12?
(d) if b2 = 0.27, s.e.(b2 ) = 0.12?
Answer:
There are 48 degrees of freedom, and hence the critical values of t at the 5 per cent,
1 per cent, and 0.1 per cent levels are 2.01, 2.68, and 3.51, respectively.
(a) The t statistic is 2.50. Reject H0 at the 5 per cent level but not at the 1 per
cent level.
(b) t = 4.58. Reject at the 0.1 per cent level.
(c) t = 0.83. Fail to reject at the 5 per cent level.
(d) t = 2.25. Reject H0 at the 5 per cent level but not at the 1 per cent level.

2.22 Explain whether it would have been possible to perform one-sided tests instead of
two-sided tests in Exercise 2.17. If you think that one-sided tests are justified,
perform them and state whether the use of a one-sided test makes any difference.
Answer:
First, there should be a discussion of whether the parameter 2 in:

EARNINGS = 1 + 2 TRAINING + u

can be assumed not to be negative. The objective of training is to impart skills. It

would be illogical for an individual with greater skills to be paid less on that
account, and so we can argue that we can rule out 2 < 0. We can then perform a
one-sided test. With 48 degrees of freedom, the critical values of t at the 5 per cent,
1 per cent, and 0.1 per cent levels are 1.68, 2.40, and 3.26, respectively.
(a) The t statistic is 2.50. We can now reject H0 at the 1 per cent level (but not at
the 0.1 per cent level).
(b) t = 4.58. Not affected by the change. Reject at the 0.1 per cent level.
(c) t = 0.83. Not affected by the change. Fail to reject at the 5 per cent level.
(d) t = 2.25. Reject H0 at the 5 per cent level but not at the 1 per cent level.
Here there is a problem because the coefficient has an unexpected sign and is
large enough to reject H0 at the 5 per cent level with a two-sided test.
In principle we should ignore this and fail to reject H0 . Admittedly, the
likelihood of such a large negative t statistic occurring under H0 is very small,
but it would be smaller still under the alternative hypothesis H1 : 2 > 0.
However, we should consider two further possibilities. One is that the
justification for a one-sided test is incorrect. For example, some jobs pay
relatively low wages because they offer training that is valued by the employee.

51
 2. Properties of the regression coefficients and hypothesis testing

Apprenticeships are the classic example. Alternatively, workers in some

low-paid occupations may, for technical reasons, receive a relatively large
amount of training. In either case, the correlation between training and
earnings might be negative instead of positive.
Another possible reason for a coefficient having an unexpected sign is that the
model is misspecified in some way. For example, the coefficient might be
distorted by omitted variable bias, to be discussed in Chapter 6.

2.27 Suppose that the true relationship between Y and X is Yi = 1 + 2 Xi + ui and

that the fitted model is Ybi = b1 + b2 Xi . In Exercise 1.13 it was shown that if
Xi = 2 Xi , and Y is regressed on X , the slope coefficient b2 = b2 /2 . How will
the t statistic for b2 be related to the t statistic for b2 ? (See also Exercise 2.15.)
Answer:
In Exercise 2.15 it was shown that s.e.(b2 ) = s.e.(b2 )/2 . Hence the t statistic is
unaffected by the transformation.
Alternatively, since we saw in Exercise 1.23 that R2 must be the same, it follows
that the F statistic for the equation must be the same. For a simple regression the
F statistic is the square of the t statistic on the slope coefficient, so the t statistic
must be the same.

2.30 Calculate the 95 per cent confidence interval for 2 in the price inflation/wage
inflation example:

pb = 1.21 + 0.82w.
(0.05) (0.10)

What can you conclude from this calculation?

Answer:
With n equal to 20, there are 18 degrees of freedom and the critical value of t at
the 5 per cent level is 2.10. The 95 per cent confidence interval is therefore:

0.82 0.10 2.10 2 0.82 + 0.10 2.10

that is:
0.61 2 1.03.
We see that we cannot (quite) reject the null hypothesis H0 : 2 = 1.

2.36 Suppose that the true relationship between Y and X is Yi = 1 + 2 Xi + ui and

that the fitted model is Ybi = b1 + b2 Xi . Suppose that Xi = 2 Xi , and Y is
regressed on X . How will the F statistic for this regression be related to the F
statistic for the original regression? (See also Exercises 1.23, 2.15, and 2.27.)
Answer:
We saw in Exercise 1.23 that R2 would be the same, and it follows that F must
also be the same.

52
 2.6. Answers to the additional exercises

2.6 Answers to the additional exercises

Note: Each of the exercises below relates to a simple regression. Accordingly, the F test
is equivalent to a two-sided t test on the slope coefficient and there is no point in
performing both tests. The F statistic is equal to the square of the t statistic and, for
any significance level, the critical value of F is equal to the critical value of t. Obviously
a one-sided t test, when justified, is preferable to either in that it has greater power for
any given significance level.

A2.1 We have:
P P P P
X i Yi X i (1 + 2 X i + ui ) 1 X i Xi ui
b2 = P 2 = P 2 = P 2 + 2 + P 2 .
Xi Xi Xi Xi
Hence:
P P  P P
1 X i X i u i 1 X i X E(u )
E(b2 ) = P 2 + 2 + E P 2 = P 2 + 2 + Pi 2 i
Xi Xi Xi Xi
assuming that X is nonstochastic. Since E(ui ) = 0, then:
P
1 Xi
E(2 ) = P 2 + 2 .
b
Xi
Thus b2 will in P
general be a biased estimator. The sign of the bias depends on the
signs of 1 and Xi . In general, we have no
P information about either of these.
However, if either 1 = 0 or X = 0 (and so Xi = 0), the bias term disappears and
2 is unbiased after all.
b

A2.2 First we need to show that E(b2 ) = 0.

P   P  P 
X i X Yi Y Xi X (1 + ui 1 u) Xi X (ui u)
b2 = P 2 = P 2 = P 2 .
Xi X Xi X Xi X
Hence, given that we are told that X is nonstochastic:
 
P
Xi X (ui u)
E(b2 ) = E

P 2
Xi X

1 X   
= P 2 E Xi X (ui u)
Xi X

1 X 
= P 2 Xi X E (ui u)
Xi X

= 0
since E(u) = 0. Thus:
 
E(1 ) = E Y 2X = 1 XE(b2 ) = 1
b b

and the estimator is unbiased.

53
 2. Properties of the regression coefficients and hypothesis testing

A2.3 In general, the researchers estimator will have a larger variance than Y and
therefore will be inefficient. However, if X = 0, the variances are the same. This is
because the estimators are then identical. Y b2X reduces to Y .

A2.4 The variance of the estimator is u2 / Xi2 whereas that of the estimator:
P

P
(Xi X)(Yi Y )
P
(Xi X)2

is:
u2 u2
P =P 2.
(Xi X)2 Xi2 nX

Thus, provided X 6= 0, u2 / Xi2 is more efficient than:

P

P
(Xi X)(Yi Y )
P
(Xi X)2

if 1 = 0 because it is unbiased and has a smaller variance. It is the OLS estimator

in this case.

If X = 0, the estimators are equally efficient because the population variance

expressions are identical. The reason for this is that the estimators are now
identical:
P P P P P
(Xi X)(Yi Y ) Xi (Yi Y ) Xi Yi Y Xi Xi Yi
= P 2 = P 2 P 2 = P 2
Xi Xi Xi Xi
P
(Xi X)2

P
since Xi = nX = 0.

Returning to the general case, if there is little variation in X in the sample,

2
P
P(Xi X) may be P small and hence the population variance of
(Xi X)(Yi Y )/ (Xi X)2 may be large. Thus using a criterion such as mean
square error, b2 may be preferable if the bias is small.

A2.5 The t statistic for the coefficient of EXP is 58.57, very highly significant. There is
little point performing a t test on the intercept, given that it has no plausible
meaning. The F statistic is 3431.0, very highly significant. Since this is a simple
regression model, the two tests are equivalent.

A2.6 The slope coefficient for every category is significantly different from zero at a very
high significance level. (The F test is equivalent to the t test on the slope
coefficient.)

54
 2.6. Answers to the additional exercises

EXP

n b2 s.e.(b2 ) t R2 F
ADM 2,815 0.0235 0.0008 28.86 0.228 832.8
CLOT 4,500 0.0316 0.0010 30.99 0.176 960.6
DOM 1,661 0.0409 0.0026 16.02 0.134 256.6
EDUC 561 0.1202 0.0090 13.30 0.241 177.0
ELEC 5,828 0.0131 0.0004 35.70 0.180 1274.8
FDAW 5,102 0.0527 0.0010 52.86 0.354 2794.7
FDHO 6,334 0.0627 0.0011 58.57 0.351 3431.0
FOOT 1,827 0.0058 0.0005 12.78 0.082 163.4
FURN 487 0.0522 0.0070 7.44 0.102 55.3
GASO 5,710 0.0373 0.0008 46.89 0.278 2198.5
HEAL 4,802 0.0574 0.0018 31.83 0.174 1013.4
HOUS 6,223 0.1976 0.0027 74.16 0.469 5499.9
LIFE 1,253 0.0193 0.0016 11.86 0.101 140.7
LOCT 692 0.0068 0.0010 6.59 0.059 43.5
MAPP 399 0.0329 0.0049 6.72 0.102 45.1
PERS 3,817 0.0069 0.0002 32.15 0.213 1033.4
READ 2,287 0.0048 0.0003 16.28 0.104 265.1
SAPP 1,037 0.0045 0.0007 6.03 0.034 36.4
TELE 5,788 0.0160 0.0003 46.04 0.268 2119.7
TEXT 992 0.0040 0.0006 7.32 0.051 53.5
TOB 1,155 0.0165 0.0016 10.56 0.088 111.6
TOYS 2,504 0.0145 0.0010 14.34 0.076 205.7
TRIP 516 0.0466 0.0043 10.84 0.186 117.5

A2.7 The t statistic, 13.29, is very highly significant. (The F test is equivalent.)

A2.8 The t statistic for height, 3.04, suggests that the effect of height on earnings is
highly significant, despite the very low R2 . In principle the estimate of an extra 41
cents of hourly earnings for every extra inch of height could have been a purely
random result of the kind that one obtains with nonsense models. However, the
fact that it is apparently highly significant causes us to look for other explanations,
the most likely one being that suggested in the answer to Exercise A1.4. Of course,
we would not attempt to test the negative constant.

A2.9 One could justify a one-sided test on the slope coefficient in the regression of the
rate of growth of employment on the rate of growth of GDP on the grounds that an
increase in the rate of growth of GDP is unlikely to cause a decrease in the rate of
growth of employment.

A2.10 One could justify a one-sided test on the slope coefficient in the regression of
weight on height in Exercise 1.9 on the grounds that an increase in height is
unlikely to cause a decrease in weight.

55
 2. Properties of the regression coefficients and hypothesis testing

A2.11 The standard error of the coefficient of GDP. This is given by:
p
b2

r  u
2

Gi G
P

bu2 , the estimator of the variance of the disturbance term, is b2

P
where u i /(n 2).
Since RSS is unchanged, bu2 =
bu2 .

We saw in Exercise A1.6 that Gi G = Gi G for all i. Hence the new standard
error is given by: p
b2

r  u 
P 2
Gi G

and is unchanged.
ESS
F =
RSS/(n 2)
where: 2
X
ESS = explained sum of squares = Ybi Yb .

bi = u
Since u bi , Ybi = Ybi and ESS is unchanged. We saw in Exercise A1.6 that RSS
is unchanged. Hence F is unchanged.

A2.12 The standard error of the coefficient of GDP. This is given by:
p
b2

r  u
2

Gi G
P

bu2 , the estimator of the variance of the disturbance term, is

P 2
where bi /(n 2).
u
We saw in Exercise 1.7 that u bi = u
bi and so RSS is unchanged. Hence bu2 =
bu2 .
Thus the new standard error is given by:
p p
bu2
1 bu2

r  = r 2 = 0.005.
P 2 2 P
2Gi 2G Gi G

F = ESS/(RSS/(n 2)) where:

X
2
ESS = explained sum of squares = Ybi Yb .

bi = u
Since u bi , Ybi = Ybi and ESS is unchanged. Hence F is unchanged.

A2.13 One way of demonstrating that Ybi = Ybi Y :

Ybi = b1 + b2 Xi = b2 (Xi X)
   
Yi Y = (1 + 2 Xi ) Y = Y 2X + 2 Xi Y = 2 Xi X .
b b b b b b

56
 2.6. Answers to the additional exercises

Demonstration that the residuals are the same:

   
ubi = Yi Ybi = Yi Y Ybi Y = u
bi .

Demonstration that the OLS estimator of the variance of the disturbance term in
(2) is equal to that in (1):
P 2 P 2
2 u
bi u
bi

bu = = =bu2 .
n2 n2
The standard error of the slope coefficient in (2) is equal to that in (1).

bu2
bu2
bu2

b2b = P 
2 = P 2 = P  b2b2 .
2 =
2 Xi
Xi X Xi X

Hence the standard errors are the same.

Demonstration that R2 in (2) is equal to R2 in (1):
P  b b 2
Yi Y
R2 = P  .
2
Yi Y

Ybi = Ybi Y and Yb = Y . Hence Yb = 0. Y = Y Y = 0. Hence:

P  b 2 P b 2
Yi Yi Y
R2 = P 2 = P  2
2 = R .
(Yi ) Yi Y

The reason that specification (2) of the second researcher is incorrect is that the
model does not include an intercept.
If the second researcher had fitted (3) instead of (2), this would not in fact have
affected his estimator of 2 . Using (3), the researcher should have estimated 2 as:
P
X Y
b2 = P i 2i .
Xi
However, Exercise 1.16 demonstrates that, effectively, he has done exactly this.
Hence the estimator will be the same. It follows that dropping the unnecessary
intercept would not have led to a gain in efficiency.

A2.14 We have: !

bX b Xi X 1 b
Ybi = b2 Xi = 2 = 2 (Xi X)

bY
bX
bY
and:
Ybi = b1 + b2 Xi = (Y b2X) + b2 Xi = Y + b2 (Xi X).
Hence:
1 b
Ybi = (Yi Y ).

bY

57
 2. Properties of the regression coefficients and hypothesis testing

Also:
1 1 b 1 1
bi = Yi Ybi =
u (Yi Y ) (Yi Y ) = (Yi Ybi ) = u
bi

bY
bY
bY
bY
and: v
u  2
s u 1 1
P 2
1 2 u
P
n2
u i
u Y n2
bi
bX
s.e.(b2 ) =
b
s.e.(b2 ).
b
P 2
= u
2 =

 
(Xi X )
t P
X i X b Y

bX

Given the expressions for b2 and s.e.(b2 ), the t statistic for b2 is the same as that
for b2 . Hence the F statistic will be the same and R2 will be the same.

A2.15 The inclusion of the fifth observation does not cause the model to be misspecified
or the regression model assumptions to be violated, so retaining it in the sample
will not give rise to biased estimates. There would be noadvantages in dropping it
P 2
and there would be one major disadvantage. Xi X would be greatly
reduced and hence the variances of the coefficients would be increased, adversely
affecting the precision of the estimates.
This said, in practice one would wish to check whether it is sensible to assume that
the model relating Y to X for the other observations really does apply to the
observation corresponding to X5 as well. This question can be answered only by
being familiar with the context and having some intuitive understanding of the
relationship between Y and X.

58
Chapter 3
Multiple regression analysis

3.1 Overview
This chapter introduces regression models with more than one explanatory variable.
Specific topics are treated with reference to a model with just two explanatory
variables, but most of the concepts and results apply straightforwardly to more general
models. The chapter begins by showing how the least squares principle is employed to
derive the expressions for the regression coefficients and how the coefficients should be
interpreted. It continues with a discussion of the precision of the regression coefficients
and tests of hypotheses relating to them. Next comes multicollinearity, the problem of
discriminating between the effects of individual explanatory variables when they are
closely related. The chapter concludes with a discussion of F tests of the joint
explanatory power of the explanatory variables or subsets of them, and shows how a t
test can be thought of as a marginal F test.

3.2 Learning outcomes

After working through the corresponding chapter in the text, studying the
corresponding slideshows, and doing the starred exercises in the text and the additional
exercises in this subject guide, you should be able to explain what is meant by:

the principles behind the derivation of multiple regression coefficients (but you are
not expected to learn the expressions for them or to be able to reproduce the
mathematical proofs)
how to interpret the regression coefficients
the FrischWaughLovell graphical representation of the relationship between the
dependent variable and one explanatory variable, controlling for the influence of
the other explanatory variables
the properties of the multiple regression coefficients
what factors determine the population variance of the regression coefficients
what is meant by multicollinearity
what measures may be appropriate for alleviating multicollinearity
what is meant by a linear restriction
the F test of the joint explanatory power of the explanatory variables

59
 3. Multiple regression analysis

the F test of the explanatory power of a group of explanatory variables

why t tests on the slope coefficients are equivalent to marginal F tests.

You should know the expression for the population variance of a slope coefficient in a
multiple regression model with two explanatory variables.

3.3 Additional exercises

A3.1 The output shows the result of regressing FDHO, expenditure on food consumed at
home, on EXP, total household expenditure, and SIZE, number of persons in the
household, using the CES data set. Provide an interpretation of the regression
coefficients and perform appropriate tests.

. reg FDHO EXP SIZE if FDHO>0

Source | SS df MS Number of obs = 6334

-------------+------------------------------ F( 2, 6331) = 2257.59
Model | 1.1521e+09 2 576056293 Prob > F = 0.0000
Residual | 1.6154e+09 6331 255164.645 R-squared = 0.4163
-------------+------------------------------ Adj R-squared = 0.4161
Total | 2.7676e+09 6333 437006.15 Root MSE = 505.14

------------------------------------------------------------------------------
FDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
EXP | .056366 .0010435 54.02 0.000 .0543204 .0584116
SIZE | 115.1636 4.341912 26.52 0.000 106.652 123.6752
_cons | 130.5997 13.53959 9.65 0.000 104.0575 157.1419
------------------------------------------------------------------------------

A3.2 Perform a regression parallel to that in Exercise A3.1 for your CES category of
expenditure, provide an interpretation of the regression coefficients and perform
appropriate tests. Delete observations where expenditure on your category is zero.

A3.3 The output shows the result of regressing FDHOPC, expenditure on food
consumed at home per capita, on EXPPC, total household expenditure per capita,
and SIZE, number of persons in the household, using the CES data set. Provide an
interpretation of the regression coefficients and perform appropriate tests.

. reg FDHOPC EXPPC SIZE if FDHO>0

Source | SS df MS Number of obs = 6334

-------------+------------------------------ F( 2, 6331) = 1572.95
Model | 202590496 2 101295248 Prob > F = 0.0000
Residual | 407705728 6331 64398.3143 R-squared = 0.3320
-------------+------------------------------ Adj R-squared = 0.3317
Total | 610296223 6333 96367.6336 Root MSE = 253.77

60
 3.3. Additional exercises

------------------------------------------------------------------------------
FDHOPC | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
EXPPC | .0480294 .0010064 47.72 0.000 .0460564 .0500023
SIZE | -26.45917 2.253999 -11.74 0.000 -30.87777 -22.04057
_cons | 283.2498 8.412603 33.67 0.000 266.7582 299.7413
------------------------------------------------------------------------------

A3.4 Perform a regression parallel to that in Exercise A3.3 for your CES category of
expenditure. Provide an interpretation of the regression coefficients and perform
appropriate tests.

A3.5 The output shows the result of regressing FDHOPC, expenditure on food
consumed at home per capita, on EXPPC, total household expenditure per capita,
and SIZEAM, SIZEAF, SIZEJM, SIZEJF, and SIZEIN, numbers of adult males,
adult females, junior males, junior females, and infants, respectively, in the
household, using the CES data set. Provide an interpretation of the regression
coefficients and perform appropriate tests.
. reg FDHOPC EXPPC SIZEAM SIZEAF SIZEJM SIZEJF SIZEIN if FDHO>0

Source | SS df MS Number of obs = 6334

-------------+------------------------------ F( 6, 6327) = 524.59
Model | 202746894 6 33791149 Prob > F = 0.0000
Residual | 407549329 6327 64414.3084 R-squared = 0.3322
-------------+------------------------------ Adj R-squared = 0.3316
Total | 610296223 6333 96367.6336 Root MSE = 253.8

------------------------------------------------------------------------------
FDHOPC | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
EXPPC | .0479717 .0010087 47.56 0.000 .0459943 .0499491
SIZEAM | -25.77747 4.757056 -5.42 0.000 -35.10291 -16.45203
SIZEAF | -32.38649 5.065782 -6.39 0.000 -42.31714 -22.45584
SIZEJM | -20.24693 5.731645 -3.53 0.000 -31.4829 -9.010967
SIZEJF | -26.66374 6.122262 -4.36 0.000 -38.66544 -14.66203
SIZEIN | -28.6047 11.75666 -2.43 0.015 -51.65174 -5.557656
_cons | 287.5695 9.280372 30.99 0.000 269.3769 305.7622
------------------------------------------------------------------------------

A3.6 Perform a regression parallel to that in Exercise A3.5 for your CES category of
expenditure. Provide an interpretation of the regression coefficients and perform
appropriate tests.

A3.7 A researcher hypothesises that, for a typical enterprise, V , the logarithm of value
added per worker, is related to K, the logarithm of capital per worker, and S, the
logarithm of the average years of schooling of the workers, the relationship being:
V = 1 + 2 K + 3 S + u
where u is a disturbance term that satisfies the usual regression model
assumptions. She fits the relationship (1) for a sample of 25 manufacturing
enterprises, and (2) for a sample of 100 services enterprises. The table provides
some data on the samples.

61
 3. Multiple regression analysis

(1) (2)
Manufacturing Services
sample sample
Number of enterprises 25 100
Estimate of variance of u 0.16 0.64
Mean square deviation of K 4.00 16.00
Correlation between K and S 0.60 0.60

P 2
1
The mean square deviation of K is defined as n
Ki K , where n is the
number of enterprises in the sample and K is the average value of K in the sample.
The researcher finds that the standard error of the coefficient of K is 0.050 for the
manufacturing sample and 0.025 for the services sample. Explain the difference
quantitatively, given the data in the table.

A3.8 A researcher is fitting earnings functions using a sample of data relating to

individuals born in the same week in 1958. He decides to relate Y , gross hourly
earnings in 2001, to S, years of schooling, and PWE, potential work experience,
using the semilogarithmic specification:

log Y = 1 + 2 S + 3 PWE + u

where u is a disturbance term assumed to satisfy the regression model assumptions.

PWE is defined as age years of schooling 5. Since the respondents were all aged
43 in 2001, this becomes:

PWE = 43 S 5 = 38 S.

The researcher finds that it is impossible to fit the model as specified. Stata output
for his regression is reproduced below:

. reg LGY S PWE

Source | SS df MS Number of obs = 5660

-------------+------------------------------ F( 1, 5658) = 1232.62
Model | 237.170265 1 237.170265 Prob > F = 0.0000
Residual | 1088.66373 5658 .192411405 R-squared = 0.1789
-------------+------------------------------ Adj R-squared = 0.1787
Total | 1325.834 5659 .234287682 Root MSE = .43865

------------------------------------------------------------------------------
LGY | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1038011 .0029566 35.11 0.000 .0980051 .1095971
PWE | (dropped)
_cons | .5000033 .0373785 13.38 0.000 .4267271 .5732795
------------------------------------------------------------------------------

Explain why the researcher was unable to fit his specification.

Explain how the coefficient of S might be interpreted.

62
 3.4. Answers to the starred exercises in the textbook

3.4 Answers to the starred exercises in the textbook

3.5 Explain why the intercept in the regression of EEARN on ES is equal to zero.
Answer:
The intercept is calculated as EEARN b2ES. However, since the mean of the
residuals from an OLS regression is zero, both EEARN and ES are zero, and hence
the intercept is zero.

3.6 Show that, in the general case, the mean of the residuals from a fitted OLS
multiple regression is equal to zero, provided that an intercept is included in the
specification. Note: This is an extension of one of the useful results in Section 1.5.
Answer:
If the model is:
Y = 1 + 2 X2 + + k Xk + u
b1 = Y b2X2 bkXk .
For observation i we have:
bi = Yi Ybi = Yi b1 b2 X2i bk Xki .
u
Hence:
b = Y b1 b2X2 bkXk
u
h i
= Y Y b2X2 bkXk b2X2 bkXk = 0.

3.16 A researcher investigating the determinants of the demand for public transport in a
certain city has the following data for 100 residents for the previous calendar year:
expenditure on public transport, E, measured in dollars; number of days worked,
W ; and number of days not worked, NW. By definition NW is equal to 365 W .
He attempts to fit the following model:
E = 1 + 2 W + 3 N W + u.
Explain why he is unable to fit this equation. (Give both intuitive and technical
explanations.) How might he resolve the problem?
Answer:
There is exact multicollinearity since there is an exact linear relationship between
W , NW and the constant term. As a consequence it is not possible to tell whether
variations in E are attributable to variations in W or variations in NW, or both.
Noting that N Wi N W = (Wi W), we have:
P  P 2 P   P  
Ei E Wi W N Wi N W Ei E N Wi N W Wi W N Wi N W
b2 = P 2 P  2 P   2
Wi W N Wi N W Wi W N Wi N W

P  P 2 P   P  
Ei E Wi W Wi W Ei E Wi + W Wi W Wi + W
= P 2 P  2 P   2
Wi W Wi W Wi W Wi + W

0
= .
0

63
 3. Multiple regression analysis

One way of dealing with the problem would be to drop N W from the regression.
The interpretation of b2 now is that it is an estimate of the extra expenditure on
transport per day worked, compared with expenditure per day not worked.

3.21 The researcher in Exercise 3.16 decides to divide the number of days not worked
into the number of days not worked because of illness, I, and the number of days
not worked for other reasons, O. The mean value of I in the sample is 2.1 and the
mean value of O is 120.2. He fits the regression (standard errors in parentheses):
b = 9.6 + 2.10W + 0.45O
E R2 = 0.72
(8.3) (1.98) (1.77)
Perform t tests on the regression coefficients and an F test on the goodness of fit of
the equation. Explain why the t tests and F test have different outcomes.
Answer:
Although there is not an exact linear relationship between W and O, they must
have a very high negative correlation because the mean value of I is so small.
Hence one would expect the regression to be subject to multicollinearity, and this is
confirmed by the results. The t statistics for the coefficients of W and O are only
1.06 and 0.25, respectively, but the F statistic:
0.72/2
F (2, 97) = = 124.7
(1 0.72)/97
is greater than the critical value of F at the 0.1 per cent level, 7.41.

3.5 Answers to the additional exercises

A3.1 The regression indicates that 5.6 cents out of the marginal expenditure dollar is
spent on food consumed at home, and that expenditure on this category increases
by $115 for each individual in the household, keeping total expenditure constant.
Both of these effects are very highly significant. Just over 40 per cent of the
variance in FDHO is explained by EXP and SIZE. The intercept has no plausible
interpretation.

A3.2 With the exception of LOCT, all of the categories have positive coefficients for
EXP, with high significance levels, but the SIZE effect varies:
Positive, significant at the 1 per cent level: FDHO, TELE, CLOT, FOOT,
GASO.
Positive, significant at the 5 per cent level: LOCT.
Negative, significant at the 1 per cent level: TEXT, FEES, READ.
Negative, significant at the 5 per cent level: SHEL, EDUC.
Not significant: FDAW, DOM, FURN, MAPP, SAPP, TRIP, HEAL, ENT,
TOYS, TOB.
At first sight it may seem surprising that SIZE has a significant negative effect for
some categories. The reason for this is that an increase in SIZE means a reduction

64
 3.5. Answers to the additional exercises

in expenditure per capita, if total household expenditure is kept constant, and thus
SIZE has a (negative) income effect in addition to any direct effect. Effectively
poorer, the larger household has to spend more on basics and less on luxuries. To
determine the true direct effect, we need to eliminate the income effect, and that is
the point of the re-specification of the model in the next exercise.

EXP SIZE

n b2 s.e.(b2 ) b3 s.e.(b3 ) R2 F
ADM 2,815 0.0238 0.0008 8.09 4.19 0.230 418.7
CLOT 4,500 0.0309 0.0010 16.39 4.50 0.178 488.2
DOM 1,661 0.0388 0.0026 52.34 14.06 0.141 136.2
EDUC 561 0.1252 0.0090 179.23 48.92 0.258 97.2
ELEC 5,828 0.0121 0.0004 18.92 1.57 0.199 725.5
FDAW 5,102 0.0538 0.0010 20.72 4.47 0.357 1,413.7
FDHO 6,334 0.0564 0.0010 115.16 4.34 0.416 2,257.6
FOOT 1,827 0.0056 0.0005 3.24 2.05 0.083 83.0
FURN 487 0.0541 0.0071 61.87 35.92 0.108 29.3
GASO 5,710 0.0347 0.0008 50.29 3.40 0.305 1,250.9
HEAL 4,802 0.0580 0.0019 9.96 8.60 0.175 507.4
HOUS 6,223 0.1997 0.0027 38.78 11.41 0.470 2,760.4
LIFE 1,253 0.0198 0.0017 9.01 8.99 0.102 70.9
LOCT 692 0.0062 0.0011 14.61 4.72 0.072 26.8
MAPP 399 0.0309 0.0050 44.48 23.94 0.110 24.4
PERS 3,817 0.0070 0.0002 2.17 1.03 0.214 519.4
READ 2,287 0.0049 0.0003 1.06 1.58 0.104 132.7
SAPP 1,037 0.0046 0.0008 3.12 3.99 0.035 18.5
TELE 5,788 0.0150 0.0004 17.92 1.47 0.287 1,161.2
TEXT 992 0.0041 0.0006 0.71 2.90 0.051 26.8
TOB 1,155 0.0161 0.0016 6.79 6.24 0.089 56.4
TOYS 2,504 0.0140 0.0010 12.19 4.88 0.078 106.2
TRIP 516 0.0450 0.0045 37.48 31.21 0.188 59.5

A3.3 Another surprise, perhaps. The purpose of this specification is to test whether
household size has an effect on expenditure per capita on food consumed at home,
controlling for the income effect of variations in household size mentioned in the
answer to Exercise A3.2. Expenditure per capita on food consumed at home
increases by 4.8 cents out of the marginal dollar of total household expenditure per
capita. Now SIZE has a very significant negative effect. Expenditure per capita on
FDHO decreases by $26 per year for each extra person in the household, suggesting
that larger households are more efficient than smaller ones with regard to
expenditure on this category, the effect being highly significant. R2 is lower than in
Exercise A3.1, but a comparison is invalidated by the fact that the dependent
variable is different.

A3.4 Nearly all of the categories have negative SIZE effects, the majority highly
significant. One explanation of the negative effects could be economies of scale, but

65
 3. Multiple regression analysis

this is not plausible in the case of some. Another might be family composition
larger families having more children. In the case of DOM, SIZE has a positive
effect, significant at the 5 per cent level. Again, this might be attributable to larger
families having more children and needing greater expenditure on childcare.

EXP SIZE

n b2 s.e.(b2 ) b3 s.e.(b3 ) R2 F
ADM 2,815 0.0244 0.0008 2.56 2.26 0.251 470.4
CLOT 4,500 0.0324 0.0012 1.07 2.91 0.151 400.8
DOM 1,661 0.0311 0.0025 18.54 7.35 0.086 78.1
EDUC 561 0.1391 0.0108 31.92 27.57 0.290 113.7
ELEC 5,828 0.0117 0.0004 17.53 0.89 0.247 953.9
FDAW 5,102 0.0528 0.0011 13.51 2.53 0.375 1,526.3
FDHO 6,334 0.0480 0.0010 26.46 2.25 0.332 1,573.0
FOOT 1,827 0.0068 0.0005 8.13 1.11 0.194 219.5
FURN 487 0.0935 0.0091 3.40 26.82 0.216 66.6
GASO 5,710 0.0308 0.0008 12.43 1.80 0.255 976.5
HEAL 4,802 0.0597 0.0020 34.16 4.99 0.197 588.5
HOUS 6,223 0.2127 0.0030 48.86 6.67 0.501 3,123.3
LIFE 1,253 0.0205 0.0017 10.33 4.65 0.131 94.4
LOCT 692 0.0062 0.0010 9.06 2.54 0.098 37.4
MAPP 399 0.0384 0.0051 15.52 12.32 0.171 41.0
PERS 3,817 0.0071 0.0003 3.96 0.63 0.228 564.0
READ 2,287 0.0052 0.0003 3.60 0.84 0.154 208.1
SAPP 1,037 0.0076 0.0010 6.71 2.61 0.090 51.1
TELE 5,788 0.0139 0.0003 9.77 0.75 0.307 1,282.6
TEXT 992 0.0041 0.0005 8.96 1.45 0.138 79.2
TOB 1,155 0.0220 0.0019 22.68 3.55 0.187 132.1
TOYS 2,504 0.0216 0.0012 8.86 2.92 0.141 205.7
TRIP 516 0.0361 0.0043 16.33 16.32 0.150 45.2

A3.5 The coefficients of the SIZE variables are fairly similar, suggesting that household
composition is not important for this category of expenditure.

A3.6 The regression results for this specification are summarised in the table below. In
the case of SHEL, the regression indicates that the SIZE effect is attributable to
SIZEAM. To investigate this further, the regression was repeated: (1) restricting
the sample to households with at least one adult male, and (2) restricting the
sample to households with either no adult male or just 1 adult male. The first
regression produces a negative effect for SIZEAM, but it is smaller than with the
whole sample and not significant. In the second regression the coefficient of
SIZEAM jumps dramatically, from $424 to $793, suggesting very strong
economies of scale for this particular comparison.
As might be expected, the SIZE composition variables on the whole do not appear
to have significant effects if the SIZE variable does not in Exercise A3.4. The

66
 3.5. Answers to the additional exercises

results for TOB are puzzling, in that the apparent economies of scale do not
appear to be related to household composition.

Category ADM CLOT DOM EDUC ELEC FDAW FDHO FOOT

EXP 0.0245 0.0309 0.0422 0.1191 0.0120 0.0531 0.0561 0.0056
(0.0008) (0.0011) (0.0026) (0.0092) (0.0004) (0.0010) (0.0011) (0.0005)
SIZEAM 37.17 12.84 141.47 120.11 23.40 29.36 129.69 2.65
(9.22) (10.33) (32.71) (107.51) (3.44) (9.88) (9.64) (4.71)
SIZEAF 40.47 12.26 67.26 58.21 35.73 45.07 105.17 9.40
(9.52) (10.95) (34.79) (107.96) (3.60) (10.17) (9.96) (5.25)
SIZEJM 1.33 17.11 114.68 413.28 12.53 24.45 126.94 1.23
(9.86) (11.41) (31.91) (107.79) (4.06) (11.53) (11.35) (4.99)
SIZEJF 48.55 29.98 93.82 287.35 8.93 26.03 105.01 6.32
(10.54) (12.15) (33.66) (103.15) (4.31) (12.05) (12.07) (5.01)
SIZEIN 34.51 2.08 441.46 123.20 4.05 61.38 95.90 16.33
(22.79) (22.20) (59.10) (289.63) (8.36) (23.77) (23.34) (11.07)
R2 0.243 0.179 0.184 0.278 0.204 0.361 0.417 0.086
F 150.1 163.0 62.1 35.6 249.2 480.2 753.6 28.5
n 2,815 4,500 1,661 561 5,828 5,102 6,334 1,827

Category FURN GASO HEAL HOUS LIFE LOCT MAPP PERS

EXP 0.0547 0.0341 0.0579 0.2022 0.0195 0.0061 0.0321 0.0071
(0.0072) (0.0008) (0.0019) (0.0027) (0.0017) (0.0011) (0.0051) (0.0002)
SIZEAM 119.30 90.70 3.01 175.23 10.54 12.02 2.41 13.99
(81.65) (7.47) (18.25) (25.24) (19.50) (9.90) (54.58) (2.23)
SIZEAF 55.42 52.23 89.64 111.39 25.43 19.16 0.75 12.33
(93.37) (7.79) (19.10) (26.12) (20.83) (10.61) (63.11) (2.34)
SIZEJM 27.44 30.83 62.83 52.32 23.28 6.41 131.15 3.33
(87.24) (8.72) (22.56) (29.65) (21.17) (12.81) (61.75) (2.59)
SIZEJF 15.06 46.24 57.94 34.65 15.65 32.97 24.87 2.10
(89.23) (9.27) (23.96) (31.58) (22.98) (15.85) (64.61) (2.71)
SIZEIN 146.90 8.90 109.08 119.91 116.37 33.48 26.25 11.30
(160.29) (18.02) (46.46) (61.40) (46.00) (25.82) (139.98) (5.32)
R2 0.110 0.310 0.181 0.475 0.109 0.077 0.116 0.228
F 9.9 427.6 177.0 937.6 25.3 9.6 8.6 187.4
n 487 5,710 4,802 6,223 1,253 692 399 3,817

Category READ SAPP TELE TEXT TOB TOYS TRIP

EXP 0.0049 0.0046 0.0148 0.0040 0.0151 0.0148 0.0448
(0.0003) (0.0008) (0.0004) (0.0006) (0.0016) (0.0010) (0.0045)
SIZEAM 6.37 1.64 29.33 7.42 30.92 39.66 64.35
(3.46) (8.26) (3.25) (5.98) (13.49) (11.19) (59.55)
SIZEAF 1.69 8.95 35.59 2.58 22.09 1.30 4.87
(3.80) (9.65) (3.38) (6.77) (13.68) (12.49) (71.23)
SIZEJM 0.63 13.21 6.38 15.90 17.42 42.46 81.61
(3.93) (9.73) (3.78) (7.51) (16.52) (11.30) (79.96)
SIZEJF 4.73 1.17 12.74 4.92 45.12 19.34 102.45
(4.26) (10.88) (4.06) (7.50) (16.82) (11.71) (91.86)
SIZEIN 18.98 19.58 26.42 19.17 2.92 50.91 294.14
(8.56) (18.58) (7.82) (14.13) (32.83) (22.49) (157.82)
R2 0.108 0.038 0.296 0.059 0.100 0.090 0.197
F 45.8 6.7 404.9 10.4 21.2 41.2 20.8
n 2,287 1,037 5,788 992 368 2,504 516

67
 3. Multiple regression analysis

A3.7 The standard error is given by:

1 1 1
bu p
s.e.(b2 ) = q .
n MSD(K) 2
1 rK,S

Data Factors
manufacturing services manufacturing services
sample sample sample sample
Number of 25 100 0.20 0.10
enterprises
Estimate of 0.16 0.64 0.40 0.80
variance of u
Mean square 4 16 0.50 0.25
deviation of K
Correlation 0.6 0.6 1.25 1.25
between K and S
Standard errors 0.050 0.025

The table shows the four factors for the two sectors. Other things being equal, the
larger number of enterprises and the greater MSD of K would separately cause the
standard error of b2 for the services sample to be half that in the manufacturing
sample. However, the larger estimate of the variance of u would, taken in isolation,
causes it to be double. The net effect, therefore, is that it is half.

A3.8 Exact multicollinearity. An extra year of schooling implies one fewer year of
potential work experience. Thus the coefficient of schooling estimates the
proportional increase in earnings associated with an additional year of schooling,
taking account of the loss of a year of potential work experience.

68
Chapter 4
Transformations of variables

4.1 Overview
This chapter shows how least squares regression analysis can be extended to fit
nonlinear models. Sometimes an apparently nonlinear model can be linearised by taking
logarithms. Y = 1 X 2 and Y = 1 e2 X are examples. Because they can be fitted using
linear regression analysis, they have proved very popular in the literature, there usually
being little to be gained from using more sophisticated specifications. If you plot
earnings on schooling, using the EAWE data set, or expenditure on a given category of
expenditure on total household expenditure, using the CES data set, you will see that
there is so much randomness in the data that one nonlinear specification is likely to be
just as good as another, and indeed a linear specification may not be obviously inferior.
Often the real reason for preferring a nonlinear specification to a linear one is that it
makes more sense theoretically. The chapter shows how the least squares principle can
be applied when the model cannot be linearised.

4.2 Learning outcomes

After working through the corresponding chapter in the text, studying the
corresponding slideshows, and doing the starred exercises in the text and the additional
exercises in this subject guide, you should be able to:

explain the difference between nonlinearity in parameters and nonlinearity in

variables

explain why nonlinearity in parameters is potentially a problem while nonlinearity

in variables is not

define an elasticity

explain how to interpret an elasticity in simple terms

perform basic manipulations with logarithms

interpret the coefficients of semi-logarithmic and logarithmic regressions

explain why the coefficients of semi-logarithmic and logarithmic regressions should

not be interpreted using the method for regressions in natural units described in
Chapter 1

perform a RESET test of functional misspecification

69
4. Transformations of variables

explain the role of the disturbance term in a nonlinear model

explain how in principle a nonlinear model that cannot be linearised may be fitted

perform a transformation for comparing the fits of models with linear and
logarithmic dependent variables.

4.3 Further material

BoxCox tests of functional specification

The theory behind the procedure for discriminating between a linear and a logarithmic
specification of the dependent variable is explained in the Appendix to Chapter 10 of
the text. However, the exposition there is fairly brief. An expanded version is offered
here. It should be skipped on first reading because it makes use of material on maximum
likelihood estimation. To keep the mathematics uncluttered, the theory will be
described in the context of the simple regression model, where we are choosing between:

Y = 1 + 2 X + u

and:
log Y = 1 + 2 X + u.
It generalises with no substantive changes to the multiple regression model.
The two models are actually special cases of the more general model:

Y1
Y = = 1 + 2 X + u

with = 1 yielding the linear model (with an unimportant adjustment to the intercept)
and = 0 yielding the logarithmic specification at the limit as tends to zero.
Assuming that u is iid (independently and identically distributed) N (0, 2 ), the density
function for ui is:
1 2 2
f (ui ) = eui /2
2
and hence the density function for Yi is:
1 2 2
f (Yi ) = e(Yi 1 2 Xi ) /2 .
2
From this we obtain the density function for Yi :
 
1 2 2 Y i  = 1 e(Yi 1 2 Xi )2 /22 Y 1 .
f (Yi ) = e(Yi 1 2 Xi ) /2 
 
i
2 Yi  2
 
The factor  Y is the Jacobian for relating the density function of Yi to that of Yi .
 i 
Yi 
Hence the likelihood function for the parameters is:
 n Y n n
1 (Yi 1 2 Xi )2 /2 2
Y
L(1 , 2 , , ) = e Yi1
2 i=1 i=1

70
 4.3. Further material

and the log-likelihood is:

n n
n X 1 X
log L(1 , 2 , , ) = log 2 2 2
(Y i 1 2 X i ) 2
+ log Yi1
2 i=1
2 i=1
n n
n 1 X X
= log 2 n log 2 (Yi 1 2 Xi )2 + ( 1) log Yi .
2 2 i=1 i=1

From the first-order condition log L/ = 0, we have:

n
n 1 X
+ 3 (Yi 1 2 Xi )2 = 0
i=1

giving:
n
2 1X

b = (Yi 1 2 Xi )2 .
n i=1
Substituting into the log-likelihood function, we obtain the concentrated log-likelihood:
n n
n n 1X n X
log L(1 , 2 , ) = log 2 log (Yi 1 2 Xi )2 + ( 1) log Yi .
2 2 n i=1 2 i=1

The expression can be simplified (Zarembka, 1968) by working with Yi rather than Yi ,
where Yi is Yi divided by YGM , the geometric mean of the Yi in the sample, for:
n
X n
X n
X
log Yi = log(Yi /YGM ) = (log Yi log YGM )
i=1 i=1 i=1

n n n
!1/n
X X Y
= log Yi n log YGM = log Yi n log Yi
i=1 i=1 i=1

n n
! n n
X Y X X
= log Yi log Yi = log Yi log Yi = 0.
i=1 i=1 i=1 i=1

With this simplification, the log-likelihood is:

  n
n 1 n X
log L(1 , 2 , ) = log 2 + log + 1 log (Yi 1 2 Xi )2
2 n 2 i=1

and it will be maximised when 1 , 2 and are chosen so as to minimise

n
(Yi 1 2 Xi )2 , the residual sum of squares from a least squares regression of the
P
i=1
scaled, transformed Y on X. One simple procedure is to perform a grid search, scaling
and transforming the data on Y for a range of values of and choosing the value that
leads to the smallest residual sum of squares (Spitzer, 1982).
A null hypothesis = 0 can be tested using a likelihood ratio test in the usual way.
Under the null hypothesis, the test statistic 2(log L log L0 ) will have a chi-squared
distribution with one degree of freedom, where log L is the unconstrained log-likelihood
and L0 is the constrained one. Note that, in view of the preceding equation:
2(log L log L0 ) = n(log RSS0 log RSS )

71
 4. Transformations of variables

where RSS0 and RSS are the residual sums of squares from the constrained and
unconstrained regressions with Y .
The most obvious tests are = 0 for the logarithmic specification and = 1 for the
linear one. Note that it is not possible to test the two hypotheses directly against each
other. As with all tests, one can only test whether a hypothesis is incompatible with the
sample result. In this case we are testing whether the log-likelihood under the
restriction is significantly smaller than the unrestricted log-likelihood. Thus, while it is
possible that we may reject the linear but not the logarithmic, or vice versa, it is also
possible that we may reject both or fail to reject both.
Example
400

300

200

100

0
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

The figure shows the residual sum of squares for values of from 1 to 1 for the wage
equation example described in Section 4.2 in the text. The maximum likelihood estimate
is 0.10, with RSS = 130.3. For the linear and logarithmic specifications, RSS was 217.0
and 131.4, respectively, with likelihood ratio statistics 500(log 217.0 log 130.3) = 255.0
and 500(log 131.4 log 130.3) = 4.20.The logarithmic specification is clearly much to be
preferred, but even it is rejected at the 5 per cent level, with 2 (1) = 3.84.

4.4 Additional exercises

A4.1 Is expenditure on your category per capita related to total expenditure per capita?
An alternative model specification.
Define a new variable LGCATPC as the logarithm of expenditure per capita on
your category. Define a new variable LGEXPPC as the logarithm of total
household expenditure per capita. Regress LGCATPC on LGEXPPC. Provide an
interpretation of the coefficients, and perform appropriate statistical tests.

A4.2 Is expenditure on your category per capita related to household size as well as to
total expenditure per capita? An alternative model specification.
Regress LGCATPC on LGEXPPC and LGSIZE. Provide an interpretation of the
coefficients, and perform appropriate statistical tests.

72
 4.4. Additional exercises

A4.3 A researcher is considering two regression specifications:

log Y = 1 + 2 log X + u (1)
and:
Y
log = 1 + 2 log X + u (2)
X
where u is a disturbance term.
Y
Writing y = log Y , x = log X, and z = log X , and using the same sample of n
observations, the researcher fits the two specifications using OLS:
yb = b1 + b2 x (3)
and:
zb =
b1 +
b2 x. (4)

Using the expressions for the OLS regression coefficients, demonstrate that
b2 =
b2 + 1.
Similarly, using the expressions for the OLS regression coefficients,
demonstrate that b1 = b1 .
Hence demonstrate that the relationship between the fitted values of y, the
fitted values of z, and the actual values of x, is ybi xi = zbi .
Hence show that the residuals for regression (3) are identical to those for (4).
Hence show that the standard errors of b2 and b2 are the same.
Determine the relationship between the t statistic for b2 and the t statistic for

b2 , and give an intuitive explanation for the relationship.
Explain whether R2 would be the same for the two regressions.

A4.4 A researcher has data on a measure of job performance, SKILL, and years of work
experience, EXP, for a sample of individuals in the same occupation. Believing
there to be diminishing returns to experience, the researcher proposes the model:
SKILL = 1 + 2 log(EXP ) + 3 log EXP 2 + u.

Comment on this specification.

A4.5 A researcher hypothesises that a variable Y is determined by a variable X and

considers the following four alternative regression specifications, using
cross-sectional data:
Y = 1 + 2 X + u (1)
log Y = 1 + 2 X + u (2)
Y = 1 + 2 log X + u (3)
log Y = 1 + 2 log X + u. (4)
Explain why a direct comparison of R2 , or of RSS, in models (1) and (2) is
illegitimate. What should be the strategy of the researcher for determining which of
the four specifications has the best fit?

73
 4. Transformations of variables

A4.6 Is a logarithmic specification preferable to a linear specification for an expenditure

function?
Use your category of expenditure from the CES data set. Define CATPCST as
CATPC scaled by its geometric mean and LGCATST as the logarithm of
CATPCST. Regress CATPCST on EXPPC and SIZE and regress LGCATST on
LGEXPPC and LGSIZE. Compare the RSS for these equations.

A4.7

. reg LGEARN S EXP ASVABC SASVABC

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 4, 495) = 22.68
Model | 23.6368302 4 5.90920754 Prob > F = 0.0000
Residual | 128.96239 495 .26053008 R-squared = 0.1549
-------------+------------------------------ Adj R-squared = 0.1481
Total | 152.59922 499 .30581006 Root MSE = .51042

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .0764243 .0116879 6.54 0.000 .0534603 .0993883
EXP | .0400506 .0096479 4.15 0.000 .0210948 .0590065
ASVABC | -.2096325 .1406659 -1.49 0.137 -.4860084 .0667434
SASVABC | .0188685 .0093393 2.02 0.044 .0005189 .0372181
_cons | 1.386753 .2109596 6.57 0.000 .9722664 1.80124
------------------------------------------------------------------------------

The output above shows the result of regressing the logarithm of hourly earnings
on years of schooling, years of work experience, ASVABC score, and SASVABC, an
interactive variable defined as the product of S and ASVABC, using EAWE Data
Set 21. The mean values of S, EXP, and ASVABC in the sample were 14.9, 6.4,
and 0.27, respectively. Give an interpretation of the regression output.

A4.8 Perform a RESET test of functional misspecification. Using your EAWE data set,
regress WEIGHT11 on HEIGHT. Save the fitted values as YHAT and define
YHATSQ as its square. Add YHATSQ to the regression specification and test its
coefficient.

4.5 Answers to the starred exercises in the textbook

4.8 Suppose that the logarithm of Y is regressed on the logarithm of X, the fitted
regression being:
log Yb = b1 + b2 log X.

Suppose X = X, where is a constant, and suppose that log Y is regressed on

log X . Determine how the regression coefficients are related to those of the original
regression. Determine also how the t statistic for b2 and R2 for the equation are
related to those in the original regression.

74
 4.5. Answers to the starred exercises in the textbook

Answer:
Nothing of substance is affected since the change amounts only to a fixed constant
shift in the measurement of the explanatory variable.
Let the fitted regression be:

log Yb = b1 + b2 log X .

Note that:
n
1X
log Xi log X
= log Xi log Xj
n j=1
n
1X
= log Xi log Xj
n j=1
n
1X
= log + log Xi (log + log Xj )
n j=1
n
1X
= log Xi log Xj
n j=1

= log Xi log X.

Hence b2 = b2 . To compute the standard error of b2 , we will also need b1 .

n
1 X
b1 = log Y b2 log X = log Y b2 (log + log Xj )
n j=1

= log Y b2 log b2 log X

= b1 b2 log .

bi is given by:
Thus the residual u

bi = log Yi b1 b2 log Xi = log Yi (b1 b2 log ) b2 (log Xi + log ) = u

u bi .

Hence the estimator of the variance of the disturbance term is unchanged and so
the standard error of b2 is the same as that for b2 . As a consequence, the t statistic
must be the same. R2 must also be the same:
P 2 P 2
2 ubi u
bi
R = 1 P  = 1 P  = R2 .
log Yi log Y log Yi log Y

4.11 RSS was the same in Tables 4.6 and 4.8. Demonstrate that this was not a
coincidence.
Answer:
This is a special case of the transformation in Exercise 4.7.

75
 4. Transformations of variables

4.14

. gen LGHTSQ = ln(HEIGHTSQ)

. reg LGWT04 LGHEIGHT LGHTSQ

Source | SS df MS Number of obs = 500

-------------+------------------------------ F( 1, 498) = 211.28
Model | 7.90843858 1 7.90843858 Prob > F = 0.0000
Residual | 18.6403163 498 .037430354 R-squared = 0.2979
-------------+------------------------------ Adj R-squared = 0.2965
Total | 26.5487548 499 .053203918 Root MSE = .19347

------------------------------------------------------------------------------
LGWT04 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
LGHEIGHT | (dropped)
LGHTSQ | 1.053218 .0724577 14.54 0.000 .9108572 1.195578
_cons | -3.78834 .610925 -6.20 0.000 -4.988648 -2.588031
------------------------------------------------------------------------------

The output shows the results of regressing, LGWT04, the logarithm of

WEIGHT04, on LGHEIGHT, the logarithm of HEIGHT, and LGHTSQ, the
logarithm of the square of HEIGHT, using EAWE Data Set 21. Explain the
regression results, comparing them with those in Exercise 4.2.
Answer:
LGHTSQ = 2 LGHEIGHT, so the specification is subject to exact
multicollinearity. In such a situation, Stata drops one of the variables responsible.

4.18

. nl (S = {beta1} + {beta2}/({beta3} + SIBLINGS)) if SIBLINGS>0

(obs = 473)

Iteration 0: residual SS = 3502.041

Iteration 1: residual SS = 3500.884
.....................................
Iteration 14: residual SS = 3482.794

Source | SS df MS
-------------+------------------------------ Number of obs = 473
Model | 132.339291 2 66.1696453 R-squared = 0.0366
Residual | 3482.7939 470 7.41019979 Adj R-squared = 0.0325
-------------+------------------------------ Root MSE = 2.722168
Total | 3615.13319 472 7.65918049 Res. dev. = 2286.658

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
/beta1 | 10.45811 5.371492 1.95 0.052 -.0970041 21.01322
/beta2 | 47.95198 125.3578 0.38 0.702 -198.3791 294.2831
/beta3 | 8.6994 15.10277 0.58 0.565 -20.97791 38.37671
------------------------------------------------------------------------------
Parameter beta1 taken as constant term in model & ANOVA table

76
 4.6. Answers to the additional exercises

The output uses EAWE Data Set 21 to fit the nonlinear model:
2
S = 1 + +u
3 + SIBLINGS
where S is the years of schooling of the respondent and SIBLINGS is the number
of brothers and sisters. The specification is an extension of that for Exercise 4.1,
with the addition of the parameter 3 . Provide an interpretation of the regression
results and compare it with that for Exercise 4.1.
Answer:
As in Exercise 4.1, the estimate of 1 provides an estimate of the lower bound of
schooling, 10.46 years, when the number of siblings is large. The other parameters
do not have straightforward interpretations. The figure below represents the
relationship. Comparing this figure with that for Exercise 4.1, it can be seen that it
gives a very different picture of the adverse effect of additional siblings. The
specification in Exercise 4.1 suggests that the adverse effect is particularly large for
the first few siblings, and then attenuates. The revised specification indicates that
the adverse effect is more evenly spread and is more enduring. However, the
relationship has been fitted with imprecision since the estimates of 2 and 3 are
not significant.

17

16
Years of schooling

15

Exercise 4.1

14

13

Exercise 4.18

12
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Siblings

4.6 Answers to the additional exercises

A4.1

. reg LGFDHOPC LGEXPPC

Source | SS df MS Number of obs = 6334

-------------+------------------------------ F( 1, 6332) = 4757.00
Model | 1502.58932 1 1502.58932 Prob > F = 0.0000
Residual | 2000.08269 6332 .315869029 R-squared = 0.4290
-------------+------------------------------ Adj R-squared = 0.4289
Total | 3502.67201 6333 .553082585 Root MSE = .56202

77
4. Transformations of variables

------------------------------------------------------------------------------
LGFDHOPC | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
LGEXPPC | .6092734 .0088338 68.97 0.000 .5919562 .6265905
_cons | .8988291 .0703516 12.78 0.000 .7609161 1.036742
------------------------------------------------------------------------------
The regression implies that the income elasticity of expenditure on food is 0.61
(supposing that total household expenditure can be taken as a proxy for permanent
income). In addition to testing the null hypothesis that the elasticity is equal to
zero, which is rejected at a very high significance level for all the categories, one
might test whether it is different from 1, as a means of classifying the categories of
expenditure as luxuries (elasticity > 1) and necessities (elasticity < 1).
The table gives the results for all the categories of expenditure.

Regression of LGCATPC on EXPPC

n b2 s.e.(b2 ) t (2 = 0) t (2 = 1) R2 RSS
ADM 2,815 1.098 0.030 37.20 3.33 0.330 1,383.9
CLOT 4,500 0.794 0.021 37.34 9.69 0.237 1,394.0
DOM 1,661 0.812 0.049 16.54 3.84 0.142 273.5
EDUC 561 1.382 0.090 15.43 4.27 0.299 238.1
ELEC 5,828 0.586 0.011 50.95 36.05 0.308 2,596.3
FDAW 5,102 0.947 0.015 64.68 3.59 0.451 4,183.6
FDHO 6,334 0.609 0.009 68.97 44.23 0.429 4,757.0
FOOT 1,827 0.608 0.027 22.11 14.26 0.211 488.7
FURN 487 0.912 0.085 10.66 1.03 0.190 113.7
GASO 5,710 0.677 0.012 56.92 27.18 0.362 3,240.1
HEAL 4,802 0.868 0.021 40.75 6.22 0.257 1,660.6
HOUS 6,223 1.033 0.014 73.34 2.34 0.464 5,378.5
LIFE 1,253 0.607 0.047 13.00 8.40 0.119 169.1
LOCT 692 0.510 0.055 9.29 8.92 0.111 86.2
MAPP 399 0.817 0.033 9.87 2.21 0.197 97.5
PERS 3,817 0.891 0.019 48.14 5.88 0.378 2,317.3
READ 2,287 0.909 0.032 28.46 2.84 0.262 809.9
SAPP 1,037 0.665 0.045 14.88 7.49 0.176 221.3
TELE 5,788 0.710 0.012 58.30 23.82 0.370 3,398.8
TEXT 992 0.629 0.046 13.72 8.09 0.160 188.2
TOB 1,155 0.721 0.035 20.39 7.87 0.265 415.8
TOYS 2,504 0.733 0.028 26.22 9.57 0.216 687.5
TRIP 516 0.723 0.077 9.43 3.60 0.147 88.9

78
 4.6. Answers to the additional exercises

A4.2

. reg LGFDHOPC LGEXPPC LGSIZE

----------------------------------------------------------------------------
Source | SS df MS Number of obs = 6334
-----------+------------------------------ F( 2, 6331) = 2410.79
Model | 1514.30728 2 757.15364 Prob> F = 0.0000
Residual | 1988.36473 6331 .314068035 R-squared = 0.4323
-----------+------------------------------ Adj R-squared = 0.4321
Total | 3502.67201 6333 .553082585 Root MSE = .56042
----------------------------------------------------------------------------
LGFDHOPC | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------+----------------------------------------------------------------
LGEXPPC | .5842097 .0097174 60.12 0.000 .5651604 .6032591
LGSIZE | -.0814427 .0133333 -6.11 0.000 -.1075806 -.0553049
_cons | 1.158326 .0820119 14.12 0.000 .9975545 1.319097
----------------------------------------------------------------------------

The income elasticity, 0.58, is now a little lower than before. The size elasticity is
significantly negative, suggesting economies of scale and indicating that the model
in the previous exercise was misspecified.
The specification is equivalent to that in Exercise 4.5 in the text. Writing the latter
again as:
LGCAT = 1 + 2 LGEXP + 3 LGSIZE + u
we have:

LGCAT LGSIZE = 1 + 2 (LGEXP LGSIZE ) + (3 + 2 1)LGSIZE + u

and so:

LGCATPC = 1 + 2 LGEXPPC + (3 + 2 1)LGSIZE + u.

Note that the estimates of the income elasticity are identical to those in Exercise
4.5 in the text. This follows from the fact that the theoretical coefficient, 2 , has
not been affected by the manipulation. The specification differs from that in
Exercise A4.1 in that we have not dropped the LGSIZE term and so we are not
imposing the restriction 3 + 2 1 = 0.

79
 4. Transformations of variables

Dependent variable LGCATPC

LGEXPPC LGSIZE

n b2 s.e.(b2 ) b3 s.e.(b3 ) R2 F RSS

ADM 2,815 1.080 0.033 0.055 0.043 0.330 692.9 3,945.2
CLOT 4,500 0.842 0.024 0.146 0.032 0.240 710.1 5,766.1
DOM 1,661 0.941 0.054 0.415 0.075 0.157 154.6 4,062.5
EDUC 561 1.229 0.101 0.437 0.139 0.311 125.9 1,380.1
ELEC 5,828 0.372 0.012 0.362 0.017 0.359 1,627.8 2,636.3
FDAW 5,102 0.879 0.016 0.213 0.022 0.461 2,176.6 3,369.1
FDHO 6,334 0.584 0.010 0.081 0.013 0.432 2,410.8 1,988.4
FOOT 1,827 0.396 0.031 0.560 0.042 0.281 356.1 1,373.5
FURN 487 0.807 0.103 0.246 0.137 0.195 58.7 913.9
GASO 5,710 0.676 0.013 0.004 0.018 0.362 1,691.8 2,879.3
HEAL 4,802 0.779 0.023 0.306 0.031 0.272 894.6 6,062.5
HOUS 6,223 0.989 0.016 0.140 0.021 0.467 2,729.5 4,825.6
LIFE 1,253 0.464 0.050 0.461 0.065 0.154 113.4 1,559.2
LOCT 692 0.389 0.060 0.396 0.086 0.138 54.9 1,075.1
MAPP 399 0.721 0.094 0.264 0.123 0.206 51.5 576.8
PERS 3,817 0.824 0.020 0.217 0.028 0.388 1,206.3 3,002.2
READ 2,287 0.764 0.034 0.503 0.047 0.297 482.8 2,892.1
SAPP 1,037 0.467 0.048 0.592 0.066 0.236 160.1 1,148.9
TELE 5,788 0.640 0.013 0.222 0.018 0.386 1,816.3 3,055.1
TEXT 992 0.388 0.049 0.713 0.067 0.246 161.0 1,032.9
TOB 1,155 0.563 0.037 0.515 0.049 0.329 282.1 873.4
TOYS 2,504 0.638 0.031 0.304 0.043 0.231 375.8 2,828.3
TRIP 516 0.681 0.083 0.142 0.109 0.150 45.3 792.8

A4.3 A researcher is considering two regression specifications:

log Y = 1 + 2 log X + u (1)

and:
Y
= 1 + 2 log X + u
log (2)
X
where u is a disturbance term.
Determine whether (2) is a reparameterised or a restricted version of (1).
(2) may be rewritten:

log Y = 1 + (2 + 1) log X + u

so it is a reparameterised version of (1) with 1 = 1 and 2 = 2 + 1.

Y
Writing y = log Y , x = log X, and z = log X , and using the same sample of n
observations, the researcher fits the two specifications using OLS:

yb = b1 + b2 x (3)

and:
zb =
b1 +
b2 x. (4)

80
 4.6. Answers to the additional exercises

Using the expressions for the OLS regression coefficients, demonstrate that
b2 =
b2 + 1.
P P
(xi x)(zi z) (xi x)([yi xi ] [y x])

b2 = P = P
(xi x)2 (xi x)2
(xi x)2
P P
(xi x)(yi y)
= P P = b2 1.
(xi x)2 (xi x)2
Similarly, using the expressions for the OLS regression coefficients, demonstrate
that b1 =
b1 .
b1 = z
b2x = (y x)
b2x = y (b
2 + 1)x = y b2x = b1 .
Hence demonstrate that the relationship between the fitted values of y, the fitted
values of z, and the actual values of x, is ybi xi = zbi .
zbi = b2 xi = b1 + (b2 1)xi = b1 + b2 xi xi = ybi xi .
b1 +
Hence show that the residuals for regression (3) are identical to those for (4).
Let u
bi be the residual in (3) and vbi the residual in (4). Then:
vbi = zi zbi = yi xi (b
yi xi ) = yi ybi = u
bi .

Hence show that the standard errors of b2 and

b2 are the same.
The standard error of b2 is:
sP sP
2
u /(n 2) vbi2 /(n 2)
Pi
b
s.e.(b2 ) = = P = s.e.(b
2 ).
(xi x)2 (xi x)2

Determine the relationship between the t statistic for b2 and the t statistic for
b2 ,
and give an intuitive explanation for the relationship.

b2
b2 + 1
tb2 = = .
s.e.(b2 ) s.e.(b
2 )

The t statistic for b2 is for the test of H0 : 2 = 0. Given the relationship, it is also
for the test of H0 : 2 = 1. The tests are equivalent since both of them reduce the
model to log Y depending only on an intercept and the disturbance term.
Explain whether R2 would be the same for the two regressions.
R2 will be different because it measures the proportion of the variance of the
dependent variable explained by the regression, and the dependent variables are
different.

A4.4 The proposed model:

SKILL = 1 + 2 log(EXP ) + 3 log(EXP 2 ) + u
cannot be fitted since:
log(EXP 2 ) = 2 log(EXP )
and the specification is therefore subject to exact multicollinearity.

81
 4. Transformations of variables

A4.5 In (1) R2 is the proportion of the variance of Y explained by the regression. In (2)
it is the proportion of the variance of log Y explained by the regression. Thus,
although related, they are not directly comparable. In (1) RSS has dimension the
squared units of Y . In (2) it has dimension the squared units of log Y . Typically it
will be much lower in (2) because the logarithm of Y tends to be much smaller
than Y .
The specifications with the same dependent variable may be compared directly in
terms of RSS (or R2 ) and hence two of the specifications may be eliminated
immediately. The remaining two specifications should be compared after scaling,
with Y replaced by Y where Y is defined as Y divided by the geometric mean of
Y in the sample. RSS for the scaled regressions will then be comparable.

A4.6 The RSS comparisons for all the categories of expenditure indicate that the
logarithmic specification is overwhelmingly superior to the linear one. The
differences are actually surprisingly large and suggest that some other factor may
also be at work. One possibility is that the data contain many outliers, and these
do more damage to the fit in linear than in logarithmic specifications. To see this,
plot CATPC and EXPPC and compare with a plot of LGCATPC and LGEXPPC.
(Strictly speaking, you should control for SIZE and LGSIZE using the
FrischWaughLovell method described in Chapter 3.)
The following Stata output gives the results of fitting the model for FDHO,
assuming that both the dependent variable and the explanatory variables are
subject to the BoxCox transformation with the same value of . Iteration
messages have been deleted. The maximum likelihood estimate of is 0.10, so the
logarithmic specification is a better approximation than the linear specification.
The latter is very soundly rejected by the likelihood-ratio test.

. boxcox FDHOPC EXPPC SIZE if FDHO>0, model(lambda)

Number of obs = 6334
LR chi2(2) = 3592.55
Log likelihood = -41551.328 Prob > chi2 = 0.000

------------------------------------------------------------------------------
FDHOPC | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
/lambda | .1019402 .0117364 8.69 0.000 .0789372 .1249432
------------------------------------------------------------------------------
Estimates of scale-variant parameters
----------------------------
| Coef.
-------------+--------------
Notrans |
_cons | 2.292828
-------------+--------------
Trans |
EXPPC | .4608736
SIZE | -.1486856
-------------+--------------
/sigma | .9983288
----------------------------

82
 4.6. Answers to the additional exercises

---------------------------------------------------------
Test Restricted LR statistic P-value
H0: log likelihood chi2 Prob > chi2
---------------------------------------------------------
lambda = -1 -50942.835 18783.01 0.000
lambda = 0 -41590.144 77.63 0.000
lambda = 1 -44053.749 5004.84 0.000
---------------------------------------------------------

A4.7 Let the theoretical model for the regression be written:

LGEARN = 1 + 2 S + 3 EXP + 4 ASVABC + 5 SA + u.
The estimate of 4 is negative, at first sight suggesting that cognitive ability has an
adverse effect on earnings, contrary to common sense and previous results with
wage equations of this kind. However, rewriting the model as:
LGEARN = 1 + 2 S + 3 EXP + (4 + 5 S)ASVABC + u
it can be seen that, as a consequence of the inclusion of the interactive term, 4
represents the effect of a marginal year of schooling for an individual with no
schooling. Since no individual in the sample had fewer than 8 years of schooling,
the perverse sign of the estimate illustrates only the danger of extrapolating
outside the data range. It makes better sense to evaluate the implicit coefficient for
an individual with the mean years of schooling, 14.9. This is
(0.2096 + 0.0189 14.9) = 0.072, implying a much more plausible 7.2 per cent
increase in earnings for each standard deviation increase in cognitive ability. The
positive sign of the coefficient of SA suggests that schooling and cognitive ability
have mutually reinforcing effects on earnings.
One way of avoiding nonsense parameter estimates is to measure the variables in
question from their sample means. This has been done in the regression output
below, where S1 and ASVABC1 are schooling and ASVABC measured from their
sample means and SASVABC1 is their interaction. The coefficients of S and
ASVABC now provide estimates of their effects when the other variable is equal to
its sample mean.
. reg LGEARN S1 EXP ASVABC1 SASVABC1
----------------------------------------------------------------------------
Source | SS df MS Number of obs = 500
-----------+------------------------------ F( 4, 495) = 22.68
Model | 23.6368304 4 5.90920759 Prob > F = 0.0000
Residual | 128.962389 495 .260530079 R-squared = 0.1549
-----------+------------------------------ Adj R-squared = 0.1481
Total | 152.59922 499 .30581006 Root MSE = .51042
----------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------+----------------------------------------------------------------
S1 | .0815188 .0116521 7.00 0.000 .0586252 .1044125
EXP | .0400506 .0096479 4.15 0.000 .0210948 .0590065
ASVABC1 | .0715084 .0298278 2.40 0.017 .0129036 .1301132
SASVABC1 | .0188685 .0093393 2.02 0.044 .0005189 .0372181
_cons | 2.544783 .0675566 37.67 0.000 2.41205 2.677516
----------------------------------------------------------------------------

83
 4. Transformations of variables

A4.8 In the first part of the output, WEIGHT11 is regressed on HEIGHT, using EAWE
Data Set 21. The predict command saves the fitted values from the most recent
regression, assigning them the variable name that follows the command, in this
case YHAT. YHATSQ is defined as the square of YHAT, and this is added to the
regression specification. Somewhat surprisingly, its coefficient is not significant. A
logarithmic regression of WEIGHT11 on HEIGHT yields an estimated elasticity of
2.05, significantly different from 1 at a high significance level. Multicollinearity is
responsible for the failure to detect nonlinearity hear. YHAT is very highly
correlated with HEIGHT.

. reg WEIGHT11 HEIGHT

----------------------------------------------------------------------------
Source | SS df MS Number of obs = 500
-----------+------------------------------ F( 1, 498) = 139.97
Model | 236642.736 1 236642.736 Prob > F = 0.0000
Residual | 841926.912 498 1690.61629 R-squared = 0.2194
-----------+------------------------------ Adj R-squared = 0.2178
Total | 1078569.65 499 2161.46222 Root MSE = 41.117
----------------------------------------------------------------------------
WEIGHT11 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------+----------------------------------------------------------------
HEIGHT | 5.369246 .4538259 11.83 0.000 4.477597 6.260895
_cons | -184.7802 30.8406 -5.99 0.000 -245.3739 -124.1865
----------------------------------------------------------------------------

. predict YHAT
. gen YHATSQ = YHAT*YHAT

. reg WEIGHT11 HEIGHT YHATSQ

----------------------------------------------------------------------------
Source | SS df MS Number of obs = 500
-----------+------------------------------ F( 2, 497) = 70.33
Model | 237931.888 2 118965.944 Prob > F = 0.0000
Residual | 840637.76 497 1691.42407 R-squared = 0.2206
-----------+------------------------------ Adj R-squared = 0.2175
Total | 1078569.65 499 2161.46222 Root MSE = 41.127
----------------------------------------------------------------------------
WEIGHT11 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------+----------------------------------------------------------------
HEIGHT | -.4995924 6.737741 -0.07 0.941 -13.73756 12.73837
YHATSQ | .0030233 .003463 0.87 0.383 -.0037807 .0098273
_cons | 114.5523 344.2538 0.33 0.739 -561.8199 790.9244
----------------------------------------------------------------------------

84