Chapter 4 Multiple regression analysis Overview This chapter introduces regression models with more than one explanatory variable. Specific topics are treated with reference to a model with just two explanatory variables, but most of the concepts and results apply straightforwardly to more general models, The chapter begins by showing how the least squares principle is employed to derive the expressions forthe regression coefficients and how the coefficients, should be interpreted. It continues with a discussion of the precision of the regression coefficients and tests of hypotheses relating to them. Next comes multicollinearity, the problem of discriminating between the effects of individual explanatory variables when they are closely related. ‘The chapter concludes with a discussion of F tests of the joint explanatory power of the explanatory variables or subsets of them, and shows how af test can be thought of as a marginal F test Learning outcomes ‘After working through the corresponding chapter in the text, studying the corresponding slideshows, and doing the starred exercises in the text and the additional exercises in this guide, you should be able to explain: ‘+ the principles behind the derivation of multiple regression coefficients (but you are not expected to learn the expressions for them or to be able to reproduce the mathematical proofs) ‘© how to interpret the regression coefficients ‘© the Frisch-Waugh graphical representation of the relationship between the dependent variable and one explanatory variable, controlling for the influence of the other explanatory variables ‘© the properties of the multiple regression coefficients ‘* what factors determine the population variance of the regression coefficients ‘© what is meant by multicollinearty and measures for alleviating it ‘+ what is meant by a linear restrition ‘+ the F test ofthe joint explanatory power of the explanatory variables ‘+ the F test of the explanatory power ofa group of explanatory variables # why ¢tests on the slope coefficients are equivalent to marginal F tess You should know the expression for the population variance of a slope coefficient in a multiple regression model with two explanatory variables, 41Elements of Econometrics 2 Additional exercises ‘A4.1 The output shows the result of regressing FDHO, expenditure on food consumed at home, on EXP, total household expenditure, and SIZE, number of persons in the household, using the CES data set. Provide an interpretation of the regression coefficients and perform appropriate tests. «eg FDHO EXP SIZE if FDHO>0; Source | 38 ae Ms Number of obs = 868, F( 2, 865) = 426.78 Model | 1.4826e+09 2 741324291 Prob $F 0.0000 Residual | 1.502Se+09 865 1736978.64 R-squared 0.4967 adj R-squared = 0.4955 Total | 2.9851e+09 867 3443039.33 Root MSE = 1317.8 FDHO | coef. std. Err. tel [95% Conf. Interval) EXP | 0372621 .0024547 15.18 0.000 0324442 -04208 SIZE | 559.7692 30.8568¢ 18.14 0.000 499.2061 620.3322 cons | 884.5901 100.1537 8.830.000 © 688.0173 1081.163 ‘A42_ Perform a regression parallel to that in Exercise Ad. for your CES category of expenditure, provide an interpretation ofthe regression coefficients and perform appropriate tests ‘A4.3. The output shows the result of regressing FDHOPC, expenditure ‘on food consumed at home per capita, on EXPPC, total household expenditure per capita, and SIZE, number of persons in the household, using the CES data set. Provide an interpretation of the regression coefficients and perform appropriate tests. eg FDHOPC EXPPC SIZE if FDHO>O; Source | ss at us Number of obs 868 F( 2, 865) = 175.68 Model | 142127276 2-71063638.2 Prob > F = 0.0000 Residual | 349895173 865 — 404503.09 Resquared = 0.2509 Adj Resquared = 0.2872 Total | 492022449 867 567499.942 Root MSE = 636.01 FDHOPC | coef. Std. Ex. t pitl [998 Conf. Interval) EXPEC | 0316606 0026915 11.76 0.000 0263779. 0369432 SIZE | “133.775 15.18071 -8.81 0.000 -163.5703 103.9797 ons | 1430.123 67.10582 21.31 0.009 =» 1298.413 561.832 ‘A4.4_ Perform a regression parallel to that in that in Exercise A4.3 for your CES category of expenditure, provide an interpretation of the regression coefficients and perform appropriate test. AAS The output shows the result of regressing FDHOPC, expenditure ‘on food consumed at home per capita, on EXPPC, total household ‘expenditure per capita, and SIZEAM, SIZEAF, SIZEJM, SIZEJF,Chapter 4; Multiple regression analysis and SIZEIN, numbers of adult males, adult females, junior males, junior females, and infants respectively, in the household, using the CES data set. Provide an interpretation of the regression coefficients and perform appropriate tess. | reg FDHOPC EXPPC SIZEAM SIZEAF SIZEIM SIZEIF SIZEIN if FDHO>O; source | 8s at us Number of obs 868 + FC 6, 861) = 59.11 Model | 142547989 6 23924664.9 Prob > F 0.0000 Residual | 348474460 961 404732.242 Resquared = 0.2918 Ad} R-squared = 0.2868 otal | 492022449 867 567499.942 Root MSE = 636.19 FDHOEC | coef. std. Err. + tl [958 Conf. Interval] EXEEC | 0319472 .0027125-«11.78 0.000 .0266234 037271 gi2zaM | -159.6329 2.79751 =4.87 0.000 224.0053 -95.26049 SIZEAF | -94.88238 37198996 -2.50 0.013 -169.4462 -20.31861 SIZEIM | -101.5105 36145485 -2.78 0,005 -173.0613 29.9597 SIZEJE | -155.5774 3749424 4.15 0.000 229.1682 81. 98662 SIZEIN | -220.7865 85.7005 -2.58 0.010 388.992 -52..58108 Neons | 1411.313 7313575 19.30 0.000 —-1267.768 ~—«1554.859 ‘A4.6. Perform a regression parallel to that in that in Exercise A4.5 for your CES category of expen regression coefficients and perform appropriate tess. re, provide an interpretation ofthe Answers to the starred exercises in the text 4.5 Question: Explain why the intercept inthe regression of EEARN on ES is equal to 0. Answer: The intercept is calculated as EEARN-b, ES However, since the mean of the residuals from an OLS regression is 0, both EEARN and ES are 0, and hence the intercept is 0. 4.9 — Question: Demonstrate that 2 is equal to 0 in multiple regression analysis. (Note: The proof is a generalization of the proof for the simple regression model, given in Section 2.7.) Answer: If the model is Y= By + Xz... + Xe, aX y a o y In observation i, YAH, 2Y,~b,~baN yo Ny 4BElements of Econometrics 44 4.13 418 bX, =6,¥, + ¥, -..-4F, =0 ‘A researcher investigating the determinants of the demand for public transport in a certain city has the following data for 100 residents for the previous calendar year: expenditure on public transport, £, measured in dollars; number of days worked, WV; and number of days not worked, NIV, By definition NI is equal to 365, —W. He attempts to fit the following model E= B+ BW + BNW u Explain why he is unable to fit this equation. (Give both intuitive and technical explanations.) How might he resolve the problem? Answer: There is exact multicollinearity since there is an exact linear relationship between 17, NIV and the constantterm. As a ‘consequence it is not possible to tell whether variations in E are attributable to variations in 1” or variations in NW, or both. Mathematically, Cov E,W Van(NI)— Cov’ E, NWPICov(W, NW) Var Var(NW7) [Cov NW”) _ Cov E,W)Var(365 - W) - Cov( E1365 W)Cov(W, [365 - W}) Var(W¥ )Var(365 ~W) — [Coviw,[365-WD} _ Cov(E,W)Var(W)—CovlE, Cov.) _0 VardW)Var()=-F-CovW.)f 0 O divided by 0 is not defined. One way of dealing with the problem would be to drop NW from the regression, ‘The interpretation of b; now is that itis an estimate of the extra expenditure on transport per day worked. ‘The researcher in Exercise 4.13 decides to divide the number of | days not worked into the number of days not worked because of illness, J, and the number of days not worked for other reasons, O. ‘The mean value of /in the sample is 2.1 and the mean value of O is 120.2. He fits the regression (standard errors in rarentheses): E = -96+ 2.10W+ 0.450 (83) (198) (1.77) 0.72 Perform £ tests on the regression coefficients and an F test on the ‘goodness of fit of the equation. Explain why the tests and F test have different outcomes. Answer: Although there is not an exact linear relationship between Wand O, they must have a very high negative correlation because the mean value of /is so small. Hence one would expect the regression to be subject to multicollinearity, and this is,Chapter 4: Multiple regression analysis confirmed by the results. The s statistics for the coefficients of and O are only 1.06 and 0.25, respectively, but the F statistic, 0.72/2 (1-0.72)/97 Fegn= 1247, which is greater than the critical value of F at the 0.1 percent level, TAL. Answers to the additional exercises Aad Aaa The regression indicates that 3.7 cents out of the marginal expenditure dollar is spent on food consumed at home, and that expenditure on this category increases by $560 for each individual inthe household, keeping total expenditure constant. Both of these effects are very highly significant, and almost half of the variance in FDHO is explained by EXP and SIZE. The intercept has no plausible interpretation With the exception of LOCT, all of the categories have positive coefficients for EXP, with high significance levels, but the SIZE effect varies: Positive, significant at the | percent level: FDHO, TELE, CLOT, FOOT, GASO. Positive, significant at the 5 percent level: Negative, significant at the 1 percent level: READ. ‘Negative, significant at the 5 percent level: SHEL, EDUC. Not significant: FDAW, DOM, FURN, MAPP, SAPP, TRIP, HEAL, ENT, TOYS, TOB. At first sight it may seem surprising that SIZE has a significant negative effect for some categories. The reason for this is that an inerease in SIZE means a reduction in expenditure per capita, if total household expenditure is kept constant, and thus SIZE has a (negative) income effect in addition to any direct effect. Effectively poorer, the larger household has to spend more on basies and less on luxuries. To determine the true direct effect, we need to eliminate the income effect, and that is the point of the re- specification of the model in the next exercise. 45Elements of Econometrics SIZE by seth) F FDHO 868 0.8373 0.0025 $59.77 3086 0.4967 4268 FDAW 827 0.0454 0.0022 -53.06 27.50 03559 2276 SHEL 867 0.1983 0.0067 -17440 83.96 0.5263 479.9 TELE 858 0.0091 0.0010 36.10.1208 0.1360 673, DOM 454 0.0217 0.0087 26.10 64.14 O0SKS 4. TEXT 482. 0.0057--«0.0007--33.15 9.10388 37.7 FURN 329 00138 0,0024 4752 35.18 0.0895 16.0 MAPP 244 0.0083 0.0019 2535—24.33.--0.09S4 12.7 SAPP 467 0.0014 0.0003. 5.63, 3.73 00839 13.2 CLOT 847 0.0371 0.0019 8798 2439-03621 239.5 FOOT 686 0.0028 0.0003. «21.24 «4.01 (0.190805 GASO 797 0.0205 0.0015. 94581867 0.2762 ISLS TRIP 309 0.0273 0.0082. 110.11. $6.17 0.1238 21.6 LOCT 172 -0.0012 0.0021. 5497 23.06 0.033529 HEAL 821 04231 0.0032 1860 4056-90674 29.6 ENT 824 00726 0.0042 9894 © 526102774 157.6 FEES 676 00335 0.0028 114713604 0.1790 73.4 TOYS 592 0.0089 0.0011 5.031333 O.N4S 38.1 READ 640.0043 «0.0003. 1586 4.06 0.1960 92.8 EDUC 288 00295 0,0055 168.13. 74.57 0.0937 14.7 TOR 368 0.0068 -«O.9014 4441629 0.0726. 143. A43 Another surprise, perhaps. The purpose of this specification is to test whether household size has an effect on expenditure per capita con food consumed at home, controlling forthe income effect of variations in household size mentioned in the answer to Exercise ‘A42. Expenditure per capita on food consumed at home increases by 32 cents out ofthe marginal dollar of total household expenditure per capita, Now SIZE has a very significant negative effect. Expenditure per capita on FDHO decreases by $134 per ‘year for each extra person inthe household, suggesting that larger households are more efficient than smaller ones with regard to expenditure on this category, the effect being highly significant. is much lower than in Exercise A4.1, but a comparison is invalidated by the fact thatthe dependent variable is different ‘AAA Several categories have significant negative SIZE effects. None has a significant positive effect, Negative, significant at the 1 percent level: FDHO, SHEL, TELE, SAPP, GASO, HEAL, READ, TOB. Negative, significant at the 5 percent level: FURN, FOOT, LOCT, EDUC. 46Chapter 4: Multiple regression analysis Not significant: FDAW, DOM, TEXT, MAPP, CLOT, TRIP, ENT, FEES, TOYS. One explanation of the negative effects could be economies of scale, but this is not plausible jn the case of some, most obviously TOB. Another might be family composition — larger families having more children. This possiblity is investigated in the next EXPPC SWE nbs sel) by sey) F FDHO $68 0.0317 0.0027 133.78 15.18 028891757 FDAW 827 0.0476 (0.0027 59.89 68.15. 0.3214 195.2 SHEL $67 0.2017 0.0075 11368 423805178 463.9 TELE $58 0.0145 0.0014 4307 7.83 0.2029 1088 DOM — 454 00243 0.0060 1.33 35.58 0.040495 TEXT — 482.0115. 0.0011 5.01 643 02191 67.2 FURN 329 0.0198 040083 43.12.2123 O.1621_ 315 MAPP 244 0.0124 0.0022 -2596 1398 0.1962 294 SAPP 467 0.0017 0.0004 =7.76. 201 0.1265 33.6 CLOT 47 0414 0.0021 2183120703327 2104 FOOT — 686 0.0034 0.0007 387 189 0.1939 82.2 GASO 797 0.0183 60015. 424987302553. 136.1 TRIP 309 0.0263 0.0044 13.06 2745 O.44T 25.9 LOCT — 172 -0.0005 0.0018 -23.84 9.16 0.0415 37 HEAL $21 0.0181 0.0036 17820 2080 0.158771 ENT 824 0.0743 0.0046 392.86 118.530.2623 146.0 FEES 616 0.0337 0.0032-2397—«1933, «0.1594 638 TOYS 592 0,095 0.0011 ~5.89 620 0.1446 49.8 READ 764 0.0080 0.0004 124922102906 1559 EDUC 288 0.0235 000088 108184745 00791122 TOB 368 0.0057 0.0016 4887-3792 0.1890 425 A4S_ It is not completely obvious how to interpret these regression results and possibly this is not the most appropriate specification for investigating composition effects. The coefficient of SIZEAF suggests that for each additional adult female in the household, expenditure falls by $95 per year, probably as a consequence of economies of scale. For each infant, there is an extra reduction, relative to adult females, of $126 per year, because infants consume less food. Similar interpretations might be given to the coefficients of the other composition variables. ‘A4.6 The regression results for this specification are summarized in the table below. In the case of SHEL, the regression indicates thatthe SIZE effect is attributable to SIZEAM. To investigate this further, the regression was repeated: (1) restricting the sample to households with at least one adult male, and (2) restricting the 47Elements of Econometrics sample to households with either no adult male or just 1 adult male. ‘The first regression produces a negative effect for SIZEAM, but itis smaller than with the whole sample and not significant. In the second regression the coefficient of SIZEAM jumps. dramatically, from -$424 to $795, suggesting very strong, ‘economies of scale for this particular comparison. 'As might be expected, the SIZE composition variables on the whole do not appear to kave significant effects if the SIZE variable does not in Exercise A4.4, The results for 7OB are puzzling, in that the apparent economies of scale do not appear to be related to household composition. Category FDHOPC FDAWPC SHELPC —TELEPC_-DOMPC _TEXTPC _ FURNPC__ MAPPPC EP 00319 0.0473 02052 0.0146 000262 0.0116 0.0203 0.0125 (0.0027) (0.0027) (0.0075) (0.0014) (0.0061) (0.0011) (0.0034) (0.0022) SIZEAM 15963-2932, 43385 48.79 133.37 236-6954 46.54 280) 248) (9037) (16.99) (83.47) (13.07), (4220) (2826) SIEAF 9488-2282 22.96.5623. 7136-1566 7952 19.74 8799) 37.59) (105.22) (19.80) (SBT) (17.36) (5443) 32.49) SIZEIM — ~101.51 Las 5370-3965 84391002 026-2234 (3645) (35.61) (100.60) (1880) (84.30) (1459) (47.01) 32.84) SIZEIF —-ASSS8_—-19.48 632-3801 23.95 1183 3624-1248 749) 667) (103.52) (19.33), (8218) (1405) (4841) 29.21) SIZEIN 220.79 -24.44 469.75 540. 17693 1734-2596 35.46 (85.70) (3.05) (236.44) ——(44.12)- (183.84) BHAT) —(B782)_—_(78.95) e 0.2918 03227 «0.5297 0.2041-——0.0503 0.2228 0.1667 0.1988. F 59.1 65.1 les 364 40 26 107 98 ” 868 27 867 858 454 482 329 244 Category SAPPPCCLOTPC FOOTPC GASOPC—TRIPPC—LOCTPCHEALPC NTPC EXP 0.0017 0.0420 0.0035. (0.0179 0.0263 -0.0005 0.0182 a.o740 (0.0004) (0.0021) (0.0003) (0.0015) (0.0044) (0.0019) (0.0037) (0.0046) SIZEAM 9.13, 2791-666 13.99 433-3364 -191.60 1438 (417) 2590) G93) (1849) (5453) (19.53) (44.43) (56.32) SIEAF 2498758931 4043. 31581023 46.92 24:53, (499) 029) (503)—@2137) (66.29) 4.15) (52.65) (64.94) SIZEIM 893 1987 258-6237 4020 S045 230.65 38.60 (463) (28.55) (428)——(20.10)—— (65.07), 2171) (50.63) (61.24) SIZEIF 863 40.08 235-6407 3498-2149 -194.56 65.74 (464) (2942) 435)-—(20.28)—— (TOSI). (2202) (51.80) (63.12) SIZEIN 10558753 -835 11258-5185 1908-24758 16.49 (114) (6580) (9.94) (4657) (19469) (70.79) (113.55) (142.40) ® 0.1290 03373——«0.1987-«0.2680 0.1472 0.0636 0.1665 0.2629 F na 13 28.1 482 87 19 21 486 ” 467 847 686, 797 309 m a2 a4 48Chapter 4: Multiple regression analysis Category FEESPC—TOYSPC—-READPC_ EDUCPC _TOBPC EXP. 0.0337 0.0096—0.0050. 0.0232 0.0056 (0.0032) (0.0012) (0.0004) (0.0090) (0.0016), SIEAM 62-1799 2183-13534 -37.28 984) (1316) (4.79) (88.87). (17.19) SIZEAF 268 3B 4.22 46.03 56.54 46.77) (1582) (S51) (103.88) (17.50) SIZEIM — 1S65—— 259 1328-10639 44.45 (4440) (13.70) (5.27) (9225) (1853) SIZEIF 3207 307-861 11936 -52.68 (4292) (13.65) (5.40) (91.60) @2.87) SIZEIN 2986 “1808-1512. 149.87 -76.25 (95.20) (30.40) (11.86) (262.13) (53.68) Rg 0.1599 0.1468 0.2969 0.0808 0.1913, F 212 168. 333) at 42 ” 676 592 164 288 368 49Elements of Econometrics Notes 50