Fin CR Lessons

O C C A S I O N A L PA P E R S E R I E S
N O 1 3 0 / O C TO B E R 2 011
SOME LESSONS FROM THE FINANCIAL CRISIS FOR THE ECONOMIC ANALYSIS
by Geoff Kenny and Julian Morgan
OCCASIONAL PAPER SERIES

NO 130 / OCTOBER 2011
SOME LESSONS FROM THE FINANCIAL CRISIS FOR THE ECONOMIC ANALYSIS 1
by Geoff Kenny and Julian Morgan

NOTE: This Occasional Paper should not be reported as representing the views of the European Central Bank (ECB). The views expressed are those of the authors and do not necessarily reflect those of the ECB.
In 2011 all ECB publications feature a motif taken from the 100 banknote.
This paper can be downloaded without charge from http://www.ecb.europa.eu or from the Social Science Research Network electronic library at http://ssrn.com/abstract_id=1791644.
1 This paper has benefited from discussions with a large number of colleagues within the European Central Bank (ECB) and the Eurosystem. We have received substantial contributions and input from ECB staff in DG-Economics, DG-Research and DG-Statistics, especially K. Christoffel, M. Ciccarelli, G. De Bondt, S. Dees, F. Fornari, E. Hahn, K. Hubrich, M. Lenza, L. Maurin, A. Meyler, F. Skudelny and O. Tristani. We would also like to thank R. Anderton, M. Banbura, G. Coenen, S. Dees, P. de Rougemont, Hans-Joachim Klckers, G. Korteweg, R. Motto, H. Pill and F. Smets for their comments. This paper has further benefited from the contributions of C. Marchini, F. Masera, L. Kezbere and M. Santoianni. In addition, some very helpful comments were received from members of the Monetary Policy Committee of the European System of Central Banks and its Working Group on Forecasting at the seminars on this topic held in 2010. Please note that the views expressed in this paper do not necessarily reflect those of the ECB.
European Central Bank, 2011 Address Kaiserstrasse 29 60311 Frankfurt am Main, Germany Postal address Postfach 16 03 19 60066 Frankfurt am Main, Germany Telephone +49 69 1344 0 Internet http://www.ecb.europa.eu Fax +49 69 1344 6000 All rights reserved. Any reproduction, publication and reprint in the form of a different publication, whether printed or produced electronically, in whole or in part, is permitted only with the explicit written authorisation of the ECB or the authors. Information on all of the papers published in the ECB Occasional Paper Series can be found on the ECBs website, http://www.ecb.europa.eu/pub/ scientific/ops/date/html/index.en.html. Unless otherwise indicated, hard copies can be obtained or subscribed to free of charge, stock permitting, by contacting [email protected] ISSN 1607-1484 (print) ISSN 1725-6534 (online)
CONTENTS
ABSTRACT NON-TECHNICAL SUMMARY 1 INTRODUCTION 2 UNDERSTANDING THE KEY DRIVERS OF THE CRISIS 3 HOW DID THE ECONOMIC ANALYSIS FARE DURING THE CRISIS? 3.1 The near-term outlook 3.2 Medium-term projections 3.3 Analysis of uncertainty and risks 3.4 Macroeconomic models 4 THE BENEFIT OF HINDSIGHT: WHAT SHOULD WE HAVE PAID MORE ATTENTION TO? 4.1 Financial factors 4.2 Non-linearities 4.3 Condence, expectations and uncertainty 4.4 Judgement 4.5 International linkages 5 WHAT IS THE WAY FORWARD? 5.1 Extend existing tools and develop new ones 5.2 Develop ways to better handle complexity 5.3 Develop further the risk analysis 6 CONCLUSIONS REFERENCES 4 5 6
CONTENTS
12 12 13 15 17
19 19 20 20 22 23 24 24 24 25 26 27
ECB Occasional Paper No 130 October 2011
ABSTRACT The economics profession in general, and economic forecasters in particular, have faced some understandable criticism for their failure to predict the timing and severity of the recent economic crisis. In this paper, we offer some assessment of the performance of the Economic Analysis conducted at the ECB both in the run up to and since the onset of the crisis. Drawing on this assessment, we then offer some indications of how the analysis of economic developments could be improved looking forward. The key priorities identied include the need to: i) extend existing tools and/or develop new tools to account for important feedback mechanisms, for instance, improved real-nancial linkages and non-linear dynamics; ii) develop ways to handle the complexity arising from the presence of multiple models and alternative economic paradigms; and iii) given the limitations of point forecasts, to further develop risk and scenario analysis around baseline projections. KEY WORDS: EURO AREA, FINANCIAL CRISIS, MACRO ECONOMIC FORECASTING JEL CLASSIFICATION: E02, E30, E2, C53
NON-TECHNICAL SUMMARY This paper draws on the experiences during the run up to, and since the onset of, the recent nancial crisis to offer some indications of how the analysis of economic developments could be improved. In the rst section, the paper recalls the key economic and nancial drivers of the crisis, highlighting the role of nancial shocks linked to the re-pricing of risk, asset prices and nancing costs. A number of important non-nancial elements are also emphasised, such as condence and uncertainty shocks and deepening international linkages. In Section 2, the predictive failure of macroeconomic tools and expert judgement widely shared by institutional and private forecasters alike (as reected in the macroeconomic projections in specic periods strongly impacted by the crisis) is documented for both short and medium-term horizons. One of the main conclusions to be drawn from the analysis here is that the errors made by forecasters largely relate to the size of the shocks impacting the economy. Nevertheless, economic tools and models as well as expert judgement also failed to identify the importance and strength of key transmission and amplication channels, especially those linked to nancial markets and uncertainty. Hence, the third section of our paper identies a number of factors which, with the benet of hindsight, could have received more attention from those conducting economic analysis during the period of crisis. These include the leading indicator properties of various nancial variables, the prevalence of the nonlinear dynamics often neglected in economic tools and the signalling aspect of condence and uncertainty indicators. In addition, there would appear to be a case for relying more on judgement than on the results of mechanical tools, particularly in the immediate aftermath of unprecedented events (such as the collapse of Lehman Brothers in September 2008).
Lastly, in the nal section of the paper, we identify a possible way forward, particularly in respect of the main priorities for developing the economic analysis. The key priorities identied include the need to: i) extend existing tools and/ or develop new tools to account for important aspects, for instance, improved real-nancial linkages and non-linear dynamics; ii) develop ways to handle the complexity arising from the presence of multiple models and alternative economic paradigms; and iii) given the limitations of point forecasts, to further develop risk and scenario analysis around baseline projections.
NON-TECHNICAL SUMMARY
INTRODUCTION
Economists, both those inside and outside policy institutions, pay considerable attention to analysing conjunctural economic developments. This helps them to better understand the current state of the economy and to make predictions about future developments. The analysis of economic developments forms a key element of the stability-oriented monetary policy strategy of the European Central Bank (ECB) aimed at achieving price stability.1 More specically, the economic analysis provides a forwardlooking perspective on the outlook for and risks to price stability over the short to medium-term and therefore complements, and can be crosschecked with, an analysis of monetary developments that is particularly suited to explaining the evolution of price developments in the medium to long run.2 In this paper, we focus on the approaches adopted for analysing the economic conjuncture and attempt to draw out some key lessons from the experiences made during the nancial crisis. Although we concentrate on the methods widely used in central banks and international organisations, the conclusions reached are also likely to be relevant for all those engaged in conjunctural analysis. In contrast to monetary analysis, economic analysis can be characterised as focusing largely on models that are based on an assessment of economic variables and, in particular, the interplay between demand and supply in goods and labour markets. In the context of the ECB, an important part of the insights emerging from the economic analysis is summarised in the regular macroeconomic projections of Eurosystem and ECB staff which are published each year in June and December and in March and September, respectively. The economic analysis also incorporates nancial information to the extent that it is relevant for this assessment. Needless to say, such information proved to be a key part of the economic analysis during the crisis, where there
was a clear tendency for nancial shocks to have an impact on the real demand and supply for goods and services and also for additional feedback effects from developments in the real economy to the nancial sector. Given the important challenges posed by the nancial crisis to the economic analysis, it now seems timely to take a step back and try to assess what possible insight can be gleaned from the performance of the economic analysis during the run-up to and since the onset of the crisis, and to already offer some indications as to how it may be improved in the light of our recent experience. The paper is structured as follows. In the rst section, we recall the key economic and nancial drivers of the crisis, highlighting the role of nancial shocks linked to risk re-pricing, asset prices and nancing costs for both rms and households. In addition, a number of important non-nancial elements are also emphasised, such as condence and uncertainty shocks, inventory adjustment as well as more intensive international linkages generated via trade. In Section 2, the poor performance of macroeconomic tools and expert judgement, as reected in the macroeconomic projections of the crisis period, is documented for both short and medium-term horizons. More importantly, we note that the large deterioration in the accuracy of macroeconomic forecasts was
1 This two-pillar approach has proved very successful, both as a device for processing and introducing a large set of complex and diverse information and for structuring the presentation of the factors underpinning the monetary policy decisions of the Governing Council. In particular, the information extracted from the two pillars is regularly presented to the public at the press conference that immediately follows the meeting in which the Governing Council takes its monetary policy decision for the euro area. It is also subsequently explained in the Monthly Bulletin of the ECB. Given the possibility of differing and even conicting messages emerging from the analysis based on these pillars, a careful cross-checking of monetary and economic developments ensures the robustness of the analysis behind monetary policy decisions. In a recent contribution, Papademos and Stark (2010) consider the possible enhancement of the monetary analysis while drawing on the lessons learnt from the nancial crisis of 2007-10. For the rapidly expanding eld of nancial stability analysis, Trichet (2011) discusses the intellectual challenges ahead.
1 INTRODUCTION widely shared by institutional and private sector forecasters alike, including the ECB and the Eurosystem. One of the main conclusions to be drawn from the analysis here is that the errors made by forecasters largely relate to the large size of the shocks impacting the economy. Having said this, however, it would appear that the various tools used (including both reduced form and more structural macroeconometric models) failed to identify the importance and strength of key transmission and amplication channels, especially those linked to nancial markets and uncertainty. Furthermore, such factors do not appear to have been adequately taken into account by experts when deciding upon the judgemental adjustments that are often made to rene the output of model-based forecasting exercises. The third section of the paper builds on the preceding discussion and attempts to identify a number of factors which, with the benet of hindsight, the economic analysis could and in our view should have paid more attention to during the crisis. These would include: i) the leading indicator properties of various nancial variables; ii) the prevalence of the non-linear dynamics that are intrinsically linked to a crisis environment and yet often neglected in our tools; and iii) the signalling aspect of condence and uncertainty indicators. In addition, it is argued that when the crisis intensied, particularly following the collapse of Lehman Brothers in September 2008, the economic analysis might have relied more on judgement rather than on the results of mechanical tools. In extracting these lessons, we recognise the possibility of hindsight bias in such assessments, which would caution against our ability to exploit the insights gained in real time and in future crises. Lastly, in the nal section of the paper, we identify some priorities for the way forward so that the economic analysis is developed further to take advantage of the lessons learnt and becomes more robust and adaptive to changing economic circumstances. The key priorities identied include: i) the need to extend existing
tools and also to develop new tools in order to account for important aspects, such as improved real-nancial linkages and non-linear dynamics; ii) to develop ways to handle the complexity arising from the presence of multiple models and alternative economic paradigms; and iii) given the limitations of point forecasts, for those conducting macroeconomic projections to consider further developing the characterisation of uncertainty surrounding the outlook as well as enriching the existing tools for identifying and quantifying the impact of important risks.
UNDERSTANDING THE KEY DRIVERS OF THE CRISIS
While some parts of the global economy already showed signs of a slowdown in growth in late 2006 and early 2007, nancial factors undoubtedly played a strong role in triggering and subsequently amplifying the macroeconomic effects of the crisis. The rst such factor was the rise in overall nancing costs within the nancial sector and, by extension, for rms and households (see Chart 1). Following a signicant deterioration of conditions in the US housing market, a sharp increase in perceived default and liquidity risk precipitated a dramatic global re-pricing of risk. The result was an outbreak of turbulence in global money markets in August 2007; a development which signalled the onset of the nancial crisis.
Chart 1 Financial market spreads
(in basis points) EURIBOR/overnight index swap loans_large/overnight index swap loans_small/overnight index swap BBB-rated corporate bonds/overnight index swap 6 5 4 3 2 1 0 2005 2006 2007 2008 2009 2010 6 5
With regard to the euro area, the nancial nature of this global shock was reected most clearly in money market spreads and also in bank lending rate spreads for loans to households and nonnancial corporations. The latter accentuated markedly after the collapse of Lehman Brothers in September 2008; a period in which there was a further sharp rise in spreads and overall nancing costs that weighed heavily on euro area growth, most notably on investment spending. A second nancial factor was the more general deterioration in asset markets beyond the US housing market. Sharp declines in stock prices (see Chart 2), and also in many countries in house prices, brought about a worsening in balance sheets for rms and households, thereby reinforcing the negative dynamics associated with increased perceptions of risk and higher
Chart 2 Stock prices
(index; 31 January 1995 = 100) euro area (in EUR) United States (in USD) emerging economies (in USD) 400 400
300 4 3 2 100 1 0 0 1995 200
300
200
100
0 2000 2005 2010
Sources: Reuters and the ECB (including ECB calculations). Note: An overnight indexed swap (OIS) is an interest rate swap where the periodic oating rate of the swap is exchanged for an overnight index. In this chart, the 3-month OIS rate, which is based on the 3-month euro overnight index average (EONIA), is used to calculate the spreads. EURIBOR refers to the 3-month euro interbank offered rate. loans_large refers to the interest rates applied by monetary nancial institutions (MFIs) to loans to non-nancial corporations of over 1 million with a oating rate and an initial rate xation period of up to one year. loans_small refers to MFI interest rates on loans to non-nancial corporations of up to (and including) 1 million with a oating rate and an initial rate xation period of up to one year. Meanwhile, BBBrated corporate bonds refers to the euro-denominated nonnancial corporate bond index calculated by Bank of America Merrill Lynch including maturities of over one year.
Source: Datastream (regional market indices).
nancing costs. Coupled with a decline in condence and real economic conditions, these developments were ultimately associated with a marked reduction in lending to rms and households in the euro area. There was also evidence of an additional nonprice mechanism at play.3 For example, according to the Bank Lending Survey (BLS) of the ECB, credit standards for loans to households and corporations tightened signicantly. This survey rst started pointing to a tightening of credit standards (realised and expected) for private sector loans during the course of 2007 and the situation worsened considerably in the second half of 2008. In the case of loans to households, bank credit tightening was particularly linked to declining prospects for the housing market and the general macroeconomic outlook. The latter was also a key factor behind the tighter funding conditions faced by rms, albeit being reinforced by industry and rmspecic considerations as well as the high costs associated with the deterioration in the balance sheets and capital positions of banks. 4 While nancial factors were certainly central to triggering and subsequently amplifying the recessionary dynamics at play, it is important to highlight the signicant non-nancial elements involved. In particular, there was a generalised collapse in business and consumer condence that was highly synchronised across the main developed economies in the Organisation for Economic Co-operation and Development (OECD). With regard to euro area rms, the associated increase in uncertainty most likely contributed to a postponement of investment spending plans. Households also exhibited a sharp increase in precautionary behaviour, as reected in the dramatic rise in the household saving rate for the euro area.5 In addition, following the collapse of Lehman Brothers, global trade went into free fall (Chart 3), thereby amplifying the international transmission of the shock and the synchronicity of its effects. The contraction of trade ows had a big impact on economies such as those in the
Chart 3 World trade and euro area exports

(annual percentage change) world trade euro area exports 20 15 10 5 0 -5 -10 -15 -20 1985 20 15 10 5 0 -5 -10 -15 -20 1991 1997 2003 2009
2 UNDERSTANDING THE KEY DRIVERS OF THE CRISIS
Sources: Eurostat and the IMF.
euro area. Ironically, the strong international linkages associated with the development of global supply chains, which had partially underpinned the robust growth in world trade in the period prior to the crisis, may actually have reinforced the downward spiral of the fourth quarter of 2008 and the rst quarter of 2009.6 Reecting these strong international linkages, the volume of euro area goods exports declined on average by over 16% in 2009, compared with 2008. Indeed, the weakness of euro area exports largely explains why the collapse in euro area GDP was signicantly larger than the average decline in output experienced during previous systemic crises in the OECD countries.7
3 4
6 7
See, for example, ECB (2009a) and Ciccarelli, Maddaloni and Peydro (2010). See, for example, Box 3 in the Monthly Bulletin of February 2009 in which the results of the previous months Bank Lending Survey are reported and analysed. Box 6 in the Monthly Bulletin of August 2009 discusses the implications of this heightened uncertainty for economic prospects in the euro area. This explanation for the severity of the crisis on global trade is discussed in Anderton and Kenny (2011). See ECB (2009b).
Another important real factor behind the sharp adjustment in output and international trade during this period was a severe adjustment in inventories, reecting the heightened costs both actual and perceived of holding or accumulating inventories in an environment of uncertain future demand. Indeed, evidence from business surveys, such as the harmonised business surveys of the European Commission and the Eurozone Manufacturing Purchasing Managers Index (PMI), is consistent with the idea of some involuntary inventory accumulation during the crisis followed by a period in which excess inventory holdings were run down.8 For example, retailers may have reduced their stocks of nished goods by purchasing fewer goods from manufacturers, while manufacturers, in turn, were likely to have reduced their own stocks of materials and other inputs.9 Lastly, the volatility of inventory adjustment in 2008 and 2009 may also have reected less willingness on the part of suppliers to extend trade credit or trade nance in an environment of increased risk aversion and heightened sensitivity to credit risk. Trade credit may also have contributed to exacerbating the above-mentioned (dramatic) decline in international trade in some countries.10 To help conclude this review of the key economic drivers of the crisis, Chart 4 below attempts to provide a graphic summary from the euro area perspective of some of the main features that have been identied to date. It is important to emphasise that this gure focuses on what were the key triggers, propagation and amplication mechanisms rather than on providing any account of what ultimately caused the crisis. With the benet of hindsight, as Chart 4 would underline, the nancial sector was the trigger for economic developments during the crisis a fact that contrasts with most of the standard business cycle analysis of the past 30 years, where the nancial sector is seen as being more passive and not always considered central to understanding cyclical dynamics. The developments that
occurred were very much in line with the strand of research which emphasises the importance of nancial frictions and non-nancial factors, such as uncertainty shocks impacting households and rms. One key element highlighted in Chart 4 is the feedback loops which may exist within the nancial sector and also between the nancial and non-nancial sectors of the economy. Another key aspect underlined is the role of policy-makers and the impact of policy measures on private sector agents. A large number of crisis management policies were implemented in both the nancial and non-nancial sectors, many of which can be assessed to have had benign effects in terms of stabilising the economic situation. Also notable, however, was the contribution of the government sector to raising uncertainty, especially regarding the build-up of scal imbalances and sovereign risk in some countries. This was particularly evidenced by the 2010 increase in the long-term government bond yields of a number of euro area countries, following a rapid deterioration in their scal position. Furthermore, Chart 4 very clearly shows the complexity of the interactions across sectors and policy agents. Such complexity poses many challenges for the economic analysis, particularly in relation to the use of models. These often make use of simplifying assumptions regarding the exogeneity of certain variables and thus effectively exclude particular feedback channels.
8 De Rougement (2011) also discusses the role of inventories during the global recession precipitated by the nancial crisis. 9 The evidence on inventory adjustment during the crisis is discussed in more detail in Box 5 of the Monthly Bulletin of May 2009. 10 Considerable challenges arise in attempting to identify particular channels, mechanisms or frictions behind the collapse in global trade. For example, Eaton, Kortum, Neiman and Romalis (2011) question whether this collapse resulted from factors impeding international transactions or, more simply, whether it was due to goods being heavily traded in international markets. Their ndings suggest that the collapse in global trade was largely related to the latter rather than specic factors impeding trade per se (although for a small number of countries such factors were in fact more relevant).
10
Chart 4 The 2007-10 financial crisis: key trigger, propagation and amplification mechanisms
2 UNDERSTANDING THE KEY DRIVERS OF THE CRISIS
Global Economy
Households
Financial Sector Risk premia Asset prices and net worth
Non-Financial Firms
Increased uncertainty and precautionary savings Reduced housing and financial wealth International confidence spillovers
Increased uncertainty Higher funding costs and borrowing constraints Lower profits and reduced or delayed investment
Governments Increased funding costs and borrowing constraints Higher mortgage default Policy interventions Fiscal imbalances and sovereign risk International confidence and trade spillovers Increased corporate bankruptcy
In conclusion, the macroeconomic developments that occurred during the nancial crisis of 2007-10 have been so dramatic and extreme that they could be seen as a once-in-a-century event a perspective which, arguably, may imply some limitation on what can be learnt for economic analysis during more normal times. Such extreme uctuations are also characteristic of an environment of panic, reecting human responses to the absence of trust and the materialisation of fear and contagion or what economists tend to call Knightian uncertainty. At the same time, the nancial crisis has clearly cast considerable doubt on the so-called Great Moderation 11, thereby suggesting that the level of macroeconomic instability and volatility recently observed may possibly be more prevalent or more frequently occurring when viewed from a forward-looking perspective.
In this respect, our future work may have much to learn from our understanding of these recent developments.
11 The Great Moderation, a term coined by Stock and Watson (2002), refers to the reduction in the volatility of the business cycles of many advanced economies that started in the mid1980s.
11
HOW DID THE ECONOMIC ANALYSIS FARE DURING THE CRISIS? THE NEAR-TERM OUTLOOK
Chart 5 Early estimates of euro area GDP from a range of factor models and bridge equations plus the Euro Zone Barometer
(quarterly percentage change) outcome average factor model/bridge equation estimate Euro Zone Barometer 2.0 1.5 1.0 0.5 0 -0.5 -1.0 -1.5 -2.0 -2.5 -3.0 Q3 Q4 2008 Q1 Q2 Q3 2009 Q4 Q1 Q2 2010 2.0 1.5 1.0 0.5 0 -0.5 -1.0 -1.5 -2.0 -2.5 -3.0
3.1
As discussed in the introduction, the economic analysis provides an important analytical input into monetary policy deliberations, generating timely and, ideally, accurate information on the economic situation and outlook. One key element in the preparation of such projections is our assessment of the current state of the economy in real time or what are sometimes referred to as early estimates of GDP.12 These include a backcast of output developments in the previous quarter (required, given the substantial lags in the publication of national accounts), a nowcast for the current quarter and a near-term forecast for the quarter to follow. A large array of tools is commonly used to make an assessment of the current economic situation and its likely evolution over the short term. These feature tools attempting to synthesise the information available for a large cross section of high frequency indicators (e.g. from surveys and nancial markets) and to extrapolate their short-term trends. The latter tools include dynamic factor models which attempt to average over a large number of data series as well as simpler bridge equations linking key indicators to the components of euro area demand.13 Chart 5 displays some early estimates of euro area GDP obtained from a range of differing specications for bridge and factor models. These have been constructed using the vintages of data that were available at the time in order to illustrate the difculties of such mechanical estimates over the period concerned. For each quarterly GDP outcome, there are six early estimates based on data at intervals of roughly two weeks (ranging from forecasts to backcasts, as described above). The dynamic factor models and bridge equations estimated use aggregate euro area data as well as data at the sectoral, expenditure component and country level.
Sources: Calculations using factor models/bridge equations and the Euro Zone Barometer of MJEconomics.
For simplicity, we report the range of outcomes for quarterly euro area GDP as a vertical line, while the mean is shown in the horizontal reddish brown line. What is striking is the clear failure of all models to capture the period of exceptional macroeconomic weakness in the fourth quarter of 2008 and the rst quarter of 2009. It is also noteworthy that broadly similar forecast errors were also made by private forecasters (as reported in the Euro Zone Barometer). Although, most estimates demonstrate some convergence towards the outcome as more short-term economic information becomes available, even the nal estimates for 2008Q4 and 2009Q1 (made available just prior to the ofcial release of GDP data) point to a signicantly higher, albeit negative, GDP growth rate than the rate of between -2.0% and -2.5% actually registered. A key failure
12 See, for instance, ECB (2008a). 13 Such tools and their application are described in some detail in Banbura, Giannone and Reichlin (2010).
12
of the economic analysis during the crisis was therefore its inability to deliver reliable information on the current state of the economy; information which could have been used to help shape the more medium-term outlook. There are a number of reasons for this poor predictive performance. For example, there is the fact that tools are estimated over sample periods where such sharp volatility in growth dynamics has not previously been observed. Also, most, if not all, of the models used are essentially designed with the assumption of a stationary environment with mean reversion in growth dynamics, while the crisis developments were undoubtedly non-stationary in nature. Complications in the identication of seasonal factors (given the large uctuation in GDP and other macroeconomic time series around the turn-of-the-year 2008-09) as well as possible changes in the signalling power of survey data (due to an exceptionally strong adjustment in consumer and corporate sentiment) may have played a further role.14 More specically, most survey data are reported as balance statistics summarising the qualitative and subjective assessments of survey respondents. As such, they may not always have a direct, stable or linear relationship with actual developments in related economic series (e.g. as regards output, consumption, employment and prices). Lastly, as the recovery started to take hold, growth dynamics may have been driven by factors that are not easily incorporated into our short-term tools. For example, a number of government policies were introduced (such as the car scrapping premium) which positively impacted growth dynamics in the second half of 2009.15 3.2 MEDIUM-TERM PROJECTIONS
Chart 6 Evolution of euro area GDP forecasts for 2009

(annual average percentage change) ECB/Eurosystem (projections range) European Commission OECD outcome Euro Zone Barometer IMF ECB Survey of Professional Forecasters Consensus Economics 3 2 1 0 -1 -2 -3 -4 -5 -6 Jan. Apr. July 2008 Oct. Jan. Apr. July 2009 Oct. 3 2 1 0 -1 -2 -3 -4 -5 -6
3 HOW DID THE ECONOMIC ANALYSIS FARE DURING THE CRISIS?
Sources: ECB, European Commission, IMF, OECD, MJEconomics and Consensus Economics. Note: The x-axis shows the release dates of various estimates.
In the midst of the nancial crisis, the imprecise nature of the information on the current and near-term state of the euro area economy was also accompanied by substantially higher medium-term projection errors for euro area GDP and, to a lesser extent, consumer price ination, as measured by the Harmonised Index of Consumer Prices (HICP).
Chart 6 plots the evolution of ECB and Eurosystem projections for euro area GDP in 2008 and 2009, together with the corresponding projections from a range of private sector entities (MJEconomics (Euro Zone Barometer), Consensus Economics and ECB Survey of Professional Forecasters) and international organisations (the IMF, the OECD and the European Commission). In contrast to the projections from other institutions, those of staff within the ECB and the Eurosystem are presented in the form of ranges rather than as point estimates. The use of ranges acknowledges the inevitable uncertainty surrounding macroeconomic projections and, based on the current method, the width of the ranges is
14 For a discussion of the challenges posed by the crisis linked to the interpretation of seasonally adjusted data, see Box 7 in the Monthly Bulletin of August 2009. 15 Box 5 in the Monthly Bulletin of December 2010 reviews developments in the euro area and the global car industry, and highlights the role of vehicle scrapping schemes and massive government bailouts in some countries.
13
calibrated such that it is consistent with a 57.5% condence interval.16 It is clear that all of the projections in this chart were strongly lagging actual developments; only in late 2008 did public and private sector institutions begin to make downward adjustments to their growth forecasts for 2009, albeit while clearly underestimating the severity of the downward spiral that was taking place at that time. As regards the projections of the ECB and the Eurosystem staff, despite the attempt to capture the role of forecast uncertainty via the publication of projection ranges, the scale of subsequent forecast errors is suggestive of the wider uncertainty prevailing in this period. Indeed, only by mid-2009 were the available forecasts coming close to the eventual outturn of a decline in average annual euro area GDP of just over 4%. A similar, though less dramatic, pattern can be seen when looking at forecasts for
Chart 7 Evolution of euro area GDP forecasts for 2010
(annual average percentage change) ECB/Eurosystem (projections range) European Commission OECD outcome Euro Zone Barometer IMF ECB Survey of Professional Forecasters Consensus Economics 2.0 1.5 1.0 0.5 0 -0.5 -1.0 -1.5 Jan. Apr. July Oct. 2009 Jan. Apr. July Oct. 2010 2.0 1.5 1.0 0.5 0 -0.5 -1.0 -1.5
2010 (see Chart 7). In this case, forecasters systematically underestimated the strength of the recovery. In the rst half of 2009, most forecasters expected very slow growth in 2010, with a fair proportion of them even anticipating a fall in output. As more positive news emerged in the second half of 2009, the forecasts were steadily revised upwards but still remained well short of the nal outcome until the last quarter of the year. The evidently lagging nature of the information contained in most projections, together with the large projection errors made, highlights the inability of standard macroeconomic tools to deliver accurate point forecasts during times of heightened economic stress. Indeed, economic developments during periods of crisis and instability are, by their very nature, exceedingly unpredictable. A relevant question with regard to the performance of forecasters is whether it is realistic to expect them to predict the timing of crises. Ex ante it may be impossible to identify the particular source, trigger mechanism and timing of a crisis event.17 To the extent that such factors could have been identied, they might merely have just brought forward the materialisation of the crisis itself. In this respect, there may be some hindsight bias in an ex post analysis of forecast errors, reecting a tendency to believe having knowledge of the key factors at the centre of the crisis that we should have identied in a more timely manner its onset. Arguably, it is in the period after the onset
16 See ECB (2009c). Various methods have been employed by staff at the ECB to compute these ranges. From the rst publication of the staff projections of the ECB and the Eurosystem (December 2000) until June 2008, the published projection ranges for each variable and horizon represented twice the mean absolute projection error constructed on the basis of an analysis of historical projection errors. These published ranges were derived using a short sample of projection errors which was not updated over time. Another, model-based method was used for the staff projections published from September 2008 to September 2009, while in December 2009 the method for calculating the ranges was further updated to take account of the most recent projection errors and to allow for some correction for very extreme observations or outliers. 17 This has been likened to seismologists anticipating the timing of an earthquake (Spaventa, 2009).
Sources: ECB, European Commission, IMF, OECD, MJEconomics and Consensus Economics. Note: The x-axis shows the release dates of various estimates.
14
of the crisis, and not before it, that the scope for improvements in predictability may lie. For example, prior to the dramatic events of September and October 2008, the possibility for enhancing our projections was probably very low. However, in the light of the key events that took place (such as the outbreak of contagion in the nancial sector), forecasters might have identied better the severity and likely evolution of the macroeconomic events that were to unfold.18 The above poor performance is partially related to inaccuracies in terms of the technical (e.g. interest rates, stock prices, exchange rates, commodity prices) or other relevant underlying assumptions or forecasts (e.g. for external demand) that are used to underpin the euro area macroeconomic projections.19 For example, the over-prediction of euro area ination in the wake of the nancial crisis is inevitably linked to errors in fundamental assumptions about oil and other commodity prices, namely that they would evolve in line with the expectations embedded in commodity futures contracts. However, as regards GDP growth, it is unlikely that the scale of the observed projection errors can be explained by errors in technical assumptions alone. Rather, given the scale of the deterioration in forecast performance highlighted by Charts 6 and 7, these errors were most likely also a reection of inadequacies in both our forecasting tools and expert judgements (as projections include judgemental adjustments). Such shortcomings may have resulted in an inability to identify the degree of persistence in the shocks impacting the euro area and/or an underestimation of their associated transmission and propagation mechanisms (e.g. the likely feedback loops between the real economy and the nancial sector highlighted in Chart 4). The preceding analysis clearly demonstrates that forecast errors were remarkably high during the crisis period, perhaps reecting both the unpredictability of events and shortcomings in our models or expert judgement. Nevertheless, and in line with the evidence in Charts 6 and 7,
it is important to note that the projections of the ECB and the Eurosystem were not systematically worse than those of other institutions over this period. In other words, the failure to predict the depth of the crisis was widespread among professional economists and not just specic to forecasters at these institutions. It should also be acknowledged that, given the truly exceptional nature of the nancial crisis, it is not surprising that forecast errors were exacerbated during this period. Moreover, even if forecasters were consistently underestimating the severity of the economic contraction, they did at least offer some reliable qualitative signals for policy-makers. For example, the staff projections of the ECB for March 2009 predicted that euro area GDP growth would be negative in 2009, ranged between -3.2% and -2.2%. What this implied was the worst recession since the end of World War II an assessment which was undoubtedly borne out by subsequent developments. 3.3 ANALYSIS OF UNCERTAINTY AND RISKS
One feature evident in Charts 6 and 7 is the relative proximity of the macroeconomic projections from the different institutions and the analysts in the private sector. Given the nature and magnitude of the shock hitting the global economy and the considerable uncertainty that existed at the time, it is perhaps surprising that there was not greater divergence between private and institutional forecasters. In particular, to the extent that agents have different models or adapt their forecasts at different speeds in response to economic news, one would expect large shocks to be associated with higher levels of forecaster disagreement.
18 Caballero (2010) emphasises the clear limits to improvements in prediction during periods of crises: Modern Cassandras will always claim to have seen the crisis coming. What they will not tell you is how many times they saw things coming that never materialized or how the specic mechanisms behind the crisis are different from those on which their predictions were based. 19 The technical assumptions underpinning the macroeconomic projections of staff in the ECB and the Eurosystem are regularly described in a box in the Monthly Bulletin. See, for example, Box 8 in the December 2010 issue of the ECB Monthly Bulletin.
15
However, the similarity of projections on the part of professional forecasters and institutions publishing projections might be seen as symptomatic of some herding behaviour. More detailed sources of information on individual forecasters do, however, suggest heightened disagreement. According to the Survey of Professional Forecasters (SPF) of the ECB, which is based on a panel of approximately 90 separate forecasters, the standard deviation of the individual one-year ahead forecasts for euro area GDP a measure of forecaster disagreement increased from below 0.2 percentage point in late 2007 to above 0.6 percentage point in the second quarter of 2009.20 Notwithstanding the above evidence on disagreement about point forecasts, there may also have been a tendency to reect the deterioration in the outlook during the crisis more in the analysis and the communication of risks surrounding projections, rather than in revisions to the baseline or central scenario. For example, in summarising the view of the Governing Council in December 2008, the Monthly Bulletin of the ECB states that the economic outlook remains surrounded by an exceptionally high degree of uncertainty. Risks to economic growth lie on the downside. One distinct message emerging from recent experience may thus be that point forecasts have limitations, especially during periods of macroeconomic instability. During such times, the analysis of the risks accompanying our outlook may prove to be a more informative source of input into monetary policy discussions. Chart 8 below provides some further evidence on the real time role of expert risk assessments during the crisis, drawing on information from the SPF conducted by the ECB. The chart shows the baseline projection from various vintages of the SPF, together with the corresponding risk balance indicator extracted from the probability forecasts also collected under the survey. The risk balance indicates
Chart 8 Risk analysis for 2009 GDP growth forecasts from the Survey of Professional Forecasters of the ECB
(probability balance and annual growth rates) risk balance (left-hand scale) baseline projection (right-hand scale) outcome (right-hand scale) balanced risk (left-hand scale) 65 3 2 60 1 55 0 -1 50 -2 -3 45 -4 40 Aug. Nov. Feb. May Aug. Nov. Feb. May Aug. Nov. 2007 2008 2009 Source: The Survey of Professional Forecasters of the ECB. Note: The risk balance is extracted from the aggregate density in the SPF and refers to the probability, in percentage terms, of observing an outcome that is lower than the average point forecast for 2009 in each survey over the period. -5
the probability of observing growth lower than the point forecast as indicated in each survey over the period. It is noteworthy that the risk analysis seems to point to a sharp increase in downside risks in late 2007 and that the SPF correctly identied more signicant downside risks over the course of 2008 (with the survey round of August 2008 being a notable exception). Also, the risk analysis highlighted an intensication of those risks in November 2008. On the other hand, given that the survey continued to predict positive growth rates for 2009 in the November 2008 round, there is clear evidence that the severity of the downside risks was underestimated during the period when the actual free fall in activity (described in Section 1)
20 Chart 11 in Section 3 of this paper highlights the upward impact of increased forecaster disagreement on overall forecast uncertainty. The increase in forecaster disagreement is very much in line with other stylised facts on the behaviour of forecaster disagreement over the business cycle. For example, Dovern et al. (2009) demonstrate that disagreement about real variables has a strong tendency to intensify during periods of recession.
16
was already taking place. Furthermore, in the February 2009 survey, when the point forecast for 2009 had been revised downwards to signal decidedly negative growth, the upside skew in the SPF risk assessment was providing a clearly erroneous signal concerning the state of the economy. Overall, the experience with the SPF probability forecasts during the crisis period is somewhat mixed and thus suggests the necessity of exploring new ways to improve the usefulness of such indicators in the future. 3.4 MACROECONOMIC MODELS
cycle dynamics. In this regard, the structural interpretation of the shocks in DSGE models means that they may have an important value in helping to interpret economic developments as they unfold. This real time interpretative function of models also applies to some of the short-term forecasting tools. For example, a signicant recent advance in the use of factor models has been the ability to trace how economic news shapes revisions to early estimates of GDP. 22 To give an idea of the interpretative value of models, Chart 9 presents the historical decomposition of euro area GDP developments using the New Area-Wide Model (NAWM) of the ECB for the period from early 2007 to mid-2010. Although the nancial sector plays only a passive role in the NAWM23, the model provides nonetheless an insightful interpretation of the crisis events as they unfolded at the time. In particular, the NAWM emphasises the strong role of international factors linked to external demand and the importance of international linkages noted earlier in Section 1 in contributing negatively to euro area GDP growth, especially during the initial phase of the downturn and when the free fall in activity occurred around the turn-of-the-year 2008-09. Considering the downturn from this perspective, it is clear from the NAWM interpretation that several important domestic factors also contributed negatively to the evolution of euro area GDP. For instance, the strong role played by risk premia in depressing domestic demand, resulting in relatively weak investment and heightened precautionary behaviour among euro area households. In addition, nominal adjustment especially downward wage
21 Economists such as Buiter (2009), Akerlof and Shiller (2009) and Krugman (2010) have argued that the current generation of micro-founded theory-based models represent a wrong turn for the economics profession. On the other hand, Lucas (2009) provides a more positive assessment of these models. 22 See Banbura, Giannone and Reichlin (2010). 23 See Christoffel, Coenen and Warne (2008). Ongoing research is aimed at extending the NAWM to include nancial frictions. Meanwhile other models in use at the ECB, such as that of Christiano, Motto and Rostagno (2003), already incorporate important nancial frictions.
The large projection errors recently experienced would imply that there is still signicant room for improvement among professional and institutional forecasters; a theme that we shall return to in the remainder of this paper. As indicated previously, the nancial crisis has also pointed to weaknesses in the macroeconomic models which support our projections. In particular, criticism has been directed at state-of-the-art dynamic stochastic general equilibrium (DSGE) models, although much of this is actually just as relevant to the other, more traditional, workhorse models used for forecasting in central banks, including those implemented within the Eurosystem.21 Mainstream macro models have been criticised for having unrealistic assumptions (e.g. perfect information and rational expectations) and for paying little or no attention to nancial frictions, the role of the banking sector and to non-linear dynamics or interrelationships. The crisis has therefore provided a strong impetus to modelling research, particularly in terms of enriching the role of the nancial sector and in attempting to relax other important assumptions, such as rationality and linearity. Despite these criticisms, DSGE and other traditional macroeconomic models remain very useful tools. For example, they can be used to help test the internal consistency of a particular economic theory with the data and may, thus, provide clues about the essential facts that need to be considered when interpreting business
17
Chart 9 NAWM decomposition of GDP growth from the vantage point of the September 2010 staff macroeconomic projection exercise (MPE) of the ECB
(quarterly percentage change and contributions in percentage points) technology demand mark-up 1.5 1.0 0.5 0.0 -0.5 -1.0 -1.5 -2.0 -2.5 -3.0 -3.5 2008 2009 2010 foreign monetary policy output growth 1.5 1.0 0.5 0.0 -0.5 -1.0 -1.5 -2.0 -2.5 -3.0 -3.5
macroeconomic volatility. This calls for further work to assess the appropriate economic tools for periods of macroeconomic stress and the extent to which the output of these tools should also reect off-model information (e.g. expert judgement) in order to help reduce forecast errors in times of extreme macroeconomic turbulence.
Source: Calculations based on Christoffel, Coenen and Warne (2008). Note: The chart shows the contribution of various groups of structural shocks to the deviation of output growth from its steady state rate of growth (0.5% quarter-on-quarter).
adjustment was relatively limited in the period immediately following the large negative shocks to output and demand. This contributed further to the prolongation of the downturn, as reected in the visibly negative impact of the mark-up shocks to GDP growth. The NAWM interpretation also helps to identify the function of negative supply side factors (technology shocks) that may have been linked to dampened prospects for euro area potential growth around the turn-of-the-year 2008-09. Lastly, the model determines the role of monetary policy measures in helping to stimulate the economy from mid-2009 onwards.24 This section has highlighted the widespread failure to predict the macroeconomic developments that occurred during the crisis as well as their severity. Although such a failure may not be all that surprising, given that existing tools were largely developed with normal business cycle uctuations in mind, it still raises questions about the appropriateness of using such tools during periods of exceptional
24 The fact that the contribution of monetary policy is not positive before mid-2009 (and is even negative) may reect that in the NAWM the policy contribution captures only the interest rate effect which is restricted by the zero lower bound. Moreover, non-standard monetary policy measures are not included in the model.
18
THE BENEFIT OF HINDSIGHT: WHAT SHOULD WE HAVE PAID MORE ATTENTION TO?
Chart 10 Turning point predictions Area-wide Leading Indicator (ALI)

(standardised deviation from trend) Area-wide Leading Indicator (ALI) Business Cycle Indicator (BCI) a) May 2007 3 2 1 0 -1 -2 3 2 1 0 -1 -2 -3 2001 2003 2005 2007 2009
The experience with macroeconomic tools and projections during the period of nancial crisis offers an important opportunity with the benet of hindsight to identify the factors and developments to which the economic analysis could have paid more attention. This insight may also help establish where improvements are required and the priority areas for future tool development. Four particular areas are highlighted below. 4.1 FINANCIAL FACTORS
4 THE BENEFIT OF HINDSIGHT: WHAT SHOULD WE HAVE PAID MORE ATTENTION TO?
-3 1999
Given the role of the nancial sector during the crisis, it would appear that economic forecasts could have been improved if more weight had been given to nancial variables. A number of studies have found, for instance, that stock prices can be informative for GDP developments.25 Indeed, the importance of nancial factors was appreciated in real-time by ECB/Eurosystem staff forecasters who took account of tighter nancing conditions emanating from wider nancial market spreads as well as tighter credit standards in their projections.26 A new monthly Area-wide Leading Indicator (ALI) for the euro area business cycle has recently been developed which seeks to exploit the possible leading indicator properties of particular nancial series.27 The ALI is derived by choosing nine leading series, including a number of nancial variables and surveybased condence measures. As demonstrated in Chart 10, it can help to predict in real time the euro area business cycle during the recession of 2008-09 and the following upturn. As can be seen from the chart, on the eve of the crisis (May 2007), the ALI was already indicating that the cyclical peak had passed. Prior to the intensication of the crisis in the autumn of 2008, the ALI was signalling a move into a below trend phase of the cycle and by mid-summer 2009 there were the rst tentative signs that the cyclical trough had passed.
b) August 2008 3 2 1 0 -1 -2 -3 1999 2001 2003 2005 2007 2009 3 2 1 0 -1 -2 -3
c) August 2009 3 2 1 0 -1 -2 -3 1999 2001 2003 2005 2007 2009 3 2 1 0 -1 -2 -3
Sources: De Bondt and Hahn (2010). Note: The BCI refers to the deviation of economic activity (measured by monthly industrial production excluding construction) from its trend level using a one-sided band pass lter. The ALI is shifted by the ve months it leads the BCI.
Aside from the clear need to develop reduced form indicators and short-term forecasting tools
25 See, for instance, De Bondt (2009) and Anderson and DAgostino (2008). 26 As indicated, for instance, in the text accompanying the ECB/ Eurosystem staff projections disclosed at the time (e.g. the December 2007 and June 2008 issues of the Monthly Bulletin). 27 Such an approach is proposed in De Bondt and Hahn (2010).
19
adequately reecting the information content of nancial variables, an additional challenge will be the incorporation of nancial frictions and nancial intermediation into forecasting models. It has long been known that nancial markets are imperfect due to, inter alia, information asymmetries between borrowers and lenders as well as the costly enforcement of nancial contracts. In this regard, research which predates the nancial crisis has highlighted the importance of such frictions in the propagation of crises and as a key amplier and source of business cycle uctuations.28 For the euro area, in particular, the role of nancial frictions in driving the cyclical dynamics has been underlined by the Christiano, Motto and Rostagno (CMR) model.29 Despite these examples, it seems fair to say that nancial frictions have not played a major role in large-scale macroeconomic models.30 Much work remains to be done in this eld in terms of assessing which frictions are most important and whether models with nancial frictions can meaningfully characterise the nature of nancial crises. It is also unclear whether such models can be useful in real time for analysing economic developments and informing policy-makers about the possible impact of differing policy responses. 4.2 NON-LINEARITIES
extent, for divergent economic dynamics, depending on the particular economic conditions prevailing at a given point in time. Such tools can also utilise the information contained in nancial variables to help identify the timing of regime changes. This type of model, when applied to the period of nancial crisis, shows a clear increase in the probability of the euro area entering a recession around six months in advance of the start of the 2008-09 recession.32 While such tools thus appear to hold some potential particularly in terms of interpreting or understanding developments it is important to bear in mind the limitations that may apply to them in practice. With the benet of hindsight, it is often possible to nd a good non-linear model that ts the data for a crisis period. However, in real time, such methods may pose substantial challenges that could limit their actual usefulness for out-of-sample prediction. In other words, such models may explain features in data that do not occur very frequently, but if these features are not present in the forecast period or the crisis in question, then they are unlikely to be of benet at that time. 4.3 CONFIDENCE, EXPECTATIONS AND UNCERTAINTY
Standard tools based on a stationary and linear environment can serve us well in providing point estimates for GDP growth and ination during normal times. However, they are far less reliable during periods of extreme economic and nancial turbulence where nonlinearities and non-stationarities are likely to be prevalent. This suggests that non-linear models are worth investigating, although there is very mixed evidence as to whether they can reliably improve forecasting performance in real time.31 The possible relevance of non-linear effects can be investigated with regime-switching models, distinguishing between periods of low and high growth. Such methods attempt to incorporate the uncertainty associated with our knowledge of the state of the economy and allow, to some
In line with the analysis presented in Sections 1 and 2, a generalised collapse in business and
28 See, for example, Bernanke and Gertler (1989) and Bernanke, Gertler and Gilchrist (1999). 29 See Christiano, Motto and Rostagno (2003). De Fiore and Tristani (2009) propose a calibrated model with asymmetric information under which shocks have a strong impact on spreads and on economic growth. In particular, the decline in investment is of the same magnitude as that observed during the recent crisis. 30 See Fagan and Morgan (2005) for a comprehensive, albeit somewhat dated, description of the main macroeconomic models used by euro area central banks. 31 Harding and Pagan (2002) nd little evidence that certain non-linearities are important to the nature of business cycles. See Tersvirta (2006) for a review of the experience with non-linear models in economic forecasting. Similarly, in an earlier review article, Clements, Franses and Swanson (2004) conclude that simple, reliable and easy to use non-linear model specication, estimation and forecasting procedures will only be readily available in the distant future. 32 See Bellgo and Ferrara (2009) for the euro area.For the US, Hubrich and Tetlow (2010) use a Markov-Switching VAR model to examine economic dynamics under regimes of nancial stress and non-stress.
20
consumer condence (both in the euro area and globally) was a central feature of the recent recession. There are mixed views among macroeconomists about the value added of survey-based measures of condence and how they should be used in practice, together with hard data and macro models and other economic tools. Traditionally, condence indicators have been treated by macroeconomists as providing some information for the short-term assessment, especially in situations where other hard indicators are lagging. But many are sceptical about whether such indicators provide more information than that contained in other variables. There have also been some examples (around the time of noteworthy events) where condence measures have reacted much more strongly than the subsequent hard data that they might have been expected to track.33 Furthermore, given the extremely low values realised by some condence measures at certain points in time, there is the possibility of non-linear response patterns from the balance statistics reported in some surveys. Nevertheless, in the recent crisis, there was clearly a strong crisis of condence and such measures generally did not provide misleading signals and could have been relied upon more as the crisis intensied in the latter part of 2008.34 Even among those macroeconomists who regard condence measures as helpful in the nearterm, condence has often been disregarded as a major driver of medium-term economic developments for which more fundamental factors (such as income, real wages and employment) play the dominant role. However, the recent crisis has certainly highlighted that condence indicators may provide important information about the development of private sector perceptions regarding uncertainty and that this is a factor that may have substantial implications for the real economy. For example, given the irreversibility of investment decisions, heightened uncertainty may be associated with a sudden sharp stop in
investment expenditures related to a connected increase in the option value of waiting. Similarly, heightened uncertainty about future income prospects may cause households to delay consumption and to increase their savings. Recent work shows that condence indicators can have some predictive power to explain euro area private consumption expenditure, even when controlling for more usual explanatory factors (such as income, wealth and employment). In particular, these indicators appear to matter especially in periods of heightened uncertainty (featuring nancial crises or geopolitical tensions) where they may play a signicant role.35 There may also be substantial non-linear effects on real activity associated with shocks to uncertainty.36 A related aspect concerns the formation of private sector expectations; something which may also be partially captured by new approaches to analysing how agents learn about economic developments. A good example of this is to be found within the learning mechanisms of the New Multi-Country Model (NMCM) of the ECB which allow the forwardlooking components driving investment and consumption decisions to adjust downwards in response to negative economic news. This analysis has also pointed to some clear divergences across the euro area regarding the impact of the crisis on households and rms expectations about economic prospects.37 Overall, there would seem to be a need to further develop indicators of uncertainty and to possibly incorporate such indicators into macroeconomic models. One potentially rich source of information on uncertainty is the SPF of the ECB which can combine the signals from measures of forecaster disagreement with
33 These would include consumer condence around the time of the Long-Term Capital Management crisis and industrial condence following the attacks of September 11. 34 See ECB (2008). 35 See Dees and Soares Brinca (2011). 36 See Bloom (2009). 37 See Dieppe, Gonzlez Pandiella, Hall and Willman (2011) and Dieppe, Gonzlez Pandiella, and Willman (2011).
21
Chart 11 Confidence channels and output uncertainty

(left-hand scale: index points; right-hand scale: percentage points) implied stock market volatility (left-hand scale) implied bond market volatility (left-hand scale) SPF uncertainty two-year ahead (right-hand scale) SPF uncertainty five-year ahead (right-hand scale) 70 60 50 0.8 40 0.7 30 0.6 20 10 0 2000 2002 2004 2006 2008 2010 0.5 0.4 1.0 0.9
has always played a role in projections and can take the form of add-factors in traditional macroeconomic models or structural shocks in DSGE models. Expert judgement is included into models to take account of elements not present in these models and it may be calibrated using another tool thought to be more reliable or to contain additional relevant information in a given set of circumstances. A key challenge in improving the nature of the judgement used in the economic analysis and in projections is to ensure that there are some rigorous foundations underpinning its formation and that expert judgement should not simply involve best guesses or unsubstantiated opinions. In large-scale models, judgement allows for many additional factors to be taken into account. These would include inuences such as condence, adjustments for possible nonlinear impacts and nancial factors, all of which could stem from other tools. In calibrating their judgement, experts should place a strong emphasis on understanding the sources and determinants of forecast errors in real time as well as their economic interpretation (e.g. whether they reect structural change or demand shocks) and thereby help to form an understanding of the likely persistence of recent shocks. In this respect, judgement can also be of help in the assessment of which tools are likely to prove most reliable at any particular point in time. The use of judgement also provides an opportunity to depart from the standard economic paradigms that are embedded in our macroeconomic models. For instance, such tools typically do not take account of departures from rational behaviour (e.g. fairness considerations in labour economics, the role of simple rules of thumb when forming expectations, information cascades, herding and panic behaviour in nancial markets and modelling of excessive risk-taking behaviour) plus the existence of highly persistent disequilibria and imbalances. Considerations of this nature can be exceedingly complex to model and it is not realistic (or even desirable) to expect them to be incorporated into
Sources: Bloomberg, the SPF of the ECB and ECB calculations.
information from the density forecasts of survey participants (see Chart 11 which also shows some data on implied volatility in nancial markets). The latter may also signal the extent to which changes in uncertainty are seen as only affecting the short to medium term, or are expected to be more persistent, given that the SPF provides information on perceived uncertainty over different time horizons, i.e. for one, two and up to ve years ahead. In this respect, a notable feature during the crisis was the widespread upward shift of uncertainty indicators relating to GDP, as observed in Chart 11. The concurrent upward shift in both short and long-term uncertainty indicators may help explain the strength of the precautionary behaviour of rms and households during the most intensive phases of the crisis. 4.4 JUDGEMENT
In its narrowest sense, judgement can be dened as what one adds to the mechanical results of a model or tool in order to produce a given projection (although it can also encompass the choice of which models or tools to use). Judgement
22
our workhorse models to a signicant extent, so it is necessary to develop auxiliary models or tools. Of course, allowing such factors to inuence our thinking and hence the judgement we put into a projection, requires an openness to depart from the standard paradigms. 4.5 INTERNATIONAL LINKAGES
The crisis has highlighted the need to pay greater attention to the international dimension of economic analysis, especially in the context of highly integrated international nancial systems and more closely integrated production processes (e.g. via global supply chains).38 Whilst many of the shortcomings in the international projections are simply reections of the same failings which can be seen in projections for the euro area, the crisis has shown that there is a particular need to pay more attention to the international transmission of both nancial and condence factors. There is also evidence of a link between US and euro area condence, as US condence shocks can have a signicant impact on euro area condence indicators in the short term.39 Lastly, as regards ination, the crisis highlighted a dramatic change in the outlook linked to sharp uctuations in global commodity prices within the context of a rapidly changing environment for global and euro area growth. This would indicate that there is considerable merit in enhancing our understanding of global inationary developments. This particularly relates to global supply and demand imbalances and commodity price developments in the context of rapid changes in global economic activity.
38 One particular aspect concerns trade. When global trade declined very sharply at the end of 2008, there was a marked decline in the volume of goods-in-transit (for instance, those subject to shipping). The consequences of this were similar to that of an inventory adjustment (which was also taking place at the time). Hence, a better understanding of the dynamics of trade ows is important, especially if such a situation were to arise once again in the future. 39 Dees and Soares Brinca (2011).
23
WHAT IS THE WAY FORWARD?
In the light of the preceding discussion, a number of key directions for future work can be emphasised. These include the need to: i) extend existing tools and develop new tools, where appropriate; ii) develop ways to better handle the complexity arising from more tools and models; and (iii) given the limited information of point forecasts, to further develop the analysis of risk and the measurement and assessment of forecast uncertainty. Certainly, such developments cannot be expected to eliminate forecast errors, particularly when economic conditions are extreme, but they should enhance our toolkit for interpreting and projecting economic developments as well as the related risks and, thereby, contribute to an enhancement of the overall economic analysis. 5.1 EXTEND EXISTING TOOLS AND DEVELOP NEW ONES
prove central to a proper understanding of developments during periods of economic and nancial instability. A major additional challenge would appear to be to identify those economic variables, estimates and forecasts likely to be of most relevance at any given point in time. The development of reliable tools capturing possibly relevant nonlinearities and the prediction or identication of turning points would also be particularly useful. With regard to larger scale models, there is a need to continue developing more realistic models, especially as concerns the treatment of nancial frictions and modelling of expectations formation. 5.2 DEVELOP WAYS TO BETTER HANDLE COMPLEXITY
The need to take better account of the factors listed in Section 3 would imply putting an emphasis (where possible) on extending existing tools to reect such factors and, where such extensions are not possible, to develop new tools that better capture them. In the case of shortterm forecasting tools, models already exist that incorporate large information sets, including many of the nancial (e.g. spreads, asset prices, loans) and non-nancial (e.g. condence, trade spillovers) factors assessed to have been important during the crisis. Nonetheless, some efforts could be directed at extending the range of available information that is incorporated into the short-term assessment tools, including high-frequency information and that gleaned from non-ofcial sources (e.g. such as Google Trends, automated teller machines and electricity consumption).40 Furthermore, new datasets linked to the Household Finance and Consumption Survey and the Survey of Access to Finance of SMEs can also be utilised.41 Although such data sources are not very timely and hence cannot be used for conjunctural analysis, they may still help inform us about the key features and structures that might
While the crisis has certainly provided much impetus to the development of new tools and the extension of existing ones along the lines noted above, a continuing challenge will be to expand models both meaningfully and tractably.42 In respect of this, it is unlikely to be feasible (e.g. due to data limitations) or optimal (e.g. due to the need to understand core model properties) to include all innovations in one all-encompassing model. This would suggest a need to further develop the existing suite of models approach, whereby the analysis embedded in a given workhorse model can be supplemented by insights from other models; ones which could be extended to incorporate additional relevant information or missing elements (e.g. nancial
40 See, for instance, Varian and Choi (2009) on Google Trends. Esteves (2009) considers the usefulness of ATM and Point of Sale (POS) information in forecasting and nds some gains in the nowcasting of non-durable consumption in Portugal. Fenz and Schneider (2009) have also investigated the use of truck mileage data in Austria. 41 See Survey Data on Household Finance and Consumption Research Summary and Policy Use in the Occasional Paper Series of the ECB (No 100, January 2009) and also the Survey on the access to nance of small and medium-sized enterprises (ECB website http://www.ecb.europa.eu/stats/money/surveys/ sme/html/index.en.html). 42 See Papademos (2010). In addition, Spaventa (2009) mentions the practical difculties of expanding models to include plausible descriptions of the nancial sector as one of the explanations for the limited progress in this area.
24
factors, condence and behavioural factors and non-linear dynamics). Consequently, an important strength of the suite of models approach is that it may help avoid a hierarchy of models that is inexible and time-invariant and where one particular model, as a rule, is considered to be superior or more reliable than the others. Instead, this approach places more of an emphasis on model selection, allowing for the fact that models which appear central to the understanding of the macro economy during some periods may prove to be less relevant or even irrelevant during others (and vice versa). The practical and conceptual limitations to the development of a single, all- encompassing model also highlight the potential value in the application of forecast combination techniques. Under forecast combination, the projections derived from different models or paradigms are averaged together in a manner which may improve the overall forecast performance. Combined forecasts may therefore help hedge against the possible poor performance of any individual forecasting model due to sudden changes in the economic environment. In this respect, combination methods that are able to identify and switch the focus to other possibly better performing models and indicators during periods of extreme volatility need to be built into the framework for the shortterm assessment. A key practical challenge is, however, the development of methods to identify in real time which models, forecasts or tools should receive the highest weight certainly, no easy task.43 5.3 DEVELOP FURTHER THE RISK ANALYSIS
In order to better understand risks, greater consideration needs to be given to the factors that might drive alternative scenarios as well as their likelihood. In this regard, there would be clear benets in drawing on the increased monitoring of disequilibria and imbalances in euro area countries for instance, those affecting public nances, asset markets, international trade and nancial ows when constructing risk scenarios that might involve a rapid correction of such imbalances. Such an emphasis on risk assessment may also call for the development of other qualitative and probabilistic indicators highlighting the likelihood of certain events (e.g. a recession or deation) or other information on forecast uncertainty that can be extracted from density forecasts (e.g. the spread or skew of the forecast distribution). More generally, it should be required that models and tools are able to provide measures of uncertainty along with their point forecasts.45 Moreover, consistent with the suite of models approach, it may not be appropriate to rely on a single measure of uncertainty from any specic tool. Indeed, as with point forecasts, there may be signicant gains from combining the information on forecast uncertainty from competing density forecasts.46 An example of this can be seen in the aggregate uncertainty measure from the Survey of Professional Forecasters of the ECB discussed in Section 3. However, further efforts are needed to explore alternative ways to optimally combine competing models or expert assessments of uncertainty.
43 Recent research related to the SPF of the ECB by Genre, Kenny, Meyler and Timmermann (2010) has also pointed to potential gains from forecast combination (see also Kenny, 2010). In particular, more optimal combination weights yield improvements in forecast performance for GDP (both on average and for the period of crisis). 44 The tendency to underestimate risk was powerfully emphasised in the book by Taleb (2007) which was written just before the onset of the recent nancial crisis. 45 As, for instance, in the Bayesian VAR tool for short-term ination forecasting of Giannone, Lenza, Momferatou, and Onorante (2010). 46 For instance, a combination of Gaussian densities will tend to exhibit non-Gaussian features such as skewness and fat tails. Such combinations may therefore better approximate underlying uncertainty if the true density is non-Gaussian. See, for example, Hall and Mitchell (2007).
5 WHAT IS THE WAY FORWARD?
The crisis has shown the need to look beyond point forecasts and pay more attention to risks.44 Moreover, the scope for improving point forecasts in real time is likely to be fairly limited particularly during crisis episodes and around turning points. Therefore, the economic analysis should place a strong emphasis and priority on risk assessments around the baseline projections.
25
CONCLUSIONS
The economics profession in general, and economic forecasters in particular, have faced some understandable criticism for their failure to predict the timing and severity of the recent economic crisis. Although many pointed to the emergence of economic imbalances, the speed and depth of the recession were still a major shock to most, if not all, economists. As we have documented, this can be seen in the widespread failure to anticipate the large decline in GDP that occurred in late 2008 and early 2009, or even to diagnose such developments correctly at the time that they were taking place. Various macroeconomic models and tools can offer plausible explanations for economic developments during the crisis, which is certainly helpful for our understanding of this period. However, as an input into forwardlooking policy deliberations, economic analysis must strive to provide more than reasonable ex post assessments of such crises and to develop tools that can also help inform us ex ante. In this context, we have argued that there is a need to develop tools which take greater account of nancial factors and also to develop and explore the usefulness of non-linear models. While it may well be the case that nancial factors and non-linear behaviour may not be important in normal times, it is essential that such models are part of the available toolkit or suite of models. Forecasters must then use their judgement or other model selection criteria to assess which model or tool is likely to be the most relevant at any specic point in time. To supplement this judgement, particularly in situations when model uncertainty is high, further emphasis could also be placed on developing ways to optimally combine diverse forecasts from differing tools. Some more general lessons, which would also apply in normal times, would seem to be the need to take greater account of international linkages and to broaden the range of information that is analysed. As regards international linkages, it is clearly necessary to look beyond the trade
channel and to examine the transmission of both nancial and condence shocks. As regards information, there is a need to extend the focus of short-term assessment tools and to nd ways to include very high frequency information and information gleaned from non-ofcial sources. Finally, the crisis has shown the necessity of looking beyond point forecasts and giving greater consideration to risk. What has been proposed above would appear to offer a promising but challenging agenda for professional economists in the coming years. However, while there seems to be some prospect of progress, it is likely that macroeconomic forecasters will continue to make signicant forecast errors in the years ahead. Predicting the timing of crises which may be linked to the bursting of asset price bubbles or the rapid correction of imbalances accumulated over a number of years is an inherently difcult if not impossible task.47 Nevertheless, by considering such factors and the risks of alternative scenarios, the economic analysis can at the very least aim to provide more useful information to policy-makers.
47 See, for instance, the recent work by Harding and Pagan (2010) on predicting recessions.
26
REFERENCES REFERENCES Akerlof, G. and Shiller, R. (2009), Animal Spirits: How Human Psychology Drives the Economy, and Why It Matters for Global Capitalism, Princeton University Press. Anderson, M. and DAgostino, A. (2008), Are sectoral stock prices useful for predicting euro area GDP? Working Paper Series, No 876, ECB. Anderton, B. and Kenny, G. (2011), Macroeconomic Performance in a Globalising Economy, in Anderton and Kenny (eds.), Macroeconomic Performance in a Globalising Economy, Cambridge University Press, Cambridge, UK. Banbura, M., Giannone, D. and Reichlin, L. (2010), Nowcasting, Working Paper Series, No 1275, ECB. Bellgo, C. and Ferrara, L. (2009), Forecasting Euro-Area Recessions Using Time-Varying Binary Response Models for Financial Variables, Document de Travail, No. 259, Banque de France. Bernanke, B. and Gertler, M. (1989), Agency costs, Net Worth, and Business Fluctuations, American Economic Review, Vol. 79, 1, 14-31. Bernanke, B., Gertler, M. and Gilchrist, S. (1999), The Financial Accelerator in a Quantitative Business Cycle Framework, Handbook of Macroeconomics, North Holland. Bloom, N. (2009), The impact of uncertainty shocks, Econometrica, Vol. 77, No. 3. Buiter, W. (2009), The unfortunate uselessness of most state of the art academic monetary economics, Financial Times, 3 March. Caballero, R. (2010), Macroeconomics after the crisis: Time to deal with the Pretense-ofKnowledge Syndrome, Working Paper Series, No. 10-16, MIT Department of Economics. Christiano, L., Motto, R. and Rostagno, M. (2003), The Great Depression and the FriedmanSchwartz Hypothesis, Journal of Money, Credit and Banking, Vol. 35 (6), pp. 1119-1197. Christoffel, K., Coenen, G. and Warne, A. (2008), The New Area-Wide Model of the Euro Area: A Micro-Founded Open-Economy Model for Forecasting and Policy Analysis, Working Paper Series, No 944, ECB. Ciccarelli, M., Maddaloni, A. and Peydro, J-L. (2010), Trusting the Bankers: A new look at the credit channel of monetary policy, Working Paper Series, No 1228, ECB. Clements, M. P., Franses, P.H. and Swanson, N.R. (2004), Forecasting economic and nancial time series with non linear models, International Journal of Forecasting, Vol. 20, pp. 169-183. Dovern, J., Fritsche, U. and Slacalek, J. (2009), Disagreement among forecasters in G7 countries, Working Paper Series, No 1082, ECB.
27
Dees, S. and Soares Brinca, P. (2011), Consumer Condence as a Predictor of Consumption Spending: Evidence for the United States and the Euro Area, ECB Working Paper No. 1349. De Bondt, G.J. (2009), Predictive content of the stock market for output revisited, Applied Economics Letters, Vol. 16 (13), pp. 1289-1294. De Bondt, G. and Hahn, E. (2010), Predicting recessions and recoveries in real time: The Euro Area Wide Leading Indicator (ALI), Working Paper Series, No 1246, ECB. De Fiore, F. and O. Tristani (2009), Optimal Monetary Policy in a Model of the Credit Channel, Working Paper Series, No 1043, ECB. De Rougement, P. (2011), Les Stocks dans la crise, Bulletin de la Banque de France, No. 183 (1). Dieppe A., Gonzlez Pandiella A. and Willman, A. (2011), The ECBs New Multi-Country Model of the Euro Area: NMCM Simulated with Rational Expectations, Working Paper Series, No 1315, ECB. Dieppe A., Gonzlez Pandiella, A., Hall, S. and Willman, A. (2011), The ECBs New MultiCountry Model of the Euro Area: NMCM with Boundedly Rational Learning Expectations, Working Paper Series, No 1316, ECB. Eaton, J., Kortum, S., Neiman, B. and Romalis, J. (2011), Trade and the Global Recession, NBER Working Paper, No. 16666, January. European Central Bank (2008), The Reliability of Survey Data During Periods of Financial Turmoil, Monthly Bulletin, Box 6, pp. 56-58, November. European Central Bank (2008a), Short-term forecasts of economic activity in the euro area, Monthly Bulletin, pp. 69-74, April. European Central Bank (2009a), Monetary policy and loan supply in the euro area, Monthly Bulletin, pp. 63-80, October. European Central Bank (2009b), The latest euro area recession in a historical context, Monthly Bulletin, pp. 97-113, November. European Central Bank (2009c), New procedures for constructing Eurosystem and ECB Staff projection ranges, December 2009 (available at www.ecb.europa.eu/pub/pdf/other/ newprocedureforprojections200912en.pdf). Esteves, P. S. (2009), Are ATM/POS data relevant when nowcasting private consumption? Banco de Portugal Working Paper, No. 25, November. Fagan, G. and Morgan, J. (2005), Econometric Models of the Euro-Area Central Banks, Edward Elgar. Fenz, G. and M. Schneider (2009), A leading indicator of Austrian exports based on truck mileage Monetary Policy & the Economy Q1/09, Oesterreichische Nationalbank, pp.44-52.
28
REFERENCES Genre, V., Kenny, G., Meyler, A. and Timmermann, A. (2010), How best to combine the forecasts in the ECB Survey of Professional Forecasters, Working Paper Series, No 1277, ECB. Giannone, D., Lenza, M., Momferatou, D. and Onorante, L. (2010), Short-term ination projections: a Bayesian VAR approach, CEPR Discussion Paper Series, No. 7746, March. Hall, S. and Mitchell, J. (2007), Combining Density Forecasts, International Journal of Forecasting, Vol. 23, pp. 1-13. Harding D. and Pagan, A. (2002), Dissecting the cycle: a methodological investigation, Journal of Monetary Economics, Vol. 49, pp. 365-381. Harding, D. and Pagan, A. (2010), Can we predict recessions? NCER Working Paper, No. 69. Hubrich K. and Tetlow, R. (2010), Financial Stress and Economic Dynamics: The Transmission of Crises, paper presented at the 2011 summer meeting of the North American Econometric Society (available at http://editorialexpress.com/conference/NASM2011/program/NASM2011. html). Kenny, G. (2010), Macroeconomic Forecasting: Can forecast combination help? Research Bulletin, No 11, November. Krugman, P. (2010), The International Finance Multiplier, The New York Times, March 22. Lucas, R. (2009), In defence of the dismal science, The Economist, Economics Focus Section, 6 August. Papademos, L. (2010), Economic modelling and policy-making: The usefulness of economic models in the policy-making process, remarks delivered at the Sixth Conference of the International Research Forum on Monetary Policy, Washington DC, 26 March. Papademos, L. and Stark, J. (2010), Enhancing the monetary analysis, ECB. Spaventa, L. (2009), Economists and economics: What does the crisis tell us? CEPR Policy Insight, No. 38. Stock, J. and Watson, M. (2002), Has the business cycle changed? NBER Macroeconomics Annual, Vol. 17, pp. 159-218. Taleb, N. N. (2007), The Black Swan: The impact of the highly improbable, Random House. Tersvirta, T. (2006), Forecasting Economic Variables with Non Linear Models, in Elliot, G. (ed.) Handbook of Economic Forecasting, Vol. 1 (8), Elsevier. Trichet, J. C. (2011), Intellectual challenges to nancial stability analysis in the era of macro prudential oversight, Banque de France Financial Stability Review, No. 15 February 2011.
29
Varian, H.R. and Choi, H. (2009), Predicting the Present with Google Trends, Google Research Blog, 2 April see http://googleresearch.blogspot.com/2009/04/predicting-present-with-googletrends.html
30

Fin CR Lessons

Uploaded by

Copyright:

Available Formats

Fin CR Lessons

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Fin CR Lessons

Uploaded by

Copyright:

Available Formats

O C C A S I O N A L PA P E R S E R I E S

by Geoff Kenny and Julian Morgan

OCCASIONAL PAPER SERIES

by Geoff Kenny and Julian Morgan

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

UNDERSTANDING THE KEY DRIVERS OF THE CRISIS

300 4 3 2 100 1 0 0 1995 200

0 2000 2005 2010

Source: Datastream (regional market indices).

ECB Occasional Paper No 130 October 2011

Chart 3 World trade and euro area exports

2 UNDERSTANDING THE KEY DRIVERS OF THE CRISIS

Sources: Eurostat and the IMF.

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

2 UNDERSTANDING THE KEY DRIVERS OF THE CRISIS

Financial Sector Risk premia Asset prices and net worth

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

Chart 6 Evolution of euro area GDP forecasts for 2009

3 HOW DID THE ECONOMIC ANALYSIS FARE DURING THE CRISIS?

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

3 HOW DID THE ECONOMIC ANALYSIS FARE DURING THE CRISIS?

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

3 HOW DID THE ECONOMIC ANALYSIS FARE DURING THE CRISIS?

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

Chart 10 Turning point predictions Area-wide Leading Indicator (ALI)

b) August 2008 3 2 1 0 -1 -2 -3 1999 2001 2003 2005 2007 2009 3 2 1 0 -1 -2 -3

c) August 2009 3 2 1 0 -1 -2 -3 1999 2001 2003 2005 2007 2009 3 2 1 0 -1 -2 -3

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

Chart 11 Confidence channels and output uncertainty

Sources: Bloomberg, the SPF of the ECB and ECB calculations.

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

WHAT IS THE WAY FORWARD?

ECB Occasional Paper No 130 October 2011

5 WHAT IS THE WAY FORWARD?

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

ECB Occasional Paper No 130 October 2011

You might also like